calibre 逆向工程实战Kindle 分层期刊 MOBI 的 Trailing Byte SequencesTBS索引编码完全解析【免费下载链接】calibreThe official source code repository for the calibre ebook manager项目地址: https://gitcode.com/GitHub_Trending/ca/calibre导读本文深入剖析 calibre 项目对 Kindle 分层期刊hierarchical periodicalMOBI 格式中 Trailing Byte SequencesTBS尾部字节序列的逆向工程成果。TBS 是附着在每个文本记录末尾的索引元数据负责描述 NCXNavigation Control XML索引条目与文本记录之间的起止关系是 Kindle 设备实现期刊 → 栏目section→ 文章article三级目录跳转的核心机制。读完本文你将掌握 TBS 的 vwi/fvwi 编码规则、六种 TBS 记录类型Opening、无节点、纯文章、栏目过渡、Ending 等的字节级布局以及 calibre 在 writer8 生成侧与 debug 解析侧对这套格式的完整实现。本文全部事实均来自 tbs_periodicals.rst 这一逆向工程笔记并结合仓库源码 writer8/tbs.py、utils.py 与 debug/mobi6.py 交叉印证。一、背景为什么要逆向 TBS在 MOBI 6/8 格式中每个文本记录text record大小通常为 4096 字节的倍数的尾部可以携带一段trailing data其中就包含 TBS。TBS 记录了哪些索引条目与当前文本记录发生接触——即哪些索引条目在该记录中开始starts、结束ends、跨越spans或完整包含completes。对于普通图书索引层级简单TBS 编码相对直接但对于分层期刊periodical其 NCX 层级为 periodical(depth 0) → section(depth 1) → article(depth 2)由于存在栏目与文章的嵌套起止关系TBS 编码变得异常复杂。Amazon 从未公开过这套格式的规范因此 calibre 作者只能通过分析 kindlegen 对样本期刊的输出进行逆向工程。原文档开篇就坦诚地写道文中所有信息与推断均来自对 kindlegen 输出样本的检查鉴于 Amazon 一般的水平无法保证这些信息是 TBS 索引的最优/最完整方式。这种务实态度贯穿整个文档——每个字段都标注了Unknown (vwi: always 0?)之类的疑问体现了严谨的逆向工程方法论。关键术语术语含义vwiVariable Width Integer变宽整数fvwi低 4 位用作标志位的 vwiflag vwiTBSTrailing Byte Sequences文本记录尾部的索引字节序列NCXNavigation Control XMLMOBI 的目录索引结构record文本记录MOBI 文件中固定大小通常 4096 字节的数据块index entry一个 NCX 索引条目对应一个目录节点期刊/栏目/文章在 debug/mobi6.py 中TBSIndexing类通过doc_typeMOBI header 的 type 字段是否为 257/258/259 来判定当前文件是否为分层期刊只有期刊才会走interpret_periodical的解析路径is_periodical self.doc_type in (257, 258, 259) if is_periodical: # Hierarchical periodical byts, a self.interpret_periodical(tbs_type, byts, dat[geom][0])二、编码基础vwi、fvwi 与序列编码表2.1 fvwi 的结构TBS 由一系列fvwi 数组成。encode_fvwi的实现位于 utils.pydef encode_fvwi(val, flags, flag_size4): Encode the value val and the flag_size bits from flags as a fvwi. This encoding is used in the trailing byte sequences for indexing. Returns encoded bytestring. ans val flag_size for i in range(flag_size): ans | flags (1 i) return encint(ans)即数值左移 4 位低 4 位留给标志位再按变宽整数encint编码。解码端对应decode_fvwiutils.py将高位移回得到数值、低 4 位还原为标志。decode_tbsutils.py随后根据标志位继续读取附加数据标志位附加数据0b1000续接标志continuation bit标记这是同一条 strand 的后续序列0b0100一个字节byte通常是记录中文章节点数量0b0010一个 vwi通常是 TBS 类型或偏移量0b0001一个 vwi通常是 EOF 标记或偏移encode_tbsutils.py是逆操作将 value 与 extra 字典中的标志位合并编码。2.2 序列编码表文档给出了从 kindlegen 输出中观察到的首个序列的编码对应关系0b1000 : Continuation bit # 续接位 0b0010 : 80 0b0011 : 80 80 0b0110 : 80 2 0b0111 : 80 2 80 0b0001 0 0b0010 0 0b0100 2注意80在 vwi 编码中即数值 0 续接标志最高位表示继续后续字节拼接数值因此80 2表示数值 2 带续接位、80 80表示数值 0 带双重续接。这些模式在后续各节的实际 TBS 字节中反复出现是解读字节流的基础。TBS 类型tbs_typeTBS 字节流中的首个 fvwi 携带类型信息debug 代码通过累加标志位得出outermost_index, extra, consumed decode_tbs(byts, flag_size3) for k in extra: tbs_type | k结合文档中的实例可归纳出各类型与记录形态的对应关系详见后续小节TBS 类型二进制适用记录形态010(2)纯单层Opening 无子节点、无节点记录父栏目 1、纯文章记录父栏目 1、Ending 记录011(3)含栏目过渡section transition的记录110(6)期刊与栏目共存Opening含栏目/文章、无节点记录父栏目1、多栏目过渡的 Ending111(7)纯文章记录且父栏目索引 1三、Opening Record起始记录起始记录是包含期刊根节点NCX 中 depth0 节点的文本记录。文档给出三种形态3.1 形态一只有期刊节点TBS 类型 2仅含期刊节点、无栏目/文章节点TBS 类型为010(2)Record #1: Starts at: 0 Ends at: 4095 Contains: 1 index entries (0 ends, 0 complete, 1 starts) TBS bytes: 82 80 Starts: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 68470) [j_xs Google reader] TBS Type: 010 (2) Outer Index entry: 0 Unknown (vwi: always 0?): 0字段含义82是 fvwi数值 0、标志0b0010携带类型 2、0b1000续接位80为附加的 vwi数值 0被注释为总是 0的未知字段。3.2 形态二期刊 一个栏目节点TBS 类型 6有期刊节点和一个栏目节点、但没有文章节点TBS 类型为110(6)Record #1: Starts at: 0 Ends at: 4095 Contains: 2 index entries (0 ends, 0 complete, 2 starts) TBS bytes: 86 80 2 Starts: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 93254) [j_xs Google reader] Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 541, Size: 49280) [Ars Technica] TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2这里多出的字节2被注释为总是 2的未知字节。3.3 形态三期刊 栏目 文章TBS 类型 6含文章信息同时包含栏目 1 节点和至少一个文章节点Record #1: Starts at: 0 Ends at: 4095 Contains: 4 index entries (0 ends, 1 complete, 3 starts) TBS bytes: 86 80 2 c4 2 Complete: Index Entry: 5 (Parent index: 1, Depth: 2, Offset: 549, Size: 1866) [Week in gaming: 3DS review, Crysis 2, George Hotz] Starts: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 79253) [j_xs Google reader] Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 541, Size: 35279) [Ars Technica] Index Entry: 6 (Parent index: 1, Depth: 2, Offset: 2415, Size: 2764) [Week in Apple: ZFS on Mac OS X, rogue tethering, DUI apps, and more] TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2 Article index at start of record or first article index, relative to parent section (fvwi): 4 [5 absolute] Number of article nodes in the record (byte): 2关键推断c4中4是记录起始处文章索引相对父栏目的偏移5 为绝对索引2是记录中文章节点数。如果记录中只有一篇文章则最后两个字节会合并为单个c0即不再有文章数量字节。3.4 含两次栏目过渡的起始记录Record #1: Starts at: 0 Ends at: 4095 Contains: 7 index entries (0 ends, 4 complete, 3 starts) TBS bytes: 86 80 2 c0 b8 c4 3 Complete: Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 564, Size: 375) [Ars Technica] Index Entry: 5 (Parent index: 1, Depth: 2, Offset: 572, Size: 367) [Week in gaming: 3DS review, Crysis 2, George Hotz] Index Entry: 6 (Parent index: 2, Depth: 2, Offset: 947, Size: 1014) [Max and the Magic Marker for iPad: Review] Index Entry: 7 (Parent index: 2, Depth: 2, Offset: 1961, Size: 1077) [iPad 2 steers itself into home console gaming territory with Real Racing 2 HD] Starts: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 35372) [j_xs Google reader] Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 939, Size: 10368) [Neowin.net] Index Entry: 8 (Parent index: 2, Depth: 2, Offset: 3038, Size: 1082) [Microsofts Joe Belfiore still working on upcoming Zune hardware] TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2 Article index at start of record or first article index, relative to parent section (fvwi): 4 [5 absolute] Remaining bytes: b8 c4 33.5 含三次栏目过渡的起始记录Record #1: Starts at: 0 Ends at: 4095 Contains: 10 index entries (0 ends, 7 complete, 3 starts) TBS bytes: 86 80 2 c0 b8 c0 b8 c4 4 Complete: Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 564, Size: 375) [Ars Technica] Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 939, Size: 316) [Neowin.net] Index Entry: 5 (Parent index: 1, Depth: 2, Offset: 572, Size: 367) [Week in gaming: 3DS review, Crysis 2, George Hotz] Index Entry: 6 (Parent index: 2, Depth: 2, Offset: 947, Size: 308) [Max and the Magic Marker for iPad: Review] Index Entry: 7 (Parent index: 3, Depth: 2, Offset: 1263, Size: 760) [OSnews Asks on Interrupts: The Results] Index Entry: 8 (Parent index: 3, Depth: 2, Offset: 2023, Size: 693) [Apple Ditches SAMBA in Favour of Homegrown Replacement] Index Entry: 9 (Parent index: 3, Depth: 2, Offset: 2716, Size: 747) [ITC: Apples Mobile Products Do Not Violate Nokia Patents] Starts: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 25320) [j_xs Google reader] Index Entry: 3 (Parent index: 0, Depth: 1, Offset: 1255, Size: 6829) [OSNews] Index Entry: 10 (Parent index: 3, Depth: 2, Offset: 3463, Size: 666) [Transparent Monitor Embedded in Window Glass] TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2 Article index at start of record or first article index, relative to parent section (fvwi): 4 [5 absolute] Remaining bytes: b8 c0 b8 c4 4对比 3.4 与 3.5 可以看出每多一次栏目过渡就会在Remaining bytes区域增加一组b8 c0 b8模式的字节对最后以c4 nn 为最后一栏的文章数收尾。四、Records with No Nodes无节点记录这类记录没有索引条目起始、结束或完整包含于其中被单个文章跨越span。subtype 为010分两种4.1 父栏目索引 1TBS 类型 6Record #4: Starts at: 12288 Ends at: 16383 Contains: 0 index entries (0 ends, 0 complete, 0 starts) TBS bytes: 86 80 2 c1 80 TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2 Article index at start of record or first article index, relative to parent section (fvwi): 4 [5 absolute] EOF (vwi: should be 0): 0关键推断如果该记录位于第一篇文章之前则 TBS 字节会简化为86 80 2即省略c1 80文章信息与 EOF 标记。4.2 父栏目索引 1TBS 类型 2Record #14: Starts at: 53248 Ends at: 57343 Contains: 0 index entries (0 ends, 0 complete, 0 starts) TBS bytes: 82 80 a0 1 e1 80 TBS Type: 010 (2) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Parent section index (fvwi): 2 Flags: 0 Article index at start of record or first article index, relative to parent section (fvwi): 14 [16 absolute] EOF (vwi: should be 0): 0与 4.1 的差异在于父栏目索引不再是隐含的 1而是通过 fvwi 显式给出a0 1解码为索引 2e1中携带的0b0001标志表示其后有一个 EOF vwi此处为 0。五、Records with Only Article Nodes纯文章记录这类记录没有栏目过渡即没有栏目结束/栏目开始配对只含一个或多个文章节点。分两种5.1 父栏目索引 1TBS 类型 7Record #6: Starts at: 20480 Ends at: 24575 Contains: 2 index entries (1 ends, 0 complete, 1 starts) TBS bytes: 87 80 2 80 1 84 2 Ends: Index Entry: 9 (Parent index: 1, Depth: 2, Offset: 16453, Size: 4199) [Vaccines success spurs whooping cough comeback] Starts: Index Entry: 10 (Parent index: 1, Depth: 2, Offset: 20652, Size: 4246) [Apples mobile products do not violate Nokia patents, says ITC] TBS Type: 111 (7) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown: \x02\x80 (vwi?: Always 256) Article at start of record (fvwi): 8 Number of articles in record (byte): 2字段说明Unknown: \x02\x80被怀疑是 vwi观察值恒为 256Article at start of record (fvwi): 8记录起始文章索引Number of articles in record (byte): 2记录内文章数量。关键推断如果记录中只有一篇文章最后两个字节会被单个字节80取代。文档还特别指出如果这是第一篇含文章的记录那么记录起始文章应当是最后一个栏目的索引kindlegen 确实这么做虽然作者认为逻辑上应该是第一个栏目索引。5.2 父栏目索引 1TBS 类型 2Record #16: Starts at: 61440 Ends at: 65535 Contains: 5 index entries (1 ends, 3 complete, 1 starts) TBS bytes: 82 80 a1 80 1 f4 5 Ends: Index Entry: 17 (Parent index: 2, Depth: 2, Offset: 60920, Size: 1082) [Microsofts Joe Belfiore still working on upcoming Zune hardware] Complete: Index Entry: 18 (Parent index: 2, Depth: 2, Offset: 62002, Size: 1016) [Rumour: OS X Lion nearing Golden Master stage] Index Entry: 19 (Parent index: 2, Depth: 2, Offset: 63018, Size: 1045) [iOS 4.3.1 released] Index Entry: 20 (Parent index: 2, Depth: 2, Offset: 64063, Size: 972) [Windows 8 system reset image leaks] Starts: Index Entry: 21 (Parent index: 2, Depth: 2, Offset: 65035, Size: 1057) [Windows Phone 7: Why its failing] TBS Type: 010 (2) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Parent section index (fvwi) : 2 Flags: 1 Unknown (vwi: always 0?): 0 Article index at start of record or first article index, relative to parent section (fvwi): 15 [17 absolute] Number of article nodes in the record (byte): 5注意此处Flags: 1与 4.2 中的Flags: 0不同——这是同一父栏目下有多篇文章的后续记录的标志。同样地如果只有一篇文章最后两个字节f4 5会被单个f0取代。六、Records with a Section Transition含栏目过渡的记录这类记录中存在从一个栏目过渡到下一个栏目的事件因此除了第一栏外记录中必然至少有一篇文章结束和一篇文章开始。TBS 类型为011(3)。6.1 第一种栏目TBS 类型 3Record #2: Starts at: 4096 Ends at: 8191 Contains: 2 index entries (0 ends, 0 complete, 2 starts) TBS bytes: 83 80 80 90 c0 Starts: Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 7758, Size: 26279) [Ars Technica] Index Entry: 5 (Parent index: 1, Depth: 2, Offset: 7766, Size: 1866) [Week in gaming: 3DS review, Crysis 2, George Hotz] TBS Type: 011 (3) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (vwi: always 0?): 0 First section index (fvwi) : 1 Extra bits: 0 First section starts Article at start of block as offset from parent index (fvwi): 4 [5 absolute] Flags: 0关键推断若记录起始处有多篇文章最后的c0会被c4 n取代n 为文章数。6.2 栏目过渡 结束栏目只有一篇文章Record #9: Starts at: 32768 Ends at: 36863 Contains: 6 index entries (2 ends, 2 complete, 2 starts) TBS bytes: 83 80 80 90 1 d0 1 c8 1 d4 3 Ends: Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 7758, Size: 26279) [Ars Technica] Index Entry: 14 (Parent index: 1, Depth: 2, Offset: 31929, Size: 2108) [Trademarked keyword sales may soon be restricted in Europe] Complete: Index Entry: 15 (Parent index: 2, Depth: 2, Offset: 34045, Size: 1014) [Max and the Magic Marker for iPad: Review] Index Entry: 16 (Parent index: 2, Depth: 2, Offset: 35059, Size: 1077) [iPad 2 steers itself into home console gaming territory with Real Racing 2 HD] Starts: Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 34037, Size: 10368) [Neowin.net] Index Entry: 17 (Parent index: 2, Depth: 2, Offset: 36136, Size: 1082) [Microsofts Joe Belfiore still working on upcoming Zune hardware] TBS Type: 011 (3) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (vwi: always 0?): 0 First section index (fvwi): 1 Extra bits (flag: always 0?): 0 First article of ending section, relative to its parents index (fvwi): 13 [14 absolute] Last article of ending section w.r.t. starting section offset (fvwi): 12 [14 absolute] Flags (always 8?): 8 Article index at start of record or first article index, relative to parent section (fvwi): 13 [15 absolute] Number of article nodes in the record (byte): 3这里出现了一个值得注意的字段序列First article of ending section结束栏目的首篇文章相对其父索引偏移 13、绝对 14与Last article of ending section相对起始栏目偏移 12、绝对 14——由于结束栏目Ars Technica索引 1下只有一篇文章索引 14两个字段指向同一篇文章。6.3 栏目过渡 结束栏目有多篇文章Record #11: Starts at: 40960 Ends at: 45055 Contains: 7 index entries (2 ends, 3 complete, 2 starts) TBS bytes: 83 80 80 a0 2 b5 4 1a f5 2 d8 2 e0 Ends: Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 34037, Size: 10368) [Neowin.net] Index Entry: 21 (Parent index: 2, Depth: 2, Offset: 40251, Size: 1057) [Windows Phone 7: Why its failing] Complete: Index Entry: 22 (Parent index: 2, Depth: 2, Offset: 41308, Size: 1050) [RIM announces Android app support for Blackberry Playbook] Index Entry: 23 (Parent index: 2, Depth: 2, Offset: 42358, Size: 1087) [Microsoft buys $7.5m worth of IPv4 addresses] Index Entry: 24 (Parent index: 2, Depth: 2, Offset: 43445, Size: 960) [TechSpot: Apple iPad 2 Review] Starts: Index Entry: 3 (Parent index: 0, Depth: 1, Offset: 44405, Size: 6829) [OSNews] Index Entry: 25 (Parent index: 3, Depth: 2, Offset: 44413, Size: 760) [OSnews Asks on Interrupts: The Results] TBS Type: 011 (3) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (vwi: always 0?): 0 First section index (fvwi): 2 Extra bits (flag: always 0?): 0 First article of ending section, relative to its parents index (fvwi): 19 [21 absolute] Number of article nodes in the record (byte): 4 -Offset from start of record to beginning of last starting section in this record (vwi)): 3445 Last article of ending section w.r.t. starting section offset (fvwi): 21 [24 absolute] Flags (always 8?): 8 Article index at start of record or first article index, relative to parent section (fvwi): 22 [25 absolute]与 6.2 的区别这里多出了两个字节1a f5编码了从记录起始到本记录中最后一个开始栏目起点的偏移量vwi值为 3445。即当结束栏目含多篇文章时需要额外的偏移字段来定位新栏目的起点。6.4 多次栏目过渡Record #9: Starts at: 32768 Ends at: 36863 Contains: 9 index entries (2 ends, 5 complete, 2 starts) TBS bytes: 83 80 80 90 1 d0 1 c8 1 d1 c b1 1 c8 1 d4 4 Ends: Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 7758, Size: 26279) [Ars Technica] Index Entry: 14 (Parent index: 1, Depth: 2, Offset: 31929, Size: 2108) [Trademarked keyword sales may soon be restricted in Europe] Complete: Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 34037, Size: 316) [Neowin.net] Index Entry: 15 (Parent index: 2, Depth: 2, Offset: 34045, Size: 308) [Max and the Magic Marker for iPad: Review] Index Entry: 16 (Parent index: 3, Depth: 2, Offset: 34361, Size: 760) [OSnews Asks on Interrupts: The Results] Index Entry: 17 (Parent index: 3, Depth: 2, Offset: 35121, Size: 693) [Apple Ditches SAMBA in Favour of Homegrown Replacement] Index Entry: 18 (Parent index: 3, Depth: 2, Offset: 35814, Size: 747) [ITC: Apples Mobile Products Do Not Violate Nokia Patents] Starts: Index Entry: 3 (Parent index: 0, Depth: 1, Offset: 34353, Size: 6829) [OSNews] Index Entry: 19 (Parent index: 3, Depth: 2, Offset: 36561, Size: 666) [Transparent Monitor Embedded in Window Glass] TBS Type: 011 (3) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (vwi: always 0?): 0 First section index (fvwi): 1 Extra bits (flag: always 0?): 0 First article of ending section, relative to its parents index (fvwi): 13 [14 absolute] Last article of ending section w.r.t. starting section offset (fvwi): 12 [14 absolute] Flags (always 8?): 8 Article index at start of record or first article index, relative to parent section (fvwi): 13 [15 absolute] -Offset from start of record to beginning ofnext starting section in this record: 1585 Last article of ending section w.r.t. starting section offset (fvwi): 12 [15 absolute] Flags (always 8?): 8 Article index at start of record or first article index, relative to parent section (fvwi): 13 [16 absolute] Number of article nodes in the record belonging to the last section (byte): 4多次过渡时每多一次过渡就重复一组Last article of ending section / Flags / Article index三元组对应字节d1 c b1与后续的c8 1最后以属于最后一栏的文章节点数byte收尾。七、Ending Record结束记录从逻辑上讲结束记录必须包含至少一篇文章结束、一个栏目结束以及期刊本身的结束。TBS 类型为010(2)Record #17: Starts at: 65536 Ends at: 68684 Contains: 4 index entries (3 ends, 1 complete, 0 starts) TBS bytes: 82 80 c0 4 f4 2 Ends: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 68470) [j_xs Google reader] Index Entry: 4 (Parent index: 0, Depth: 1, Offset: 51234, Size: 17451) [Slashdot] Index Entry: 43 (Parent index: 4, Depth: 2, Offset: 65422, Size: 1717) [US ITC May Reverse Judges Ruling In Kodak vs. Apple] Complete: Index Entry: 44 (Parent index: 4, Depth: 2, Offset: 67139, Size: 1546) [Google Starts Testing Google Music Internally] TBS Type: 010 (2) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Parent section index (fvwi): 4 Flags: 0 Article at start of block as offset from parent index (fvwi): 39 [43 absolute] Number of nodes (byte): 2字段解读Ends中包含了期刊根节点索引 0、栏目节点索引 4与文章节点索引 43的结束Article at start of block相对父索引偏移 39、绝对索引 43Number of nodes (byte): 2是待结束的节点数量。关键推断如果记录只有单篇文章结束最后两个字节f4 2会被单个f0取代。若最后一条记录含多次栏目过渡则其为类型 6布局与 Opening 记录对称Record #9: Starts at: 32768 Ends at: 34953 Contains: 9 index entries (3 ends, 6 complete, 0 starts) TBS bytes: 86 80 2 1 d0 1 c8 1 d0 1 c8 1 d0 1 c8 1 d0 Ends: Index Entry: 0 (Parent index: -1, Depth: 0, Offset: 215, Size: 34739) [j_xs Google reader] Index Entry: 1 (Parent index: 0, Depth: 1, Offset: 7758, Size: 26279) [Ars Technica] Index Entry: 14 (Parent index: 1, Depth: 2, Offset: 31929, Size: 2108) [Trademarked keyword sales may soon be restricted in Europe] Complete: Index Entry: 2 (Parent index: 0, Depth: 1, Offset: 34037, Size: 316) [Neowin.net] Index Entry: 3 (Parent index: 0, Depth: 1, Offset: 34353, Size: 282) [OSNews] Index Entry: 4 (Parent index: 0, Depth: 1, Offset: 34635, Size: 319) [Slashdot] Index Entry: 15 (Parent index: 2, Depth: 2, Offset: 34045, Size: 308) [Max and the Magic Marker for iPad: Review] Index Entry: 16 (Parent index: 3, Depth: 2, Offset: 34361, Size: 274) [OSnews Asks on Interrupts: The Results] Index Entry: 17 (Parent index: 4, Depth: 2, Offset: 34643, Size: 311) [Leonard Nimoy Turns 80] TBS Type: 110 (6) Outer Index entry: 0 Unknown (vwi: always 0?): 0 Unknown (byte: always 2?): 2 Article index at start of record or first article index, relative to parent section (fvwi): 13 [14 absolute] Remaining bytes: 1 c8 1 d0 1 c8 1 d0 1 c8 1 d0八、源码实现calibre 如何生成与解析 TBS8.1 生成侧writer8 的完整算法链calibre 的 MOBI8 写出器在 writer8/main.py 的create_indices方法中构建索引扁平化 ToC将 OEB 的 ToC 深度优先展平为entries列表为每个节点记录index、depth来自ncx_hlvl、parent、first_child、last_child、offset文本流中的字节偏移等字段线性化排序Kindle 要求条目按(depth, offset)排序calibre 通过该排序将非线性 ToC 线性化源码注释明确说明非线性 ToC 会导致栏目间跳转失效计算长度get_next_start用同一或更浅深度、偏移更大的下一条目减去当前条目偏移得到每个条目的length生成 TBS调用apply_trailing_byte_sequences(entries, self.records, self.uncompressed_record_lengths)writer8/tbs.py返回值存入self.has_tbs。apply_trailing_byte_sequences内部依次执行函数职责collect_indexing_data对每条文本记录筛选出与之接触的索引条目并用fill_entry标记为 starts/ends/spans/completesseparate_strandspopulate_strand将条目组织成strand——从顶层索引向下逐级递进的层级链同一父节点下连续编号的兄弟条目可合并进同一 strand对应0b100标志encode_strands_as_sequences将 strands 编码为序列列表首条序列携带 TBS 类型0b10标志、条目数0b100、跨度标记0b1后续 strand 使用0b1000续接位sequences_to_bytes调用encode_tbs将每个序列转为 fvwi 字节仅首条序列 flag_size 为 3其余为 4因为后续序列可能需要0b1000标志calculate_all_tbs汇总所有记录生成{record_index: tbs_bytes}映射值得注意的细节encode_strands_as_sequences中有一段被False and禁用的逻辑源码注释写道I cant figure out exactly when kindlegen decides to insert this, so disable it for now无法确定 kindlegen 何时插入0b1 length_offset扩展暂时禁用——这正对应原文档中多处Unknown字段的未解之谜。此外calculate_all_tbs对NegativeStrandIndex异常有降级处理当非首条 strand 出现负索引时改用tbs_type5重新编码writer8/tbs.py这是对 kindlegen 在特殊 ToC 下行为的兼容策略。8.2 解析侧debug 工具如何反解 TBSMOBI 调试工具 debug/mobi6.py 的TBSIndexing类实现了与文档完全对应的解码逻辑dump_recordmobi6.py输出Record #N: Starts at: ... Ends at: ...、Contains: N index entries (n ends, n complete, n starts)、TBS bytes: ...以及 Ends/Complete/Starts 三个条目列表——这正是原文档中所有示例的排版格式interpret_periodicalmobi6.py针对期刊type 257/258/259进一步解析read_starting_section读取记录起始栏目索引若标志含0b0100则同时读取该栏文章数若含0b0001则读取 EOF 标记read_section_transitions循环读取栏目过渡——遇0b1000标志表示上一栏最后一篇文章并切换到下一栏否则表示本栏首篇文章0b0100给出文章数0b0001给出到下一栏起点的偏移解析结果直接生成人类可读的行与文档中的字段描述一一对应如First article in this record of section N (relative to its parent section): ai [absolute]该调试器还可将解析结果按 TBS 类型分类写入tbs_type_{n}.txt文件便于批量比对 kindlegen 与 calibre 的输出差异。8.3 期刊结构检测生成侧在写出前还会通过 utils.py 的detect_periodical校验 ToC 是否符合期刊结构depth1 的节点必须是article、depth2 必须是section、depth3 必须是periodical且不允许超过 3 层——这与 TBS 编码中depth 0/1/2 三级索引的假设完全吻合。九、逆向工程的方法论与遗留问题原文档通篇体现了严谨的逆向工程态度值得读者借鉴以观测数据为准所有字节模式都来自 kindlegen 的真实输出而非猜测显式标注不确定性Unknown (vwi: always 0?)、Flags (always 8?)、Extra bits (flag: always 0?)等标注区分了已确认与仅观察的事实给出可证伪的推断如若只有一篇文章则c4 2→c0、若在首篇文章之前则省略c1 80等均可通过构造样本验证识别未解之谜0b0010标志在 section transition 场景下的含义尚未破解read_section_transitions中遇到该标志直接抛出ValueError(Dont know how to interpret flag 0b0010...)。文档同时给出了明确的适用边界这些推断仅针对 kindlegen 输出的样本期刊不保证是所有可能性的最全/最优编码calibre 生成端在遇到无法处理的情况时会通过异常降级tbs_type 5或直接接受不完美的输出。十、总结Trailing Byte Sequences 是 Kindle 期刊 MOBI 中文本记录 ↔ 索引条目关系的唯一边界契约。通过 tbs_periodicals.rst 这份逆向工程笔记可以完整掌握六类记录Opening、无节点、纯文章、栏目过渡、多栏目过渡、Ending的 TBS 字节布局以及 vwi/fvwi 编码、0b1000续接位、文章数/偏移/EOF 等字段的语义而 writer8/tbs.py、utils.py 与 debug/mobi6.py 则分别给出了这套格式在 calibre 中的生成、编解码与诊断实现三者互为印证构成了从逆向分析到工程落地的完整闭环。对于任何需要自行实现 MOBI 期刊索引生成或解析的开发者本文档与源码都是不可多得的参考资料。【免费下载链接】calibreThe official source code repository for the calibre ebook manager项目地址: https://gitcode.com/GitHub_Trending/ca/calibre创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考