User:BenLubar/save research

Partly based on User:Rick/Save research and User:Andux/Format research.

Primitives

intn	an n bit little endian two's complement signed integer
uintn	an n bit little endian unsigned integer
string	uint16 (length) followed by length bytes of CP437-encoded text.
list of type	uint32 (count) followed by count elements of type. can be nested.
n-array of type	n elements of type. n is always a constant.
optional type	uint8 (flag); if flag is 0, nothing follows; if flag is 1, an element of type type follows.
mapping from type1 to type2	uint32 (count) followed by count instances of a type1 followed by a type2, alternating.

Data structures

name

string	first name
string	nickname
7-array of int32	word index (index into WORD string table or -1)
7-array of uint16	word form
uint32	language (index into TRANSLATION string table)
int16	unknown

version header

uint32	version (this page only concerns version 1404)
uint32	compression. when 0, the remainder of the file is verbatim. when 1, the remainder of the file is compressed with DEFLATE
uint32	length. only present when compression is nonzero.

Files

world.dat

world header

version header	version header
uint16	always zero?[Verify]
23-array of int32	unknown
optional name	the English part of the world name, like "The Realm of Daggers"
uint8	unknown
int16	unknown
3-array of int32	unknown
string	the gibberish part of the world name, like "Thadar Rabin"
4-array of list of list of string	generated raws
20-array of list of string	string tables: inorganic, plant, body, bodygloss, creature, item, building, entity, word, symbol, translation, color, shape, color_pattern, reaction, material_template, tissue_template, body_detail_plan, creature_variation, and interaction
mapping from uint32 to uint32	unknown
uint32	always zero?[Verify]
14-array of list of uint32	unknown

books

Still working on this; here's what I have so far:

the byte sequence d0 8a d0 8a d0 8a 00 00
a 32-bit bitfield - 0x00000804 has 6 bytes at the end that 0x00000004 doesn't. (A)
there are 48 bits, all of them zeroes.
there's a 32 bit integer that monotonically increases. (B)
the following 32 bit integers: 0, 1, 1.
B is repeated.
the following 32 bit integers: -1, -1, 1.
24 bits, all of them zeroes.
the 16 bit integer 0x2742 (ASCII B', which looks like "Book"(?))
the following 32 bit integers: 0, 0, -1.
16 bit integer. (C)
if C is 0x00, this book is bound with an INORGANIC material.
if C is 0x25, this book is bound with CREATURE leather.

32 bit signed integer. (D)
D is the index into the string table mentioned by C that further describes the material binding the book.
For example, if the INORGANIC string table has "QUARTZITE" in position 178 and C=0, D=178, this book is bound with quartzite.

the following 16 bit integer: -1.

16 bit integer. (G)
if G is 0, this is a book and the string H is a title.
if G is 5, this is a slab and the string H is a description.
A will have the 0x800 bit set iff G is 5.

the following 16 bit integers: 0, 0.
32 bit integer. (E)
the following 32 bit integer: -1.
32 bit integer (bitfield). (F)


if F & 2,
        74 bytes (TODO)
if F & 1,
        40 bytes (TODO)

a string. (H)

if A & 0x800
        six bytes