ESOMod:DAT File Format

DAT files basically just contain a short header and then zero or more compressed files. They are found in a few places:
:* \game\client\
::* game0000.dat
:* \depot\
::* eso0000.dat to eso0215.dat
:* \vo_en\
::* esoaudioen0000.dat to esoaudioen0003.dat

For each of the three sets of files there is a single MNF file in the same directory with a matching name (game.mnf, eso.mnf, esoaudioen.mnf).


File Format
The overall DAT format is simply:

     [File Header (14 bytes)] 
     [0...N Compressed Files] 

File Header
The DAT file header appears to be a fixed size of 14 (0x0E) bytes and contains a few fields (numbers in little endian):

     byte[4] MAGIC_WORD[4]     = "PES2"
     word    PES_VERSION       = 1 
     dword   Unknown2          = 0 /* seems to be 0x0 in all dat files */
     dword   OffsetToFirstFile = 0x0E /* always length of this header */

It is assumed that the last field is the offset to the first sub-file as all DATs have a value of 0x0000000E in this field.

Compressed Files
Directly following the header are 0 or more files either uncompressed or compressed with the zLib, Snappy, or Oodle formats. 

Oodle
Oodle compression was introduced in update 25 and in MNF files having a version field of 3 or higher. Oodle compression can be done in addition to zLib or Snappy compression (Oodle decompression should always be done first). To recognize Oodle formats look for the following first two byte values:

:* 8C 06
:* 8C 0A
:* CC 06
:* CC 0A

To decompress Oodle data find the ""oo2core_8_win64.dll" file somewhere online and dynamically load it as follows (C++ code example):

     typedef int __stdcall OodleLZ_Compress_Func(uint fmt, byte* buffer, int bufferSize, byte* outputBuffer, int level, void* unused1, void* unused2, void* unused3);
     typedef int __stdcall OodleLZ_Decompress_Func(byte* buffer, int bufferSize, byte* outputBuffer, int outputBufferSize, int a, int b, int c, void* d, void* e, void* f, void* g, void* h, void* i, int threadModule);
     
     OodleLZ_Compress_Func*   g_OodleCompressFunc;
     OodleLZ_Decompress_Func* g_OodleDecompressFunc;
     
     HINSTANCE mod = LoadLibrary("oo2core_8_win64.dll");
     
     g_OodleCompressFunc   = (OodleLZ_Compress_Func   *) GetProcAddress(mod, "OodleLZ_Compress");
     g_OodleDecompressFunc = (OodleLZ_Decompress_Func *) GetProcAddress(mod, "OodleLZ_Decompress");
     
     byte* pOutputBuffer = new byte[UncompressedSize];
     int BytesDecompressed = g_OodleDecompressFunc(pCompressedData, CompressedSize, pOutputBuffer, UncompressedSize, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3);

Note that the UncompressedSize must exactly match that used in the file or the data will fail to decompress (returns 0). This occurs in the first file from eso00000.dat in update 25 (uncompressed size stored in MNF is larger than the actual uncompressed size of the data.

Some Oodle decompressed files have unknown header data before  the actual file data. If the first dword in the uncompressed data is 0x00000000 then it indicates the presence of the header data which has the following format:

     dword Zero
     word  Unknown
     word  Size1
     byte  Data1[Size1]  
     dword Size2          // Usually 0x00000028
     byte  Data2[Size2]

zLib/Snappy
Files compressed with zLib seem to always have the first two bytes as 78 9C (default compression) but in theory this could take a variety of values. Files continue one after another with no extra data between them until the end of the DAT file.

The data can be uncompressed either using the ZLib library (exactly as shown in their usage example) or the Snappy library.


Subfile Formats
The actual content of the DAT sub-file appears to have one of two formats depending on the value of Unknown2 in the MNF file header.

Unknown2 0x06
This format, currently only used in the GAME.MNF file, has a small header and two variable sized records before the actual file data:

    dword Null1 = 0x00000000
    dword Size1                //Ranges from 0x1B8 to 0x1BC
    byte  Data1[Size1]
    dword Size2                //Always 0x28
    byte  Data2[Size2]
    byte  FileData[]

The Data1/Data2 formats are unknown. They look somewhat like compressed data but are not valid snappy or zlib formats.

This format was introduced in the 8/2/2014 beta patch.

Unknown2 Other
This is the typical/default format where the data is simply the entire sub-file (no header or footer bytes).

Notes
:* There doesn't appear to be any way of directly accessing a random file in the DAT file without processing all previous files.
:* The MNF_File_Format contains additional information related to the DAT files including a file index holding offsets to sub-files with the DAT.
:* The last file in game0000.dat is a ZOSFT (Zenimax Online Studios File Table) which contains a nul-terminated list of filenames among other binary data. Unfortunately the number of filenames does not correlate exactly to the number of compressed files in the DAT file. The ZOSFT is itself composed of several compressed data blocks followed by the filename text.
:* Some files store uncompressed data (ex: eso0093.dat). These appear to have B8 D6 0A as the first three bytes of their data followed by the uncompressed file.
:* The patch for the 8/2/2014 beta appears to have changed the format of the sub-file data for game0000.dat (the overall DAT format hasn't changed). The new format appears to be (header bytes are in big endian bit format):
     dword Null1 = 0x00000000
     dword Size1
     byte  Data1[Size1]
     dword Size2
     byte  Data2[Size2]
     byte  FileData[]    //Same as pre-patch
::* The Data1/Data2 formats are unknown. They look like compressed data but are not valid snappy or zlib formats.
::* DataSize1 ranges from 0x1B8 to 0x1BC
::* DataSize2 is always 0x28
::* This DAT format might be identified by the value of 0x6 in Unknown2 of the MNF file header.

File Contents
Moved to