Code change:
4. <del> The "br-d" content encoding identifies a resource that is a </del> <ins> The "dcb" content encoding identifies a resource that is a </ins> "Dictionary-Compressed Brotli" stream. <del> A "Dictionary-Compressed Brotli" stream is a Brotli RFC7932 stream for a response that has been compressed with an external dictionary using a compression window not larger than 16 MB. </del> <ins> A "Dictionary-Compressed Brotli" stream has a fixed header that is followed by a Shared Brotli SHARED-BROTLI stream.  The header consists of a fixed 4 byte sequence and a 32 byte hash of the external dictionary that was used.  The Shared Brotli stream is created using the referenced external dictionary and a compression window that is at most 16 MB in size. The 36-byte fixed header is as follows: </ins> <del> Clients that announce support for br-d content encoding MUST be able </del> <ins> Clients that announce support for dcb content encoding MUST be able </ins> to decompress resources that were compressed with a window size of up to 16 MB.

Related discussion:
text: This eliminates the response header and moves the hash of the dictionary into the payload of the response. Specifically, it adds 36-byte header in front of the compressed stream which is a 4 or 8-byte signature followed by the 32-byte SHA-256 hash digest. The signature is different for the two different compressions: Dictionary-Compressed Brotli: (0xFF followed by ) Dictionary-Compressed Zstandard: (32-byte skippable Zstandard frame) This also changes the content-encoding names for the different schemes to make it clear that they are a new format and not a raw brotli or Zstandard stream ( and ). From the cli, the stream is equivalent to: or
FWIW, it doesn't look like it would be particularly difficult to use a skippable frame instead of a custom header in the Zstandard case if that's the way we want to go. It's 4-bytes bigger (for the frame size) so it will essentially have an 8-byte "magic number" (since the size will always be the same). It won't help with the creation of the files but if there's benefit to have the existing tooling be able to decode them (without verifying the hash because it will just skip it) we can go that route. On the client side, it feels like it will be cleaner on the implementation to use the same flow for verifying the dictionary hash and magic number for both encodings without having to deal with different sized magic numbers (and always buffer and remove the first 36 bytes) but it's not a significant difference.
Still LGTM
NAME could you please take a look and make sure I got the byte sequence for the skippable frame correct? I tested with the zstd cli and it was able to decode the file with the header without issue when created like this:
Thanks. Updated and added an example with the zstd cli.
This is a discussion we've had several times when defining content codings, but it seems like there is never really a single answer. Should the content coding be self-describing, or can it rely on metadata in fields? The compression dictionary work uses header fields to identify which compression dictionary is in use. Originally, the client would indicate a dictionary and the server would indicate use of that dictionary by choosing the content coding.