Draft: draft-ietf-quic-qpack

Commentary: A dynamic table entry that has not been acknowledged by the decoder must not be evicted by the encoder , even if there has never been any references to it . This is already alluded to in the current spec at by the phrase " acknowledged by the decoder " , but since " acknowledged " is used interchangeably for acknowledging the insertion instruction and acknowledging references , a reader could understand this to mean " all references acknowledged " . In addition , at three other places eviction of entries with unacknowledged references are banned , with unambigously no ban on eviction of unacknowledged entries with no references . I argue that more explicit language would be beneficial . This issue has already been discussed at and below . Note that avoiding this kind of corruption could also be achieved by banning an encoder from inserting entries that it does not immediately reference , but this is probably way too restrictive . For example , an encoder might wish to achieve best latency and speculatively improve future compression ratio at the expense of potentially wasting current bandwidth by encoding header fields as literals while inserting them at the same time for potiential future use . This is already alluded to in the spec at multiple places . Below is a specific scenario when this issue might cause corruption :   is 5 . Suppose that there is a fresh connection , with an empty dynamic table , and there is a header list to be sent that has two header fields which fit in the dynamic table together . Suppose one encoder implementation inserts these two header fields into the dynamic table , and encodes the header list using two Indexed Header Field instructions , referring to absolute indices 1 and 2 . This header block will have an abstract Largest Reference value of 2 , which is encoded as 3 on the wire . A spec compliant decoder should have no difficultly correctly decoding this header block . Suppose another encoder implementation starts by adding 10 unique entries , all different from the two header fields in the header list to be sent , to the dynamic table , but does not emit references to any of them . Of course in the end only the last couple will stay in the dynamic table , the rest will be evicted . This can be considered spec compliant depending on how one interprets the spec , see above . Suppose that after this , this encoder implementation encodes our header list by inserting both header fields into the dynamic table , with absolute indices 11 and 12 , and using two Indexed Header Field instructions on the request stream . The abstract Largest Reference value is 12 , which is encoded as 3 on the wire since 2 * MaxEntries = = 10 . Now suppose that out of the 12 instructions on the encoder stream , only the first two arrive before all the request stream data are received . The remaining 10 instructions on the encoder stream are delayed on the wire . Now a spec compliant decoder sees the same header block on the request stream , bit by bit , as above , and two insertions on the encoder stream . It will emit dynamic table entires 1 and 2 , which are different from the encoded header list . 
 You should really create a new issue for these things . Because this comment relates only to the issue , not the proposed fix . I have n't read the proposed changes , though I will if you think that there is a fix in there that is worth keeping after this . Anyhow , the example is invalid here : Because : And you can only ever fit 5 entries in that table . 
 That 's exactly the issue : " it has first been acknowledged by the decoder " is ambigous . " Acknowledged " is being used throughout the text with two different meanings , most often to refer to references , not insertions . The ban on evicting an entry the insertion of which has not been acknowledged , that you are quoting , only occurs once out of four different locations talking about when an entry can be evicted . The other three places clearly only ban evicting an entry with unacknowledged references ( note the use of the same word ) , giving the impression that acknowledgement of the insertion is not necessary before eviction . Since the design is correct if you read it right , but wording is not as clear as I think it should be , I consider this issue editorial . The change I propose in this PR is meant to be an improvement on clarity . 
 OK , having read the changes , I agree with them . They consolidate the fix by creating two conditions that cause an entry to be " blocking " . 
 Rebased on . Please let me know if anybody can come up with a term better than " blocking " which is confusing given that streams can be " blocked " . 
 Thank you for your feedback . 
 Thanks for this . I do n't have a better name for blocking entries right now , but I will keep thinking on it as I agree it could confuse people even more . 
 This looks reasonable .
Paragraph (version A):
An encoder MUST NOT insert an entry into the dynamic table (or
duplicate an existing entry) if doing so would evict an entry with
unacknowledged references.  For header blocks that might rely on the
newly added entry, the encoder can use a literal representation.
Paragraph (version B):
A dynamic table entry is considered blocking and cannot be evicted
until its insertion has been acknowledged and there are no
outstanding unacknowledged references to the entry.  In particular, a
dynamic table entry that has never been referenced can still be
blocking.
Paragraph (version B):
A blocking entry is unrelated to a blocked stream, which is a
stream that a decoder cannot decode as a result of references to
entries that are not yet available.  Any encoder that uses the
dynamic table has to keep track of blocked entries, whereas
blocked streams are optional.
Paragraph (version B):
An encoder MUST NOT insert an entry into the dynamic table (or
duplicate an existing entry) if doing so would evict a blocking
entry.  In this case, the encoder can send literal representations of
header fields.
Paragraph (version A):
Determining which entries are too close to eviction to reference is
an encoder preference.  One heuristic is to target a fixed amount of
available space in the dynamic table: either unused space or space
that can be reclaimed by evicting unreferenced entries.  To achieve
this, the encoder can maintain a draining index, which is the
smallest absolute index in the dynamic table that it will emit a
reference for.  As new entries are inserted, the encoder increases
the draining index to maintain the section of the table that it will
not reference.  If the encoder does not create new references to
entries with an absolute index lower than the draining index, the
number of unacknowledged references to those entries will eventually
become zero, allowing them to be evicted.
Paragraph (version B):
Determining which entries are too close to eviction to reference is
an encoder preference.  One heuristic is to target a fixed amount of
available space in the dynamic table: either unused space or space
that can be reclaimed by evicting non-blocking entries.  To achieve
this, the encoder can maintain a draining index, which is the
smallest absolute index in the dynamic table that it will emit a
reference for.  As new entries are inserted, the encoder increases
the draining index to maintain the section of the table that it will
not reference.  If the encoder does not create new references to
entries with an absolute index lower than the draining index, the
number of unacknowledged references to those entries will eventually
become zero, allowing them to be evicted.
Paragraph (version A):
Knowledge that a header block with references to the dynamic table
has been processed permits the encoder to evict entries to which no
unacknowledged references remain, regardless of whether those
references were potentially blocking (see blocked-insertion).  When a
stream is reset or abandoned, the indication that these header blocks
will never be processed serves a similar function; see stream-
cancellation.
Paragraph (version B):
Knowledge that a header block with references to the dynamic table
has been processed permits the encoder to evict entries to which no
unacknowledged references remain (see blocked-insertion).  When a
stream is reset or abandoned, the indication that these header blocks
will never be processed serves a similar function (see stream-
cancellation).
Paragraph (version A):
Before a new entry is added to the dynamic table, entries are evicted
from the end of the dynamic table until the size of the dynamic table
is less than or equal to (table capacity - size of new entry) or
until the table is empty.  The encoder MUST NOT evict a dynamic table
entry unless it has first been acknowledged by the decoder.
Paragraph (version B):
Before a new entry is added to the dynamic table, entries are evicted
from the end of the dynamic table until the size of the dynamic table
is less than or equal to (table capacity - size of new entry) or
until the table is empty.  The encoder MUST NOT evict a blocking
dynamic table entry (see blocked-insertion).
Paragraph (version A):
Reducing the dynamic table capacity can cause entries to be evicted
(see eviction).  This MUST NOT cause the eviction of entries with
outstanding references (see reference-tracking).  Changing the
capacity of the dynamic table is not acknowledged as this instruction
does not insert an entry.
Paragraph (version B):
Reducing the dynamic table capacity can cause entries to be evicted
(see eviction).  This MUST NOT cause the eviction of blocking entries
(see blocked-insertion).  Changing the capacity of the dynamic table
is not acknowledged as this instruction does not insert an entry.