Draft: draft-irtf-nwcrg-coding-and-congestion-01

Commentary: Thanks so much for doing this ! If you incorporate my comments , you 've written exactly what I wanted to do , you 've just taken the work from me :) 
 SD : We discussed in many RFC congestion control . I encourage you to include scalable congestion control and the IETF history on the reaction to ECN signals . EL : Good point . 
 I was talking to   afterwards , and  told me what I SHOULD have said , was that the sender should perform whatever its response to congestion would be - if it 's an old - style sender , treat recovery as loss , but if it supports scalable congestion control ( L4S is one example ) , it does n't have to treat recovery as loss - it just needs to do something appropriate . 
  's point sounds like " the congestion control reaction is what it is , it has nothing to do with coding " , and I 'd agree with that . I would therefore not talk much about different types of cc . in the document , as this may just add more confusion . 
 I think that this is covered by the current version of the document . " The server MUST be able to detect the " repaired packet " signal . The base solution does not describe how the server reacts to such signal . " 
 I may have forgotten things over the holidays , so I 'm not sure how clear the document is on that ... but I do n't like the sentence that you quote because it sounds like " repaired packet " is a different kind of signal than " lost ( and not repaired ) packet " . For congestion control , that distinction should n't exist . I see coding and CC as two separate layers , benefiting from different signals . 
 If I remember well , we formulated this sentence this way : 1 ) to start reactions and discussions as it is quite a strong claim as Nico said it during the meeting 2 ) adding the " repaired " signal does not seem to me to interfere with the other signals : it is just an additional signal that can be ignored by the congestion control . A " repaired " packet does not imply a lost packet ( the repair could be due to re - ordering ) , a lost packet will be determined by the loss detection mechanism . So in my opinion , we can have different cases for repaired packets : a lost and repaired packet : the packet has been both repaired and detected as lost ( the loss detection may happen before of after the loss detection , it does not matter ) . Even if the packet is repaired , it can still be marked as lost by the loss detection mechanism because " repair " does not mean the same as an acknowledgement . a non - lost repaired packet ( re - ordering implied the repair of the packet but the original packet has been received afterwards ) A loss - based congestion control will likely attach itself to the " lost packet " event that will be triggered regardless of the fact that the packet has been repaired , so I do n't think it will be perturbated in any way by the " repaired packet " event . In my opinion , a congestion control mechanism could also attach itself to the " repaired " event to do whatever it wants with this additional signal ( I have no idea on what it could do with this signal , but a new congestion control could define some heuristics based on the different signals occuring during the connection ) . I 'm saying this with QUIC in mind , it may be more difficult to not mix the " repaired " and " lost " events with TCP ( but is seems doable to me a - priori ) . 
 I think I 'll insist on this point . This is exactly the clarity that 's missing in this document : because coding and congestion control are discussed as a conflated whole , it sounds as if , e.g. , the difference between " lost but repaired " and " lost but not repaired " might matter to a congestion control mechanism . You even suggest that a new control could do something useful with it ; why ? ? What value does it add , when a congestion control mechanism tries to find the right sending rate based on how full the network is ? The two are really entirely separate and should be discussed and presented as such . Implementations may produce the signals you describe , such that their senders can then pick them apart to do what is most reasonable ; that does n't change the fact that conceptually , these are completely separate and discussing them as a combo makes it all confusing and even misleading . This is exactly the confusion the NWCRG has been dealing with for so long . Let 's please clear this up once and for all and split these beasts . 
 I think we need to be sure about the current scope of the draft then . The main idea we wanted to bring with the other guys ( correct me if I 'm wrong ) was to avoid interferences between the coding and congestion control mechanism , especially avoiding a receiver that simply ACKs recovered packets , making the sender think that they have been successfully received . I agree with you , defining a " repaired " signal may lead to think that we consider those two mechanisms as whole . So why defining a " repaired " signal instead of saying that " acknowledging a recovered packet should not be done " ? We discussed this with and and we concluded that defining the " repaired " signal seemed easier to incorporate within a protocol such as TCP compared to the other solution that requires major design changes . With QUIC , we can incorporate the two solutions without any problem . I agree that the current state of the draft is misleading I am OK to change that . We need however to keep in mind the feasibility of the solution on existing protocols . When we started to elaborate the draft with and ( again , correct me if I 'm wrong ) and until now , we decided that determining what the congestion control behaviour should be was out - of - scope for the document . The scope was just the following : we should not hide the recovery / loss signal so that the already existing congestion control mechanisms react the exact same way to losses whether there is coding or not . For the rest , we did not take any decision . This is why you will see many different behaviours described in the current section 6 . of the draft ( e.g. 6.2 . ) . Until now , we never discussed whether these solutions are good or bad , as they are congestion control behaviours and we are working on the coding component ( more specifically what signalling should be done between coded hosts ) . We just presented them . Because the approach of the document was to avoid interferences between the coding and congestion control mechanisms , not to decide what a congestion control should do or what set of signals it should or should not use . But do n't get me wrong , I am not saying that using the " repaired " signal for a congestion control is a good idea . I am fine with the idea of recommending to not use such a signal in congestion controls , but to me , it redefines the scope of the document and might imply other groups than the nwcrg . 
 I think what I 'm suggesting is less problematic than you may believe . To me , some parts of the draft are currently messy because of how they mix the two things - e.g. fig . 2 really hurts my feelings , as a cc . person :) Do n't get me wrong , I agree with many things in the draft , and with most of what you say in your comments here too - e.g. this : " especially avoiding a receiver that simply ACKs recovered packets , making the sender think that they have been successfully received . " is definitely an important goal ! This statement from your comment best describes why we seem to have a disconnect : " I agree that the current state of the draft is misleading I am OK to change that . We need however to keep in mind the feasibility of the solution on existing protocols . " because what I 'm proposing changes implementation feasibility in no way . This is just conceptual - how we talk about things . Coding can ( sigh ... if it MUST ... ) fit into TCP because TCP does error control , not because TCP does congestion control . I can see one useful input from coding to cc : an " importance " ( or urgency ? ) associated with a data block . I can see useful inputs from cc to coding : network measurements ( which is what cc carries out ) . That 's how these things should be described IMO - conceptually separate blocks with interfaces . Implementations can definitely build a compound " repaired " signal which has a meaning to both blocks at the same time , and I do n't object to giving implementation examples or advice after introducing things cleanly and separately . But these , concept and implementation , are also two separate things . As for the scope , that 's nothing to worry about . This is not about telling a cc . mechanism what to do - it 's about explaining the relationship between coding and cc , and that wo n't change if we re - structure things as I suggest . I 'm beginning to think that I should just make a PR where I update everything the way I think it should be , and then we can have a much more focused discussion ? Would that be ok ? ( next week or so ... probably not before ) 
 100 % agree . I 'm beginning to think that I should just make a PR where I update everything the way I think it should be , and then we can have a much more focused discussion ? Would that be ok ? ( next week or so ... probably not before ) Perfect for me , I do not plan to modify the draft during that time anyway . 
 +1 I think that the separation of the " congestion " and " coding " channels can help in clarifying there is a difference here . +1 We need to make that clear in the document . ( next week or so ... probably not before ) Ok - I may do some update on master branch before next week 
 Just to be clear " lost but repaired " is an erased packet rebuilt at destination , erased means it thus can be either a reordered or a lost packet . 
 Okay , but then there 's also the case of " lost but NOT repaired " , which is also an erased packet , i.e. either reordered or lost . These two cases are , and should absolutely be , the same for a congestion controller . This is why I do n't like feeding " repaired " into a CC . 
 OK I think I see the problem : you are on the CC - side and we are on the NC - side :) First , I recall we agreed that there are two different signaling paths : one for CC one for NC . A lost but not repaired packet is an erased packet and we do n't have any supplementary info at the sender side to provide because the CC will provide it . A lost but rebuilt is a lost packet lost or reordered and we must provide the info at the sender side because the CC wo n't provide it . 
 Oh yeah , I see that you ( NCers ) need the signal ! But it needs to be separate from the CC signal . I 'm glad to see that you say this was agreed - and the early part of the draft reflects that , but later in the draft ( from section 6 ) things get really mixed up , which is why propose to remove sections 6 and 7 . 
 Not only , when I say " /A lost but rebuilt is a lost packet lost or reordered and we must provide the info at the sender side because the CC wo n't provide it./ " means that the NC receiver needs to alert the sender CC that a lost or a reorder ( following the CC flavor used ) occurred . This allows the sender to react in a compliant manner as TCP will do . One possible solution would be the NC receiver marks ECN when a packet is rebuilt to inform the sender as explained in my previous NWCRG talk : 
 That 's very implementation oriented . Of course , if you want to hack your stuff into TCP and leverage TCP ACKs yet convey that CC . should reduce its rate , you can use tricks like ECN . I would prefer to first discuss these things as entirely conceptually separate - and then , once it 's all neat and clean , discuss possible implementation strategies such as this one . 
  , I showed this just to be sure we lay on the same thread and that all of us ( whatever the point of view is from CC or NC ) are speaking the same language . 
 ACK ! 
 One question still remains , what do we do with section 6 which is also implementation oriented and cross - layer . As far as I understand your point , we should not speak or propose such cross - layer schemes as both layers : CC and NC should remain independent right ? Furthermore I 'm not happy with the beginning " Delay - based congestion controls ignore packets that have been repaired with coding . " . If we lay on the same logic , delay - based should not ignore these packets as a rebuilt one , following the code used and ( n , k ) length might artificially increase the e2e delay . In that case , one could argue that a rebuilt packet implicitly signal a rebuilt packet to the sender as the delay increase , but this really depends on the kind of code used and its parameters . The more I read this section the more I think we should reconsider it . 
 cross - layer schemes as both layers : CC and NC should remain independent right ? Yes - I think we CAN then have a section ( a new one ! I would really rather just dump the current text of sections 6 and 7 ) that VERY carefully discusses how to implement things together . But then there would have to be text saying that this is implementation only , still the signals are conceptually separate , something like that . For the next version of the draft , I would n't even do that but just clean things up ( i.e. , remove these sections ) . controls ignore packets that have been repaired with coding . " . If we lay on the same logic , delay - based should not ignore these packets as a rebuilt one , following the code used and ( n , k ) length might artificially increase the e2e delay . In that case , one could argue that a rebuilt packet implicitly signal a rebuilt packet to the sender as the delay increase , but this really depends on the kind of code used and its parameters . You should look at my pull request . Let 's see if this URL survives copy+pasting : Here , this paragraph now begins with : " [ MICHAEL : I would remove this section entirely : it only adds to confusion . " :-) 
 Got all your comments from sections 6 - 7 and others , I think I see what you mean . Let 's restart first considering this comment -[MICHAEL : So far , it has n't been clear that we 're going to describe a " solution " . A solution to which problem ? I thought we 're only discussing how FEC and CC relate . ] Let 's re - discuss only how FEC and CC relate , this is what we attempted to do but may be badly . 
 So I guess the way ahead is to discuss - in the PR - what changes to the PR are desired . If I do n't hear any more details by next week , I 'll submit an update to the PR where , instead of commenting that sections 6 and 7 should be removed ( and why ) , these sections are gone . What do y' all think about the new diagram , on page 4 , and the two paragraphs just below it ? This captures how I think about things . Mainly , my take - away is that yes , there may be meaningful interaction between FEC and CC , but no , it 's not obviously a matter of sharing the " raw inputs " . Instead , e.g. ( as one example I 'm mentioning ) , unequal error protection can be an input to CC , as a result of what a fountain code produces . ( an example : chapter 8 of this Ph.D. thesis : ... yes , this is me showing off a little , trying to make the point that I 'm not COMPLETELY coding illiterate :-) only 90 % ! ) 
 There is a fundamental difference between UEP and what we are targeting here , the application of UEP is similar to    so there is no interaction at all with a CC as both work at the application layer and I think nothing really new to discuss . Furthermore , UEP needs information about the flow data packet itself which is not our case as we are flow agnostic . In brief , this draft does not tackle    . The problem is the use of FEC , RLNC , Tetrys or Sliding Window Code ( RFC8680 and 81 ) below or inside the transport layer . The diagram page 4 shows such    application ( BTW I would change " information about repaired block " by " information about repaired ratio " to be generic and compliant with SW codes ) . Consider that you reverse both blocks FEC and CC in your figure , do you see what happens in that case ? 
 This is getting more interesting . About UEP : interaction at all with a CC as both work at the application layer and I think nothing really new to discuss I think that UEP can assign different priorities to blocks of data , such that they can be more or less important , which could lead to a CC decision to treat some of them with LBE , for example ? ( RFC8680 and 81 ) below or inside the transport layer . while I 'm admittedly thinking " above " . But I placed them next to each other to avoid having to make a layering decision , as it 's about separating functionality more than it is about which one is on top of which ( which strikes me as more of an implementation decision and less conceptual ) . " information about repaired block " by " information about repaired ratio " to be generic and compliant with SW codes ACK , will do ! About the diagram on page 4 : both blocks FEC and CC in your figure , do you see what happens in that case ? No , I do n't get it - the idea was to show them side by side so they do n't convey a sequence of operation . So in my interpretation , changing their position wo n't change anything . Or what do you mean ? 
 I think I get a good take - away from this conversation : I keep talking about separating functionality , you keep talking about layering . And , I think we all agree that we want a section discussing implementation possibilities . So , my proposal is to discuss layering there , regarding the transport layer where CC is usually done . 1 ) FEC above transport , 2 ) FEC inside transport , 3 ) FEC below transport . Then we can have a discussion about various issues in this section , but this way it 's a clean framework for such text . 
 I like your proposition of layering . I think that the problem of the first version of the draft was that we tried to consider everything at once . This should be a lot more clear with such a séparation . 
 Great ! I wanted to do it now but I 'm running out of time ... early next week , promise ! 
 Le 17/01/2020 à 09:22 , mwelzl a écrit : Perfect ! I think we start to converge . 
 Sorry for my late answer - I was sick at the end of the week and could hardly manage all I wanted to deal with .. I caught up with the discussions and think there are some things that we all agree on . I will then merge the pull request so that we can separately work on the document . 
 I have tried to clean and reorganize the document based on what I think we agree on . See PR Let me know what you think . In case you agree , I guess I can merge it with master so that we have a new basis to go on with . 
 Once PR is merged to master , I propose to close this issue . 
 GREAT , thanks ! ! !
Coding is a reliability mechanism that is distinct and separated from
the loss detection of congestion controls.  Using coding can be a
useful way to better deal with tail losses or with networks with non-
congestion losses.  However, coding mechanisms should not hide
congestion signals.  This memo offers a discussion of how coding and
congestion control interact.
This document represents the collaborative work and consensus of the
Coding for Efficient Network Communications Research Group (NWCRG);
it is not an IETF product and is not a standard.
There are cases where deploying coding improves the quality of the
transmission.  As an example, the server may hardly detect tail
losses that impact may impact the application layer [MICHAEL: I don't
understand this.  Why would it not detect them?  And why is detecting
them the problem?].  Another example are networks where non-
congestion losses are persistent and prevent a sender from exploiting
the link capacity.  RFC5681 defines TCP as a loss-based congestion
control; because coding repairs such losses, blindly applying it may
easily lead to an implementation that also hides a congestion signal
to the sender.  It is important to ensure that such information
hiding does not occur.
Coding is a reliability mechanism that is distinct and separated from
the loss detection of congestion controls.  This memo discusses
simple best practices on how coding and congestion control mechanisms
could coexist.  One objective is to help the research community
consider congestion control aspects when proposing coding solutions.
The proposed recommendations apply for coding at the transport or
application layer.  Coding for tunnels is out of scope for the
document.
fig:sep-channel presents the notations that will be used in this
document and introduce the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries data packets (from a server to a client) and a potential
information signaling the packets that have been received (from the
client to the server).  The Forward Erasure Correction channel
carries coded packets (from the server to the client) and a
potiential information signaling the packets that have been repaired
(from the client to the server).  It is worth pointing out that there
are cases where these channels are not separated.
Coding is a reliability mechanism that is distinct and separated from
the loss detection of congestion controls.  Using coding can be a
useful way to better deal with tail losses or with networks with non-
congestion losses.  However, coding mechanisms should not hide
congestion signals.  This memo offers a discussion of how coding and
congestion control can coexist.
There are cases where deploying coding improves the quality of the
transmission.  As an example, it may take time for the server to
detect tail losses and this would impact the experience of
applications using short flows.  Another example are networks where
non-congestion losses are persistent and prevent a sender from
exploiting the link capacity.
Coding is a reliability mechanism that is distinct and separated from
the loss detection of congestion controls.  RFC5681 defines TCP as a
loss-based congestion control; because coding repairs such losses,
blindly applying it may easily lead to an implementation that also
hides a congestion signal to the sender.  It is important to ensure
that such information hiding does not occur.
This memo offers a discussion of how coding and congestion control
can coexist.  One objective is to help the research community
consider congestion control aspects when proposing coding solutions.
This document represents the collaborative work and consensus of the
Coding for Efficient Network Communications Research Group (NWCRG);
it is not an IETF product and is not a standard.  The proposed
recommendations apply for coding at the transport or application
layer.  Coding for tunnels is out of scope for the document.
fig:sep-channel presents the notations that will be used in this
document and introduce the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries data packets (from a server to a client) and potential
information signaling the packets that have been received (from the
client to the server).  The Forward Erasure Correction channel
carries coded packets (from the server to the client) and a
potiential information signaling the packets that have been repaired
(from the client to the server).  It is worth pointing out that there
are cases where these channels are not separated.
As this diagram shows, the inputs to FEC (data to work upon, and
signaling from the receiver about losses and/or repaired blocks) are
distinct from the inputs to CC.  The latter calculates a sending rate
or window from network measurements, and it takes the data to send as
input, sometimes along with application requirements such as upper/
lower rate bounds, periods of quiescence, or a priority.
fig:sep-entities shows a example where FEC is on top of the CC, but
interactions between the FEC and CC are possible.  In this example,
the inputs to FEC (data to work upon, and signaling from the receiver
about losses and/or repaired blocks) are distinct from the inputs to
CC.  The latter calculates a sending rate or window from network
measurements, and it takes the data to send as input, sometimes along
with application requirements such as upper/lower rate bounds,
periods of quiescence, or a priority.
It is not clear that the ACK signals feeding into a congestion
control algorithm are useful to FEC in their raw form, and vice versa
- information about repaired blocks may be quite irrelevant to a CC
algorithm.  However, there can be meaningful other interactions
(indicated by the horizontal double arrow) between the two entities,
usually as a result of their operation rather than by relaying their
own raw inputs.  For example, the network measurements carried out by
CC can yield a longer-term statistical measure such as a loss ratio
which is useful input for a FEC coding scheme.  Similarly, unequal
error protection using fountain codes can be used to assign different
priorities to blocks of data, and these priorities can be honored by
a CC mechanism.
It is not clear that the ACK signals feeding into a congestion
control algorithm are useful to FEC in their raw form, and vice versa
- information about repaired blocks may be quite irrelevant to a CC
algorithm.  However, there can be meaningful other interactions
(indicated by the horizontal double arrow) between the two entities,
usually as a result of their operation rather than by relaying their
own raw inputs.
The client indicates to the server that one or multiple packets
have been repaired using a coding scheme.  The "repaired packet"
signal does not guarantee that the packet actually needed to be
repaired, since it could have been delayed but not lost.
The server must be able to detect the "repaired packet" signal.
The base solution does not describe how the congestion control
reacts to such signal.
The rationale behind this solution is the following: the server has
more information on the congestion status and the application
characteristics.  Moreover, congestion control should not be split
between multiple entities, as otherwise non-optimized decisions may
be taken.
The proposed solution applies for coding in the transport and
application layers.  The proposed approach is in-line with the one in
I-D.swett-nwcrg-coding-for-quic.  The proposed solution does not
apply for the interaction between coding under the transport layer
(i.e. not end-to-end), such as coding for tunnels.
This section presents solutions for a server to add application
coding.
In this solution, the coded packets are sent on top of what is
allowed by a congestion window.  Examples of the solution could be
adding a given percentage of the congestion window as supplementary
packets or sending a given amount of coded packets at a given rate.
The redundancy flow can be decorrelated from the congestion control
that manages source packets: a secondary congestion control can be
introduced, such as in coupled congestion control for RTP media I-
D.ietf-rmcat-coupled-cc.  An example would be to exploit a lower than
best-effort congestion control RFC6297.
The advantage of such a solution is that coding would help in
challenging cases where transmission losses are persistent.
The drawback of such a solution is that it may result in coding
solutions being unfair towards non-coding solutions.  This solutions
may result in adding congestion in congested networks.
In this solution, the coded packets are sent within what the
congestion window allows, such as in CTCP.  Examples of the solution
would be sending coded packets when there is no more data to transmit
or preferably send coded packets instead of the following packets in
the send buffer.
The advantage of this solution is that it does not add more traffic
than the congestion window allows.  Indeed, all traffic (source and
redundancy) is controlled by one congestion control only and TCP
metrics for fairness can be indifferently applied in this case.
The main drawback is the decrease of goodput if coded packets are
sent but are not used at the client side.
The advantages of the solution are (1) that coding mechanisms do not
hide congestion signals, such as packets voluntary dropped by a AQM
RFC7567 and (2) packets may be repaired faster than with a
traditional retransmission mechanisms.
The drawback of this solution is that, if there is a high non-
congestion loss rate, the congestion control throughput may decrease
drastically.  Reporting this amount of loss to a server may reduce
the application goodput when there is no actual congestion.
The advantage of the solution is that in cases where there is no
actual congestion, coding could help in improving the transmission
without ignoring congestion signals.
The main drawback is the precised design of the solution and its
interaction with AQM mechanisms RFC7567.  Moreover there may be
fairness issues since AIMD convergence may not be guaranteed.
The advantage of this solution is that coding would provided
substantial benefits in cases where there are transmission losses.
However, this solution hides potential congestion losses, making it
unfair to other congestion controls.
fig:fec-above illustrates the FEC above the transport case.
fig:fec-above presents an architecture where FEC is on top of the
transport.  The coded packets are sent within what the congestion
window allows.
fig:fec-in illustrates the FEC within the transport case.
fig:fec-in presents an architecture where FEC is within the
transport.  The coded packets are sent within what the congestion
window allows, such as in CTCP.  Examples of the solution would be
sending coded packets when there is no more data to transmit or
preferably send coded packets instead of the following packets in the
send buffer.
fig:fec-below illustrates the FEC below the transport case.
fig:fec-below presents an architecture where FEC is below the
transport.  The coded packets are sent on top of what is allowed by
the congestion control.  Examples of the solution could be adding a
given percentage of the congestion window as supplementary packets or
sending a given amount of coded packets at a given rate.  The
redundancy flow can be decorrelated from the congestion control that
manages source packets: a secondary congestion control could be
introduced underneath the FEC layer, such as in coupled congestion
control for RTP media I-D.ietf-rmcat-coupled-cc.  An example would be
to exploit a lower than best-effort congestion control RFC6297.