Draft: draft-irtf-nwcrg-coding-and-congestion-05

Commentary: The discussion of fairness should point to the presentation from  on " do not harm others " but still mention  mechanisms . It may be wort moving this discussion up in the document . 
    Fairness Index : Setting the Bar For The Deployment of Congestion Control Algorithms 
 Done in PR 
 The case of coding above transport should be explicit on the document that it may not be relevant since the recovered symbols would not be known to the transport . Maybe mention cases where UDP is blocked by firewalls and TCP fall back is introduced ? 
 Done in 
 Coding above transport is relevant :    is above transport . May be you mean " out of scope " of the present document ? Or    schemes are not considered here because it does not impact on CC ? 
 The text does not mention the ' relevant ' : ' < tThis approach requires that the transport protocol does not implement a fully reliable data transfer service ( e.g. , based on lost packet retransmission ) . QUIC with unreliable datagram extension < xref target="I - D.ietf - quic - datagram"</xrefis an example of a protocol for which this approach is relevant . In cases where QUIC traffic is blocked and a fallback to TCP mechanism is proposed , there is a risk for bad interactions between the TCP reliable mechanisms and coding schemes . For reliable transfers , coding usage does not guarantee better performance and would mainly reduce goodput for large file transfers . Moreover , the recovered symbols may not be known to the transport.</t ' 
 The discussion is spread in the document and should be better organized to avoid repetitions . 
 Done in PR 
 The following : I attach the label as it might require some time to rethink the content of these 2 sections but I think it is important to fix it later at some point . :-) 
  proposed in PR 
 In section 3.2 , the following paragraph : I think we should not mention UDP as it does not integrate a congestion control , so I do n't think we could consider it in our FEC - above - transport category ( maybe should we rename those categories by FEC above congestion control instead , but later ) . maybe should we mention FEC over QUIC datagrams instead : QUIC datagrams are congestion - controlled but unreliable :-) 
 I think the discussion on FEC over unreliable QUIC datagrams is very interesting to have in the document - but maybe later 
 I think the FEC above transport is more general and covers multiple aspects that could be included in the transport layer 
 I agree with Francois ' argument of not mentioning UDP - indeed it does n't fit , and QUIC with datagrams fits better . But I also agree with doing this later - so my suggestion is to leave it with UDP for now , until we introduce QuIC : it 's easier to understand with UDP being mentioned , I think . 
 Knowing there are drafts on FEC and QUIC it could be wise to leave the related discussion there .       in PR 
  made a comment on partial reliability . Should we need a new subsection 6.X to discuss this ? 
 I do n't get this :  's comment was about partial order , not partial reliability . Quote : One important thing to consider is reordering and not only reliability alone , we could also consider Partial Order Connection ( POC ) and Partial Order Transport Service ( see   works ) . Spurious retransmission is not only linked to reliability but reordering . 
 If this issue ends up being about ordering , and not partial reliability : From  ( copied over from a PR ): There is something we do not mention here concerning the reliable transport that deliver data in - order : with a reliable transport , the data are delivered in order . I can see two possible cases : Above a single TCP connection : the repair packets wo n't ever be delivered to the applications if there is a loss occurring before them . They are thus useless as they are delivered once the loss is receovered . Above multiple streams ( e.g. QUIC streams or one TCP connection for the source and one for the repair symbols ): the repair symbols are less effective : if there is a hole in the data ( ex : seq 1 and 3 are received but not 2),you will need to consider 3 a lost too as it wo n't be delivered to the upper layer . 
 An answer from me to this : there are transports that can deliver data out - of - order . SCTP can do that , and it 's exposed in the interface of  ( draft - ietf - taps - interface ) . Multi - streaming is closely related to that : if you take out - of - order message delivery and just add stream identifiers to ensure that , per stream , messages are delivered in order , you have it . This means that FEC could perhaps be applied across streams in case of a multi - streaming transport . I think this just complicates the discussion , let 's not go there in the draft . 
 Thank you for summing up the discussion here . I think we already quite an important number of new sections in this document - we may leave the topic on out - of - order packets on the side and focus on what we have at the moment . 
 Ok , but do we agree that this is about " partial ordering " , not " partial reliability " ? In this case , we should probably rename this issue to avoid further confusion . 
 Text proposed in PR
Forward Erasure Correction (FEC) is a reliability mechanism that is
distinct and separate from the retransmission logic in reliable
transfer protocols such as TCP.  Using FEC coding can help deal with
transfer tail losses or with networks having non-congestion losses.
However, FEC coding mechanisms should not hide congestion signals.
This memo offers a discussion of how FEC coding and congestion
control can coexist.  Another objective is to encourage the research
community to also consider congestion control aspects when proposing
and comparing FEC coding solutions in communication systems.
Forward Erasure Correction (FEC) is a reliability mechanism that is
distinct and separate from the retransmission logic in reliable
transfer protocols such as TCP.  Using FEC coding can help deal with
losses at the end of transfers or with networks having non-congestion
losses.  However, FEC coding mechanisms should not hide congestion
signals.  This memo offers a discussion of how FEC coding and
congestion control can coexist.  Another objective is to encourage
the research community to also consider congestion control aspects
when proposing and comparing FEC coding solutions in communication
systems.
There are cases where deploying FEC coding improves the performance
of a transmission.  As an example, it may take time for the sender to
detect transfer tail losses (losses that occur at the end of a
transfer, where e.g., TCP obtains no more ACKs to repair the loss via
retransmission quickly).  Allowing the receiver to recover such
losses instead of having to rely on a retransmission could improve
the experience of applications using short flows.  Another example
are networks where non-congestion losses are persistent and prevent a
sender from exploiting the link capacity.
There are cases where deploying FEC coding improves the performance
of a transmission.  As an example, it may take time for the sender to
detect transfer tail losses (losses that occur at the end of a
transfer, where e.g., TCP obtains no more ACKs to repair the loss via
retransmission quickly).  Allowing the receiver to recover such
losses instead of having to rely on a retransmission could improve
the experience of applications using short flows.  Another example is
a network where non-congestion losses are persistent and prevent a
sender from exploiting the link capacity.
FEC coding and congestion control can be seen as two separate
channels.  In practice, implementations may mix the signals that are
exchanged on these channels.  This memo offers a discussion of how
FEC coding and congestion control can coexist.  Another objective is
to encourage the research community also to consider congestion
control aspects when proposing and comparing FEC coding solutions in
communication systems.  This document does not aim at proposing
guidelines for characterizing FEC coding solutions.
The proposed document considers an end-to-end unicast data transfer
with FEC coding at the application (above the transport), within the
transport or directly below the transport.  The typical application
scenario considered in the current version of the document is a
client browsing the web or watching a live video.  This memo may be
extended to cases with multiple paths.
FEC coding and congestion control can be seen as two separate
channels.  In practice, implementations may mix the signals that are
exchanged on these channels.  This memo offers a discussion of how
FEC coding and congestion control coexist.  Another objective is to
encourage the research community also to consider congestion control
aspects when proposing and comparing FEC coding solutions in
communication systems.  This document does not aim at proposing
guidelines for characterizing FEC coding solutions.
We consider an end-to-end unicast data transfer with FEC coding at
the application (above the transport), within the transport or
directly below the transport.  The typical application scenario
considered in the current version of the document is a client
browsing the web or watching a live video.  This memo may be extended
to cases with multiple paths.
fig:sep-channel presents the notations that will be used in this
document and introduces the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries source packets from a sender to a receiver, and packets
signaling information about the network (number of packets received
vs. lost, ECN marks, etc.) from the receiver to the sender.  The
Forward Erasure Correction channel carries repair symbols (from the
sender to the receiver) and potential information signaling which
packets have been repaired (from the receiver to the sender).  It is
worth pointing out that there are cases where these channels are not
separated.
fig:sep-channel presents the notations that will be used in this
document and introduces the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries source packets from a sender to a receiver, and packets
signaling information about the network (number of packets received
vs. lost, ECN marks, etc.) from the receiver to the sender.  The
Forward Erasure Correction channel carries repair symbols (from the
sender to the receiver) and potential information signaling which
packets have been repaired (from the receiver to the sender).  There
are cases where these channels are not separated.
In the case of an unreliable data transfer, the transport layer
may provide a non-reliable transport service (e.g.  UDP or DCCP
RFC4340 or a partially reliable transport protocol such as SCTP
with partial reliability RFC3758).  Depending on the amount of
redundancy and network conditions, there could be cases where it
becomes impossible to carry traffic.
In the case of an unreliable data transfer, the transport layer
may provide a non-reliable transport service (e.g.  UDP or DCCP
RFC4340 or a partially reliable transport protocol such as SCTP
with partial reliability RFC3758) or QUIC with the unreliable
datagram extension I-D.ietf-quic-datagram.  Depending on the
amount of redundancy and network conditions, there could be cases
where it becomes impossible to carry traffic.
The advantage of this approach is that the FEC overhead does not
contribute to congestion in the network.  When congestion control is
implemented at the transport layer, the repair symbols are sent
following the congestion window.  This approach can result in
improved quality of experience for latency sensitive applications
such as VoIP.
This approach requires that the transport protocol does not implement
a fully reliable data transfer service (e.g., based on lost packet
retransmission).  UDP is an example of a protocol for which this
approach is relevant.  For reliable transfers, coding usage does not
guarantee better performance and would mainly reduce goodput for
large file transfers.
The advantage of this approach allows a joint optimization between
the CC and the FEC.  Moreover, the transmission of repair symbols
does not add congestion in potentially congested networks but helps
repair lost packets (such as tail losses).
For reliable transfers, including redundancy reduces goodput for
large file transfers but the amount of repair symbols can be adapted,
e.g. depending on the congestion window size.  There is a trade-off
between the cost in capacity used to transmit source packets and the
benefits brought out by transmitting repair symbols (e.g. unlocking
the receive buffer if this is limiting).  The coding ratio needs to
be carefully designed.  For small files, sending repair symbols when
there is no more data to transmit could help to reduce the transfer
time.  In general, sending repair symbols could avoid a silent period
between the transmission of the last packet in the send buffer and 1)
firing the retransmission of lost packets, or 2) the transmission of
new packets.
The advantage of this approach is that the FEC overhead does not
contribute to congestion in the network.  When congestion control is
implemented at the transport layer, the repair symbols are sent
following the congestion window.  This approach can result in
improved quality of experience for latency sensitive applications
such as VoIP or any not-fully reliable services.
This approach requires that the transport protocol does not implement
a fully reliable data transfer service (e.g., based on lost packet
retransmission).  QUIC with unreliable datagram extension I-D.ietf-
quic-datagram is an example of a protocol for which this approach is
relevant.  In cases where QUIC traffic is blocked and a fallback to
TCP mechanism is proposed, there is a risk for bad interactions
between the TCP reliability mechanisms and coding schemes.  For
reliable transfers, coding usage does not guarantee better
performance and would mainly reduce goodput for large file transfers.
Moreover, the recovered symbols may not be known to the transport.
The discussion depends on application needs.  The only case where
adding useless repair symbols does not obviously result in reduced
goodput is when the application needs a limited amount of goodput
(e.g., VoIP traffic).  In this case, the useless repair symbols would
only impact the amount of data generated in the network.  Extra data
in the network can, however, increase the likelihood of increasing
delay and/or packet loss, which could provoke a congestion control
reaction that would degrade goodput.
Whether the transport protocol includes a reordering mechanism or
not, the FEC mechanism does not require to implement a reordering
mechanism if the application does not require it.  However, if the
application requires in-order delivery of packets, a reordering
mechanism at the client is required.
The application may require partial reliability only.  In this case,
the coding rates of the FEC mechanisms could be adapted accordingly
based on inputs of the application and the trade-off between latency
and packet loss.
The advantage of this approach allows a joint optimization of the CC
and the FEC.  Moreover, the transmission of repair symbols does not
add congestion in potentially congested networks but helps repair
lost packets (such as tail losses).
For reliable transfers, including redundancy reduces goodput for
large file transfers but the amount of repair symbols can be adapted,
e.g. depending on the congestion window size.  There is a trade-off
between the cost of capacity used to transmit source packets and the
benefits brought out by transmitting repair symbols (e.g. unlocking
the receive buffer if this is limiting).  The coding ratio needs to
be carefully designed.  For small files, sending repair symbols when
there is no more data to transmit could help to reduce the transfer
time.  In general, sending repair symbols could avoid a silent period
between the transmission of the last packet in the send buffer and 1)
firing the retransmission of lost packets, or 2) the transmission of
new packets.
The addition of coding within the flow may impact the congestion
control mechanism and hide congestion losses.  Specific interaction
between congestion controls and coding schemes can be proposed (see
subsec:cc-recov-interaction_in, subsec:cc-nc-interaction_in and
subsec:cc-useless-interaction_in).  If no specific interaction is
introduced, the coding scheme may hide congestion losses from the
congestion controller and the description of sec:fec-below may apply.
If the FEC and CC channels are decoupled, the endpoint may exploit
different protocols for each channel.  The channels may be coupled
and one single protocol may be exploited.  In both cases, the
receiver can differentiate source packets and repair symbols.  The
receiver may indicate both the number of source packets received and
repair symbols that were actually useful in the recovery process of
packets.
There is an important flexibility in the trade-off, inherent to the
use of coding, between (1) reducing goodput when useless repair
symbols are transmitted and (2) helping to recover sooner from
transmission and congestion losses.  The receiver may indicate to the
sender the number of packets that have been received or recovered.
The sender may exploit this information to tune the coding ratio.  As
one example of flexibility of this case, coupling an increased
transmission rate with an increasing or decreasing coding rate could
be envisioned.  A server may use an increasing coding rate as a probe
of the channel capacity and adapt the congestion control transmission
rate.
The application may require in-order delivery of packets.  In this
case, both FEC and transport layer mechanisms should guarantee that
packets are delivered in order.  If partial ordering is requested by
the application, both the FEC and transport could release the
constraints related to in-order delivery of packets : reordering
mechanisms at the receiver may not be necessary.
The application may require partial reliability.  In this case, the
transport and FEC mechanisms could be conjointly designed.  As one
example, the reliability offered by FEC may be sufficient and no
retransmission required.  This depends on application requirements
and the trade-off between latency and loss.
In this case, including redundancy adds congestion without reducing
goodput but leads to potential fairness issues.  The effective
bitrate is indeed higher than the CC's computed fair share due to the
sending of repair symbols and the losses are hidden from the
transport.  This may cause a problem for loss-based congestion
detection, but it is not a problem for delay-based congestion
detection.
Including redundancy adds traffic without reducing goodput but leads
to potential fairness issues.  The effective bitrate is indeed higher
than the CC's computed fair share due to the sending of repair
symbols and the losses are hidden from the transport.  This may cause
a problem for loss-based congestion detection, but it is not a
problem for delay-based congestion detection.
The objective of this section is to further detail some aspects that
have been expressed in previous discussion subsections.
The addition of coding within the flow may impact the congestion
control mechanism and hide congestion losses.  Specific interaction
between congestion controls and coding schemes can be proposed (see
subsec:cc-recov-interaction, subsec:cc-nc-interaction and subsec:cc-
useless-interaction).  If no specific interaction is introduced, the
coding scheme may hide congestion losses from the congestion
controller and the description of subsub:fec_below may apply.
In this case, the coding scheme may hide congestion losses from the
congestion controller.  There are cases where this can drastically
reduce the goodput of non-coded flows.  Depending on the congestion
control, it may be possible to signal to the congestion control
mechanism that there was congestion (loss) even when a packet has
been recovered, e.g. using ECN, to reduce the impact on the non-coded
flows (see subsec:cc-recov-inter-below and TENTET).
The objective of this subsection is to describe potential
interactions between the congestion control and the recovered
symbols.
If the two FEC and CC channels are decoupled, the endpoint may
exploit different protocols for each channel.  The channels may be
coupled and one single protocol may be exploited.  In both cases, the
receiver can differentiate source packets and repair symbols.  The
receiver may indicate both the number of source packets received and
repair symbols that were actually useful in the recovery process of
packets.
The coding scheme may hide congestion losses from the congestion
controller.  There are cases where this can drastically reduce the
goodput of non-coded flows.  Depending on the congestion control, it
may be possible to signal to the congestion control mechanism that
there was congestion (loss) even when a packet has been recovered,
e.g. using ECN, to reduce the impact on the non-coded flows (see
subsec:cc-recov-interaction_below and TENTET).
This section discusses to what extent the interaction between the
congestion control and the coding rates is possible.
In this case, there is an important flexibility in the trade-off,
inherent to the use of coding, between (1) reducing goodput when
useless repair symbols are transmitted and (2) helping to recover
sooner from transmission and congestion losses.  As explained in
subsec:cc-recov-inter-in, the receiver may indicate to the sender the
number of packets that have been received or recovered.  The sender
may exploit this information to tune the coding ratio.  As one
example of flexibility of this case, coupling an increased
transmission rate with an increasing or decreasing coding rate could
be envisioned.  A server may use an increasing coding rate as a probe
of the channel capacity and adapt the congestion control transmission
rate.
In this case, the coding rate can be tuned depending on the number of
recovered symbols and the rate at which the sender transmits data.
The coding scheme is not aware of the congestion control
implementation, making it hard for the coding scheme to apply the
relevant coding rate.
There are cases where useless repair symbols may be transmitted.
These impact on the network load and may reduce the goodput of the
flow without concrete gains.
In this case, the discussion depends on application needs.  The only
case where adding useless repair symbols does not result in reduced
goodput is when the application needs a limited amount of goodput
(e.g., VoIP traffic).  In this case, the useless repair symbols would
only impact the amount of data generated in the network.
In this case, the useless repair symbols only impact the load of the
network without actual gain for the coded flow.
This section provides a simplified state-of-the art of the activities
related to congestion control and coding.  The objective is to
identify open research questions and contribute to advice when
evaluating coding mechanisms.
The coding rate can be tuned depending on the number of recovered
symbols and the rate at which the sender transmits data.  The coding
scheme is not aware of the congestion control implementation, making
it hard for the coding scheme to apply the relevant coding rate.
The useless repair symbols only impact the load of the network
without actual gain for the coded flow.  That being said, using
feedback signaling, FEC mechanisms can measure the actually used
symbols and tune the amount of useless repair symbols.
The transport above the FEC channel may support out-of-order delivery
of packets: reordering mechanisms at the receiver may not be
necessary.  In cases where the transport requires in-order delivery,
the FEC channel may need to implement a reordering mechanism
otherwise there may be spurious retransmissions at the transport
level.
The transport or application layer above the FEC channel may require
partial reliability only.  In this case, FEC may provide an
unnecessary service if it is not aware of the reliability
requirements.
This section provides a short state-of-the art overview of activities
related to congestion control and coding.  The objective is to
identify open research questions and contribute to advice when
evaluating coding mechanisms.
For the FEC above transport case: RFC8680.
The research questions should be mapped following the organization of
this document.  In all these three use-cases, open questions remain.
There is a general trade-off, inherent to the use of coding, between
(1) reducing goodput when useless repair symbols are transmitted and
(2) helping to recover from transmission and congestion losses.
There is a general trade-off, inherent to the use of coding, between
(1) reducing goodput when useless repair symbols are transmitted and
(2) helping to recover from transmission and congestion losses.
For the FEC within transport case, recovering lost symbols may hide
congestion losses to the congestion control.  Some existing solutions
already propose to disambiguate acked packets from rebuilt packets
QUIC-FEC.  New signalling methods and FEC-recovery-aware congestion
controls could be proposed.
For the FEC below transport case, there are opportunities for
introducing interaction between congestion control and coding schemes
to improve the quality of experience while guaranteeing fairness with
other flows.  An open question also resides in the relevance of FEC
when there are multiple streams that exploit the FEC channel.
For the FEC within transport case, recovering lost symbols may hide
congestion losses to the congestion control.  Some existing solutions
already propose to disambiguate acked packets from rebuilt packets
QUIC-FEC.  New signaling methods and FEC-recovery-aware congestion
controls could be proposed.
For the FEC below transport case, there are opportunities for
introducing interaction between congestion control and coding schemes
to improve the quality of experience while guaranteeing fairness with
other flows.  New signaling methods and FEC-recovery-aware congestion
controls could be proposed.  An open question also resides in the
relevance of FEC when there are multiple streams that exploit the FEC
channel.
The discussion provided in this document aims at encouraging the
research community to also consider congestion control aspects when
proposing and comparing FEC coding solutions in communication
systems.  As one example, this draft proposes discussions on the
impact of the proposed FEC solution on congestion control, especially
loss-based congestion control mechanisms.  When a research work aims
at improving the throughput by hiding the packet loss signal from the
congestion control, the authors should 1) discuss the advantages of
using the proposed FEC solution compared to replacing the congestion
control by one that ignores a portion of the encountered losses, 2)
critically discuss the impact of hiding packet loss from the
congestion control mechanism.
The discussion provided in this document aims at encouraging the
research community to also consider congestion control aspects when
proposing and comparing FEC coding solutions in communication
systems.  As one example, this draft proposes discussions on the
impact of the proposed FEC solution on congestion control, especially
loss-based congestion control mechanisms.  When a research work aims
at improving throughput by hiding the packet loss signal from
congestion control, the authors should 1) discuss the advantages of
using the proposed FEC solution compared to replacing the congestion
control by one that ignores a portion of the encountered losses, 2)
critically discuss the impact of hiding packet loss from the
congestion control mechanism.