Draft: draft-irtf-nwcrg-coding-and-congestion-07

Commentary: Thank you very much - this helps a lot ! 
 Some sentences are not clear and the document needs to be re written in many parts .
Forward Erasure Correction (FEC) is a reliability mechanism that is
distinct and separate from the retransmission logic in reliable
transfer protocols such as TCP.  Using FEC coding can help deal with
losses at the end of transfers or with networks having non-congestion
losses.  However, FEC coding mechanisms should not hide congestion
signals.  This memo offers a discussion of how FEC coding and
congestion control can coexist.  Another objective is to encourage
the research community to also consider congestion control aspects
when proposing and comparing FEC coding solutions in communication
systems.
Forward Erasure Correction (FEC) is a reliability mechanism that is
distinct and separate from the retransmission logic in reliable
transfer protocols such as TCP.  FEC coding can help deal with losses
at the end of transfers or with networks having non-congestion
losses.  However, FEC coding mechanisms should not hide congestion
signals.  This memo offers a discussion of how FEC coding and
congestion control can coexist.  Another objective is to encourage
the research community to also consider congestion control aspects
when proposing and comparing FEC coding solutions in communication
systems.
There are cases where deploying FEC coding improves the performance
of a transmission.  As an example, it may take time for the sender to
detect transfer tail losses (losses that occur at the end of a
transfer, where e.g., TCP obtains no more ACKs to repair the loss via
retransmission quickly).  Allowing the receiver to recover such
losses instead of having to rely on a retransmission could improve
the experience of applications using short flows.  Another example is
a network where non-congestion losses are persistent and prevent a
sender from exploiting the link capacity.
Coding is a reliability mechanism that is distinct and separate from
the loss detection of congestion controls.  RFC5681 defines TCP as a
loss-based congestion control; since FEC coding repairs such losses,
blindly applying it may easily lead to an implementation that also
hides a congestion signal from the sender.  It is important to ensure
that such information hiding does not occur.
There are cases where deploying FEC coding improves the performance
of a transmission.  As an example, it may take time for a sender to
detect transfer tail losses (losses that occur at the end of a
transfer, where, e.g., TCP obtains no more ACKs that would enable it
to quickly repair the loss via retransmission).  Allowing the
receiver to recover such losses instead of having to rely on a
retransmission could improve the experience of applications using
short flows.  Another example is a network where non-congestion
losses are persistent and prevent a sender from exploiting the link
capacity.
Coding is a reliability mechanism that is distinct and separate from
the loss detection of congestion controls.  RFC5681 defines the loss-
based congestion control of TCP; since FEC coding repairs such
losses, blindly applying it may easily lead to an implementation that
also hides a congestion signal from the sender.  It is important to
ensure that such information hiding does not occur.
We consider an end-to-end unicast data transfer with FEC coding at
the application (above the transport), within the transport or
directly below the transport.  The typical application scenario
considered in the current version of the document is a client
browsing the web or watching a live video.
We consider an end-to-end unicast data transfer with FEC coding in
the application (above the transport), within the transport or
directly below the transport.  A typical scenario for the
considerations in this document is a client browsing the web or
watching a live video.
fig:sep-channel presents the notations that will be used in this
document and introduces the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries source packets from a sender to a receiver, and packets
signaling information about the network (number of packets received
vs. lost, ECN marks, etc.) from the receiver to the sender.  The
Forward Erasure Correction channel carries repair symbols (from the
sender to the receiver) and information from the receiver to the
sender (e.g. signaling which packets have been repaired, loss rate
prior and/or after decoding, etc.).  There are cases where these
channels are not separated.
fig:sep-channel presents the notations that will be used in this
document and introduces the Congestion Control (CC) and Forward
Erasure Correction (FEC) channels.  The Congestion Control channel
carries source packets from a sender to a receiver, and packets
signaling information about the network (number of packets received
vs. lost, Explicit Congestion Notification (ECN) marks, etc.) from
the receiver to the sender.  The Forward Erasure Correction channel
carries repair symbols (from the sender to the receiver) and
information from the receiver to the sender (e.g. signaling which
packets have been repaired, loss rate prior and/or after decoding,
etc.).
In the case of an unreliable data transfer, the transport layer
may provide a non-reliable transport service (e.g.  UDP or DCCP
RFC4340 or a partially reliable transport protocol such as SCTP
with partial reliability RFC3758) or QUIC with the unreliable
datagram extension I-D.ietf-quic-datagram.  Depending on the
amount of redundancy and network conditions, there could be cases
where it becomes impossible to carry traffic.
In the case of a reliable data transfer, the transport layer may
implement a retransmission mechanism to guarantee the reliability
of the file transfer (e.g.  TCP).  Depending on how the FEC and CC
functions are scheduled (FEC above CC, FEC in CC, FEC below CC),
the impact of reliable transport on the FEC reliability mechanisms
is different.
The application layer may be composed of several streams above each
FEC and transport layers instances.  The different streams could
exploit different paths between the sender and the receiver or not.
The document considers one application layer stream as input packets
above a combination of FEC and transport layers.  The case of
multiple application level streams above multiple FEC and transport
layers instances is currently out of the scope of the document and
not further described.
End users may share a bottleneck that may not be ruled by a quality
of service mechanism that should ensure the fairness between the
different flows.  In this case, the share of available capacity
between single flows can help assess when one flow starves the other.
As one example, for residential accesses, the data-rate can be
guaranteed for the customer premises equipment, but not necessarily
for the end user.  The quality of service that guarantees fairness
between the different clients can be seen as a policy concern I-
D.briscoe-tsvarea-fair.
While past efforts have focused on achieving fairness, quantifying
and limiting harm caused by new algorithm (or algorithms with coding)
is more practical BEYONDJAIN.  This document considers fairness as
the impact of the addition of coded flows on non-coded flows when
they share the same bottleneck.
This document assumes that the non-coded flows respond to congestion
signals from the network.  This document does not aim at contributing
to the definition of fairness at a wider scale.
fig:fec-above present an architecture where FEC operates on top of
the transport.
The advantage of this approach is that the FEC overhead does not
contribute to congestion in the network.  When congestion control is
implemented at the transport layer, the repair symbols are sent
following the congestion window.  This approach can result in
improved quality of experience for latency sensitive applications
such as VoIP or any not-fully reliable services.
This approach requires that the transport protocol does not implement
a fully reliable data transfer service (e.g., based on lost packet
retransmission).  QUIC with unreliable datagram extension I-D.ietf-
quic-datagram is an example of a protocol for which this approach is
relevant.  In cases where QUIC traffic is blocked and a fallback to
TCP mechanism is proposed, there is a risk for bad interactions
between the TCP reliability mechanisms and coding schemes.  For
reliable transfers, coding usage does not guarantee better
performance and would mainly reduce goodput for large file transfers.
Moreover, the recovered symbols may not be known to the transport.
The addition of coding within the flow does not impact on the
interaction between coded and non-coded flows.  This interaction
would mainly depend on the congestion controls embedded in each host.
The congestion control may not be able to differentiate repair
symbols from actual source packets.  The relevance of adding coding
at the application layer is related to the needs of the application.
For real-time applications, this approach may reduce the number of
retransmissions.  The usage of a non-reliable transport is more
adequate in this case.
The coding rate applied at the application layer mainly depends on
the available capacity given by the congestion control underneath.
The coding rate could be adapted to avoid adding overhead when the
minimum required data rate of the application is not provided by the
congestion control underneath.  When it is not the case, coding would
reduce packet losses and improve the quality of experience.
The discussion depends on application needs.  The only case where
adding useless repair symbols does not obviously result in reduced
goodput is when the application needs a limited amount of goodput
(e.g., VoIP traffic).  In this case, the useless repair symbols would
only impact the amount of data generated in the network.  Extra data
in the network can, however, increase the likelihood of increasing
delay and/or packet loss, which could provoke a congestion control
reaction that would degrade goodput.
Whether the transport protocol includes a reordering mechanism or
not, the FEC mechanism does not require to implement a reordering
mechanism if the application does not require it.  However, if the
application requires in-order delivery of packets, a reordering
mechanism at the client is required.
The application may require partial reliability only.  In this case,
the coding rates of the FEC mechanisms could be adapted accordingly
based on inputs of the application and the trade-off between latency
and packet loss.  Partial reliability impacts the type of FEC and
type of codec that can be used.
The transport layer may provide an unreliable transport service
(e.g.  UDP or DCCP RFC4340) or a partially reliable transport
service (e.g.  SCTP with the partial reliability extension RFC3758
or QUIC with the unreliable datagram extension I-D.ietf-quic-
datagram).  Depending on the amount of redundancy and network
conditions, there could be cases where it becomes impossible to
carry traffic.
The transport layer may implement a retransmission mechanism to
guarantee the reliability of a data transfer (e.g.  TCP).
Depending on how the FEC and CC functions are scheduled (FEC above
CC, FEC in CC, FEC below CC), the impact of reliable transport on
the FEC reliability mechanisms is different.
The application layer may be composed of several streams above FEC
and transport layers instances.  The different streams could exploit
different paths between the sender and the receiver or not.  The
document considers one application layer stream as input packets
above a combination of FEC and transport layers.  The case of
multiple application level streams above multiple FEC and transport
layers instances is out of the scope of the document and not further
described.
Traffic from or to different end users may share various types of
bottlenecks.  When such a shared bottleneck does not implement some
form of flow protection, the share of the available capacity between
single flows can help assess when one flow starves the other.
As one example, for residential accesses, the data rate can be
guaranteed for the customer premises equipment, but not necessarily
for the end user.  The quality of service that guarantees fairness
between the different clients can be seen as a policy concern I-
D.briscoe-tsvarea-fair.
While past efforts have focused on achieving fairness, quantifying
and limiting harm caused by new algorithms (or algorithms with
coding) is more practical BEYONDJAIN.  This document considers
fairness as the impact of the addition of coded flows on non-coded
flows when they share the same bottleneck.  It is assumed that the
non-coded flows respond to congestion signals from the network.  This
document does not contribute to the definition of fairness at a wider
scale.
fig:fec-above presents an architecture where FEC operates on top of
the transport.
The advantage of this approach is that the FEC overhead does not
contribute to congestion in the network.  When congestion control is
implemented at the transport layer, the repair symbols are sent
following the congestion window or rate determined by the CC
mechanism.  This can result in improved quality of experience for
latency sensitive applications such as VoIP or any not-fully reliable
services.
This approach requires that the transport protocol does not implement
a fully reliable data transfer service (e.g., based on lost packet
retransmission).  QUIC with unreliable datagram extension I-D.ietf-
quic-datagram is an example of a protocol for which this is relevant.
In cases where QUIC traffic is blocked and a fall-back to TCP is
proposed, there is a risk for bad interactions between TCP's full
reliability and coding schemes.  For reliable transfers, coding usage
does not guarantee better performance; instead, it would mainly
reduce goodput.
The addition of coding within the flow does not influence the
interaction between coded and non-coded flows.  This interaction
would mainly depend on the congestion controls associated with each
flow.
The congestion control mechanism may not be able to differentiate
repair symbols from actual source packets.  The relevance of adding
coding at the application layer is related to the needs of the
application.  For real-time applications using an unreliable or
partially reliable transport, this approach may reduce the number of
losses perceived by the application.
The coding rate applied at the application layer mainly depends on
the available rate or congestion window given by the congestion
control underneath.  The coding rate could be adapted to avoid adding
overhead when the minimum required data rate of the application is
not provided by the congestion control underneath.  When the
congestion control allows sending faster than the application needs,
adding coding can reduce packet losses and improve the quality of
experience (provided that an unreliable or partially reliable
transport is used).
The discussion depends on application needs.  The only case where
adding useless repair symbols does not obviously result in reduced
goodput is when the application rate is limited (e.g., VoIP traffic).
In this case, useless repair symbols would only impact the amount of
data generated in the network.  Extra data in the network can,
however, increase the likelihood of increasing delay and/or packet
loss, which could provoke a congestion control reaction that would
degrade goodput.
Irrespective of the transport protocol, a FEC mechanism does not
require to implement a reordering mechanism if the application does
not need it.  However, if the application needs in-order delivery of
packets, a reordering mechanism at the receiver is required.
The application may require partial reliability.  In this case, the
coding rate of a FEC mechanism could be adapted based on inputs from
the application and the trade-off between latency and packet loss.
Partial reliability impacts the type of FEC and type of codec that
can be used.
fig:fec-in presents an architecture where FEC operates within the
transport.  The repair symbols are sent within what the congestion
window allows, such as in CTCP.
The advantage of this approach allows a joint optimization of the CC
and the FEC.  Moreover, the transmission of repair symbols does not
add congestion in potentially congested networks but helps repair
lost packets (such as tail losses).
For reliable transfers, including redundancy reduces goodput for
large file transfers but the amount of repair symbols can be adapted,
e.g. depending on the congestion window size.  There is a trade-off
between1) the capacity that could have been exploited to transmit
source packets and 2) the benefits brought out by transmitting repair
symbols (e.g. unlocking the receive buffer if this is limiting).  The
coding ratio needs to be carefully designed.  For small files,
sending repair symbols when there is no more data to transmit could
help to reduce the transfer time.  In general, sending repair symbols
could avoid a silent period between the transmission of the last
packet in the send buffer and 1) firing the retransmission of lost
packets, or 2) the transmission of new packets.
The addition of coding within the flow may impact the congestion
control mechanism and hide congestion losses.  Specific interaction
between congestion controls and coding schemes can be proposed (see
subsec:cc-recov-interaction_in, subsec:cc-nc-interaction_in and
subsec:cc-useless-interaction_in).  If no specific interaction is
introduced, the coding scheme may hide congestion losses from the
congestion controller and the description of sec:fec-below may apply.
fig:fec-in presents an architecture where FEC operates within the
transport.  The repair symbols are sent within what the congestion
window or calculated rate allows, such as in CTCP.
The advantage of this approach is that it allows a joint optimization
of CC and FEC.  Moreover, the transmission of repair symbols does not
add congestion in potentially congested networks but helps repair
lost packets (such as tail losses).
For reliable transfers, including redundancy reduces goodput for long
transfers but the amount of repair symbols can be adapted, e.g.
depending on the congestion window size.  There is a trade-off
between 1) the capacity that could have been exploited by application
data instead of transmitting source packets, and 2) the benefits
derived from transmitting repair symbols (e.g. unlocking the receive
buffer if it is limiting).  The coding ratio needs to be carefully
designed.  For small files, sending repair symbols when there is no
more data to transmit could help to reduce the transfer time.
Sending repair symbols can avoid the silence period between the
transmission of the last packet in the send buffer and 1) firing a
retransmission of lost packets, or 2) the transmission of new
packets.
The addition of coding within the transport may impact the congestion
control mechanism and hide congestion losses.  Specific interaction
between congestion controls and coding schemes can be proposed (see
subsec:cc-recov-interaction_in, subsec:cc-nc-interaction_in and
subsec:cc-useless-interaction_in).  If no specific interaction is
introduced, the coding scheme may hide congestion losses from the
congestion controller and the description of sec:fec-below may apply.
The receiver can differentiate source packets and repair symbols.
The receiver may indicate both the number of source packets received
and repair symbols that were actually useful in the recovery process
of packets.
There is an important flexibility in the trade-off, inherent to the
use of coding, between (1) reducing goodput when useless repair
symbols are transmitted and (2) helping to recover sooner from
transmission and congestion losses.  The receiver may indicate to the
sender the number of packets that have been received or recovered.
The sender may exploit this information to tune the coding ratio.  As
one example of flexibility of this case, coupling an increased
transmission rate with an increasing or decreasing coding rate could
be envisioned.  A server may use an decreasing coding rate as a probe
of the channel capacity and adapt the congestion control transmission
rate.
The receiver can differentiate between source packets and repair
symbols.  The receiver may indicate both the number of source packets
received and repair symbols that were actually useful in the recovery
process of packets.
There is an important flexibility in the trade-off, inherent to the
use of coding, between (1) reducing goodput when useless repair
symbols are transmitted and (2) helping to recover from losses
earlier than with retransmissions.  The receiver may indicate to the
sender the number of packets that have been received or recovered.
The sender may use this information to tune the coding ratio.  For
example, coupling an increased transmission rate with an increasing
or decreasing coding rate could be envisioned.  A server may use a
decreasing coding rate as a probe of the channel capacity and adapt
the congestion control transmission rate.
The application may require in-order delivery of packets.  In this
case, both FEC and transport layer mechanisms should guarantee that
packets are delivered in order.  If partial ordering is requested by
the application, both the FEC and transport could release the
constraints related to in-order delivery of packets : reordering
mechanisms at the receiver may not be necessary.
The application may require partial reliability.  In this case, the
transport and FEC mechanisms could be conjointly designed.  As one
example, the reliability offered by FEC may be sufficient and no
retransmission required.  This depends on application requirements
and the trade-off between latency and loss.  Partial reliability
impacts the type of FEC and type of codec that can be used.
The application may require in-order delivery of packets.  In this
case, both FEC and transport layer mechanisms should guarantee that
packets are delivered in order.  If partial ordering is requested by
the application, both the FEC and transport could relax the
constraints related to in-order delivery: reordering mechanisms at
the receiver may not be necessary.
The application may require partial reliability.  In this case, the
transport and FEC mechanisms could be conjointly designed.  As one
example, the reliability offered by FEC may be sufficient, with no
retransmission required.  This depends on application needs and the
trade-off between latency and loss.  Partial reliability impacts the
type of FEC and type of codec that can be used.
fig:fec-below presents an architecture where FEC is applied end-to-
end below the transport layer, but above the link layer.  Note that
it is common to apply FEC at the link layer, in which it contributes
to the total capacity that a link exposes to upper layers.  This
application of FEC is out of scope of this document.  In the scenario
considered here, the repair symbols are sent on top of what is
allowed by the congestion control.
Including redundancy adds traffic without reducing goodput but leads
to potential fairness issues.  The effective bitrate is indeed higher
than the CC's computed fair share due to the sending of repair
symbols and the losses are hidden from the transport.  This may cause
a problem for loss-based congestion detection, but it is not a
problem for delay-based congestion detection.
fig:fec-below presents an architecture where FEC is applied end-to-
end below the transport layer, but above the link layer.  Note that
it is common to apply FEC at the link layer, where it contributes to
the total capacity that a link exposes to upper layers.  This
application of FEC is out of scope of this document.  In the scenario
considered here, the repair symbols are sent on top of what is
allowed by the congestion control.
Including redundancy adds traffic without reducing goodput but incurs
potential fairness issues.  The effective bitrate is higher than the
CC's computed fair share due to the transmission of repair symbols,
and losses are hidden from the transport.  This may cause a problem
for loss-based congestion detection, but it is not a problem for
delay-based congestion detection.
Examples of the solution could be adding a given percentage of the
congestion window as supplementary symbols or sending a given amount
of repair symbols at a given rate.  The redundancy flow can be
decorrelated from the congestion control that manages source packets:
a separate congestion control entity could be introduced to manage
the amount of repaired packets to transmit on the FEC channel.  The
separate congestion control instances could be made to work together
while adhering to priorities, as in coupled congestion control for
RTP media RFC8699 in case all traffic can be assumed to take the same
path, or otherwise with a multipath congestion window coupling
mechanism as in Multipath TCP RFC6356.  Another possibility would be
to exploit a lower than best-effort congestion control RFC6297 for
repair symbols.
Examples of the solution could be to add a given percentage of the
congestion window or rate as supplementary symbols, or to send a
fixed amount of repair symbols at a fixed rate.  The redundancy flow
can be decorrelated from the congestion control that manages source
packets: a separate congestion control entity could be introduced to
manage the amount of repaired packets to transmit on the FEC channel.
The separate congestion control instances could be made to work
together while adhering to priorities, as in coupled congestion
control for RTP media RFC8699 in case all traffic can be assumed to
take the same path, or otherwise with a multipath congestion window
coupling mechanism as in Multipath TCP RFC6356.  Another possibility
would be to exploit a lower than best-effort congestion control
RFC6297 for repair symbols.
The congestion control may not know what is going on in the network
underneath and whether a coding scheme is introduced or not.  The
congestion control may behave as if no coding scheme is introduced.
The only way for a coding channel to indicate that symbols have been
recovered is to exploit existing signaling that is understood by the
congestion control mechanism.  An example would be to indicate to a
TCP sender that a packet has been recovered (i.e., congestion has
occurred), by using ECN signaling TENTET.
The coding rate can be tuned depending on the number of recovered
symbols and the rate at which the sender transmits data.  The coding
scheme is not aware of the congestion control implementation, making
it hard for the coding scheme to apply the relevant coding rate.
The useless repair symbols only impact the load of the network
without actual gain for the coded flow.  That being said, using
feedback signaling, FEC mechanisms can measure the actually used
symbols and adjust the coding rate.
The transport above the FEC channel may support out-of-order delivery
of packets: reordering mechanisms at the receiver may not be
necessary.  In cases where the transport requires in-order delivery,
the FEC channel may need to implement a reordering mechanism
otherwise there may be spurious retransmissions at the transport
level.
The congestion control may not be aware of the existence of a coding
scheme underneath it.  The congestion control may behave as if no
coding scheme had been introduced.  The only way for a coding channel
to indicate that symbols have been lost but recovered is to exploit
existing signaling that is understood by the congestion control
mechanism.  An example would be to indicate to a TCP sender that a
packet has been received, yet congestion has occurred, by using ECN
signaling TENTET.
The coding rate can be tuned depending on the number of recovered
symbols and the rate at which the sender transmits data.  If the
coding scheme is not aware of the congestion control implementation,
it is hard for the coding scheme to apply the relevant coding rate.
Useless repair symbols only impact the load on the network without
actual gain for the coded flow.  Using feedback signaling, FEC
mechanisms can measure the ratio between actually used and useless
symbols, and adjust the coding rate.
The transport above the FEC channel may support out-of-order delivery
of packets: reordering mechanisms at the receiver may not be
necessary.  In cases where the transport requires in-order delivery,
the FEC channel may need to implement a reordering mechanism.
Otherwise, spurious retransmissions may occur at the transport level.
For the FEC within transport case, recovering lost symbols may hide
congestion losses to the congestion control.  Some existing solutions
already propose to disambiguate acked packets from rebuilt packets
QUIC-FEC.  New signaling methods and FEC-recovery-aware congestion
controls could be proposed.
For the FEC within transport case, recovering lost symbols may hide
congestion losses from the congestion control.  Some existing
solutions already propose to disambiguate acked packets from rebuilt
packets QUIC-FEC.  New signaling methods and FEC-recovery-aware
congestion controls could be proposed.
The contribution to research questions should be mapped following the
organization of this document.  Otherwise, this may lead to wrong
assumptions on the validity of the proposal and wrong ideas about the
relevance of coding for a given use case.
Our advice is that new research contributions should be mapped
following the organization of this document.  Otherwise, this may
lead to wrong assumptions on the validity of the proposal and wrong
ideas about the relevance of coding for a given use case.