Draft: draft-ietf-taps-interface

Commentary: Satisfies Support partial sends more naturally by splitting out into Message , messageData , and endOfMessage flag . Do the same for Receive Receive should also take minIncompleteLength to specify how partial receives are treated ( or not , by default ) 
 Looks like a good approach . I have a general question about setting Message Properties : This Pull Request gets rid of an explicit MessageProperties object and instead sets any Message Properties directly on the Message . For ( Pre-)Connections , we do have an explicit TransportProperties object , which has the advantage that it can be reused across multiple Connections without being tied to any specific Connection . I think that the same would be useful for Messages , e.g. , across multiple threads . What is the advantage of setting properties directly on a Message , versus having an explicit MessageProperties object , which is then assigned to the Message ? 
 - your idea sounds interesting , but from your description I do n't understand why this reduces complexity in the API . I think the complexity is related to ordering . If we assume that , on the receiver side , message fragments are only delivered in order , things become easier . Then we may only need an extra signal to tell the receiver that remaining fragments will never arrive . 
 I like the general idea and I agree having an explicit message object really is a way out , but I also think the API in this PR exposes too many details and makes the " common " case unnecessarily complicated . I like Idea to move that partial send complexity into the message object . 
 on second thought , let me +1 on " move partial send complexity into the message object " : to send a partial message , get a message and keep writing bytes to receive a partial message , get a message that indicates that the octets read from it are not the whole message . 
 Right , having Message objects be the active objects for sending and receiving is the other option . That 's the one that we had implemented back when we were working on Minion , years ago . It certainly can work , but adds a ton of complexity to the application workflow , and reduces the ability to have meaningful back pressure and flow control . I think that to a large degree , this PR is just writing out the underlying way that sending and receiving in the current draft works . Here 's the long form of sending that works for partial and complete : Send(messageContext , messageData , endOfMessage ) The default for endOfMessage should be true , therefore most applications will just see : Send(messageContext , messageData ) And the the default for messageContext should be the default parameters for a new message , so most applications ( using UDP , for example ) will just see : Send(messageData ) The same is true for receive . We can give the variants of Send to be partial or not different names , but they boil down to convenience / default sets of the same thing . The key points here are these : properties of a message need to be the same across multiple partial sends if there are multiple partial sends . If the application is streaming a long message , then the properties of that message apply to all of the data . completion handlers for Send actions that provide notions of back pressure and errors must be per data chunk sent , not per message . If I am streaming a long message , then I will likely need to handle flow control , and the completion handler for send ca n't just be on the whole message , it needs to be on the partial chunks . If you move to sending data on a new Message object rather than the Connection , you would need to have back pressure on those sends too . It becomes very complicated for the application : Connection . Send(Message ) -MessageSentHandler[May have error but ca n't be used for flow control ] Message . Send(bytes ) -DataSentHandler[Used for per - message flow control , which now becomes difficult to coordinate with other messages if you have a stream with HOL blocking ] Going down this road means that partial sends become front - and - center for applications dealing with the API , even when they do n't want it , and adds an extra layer of indirection just to get basic bytes on the wire for a transport . The approach in this PR ends up being a way to allow expressiveness for partial sends when they are needed , but allows applications to ignore it most of the time with defaults and write the simplest code . 
 To date we 've avoided adding alternate action and event signatures to this API , leaving what gets defaulted and how up to the platform . Perhaps we should explicitly suggest / define which parts of the API should be defaulted out ... I think that would go a long way to reducing the apparent complexity of this approach . 
 Right , having the default support makes things much easier . For languages that support defaults ( Swift , for us ) , you can do the scheme like I had above . For languages like C , you can write out the common sets of calls , which all can be implemented using the most complex : 
 +1 on introducing defaults . The API is beginning to get quite complex . Maybe we should explicitly split up the send section in " Basic Send Operations " and " Advanced Send Functionality " or something like that . Splitting up other relevant sections in a similar way . 
 Sure , I can work tomorrow on splitting out the definition of the simple send operations , versus the more full story for partial sends . We can mention as an implementation detail that everything in the simpler approach for atomic messages can be expressed in terms of defaults in the variants that support partial sends , and leave the specific symbol bindings as a per - language choice . 
 +1 to recommendation . 
 addressing comment here would make this ready to land IMO . I 'd like to get this and in soon so we can submit -01 in time , and keep working on this in person in Montreal . 
 Okay , just updated the Sending side to break out basic sends , sending with context , and partial sends . I 'll do a bit more work on receiving too today , but take a look at the sending side and see if that addresses the comments there . 
 Thanks for the restructuring , I like the new structure ! I find it much more clear and readable . One question on the Events in the relation to the partial sends . If I understand it correctly Sent Events relate to specific send calls whereas Expired Events relate to an entire Message . Not sure about SendError Events ? Seems there it may depend on what the error is . Not sure if this is something that needs to be clarified further . 
 I personally think it 's simplest if the events correspond to Send events , so if you did a 2 partial sends for a single message that expired , you should get two Expired events if neither could be sent , etc . 
 Makes sense , thanks . 
 I 've the rest of my reworked text , to bring Receives into the same style as Send ( with the basic approach up front , and partial phrased as an extension later on ) . Thanks for all of the good feedback , I think this text is certainly clearer ! I some text about how Send events are delivered when there are multiple Sends per Message , please check . I text to explain that receiving is only in - order contiguous chunks , and clarified that you ca n't span a message over multiple datagrams directly , etc . 
 Thanks , the text looks good to me ! 
 This LGTM now , thanks a lot for doing this ! 
 wrote : The key points here are these : The completion handlers for Send actions that provide notions of back pressure and errors must be per data chunk sent , not per message . If I am streaming a long message , then I will likely need to handle flow control , and the completion handler for send ca n't just be on the whole message , it needs to be on the partial chunks . Agree with these . If you move to sending data on a new Message object rather than the Connection , I was n't arguing to move the sending data onto the Message object . In my approach , the Connection would still handle sending the data , but rather than have the application provide chunks of data , I was suggesting to have the connection pull data from the message object as needed . you would need to have back pressure on those sends too . It becomes very complicated for the application : Do n't we need that back pressure anyway ? If one sends a message object from which data is pulled as needed , then I agree the connection would need to generate events before the final event . But , we do n't avoid that need by splitting the data up at the application level and making partial send requests : with a transport like TCP the connection can always block , and will need to generate a event , no matter whether the application has split the data into chunks before calling send or not . It 's perhaps easier for the application to change what it sends , or cancel the send , if it send chunk - by - chunk – that I agree – but otherwise I do n't see a complexity saving . 
 Regarding backpressure , you 're right that we do need backpressure on all chunks that are sent . However , by allowing partial sends to be expressed in the native send action of the Connection , they automatically get the same backpressure that complete sends get . Each Sent event is one - to - one with a Send action , whether partial or complete . 
 Okay , I think based on the reviews we 're in a good state to land this , and I do want to unblock merging for . As such , I 'm going to go ahead and merge . 
 Should we specify in the API more clearly if properties around sending apply more to the Send ( ) operation or the Message itself ? In my implementation , I 'm treating things like " Idempotent " as per - send , since I could technically have a bit of a message at the beginning I want to do 0 - RTT with , but the rest of the data may not be fully idempotent . However , ordering and reliability ( and likely niceness ) is a property of the entire Message , and so if I 'm sending a Message in multiple Send()s , we want the same properties across all calls . 
 Message properties are : niceness , ordering / antecedents , etc , etc Send properties are : end - of - message , end - of - connection , etc . 
 That introduces a third kind of properties – do we want this kind of complexity ? In my opinion , we should only have Send Properties . Whether they can be applied to partial messages is transport - protocol and implementation specific . So if your TCP stream is a single message ( no framers / de - framers ) , one can indeed only mark the first part as idempotent . The consequence is that we need to think about when this will raise an error . 
 I 'd say it should be per - message , not per - send , and I also agree that we should keep things simple . In particular , from Tommy , sounds really messy to me ! Why not have two separate messages instead , an idempotent one and a non - idempotent one ? What 's the argument for dividing a message into two separate parts with different properties ? 
 As long as we treat TCP streams as a large message , we need to support Send Properties on partial messages , e.g. , for 0 - RTT 
 i thought this tcp - stream = message idea was finally behind us ? ? that was agreed between tommy and me in london , at least 
 for bookkeeping purposes : ISTM that addresses this one , too . 
 As far as I am concerned , and seem to have fixed this issue . 
 and 
 I see where this is coming from , but I do n't like the proposed API – it exposes a lot of low - level details about partial messages that I was hoping we 'd get away from with the new API . I wonder if we should regard messages as more active objects ? That is , a is something that knows whether it 's complete or not , and that can produce more data as appropriate . The API could then be to a object once , then internally the system will repeatedly call on the Message object until it returns that it 's complete . This makes the conceptual model easier : you just send a message . The complexity of dealing with partial messages , etc . , is then pushed into the message for doing this ... this seems to be the simplest way to handle message non - atomicity while still providing for atomic(-like ) sends and receives in the general case . All of my problems with what 's here are about the elegance of the interface the programmer will see ( a " Message " that is separate from the content seems misnamed ; this adds complexity that we should make sure everyone knows can be papered over with syntactic sugar in the common case ) . I 'm happy with landing this as - is for Montreal and filing an issue to discuss in person , though .
Paragraph (version A):
If Send is called on a Connection which has not yet been established,
an Initiate Action will be implicitly performed simultaneously with
the Send.  Used together with the Idempotent property (see send-
idempotent), this can be used to send data during establishment for
0-RTT session resumption on Protocol Stacks that support it.
Paragraph (version A):
Like all Actions in this interface, the Send Action is asynchronous.
Paragraph (version B):
Once a Connection has been established, it can be used for sending
data.  Data is sent in terms of Messages, which allow the application
to communicate the boundaries of the data being transferred.  By
default, Send enqueues a complete Message, and takes optional per-
Message parameters (see send-basic).  All Send actions are
asynchronous, and deliver events (see send-events).  Sending partial
Messages for streaming large data is also supported (see send-
partial).
Paragraph (version B):
The most basic form of sending on a connection involves enqueuing a
single Data block as a complete Message, with default send
parameters.  Message data is created as an array of octets, and the
resulting object contains both the byte array and the length of the
array.
Paragraph (version B):
The interpretation of a Message to be sent is dependent on the
implementation, and on the constraints on the Protocol Stacks implied
by the Connection's transport parameters.  For example, a Message may
be a single datagram for UDP Connections; or an HTTP Request for HTTP
Connections.
Paragraph (version B):
Like all Actions in this interface, the Send Action is asynchronous.
There are several events that can be delivered in response to Sending
a Message.
Paragraph (version B):
Note that if partial Sends are used (send-partial), there will still
be exactly one Send Event delivered for each call to Send.  For
example, if a Message expired while two requests to Send data for
that Message are outstanding, there will be two Expired events
delivered.
Paragraph (version A):
The Send Action takes per-Message send parameters which control how
the contents will be sent down to the underlying Protocol Stack and
transmitted.
Paragraph (version A):
If Send Parameters should be overridden for a specific Message, an
empty sent parameter Object can be acquired and all desired Send
Parameters can be added to that Object.  A sendParameters Object can
be reused for sending multiple contents with the same properties.
Paragraph (version A):
The Send Parameters share a single namespace with the Transport
Parameters (see transport-params).  This allows the specification of
Protocol Properties that can be overridden on a per-Message basis.
Paragraph (version A):
Send Parameters may be inconsistent with the properties of the
Protocol Stacks underlying the Connection on which a given Message is
sent.  For example, a Connection must provide reliability to allow
setting an infinitie value for the lifetime property of a Message.
Sending a Message with Send Properties inconsistent with the
Transport Preferences on the Connection yields an error.
Paragraph (version B):
Applications may need to annotate the Messages they send with extra
information to control how data is scheduled and processed by the
transport protocols in the Connection.  A MessageContext object
contains parameters for sending Messages, and can be passed to the
Send Action.  Note that these properties are per-Message, not per-
Send if partial Messages are sent (send-partial).  All data blocks
associated with a single Message share properties.  For example, it
would not make sense to have the beginning of a Message expire, but
allow the end of a Message to still be sent.
Paragraph (version B):
The simpler form of Send that does not take any MessageContext is
equivalent to passing a default MessageContext with not values added.
Paragraph (version A):
Final is a boolean property.  If true, this Message is the last one
that the application will send on a Connection.  This allows
underlying protocols to indicate to the Remote Endpoint that the
Connection has been effectively closed in the sending direction.  For
example, TCP-based Connections can send a FIN once a Message marked
as Final has been completely sent.  Protocols that do not support
signalling the end of a Connection in a given direction will ignore
this property.
Paragraph (version B):
It is not always possible for an application to send all data
associated with a Message in a single Send Action.  The Message data
may be too large for the application to hold in memory at one time,
or the length of the Message may be unknown or unbounded.
Paragraph (version B):
Partial Message sending is supported by passing an endOfMessage
boolean parameter to the Send Action.  This value is always true by
default, and the simpler forms of send are equivalent to passing true
for endOfMessage.
Paragraph (version B):
The following example sends a Message in two separate calls to Send.
Paragraph (version A):
Once a Connection is established, Messages may be received on it.
The application can indicate that it is ready to receive Messages by
calling Receive() on the Connection.
Paragraph (version A):
Receive takes a ReceiveHandler, which can handle the Received Event
and the ReceiveError error.  Each call to Receive will result in at
most one Received event being sent to the handler, though
implementations may provide convenience functions to indicate
readiness to receive a larger but finite number of Messages with a
single call.  This allows an application to provide backpressure to
the transport stack when it is temporarily not ready to receive
messages.
Paragraph (version A):
Receive also takes an optional maxLength argument, the maximum size
(in bytes of data) Message the application is currently prepared to
receive.  The default value for maxLength is infinite.  If an
incoming Message is larger than the minimum of this size and the
maximum Message size on receive for the Connection's Protocol Stack,
it will be received as a partial Message.  Note that maxLength does
not guarantee that the application will receive that many bytes if
they are available; the interface may return partial Messages smaller
than maxLength according to implementation constraints.
Paragraph (version A):
The Message Object must provide some method to retrieve an octet
array containing application data, corresponding to a single message
within the underlying Protocol Stack's framing.  See receive-framing
for handling framing in situations where the Protocol Stack provides
octet-stream transport only.
Paragraph (version B):
Receive takes two parameters to specify the length of data that an
application is willing to receive, both of which are optional and
have default values if not specified.
Paragraph (version B):
By default, Receive will try to deliver complete Messages in a single
event (receive-complete).
Paragraph (version B):
Each call to Receive will be paired with a single Receive Event,
which can be a success or an error.  This allows an application to
provide backpressure to the transport stack when it is temporarily
not ready to receive messages.
Paragraph (version A):
Note that in the absence of message boundary preservation and without
deframing, the entire Connection is represented as one large message
of indeterminate length.
Paragraph (version B):
The ReceiveError event passes an optional associated messageContext.
This may indicate that a Message that was being partially received
previously, but had not completed, encountered and error and will not
be completed.
Paragraph (version A):
The application may check a property to determine if a received
Message is the Final Message on a Connection.  For any Message that
is marked as Final, the application can assume that there will be no
more Messages received on the Connection once the Message has been
completely delivered.  This corresponds to the Final property that
may be marked on a sent Message send-final.
Paragraph (version A):
Any calls to "Receive" once the Final Message has been delivered will
result in errors.
Paragraph (version B):
Any calls to Receive once the Final Message has been delivered will
result in errors.