Change:
Suggestion to address and in the API doc.
Reading through the API, I noted that there is, amongst the various connection states, a state that is defined but never mentioned again in any of the drafts: "Closing". I am unclear how this state is useful to the application, and how an implementation would determine it. If the connection is closed from the remote side, like a RST, then we'll move right to Closed. If the connection is closed from the local side, I guess we can move into the Closing state before getting an ack that the remote side knows that we've closed, but at that point, I'm not going to be able send or receive any more on the connection anyhow. Do we have a good use case for Closing? If so, can we get more text? If not, can we remove it? For reference, we don't have any corresponding state in our implementation, and it's never come up as a need.
I also don't see a good use case and would rather see it removed. I guess the only case worth even thinking about is the local close one. Here, our "close" semantics don't promise any use for any Closing state anyway, and since (as you say) you can't send or receive anymore on a closing connection, the only point is resource optimization - to avoid opening a new connection when we can wait until the previous one is closed, stuff like that. Even that idea creates a protocol dependency in the application (because some protocols will have delay on closing, but some will just terminate), which IMO makes the state a bad idea.
Right, if you wanted to know you were locally 'closing', you could just know that you called and haven't gotten the state yet.
So for me, the state is between someone called and the sending of the event – might be interesting for concurrent threads to not fall into race issues, but besides this not very useful. Side question: can I clean up all my connection related state in the event handler and is the event guaranteed to fire eventually?
This seems implementation-internal, which indicates to me we can probably get rid of the state. I think that's a question for / which I haven't got around to thinking about yet.
NAME yes, that seems to be the only meaning of the "closing" state, but it doesn't really even provide any threading benefit. If you want to make sure that the connection has closed, just wait for the state—anything after you invoke Close() before the transition to is effectively . With regards to the question about the final event, I think there MUST be some final event that means that everything is cleaned up from the application's perspective so the memory can be released, etc. In the current model, seems to be that correct event. In our implementation, we call this , which only happens after the client of the API calls the / function, to distinguish from a remote-initiated close. So, to be most safe, you may want a state for invalidated/cancelled/never-delivering-any-events-again.
(catching-up on old discussions) I'm confused: we surely need a name for the time between calling and getting the event. What would this be, other than the state? I don't understand how we can remove this.
I agree with NAME Depending on the case, it may not map to much going on in code, you may just get an immediate "Closed" instead after calling Close() - but not always. So then, what's the reason to not give this time period a name, and what else is it but a state?
Maybe we should add a hint to this state, that it is not particularly useful from a programmers perspective, but necessary in the presence of concurrency.
In , I've added a section on ordering of API events (please have a look at it). It talks about guarantees the API makes about which events must occur in which order, but it doesn't talk about guarantees the application must make in its ordering of operations. If we added this, we could talk about the state machine there. It does seem to me that if we add such a discussion, that after is not allowed. This suggests that is a state.
I don't see a need to have a state be set to enforce that must not be called after . If everything is an asynchronous operation, and states are signaled asynchronously, etc, then there still may be a race if you call then immediately; you most likely will not have received the state transition to yet, but it's still an invalid sequence of calls, and the send should fail. NAME while I agree that closing is a state that may exist internally for some protocols, I don't see any value to delivering a signal to an application that they are in the closing state. That's an extra event delivery that necessitates to action and provides no new information. It also is a source of confusion if some protocols use a closing state, but others don't (because UDP should just go to immediately).
I agree with NAME that conceptually you always pass through the Closing state after calling Close() and before receiving the Closed event, even if this transition may sometimes be practically immediate as when you use UDP. Even if you use UDP the TAPS system may need to perform some internal cleanup before triggering the Closed event though. I agree with NAME that you do not need to signal that you are in the Closing state to the application, but I also do not see that this is suggested anywhere. If the application asks what state it is in, then Closing is a possible return value, but that does not create an extra event delivery? So I do not see that the Closing state causes any problem or extra overhead.
I think that state changes do need to be events signaled to the application. Knowing that a connection is ready should not be polled—it is an event. Similar for closed, etc. there should not be a special case for closing, but closing should not be a fired event at all!
Why not have a special case? True, I struggle to imagine other state changes that an application doesn't care about, but so be it - "closing" is no less of a state just because it's not important enough to apps to be signalled to them. (side note: I tag this message as "I offer my opinion but I also don't care much about this")
Why would we not signal to the application? We need some signal that the stack is acting on the request, since this implies that events pertaining to an active connection will no longer be delivered and the application can free up those resources. Especially since, for some protocols, the event could come some significant time after .
Splitting this into and could simplify tge However, I'm a little confused about this: happens after is called, but before happens. ISTM you can still receive things after (i.e., there can still be events fired from pending receive calls), but is it legal to call and ?
NAME If I some data then call , when do I enter the closing state? The event could mean "I got your close request, and will send any outstanding data then close the connection" or it could mean "I've send all outstanding data and am in the process of closing the connection". If the former, I'd expect to get Message Sent events after , if the latter I'd not expect such events.
I thought you enter the Closing state immediately after calling Close() and you can no longer call Send() or Receive() after this point and a Closing<> event would have the former meaning ("I got your close request, and will send any outstanding data then close the connection"). That's why I did not see the need to signal this state transition to the application. I agree we need to signal the application when all outstanding data has been sent and resources can be freed. I thought this would be captured by the Closed<> event. But if the Closed<> event relies on the other end it may be good to separate them. In relation to the need for signaling all state transitions, do we not have a similar situation for Establishing. The Establishing state is I assume entered when an application calls Initiate(), Listen() or Rendezvous(). We do not signal this state transition to the application as far as I understand. We signal when the connection is ready and we move to the Established state, correct?
Linking this to operation ordering as in might make the whole concept of application-exposed connection state make more sense.
having submitted , it's pretty clear that the Established and Closing states are identical from the application's standpoint with respect to which events are received. It may be that Closing is simply equivalent to Established s.t. both "can send" and "can receive" are False. That raises the quesiton of whether or not we should split Established into four states: Bidirectional Send-Only Receive-Only Closing
I don't see the point of this state split proposal - but if you like this better I don't mind it either.
NAME I agree with the overall assessment that Closing is the same as Established, with the distinction that I/O doesn't work. There are other cases in which I/O may not work temporarily in the middle of a connection too, and I wonder if that should be related. For example, if you have a migratable protocol like MPTCP or QUIC, and you lose all interfaces, you can't actually send or receive for a time (although sends and receive calls may be buffered), and you may end up timing out and closing, or you may end up re-establishing over another link.
ahhhh yep. "no I/O available" can actually happen for a range of stack/transport system implementation specific reasons, not just because it's closing, so I do think we need a state but it needs a different name. It also makes sense to keep "can send" and "can receive" separate read-only booleans, because they happen for different reasons which are not necessarily state-related. We have four states then: Establishing: connection is not yet up and running; local allocation is complete and resolution/racing is in progress, or a listener is up and waiting for connections. Established: connection is up and running and I/O is possible in at least one direction; there exists at least one transport stack attached to the connection Dormant: the transport system is maintains state for a connection, but there exist no active transport stacks attached to the connection, and the transport system is not currently attempting to re-establish the connection. Modeling this probably goes hand in hand with the text for . Closed: the transport system has shut down the connection; it is dead, and eligible for GC etc.
and are different though: the connection doesn't come back from , but it does from . Also: is it or do we just transition back to in this case? Is it just for migration?
Dormant might also happen in an environment where the transport system knows it can resume. Migration, yes, but maybe migration during sleep (i.e., you closed your laptop on a download then landed in another country and opened it again -- if the endpoints know about each other they can still resume). As to whether Dormant isn't just Establishing, it depends on whether we need the application to distinguish between "the transport system has at least one candidate it is racing, sends will be buffered and receives will queue" and "the transport system has no candidates, but attempts to send or receive may result in a transition to Establishing". Rereading what I wrote above, I think I got the equivalence between Dormant and Closing wrong -- they're actually totally separate. But I still don't think I see the utility of Closing on its own from a properties management standpoint-- you're Established, you Close (or the remote peer does), then you're Closed (and subject to GC if you're that sort of language :) ).
Whether or not has utility, it exists as a state: for many transports there is a time period between starting to tear down the connection, and having completed tearing down the connection. That's the state, whether or not its useful. I don't see that pretending it doesn't exist is a good idea.
ok i buy that. So we have five: establishing, established, dormant (which might just be establishing), closing and closed (which act the same from the interface's standpoint, but are semantically different...)
It seems this discussion has concluded, without a need to add new text (because we have establishing, established, closing and closed already covered).
As far as I can tell there's no way for an application to indicate that it wants to stop listening for an incoming connection. As with my , this will need a way to know that it's complete, in case there's a race with Preconnection -> ConnectionReceived<> on another thread.
fixed by
An application needs a way to know when its Close() or Abort() call is complete, guaranteeing it won't receive any further events on the connection and all current event processing is done. This is necessary so the application can clean up any application state related to the connection. Note that in a multi-threaded environment, events can be executing on other threads at the same time that the Close() or Abort() call is called; in addition, an application can potentially call Close() or Abort() from within an event processing function itself.
The intention (in the API) is that the Closed<> and Aborted<> events fire after these are complete, whether the close or abort was locally or remotely initiated.
fixed by
The API document needs to specify the relative timing and ordering (i.e., for appropriate thread models, the threading properties) of events. E.g., I think it should be guaranteed that a connection's Received<> event isn't called until its initiating event (Connected<>, ConnectionReceived<>, or RendezvousDone<>) has finished processing; and only one Received<> event is invoked at a time for a given connection, with the events being invoked with the appropriate logical order is required by the connection semantics. Similarly, only one of a connection's Sent<> events should be invoked at a time. However, I don't think there needs to be any restriction against Sent<> and Received<> being invoked simultaneously on the same connection. No other events should be received for a connection after Closed<> or ConnectionError<>. For different connections, It should be allowed for their events to be invoked simultaneously, even if the connections are entangled with each other.
One particular consideration that came up in the May 16th call is that this last event (presumably Closed<> or ConnectionError<> or Aborted<> or whatever) needs to be enqueued to be executed after previous callbacks complete, rather than being executed synchronously on the stack of another callback.
fixed by

Discussion:
further parameters may be bound to the Preconnection, and no subsequent establishment call may be made on the Preconnection. <ins> Listening continues until the global context shuts down, or until the Stop action is performed on the same Preconnection: After Stop() is called, the preconnection can be disposed of. </ins> The ConnectionReceived Event occurs when a Remote Endpoint has established a transport-layer connection to this Preconnection (for Connection-oriented transport protocols), or when the first Message

if specified) cannot be resolved, or when the application is prohibited from listening by policy. <ins> A Stopped event occurs after the Preconnection has stopped listening. </ins> 6.3. Simultaneous peer-to-peer Connection establishment is supported by

11. <ins> As this interface is designed to be independent of concurrency model, the details of how exactly actions are handled, and on which threads/ callbacks events are dispatched, are implementation dependent. However, the interface does provide the following guarantees about the ordering of operations: Received<> will never occur on a Connection before a Ready<> event on that Connection, or a ConnectionReceived<> or RendezvousDone<> containing that Connection. No events will occur on a Connection after a Closed<> event, an InitiateError<> or ConnectionError<> on that connection.  To ensure this ordering, Closed<> will not occur on a Connection while other events on the Connection are still locally outstanding (i.e., known to the interface and waiting to be dealt with by the application).  ConnectionError<> may occur after Closed<>, but the interface must gracefully handle the application ignoring these errors. Sent<> events will occur on a Connection in the order in which the Messages were sent (i.e., delivered to the kernel or to the network interface, depending on implementation). 12. </ins> RFC-EDITOR: Please remove this section before publication. This document has no Actions for IANA. <del> 12. </del> <ins> 13. </ins> This document describes a generic API for interacting with a transport services (TAPS) system.  Part of this API includes