Project: Pipy (Orchestration & Management / Service Proxy)
File: connection-id-cache.md

QUIC Connection ID Cache ======================== The connection ID cache is responsible for managing connection IDs, both local and remote. Remote Connection IDs For remote connection IDs, we need to be able to: add new IDs per connection; pick a non-retired ID associated from those available for a connection and select a connection ID by sequence number and retire that and all older IDs. The cache will be implemented as a double ended queue as part of the QUIC_CONNECTION object. The queue will be sorted by sequence number and must maintain a count of the number of connection IDs present. There is no requirement to maintain a global mapping since remote IDs are only used when sending packets, not receiving them. In MVP, a many-to-1 matching of Connection IDs per Connection object is required. Refer third paragraph in [5.1]. When picking a non-retired connection ID for MVP, the youngest available will be chosen. Local Connection IDs -- For local connection IDs, we need to be able to: generate a new connection ID and associate it with a connection; query if a connection ID is present; for a server, map a connection ID to a QUIC_CONNECTION; drop all connection IDs associated with a QUIC_CONNECTION; select a connection ID by sequence number and retire that and all older IDs and select a connection ID by sequence number and drop that and all older IDs. All connection IDs issued by our stack must be the same length because short form packets include the connection ID but no length byte. Random connection IDs of this length will be allocated. Note that no additional information will be contained in the connection ID. There will be one global set of local connection IDs, `QUICROUTETABLE`, which is shared by all connections over all SSL_CTX objects. This is used to dispatch incoming datagrams to their correct destination and will be implemented as a dictionary. For MVP, it would be sufficient to only use a zero length connection ID. For MVP, a connection ID to QUIC_CONNECTION mapping need not be implemented. Post MVP, funnelling all received packets through a single socket is likely to be bottleneck. An alternative would be receiving from <host address, source port> pairs. For MVP, the local connection ID cache need only have one element. I.e. there is no requirement to implement any form of lookup. Routes A pair of connection IDs identify a route between the two ends of the communication. This contains the connection IDs of both ends of the connection and the common sequence number. We need to be able to: select a connection ID by sequence number and retire that and all older IDs and select a connection ID by sequence number and drop that and all older IDs. It is likely that operations on local and remote connection IDs can be subsumed by the route functionality. ID Retirement Connection IDs are retired by either a [NEWCONNECTIONID] or a [RETIRECONNECTIONID] frame and this is acknowledged by a RETIRECONNECTIONID or a NEWCONNECTIONID frame respectively. When a retirement frame is received, we can immediately remove the IDs covered from our cache and then send back an acknowledgement of the retirement. If we want to retire a frame, we send a retirement frame and mark the IDs covered in our cache as retired. This means that we cannot send using any of these IDs but can still receive using

Once our peer acknowledges the retirement, we can remove the IDs. It is possible to receive out of order packets after receiving a retirement notification. It's unclear what to do with these, however dropping them seems reasonable. The alternative would be to maintain the route in a deletable state until all packets in flight at the time of retirement have been acked. API QUIC connection IDs are defined in #18949 but some extra functions are available: ```c / QUIC connection ID representation. / typedef struct quicconnid_st { unsigned char id_len; unsigned char id[QUICMAXCONNIDLEN]; / likely required later, although this might not be the ideal location / unsigned char reset_token[16]; / stateless reset token is per conn ID / } QUICCONNID; static osslunused osslinline int osslquicconnideq(const QUICCONNID *a, const QUICCONNID *b); / New functions / int osslquicconnidset(QUICCONNID cid, unsigned char id, unsigned int id_len); int osslquicconnidgenerate(QUICCONNID *cid); ``` ```c typedef struct quicremoteconnidcachest QUICREMOTECONNID_CACHE; /* Allocate and free a remote connection ID cache */ QUICREMOTECONNIDCACHE *osslquicremoteconnidcachenew( sizet idlimit / [active_connection_id_limit] / ); void osslquicremoteconnidcachefree(QUICREMOTECONNIDCACHE *cache); /* Add a remote connection ID to the cache */ int osslquicremoteconnidcacheadd(QUICREMOTECONNIDCACHE *cache, const QUIC_CONNECTION *conn, const unsigned char *conn_id, sizet connid_len, uint64t seqno); /* Query a remote connection for a connection ID. Each connection can have multiple connection IDs associated with different routes. This function returns one of these in a non-specified manner. */ int osslquicremoteconnidcacheget0connid( const QUICREMOTECONNIDCACHE *cache, const QUICCONNECTION *conn, QUICCONN_ID cid); /* Retire remote connection IDs up to and including the one determined by the sequence number. */ int osslquicremoteconnidcacheretire( QUICREMOTECONNIDCACHE *cache, uint64t seqno); /* Remove remote connection IDs up to and including the one determined by the sequence number. */ int osslquicremoteconnidcacheremove( QUICREMOTECONNIDCACHE *cache, uint64t seqno); ``` ```c typedef struct quiclocalconnidcachest QUICLOCALCONNID_CACHE; /* Allocate and free a local connection ID cache */ QUICLOCALCONNIDCACHE *osslquiclocalconnidcachenew(void); void osslquiclocalconnidcachefree(QUICLOCALCONNIDCACHE *cache); /* Generate a new random local connection ID and associate it with a connection. For MVP this could just be a zero length ID. */ int osslquiclocalconnidcachenewconnid(QUICLOCALCONNIDCACHE *cache, QUIC_CONNECTION *conn, QUICCONNID cid); /* Remove a local connection and all associated cached IDs */ int osslquiclocalconnidcacheremoveconn(QUICLOCALCONNID_CACHE *cache, const QUIC_CONNECTION *conn); /* Lookup a local connection by ID. Returns the connection or NULL if absent. */ QUICCONNECTION *osslquiclocalconnidcacheget0conn( const QUICLOCALCONNIDCACHE *cache, const unsigned char *connid, sizet connidlen); /* Retire local connection IDs up to and including the one specified by the sequence number. */ int osslquiclocalconnidcacheretire( QUICLOCALCONNIDCACHE *cache, uint64t fromseq_no); /* Remove local connection IDs up to and including the one specified by the sequence number. */ int osslquiclocalconnidcacheremove( QUICLOCALCONNIDCACHE *cache, uint64t fromseq_no); ``` Additional status and source information is also included. ```c typedef struct quicroutest QUIC_ROUTE; typedef struct quicroutetable QUICROUTETABLE; struct quicroutest { QUIC_CONNECTION *conn; QUICCONNID local; QUICCONNID remote; uint64t seqno; / Sequence number for both ends / unsigned int retired : 1; / Connection ID has been retired / / Later will require / BIOADDR remoteaddress;/ remote source address / }; QUICROUTETABLE *osslquicroutetablenew(void); void osslquicroutetablefree(QUICROUTETABLE *routes); ``` ```c int osslroutetableaddroute(QUICROUTETABLE *cache, QUICROUTETABLE *route); ``` ```c /* Query a route table entry by either local or remote ID */ QUICROUTE *osslroutetableget0routefrom_local( const QUICROUTETABLE *cache, const unsigned char *connid, sizet connidlen); QUICROUTE *osslroutetableget0routefrom_remote( const QUICROUTETABLE *cache, const unsigned char *connid, sizet connidlen); ``` ```c /* Retire by sequence number up to and including the one specified. */ int osslquicroutetableretire(QUICROUTETABLE *routes, QUIC_CONNECTION *conn, uint64t seqno); /* Delete by sequence number up to and including the one specified. */ int osslquicroutetableremove(QUICROUTETABLE *routes, QUIC_CONNECTION *conn, uint64t seqno); ```