Code change:
by explicitly specifying the namespace. Also consistently changes to the (b) notation to indicate a variable length sequence of bytes.
This PR got lost when merging 289.
Assume a setup with two publishers, a single relay and one subscriber. P1 ANNOUNCE's track namespace fooba P2 ANNOUNCE's track namespace foo S1 sends a SUBSCRIBE REQUEST for Full Track Name How does the relay know whether is Track Namespace: and Track Name: , or Track Namespace: and Track Name , and hence which publisher to route the subscribe to? I see a few possibilities: 1) The relay should not allow this scenario (via namespace construction and/or authorization). If so the draft should include text explaining that this is disallowed (normatively?) or "bad things(tm)" happen. 2) Change SUBSCRIBE REQUEST to be able to disambiguate the Track Namespace from the Track Name. This could be done by sending name and namespace in separate message fields, or having stricter rules for the structure of Full Track Name such that a relay can extract the constituent parts. 3) Explicitly state that Full Track Names are matched to namespaces via "longest prefix match".
Option 3 looks interesting. P1 can use a longer track namespace to steer the sub request from P2. Like a traffic engeering for content.
I like option 2. I don't agree with treating track namespace as merely a string prefix. Even if you go with option 3, there's still a race condition where a late ANNOUNCE will cause a SUBSCRIBE to an unintended track. I want to give the track namespace more properties so it's treated more like a broadcast identifier. A few folks are headed down this path without realizing it, such as proposing that there's a single catalog per track namespace. I don't see how that would even work if ANNOUNCE is used purely for routing like a BGP announcement.
NAME : are you saying you believe that the relay resolves the described ambiguity by using 'longest prefix' matched (eg: option 3)? Another option I hadn't considered based on this comment is that the relay would send OBJECTs from BOTH publishers since they both match the Full Track Name. This will break in the current design, since each publisher has their own Object Sequence and Group Sequence and when merged the consumer will be left with garbage. It also violates the logic described that says a Full Track Name can be used as a cache key.
Yeah that seems quite problematic NAME Let's say a relay have two connected publishers who send IN ORDER: A subscriber connects and issues: What happens next? The relay forwards the SUBSCRIBE to publisher 1. The relay returns SUBSCRIBE ERROR notfound The relay forwards the SUBSCRIBE to publisher 2. The relay forwards the SUBSCRIBE to publisher 1&2. These are in the order of Alan's options. I added option 4 based on Suhas' comment. Would occur if the relay rejects publisher2's ANNOUNCE because of thr shared prefix. Would occur because the track namespace is an ID, not a prefix, and must fully match an ANNOUNCE. Would occur because publisher2 has the longer path prefix. Would occur because both publishers could_ produce the requested track. And to throw a wrench in things, a third publisher joins.
There's also the case where the relay replies to the subscribe requests on behalf of the publisher(s), so it doesn't have to make the routing decision at subscription time. But it does need to decide where to forward OBJECTs when they arrive.
Implementing this now so I have some more thoughts. We should explicitly support UNANNOUNCE and UNSUBSCRIBE, which implicitly occur when the connection is dropped. On ANNOUNCE, the relay MUST ensure that the namespace not a subset/superset of an existing ANNOUNCE. On SUBSCRIBE, the relay looks up the track based on the prefix and appends itself to a list of subscribers. If not found, it looks up the namespace and creates a track entry if valid. On UNSUBSCRIBE, the relay looks up the track based on the full name and removes itself from a list of subscribers. On UNANNOUNCE, the relay deletes any namespaces and tracks matching the prefix. A radix tree is used for both so the relay can iterate through any values that match the given prefix. This is a global radix tree, similar to how QUIC connections are identified when variable length connection IDs are supported. On ANNOUNCE, the relay inserts the upstream into the upstreams hash map and errors on a duplicate. On SUBSCRIBE, the relay looks up the namespace and then the track, creating an entry if not found. On UNSUBSCRIBE, the relay looks up the namespace and then the track, deleting an entry if found. On UNANNOUNCE, the relay deletes from the namespace and track hash map. This is a strictly better than option 1. It provides the same functionality but faster, and without the restriction on namespace prefixes. Runs into issues with ANNOUNCE. example: and are subscribed to different publishers, despite using THE SAME NAME. This invalidates the claim that the track name is the cache key. We can't migrate to the new longer prefix from unless there's a guarantee they are producing the same content. A similar issue occurs when UNANNOUNCE is fired, when happens implicitly when the connection is dropped: Each subscription needs to keep reference to the track when the SUBSCRIBE was received, instead of doing a lookup based on full track name when each OBJECT is received. This will cause havok when resubscriptions occur, for example with ABR, as switching away from and back to a track MAY result in different content. It's not viable. The subscription would intermingle objects from separate publishers that happen share a prefix, all while using the same . I really don't want to support subscribing based on wildcards or prefixes anyway. This is also the slowest option, as there needs to be a global RadixTree that maps tracks to subscriptions on receipt of each object. All other options listed above would keep an array of subscribers per track, resolved using a (shardable) HashMap.
I think this seems like the right approach. Having 2 producers with conflicting namespaces is a bad idea, unless the authz policy and application is somehow OK with the consequences. I have added to clarify the thinking further
I'm of the view of two things 1) apps should not do this if it is a problem for that app usage of relay 2) relay should use the latest "announce" to override any earlier announces that are more specific ( or the same )
Discussed at IETF 117: Support in the room for changing "Full Track Name" from a single string to a tuple of Namespace and Track Name, and matching on Namespace exactly.
IIRC, my recollection was not the same, also I don't think we concluded that the namespace match exactly. The discussion went over what the authz model defines and how that impacts the flow. I wonder how much the transport draft can mandate how a given relay application should implement the business rules. The transport draft must just indicate relay behavior and the course of actions to take for what happens when " if match succeeds vs match fails ", which is what is in the draft today already
Uh, my understanding of what we talking about what not at all removing Full Track Name - it was about we did not need to define a separator for concatenating things in a tuple. I don't agree there was support for chaning Full Track Name to Namesapce and Track Name tuple and only doing full matches. That is not going to work for many of the use cases and not what we are discussing.
I thought what came out of this discussion is 1) we do not know how duplicate ANNOUNCE work even when the names are identical ( do we keep first, last , error etc ) 2) once we figure that out, we can start to deal with the more complicated issue of partial match or overlapping Perhaps I have this recollection wrong but if so point me at right place in recording for that.
My motivation for ANNOUNCE was for the contribution case. A client like OBS will connect to the server, ANNOUNCE a new catalog track, and then the server will issue a corresponding SUBSCRIBE to get the data flowing. This is modeled after the QUICR PUBLISHINTENT message, except that data doesn't flow until the other side issues a SUBSCRIBE. However, announcing a namespace prefix subverts this behavior: The relay doesn't know if an ANNOUNCE is a prefix for routing, or a single broadcast being published. The relay effectively needs a business agreement with each client to determine. I think we should split this ANNOUNCE functionality into separate messages if just for the sake of discussion:
In the case of , a tuple makes perfect sense as it mirrors the arguments to SUBSCRIBE. This message would be primarily used for contribution to "push" a new broadcast. It also makes sense as a reconnect signal where the last message wins. In the case of , matching the longest prefix might make sense instead. I know very little about BGP but maybe there's some inspiration there. This message would primarily be used for inter-relay routing.
I assumed that full track name was a tuple already, and "full track name as a concatenation" was a notational convenience. Now that I read the draft more carefully, it does send full track name in messages, rather than a tuple of track name and track NS. As far as I'm concerned, this is a mistake -- I don't think this actually works, and I have no idea how one is supposed to implement that. I believe "error" is actually equivalent to "keep first" here. I think it's a safety vs practicality trade-off. On one hand, if two different programs announce the same track, it will not work, since they will publish conflicting objects under the same name; if we disallow re-announcing, we will prevent that. On the other hand, re-announcing makes reconnects more practical.
NAME Sorry if I misread the room and/or the author's meeting. Let's continue discussing. That is being discussed in / , let's keep this issue focused on the SUBSCRIBE REQUEST ambiguity?
Individual comment: Just to illustrate the problem further here, I wrote a short sketch of a generic relay (python syntax): So the concrete proposal I have is to change to have two fields, tracknamespace and trackname. Then the first bit would change to: This is not to say the every relay has to do matches in this way. By squishing the namespace and name together it's not possible to implement a relay that does exact matching. If we put the tuple in , a relay that wants non-exact matching rules based on full track name can always construct it by concatenating namespace and track_name together. NAME Can you say more about the use cases that would break with a change like this, or suggest an alternate way to implement an exact matching relay?
In a recent editor's call there was some discussion, which split into three separate issues - capturing it here: 1) SUBSCRIBE REQUEST should have a way to denote which part of the Full Track Name is the namespace. Either a Track Namespace Length field or just transmitting Track Namespace and Track Name in two fields. There was consensus to add this. 2) Does relay matching behavior (eg: exact match vs longest prefix vs ??) need to be negotiated in band? () 3) Is there a way to subscribe to more than one track at a time (eg: wildcard subscribe, ). I'd like to keep this issue focused on 1.

Related discussion:
text: A "SUBSCRIBE OK" control message is sent for successful subscriptions. <del> Full Track Name: Identifies the track for which this response is provided. </del> <ins> Track Namespace: Identifies the namespace of the track as defined in (track-name). Track Name: Identifies the track name as defined in (track-name). </ins> Track ID: Session specific identifier that is used as an alias for the Full Track Name in the Track ID field of the OBJECT (message-