Original text:
6.12. <del> The Track Request Parameters identify properties of the track requested in either the ANNOUNCE or SUSBCRIBE REQUEST control messages.  The peers MUST close the connection if there are duplicates.  The Parameter Value Length field indicates the length of the Parameter Value. </del>  <del> The format of "Track Request Parameter" is as follows: </del>  6.12.1. <del> The GROUP SEQUENCE parameter (key 0x00) identifies the group within the track to start delivering objects.  The publisher MUST start delivering the objects from the most recent group, when this parameter is omitted.  This parameter is applicable in SUBSCRIBE REQUEST message. </del>  6.12.2. <del> The OBJECT SEQUENCE parameter (key 0x01) identifies the object with the track to start delivering objects.  The "GROUP SEQUENCE" parameter MUST be set to identify the group under which to start delivery.  The publisher MUST start delivering from the beginning of the selected group when this parameter is omitted.  This parameter is applicable in SUBSCRIBE REQUEST message. </del>  6.12.3.  AUTHORIZATION INFO parameter (key 0x02) identifies track's authorization information.  This parameter is populated for cases where the authorization is required at the track level.  This

Reviewer comments:
This is a initial PR to get some basic definition of subscription hints. I would like to get us to an agreement for the basic proposal and then discuss further on error codes, edge cases and impacts/interactions with other messages (Sub Fin, RST, UPD)
Can you mark this as so it shows as having a PR attached (also possibly 111 and 260)? Also recommend closing as this replaces it.
I feel proposal is more ambiguous though. I agree we can make the proposed ending in this PR to make it easier to encode/decode. I have to get higher level idea agreed upon There are subscription hints. 2 hints are defined in this PR a) StartPoint b) Interval There may be hints we might want to add in future and hence a Hint structure identifies a type and value StartPoint hint can support 4 modes. These modes are only applicable when the HintType is StartPoint Interval hint is totally separate hintType that defines ranges of groups/objects
Having a registry is the idea and I was thinking once we get to agreement of Hint strucutres , then we can follow with defining error codes, registries for types and cleaning up track request params
Payload(b) is defined to have len and value . Here is the definition as defined in the draft today x (b): : Indicates that x consists of a variable length integer, followed by that many bytes of binary data.
So pushed few more commits cleaning up somethings and considering suggestion from Alan on TrackOffset and removing text on error. Thanks everyone for the review feedback Would love to hear feedback on the whole text.
Given the length of 245. Could you please summarize the use-cases to help focus the discussion. In the order of small to big, my suggestion of the order would be "3, 4, 5, 2, 1 "
There are 4 examples right at the end of 245, I copied them in this review comment: URL We can discuss tomorrow.
Alan proposed we use 6 parameters. Start Object Start Group Start relative / absolute flag End Object End Group End relative / absolute flag Preference for not two ways to say same thing. Use 0 to indicate current. Offset encoding. You can subscribe to the track twice with two different hints if that is what you want. Seems good consensus on call for this. Further conversation got to the point that this fails to be able to do both the first object of current group and current object in current group. So we need to do that too. Update on this luke porposed make rel/abs flag for both object and group which solves this
My proposal is 4 parameters, combining the flag with the value: Start Group + Relative/Absolute Flag Start Object + Relative/Absolute Flag End Group + Relative/Absolute Flag End Object + Relative/Absolute Flag I'm using the lower bits of the VarInt to save some bytes and make parsing easier. Otherwise you have to deal with missing the corresponding sequence/mode parameter. I think it's even easier if these 4 VarInts are just encoded as part of SUBSCRIBE instead of being parameters. There would be a None flag if you don't want an end group and/or object.
+1 on making it part of Subscribe.
Regardless of the approach, I think these are the use-cases we MUST support: I think these are use-cases worth considering: And not mentioned in this PR, but I do think we need UNSUBSCRIBE hints.
New proposal: Range requests would be accomplished with a SUBSCRIBE followed by an immediate UNSUBSCRIBE. This is a race though, and a reason to not take this approach. Keep-alive would be kinda funny if only the latest UNSUBSCRIBE is active.
In addition to what is summarized above, we had a discussion about the way to express relative ranges. In the case of the start hint, the well understood scenarios that I heard are: start at a specific point (specific group, specific object) start at the current point (current group, current object) start at the beginning of the current group (e.g., because the receiver needs all the frames from the beginning of the current group to synchronize its decoder) start at the beginning of the next group (e.g., because the receiver does not want to process old frames, and would rather start clean on a group start) We can make it more complicated, and we have proposals such as: start N (groups, objects) before the current one (in the past) start N (groups, objects) after the current one (in the future). I think that these relative offsets bring a lot of complexity, and a lot of corner cases. They are somewhat undefined, because the subscriber does not know what the current group or object is. If it knew, it could just ask for a specific start point. The combination of groups and objects is a bit weird, because the subscriber does not know how many objects there are in the past groups. The syntax allows for silly requests, like "start 1,000,000 groups in the past", or requests that are latent DOS attack like "start 1,000,000 groups in the future", which means "keep state in memory forever". There is a potential simplification there...
I want to keep the scope simple and focussed. When i presented on Subscription hints, there were few real use-cases that had a need for hints User joining a mid call and want to start from the beginning of the group to catchup or wait until next group User rejoins mid call and want wherever the current state of group/object is User disconnected and joining at a known point and want to catch up from there ( say to 2x catchup render) User wants to buffer a few seconds from past to keep the player go smoothly User asking for range of frames and be done with it as catch up Can we define a system that works for all possible combinations ever - Oh YEAH .. But do we really need it - I DOUBT IT I am waiting on Alan's proposal for the promised simplification. NAME no pressure :-)
I was pretty busy today but have been written up what we discussed during the call - using 6 track request parameters offering the full range of options that are available in this PR as is. By using defaults when a param is omitted, there was only one invalid combination, which I like. The logic to determine what group and object to actually start serving is about 20 lines of pseudo code, which I added as an appendix. I'm not sure what to make of the discussion on this PR. I'll write a second PR that is simpler and just includes the 4 basic modes, and maybe something for range. Stay tuned. Would it be better to submit both as separate PR's, and leave this proposal as is, or should I push what we discussed this morning here?
Its ok to push here .. how easy would it be merge both into a single commit ?
Commenting on Luke's table of use cases: Love this way of enumerating and sorting the use cases. Thanks. All the use case I think are critical are covered in the first table and nothing that I did not care about seemed like it was going to complicated implementations so my first reaction was YES. But looking at it, we were getting close to covering 100% of all the cases and since relays are likely to upgrade slower than clients, I just came down to we should just support all the possible combinations even if it is not clear there is s use case for it today - that will actually make implementations simpler. I think the proposed solutions are starting to look like they do all the cases anyways. By all the cases, I mean a subscribe can Start at absolute or relative for both group and object. Relative can have delta to current. Start at absolute or relative or fin for both group and object. Relative can have delta to current.
On the topic of do we need unsubscribe hints .... How I think of it is subscribe installs some state on the relay or publisher telling it what to do. It can say what the first and last object is that should be forwarded. However, I think as unsubscribe as the message that removes all that state. If we start making unsubscribe be "forward until you see object X" then remove all the state, we get into questions like what happens if object X-1 arrives after X. That said, I do agree with the use case that Luke is getting at of things like. Client was subscribed to something forever, but now I only want to be subscribe up to the end of the current group so that the client can do a clean switch over to some different subscription at the group boundary
I was thinking about use case for things like text media some of the stuff with the game moves draft that are not audio and video. It is pretty common to want to be able to get the last N things in a group first then later go get the earlier things in a group. The usual rest style pagination type thing where. Imagine you had a group where each object was a message in a chat, you might only want the most recent 10 messages but then when user scrolled back, to get the rest. The more I thought about this, I think we have a requirement for start group notes -- -- absolute Get last N object in a given group
Yeah, I'm thinking we should use RESUBSCRIBE instead of adding hints to UNSUBSCRIBE. UNSUBSCRIBE would always be an immediate and cleanup any state, while RESUBSCRIBE would be used to update the start/end bounds.
I took the liberty of adding a 3rd proposal to the PR but did not change anything else in the PR.
+1 - that does seem like it will be a very good way to deal with it
NAME I wrote this up on my phone before noticing your commit. It's basically your proposal 3, but not using optional parameters. It would be encoded as: I really like that Alan wrote some pseudo-code, although I agree that it shouldn't be in the final draft. The devil is in the details with this encoding, and I think this scheme is both the simplest and the most powerful. It's also the smallest on the wire if that's any consolation prize.
And I thought it would be a bug to specify startobject=none with my encoding, but it actually seems like a valid use-case. start group notes -- -- none Check if a track exists You would get a SUBSCRIBEOK if the track exists, but the subscription won't actually do anything. It kinda seems useful for specifying intent too. start group notes -- -- N Start warning the cache for group N
My proposal in one of the authors call was Subscribe Update message.
So NAME proposal in the thread above that is similar to mine just seems better than what I proposed. It covers all the cases, it is easy to describe and validate work and does not have a bunch of uses cases. I would put in a very weak argument for instead of taking the bottom two bits to encode the (absolute, relative, future, none ) , instead just put that into it's own VarInt. I would not argue very much for that because this is all going to get put in 5 lines of code I will never look at again and it's does not turn up on the wire in something that has to be small. Probably the worst part of is is someone looking at network dump might be confused but that just means the wireshark dissector needs to know about how to the >>2 works. I'd be totally fine with proposal as is. Some strong points I see for this: 1) we could merge it by Monday 2) It covers all the uses case we can imagine now plus has a high chance of covering the future cases we can not imagine now 3) It is simple to understand, think about, and test
A 0-16 range would work for the majority of relative/future hints, so it would definitely save that byte. But I'm fine keeping it verbose for now; the important part is the reusable encoding. would be useful for checking track existence without receiving OBJECTs. But let's rule it out for now because it would mean a subscription that will never match objects, which would need some extra text. would be same as but that's not obvious. We can also rule it out. is required to specify the FIN of a group. ex. end at the last object of group 69. is invalid though. I think it's a hint right now, since the publisher can choose to drop objects/groups on a whim. Even if you ask to start at group 3, you might never get it for congestion control reasons. But the bigger issue is that a subscriber doesn't know the upstream's expiration time. A publisher may only cache the last 3 groups, so a subscriber can't just ask for the the last 8 groups and expect to receive them all. Throwing an error in that situation seems futile. But I think the publisher should signal the (absolute) start/end range in SUBSCRIBEOK, which may be a subset of the requested range. Then it would be a start/end range request, resulting in a start/end range response.
I like the direction that Alan has take Lukes stuff. To bikeshed the Hint. I think "Location" would be a better name than "Hint". I think this is not really a hint but more a filter that relay will not send me objects on this subscription before the Start location or after the end Location. So if you want my bikeshedy 0.000002 cents, I would define Location { GroupMode (i), GroupValue (i), ObjectMode (i), ObjectValue (i) } ObjectModes = None, Absolute, Past, Future GroupModes = None, Absolute, Past, Future The the subscribe would have a Start and End Location. Location become a clean way to specify a specific a specific object in a track and may turn out to be useful in other parts of the spec.
I'd like to take a moment to reflect that the notation we are using for is both shitty and bad. Take Hint { Mode (i), [Value (i)] } Modes = None, Relative, Next, Absolute By shitty, I mean super non intuitive. What does (i) mean? Is [X] optional or an array? But no, that would be ... Can I have more than one X? If I had Mode(2) followed by [Value(i)], would it be bit packed or would the Mode be padded out to a byte? And what type is Mode. Or if Mode is a type what is then encoding of all that crap. Most schema languages are fairly intuitive to read and guess what they might mean. This is not. Then there is the bad. Say I want to define a type called Mode then have two fields in a record called ObjectMode and GroupMode both of type Mode. What do I do? Location { GroupMode (i), ObjectMode (i) } ObjectModes = None, Absolute, Past, Future GroupModes = None, Absolute, Past, Future This can not possibly be the right way to do it. I mean now that Mode is not reusable and repeats itself. If only we had a well defined language for expressing this. Oh wait we do URL
I went ahead and enumerated the permutations group notes -- absolute range request absolute tail of group absolute absolute full group relative head of live group relative tail of live group relative next audio sample(s) relative full group future head of future group future future future future group(s) none tail of track none The ones labeled with audio are only when you have partially independent samples. For example, each audio GROUP is 2s long and each audio OBJECT is 21ms. This would get you the last 105ms of audio (target initial buffer) most of the time. However, you could get unlucky and a new group could have started with less than that available, but it doesn't really matter if the target buffer is small because the player will just wait. Personally, I want to remove this use-case. If objects can be independent, then they should be separate groups IMO. So each audio frame would be a group of size 21ms. You could then combine the group/object mode since object boundaries can either be absolute or none.
I pushed a new commit with PROPOSAL 4, which we have been discussing, pulling the "best of" text and examples from the other proposals. I deleted the PROPOSAL 1 sample code and didn't add code for 4. There's one DISCUSS, whether we should allow Start/EndObject to be None and have defaults, or require them to be specified.
Currently, we have GROUP SEQUENCE and OBJECT SEQUENCE parameters that allow subscribing to a track at a specific offset. As far as I can tell, we do not actually describe what happens when those are not present. I imagine what most subscribers would want is to receive most recent N groups when they subscribe (for some subscriber-specified N that is determined by how large they want their buffer to be), and we should add a parameter for the said N.
This is a special case of Subscription hint issue IIUC () Wondering if we should discuss together
I hit this in moq-chat when traversing a non-caching relay - a client's subscription to a catalog track came after the most recent group/object was published, so the client didn't get a catalog. Do all relays need to implement caching of some kind in order to satisfy late-comer requests? Or something else?
I think the default behavior should be to cache/serve the latest group. It's basically required for any sort of catalog. But I do think we should support cache-less senders. They can basically only support from and will ignore requests to start subscriptions further back.
Subscribers needs a way to hint the publishers on when do they have the want the objects to delivered to them within the requested track. This is needed in case due to the fact that different media types can be handled differently by the application ( audio can be delivered to the application immediately, video needs some form of key-frame to have the best experience or something more specific that the application wants ) In that effect , i want to propose an experimental extension to the Track Request Parameters called "SubscriptionHint" with the following variants. latest (0): The "latest" subscription hint identifies delivery of the objects for the requested track to begin from the most recent group and object sequence. catchup (1): The "catchup" subscription hint identifies delivery of the objects for the requested track from the beginning of the group provided in the GroupSequence track request param. waitup(2): The "waitup" subscription hint identifies delivery of the objects for the requested track from the beginning of the new group.
One of the reason this is key, is often a client is doing a subscribe to a track with a different bandwidth because there is not enough bandwidth for the current track. ( the case where a client is receiving a high bandwidth track, and wants to move a low bandwidth track of the same stuff ). In this case, there is already congestion happening and the client wants to move to the new track quickly without making the congestion worse. Hints along the lines of this really help in this critical case.
Oof yeah this is a difficult problem that we need to solve. A new player can start X seconds behind the live playhead, where X is the target size of the jitter buffer. Real-time will set this to 0. An existing player can switch between tracks at timestamp Y. (NAME example) Neither the player nor the relay knows the mapping from group timestamp. Tracks might not have aligned groups; different offsets, frequencies, boundaries, etc.
My issue with this proposal is that (0) latest and (2) waitup are based on the relay's cache and are extremely racey. The assumption is that the relay always has the latest group/object in cache. The reality is that it often won't, nor does it even know what the "latest" group is supposed to be. For example: A player is subscribed to 720p and encounters congestion after . It wants to subscribe to 360p and it has three choices: If the latest object in cache is , then we receive some undecodable objects until starts. It's wasteful and exacerbate congestions, but hopefully for not too long. If the latest object in cache is , then is undecodable and and playback stalls until is generated in a few seconds (oof). If there's no cache, we could forward the SUBSCRIBE upstream with the and run into either of the two above scenarios. Note: the upstream may have it's own cache; it won't necessarily be the origin. If there's a partial cache, the "latest" object might actually be in the past. For example, another subscriber issued a SUBSCRIBE for 100ms ago, so our latest in cache might be . We incorrectly start delivering at and exacerbate congestion. (oof) Note that there's no way for a relay to know when it's transitioned from a partial cache (not caught up to live playhead) to a full cache (caught up to live playhead). Issuing two separate subscriptions, one for each mode, would be the only option, but that would temporarily double bandwidth usage (bad for first hop in particular). This works the best because it doesn't depend on what is in the relay's cache. It knows to always start at . However this requires groups to be aligned across different tracks (ex. HLS). If 720p does not correspond to 360p , then this isn't an option. It's also not an option for startup since the player doesn't know the latest group number. The player needs to somehow know which timestamps correspond to which group for each track. If the latest object in cache is , then we correctly wait until starts. If the latest object in cache is , then playback stalls until is generated in a few seconds (oof). If there's no cache, we could forward the SUBSCRIBE to the upstream and run into either of the two above scenarios. Note: the upstream may have it's own cache; it won't necessarily be the origin. If there's a partial cache, the "latest" object might actually be in the past. For example, another subscriber issued a SUBSCRIBE for 100ms ago, so our latest in cache might be . We incorrectly start delivering at and exacerbate congestion. (oof) Basically the same problems as (0), but on the other side of the coin flip.
That was a wall of text; hopefully I explained it well enough. Like I said initially, the fundamental issue is that neither the player nor the relay know the mapping from group timestamp. What if we resolved that? contains a timestamp field contains a relative/absolute timestamp. contains an optional timestamp for graceful termination. This suffers from some rounding/alignment issues but it's fine. Relative timestamp requests are also problematic with partial caches. track contains group/timestamp pairs. contains start group and optional end group. contains optional end group for graceful termination. This is absolutely the best option for relays. It knows what groups are missing in the cache and can fill them on request. It also means the relay can have multiple SUBSCRIBE requests upstream for the same track without fear of duplication. The problem is that the media protocol needs to produce a live group listing kinda like HLS. This incurs an extra RTT on startup, although it's super useful for seeking backwards and DVR.
I think there are scalability and flexibility benefits to keeping relays agnostic to time and in fact to any internal characteristics of the objects they are serving. Keeping them decoupled allows the content to change (and the complete timing mechanism if so desired), without having to change the relay networks. The introduction of relay’s requiring timestamp knowledge stems from the original assertion that I think this need not be true. We can teach players the relationship between group and timestamp without too much difficulty. We do this all the time with HLS and DASH. In both cases the HTTP servers delivering these formats know nothing about the relationship between segments and time, yet we are able to build players which seek to precise points in a stream. We can do something similar with our streaming formats and the ideas already presented in this thread by Suhas and Luke. One difference is that I would make would be to allow the group offset described by Suhas to be either an absolute or relative one. As a proposal, what if the WARP catalog listed two things: The GROUP duration in milliseconds. The name of a track which described GROUP number/timestamp pairs (exactly the track suggested by Luke above). We assume that this timeline track allows delta updates, perhaps describing the complete DVR window every minute at a group boundary, with incremental updates in between. Use cases: Start as close to the live edge as possible a. Since the last group represents the last independently accessible point in the stream, the player would issue a plain SUBSCRIBE b. The relay would serve content starting with the last group it has cached (its default behavior). c. If the relay does not have this content cached, and there is no pending subscription, then it would issue its own SUBSCRIBE upstream. Start with a 5s buffer a. The player would calculate how many group’s it would need to guarantee a 5s buffer. Assuming the advertised group duration was 2000, then it would need 3 groups. Using zeroth based relative group addressing, it would issue SUBSCRIBE (-2). b. The relay would serve from the 3rd last group that it had cached. c. If the relay does not have the content cached, it would send its own SUSBCRIBE(-2) upstream. If it has an open subscription, but has only received 2 groups, then it should wait for the 3rd before responding. Seek to a wall clock time of T seconds a. The player would subscribe to the /timelines track, using SUBSCRIBE so that it receives the last full DVR description. b. Based on the data in the timeline track, the player calculates that it needs a group number of 5321. It issues a SUBSCRIBE(5321). c. The relay recognizes that this is an absolute reference to a GROUP (as all relative ones are negative) and returns content starting at GROUP . Player encounters congestion at group 53 on 720p and wants to switch to 360p track a. WARP places the packaging restriction that group numbers across tracks must carry the same media time offset (it actually has this requirement already) b. The player knows it was playing group 53 when it encountered slowness. Its next chance for a switch will be at a group boundary. Since it knows the group duration from the catalog, it can decide whether it is better off waiting for the next group, or reloading from the same group. In this case it is near the end of the group, so it decides to fetch the next group. c. It issues a SUBSCRIBE (54) for the 360p track and a UNSUBSCRIBE(53) for the 720p. Cheers Will
Absolutely NAME The main difficulty with relative requests is deduplication: a relay has trouble merging subscriptions that use absolute and relative groups when the cache is empty. ex. SUBSCRIBE 480p -3 (50ms delay) SUBSCRIBE 480p 351 The relay doesn't know if the existing upstream subscription (-3) will fulfill both. In fact the relay doesn't know if 351 is even remotely close; it could be a group from minutes/hours ago. We don't want to download the same objects twice, but maybe that's okay when the cache is empty? I think we either: only use only absolute groups or add a way to OR an existing SUBSCRIBE
Thinking about it some more. If the player only knows the duration of each group (fixed), then it could use relative timestamps, but it's still a massive headache for the relay. Suppose two SUBSCRIBES from separate viewers: A relay with an empty cache issues the upstream. It starts getting some OBJECTs: . And then it gets a from a viewer. It would be an error to serve those three objects from cache. Why? Who knows if they were the latest three or from an hour ago. The relay could hypothetically augment the existing upstream subscription: . However, it still doesn't know which groups correspond to -3. No additional objects would be sent if 6330 actually was the live playhead, and the relay wouldn't know where to start the -3 subscribe. IMO the relay needs a way of converting relative groups to absolute groups for this to work. Maybe SUBSCRIBE_OK contains that translation? ex. . It's still complicated though.
You're indeed right. This the fatal weakness of the relative request approach. If the relay has the latest group cached and an active subscription, it works fine, but the player can never be sure of that and the relay can never disambiguate fresh from stale content. If the relay can do that translation, then why couldn't the player do the same and just request an absolute group to begin with? What are some solutions to this problem? Players always make absolute requests. Here are two ways to do this: a. The catalog instructs the player in how to calculate the group number for any given time, for example, via a start time and then modulus the group duration. To avoid players having to have an accurate clock, the SETUP response from the server could echo back the current time. This is essentially a DASH approach to timing. b. The player subscribes to the /timeline track which describes the wallclock time and media time associated with the start of each group. This is similar to an HLS approach and has the downside of requiring 1xRTT before starting to discover the latest group. It has an advantage that it can describe groups of varying duration and Each group carries a timestamp in its header indicating the UTC time at which it was produced. This is not violating any decoupling of relays and payload because the time of production is core characteristic of any payload and is independent of the internal media time. With these timestamps in place, the relay can judge freshness. A player can issue SUBSCRIBE (-5) which means gives me this track starting with any group whose timestamp T > (now -5). We could also consider SUBSCRIBE (from: 1695571105212), which would require the player to have an accurate clock (although SETUP could be used to address that). Note that this scheme also works for DVR seeking if the player knows how to translate group number to UTC time. Same as [2] except that we add both a media timestamp and a UTC timestamp to each group. This gives the flexibility to ask relays to begin subscriptions at both media time offset and absolute and relative UTC offsets. The downsides are that it binds moq-transport to internal properties of payloads. One way round this is in core moq-transport we can specify two "index fields" in the header of each group. These fields must be numeric and monotonically increasing. Relays don't know that these fields represent, however they can service subscription requests that reference them. The streaming format would then bind index A to UTC time and index B to media time. Other formats may bind different attributes, but all could still work with generic relays. After writing these out , I'm in favor of [3] with generic indexes.
Yeah exactly. UTC timestamps get annoying because of clock synchronization. It's also annoying because the player has to know the latency to the origin; requesting media that was generated 200ms ago results in a very different experience if the origin is 10 miles away versus 10k miles away. I like the idea of delegating the group timestamp mapping to the application. The catalog could contain a formula (like DASH) or it could maintain a /timeline track. That being said, we still run into the same issue; how are you supposed to request the latest group in the /timeline track? Do you always request ? And is the latest, or do you need to wait until you receive the full /timeline? I think there's a missing piece. If I were to implement this in HTTP, I would make a GET request with an ETAG to get the latest group/object sequence. There's no equivalent in MoQ which is problematic. I think we still need relative group requests, but the relays need an authoritative source somehow; it can't decide what is "latest" from cache.
I think that's not what it meant. By "latest", relays starts delivering whatever it has in the cache as the most recent group/object and it doesn't need to remember anything about it other than they happen to be in cache. "waitup" is saying given everything once you get objects from current group +1.
+1 on keeping the relays not aware of media characteristics. Since its complex and may not apply the same way to all the types of media getting transported (live vs dvr vs chat vs catalog vs something new). +1 to keep players aware of mapping from media time to group number. I would be interested in knowing where this will not work though. I agree with Will that the warp catalog can specify the needed relationship here Also the susbcribe message today has "Group/Object Id" in its track request parameters. I would proposes merging Will's usecases (2) and (3) into given a Time T, find the group number you want to start fetching the media Issue a subscribe with that group number Relay can decide if the requsted group number is behind or ahead of the current group and if it is in cache serve them accordingly Basically i think we don't need offset. But would love to hear from NAME where we can't do away with absolute groups
We need to be clear on the term latest. In my proposal of Subscription Hints "Latest" is in terms of latest in the cache. So if the cache is current running group-45, object-32, then that's the latest. And waitup is , wait until you see group number change before giving me objects. I think with the hints listed, groupid & objectid being in the track request parameters and player providing the mapping function from timeline to group, we can address most of the use-cases being raised here. I would interested to learn what use-cases cannot be addressed with the proposed scheme
Can you give an example using the "latest" mode? Would you use it for playback start or switching between tracks or both? As I understand, groups are independent but objects are not. would be a P-frame or catalog delta. I don't understand why you would ever want to start your subscription at an undecodable object. Starting at or would work, but never .
For the video delivery case, I would use "waitup" if i have no idea what the current group is. Since it will start delivery the objects from the beginning of the next group. I would use latest for audio use-case where the is fine for me to either render or conceal. Again we can enumerate the use-cases we want to address and I am happy to share my thinking around how the hints help. As part of exercise , we may come out deciding if we need more granular things than what is being proposed here. NAME could you please share one liner for the use-cases you have been considering from a subscribe perspective ?
The key point we're forgetting in the freshness quandary (or perhaps it is implicit) is that the relay knows whether it has an open subscription for a given track. This is a key bit of information. If it does have an open subscription, then the objects it has in cache are by definition the latest, and hence any 'relative' or 'latest' queries can be satisfied. If it does not have an open subscription, then it cannot make any determination about the freshness of any objects, even if it has some in cache and it must renew its subscription before responding. To illustrate this, consider a relay which has groups 637,638,639 in cache. Five minutes later another client joins issues a SUBSCRIBE (latest). The response of the relay should be different in the two cases below: It has an open subscription for that track - in this case it knows that those are still the latest objects available for that track and so it would return 639 It does not have an open subscription for that track (because the prior client that subscribed went away) - in this case the relay must make a SUBSCRIBE(latest) request upstream and return the first group it receives (perhaps 763) to the client. The advantage of relative requests is that the player does not need any understanding of the relationship between group number and time. This makes for simple players, as long as they don't need to seek behind live with any accuracy. The relative datum, in this case the "live edge", is constantly changing, and for players that need to seek back accurately this can be a problem. Depending on when you make your request and how long it is queued by intermediaries, you may get two different answers. Absolute groups make precise group selection for more deterministic and are robust. For live-only players (such as the web meeting use case), we probably could get by with relative addressing alone. However since our charter includes intermediate and higher levels of latency, we should design moq-transport such that it is flexible enough to handle both relative and absolute group addressing.
if there are no upstream subscriptions active and hence no published data is being received , then there are couple of things to note What's in the cache is the current source of truth and it needs to be served yes Relay may forward the subscription upstream and feed the data that arrives towards the subscriber ( and also add it to the cache) This will keep "latest" property preserved. Players need to handle gaps that might be caused for several reasons including relay cache purges, publisher going away/reconnecting, subscribers loosing state, relay reboots. I see these as player level details and not specific to moqt. Isn't it ?
It's not a source of truth at all. It may be 5 minutes old or 50 minutes old. Without the active subscription, there is no guarantee about its status at all other than it was previously served to some client and hasn't yet fallen out of cache. Why would a client wanting "latest" want that and if it was sent, how would the client know that it wasn't the latest. It would start rendering and then jump ahead when the true latest content arrived . If you need a subscription-hint for give-me-anything-in-your-cache-not-matter-how-old-it-is, we can create that, but thats different from "latest" and I'm not clear on its utility.
In particular, I would like to see MoQ also support some improvements for DVR or even VoD playback, for example: faster frame-accurate seeking. We don't necessarily need the protocol to provide a complete implementation, but having the ability to absolutely address groups would be especially useful in that context. One concern I have about adding timestamps into the mix is in how they would apply to content being played back at arbitrary points in time relative to the original recording time.
Here's my current thought. SUBSCRIBE can contain a relative or absolute start group. A bit flag is used to specify if it's an absolute or relative request. N means serve group N and onwards +1 means serve the next latest group (waitup) -0 means serve the latest group (catchup) -N means serve N groups prior to latest SUBSCRIBEOK contains the latest and . Useful for players: it knows it should start rendering after decoding object=K Useful for relays: enough information to serve relative requests from cache. Relays can compute "latest" if it has an active subscription: if SUBSCRIBE active: if SUBSCRIBE inactive: You can serve relative subscriptions from cache only once you have an active SUBSCRIBE and the corresponding SUBSCRIBEOK. If you don't have an active subscription, then you need to SUBSCRIBE again. The SUBSCRIBEOK will tell the relay how stale the cache is and what can still be served.
Since URL can only be equal to but never greater than URL, this simplifies to Another another topic, we should also allow the use case of subscribing to a stream before any groups have been produced. In this case we need some convention to convey that no groups are yet available, for example group=null and object=null
Nah, is updated on receipt of a new object. It's used for any new relative subscriptions. Here's a full example from the relay <- relay POV. The edge receives a SUBSCRIBE -3 from a viewer or downstream and has an empty cache: The subscription hasn't caught up to the live edge yet, but now the relay has a snapshot of how to map relative requests to absolute requests. If the relay receives a relative SUBSCRIBE during this state, it will know what can be served from cache. For example, a new SUBSCRIBE asking for 0 will not receive 6743 thanks to this hint. It's the only object in cache at the moment, so it would have been served with or but not with this approach. The SUBSCRIBEOK says that 6745 is the latest so we just have to wait for the data to finish transfering. This snapshot of latest will go stale, but the active subscribe will catch up soon. The snapshot exists mainly to support the first ~100ms where the subscribe is backfilling. In the meantime, the relay keeps receiving groups, possibly out of order: Upon receiving 6746, the relay updates the value since it's now larger than the snapshot from SUBSCRIBEOK. A new downstream SUBSCRIBE asking for -1 now will get 6745 and 6746 from cache. Note that there's no new upstream SUBSCRIBE; we can continue to serve relative requests indefinitely from cache unless they are too far in the past. Just to complete the example, let's suppose the relay UNSUBSCRIBEs right now for whatever reason. 6742-6746 are still in cache but we've wiped our "latest" value since the subscription is no longer active. The relay could get either a new absolute or relative downstream subscriber: : the relay serves 6745-6746 from cache while SUBSCRIBE 6747 is sent upstream. : the relay doesn't know if it can serve from cache since the active subscription has ended. The is sent upstream and the relay behavior depends on the resulting : : both 6745 and 6746 are served from cache : only 6746 can be served from cache : the cache is stale and can't be used In the first two cases, the relay would receive a duplicate copy of the objects it already has cached. That's probably fine, but a way to avoid this would be to subscribe based on both absolute and relative groups:
Please do not use these tricks of encoding meaning as the sign of an integer. It looks like a premature optimization, and in fact it is not: there is no syntax defined to encode negative integer, you will end up with a complement to 2 64 bit integer encoded on 8 bytes. It will be much cleaner and also more efficient to encode the hint as a tuple: meaning: an enum value describing what the subscriber wants, such as start at the current group, the next group, the first group, a specific group, or a specific location. additional parameters if needed, such as actual group ID if not "current", "next" or "first", and actual object ID if needs to be specified.
Consensus at the interim is that we definitely want this and we both want to relative (n groups back) and range requests (ie: group n to group m), which could be open-ended. Joining from the currently ongoing group (startpoint=-1 in slides), and the next group (startpoint=1) had strong interest. start_point=0 starts from the current object within a group and there is contention about whether it's necessary given groups are intended as start points, but there was enough interest in the room that it was felt the correct way forward was to support it and then we can revisit the issue in the future if needed. Additionally, there is interest in object granularity, particularly for the range requests. A solution should and .
+1 to christian and happy to help with the PR . thanks
Here's what I was thinking: These fields could be optional via a parameter, message type flag, or a . My preference would be the mode because it's easier to decode and encode. It would take 4 bytes to specify all of the start/end boundaries versus up to 12 bytes if they're all parameters. Here's how the modes from would map: I think this is simpler and more powerful. A publisher just needs to write a function to convert a relative boundary into an absolute one based on the latest value in the cache. This might be an asynchronous process for a relay if the cache is empty or new. Here's some things you can't do with the proposed encoding in : Start 4 groups in the past and end 2 in the future: Start at group 69 and end at latest: Start at 69 and end 4 groups in the past (might be a noop): Keep refreshing a relative subscription while it's needed:
Some use-case like catchup , go back to a given object for layer refresh or server the game state from a given object , and similar scenarios , would need a name to ask for the object. It would be good for the protocol to be able to subscribe to not just track and group, but also object.
As an individual: +1. An application should be able to request a single object. Maybe even a byte range of an object but I would punt that until an actual use case materializes.
Can you elaborate on the use-case? Resumption after a connection is severed makes sense. We need to solve that for both the contribution and distribution use-case. NAME has some strong opinions here. I don't quite understand the second use-case. Why would a consumer need to refetch a single object, but none of the objects it could depend on? It only makes sense when there's a hole in the group, but why did that happen and how does the subscriber know about it?
And to be clear, I also think we should have a GET equivalent, but I want to make sure I understand the ask.
The encoder might have skipped some frames due to input loss or another issue, but maybe they are available from a redundant encoder and you would like to "fill the gaps"? The subscriber only sees the gap, it does not need to know the dependency structure.
Two uses cases I am thinking, both on distribution side: Use case A: client join a session that is in progress and wants to join Track A. The client wants to start playing before the next the GOP starts so it subscribes to Track A with the most recent group and wants to get all the objects in the group. Use case B: client is watching Track A and has received all the objects up to group 123 object 45 and looses wifi and switched to 5G. At this point it forms a new subscription to Track A but it wants to start at group 123 object 46 because it already has the stuff before that. If it had to wait to received all the stuff in group 123 over again, it would add delay as it switched networks. There are some more uses from scalable codecs but if we can solve the above two cases, we will probably solve the toher uses cases as well.
I think the solution to this is simply that the Subscribe can have some sort of flag that indicates if the clients wants just future objects, or if wants to catch up on all the objects in the current group, or wants all the objects after some particular objectID
There isn't a way to GET a single object - you can SUBSCRIBE to the track starting from that object, and perhaps issues an UNSUBSCRIBE after that. I'm going to leave this open for now.
Will suggested solving this as part of and have both a start group and end group.
Merge it :-)

Revised text:
6.12.  <ins> Subscription hints provide a way for a subscriber to indicate, to the publisher, its preferences for receiving objects from a given track. </ins>  <ins> Subscription Hints have the following structure: The "HintType" parameter identifies one of the following values: "HintValue" contains "HintValueLength" bytes, which encode attributes as specified by the HintType.  If the HintType does not need extra hint attributes, the HintValueLength is 0, and the HintValue is empty. Future versions of the specification may define more hint types as needed. In the sections below, a publisher's current state of the track is defined by the most recent group and object sequence received (or active in the cache), if available, at the time of request. </ins> 6.12.1.  <ins> "Current" subscription hint specifies the start point for object delivery from the beginning of the current group.  Current hint type defines no further hint attributes. </ins> 6.12.2.  <ins> "Now" subscription hint specifies the start point for object delivery from the most recent object of the current group.  Now hint type defines no further hint attributes. </ins> 6.12.3. <ins> "RelativeStartPrevious" subscription hint specifies the start point for object delivery from an earlier group relative to the current group.  The "GroupCount" hint attribute specifies the number of groups to go back from the current group to determine the start point. 6.12.4. "RelativeStartNext" subscription hint specifies the start point for object delivery to a future group relative to the current group.  The "GroupCount" hint attribute specifies the number of groups to wait on before delivering the objects. 6.12.5. The "AbsoluteStart" subscription hint allows subscribers to specify an absolute point in the track to start delivering objects, as indicated by the "TrackOffset" hint attribute. The structure for AbsoluteStart hint value is as follows: TrackOffset: Identifies the group and the object sequence value within the track as the start point for the delivery.  TrackOffset is defined as below. "ObjectSequence" set to 0x0 implies until the end of the group identified in "GroupSequence". 6.12.6. The "AbsoluteInterval" subscription hint allows subscribers to request for range of objects by specifying values pertaining to start and end group/object sequences. The AbsoluteInterval hint value has the following structure: The end track offset of the range is exclusive. 6.13. The Track Request Parameters identify properties of the track requested in either the ANNOUNCE or SUSBCRIBE REQUEST control messages.  The peers MUST close the connection if there are duplicates.  The Parameter Value Length field indicates the length of the Parameter Value. The format of "Track Request Parameter" is as follows: 6.13.1. </ins> AUTHORIZATION INFO parameter (key 0x02) identifies track's authorization information.  This parameter is populated for cases where the authorization is required at the track level.  This