jiebi/CodeConvo huggingface dataset:
. Since I wasn't sure if we can make normative references from an appendix, RFC5234 (ABNF) and RFC4648 (Base64) has been added as informative references.
NAME Thank you for your review. I believe that I have addressed all four issues in the commit above.
NAME Would it make sense to also describe the cookie-based method in this spec?
While we could do that, I am kind of negative against it. We need a specification for Cache-digest using HTTP/2 frames or HTTP/1 headers, since it needs to be defined as an interaction between a server and a client (that has the knowledge of what is being cached). OTOH, we do not need a specification for a Cache-Digest cookie, since it is totally controlled by the server. So it is entirely up to each server implementation to use whatever format it wants. Also, it is not easy to find out the right way to implement a cookie-based Cache-Digest that would work on all web browsers, since the timing the browser consumes the cookies sent from the server depend on the timing of the response. To summarize, the relatively low necessity and the technical difficulty makes me hesitate in pursuing the idea.
The change switches the order so that there is a more natural progress from presenting the incremental default of false, to discussing how a server could act on non-incremental requests, then how they might act on incremental requests.
(Emphasis mine.) I think that the goal of this text was to establish an expectation. That is, if a request is marked as incremental, there is no expectation that a server will attempt to send the entirety of a response before starting on other responses. It's a negation of the standard expectation that exists for : responses are transmitted in their entirety where possible. Consider rewording this bit.
Some suggestions, but I think that this is an improvement.Thank you for the PR. LGTM modulo the points below.
Addresses feedback in issue NAME
Feedback from Dean Saxe on WGLC Section 3.3.1 the first sentence should note that the QR code is associated with the particular service (Netflix, AppleTV, Disney+). Readers could assume that the QR codes originate from the TV manufacturer’s service alone as written. URL
Adressed in Rifaat, I have a few minor nits in the doc, nothing of significant concern for WGLC. When describing the visuals documenting the flows, there is a step that includes “The user authenticates to the authorization server”. In each case this should include verbiage to indicate that this is only necessary if the user is unauthenticated, e.g. “If unauthenticated, the user authenticates to the authorization server…”. Specific sections include 3.1.1, 3.1.2, 4.1.1, 4.1.2 Section 3.1.3 the final sentence notes the authorization data may be delivered as a text message or via a mobile app. This is inconsistent with the methods mentioned in the first paragraph, which includes email and text messages. I suggest being clear that these are example mechanisms and not a full list of mechanisms by which codes can be delivered. Section 3.3.1 the first sentence should note that the QR code is associated with the particular service (Netflix, AppleTV, Disney+). Readers could assume that the QR codes originate from the TV manufacturer’s service alone as written. Section 4.3.9 reads, “… using an e-mail campaign etc.” Should this be rewritten, “using an e-mail campaign, for example.”? Section 6.2.3 discusses FIDO CTAP 2.2. This document is still in review draft 01. We should note that the document is not final as of today. Section 6.2.3.5 could be softened a bit. The first sentence should include, “… and a suitable FIDO credential is not available on the consumption device.” In most patterns, this mechanism is used to bootstrap a new credential on the device, rather than using this mechanism for authN every time. Authors, if you have any questions please let me know.
Thanks, -dhs Dean H. Saxe, CIDPRO (he/him) Senior Security Engineer, AWS Identity Security Team | Amazon Web Services (AWS) E: EMAIL | M: 206-659-7293 From: OAuth on behalf of Rifaat Shekh-Yusef Date: Monday, April 22, 2024 at 7:57 AM To: oauth Subject: RE: [EXTERNAL] [OAUTH-WG] WGLC for Cross-Device Flows BCP CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. We have not received any feedback on this document so far. This is a reminder to review and provide feedback on this document. If you reviewed the document, and you do not have any comments or concerns, it would be great if you can send an email to the list indicating that. Regards, Rifaat All, This is a WG Last Call for the Cross-Device Flows BCP document. URL Please, review this document and reply on the mailing list if you have any comments or concerns, by April 29th. Regards, Rifaat & Hannes
PR updated, PTAL
This came out of discussion on When a relay receives an SUBSCRIBE it may need to issue its own SUBSCRIBE to a publisher that has previously ANNOUNCE'd a namespace matching that track. The current draft doesn't specify how this match is done. A relay could implement an exact match on Track Namespace (requires a tweak to SUBSCRIBE), or could do a longest prefix match on track name vs all ANNOUNCE'd namespaces, or possibly something else. Is it ok to leave this up to implementations or should the draft specify this behavior more exactly? If more than one matching type is allowed, does it need to be negotiated, and by whom (publisher, subscriber, or both)?
An issue that came up in interop that is somewhat related - NAME relay was implemented to take the namespace from the connect URL, because in his use-case, ANNOUNCE can be made redundant with the connect URL (eg it contains a namespace and enough auth). However, this caused an interop problem because my chat application assumes a certain functionality from relays, and that includes supporting ANNOUNCE messages to determine routing to publishers. Do we need to be more prescriptive about relay behavior?
ANNOUNCE is what my implementation expects as well. A producer can announce track namespaces at any point (after being connected). Overloading connect URL with track info is limiting
This is going to be a slightly long comment but this is how I think this should work. The catalog has two fields for the track namespace , and track name. We also specify in the catalog a way to form a URL from these two fields. See Teds proposal and that is catalog specific. The things we put in these track namespaces and track name fields are already canonicalized in a way defined by the catalog and URL. When the application using the moq transport puts these two fields into message, they are treated as just a bunch of bits and any comparison of them is done with bitwise compare. A relay that needs to check if a namespace matches another does not need to worry about string prep or escape encoding of the bits, the will all have been done when the catalog was formed. This allows the relays to work faster by just doing bitwise compare yet at the same time allows us to use URI at the application level. The transport draft just needs to deal with a bag of bits.
The catalog draft pushes the work to the URL definition, and the URL definition does the heavy lifting but that is where this type of stuff should be
NAME Just to clarify that when you say "bitwise compare" you also mean exact matching the entire namespace field from announce to the one in subscribe?
I would like to decouple the media layer from the transport layer. A tool like ffmpeg should be able to generate a MoQ catalog without knowing the namespace it will be served from, much like a tool like zip should be able to generate a file without knowing the path it will be stored. HLS/DASH provides this functionality with absolute or relative paths. If the playlist contains relative paths, then it can be served from any HTTP host/path provided the segments are served from the same host/path. If you want to decouple the two (ex. application serves playlist, CDN serves segments), then you use absolute URLs. To accomplish the same thing, the namespace should be optional in a MoQ catalog. This would mean media tracks are served from the same namespace as the catalog: whatever that might be.
To expand further, I think that MoQ broadcasts should be namespace agnostic: a broadcast ingested with namespace XXX is NOT required to be delivered as XXX. I understand where Cullen is coming from: an application like WebEx can use the same namespace for contribution/distribution because it controls the full pipeline. The application has all of the information required for all future routing/identification so it can set it on the first hop and expect any relays to blindly proxy. However, for a more generic or distributed system, this is not true. A client like OBS knows nothing about how the broadcast will eventually be distributed, fanned out, or the requirements of potentially multiple CDNs that might be used, nor will those requirements be the same between vendors (present or future). So I don't think we should require that input namespace == output namespace. In the HTTP world, this would be like requiring that all HTTP paths match the origin. It's quite restrictive and just means a CDN needs find another way to perform routing via a custom header/parameter. I propose we emulate the HTTP model where there's non in-band notification that a path/namespace was modified. An origin might serve and a CDN might decide to serve it as .
Any namespace remapping from XXX -> YYY is business logic and not signaled over the wire. If you don't want any remapping, then that's a business agreement between you and the CDN, wherein you may need to encode namespaces in a vendor-specific way.
I think we have agreement on that from past discussions. Can the catalog draft have a mechanism to inherit the namespace for tracks from somewhere else, like a relative URL in an HTML page? I'm not sure this is relevant for how relays perform matching between SUBSCRIBE and ANNOUNCE though. The only available information in announce is the namespace.
Yeah I'm probably rambling about something unrelated. But in order to support namespace remapping, the downstream ANNOUCE/SUBSCRIBE won't necessarily match the upstream ANNOUCE/SUBSCRIBE.
This is a useful property. I opened to address this issue and PR URL to fix it.
So if the input full track name does not equal output full track name ( for some version of matching the two ), what is it that tells the relays there is mapping between the two. I'm not objecting to some other way to do this, I just don't understand what it is that tells the relays to do that. Keep in mind relays do not need to read the catalog.
To clarify the "bitwise compare" ... yes I mean exact matching the entire namespace field from announce to the one in subscribe? If the namespace was utf-8 or even a url restricted utf-8, that would match. I am just aruging that a relay should not have to verify every string is a valid utf-8 string, it should just match the bits.
HTTP proxies will not modify the path by default. A downstream request for arrives, there's some rules to match it to a backend, and then an upstream request for is sent. A simple MoQ relay would work the same way. However, you can configure a HTTP proxy to perform special behavior, including rewriting the path. Note that a the HTTP server does rewrite the HLS/DASH playlist provided they are using relative URLs since both the playlist and segment would match the same prefix rules. For example, here's a simple nginx configuration: I would allow the same exact thing for MoQ, but of course use our terminology instead. I think a key improvement with MoQ over HTTP is actually the namespace/name tuple.
I propose: A relay SHOULD NOT change the track name. That means a producer can make a track called "video_480p" and reference it in a catalog, inheriting the current namespace. A relay MAY change the track namespace when negotiated out-of-band. It's RECOMMENDED that an application support being served from arbitrary namespaces, including using relative namespaces when tracks reference each other. Basically, the track name is controlled by the application (encoder/decoder) while the track namespace is controlled by the relay (hop-by-hop). This makes a ton of sense if the namespace is primarily meant for routing (critical for relays), and it allows relays to do stuff like encode the route or a unique ID into the namespace itself.
Going back to the original issue Alan posted (if I understand it correctly), I think we need to support out-of-band negotiation given the nature of fanout. Maybe we can provide something in-band but it won't always be possible. For example, an OBS client sends: In theory, the server could reply with the information required to watch the broadcast (final hop), including any remapping along the way: But this gets really messy when you factor in auth, multiple CDNs, multiple edge hosts, and just the nature of ingest being on the other side of the pipe from distribution. In this example, I think the application has to negotiate with Akamai on how they need to ingest and how they need to egress. I'm not opposed to using extensions to make this mapping explicit though. It would be cool to have a scheme that defines routing/authentication using globally unique namespace, kind of like a decentralized CDN. But it shouldn't be required as part of the base protocol and there's a LOT of sharp edges to figure out.
Individual Comment: My opinion is that anything that does this kind of rewriting is out-of-band, at least for now? For a completely generic relay input=output.
Yes. I opened issue URL to address this and PR URL to fix it. I really like the flexibility that relative names bring. We know this works at scale for current web and media delivery across disparate systems. I'd hope that relative track names are the normal mode of operation. We should allow absolute namespaces in catalogs to allow mixing between sources, however I think this will be an edge case. By default, a relay SHOULD NOT modify the track namespace or track name.
However, what a distribution network does within the bounds of its own network is up to it. As it restores the original namespace and name at its boundaries (to match the catalog), then external systems cannot tell and distribution intent is preserved. Additionally, there will be business logic and distribution agreements between CDNs, coordinated by the content distributor, to rewrite namespaces between networks. Such namespace rewriting is common between networks and should not be prohibited. Here's how I envisage a scalable solution working. When a twitch user wants to use OBS to publish, they would log in to URL and authenticate against the Twitch CMS. It would vend them two URLS: a connect and publish URL a playback URL to share with their followers The OBS client would use this to The client then publishes twitch/kixelated/catalog, a track which describes two relative tracks - audio and video. The playback client connects to URL and subscribes to premiergaming/gamer34/catalog. After parsing the catalog it subscribes to The Fastly relays have been configured by Twitch, out-of--band, so that when they go forward to origin (which is Akamai), they do a namespace rewrite to The subscriptions make their way back to the OBS instance. The encoder prepare Moqt Objects with in their headers. As these are received by Fastly, business logic rewrites them to , caches them as such and they are delivered to the client.
As per the discussions in person, it is more nuanced than renaming few bits in the header. It might break e2e encryption /auth for example or reencryption in some cases. As this is very application specific, one needs to be take into consideration the costs as it may not be just rewrite few bits in and out of network edges. The more appropriate or clearer framing may be - whenever the name changes, it is ,in a way, republishing the content with a new name ..
just to note, the network wouldn't have access to the catalog
Conclusion is that we'll use exact matching like specified in
Thanks for the update I'm not sure how to wordsmith this but +1 to what it is saying.
Assuming URL is merged, that document will register everything that is needed for SCITT Receipts (a profile of COSE Receipts). There is clean up work to do in the architecture. Remove the editorial notes regarding 394. Review IANA considerations (AFAIK, 394 was never requested for registration in the document, despite the editor notes).
I've merged the PR, but we need to publish a new version to the data tracker. Ideally other authors (of COSE Receipts / COMETRE) will do an editorial pass, and then push a new version to the data tracker.
I'd personally prefer to separate the mediatype exclusion (which I agree with removing) to a different PR, but would approve both anyway. LGTM
The other option is we can use a delta encoding to make it impossible to specify an earlier object, but I think that would make it more complex.
Good point, I changed it to the same or later object
As mentioned in the sprawling , I think we should allow empty subscriptions, aka subscriptions that match no objects. It would be nice to have a way to check if a track exists without receiving any OBJECTs. The MoQ equivalent of a HTTP HEAD request. It could be useful to pre-warm the cache. An empty subscription could be the equivalent of "I'm interested, but don't have the bandwidth to receive objects yet". Again, similar to HTTP HEAD requests. For example, a client could issue a to accomplish two separate things: The track exists if it gets a SUBSCRIBE_OK. The encoder/CDN starts producing the track. The client could resubscribe at any point to immediately start receiving the track.
Can the same be accomplished with: or any other range request where start==end? Is it ok for the publisher sends SUBSCRIBEOK and SUBSCRIBEFIN, or does your use case require the publisher to hold the subscription open? Adding a new message is also an option.
I think there's a few potential use-cases: Check if the track exists. Check if the track is active. Check if the track is active, and get notified when the track is closed. Check if the track exists and group/object N exists. "exists" vs "active" is tricky. If you request absolute/0, then it's totally valid for the relay to return OK so long as the objects haven't expired yet, even if the publisher sent a FIN a long time ago. But what should the relay do if absolute/0 has expired? Should it return an error, even if the track is still actively producing objects? Requesting relative/0 is better, although what if that object has expired too? For example, a muted audio track. IMO ? SUBSCRIBE start/end=none ... UNSUBSCRIBE SUBSCRIBE start/end=none ... SUBSCRIBEFIN SUBSCRIBE start/end=absolute/N ... SUBSCRIBEFIN
And a more concrete use-case that NAME and NAME might like. Let's say I have 3 renditions: .
The client starts watching a broadcast: This is a hint to the relay that the 720p track exists and it should start warming the cache. The relay is free to ignore the hint, but at the very least it will perform authentication. When the client decides that it has enough bandwidth for 720p, it will switch up at an aligned group: If the relay warmed the cache, then this switch is instantaneous. If the relay ignored the hint, then it takes at least an RTT to origin to fetch the 720p track. The time it takes to switch renditions matters more when switching down in response to congestion. The buffer will deplete while the CDN fetches the rendition from origin, which pre-warming subscriptions will avoid. And just to complete the example, you have to pre-warm the 480p SUBSCRIBE again. This could either be done with multiple subscriptions, or waiting until the SUBSCRIBE_FIN arrives and then subscribing again. I'm not quite sure which approach is better.
I'm a little concerned about this use case, for the reasons below: Is the assumption that the catalog, which declares the existence of the track, is not trustable? How? If the relay went forward to the origin with the same start=none request then the origin should not actually send anything and therefore the cache would not fill. The only way this works is if the relay goes forward with a different request with some !none start value. This non-deterministic behavior (ask one component for one thing and it asks for somehting else) can lead to scaling and debugging problems. I appreciate the benefits of cache warming, for fast start especially when the the edge relay is far from the origin. There are other ways to achieve this. One way can be by having a "warming client" make a conventional subscription request. This client may be an agent hosted on the edge relay for this purpose and its APi would be offered by the CDN and is outside the scope of moqt. As a third point, empty subscriptions are also a DDOS attack vector. As a client, if can make a hundred requests for streams that all then flow to a relay, but no content actually comes to me, it is a highly asymmetric (and therefore dangerous) DDOS tool.
I think it's useful for applications that don't use a catalog. ex.
Alan's chat protocol could check if a chat room or participant exists by performing an empty subscribe. The relay doesn't have to forward ranges verbatim. Just like if a CDN receives a HEAD request, the relay MAY issue a GET request to the origin instead. I think this is actually pretty common? In the MoQ case, the relay could request instead of from the origin. It's completely up to the relay; it would only do this if there's sufficient bandwidth/cache available. It's a little bit of business logic though; maybe the customer can choose to enable it. Just keep in mind that we don't want the relay to have access to the catalog. So you're asking applications to provide this functionality, requiring a custom service worker, a custom player, and hooks to notify the service worker on player start/stop. And the service worker itself doesn't actually want to receive these objects, so yeah there would need to be a non-standard API to actually warm a track. Plus it gets more complicated when you through in alternative renditions. Maybe the client has no intention of downloading AV1 tracks, or maybe it has no intention of downloading Japanese subtitles. This sort of selection criteria should be signaled to the service worker; pre-warming all tracks is a waste. I think this is the type of thing we build into the protocol instead of asking every application make a unique implementation. Warming would be completely optional for both the publisher and subscriber. I don't quite understand. The CDN would deduplicate subscriptions so only one would ever reach the origin for a given track. If the relay already has an active subscription for the 480p track, then it wouldn't need to issue another subscribe upstream to serve empty subscriptions. But if you're talking about using random track names, a client can already SUBSCRIBE to random track names (just like random HTTP paths) to DoS the origin. Or maybe you're talking about a client potentially pre-warming a bunch of valid tracks. Like somebody wrote a bot to scrape all Twitch broadcasts with zero viewers and issued an empty subscribe to them all. The CDN is completely in charge of determining if it pre-warms empty subscribes, which it should not do if it's near capacity or detects abuse. But I don't think this is unique to empty subscribes.