Session description message extensions

Session description message extensions include information describing a multimedia presentation or a single media presentation. The session description message is typically sent from one device to another device to which the multimedia or single media presentation will be streamed. Several extensions are described that extend the information that can be included in the session description message.

TECHNICAL FIELD

This invention relates to streaming media and data transfers, and particularly to session description message extensions.

BACKGROUND

Content streaming, such as the streaming of audio, video, and/or text is becoming increasingly popular. The term “streaming” is typically used to indicate that the data representing the media is provided over a network to a client computer on an as-needed basis rather than being pre-delivered in its entirety before playback. Thus, the client computer renders streaming content as it is received from a network server, rather than waiting for an entire “file” to be delivered.

The widespread availability of streaming multimedia content enables a variety of informational content that was not previously available over the Internet or other computer networks. Live content is one significant example of such content. Using streaming multimedia, audio, video, or audio/visual coverage of noteworthy events can be broadcast over the Internet as the events unfold. Similarly, television and radio stations can transmit their live content over the Internet.

The Session Description Protocol (SDP), Network Working Group Request for Comments (RFC) 2327, April 1998, is a text-based format used to describe properties of a multimedia presentation, referred to as a “session”, and properties of one or more media streams contained within the presentation. SDP has been developed as an application level protocol intended for describing multimedia sessions for the purposes of session announcement, session invitation, and other forms of multimedia session initiation. SDP can be used in accordance with other protocols, such as the Real-Time Streaming Protocol (RTSP) or the HyperText Transfer Protocol (HTTP), to describe and/or negotiate properties of a multimedia session used for delivery of streaming data.

SDP, however, is limited in the properties of the multimedia presentation that can be described. One solution to such limitations is to embed a different file header, such as an ASF (Advanced Streaming Format or Advanced Systems Format) file header, within an SDP message. Such a solution is described, for example, in U.S. Patent Application Ser. No. 2003/0236912, entitled “System and Method for Embedding a Streaming Media Format Header Within a Session Description Message”. However, embedding such a file header can create problems, such as requiring the recipient of such an SDP message to be able to parse and understand the different file header, and increasing the size of the SDP message that includes the different file header. Thus, it would be beneficial to have a way to overcome these limitations of SDP without requiring recipients to be able to parse and understand an embedded different file header (such as ASF).

SUMMARY

Session description message extensions are described herein. These extensions extend the information that can be included in a session description message for streaming media.

In accordance with certain aspects, the session description message extensions include one or more of: a mutual exclusivity extension that specifies which of multiple media streams of a multimedia (or single media) presentation are mutually exclusive, a marker extension that specifies an index into a multimedia presentation, a script command extension that specifies data that is to be presented during playback of a multimedia presentation, a leaky bucket extension that specifies how much data of a multimedia presentation to buffer when playing the multimedia presentation at a particular bit rate, a caching information extension that specifies information about caching of a multimedia presentation, an encapsulation extension that specifies codec parameters for one or more of a plurality of different codecs, a bandwidth sharing extension that specifies media streams of a multimedia presentation that share bandwidth, or a stream prioritization extension that specifies which media streams of a multimedia presentation should have priority over which other media streams of the multimedia presentation.

DETAILED DESCRIPTION

Session description message extensions are described herein. A session description message includes information describing a multimedia presentation or a single media presentation. The session description message is typically sent from a server device to a client device to which the multimedia presentation will be streamed. Several extensions are described herein that extend the information that can be included in the session description message.

FIG. 1illustrates an example network environment100that can be used to stream media using the session description message extensions described herein. In environment100, multiple (a) client computing devices102(1),102(2), . . . ,102(a) are coupled to multiple (b) server computing devices104(1),104(2), . . . ,104(b) via a network106. Network106is intended to represent any of a variety of conventional network topologies and types (including wired and/or wireless networks), employing any of a variety of conventional network protocols (including public and/or proprietary protocols). Network106may include, for example, the Internet as well as possibly at least portions of one or more local area networks (LANs).

Computing devices102and104can each be any of a variety of conventional computing devices, including desktop PCs, workstations, mainframe computers, Internet appliances, gaming consoles, handheld PCs, cellular telephones, personal digital assistants (PDAs), etc. One or more of devices102and104can be the same types of devices, or alternatively different types of devices.

Server devices104can make any of a variety of data available for streaming to clients102. The term “streaming” is used to indicate that the data representing the media is provided over a network to a client device and that playback of the content can begin prior to the content being delivered in its entirety (e.g., providing the data on an as-needed basis rather than pre-delivering the data in its entirety before playback). The data may be publicly available or alternatively restricted (e.g., restricted to only certain users, available only if the appropriate fee is paid, etc.). The data may be any of a variety of one or more types of content, such as audio, video, text, animation, etc. Additionally, the data may be pre-recorded or alternatively “live” (e.g., a digital representation of a concert being captured as the concert is performed and made available for streaming shortly after capture).

A client device102may receive streaming media from a server104that stores the streaming media content as a file, or alternatively from a server104that receives the streaming media from some other source. For example, server104may receive the streaming media from another server that stores the streaming media content as a file, or may receive the streaming media from some other source (e.g., an encoder that is encoding a “live” event).

As used herein, streaming media refers to streaming one or more media streams from one device to another (e.g., from a server device104to a client device102). The media streams can include any of a variety of types of content, such as one or more of audio, video, text, and so forth.

FIG. 2illustrates example client and server devices that can stream media content using the session description message extensions described herein. Client device102includes a streaming media player142configured to access a streaming module144of server device104. Media content including session description message(s)146is streamed from streaming module144to streaming media player142, where the steaming media can be presented to a user of client device102. The streaming media146can be streamed in accordance with different protocols, such as the Real-Time Streaming Protocol (RTSP) or the HyperText Transport Protocol (HTTP). Although not shown inFIG. 2, one or more additional devices (e.g., firewalls, routers, gateways, bridges, etc.) may be situated between client device102and server device104.

One or more session description messages are used to describe properties of the multimedia presentation streamed as streaming media146. Different media formats or protocols can be used for such session description messages. An example of such a media format is the Session Description Protocol (SDP), it Network Working Group Request for Comments (RFC) 2327, April 1998. In certain embodiments, the session description message discussed herein is a message in accordance with the SDP format described in RFC 2327.

Although different formats can be used to describe properties of the multimedia presentation, one or more session description messages are sent from streaming module144to streaming media player142that include identifier(s) of the properties. A single session description message may be sent by streaming module144for a particular multimedia presentation, or alternatively multiple session description messages may be sent. If multiple session description messages are sent, the multiple messages may include the same information, different information, or overlapping information.

A session description message includes, for example, one or more of: descriptions of each media stream available in the multimedia presentation (e.g., indicating the type of stream (e.g., video or audio), a bit-rate of each media stream, a language used in the stream, etc.); error correction information; security/authentication information; encryption information; or digital rights management (DRM) information; etc.

FIG. 3illustrates an example session description message format. Although illustrated as a specific example inFIG. 3, the session description message could have a format with fields or portions in different orders, or alternatively spread across different messages.

Session description message180includes a session level description portion182and zero or more media level description portions184. Session level description portion182includes one or more fields having data that applies to the whole session and all media streams that are part of the session. Each media level description portion182, on the other hand, includes one or more fields having data that applies only to a single media stream.

The data fields in media level description portion182describe properties for particular media streams. These properties may be in addition to properties described in session level description portion182, or in place of properties described in session level description portion182. For example, one or more properties in a particular media level description portion182may override, for the particular media stream associated with that particular media level description portion182, properties identified in session level description portion182.

Session description message180, and the structure of message180is discussed in additional detail below specifically with respect to SDP. It is to be appreciated that these specific structures are only examples, and that the session description message can take different forms.

Session level description portion182begins with a particular field, referred to as the protocol version field. Similarly, media level description portions184each start with a particular field, referred to as a media name and transport address field. In certain embodiments, multiple fields of the same type may be included in a session description message (e.g., a single session description message may have two or more attribute fields).

Table I below illustrates example fields that may be included in session level description portion182. Table I includes a name for each example field, an abbreviation or type for each example field, and a brief discussion of each example field. In certain embodiments, the protocol version field, the owner/creator and session identifier field, the session name field, and the time description field are required whereas all other fields in Table I are optional.

TABLE INameTypeDescriptionprotocol versionv =The version of the SDP.origino =The originator of the session (e.g., user nameand address of the user's host), plus a sessionid and a session version number.session names =The name of the session.session informationi =Information about the session.URI of descriptionu =A pointer to additional information aboutthe session.email addresse =Email address of person responsible forthe session.phone numberp =Phone number of person responsible forthe session.connectionc =Connection data describing the connectioninformationfor the session, such as network type, typeof addressing being used, and aconnection address.bandwidthb =The proposed bandwidth to be used by theinformationsession.time descriptionSee Table II below.time zonez =Specifies adjustment times and offsets toadjustmentsallow for daylight-saving time.encryption keyk =Indicates the mechanism to be used to obtainan encryption key for the session by externalmeans, or from an included encodedencryption key.attributea =Attribute of the session extending the SDP.

Table II below illustrates the time description field in additional detail. Table II includes a name for each field in the time description field, an abbreviation or type for each field in the time description field, and a brief discussion of each field in the time description field. The time the session is active field is required whereas the zero or more repeat times field is optional.

TABLE IINameTypeDescriptiontime the session ist =The start and stop times for the session.activezero or more repeatr =Specifies repeat times for the session.times

Table III below illustrates example fields that may be included in a media level description portion184. Table III includes a name for each example field, an abbreviation or type for each example field, and a brief discussion of each example field. In certain embodiments, the media announcement field is required whereas all other fields in Table III are optional.

TABLE IIINameTypeDescriptionmediam =The media type of the media stream, the transportannouncementport to which the media stream will be sent, thetransport protocol for the media stream, and themedia format(s) for the media stream.media titlei =Information about the media stream (e.g., a labelfor the media stream).connectionc =Connection data describing the connection for theinformationmedia stream, such as network type, type ofaddressing being used, and a connection address.bandwidthb =The proposed bandwidth to be used by the mediainformationstream.encryption keyk =Indicates the mechanism to be used to obtain anencryption key for the media stream by externalmeans, or from an included encoded encryptionkey.attributea =Attribute of the media stream extending the SDP.

In certain embodiments, the session description message extensions discussed herein are included as attribute fields in the session level description182and/or one or more media level descriptions184.

FIG. 4is a flowchart illustrating an example process200for extending a session description message for a multimedia presentation. Process200may be performed in software, firmware, hardware, or combinations thereof. Process200is typically performed by a source of the multimedia presentation (e.g., an encoder device that is generating the multimedia presentation). For stored multimedia presentations, the information included in the session description message extensions is typically also stored along with the multimedia presentation or alternatively elsewhere. When the multimedia presentation is to be retrieved and streamed to a client device, the stored information can be retrieved and the session description message, with extensions, generated. For live multimedia presentations, the information included in the session description message is added to the session description message (e.g., by a human user or automatically by a computing device) at the beginning of the multimedia presentation and/or during the multimedia presentation (e.g., information may be added to or changed in the session description message as the live multimedia presentation changes).

Initially, the information to be included in the session description message is identified (act202). This information can be identified in one or more different manners. One manner in which this information can be identified is retrieval from a file or record. In certain embodiments, at least some of the information is stored in a file or record associated with the multimedia presentation that the session description message is to describe. This file or record is accessed in act202to retrieve the information stored therein.

Another manner in which this information can be identified is receipt from a human user. In certain embodiments, at least some of the information is received from a human user. These user inputs are used in act202as at least some of the information to be included in the session description message.

Another manner in which this information can be identified is automatic detection. In certain embodiments, at least some of the information can be identified automatically by a computing device by analyzing the source of the multimedia presentation or the multimedia presentation itself. This automatically detected information is used in act202as at least some of the information to be included in the session description message.

The extensions to be added to the session description message in order to include all of the identified information are then identified (act204). These extensions can be identified in different manners. One manner in which these extensions can be identified is automatically based on the identified information. In this manner, given the information that is identified, the particular extensions to be used to include that information in the session description message are automatically identified. For example, a table or record mapping information to extension may be accessed, the device performing the automatic configuration may be configured or programmed with the appropriate extensions for particular information, and so forth.

Another manner in which these extensions can be identified is identification of the extension information with the information itself. For example, when the information is identified in act202, the information may also include the identification of which extension to use to include the information in the session description message.

The session description message is then generated including the extensions identified in act204and the information identified in act202(act206). Once generated, the session description message can be sent to the client device to inform the client device of the information describing the multimedia presentation associated with the session description message.

A wide variety of information can be included in the session description message extensions. In certain embodiments, these extensions include information that would also be included in an ASF file header associated with the multimedia presentation. By using the session description message extensions to include such information, the ASF file header would not need to be communicated to the client device. This alleviates the need for the client device to be able to understand the ASF file header, can reduce the size of the session description message by not embedding the entirety of the ASF file header in the session description message, and can eliminate the need to generate an ASF file header in situations where an ASF file header has not already been generated.

It should be noted that the session description message extensions discussed herein are separate from the ASF file header. These session description message extensions are defined as being separate from any ASF file header (i.e., these session description message extensions are not simply part of an ASF file header that is embedded in a session description message). These session description message extensions can also be used in situations where there is no ASF file header associated with the multimedia presentation.

Table IV below is a summary of several extensions that can be included in a session description message. In certain embodiments, each of these extensions are added using an attribute field of the session description. The discussions of these extensions following Table IV indicate whether these extensions are typically found in session level description182or media level description184, although alternatively these extensions could be located in either session level description182or media level description184.

TABLE IVNameDescriptionMutualSpecifies whether media streams are mutually exclusive.ExclusivityMarkerUser-friendly indexes into the media stream(s).ScriptSpecifies text (or other data) that is to be displayed or anCommandidentifier of a web page that should be displayed at aparticular time during playback of the media stream(s).LeakyWhen playing back variable bit-rate content at a constantBucketbit-rate, specifies how much data of the media stream(s) theclient should buffer.CachingSpecifies information about caching of the media stream(s),Informationsuch as whether the stream(s) can be cached, how long thestream(s) are considered fresh, etc.RTPSpecifies codec initialization parameters generically acrossEncap-a broad variety of codecs for data transmitted over RTP.sulationBandwidthSpecifies streams that share bandwidth so that theSharingmaximum bandwidth of the set of specified streams is lessthan the sum of the maximum bandwidths of the individualstreams.StreamSpecifies which streams should have priority over whichPrioritizationother streams (e.g., which streams should or should not bedropped in the event of network congestion).

Each of the extensions included in the summary of Table IV are described in the following in additional detail. Various example syntaxes are discussed below for the various extensions. It is to be appreciated that these are only examples, and that other syntaxes can alternatively be used. For example, the parameters or the order of the parameters may be changed. By way of another example, the names of specific attributes may be changed (e.g., the name for the mutual exclusivity attribute discussed below may be changed to something other than “a=group”).

It should be noted that references are made herein to identifications of particular media streams. These identifications can be made at different times. In certain embodiments, identifiers of the media streams are created by an author or designer (or alternatively automatically) when the media stream is encoded (e.g., for a live broadcast) or when the media stream is stored (e.g., as an ASF file). In other embodiments, identifiers of the media streams are created when the streaming of the multimedia presentation (or single media presentation) from the server device begins.

The identifiers themselves can be generated in any of a variety of manners. In certain embodiments, the identifiers are identifiers of RTP sessions. In these embodiments, individual media streams are streamed from the server device to the client device using the Realtime Transport Protocol (RTP) as described in Network Working Group Request for Comments (RFC) 3550, July 2003. RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. Each media stream is streamed over a different RTP session. These RTP sessions can be assigned numbers in different manners, such as by using the “a=mid:” attribute as described in the Grouping of Media Lines in the Session Description Protocol (SDP) Network Working Group Request for Comments (RFC) 3388, December 2002. In other embodiments RTP sessions can be assigned numbers in different manners, such as by using an “a=stream:” attribute or some other attribute. Alternatively, the identifiers may be generated by simply using the same identifiers as the ASF Stream IDs, as assigned in the ASF file header.

Mutual Exclusivity

The mutual exclusivity extension specifies whether media streams are mutually exclusive. In certain implementations, streams are mutually exclusive based on bit rate and language. For example, a multimedia presentation may have multiple streams of the same content (e.g., audio or video) that are encoded at different bit rates. Only one of these streams at a time, having the desired bit rate, should be selected for streaming to the client device. By way of another example, a multimedia presentation may have multiple streams of the same content (e.g., audio) that are in different languages. Only one of these streams at a time, in the desired language, should be selected for streaming to the client device.

In certain embodiments, the parameters for the mutual exclusivity extension are obtained by accessing the Bitrate Mutual Exclusion Object or Advanced Mutual Exclusion Object in an ASF file header associated with the multimedia presentation. The parameters identified in the ASF file header are retrieved and used as the parameters of the mutual exclusivity extension. If there are multiple Bitrate Mutual Exclusion Objects and/or Advanced Mutual Exclusion Objects in the ASF file header, then a separate mutual exclusivity extension is included in the session description message for each of those multiple Bitrate Mutual Exclusion Objects and/or Advanced Mutual Exclusion Objects.

The mutual exclusivity extension is included in the session level description using an “a=group” attribute. The syntax for the “a=group” attribute is defined as follows:
a=group:ExclusionType StreamNumbers

The ExclusionType parameter can be set to different values to indicate the type of the mutual exclusivity. For example, the ExclusionType parameter may be set to “MBR” to indicate that the streams are mutually exclusive based on bit rate, but set to “LANG” to indicate that the streams are mutually exclusive based on language. In certain embodiments, when basing the mutual exclusivity extension on the information in an ASF file header, if the value of the “Exclusion Type” GUID in the Bitrate Mutual Exclusion Object or Advanced Mutual Exclusion Object of the ASF file header is ASF_Mutex_Language then the ExclusionType parameter is set to “LANG”, and if the value of the “Exclusion Type” GUID in the Bitrate Mutual Exclusion Object or Advanced Mutual Exclusion Object of the ASF file header is ASF_Mutex_Bitrate then the ExclusionType parameter is set to “MBR”.

The StreamNumbers parameter contains a list of streams that the mutual exclusion applies to. These streams are identified, for example, by their RTP sessions. In certain embodiments, when basing the mutual exclusivity extension on the information in an ASF file header, for each stream in the “Stream Numbers” field of the Bitrate Mutual Exclusion Object or Advanced Mutual Exclusion Object of the ASF file, the number assigned to the corresponding RTP session is added to the list of streams in the StreamNumbers parameter.

Marker

The marker extension specifies user-friendly indexes into the media stream(s). These user-friendly indexes are intended to be easily understood by human readers. The marker extension is included in the session level description using an “a=marker” attribute.

In certain embodiments, the parameters for the marker extension are obtained by accessing the Marker Object in an ASF file header associated with the multimedia presentation. The parameters identified in the ASF file header are retrieved and used as the parameters of the marker extension. If there are multiple Markers in the ASF file header, then a single marker extension can be used to identify all of the multiple Markers. Alternatively, a separate marker extension may be included in the session description message for each of those multiple Markers.

The syntax for the “a=marker” attribute is defined as follows:
a=marker:name time;desc

The name parameter specifies the name of the marker being specified by the marker extension. In certain embodiments, when basing the marker extension on the information in an ASF file header, the value of “Name” in the Marker Object of the ASF file header is UTF-8 encoded and used as the value of the name parameter in the marker extension.

The time parameter specifies the presentation time of the marker. In certain embodiments, when basing the marker extension on the information in an ASF file header, the value of “Presentation Time” in the Marker Object of the ASF file header is used as the value of the time parameter in the marker extension. In certain implementations, if the ASF file header specifies an amount of time to buffer data before starting to play the multimedia presentation (e.g., in the “Preroll” field of the File Properties Object), then that specified amount of time is subtracted from the value of “Presentation Time” in the Marker Object of the ASF file header, and the resulting difference is used as the value of the time parameter in the marker extension.

The desc parameter specifies the user-friendly index that is to be presented to the user as the marker. In certain embodiments, when basing the marker extension on the information in an ASF file header, the value of “Marker Description” in the Marker Object of the ASF file header is UTF-8 encoded and used as the value of the desc parameter in the marker extension.

The “a=marker” attribute may optionally include time and desc parameters for multiple markers. Each parameter pair is separated from the previous parameter pair by a space character (or alternatively by some other character(s), such as a comma, colon, period, etc.).

Script Command

The script command extension specifies text (or other data) that is to be displayed at a particular time during playback of the media stream(s), or specifies an identifier of a web page that should be displayed (e.g., within a web browser) at a particular time during playback of the media stream(s). The script command extension is included in the session level description using an “a=script” attribute.

In certain embodiments, the parameters for the script command extension are obtained by accessing the Script Command Object in an ASF file header associated with the multimedia presentation. The parameters identified in the ASF file header are retrieved and used as the parameters of the script command extension. If there are multiple Commands in the ASF file header, then a single script command extension can be used to identify all of the multiple Commands. Alternatively, a separate script command extension may be included in the session description message for each of those multiple Commands.

The syntax for the “a=script” attributes is defined as follows:
a=script:type time;command

The type parameter specifies the name of a type of command being specified by the script command extension. In certain embodiments, when basing the script command extension on the information in an ASF file header, the value of “Command Type Name” in the Script Command Object of the ASF file header is UTF-8 encoded and used as the value of the type parameter in the script command extension. If a Script Command Object in the ASF file header can contain multiple “Command Type Fields”, then a separate script command extension is used for each such field.

The time parameter specifies the presentation time of the command. In certain embodiments, when basing the script command extension on the information in an ASF file header, the value of “Presentation Time” in the Script Command Object of the ASF file header is used as the value of the time parameter in the script command extension. In certain implementations, if the ASF file header specifies an amount of time to buffer data before starting to play the multimedia presentation (e.g., in the “Preroll” field of the File Properties Object), then that specified amount of time is subtracted from the value of “Presentation Time” in the Script Command Object of the ASF file header, and the resulting difference is used as the value of the time parameter in the script command extension.

The command parameter specifies the text (or other data) that is to be displayed or the identifier of the web page(s) that should be displayed (e.g., within a web browser). In certain embodiments, when basing the script command extension on the information in an ASF file header, the value of “Command Name” in the Script Command Object of the ASF file header is UTF-8 encoded and used as the value of the command parameter in the script command extension.

The “a=script” attribute may optionally include time and command parameters for multiple script commands. Each parameter pair is separated from the previous parameter pair by a space character (or alternatively by some other character(s), such as a comma, colon, period, etc.).

Leaky Bucket

The leaky bucket extension specifies, when playing back variable bit-rate content at a constant bit-rate, how much data of the media stream(s) the client should buffer. A leaky bucket refers to a buffer that empties at a constant positive leak rate, into which bits can be dumped at irregular times and in irregular amounts. A leaky bucket is specified by its leak rate (e.g., in bits per second) and its size or capacity (e.g., in bits). The leaky bucket extension is included in the session level description using an “a=bucketinfo” attribute.

In certain embodiments, the parameters for the leaky bucket extension are obtained by accessing the Extended Content Description Object in an ASF file header associated with the multimedia presentation. The Extended Content Description Object includes a name-value pair named “ASFLeakyBucketPairs”. The value of the “ASFLeakyBucketPairs” name-value pair is the leaky bucket information, which is used as the parameters for the leaky bucket extension. Alternatively, the parameters for the leaky bucket extension may come from elsewhere in an ASF file header, such as the Extended Stream Description Object. If there are multiple such Extended Content Description Objects having “ASFLeakyBucketPairs” name-value pairs (and/or multiple such “ASFLeakyBucketPairs” name-value pairs within an Extended Content Description Object) in the ASF file header, then a single leaky bucket extension can be used to identify all of the multiple “ASFLeakyBucketPairs” name-value pairs. Alternatively, a separate leaky bucket extension may be included in the session description message for each of those multiple Extended Content Description Objects or multiple “ASFLeakyBucketPairs” name-value pairs.

The syntax for the “a=bucketinfo” attributes is defined as follows:
a=bucketinfo:R1;B1R2;B2
The R1 parameter specifies the bit rate (the leak rate) for the first leaky bucket, and the B1 parameter specifies the size (e.g., in milliseconds) of the first leaky bucket. The R2 parameter specifies the bit rate (the leak rate) for the second leaky bucket, and the B2 parameter specifies the size of the second leaky bucket. Although only two leaky buckets are shown in the syntax definition, the “a=bucketinfo” attribute may provide parameter pairs Rx;Bx for an unlimited number (x) of leaky buckets. Each parameter pair is separated from the previous pair by a space character (or alternatively some other character, such as a comma, colon, period, etc.).

In certain embodiments, when basing the leaky bucket extension on the information in an ASF file header, the value of the “ASFLeakyBucketPairs” name-value pair in the Extended Content Description Object of the ASF header file is used as the value of the R and B parameters in the leaky bucket extension. The first part of the “Descriptor Value” (e.g., the first four bytes) specifies the bit rate for the bucket (the R parameter), while the second part of the “Descriptor Value” (e.g., the second four bytes) specifies the size for the bucket (the B parameter).

Caching Information

The caching information extension specifies information about caching of the media stream(s), such as whether the stream(s) can be cached, how long the stream(s) are considered fresh, and so forth. In certain embodiments, the caching information is determined based on the caching policy implemented by the server device (e.g., server device104). The caching information extension is included in the session level description using an “a=cachectrl” attribute.

The syntax of the “a=cachectrl” attribute is as follows:
a=cachectrl:Rights

The Rights parameter is one or more directives, separated by space characters (or alternatively some other characters, such as commas, periods, semicolons, colons, etc.). A directive can be either a name-value pair, where the name and the value are separated by an “=” character (or alternatively some other character), or a single “name”, without a “value”.

Different directives can be included in the Rights parameter. In certain embodiments, these directives include a no-cache directive, a no-splitting directive, and an expiration directive.

The no-cache directive is defined as “no-cache”, and indicates to the client device, as well as any intermediate devices between the server and client devices, that the device is not allowed to cache the media stream(s) for playback at a later time.

The no-splitting directive is defined as “no-splitting”, and indicates to the client device, as well as any intermediate devices between the server and client devices, that the device is not allowed to rebroadcast or redistribute the content. Splitting of streaming media content refers to the same content being communicated to multiple clients. The no-splitting directive indicate that such splitting is not allowed.

The “a=cachectrl” attribute may also include an expiration directive, which is a name-value pair that is defined as follows:
expiration=Timeout
The Timeout parameter specifies when cached copies of the media stream(s) are considered stale (no longer fresh). The Timeout parameter may specify a period of time (e.g., a number of seconds, minutes, days, etc.) until the cached copies are considered stale, or alternatively may specify a particular date and/or time (e.g., Apr. 15, 2004 at 3:00 pm) when the cached copies become stale. In certain embodiments, a device with a cached copy is not allowed to use stale media stream(s). In other embodiments, the device can use a stale cached copy of a media stream(s), but the device should attempt to determine if a newer version of the media stream(s) is available, and use that newer version instead if possible.

If the expiration directive is not included, then the media stream(s) do not become stale. If the “a=cachectrl” attribute is not included, then caching and splitting of the media stream(s) is allowed, and cached content does not become stale.

RTP Encapsulation

The RTP encapsulation extension specifies codec initialization parameters generically across a broad variety of codecs for data transmitted over RTP. A variety of different codecs can be used with RTP, and these different codecs oftentimes have their own encapsulation schemes and their own initialization parameters. The RTP encapsulation extension allows the same syntax (which is the RTP encapsulation extension) to describe the properties of a variety of different codecs, thereby alleviating the need for many different syntaxes to describe many different codes. The RTP encapsulation extension is included in the media level description using an “a=fmtp” attribute.
a=fmtp:pt MIME config=config-info

The pt parameter is the RTP payload type number, which identifies the format or type of the RTP payload. In certain implementations, any number between 0 and 127 is valid.

The MIME parameter is an identifier of the codec used in the media stream. In certain embodiments, the MIME parameter is a MIME (Multipurpose Internet Mail Extensions) type that is formed according to the rules in “WAVE and AVI Codec Registries”, Network Working Group Request for Comments (RFC) 2361, June 1998. In other embodiments, the MIME parameter can take the following form:
application/vnd.asf;codec=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
where the parameter xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is replaced by an identifier of the codec. In certain implementations, this identifier is the codec GUID from the ASF file. The identifier may be stored in the ASF file as, for example, the Media Type GUID from the Stream Properties Object in the ASF file, or the Type-Specific Data section of the Stream Properties Object in the ASF file.

The config=config-info parameter identifies the codec initialization data, where config-info is replaced by a series of sub-parameters. In certain implementations, the sub-parameters are Base64 encoded. These sub-parameters are separated by “/” characters, or alternatively other characters (e.g., periods, commas, colons, semi-colons, etc.). The syntax of config-info is as follows:
MajorType/FixedSizeSamples/TemporalCompression/SampleSize/FormatType/FormatData

The MajorType sub-parameter is an identifier (e.g., a GUID (Globally Unique ID)) that identifies the type of the media stream. In certain implementations, valid values include “73647561-0000-0010-8000-00AA00389B71” for an audio stream, and “73646976-0000-0010-8000-00AA00389B71” for a video stream.

The FixedSizeSamples sub-parameter indicates whether the codec uses fixed size samples (packets). In certain implementations, the FixedSizeSamples sub-parameter is “1” if the codec uses fixed size media samples (packets), or “0” otherwise. For example, usually this field will be “1” for audio and uncompressed video, but “0” for compressed video.

The TemporalCompression sub-parameter indicates whether the stream uses temporal compression. In certain implementations, the TemporalCompression sub-parameter is “1” if the stream uses temporal compression, meaning that not all media samples are “key-frames”. Otherwise, it is set to “0”. This field may be set to “1” for many types of compressed video, because such streams may have non-key frame media samples.

The SampleSize sub-parameter indicates the size of the media samples when the codec uses fixed size samples (packets). In certain implementations, the SampleSize sub-parameter is a decimal number which gives the size of the media samples when FixedSizeSamples is “1”. The SampleSize sub-parameter can be set to “0”, and should be ignored, when FixedSizeSamples is “0”.

The FormatType sub-parameter is an identifier (e.g., a GUID) that identifies the format of the FormatData sub-parameter. The FormatData sub-parameter is the codec initialization data identified by the FormatType sub-parameter. Any of a variety of different structures that describe properties of a codec can be used for the FormatData sub-parameter. For example, the data structure describing properties of a codec for an audio stream may include information about the audio sample rate and the number of bits per sample. By way of another example, the data structure describing properties of a codec for a video stream may include information about the width and height (e.g., in pixels) of the video information, as well as the pixel aspect ratio. In certain implementations, each byte in the codec initialization data is represented as two hexadecimal digits in SDP. Alternatively, other representations may be used such as Base64 encoding.

Bandwidth Sharing

The bandwidth sharing extension specifies streams that share bandwidth in such a way that the maximum bandwidth of the set of specified streams is less than the sum of the maximum bandwidths of the individual specified streams. The bandwidth sharing extension is included in the session level description using an “a=bwshare” attribute.

In certain embodiments, the parameters for the bandwidth sharing extension are obtained by accessing the Bandwidth Sharing Object in an ASF file header associated with the multimedia presentation. The parameters identified in the ASF file header are retrieved and used as the parameters of the bandwidth sharing extension. If there are multiple Bandwidth Sharing Objects in the ASF file header, then a separate bandwidth sharing extension is included in the session description message for each of those multiple Bandwidth Sharing Objects.

The syntax for the “a=bwshare” attribute is defined as follows:
a=bwshare:SharingTypeR BSessionList

The SharingType parameter can be set to different values to indicate the type of sharing. For example, the SharingType parameter may be set to “PART” to indicate that any number of the specified streams may be streaming data at any given time, but set to “EXCL” if only one of the specified streams may be streaming data at any given time. In certain embodiments, when basing the bandwidth sharing extension on the information in an ASF file header, if the value of the “Sharing Type” GUID in the Bandwidth Sharing Object of the ASF file header is ASF_Bandwidth_Sharing_Partial then the SharingType parameter is set to “PART”, and if the value of the “Sharing Type” GUID in the Bandwidth Sharing Object of the ASF file header is ASF_Bandwidth_Sharing_Exclusive then the SharingType parameter is set to “EXCL”.

The R parameter specifies the leak rate (e.g., in bits per second) of a leaky bucket that contains the data portion of all of the streams without overflowing. The size of the leaky bucket is specified as the B parameter. In certain embodiments, when basing the bandwidth sharing extension on the information in an ASF file header, the R parameter is the value of the “Data Bitrate” field in the ASF file header, expressed as a decimal number.

The B parameter specifies the size (e.g., in milliseconds) of the leaky bucket. In certain embodiments, when basing the bandwidth sharing extension on the information in an ASF file header, the B parameter is the value of the “Buffer Size” field in the ASF file header, expressed as a decimal number.

The SessionList parameter consists of one or more decimal numbers, separated by space characters. Each number identifies one of the specified streams that is covered by the bandwidth sharing relationship expressed by this “a=bwshare” attribute. The streams are identified by their RTP session numbers.

Stream Prioritization

The stream prioritization extension specifies which streams in the multimedia presentation should have priority over which other streams. This prioritization can be used, for example, to identify which streams should or should not be dropped in the event of network congestion.

The stream prioritization extension is included in the session level description using an “a=prio” attribute. The “a=prio” attribute lists each RTP session in priority order, with the highest priority RTP session listed first. The “a=prio” attribute includes a list of parameters that are identifiers of the RTP sessions.

Certain RTP sessions can be identified as mandatory, which indicates that those certain sessions are not supposed to be dropped in response to network congestion. Mandatory RTP sessions are indicated in the “a=prio” attribute by appending the character ‘m’ after the RTP session number.

The following example illustrates the syntax of the “a=prio” attribute.
a=prio:1m2m3
In this example, RTP sessions 1 and 2 are mandatory. RTP session 3 has lower priority than the other sessions, and may be dropped in the event of network congestion.

In certain embodiments, the stream priorities are obtained by accessing the Stream Prioritization Object in an ASF file header associated with the multimedia presentation. The streams identified in the ASF file header correspond to RTP sessions, and the stream priorities identified in the ASF file header are used as the priorities for the corresponding RTP sessions. If the “Mandatory Flag” is set for a particular ASF stream, then the corresponding RTP session is also considered mandatory.

Referring now toFIG. 5,FIG. 5illustrates a general computer environment300, which can be used to implement the techniques described herein. The computer environment300is only one example of a computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the computer and network architectures. Neither should the computer environment300be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary computer environment300.

Computer environment300includes a general-purpose computing device in the form of a computer302. Computer302can be, for example, a client device102or server device104ofFIGS. 1 and 2. Computer302can also be an encoder device that is the source of a multimedia presentation. The components of computer302can include, but are not limited to, one or more processors or processing units304, a system memory306, and a system bus308that couples various system components including the processor304to the system memory306.

Computer302typically includes a variety of computer readable media. Such media can be any available media that is accessible by computer302and includes both volatile and non-volatile media, removable and non-removable media.

The system memory306includes computer readable media in the form of volatile memory, such as random access memory (RAM)310, and/or non-volatile memory, such as read only memory (ROM)312. A basic input/output system (BIOS)314, containing the basic routines that help to transfer information between elements within computer302, such as during start-up, is stored in ROM312. RAM310typically contains data and/or program modules that are immediately accessible to and/or presently operated on by the processing unit304.

Computer302may also include other removable/non-removable, volatile/non-volatile computer storage media. By way of example,FIG. 5illustrates a hard disk drive316for reading from and writing to a non-removable, non-volatile magnetic media (not shown), a magnetic disk drive318for reading from and writing to a removable, non-volatile magnetic disk320(e.g., a “floppy disk”), and an optical disk drive322for reading from and/or writing to a removable, non-volatile optical disk324such as a CD-ROM, DVD-ROM, or other optical media. The hard disk drive316, magnetic disk drive318, and optical disk drive322are each connected to the system bus308by one or more data media interfaces326. Alternatively, the hard disk drive316, magnetic disk drive318, and optical disk drive322can be connected to the system bus308by one or more interfaces (not shown).

Any number of program modules can be stored on the hard disk316, magnetic disk320, optical disk324, ROM312, and/or RAM310, including by way of example, an operating system326, one or more application programs328, other program modules330, and program data332. Each of such operating system326, one or more application programs328, other program modules330, and program data332(or some combination thereof) may implement all or part of the resident components that support the distributed file system.

A user can enter commands and information into computer302via input devices such as a keyboard334and a pointing device336(e.g., a “mouse”). Other input devices338(not shown specifically) may include a microphone, joystick, game pad, satellite dish, serial port, scanner, and/or the like. These and other input devices are connected to the processing unit304via input/output interfaces340that are coupled to the system bus308, but may be connected by other interface and bus structures, such as a parallel port, game port, or a universal serial bus (USB).

A monitor342or other type of display device can also be connected to the system bus308via an interface, such as a video adapter344. In addition to the monitor342, other output peripheral devices can include components such as speakers (not shown) and a printer346which can be connected to computer302via the input/output interfaces340.

Computer302can operate in a networked environment using logical connections to one or more remote computers, such as a remote computing device348. By way of example, the remote computing device348can be a personal computer, portable computer, a server, a router, a network computer, a peer device or other common network node, and the like. The remote computing device348is illustrated as a portable computer that can include many or all of the elements and features described herein relative to computer302.

Logical connections between computer302and the remote computer348are depicted as a local area network (LAN)350and a general wide area network (WAN)352. Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet.

When implemented in a LAN networking environment, the computer302is connected to a local network350via a network interface or adapter354. When implemented in a WAN networking environment, the computer302typically includes a modem356or other means for establishing communications over the wide network352. The modem356, which can be internal or external to computer302, can be connected to the system bus308via the input/output interfaces340or other appropriate mechanisms. It is to be appreciated that the illustrated network connections are exemplary and that other means of establishing communication link(s) between the computers302and348can be employed.

In a networked environment, such as that illustrated with computing environment300, program modules depicted relative to the computer302, or portions thereof, may be stored in a remote memory storage device. By way of example, remote application programs358reside on a memory device of remote computer348. For purposes of illustration, application programs and other executable program components such as the operating system are illustrated herein as discrete blocks, although it is recognized that such programs and components reside at various times in different storage components of the computing device302, and are executed by the data processor(s) of the computer.

Although the description above uses language that is specific to structural features and/or methodological acts, it is to be understood that the invention defined in the appended claims is not limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the invention.