Patent Publication Number: US-2007112811-A1

Title: Architecture for scalable video coding applications

Description:
BACKGROUND  
      People use many different types of devices to access services over the Internet and other networks. Increasingly, people use these devices to access motion video from entertainment sources, news services, and other providers.  
       FIG. 1  illustrates a number of different digital devices  102 - 108  that may be used to access media content. Generally, media content is accessed by devices  102 - 108  from servers  110  and  112  via a network  114 , such as a local area network or a wide area network, such as the Internet. The personal computer  102  may be associated with a large, high-resolution display. Alternatively, as shown in  FIG. 1 , the personal computer  102  may be a media center computer associated with a high resolution television display or video monitor  122 , or a video projector (not shown). Thus, the user of the personal computer  102  is likely to want video content displayed at its highest resolution to take advantage of the capabilities of the video monitor  122 .  
      On the other hand, the portable display  124  of the portable computer  104  may not provide resolution comparable to that of the video monitor  122  of the personal computer  102 . Alternatively, even if the portable display  124  does support high resolution graphics, the portable computer  104  may be coupled with the network  114  via a lower bandwidth connection. As a result, the user may have to sacrifice resolution in exchange for being able to access the media content without having to wait an intolerably long time to receive the media content and to be able to play the media content continuously, and without pauses or delays.  
      In addition to the personal computer  102  and the portable computer  104 , handheld devices  106  and  108  also are used to access Internet services. For example, a personal digital assistant  106  includes a touchscreen display  126 , measuring a few inches on each side, that permits access to video content, albeit it at only a portion of the resolution available with the personal computer  102  and the portable computer  104 . Even a smaller device, such as wireless telephone  108 , includes a phone display  128  usable to access media services and present video content to a user. Users may also access media content using gaming systems and other portable and non-portable devices.  
      Historically, the broad range of devices  102 - 108  seeking to access media content on servers  110  and  112  has posed a problem for content providers. More specifically, because of the wide range of displays  122 - 128  used by devices  102 - 108 , media content providers have had to make media content available in different formats. For example, high resolution video content had to be made available to users with high resolution video monitors  122 , while lower resolution video content had to be made available to users using devices with lower resolution displays or via slower network connections. Conventionally, content providers maintained the video content in multiple resolution formats selectable by a user. Alternatively, the format might be determined automatically based on information available to the host about the device or the connection used to access the media content.  
      The problem of servers  110 - 112  having to maintain and selectively communicate multiple different video content formats is addressed by scalable image formats. For one example, the Joint Photographic Experts Group 2000 (“JPEG 2000”) format specifies a codestream that is scalable not only in resolution, but for each of a number of different access types including tile, layer, quality component, precinct, bit rate, and peak signal to noise ratio. The codestream is scalable at a number of levels within each of these access types. A single codestream can be accessed by different devices to present images or video adapted to levels each of the devices is configured to support for each access type. Thus, one image codestream can be stored and provided to any device supporting the scalable codestream.  
       FIGS. 2A-2E  are a series of diagrams illustrating the nature of video frames presented in a JPEG 2000 codestream.  FIG. 2A  shows a frame  200  as a user views it: image  200  is comprised of an array of elements  202 . However, the actual structure of the scalable codestream is not so simply organized.  
       FIG. 2B  is an array  210  of differently sized data blocks  212 , representing how different resolution levels are represented in the codestream. As a function of discrete wavelet transformation, accessing different resolution levels involves accessing different blocks of data  212  in the codestream. As shown in  FIG. 2C , to access a lowest supported resolution level, only a first data block  222  of array  220  is accessed. As shown in  FIG. 2D , to access a next highest resolution level, data block  222  and a series of three adjoining second data blocks  232  are accessed. To access a next highest resolution level, as shown in  FIG. 2E , first data block  222 , second data blocks  232 , and a series of adjoining third data blocks  242  all must be accessed. Adjoining groups of data blocks are accessed until the highest available image resolution is reached. Thus, data blocks used in presenting higher levels of resolution encompasses data are used in presenting lower levels of resolution.  
      With a scalable codestream, of which the JPEG 2000 codestream described above is just one example, all the devices  102 ,  104 ,  106  and  108 , can access the same codestream and present media content on their associated displays  122 ,  124 ,  126 , and  128 , respectively. Thus, differently enabled devices can present media content at access levels as high or as low as users&#39; hardware systems, available bandwidth, and preferences allow.  
      Scalable codestreams allow for the possibility of differently enabled devices accessing the codestreams at various levels, but sending and receiving the full codestreams may not be very efficient. For example, when a receiving system requests access to media content, a sending device may deliver the entirety of the codestream so that the receiving system can access the media content at the highest access levels it supports. However, if the user uses a device with a low resolution display or has only a low-bandwidth network connection, and thus will access the codestream at lower access levels, transmission of the entire codestream may be wasteful. On the other hand, if the user selects a non-scalable codestream to save downloading time, but then wishes to access the codestream at higher access levels, the user will have to reinitiate access to the codestream at the higher access level, and acquire the codestream all over again.  
     SUMMARY  
      An architecture provides adaptive access to scalable media codestreams. Minimum coding units from the codestream to facilitate presentation of the media content at a selected access level are collected in packettes. The data needed for the packettes are identified and assembled by a peering subsystem or peer layer that supplements a conventional architecture in a sending system. The packettes are communicated to one or more receiving systems, such as by collecting the packettes into transport packets recognized by conventional architecture. The peering subsystem or peering layer of a receiving system unpacks the packettes needed to support the desired access level to the media content. The peer subsystems or peer layers communicate between systems to effect changes in the packettes provided to adapt access levels or avoid waste of network resources. The architecture supports applications including multiple access level streaming of media content, device roaming, and time-dependent access level shifting.  
      This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.  
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
      The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit of a three-digit reference number and the two left-most digits of a four-digit reference number identify the figure in which the reference number first appears,. The use of the same reference numbers in different figures indicates similar or identical items.  
       FIG. 1  (Prior Art) represents a network in which different devices with different display capabilities used to access content on the same servers.  
       FIGS. 2A-2E  (Prior Art) illustrate data blocks used in presenting image content at different access levels.  
       FIG. 3  is a block diagram of two systems using an embodiment of an architecture to adaptively send and receive media content.  
       FIG. 4  is a block diagram of a computer network protocol adapted to include a peer layer.  
       FIG. 5  is a block diagram of a packette used to exchange one or more minimum coding blocks of data used by a scalable codestream.  
       FIG. 6  is a block diagram of subsystems of a sending system and a receiving system according to an architecture for adaptively exchanging scalable codestreams.  
       FIG. 7  is a flow diagram of a process for both adaptively sending and adaptively receiving the content of a scalable codestream.  
       FIG. 8  is a network in which multiple receiving systems with different capabilities adaptively receive the content of a scalable codestream.  
       FIG. 9  is a flow diagram of a mode of adaptively receiving the content of a scalable codestream used by each of the systems of  FIG. 8 .  
       FIG. 10  is a network in which access to content is migrated between receiving systems.  
       FIG. 11  is a flow diagram of a mode of adaptively receiving the content of a scalable codestream as it is migrated between systems having different capabilities.  
       FIG. 12  is a network in which access to content of a scalable codestream provided by one system is adaptively received over time by a second system.  
       FIG. 13  is a flow diagram of a mode of adapting an access level used in recording a program based on storage space available for recording the program.  
       FIG. 14  is a flow diagram of a mode of recording a program at a desired access level by reducing access levels used in storing previously-recorded programs by truncating data supporting the original access levels.  
       FIG. 15  is a functional diagram of a computing system environment suitable for use in adaptively receiving or sending content of a scalable codestream. 
    
    
     DETAILED DESCRIPTION  
      To take best advantage of scalable codestreams for video and other forms of media content, an adaptive architecture for sending and receiving the video content is desired. In the case of motion video content, a scalable codestream allows for video content to be accessed at the level of resolution—or access levels of other types—up to the highest resolution supported by the codestream and the accessing device. Alternatively, the scalable codestream also allows the device to access the codestream at a lower access level to permit faster access to the content, such as to allow for workable access over lower bandwidth connections. Desirably, the systems using an embodiment of an adaptive architecture to adjust the selected access level based on capabilities of the device currently used to access the content, available bandwidth, user preferences, or other factors.  
      An architecture that permits adaptive communication between the sending and receiving devices facilitates a number of features. First, the architecture promotes scalable distribution. For example, scalable digital rights management facilitates charging for media content based on the access level selected. The architecture allows the receiving system to adapt to access the content of the scalable codestream at the permitted level. Also, in a video teleconference situation or other situations where different participants use devices with different media capabilities, the codestream can be adaptively processed so that each user is able to access the codestream at an optimal resolution. However, unlike conventional access to a non-scalable codestream, the sending and/or receiving devices can send and receive, respectively, only desired portions of the scalable codestream to reduce waste of computing or communication resources.  
      Correspondingly, not only does the architecture allow multiple systems to receive and selectively access a codestream from a single sending system, but it also allows a single system to receive a codestream from multiple sources. As is understood in the art and has been previously described in connection with  FIGS. 2A through 2E , a scalable codestream includes discrete, constituent elements that represent different access levels to the content presented by codestream. Thus, multiple systems may send different constituent portions of the codestream to a receiving system, allowing the receiving system to receive all of the constituent elements to allow the receiving system to access the codestream at high access levels more quickly than would be possible if the codestream were being provided by a single sending system.  
      Second, the architecture provides migration of a codestream from one device to another. Thus, the architecture permits “device roaming.” For example, if a user originally receives the content using a high resolution device, such as a portable computer, but then switches to a lower resolution handheld device, such as a personal digital assistant, handheld computer, mobile phone, or gaming system, access to the content of the codestream is automatically adapted to allow the user to continue to access the content regardless of the changes of capability between the devices. A user can switch back and forth between receiving devices. Thus, a user viewing one program on a high resolution video monitor while monitoring a second program on a lower resolution handheld device can switch which program is being viewed on each device.  
      Adaptive distribution also promotes time-dependent scaling of content. For example, if a user is recording a program and is running out of storage space, the adaptive distribution allows the receiving device to reduce the access level of the content being received to be able to fit the program into available storage space. Similarly, if a user is receiving multiple files, the files may all be received at a reduced access level to provide the user with all at least low access level versions of all the content requested. Then, when the user later is receiving files again or is accessing the files, additional data to increase the access level is received to supplement the original data received. Also, for users performing video content analysis and editing, users can retrieve, edit, and sort the content based on lower-access level content, and when the piece has been edited, additional data to supplement the lower-access level content is retrieved.  
      The functions are facilitated by systems including a peering subsystem, a network engine, a kernel, and a communications control subsystem to monitor and adapt codestream communications being sent and received, as is described in detail below.  
      Architecture Facilitating Adaptive Communication of Video Content  
       FIG. 3  is a high-level block diagram of an embodiment of an architecture operable to exchange data and control signals to take advantage of scalable codestreams.  FIG. 3  includes two systems, System A  300  and System B  350 , each of which uses an embodiment of the architecture and is able to communicate with one another over a network  302 . System A  300  and System B  350  are also able to communicate over the network  302  with other systems, including System C  304  through System n  306 . The network  302  is depicted as a single network through which each of the systems  300 ,  304 ,  306 , and  350  communicates. However, the systems can communicate peer-to-peer over different networks. For example, while System A  300  and System B  350  may communicate over a wide-area network such as the Internet, System B  350  might communicate with System C  304  via a Bluetooth network or some other form of network medium.  
      More specifically, an embodiment of the architecture illustrated in System A  300  and System B  350  allows the systems to monitor network and device capabilities and exchange control signals to optimize the exchange of data signals. Both System A  300  and System B  350  include four general subsystems: controller subsystems  310  and  360 , network engines  320  and  370 , peering subsystems  330  and  380 , and kernels  340  and  390 . System A  300  and System B  350  also direct coding subsystems  345  and  395 , respectively. File coding subsystems  345  and  395  may include any of a number of known coding systems used to encode and decode files for data transmission.  
      By way of overview, the controller subsystems  310  and  360  provide user control of media flow, such as by launching media applications and processing user commands. The controller subsystems  310  and  360  also monitor resources to control or restrict the data flow to levels that the local system can accommodate. The controller subsystems  310  and  360  also cooperate between the server side and the client side to monitor quality of service (QoS) reports and requests to adapt the flow of data between the sending or server side and the receiving or client side. Thus, to facilitate transmission of one codestream to multiple receiving systems or to facilitate reception of constituent elements of a codestream from multiple sending systems, the controller subsystems  310  and  360  exchange control information about the codestreams or portions of codestreams being sent and received by the systems participating in the communication.  
      The network engines  320  and  370  control data transmission operations. For example, on the receiving the network engine performs packet reordering, sending non-acknowledgment (NACK) messages for missing transport packets, transmits QoS information and client requests, and similar functions. On the sending system, in addition to receiving QoS information and client requests, the network engine directs data transmission, maintains a re-sending buffer for transport layer packets that are not acknowledged by the client, and handles automatic repeat request (ARQ) processing.  
      In one embodiment, System A  300  and System B  350  each include multiple network engines  320  and  370 , respectively, to support collaborative streaming. Collaborative streaming, as previously described, allows one system to send or receive multiple codestreams. As will be described below in connection with  FIG. 6 , the network engines  320  and  370  include data channels, buffers, and other components to send data to and receive data from other systems.  
      System A  300  and System B  350  also include peering subsystems  330  and  380 . As is explained in further detail below, the peering subsystems on the sending system receives the codestream from the file encoding subsystem  345  and selects portions of the coding units in the codestream. The selected portions are included in transport packets to be sent to one or more receiving systems. The peering subsystem  330  handles this selection and packettization of the selected portions, thus the process is transparent to the file coding subsystem  345  on the sending system, System A  300 . In one embodiment, the peering system also performs forward error correction (FEC) on the transport layer packets.  
      On a client side, the peering subsystem  380  unpacks the selected coding units of data from the incoming transport layer packets for processing by the file decoding system. In one embodiment, the peering subsystem  380  also performs inverse FEC on the incoming transport layer data packets.  
      System A  300  and System B  350  also include kernel subsystems  340  and  390 , respectively. On a sending system, the kernel subsystem multiplexes data and control data. On a client side, the kernel subsystem includes a memory and/or storage cache for receiving and staging the data for access by the receiving system.  
      The control subsystems  310  and  360 , network engines  320  and  370 , peeing subsystems  330  and  380 , and kernel subsystems  340  and  390  communicate with one another to adaptively extract, send, receive, and unpack the selected coding units from the scalable codestream to provide efficient and flexible access to desired media content. The access is efficient because coding units that are not used are not sent and/or accessed. The access is flexible because the peering subsystems  330  and  380  can adjust the number of coding units that are exchanged and/or accessed during the exchange of the media content based upon the control signals exchanged by the other subsystems. Accordingly, the selected access level of the codestream can be changed during the transmission, without restarting, rebuffering, or otherwise reinitiating the exchange of media content.  
       FIGS. 4 and 5  illustrate two additional attributes of an embodiment of the architecture. First,  FIG. 4  shows an additional logical layer used by an embodiment of the architecture in a layered communications protocol in both a server or other sending system  400  and a client or other receiving system  450 . A conventional sending system observing an International Standards Organization Open Systems Interconnection (ISO-OSI) protocol typically includes an application layer  402 , a presentation layer  404 , a session layer  406 , a transport layer  408 , a network layer  410 , a data link layer  412 , and a physical layer  414 . Similarly, a conventional receiving system includes an application layer  452 , a presentation layer  454 , a session layer  456 , a transport layer  458 , a network layer  460 , a data link layer  462 , and a physical layer  464 . The physical layer  414  of the sending system  400  is in communication with the physical layer  464  over a network medium  430 .  
       FIG. 4  shows an additional layer added to the sending system  400  and the receiving system  450  to make the architecture transparent not only to the application layers  402  and  452 , but to the other layers of the protocol as well. In one embodiment, a peer layer  420  is added to the sending system  400  above the transport layer  408 , and a corresponding peer layer  470  is added to the receiving system  450  above the transport layer  458 . The peer layers  420  and  470  work with packettes to adaptively transmit and receive media codestreams according to an embodiment of the architecture.  
      The peer layers  420  and  470  provide multiple advantages. First, for example, the peer layers  420  and  470  allow for the packing and unpacking, respectively, of minimum units of data into packeftes that can be included in transport packets that are assembled by the transport layer  408  and other layers of the sending system  400  in sending data via the network medium  430  to the receiving system  450 . Upon receiving transport packets from the transport layer  458  of the receiving system  450 , the peer layer  470  of the receiving system  450  then unpacks the packettes from the transport packets transmitted by the sending system  400 . In other words, packettes are packed and unpacked by the peer layers  420  and  470  and provided to the transport layers  408  and  458 , respectively. Accordingly, embodiments of the architecture are not dependent on any particular packetization mechanism that may be used by the transport layers  420  and  470 , or by other layers in the systems. Moreover, the assembly and unpacking of packettes is performed transparently with regard to the other layers.  
      Another advantage of adding the peer layers  420  and  470  is that their presence decouples the application layers  402  and  452  from the transport layers  408  and  458 , respectively. Decoupling these layers supports functions such as device roaming, which is mentioned above and described in more detail below. The peer layers  420  and  470  perform the selective assembly and selection of packettes to permit the application layers  402  and  452  to engage the codestream at an access level that each system can accommodate transparently with regard to either the application layers  402  and  452  and the transport layers  408  and  458 .  
      In the alternative, instead of adding the peer layers  420  and  470  to the seven-layer ISO-OSI protocol, the peer layers  420  and  470  can incorporate functions of other layers and supplant those layers, and other layers may be combined as well. To name just one example, a three-layer model may be appropriate for both the sending system and the receiving system, where the three layers, include an application layer, a peer layer, and a transport layer. In such a model, the peer layer collects selected coding blocks into packettes to support the applications, and collects the packettes into transport packets to be communicated by the transport layer. Thus, comparable to the eight-layer model derived by adding a peer layer, the three-layer model includes a peer layer to support the media applications, while forming or unpacking the packettes in a manner that is independent of and transparent to the application and transport layers.  
       FIG. 5  shows a diagram of a packette  500  used in an embodiment of the architecture. The packette  500  is a coding unit used by an embodiment of the architecture to exchange data between the sending system  400  and the receiving system  450  ( FIG. 4 ). The packette  500  includes a data section  510 , including one or more minimum coding units of data. A minimum coding unit suitably includes a sequence of bits which are syntactically compliant with the specific codestream format. Preferably, the minimum coding units are related to the minimum block division of a frame, image, or other media unit. Thus, in the example of video coding, the minimum coding unit can be a macroblock. A header  520  is appended to the data section  510 . The header  520  includes some assisting information identifying the contents of the packette  500 .  
      Embodiments of the architecture are not limited to any particular packette structure, and there are multiple possibilities for forming the packettes  500 . To name one example, packettes  500  may be formed by specifying a fixed maximum data length for data section  510 , and including as many minimum coding units as will fit into to it. Alternatively, packettes  500  may include a fixed number of minimum coding units. The minimum coding unit itself is application configurable. For example, the minimum coding structure may include a single macroblock for video coding. Alternatively, the minimum coding structure may include multiple macroblocks.  
      The header  520  of the packette  510  may include the packette length, stated in bytes or other units recognized by the architecture. The header  520  also may include a frame number or time stamp of the frame of which the minimum coding units are a part, a bit plane number, a starting macroblock index number, an end macroblock index number, a number of useful bits in the last byte, and other information. When the header  520  includes macroblock index numbers, the data section  510  includes the bits representing the macroblocks from the starting index through the end index specified in the header  510 . Thus, when the header  520  includes the starting macroblock index and end macroblock index, the header  520  provides a metric for the quality of a received frame. Alternatively, however, the header  520  may include a quality index field to indicate the quality of the frame made available upon successfully receiving the current packette and the preceding packettes including data representing the frame.  
      Detailed Example of Architecture Facilitating Adaptive Communication  
       FIG. 6  is a functional block diagram of an exemplary embodiment of systems employing an embodiment of the architecture to exchange data and control signals to take advantage of scalable codestreams. The data signals are represented by solid lines, while the control signals are represented by dotted lines. More specifically,  FIG. 6  is a functional block diagram that illustrates the interrelationships between subsystems when a server or sending system is providing a codestream to a client or receiving system. It should be noted that the subsystems of the sending system and the receiving system are distributed throughout the functional block diagram of  FIG. 6  to illustrate the functional interrelationship between the subsystems.  
      Ultimately, the data is passed from a file encoding system  696  on a system sending the data to a file decoding system  698  on a system requesting and receiving the data. A multitude of coding and decoding systems are known in the art, and embodiments of the architecture for adaptive communication of scalable codestreams are not limited to any particular encoding and decoding topologies.  
      A receiver control subsystem  600  and a sender control subsystem  610  cooperate to control the codestream transmission between the sending system and the receiving system. Both the receiver control subsystem  600  and the sender control subsystem  610  support connection managers  602  and  612 , respectively. In one embodiment, the connection managers  602  and  612  are the only daemon threads that are always running in order to monitor a port for codestream-related communications.  
      The connection manager  602  on the receiving system provides an interface that allows a user to launch an application that uses the scalable codestreams. The connection manager  602  also accepts user commands, such as START, PAUSE, STOP, and similar media control commands. The connection manager  612  on the sending system responds to communications generated by the connection manager  602  on the receiving system. The connection managers  602  and  612  communicate with each other, and with other peer connection managers, to perform security checks for user authentication, identification verification, and similar functions. In addition, the connection managers  602  and  612  provide input to the receiver controller  604  and the sender controller  614 , respectively.  
      The receiver controller  604  manages functions on the receiving system in cooperation with the sender controller  614  on the sending system. The receiver controller  604  receives data from the sender kernel  680  that multiplexes the codestream regarding the quality of the transmission from the sending system. The receiver controller  604  also receives QoS information from the receiver network engine  620 , and generates quality of service reporting information to the sending controller  610 . The receiver controller  604  engages a peer coordinator  608  that provides information to the receiver network engine  620  to coordinate among peers for better cooperative streaming, according to the network status among peers and user commands.  
      The receiver controller subsystem  600  also includes a local controller  606 . The local controller  606  determines which packettes will be selected to the sender kernel  60  to be multiplexed and delivered to the receiver kernel  6  where the codestreams will be cached on the receiving system. The local controller  606  receives input  618  that may include input from a user seeking smoother motion, better quality, or other attributes during the presentation of the codestream on the receiving system.  
      In addition, the local controller  606  may receive input  618  regarding the processing status of the receiving system. Thus, if the receiving system does not have the capability to store or process the data being received, or if bandwidth is limited, further input  618  is provided to the local controller  606  to indicate that the playback quality should be reduced. The local controller  606  communicates with a packettes selector  692  to reduce the number of packettes being transmitted. Thus, the local controller  606  can restrict the number of packettes to be selected so that the playback can last longer while at a degraded quality.  
      The sender control subsystem  610 , in addition to the connection manager  612 , also includes a sender controller  614 . The sender controller  614  communicates with receiver controller  604 . The sender controller  614  receives client requests to access media and the QoS status from the sender network engine  640 . The sender controller  614  communicates the desired media quality to the local controller  606 . The sender controller  614  communicates to the scheduler  666  of the sender peering subsystem  660  to perform scheduling, with possible cross-frame optimization of packets considering the client request. Cross-frame optimization can still be performed if such optimization information is passed at regular interval for multiple frames, or the scheduler  666  can be permitted to perform the optimization. The sender controller  614  also specifies a quality claim specifying the capability of the receiving system that can be used by other systems in performing peer coordination.  
      The receiving system also includes a receiver network engine  620 . The receiver network engine  620  includes a data channel  622  that engages a transport layer or directly performs data transmission using user datagram protocol (UDP), rapid transport protocol (RTP), or other protocols. The data channel  622  includes a non-acknowledgement generator (NACK)  624  to communicate to signal the sending system when missing transport packets are identified. The data channel  622  maintains a receiving buffer to perform necessary reordering if transport packets are out of order. The data channel  622  also estimates the network status parameters to monitor QoS and communicate the QoS information to the receiver controller  604 .  
      The receiver network engine  620  also includes a control channel  630 . The control channel  620  receives QoS reports  632  and receives client requests  634 . The control channel  620  also works with the transport layer to control or directly communicates control information using TCP, RTCP, or other protocols.  
      The sending system also includes a sender network engine  640 . Like the receiver network engine  620 , the sender network engine  640  includes a data channel  642  that maintains a sending buffer  644  for holding transport packets. Directly, or in concert with a transport layer, the data channel  642  performs data transmission using UDP, RTP, or other protocols. The data channel  642  also maintains a re-sending buffer  646  for base layer transport packets or other packets subject to ARQ. In one embodiment, two resending queues are maintained: a first queue for transport packets that will be resent if missing, and a second queue for transport packets for which ARQ is not necessary. The data channel  642  also includes an ARQ handler  648  to resend transport packets as needed.  
      The sender network engine  640  also includes a control channel  650 . The control channel  650  maintains a QoS handler  652  to receive QoS parameters from client systems. The control channel  650  also maintains a client request handler  654  to receive client requests. In addition, the control channel  650  includes a quality claim handler  656  to track quality claims regarding peer clients.  
      Both the sending and receiving systems also include peering systems. A sender peering system  660  performs includes a packetizer  662  and a forward error correction (FEC) handler  664 . The packetizer  662  forms the selected data into packettes, and collects the packettes into transport packets as designated by the scheduler  666 . The FEC handler  664  performs forward error correction of the packets. A receiver peering system  670  includes an inverse FEC handler  672  to perform inverse error correction. The receiver peering system  670  also includes an unpacker  674  to unpack the packettes from the transport packets generated by the packetizer  662 .  
      The sending and receiving systems each also include a kernel. The sender kernel  680  is a multiplexer that includes a packette serializer  682  and a quality summarizer  684  that provides a quality summary for each access unit. The receiver kernel  690  is a caching system that includes a memory  6  and disk storage  690 . The cache size determines the number of asynchronous clients that a single streaming session of a program can support. If the cache size is large enough to hold the whole streaming file, then all the clients can be supported with one instance of the streaming server. For a specific client, a quality constraint can be specified by the local controller  606  to the packettes selector  692  so that only the necessary number of packettes is passed to the sender kernel  680 . When multiple clients are to be supported with a single session, the quality constraint should be specified at the highest level needed to serve the client specifying the highest access level.  
      The block diagram also graphically depicts where a plurality of application program interfaces (APIs) engage the subsystems to allow applications to integrate with the peering subsystems and the kernels to control the access levels used in communicating the media content. More specifically,  FIG. 6  shows three input APIs, I- 1   601 , I- 2   603 , and I- 3   605 , and three output APIs, O- 1   607 , O- 2   609 , and O- 3   611 .  
      API I- 1  affects the flow of the packettes from the packettes selector  692  to the packette serializer  682 . API I- 2   603  controls the flow of data from the file encoding system  696  to the packette serializer  682 . API I- 3   605  controls the flow of data from the unpacker  674  to the packette serializer  682 . API O- 1   607  and O- 2   609  both affect the flow of data from the memory  688  of the receiver kernel  686  to the file decoding subsystem  698 . API O- 3   611  affects the flow of data to the packetizer  662 . Thus, applications can use these APIs to control what data are sent or are accessed in presenting or accessing codestreams at different access levels.  
      Process of Facilitating Adaptive Communication  
       FIG. 7  is a flow diagram of a mode of facilitating adaptive communication using an architecture as described in  FIGS. 3 and 6 , or a similarly capable system. The process  700  illustrates actions performed by the sending system and the receiving system, including actions performed cooperatively by both systems.  
      The process  700  begins at  702  with a client initiating a request for media content. At  704 , an initial access level is identified. The initial access level may be a default level or determined by the receiving system and/or the sending system based on preferences, processing and bandwidth capabilities, or other factors. At  706 , the media codestream is accessed and, at  708 , the data to be included in the packettes is identified. At  710 , the packettes are assembled. At  712 , the packettes are serialized. At  714 , the packettes collected in transport packets. At  716 , a number of the transport packets are collected in a buffer from which they can be resent if the packets are not received and thus are not acknowledged by the receiving system. At  718 , the transport packets are sent to one or more receiving systems.  
      At  720 , it is determined if a packet is not acknowledged by one or more receiving systems. If so, at  722 , the missed packet is retrieved from the buffer for resending, and the process  700  loops to  718  to resend the missed packet. On the other hand, if all packets are acknowledged, at  724 , it is determined if an access level change is indicated as a result of a user input, system conditions, or other factors. If so, at  726 , packette specifications are adjusted. Such an adjustment may involve additional coding units being included in packettes, or more packettes being selected for sending. The process  700  loops to  708  for the data to be included in the packettes to be identified based on the indicated change.  
      On the other hand, if it is determined at  724  that no access level change has been initiated, moving to actions performed by the receiving system, at  728 , missed packets are reported by sending NACK messages to the sending system (which are detected at  720  as previously described). At  730 , the quality of service is monitored for possible changes in the access level of the media content. At  732 , the packettes are unpacked from the transport layers. At  734 , the codestream is generated for presentation by the receiving system.  
      At  736 , it is determined if an access level change is indicated either by system conditions or a user selection. If so, at  738 , the packette selection is adjusted on the local system. Thus, if a local access level change is indicated, without changing the packettes being sent from the sending system, the local system can adjust the access level. The access level may be changed to a higher level than is currently being presented on the receiving system if a sufficient number of packettes are being sent and/or the packettes include sufficient coding units to permit a higher access level. If the access level is to be reduced, the access level can be reduced by reducing the number of packettes being accessed.  
      In addition, at  740 , an access level change also may be communicated to a sending system. Thus, if the receiving system is reducing the access level, the reduction may be communicated to the sending system to reduce the number of packettes being sent to reduce processing and/or bandwidth being used. If multiple receiving systems are receiving the codestream, the codestream may be unchanged.  
      On the other hand, if it is determined at  736  that no access level change is indicated, the process loops to  708  to continue the identification of data to be included in packettes to be provided by the sending system.  
      Exemplary Supported Applications  
      Embodiment of adaptive architectures as previously described support a number of enhanced media access applications, including providing multiple access level streaming of media content, device roaming, and time-dependent access level shifting.  
      To illustrate multiple access level streaming,  FIG. 8  shows a network of a sending system  800  communicating media content via a network  810  to a plurality of systems  820  and  830  having different capabilities for presenting media content. Using an architecture such as that illustrated in  FIG. 6 , the access level employed by a receiving system can be changed on the local system and changes in access levels also can be communicated between sending and receiving systems.  
      More specifically, in  FIG. 8  one of the receiving systems is handheld computer  820 , while the other receiving system is a personal computer  830 , such as a media computer, coupled with a high resolution video monitor  840 . For purposes of the example of  FIG. 8 , it can be assumed that systems  820  and  830  employ different access levels because each of the systems features a differently-enabled display. However, a difference in desired access level may be a function of different network bandwidths available to the systems, different digital rights management authorization between the systems, or other reasons. In the example of  FIG. 8 , the media content may be provided by a sending system in a one-way transaction, or the media content may be part of an interactive video conference call.  
       FIG. 9  is a flow diagram of a process  900  employed in the interaction between the serving system  800  and each of the receiving systems  820  and  830 . At  902 , the sending system identifies the capabilities of the receiving systems in order to determine the appropriate access level of the media content to be provided. For example, if all of the receiving systems are limited by bandwidth, processing, display, or other limitations, the sending system may provide the media content at a reduced access level to avoid unnecessary use of processing and bandwidth resources. At  904 , based on the highest current access level supported by the receiving systems, the sending system identifies packette parameters, including the number of packettes to be sent. In one mode, the access level is selected based on the highest capabilities among the receiving systems. Accordingly, each of the receiving systems can locally adjust the access level up to the highest level provided or to a lower level without affecting the access level provided to other receiving systems.  
      At  906 , the packettes are assembled, collected, and transmitted. At  908 , each of the receiving systems selects which packettes or which coding units included in the packettes will be accessed to determine the local access level. Each of the receiving systems, based on capabilities or user preferences, can access the packettes to present the media content at an access level up to the highest access level made possible by the packettes sent by the sending system, or at a lower access level.  
      In one mode, at  910 , it is determined if the current access level exceeds the highest capability access level that is currently being used. If so, at  912 , the current access level is reduced to the highest access level being used to reduce unnecessary use of processing and/or bandwidth resources. The process  900  then loops to  904  for the packette parameters to be changed. On the other hand, at  910  if it is determined that the current access level does not exceed the highest capability access level being used, the routine  900  loops to  906  for the packettes to continue to be assembled, collected, and transmitted.  
      It will be appreciated that if even one other receiving system can access the media content at the highest possible access level based on the packettes being sent, that no one receiving system should be able to undermine the access level of another system. However, as in the exemplary case of  FIG. 8 , the personal computer  830  with the high resolution video monitor  840  is capable of presenting higher resolution images than handheld computer  820 . The original access level identified by the sending system at  904  is selected to serve the more capable system  830 , which is likely a higher access level than can be supported by handheld system  820 . Then, if the access level at which the media content is engaged on the more capable system  830 , it would be a waste of resources for the sending system to present the media content at any higher level. Accordingly, at  912 , the change in access level is reported to the sending system and, the process loops to  904  for the packette parameters to be changed.  
      To illustrate device roaming,  FIG. 10  shows a network of a sending system  1000  communicating media content via a network  1010  to a plurality of systems  1020  and  1030  that may have equivalent or different capabilities for presenting media content as previously described in connection with  FIG. 8 . Again, handheld device  1020  is assumed to have lesser media access capabilities than personal computer  1030  which is associated with a high resolution video monitor  1040 . In contrast to the example of  FIG. 8 , however, in  FIG. 10 , the content  1050  being received by the receiving devices  1020  and  1030  may not be the same, and the content  1050  is migrated from one of the other receiving devices to the other, or the content received by each receiving device is swapped. For example, a user may be viewing a media program on the handheld device  1020 , but then decides she wishes to access the media program on a higher resolution device. Alternatively, the user may be viewing one media program on the personal computer  1030  and high resolution video monitor  1040 , while monitoring a second program on the handheld device  1020 , and the user may wish to swap the media programs between the receiving devices  1020  and  1030 .  
       FIG. 11  shows a flow diagram of a process  1100  employed in the migrating a codestream between receiving devices. At  1102 , an access level is selected for providing media content to a first system. The access level, as previously described, is based on user preferences, system capabilities, digital rights management permissions, or other factors. In addition, although not shown in  FIG. 11 , if a different system is being used to access different media content, the access level is determined for the different system in the same manner. Once the access level is selected (for each of one or more systems) at  1104 , packettes are formed and sent for presenting the media content at the currently selected access level(s). At  1106 , the media content is presented.  
      At  1108 , it is determined if the content from one (or both) of the systems is to be routed to a different system, for example, if the user elects to migrate or swap media content from one device to another. If not, the process  1100  loops to  1104  for packettes to be continued to be formed and sent at the current access level(s). On the other hand, if it is determined at  1108  that the content is to be rerouted to a different system, at  1110 , a desired or appropriate access level for the different system is identified. At  1112 , it is determined if a change in the packette parameters is indicated based on the access level identified for the different system. If not, the process loops to  1104  where packettes will continue to be presented to support the current access level. However, if a change in the packette parameters is indicated, at  1114  the packette parameters are changed, and the process again loops to  1104  for packettes to be formed at the new current access level.  
      Although not expressly shown in  FIG. 11 , the case where content is swapped between the handheld device  1020  ( FIG. 10 ) and the personal computer  1030 , there are two migrations that take place. One migration involves the content being accessed by the handheld device  1020  being migrated to the personal computer  1030 , and the other migration the content being accessed by the personal computer being migrated to the handheld device  1020 . Based on the earlier descriptions, it will be appreciated that the second migration to the handheld device will not necessarily involve a change in how the packettes are prepared; how the packettes are accessed on the handheld device  1020  can be used to control the access level at the local level. Only if the packette parameters are insufficient to support a higher level of access do the packette parameters need to be changed.  
      To illustrate time-dependent access level shifting,  FIG. 12  shows an example of a network including a sending system  1200  communicating media content via a network  1210  to a personal computer  1220 , acting as a digital video recorder, at time T=0:00 on a timeline  1230 , and continuing to send media content to the same personal computer at time T=2:00, represented as personal computer  1240 . The media content is a program  1250  that is 3:00 hours in length. The storage remaining at time T=0:00  1260  is 2:30 at a first access level Q 1 , and 5:00 at a second access level Q 2 . It is assumed that the user elected to record the program  1250  at Q 1 , a higher access level, and the personal computer  1220  will attempt honor that selection. However, to prevent the user from losing the end of the program  1250 , the personal computer is equipped to perform “disk squeezing,” by automatically shifting to a reduced second access level Q 2  as needed to “squeeze” the program onto the disk. Alternatively, if the user desires to retain access level Q 1  for the whole program, the receiving system may retroactively perform “disk squeezing” of some older stored programs to make room for the new program to be stored at Q 1 .  
      In the example of  FIG. 12 , it is assumed that the personal computer  1220  at time T=0:00 records the media content at higher access level Q 1 . However, upon reaching a threshold that is automatically determined or manually predetermined, the access level being used by personal computer  1240  in the recording is reduced. For example, the access level is reduced at time T=2:00 to access level Q 2 . As previously described, the receiving system can reduce the access level by reducing the number of packettes accessed.  
       FIG. 13  is a flow diagram of a process  1300  employed in recording, and potentially reducing the storage size of a program by truncating data used to support higher access levels. At  1302 , the media content received from the sending system is recorded at an originally established access level. At  1304 , the remaining storage capacity of the receiving system is monitored. At  1306 , it is determined if the storage capacity will fall short of the quantity needed to record the media content. If not, the process  1300  loops to  1302  to continue recording the media content at the originally established access level.  
      On the other hand, if it is determined at  1306  that the storage capacity will be insufficient at the current access level, at  1308 , the number of packettes accessed is reduced to change the access level used in recording the media content. At  1310 , the remainder of the media content is recorded at the reduced access level.  
       FIG. 14  is a flow diagram of a process  1400  employed in recording, and as needed, reducing the access levels used in storing previously-recorded programs to accommodate a newly recorded program. Storage space can be freed by truncating data used to support the access levels originally used in storing the previously-recorded programs, making that storage available for new programs. At  1402 , the media content received from the sending system is recorded at an established access level. At  1404 , the remaining storage capacity of the receiving system is monitored. At  1406 , it is determined if the storage capacity will fall short of the quantity needed to record the media content. If not, the process  1400  loops to  1402  to continue recording the media content.  
      On the other hand, if it is determined at  1406  that the storage capacity will be insufficient, at  1408 , previously-recorded programs that can be reduced to lower access levels are identified. A number of criteria can be established to determine which programs can be permissibly reduced in access level. For example, programs that have already been viewed or that are flagged as having low importance may be identified for data truncation. At  1410 , the identified programs are condensed in size by re-storing the programs at reduced access levels. The process  1400  then loops to  1402  to continue recording the new program in the storage space freed by reducing the storage space consumed by the previously-recorded programs.  
      These are just some of the applications enabled by an architecture providing adaptive access to media content. Many other applications are similarly supported. To cite one additional example, an embodiment of the architecture supports incremental streaming and enhancement of stored media content. Taking the example of  FIG. 12 , if media content was recorded at a reduced access level for the last hour of the program, the user may wish to free storage and re-record the last hour of the media content at a higher access level for quality consistency. In this case, because packettes are stored to provide access at the lower access level, additional packettes will need to be acquired to supplement the already stored packettes, instead of the entire codestream needing to be re-sent and re-recorded.  
      Computing System for Implementing Exemplary Embodiments  
       FIG. 15  illustrates an exemplary computing system  1500  for implementing embodiments of an architecture supporting adaptive access to scalable codestreams. The computing system  1500  is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of exemplary embodiments of an architecture supporting adaptive access to scalable codestreams or other embodiments. Neither should the computing system  1500  be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary computing system  1500 .  
      An architecture supporting adaptive access to scalable codestreams may be described in the general context of computer-executable instructions, such as program modules, being executed on computing system  1500 . Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. Moreover, those skilled in the art will appreciate that the architecture supporting adaptive access to scalable codestreams may be practiced with a variety of computer-system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable-consumer electronics, minicomputers, mainframe computers, and the like. The architecture supporting adaptive access to scalable codestreams may also be practiced in distributed-computing environments where tasks are performed by remote-processing devices that are linked through a communications network. In a distributed-computing environment, program modules may be located in both local and remote computer-storage media including memory-storage devices.  
      With reference to  FIG. 15 , an exemplary computing system  1500  for implementing the architecture supporting adaptive access to scalable codestreams includes a computer  1510  including a processing unit  1520 , a system memory  1530 , and a system bus  1521  that couples various system components including the system memory  1530  to the processing unit  1520 .  
      Computer  1510  typically includes a variety of computer-readable media. By way of example, and not limitation, computer-readable media may comprise computer-storage media and communication media. Examples of computer-storage media include, but are not limited to, Random Access Memory (RAM); Read Only Memory (ROM); Electronically Erasable Programmable Read Only Memory (EEPROM); flash memory or other memory technology; CD ROM, digital versatile discs (DVD) or other optical or holographic disc storage; magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices; or any other medium that can be used to store desired information and be accessed by computer  1510 . The system memory  1530  includes computer-storage media in the form of volatile and/or nonvolatile memory such as ROM  1531  and RAM  1532 . A Basic Input/Output System  1533  (BIOS), containing the basic routines that help to transfer information between elements within computer  1510  (such as during start-up) is typically stored in ROM  1531 . RAM  1532  typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit  1520 . By way of example, and not limitation,  FIG. 15  illustrates operating system  1534 , application programs  1535 , other program modules  1536 , and program data  1537 .  
      The computer  1510  may also include other removable/nonremovable, volatile/nonvolatile computer-storage media. By way of example only,  FIG. 15  illustrates a hard disk drive  1541  that reads from or writes to nonremovable, nonvolatile magnetic media, a magnetic disk drive  1551  that reads from or writes to a removable, nonvolatile magnetic disk  1552 , and an optical-disc drive  1555  that reads from or writes to a removable, nonvolatile optical disc  1556  such as a CD-ROM or other optical media. Other removable/nonremovable, volatile/nonvolatile computer-storage media that can be used in the exemplary operating environment include, but are not limited to, magnetic tape cassettes, flash memory units, digital versatile discs, digital video tape, solid state RAM, solid state ROM, and the like. The hard disk drive  1541  is typically connected to the system bus  1521  through a nonremovable memory interface such as interface  1540 . Magnetic disk drive  1551  and optical dick drive  1555  are typically connected to the system bus  1521  by a removable memory interface, such as interface  1550 .  
      The drives and their associated computer-storage media discussed above and illustrated in  FIG. 15  provide storage of computer-readable instructions, data structures, program modules and other data for computer  1510 . For example, hard disk drive  1541  is illustrated as storing operating system  1544 , application programs  1545 , other program modules  1546 , and program data  1547 . Note that these components can either be the same as or different from operating system  1534 , application programs  1535 , other program modules  1536 , and program data  1537 . Typically, the operating system, application programs, and the like that are stored in RAM are portions of the corresponding systems, programs, or data read from hard disk drive  1541 , the portions varying in size and scope depending on the functions desired. Operating system  1544 , application programs  1545 , other program modules  1546 , and program data  1547  are given different numbers here to illustrate that, at a minimum, they can be different copies. A user may enter commands and information into the computer  1510  through input devices such as a keyboard  1562 ; pointing device  1561 , commonly referred to as a mouse, trackball or touch pad; a wireless-input-reception component  1563 ; or a wireless source such as a remote control. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit  1520  through a user-input interface  1560  that is coupled to the system bus  1521  but may be connected by other interface and bus structures, such as a parallel port, game port, IEEE 1394 port, or a universal serial bus (USB)  1598 , or infrared (IR) bus  1599 . As previously mentioned, input/output functions can be facilitated in a distributed manner via a communications network.  
      A display device  1591  is also connected to the system bus  1521  via an interface, such as a video interface  1590 . Display device  1591  can be any device to display the output of computer  1510  not limited to a monitor, an LCD screen, a TFT screen, a flat-panel display, a conventional television, or screen projector. In addition to the display device  1591 , computers may also include other peripheral output devices such as speakers  1597  and printer  1596 , which may be connected through an output peripheral interface  1595 .  
      The computer  1510  will operate in a networked environment using logical connections to one or more remote computers, such as a remote computer  1580 . The remote computer  1580  may be a personal computer, and typically includes many or all of the elements described above relative to the computer  1510 , although only a memory storage device  1581  has been illustrated in  FIG. 15 . The logical connections depicted in  FIG. 15  include a local-area network (LAN)  1571  and a wide-area network (WAN)  1573  but may also include other networks, such as connections to a metropolitan-area network (MAN), intranet, or the Internet.  
      When used in a LAN networking environment, the computer  1510  is connected to the LAN  1571  through a network interface or adapter  1570 . When used in a WAN networking environment, the computer  1510  typically includes a modem  1572  or other means for establishing communications over the WAN  1573 , such as the Internet. The modem  1572 , which may be internal or external, may be connected to the system bus  1521  via the network interface  1570 , or other appropriate mechanism. Modem  1572  could be a cable modem, DSL modem, or other broadband device. In a networked environment, program modules depicted relative to the computer  1510 , or portions thereof, may be stored in the remote memory storage device. By way of example, and not limitation,  FIG. 15  illustrates remote application programs  1585  as residing on memory device  1581 . It will be appreciated that the network connections shown are exemplary, and other means of establishing a communications link between the computers may be used.  
      Although many other internal components of the computer  1510  are not shown, those of ordinary skill in the art will appreciate that such components and the interconnections are well-known. For example, including various expansion cards such as television-tuner cards and network-interface cards within a computer  1510  is conventional. Accordingly, additional details concerning the internal construction of the computer  1510  need not be disclosed in describing exemplary embodiments of the key management process.  
      When the computer  1510  is turned on or reset, the BIOS 1533, which is stored in ROM  1531 , instructs the processing unit  1520  to load the operating system, or necessary portion thereof, from the hard disk drive  1541  into the RAM  1532 . Once the copied portion of the operating system, designated as operating system  1544 , is loaded into RAM  1532 , the processing unit  1520  executes the operating system code and causes the visual elements associated with the user interface of the operating system  1534  to be displayed on the display device  1591 . Typically, when an application program  1545  is opened by a user, the program code and relevant data are read from the hard disk drive  1541  and the necessary portions are copied into RAM  1532 , the copied portion represented herein by reference numeral  1535 .  
     CONCLUSION  
      Although exemplary embodiments have been described in language specific to structural features and/or methodological acts, it is to be understood that the appended claims are not necessarily limited to the specific features or acts previously described. Rather, the specific features and acts are disclosed as exemplary embodiments.