Patent Publication Number: US-8126048-B2

Title: Recording streaming delta-encoded data

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application is also related to application Ser. No.: 12/051,430, filed Mar. 19, 2008. 
     BACKGROUND 
     The present disclosure relates generally to the processing of streaming delta-encoded data, and more particularly to the recording of streaming delta-encoded data. 
     Various compression techniques have been developed to enable efficient transmission of data streams such as video streams. One family of these compression techniques is referred to as delta encoding. In delta encoding, after encoding and transmitting a complete picture, referred to as an intraframe or I Frame, only the differences between subsequent pictures are encoded and transmitted. These differences are referred to as interframes, and in various compression protocols include predicted frames (P Frames), Bi-predictive frames (B Frames), and the like. 
     Different transport-layer protocols can be used to transport packets of delta-encoded data. For example, User Datagram Protocol (UDP) can be used. But because UDP does not guarantee reliability or ordering, it is necessary to transmit I Frames frequently in order to supply data missing due to lost frames. The use of UDP also poses significant security risks. Therefore UDP packets are generally not permitted to traverse the network firewalls employed by most businesses, rendering UDP unsuitable for business communications. 
     Another transport-layer protocol that can be used to transport packets of delta-encoded data is Transmission Control Protocol (TCP). Because TCP provides reliable, in-order delivery of packets, it does not pose the security issues associated with UDP. In addition, it is only necessary to transmit one I Frame at the beginning of each stream of delta-encoded data. 
     SUMMARY 
     In general, in one aspect, an embodiment features an apparatus comprising: an input circuit adapted to receive a stream of delta-encoded data; a record circuit adapted to store the stream of delta-encoded data in a stream file; a current frame circuit adapted to maintain a current frame of the data based on the stream of delta-encoded data; a record frame circuit adapted to store the current frame in a frame file as an intraframe with a frame pointer at a plurality of different index times, wherein each frame pointer indicates a location in the stream file associated with the respective index time; and a record index circuit adapted to store the index times sequentially in an index file, and adapted to store an index pointer for each index time, wherein each index pointer indicates a location in the frame file of the intraframe associated with the respective index time. 
     Embodiments of the apparatus can include one or more of the following features. In some embodiments, the stream of delta-encoded data includes a plurality of packets of video data; and wherein the record circuit stores the packets in the stream file. In some embodiments, the video data complies with at least one of: one or more of the Video Coding Experts Group (VCEG) standards; and one or more of the Moving Picture Experts Group (MPEG) standards. In some embodiments, the stream of delta-encoded data includes a plurality of real-time protocol (RTP) packets, wherein each RTP packet includes at least a portion of one of the packets of the video data; and wherein the record circuit stores the RTP packets in the stream file. In some embodiments, the RTP packets are received using transmission control protocol/Internet protocol (TCP/IP). 
     In general, in one aspect, an embodiment features an apparatus comprising: input means for receiving a stream of delta-encoded data; recording means for storing the stream of delta-encoded data in a stream file; current frame means for maintaining a current frame of the data based on the stream of delta-encoded data; record frame means for storing the current frame in a frame file as an intraframe with a frame pointer at a plurality of different index times, wherein each frame pointer indicates a location in the stream file associated with the respective index time; and record index means for storing the index times sequentially in an index file, and for storing an index pointer for each index time, wherein each index pointer indicates a location in the frame file of the intraframe associated with the respective index time. 
     Embodiments of the apparatus can include one or more of the following features. In some embodiments, the stream of delta-encoded data includes a plurality of packets of video data; and wherein the means for recording stores the packets in the stream file. In some embodiments, the video data complies with at least one of: one or more of the Video Coding Experts Group (VCEG) standards; and one or more of the Moving Picture Experts Group (MPEG) standards. In some embodiments, the stream of delta-encoded data includes a plurality of real-time protocol (RTP) packets, wherein each RTP packet includes at least a portion of one of the packets of the video data; and wherein the record circuit stores the RTP packets in the stream file. In some embodiments, the RTP packets are received using transmission control protocol/Internet protocol (TCP/IP). 
     In general, in one aspect, an embodiment features a computer-readable media embodying instructions executable by a computer to perform a method comprising: receiving a stream of delta-encoded data; storing the stream of delta-encoded data in a stream file; maintaining a current frame of the data based on the stream of delta-encoded data; storing the current frame in a frame file as an intraframe with a frame pointer at a plurality of different index times, wherein each frame pointer indicates a location in the stream file associated with the respective index time; and storing the index times sequentially in an index file, and for storing an index pointer for each index time, wherein each index pointer indicates a location in the frame file of the intraframe associated with the respective index time. 
     Embodiments of the computer-readable media can include one or more of the following features. In some embodiments, the stream of delta-encoded data includes a plurality of packets of video data; and the method further comprises storing the packets in the stream file. In some embodiments, the video data complies with at least one of: one or more of the Video Coding Experts Group (VCEG) standards; and one or more of the Moving Picture Experts Group (MPEG) standards. In some embodiments, the stream of delta-encoded data includes a plurality of real-time protocol (RTP) packets, wherein each RTP packet includes at least a portion of one of the packets of the video data; and wherein the method further comprising storing the RTP packets in the stream file. In some embodiments, the RTP packets are received using transmission control protocol/Internet protocol (TCP/IP). 
     In general, in one aspect, an embodiment features a method comprising: receiving a stream of delta-encoded data; storing the stream of delta-encoded data in a stream file; maintaining a current frame of the data based on the stream of delta-encoded data; storing the current frame in a frame file as an intraframe with a frame pointer at a plurality of different index times, wherein each frame pointer indicates a location in the stream file associated with the respective index time; and storing the index times sequentially in an index file, and for storing an index pointer for each index time, wherein each index pointer indicates a location in the frame file of the intraframe associated with the respective index time. 
     Embodiments of the method can include one or more of the following features. In some embodiments, the stream of delta-encoded data includes a plurality of packets of video data; and the method further comprises storing the packets in the stream file. In some embodiments, the video data complies with at least one of: one or more of the Video Coding Experts Group (VCEG) standards; and one or more of the Moving Picture Experts Group (MPEG) standards. In some embodiments, the stream of delta-encoded data includes a plurality of real-time protocol (RTP) packets, wherein each RTP packet includes at least a portion of one of the packets of the video data; and wherein the method further comprising storing the RTP packets in the stream file. In some embodiments, the RTP packets are received using transmission control protocol/Internet protocol (TCP/IP). 
     In general, in one aspect, an embodiment features a data structure embodied in a computer-readable storage medium, the data structure comprising: a stream file comprising a stream of delta-encoded data; a frame file comprising a plurality of intraframes each representing the delta-encoded data preceding a different index time, and a frame pointer for each of the intraframes, wherein each frame pointer indicates a location in the stream file for the index time corresponding to the respective intraframe; and an index file comprising the index times arranged in sequence, and an index pointer for each of the index times, wherein each index pointer indicates a location in the frame file for the intraframe corresponding to the respective index time. 
     Embodiments of the data structure can include one or more of the following features. In some embodiments, the stream of delta-encoded data includes a plurality of packets of video data. In some embodiments, the video data complies with at least one of: one or more of the Video Coding Experts Group (VCEG) standards; and one or more of the Moving Picture Experts Group (MPEG) standards. In some embodiments, the stream of delta-encoded data includes a plurality of real-time protocol (RTP) packets, wherein each RTP packet includes at least a portion of one of the packets of the video data. 
     The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. 
    
    
     
       DESCRIPTION OF DRAWINGS 
         FIG. 1  shows a recording apparatus for recording a stream of delta-encoded data according to one embodiment. 
         FIG. 2  shows a recording process for the recording apparatus of  FIG. 1  according to one embodiment. 
         FIG. 3  shows a data structure for storing delta-encoded data according to one embodiment. 
         FIG. 4  shows a playback apparatus for playing a recorded stream of delta-encoded data according to one embodiment. 
         FIG. 5  shows a playback process for the playback apparatus of  FIG. 4  according to one embodiment. 
     
    
    
     The leading digit(s) of each reference numeral used in this specification indicates the number of the drawing in which the reference numeral first appears. 
     DETAILED DESCRIPTION 
     Embodiments of the present invention provide techniques for recording and playback of streaming delta-encoded data. The disclosed techniques enable operations such as seek operations during playback. The streaming delta-encoded data can represent video data such as a video channel of a video conference and the like. The streaming delta-encoded data can be transmitted using protocols such as real-time transport protocol (RTP), transmission control protocol (TCP), Internet protocol (IP), and the like. 
     Because TCP provides reliable delivery of packets, only one intraframe need be transmitted for each stream of delta-encoded data. The intraframe is transmitted first, and only interframes are required thereafter. While this technique provides very efficient bandwidth usage, it can make some playback operations, such as seek, fast-forward, and rewind, very expensive in terms of processing resources and time required. For example, to seek to a point in a recorded video stream, it would be necessary to process the stream from the beginning in order to process the initial intraframe. 
       FIG. 1  shows a recording apparatus  100  for recording a stream of delta-encoded data according to one embodiment. Although in the described embodiments, the elements of recording apparatus  100  are presented in one arrangement, other embodiments may feature other arrangements, as will be apparent to one skilled in the relevant arts based on the disclosure and teachings provided herein. For example, the elements of recording apparatus  100  can be implemented in hardware, software, or combinations thereof. In some embodiments, recording apparatus  100  can be used to record a videoconference. 
     Referring to  FIG. 1 , recording apparatus  100  includes a memory  102 , an input circuit  104  adapted to receive a stream of delta-encoded data  106 , a record circuit  108  adapted to store stream  106  in a stream file  110 , a current frame circuit  112  adapted to maintain a current frame  114  of the data based on stream  106 , a record frame circuit  116  adapted to store current frame  114  in a frame file  118  at a plurality of different index times, and a record index circuit  120  adapted to store the index times sequentially in an index file  122 . Recording apparatus  100  can also include a clock  124  to generate a time basis for the index times. 
       FIG. 2  shows a recording process  200  for recording apparatus  100  of  FIG. 1  according to one embodiment. Although in the described embodiments, the elements of process  200  are presented in one arrangement, other embodiments may feature other arrangements, as will be apparent to one skilled in the relevant arts based on the disclosure and teachings provided herein. For example, in various embodiments, some or all of the steps of process  200  can be executed in a different order, concurrently, and the like. 
     Referring to  FIG. 2 , input circuit  104  receives a stream of delta-encoded data  106  (step  202 ). Record circuit  108  stores the stream of delta-encoded data  106  in stream file  110  (step  204 ). In some embodiments, delta-encoded data  106  represents packetized video data. That is, delta-encoded data  106  includes a plurality of packets of video data. In these embodiments, record circuit  108  stores the packets in stream file  110 . In some embodiments, the video data complies with one or more standards such as the Video Coding Experts Group (VCEG) standards, the Moving Picture Experts Group (MPEG) standards, and the like. For example, the video data can comply with standards such as H.261, H.263, H.264, and the like. In other embodiments, delta-encoded data  106  can represent other sorts of data. 
     In some embodiments, the stream of delta-encoded data  106  includes a plurality of real-time transport protocol (RTP) packets, where each RTP packet includes at least a portion of one of the packets of the video data. Each RTP packet includes a timestamp and a sequence number. In these embodiments, record circuit  108  stores the RTP packets in stream file  110  in sequence according to the timestamps and/or sequence numbers. The RTP packets can be received using TCP/IP. 
     Based on the stream of delta-encoded data  106 , current frame circuit  112  maintains a current frame  114  of the data  106  (step  206 ). For example, current frame circuit  112  stores the initial intraframe of the stream as the current frame  114 , and then applies the changes represented by the subsequent intraframes of the stream to the current frame  114 . 
     Record frame circuit  116  occasionally stores the current frame  114 , along with a frame pointer FP, in frame file  118  (step  208 ). Record frame circuit  116  can store the current frame  114  at regular intervals, for example every 10 seconds. Of course other intervals can be used. Each of the stored frames is referred to herein as an intraframe IF. The time at which an intraframe IF and the corresponding frame pointer FP are stored is referred to as an index time IT. The index times IT can be calculated as elapsed times from the beginning of reception of the stream of delta-encoded data  106 . Each frame pointer FP indicates a location in stream file  110  associated with the respective index time IT. 
     Record index circuit  120  stores the index times IT sequentially in index file  122 , along with an index pointer IP for each index time (step  210 ). Each index pointer IP indicates the location in frame file  118  of the intraframe IF associated with the respective index time IT. 
       FIG. 3  shows a data structure  300  for storing delta-encoded data according to one embodiment. Data structure  300  can be embodied in a computer-readable storage medium. For example, referring to  FIG. 1 , data structure  300  can be embodied in memory  102 . Data structure  300  can be produced by recording apparatus  100  of  FIG. 1 . Referring again to  FIG. 3 , data structure  300  includes stream file  110 , frame file  118 , and index file  122 . 
     Stream file  110  includes a recorded stream of delta-encoded data  106 , which can be stored in the form of RTP packets. In  FIG. 3 , stream file  110  includes M RTP packets labeled RTP( 0 )-RTP(M) where packet RTP( 0 ) is the first RTP packet in the stream of delta-encoded data  106 . 
     Frame file  118  includes a plurality of intraframes IF each representing the delta-encoded data  106  preceding a different index time IT. Frame file  118  also includes a frame pointer FP for each of the intraframes IF. In  FIG. 3 , frame file  118  includes N intraframes IF( 0 )-IF(N), each associated with one of N frame pointers FP( 0 )-FP(N). Each frame pointer FP indicates a location in stream file  110  for the index time IT corresponding to the respective intraframe IF. For example, the first intraframe (IF) can represent the initial frame of data in the stream of delta-encoded data  106 . In that example, intraframe IF( 0 ) is stored at index time IT( 0 ), along with a frame pointer FP( 0 ) that points to the location of the first RTP packet RTP( 0 ) in stream file  110 . 
     Index file  122  includes the index times IT arranged in sequence, and an index pointer IP for each of the index times IT. In  FIG. 3 , index file  122  includes N index times IT( 0 )-IT(N), each associated with one of N index pointers IP( 0 )-IP(N). Each index pointer IP indicates a location in frame file  118  for the intraframe IF corresponding to the respective index time IT. For example, the first index time IT( 0 ) is the time at which the first intraframe IF( 0 ) was generated, so the associated index pointer IP( 0 ) points to the location of intraframe IF( 0 ) in frame file  118 . 
       FIG. 4  shows a playback apparatus  400  for playing a recorded stream of delta-encoded data according to one embodiment. Playback apparatus  400  can be used to play back a stream of delta-encoded data  106  stored in the data structure  300  of  FIG. 3 . Although in the described embodiments, the elements of playback apparatus  400  are presented in one arrangement, other embodiments may feature other arrangements, as will be apparent to one skilled in the relevant arts based on the disclosure and teachings provided herein. For example, the elements of playback apparatus  400  can be implemented in hardware, software, or combinations thereof. In some embodiments, playback apparatus  400  can be used to play a recorded videoconference. 
     Referring to  FIG. 4 , playback apparatus  400  includes a memory  402 , a seek circuit  404  adapted to receive a seek time ST for a stream of delta-encoded data  106  stored in a stream file  110 , a playback index circuit  406  adapted to select an index time IT based on the seek time ST, and adapted to retrieve an index pointer IP associated with the index time IT from an index file  122 , a playback frame circuit  408  adapted to select an intraframe IF based on the index pointer IP, and adapted to retrieve the intraframe IF, and a frame pointer FP associated with the intraframe IF, from a frame file  122 , and an output circuit  410  adapted to transmit a stream of delta-encoded data  420  based on the intraframe IF and the delta-encoded data  106  stored in stream file  110  following a location indicated by the frame pointer FP. 
     In some embodiments, output circuit  410  can transmit data in the form of RTP packets, for example as stored in stream file  110  of data structure  300  of  FIG. 3 . In such embodiments, output circuit  410  can include a timestamp circuit  412  adapted to modify the timestamps of the RTP packets and a sequence circuit  414  adapted to modify the sequence numbers of the RTP packets. In some embodiments, output circuit  410  can include a frame generation circuit  416  adapted to generate a frame based on the intraframe IF and the delta-encoded data  106  in the stream file  110  located between the index time IT and the seek time ST. 
       FIG. 5  shows a playback process  500  for playback apparatus  400  of  FIG. 4  according to one embodiment. Although in the described embodiments, the elements of process  500  are presented in one arrangement, other embodiments may feature other arrangements, as will be apparent to one skilled in the relevant arts based on the disclosure and teachings provided herein. For example, in various embodiments, some or all of the steps of process  500  can be executed in a different order, concurrently, and the like. 
     Referring to  FIG. 5 , seek circuit  404  receives a seek time ST for a stream of delta-encoded data  106  stored in stream file  110  (step  502 ). For example, the seek time can be provided by a user during playback of the stream of delta-encoded data  106 . 
     Playback index circuit  406  selects an index time IT based on the seek time ST (step  504 ). In some embodiments, the user can be restricted to choosing seek times ST that exactly match the index times IT. In such embodiments, playback index circuit  406  simply uses seek time ST as the index time IT. But in embodiments where the user can choose any seek time ST, playback index circuit  406  can select the index time IT immediately preceding the seek time ST. For example, if the index times IT occur at 10-second intervals and the seek time ST=45 seconds, playback index circuit  406  can select the index time IT( 3 )=40 seconds. 
     Playback index circuit  406  retrieves the index pointer IP associated with the selected index time IT from index file  122  (step  506 ). For example, if the selected index time is IT( 3 ), playback index circuit  406  retrieves the index pointer IP( 3 ). 
     Playback frame circuit  408  selects and retrieves an intraframe IF, and the frame pointer FP associated with the intraframe IF, based on the selected index pointer IP (step  508 ). For example, if the selected index pointer is IP( 3 ), playback frame circuit  408  retrieves intraframe IF( 3 ) and frame pointer FP( 3 ). 
     Output circuit  410  transmits a stream of delta-encoded data  420  based on the intraframe IF and the delta-encoded data  106  stored in stream file  110  following the location indicated by the frame pointer FP (step  510 ). Output circuit  410  can transmit the stream of delta-encoded data  420  as RTP packets. Output circuit  410  can transmit the RTP packets using TCP/IP. 
     In embodiments where seek times ST are restricted to index times IT, output circuit  410  simply transmits the intraframe IF followed by the delta-encoded data  106  in stream file  110  following index time IT. For example, if the selected index pointer is IP( 3 ), output circuit  410  transmits intraframe IF( 3 ) followed by the delta-encoded data  106  in stream file  110  following the location indicated by frame pointer FP( 3 ). 
     In embodiments where seek times ST are unrestricted, frame generation circuit  416  can first generate a frame for the seek time ST based on the intraframe IF and the delta-encoded data  106  in stream file  110  located between the index time IT and the seek time ST. Output circuit  410  then transmits the frame followed by the delta-encoded data  106  in stream file  110  following seek time ST. For example, if the seek time ST is 45 seconds, and the selected index time IT( 3 )=40 seconds, frame generation circuit  416  generates a frame based on intraframe IF( 3 ) and the 5 seconds of delta-encoded data  106  in stream file  110  located between t=40 seconds and t=45 seconds. Output circuit  410  then transmits the frame followed by the delta-encoded data  106  in stream file  110  following t=45 seconds. 
     In embodiments where delta-encoded data  106  is stored in the form of RTP packets, output circuit  410  transmits the RTP packets. However, the recipient of the RTP packets may expect the timestamps and sequence numbers of the stream of delta-encoded data  420  transmitted by output circuit  410  to appear as a continuous RTP stream. That is, the recipient may expect the timestamps and sequence numbers of the RTP packets in stream  420  to increase in value despite whatever operations such as seek, fast-forward, rewind, and the like, are taking place at playback apparatus  400 . Therefore output circuit  410  can include timestamp circuit  412  to modify the timestamps, and sequence circuit  414  to modify the sequence numbers, of the RTP packets before transmitting the RTP packets. 
     The invention can be implemented in digital electronic circuitry, or in computer hardware, firmware, software, or in combinations of them. Apparatus of the invention can be implemented in a computer program product tangibly embodied in a machine-readable storage device for execution by a programmable processor; and method steps of the invention can be performed by a programmable processor executing a program of instructions to perform functions of the invention by operating on input data and generating output. The invention can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. Each computer program can be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language if desired; and in any case, the language can be a compiled or interpreted language. Suitable processors include, by way of example, both general and special purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory and/or a random access memory. Generally, a computer will include one or more mass storage devices for storing data files; such devices include magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and optical disks. Storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including by way of example semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks. Any of the foregoing can be supplemented by, or incorporated in, ASICs (application-specific integrated circuits). 
     A number of implementations of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other implementations are within the scope of the following claims.