Source: https://patents.google.com/patent/US9906785B2/en
Timestamp: 2019-01-23 22:30:46
Document Index: 136385849

Matched Legal Cases: ['art 2', 'art 2', 'art 1', 'art1', 'art 1', 'art1']

US9906785B2 - Systems, methods, and media for transcoding video data according to encoding parameters indicated by received metadata - Google Patents
Systems, methods, and media for transcoding video data according to encoding parameters indicated by received metadata Download PDF
US9906785B2
US9906785B2 US13841943 US201313841943A US9906785B2 US 9906785 B2 US9906785 B2 US 9906785B2 US 13841943 US13841943 US 13841943 US 201313841943 A US201313841943 A US 201313841943A US 9906785 B2 US9906785 B2 US 9906785B2
US13841943
US20140269927A1 (en )
Ivan Vladimirovich Naletov
Sergey Zurpal
Methods, systems, and computer readable media for transcoding video data based on metadata are provided. Methods for transcoding video data using meta data are provided that include receiving portions of encoded video encoded according to varying encoding schemes. The portions of encoded video can be re-encoded according to different encoding schemes. Different encoding schemes can be selected according to received metadata. The received metadata can indicate transcoding parameters for use in re-encoding portions of encoded vide. The portions of encoded video can be received from media content sources and the metadata can be received from a media metadata source. In various embodiments, the methods can be performed by a transcoding device and/or implemented via a computer readable media and at least one processing unit.
Transcoding is an important task in video distribution applications. For example, a transcoder can receive input video data having a first format and convert the input video data into video data having a second format. More particularly, for example, the first format and the second format can correspond to different video coding standards, such as Motion JPEG, JPEG 2000, MPEG-2, MPEG-4, H.263, H.264, AVC, High Efficiency Video Coding (HEVC), etc. Alternatively or additionally, the first format and the second format can have different bitrates and/or resolutions.
There are many current approaches to transcoding video data. For example, a transcoder can decode video data compressed in a first format into raw video data and re-encode the raw video data into a second format. More particularly, for example, the transcoder can estimate encoding parameters and re-encode the raw video data using the estimated encoding parameters. The estimation of encoding parameters within a transcoder is very time-consuming.
Accordingly, new mechanisms for transcoding video data are desirable.
In view of the foregoing, systems, methods, and media for transcoding video data using metadata are provided.
In some embodiments, methods for transcoding video data using metadata are provided, the methods comprising: receiving a first plurality of encoded images from a storage device; decoding the first plurality of encoded images based on a first coding scheme to generate a plurality of decoded images; receiving a plurality of encoding parameters from the storage device; and encoding the plurality of decoded images into a second plurality of encoded images based on a second coding scheme and the plurality of encoding parameters.
In some embodiments, systems for transcoding video data using metadata are provided, the systems comprising: processing circuitry configured to: receive a first plurality of encoded images from a storage device; decode the first plurality of encoded images based on a first coding scheme to generate a plurality of decoded images; receive a plurality of encoding parameters from the storage device; and encode the plurality of decoded images into a second plurality of encoded images based on a second coding scheme and the plurality of encoding parameters.
In some embodiments, non-transitory media containing computer-executable instructions that, when executed by a processing circuitry, cause the processing circuitry to performing a method for transcoding video data are provided, the method comprising: receiving a first plurality of encoded images from a storage device; decoding the first plurality of encoded images based on a first coding scheme to generate a plurality of decoded images; receiving a plurality of encoding parameters from the storage device; and encoding the plurality of decoded images into a second plurality of encoded images based on a second coding scheme and the plurality of encoding parameters.
FIG. 1 shows a generalized block diagram of an example of an architecture of hardware that can be used in accordance with some embodiments of the invention;
FIG. 2 shows a block diagram of an example of storage device and transcoder in accordance with some embodiments of the invention;
FIG. 3 shows a flow chart of an example of a process for transcoding video data in accordance with some embodiments of the invention;
FIG. 4 shows a flow chart of an example of a process for decoding video data in accordance with some embodiments of the invention; and
FIG. 5 shows a flow chart of an example of a process for encoding video data in accordance with some embodiments of the invention.
This invention generally relates to mechanisms (which can be systems, methods, media, etc.) for transcoding video data based on metadata. In some embodiments, the mechanisms can be used to transcode video data having a first format into video data having a second format.
In some embodiments, the mechanisms can receive a compressed bitstream and media metadata. The mechanisms can decompress the compressed bitstream and generate decoded video data based on a first coding scheme. The mechanisms can then encode the decoded video data based on a second coding scheme.
In some embodiments, the media metadata can include any suitable data. For example, the media metadata can include a set of coding parameters that can be used to encoding video data. More particularly, the media metadata can include information about one or more video scenes, such as a scene change indication signal, the number of frames between two scenes, the type of a video scene, etc. The media metadata can also include motion data, intra-prediction information, picture complexity information, etc. about video data.
In some embodiments, the mechanisms can encode the decoded video data using the media content data. For example, the mechanisms can generate a prediction image based on the motion data, the intra-prediction information, etc. As another example, the mechanisms can perform rate-control on the decoded video data based on the information about the video scenes, picture complexity information, etc.
Turning to FIG. 1, a generalized block diagram of an example 100 of an architecture of hardware that can be used in accordance with some embodiments is shown. As illustrated, architecture 100 can include a media content source 102, a media encoder 104, a media metadata source 106, a communications network 108, a storage device 110, a transcoder 112, and communications paths 114, 116, 118, 120, 122, 124, 126, 128, and 130.
Media content source 102 can include any suitable device that can provide media content. For example, media content source 102 can include one or more suitable cameras that can be configured to capture still images or moving images. As another example, media content source 102 can include one or more types of content distribution equipment for distributing any suitable media content, including television distribution facility equipment, cable system head-end equipment, satellite distribution facility equipment, programming source equipment (e.g., equipment of television broadcasters, such as NBC, ABC, HBO, etc.), intermediate distribution facility equipment, Internet provider equipment, on-demand media server equipment, and/or any other suitable media content provider equipment. NBC is a trademark owned by the National Broadcasting Company, Inc., ABC is a trademark owned by the ABC, INC., and HBO is a trademark owned by the Home Box Office, Inc.
Media content source 102 may include a remote media server used to store different types of content (including video content selected by a user), in a location remote from any of the user equipment devices. Systems and methods for remote storage of content, and providing remotely stored content to user equipment are discussed in greater detail in connection with Ellis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, which is hereby incorporated by reference herein in its entirety.
Media encoder 104 can include any suitable circuitry that is capable of encoding media content. For example, media encoder 104 can include one or more suitable video encoders, audio encoders, video decoders, audio decoders, etc. More particularly, for example, media encoder 104 can include one or more video encoders that can encode video data including a set of images in accordance with a suitable coding standard, such as Motion JPEG, JPEG 2000, MPEG-2, MPEG-4, H.263, H.264, AVC, High Efficiency Video Coding (HEVC), etc. As referred to herein, an image can have any suitable size and shape. For example, an image can be a frame, a field, or any suitable portion of a frame or a field, such as a slice, a block, a macroblock, a set of macroblocks, a coding tree unit (CTU), a coding tree block (CTB), etc.
Media metadata source 106 can include any suitable circuitry that is capable of providing metadata for media content. The metadata for media content can include any suitable information about the media content. For example, the metadata can include one or more coding parameters that can be used by suitable encoding circuitry and/or suitable decoding circuitry to encode and/or decode video data including multiple video frames.
In a more particular example, the metadata can include information about one or more video scenes, each of which can be composed of a set of images that have similar content. More particularly, for example, the metadata can include scene change information that can indicate the start and/or end of one or more scene changes in the video data. In some embodiments, the metadata can also include a set of parameters that can indicate the type of each of the scene changes, such as a shot change, a fading change, a dissolving change, etc. In some embodiments, the metadata can include the number of images between two scene changes. For example, the metadata can include the number of images between two consecutive scene changes, two scene changes of a given type (e.g., such as two shot changes), etc.
In another more particular example, the media metadata can include picture complexity information. The picture complexity information can include any suitable information about the spatial and/or temporal complexity of an image, such as a frame, a field, a slice, a macroblock, a sub-macroblock, a CTU, a CTB, etc.
In some embodiments, for example, the picture complexity information can include spatial complexity of an image that can indicate the amount of intra-distortion across the image. The amount of intra-distortion can be measured in any suitable manner. For example, the amount of intra-distortion of the image can be measured based on the variances of pixel values, luminance, brightness, or other characteristics of the image using a suitable metric, such as the mean absolute difference (MAD), the mean square error (MSE), etc. In some embodiments, the spatial complexity of a frame can be measured using the sum of the spatial complexity of the macroblocks and/or CTUs of the frame. In some embodiments, the picture complexity information can include a map of spatial complexity distribution within a frame for each frame of the video data.
In some embodiments, for example, the picture complexity information can include temporal complexity of an image that can indicate the amount of motion between the image and one or more reference images. The amount of motion can be represented in any suitable manner. For example, the amount of motion between the image and a reference can be measured using a suitable difference metric, such as the sum of the absolute difference (SAD), the sum of the squared difference (SSD), the mean absolute difference (MAD), the sum of absolute transformed differences (SATD), etc. More particularly, for example, the temporal complexity of a frame can be represented as the SAD, SSD, MAD, SATD, etc. between two consecutive frames. In some embodiments, the picture complexity information can include a map of temporal complexity distribution within a frame for each frame of the video data.
In yet another more particular example, the metadata can include motion data about the video data. The motion data can be generated in any suitable manner and can include any suitable data about changes among video frames due to object motions, camera motions, uncovered regions, lighting changes, etc. More particularly, for example, media metadata source 106 can generate a motion vector map for each video frame of the media content, motion characteristics (e.g., high motion, slow motion, etc.) of one or a set of frames, the number of B-frames between two P-frames, etc. In some embodiments, the motion data can be generated based on a suitable motion estimation algorithm, such as a block matching algorithm, an optical flow algorithm, a sub-pixel motion estimation algorithm, a hieratical block matching algorithm, etc. For example, in some embodiments, the motion vector map can include a set of integer vectors corresponding to each integer pixel of a video frame. As another example, the motion vector map can include a set of fractional motion vectors corresponding to each sub-pixel of the video frame (e.g., ½ pixel, ¼ pixel, ⅛ pixel, etc.). In some embodiments, the media metadata can also include one or more reference lists that can contain a set of frames that can serve as reference frames.
As yet another example, the media metadata can include intra-prediction data about the media content. The intra prediction data can include any suitable data that can be used for intra prediction under a suitable coding standard. For example, the intra-prediction data can include a set of candidate intra prediction modes, such as a vertical mode, a horizontal mode, a DC mode, a diagonal down-left mode, a diagonal down-right mode, a vertical-right mode, a horizontal-down node, a vertical-left mode, a horizontal-up mode, a plane mode, an intra-angular mode, etc. Additionally, the intra-prediction data can include a coding cost and/or distortion corresponding to each intra-prediction mode.
In some embodiments, the media metadata can be stored based on the play order of the video frames.
Storage device 110 can be any suitable digital storage mechanism in some embodiments. For example, storage 110 can include any device for storing electronic data, program instructions, computer software, firmware, register values, etc., such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. Storage 110 may be used to store media content, media metadata, media guidance data, executable instructions (e.g., programs, software, scripts, etc.) for providing an interactive media guidance application, and for any other suitable functions, and/or any other suitable data or program code, in accordance with some embodiments. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions), in some embodiments. In some embodiments, storage 110 can store media content, encoded video data, and/or metadata provided by media content source 102, media encoder 104, and/or media metadata source 106.
Transcoder 112 can include any suitable circuitry that is capable of converting input media content having a first format into media content having a second format. For example, transcoder 112 can include a suitable video transcoder that can convert a first set of images that are encoded in accordance with a first coding scheme into a second set of images that are encoded in accordance with a second coding scheme. In some embodiments, the first coding scheme and the second coding scheme may have different target bitrates. In some embodiments, the first set of encoded images and the second set of encoded images may have different resolutions, such as spatial resolutions, temporal resolutions, quality resolutions, etc. In some embodiments, the first coding scheme and the second coding scheme may correspond to different coding standards, such as Motion JPEG, JPEG 2000, MPEG-2, MPEG-4/AVC, H.263, H.264, High Efficiency Video Coding (HEVC), etc. More particularly, for example, in some embodiments, transcoder 112 can convert a set of images encoded based on MPEG-2 standard into a set of images encoded based on HEVC standard.
In some embodiments, communications network 108 may be any one or more networks including the Internet, a mobile phone network, a mobile voice, a mobile data network (e.g., a 3G, 4G, or LTE network), a cable network, a satellite network, a public switched telephone network, a local area network, a wide area network, a fiber-optic network, any other suitable type of communications network, and/or any suitable combination of communications networks.
In some embodiments, media content source 102, media encoder 104, media metadata source 106, storage device 110, and transcoder 112 can be implemented in any suitable hardware. For example, each of media content source 102, media encoder 104, media metadata source 106, storage 126, and transcoder 112 can be implemented in any of a general purpose device such as a computer or a special purpose device such as a client, a server, mobile terminal (e.g., mobile phone), etc. Any of these general or special purpose devices can include any suitable components such as a hardware processor (which can be a microprocessor, digital signal processor, a controller, etc.).
In some embodiments, each of media content source 102, media encoder 104, media metadata source 106, storage device 110, and transcoder 112 can be implemented as a stand-alone device or integrated with other components of architecture 100.
In some embodiments, media content source 102 can be connected to media metadata source 106 through communications path 114. In some embodiments, media encoder 104 can be connected to media content source 102 and media metadata source 106 through communications paths 116 and 118, respectively. In some embodiments, communications network 108 can be connected to media content source 102, media encoder 104, media metadata source 106, storage device, and transcoder 112 through communications paths 120, 122, 124, 126, and 128, respectively. In some embodiments, storage device 110 can be connected to transcoder 112 through communications path 130.
Communications paths 116, 118, 120, 122, 124, 126, 128, and 130 may separately or together include one or more communications paths, such as, a satellite path, a fiber-optic path, a cable path, a path that supports Internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths, in some embodiments.
Turning to FIG. 2, a block diagram of an example 200 of storage device 110 and transcoder 112 of FIG. 1 in accordance with some embodiments of the disclosure is shown.
As illustrated, transcoder 112 may include a decoding circuitry 202, an encoding circuitry 204, a video-data storage 206, and communication paths 208, 210, 212, and 214.
Decoding circuitry 202 can include any suitable circuitry that is capable of performing video decoding. For example, decoding circuitry 202 can include one or more decoders that can decode a set of encoded images based on a suitable coding standard, such as MPEG-2, MPEG-4, AVC, H.263, H.264, HEVC, etc.
Encoding circuitry 204 can include any suitable circuitry that is capable of performing video encoding. For example, encoding circuitry 204 can include one or more suitable encoders that can encode a set of images based on a suitable coding standard, such as MPEG-2, MPEG-4, AVC, H.263, H.264, HEVC, etc. In some embodiments, encoding circuitry 204 can also include scaler circuitry for upconverting and/or downconverting content into a preferred output format.
Decoding circuitry 202 can be connected to encoding circuitry 204 through communication path 210. Encoding circuitry 204 can be connected to video storage 206 through communication path 214. Transcoder 112 may be connected to media storage 110 through communication paths 208 and 212.
Each of decoding circuitry 202 and encoding circuitry 204 can include any suitable processing circuitry. As referred to herein, processing circuitry can be any suitable circuitry that includes one or more microprocessors, microcontrollers, digital signal processors, programmable logic devices, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), hardware processors, etc., and may include a multi-core processor (e.g., dual-core, quad-core, hexa-core, or any suitable number of cores) or a supercomputer, in some embodiments. In some embodiments, processing circuitry may be distributed across multiple separate processors or processing units, such as, for example, multiple of the same type of processing units (e.g., two Intel Core i7 processors) or multiple different processors (e.g., an Intel Core i5 processor and an Intel Core i7 processor).
Video data storage 206 can be any suitable digital storage mechanism in some embodiments. For example, video data storage 206 can include any device for storing electronic data, program instructions, computer software, firmware, register values, etc., such as random-access memory, read-only memory, hard drives, optical drives, digital video disc (DVD) recorders, compact disc (CD) recorders, BLU-RAY disc (BD) recorders, BLU-RAY 3D disc recorders, digital video recorders (DVR, sometimes called a personal video recorder, or PVR), solid state devices, quantum storage devices, gaming consoles, gaming media, or any other suitable fixed or removable storage devices, and/or any combination of the same. Video data storage 206 may be used to store media content, media guidance data, executable instructions (e.g., programs, software, scripts, etc.) for providing an interactive media guidance application, and for any other suitable functions, and/or any other suitable data or program code, in accordance with some embodiments. Nonvolatile memory may also be used (e.g., to launch a boot-up routine and other instructions), in some embodiments.
Each of storage device 110, decoding circuitry 202, encoding circuitry 204, and video-data storage 206 can be provided as a stand-alone device or integrated with other components of architecture 200.
In some embodiments, storage device 110 can be connected to decoding circuitry 202 and encoding circuitry 204 through path paths 208 and 210, respectively. In some embodiments, decoding circuitry 202 can be connected to encoding circuitry 204 through communications path 212. In some embodiments, encoding circuitry 204 can be connected to video-data storage 206 through communications path 214.
Communications paths 208, 210, 212, and 214 may separately or together include one or more communications paths, such as, a satellite path, a fiber-optic path, a cable path, a path that supports internet communications (e.g., IPTV), free-space connections (e.g., for broadcast or other wireless signals), or any other suitable wired or wireless communications path or combination of such paths, in some embodiments
In some embodiments, transcoder 112 can also include a demultiplexer circuitry (not shown in FIG. 2). The demultiplexer circuitry can be any suitable circuitry that is capable of demultiplexing a media content transport stream (TS). For example, the demultiplexer circuitry can receive a TS from storage 110 and demultiplex the TS into a video stream, an audio stream, program and system information protocol data stream, etc. The demultiplexer circuitry can also pass the video stream to decoding circuitry 202.
Turning to FIG. 3, a flow chart of an example 300 of a process for transcoding video data in accordance with some embodiments of the disclosure is shown. In some embodiments, process 300 can be implemented by transcoder 112 as illustrated in FIGS. 1 and 2.
As illustrated, process 300 can start by receiving a compressed bitstream at 302. The compressed bitstream can include any suitable data and can be received in any suitable manner. For example, the compressed bitstream can include video data generated based on any suitable coding standard, such as Motion JPEG, JPEG, MPEG-2, MPEG-4, H.263, H.264, HEVC, etc. More particularly, for example, the video data can include encoded images, decoding parameters, header information, etc. In some embodiments, each of the encoded images can include one or more quantized transform coefficients.
In some embodiments, for example, the compressed bitstream can be received from storage 110 as illustrated in FIGS. 1 and 2. Alternatively or additionally, the compressed bitstream can be received from media encoder 104 and/or media content source 102.
Next, at 304, transcoder 112 can decompress the compressed bitstream and generate decoded video data. The compressed bitstream can be decompressed and the decoded video data can be generated in any suitable manner. For example, transcoder 112 can decompress the compressed bitstream and generate multiple decoded images based on a suitable coding standard, such as Motion JPEG, JPEG 2000, MPEG-2, MPEG-4, H.263, H.264, HEVC, etc. In some embodiments, the decoded images can have any suitable color format, such as RGB, YCrCb, YUV, etc.
More particularly, for example, each of the decoded images can be generated using a process 400 as illustrated in FIG. 4. In some embodiments, for example, process 400 can be implemented by decoding circuitry 202 of transcoder 112 (FIG. 2).
As shown, at 402, decoding circuitry 202 can perform entropy decoding on the compressed bitstream and extract the quantized transform coefficients associated with each of the encoded images, decoding parameters (e.g., quantization parameters, coding modes, macroblock partition information, motion vectors, reference lists, etc.), header information, etc.
At 404, decoding circuitry 202 can perform inverse quantization on the quantized transformed coefficients associated with a current encoded image to generate one or more transform coefficients. The inverse quantization can be performed in any suitable manner. For example, decoding circuitry 202 can multiply each of the quantized transform coefficients by a suitable quantization parameter. In some embodiments, for example, decoding circuitry 202 can obtain the quantization parameter from the decoding parameters.
At 406, decoding circuitry 202 can perform an inverse transform on the transform coefficients to generate a decoded residual image for the current encoded image. The inverse transform can be performed in any suitable manner. For example, the inverse transform can be an inverse Discrete Cosine Transform (IDCT).
Next, at 408, decoding circuitry 202 can generate a prediction image for the current encoded image. The prediction image can be calculated in any suitable manner. For example, decoding circuitry 202 can generate the prediction image based on a suitable inter-prediction method by referring to one or more previously decoded frames. More particularly, for example, decoding circuitry 202 can perform motion compensation on one or more previously decoded frames and produce a motion compensated reference image as the prediction image. In a more particular example, decoding circuitry 202 can locate a previously decoded image or a portion of the previously decoded image as a reference image for the current encoded image using a motion vector. The reference image can then be used as the motion compensated prediction for the current image. In another more particular example, decoding circuitry 202 can locate two reference images for the current encoded image using one or more motion vectors. Decoding circuitry 202 can then calculate a prediction image for the current encoded image based on the reference images. More particularly, for example, the prediction image can be a weighted prediction of the two reference images.
As another example, decoding circuitry 202 can generate the prediction image based on a suitable intra-prediction method by referring to one or more previously decoded pixels in the same frame. More particularly, for example, decoding circuitry 202 can perform spatial extrapolation to produce an intra-prediction image for the current encoded image. In some embodiments, one or more prediction images can be formed by extrapolating previously decoded pixels of the current frame in any suitable direction, such as vertical, horizontal, diagonal down-left, diagonal down-right, vertical-left, horizontal-down, vertical right, horizontal-up, etc.
At 410, decoding circuitry 202 can generate a decoded image for the current encoded image based on the residual image and the prediction image. The decoded image can be generated in any suitable manner. For example, decoding circuitry 202 can add the prediction image to the decoded residual image to produce the decoded image.
Turning back to FIG. 3, at 306, transcoder 112 can receive media metadata. The media metadata can include any suitable data and can be received in any suitable manner. For example, the media metadata can be the metadata produced by media metadata source 106, as described above in connection with FIG. 1. More particularly, for example, the media metadata can include information about video scenes (e.g., scene change information, the number of the frames between scene changes, the type of a scene change, the number of B-frames between two P-frames, picture complexity information, etc.), motion data about the media content (e.g., motion vector maps, reference lists, etc.), intra-prediction data (e.g., a set of candidate intra-prediction modes, the coding cost and/or distortion corresponding to each candidate intra-prediction mode, etc.), etc.
In some embodiments, for example, encoding circuitry 204 (FIG. 2) can receive the media metadata from storage 110. In some embodiments, encoding circuitry 204 can receive the media metadata from media metadata source 106 through communications network 108 as illustrated in FIG. 1.
At 308, transcoder 112 can encode the decoded video data using the media metadata based on a second coding scheme. The decoded video data can be encoded in any suitable manner. For example, transcoder 112 can encode the decoded images into a set of encoded images based on any suitable coding standard, such as MPEG-2, MPEG-4, H.263, H.264, HEVC, etc. As another example, transcoder 112 can encode the decoded video data into a compressed bitstream including a set of encoded images that has a given bitrate. As yet another example, encoding circuitry 204 can encode the decoded images into a set of encoded images that has a given resolution, such as a spatial resolution, a temporal resolution, a quality resolution, etc.
More particularly, for example, transcoder 112 can generate each of the encoded images using a process 500 as illustrated in FIG. 5. In some embodiments, process 500 can be implemented by encoding circuitry 204 of transcoder 112.
At 502, encoding circuitry 204 can receive the set of decoded images and the media metadata. The set of decoded images and the media metadata can be received in any suitable manner. For example, encoding circuitry 204 can receive the set of decoded images from the decoding circuitry 202 and receive the media metadata from storage device 110.
At 504, encoding circuitry 204 can divide a decoded image into one or more suitable coding units based on the second coding scheme. Each of the coding units can have any suitable size and shape and can be obtained in any suitable manner. In some embodiments, for example, the second coding scheme can include the HEVC coding standard. Encoding circuitry 204 can divide a video frame into multiple coding tree units (CTU), each of which can have a size of 8×8, 16×16, 32×32, 64×64, etc. In some embodiments, each of the CTUs can be partitioned into multiple coding tree blocks (CTBs), each of which can have a size of 4×4, 8×8, 16×16, etc. based on the size of the CTU. In some embodiments, each of the CTBs can be further partitioned into multiple coding blocks (CBs) and coding units (CUs).
At 506, encoding circuitry 204 can generate a prediction image for a coding unit. The prediction image can be generated in any suitable way. For example, encoding circuitry 204 can generate the prediction image based on the media metadata such as scene change information, motion data, picture complexity information, intra-prediction information, etc.
In some embodiments, for example, encoding circuitry 204 can generate the prediction image based on a suitable inter-prediction method by referring to one or more reference images. More particularly, for example, encoding circuitry 204 can calculate one or more suitable motion vectors for the coding unit based on the motion vector map corresponding to the coding unit. Encoding circuitry 204 can then generate a motion compensated prediction image for the coding unit based on the motion vectors by referring to one or more reference images. In some embodiments, the motion compensated prediction image can be generated based on one reference frame that can be located using the reference frame lists. For example, encoding circuitry 204 can locate a region in the reference frame as a reference image for the coding unit based on a motion vector. The reference image can then be used as a prediction image for the coding unit. In some embodiments, the motion compensated prediction image can be generated based on two reference frames that can be located using the reference frame lists. For example, encoding circuitry 204 can generate two reference images by locating a region in each of the two reference frames, respectively, based on one or more motion vectors. Encoding circuitry 204 can then produce a prediction for the coding unit using the two reference images. More particularly, for example, the prediction for the coding unit can be a weighted prediction of the two reference images.
In some embodiments, encoding circuitry 204 can generate the predicted image based on a suitable intra-prediction method. The intra-prediction can be performed in any suitable manner. For example, encoding circuitry 204 can generate an intra-prediction image for the coding unit based on the media metadata, such as the intra-prediction data including the set of candidate intra-prediction modes, the coding cost and/or distortion corresponding to each intra-prediction mode, etc. More particularly, for example, encoding circuitry 204 can determine a sub-set of the candidate intra-prediction modes that can be used in accordance with the second coding scheme. Additionally, encoding circuitry 204 can select an intra-prediction mode from the sub-set of candidate intra-prediction modes based on the coding costs and/or distortion corresponding to each of the sub-set of candidate intra-prediction modes. Encoding circuitry 204 can then generate a prediction image for the coding unit based on the selected intra-prediction mode. More particularly, for example, encoding circuitry 204 can predict each pixel of the coding unit by extrapolating pixel samples in a direction defined by the intra-prediction mode.
At 508, encoding circuitry 204 can generate a residual image for the coding unit. The residual image can be generated in any suitable manner. For example, the residual image can be generated at 506 by subtracting the prediction image generated at from the original image of the coding unit.
At 510, encoding circuitry 204 can perform a transform on the residual image and generate a set of transform coefficients. The set of transform coefficients can be generated in any suitable manner. For example, encoding circuitry 204 can perform a Discrete Cosine Transform (DCT) on the residual image and generate a set of DCT coefficients.
At 512, encoding circuitry 204 can perform quantization on the set of transform coefficients. The quantization can be performed in any suitable manner. For example, encoding circuitry 204 can determine a suitable quantization parameter (QP) for a coding unit based on a target bitrate of the second coding scheme. Encoding circuitry 204 can then quantize the transform coefficients using the QP. The target bitrate can be any suitable bitrate, such as a constant bitrate, a variable bitrate, etc. A QP can be determined in any suitable manner. In some embodiments, for example, encoding circuitry 204 can reduce the bitrate of a compressed bitstream by increasing QP or increase the bitrate of a compressed bitstream by decreasing QP. In some embodiments, for example, an I-frame can be encoded using most bits, followed by a P-frame and a B-frame.
In some embodiments, encoding circuitry 204 can determine a QP based on the media metadata (e.g., scene change information, the number of frames between two scenes, the type of each scene change, picture complexity information, etc.), the target bitrate in accordance with the second coding scheme, etc.
For example, encoding circuitry 204 can determine a QP for a group of pictures (GOP) based on the media metadata. The QP can be determined for the GOP in any suitable manner. More particularly, for example, encoding circuitry 204 can determine the structure of a GOP (e.g., the length of the GOP, the distance between P-frames, the distance between I-frames, etc.) based on the media metadata and determine the QP for the GOP based on the structure of the GOP.
In some embodiments, encoding circuitry 204 can calculate the number of bits available to encode the GOP based on the structure of a GOP, the frame rate of the video data, the target rate, etc. Encoding circuitry 204 can then calculate a QP for the GOP based on the number of bits available to encode the GOP. More particularly, for example, the QP can be calculated based on a suitable model that can define the relation between the QP and the target rate, such as a rate-distortion model, a rate-distortion optimization model, etc.
In some embodiments, encoding circuitry 204 can determine the structure of GOP based on the media metadata, such as scene information, the number of frames between two scene changes, the number of B-frames between two P-frames, etc.
In a more particular example, the first frame of the GOP can be an I-frame that can be located using the scene change information. More particularly, for example, the first frame of the GOP can correspond to the start of a video scene.
In another more particular example, the length of the GOP, i.e., the number of frames in the GOP, can be determined based on the number of frames between two scene changes. In some embodiments, the length of the GOP can be equal to the number of frames between two adjacent scene changes. In some embodiments, the length of the GOP can be equal to the number of frames between two given scene changes, e.g., two shot changes, etc.
In yet another more particular example, the distance between P-frames in the GOP can be determined based on the number of B-frames between two P-frames included in the media metadata. In a more particular example, the GOP can include a set of frames IBBPBBP . . . where the distance between P-frames is three.
As another example, encoding circuitry 204 can determine a QP for the coding unit based on the media metadata. More particularly, for example, encoding circuitry 204 can determine the complexity of the coding unit using the picture complexity information (e.g., the maps of spatial complexity, the maps of motion complexity, etc.). Encoding circuitry 204 can then calculate a target number of bits that are available to encode the coding unit based on the complexity of the coding unit. In some embodiments, for example, more bits can be allocated to a coding unit having relatively high complexity while fewer bits can be allocated to a coding unit having relatively lower complexity.
Additionally, encoding circuitry 204 can determine a QP for the coding unit to produce the target number of bits. More particularly, for example, the QP can be calculated based on a suitable model that can define the relation between the QP and the target rate, such as a rate-distortion model, a rate-distortion optimization model, etc.
Next, at 514, encoding circuitry 204 can perform entropy encoding on the quantized transform coefficients. The entropy encoding can be performed in any suitable manner. For example, encoding circuitry 204 can perform the entropy encoding using a suitable variable length encoding method.
It should be noted that the above steps of the flow diagrams of FIGS. 3-5 may be executed or performed in any order or sequence not limited to the order and sequence shown and described in the figures. Furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3-5 may be executed or performed substantially simultaneously where appropriate or in parallel to reduce latency and processing times. And still furthermore, it should be noted, some of the above steps of the flow diagrams of FIGS. 3-5 may be omitted.
1. A method for transcoding video data using a transcoding device, the method comprising:
receiving a portion of encoded video that includes a first plurality of encoded images encoded according to a first encoding scheme from a media content source using a transcoding device;
decoding the portion of encoded video based on the first encoding scheme to generate a decoded portion of video including a plurality of decoded images using the transcoding device;
receiving media metadata at the transcoding device over a communications network from a remote media metadata source, the media metadata being generated and stored on the remote media metadata source prior to decoding the portion of encoded video, wherein the media metadata comprises:
a set of transcoding parameters that define a second encoding scheme, wherein the first encoding scheme and the second encoding scheme have different bit rates; and
at least two types of data from the group consisting of motion data, picture complexity information, a number of frames between two scene changes, a set of motion vectors, and a set of intra-prediction modes for use in encoding the decoded portion of video according to a second encoding scheme using the transcoding device; and
encoding the plurality of decoded images of the decoded portion of video into a second plurality of encoded images based on the second encoding scheme defined by the received media metadata using the transcoding device by performing at least the following operations for each particular decoded image in the plurality of decoded images:
generating a prediction image for each of a plurality of coding units of the particular decoded image using at least one of the motion data, the set of motion vectors, or a set of intra-prediction modes within the received media metadata according to the second encoding scheme;
performing transforms on residual images of the plurality of coding units to generate sets of transform coefficients based on the second encoding scheme;
performing quantization on the sets of transform coefficients by:
determining a number of bits to encode a group of pictures (GOP) based at least in part on at least one of the picture complexity information or the number of frames between two scene changes;
determining a quantization parameter based on the number of bits; and
quantizing the generated set of transform coefficients according to the second encoding scheme using the quantization parameter; and
performing entropy encoding on the sets of quantized transform coefficients to generate images for the second plurality of encoded images.
2. The method of claim 1, wherein the media metadata source is not an encoder, and wherein the set of transcoding parameters within the received metadata include at least one of a number of B-frames between two P-frames, and a scene change detection signal.
3. The method of claim 2, further comprising determining the number of bits based at least in part on the number of B-frames between two P-frames.
4. The method of claim 3, further comprising determining the number of bits based at least in part on the scene detection signal indicated by the received media metadata.
5. The method of claim 1, wherein the first encoding scheme and the second encoding scheme have different resolutions.
6. The method of claim 1 further comprising dividing each particular decoded image in the plurality of decoded images into a plurality of coding units based on the second encoding scheme.
7. A system for transcoding video data, the system comprising:
a non-transitory memory storing a transcoding application;
wherein the transcoding application directs the processing circuitry to:
receive a portion of encoded video that includes a first plurality of encoded images encoded according to a first encoding scheme from a media content source;
decode the portion of encoded video based on the first encoding scheme to generate a decoded portion of video including a plurality of decoded images;
receive media metadata over a communications network from a remote media metadata source, the media metadata being generated and stored on the remote media metadata source prior to decoding the portion of encoded video, wherein the media metadata comprises:
at least two types of data from the group consisting of motion data, picture complexity information, a number of frames between two scene changes, a set of motion vectors, and a set of intra-prediction modes for use in encoding the decoded portion of video according to a second encoding scheme; and
encode the plurality of decoded images of the decoded portion of video into a second plurality of encoded images based on the second encoding scheme defined by the received media metadata by performing at least the following operations for each particular decoded image in the plurality of decoded images:
8. The system of claim 7, wherein the media metadata source is not an encoder, and wherein the set of transcoding parameters within the received metadata include at least one of a number of B-frames between two P-frames, and a scene change detection signal.
9. The system of claim 8, wherein the processing circuitry is further configured to determine the number of bits based at least in part on the number of B-frames between two P-frames.
10. The system of claim 9, wherein the processing circuitry is further configured to determine the number of bits based at least in part on the scene detection signal indicated by the received media metadata.
11. The system of claim 7 further comprising dividing each particular decoded image in the plurality of decoded images into a plurality of coding units based on the second encoding scheme.
12. The system of claim 7, wherein the first encoding scheme and the second encoding scheme correspond to different video encoding standards.
13. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processing circuitry, cause the processing circuitry to perform a method for transcoding video data, the method comprising:
receiving a portion of encoded video that includes a first plurality of encoded images encoded according to a first encoding scheme from a media content source;
decoding the portion of encoded video based on the first encoding scheme to generate a decoded portion of video including a plurality of decoded images;
receiving media metadata over a communications network from a remote media metadata source, the media metadata being generated and stored on the remote media metadata source prior to decoding the portion of encoded video, wherein the media metadata comprises:
encoding the plurality of decoded images of the decoded portion of video into a second plurality of encoded images based on the second encoding scheme defined by the received media metadata by performing at least the following operations for each particular decoded image in the plurality of decoded images:
14. The non-transitory computer-readable medium of claim 13, wherein the media metadata source is not an encoder, and wherein the set of transcoding parameters within the received metadata include at least one of a number of B-frames between two P-frames, and a scene change detection signal.
15. The non-transitory computer-readable medium of claim 14, wherein the method further comprises determining the number of bits based at least in part on the number of B-frames between two P-frames.
16. The non-transitory computer-readable medium of claim 15, wherein the method further comprises determining the number of bits based at least in part on the scene detection signal indicated by the received media metadata.
17. The non-transitory computer-readable medium of claim 13 further comprising dividing each particular decoded image in the plurality of decoded images into a plurality of coding units based on the second encoding scheme.
18. The non-transitory computer-readable medium of claim 13, wherein the first encoding scheme and the second encoding scheme have different resolutions.
19. The non-transitory computer-readable medium of claim 13, wherein the first encoding scheme and the second encoding scheme correspond to different video coding standards.
US13841943 2013-03-15 2013-03-15 Systems, methods, and media for transcoding video data according to encoding parameters indicated by received metadata Active 2033-10-26 US9906785B2 (en)
US13841943 US9906785B2 (en) 2013-03-15 2013-03-15 Systems, methods, and media for transcoding video data according to encoding parameters indicated by received metadata
US15905695 US20180262757A1 (en) 2013-03-15 2018-02-26 Systems, Methods, and Media for Transcoding Video Data According to Encoding Parameters Indicated by Received Metadata
US15905695 Continuation US20180262757A1 (en) 2013-03-15 2018-02-26 Systems, Methods, and Media for Transcoding Video Data According to Encoding Parameters Indicated by Received Metadata
US20140269927A1 true US20140269927A1 (en) 2014-09-18
US9906785B2 true US9906785B2 (en) 2018-02-27
ID=51526955
US13841943 Active 2033-10-26 US9906785B2 (en) 2013-03-15 2013-03-15 Systems, methods, and media for transcoding video data according to encoding parameters indicated by received metadata
US15905695 Pending US20180262757A1 (en) 2013-03-15 2018-02-26 Systems, Methods, and Media for Transcoding Video Data According to Encoding Parameters Indicated by Received Metadata
US (2) US9906785B2 (en)
US20170236546A1 (en) * 2016-02-16 2017-08-17 Arris Enterprises Llc Efficient just-in-time transcoding by parameter extraction
JPH0846902A (en) 1994-07-29 1996-02-16 Nippon Telegr & Teleph Corp <Ntt> Picture storage communication device
WO1997031445A2 (en) 1996-02-09 1997-08-28 Geo Interactive Media Group, Ltd. Method and system of building and transmitting a data file for real time play of multimedia
JPH11164307A (en) 1997-11-28 1999-06-18 Victor Co Of Japan Ltd Moving image coder, moving image decoder, moving image coding method and moving image decoding method
JP2001346165A (en) 1999-12-15 2001-12-14 Sanyo Electric Co Ltd Image processing method and image processing unit utilizing this method and television receiver
US20100057928A1 (en) 2008-08-29 2010-03-04 Adobe Systems Incorporated Dynamically Altering Playlists
WO2010089962A1 (en) 2009-02-06 2010-08-12 日本電気株式会社 Stream signal transmission device and transmission method
WO2011042898A1 (en) 2009-10-05 2011-04-14 I.C.V.T Ltd. Apparatus and methods for recompression of digital images
US20110116772A1 (en) 2009-11-13 2011-05-19 Samsung Electronics Co., Ltd. Method and apparatus for providing trick play service
WO2011135558A1 (en) 2010-04-29 2011-11-03 I.C.V.T. Ltd Apparatus and methods for re-compression having a monotonic relationship between extent of comprission and quality of compressed image
US20120005368A1 (en) 2010-06-30 2012-01-05 Cable Television Laboratories, Inc. Adaptive bit rate method and system using retransmission and replacement
US20120036544A1 (en) 2010-08-05 2012-02-09 Qualcomm Incorporated Signaling Attributes for Network-Streamed Video Data
US20120144445A1 (en) 2010-12-03 2012-06-07 General Instrument Corporation Method and apparatus for distributing video
US20120263434A1 (en) 2011-04-14 2012-10-18 Cisco Technology, Inc. Per-Subscriber Adaptive Bit Rate Stream Management Method
WO2012171113A1 (en) 2011-06-14 2012-12-20 Zhou Wang Method and system for structural similarity based rate-distortion optimization for perceptual video coding
US20130007223A1 (en) 2006-06-09 2013-01-03 Qualcomm Incorporated Enhanced block-request streaming system for handling low-latency streaming
US20130094565A1 (en) 2011-10-17 2013-04-18 Google Inc. Rate-distortion-complexity optimization of video encoding guided by video description length
US20130182952A1 (en) 2010-09-17 2013-07-18 I.C.V.T Ltd. Method of classifying a chroma downsampling error
US20130223812A1 (en) 2012-02-26 2013-08-29 Antonio Rossi Streaming video navigation systems and methods
US20140143440A1 (en) 2012-11-20 2014-05-22 General Instrument Corporation Method and apparatus for streaming media content to client devices
WO2015031982A1 (en) 2013-09-06 2015-03-12 Wang, Zhou Method and system for objective perceptual video quality assessment
US20110291723A1 (en) 2009-02-06 2011-12-01 Kiyoshi Hashimoto Stream signal transmission device and transmission method
US20120201476A1 (en) 2009-10-05 2012-08-09 I.C.V.T. Ltd. Apparatus and methods for recompression of digital images
US20150049957A1 (en) 2009-10-05 2015-02-19 I.C.V.T. Ltd. Apparatus and methods for recompression of digital images
EP2486517A1 (en) 2009-10-05 2012-08-15 I.C.V.T Ltd Apparatus and methods for recompression of digital images
EP2486727A1 (en) 2009-10-05 2012-08-15 I.C.V.T Ltd A method and system for processing an image
US20110222786A1 (en) 2009-10-05 2011-09-15 I.C.V.T Ltd. Classifying an image's compression level
EP2564354A1 (en) 2010-04-29 2013-03-06 I.C.V.T Ltd Apparatus and methods for re-compression having a monotonic relationship between extent of comprission and quality of compressed image
US20130064466A1 (en) 2010-04-29 2013-03-14 I.C.V.T. Ltd Apparatus and methods for re-compression having a monotonic relationship between extent of compression and quality of compressed image
WO2012035534A2 (en) 2010-09-17 2012-03-22 I.C.V.T Ltd. Downsizing an encoded image
US20130170764A1 (en) 2010-09-17 2013-07-04 I.C.V.T Ltd. Downsizing an encoded image
EP2616991A2 (en) 2010-09-17 2013-07-24 I.C.V.T Ltd Downsizing an encoded image
EP2617192A2 (en) 2010-09-17 2013-07-24 I.C.V.T Ltd A method of classifying a chroma downsampling error
EP2716048A1 (en) 2011-06-01 2014-04-09 Zhou Wang Method and system for structural similarity based perceptual video coding
US20140140396A1 (en) 2011-06-01 2014-05-22 Zhou Wang Method and system for structural similarity based perceptual video coding
US20140119432A1 (en) 2011-06-14 2014-05-01 Zhou Wang Method and system for structural similarity based rate-distortion optimization for perceptual video coding
EP2721826A1 (en) 2011-06-14 2014-04-23 Zhou Wang Method and system for structural similarity based rate-distortion optimization for perceptual video coding
EP2751990A1 (en) 2011-08-29 2014-07-09 I.C.V.T Ltd Controlling a video content system
US20140211859A1 (en) 2011-08-29 2014-07-31 I.C.V.T. Ltd. Controlling a video content system
US20140177734A1 (en) 2011-08-29 2014-06-26 I.C.V.T Ltd. Controlling a video content system
US20140355668A1 (en) 2012-01-23 2014-12-04 I.C.V.T. Ltd. Method and System for Controlling Video Frame Encoding
EP2807821A2 (en) 2012-01-23 2014-12-03 I.C.V.T Ltd Method and system for controlling video frame encoding
US20150063693A1 (en) 2012-03-28 2015-03-05 I.C.V.T. Ltd. Controlling a compression of an image according to a degree of photo-realism
US20150334435A1 (en) 2013-05-30 2015-11-19 Sonic Ip, Inc. Network Video Streaming With Trick Play Based on Separate Trick Play Files
"Best Practices for Multi-Device Transcoding", Kaltura Open Source Video, Printed on Nov. 27, 2013 from knowledge.kaltura.com/best-practices-multi-device-transcoding, 13 pgs.
"IBM Closes Cryptolopes Unit," Dec. 17, 1997, CNET News, Printed on Apr. 25, 2014 from http://news.cnet.com/IBM-closes-Cryptolopes-unit/2100-1001-3206465.html, 3 pages.
"IBM Spearheading Intellectual Property Protection Technology for Information on the Internet; Cryptolope Containers Have Arrived", May 1, 1996, Business Wire, Printed on Aug. 1, 2014 from http://www.thefreelibrary.com/IBM+Spearheading+Intellectual+Property+Protection+Technology+for...-a018239381, 6pg.
"Information Technology-Coding of Audio Visual Objects-Part 2: Visual" International Standard, ISO/IEC 14496-2, Third Edition, Jun. 1, 2004, pp. 1-724.
"Netflix turns on subtitles for PC, Mac streaming", Printed on Mar. 26, 2014, 3 pgs.
"Supported Media Formats", Supported Media Formats, Android Developers, Printed on Nov. 27, 2013 from.developer.android.com/guide/appendix/media-formats.html, 3 pgs.
"Thread: SSME (Smooth Streaming Medial Element) config.xml review (Smooth Streaming Client configuration file)", printed Mar. 26, 2014 from, 3 pgs.
"Transcoding Best Practices", From movideo, Printed on Nov. 27, 2013 from code.movideo.com/Transcoding-Best-Practices, 5 pgs.
"Using HTTP Live Streaming", iOS Developer Library, Retrieved from: http://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html#//apple-ref/doc/uid/TP40008332-CH102-SW1, 10 pgs. (2014).
"IBM Closes Cryptolopes Unit," Dec. 17, 1997, CNET News, Printed on Apr. 25, 2014 from http://news.cnet.com/IBM-closes-Cryptolopes-unit/2100-1001—3206465.html, 3 pages.
"Information Technology-Coding of Audio Visual Objects—Part 2: Visual" International Standard, ISO/IEC 14496-2, Third Edition, Jun. 1, 2004, pp. 1-724.
"Transcoding Best Practices", From movideo, Printed on Nov. 27, 2013 from code.movideo.com/Transcoding—Best—Practices, 5 pgs.
"Using HTTP Live Streaming", iOS Developer Library, Retrieved from: http://developer.apple.com/library/ios/#documentation/networkinginternet/conceptual/streamingmediaguide/UsingHTTPLiveStreaming/UsingHTTPLiveStreaming.html#//apple—ref/doc/uid/TP40008332-CH102-SW1, 10 pgs. (2014).
Akhshabi et al., "An Experimental Evaluation of Rate-Adaptation Algorithms in Adaptive Streaming over HTTP", MMSys'11, Feb. 23-25, 2011, 12 pgs.
Author Unknown, "Blu-ray Disc-Blu-ray Disc-Wikipedia, the free encyclopedia", printed Oct. 30, 2008 from http://en.wikipedia.org/wiki/Blu-ray-Disc, 11 pgs.
Author Unknown, "MPEG-4 Video Encoder: Based on International Standard ISO/IEC 14496-2", Patni Computer Systems, Ltd., Publication date unknown, 15 pgs.
Author Unknown, "Blu-ray Disc—Blu-ray Disc—Wikipedia, the free encyclopedia", printed Oct. 30, 2008 from http://en.wikipedia.org/wiki/Blu-ray—Disc, 11 pgs.
Blasiak, Ph.D., Darek "Video Transrating and Transcoding: Overview of Video Transrating and Transcoding Technologies,", Ingenient Technologies, TI Developer Conference, Aug. 6-8, 2002, 22 pgs.
Deutscher, "IIS Transform Manager Beta-Using the MP4 to Smooth Task", Retrieved from: https://web.archive.org/web/20130328111303/http://blog.johndeutscher.com/category/smooth-streaming, Blog post of Apr. 29, 2011, 14 pgs.
Deutscher, "IIS Transform Manager Beta—Using the MP4 to Smooth Task", Retrieved from: https://web.archive.org/web/20130328111303/http://blog.johndeutscher.com/category/smooth-streaming, Blog post of Apr. 29, 2011, 14 pgs.
European Search Report Application No. EP 08870152, Search Completed May 19, 2011, dated May 26, 2011, 10 pgs.
European Search Report for Application 11855103.5, search completed Jun. 26, 2014, 10 pgs.
Informationweek: Front End: Daily Dose, "Internet on Wheels", Jul. 20. 1999, Printed on Mar. 26, 2014, 3 pgs.
Inlet Technologies, "HTTP versus RTMP", 3 page.
International Preliminary Report on Patentability for International Application No. PCT/US2011/068276, International Filing Date Dec. 31, 2011, dated Mar. 4, 2014, 23 pgs.
International Preliminary Report on Patentability for International Application PCT/US2013/043181, Report issued Dec. 31, 2014, dated Jan. 8, 2015, 11 Pgs.
International Search Report and Written Opinion for International Application No. PCT/US2010/56733, Completed Jan. 3, 2011, dated Jan. 14, 2011, 9 pgs.
International Search Report and Written Opinion for International Application No. PCT/US2013/043181, International Filing Date May 29, 2013, Search Completed Nov. 27, 2013, dated Dec. 6, 2013, 12 pgs.
International Search Report and Written Opinion for International Application PCT/US2011/066927, International Filing Date Dec. 22, 2011, Report Completed Apr. 3, 2012, dated Apr. 20, 2012, 14 pgs.
International Search Report and Written Opinion for International Application PCT/US2011/067167, International Filing Date Dec. 23, 2011, Report Completed Jun. 19, 2012, dated Jul. 2, 2012, 11 pgs.
International Search Report and Written Opinion for International Application PCT/US2011/068276, International Filing Date Dec. 31, 2011, Report completed Jun. 19, 2013, dated Jul. 8, 2013, 25 pgs.
International Search Report for International Application No. PCT/US07/63950, completed Feb. 19, 2008; dated Mar. 19, 2008, 3 pgs.
International Search Report for International Application No. PCT/US08/87999, completed Feb. 7, 2009, dated Mar. 19, 2009, 2 pgs.
International Search Report for International Application No. PCT/US2005/025845 filed Jul. 21, 2005, report completed Feb. 5, 2007 and dated May 10, 2007, 3 pgs.
Nelson, Mark "Arithmetic Coding+ Statistical Modeling= Data Compression: Part 1-Arithmetic Coding", Doctor Dobb's Journal, Feb. 1991, printed from http://www.dogma.net/markn/articles/arith/part1.htm; printed Jul. 2, 2003, 12 pgs.
Nelson, Mark "Arithmetic Coding+ Statistical Modeling= Data Compression: Part 1—Arithmetic Coding", Doctor Dobb's Journal, Feb. 1991, printed from http://www.dogma.net/markn/articles/arith/part1.htm; printed Jul. 2, 2003, 12 pgs.
Nelson, Michael, "IBM's Cryptolopes", Complex Objects in Digital Libraries Course, Spring 2001, Retrieved from http://www.cs.odu.edu/˜mln/teaching/unc/inls210/?method=display&pkg-name=cryptolopes.pkg&element-name=cryptolopes.ppt, 12 pages.
Nelson, Michael, "IBM's Cryptolopes", Complex Objects in Digital Libraries Course, Spring 2001, Retrieved from http://www.cs.odu.edu/˜mln/teaching/unc/inls210/?method=display&pkg—name=cryptolopes.pkg&element—name=cryptolopes.ppt, 12 pages.
Noe, A.,"Matroska File Format (under construction!)", Retrieved from the Internet: URL:http://web.archive.orgweb/20070821155146/www.matroska.org/technical/specs/matroska.pdf [retrieved on Jan. 19, 2011], Jun. 24, 2007, 1-51.
Office Action for U.S. Appl. No. 13/223,210, dated Apr. 30, 2015, 14 pgs.
Office Action for U.S. Appl. No. 14/564,003, dated Apr. 17, 2015, 28 pgs.
Schulzrinne, H et al., "Real Time Streaming Protocol 2.0 (RTSP): draft-ietfmmusic-rfc2326bis-27", Music Working Group of the Internet Engineering Task Force (IETF), Mar. 9, 2011, 296 pages.
Tan, Yap-Peng et al., "Video transcoding for fast forward/reverse video playback", IEEE ICIP, 2002, pp. 1-713 to 1-716.
The Official Microsoft IIS Site, "Smooth Streaming Client," 4 pages.
U.S. Appl. No. 13/224,298, "Final Office Action Received", dated May 19, 2014, 27 pgs.
U.S. Appl. No. 13/905,804, "Non-Final Office Action Received", dated Jul. 25, 2014, 15 pgs.
U.S. Appl. No. 13/905,804, "Notice of Allowance", dated Aug. 12, 2015, 8 pgs.
Unknown, "Entropy and Source Coding (Compression)", TCOM 570, 1999-9, pp. 1-22.
Written Opinion for International Application No. PCT/US2007/063950 filed Mar. 14, 2007, report completed Mar. 1, 2008; report dated Mar. 19, 2008, 6 pgs.
Written Opinion for International Application No. PT/US2005/025845 filed Jul. 21, 2005, report completed Feb. 5, 2007 and dated May 10, 2007, 5 pgs.
Written Opinion of the International Searching Authority for International Application No. PCT/US08/87999, date completed Feb. 7, 2009, dated Mar. 19, 2009, 4 pgs.
US20140269927A1 (en) 2014-09-18 application
US20180262757A1 (en) 2018-09-13 application
KR20060083053A (en) 2006-07-20 Method for setting reference frame order for multi-view coding and computer readable medium storing thereof
US20130182769A1 (en) 2013-07-18 Efficient storage of motion information for high efficiency video coding
US20070009030A1 (en) 2007-01-11 Method and apparatus for video encoding and decoding, and recording medium having recorded thereon a program for implementing the method
US20140140406A1 (en) 2014-05-22 Devices and methods for processing of non-idr related syntax for high efficiency video coding (hevc)
US20120063515A1 (en) 2012-03-15 Efficient Coding of Video Parameters for Weighted Motion Compensated Prediction in Video Coding
US20120224643A1 (en) 2012-09-06 Video decoder with pipeline processing and methods for use therewith
US20120269265A1 (en) 2012-10-25 Method and arrangement for video coding
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:NALETOV, IVAN VLADIMIROVICH;ZURPAL, SERGEY;SIGNING DATES FROM 20140306 TO 20140313;REEL/FRAME:032490/0798