Prediction mode-based block ordering in video coding

Video data streams can be encoded and decoded using inter or intra prediction. The blocks of a frame can be divided into groups of blocks to be inter predicted and blocks to be intra predicted, and the blocks to be inter predicted are encoded first. The availability of data from the inter predicted blocks can improve the performance of intra prediction over processing the blocks in the scan order since more pixel data is available for intra prediction of some blocks. For example, when the scan order is raster scan order, intra prediction of a block can use pixels peripheral to the bottom and right sides of the block in addition to the pixels peripheral to the top and left sides of the block.

TECHNICAL FIELD

This disclosure relates to encoding and decoding visual data, such as video stream data, for transmission or storage and subsequent display, with particular reference to multi-user video conferencing.

BACKGROUND

Digital video streams typically represent video using a sequence of frames or still images. Each frame can include a number of blocks, which in turn may contain information describing the value of color, brightness or other attributes for pixels. The amount of data in a typical video stream is large, and transmission and storage of video can use significant computing or communications resources. Various approaches have been proposed to reduce the amount of data in video streams, including compression and other encoding techniques.

SUMMARY

Disclosed herein are aspects of systems, methods and apparatuses for encoding a video stream. One implementation of a method for encoding a video stream includes identifying, in a frame of the video stream, a first group of blocks to be encoded using inter prediction, and identifying, in the frame, a second group of blocks to be encoded using intra prediction, the second group of blocks including at least one block that is located in the frame at a position that precedes, in a scan order of the frame, at least one block of the first group of blocks. The method also includes at least partially encoding, using inter prediction, the first group of blocks to form a first group of encoded blocks, at least partially decoding the first group of encoded blocks to form a first group of decoded blocks, encoding, using intra prediction, the second group of blocks using at least one block of the first group of decoded blocks, and inserting the first group of encoded blocks and the second group of encoded blocks into an encoded bitstream.

A method for decoding a video bitstream according to the teachings herein includes identifying, in a frame in the video stream, a first group of encoded blocks that were encoded using inter prediction, and identifying, in the frame, a second group of encoded blocks that were encoded using intra prediction, the second group of encoded blocks including at least one block that is located in the frame at a position that precedes, in a scan order, at least one block of the first group of encoded blocks. The method includes decoding, using inter prediction, the first group of encoded blocks to form a first group of decoded blocks, and decoding, using intra prediction, the second group of encoded blocks using at least one block of the first group of decoded blocks.

Another implementation of the teachings herein is an apparatus for encoding a video stream, including a memory and a processor. The processor is configured to execute instructions stored in memory to identify, in a frame of the video stream, a first group of blocks to be encoded using inter prediction, and identify, in the frame, a second group of blocks to be encoded using intra prediction, the second group of blocks including at least one block that is located in the frame at a position that precedes, in a scan order of the frame, at least one block of the first group of blocks. The processor is also configured to at least partially encode, using inter prediction, the first group of blocks to form a first group of encoded blocks, at least partially decode the first group of encoded blocks to form a first group of decoded blocks, encode, using intra prediction, the second group of blocks using at least one block of the first group of decoded blocks, and insert the first group of encoded blocks and the second group of encoded blocks into an encoded bitstream.

Variations in these and other aspects and implementations will be described in additional detail hereafter.

DETAILED DESCRIPTION

Digital video is used for various purposes including, for example, remote business meetings via video conferencing, high definition video entertainment, video advertisements, and sharing of user-generated videos. Image and video compression is can improve the efficiency of data transmission and storage of digital video. Compression techniques can be used to reduce the amount of information to be transmitted or stored. Internet based multimedia services such as streaming video web sites can rely on good compression technology to improve the quality of service and control the cost of bandwidth and content delivering at the same time.

In video compression, a block-based encoder-decoder system (codec) can first divide an image frame into blocks. The encoder can scan (e.g., in raster scan order) the blocks in the frame and pick the best prediction mode for each block based on previously-processed block. The encoder can subtract the predicted block from the block and encode the prediction residual. Aspects of this disclosure describe a new coding scheme that performs an extra pass through the blocks before prediction coding so as to re-order the encoding of blocks based on the prediction modes used. In the re-ordering, the blocks within an image frame using inter prediction modes are at least partially encoded and decoded first, and then the blocks using intra prediction modes are encoded. By such re-ordering, the encoder can have more information available from surrounding blocks to improve the quality of intra prediction, and can improve the overall coding efficiency. A decoder can perform the same re-ordering of blocks for decoding, relying on bits included in the encoded video bitstream to indicate which blocks can be decoded using inter prediction and which blocks can be decoded using intra prediction.

Grouping blocks in to two groups for encoding or decoding can permit the use of intra prediction modes where pixel data from more than two sides of a block can be used to form a prediction block. In some intra prediction modes, such as where blocks of a frame are processed in raster scan order, intra prediction modes are limited to modes using pixel data from blocks occurring before the block to be predicted in the raster scan order. Identifying blocks to be encoded or decoded using inter prediction and at least partially encoding these blocks first permits the use of pixel data from blocks on all four sides of a block to be used in prediction in some cases, thereby improving the performance of the encoding or decoding process.

First discussed below are environments in which aspects of this disclosure can be implemented.

FIG. 1is a schematic of a video encoding and decoding system100. An exemplary transmitting station112can be, for example, a computer having an internal configuration of hardware such as that described inFIG. 2. However, other suitable implementations of transmitting station112are possible. For example, the processing of transmitting station112can be distributed among multiple devices.

A network128can connect the transmitting station112and a receiving station130for encoding and decoding of a video stream. Specifically, the video stream can be encoded in transmitting station112and the encoded video stream can be decoded in receiving station130. Network128can be, for example, the Internet. Network128can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network or any other means of transferring the video stream from transmitting station112to, in this example, receiving station130.

Receiving station130, in one example, can be a computer having an internal configuration of hardware such as that described inFIG. 2. However, other suitable implementations of receiving station130are possible. For example, the processing of receiving station130can be distributed among multiple devices.

Other implementations of video encoding and decoding system100are possible. For example, an implementation can omit network128. In another implementation, a video stream can be encoded and then stored for transmission at a later time to receiving station130or any other device having memory. In one implementation, receiving station130receives (e.g., via network128, a computer bus, and/or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an exemplary implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over network128. In another implementation, a transport protocol other than RTP may be used, e.g., an HTTP-based video streaming protocol.

FIG. 2is a block diagram of an exemplary computing device200that can implement a transmitting station or a receiving station. For example, computing device200can implement one or both of transmitting station112and receiving station130ofFIG. 1. Computing device200can be in the form of a computing system including multiple computing devices, or in the form of a single computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

A CPU224in computing device200can be a conventional central processing unit. Alternatively, CPU224can be any other type of device, or multiple devices, capable of manipulating or processing information now-existing or hereafter developed. Although the disclosed implementations can be practiced with a single processor as shown, e.g., CPU224, advantages in speed and efficiency can be achieved using more than one processor.

A memory226in computing device200can be a read only memory (ROM) device or a random access memory (RAM) device in an implementation. Any other suitable type of storage device can be used as the memory226. Memory226can include code and data227that is accessed by CPU224using a bus230. Memory226can further include an operating system232and application programs234, the application programs234including at least one program that permits CPU224to perform the methods described here. As shown, for example, application programs234can include applications1through N, which further include a video stream decoding application that performs a method described here. Computing device200can also include a secondary storage236that can be, for example, a memory card used with a mobile computing device200. Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in secondary storage236and loaded into memory226as needed for processing.

Computing device200can also include one or more output devices, such as a display228. Display228may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. Display228can be coupled to CPU224via bus230. Other output devices that permit a user to program or otherwise use computing device200can be provided in addition to or as an alternative to display228. When the output device is or includes a display, the display can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) or light emitting diode (LEI)) display, such as an OLED display.

Computing device200can also include or be in communication with an image-sensing device238, for example a camera, or any other image-sensing device238now existing or hereafter developed that can sense an image such as the image of a user operating computing device200. Image-sensing device238can be positioned such that it is directed toward the user operating computing device200. In an example, the position and optical axis of image-sensing device238can be configured such that the field of vision includes an area that is directly adjacent to display228and from which display228is visible.

Computing device200can also include or be in communication with a sound-sensing device240, for example a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near computing device200. Sound-sensing device240can be positioned such that it is directed toward the user operating computing device200and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates computing device200.

AlthoughFIG. 2depicts CPU224and memory226of computing device200as being integrated into a single unit, other configurations can be utilized. The operations of CPU224can be distributed across multiple machines (each machine having one or more of processors) that can be coupled directly or across a local area or other network. Memory226can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of computing device200. Although depicted here as a single bus, bus230of computing device200can be composed of multiple buses. Further, secondary storage236can be directly coupled to the other components of computing device200or can be accessed via a network and can comprise a single integrated unit such as a memory card or multiple units such as multiple memory cards. Computing device200can thus be implemented in a wide variety of configurations.

FIG. 3is a diagram of an example of a video stream350to be encoded and subsequently decoded. Video stream350includes a video sequence352. At the next level, video sequence352includes a number of adjacent frames354. While three frames are depicted as adjacent frames354, video sequence352can include any number of adjacent frames354. Adjacent frames354can then be further subdivided into individual frames, e.g., a single frame356. At the next level, single frame356can be divided into a series of blocks358, which can contain data corresponding to, for example, 16×16 pixels in frame356. The blocks358can also be arranged in planes of data. For example, a corresponding block358in each plane can respectively contain luminance and chrominance data for the pixels of the block358. Blocks358can also be of any other suitable size such as 16×8 pixel groups or 8×16 pixel groups and can be further subdivided into smaller blocks depending on the application. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

FIG. 4is a block diagram of an encoder470in accordance with an aspect of this disclosure. Encoder470can be implemented, as described above, in transmitting station112such as by providing a computer software program stored in memory, for example, memory226. The computer software program can include machine instructions that, when executed by a processor such as CPU224, cause transmitting station112to encode video data in the manner described inFIG. 4. Encoder470can also be implemented as specialized hardware included, for example, in transmitting station112. Encoder470has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream488using input video stream350: an intra/inter prediction stage472, a transform stage474, a quantization stage476, and an entropy encoding stage478. Encoder470may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. InFIG. 4, encoder470has the following stages to perform the various functions in a reconstruction path: a dequantization stage480, an inverse transform stage482, a reconstruction stage484, and a loop filtering stage486. Other structural variations of encoder470can be used to encode video stream350.

When video stream350is presented for encoding, each frame356within the video stream350can be processed in units of blocks358. At the intra/inter prediction stage472, each block can be encoded using intra-frame prediction or inter-frame prediction. In any case, a prediction block can be formed. In the case of intra-frame prediction, also called intra prediction herein, a prediction block can be formed from spatially nearby blocks in the current frame that have been previously encoded and reconstructed. In the case of inter-frame prediction (also called inter prediction herein), a prediction block can be formed from one or more blocks of previously-constructed reference frame(s) or temporally nearby frame(s) as identified by a respective motion vector.

Next, still referring toFIG. 4, the prediction block can be subtracted from the current block at intra/inter prediction stage472to produce a residual block (also called a residual). Transform stage474transforms the residual into transform coefficients in, for example, the frequency domain. Examples of block-based transforms include the Karhunen-Loève Transform (KLT), the Discrete Cosine Transform (DCT), and the Singular Value Decomposition Transform (SVD). In one example, the DCT transforms the block into the frequency domain. In the case of DCT, the transform coefficient values are based on spatial frequency, with the lowest frequency (DC) coefficient at the top-left of the matrix and the highest frequency coefficient at the bottom-right of the matrix.

Quantization stage476converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. The quantized transform coefficients are then entropy encoded by entropy encoding stage478. The entropy-encoded coefficients, together with other information used to decode the block, which may include for example the type of prediction used, motion vectors and quantizer value, are then output to compressed bitstream488. Compressed bitstream488can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. Compressed bitstream488can also be referred to as an encoded video stream and the terms are used interchangeably herein.

The reconstruction path inFIG. 4(shown by the dotted connection lines) can be used to ensure that both encoder470and a decoder500(described below) use the same reference frames to decode compressed bitstream488. The reconstruction path performs functions that are similar to functions that take place during the decoding process that are discussed in more detail below, including dequantizing the quantized transform coefficients at dequantization stage480and inverse transforming the dequantized transform coefficients at inverse transform stage482to produce a derivative residual block (also called a derivative residual). At reconstruction stage484, the prediction block that was predicted at the intra/inter prediction stage472can be added to the derivative residual to create a reconstructed block. Loop filtering stage486can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

Other variations of encoder470can be used to encode compressed bitstream488. For example, a non-transform based encoder470can quantize the residual signal directly without transform stage474. In another implementation, an encoder470can have quantization stage476and dequantization stage480combined into a single stage.

FIG. 5is a block diagram of a decoder500in accordance with an implementation. Decoder500can be implemented in receiving station130, for example, by providing a computer software program stored in memory226. The computer software program can include machine instructions that, when executed by a processor such as CPU224, cause receiving station130to decode video data in the manner described inFIG. 5. Decoder500can also be implemented in hardware included, for example, in transmitting station112or receiving station130.

Decoder500, similar to the reconstruction path of encoder470discussed above, includes in one example the following stages to perform various functions to produce an output video stream516from compressed bitstream488: an entropy decoding stage502, a dequantization stage504, an inverse transform stage506, an intra/inter prediction stage508, a reconstruction stage510, a loop filtering stage512and a deblocking filtering stage514. Other structural variations of decoder500can be used to decode compressed bitstream488.

When compressed bitstream488is presented for decoding, the data elements within compressed bitstream488can be decoded by entropy decoding stage502(using, for example, arithmetic coding) to produce a set of quantized transform coefficients. Dequantization stage504dequantizes the quantized transform coefficients, and inverse transform stage506inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by inverse transform stage482in encoder470. Using header information decoded from compressed bitstream488, decoder500can use intra/inter prediction stage508to create the same prediction block as was created in encoder470, e.g., at intra/inter prediction stage472. At reconstruction stage510, the prediction block can be added to the derivative residual to create a reconstructed block. Loop filtering stage512can be applied to the reconstructed block to reduce blocking artifacts. A postprocessing stage can be applied to the reconstructed block to further refine the image. In this example, deblocking filtering stage514is applied to the reconstructed block to reduce blocking distortion, and the result is output as output video stream516. Output video stream516can also be referred to as a decoded video stream and the terms are used interchangeably herein.

Other variations of decoder500can be used to decode compressed bitstream488. For example, decoder500can produce output video stream516without post-processing such as deblocking filtering stage514.

FIG. 6is a flowchart of a process600for encoding a video stream according to an aspect of this disclosure. In this example, the video stream encodes blocks using inter-frame prediction first and then encodes blocks using intra-frame prediction. Process600can be implemented in an encoder such as encoder470to implement prediction mode block ordering to encode a video stream. Process600can be implemented, for example, as a software program that is executed by computing devices such as transmitting station112or receiving station130. The software program can include machine-readable instructions that are stored in a memory such as memory226that, when executed by a processor such as CPU224, cause the computing device to perform process600. Process600can also be implemented using hardware. As explained above, some computing devices may have multiple memories and multiple processors, and the steps of process600may in such cases be distributed using different processors and memories. Use of the terms “processor” and “memory” in the singular herein encompasses computing devices that have only one processor or one memory as well as devices having multiple processors or memories that may each be used in the performance of some but not necessarily all of the recited steps.

For simplicity of explanation, process600is depicted and described as a series of steps. However, steps in accordance with this disclosure can occur in various orders and/or concurrently. Additionally, steps in accordance with this disclosure may occur with other steps not presented and described herein. Furthermore, not all illustrated steps may be required to implement a method in accordance with the disclosed subject matter.

At step602, a first group of blocks is identified in a frame of the video stream. By identified we mean selected, chosen, determined or otherwise identified in any manner whatsoever. The first group of blocks are blocks to be encoded using inter prediction. As described above and in relation toFIGS. 4 and 5, inter prediction includes using pixel data from another frame to predict the pixel data in a block. In one example of implementing the teachings herein, the first group of blocks is identified by examining blocks of the frame in the scan order of the frame, and each block to be encoded using inter prediction is added to the first group of blocks.

At step604, a second group of blocks is identified that are to be encoded using intra prediction. The second group of block can be identified in the same scan order of blocks as the first group. That is, when examining the blocks in the frame in the scan order of the frame, when a block is identified as to be encoded using intra prediction, that block is added to the second group of blocks. Accordingly, steps602and604may be performed during a single scan of the blocks of the frame by analyzing the blocks in the scan order for the optimal prediction mode and sorting the blocks into groups once that optimal prediction mode is selected.

Which prediction mode to use for a block can be determined by trying different prediction modes and comparing the results. For example, the sum of absolute differences for the resulting residual blocks for the various prediction modes can be compared. The prediction mode with the smallest residual can be selected for a given block. Note that although inter prediction is described generally as using block(s) of another frame to predict a block of the current frame, this disclosure contemplates that a current block may be encoded using inter prediction within the current frame through the use of a motion vector and another block within the current frame. Such a block would be included within the first group of blocks. The intra prediction modes tested as part of the identification in steps602and604can be restricted to those conventionally used with the scan order of the frame.

In identifying the second group of blocks, at least one block is identified that occurs in the scan order before at least one block of the first group. Blocks of a frame are scanned in particular orders. A typical order to use is raster scan order, where blocks of a frame are arranged in a rectangular array of rows and columns and the blocks of the array are accessed one at a time starting from the upper left hand corner and accessed in row order from the top row and moving down. Any scan order can be used with the teachings herein, but once a scan order is selected, a block in the second group would precede a block in the first group in the scan order if both groups were included in the same scan.

By dividing the blocks of the current frame to be encoded into two groups of blocks, those to be encoded using inter prediction and those to be encoded using intra prediction, the order of encoding the blocks can be manipulated to provide better prediction for intra coded blocks, and hence improving coding efficiency.

More particularly, and as described above in relation toFIGS. 4 and 5, intra prediction includes using pixel data from blocks peripheral to the block to be encoded to predict the pixel values in the block. The blocks used to form the prediction block are often encoded and decoded before being used for prediction. That is, since encoding and decoding can be lossy operations, the pixel values in the encoded and decoded block will not be exactly equal to the pixel values of the original block. By encoding and decoding the block before using it as a prediction block to predict another block, the encoder can use the same pixel values that a decoder will use to intra predict the same block. In the case of coding in raster scan order, for example, intra prediction often uses only pixels from above and to the left of the current block to form the prediction block. This arrangement guarantees that the pixel data of the blocks occurring before the block to be predicted will have been at least partially encoded and decoded before being used for prediction.

According to the teachings herein, the encoder can encode the first set of blocks first. These inter predicted blocks can then be decoded to form reconstructed, or decoded, blocks that can be used for intra prediction of the second set of blocks. In this way, the intra prediction modes for those blocks in the second set of blocks can be expanded to include intra prediction modes using blocks in any position relative to the current blocks where at least some of the blocks (i.e., earlier intra coded blocks in the scan order and the inter coded blocks) have already been encoded and decoded for prediction.

At next step606, the first group of blocks is at least partially encoded. Generally, this partial encoding is lossy, meaning that reversing the encoding steps will not result in exactly the same pixel values as input. As shown inFIG. 4, for example, encoding a block of video data can include forming a prediction block, in this case from another frame or the current frame using motion vector(s), subtracting the prediction block from the block to be encoded, transforming the block using a transform as described above and then quantizing the transform data. The encoding of the first group of blocks can occur in the scan order, skipping those blocks belonging to the second group of blocks. The partially-encoded blocks from the first group of blocks may also be referred to as encoded blocks.

At step608, the encoded blocks generated by the first group of blocks are partially decoded by reversing the lossy steps in encoding. In this example, this involves de-quantizing, inverse transforming and adding the inverse transformed block to the prediction block generated using inter prediction as described with reference to the reconstruction loop ofFIG. 4. This yields pixel data that is equal to the pixel data formed when decoding the blocks at the decoder. An encoder can maintain a copy of the partially-encoded blocks in memory, for example, while performing the remaining steps in process600before completing the encoding of the blocks to include in the output video bitstream.

At step610, the second group of blocks is encoded using intra prediction and at least some of the partially encoded and decoded first group of first blocks. Intra prediction uses pixels from blocks peripheral to a block to predict the pixel values within a current block. This process may be performed in the scan order after some or all blocks in the first group of blocks are encoded and decoded. As mentioned above, having encoded and decoded results from inter predicted blocks can improve the performance of intra prediction coded blocks by permitting additional prediction modes to be included in the encoding process. Accordingly, step610can include re-calculating the optimal intra prediction mode choice for each block to be encoded using intra prediction. Some of these intra prediction modes may use information from inter coded blocks that would have been coded after a current block if all blocks were encoded in the scan order or a predefined coding order. As a result, intra coding of the current block can make use of reconstructed pixel values from inter coded blocks that would have been previously been encoded after the current block as the inter coded blocks are already processed. The availability of these reconstructed pixel values may help improve the prediction quality when using intra prediction modes, therefore improving the overall coding efficiency of the video frame.

FIG. 8can be used to explain this process.FIG. 8is a diagram of blocks to be encoded or decoded according to aspects of this disclosure. In this example, the blocks are processed in raster scan order. If the blocks of a frame800were conventionally encoded in raster scan order, predicting current block (X, Y) may be performed using pixels in one or more of row802and column804belonging to blocks (X−1, Y−1), (X, Y−1), (X+1, Y−1) and (X−1, Y). Since these blocks, and hence the pixels of row802and column804are encoded and decoded before encoding the current block, they can be desirably used to form prediction blocks taking the lossy nature of the encoding into account. For example, in a vertical intra prediction mode, the pixel values of row802can be reproduced for each row of a prediction block to form the prediction block. In a horizontal prediction intra prediction mode, the pixel values of column804can be reproduced for each column of a prediction block to form the prediction block. Various diagonally-based intra prediction modes are available, at least some of which can use pixel values from both row802and column804.

On the other hand, when blocks are encoded in raster scan order, the pixels from blocks (X+1, (X−1, Y+1), (X, Y+1) and (X+1, Y+1) would not be encoded and decoded at the time current block (X, Y) is encoded. Therefore, pixels from row808and column806are not available for intra prediction of current block (X, Y). According to implementations of the teachings herein, some or all of blocks (X+1, Y), (X−1, Y+1), (X, Y+1) and (X+1, Y+1) may be encoded out of order, i.e., as part of the first group of blocks, so as to provide pixels adjacent to current block (X, Y) for additional intra prediction modes. In an alternative vertical prediction mode, for example, the pixel values of row808can be used to form the prediction block. Similarly, in an alternative horizontal prediction mode, the pixel values of column806can be used to form the prediction block. Other prediction modes may be available using combinations of pixels values of row808and column806, row808and column804, and row802and column806, for example.

Current block (X, Y) is one of the second group of blocks in this example. When encoding current block (X, Y) in step610, an optimal intra prediction mode selected for the encoding may be one that uses any of these modes. Where some or all of blocks (X+1, Y), (X−1, Y+1), (X, Y+1) and (X+1, Y+1) are encoded in step606as part of the first group of blocks, the optimal intra prediction mode may be different from that determined when intra prediction is performed using only those modes associated with pixels of row802and column804.

Referring again toFIG. 6, encoding the second group of blocks in step610involves the same lossy coding process of step606. Step610may also include an additional lossless encoding step, such as entropy coding the quantized residual block Although not shown inFIG. 6, encoding of the blocks of the first group of blocks can also be completed by, for example, entropy coding the quantized residual block. The entropy coded, quantized residual blocks can then be packetized into a video bitstream for transmission or storage and subsequent decoding. The prediction modes can also be entropy encoded and included as part of the compressed video data.

FIG. 7is a flowchart of a process700for decoding a video bitstream according to another aspect of this disclosure. Broadly, in process700, the first group of blocks of the video bitstream are decoded using inter prediction and then the second group of blocks of the video bitstream are decoded using intra prediction. Process700can be implemented in a decoder such as decoder500according to aspects of disclosed implementations. Process700can be implemented, for example, as a software program that is executed by computing devices such as transmitting station112or receiving station130. The software program can include machine-readable instructions that are stored in a memory such as memory226that, when executed by a processor such as CPU224, cause the computing device to perform process700. Process700can also be implemented using hardware. As explained above, some computing devices may have multiple memories and multiple processors, and the steps of process700may in such cases be distributed using different processors and memories.

For simplicity of explanation, process700is depicted and described as a series of steps. However, steps in accordance with this disclosure can occur in various orders and/or concurrently. Additionally, steps in accordance with this disclosure may occur with other steps not presented and described herein. Furthermore, not all illustrated steps may be required to implement a method in accordance with the disclosed subject matter.

At step702, process700identifies a first group of first blocks that can be decoded using inter prediction. At step704, process700identifies a second group of blocks that can be decoded using intra prediction. As discussed above in relation toFIG. 6, the blocks of the frame can be identified in a scan order, which, for example, can be raster scan order. In this example, the decoder can perform entropy decoding to produce prediction modes for every block. All blocks are then grouped into two groups according to the prediction mode used. Accordingly, steps702and704may be performed as one step.

The blocks that can be decoded using each prediction method can be identified using bits included in the video bitstream by the encoder at the time the blocks were encoded, for example. These bits are included in the encoded video bitstream by an encoder to direct a decoder as to which prediction mode to use. As a result, blocks can be sorted into groups for decoding without requiring additional bits in the video bitstream beyond the bits typically included to identify the prediction mode.

At step706, the first group of blocks is decoded using inter prediction. For example, each entropy decoded residual block is inverse transformed and dequantized to form a residual block. The decoder generates the prediction block for the current block using inter prediction, and the current block is reconstructed by adding the prediction block to the residual block as described with respect toFIG. 5. The first group of blocks can be decoded in the scan order for the frame, e.g., raster scan order.

At step708, the second group of blocks is decoded using intra prediction and, depending on the intra prediction mode, the blocks decoded using inter prediction. For example, each entropy decoded residual block is inverse transformed and dequantized to form a residual block. The decoder generates the prediction block for the current block using intra prediction, and the current block is reconstructed by adding the prediction block to the residual block as described with respect toFIG. 5. The second group of blocks can be decoded in the scan order for the frame, e.g., raster scan order.

In this example, the processing of the blocks is performed according to raster scan order, i.e., from top to bottom and left to right. In other cases, processing of the blocks may be according to another predefined scan order, and the choice of such order is also encoded, so that the decoder can process the blocks in same order. No order definition other than the scan order needs to be encoded or transmitted.

According to the teachings herein, intra predicted blocks may use reconstructed pixel values from inter predicted blocks even when those inter predicted blocks would have been encoded/decoded after them in the normal scan order. By re-ordering the encoding and decoding of blocks based on their prediction modes, the encoder can effectively change the data dependency of the blocks in the encoding/decoding process. The blocks encoded later in a frame can, in this way, use all reconstructed pixel information from previously encoded blocks for improving the quality of prediction, therefore improve the coding efficiency.

The aspects of encoding and decoding described above illustrate some exemplary encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

Implementations of transmitting station112and/or receiving station130(and the algorithms, methods, instructions, etc., stored thereon and/or executed thereby, including by encoder470and decoder500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (EP) cores, application-specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of transmitting station112and receiving station130do not necessarily have to be implemented in the same manner.

Further, in one aspect, for example, transmitting station112or receiving station130can be implemented using a general purpose computer or general purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms and/or instructions described herein. In addition or alternatively, for example, a special purpose computer/processor can be utilized which can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

Transmitting station112and receiving station130can, for example, be implemented on computers in a video conferencing system. Alternatively, transmitting station112can be implemented on a server and receiving station130can be implemented on a device separate from the server, such as a hand-held communications device. In this instance, transmitting station112can encode content using an encoder470into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by transmitting station112. Other suitable transmitting station112and receiving station130implementation schemes are available. For example, receiving station130can be a generally stationary personal computer rather than a portable communications device and/or a device including an encoder470may also include a decoder500.