Apparatus, medium, and method for processing neighbor information in video decoding

Provided is an apparatus, medium, and method for processing neighbor information in a video decoder that can minimize the number of memory accesses. The apparatus includes a neighbor information providing unit and a storage unit. If at least one spatially neighboring block of a current block exists in memory, the neighbor information providing unit can access information of all neighbor blocks from the memory and provides the accessed information as neighbor information. The storage unit stores the neighbor information provided by the neighbor information providing unit and outputs the stored neighbor information to the plurality of modules.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of Korean Patent Application No. 10-2004-0091495, filed on Nov. 10, 2004, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

Embodiments of the present invention relate to an apparatus, medium, and method for processing neighbor information in video decoding, and more particularly, to an apparatus, medium, and method for processing neighbor information in a video decoder that minimizes the number of memory accesses.

2. Description of the Related Art

Video decoders decompress images that are encoded in a compressed form. The video decoders are included in such devices as a video codec or an H.264 codec.FIG. 1illustrates a block diagram of an H.264 decoder included in an H.264 codec.

The H.264 decoder shown inFIG. 1uses an entropy-decoder102, a dequantization and inverse transformation unit103, a motion prediction unit104, and a deblocking filter105, in a pipeline form in order to improve its speed. Here, when the entropy-decoder102entropy-decodes an nthmacroblock, the dequantization and inverse transformation unit103dequantizes and inversely transforms an (n−1)thmacroblock, the motion prediction unit104performs motion prediction on an (n−2)thmacroblock, and the deblocking filter105performs deblocking-filtering on an (n−3)thmacroblock.

These modules, and others such as a parser101, are supposed to access a memory107using a common bus106. The memory107typically store information in macroblock or block units. Thus, the entropy-decoder102, the dequantization and inverse transformation unit103, and the motion prediction unit104, and the deblocking filter105access the memory107through the common bus106when information of spatially neighboring macroblocks or blocks is required.

However, since the modules operate in the form of a macroblock-based pipeline, as described above, several of the modules can access the memory107at the same time. Consequently, collisions occur in the common bus106, resulting in delays in data transmission. Also, such operations by each module has it's own latency, as shown inFIG. 2, from generating an address to access the memory107to receiving actual data. The latency ranges from 8 to 9 clock cycles. Such problems inevitably limit improvements in the processing speed of the H.264 decoder.

Also, the bit size of most of the neighbor information that is accessed in the memory107is smaller than that of the common bus106. Thus, as the number of accesses of the memory107to obtain the neighbor information increases, the common bus106operates less efficiently.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide an apparatus, medium, and method for processing neighbor information in video decoding minimizing the number of memory accesses.

To achieve the above and/or other aspects embodiments of the present invention include an apparatus to process neighbor information in a video decoder having a plurality of modules operating in a pipeline form and a memory storing information in units of at least a block, including a neighbor information providing unit to access respective information of at least one spatially neighboring block of a current block from the memory and provide the information as neighbor information, and a storage unit, distinguished from the memory, to store the neighbor information, provided by the neighbor information providing unit, and output the stored neighbor information to at least one of the plurality of modules for video decoding.

The storage unit may have a structure in which respective registers in the storage unit are assigned to respective modules. In addition, the respective registers may be assigned to the respective modules are consecutively connected.

In addition, the memory may be accessed through a bus of a the video decoder and the storage unit may output the stored neighbor information to the plurality of modules outside of the bus.

Further, when the plurality of modules include a dequantization and inverse transformation module, a motion prediction module, and a deblocking filter module, the storage unit may further include a first register to store respective neighbor information used by the dequantization and inverse transformation module, the motion prediction module, and the deblocking filter module, a a second register to store neighbor information used by the motion prediction module and the deblocking filter module, and a third register to store neighbor information used by the deblocking filter module, wherein the first register transmits respective neighbor information used by the motion prediction module and the deblocking filter module to the second register, and the second register transmits respective neighbor information used by the deblocking filter module to the third register.

The first register, the second register, and the third register may further store information processed in slice units, and the information processed in slice units is provided by a parser included in the video decoder.

The plurality of modules may include an entropy-decoder module, a dequantization and inverse transformation module, a motion prediction module, and a deblocking filter module, with the neighbor information providing unit providing respective neighbor information used by the entropy-decoder module, the dequantization and inverse transformation module, the motion prediction module, and the deblocking filter module.

The neighbor information providing unit may access the memory to store an entropy-decoded result obtained from the entropy-decoder in the memory.

To achieve the above and/or other aspects embodiments of the present invention include an apparatus to process neighbor information in a video decoder having a plurality of modules operating in a pipeline form, and a bus used to access a memory connected to the plurality of modules and to store information in units of at least a block, the apparatus including a neighbor information providing unit to access respective information of spatially neighboring blocks of a current block from the memory via the bus and provide the accessed information as neighbor information, and a storage unit, distinguished from the memory, to store the neighbor information provided by the neighbor information providing unit and output the stored neighbor information to at least one of the plurality of modules for video decoding.

All respective spatially neighboring blocks may be read from the memory as the respective information of the spatially neighboring blocks. In addition, the memory may be accessed through a bus of a the video decoder and the storage unit may output the stored neighbor information to the plurality of modules outside of the bus.

To achieve the above and/or other aspects embodiments of the present invention include a method for processing neighbor information, through a plurality of modules operating in a pipeline form, accessing a memory storing information in units of at least a block, the method including reading respective information of spatially neighboring blocks of a current block from the memory, as respective neighbor information, storing in a storage unit, distinguishable from the memory, the respective neighbor information for respective modules, and transmitting the respective neighbor information from the storage unit to the plurality of modules, respectively, for video decoding.

All spatially neighboring blocks may be read from the memory as the respective information of the spatially neighboring blocks. In addition, the memory may be accessed through a bus of a corresponding video decoder and the storage unit may output the stored neighbor information to the plurality of modules outside of the bus.

When the plurality of modules include a dequantization and inverse transformation module, a motion prediction module, and a deblocking filter module, the transmitting of the neighbor information may include (a) storing in the storage unit respective neighbor information used by the dequantization and inverse transformation module, the motion prediction module, and the deblocking filter module among the read respective neighbor information, (b) storing in the storage unit respective neighbor information used by the motion prediction module and the deblocking filter module among the respective neighbor information of (a), and (c) storing in the storage unit respective neighbor information used by the deblocking filter module among the respective neighbor information of (b). Here, (a), (b), and (c) may be performed in sequence.

To achieve the above and/or other aspects embodiments of the present invention include at least one medium including computer readable code to implement at least an aspect of this method.

To achieve the above and/or other aspects embodiments of the present invention include at least one medium including computer readable code to implement a processing of neighbor information through a plurality of modules operating in a pipeline form, and an accessing of a memory storing at least block-based information, the method including reading respective information of spatially neighboring blocks of a current block from the memory, as respective neighbor information, storing in a storage unit, distinguishable from the memory, the respective neighbor information for respective modules, and transmitting the respective neighbor information from the storage unit to the plurality of modules, respectively, for video decoding.

The memory may be accessed through a bus of a corresponding video decoder and the storage unit may output the stored neighbor information to the plurality of modules outside of the bus.

To achieve the above and/or other aspects embodiments of the present invention include a video decoder means for decoding video through a plurality of module means operating in a pipeline form and a memory means for storing information in units of at least a block, including a means for accessing respective information of at least one spatially neighboring block of a current block from the memory means and for providing the information as neighbor information, and a storage means, distinguished from the memory means, for storing the neighbor information and outputting the stored neighbor information to at least one of the plurality of module means for the decoding of the video.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 3illustrates a video decoder processing neighbor information, according to an embodiment of the present invention. Referring toFIG. 3, the video decoder includes a parser301, a bus302, a memory303, an entropy-decoder304, a neighbor information providing unit305, a storage unit306, a dequantization and inverse transformation unit310, a motion prediction unit311, and a deblocking filter312. The storage unit306includes first through third registers307through309.

The video decoder includes the memory303, but the memory303may also be an external buffer of the video decoder, for example. Also, the parser301, the entropy-decoder304, the dequantization and inverse transformation unit310, the motion prediction unit311, and the deblocking filter312operate in a pipeline form.

First, a bitstream input to the video decoder is composed of a video coding layer (VCL) network abstraction layer (NAL) and a non-VCL NAL. The VCL NAL is composed of a slice header and data, and the data is composed of various macroblocks (MB).

The parser301parses the bitstream input in the above-described format into slice units, and transmits the parsing result to the first register307, while providing the same to the memory303through the bus302. The parsing result includes information defined by slice units, such as a slice number. The parser301transmits the input bitstream to the entropy-decoder304.

The entropy-decoder304entropy-decodes the received bitstream into macroblock units. At this time, if at least one spatially neighboring macroblock of the current macroblock exists, the entropy-decoder304provides identification information of the current macroblock and information indicating the range of the neighboring macroblocks of the current macroblock (e.g., macroblock adaptive frame/field (Mbaff)) to the neighbor information providing unit305, and entropy-decodes the received bitstream using neighbor information provided by the neighbor information providing unit305.

The neighbor information providing unit305accesses the memory303through the bus302based on the identification information of the current macroblock and the information indicating the range of the neighboring macroblocks, and then reads information of neighboring macroblocks or 4×4 blocks of the current macroblock.

For example, when a macroblock18is the current macroblock, as shown inFIG. 4, the neighbor information providing unit305can read information of macroblocks0through13,16, and17, for example, from the memory303. Here, when the current macroblock is a hatched macroblock, as shown inFIG. 5, the neighbor information providing unit305can read information of hatched 4×4 blocks from the memory303.

Also, the neighbor information providing unit305can select neighboring macroblocks to be read from the memory303from among neighboring macroblocks shown inFIG. 4, based on macroblock adaptive frame/field (Mbaff) information provided by the entropy-encoder304. In other words, if Mbaff is 0, the neighbor information providing unit305selects 4 hatched macroblocks NMB[1], NMB[3], NMB[5], and NMB[6] for the current macroblock NMB[8], as shown inFIG. 6A, as neighboring macroblocks to be read from among the neighboring macroblocks shown inFIG. 4, and reads information from the memory303in macroblock units. If Mbaff is 1, the neighbor information providing unit305selects 8 hatched macroblocks NMB[0], NMB[1], NMB[2], NMB[3], NMB[4], NMB[5], NMB[6], and NMB[7] for the current macroblock NMB[8], as shown inFIG. 6B, or 9 hatched macroblocks NMB[0], NMB[1], NMB[2], NMB[3], NMB[4], NMB[5], NMB[6], and NMB[7], NMB[8] for the current macroblock NMB[9], as shown inFIG. 6C, as neighboring macroblocks to be read from among the neighboring macroblocks shown inFIG. 4, and reads information of the selected macroblocks from the memory303in macroblock units.

If the memory303is an external buffer, the bus302may be defined as an external bus in the H.264 codec. If the memory303is included in the video decoder, the bus302may be defined as an internal bus in the H.264 codec, for example.

The memory303can store information in macroblock or 4×4 block units. If the information is stored in macroblock units, the memory303can further store information in the format shown inFIG. 7, for example. InFIG. 7, information of a macroblock has been illustrated based on information of the macroblock being made up of 32 bits.

InFIG. 7, MBI represents a flag indicating whether a corresponding macroblock is an intrablock, MBE represents a flag indicating whether an error occurs in the corresponding macroblock, Slice_num represents a slice number (Slice_nr), MBS represents a skip flag indicating whether the corresponding macroblock is a skipped macroblock, MBF represents information indicating that the corresponding macroblock is a macroblock field (MB_Field), Qpc represents a quantization coefficient of chrominance, Qpy represents a quantization coefficient of luminance, coded_block_pattern represents a value indicating whether the coefficient of the corresponding macroblock is 0, c_ipred_md represents information indicating an intra-prediction mode of chrominance, the B8pdir[n] represent a prediction direction of an nth8×8 block in the corresponding macroblock, the B8mode[n] represent type information of the nth8×8 block in the corresponding macroblock, cbp_bits represents code block flag information used for context-based adaptive binary arithmetic coding (CABAC), LFBetaOFFset represents beta offset information used in the deblocking filter312, LFAlphaCOOFFset represents AlphaCO offset information used in the deblocking filter312, LFD_idc represents identification information of the deblocking filter312, and CBP_blk represents code block flag information used for the deblocking filter312.

As an example, when the neighbor information providing unit305reads information of 4 macroblocks, it reads information of a macroblock which has a format like that shown inFIG. 7, from the memory303, 4 times.

The neighbor information providing unit305provides neighbor information used by the entropy-decoder304, from among the read neighbor information, to the entropy-decoder304, and transmits neighbor information used by the dequantization and inverse transformation unit310, the motion prediction unit311, and the deblocking filter312, to the storage unit306, for example.

The neighbor information used by the entropy-decoder304is defined by the below Table 1.

TABLE 1Neighbor information forentropy-decoderMeaningMvLX[ ]Motion vector in 4 × 4 partition units for motion vectorpredictionIntra4×4_pred_modeIntra prediction mode in 4 × 4 partition units forobtaining 4 × 4 intra prediction modeTotalCoeffThe number of non-zero coefficients in 4 × 4 partitionunits for VLC of residual data in context-basedadaptive variable length coding (CAVLC)MvdLX[ ]Motion vector difference in 4 × 4 partition units for thecontext of Mvd of CABACReIdx [ ][ ]Reference index in 4 × 4 partition units for the contextof reference indexs of CABAC and a reference picturereferring to MV.Skip_flagInformation in macroblock units for obtaining contextinformation for Mb_skipMb_fieldMacroblock field/frame information for obtaining alocation of a neighboring block in an Mbaff andinformation for obtaining the context for a macroblockfield (mb_field).Mb_typeInformation in macroblock units for obtaining contextinformation for a macroblock type (Mb_type)CbpCoded block flag for CABAC (1 bit flag per 8 × 8 block)Cbp_bitsCoded block flag for CABAC (1 bit flag per 4 × 4 blockfor DC and AC coefficient)C_ipred_modeIntra chroma prediction mode(Intra_chroma_pred_mode) information for CABACSlice_nrSlice number information for checking availability of a macroblockIntra_blockInformation indicating a macroblock is an intra- or inter-block

Upon receiving the result of entropy-decoding the current macroblock from the entropy-decoder304, the neighbor information providing unit305writes the entropy-decoded result to the memory303through the bus302.

The storage unit306includes the respective first through third registers307through309, which are consecutively connected and each assigned to a module, for example. In other words, the first register307is assigned to the dequantization and inverse transformation unit310, the second register308is assigned to the motion prediction unit311, and the third register309is assigned to the deblocking filter312. Since the neighbor information stored in the first through third registers307through309is used as parameters for the corresponding modules, the first through third registers307through309may be defined as parameter registers.

Upon receiving neighbor information from the neighbor information providing unit305, the first register307stores the received neighbor information. Here, the received neighbor information includes neighbor information used by the dequantization and inverse transformation Unit310, the motion prediction unit311, and the deblocking filter312.

The first register307transmits the neighbor information used by the motion prediction unit311and the deblocking filter312, from among its stored neighbor information, to the second register308. The neighbor information used by the motion prediction unit311is defined by the below Table 2.

TABLE 2Neighbor informationrequired for predictionMeaningMb_fieldMacroblock field/frame information for obtaininga location of a neighboring block in an MbaffSlice_nrSlice number information for checkingavailability of a macroblock

The second register308transmits the neighbor information used by the deblocking filter312, from among its stored neighbor information, to the third register309. The third register309then stores the received neighbor information. The neighbor information used by the deblocking filter312is defined by the below Table 3.

TABLE 3Neighborinformation forthe strength of thedeblocking filterMeaningMb_fieldMacroblock field/frame information for obtaining alocation of a neighboring block in an MbaffMvLX [ ]Motion vector in 4 × 4 partition units used forcalculating a filter strengthrefAdr [ ] [ ]Reference picture in 4 × 4 partition units used forcalculating a filter strengthLFDisableIdcIndicator for disabling the deblocking filteraccording to conditionsMb_typeIntra/inter mode information used for calculating afilter strengthcbp_blkInformation indicated whether there is a non-zerocoefficient of a macroblock in calculation of a filterstrength (1-bit flag per 4 × 4 block only forluminance)Slice_nrSlice number information for checking availabilityof a macroblock

The dequantization and inverse transformation unit310dequantizes the received entropy-decoded data with reference to the neighbor information stored in the first register307, and inversely transforms the dequantized result. The result is residual information which is transmitted to the motion prediction unit311and, at the same time, for example, to the memory303through the bus302.

Upon receiving the residual information of the current macroblock, the motion prediction unit311predicts motion between pictures with reference to the neighbor information stored in the second register308, and reconstruct the original image with respect to the current macroblock using the predicted motion between the pictures and the residual information. The reconstructed result is transmitted to the memory303through the bus deblocking filter312and the bus302.

The deblocking filter312deblocking-filters the reconstructed image with respect to the current macroblock, input from the motion prediction unit311, outputs the deblocking-filtered result, and stores the same in the memory303through the bus302.

A pipeline operation of the entropy-decoder304, the neighbor information providing unit305, the dequantization and inverse transformation unit310, the motion prediction unit, and the deblocking filter312is further illustrated inFIG. 8, according to an embodiment of the present invention. As can be seen from the stages defined inFIG. 8, the entropy-decoder304and the neighbor information providing unit305process the same macroblock.

Referring to stage (X+3) ofFIG. 8, in this embodiment of the present invention, when the entropy-decoder304and the neighbor information providing unit305operate on an (N+3)thmacroblock, the dequantization and inverse transformation unit310operates on an (N+2)thmacroblock, the motion prediction unit311operates on an (N+1)thmacroblock, and the deblocking filter312operates on an Nthmacroblock.

FIG. 9is a flowchart illustrating a processing of neighbor information, according to an embodiment of the present invention.

If at least one spatially neighboring macroblock of a current macroblock exists, the neighbor information providing unit305reads information of all neighboring macroblocks of the current macroblock, as used by each module of the video decoder, from the memory303, e.g., such as in operation901.

The neighbor information providing unit305provides information used by the entropy-decoder304, from among the read information of the neighboring macroblocks (referring to as neighbor information), to the entropy-decoder304, and at the same time, transmits neighbor information used by the dequantization and inverse transformation unit310, the motion prediction unit311, and the deblocking filter312to the storage unit306, e.g., such as in operation902.

The storage unit306separately stores the neighbor information transmitted from the neighbor information providing unit305, to transmit its stored neighbor information to each module that uses its corresponding neighbor information, such as in operation903. First, the storage unit306stores the neighbor information used by the dequantization and inverse transformation unit310, the motion prediction unit311, and the deblocking filter312as neighbor information that can be referred to by the dequantization and inverse transformation unit310. The storage unit306then stores neighbor information used by the motion prediction unit311and the deblocking filter312, as neighbor information that can be referred to by the motion prediction unit311. Next, the storage unit306stores neighbor information used by the deblocking filter312as neighbor information that can be referred to by the deblocking filter312. In this way, the storage unit306sequentially and separately stores the neighbor information transmitted from the neighbor information providing unit305.

In operation904, the entropy-decoder304, the dequantization and inverse transformation unit310, the motion prediction unit311, and the deblocking filter312performs corresponding operations with reference to the stored neighbor information, such as the neighbor information stored in operation903.

As described above, in a video decoder embodiment that operates each module in a pipeline form, neighbor information is accessed in macroblock units from a memory, the neighbor information is separately stored for each module's reference to the accessed neighbor information, and each module in the video decoder is allowed to refer to the stored neighbor information, thereby minimizing a number of accesses to the memory. Furthermore, it is possible to prevent bus collisions resulting from memory access to obtain neighbor information, and thereby prevent delays in data transmission caused by such collisions. Therefore, the processing speed of the video decoder can be improved.

Also, by minimizing the inefficient operation of the bus between the memory and each module and the number of memory accesses, the hardware modules of the video decoder can be designed more easily.

A computer readable code, such as coding, instructions, and programs, to implement the processing of neighbor information, and/or aspects thereof, according to embodiments of the present invention, can also be included in/on a medium, such as a computer readable recording medium, for example. The medium may be any data storage/transmitting device that can store data that can be thereafter be read by a computer. Examples of the medium may include any of a read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and carrier waves, for example.

The medium may also be distributed over network coupled computer systems so that the computer readable code is stored/transmitted and implemented in a distributed fashion.