Source: http://www.google.com/patents/US5684534?dq=6650327
Timestamp: 2017-07-24 17:58:52
Document Index: 733378395

Matched Legal Cases: ['art 1300', 'art 1300', 'art 1300', 'art 1300', 'art 1300', 'art 1300']

Patent US5684534 - Task-splitting dual-processor system for motion estimation processing - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsA video processor system has separate and independent video processors for performing a variety of video processor functions required for encoding and decoding video signals. Each of the separate video processors performs its own individual set of video processor functions. During the encode process...http://www.google.com/patents/US5684534?utm_source=gb-gplus-sharePatent US5684534 - Task-splitting dual-processor system for motion estimation processingAdvanced Patent SearchTry the new Google Patents, with machine-classified Google Scholar results, and Japanese and South Korean patents.Publication numberUS5684534 APublication typeGrantApplication numberUS 08/067,864Publication dateNov 4, 1997Filing dateMay 26, 1993Priority dateMay 26, 1993Fee statusPaidPublication number067864, 08067864, US 5684534 A, US 5684534A, US-A-5684534, US5684534 A, US5684534AInventorsKevin Harney, Mike S. Kelly, Gary LoeserOriginal AssigneeIntel CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (16), Referenced by (52), Classifications (21), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetTask-splitting dual-processor system for motion estimation processing
US 5684534 AAbstract
A video processor system has separate and independent video processors for performing a variety of video processor functions required for encoding and decoding video signals. Each of the separate video processors performs its own individual set of video processor functions. During the encode process the first video processor performs motion estimation to provide motion estimation information which it applies to the second video processor. The second video processor receives the motion estimation information and performs forward and inverse discrete cosine transforms, quantization and dequantization, frame addition and frame differencing, as well as run length encoding. The run length encoding operation produces run/value pairs which are then applied to the first video processor. The first video processor performs variable length encoding upon the run/value pairs. During the decoding process the first video processor performs a variable length decode and applies the variable length decoded data to the second video processor. The second video processor performs run length decoding, dequantization, inverse discrete cosine transforms and frame addition according to the received variable length data. The inverse transformed data produced by this operation is then applied to the first video processor.
1. A video processor task-splitting system having system memory for performing a plurality of video processor functions, comprising:first video processor means for performing a first selected portion of said plurality of video processor functions; second video processor means comprising a compression/decompression accelerator, connected to said first video processor means, for performing a second selected portion of said plurality of video processor functions different from said first selected portion of said plurality of video processor functions; said first video processor means having means for receiving data from said system memory, and for performing motion estimation upon said received data, and for applying motion estimation information to said second video processor means in response to said motion estimation; said second video processor means having means for receiving said motion estimation information, and for performing a discrete cosine transform, and for quantization operations in accordance with said motion estimation information to provide encoded information, and for applying said encoded information to said first video processor means; and, said first video processor means having means for receiving said encoded information, and for performing variable length encoding upon said encoded information, and for storing variable length encoded data in said system memory. 2. The video processing system of claim 1, wherein said second video processor means further comprises frame add means and frame subtract means.
3. The video processing system of claim 2, wherein said second video processor means further comprises run length encoding means and run length decoding means.
4. The video processor system of claim 3, wherein said first video processor means further comprises programmable processor means and program control means.
5. The video processor system of claim 2, wherein said second video processor means further comprises a loop filter.
6. In a video processor task-splitting system having system memory, a method for performing a plurality of video processor functions, the method comprising the steps of:(a) performing, with a first video processor, a first selected portion of said plurality of video processor functions; (b) performing, with a second video processor comprising a compression/decompression accelerator connected to said first video processor, a second selected portion of said plurality of video processor functions different from said first selected portion of said plurality of video processor functions; (c) receiving data from said system memory with said first video processor, performing motion estimation with said first video processor upon said received data, and applying motion estimation information with said first video processor to said second video processor means in response to said motion estimation; (d) receiving said motion estimation information with said second video processor, performing a discrete cosine transform with said second video processor, performing quantization operations with said second video processor in accordance with said motion estimation information to provide encoded information, and applying said encoded information to said first video processor with said second video processor; and (e) receiving, with said first video processor, said encoded information, performing variable length encoding upon said encoded information with said first video processor, and storing, with said first video processor, variable length encoded data in said system memory. 7. The method of claim 6, wherein said second video processor comprises frame add means and frame subtract means.
8. The method of claim 7, wherein said second video processor further comprises run length encoding means and run length decoding means.
9. The method of claim 8, wherein said first video processor comprises programmable processor means and program control means.
10. The method of claim 7, wherein said second video processor comprises a loop filter.
It is well-known to provide hardware circuitry for performing video encoding and decoding operations. In particular, it is known to provide hardware circuitry to perform frame adds and frame subtracts, run length encoding and decoding, forward and inverse discrete cosine transforms, and quantization/dequantization operations as well as motion estimation and variable length encoding and decoding. When greater processing power was required or when improved algorithms became available in these systems, it was thus necessary to perform hardware redesign and provide new integrated circuit chips. These redesigns were very expensive and slow.
However, many of these operations were much too complex to be performed by software since executing software capable of performing these tasks would require too much time to be useful for video processing. An exception to this is the processing required for motion estimation. However, improvements in motion estimation methods still required extension hardware redesign since these operations were embedded in the overall hardware circuitry. The motion estimation process required a great deal of memory space because several different images had to be buffered. These images included a current image being encoded, the previous image, a companded image provided by decoding the data being encoded and transmitted, and the new digitized video data being received by the system. Thus four full size image buffers were required.
In addition, it is a well known to provide quantization within these systems. In a typical system the number of quantizations may be one for the intrablock encoded DC coefficient and thirty-one for all other coefficients. Within a macroblock the same quantization is used for all coefficients except the intrablock encoded DC quantization. The decision levels may not be defined. The intrablock encoded dc coefficient is nominally the transform value linearly quantized with a step size of eight and no dead zone. Each of the other thirty-one quantizations is also nominally linear but with a central dead zone around zero and with a step size of an even value in the range two to sixty-two. In these systems the full dynamic range of the transformed coefficients cannot be represented for smaller quantization step sizes.
Values that are quantized in this manner may be dequantized in the following manner. For all coefficients other than the intrablock encoded DC quantization the reconstruction levels, REC, are in the range of -2048 to 2047 and are given by clipping the results of the following equations:
REC=QUANT*(2*LEVEL+1);LEVEL&gt;0 QUANT="odd"
REC=QUANT*(2*LEVEL-1);LEVEL&lt;0 QUANT="odd"
REC=QUANT*(2*LEVEL+1)-1;LEVEL&gt;0 QUANT="even"
REC=QUANT*(2*LEVEL-1)+1;LEVEL&lt;0 QUANT="even"
REC=0;LEVEL=0
Where QUANT ranges from one to thirty-one. These reconstruction levels are symmetrical with respect to the sign of LEVEL except for the values 2047 and -2048.
In the case of blocks which are intrablock encoded the first coefficient is nominally the transform DC value linearly quantized with a step size of eight and no dead zone.
The resulting values are represented with eight bits. A nominally black block provides the value 0001 0000 and a nominally white block yields 1110 1011. The codes 0000 0000 and 1000 0000 are not used. The reconstruction level of 1024 is coded as 1111 1111. Coefficients after the last non-zero one are not transmitted.
A video processor system is provided with two separate and independent video processors for performing a variety of video processor functions required for encoding and decoding video signals. Each of the separate video processors performs its own individual set of video processor functions. During the encode process the first video processor performs motion estimation to provide motion estimation information which it applies to the second video processor. The second video processor receives the motion estimation information and performs forward and inverse discrete cosine transforms, quantization and dequantization, frame addition and frame differencing, as well as run length encoding. The run length encoding operation produces run/value pairs which are then applied to the first video processor. The first video processor performs variable length encoding upon the run/value pairs. During the decoding process the first video processor performs a variable length decode and applies the variable length decoded data to the second video processor. The second video processor performs run length decoding, dequantization, inverse discrete cosine transforms and frame addition according to the received valuable length data. The inverse transformed data produced by this operation is then applied to the first video processor.
FIGS. 3A, B show a block diagram representation of the dataflow of the system and method of FIG. 2 when it is adapted to encode digital video signals and physical buffer memories for use in this process.
FIGS. 14A-C show flowchart representations of a motion estimation method and an alternate embodiment thereof for use in the dataflow of FIG. 3.
FIGS. 21A,B show block diagram representations of alternate embodiments of a system for testing data and selectably performing a discrete cosine transform suitable for use in the system of FIG. 1.
Referring now to FIG. 1, there is shown remote video processing system 100 including a general digital video processor 112 and a specialized digital video processor 120, also known as a compression/decompression accelerator 120. The bus structure of remote video processing system 100, including address bus 116 and data bus 118, permits easy interconnection of the components of a multimedia display system without using a host processor bus. For example, the local video data of video camera 127 may be received by video processing system 100 by way of line 129 and captured, digitized, subsampled and scaled by video capture 126. The signals provided by video capture 126 may then be processed by system 100 for transmission to merge logic video by way of system output line 132 and capture 128. All of these operations are performed by remote video processing system 100 without use of a host processor bus or an industry standard bus such as bus 140 which may couple remote system 100 to a host computer by way of bus 138. Because system 100 interfaces a source of video data such as video camera 127 to bus 140, system 100 may be understood to be a remote video interface system 100.
Within video processing system 100 digital video processor 112 performs software processing while accelerator 120 does hardwired functions. Thus video processor 112 includes the devices known to those skilled in the art which are effective to permit execution of program instructions. A digital video processor of this type is taught in "Reconfigurable Interrupt Device", U.S. Ser. No. 08/051,038 filed Apr. 21, 1993, by Farrell et al. and incorporated by reference herein. Nucleus gate array 121 performs the required memory interface functions for integrated system 100. For example, VRAM emulation block 124 makes nucleus gate array 121 function like DRAM with respect to capture 128. Video processing system 100 is therefore limited to the memory configurations supported by nucleus gate array 121.
TABLE I______________________________________1  2  12  4  21  2  1______________________________________
TABLE II______________________________________0    1       1       1     1     1     1     01    2       2       2     2     2     2     11    2       2       2     2     2     2     11    2       2       2     2     2     2     11    2       2       2     2     2     2     11    2       2       2     2     2     2     11    2       2       2     2     2     2     10    1       1       1     1     1     1     0______________________________________
value=d/(2×Q)                                        Equation (1)
R=256/(2×Q)
value=(d×R)&gt;&gt;8
value=255if (dis 1024)
value=d&gt;&gt;3.
d=Q×(2×value +sign(value                       Equations (2)
d=8×value                                            Equations (3)
TABLE III______________________________________1    2       6       7     15    16    28    293    5       8       14    17    27    30    434    9       13      18    26    31    42    4410   12      19      25    32    41    45    5411   20      24      33    40    46    53    5521   23      34      39    47    52    56    6122   35      38      48    52    57    60    6236   37      49      50    58    59    63    64______________________________________
Referring now to FIGS. 3A,B, there are shown encode dataflow 300 for performing the encoding of data within compression/decompression accelerator 120 of the present invention and physical memory buffers 350, 352, 354. Within encode dataflow 300 current image block 302 is applied to motion estimation block 112a for a coding decision in order to permit system 100 to transmit as little data as possible. For this purpose, it will be understood that it is sometimes more efficient to estimate the displacement between one frame and the next and transmit only the displacement. This is understood to be motion estimation encoding. However, if there is a great deal of difference between frames, it is less efficient to transmit the displacement and the block is encoded based only upon itself. This is understood to be intrablock encoding. This determination, and therefore the determination whether to apply loop filter 210, is made by software coding decision block 112a.
It will thus be understood that the operations of both motion estimation block 112a and variable length encode 112b may be performed by this video processor system. Thus the functions of encode dataflow 300 are partitioned as follows: (1) the software functions including motion estimation, Huffman encoding and decoding, as well as memory management operations involving moving blocks of data into and out of memory 114, are performed by video processor 112, and (2) the functions which would be computationally intensive such as discrete cosine transforms, frame add/subtract, quantizing and dequantizing and run length encoding and decoding are hardwired in circuitry within accelerator 120. These partitioned functions of processor 112 and accelerator 120 are synchronized within remote video processing system 100.
The latter group of computationally intensive functions tend to be rather rigidly specified in current standards. Since it is not possible to change them for this reason there is little improvement in visual quality possible in these functions. However, as new algorithms or more powerful video processors become available the performance of the former group of functions may be improved relatively easily. Thus the performance of video processing system 100 may be improved without hardware redesign. Additionally, new operations may be added. For example, pre-filtering and post-filtering operations may be performed by video processor 112 if time permits.
It will be understood that motion estimation coding decision block 112a performs the motion estimation process within encode dataflow 300. Additionally it performs the variable length encode of the local image and the variable length decode of remote compressed bit stream 601. Thus, the functions of block 112a may be performed by a general purpose video processor such as video processor 112. Compression/decompression accelerator 120 of the present invention performs all of the remaining functions of encode dataflow 300 and decode dataflow 600.
It will be understood that video processor 112 must set accelerator pointers 804a,c,d equal to processor pointers 820a,c,d so that accelerator 120 and block 112 may agree regarding where circular buffer 322 begins and ends and where current list 812 ends. Thus, for example, after video processor 112 writes one or more blocks of data, and updates its own write pointer 820e in a post increment manner, it updates accelerator pointer INCEND 804d.
Referring now to FIGS. 10, 11, there are shown forward discrete cosine transform computation flow 1000 and inverse discrete cosine transform computation flow 1100 of the present invention. Discrete cosine transform computation flows 1000, 1100 may be performed by transform blocks 230a,b respectively of dataflows 300, 600. Additionally discrete cosine transform computation flows 1000, 1100 may be performed by selectable transform block 230 under the control of forward/inverse transform control line 228.
&#934;!= C! X!.                                            Equation (4)
In Equation (4) matrix X! is the input data matrix applied to transform block 230 or transform block 230a, matrix C! is the discrete cosine transform matrix, and matrix Φ! is the transformed output matrix which is applied to transform output line 236 by transform block 230.
X!= C!T  &#934;!.                                     Equation (5)
In Equation (5) matrix Φ! is the transformed input matrix received by way of transform input line 223, matrix X! is the output matrix applied to transform output line 234, and matrix C!T is the inverse discrete cosine transform matrix.
The individual coefficients cm of the discrete cosine transform matrix C! may be expressed as: ##EQU1## In Equation (6) N is the order of the discrete cosine transform performed within transform computation flows 1000, 1100 and m and n are the row and column indices, respectively, of the discrete cosine transform matrix C! wherein m and n have the values 0, 1, 2, . . . , N-1. The constant km has a value of one divided by the square root of two if the row index m has a value of zero. The constant Km has the value of one if the row index m is not zero.
Solving Equation (6) when the order N of the discrete cosine transform is eight yields the following discrete cosine transform coefficients cm : ##EQU2## Under these circumstances the discrete cosine transform matrix C! of Equation (4) may be formed in accordance with Equation (6) and Equations (7) as follows: ##EQU3##
It will be understood that eight multiply/accumulate operations are required to perform this transform for each data point within input data matrix X!. Therefore 64×8=512 multiply/accumulate operations are required for a one-dimensional discrete cosine transform. For a two-dimensional discrete cosine transform 1024 multiply/accumulate operations are required.
Forward discrete cosine transform flow 1000 of the present invention executes a fast forward discrete cosine which is a faster and more efficient variation of the transform represented by Equation (4). In forward discrete cosine transform flow 1000, the order N of the transform is eight. It will be understood by those skilled in the art that the transform performed by forward transform flow 1000 is a fast forward transform of the type described with respect to transform block 230a. It is performed by selectable discrete cosine transform block 230 when selectable transform block 230 is in the encode mode. This fast forward transform may be expressed as: ##EQU4## wherein the elements of submatrices CA ! and CB ! are obtained from the discrete cosine transform matrix using sparse matrix factorization techniques known to those skilled in the art and the coefficients of Equations (8) are given by: ##EQU5## The elements of vectors X'! and X"! or submatrices X'! and X"! are formed by respectively adding and subtracting the high order data points of matrix X! and the low order data points of matrix X!.
Transform computation flow 1000, performed by selectable discrete cosine transform block 230, is effective to receive the input data matrix X! and apply the forward discrete cosine transform matrix C! to input data matrix X! to provide the forward transformed matrix Φ! as set forth in Equation (4) and Equations (8). In order to perform these operations the low order data points x0 -x3 of an input word x are selected to form a subword. These data points may be any number of bits wide. The data points of this subword are placed into circular input registers 1006. Similarly, the high order data points x4 -x7 of the input word are selected to form another subword. The data points of this subword are placed into circular input registers 1032.
As data points x0 -x3 are successively applied to addition node 1008 and subtraction node 1036 they are also applied to the input of circular registers 1006 by way of a loop formed by register output line 1004. Similarly, data points x4 -x7 are successively applied to the input of circular registers 1032 by way of a loop formed by register output line 1030 as they are applied to addition node 1008 and subtraction node 1036.
The timing of the presentation of each of these data points is controlled in a manner understood by those skilled in the art to provide the sums x0 +x7, x1 +x6, x2 +x5, and x3 +x4, which are the elements of submatrix X'! of Equations (8), at the output of addition node 1008. In a similar manner the differences x0 -x7, x1 -x6, x2 -x5, x3 -x4, which are the elements of submatrix X"! of Equations (8), are formed at the output of subtraction node 1036. It will be understood that, acting cooperatively, register output lines 1004, 1030 and nodes 1008, 1036 operate as a conventional butterfly adder operating upon a series of pairs of input data points. In a conventional butterfly adder, two inputs are received and two outputs are provided, one output being the sum of the inputs, the other the difference. In the case of flow 1000, these inputs proceed through input circular buffers 1004, 1032 to produce the values of submatrices X'! and X"!. The output of nodes 1008, 1036 are then successively applied to multiplication nodes 1014, 1040 within transform computation flow 1000.
The coefficients of submatrix CA ! are applied to submatrix X'! received by multiplication node 1014 from addition node 1008 for multiplication within node 1014 in accordance with Equations (8). The coefficients of the submatrix CA ! are applied to multiplication node 1014 by coefficient register 1010. The matrix partial product terms thus formed by multiplication node 1014 are then applied to addition node 1016 within transform computation flow 1000.
In a similar manner submatrix X"! received by multiplication node 1040 from subtraction node 1036 is multiplied within node 1040 by the coefficients of submatrix CB ! in accordance with Equations (8). The coefficients of submatrix CB ! are applied to multiplication node 1040 by coefficient register 1042. The matrix partial product terms thus formed by multiplication node 1040 are applied to addition node 1044 within transform computation flow 1000.
The output of register blocks 1018, 1046 are also applied to register blocks 1024, 1050, respectively, within forward transform computation flow 1000 which may be implemented by selectable discrete cosine transform block 230. It will be understood that the output of register block 1024, which appears on register output line 1026, is the set of even numbered transformed data points .o slashed.6, .o slashed.4, .o slashed.2, .o slashed.0 of Equations (8). Additionally, it will be understood that the output of register block 1050, which appears on register output line 1052, is the set of odd numbered transformed data points .o slashed.5, .o slashed.3, .o slashed.1 of Equations (8). The values of register output lines 1026, 1052 are applied to flow output multiplexer 1054 in order to be multiplexed, reordered and applied by way of the forward flow output line 236. It is believed that performing a fast forward discrete cosine transform using the system and method of computation flow 1000 provides a transformed output signal several clock cycles faster than the known prior act.
In a similar manner inverse discrete cosine transform computation flow 1100 of the present invention executes a fast inverse discrete cosine transform wherein the order N of the transform is eight. It will be understood by those skilled in the art that the transform performed by forward transform computation flow 1100 is the type of transform performed by selectable discrete cosine transform block 230 when transform block 230 is in the decode mode. This inverse discrete cosine transform matrix C!T may be mathematically expressed as: ##EQU6## wherein the submatrices CA !T and CB !T are obtained from the discrete cosine transform matrix using sparse matrix factorization techniques.
Computation flow 1100 of selectable discrete cosine transform block 230 is effective to receive an inverse transformed matrix Φ! and apply an inverse discrete cosine transform matrix C!T to provide an output data matrix X!. Computation flow 1100 is adapted to be a much faster variation of the transform set forth in Equation (4) and Equations (9). In order to perform these operations the even transformed data points .o slashed.0, .o slashed.2, .o slashed.4, .o slashed.6, of the values of transformed matrix Φ! are placed in circular input registers 1106 and the odd transformed data points .o slashed.1, .o slashed.3, .o slashed.5, .o slashed.7 reside in circular input registers 1132.
As even transformed data points .o slashed.0, .o slashed.2, .o slashed.4, .o slashed.6 residing in circular input registers 1104 are applied to multiplication node 1114 they are simultaneously applied to the input of circular registers 1106 by register output line 1104. As odd numbered transformed data points .o slashed.1, .o slashed.3, .o slashed.5, .o slashed.7, of circular input registers 1132 are applied to multiplication node 1140 they are also simultaneously applied back to the input of circular registers 1130 by way of a loop formed by register output line 1130.
The value applied to multiplication node 1114 from input registers 1106 is multiplied within multiplication node 1114 by the coefficients of submatrix CA !T in accordance with Equations (8). The coefficients submatrix CA !T are applied to multiplication node 1114 by coefficient register 1110. The product formed by multiplication node 1114 is then applied to addition node 1116.
In a similar manner the value received by multiplication node 1140 from input registers 1130 is multiplied within node 1140 by the coefficients of submatrix CB !T in accordance with Equations (8). The coefficients of submatrix CB !T are applied to multiplication node 1140 from coefficient register 1142. The product thus formed by multiplication node 1140 is applied to addition node 1144.
It will be understood that the output of register block 1124, which appears on register output line 1126, includes the low order bits x0, x1, x2, x3 of Equations (9). Additionally, it will be understood that the output of register block 1150, which appears on line 1152, includes the high order data points x4, x5, x6, x7 of Equations (9). The values of lines 1126, 1152 are applied to output multiplexer 1154 in order to be multiplexed and reordered to be applied to frame add 235 by way of output line 234.
Within selectable discrete cosine transform block 230 data from block memory 1242 is applied by way of data bus 1238 to input data buffer/shifter 1212. Input data buffer/shifter 1212 is effective to arrange and order input data either as shown with respect to input circular registers 1006, 1032 during a forward transform or as shown with respect to input circular buffers 1106, 1132 during an inverse transform. This arranging and ordering of the input data thus provides input matrices X'! and X"! of Equations (8) or input matrices Φ'! and Φ"! of Equations (9) depending upon whether a forward transform or an inverse transform is performed by selectable discrete cosine transform block 230.
When selectable transform block 230 is in the inverse mode, transform control line 228 causes array input multiplexer 1208 to select the data of multiplexer line 1214 and apply the data of line 1214 to multiplier/accumulator array 1210. Thus the input data from buffer/shifter 1212 is applied substantially directly to multiplexer/accumulator array 1210 when selectable transform block 230 is in the decode mode. The coefficient matrices CA !T and CB !T are then applied to array 1210 from coefficient registers 1204 in order that array 1210 may perform the operations of Equations (9) as previously described with respect to inverse computation flow 1100. Transformed data in output data buffer 1230 may be transmitted to block memory 1242 by way of bus 1238.
For example, the same hardware elements within transform block 230 may serve both as registers 1006 and as registers 1106, as registers 1032 and registers 1132. These registers together may form buffer 1212. Similarly, the same hardware elements within transform block 230 may serve both as adder 1008 and as adder 1122, as subtractor 1036 and subtractor 1148. These arithmetic elements operating cooperatively may serve as butterfly adder 1218. Likewise coefficient matrices CA ! and CB !T may be the same hardware circuitry which is merely accessed differently depending on the mode of selectable transform block 230.
Referring now to FIG. 13, there is shown stepping direction chart 1300. Stepping direction chart 1300 represents a center position Pc surrounded by four positions P1 -P4 along with a plurality of stepping directions 1302-1316. Stepping directions 1302-1316 represent the directions that positions Pc, P1 -P4 may move from one frame to another frame during a display of remote video processor system 100. For example, if positions Pc, P1 -P4 of stepping direction chart 1300 move directly upwards from one frame to the next, stepping direction 1302 represents their displacement. If positions Pc, P1 -P4 move directly to the right, their motion is represented by stepping directions 1306. Stepping direction 1304 represents the motion of position Pc, P1 -P4 when the positions of stepping direction chart 1300 move to the upper right. In the manner, eight different directions are represented by stepping directions 1302-1316 of stepping direction chart 1300.
Referring now to FIGS. 14A-C, there is shown a flow chart representation of motion estimation method 1400. Also show is a representation of motion estimation method 1440 which is an alternate embodiment of motion estimation method 1400. The function of motion estimation methods 1400, 1440 is finding the best match for a target region during a predetermined period of time. Motion estimation methods 1400, 1440 may be applied to positions Pc, P1 -P4 of motion vector chart 1300 to determine which stepping direction 1302-1316 best represents the motion of positions Pc, P1 -P4 from one frame to another. It will be understood that both motion estimation methods 1400, 1440 may be used in encoding dataflow 300 of remote video processing system 100 of the present invention.
Operation of motion estimation method 1400 begins with a determination in decision 1402 whether center position Pc or position Pc of the current frame is a better match with the center position of the previous frame. This determination, as well as the determination of each of the remaining candidate positions tested in method 1400, requires a comparison of two hundred fifty-six pixels values of a sixteen-by-sixteen pixel block in the preferred embodiment of encode dataflow 300. If position P1 is a better match than center position Pc the best horizontal position PH is determined to be position P1 as shown in block 1404 by motion estimation method 1400.
If position P1 is not a better match than position Pc a determination is made in decision 1406 whether position P2 is a better match with the center position of the previous frame than center position Pc of the current frame. If position P2 is a better match than center position Pc the best horizontal position PH is made equal to position P2 as shown in block 1408. If neither position P1 nor position Pc is a better match than center position Pc then center position Pc is selected as the best horizontal position VH as shown in block 1410. Thus, when operation of motion estimation method 1400 arrives at point 1411, a determination has been made which of the three horizontal positions Pc, P1, P2 of the current frame has the greatest improvement from the center position. The best horizontal position PH is the one determined to be the best match.
A determination is then made within motion estimation method 1400 whether position P3 is a better match than center position Pc in decision 1412. If position P3 is a better match it is selected as the best vertical position Pv as shown in block 1414. If position P3 is not a better match than center position Pc a determination is made in decision 1416 whether position P4 is a better match than center position Pc. If position P4 is a better match, as determined in decision 1416, it is selected as the best vertical position Pv as shown in block 1418. If neither position P3 nor position P4 1416, is a better match than center position Pc, as determined in decisions 1412, center position Pc is selected as the best vertical position Pv as shown in block 1420. Thus, when operation of motion estimation method 1400 arrives at point 1421 the best vertical position Pv has been set equal to either center position Pc, position Ps or position P4. Operation of motion estimation method 1400 then proceeds, by way of off-page connector 1422, to on-page connector 1423.
When execution of motion estimation method 1400 arrives at block 1424, by way of off-page connector 1422 and on-page connector 1423, a stepping direction 1302-1316 is determined by motion estimation method 1400. This stepping direction 1302-1316 is based upon best horizontal position PH, as determined by blocks 1404, 1408, 1410 depending on whether the best horizontal match is position Pc, P1 or P2. Additionally, the stepping direction 1302-1316 determined in block 1424 is based upon the best vertical position Pv as determined in blocks 1414, 1418, 1420 depending on whether the best vertical match was position Pc, P3 or P4.
For example, if the best horizontal position PH selected by motion estimation method 1400 in decision 1402 is position P1 and the best vertical position Pv is center position Pc, as determined in block 1420, the stepping direction 1302-1316 determined in block 1424 is stepping direction 1314. This results from horizontal motion to the left and no vertical motion. If, for example, position P1 is selected in decision 1402 and position P4 is selected as the best vertical position Pv in decision 1426, stepping direction 1312 is determined in block 1424. This results from horizonal motion to the left and downward vertical motion.
If center position Pc is determined to be both the best horizontal position PH and the best vertical position Pv within motion estimation method 1400, it will be understood that the best match from one frame to the next may be achieved with no motion. This indicates no motion of positions Pc, P1 -P4 from one frame to the next. Thus there are nine possible outcomes of motion estimation method 1400 even though only eight stepping directions 1302-1316 are defined. The selection of stepping direction 1302-1316, in accordance with best vertical positions Pv and best horizontal position VH, is set forth in Table VI.
TABLE VI______________________________________                SteppingPv        PH                Direction______________________________________Pc        Pc                N/APc        P1                1314Pc        P2                1306P3        Pc                1302P3        P1                1316P3        P2                1304P4        Pc                1310P4        P1                1312P4        P2                1308______________________________________
When stepping direction 1302-1316 is determined in block 1424 of motion estimation method 1400, the operation of motion estimation method 1400 includes movement in the direction of stepping direction 1302-1316 formed therein as shown in block 1426. A determination is then made in decision whether the movement indicated in block 1426 results in an improvement in the least mean square error from one frame to the next. If no improvement is obtained, it may be determined that the least mean square error has been minimized and that the best match has been found. When the best match has been found, as indicated by decision 1428, execution of motion estimation method 1400 proceeds to end terminal 1432 by way of path 1429. At this point motion estimation method 1400 may have determined the minimum error between the image to be matched and the reconstructed image.
If movement in the direction indicated stepping direction 1302-1316 as determined in block 1424 results in improvement of the match between frames, as indicated for example by determining the least mean square error, a determination is made in decision 1430 whether motion estimation method 1400 has timed out. The time out duration of decision 1430 may be based upon a predetermined number of processor cycles or upon a predetermined amount of time during which video processing system 100. This predetermined number of processor cycles is the number allotted for video processor 112 to search for the best match between the images of one frame and another. During this predetermined duration, motion estimation method 1400 may literatively approach the best match.
If the match between frames continues to improve and there is more time to search for the best match, as determined in decisions 1428, 1430, execution of motion estimation method 1400 proceeds by way of off-page connector 1434 to on page connector 1401. From on-page connector 1401, motion estimation method again searches for the best horizontal match PH, in decisions 1402, 1406, and the best vertical match Pv, in decisions 1412, 1416, as previously described. When best horizontal and vertical positions PH, Pv are determined by decisions 1402, 1406, 1412, 1416, a new stepping direction 1302-1316 is determined in block 1424 and a determination is again made whether movement in the direction indicated by new stepping direction 1302-1316 results in an improvement.
Motion estimation method 1440 is an alternate embodiment of motion estimation method 1400 as previously described. In this alternate embodiment of motion estimation method 1400, execution proceeds from off-page connector 1422 to on-page connector 1442 of method 1440. When execution of motion estimation method 1440 proceeds by way of on-page connector 1442, a determination is made in decision 1444 whether center position Pc provides the best match in both the horizontal and vertical directions.
TABLE VII______________________________________   START:   Try P1   If P1 is worse than Pc     try P2   Try P3   If P3 is worse than Pc     try P4   Determine a stepping direction from     best of P1, P2, Pc     best of P3, P4, Pc   Step in stepping direction until     no more improvement with time out     check.   Go to START______________________________________
Improved dequantization system 1500 assumes that value ≠0 in Equations (2) because zero is not a legal input into system 1500. Dequantization system 1500 also assumes that 1≦Q≦31 and -127 ≦ value ≦+127. In order to develop the computational flow of dequantization system 1500, dequantization Equations (2) may be rewritten as follows:
Q even
d=Q*(2*value+sign(value))-sign(value)                      Equations (10)
Q odd
d=Q*(2*value+sign(value)).
It will be seen from Equations (10) that the two inputs into a multiplier in a system performing dequantization according to Equations (10) are (a) Q, and (b) 2*value+sign(value). It will be understood by those skilled in the art that the multiplier input quantity 2*value+sign(value), as set forth in Equations (10), must be between negative two hundred fifty-five and positive two hundred and fifty-five. Furthermore, it should be noted that the quantity may be expressed as set forth in Equations (11). It will also be understood that 2*value must be even and that 2*value+1 must therefore be odd. Likewise 2*(value-1) must be even and 2*(value-1)+1 must be odd.
sign(value)=+1d=2*value+1                                  Equations (11)
sign(value)=+1d=2*(value-1)+1
The result of the multiplication Q*(2*value+sign(value)) of Equations (10) is set forth in Equations (12) ##EQU7##
Thus the eight bit input, value, is received by dequantization system 1500 by way of dequantization input bus 1502 and applied to conditional decrementing device 1506. The borrow input of value of input bus 1502 is applied to sign bit input 1510 of decrementing device 1506. Conditional decrementing device 1506 either decrements the value received by way of input bus 1502 or passes it through unchanged depending on the sign bit of the input value as received at sign bit input 1510. Conditional decrementing device 1506 applies the result, either decremented or passed straight through, to output bus 1514.
At point 1534 of dequantization system 1500 the quantity multout, provided by multiplication device 1526, is shifted one position to the left to provide the quantity multout/2 as set forth in Equations (12). This value is then applied, byway of input bus 1538, to conditional decrementing device 1546. Conditional decrementing device 1546 receives a control signal at control input 1542 by way of control line 1544. The control signal of line 1544 is high when value >0 and Q is even.
Conditional decrementing device 1546 either decrements the quantity received on input bus 1538 or passes it through unchanged according to this control signal as previously described with respect to conditional decrementing device 1506 and sign bit input 1510. The thirteen bit result of this conditional decrement operation is provided by decrementing device 1546 at output bus 1550. A hardwired value of one is applied to the signal of output bus 1550 at point 1554. The fourteen bit result is applied to block 1558 where it may be clipped, and where special cases such as zero value inputs and interblock DC encoding may be handled.
Referring now to FIG. 16, there is shown a more detailed representation of the various memories and controls associated with bus interface 200. Bus interface 200 is provided with four buffer memories 204, 206, 240, 248. Memories 204, 206, 240, 248 buffer the incoming data to be processed by compression/decompression system 120 as well as the results of operations performed by compression/decompression system 120 which are written to other devices within remote video processing system 100 using buses 116, 118.
Zig-zag memory 248 stores the run/value pairs generated cooperatively by zig-zag/quantization block 238 and run length encode block 246 as previously described. The run/value pairs are read from zig-zag memory 248 by bus interface 200 for writing data bus of remote video processing system 100. Memory 248 contains its own address incrementer.
Within add/subtract unit 1700, data received from memory 204 is zero extended and complemented by blocks 1702, 1704. The results are multiplexed in block 1710 according to whether frame differencing is being performed. The output result of block 1710 is multiplexed in block 1712 with the complement of data from selectable transform device 230 by way of complement block 1706. This output result is added in block 1716 with the multiplexed output result of multiplexer block 1714. The output of multiplexer block 1714 may be zero or zero extended data from block 1708.
To prevent quantization distortion of transform coefficient amplitudes causing arithmetic overflow within encode dataflow 300 and decode dataflow 600, a clipping or saturate function is included in block 1718 of add/subtract unit 1700. The saturation function is applied to a reconstructed image which is formed by summing the prediction and the prediction error during the encoding mode of compression/decompression accelerator 120. Results are saturated between zero and two hundred fifty-five.
When the filtering process is begun, vertical filter state 1844 is entered by way of pathway 1832. The first pass, or the vertical pass, of loop filter 210 scans an eight-by-eight input matrix of pixel values starting from the top of the matrix and proceeding to the bottom. These vertical scans start at the top left corner of the matrix which may be designated location 0,0. Three buffer registers 1902, 1904, 1906 are used to store the pixels at the input of filter adder 1908. When first register 1902 and second register 1904 are filled, processing by loop filter 210 begins and the partially processed values are written back loop filter memory 206 by way of data line 209. At the end of the vertical state 1844, loop filter memory 206 is filled with sixty-four ten bit intermediate values. The bottom two bits of each intermediate value are the fractional bits which maintain precision for the next pass of loop filter 210.
Referring now FIGS. 19, 20, there are shown adder unit 1900 and address unit 2000 of loop filter 210. Address generator 2002 within address unit 2000 generates all the addressing for loop filter memory 206 and current image memory 204 accesses. The addresses are generated sequentially and may range from 00H to 3fH. Address generator has three main functions. The first function of address generator 2000 is to generate horizontal pass addresses for loop filter 210. Generator 2002 produces sequentially ascending addresses from 00H to 3fH during the horizontal pass. Whenever a corner or side address is processed by adder 1900 the output of adder 1900 is ignored and the contents of 1904 are copied instead. The addresses generated are used to read from loop filter memory 206 and to read from current image memory 204.
Another function of address generator 2002 is generating read addresses of loop filter memory 206 when selectable loop filter 210 is bypassed within accelerator 120. When this occurs adder unit 1900 is disabled and the addresses generated are used to read of loop filter memory 206 and current image memory 204. The data is sent to the frame add/subtract block 1700 after being sent through disabled adder 1908. The addresses generated are all in the horizontal read mode, from left to right.
A determination is made when a corner, a top row, or a bottom row is being processed dividing the output of counter 2002 into two sets of three bits each using buses 2003. Buses 2003 are applied to multiplexer 2004 and wrapped. Thus, in summary, address generator 2000 of selectable filter 310 includes counter 2002, two registers 2006, 2008 and byte-swap multiplexer 2004. Address register 200 generates the addresses in loop filter memory 206 from which the filter input data is read from and the addresses where the intermediate filtered result are stored. It also generates addresses for reading from current image memory 204, for writing to selectable discrete cosine transform device 230, and for writing decoded image memory 240. During the vertical pass of loop filter 210, the upper three bits of the address are swapped with the lower three bits by multiplexer 2004. This results in a top-to-bottom read instead of a left-to-right read of the eight-by-eight pixel matrix being processed by loop filter 210.
Adder unit 1900 includes three input twelve bit adder 1908. In addition to three twelve bit inputs adder 1908 has a single output. Two of the inputs of adder 1908 are connected directly to the outputs of registers 1902, 1906 and the third input of adder 1908 is from register 1904. Register 1904 contains the pixel that is currently being processed. Registers 1902, 1904, 1906 may be standard ten bit registers. During the first pass, they may contain only eight bit values with two extra bits padded on. The output of registers 1902, 1906 go directly to adder 1908. The output register 1904 is applied to a multiply-by-two operator before being applied to adder 1908. The multiply operation is done by mapping the bits up by one.
Controller 213 is responsible for handling the various inputs from other units and decoding them to provide controls for adder unit 1900 and address generator 2000. For example, controller 213 transmits control signals to address generator 2000 to start counting. It also enables and disables adder 1908. Based on control signals received by controller 213, controller 213 may put loop filter 210 into the bypass mode. Filter 210 then acts as an address generator for frame add/subtract unit 1700. Loop filter controller 213 of selectable loop filter 210 includes its own state machine and some miscellaneous logic. The state machine of controller 213 sequences the various filter and read/write processes.
Referring now to FIGS. 21A,B, there are shown data testing discrete cosine transform systems 2100a,b. Discrete cosine transform systems 21a,b are alternate embodiments of a transform system which may be provided within compression/decompression accelerator 120. Within data testing discrete cosine transform systems 2100a,b, a determination is made of the input data which is applied to transform systems 2100a,b. When predetermined input data is detected, the discrete cosine transform operations are not performed within transform systems 2100a,b. In these cases pre-determined known output results corresponding to the detected input data are applied to output line 236 of data testing discrete cosine transform systems 2100a,b. It will be understood that the data testing and the selective performing of the discrete cosine transform within data testing discrete cosine transform systems 2100a,b may be performed for both forward discrete cosine transforms and inverse discrete cosine transforms. Furthermore, the data testing of the present invention may be performed within selectable discrete cosine transform device 230 of accelerator 120.
Within data testing discrete cosine transform systems 2100a,b, input data is received at input bus 2102 and applied to data storage 2106. The input values stored in data storage 2106 are applied to discrete cosine transform computation block 2114 by way of bidirectional data bus 2110. Discrete cosine transform computation block 2114, after performing the discrete cosine transform upon the data received by way of data bus 2110, applies the results of the transform back to data storage 2106 by way of bidirectional data bus 2110. The transformed data may later be applied to quantization block 238 by way of output line 236 as previously described.
When input data within transform system 2100a is applied from data storage 2106 to transform computation block 2114 by way of bidirectional data bus 2110, detection circuit 2126 monitors the data which is received by block 2114 for transformation. Detection circuit 2126 detects input data which produces a known result when operated upon in accordance with transforms such as those described by Equation (4) or Equation (5).
For example, detection circuit 2126 may detect an input matrix on data bus 2110 having rows or columns formed entirely of zeros. In this case no transform is necessary. Additionally, detection circuit 2126 may detect matrices wherein all of the values of an entire row are non-zero and equal to each other or all of the values of an entire column are equal to each other. This corresponds to the DC case wherein a known constant output value may be inserted. Additionally, detection circuit 2126 may be adapted to detect an entire block of constant data being applied by data storage 2106 to transform computation block 2114. Due to the quantization operations, and other operations normally performed in encoding and decoding processes, the cases detected by detection circuit 2126 may occur quite frequently.
As previously described, when data corresponding to the cases detected by detection circuit 2126 is applied to discrete cosine transform computation block 2114 the results of applying transforms such as Equation (4) or Equation (5) are known. Thus, these results may be pre-calculated and stored within transform systems 2100a,b. Transform system 2100a may store output values corresponding to any number of differing cases which detection circuit 2126 may be adapted to detect.
When these cases are detected, detection circuit 2126 applies a detection signal to control circuit 2130. In response to this detection signal control circuit 2130 applies a control signal to discrete cosine transform computation block 2114 by way of control line 2134. The control signal transmitted by way of control line 2134 instructs discrete cosine transform computation block 2114 not to perform the transform operations. Additionally, the signal of control line 2134 may provide computation block 2114 with the pre-calculated output results corresponding to the input case detected by detection circuit 2126. In an alternate embodiment of data testing discrete cosine transform system 2100a, discrete cosine transform computation block 2114 may store these known output values itself and provide them in response to the control signal from control circuit 2130.
The transform output results, which are obtained without actually performing the discrete cosine transform operations within computation block 2114, are then applied to data storage 2106 by way of bidirectional bus 2110. It is believed that this method saves approximately ten to twelve cycles compared with actually performing the computations within transform computation block 2114 when detection circuit 2126 detects the cases described above.
Within data testing discrete cosine transform system 2100b, detection circuit 2122 is applied to input bus 2102 rather than bidirectional data bus 2110. Detection circuit 2122 may be adapted to detect any of the cases detected by detection circuit 2126. When detection circuit 2122 detects these cases on input bus 2102 of system 2100b, it applies a control signal to control circuit 2130 as previously described with respect to detection circuit 2126. In a similar manner detection circuit 2122 may store the pre-calculated output results and apply them to computation block 2114 by way of control lines 2134. Alternatively, control circuit 2122 may signal computation block 2114 to provide pre-calculated output values which are stored in block 2114. Within system 2100b each row and each column of an input matrix may be tagged to indicate its status as determined by detection circuit 2122 during processing. Thus a conventional eight by eight input matrix received by transform system 2200b may be enlarged to a nine by nine matrix wherein the positions within one row and one column may be used to indicate, for example, whether the row or column is formed of zeros or other equal values.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4706265 *Oct 30, 1985Nov 10, 1987Nec CorporationCode converting system and method for band compression of digital signalsUS4780761 *Jun 2, 1987Oct 25, 1988Eastman Kodak CompanyDigital image compression and transmission system visually weighted transform coefficientsUS4849810 *Jun 2, 1987Jul 18, 1989Picturetel CorporationHierarchial encoding method and apparatus for efficiently communicating image sequencesUS5019901 *Mar 26, 1990May 28, 1991Matsushita Electric Industrial Co., Ltd.Image motion vector sensor for sensing image displacement amountUS5073820 *Oct 23, 1990Dec 17, 1991Olympus Optical Co., Ltd.Image data coding apparatus and coding methodUS5113255 *May 11, 1990May 12, 1992Matsushita Electric Industrial Co., Ltd.Moving image signal encoding apparatus and decoding apparatusUS5126841 *Oct 15, 1990Jun 30, 1992Matsushita Electric Industrial Co., Ltd.Motion compensated prediction interframe coding systemUS5150209 *May 10, 1991Sep 22, 1992Picturetel CorporationHierarchical entropy coded lattice threshold quantization encoding method and apparatus for image and video compressionUS5151784 *Apr 30, 1991Sep 29, 1992At&T Bell LaboratoriesMultiple frame motion estimationUS5173773 *Nov 8, 1991Dec 22, 1992Victor Company Of Japan, Ltd.Moving picture signal progressive coding systemUS5198901 *Sep 23, 1991Mar 30, 1993Matsushita Electric Corporation Of AmericaDerivation and use of motion vectors in a differential pulse code modulation systemUS5237410 *Nov 29, 1991Aug 17, 1993Matsushita Electric Industrial Co., Ltd.Video signal encoding apparatus utilizing control of quantization step size for improved picture qualityUS5251028 *Aug 8, 1991Oct 5, 1993Matsushita Electric Corporation Of AmericaApparatus for reducing quantization artifacts in an interframe hybrid coding system with motion compensationUS5329318 *May 13, 1993Jul 12, 1994Intel CorporationMethod for optimizing image motion estimationUS5436666 *May 21, 1993Jul 25, 1995Intel CorporationLimited-domain motion estimation/compensation for video encoding/decodingUS5448310 *Apr 27, 1993Sep 5, 1995Array Microsystems, Inc.Motion estimation coprocessor* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS5923375 *Feb 13, 1997Jul 13, 1999Sgs-Thomson Microelectronics S.R.L.Memory reduction in the MPEG-2 main profile main level decoderUS5964864 *Apr 3, 1996Oct 12, 1999Ricoh CompanyInformation processing apparatus for separately processing code information and pattern informationUS6002441 *Oct 28, 1996Dec 14, 1999National Semiconductor CorporationAudio/video subprocessor method and structureUS6226410 *Jun 30, 1997May 1, 2001Intel CorporationMethod and apparatus for providing image and video coding with iterative post-processing using transmitted step size informationUS6353680 *Oct 4, 1999Mar 5, 2002Intel CorporationMethod and apparatus for providing image and video coding with iterative post-processing using a variable image model parameterUS6408029Mar 22, 1999Jun 18, 2002Intel CorporationMethod and apparatus for simplifying real-time data encodingUS6501799 *Aug 4, 1998Dec 31, 2002Lsi Logic CorporationDual-prime motion estimation engineUS6574278Mar 22, 1999Jun 3, 2003Intel CorporationMethod and apparatus for performing real-time data encodingUS6614845 *Sep 30, 1997Sep 2, 2003Verizon Laboratories Inc.Method and apparatus for differential macroblock coding for intra-frame data in video conferencing systemsUS6760478 *Jul 10, 2000Jul 6, 2004Hewlett-Packard Development Company, L.P.Method and apparatus for performing two pass quality video compression through pipelining and buffer managementUS6876704 *Jun 6, 2001Apr 5, 2005Qualcomm, IncorporatedApparatus and method for encoding and computing a discrete cosine transform using a butterfly processorUS6904174 *Dec 11, 1998Jun 7, 2005Intel CorporationSimplified predictive video encoderUS6963613 *Apr 1, 2002Nov 8, 2005Broadcom CorporationMethod of communicating between modules in a decoding systemUS7006698Nov 25, 2003Feb 28, 2006Hewlett-Packard Development Company, L.P.Method and apparatus for compressing a video imageUS7046734Jan 8, 2003May 16, 2006Intel CorporationMethod and apparatus for performing real-time data encodingUS7180946 *Sep 20, 2002Feb 20, 2007Koninklijke Philips Electronics N. V.Circuit for filtering video data in the frequency domainUS7215384Mar 22, 1999May 8, 2007Intel CorporationMethod and apparatus for simplifying field prediction motion estimationUS7231091May 16, 2005Jun 12, 2007Intel CorporationSimplified predictive video encoderUS7263127Mar 22, 1999Aug 28, 2007Intel CorporationMethod and apparatus for simplifying frame-based motion estimationUS7428339 *Nov 7, 2003Sep 23, 2008Arcsoft, Inc.Pseudo-frames for MPEG-2 encodingUS7649939Sep 20, 2004Jan 19, 2010Qualcomm IncorporatedApparatus and method for decoding and computing a discrete cosine transform using a butterfly processorUS8009174 *Mar 15, 2006Aug 30, 2011Koninklijke Philips Electronics N.V.Processing a data array with a meandering scanning order using a circular buffer memoryUS8031777Mar 21, 2006Oct 4, 2011Apple Inc.Multipass video encoding and rate control using subsampling of framesUS8233535Aug 4, 2006Jul 31, 2012Apple Inc.Region-based processing of predicted pixelsUS8253750 *Jul 8, 2010Aug 28, 2012Nvidia CorporationDigital media processorUS8295343Jun 23, 2006Oct 23, 2012Apple Inc.Video bit rate control methodUS8355447 *Dec 19, 2007Jan 15, 2013Vixs Systems, Inc.Video encoder with ring buffering of run-level pairs and methods for use therewithUS8780997Aug 4, 2006Jul 15, 2014Apple Inc.Regulation of decode-side processing based on perceptual maskingUS9049451Jul 30, 2012Jun 2, 2015Apple Inc.Region-based processing of predicted pixelsUS9201899 *Sep 11, 2012Dec 1, 2015Panasonic Intellectual Property Management Co., Ltd.Transposition operation device, integrated circuit for the same, and transposition methodUS9706201May 27, 2015Jul 11, 2017Apple Inc.Region-based processing of predicted pixelsUS20020181027 *Jun 6, 2001Dec 5, 2002Larocca JudithApparatus and method for encoding and computing a discrete cosine transform using a butterfly processorUS20030076882 *Sep 20, 2002Apr 24, 2003Jean GobertCircuit for filtering video data in the frequency domainUS20030118117 *Jan 8, 2003Jun 26, 2003Mcveigh Jeffrey S.Method and apparatus for performing real-time data encodingUS20030185305 *Apr 1, 2002Oct 2, 2003Macinnis Alexander G.Method of communicating between modules in a decoding systemUS20040131267 *Nov 25, 2003Jul 8, 2004Adiletta Matthew JamesMethod and apparatus for performing quality video compression and motion estimationUS20040179599 *Mar 13, 2003Sep 16, 2004Motorola, Inc.Programmable video motion accelerator method and apparatusUS20040190625 *Mar 13, 2003Sep 30, 2004Motorola, Inc.Programmable video encoding accelerator method and apparatusUS20050100231 *Nov 7, 2003May 12, 2005Xingguo WangPseudo-frames for MPEG-2 encodingUS20050265615 *May 16, 2005Dec 1, 2005Michael KeithSimplified predictive video encoderUS20050276329 *Aug 15, 2005Dec 15, 2005Adiletta Matthew JMethod and apparatus for performing motion compensation on video dataUS20060056517 *Nov 8, 2005Mar 16, 2006Macinnis Alexander GMethod of communicating between modules in a decoding systemUS20070116117 *Jun 15, 2006May 24, 2007Apple Computer, Inc.Controlling buffer states in video compression coding to enable editing and distributed encodingUS20070116124 *Aug 4, 2006May 24, 2007Apple Computer, Inc.Regulation of decode-side processing based on perceptual maskingUS20070116126 *Mar 21, 2006May 24, 2007Apple Computer, Inc.Multipass video encoding and rate control using subsampling of framesUS20070116437 *Aug 4, 2006May 24, 2007Apple Computer, Inc.Region-based processing of predicted pixelsUS20070252732 *Apr 30, 2007Nov 1, 2007Tomo KitaguchiVideo reproduction apparatusUS20080215644 *Mar 15, 2006Sep 4, 2008Koninklijke Philips Electronics, N.V.Processing A Data Array With A Meandering Scanning Order Using A Circular Buffer MemoryUS20090161764 *Dec 19, 2007Jun 25, 2009Xu Gang ZhaoVideo encoder with ring buffering of run-level pairs and methods for use therewithUS20140003742 *Sep 11, 2012Jan 2, 2014Takashi NishimuraTransposition operation device, integrated circuit for the same, and transposition methodCN101147396BMar 15, 2006Dec 14, 2011皇家飞利浦电子股份有限公司使用循环缓存存储器以曲折扫描顺序处理数据阵列的方法和设备WO2006100625A1 *Mar 15, 2006Sep 28, 2006Koninklijke Philips Electronics N.V.Processing a data array with a meandering scanning order using a circular buffer memory* Cited by examinerClassifications U.S. Classification375/240.25, 375/E07.1, 375/E07.256, 375/240.03, 375/240.14, 375/E07.093, 375/E07.211, 375/240.2, 375/E07.105International ClassificationH04N7/36, H04N7/50, H04N7/26Cooperative ClassificationH04N19/51, H04N19/61, H04N19/43, H04N19/42European ClassificationH04N7/26M2, H04N7/26L, H04N7/26L4, H04N7/36C, H04N7/50Legal EventsDateCodeEventDescriptionNov 18, 1993ASAssignmentOwner name: INTEL CORPORATION, OREGONFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HARNEY, KEVIN;KELLY, MICHAEL S.;LOESER, GARY;REEL/FRAME:006768/0279;SIGNING DATES FROM 19930727 TO 19931004Apr 19, 2001FPAYFee paymentYear of fee payment: 4May 4, 2005FPAYFee paymentYear of fee payment: 8Apr 29, 2009FPAYFee paymentYear of fee payment: 12RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services