Source: https://patents.google.com/patent/US20030206656A1/en
Timestamp: 2019-04-26 12:02:14
Document Index: 482095499

Matched Legal Cases: ['arts 2102', 'art 2101', 'arts 2105', 'arts 2101', 'arts 2107', 'art.\n2', 'art.\n5', 'art.\n8']

US20030206656A1 - Method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part - Google Patents
Method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part Download PDF
US20030206656A1
US20030206656A1 US09/800,633 US80063301A US2003206656A1 US 20030206656 A1 US20030206656 A1 US 20030206656A1 US 80063301 A US80063301 A US 80063301A US 2003206656 A1 US2003206656 A1 US 2003206656A1
US09/800,633
US6925209B2 (en
2001-02-15 Priority to US09/784,928 priority Critical patent/US6898323B2/en
2001-03-06 Application filed by Schwartz Edward L., Gormish Michael J., Martin Boliek, Wu Kok Gi filed Critical Schwartz Edward L.
2001-03-06 Priority to US09/800,633 priority patent/US6925209B2/en
2003-11-06 Publication of US20030206656A1 publication Critical patent/US20030206656A1/en
2005-08-02 Publication of US6925209B2 publication Critical patent/US6925209B2/en
This is a divisional of application Ser. No. ______, filed on Feb. 15, 2001, entitled “A Memory Usage Scheme for Performing Wavelet Processing,” and assigned to the corporate assignee of the present invention.[0001]
The present invention relates to the field of compression and decompression; more particularly, the present invention relates to outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part. [0002]
A method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part is described. In one embodiment, the method comprises coding of each tile of an input codestream, outputting packets of each tile as part of an output codestream as a first complete tile-part, storing remaining layers of each tile in a buffer, and outputting additional packets of each tile from the remaining layers as a second complete tile-part. [0014]
FIG. 23 illustrates an example in which transform has 5 levels and the data is divided up into layers [0040] 0-3.
Similarly, the results of applying the wavelet transform to two rows of LL coefficients at level [0066] 1 (solid lines) are the coefficients in two rows namely LL coefficients (331), LH coefficients (332), HL coefficients (333) and HH coefficients (334) at level 2. The width and height for level 2 coefficients is a quarter that of input tile 301. The line offset for level 2 is four times the tile width (since each coefficient row is from an area corresponding to two level 1 LL rows or four input lines). Thus, four extra lines of memory are needed to use the same memory that is storing the input tile to store the L2 coefficients. Note that if a third decomposition level was being performed, an additional 8 lines would be needed. Thus, in this example, a total of 14 extra lines are needed to enable reuse of the memory that stores an input time and has two levels of decomposition applied thereto. A general formula may be used to determine the number of extra lines is as follows:
2[0067] (maxlevel+1)−2.
To access subbands, such as the LL, LH, HL and HH subbands, only a starting pointer and the offset between rows/lines are necessary. The height and width are also needed to know when to stop when accessing a tile. [0068]
As the number of decomposition levels increases, some rows at the bottom of memory become unused. That is, the lines of memory below the L[0069] 1 coefficients after the first decomposition level become unused, the lines of memory below the L2 coefficients after the second decomposition level become unused, etc. In one embodiment, this extra space may be reused.
FIGS. [0070] 3D-3F illustrate the corresponding inverse transform memory usage in which additional lines store the results of applying an inverse transform and those additional lines are in the memory behind the direction the inverse transform is being performed.
FIG. 4A shows one embodiment of a single memory where the input and the various levels can be stored during application of a forward transform. Referring to FIG. 4A, locations for the input tile, level [0071] 1 coefficients, level 2 coefficients, and level 3 coefficients is shown with the added 2, 4 and 8 lines respectively. FIG. 4B shows a similar single memory embodiment where the input coefficients of various levels of the transform can be stored along with the output during application of an inverse transform.
Table 1 shows the amount of memory required for various transform levels for a 256×256 tile for separate memories and reused memory. [0072]
For reused memory, the amount listed is the additional new memory used for that level. For this example, reusing memory for levels [0073] 1, 2 and 3 saves memory. Level 4 may use a separate memory. The memory for levels 4, 5 and 6 could be placed in a single memory after level 3 has been generated or in a completely different and separate memory. The amount of memory necessary is 38×32, which is less than 5×256. Because there are two unused lines after generating the level 1 coefficients (i.e., the memory that stored the last two lines of input data), a small memory savings can be achieved by letting the levels 4, 5 and 6 reuse these two lines. This is particularly important because the number of additional lines for levels 4, 5, and 6 is 16, 32 and 64, and the extra space between the lines will be twice as far and half as wide as the level before.
In one embodiment, coefficients from levels [0074] 4, 5, and 6 are packed in a smaller memory structure, such as storage area 450 in FIG. 4. Referring to FIG. 4, the level 4 coefficients are stored in an area having a height equal to the tile height divided by 8 (23 where 3 corresponds to the number of levels) and a width equal to the tile width w divided by 8 (23 where 3 corresponds to the number of levels previously stored elsewhere). An additional two lines 451 are all that is needed to store level 5 coefficients in the same necessary storage area. Similarly, an additional four lines is all that is necessary to accommodate using this memory storage area for the level 6 coefficients. Note that no lines are skipped when storing the coefficients. In one embodiment in which a 256×256 tile is being processed, the extra 5 lines at the bottom of storage area 430, two lines 421 and approximately 4.75 lines 422 are used to accommodate storage area 450. As shown, the approximate by three lines 422 represent allocated memory or in addition to that necessary to store the input tile. In this manner, the storage area for the input tile is almost completely reused.
In one embodiment, to use a very little, or potentially minimum, memory, level [0075] 6 is stored separately from levels 4 and 5. However, this only saves 64 bytes of memory.
Decoding is similar in that the results of applying an inverse transform are written ahead of where the decoding processing logic is reading, with the only notable difference being that the start is from the highest level to the lowest level, such as level [0084] 6 to level 1 in the example above. In such a case, the input tile ends up at the top of the memory structure. The extra lines to accommodate the memory reuse are in decreasing order. For example, using the structure of FIG. 4B, 8 lines would be necessary to create the L2 coefficients from the L3 coefficients, 4 extra lines would be necessary to create the L1 coefficients from the L2 coefficients and 2 extra lines would be necessary to create the input tile from the L1 coefficients.
After the transform is applied to coefficient buffers [0088] 502 1-502 N the context model 503 and entropy coder 505 can perform further compression processing on the already transformed data. The coded data is buffered in coded data memory 505.
In this codestream, packets are grouped based on the layer in which they reside. The first 8 packets belong to Layer [0107] 0. The following 8 packets belong to Layer 1.
After the layer progressive codestream is converted to resolution progression, in the new codestream, packets are grouped based on resolution. Such a grouping is shown in Table 5. The first 4 packets belong to resolution [0109] 0, the next 4 packets to resolution 1, and so on.
where layer=the number of layers in a codestream, [0112]
resolution=the number of decomposition levels in a codestream, and [0113]
component=the number of components in a codestream [0114]
Data Hiding (Sideband Information) in JPEG2000 Coding [0115]
Bit hiding allows sideband information to be transmitted without increasing the file size. Sideband information that does increase file size but does not break naive decoders might also be valuable (although the COM marker defined by the JPEG 2000 standard might be used instead). [0116]
Some marker segments, packet headers and packets are padded out to the nearest byte. Examples of the JPEG 2000 marker segments include PPM, PPT, PLM, and PLT. In addition, some marker segments can be longer than they need to be including QCD, QCC, and POC. In all of these cases, the padded data values are not defined. [0117]
Several proprietary coding schemes could use this semi-randomly located undefined data to provide a number of important types of information including, but not limited to, decoding and filtering hints, ownership, segmentation hints, and so on. A hint might include an index to a particular enhancement scheme. For example, if it is known that an image is mostly text, a value may be sent that indicates that a first post-processing filter is to be used. On the other hand, if the area is mostly a graphic image, then a value may be sent that indicates that a second post-processing filter is to be used. [0118]
The following are places where bits may be hidden or sideband information may be stored in the codestream. [0119]
arithmetic coder (AC) termination (without predictable termination) [0120]
end of packet header rounding to byte [0121]
after last packet, before next tile [0122]
tag tree construction by not always using minimum [0123]
packet header Lblock signalling [0124]
LSB parity for codeblocks (refinement pass only, cleanup pass only, all) [0125]
QCD, QCC extra subbands, POC. [0126]
For example, with respect to hiding data using AC termination, 0 to 7 bits are provided, at least, everytime the coder is terminated. However, this could be extended for a few bytes. These extra bits and bytes may be used for sending extra information. [0127]
With respect to each packet header, the end of a packet header is rounded to a byte boundary. Therefore, there may be 1 to 7 bits that may be available for sending extra information at times when rounding would have been necessary. Similarly, each packet is rounded to a byte boundary, thereby providing 1 to 7 bits (assuming that rounding would have been necessary). Also the last packet in a tile-part can be extended a few bytes. These extra bytes may be used to send additional information. [0128]
The length of the compressed data for a code-block can be given in the packet header with a non-minimum representation. The choice of representation (e.g., a non-minimum representation) could be used for indicating other information. [0129]
With respect to tag tree data hiding, packet headers of the JPEG 2000 standard use tag trees for coding first inclusion and zero bitplane information. When there are multiple codeblocks, tag trees are like a quadtree of minimum values. For example, in the case of 16 codeblocks in a 4×4 arrangement in a packet, the arrangement may be as follows: [0130]
An example tag tree, which is minimal for the 4×4 arrangement above is as follows: [0131]
3 0 2 7 4 71 0
in which “3” is added to every codeblock's value, and “0”, “2”, “5” and “2” are each added to the 4 corresponding codeblocks. Finally, there is one value per codeblock. That is, the minimal tag tree is created by taking the first 2×2 group in the 4×4 arrangement above and look at minimum value is out of the four values. In this case, for the 2×2 block [0132]
the minimum value is 3. This is then performed on the other 2×2 blocks. Then these identified minimum values are evaluated again to determine their minimum, which would be “3” in the example. Then the minimum value is subtracted from the four minimum values to create the following [0133]
Then, for the remaining numbers in the 4×4, the number 3 is subtracted from each value along with the value in the 2×2 that corresponds to the particular value in the 4×4 arrangement, thereby resulting in the tag tree above. The first row adds up as follows: [0134]
10=3+0+7 [0135]
7=3+0+4 [0136]
12=3+2+7 [0137]
15=3+2+10 [0138]
A variable length code may be used that efficiently represents small numbers. [0139]
An example of a tag tree that is not minimal is as follows: [0140]
(Note that representing “3”, “0”, “2”, “5” and “2” might use less bitstream data than “2”, “1”, “3”, “6” and “3”.) [0141]
Once a tag tree representation has been made, a determination can be made as to whether the representation is minimal or not based on whether there is a zero in the 2×2 block. Therefore, this information is hidden. For example, the 1 bit block represents the 1 in the 2×2 block above indicates it is not part of a minimal tag tree, but can be used to convey some particular information to a decoder. Likewise if a 2 was the minimal value in the 2×2 block, such a fact may convey different information to a decoder. [0142]
The JPEG 2000 POC, QCD, and QCC markers can have redundant entries. It is as if the codestream were quantized and the markers were not rewritten. For example, the QCD and QCC markers have values for a number of subbands specified by the syntax of the marker. If there are fewer subbands actually coded in the bitstream, data may be hidden in the values used for the missing subbands. The redundant entries may be replaced and used for hidden or sideband information. [0143]
The hidden or sideband information may include post-processing hints (such as, for example, sharpen this tile with a specified filter or strength, or smooth, or perform optical character recognition (OCR) on this region, etc.), decoding hints, security (such as, for example, an encryption key for decoding the remainder of the image or another image, etc.) codestream identification (such as, for example, labeling POTUS as the originator of the file, etc.) and/or other information. [0144]
Use of Layers when Encoding [0145]
Layers are part of the JPEG standard. In one embodiment, sideband information, possibly in a COM marker, is used by the decoder to allow selecting of layers during decoding. The sideband information may be used to select layers for postcompression quantization to meet rate/distortion targets for different viewing distances, different resolutions, different regions of interest, different frequency content for analysis (e.g., finding edges of text). [0146]
In one embodiment, the layers are predefined based on rate. For example, the first layer represents a 1-bit per pixel image, while the second layer represents a 2-bit per pixel image, etc. Therefore, the layers run from the lowest quality to the highest quality. Likewise, target rates can be met for lower resolutions as well. [0147]
The sideband information may be stored in a marker segment of the codestream. In one embodiment, the JPEG 2000 comment (COM) marker is used to provide information about the layers. Specifically, the COM marker may be used to indicate the number of bytes for each resolution and/or rate across the entire image or a relative number of bytes for each additional layer. Table 6 indicates each layer and its resolution in the number of bytes across the tile in an image. Such a table may have distortion values instead. [0148]
In another embodiment, the ordering could be by layer. Thus, the information above is consolidated for each level (not segregated by level or component), as shown below: [0149]
Ordering by layer=0 bytes=7959 bitrate=0.971558 PSNR=30.7785 [0150]
Ordering by layer=1 bytes=10877 bitrate=1.327759 PSNR=32.0779 [0151]
Ordering by layer=2 bytes=16560 bitrate=2.021484 PSNR=35.7321 [0152]
Distortion by layers can be based on PSNR. For example, [0153]
layer=0 PSNR=30.7785 [0154]
layer=1 PSNR=32.0779 [0155]
layer=2 PSNR=35.7321 [0156]
In an alternative embodiment, such information may be hidden in the codestream as described above. The information may be used to control rate distortion. [0157]
In another embodiment, the layers may be predefined for a particular viewing distance. In such a case, the data is divided into layers from the highest frequency, lowest resolution to the lowest frequency, highest resolution. [0158]
In one embodiment, the layer information indicates the summation of bits across the entire image for that layer and all previous layers (for example the 16,011 bits listed next to layer [0159] 1 indicates the total number of bits for layer 0 and layer 1). Alternatively, bytes, words, kilobytes, or other units of memory or rate could be used instead of bits. Table 7 shows this type of absolute rate information.
Table 8 shows relative rate information. Layer [0160] 0 has 4096 bits, layer 1 has 11,915 bits, etc.
For example, if only 75,000 bytes may be allowed in the decoded image, then all that can be decoded (as the 1,000,000 bytes tabulated with layer [0162] 6 includes 500,000 bytes of layers 0-5) is through layer 5 and half of importance layer 6. In some embodiments, no packets from layer 6 would be included. In other embodiments, some packets from layer 6 would be included and others would be replaced by zero packets so that the total amount of layer 6 data was approximately 250,000 bytes.
FIG. 22 illustrates an example of layering for a 5,3 irreversible transform with three levels, MSE or similar. Referring to FIG. 22, there are 45 layers shown. Each additional layer improves MSE in an order that gives good rate-distortion for MSE. [0163]
FIG. 23 illustrates another example in which transform has 5 levels and the data is divided up into layers [0164] 0-3. Layer 0 corresponds to the thumbnail version, layers 0-1 correspond to the monitor (or screen) resolution, layers 0-2 correspond to the print resolution, and layers 0-3 correspond to lossless.
In an alternative embodiment, the layers may be predefined for some other distortion metric (e.g., MSE, weighted MSE, sharpness of text, etc.) The decoder uses the information regarding the layers from the codestream to select layers to generate an image. The decoder knowing what the desired viewing characteristics from the application or implementation (see Table 9 below), and using the information from the codestream specifying the layers, can quantize the codestream in order to display an image at the correct viewing distance. FIG. 9 illustrates such a decoder. Referring to FIG. 9, decoder [0165] 901 receives a codestream and includes quantization logic 902 that examines the COM marker and uses information about the viewing distance it is at stored in storage 903 to generate quantized codestream 904 via, for example, selecting the proper layers. Quantized codestream 904 is decoded by decoding logic 905 (e.g., a JPEG 2000 decoder) after selecting layers to generate an image data 906. A naive decoder would simply ignore the data in the comment marker.
FIG. 10 is a flow diagram of a process for using layers when decoding. The process is performed by processing logic that may comprise hardware (e.g., dedicated logic, circuitry, etc.), software (such as is run by, for example, a general purpose computer or a dedicated machine), or a combination of both. [0166]
Referring to FIG. 10, the process begins by processing logic receiving a codestream of compressed logic data (processing block [0167] 1001). The image data is organized into multiple layers, each of which comprises coded data that adds visual value to the image (e.g., look sharper, better defined, better contrast, etc.). Next processing logic selects one or more layers for quantization based on sideband information (processing block 1002). After selection, processing logic decompresses the non-quantized layers of the codestream (processing block 1003).
Editing of Tiles, Tile-parts, or Packets [0168]
Once a codestream is created, it may be desirable to edit parts of the image. That is, for example, after performing encoding to create the codestream, a set of tiles may be decoded. After decoding the set of times, editing may be performed, followed by encoding the set of tiles with the edits to the same size as the encoded tiles were prior to their decoding. Examples of typical editing include sharpening of text and removing “red-eye.” The JPEG 2000 codestream can be edited in memory or in a disk file system without rewriting the entire codestream. [0169]
FIG. 11 is a flow diagram of one embodiment of an editing process. The process is performed by process logic that may comprise hardware (e.g., dedicated logic, circuitry, etc.), software (such as is run by, for example, a general purpose computer or a dedicated machine), or a combination of both. [0170]
Referring to FIG. 11, processing logic initially determines the tiles, tile-parts, or packets that cover the area, resolution, components, and/or precincts to be edited and decodes them (processing block [0171] 1101). This determination may be made in response to a user selecting an area and/or working resolution. The determination may use editing information for a higher resolution to determine which parts or tiles cover the portion to be edited. Once decoding has been completed, processing logic performs the desired edits (processing block 1102).
After performing the desired edits, processing logic recompresses the data into coded data (processing block [0172] 1103) and creates a replacement tile, tile-part, or packet for the codestream (processing block 1104). In one embodiment, in creating the replacement tile, tile-part, or packet, processing logic pads out the data with bytes at the end of the codestream if the new data is smaller than the unedited version of the data to make the replacement tile, tile-part or packet the same size as the unedited version.
In an alternative embodiment, processing logic may use a marker, or tag, such as a COM marker segment of the appropriate length instead of the padding. The COM marker could be used to fill space or could contain information that the encoder wanted to include. It could contain information such as, for example, sideband information described herein or a copyright license for an image or text or other file format information. [0173]
In one embodiment, in creating the replacement tile, tile-part, or packet, processing logic truncates the last packets for any or all components until the data fits in the codestream if the new data is larger than the unedited version of the data. [0174]
Editing of an image may be performed by changing coded data for tiles, tile-parts, or codeblocks. In one embodiment, editing is performed without changing file size by quantizing instead of expanding. In another embodiment, a predetermined amount of extra space is allocated per tile or per codeblock to allow for a predetermined amount of expansion. In still another embodiment, coded data may be put at end of files by manipulating tile headers and putting invalid tile data in COM markers. [0175]
Note that if there are subsequent tile-parts that depend on the data in the portion of the codestream that is being edited, these tile-parts may become useless in the codestream. An indication of this useless data may be noted to the decoder by one of several methods. These methods involve inserting or modifying information in the codestream to indicate the presence and/or location of the useless data. In one embodiment, the application uses a status buffer to indicate that the data in tile-parts subsequent to an edited tile-part may be useless. The status buffer may be in workspace memory and describes dependencies between packets. If an earlier packet is altered, the subsequent packets cannot be decoded as is. These subsequent packets must be edited accordingly or eliminated. In another embodiment, such an indication may be made by zeroing out the data section of those tile-parts and/or creating a PPT marker segment that denotes no data. [0176]
Optimal Encoder Quantization [0177]
During encoding, unquantized coefficients from some or all subbands may be divided by a value of Q to create the quantized coefficient values. This value Q may have a wide range of values. Typical encoders quantize a number of the values in a single particular range of values is made equal to one single coefficient value. In essence, all the coefficients in the particular range are quantized to the same value. This can be exemplified by FIG. 12 which shows that the range of values is often in a bell shaped curve and that all of the values in the particular range, such as range R[0178] 1 are sent to the decoder as one quantized value, such as R1′, and the decoder will reconstruct these values to a particular value. Assume a decoder reconstructs these values to a predetermined value (e.g., floor (½ min+½ max), or min+½ Q, where Q is the quantization step size). For example, if the range of values is between 16 and 31, then the decoder may assume the value is 24. In one embodiment, instead of using ½ as the value, another value is selected, such as floor (⅜ min+⅝ max), or min +⅜ Q, where Q is the quantization step size. Therefore, if the range is from 16 to 31, then it is assumed that the decoder will reconstruct the value to 22, instead of 24.
In some cases, two spatially adjacent coefficients may be close to each other numerically yet in separate quantization bins, such as coefficient values [0179] 1201 of range R2 and 1202 of range R1 in FIG. 12. The results of the quantization may cause an artifact to occur. In one embodiment, for coefficients near a boundary between two quantization bins, the encoder selects a bin such as Range R1 into which a coefficient, such as coefficient 1201, will be quantized so that it is consistent with neighbors, such as coefficient 1202. This helps avoid artifacts. That is, this technique reduces distortion yet may increase rate, particularly when a coefficient is moved from a smaller bin to a higher bin.
Flicker Reduction for Motion JPEG [0180]
At times, flicker occurs when applying wavelet compression to motion sequences. An example of such flicker may include the image getting brighter or darker in areas or the appearance of edges changing in successive frames as the motion sequence is played (mosquito noise around the edges). The flicker may be due to the application of different local quantization to successive frames of a motion sequence or to noise exacerbated by quantization that is viewed temporarily. [0181]
To reduce flicker, coefficients that are in the same position and close to the same value in successive frames are forced to the same value. That is, the coefficients values in successive frames are set to a predetermined value. This is essentially a form of quantization that is applied during encoding. FIG. 13 is a flow diagram of one embodiment of a process to reduce flicker. [0182]
A test of whether to apply such quantization to a coefficient value in a subsequent frame is based on the quantization that was performed on the coefficient in the previous frame. Thus, the encoder is utilizing frame dependency to eliminate flicker while the decoder decodes data frame by frame independently. [0183]
In one embodiment, in order to reduce flicker in motion JPEG, coefficient values are modified (quantized) based on their relationship with each other with respect to a threshold. For example, if Dn and Dn+1 are the corresponding coefficient (same spatial location and same subband) in two frames before quantization, if D′n and D′n+1 represent these coefficients after quantization, if Q(•) are scalar quantization, and if the value T is a threshold, then the following may be applied: [0184]
For example, the value T may be twice the quantization step size. Other values of T include, but are not limited to, {square root}{square root over (2)}Q, 1.5Q, 2{square root}{square root over ([0185] 2)}Q.
One of the coefficient values may be modified to be either a predetermined closeness to another coefficient value. The closeness may be determined by some threshold. The threshold may be user set or adaptive based on some criteria. The threshold could be different based on the subband and, perhaps, on the persistance of the particular value (number of frames that this coefficient is close). In one embodiment, the coefficient value is set equal to the other coefficient value. In alternative embodiments, the coefficient is set to be within the quantization bin size of the other coefficient value or twice the quantization bin size. [0186]
FIG. 14 illustrates one embodiment of an encoder (or portion thereof) that performs the quantization described above. Referring to FIG. 14, a quantizer [0187] 1400 receives coefficients 1410 for frames of a motion sequence from a wavelet transform (not shown). The coefficients are received by quantization logic 1401 which compares a threshold value stored in memory 1401 to coefficient values for the previous frame that are stored in memory 1403 to coefficients 1410 with a scalar quantizer Q applied from memory 1404.
Quantization logic [0188] 1401 may comprise comparison hardware (e.g., logic with gates, circuitry, etc.) or software to perform the comparison. This comparison hardware and software may implement a subtractor or subtraction operation. The results are a quantized codesteam (assuming some values have been changed.)
This may be applied over two or more frames. Also the comparison is not limited to two consecutive frames. The comparison can be over 3, 4, 5, etc., frames, for example, to determine if a variance exists. FIG. 24 illustrates one example in which values in a first and third frame are used to set the value in the second frame. [0189]
Note that the quantization can also be codestream quantization with a code block-based rule. [0190]
Rate Control, Quantization, and Layering [0191]
In one embodiment, selective quantization of coefficients can be performed during encoding by setting a subset of the refinement bits to be the more probable symbol (MPS). This may be performed at a user selected bitplane. For examples, if there is text on a background image, with a goal of having sharp text images while minimizing coded data required for the background, the refinement bits that are set to MPS are those that do not effect text for the last bitplane, while using the actual value for bits that effect text. [0192]
Such a quantization scheme may be used to implement non-uniform quantization step sizes. For example, if one wanted to have a background with fewer bits, setting the refinement bits to the MPS could operate as a form of quantization. This quantization scheme causes some level of distortion but lowers the bit rate necessary to transfer the codestream. [0193]
Note that although this technique may be applied to bits generated during the refinement pass, the technique has application to other compression schemes (e.g., lists generated during subordinate passes, tail bits of CREW of Ricoh Silicon Valley, Menlo Park, Calif., MPEG IV texture mode, etc.). [0194]
In one embodiment, the same technique may be applied to other changes between frames. That is, in one embodiment, a change due to a rate distortion in one frame may be performed in a subsequent frame to avoid distortion effects. [0195]
Rate Control and Quantization [0196]
In one embodiment, user specified quantization is provided. For a 3 level transform for one component, 7 quantization values are sufficient: level [0197] 1 HH, level 1 HL and LH, level 2 HH, level 2 HL and LH, level 3 HH, level 3 HL and LH, and level 3 LH.
If quantization values are bitplanes to truncate (which is equivalent to scalar quantization by powers of 2), 3-bit values (0 . . . 7) are sufficient for most applications. (For image components with depth 12-bits or more and 5 or more transform levels, perhaps higher quantizations might be useful.) Values 0 . . . 6 could be used to specify the number of bitplanes to truncate and 7 could be used to mean discard all bitplanes. The three bit values may be written to a controller that controls compression (or decompression) hardware (e.g., JPEG2000 compatible hardware) to perform the quantization. [0198]
For 3 component color quantization: [0199]
21 values can be used with separate values for each component, [0200]
14 values can be used, 7 for luminance and 7 for chrominance, [0201]
17 values can be used for 4:1:1 subsampled data, 7 for luminance and 5 for each chrominance component, [0202]
12 values can be used for 4:1:1 subsampled data, 7 for luminance and 5 for chrominance, [0203]
19 values can be used for 4:2:2 subsampled data, 7 for luminance and 6 for each chrominance component, and [0204]
13 values can be used for 4:2:2 subsampled data, 7 for luminance and 6 for chrominance. [0205]
Since 21*3=63 bits is less than 8 bytes, transferring or storing the quantization uses little resources. A central processing unit (CPU) might select one predetermined quantizer from a table and write it to a CPU or other controller controlling special purpose JPEG 2000 hardware (a chip) for each frame of a motion JPEG 2000 video sequence. Alternatively, one implementation of JPEG 2000 might have a small memory that holds 8 or 16 different quantizers that could be selected for each frame. [0206]
Quantizers can also be used to assign bitplanes to layers. For example, Q[0207] 0, Q1, and Q2 may be quantizers that specify bitplanes of coding pass to quantize. Quantizer Q0 causes the most loss, while quantizer Q2 causes the least loss. Layer 1 is all the data quantized by Q0 but not quantized by Q1. Layer 2 is all the data quantized by Q1 but not quantized by Q2. Layer 3 is all the data quantized by Q2.
Simple Quantization [0208]
FIGS. 17 and 18 show example quantizers (label A . . . Q) for the 3-level 5/3 transform as the number of coefficient LSBs to truncate or not code. Truncating N bitplanes is equivalent to a scalar quantizer of 2[0209] N. The subband where the quantization changes with respect to the previous quantizer is highlighted with a dashed box. The quantizers D, K and Q all have the same relationship between the subbands. Other quantizers might be used that are better for MSE or for other distortion metrics.
The exemplary Verilog below converts a single quantization value “q” into seven quantizers (number of LSBs to truncate). The variable q[0210] —1_HH is used for level 1 HH coefficients, the variable q—1_H is used for level 1 HL and LH coefficients, etc. Some consecutive values of q result in the same quantizer: 0 and 1; 2 and 3; 4 and 5; 8i+6 and 8i+7 for all integers i with i≧0.
Human Visual System Weighting for Color and Frequency [0211]
Table 9 shows additional bitplanes to quantize (e.g., truncate) for luminance to take advantage of the frequency response of the Human Visual System (from Table J-2 of the JPEG 2000 standard). A viewing distance of 1000 pixels might be appropriate for viewing images on a computer monitor. Larger viewing distances might be appropriate for print images or television. [0212]
Additionally chrominance may be quantized more heavily than luminance. [0213]
FIG. 19 shows a quantization that starts with FIG. 17(D) and then adds frequency weighting for a 1000 pixel viewing distance (to both luminance and chrominance), keeps [0214] 3LL chrominance unchanged, discards 1HL and 1HH chrominance for 4:2:2 and additional 2 bitplanes are discarded for the remaining chrominance.
Sharp text without ringing artifacts is more desirable than exact gray value for text/background. That is, if a gray level is supposed to be at 50% (for example), and is instead at 60%, it is often not visually objectionable if the image is of text. In one embodiment, the LL (DC) coefficients are quantized more heavily for text than for non-text images at low bitrate. For example, for an 8-bit image component, a quantiation step size of 8, 16 or 32 might be used for text only regions and a quantization step size of 1, 2 or 4 might be used for regions containing non-text. This allows more fidelity for the high frequency coefficients, thereby resulting in text with sharp edges. [0215]
Using Quantizers to Divide Things into Layers [0216]
Table 10 shows 16 example quantizers. Quantizer [0217] 15 is lossless. Quantizer 8 is the same as FIG. 19. These can be used divide the subband bitplanes into layers.
Layer [0218] 0 contains all data not quantized away by quantizer 0. This would be luminance data only: all of 3LL; all but 4 bitplanes of 2HL, 2LH, 3HL, 3LH and 3HH; all but 5 bitplanes of 2HH and all but 6 bitplanes of 1HL and 1LH. Layer 1 contains all data not in layer 0 and not quantized away by quantizer 1. This would be luminance bitplanes 5 for 1HL and 1LH, bitplane 4 for 2 HH, bitplane 3 for 3HL and 3LH; all 3LL chrominance; all but 5 bitplanes for chrominance 3HL and 31H; and all but 6 bitplanes for chrominance 2HL, 2LH and 3HH. Finally, layer 15 would contain the LSB of 1LH chrominance.
Rate Control with Multiple Layers and Tile-Parts [0219]
There several well known techniques for rate control in compression systems. The simplist method is to pick a distortion for every “unit” compressed (a unit may be an 8×8 block in JPEG, a frame in a motion sequence, a tile of a single image, a subband of a tile in a wavelet coded image, etc.). If the distortion selected leads to a bitrate higher than the desired average bitrate, the distortion allowed is increased for new units as they are compressed. If the distortion selected leads to a bit rate lower than the desired average bitrate, the distortion allowed is decreased for new units as they are compressed. [0220]
A more complex method buffers the compressed data from some number of “units.” The bitrate and/or distortion for each unit at each distortion level is stored. Then the distortion to allow across all the units in the buffer is determined when the buffer is full. If the buffer is sufficient to contain the entire image, extremely high quality results can be obtained. In JPEG 2000, layers are designed to contain increments to quality. Thus, selecting a distortion can mean selecting the number of layers to use for each code block or tile. A complete description of this type of rate control is in, David Taubman, “High Performance Scalable Image Compression with EBCOT,” IEEE Transactions on Image Processing. [0221]
There are several disadvantages to this process. One disadvantage is that a buffer memory for the entire codestream is required. A second disadvantage is that the latency (time until any of the codestream is output) is high. A third disadvantage is that the second pass could take large amount of time. [0222]
To mitigate these problems, each tile of a JPEG 2000 codestream is encoded as described above with at least two layers. At the completion of encoding each tile, a number of packets (e.g., layer, resolution, precinct, tile-component) are output to the codestream as a complete tile-part. The remaining layers are stored in the buffer. A second pass through the remaining coded data in the buffer is optional. During this second pass, extra packets from each tile are appended to the codestream as complete tile-parts as space or time allows. If in a fixed-rate application, then only packets within the given rate are appended. If in a fixed time application, then only number of cycles allowed. One embodiment of this process is shown in FIG. 15A. Thus, these can be the 2 complete tile-parts output for each tile. [0223]
FIG. 15B illustrates a number of layers, layers [0224] 1-n. Layer 1 is output on the first pass, and the remaining layers are most likely below fixed-time or fixed-rate time limits. Layer 2 may be output on a second pass within fixed-time or fixed-rate requirements while achieving similar distortion over all the components.
The above process is advantageous in that it allows the buffer to store a fraction of the coded data, the first data can be output (transmitted or stored) sooner, and the second pass through the data can be faster because there is less data to process. Also less memory is required for buffering. [0225]
The criterion for selecting which packets go into the first set of tile-parts can be similar to any other rate control algorithm. In one embodiment, the rate of packets can be less than the desired average bitrate for the whole image. For example, if a final compressed bitstream at 2.0 bpp is desired, the first pass could place 1.5 bpp for every tile in the codestream, and buffer 1 bpp for every tile. [0226]
The second pass can select from the remaining data the packets to place in the second tile part of each tile. Thus, to obtain a 2.0 bpp average encoding, some tiles that had high distortion after the first pass could receive all the remaining data saved for the tile, while other tile parts which had low distortion after the first pass might not have any additional data transmitted. [0227]
Rate Control for Compressed Codestream Data [0228]
Some rate control techniques described herein include rate control performed on a compressed codestream based on a request implemented by selecting some number of layers to keep in the codestream. A parser may be used to produce a new codestream which shows the bitrate based on layers. This bitrate is equal to or less than the bitrate specified by the request. [0229]
The parser may use a data structure referred to herein as a “packet structure.” Note that this data structure may be used for other purposes such as, for example, the versatile pocket data structure described below. In one embodiment, the packet structure includes a packet start pointer and packet length. It also contains a tile number, a resolution, a component, layer, and a precinct the packet belongs to. Finally, it also consists of a selection flag. This flag, when set to a predetermined value (e.g., 1), indicates if the packet is selected in the array for writing out to a new codestream. [0230]
In one embodiment, packets are read in sequential order from a codestream based on the progression order information indicated by the COD marker. [0231]
The number of bytes is computed based on the bitrate desired by the request. The number of bytes belonging to layer [0232] 0 is added up to a total. Then this total of bytes is compared with the number of bytes desired. If the total is less than the number of bytes desired, one additional layer is added to the total. The process continues until the total is equal to or greater than the number of bytes desired or all packets have been added.
During the process, those packets which have been added to the total, are marked as selected by the selection flag in the structure. [0233]
If the total is equal to the number of bytes desired, the addition process is stopped. If the total exceeds the number of bytes desired, the packets in the last layer added are subtracted from the total. This is done to guarantee that the bitrate is below the bitrate desired. Consequently, during the subtraction step, packets which have been subtracted from the total are marked unselected. [0234]
In one embodiment, the related markers such as SOT, COD, PLT are updated according to the request. Packets are written to the new codestream. The packet structure may be created using the following: [0235]
Versatile Packet Data Structure [0236]
The same packet data structure described above can be used to facilitate other parsing options, once packets are read into the structure. [0237]
For resolution parsing, the packets which are to be excluded are marked unselected. For example, given a 4 resolution codestream, and a request is to produce a 3-resolution codestream, a parser marks all packets which belong to resolution [0238] 4 unselected. Then the newly produced codestream contains only packets from resolution 1 up to resolution 3.
Similarly, for component parsing, progression conversion parsing, quality parsing can be performed step by step processing the packets in the structure. [0239]
The packet data structure can handle complex requests. For example, a request which requires the parser to produce a codestream which has a 3-resolution, 2-layer, and 1-component codestream. [0240]
Clipping After Each Inverse Transform [0241]
As a result of quantization performed on wavelet coefficients, the final decoded pixels are often outside of the original range of allowed pixels from the specified bit depth. Typically, these pixels are clipped to the original range so that further image processing or display devices can use the original bit depth. [0242]
For example, an eight bit image has pixel values between 0 and 255, inclusive. After lossy compression is used, the decoded image may contain values like −5 and 256. To provide an eight bit output, these values are clipped to 0 and 255 respectively. This clipping procedure always reduces pixel wise distortion because the original image did not contain pixels outside of the clipping bounds. This procedure is well known and recommend by the JPEG 2000 standard. [0243]
In addition to the bounds on the final output samples, there are bounds on the values coefficients can assume at the various stages of the wavelet transform. Just as quantization can change the final decoded samples to lie outside the original bounds, quantization can change the partially transformed wavelet coefficients to lie outside their original bounds. If these coefficients are clipped to their original bounds, distortion will decrease. [0244]
For example, after a horizontal (one dimensional) 5-3 reversible transform as specified by JPEG 2000 with 8 bit input samples, the maximum value of the low pass coefficient is +191, and the minimum possible value is −191. The high pass coefficient must be between −255 and 255 inclusive. After the vertical one dimensional transform, the Low-Low coefficients are bounded by −286 and 287. Thus when decoding an eight bit image, when the first level low-low pass coefficients are generated (by the inverse wavelet transfrom from a higher level), the coefficients can be clipped to −286 and +287, and distortion will decrease. Likewise after the first level vertical inverse transformation is done, the low pass coefficients can be clipped to −191, +191, and the high pass coefficients can be clipped to −255, 255. [0245]
For each subband, each filter, each transform level, and each image depth, there is a different maximum and minimum value for the coefficients. These maximum and minimum values can be computed by finding the signal that leads to the maximum and minimum and running the forward compression system and recording the maxima. The signals that lead to extreme values come from inputs where each pixel is either a maximum or minimum. Which pixels should be maximum and which pixels should be minimum can be determined by convolving sequences which are −1 when the wavelet coefficient is negative and +1 when the wavelet coefficient is negative. For the 5-3 filter used in JPEG 2000 Part I, the low pass signal of interest is [−1 +1 +1 +1 −1] and the high pass signal is [−1 +1 −1]. [0246]
The signal (image) which will generate the largest LL value is: [0247]
(where +1 must be replaced by the input maximum (e.g., 255) and −1 must be replaced by the input minimum (e.g., 0). [0248]
For irreversible filters, it is not necessary to actually run the system to determine the maxima, simply convolving the wavelet coefficients is sufficient. For the reversible 5-3 filter, however, the floor function is used in the computation of coefficients and is also used to determine the correct maxima. [0249]
Note that this may be used for other filters (e.g., a 9-7 filter). [0250]
FIG. 28 is a flow diagram of one embodiment of a process for applying an inverse transform with clipping on partially transformed coefficients. The process is performed by processing logic, which may comprise hardware (e.g., circuitry, dedicated logic, etc.), software (such as that which runs on a general purpose computer system or a dedicated machine), or a combination of both. [0251]
Referring to FIG. 28, processing logic applies a first level inverse transform to coefficients (processing block [0252] 2801). Thereafter, processing logic clips the partially transformed coefficients to a predetermined range (processing block 2802). Next, processing logic applies a first level inverse transform to the clipped coefficients (processing block 2803) and clips the partially transformed coefficients to a predetermined range (processing block 2804), which is different than the range in processing block 2802. Again, processing logic applies a first level inverse transform to clipped coefficients (processing block 2805) and clips the partially transformed coefficients to still another predetermined range (processing block 2806).
Simplified Colorspace Handling [0253]
A typical decoding process including color management is shown in FIG. 25. Referring to FIG. 25, a file with a file format (e.g., a file format described in the JPEG 2000 standard) containing a restricted ICC profile is provided to a decoding device. Decompression block [0254] 2501 decompresses the file by taking the codestream portion of the file and performing context modeling, entropy decoding, and applying an inverse wavelet transform, but does not perform color space operations. If the codestream indicates the RCT or ICT component transform should be used to decode the codestream, these will be performed by block 2502. That is, inverse RCT/ICT block 2502 takes the components and the “RCT Y/N” indication (RCT if yes, ICT is no) and performs the specified inverse transform and provides (non-display) RGB pixels. (If specified by the syntax, inverse level shifting is also performed.)
Finally, the ICC color profile from the file format along with information about the display device will be used to produce the output pixels. [0255]
Inverse ICC block [0256] 2503 receives the (non-display) RGB pixels and the ICC profile and applies an inverse color space transform to provide display RGB pixels.
FIG. 26 illustrates one embodiment of a non-preferred camera encoder. Referring to FIG. 26, a camera generates YCrCb pixels. A converter [0257] 2602 converts the YCrCb pixels to RGB pixels and provides those two a typical JPEG 2000 encoder. The encoder comprises a RCT to ICT converter 2603 followed by a compressor 2604. The compressor generates an ICCA for codestream.
FIG. 27 illustrates one embodiment of a simpler camera encoder. That is, instead of including RCT/ICT converter [0258] 2603 and compressor 2604, a simple camera encoder includes only compressor block 2702. Referring to FIG. 27, a camera 2701 generates YCrCb pixels and provides them to compressor 2702. Compressor comprises a JPEG 2000 encoder without an RCT conversion and generates an ICCB codestream with RCT equaling 1 (with syntax signaling that the inverse RCT should be used on decoding). The relationship between ICCB and ICCA is given by the following equation:
where ∘ represents function composition. [0259]
Restricted ICC profiles are “syntaxes” for functions on pixels. A camera will typically write the same profile for all images, so ICC[0260] B is computed offline, and copied into each output file. In a prior art system there must be HW for YCrCb−1 and RCT/ICT which operates on every pixel.
Coding 4:2:2 and 4:1:1 Data as 4:4:4 Data with Quantization [0261]
The JPEG 2000 standard is typically used to handling data in a 4:4:4 format. It is not capable of describing how to reconstruct data in 4:1:1 or 4:2:2 formats in a 4:4:4 format for output. In one embodiment, when encoding 4:1:1 data, the encoder treats [0262] 1 HL, 1 LH and 1 HH coefficients as zero. When encoding 4:2:2 data, the encoder treats 1 HL and 1 HH coefficients as zero. Thus, with all information in the extra subbands quantized to zero, a decoder is able to receive the codestream in a way it expects. In other words, the encoded data resembles 4:4:4 data that has been heavily quantized.
File Order for Thumbnail, Monitor, Printer, and Full Resolution and Quality [0263]
Multiple images at multiple resolutions are important in many image processing situations. Depending on the application, a user may want to select different images of different resolutions. For example, thumbnail images may be used as an index into a large number of images. Also, a screen resolution image may be the image used to send to a monitor for display thereon. A print resolution image may be of lower quality for printer applications. [0264]
In one embodiment, a codestream of an image is organized into sections so that different versions of the image, such as, for example, a thumbnail version, a screen version, a print version and a lossless version, is progressive by quality. [0265]
In one embodiment, the packets are arranged such that certain packets correspond to particular resolutions such as a thumbnail. The combination of these packets with other packets represents the monitor resolution image, which when combined with other packets may represent the printer version, etc. Using the POC and tile parts, portions of a codestream may be grouped together. For example, all the tiles of the thumbnail size may be grouped together followed by tiles for another resolution followed by tiles of another resolution, etc. FIG. 21 illustrates an example progression with tile parts for a single server. Each tile's thumbnail is grouped in tile-parts at the beginning of a file. FIG. 21A illustrates that tile-part [0266] 2101 is the only portion that is used for a thumbnail image. FIG. 21B illustrates that for a monitor resolution, tile-parts 2102-2104 have been included with tile-part 2101. FIG. 21C illustrates that for a printer resolution, tile-parts 2105 and 2106 have been included with tile-parts 2101-2104. Lastly, FIG. 21D illustrates that for a lossless version of the data, the remaining three tile-parts 2107-2108 are included with the rest of the tile-parts. These sets of tile parts may be placed on a server in this progressive order.
One embodiment of the process for accessing the groupings of tile parts is shown in FIG. 16. The process may be performed by processing logic that may comprise hardware (e.g., dedicated logic, circuitry, etc.), software (such as is run on a general purpose computer system or a dedicated machine), or a combination of both. The following steps assume that the image has been transformed with sufficient resolution levels and layers to divide the image into the four sizes. [0267]
Referring to FIG. 16, processing logic initially determines the correct resolution and layering for the thumbnail (processing block [0268] 1601). In one embodiment, to determine the correct resolution and layering for the thumbnail, processing logic creates a POC constrained to that resolution and layer for each tile and then creates a set of tile-parts and places this POC for each tile in the codestream.
Next, processing logic repeats processing block [0269] 1601 for the monitor resolution given that the thumbnail packets are already in the codestream (processing block 1602). Then, processing logic repeats processing block 1601 for the printer resolution given that the monitor packets are already in the codestream (processing block 1603).
Lastly, processing logic creates a POC marker with the extremes of the resolutions and layers for each tile (processing block [0270] 1604). In one embodiment, creating the POC with the extremes of the resolutions and layers is performed by creating a fourth set of tile-parts with the remaining tile-parts for a lossless version.
Note that the particular orders of the packets defined in the POCs are not of importance, only the limits. [0271]
An Exemplary Computer System [0272]
FIG. 20 is a block diagram of an exemplary computer system. Referring to FIG. 20, computer system [0273] 2000 may comprise an exemplary client 150 or server 100 computer system. Computer system 2000 comprises a communication mechanism or bus 2011 for communicating information, and a processor 2012 coupled with bus 2011 for processing information. Processor 2012 includes a microprocessor, but is not limited to a microprocessor, such as, for example, Pentium™, PowerPC™, Alpha™, etc.
System [0274] 2000 further comprises a random access memory (RAM), or other dynamic storage device 2004 (referred to as main memory) coupled to bus 2011 for storing information and instructions to be executed by processor 2012. Main memory 2004 also may be used for storing temporary variables or other intermediate information during execution of instructions by processor 2012.
Computer system [0275] 2000 also comprises a read only memory (ROM) and/or other static storage device 2006 coupled to bus 2011 for storing static information and instructions for processor 2012, and a data storage device 2007, such as a magnetic disk or optical disk and its corresponding disk drive. Data storage device 2007 is coupled to bus 2011 for storing information and instructions.
Computer system [0276] 2000 may further be coupled to a display device 2021, such as a cathode ray tube (CRT) or liquid crystal display (LCD), coupled to bus 2011 for displaying information to a computer user. An alphanumeric input device 2022, including alphanumeric and other keys, may also be coupled to bus 2011 for communicating information and command selections to processor 2012. An additional user input device is cursor control 2023, such as a mouse, trackball, trackpad, stylus, or cursor direction keys, coupled to bus 2011 for communicating direction information and command selections to processor 2012, and for controlling cursor movement on display 2021.
Another device that may be coupled to bus [0277] 2011 is hard copy device 2024, which may be used for printing instructions, data, or other information on a medium such as paper, film, or similar types of media. Furthermore, a sound recording and playback device, such as a speaker and/or microphone may optionally be coupled to bus 2011 for audio interfacing with computer system 2000. Another device that may be coupled to bus 2011 is a wired/wireless communication capability 2025 to communication to a phone or handheld palm device.
Note that any or all of the components of system [0278] 2000 and associated hardware may be used in the present invention. However, it can be appreciated that other configurations of the computer system may include some or all of the devices.
Whereas many alterations and modifications of the present invention will no doubt become apparent to a person of ordinary skill in the art after having read the foregoing description, it is to be understood that any particular embodiment shown and described by way of illustration is in no way intended to be considered limiting. Therefore, references to details of various embodiments are not intended to limit the scope of the claims which in themselves recite only those features regarded as essential to the invention. [0279]
coding of each tile of an input codestream;
outputting a plurality of packets of said each tile as part of an output codestream as a first complete tile-part;
outputting additional packets of said each tile from the remaining layers as a second complete tile-part.
2. The method defined in claim 1 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-p art is based on total bandwidth for first and second passes.
3. The method defined in claim 1 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-part is based on size of the buffer.
means for coding of each tile of an input codestream;
means for outputting a plurality of packets of said each tile as part of an output codestream as a first complete tile-part;
means for outputting additional packets of said each tile from the remaining layers as a second complete tile-part.
5. The apparatus defined in claim 4 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-part is based on total bandwidth for first and second passes.
6. The apparatus defined in claim 4 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-part is based on size of the buffer.
code of each tile of an input codestream;
output a plurality of packets of said each tile as part of an output codestream as a first complete tile-part;
output additional packets of said each tile from the remaining layers as a second complete tile-part.
8. The article of manufacture defined in claim 7 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-part is based on total bandwidth for first and second passes.
9. The article of manufacture defined in claim 7 wherein selection of packets of the plurality of packets for inclusion in the first complete tile-part is based on size of the buffer.
performing rate control on the original compressed codestream to created a new compressed codestream in response to a request by selecting a number of layers to remain in the codestream, including a parser using a packet structure containing a selection flag and setting a selection flag for each packet of the original codestream to a predetermined value to indicate whether said each packet is to be included in the new compressed codestream.
12. The method defined in claim 10 further comprising writing the new codestream based on whether selection flags of packets for the original compressed codestream are set.
13. The method defined in claim 10 further comprising reading packets in sequential order from the compressed codestream based on order information indicated in a marker.
14. The method defined in claim 13 wherein the order information comprises progression order information and the marker comprises a COD marker.
means for performing rate control on the original compressed codestream to created a new compressed codestream in response to a request by selecting a number of layers to remain in the codestream, including a parser to use a packet structure containing a selection flag and to set a selection flag for each packet of the original codestream to a predetermined value to indicate whether said each packet is to be included in the new compressed codestream to control whether individual layers are included in the new compressed codestream.
means for adding the total number of bytes associated with at least one layer, starting with the highest level layer and adding a new layer until the total number of bytes associated with the one or more layers whose bytes have been added is equal to or greater than the number of bytes desired or bytes for all layers have been added;
means for subtracting bytes associated with a last added layer when the total number of bytes associated with the one or more layers whose bytes have been added is greater than the number of bytes desired; and
17. The apparatus defined in claim 15 further comprising means for writing the new codestream based on whether selection flags of packets for the original compressed codestream are set.
18. The apparatus defined in claim 10 further comprising means for reading packets in sequential order from the compressed codestream based on order information indicated in a marker.
19. The apparatus defined in claim 18 wherein the order information comprises progression order information and the marker comprises a COD marker.
perform rate control on the original compressed codestream to created a new compressed codestream in response to a request by selecting a number of layers to remain in the codestream, including a parser using a packet structure containing a selection flag and setting a selection flag in each packet of the original codestream to a predetermined value to indicate whether said each packet is to be included in the new compressed codestream.
21. The article of manufacture defined in claim 20 further comprising instructions which, when executed by the machine, cause the machine to:
US09/800,633 2001-02-15 2001-03-06 Method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part Active 2022-12-02 US6925209B2 (en)
US09/784,928 US6898323B2 (en) 2001-02-15 2001-02-15 Memory usage scheme for performing wavelet processing
US09/800,633 US6925209B2 (en) 2001-02-15 2001-03-06 Method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part
US09/784,928 Division US6898323B2 (en) 2001-02-15 2001-02-15 Memory usage scheme for performing wavelet processing
US20030206656A1 true US20030206656A1 (en) 2003-11-06
US6925209B2 US6925209B2 (en) 2005-08-02
US09/784,928 Active 2022-06-03 US6898323B2 (en) 2001-02-15 2001-02-15 Memory usage scheme for performing wavelet processing
US09/800,932 Active 2023-02-17 US7477792B2 (en) 2001-02-15 2001-03-06 Method and apparatus for performing progressive order conversion
US09/800,530 Expired - Fee Related US7095900B2 (en) 2001-02-15 2001-03-06 Method and apparatus for performing scalar quantization with a power of two step size
US09/801,218 Active 2023-02-04 US6904178B2 (en) 2001-02-15 2001-03-06 Method and apparatus for eliminating flicker by quantizing values based on previous quantization
US09/801,360 Expired - Fee Related US7079690B2 (en) 2001-02-15 2001-03-06 Method and apparatus for editing an image while maintaining codestream size
US09/801,361 Active 2022-12-03 US6898325B2 (en) 2001-02-15 2001-03-06 Method and apparatus for clipping coefficient values after application of each wavelet transform
US09/800,633 Active 2022-12-02 US6925209B2 (en) 2001-02-15 2001-03-06 Method and apparatus for outputting a codestream as multiple tile-part outputs with packets from tiles being output in each tile-part
US09/800,934 Expired - Fee Related US7062103B2 (en) 2001-02-15 2001-03-06 Method and apparatus for specifying quantization based upon the human visual system
US09/800,832 Expired - Fee Related US6973217B2 (en) 2001-02-15 2001-03-06 Method and apparatus for sending additional sideband information in a codestream
US09/800,831 Expired - Fee Related US7072520B2 (en) 2001-02-15 2001-03-06 Method and apparatus for selecting layers for quantization based on sideband information
US09/800,687 Expired - Fee Related US6983075B2 (en) 2001-02-15 2001-03-06 Method and apparatus for performing selective quantization by manipulation of refinement bits
US10/958,895 Expired - Fee Related US7031537B2 (en) 2001-02-15 2004-10-04 Method and apparatus for specifying quantization based upon the human visual system
US11/081,281 Expired - Fee Related US7164804B2 (en) 2001-02-15 2005-03-15 Method and apparatus for eliminating flicker by quantizing values based on previous quantization
TW569159B (en) * 2001-11-30 2004-01-01 Inst Information Industry Video wavelet transform processing method
AU2003248470B2 (en) * 2002-09-30 2008-02-28 Canon Kabushiki Kaisha Digital Video Compression
TWI456620B (en) * 2012-12-28 2014-10-11 Delta Electronics Inc Power apparatus of x-ray tube, power system with the power apparatus, and method of operating the same
US3275531A (en) * 1960-07-28 1966-09-27 Knapsack Ag Purification of chloroprene by plural stage distillation and the addition of polymerization inhibitors
NO151655C (en) * 1983-02-18 1985-05-15 Holger B Nygaard Immersion suit.
AUPP918699A0 (en) * 1999-03-12 1999-04-15 Canon Kabushiki Kaisha Encoding method and appartus
AUPR192800A0 (en) * 2000-12-06 2001-01-04 Canon Kabushiki Kaisha Digital image compression and decompression
2001-02-15 US US09/784,928 patent/US6898323B2/en active Active
2001-03-06 US US09/800,932 patent/US7477792B2/en active Active
2001-03-06 US US09/800,530 patent/US7095900B2/en not_active Expired - Fee Related
2001-03-06 US US09/801,218 patent/US6904178B2/en active Active
2001-03-06 US US09/801,360 patent/US7079690B2/en not_active Expired - Fee Related
2001-03-06 US US09/801,361 patent/US6898325B2/en active Active
2001-03-06 US US09/800,633 patent/US6925209B2/en active Active
2001-03-06 US US09/800,934 patent/US7062103B2/en not_active Expired - Fee Related
2001-03-06 US US09/800,832 patent/US6973217B2/en not_active Expired - Fee Related
2001-03-06 US US09/800,831 patent/US7072520B2/en not_active Expired - Fee Related
2001-03-06 US US09/800,687 patent/US6983075B2/en not_active Expired - Fee Related
2001-12-11 EP EP01129242A patent/EP1233624A1/en not_active Ceased
2002-02-13 JP JP2002035886A patent/JP4177583B2/en not_active Expired - Fee Related
2004-10-04 US US10/958,895 patent/US7031537B2/en not_active Expired - Fee Related
2005-03-15 US US11/081,281 patent/US7164804B2/en not_active Expired - Fee Related
US6898325B2 (en) 2005-05-24
US6973217B2 (en) 2005-12-06
JP2002359850A (en) 2002-12-13
US20030206661A1 (en) 2003-11-06
US7062103B2 (en) 2006-06-13
US20040028284A1 (en) 2004-02-12
US7079690B2 (en) 2006-07-18
US6904178B2 (en) 2005-06-07
US6983075B2 (en) 2006-01-03
JP4177583B2 (en) 2008-11-05
US20050041878A1 (en) 2005-02-24
US7031537B2 (en) 2006-04-18
US20030210827A1 (en) 2003-11-13
US20040057628A1 (en) 2004-03-25
US20030215150A1 (en) 2003-11-20
US7095900B2 (en) 2006-08-22
US20020168112A1 (en) 2002-11-14
US20030215146A1 (en) 2003-11-20
US7072520B2 (en) 2006-07-04
US7477792B2 (en) 2009-01-13
US20040120585A1 (en) 2004-06-24
US20030210826A1 (en) 2003-11-13
US7164804B2 (en) 2007-01-16
US20030219166A1 (en) 2003-11-27
EP1233624A1 (en) 2002-08-21
US6925209B2 (en) 2005-08-02
US20050163388A1 (en) 2005-07-28
US6898323B2 (en) 2005-05-24
JP5683350B2 (en) 2015-03-11 Method for providing an interactive remote browsing of compressed digital video signals and systems