Abstract:
Most output display devices such as printers and displays require certain minimum timings for delivery of data to be displayed. The use of compressed image stores as input to these displays requires that the image be decompressed within the minimum time limits as required by the display. The present invention discloses an effective method for ensuing that these minimum timings are met through the division of the image into bands (B1,B2, . . . Bn) and the recompression of those bands that fail to meet the minimum timing requirements of the output display device.

Description:
The present invention relates to the display of compressed images on a synchronous output device such as a colour computer display and/or colour printers, and, in particular, to the display of colour images on a raster colour display apparatus. 
     BACKGROUND ART 
     Some attributes of the prior art will now be discussed with reference to FIG. 1 and FIG. 2 in which: 
     FIG. 1 illustrates a first example decompression method for displaying a compressed image, and 
     FIG. 2 illustrates a second example decompression method for displaying a compressed image. 
     FIG. 1 illustrates a compressed image memory 1 that holds a compressed image which is decompressed by a decompression apparatus 2 in a pipeline fashion with the output being forwarded for display on a display device 3. Often the display device has certain display requirements, such as, for example, each pixel of a given line having to be forwarded to the display device 3 within a minimum time period for the display to function properly. For example, such is the case in rasterised displays. 
     If the decompression of a particular portion of the compressed image takes longer than the maximum time period for the forwarding to the display device 3, then the compressed image will not be able to be displayed according to the method used in FIG. 1 as it will not be decompressed in time. In a display device 3 which requires synchronous input of uncompressed data it is necessary to maintain a consistent output data rate from the decompression process. It has been found that if the volume of compressed data is too large, then it is not possible to successfully decompress all the data at the data rate required for real time operation. Thus, the synchronous nature of the system cannot be maintained and the image cannot be successfully displayed. 
     Referring now to FIG. 2 there is shown a second example decompression method for displaying a compressed image. This method uses a batch arrangement designed to overcome the problems of the first method by decompressing, using a decompression apparatus 5, a compressed image stored in a compressed image memory 4, and storing the decompressed image in a decompressed image store 6, which can then be forwarded to a display device 7 within the time requirements. However, this method fails to use the advantages of a compressed frame store as the entire image must be decompressed first and a large storage means is required to store the decompressed image before it is forwarded to the display device 7. The need for such a large storage means often results in an undesirable and significant increase in expense of a corresponding computer graphics system using such an arrangement. 
     SUMMARY OF THE INVENTION 
     It is an object of the present invention to provide a means for decompressing a compressed image wherein the compressed image will always be decompressed within the real time requirements for the display on a display device. 
     In accordance with a first aspect of the present invention, there is provided a method for the real-time display of a compressed image, the method comprising the steps of: 
     (a) dividing the image into a plurality of bands, 
     (b) compressing each of said bands using a first compression factor. 
     (c) determining those invalid bands that cannot be decompressed within a first predetermined time and 
     if the number of invalid bands is less than a first predetermined number, then filtering each band so that its decompression time is less than said first predetermined time, otherwise 
     if the number of invalid bands is greater than or equal to said first predetermined number, recompressing the whole image with a second compression factor such that each band can be decompressed in a shorter time interval less than said first predetermined time. 
     In accordance with a second aspect of the present invention, there is provided a method for the real-time display of a compressed image, the method comprising the steps of: 
     (a) dividing the image into a plurality of bands. 
     (b) compressing each of said bands using a first compression factor. 
     (c) determining those invalid bands that cannot be decompressed within a first predetermined time and 
     if the number of invalid bands is less than a first predetermined number, then filtering each band so that its decompression time is less than said first predetermined time, otherwise 
     if the number of invalid bands is greater than or equal to said first predetermined number, recompressing the whole image with a second compression factor such that each band can be decompressed in a shorter time interval and applying step (c) to said recompressed image. 
     Preferably said filtering includes convolution of an invalid band with a predetermined convolution matrix. 
     Preferably, said first predetermined number is chosen to be a proportion of the total number of bands in the image. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     A preferred embodiment of the present invention will now be described with reference to the remaining drawings in which: 
     FIG. 1 illustrates a first example of decompression method for displaying a compressed image. 
     FIG. 2 illustrates a second example of decompression method for displaying a compressed image. 
     FIG. 3 illustrates an image made up of a number of bands. 
     FIG. 4 illustrates the same image as FIG. 3 made up of compressed bands. 
     FIG. 5 illustrates a 3×3 convolution matix used in the preferred embodiment. 
     FIG. 6 illustrates an example decompressed band, and 
     FIGS. 7 to 10 are flowcharts which illustrate the method used in the preferred embodiment. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     The preferred method of compression of the image is the compression method as devised by the CCITT/ISO Joint Photographic Expert&#39;s Group (JPEG) based on the Discrete Cosine Transform and Huffman encoding processes and published in ISO/IEC JTC1/SC2/WG8 JPEG Technical Specification Rev 5 Jan. 16, 1990. 
     One implementation of a JPEG processor system has been implemented in silicon by C-Cube Microsystems as the CL550B device. The JPEG method allows the size of the compressed data relative to the uncompressed data to be selected (the compression ratio) and implementations of the JPEG algorithm such as the CL550B require a finite time to decompress a compressed image that has been compressed using the JPEG method. 
     Referring now to FIG. 3 and FIG. 4 there is shown an input image 8 (FIG. 3) and its corresponding compressed image 9 (FIG. 4). For the purposes of the preferred embodiment the JPEG compressed image 9 is organized in bands CB1, CB2, CB3 . . . CBn, corresponding to fixed size image bands B1, B2, B3, . . . Bn in the input image 8. The bands can, for example, contain eight scanlines 13 of image data as in the standard JPEG algorithm and a compressed image is comprised of a large number of compressed bands. 
     The situation arises that some of these bands may not successfully decompress at the synchronous rate required. 
     Firstly a determination of those bands of a current image which are bad must be made. A band is considered &#34;bad&#34; if the time taken for it to be decompressed is greater than some predetermined time, that time being the maximum time required to get a band ready to send to a display device, such as a colour printer or video display. 
     Preferably, if a band has been previously decompressed, the time required to decompress the band is stored with the band itself or in a separate table of all the band decompression times for a current image. If however, there is no timing information available for some or all bands, a test-print operation can be performed in order to determine the band decompression timings. 
     If the number of bad bands requiring fixing is relatively low, for example less then say 5%, each bad band can be altered so that its decompression time is less than the predetermined time. One method of reducing the decompression time without substantially altering the original bad band is to reduce the amount of high frequency content in the bad band data. This can be achieved by blurring the luminance information, thereby improving the compressiblity of the current image. This can be done using a pixel averaging technique such as convolution over a group of pixels. 
     As the JPEG compressed bad band data cannot be manipulated directly, it is necessary to decompress the JPEG bad band data, perform the convolution operation, and then recompress the data. The overall effect of this is blurring of the image in the bad bands. In the preferred embodiment, image data has the high frequency content of its luminance component reduced by pixel averaging over a certain area. 
     Referring now to FIG. 5 there is shown a 3×3 convolution matrix 10 used in the preferred embodiment. The convolution matrix 10 consists of a current pixel position containing the value eight (8) and neighbouring pixel elements containing the value one (1). 
     Referring now to FIG. 6 there is shown an example of a decompressed band 11. When the convolution matrix 10 of FIG. 5 is applied to the decompressed band 11, each pixel 12 is averaged by multiplying itself by 8 and adding all its neighbours to the result before dividing the new result by 16 and storing it back in the current decompressed band. This particular convolution process has been designed to operate highly efficiently on a binary based digital processor. 
     If a substantial number of the bands of a current image are bad bands (say greater than 5%), then the entire image can be decompressed and recompressed at a higher compression ratio, which results in a lowering of the quality of the overall image, but an advantageous decrease in decompression time. The decision to recompress the entire image can be made to coincide with situations where the full recompression process will take less time than the blurring of a small number of bad bands in the input image. A higher compression ratio results in a decrease in the decompression time for each given band of the image, thereby reducing the likelihood of the occurrence of bad bands. 
     Once the image has been recompressed, the process of location of bad bands or further recompression can be applied repeatedly to the current image until a final compressed image is obtained which meets those real-time printing requirements that may be required. 
     It should be noted that the above process of blurring and recompression are destructive operations in that the image is left permanently altered after a printing or display operation that has had to alter the image. Hence, the saving of the current image (or rendering onto it) after such a print will be saving (or rendering onto) an altered image. 
     Appendix A shows a pseudo code implementation of the recompression technique of the preferred embodiment for implementation on a computer system. The method disclosed can be implemented on any computer system configured with a display device having certain minimum synchronous display rate requirements as elucidated above. 
     Referring now to FIGS. 7 to 10, there is shown flowcharts of the pseudo-code of Appendix A. 
     FIG. 7 shows the overall main program &#34;Ensure Current Image Printable&#34;. After initialization of variables 20, a procedure Get Maximum Decompress Time 21 is called to determine the number of &#34;Bad Bands&#34; which cannot be decompressed in time. If the number of bad bands exceeds a predetermined number 22 the entire image is recompressed 23 at a higher compression ratio. Otherwise individual bad bands are blurred 24 to reduce their compression time. The program then enters a return loop 25 to again determine if there are any remaining bad bands. 
     Referring now to FIG. 8, there is shown the procedure Get Maximum Decompress Time 21 of FIG. 7 in more detail. This procedure first performs a test print 26 for all bands for which decompression timings are not available and then, for all bands in the image 27, if the decompression time of the band exceeds a threshold, the number of bad bands (initially set to zero) is incremented. 
     Referring now to FIG. 9, there is shown the procedure Blur Bad Bands 24 of FIG. 7. 
     After initialization, of the variables 29, each band in the image 30 is checked 31 to determine if it&#39;s decompression time exceeds a predetermined threshold. If this is the case then the band is blurred. If the blurring is unsuccessful, the blur band procedure sets unsuccessful blurring to true (thereby causing the image to be recompressed), and exits. 
     Referring now to FIG. 10, there is shown the procedure Blur Band 32 of FIG. 9 in more detail. The blurring of a band involves the decompression of the band 34, it&#39;s convolution of the decompressed band with a convolution matrix previously described 35 and it&#39;s recompression 36. 
     The foregoing describes only one embodiment of the present invention, and modifications, obvious to those skilled in the art, can be made thereto without departing from the scope of the present invention. 
     APPENDIX A 
     The following paragraphs set out a pseudo code representation of the preferred embodiment. The main program is set out under the heading `Ensure Current Image Printable`. This program calls three sub-programs being `Blur Bad Bands`, `Get Maximum Decompress Time` and `Blur Band`. These are set out under separate headings. 
     Ensure Current Image Printable 
     Given a band-decompress-time threshold, ensure that all those bands in the current image are printable. This means to look at the decompress times (if they exist) and ensure they are less than or equal to the band-decompress-time given. The current image is backed up automatically to make error-recovery easier, but after successful completion the backup is thrown away. Thus any change is permanent. 
     
         ______________________________________ENSURE CURRENT IMAGE PRINTABLESet the maximum decompress time to zero.Set the maximum compress size to zero.Reset &#34;compresses are failing&#34; global flagSet the number of bands we have to repair to zero.(Keep looping until the current image has been altered enough to beprintable)LOOPIF need to recompressTHENCALL RECOMPRESS AT A WORSE QUALITY specifying thenew reduced quality and returning whether or not furtherrecompression is required and, if so, the new quality to applyCONTINUE the loopENDIFIF GET MAXIMUM DECOMPRESS TIME giving threshold timingvalue returns a non-zero number of bands with bad timingsTHENIF the number of bands exceeding limit &lt;=0.05 * total number bandsTHEN(It is required to blur some of the bands)CALL BLUR BAD BANDS giving decompress timingthreshold and returning the boolean requirement for furtherrecompressELSESet need to recompress to TRUEWork out new quality based on maximum decompress timingENDIFENDIFIF there is no need to recompressTHENBREAK the loopENDIFENDLOOPRETURN CURRENT IMAGE PRINTABLEEND ENSURE CURRENT IMAGE PRINTABLE______________________________________ (1)Note that if the current image does not currently have a defined quality value (eg, we&#39;ve loaded it from an external file) then it is required to guess one based on, say, the Q factor. If the quality is already zero it is required to return an error &#34;cannot print even at zero quality (2)The 5% value is a config file parameter. 
    
     Blur Bad Bands 
     Given a band decompress timing threshold, blur the content of any band that has a decompress timing greater than the nominated threshold. If any band after blurring *still* has bad decompress timing, report this to our caller as a setting of &#34;need to recompress&#34; along with a new quality. 
     
         ______________________________________BLUR BAD BANDSSet &#34;need to recompress&#34; to FALSEReset any &#34;compresses are failing&#34; global flagSet &#34;max compress size&#34; to zeroCALL CREATE NEW CURRENT IMAGE indicating to leave thequality unchanged.Allocate a new band that can contain an entire decompressedband of the current image.FOR all bands in the current imageIF the band exists and has a decompress-time larger than theband decompress timing threshold givenTHENCALL BLUR BAND giving the band number in the current imageand the newly allocated band.IF compresses are failingORthe band decompress timing is larger than the thresholdTHENSet &#34;need to recompress&#34; to TRUEWork out new quality based on either timing or max compresssize.BREAK out of the loopENDIFENDIFENDFORFree the band allocated to hold an uncompressed current image band.IF &#34;abort&#34; is setTHENCALL DROP WORKING IMAGE INFO specifying new currentimageRETURN BLURRING FAILED &#34;aborting&#34;ELSEIF &#34;need to recompress&#34; is set to TRUETHEN(drop the new image, and try recompressing everything . . . )CALL DROP WORKING IMAGE INFO specifying new currentimageELSE(Things have completed -- it is possible to get rid of the originalcurrent image, replacing it with the new current image)CALL TRANSFER NEW IMAGE TO CURRENT IMAGEENDIFRETURN OK with &#34;need to recompress&#34; settingEND BLUR BAD BANDS______________________________________ 
    
     Get Maximum Decompress Time 
     Given a band decompress time threshold, return the number of bands within the current image that exceed the threshold and what the maximum decompress timing is. All bands should be fixed already (i.e., decompress to the correct current image height). If any band does not have decompress timing info, it will be decompressed to get it. It is assumed that a blank band will never exceed any decompress timing threshold. 
     
         ______________________________________GET MAXIMUM DECOMPRESS TIMESet number-of-exceeding-bands to zeroSet maximum-decompress-time to zero(Ensure all bands have timing information)FOR all bands in the current image that exist and have nodecompress-timing-informationAdd band (and possibly text-detect) to those to be test printedENDFORIF the number of bands to be test printed is non-NULLTHENFOR all bands to be test printed  Decompress band and get decompression timingsENDIF(Actually look at band timings now -- they should all be there)FOR all bands in current imageIF the band existsTHENIF the decompress-time is larger than the specified threshold givenTHENRecord as maximum-decompress-time if greaterIncrement the number-of-exceeding-bandsENDIFENDIFENDFORRETURN the maximum-decompress-time and number-of-exceeding-bandsEND GET MAXIMUM DECOMPRESS TIME______________________________________ 
    
     Blur Band 
     Given a band number and a band comprising an uncompressed band buffer, decompress the band, blur it, and then compress it back into the current image. This is performed before a print operation on bad bands. The compress is synchronised so that the decompress time for the band is reported to our caller.