Patent Publication Number: US-2016249049-A1

Title: Performance of the JPEG Estimated Spectrum Adaptive Postfilter (JPEG-ESAP) for Low Bit Rates

Description:
STATEMENT OF FEDERAL RIGHTS 
     The invention described herein was made by employees of the United States Government and may be manufactured and used by or for the Government for Government purposes without the payment of any royalties thereon or therefore. 
    
    
     FIELD 
     The present invention generally relates to image enhancement, and more particularly, to frequency-based, pixel-adaptive filtering for low bit rates for JPEG format color images. 
     BACKGROUND 
     JPEG, a lossy image compression standard based on the discrete cosine transform (DCT), was introduced in 1992. JPEG is firmly entrenched in digital cameras (e.g., Nikon™), smart phones (e.g., Android RAZR™), and web browsers (e.g., Internet Explorer™, Firefox™, etc.). Additionally, JPEG is the basis of today&#39;s MPEG-2 high definition television (HDTV) digital satellite news gathering (DSNG) broadcast industry and H.262 video compression. 
     JPEG-2000 (i.e., JP2), also a lossy image compression standard, but based on the discrete wavelet transform (DWT), was published in 1999. JPEG-2000 has been a JPEG-competing standard since its inception. In 2005, the U.S. Library of Congress embraced JPEG-2000 for permanent electronic storage of digitized images. JPEG-2000 is also supported by other smart phone brands (e.g., Apple iPhone™) 
     The estimated spectrum adaptive postfilter (ESAP) reuses the DCT coefficients of a decoded grayscale JPEG image to postfilter and enhance the image. ESAP pixel-adaptive filters minimize the JPEG 8×8 blocking distortion without requiring any additional bit stream overhead. A key improvement is that ESAP not only adaptively filters out the JPEG blocking distortion, but also does not introduce the ringing artifacts inherent to JPEG-2000. ESAP complies with the Independent JPEG Group (IJG) Software™, Release 8c, found at http://www.ijg.org. Baseline JPEG uses Huffman entropy coding, whereas JPEG-2000 and ESAP-AC both use arithmetic entropy coding (AC). 
     Most of the quality improvement of JPEG-2000 over baseline JPEG images is likely due to the arithmetic entropy coding utilized by JPEG-2000 versus the Huffman coding initially used by JPEG. Similarly, when JPEG uses AC instead of Hoffman coding, as in the IJG Software™ command example:
         cjpeg -quality 5 -arithmetic img.ppm img.jpg,       

     a remarkable visual improvement of the JPEG image is observed. When the image is further filtered by the pixel-adaptive ESAP process, the ESAP-AC image visual and peak signal-to-noise ratio (PSNR) quality either matches or surpasses the quality of JPEG-2000. The excellent DCT energy compaction properties are thus exulted with usage of AC. 
     A previous version of the ESAP algorithm was applied to grayscale images only. ESAP, in its most recent version, operates in the RGB color image space. 
     SUMMARY 
     Certain embodiments of the present invention may provide solutions to the problems and needs in the art that have not yet been fully identified, appreciated, or solved by conventional image processing algorithms. For example, some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images. 
     In an embodiment, a computer-implemented method includes creating JPEG quantization coefficients, by a computing system, for each respective color plane of a plurality of color planes. The computer-implemented method also includes JPEG decoding each color plane, by the computing system, to generate YCbCr DCT coefficients for ESAP postfiltering. The computer-implemented method further includes ESAP postfiltering each of the color planes, by the computing system, and outputting a JPEG image file and an ESAP postfiltered image file, by the computing system. 
     In another embodiment, a computer program is embodied on a non-transitory computer-readable medium. The program configured to cause at least one processor to convert an input image to a portable pix map (PPM) format and decompose the PPM image into color planes in portable graymap (PGM) format. The computer program is also configured to cause the at least one processor to create JPEG quantization coefficients for each respective color plane, JPEG decode each color plane to generate YCbCr DCT for ESAP postfiltering and ESAP postfilter each color plane. The computer program is further configured to cause the at least one processor to combine PGM JPEG planes for each color, combine PGM ESAP planes for each color into PPM, and output a JPEG image file and an ESAP postfiltered image file. 
     In yet another embodiment, an apparatus includes memory storing computer program instructions and at least one processor configured to execute the computer program instructions stored in the memory. The at least one processor is configured to receive a video comprising a series of images. For each of the received images, the at least one processor is also configured to create JPEG quantization coefficients for each respective color plane of a plurality of color planes, JPEG decode each color plane to generate YCbCr DCT coefficients for ESAP postfiltering, ESAP postfilter each of the color planes, and output a JPEG image file and an ESAP postfiltered image file. The at least one processor is further configured to output a sequence of PPM images and re-code the sequence of PPM images into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       In order that the advantages of certain embodiments of the invention will be readily understood, a more particular description of the invention briefly described above will be rendered by reference to specific embodiments that are illustrated in the appended drawings. While it should be understood that these drawings depict only typical embodiments of the invention and are not therefore to be considered to be limiting of its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings, in which: 
         FIG. 1  is a block diagram illustrating the grayscale ESAP decoder algorithm. 
         FIG. 2  illustrates a series of Lena images showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention. 
         FIG. 3  illustrates the Lena, peppers, airplane, Barbara, and mandrill images referred to in Table 2 that were processed according to an embodiment of the present invention. 
         FIG. 4  is a flowchart illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. 
         FIG. 5  is a block diagram of a computing system configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS 
     Some embodiments of the present invention pertain to using JPEG-ESAP for frequency-based, pixel-adaptive filtering for low bit rates (i.e., less than 0.25 bits/pixel) for JPEG format color images. For RGB, for example, an image is decomposed into three color bands—red, green, and blue. However, other bands may be used in certain embodiments, for instance, RGBY. The JPEG-ESAP algorithm is then applied for each band (e.g., once for red, once for green, and once for blue in RGB) and the output of each application of the algorithm is rebuilt as a color image by combining the ESAP filtered RGB planes (for RGB schemes) in grayscale portable graymap (PGM) format into color portable pix map (PPM) format. 
     ESAP estimates two-dimensional (2-D) pixel-adaptive bandwidths directly from the dequantized DCT coefficients without requiring any additional overhead. The ESAP decoder algorithm  100  that is applied to each color plane is shown in  FIG. 1 . For more detail on a grayscale implementation, see Linares I., Mersereau R. M., and Smith M. J. T., “ JPEG Estimated Spectrum Adaptive Postfilter Using Image - Adaptive Q - Tables and Canny Edge Detectors ,” ISCAS &#39;96 International Symposium on Circuits and Systems, IEEE, Atlanta, Ga., pp. 722-725, Vol. 2, May 12, 1996, as well as U.S. Pat. No. 6,760,487. 
     A simple, one-pass, non-iterative process is used for w [m] bandwidth estimation and for ESAP h m [n] filtering. Each DCT block 2-D bandwidth is vertically and horizontally separable and determined from Table 1 below. 
     
       
         
           
               
               
               
             
               
                   
                   
               
               
                   
                 Highest DCT Coefficient 
                 Normalized 1-D Bandwidth 
               
               
                   
                   
               
             
            
               
                   
                 c 0   
                 0.125π 
               
               
                   
                 c 1   
                 0.250π 
               
               
                   
                 c 2   
                 0.375π 
               
               
                   
                 c 3   
                 0.500π 
               
               
                   
                 c 4   
                 0.625π 
               
               
                   
                 c 5   
                 0.750π 
               
               
                   
                 c 6   
                 0.875π 
               
               
                   
                 c 7   
                 1.000π 
               
               
                   
                   
               
            
           
         
       
     
       FIG. 2  illustrates a series of images  200  showing output of the 2-D, 1:8 ESAP frequency up sampler for the blue plane, according to an embodiment of the present invention. Original image  210  is in color, but is shown in grayscale here. The 2-D interpolated vertical frequency (IVF) image  220  and the interpolated horizontal frequency (IHF) image  230 , which are grayscale, show the interpolated DCT bandwidths w[m] scaled for 8-BPP display. 
     The IVF and IHF then drive a separable, pixel-adaptive, non-causal 2-D h m [n] Hamming window low pass filter that is applied in a raster manner to each pixel and its surrounding 17×17 area of support. This filtering minimizes the DCT blocking while performing negligible filtering in the high frequency areas of the image consisting of edges or highly textured areas. This process takes advantage of the ability of the human visual system to mask the JPEG quantization noise present in the high frequency areas. 
     Some experimental observations regarding JPEG, ESAP-AC, and JPEG-2000 based on the processed color test images include, but are not limited to, the following: (1) the ESAP-AC PSNR is always higher than the DWT-based JPEG-2000 images compressed at similar bit rates, for typical test images; (2) ESAP-AC performs equally to or better than JPEG-2000 images; (3) ESAP visual quality is better than JPEG-2000 for high frequency content images; (4) ESAP visual quality is comparable to JPEG-2000 for low frequency content images, and the arithmetic coding entropy option for JPEG should be used rather than the baseline Huffman coding option; (5) JPEG AC provides a better quality image than JPEG Huffman coding at comparable bit rates; (6) ESAP-AC should be turned on for bit rates below 0.5 BPP; (7) for bit rates higher than 0.5 BPP, JPEG images should only choose to use JPEG-AC without ESAP post-processing; (8) ESAP may be implemented in fast electronics, such as FPGAs, ASICs, or DSPs, which could reduce the processing time by several orders of magnitude; (9) ESAP is an optional post-processing step at the receiver or decoder fully capable of removing the JPEG/MPEG-2 blocking distortion; and (10) ESAP does not increase the coded stream bit rate. 
     Some embodiments are also potentially applicable to MPEG-2 video. An MPEG-2 Group of Pictures (GOP) is a sequence of JPEG-like, DCT coded intraframes (I), bidirectional (B) frames, and forward predicted (P) frames. All types of frames may be coded with JPEG-AC, and then ESAP may be applied to filter the decoded frames. A typical MPEG-2 CCIR-601 GOP sequence with an I-frame distance of 15 and a P-frame distance of 3, such as IBBPBBPBBPBBPBB, has an I-frame bit rate of about 1.2 BPP, a P-frame bit rate of 0.6 BPP, and a B-frame bit rate of 0.23 BPP. The equivalent average bit rate for this GOP sequence is 0.38 BPP. 
     By extension to each frame, applying JPEG-AC and ESAP to MPEG-2 frames in a GOP sequence could reduce the MPEG-2 bit rate by a factor of two without detrimentally affecting the video quality, and while retaining an equivalent video quality at an approximate average bit rate of 0.19 BPP. Comparative objective PSNR among JPEG, ESAP, and JPEG-2000 versions of a series of images  300  at 0.20, 0.25, 0.50, and 1.00 BPP is shown in Table 2 below with respect to the Lena  310 , peppers  320 , airplane  330 , Barbara  340 , and mandrill  350  images. 
     
       
         
           
               
            
               
                   
               
               
                 COMPARATIVE OBJECTIVE PSNR AMONG 
               
               
                 JPEG, ESAP, AND JPEG-2000 IMAGES 
               
            
           
           
               
               
               
               
               
               
               
            
               
                   
                   
                   
                   
                   
                   
                 Δ 
               
               
                   
                 Base- 
                   
                 Δ PSNR 
                   
                   
                 PSNR 
               
               
                   
                 line 
                 ESAP- 
                 ESAP- 
                   
                 Δ PSNR 
                 ESAP 
               
               
                   
                 JPEG 
                 AC 
                 AC over 
                 JP2 
                 JP2 over 
                 over 
               
               
                   
                 PSNR 
                 PSNR 
                 JPEG 
                 PSNR 
                 JPEG 
                 JP2 
               
               
                   
                   
               
            
           
           
               
               
               
               
               
               
               
            
               
                 Image @ 
                   
                   
                   
                   
                   
                   
               
               
                 0.20 BPP 
               
               
                 Lena 
                 26.15 
                 30.88 
                 4.72 
                 29.41 
                 3.26 
                 1.47 
               
               
                 Peppers 
                 24.34 
                 29.62 
                 5.28 
                 27.74 
                 3.41 
                 1.88 
               
               
                 Airplane 
                 24.08 
                 30.45 
                 6.38 
                 28.70 
                 4.63 
                 1.75 
               
               
                 Barbara 
                 22.19 
                 25.59 
                 3.40 
                 25.41 
                 3.22 
                 0.18 
               
               
                 Mandrill 
                 19.76 
                 21.55 
                 1.79 
                 20.88 
                 1.13 
                 0.66 
               
               
                 Image @ 
               
               
                 0.25 BPP 
               
               
                 Lena 
                 28.55 
                 31.65 
                 3.10 
                 30.11 
                 1.56 
                 1.54 
               
               
                 Peppers 
                 27.59 
                 30.48 
                 2.89 
                 28.44 
                 0.85 
                 2.04 
               
               
                 Airplane 
                 27.69 
                 31.55 
                 3.87 
                 29.55 
                 1.86 
                 2.01 
               
               
                 Barbara 
                 23.91 
                 26.47 
                 2.56 
                 26.36 
                 2.46 
                 0.10 
               
               
                 Mandrill 
                 20.52 
                 22.00 
                 1.48 
                 21.33 
                 0.81 
                 0.67 
               
               
                 Image @ 
               
               
                 0.50 BPP 
               
               
                 Lena 
                 32.84 
                 33.87 
                 1.03 
                 32.39 
                 −0.55 
                 1.58 
               
               
                 Peppers 
                 31.76 
                 32.43 
                 0.67 
                 30.70 
                 −1.06 
                 1.74 
               
               
                 Airplane 
                 33.05 
                 34.77 
                 1.72 
                 32.69 
                 −0.36 
                 2.08 
               
               
                 Barbara 
                 27.94 
                 30.04 
                 2.10 
                 29.46 
                 1.53 
                 0.57 
               
               
                 Mandrill 
                 22.96 
                 23.90 
                 0.94 
                 22.90 
                 −0.06 
                 1.00 
               
               
                 Image @ 
               
               
                 1.00 BPP 
               
               
                 Lena 
                 35.68 
                 36.29 
                 0.61 
                 34.40 
                 −1.28 
                 1.89 
               
               
                 Peppers 
                 33.93 
                 34.23 
                 0.30 
                 32.56 
                 −1.37 
                 1.67 
               
               
                 Airplane 
                 37.18 
                 38.14 
                 0.96 
                 36.00 
                 −1.18 
                 2.14 
               
               
                 Barbara 
                 32.86 
                 34.36 
                 1.50 
                 32.94 
                 0.08 
                 1.42 
               
               
                 Mandrill 
                 25.44 
                 26.33 
                 0.89 
                 24.92 
                 −0.52 
                 1.41 
               
               
                   
               
            
           
         
       
     
     A visual and PSNR comparison of baseline JPEG images coded at 1.0 BPP (Table 2, fourth table) versus the quality of the same images coded at half the bit rate (0.5 BPP in the third table) using ESAP-AC (column 3), demonstrates the two times compression potential of ESAP-AC versus baseline JPEG at one times compression. Similarly, comparing column 3 (ESAP-AC) versus column 5 (JPEG-2000 or JP2) images compressed at the same bit rate shows the improved visual quality of ESAP-AC over baseline coded JP2 images. The same comparisons can be made for the 0.5 versus 0.25 BPP rates as well. All of the images listed in Table 2 can also be viewed in full detail at the NASA ITPO website http://techtransfer.gsfc.nasa.gov/ft_tech_esap.shtm. The website details the PSNR, BPP, and CR statistics for each image as well. 
     ESAP can also remove blocking artifacts in MPEG-2 GOP frames. As a related example of MPEG-2 laboratory video processing, frame sequences extracted from MPEG-2 video using ffmpeg software, then compressed at half or a third of their original bit rate using ffmpeg, and in parallel, also post-processed with ESAP-AC before reassembling into an ESAP-AC ffmpeg video sequence, show significant video improvement, with imperceptible or minimized DCT blocking over their respective 2× and 3× compressed MPEG-2 sequences. In these ffmpeg video experiments, it was verified that the ESAP-AC MPEG-2 video sequences compare very well in visual quality against their respective MPEG-4 sequences coded at the same 2× and 3× video compression ratios. 
     This breakthrough bit rate reduction creates exciting new opportunities for digital video content service providers, webcasting, and satellite video. Some such opportunities include, but are not limited to, the following: (1) ESAP-AC could make 50% of the digital bandwidth reduction available to approximately double the number of digital MPEG-2 HDTV terrestrial channels over the same bandwidth without changing the broadcasting infrastructure; (2) ESAP-AC could also allow the extra 50% bandwidth to seamlessly integrate, multiplex, and provide UHDTV MPEG-4 content alongside HDTV MPEG-2 over the same digital pipe for dual, HDTV-UHDTV compatible receiver sets; (3) decoupled coexistence of both HDTV standards should provide many more choices for commercial video providers and consumers; and (4) Earth orbiting and planetary spacecraft video would certainly benefit from lower bit rates as well that could improve the SNR of the received signal. 
     As discussed above, the ESAP algorithm may be applied to RGB color images. The ESAP-AC PSNR is always higher than default settings JPEG-2000 images compressed at similar bit rates. ESAP-AC post-processing yields better PSNRs than default settings compressed JPEG-2000 images due to ESAP&#39;s excellent ability to remove DCT blocking artifacts without introducing the DWT ringing inherent to JPEG-2000. 
     ESAP Algorithm for RGB Jpeg Image Processing 
     An embodiment of the ESAP algorithm may be implemented in the IJG Software™ discussed above using the cjpeg -arithmetic option and the ImageMagick™ convert utility, which can be found at http://www.imagemagick.org. The pseudocode is shown in the esap_script below: 
     
       
         
           
               
               
             
               
                   
               
             
            
               
                 esap_script 
                 image.fmt  QF  esap_img.ppm 
               
            
           
           
               
            
               
                 // The input image format “fmt” is any of: 
               
               
                 // PPM, PGM, GIF, JPG, TIFF, BMP, etc. The JPEG quality factor “QF” 
               
               
                 // is a [0..100] integer. The output is the filtered esap_img.ppm 
               
               
                 image. 
               
               
                 //  1. Convert input image to a portable pix map (PPM) format, 
               
               
                 // and then decompose into RGB planes in portable graymap (PGM) 
               
               
                 format. 
               
            
           
           
               
               
               
            
               
                 convert 
                  image.fmt 
                 image.ppm 
               
               
                 ppm2rgb 
                 image.ppm 
                 image.r.pgm  image.g.pgm  image.b.pgm 
               
            
           
           
               
            
               
                 // 2. Create JPEG RGB Quantization Coefficients 
               
               
                 cjpeg  -quality QF -arith image.r.pgm   image.r.jpg 
               
               
                 cjpeg  -quality QF -arith image.g.pgm  image.g.jpg 
               
               
                 cjpeg  -quality QF -arith image.b.pgm  image.b.jpg 
               
               
                 // 3. JPEG decode each R, G, and B planes. Then ESAP-filter each plane. 
               
               
                 // The modified djpeg decoder generates the YCbCr DCT coefficients 
               
               
                 // necessary to perform ESAP postfiltering. 
               
            
           
           
               
               
            
               
                 djpeg 
                 image.r.jpg  image.r.djpg.pgm 
               
               
                 esap 
                 image.r.djpg.pgm  DCTcoeffs.r  esap_image.r.pgm 
               
               
                 djpeg 
                 image.g.jpg  image.g.djpg.pgm 
               
               
                 esap 
                 image.g.djpg.pgm  DCTcoeffs.g  esap_image.g.pgm 
               
               
                 djpeg 
                 image.b.jpg  image.b.djpg.pgm 
               
               
                 esap 
                 image.b.djpg.pgm  DCTcoeffs.b  esap_image.b.pgm 
               
            
           
           
               
            
               
                 // 4. Combine RGB PGM JPEG planes into ppm 
               
            
           
           
               
               
            
               
                 rgb2ppm 
                  image.r.djpg.pgm  image.g.djpg.pgm 
               
               
                   
                 image.b.djpg.pgm\ 
               
               
                   
                 image.djpg.ppm 
               
            
           
           
               
            
               
                 // 5. Combine RGB PGM ESAP planes into ppm 
               
            
           
           
               
               
            
               
                 rgb2ppm 
                  esap_image.r.pgm  esap_  image.g.pgm 
               
               
                   
                 esap_image.b.pgm\ 
               
               
                   
                 esap_image.ppm 
               
            
           
           
               
            
               
                 // 6. Compute grayscale Mean Square Errors (MSEs) for each color plane 
               
               
                 // and output results to file 
               
            
           
           
               
               
               
               
            
               
                 mse 
                 image.r.pgm 
                 image.r.djpg.pgm 
                 &gt;&gt; file.txt 
               
               
                 mse 
                 image.g.pgm 
                 image.g.djpg.pgm 
                 &gt;&gt; file.txt 
               
               
                 mse 
                 image.b.pgm 
                 image.b.djpg.pgm 
                 &gt;&gt; file.txt 
               
               
                 mse 
                 image.r.pgm 
                 esap_image.r.pgm 
                 &gt;&gt; file.txt 
               
               
                 mse 
                 image.g.pgm 
                 esap_image.g.pgm 
                 &gt;&gt; file.txt 
               
               
                 mse 
                 image.b.pgm 
                 esap_image.b.pgm 
                 &gt;&gt; file.txt 
               
            
           
           
               
            
               
                 // 7. Compute the peak signal-to-noise ratios (PSNRs) as a function of the 
               
               
                 // above MSEs for the JPEG and ESAP images. Also compute PPM 
               
               
                 // compression factor (CF) X:1 ratios and bit rates (BPPs). 
               
               
                 compute_psnr_cf &gt;&gt; file.txt 
               
               
                   
               
            
           
         
       
     
     The PSNR of an RGB image is defined as: 
       PSNR RGB =10*log 10 (255 2 /(MSE R +MSE G +MSE B )/3)  (1)
 
     For each color plane c={R, G, or B}, the mean-square-error (MSE) is: 
     
       
         
           
             
               
                 
                   
                     MSE 
                     C 
                   
                   = 
                   
                     1 
                      
                     
                       / 
                     
                      
                     
                       N 
                       1 
                     
                      
                     
                       N 
                       2 
                     
                      
                     
                       
                         ∑ 
                         
                           
                             n 
                             1 
                           
                           = 
                           0 
                         
                         
                           
                             N 
                             1 
                           
                           - 
                           1 
                         
                       
                        
                       
                         
                           ∑ 
                           
                             
                               n 
                               2 
                             
                             = 
                             0 
                           
                           
                             
                               N 
                               2 
                             
                             - 
                             1 
                           
                         
                          
                         
                           
                             ( 
                             
                               
                                 
                                   
                                     x 
                                     1 
                                   
                                    
                                   
                                     [ 
                                     
                                       n 
                                       1 
                                     
                                     ] 
                                   
                                 
                                  
                                 
                                   [ 
                                   
                                     n 
                                     2 
                                   
                                   ] 
                                 
                               
                               - 
                               
                                 
                                   
                                     x 
                                     2 
                                   
                                    
                                   
                                     [ 
                                     
                                       n 
                                       1 
                                     
                                     ] 
                                   
                                 
                                  
                                 
                                   [ 
                                   
                                     n 
                                     2 
                                   
                                   ] 
                                 
                               
                             
                             ) 
                           
                           2 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
       FIG. 4  is a flowchart  400  illustrating a process for improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. The process begins with converting an input image to a PPM format at  405  and then decomposing the ppm image into color planes in PGM format at  410 . Next, JPEG quantization coefficients are created for each respective color plane at  415 . In some embodiments, the colors of the color planes may be RGB, RGBY, or any other suitable color scheme. 
     Each color plane is JPEG decoded at  420  and ESAP postfiltered at  425 . The decoder generates the YCbCr DCT coefficients necessary to perform ESAP postfiltering. The PGM JPEG planes for each color are combined at  430  and the PGM ESAP planes for each color are combined into PPM at  435 . 
     Grayscale MSEs are computed for each color plane at  440  and the results are output to files at  445 . The purpose of this output is to compute a single color PSNR metric for each decoded and each postfiltered color image. The PSNRs are computed for the JPEG and ESAP images at  450  and the PPM CF X:1 ratios and BPPs are computed at  455 . The results of the computations are then output to file at  460 . The purpose of this output is to gather and record PSNR performance metrics as a function of the compression ratio (or equivalently, as a function of the BPP rate) for each processed image. 
     The final output of the color JPEG-ESAP processing is a JPEG image file and an ESAP postfiltered image file, which are created at  465 . Baseline Huffman entropy coding, or alternatively, arithmetic entropy coding may be used for the creation of both images. Also performance metrics may be computed and appended to a file for each processed image. For video, such as MPEG-2, the process may be repeated for multiple frames of the video or all frames of the video, as shown by the dashed line. The final output of MPEG-2 ESAP video processing is a sequence of PPM images that can be re-coded into an enhanced blocking distortion-minimized MPEG-2 or MPEG-4 video sequence at  470 . 
       FIG. 5  is a block diagram of a computing system  500  configured to perform improved JPEG-ESAP processing of color images, according to an embodiment of the present invention. Computing system  500  includes a bus  505  or other communication mechanism for communicating information, and processor(s)  510  coupled to bus  505  for processing information. Processor(s)  510  may be any type of general or specific purpose processor, including a central processing unit (“CPU”) or application specific integrated circuit (“ASIC”). Processor(s)  510  may also have multiple processing cores, and at least some of the cores may be configured to perform specific functions. Multi-parallel processing may be used in some embodiments. Computing system  500  further includes a memory  515  for storing information and instructions to be executed by processor(s)  510 . Memory  515  can be comprised of any combination of random access memory (RAM), read only memory (ROM), flash memory, cache, static storage such as a magnetic or optical disk, or any other types of non-transitory computer-readable media or combinations thereof. Additionally, computing system  500  includes a communication device  520 , such as a transceiver and antenna, to wirelessly provide access to a communications network. 
     Non-transitory computer-readable media may be any available media that can be accessed by processor(s)  510  and may include both volatile and non-volatile media, removable and non-removable media, and communication media. Communication media may include computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. 
     Processor(s)  510  are further coupled via bus  505  to a display  525 , such as a Liquid Crystal Display (LCD), for displaying information to a user. A keyboard  530  and a cursor control device  535 , such as a computer mouse, are further coupled to bus  505  to enable a user to interface with computing system. However, in certain embodiments such as those for mobile computing implementations, a physical keyboard and mouse may not be present, and the user may interact with the device solely through display  525  and/or a touchpad (not shown). Any type and combination of input devices may be used as a matter of design choice. 
     Memory  515  stores software modules that provide functionality when executed by processor(s)  510 . The modules include an operating system  540  for computing system  500 . The modules further include an improved JPEG-ESAP module  545  that is configured to perform JPEG-ESAP processing of color images. Computing system  500  may include one or more additional functional modules  550  that include additional functionality. 
     One skilled in the art will appreciate that a “system” could be embodied as an embedded computing system, a personal computer, a server, a console, a personal digital assistant (PDA), a cell phone, a tablet computing device, a cable TV set top box, an HDTV or UHDTV set, a digital video disk (DVD) player, a Blu-ray player, or any other suitable computing device, or combination of devices. Presenting the above-described functions as being performed by a “system” is not intended to limit the scope of the present invention in any way, but is intended to provide one example of many embodiments of the present invention. Indeed, methods, systems and apparatuses disclosed herein may be implemented in localized and distributed forms consistent with computing technology, including cloud computing systems. 
     It should be noted that some of the system features described in this specification have been presented as modules, in order to more particularly emphasize their implementation independence. For example, a module may be implemented as a hardware circuit comprising custom very large scale integration (“VLSI”) circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, graphics processing units, or the like. 
     A module may also be at least partially implemented in software for execution by various types of processors. An identified unit of executable code may, for instance, comprise one or more physical or logical blocks of computer instructions that may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module. Further, modules may be stored on a computer-readable medium, which may be, for instance, a hard disk drive, flash device, RAM, tape, or any other such medium used to store data. 
     Indeed, a module of executable code could be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network. 
     The process steps performed in  FIG. 4  may be performed by a computer program, encoding instructions for the nonlinear adaptive processor to perform at least the process described in  FIG. 4 , in accordance with embodiments of the present invention. The computer program may be embodied on a non-transitory computer-readable medium. The computer-readable medium may be, but is not limited to, a hard disk drive, a flash device, a random access memory, a tape, or any other such medium used to store data. The computer program may include encoded instructions for controlling the nonlinear adaptive processor to implement the process described in  FIG. 4 , which may also be stored on the computer-readable medium. 
     The computer program can be implemented in hardware, software, or a hybrid implementation. The computer program can be composed of modules that are in operative communication with one another, and which are designed to pass information or instructions to display. The computer program can be configured to operate on a general purpose computer, or an ASIC. 
     It will be readily understood that the components of various embodiments of the present invention, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations. Thus, the detailed description of the embodiments of the present invention, as represented in the attached figures, is not intended to limit the scope of the invention as claimed, but is merely representative of selected embodiments of the invention. 
     The features, structures, or characteristics of the invention described throughout this specification may be combined in any suitable manner in one or more embodiments. For example, reference throughout this specification to “certain embodiments,” “some embodiments,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in certain embodiments,” “in some embodiment,” “in other embodiments,” or similar language throughout this specification do not necessarily all refer to the same group of embodiments and the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. 
     It should be noted that reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the present invention should be or are in any single embodiment of the invention. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussion of the features and advantages, and similar language, throughout this specification may, but do not necessarily, refer to the same embodiment. 
     Furthermore, the described features, advantages, and characteristics of the invention may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize that the invention can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the invention. 
     One having ordinary skill in the art will readily understand that the invention as discussed above may be practiced with steps in a different order, and/or with hardware elements in configurations which are different than those which are disclosed. Therefore, although the invention has been described based upon these preferred embodiments, it would be apparent to those of skill in the art that certain modifications, variations, and alternative constructions would be apparent, while remaining within the spirit and scope of the invention. In order to determine the metes and bounds of the invention, therefore, reference should be made to the appended claims.