METHOD AND APPARATUS FOR PIXEL LEVEL LUMINANCE ADJUSTMENT

Abstract: A method for adjusting the luminance IMHI of pixels in a high resolution image includes first forming a low resolution image of the original by grouping pixels into superpixels such as 8x8 blocks. A low resolution luminance correction value delta LO is determined for each superpixel by comparing the luminance of the superpixel to both a predetermined darkness threshold TD and a predetermined brightness threshold TB. The low resolution luminance correction value delta LO is modified to yield delta 'LO for each superpixel by forming islands of the superpixels and smoothing delta LO between superpixels within each island. Finally, a high resolution luminance correction value delta HI is calculated for each pixel of the original high resolution image by comparing the luminance IMHI of each pixel to both TD and TB, keeping in mind that delta HI is generated through a series of mathematical computations as a function of both the pixel luminance IMHI and the low resolution luminance correction value delta LO. Each pixel of the original image is adjusted by the corresponding high resolution luminance correction value delta HI and a luminance adjusted image is produced.

Description: METHOD AND APPARATUS FOR PLXEL LEVEL LUMINANCE ADJUSTMENT FIELD OF THE INVENTION This invention relates generally to a method and apparatus for improving the quality of photographic images and, more particularly, to improving digital images by pixel level brightness adjustments. BACKGROUND OF THE INVENTION Anyone acquiring an image needs to have a permanent record which faithfully reproduces the original subject or scene, or at least those aspects of the subject or scene which are considered most important. The quality of the reproduction is judged by visually comparing the hardcopy with the original scene. In making this judgment an observer in part compares the brightness of various points in the hardcopy with the brightness of corresponding points in the original scene and thereby forms a subjective opinion about the quality of the reproduction. Exact subjective tone reproduction requires that the brightness of each point in the hardcopy equals that of the brightness of a corresponding point in the original scene. However, as those skilled in the arts know, exact subjective tone reproduction is extraordinarily difficult and inconvenient to achieve because hardcopy media from printers, and media from other devices (such as electronic cameras, scanners, monitors, etc.) are generally viewed at illumination levels which are significantly less than those under which an original was created - original photographs typically being about 1/100 that of the original scene. Also, most hardcopy or electronic media have a limited ability to capture the range of tones which typically exist in nature. Nevertheless, satisfactory subjective tone reproduction can, in part, be obtained if the perceptible brightness of the image on the medium appears similar to the brightness of the original scene. To assure that the foregoing conditions are satisfied depends, in part, on properly matching the luminance range of the scene to the tone scale of the medium, taking into account the particular scene characteristics to be emphasized, prevailing scene lighting conditions, and the medium reproduction characteristics. The issue of matching the luminance range of the scene to the tone scale of the medium was addressed in the Scene Analysis method of U.S. Patent Application No. 08/414,750 filed March 31, 1995 by James R. Boyack and Andrew K. Juenger. Specifically, an image is partitioned into blocks and certain blocks are combined into sectors. An average luminance block value is determined for each block and a difference is determined between the maximum and minimum average luminance block values for each sector. If the difference exceeds a predetermined threshold value, then the sector is labeled as an active sector and an average luminance sector value is obtained from maximum and minimum average luminance block values. All active sectors of the image are plotted versus the average luminance sector values in a histogram, then the histogram is shifted via some predetermined criterion so that the average luminance sector values of interest will fall within a destination window corresponding to the tonal reproduction capability of a destination application. The result of the Scene Analysis is a global balancing of the image brightness by large scale luminance adjustment. Certain areas of the image typically still require luminance adjustments following Scene Analysis. These adjustments highlight and enhance specific regions of interest in the image in order to prevent loss of details and the subsequent reduced image quality. For instance, scenes with a large shaded region adjacent to a bright region are especially difficult to develop or display without loss of some image quality in either or both of the dark and bright regions. This type of luminance adjustment was addressed in U.S. Patent No. 5,235,434 issued August 10, 1993 to Munib Wober. The method of patent '434 includes a first pas: through an image compensation stage, the image being partitioned into groups of pixels called superpixels. Each superpixel is individually processed to determine whether it is part of a larger grouping (i.e. region) of superpixels which all fall beyond predefined darkness or brightness threshold values. Each superpixel is given a SIZE value and an AVERAGE luminance value for all pixels resident in the given superpixel. In a second pass through the image compensation stage, the amount of luminance adjustment for each region of superpixels is determined in relation to both the corresponding SIZE and AVERAGE values. For small and intermediate regions, a smoothing function is applied to prevent the appearance of artifacts along the borders of the superpixel. Further luminance adjustment is sometimes necessary at the pixel level. Thus, the present invention is primarily directed at an improved system and method for performing pixel level luminance adjustments of a digital image which is independent of large area luminance averaging. This and other objects will become apparent in view of the following description, drawings and claims. SUMMARY OF THE INVENTION A method for adjusting the luminance IM HI of pixels in a high resolution image includes first forming a low resolution image of the original by grouping pixels into superpixels such as 8x8 blocks. A low resolution luminance correction value 6L O is determined for each superpixel by comparing the luminance of the superpixel to both a predetermined darkness threshold T D and a predetermined brightness threshold Tβ. The low resolution luminance correction value 5 LO is modified to yield 5'L O for each superpixel by forming islands of the superpixels and smoothing δ o between superpixels within each island. Finally, a high resolution luminance correction value δrø is calculated for each pixel of the original high resolution image by comparing the luminance IM HI of each pixel to both T D and T B , keeping in mind that 5H I is generated through a series of mathematical computations as a function of both the pixel luminance IMH I and the low resolution luminance correction value δ Each pixel of the original image is adjusted by the corresponding high resolution luminance correction value 5 HI and a luminance adjusted image is produced. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic block diagram of a system for compressing the digital image data of a scene including apparatus for pixel level luminance adjustment according to the present invention; Figure 2 is a block diagram overview of the inventive method for pixel level luminance adjustment; Figure 3 is a flow diagram of part of block 100 of the Figure 2 method for brightening a superpixel which has a luminance value below some predetermined darkness threshold TD; Figure 4 is a flow diagram of part of block 100 of the Figure 2 method for darkening a superpixel which has a luminance value above some predetermined brightness threshold T B ; Figures 5A-5E show different size superpixels corresponding to SIZE(ij) values of 1 to 5; Figure 6 is a flow diagram of block 200 of the method of Figure 2; Figure 7A is a representation of SIZE(ij) values in an image; Figure 7B is a representation of DOMAIN(ij) values of an image; Figure 8 is a flow diagram of block 300 of the method of Figure 2; Figure 9 is a flow diagram of the steps within block 330 that occur when the high resolution pixel luminance IM H i(m,n) is greater than the darkness threshold T D and less than the brightness threshold TB; Figure 10 is a flow diagram of the steps within block 360 for setting AVG B and AVGD levels for a high resolution pixel; Figure 11 is a flow diagram of the steps within block 340 for brightening a high resolution pixel when the luminance IM H i(m,n) of the pixel is less than the darkness threshold TD; and Figure 12 is a flow diagram of the steps of block 340 for darkening a high resolution pixel when the luminance IM H i(m,n) of the pixel is greater than the brightness threshold Tβ. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 1. A System For Pixel Level Luminance Adjustment FIG. 1 shows one preferred embodiment of a system 10 for capturing an image 12, compressing and storing the image for display at a later time, and decompressing the stored image and displaying the same. In the illustration the image 12 comprises a tree 14 which is illuminated by ambient light (arrow 16) casting a shadow 18 on the ground as illustrated. An object 20 is located in the shadow 18. The illustrative image 12 has a large dynamic range. If the object 20 is located partially or entirely within the shadow 18, but for the present invention, some information will normally be lost. It should be understood that the object 20 may be in a bright area (not shown) and the principles of the invention would be equally applicable for rendering the object visible as hereinafter described. The image 12 is captured by an input device such as a CCD camera 22 and is processed by an image processing device or compression stage 24. The camera 22 may be directly coupled to an analog to digital converter 26 in the compression stage 24 which produces a digital image signal. A storage device 28 receives and stores the data, and a central processing unit 30 clocks the image signal to the storage device 28 and governs the operation of other devices. The stored image data is fed to a discrete cosine transform or DCT 32 which converts the digital information into a series of coefficients. The transformed data is thereafter coupled to a human visibility matrix filter 34 which discards certain high frequency coefficients considered to be relatively unimportant to good picture quality. For example, certain high frequency color information is filtered out because it is not readily perceived by the human eye and thus the information may be simplified for later processing. The transformed and filtered digital data is thereafter quantized in a quantizer 36 which establishes selected levels of the digital data in a given range of discrete available levels. The quantized data is then encoded by means of a Huffman coding device 38 which compresses the quantized data by a known encoding technique. It should be understood that coding algorithms other than Huffman coding may be employed if desired. The encoded information may then be thereafter stored in a storage device such as a static RAM (SRAM) 40. If desired at some later time, the information in SRAM 40 may be fed to an image processing apparatus or decompression stage 42 by any convenient transmission link 43. Here a Huffman decoding device 44 decodes the data which is then coupled to an inverse quantizer 46. In accordance with the present invention the data stored in storage device 48 is coupled to an image compensation stage 50 which electronically processes the data according to the inventive method so as to selectively adjust the brightness of each pixel of the image 12 within the shadow 18, thereby enhancing the image and retrieving otherwise lost information. The compensated data is converted to digital data by means of inverse DCT 52. Thereafter the data may be converted to an analog signal by digital to analog converter 54 which is coupled to display 56. A modified version of the original image 12 is reproduced as displayed image 12' including the tree 16', the shadow 18' and the object 20' which is enhanced and more clearly visible in display 54. CPU 58 in the compression stage 42 regulates d e various devices therein as illustrated and may manipulate the data as hereinafter described. The general description of the overall system 10 illustrated in FIG. 1 employs processing techniques designed to reduce the amount of computation and thereby speed up the process of selectively adjusting the image 12. In image compensation stage 50, each block of pixels is transformed using the DCT 32 so that the information has the form of a series of coefficients. In accordance with a technique employed in the present invention, the first coefficient of each block of pixels in the domain of the DCT is referred to as the DC (direct current) value and represents an approximate brightness level of the pixel. When the data is decompressed it may likewise be stored in a similar format. If the DC value is within an acceptable level of brightness as defined by darkness threshold TD and brightness threshold TB, the luminance level of the block of pixels is not modified. Under certain conditions, if the DC value is outside of an acceptable brightness level, the pixel is modified to bring out the luminance information. In accordance with the invention, an algorithm or process is employed which modifies or selectively changes the DC value (i.e. the luminance value) by an amount δ ) in the DCT domain for each block of pixels stored in storage device 48. It should be understood that the image compensation stage 50 may be alternatively employed conveniently in the compression stage 24 between human visibility matrix filter 34 and quantizer 36, or directly after the A/D converter 26. This is especially useful where it is desired to form and transmit a corrected image such as a TV image to a plurality of receivers which do not have image compensation capability. The corrected image would then simply be decompressed without further image adjustment in decompression stage 42. 2. A Method For Pixel Level Luminance Adjustment The inventive pixel level luminance adjustment method can be thought of as consisting of three parts as shown in Figure 2. The first part 100 determines the parameters SIZE(ij) and δLo- SIZE(i ) is an integer value designating the size of the neighborhood of the superpixels that were used in testing for a particular superpixel having a luminance S0(ij), which is being considered for estimating the amount of luminance adjustment necessary. The values IM O (ΪJ), which is the low resolution luminance of a superpixel, and 6 LO> which is the low resolution luminance correction value of a superpixel, are identical, respectively, to the values S0(ij) and AVERAGE(ij) calculated in U.S. Patent No. 5,235,434. The low resolution luminance correction value δι > is calculated for each superpixel, then it is applied to each low resolution pixel within that superpixel. The second part of the inventive luminance adjustment method creates islands of pixels which require luminance adjustment, as well as determining a modified low resolution luminance correction value δ'Lo based on the islands that were created. The third part of the inventive luminance adjustment method calculates a high resolution luminance correction value δHi and adds it to the luminance value for each pixel of the high resolution image IM H ι to generate a high resolution luminance adjusted image IM' HI - Detailed steps of the first part, block 100, are shown in Figure 3 for determining Sl(ij) by comparing superpixel luminance S0(ij) with darkness threshold TD, and in Figure 4 for determining Sl(ij) by comparing superpixel luminance S0(ij) with brightness threshold Tβ. In other words each superpixel, S0(ij), of the low resolution image, IM O , must traverse through both the steps of Figure 3 which tests whether the superpixel is too dark and needs to be brightened, and Figure 4, which tests whether the superpixel is too bright and needs to be darkened. Equation (1) is useful for determining whether the luminance level of a superpixel of interest is within an acceptable range. S0(i j) is defined as the luminance of the superpixel of interest, where 0 < i < X, 0 < j < Y, for row i, column j in a low resolution image containing X total rows and Y total columns (i, j, X and Y are integers). Sl(i ) as defined in equation (1) is an indicator of whether the luminance S0(ij) of the superpixel is greater than, equal to, or less than some predetermined luminance threshold. Sl(ij)=threshold-SO(ij) (1) The first threshold useful in equation (1) is the darkness threshold T D defined as the minimum luminance acceptable for a given superpixel. The other threshold useful in equation (1) is the brightness threshold Tβ defined as the maximum luminance acceptable for a given superpixel. When the luminance value of any superpixel is below the darkness threshold TD, the luminance of the given superpixel will be raised towards the darkness threshold TD- Similarly, when the luminance value of any superpixel is above the brightness threshold Tβ, the luminance value of the superpixel will be lowered towards the brightness threshold Tβ. As defined, the value Sl(ij) could be negative, positive or zero. If luminance S0(ij) is being compared to the darkness threshold TD, then a positive value of Sl(ij) means that the luminance of the superpixel is below the darkness threshold T D and Sl(ij) must be raised. If S0(ij) is being compared to the brightness threshold T B , then a negative value of Sl(ij) means that the luminance of the superpixel is above the brightness threshold Tβ and the luminance S0(ij) must be lowered. Block 100 of Figure 2 first follows the steps of Figure 3 to determine whether the superpixel of interest requires brightening. If Sl(ij) is less than or equal to zero, then the luminance S0(ij) of the superpixel is greater than or equal to some predetermined darkness threshold T D and no luminance adjustment is necessary for that superpixel. The low resolution luminance correction value O LO> and SIZE(i j) are both set equal to zero in block 132. However, if Sl(ij) is greater than zero in block 120, then the luminance S0(ij) is lower than the darkness threshold TD and the superpixel must be brightened. At this point in time it is necessary to determine what degree of brightening is necessary. This is done by increasing the size of the neighborhood around the superpixel, i.e. increasing the radius of the search of the pixels surrounding the superpixel of interest, then individually testing in blocks 122, 124, 126 and 128 each of the surrounding low resolution pixels against the darkness threshold TD- An average low resolution luminance correction value b j for the superpixel is determined in blocks 134, 136, 138, 140 and 130 by averaging the luminance of all the surrounding low resolution pixels. Block 122 tests if SI is greater than zero for all low resolution pixels of the 3x3 group shown in Figure 5B. If block 122 is false for any low resolution pixel of Figure 5B, then SIZE(i j) is set equal to 1 and the low resolution luminance correction value δι ι is set equal to Sl(ij) in block 134. If block 122 is true for all low resolution pixels of Figure 5B, then a larger group of surrounding low resolution pixels will be tested in block 124. Block 124 tests if SI is greater than zero for all 25 low resolution pixels of Figure 5C. If block 124 is false for any pixel of Figure 5C, then block 136 sets SIZE(i j) equal to 2 and the low resolution luminance correction value duo equal to 1/9 of the value of all the low resolution pixels within the 3x3 region according to equation (2). If block 124 is true for all pixels of Figure 5C, then a larger group of surrounding low resolution pixels will be tested in block 126. If block 126 is false for any low resolution pixel of Figure 5D, then block 138 sets SIZE(ij) equal to 3 and the low resolution luminance correction value 6 LO equal to the average value of all the low resolution pixels in the 5x5 region according to equation (3). &o(i, ) = 1/2 l(ι, ) (3) If block 126 is true for all pixels of Figure 5D, then a larger group of surrounding low resolution pixels will be tested in block 128. If block 128 is false for any low resolution pixel of Figure 5E, then block 140 sets SIZE(ij) equal to 4 and the low resolution luminance correction value δ \ χ > equal to the average value of all the low resolution pixels in the 7x7 region according to equation (4). If block 128 is true for all low resolution pixels of Figure 5E, then block 130 sets SIZE(ij) to 5 and the low resolution luminance correction value δ L o equal to the average value of all the low resolution pixels in the 9x9 region as shown in equation (5). The outputs to blocks 132, 134, 136, 138 and 140 all feed into block 142 which generates a smoothing function such as the one given in equation (6). SMOOTH(k) = V(k - l) / 4, (6) where 0 < k < 5 The luminance S0(i j) is then adjusted according to the smoothing function so that S0(ij) = S0(ij) + {SMOOTH(k)*δ LO (ij)}. After it is determined whether the superpixel is too dark and needs to be brightened (as described above in coordination with Figure 3), then it must be ascertained whether the superpixel is too bright and needs to be darkened. This is done according to the flow diagram of Figure 4. In block 150, if Sl(ij) is greater than or equal to zero, then the superpixel luminance S0(ij) is less than or equal to some predetermined brightness threshold Tβ and no luminance adjustment is necessary for that superpixel. The low resolution luminance correction value oYo, and SIZE(i j) are both set equal to zero in block 162. However, if Sl(ij) is less than zero in block 150, then the luminance S0(ij) is greater than the brightness threshold TB and the superpixel must be darkened. At this point in time it is necessary to determine what degree of darkening is necessary for the superpixel of interest. This is done by increasing the size of the neighborhood around the superpixel by increasing the radius of the search of the pixels surrounding the superpixel of interest, then individually testing in blocks 152, 154, 156 and 158 each of the surrounding low resolution pixels against the brightness threshold Tβ. An average low resolution luminance correction value 6 L O for the superpixel of interest is determined in blocks 164, 166, 168, 170 and 160 by averaging the luminance of all the low resolution pixels of the superpixel. Block 152 tests if SI is less than zero for all low resolution pixels of the 3x3 group as shown in Figure 5B. If block 152 is false for any of these pixels, then SIZE(ij) is set equal to -1 and the low resolution luminance correction value δ o is set equal to Sl(ij) in block 164. If block 152 is true for all of the low resolution pixels of Figure 5B, then a larger group of surrounding low resolution pixels will be tested in block 154. Block 154 tests if SI is less than zero for all 25 low resolution pixels of Figure 5C. If block 154 is false for any low resolution pixel of Figure 5C, then block 166 sets SIZE(ij) equal to -2 and the low resolution luminance correction value 6L O equal to 1/9 of the value of all the low resolution pixels within the 3x3 region according to equation (2). If block 154 is true for all the low resolution pixels of Figure 5C, then a larger group of surrounding low resolution pixels will be tested in block 156. If block 156 is false for any low resolution pixel of Figure 5D, then block 168 sets SIZE(ij) equal to - 3 and the low resolution luminance correction value δ o equal to the average value of all the low resolution pixels in the 5x5 region according to equation (3). If block 156 is true for all pixels of Figure 5D, then a larger group of surrounding low resolution pixels will be tested in block 158. If block 158 is false for any low resolution pixel of Figure 5E, then block 170 sets SIZE(ij) equal to -4 and the low resolution luminance correction value δ o equal to the average value of all the low resolution pixels in the 7x7 region according to equation (4). If block 158 is true for all pixels of Figure 5E, then block 160 sets SIZE(ij) to -5 and the low resolution luminance correction value δ t o equal to the average value of all the low resolution pixels in the 9x9 region as shown in equation (5). The outputs to blocks 162, 164, 166, 168 and 170 all feed into block 172 which generates a smoothing function such as the one given in equation (6). The luminance S0(ij) is then adjusted in block 172 so that S0(ij) = S0(ij) + SMOOTH(k)*δ L o(ij). The second part 200 of the inventive luminance adjustment method, shown in Figure 6, creates islands of adjacent regions which all require luminance adjustment, as well as determining a modified low resolution luminance correction value δ'u First, SIZE(ij) is binarized for each region in block 202, i.e. given a value DOMAIN(ij) equal to 0 or 1, according to the following rule: if SIZE(ij)=0, then DOMAIN(ij) is set equal to 0; otherwise, if SIZE(iJ)≠0, then DOMAIN(i j) is set equal to 1. In other words, DOMAIN(ij) is set to 1 in the locations of the superpixels that require luminance correction. An example of that is shown in Figures 7 A and 7B. Figure 7A shows SIZE values for superpixels in an image and Figure 7B shows corresponding DOMAIN values for the superpixels. For instance, Figure 7A shows a value SIZE=-3 for one element of region 704. The corresponding element in block 704 of Figure 7B has a value DOMAIN=l, since SIZE≠O according to the above rule. The other element of region 704 in Figure 7A has a value SIZE=-2 so again, according to the above rule, SIZE≠O and the corresponding element in block 704 of Figure 7B has a value DOMAIN=l. The results of the binarization of SIZE for each superpixel of Figure 7 A are shown as superpixel DOMAIN values in Figure 7B. Block 208 creates and numbers, i.e. labels, islands of adjacent regions where DOMAIN(ij)=l . A collection of superpixels are said to form an island if all the elements in the island are connected to one or more other elements in the island. Figure 7B shows an example with three islands where each island is labeled with a number (in this case 702, 704 and 706). ISLANDNUM 706 in Figure 7B is an island of four adjacent elements or superpixels; ISLANDNUM 704 is an island of two adjacent superpixels; and ISLANDNUM 702 is an island of one superpixel. After creating and labeling each island, block 210 generates two parameters labeled ISLANDSIZE and AVG ISL - ISLANDSIZE is calculated for each island, and is defined as the maximum value of SIZE(ij) within the given island. ISLANDSIZE (of ISLANDNUM) = max SIZE(i j) (7) where (ij)e ISLANDNUM. For instance, if the four elements of ISLANDNUM 706 in Figure 7A have SIZE values of 1, 3, 2 and 2 as shown, then ISLANDSLZE=3 for ISLANDNUM 706. The value AVG ISL is calculated for each separate island and is defined as the sum of the low resolution luminance correction values δw for each island divided by the number of elements in the island. Mathematically for each ISLANDNUM, AVGιsL = χ& 0 (/,7) (8) where (i j)e ISLANDNUM. In block 212 the modified low resolution luminance correction value 6' LO is generated utilizing a smoothing function for providing smooth luminance transition between adjacent regions of the image. One such smoothing function is shown in equation (9). SMOOTH(ISLANDSIZE) = (ISLANDSIZE - 1 } R (9) where R is a rational number less than 1. Thus, the modified low resolution luminance correction value δ'Lo for each region is determined as δ'io = SMOOTH(ISLANDSIZE)* AVGISL ( 10) The third part 300 of the inventive luminance adjustment method, detailed in Figure 8, calculates a high resolution luminance correction value δrø to be added to the luminance IMHi(m,n) for each pixel of the high resolution image IMH I which in turn allows the generation of a high resolution luminance adjusted image IM'H I , where 0<m<XDIM and 0<n<YDIM. Constants XDIM, YDIM, T D , T B , SLOPE B and SLOPED are initialized in block 320. The value of XDIM is the total number of rows of high resolution pixels in the image; YDIM is the total number of columns of high resolution pixels in the image; TD is the predetermined darkness threshold; Tβ is the predetermined brightness threshold; SLOPEβ pertains to the transition or smoothing between a bright pixel having a luminance greater than Tβ and an adjacent pixel; and SLOPED pertains to the transition or smoothing between a dark pixel having a luminance less than Tβ and an adjacent pixel. Each pixel of the high resolution image will be tested in block 300 via the loop between blocks 332 and 322. The variables m and n are incremented in either block 322 or 332 after each pass through the loop until the condition of block 332 is true, at which time all pixels within the image have been tested and the process moves on to block 340. If the pixel of interest at location (m,n), which is being tested in decision block 324, is within a region where SIZE=0, then the luminance IM H i(m,n) of the pixel is both greater than or equal to the darkness threshold TD and less than or equal to the brightness threshold T B . The process thereafter continues in block 330. However, if SIZE≠O in block 324, then block 326 tests whether SIZE>0. If block 326 is true, then the luminance IM H i(m,n) of the given high resolution pixel is too low (i.e. less than the darkness threshold TD) and the high resolution pixel will be brightened in block 340. If block 326 is false, then SIZEO and the luminance IMm(m,n) of the given high resolution pixel is too high (i.e. greater than the brightness threshold Tβ), so that the pixel will be darkened in block 350. Once every pixel of the high resolution image IM HI has been processed as described above, then the complete image can be reconstructed in block 328, including of course all the appropriate corrections δm. Figure 9 is a detailed flow diagram of the steps involved in block 330 when the luminance of a high resolution pixel is both greater than or equal to the darkness threshold T D and less than or equal to the brightness threshold TB- Block 360 sets both AVGβ and AVGD to zero and loops within a specified neighborhood for each pixel of the high resolution image IMH I {XDIM, YDIM}. AVGD is a variable to indicate that the pixel being tested is in the transition region between SIZE=0 and SIZE=1. AVG B is a variable to indicate that the pixel being tested is in the transition region between SIZE=0 and SIZE=-1. Both AVG D and AVG B are set to the non-zero value of the corresponding du . The details of block 360 are shown in Figure 10 where both AVGβ and AVG D are first initialized to zero in block 361. Decision block 362 determines whether the average luminance AVG D of pixels in a 3x3 neighborhood (in which the subject high resolution pixel is centered) is less than the low resolution luminance correction value &u - If block 362 is true, then the average luminance AVG D is set equal to the low resolution correction value o^o in block 366. If block 362 is false, then decision block 364 deteπnines whether the average luminance AVG B of pixels in a 3x3 neighborhood (in which the subject high resolution pixel is centered) is greater than the low resolution luminance correction value δι )- If block 364 is true, then the average luminance AVGβ is set equal to the low resolution correction value δ L o in block 368. Once AVGB and AVG D are set in block 360, then decision block 371 shown in Figure 9 determines whether either AVG D ≠O or AVG B ≠0. If block 371 is false, then the process continues to block 332. However, if block 371 is true, then variables P D and PB are set in block 370. PD is an estimated value of the difference in luminance between the pixel of interest and the darkness threshold T D . P B is an estimated value of the difference in luminance between the pixel of interest and the brightness threshold T B . In block 370, FK = abs(SIZE(i j)) (11) FK5 = (FK-5)/4 (12) FKl = (FK-l)/4 (13) PD = T D - (AVG D SLOPE D ) ( 14) PB = T B - (AVG B /SLOPE B ) (15) The variables FK, FK5, FKl, P D and PB are used for feathering δι.o to ensure smooth transition between pixels. Feathering is defined as smoothly combining portions of an image, i.e. integrating portions of an image while ensuring a smooth transition therebetween. FK, FKl and FK5 are particularly used for feathering within regions having different SIZE values. Once P D and P B are set in block 370, then decision block 372 determines whether the pixel luminance value IM H i(m,n) is both less than the darkness threshold T D , and less than or equal to P D . If block 372 is true, then block 376 adjusts the high resolution pixel luminance value IM H i(m,n) by AVG D - If block 372 is false, then decision block 373 determines whether the high resolution pixel luminance value IM H i(m,n) is both less than the darkness threshold TD and greater than P D - If block 373 is true, then block 377 adjusts the pixel luminance value IM H i(m,n) by the modified dark level luminance average MODAVGD- MODAVGD = AVGD * ((T D -IMHi(m,n))/(T D -P D )) (16) If block 373 is false, then decision block 374 determines whether die pixel luminance value IM H! (m,n) is both greater than the brightness threshold T B and greater than or equal to Pβ. If block 374 is true, then block 378 adjusts the pixel luminance value IMm(m,n) by the average bright level luminance AVG B . If block 374 is false, then decision block 375 determines whether the pixel luminance value IMπι(m,n) is both greater than the brightness threshold T B and less than P B - If block 375 is true, then block 379 adjusts the pixel luminance value IM H i(m,n) by the modified luminance average MODAVG B . MODAVG B = AVGB * ((T B -IM H ι(m,n))/(T B -P B )) (17) If block 375 is false, then the method continues with block 332. Returning to Figure 8, if the luminance value of any high resolution pixel is too dark, i.e. below the darkness threshold T D , then that pixel is brightened in block 340 by raising its luminance level by δ H i. The details of this operation are shown in the flow diagram of Figure 11. Block 380 sets FK=abs(SIZE(ij)); FKl=(FK-l)/4; FK5=(FK- 5)/4; and P D =T D -(6 LO SLOPE D ). All of these variables have been previously defined. In decision block 382, if the luminance value of the high resolution pixel of interest is both less than the darkness threshold T D and less than or equal to PD, then block 386 adjusts the luminance value by adding 6 LO to the original luminance value. If the inquiry of block 382 is false, then decision block 384 tests whether the luminance is both less than the darkness threshold T D and greater than PD- If block 384 is true, then block 387 adjusts the luminance value of the corresponding pixel by adding MODOL OI to IMHi(m,n). MODoYoi = δLo*(T D +FK5*IMHi(m,n)-FKl *P D )/(T D -P D ) (18) If block 384 is false, then decision block 385 determines whether both the high resolution pixel luminance is greater than or equal to darkness threshold TD and SIZE≥l. If block 385 is true, then the high resolution pixel luminance is adjusted by adding MODδ w to the original pixel luminance. MODδLθ2 = O' LO *(T D +FK5*TD-FK1 *P D )/(TD-PD) (19) If block 385 is false, then the process continues with block 332. If the luminance value of a high resolution pixel is too bright, i.e. above the brightness threshold Tβ, then that pixel is darkened in block 350 by lowering its luminance level. The details of this operation are shown in the flow diagram of Figure 12. Block 390 sets FK=abs(SIZE(iJ)), FKl=(FK-l)/4 and FK5=(FK-5)/4. Also, PB=T B -(δLo SLOPE B ). Decision block 392 determines if the high resolution pixel luminance is both greater than brightness threshold T B and greater than or equal to P B . If block 392 is true, then the high resolution pixel luminance is adjusted by adding δ w to the original high resolution pixel luminance in block 395. If block 392 is false, decision block 393 determines whether the high resolution pixel luminance is both greater than brightness threshold T B and less than PB. If block 393 is true, then the high resolution pixel luminance is adjusted in block 396 by adding the modified correction value MODδ L o 3 to the original pixel luminance value. MODδ L03 = δ' L0 *(T B +FK5*IMHi(m,n)-FKl *P B )/(T B -PB) (20) If block 393 is not true, then decision block 394 determines whether both the pixel luminance is less than or equal to the brightoess threshold Tβ and SIZE<-1. If block 394 is true, then the high resolution pixel luminance is adjusted in block 397 by adding the modified correction value MODδixw to the original high resolution pixel luminance. MODδ M = δ'Lo*(TB+FK5*T B -FKl *P B )/(T B -P B ) (21 ) If block 394 is false, then the process continues with block 332. Note that the third part 300 of the inventive method calculates the high resolution correction value δm (as indicated in Figure 2) for each pixel of the original image. Each pixel in the image is thereafter adjusted by 6 HJ to form a new luminance adjusted image. It is understood that the high resolution correction value δHi is labeled in many different ways throughout the specification and drawings. For instance in Figure 9, δ H j is labeled as AVG D in block 376, MODAVG D in block 377, AVG B in block 378 and MODAVG B in block 379. In Figures 11 and 12, δm is labeled as δ in blocks 386 and 395, MODδLoi in blocks 387 and 396, and MODδuM i blocks 388 and 394. While there has been described what at present are considered to be the preferred embodiments of the present invention it will be apparent to those skilled in the art that various changes and modifications may be made therein without departing from the invention and it is intended in the accompanying claims to cover all such changes and modifications as fall within the true spirit and scope of the invention.

Claims: WHAT IS CLAIMED IS: 1. A method for adjusting luminance IM HI of pixels in a high resolution image, comprising the steps of: forming a low resolution image of said high resolution image by grouping said pixels into superpixels; generating a low resolution luminance correction value δ L o for each supeφixel by comparing the luminance of said superpixel to both a predetermined darkness threshold T D and a predetermined brightness threshold T B ; generating a modified low resolution correction value δ o for each said supeφixel by forming islands of said supeφixels and smoothing O LO between supeφixels within each island; generating a high resolution luminance correction value δrø for each pixel of said high resolution image by comparing the luminance IM HI of each said pixel to both TD and T B , said δ^ being generated as a function of both IM HI and δι,o; and generating a luminance adjusted image by combining the luminance of each said pixel with the corresponding high resolution luminance correction value 6 HI - AMENDED CLAIMS [received by the International Bureau on 3 April 1997 (03.04.97); original claim 1 replaced by new claims 1-6 (2 pages)] 1. A method for adjusting luminance values IMHI of pixels in a high resolution image, comprising the steps of: forming a low resolution image of said high resolution image by grouping the pixels into superpixels; for selected said superpixels having luminance values exceeding a predetermined thresholό alue. generating non-zero low resolution luminance correction values OLD by averaging luminance values of superpixels surrounding each said selected superpixel; generating a modified low resolution correction value 5'LO for each said selected superpixel by foπning islands of adjacent said selected supeφixels and smoothing 6 O between said adjacent selected supeφixels within each island; generating a high resolution luminance correction value SHI for each pixel of the high resolution image by comparing the luminance values IMHI of each said pixel to said predetermined threshold value and generating δtu as a function of both IMHI and δ , and generating a luminance adjusted image by combining the luminance values IMHI of each said pixel with said corresponding high resolution luminance correction value HJ. 2. Toe method of claim 1, wherein said predetermined ihreshold value is one of a darkness threshold To and a brightness threshold TB. 3. A method for adjusting luminance values of pixels of an image, comprising the steps of: forming a low resolution version of said image by grouping said pixels into supeφixels according to a predetermined pixel classification; generating a low resolution correction value 5LO for each said supeφixel by comparing a luminance value of each supeφixel to a preo termined threshold value; generating a modified low resolution correction value 6'LO for each said supeφixel by smoothing SLO between neighboring supeφixeis; generating a high resolution correction value SHI for each of said pixels as a function of both said corresponding superpixel luminance value and 5LO; and generating the adjusted luminance values of pixels of the image by combining om with the luminance value for each pixel. 4. The method of claim 3, wherein said predetermined threshold value is one of a darkness threshold To and a brightness threshold TB. 5. A system for pixel level luminance adjustment of an image, comprising: means for forming a low resolution image of said high resolution image by grouping die pixels into supeφixels; means for generating non-zero low resolution luminance correction values SLO for selected said supeφixels having luminance values exceeding a predetermined threshold value, by averaging luminance values of superpixels surrounding each said selected superpixel; means for generating a modified low resolution correction value 5'LO for each said selected supeφixel by forming islands of adjacent said selected superpixels and smoothing δ o between said adjacent selected supeφixels within each island; means for generating a high resolution imniπance correction value orø for each pixel of the high resolution image by comparing the luminance values IMHI of each said pixel of the image to said pn^termined threshold value and generating SHI as a function of both corresponding values IMHI and 5L ; and means for generating a luminance adjusted image by combining said luminance values IMHI of each said pixel with said corresponding high resolution luminance correction value ©HJ. 6. The system of claim 5, wherein said predetermined threshold value is one of a darkness threshold T D and a brightness threshold TB-

Inventors: HAJJAHMAD IBRAHIM, WOBER MUNIB A, HAJJAHMAD, IBRAHIM, WOBER, MUNIB, A.
Applicants: POLAROID CORP, POLAROID CORPORATION
Application year: 1996
Main classification: G06T5
IPC codes: G06T5, G06T9, H04N1