Patent Description:
More specifically, when an HDR image, such as shown in <FIG>, is displayed on a standard monitor that has <NUM> bits per pixel, the HDR image looks very flat because the <NUM> bits per pixel results in a low dynamic range image (i.e., an image in which the contrast between bright and dark regions is relatively limited and medium brightness levels tend to be close together). That is, the reason why HDR images appear to have low dynamic range is because the wide range of brightness has to be compressed to fit within a much smaller range of brightness. For example, an HDR image having <NUM> bits per pixel image (or otherwise described as <NUM>-bit pixel image) has <NUM> (<NUM>^<NUM>) brightness levels for each pixel as compared to a LDR image having <NUM> bits per pixel image only has <NUM> (<NUM>^<NUM>) brightness levels. As a result, there is an overall lack of contrast, which results in flatness and potentially other visual artifacts in the LDR images. As shown in <FIG>, the individual's face <NUM> is difficult to see and features in the grass within the shadow <NUM> because the contrast is narrow.

An HDR image can be captured by an HDR camera or the HDR image may be created (not captured) from many LDR images captured by a standard camera by capturing three or more photos with different exposure levels. For example, if three LDR images are captured, one of the images may be properly exposed, while the other two photos are often overexposed and underexposed. These three images typically capture suitable details in the highlights and the shadows of the scene. However, the problem is that the images then have to be combined to correct the bright and dark regions so that details in those regions are properly visible.

With further regard to <FIG>, tone mapping is a technique that adjusts contrast locally so that each region of the image uses the whole brightness range for maximum contrast. Tone mapping is also often used in image processing and in computer graphics to map one set of colors to another set of colors in order to approximate the appearance of high dynamic range images in a medium or in another image that has a more limited dynamic range. Tone mapping is often an improvement over other image processing techniques that map the whole image into the LDR brightness range because the bright and dark regions are often not properly corrected. <FIG> is an image of the scene <NUM> of <FIG> that was processed using conventional tone mapping. Tone mapping, however, is a process that is generally complex and time-consuming. In this case, the tone mapping causes the lighting and detail to be much better as exemplified by the added detail in the face <NUM> and shadow <NUM> in the grass of <FIG> as compared to the face <NUM> and shadow <NUM> of<FIG>.

As a result of modern day cameras having the ability to capture multiple images in quick succession and with different exposure levels, algorithms have been created for combining multiple images into a single image in which each of the input image information is involved without producing any image artifacts, such as halos. The algorithms determine a proper exposure for each region of the image such that a maximum level of information may be presented in the image. However, such image processing techniques are not good for capturing images of a scene in which motion occurs (e.g., object moving, converge moving, or both moving) because successive images a moving object will show to be in different positions within each of the different images.

With regard to <FIG>, four different images of the same scene <NUM> are shown to have been captured at different exposure levels in the images 202a-202d (collectively <NUM>). In <FIG>, the exposure level is low or short, so that the image is dark, but because of the brightness of lamps 204a and 204b (collectively <NUM>), good resolution of the lamps <NUM>. However, the remainder of the image 202a away from the lamps <NUM> is dark, so less detail for the remainder of the image 202a exists. As the exposure level is increased in <FIG>, more detail is shown at the lamps <NUM>, but the remainder of the scene <NUM> is still limited in detail due to the darkness. As shown in <FIG>, the lamps <NUM> are beginning to become oversaturated because the exposure has further been increased, but resolution of a carpet <NUM> is beginning to show because of increased exposure level. And, as shown in <FIG>, as the exposure level is further increased, even more resolution of the carpet <NUM>, chairs <NUM>, and other areas of the scene <NUM>, but also because of the increased exposure, the image region at the lamps <NUM> are oversaturated and resolution is decreased or completely lost. These four images <NUM> may be combined using a conventional tone mapping algorithm that combines the multiple images 202a-202d to generate an output image 202e of <FIG> with the best resolution of each of the various areas of the scene <NUM>. As shown in the image 202e, the lamps <NUM> are not oversaturated with light and the carpet <NUM> is not under-saturated with light, thereby generating an image 202e with the maximum amount of information of the scene <NUM>. In summary, the first step includes computing an HDR image from the multiple LDR images, and the second step includes performing tone mapping from the multiple images in order to obtain an LDR detailed output image to obtain an LDR image with locally adjusted lighting contrast in order to have a maximum level of detail of each object in the scene <NUM> in the LDR image. In this case, the lamps <NUM> are derived from the first image 202a, which was captured with the smallest exposure time, while the carpet <NUM> and chairs <NUM>, among other features, are derived from the last image 202d, which was captured with the largest exposure time, and the table <NUM>, the couch <NUM>, and chairs <NUM> are derived from the middle images 202b and 202c, which were captured with medium exposure times between the first and last images 202a and 202d.

As previously indicated, the ability to perform tone mapping requires multiple images to be captured, and if a non-static scene exists in which there is motion in the scene or there is motion with the camera relative to the scene, then it is not possible to use the tone mapping algorithm because each image would contain motion of one or more objects in the scene. Moreover, modern image sensors generate image data with more than eight bits per pixel (HDR images) while software applications process <NUM>-bit images (LDR images). As a result, to accommodate the different bits per pixel, less significant bits of the HDR input images are discarded, which means that information is being lost, such that flat images and images that have halo effects where lighting quickly transitions from dark to bright or vice versa results. As such, there is a need for new systems and processes that do not have the limitations of existing image processing (e.g., tone mapping) of HDR images to LDR images. The following documents discuss this topic: <NPL>; <NPL>; <NPL>; and <CIT>.

To overcome the problem of flatness of LDR images derived from HDR images and having to capture multiple HDR images to produce an LDR image, a single shot HDR image processing system may be utilized. Because the mathematics of the system are relatively simplistic, the system may be performed using hardware or an embedded firmware system that operates at real-time or near-real-time speeds. By being able to perform the image processing from a single image, the problem of motion in the scene is eliminated as compared to conventional tone mapping solutions.

The present invention relates to a computer-implemented method of processing an image according to claim <NUM>. The image data having M-bits per pixel is an HDR image, and the processed image is an LDR image having <NUM> bits per pixel.

The present invention relates to a system for processing an image according to claim <NUM>. The image data having M-bits per pixel is an HDR image, and the processed image is an LDR image having <NUM> bits per pixel.

Generating image data with M-bits per pixel includes generating image data having more than eight bits per pixel.

Generating a plurality of sets of simulated image data includes generating at least two sets of simulated image data.

Generating at least two sets of simulated image data includes generating each of the sets of simulated image data by:.

Preferably, saturating the sets of scaled image data includes limiting the pixel values to a maximum value of <NUM>.

Preferably, a level of detail of each of the sets of simulated image data is computed by:.

Preferably, the method further comprises blending the sets of detailed image data to produce the processed image having N-bits.

Capturing an image of a scene includes capturing a high dynamic range (HDR) image, and wherein producing the processed image includes producing a low dynamic range (LDR) image.

Preferably, blending the sets of detailed image data includes:.

Preferably, the method further comprises mixing the sets of blended image data by:.

Said image sensor, in generating image data with M-bits per pixel, is further configured to generate image data having more than eight bits per pixel.

Said electronics, in generating a plurality of sets of simulated image data, are further configured to generate at least two sets of simulated image data.

Said electronics, in generating at least two sets of simulated image data, are further configured to:.

Preferably, said electronics, in saturating the sets of scaled image data, are further configured to limit the pixel values to a maximum value of <NUM>.

Preferably, said electronics are further configured to compute a level of detail of each of the by being configured to:.

Preferably, said electronics are further configured to blend the sets of detailed image data to produce the processed image having N-bits.

Said electronics, in capturing an image of a scene, are further configured to capture a high dynamic range (HDR) image, and wherein said electronics, in producing the processed image, are further configured to produce a low dynamic range (LDR) image having <NUM> bits per pixel.

Preferably, said electronics, in blending the sets of detailed image data, are further configured to:.

Preferably, said electronics are further configured to mix the sets of blended image data by being configured to:.

Illustrative embodiments of the present invention are described in detail below with reference to the attached drawing figures, wherein:.

With regard to <FIG>, a block diagram of a hardware system <NUM> that may be utilized to implement a system to execute a single-shot high dynamic range (HDR) algorithm as provided in <FIG> is shown. The system <NUM> is shown to include an M-bit image sensor <NUM> that captures images of a scene and generates M-bits per pixel data. A field programmable gate array <NUM> may be configured with a hard processing system (HPS) <NUM> that includes a microprocessor <NUM> and double data rate (DDR) memory controller <NUM> for controlling a DDR memory <NUM>. A sensor manager <NUM> and single-shot HDR unit <NUM> may be included to control operation of the image sensor <NUM>, communicate M-bit HDR image data <NUM> to the single-shot HDR unit <NUM>, which outputs <NUM>-bit LDR image data <NUM>. The <NUM>-bit LDR image data is communicated to the DDR memory controller <NUM> for processing by the microprocessor <NUM>. By including the DDR memory controller <NUM>, the microprocessor <NUM> and FPGA <NUM> may access the same data. The image sensor <NUM> outputs M-bits per pixel, where M is greater or equal to <NUM>, while the single-shot HDR unit <NUM>, as detailed in <FIG>, may be implemented in hardware to process the image pixels on-the-fly (i.e., real-time or near-real-time) without any computational load for the microprocessor <NUM> and adding only some negligible latency to the image acquisition.

A core principle of the single-shot HDR algorithm is to capture a single image with M-bits per pixel, where M><NUM>, and generate an LDR detailed output image with <NUM>-bits per pixel. Moreover, the single-shot HDR algorithm is able to function without performing multiple image acquisitions with different exposure times, as performed by conventional tone mapping algorithms, as previously described.

More specifically, the single-shot HDR algorithm may operate in the following manner:.

It is further noted that the single-shot HDR algorithm does not lead to the same LDR-detailed output image that would be obtained performing K different image acquisitions with K different exposure times, and the reason is not only a due to capturing an image of a non-static scene. Even if a static-scene were captured, computing the K <NUM>-bit images from the M-bit input image using the less significant bits introduces more noise than acquiring K images with different exposure times. As shown hereinbelow, the results are more than acceptable so the use of a single-shot HDR algorithm provides for computing the LDR detailed output images with a single image acquisition.

With regard to <FIG>, a block diagram of a single-shot HDR algorithm <NUM>, shown as a high-level view <NUM> that (i) performs image processing on a single HDR input image <NUM> of M-bits per pixel (e.g., <NUM> bits per pixel) and (ii) creates simulated images 404a-404c (collectively <NUM>) with different simulated exposure levels that are used to produce an LDR detailed output image <NUM> are shown. As indicated, the simulated images <NUM> are <NUM>-bits per pixel.

The single-shot HDR algorithm <NUM> includes three stages, including Stage-<NUM>, Stage-<NUM>, and Stage-<NUM>, within which different functions of the algorithm <NUM> are performed. It should be understood that the number of blocks and starting and ending locations of the blocks may vary and are not limiting.

Stage-<NUM> may include a set of multiply-saturate blocks 408a-408c (collectively <NUM>), in this case three multiplier-saturate blocks <NUM>. The blocks <NUM> are configured to simulate different lighting exposure levels to produce the respective simulated images <NUM>. It should be understood that the number of blocks <NUM> may be two or more blocks, as well, to increase the number of simulated images (i.e., the more blocks <NUM>, the more simulated images). The simulated images <NUM> may be communicated to Stage-<NUM> for processing.

Stage-<NUM> includes level of detail blocks 410a-410c (collectively <NUM>) that generate images 412a-412c (collectively <NUM>) having different levels of detail. The different levels of detail in the images <NUM> are created using mathematical functions. It should be understood that the number of detail blocks <NUM> may be two or more blocks, as well, to increase the number of detailed images (i.e., the more blocks <NUM>, the higher levels of detail that may be generated and/or analyzed). The images <NUM> may be communicated to Stage-<NUM> for processing.

Stage-<NUM> includes two main blocks, including a blend function <NUM> and mix function <NUM>. The blend function <NUM> uses the level of detail of each pixel in each image and performs mathematical computations to create blended images 418a-418c (collectively <NUM>). The blended images <NUM> and simulated images <NUM> may be mixed by the mixing function <NUM> to produce the LDR detailed output image <NUM>.

With regard to <FIG>, a block diagram of a multiply function <NUM> and saturate function <NUM> performed within the multiply-saturate blocks <NUM> of Stage-<NUM> of the algorithm of <FIG> is shown. Input into the multiply function <NUM> is the M-bit HDR input image <NUM> having M-bits per pixel along with a multiplier value V#i, where each of the multiplier values V#i may be set by a user or automatically selected by the system based on lighting or other conditions of the camera and/or scene. To generate simulation images with different lighting values, each of the multiplier-saturate functions 408a-408c may be provided with different multiplier values V#i, as previously described. As shown, the M-bit HDR input image <NUM> may result in a P-bit image <NUM> after the multiply function <NUM>, and the saturate function <NUM> may reduce the number of bits per pixel from P-bits to <NUM>-bits per pixel by using the less significant bits, as previously described.

With regard to <FIG>, a block diagram of the functions performed within the level of detail blocks <NUM> of Stage-<NUM> of the algorithm of <FIG> is shown. The functions may include a smooth function <NUM>, gradient function <NUM>, and average function <NUM>. The smooth function <NUM> may be implemented using a low-pass filter, Gaussian filter, and/or any other filter that smoothens data, as understood in the art. The gradient function <NUM> may utilize a gradient filter or any other mathematical function that computes lighting gradients within an image. The average function <NUM> may be implemented using a low-pass filter or any other filter that performs averaging within image data. For example, the average function <NUM> may be configured to compute an average gray level of each pixel, such as by averaging a pixel and neighboring pixels of the pixel of the gradient images. That is, the average function <NUM> may calculate a mean value of the gradient values resulting from the gradient function <NUM>. Output from the level of detail block <NUM> is the detail image 412a. It should be understood that each of the level of detail blocks <NUM> may have the same or similar functions for generating the detailed images <NUM>.

With regard to <FIG>, a block diagram of functions used to perform the blend function <NUM> within Stage-<NUM> of the algorithm <NUM> of <FIG> is shown. The blend function <NUM> may be configured to receive the detailed images <NUM> and output blended images <NUM>. More particularly, the blend function <NUM> mixes the gray level values of each image, and executes a blending algorithm by computing a ratio between each detail in each of the images against the total detail of the images. The mixing function then smooths the images in order to obtain an LDR-detailed output image with smooth transitions between different exposures. In operation, the detailed images <NUM> are summed together using a sum function <NUM> to output a summed image <NUM>. Divide functions 706a-706c (collectively <NUM>) may divide the respective detailed images <NUM> by the summed image <NUM>. Each of the outputs from the divide functions <NUM> may be smoothed by smooth functions 708a-708c (collectively <NUM>) to generate the respective blended images 418a. The blend function <NUM> may be used to determine which is the best image from the three computed images, and scores each image.

With regard to <FIG>, a block diagram of the mix function <NUM> performed within Stage-<NUM> of the single-shot HDR algorithm of <FIG> is shown. The mix function <NUM> may be configured to receive and multiply the respective detailed images <NUM> produced by the level of detail blocks <NUM> and blended images <NUM> produced by the blend function <NUM> using multiply functions 820a-802c (collectively <NUM>). The results from the multiply functions <NUM> may be summed by a summing function <NUM> to produce the <NUM>-bit (LDR) detailed output image <NUM>, or more precisely, a weighted average of the pixel values of the different exposure time images.

In <FIG>, K=<NUM>, so there are only <NUM> image processing chains, and each function within the chain has <NUM> corresponding functions. In general, however, there may be <NUM> to <NUM> image processing chains (theoretically there may be even more than <NUM>, but the state-of-the-art HDR algorithms usually do not exceed <NUM>).

The multiplier values have to be chosen according to M. In particular, the minimum value is equal to <NUM> while the maximum value is equal to <NUM>^(M-<NUM>). For example, if the image sensor produces images with <NUM> bits per pixel, then the multiplier values are typically selected between <NUM> and <NUM>.

With regard to <FIG>, sets of image data 900a-900c (collectively <NUM>) with simulated lighting that are output from Stage-<NUM> of the algorithm <NUM> of <FIG> are shown. Most notably, barcode labels 902a, 902b, and 902c are shown with increasing brightness as the simulated lighting produced by the multiplier values V#i are successively increased for each of the different multiply functions (e.g., multiply function <NUM> of <FIG>) used in the multiply saturate blocks <NUM> of <FIG>. It can also be seen how other barcodes in the background <NUM> are difficult to see in the first set of input data 900a, but are easier to see in the second and third set of input data 900b and 900c.

With regard to <FIG>, sets of smoothed image data 1000a-1000c (collectively <NUM>) derived from the image data <NUM> of <FIG> that have been smoothed by a smoothing function <NUM> of <FIG> and within Stage-<NUM> of <FIG> are shown. Applying a smoothing function helps to eliminate sharp edges in the image data <NUM>.

With regard to <FIG>, sets of gradient image data 1100a-1100c (collectively <NUM>) generated by a gradient algorithm <NUM> of <FIG> and within Stage-<NUM> of <FIG> are shown. Notably, the gradient data <NUM> highlights barcodes, such as barcodes 1102a-1102c (collectively <NUM>) in the barcode labels 902a-902c. It can be seen that as the brightness increases the image data 1100a-1100c, the barcodes <NUM> alter in appearance, which may mean that if the first barcode 902a is not decodable because the image is too dark and the last barcode 902c is not decodable because the image is too bright, then the middle barcode 902b is decodable because the simulated lighting strikes a good balance between being too dark and too bright. The level of detail of the barcodes <NUM> is shown to vary across different simulated lighting. The multiplier V#i of <FIG> may be selectable to optimize such simulated lighting functionality.

With regard to <FIG>, sets of averaged image data 1200a-1200c (collectively <NUM>) generated by an average function <NUM> of <FIG> performed in the level of detail blocks <NUM> within Stage-<NUM> of the algorithm of <FIG> are shown. The average function <NUM> smoothens the features in the image data <NUM> of <FIG>. The averaged image data <NUM> is output from Stage-<NUM> and input to Stage-<NUM>.

With regard to <FIG>, sets of blended image data 1300a-1300c (collectively <NUM>) output by the blend function <NUM> of Stage-<NUM> of the single-shot HDR algorithm <NUM> of <FIG> are shown. The image data <NUM> is shown to be quite different as the backgrounds 1302a-1302c transition from dark to bright to gray due to the blending functionality to help determine which is the best image from the three computed images.

With regard to <FIG>, blended image data <NUM> that is output from the mix function <NUM> of the algorithm of <FIG> is shown. The image data <NUM> is a result of the weighted sum average of the sets of simulated image data <NUM> and blended image data <NUM>. When compared to the sets of simulated image data of <FIG>, it is clear that barcodes 1402a-1402c in the background are more readable and the barcode 1402d on the barcode label <NUM> are each more visible, and hence more likely to be decodable (assuming the codes are in focus).

With regard to <FIG>, sets of image data 1500a-1500c (collectively <NUM>) generated by stimulating exposure levels and applied to a single HDR image with the focus on the nearest code are shown. The captured image data is generated by a 1280x960 <NUM>-bit pixels with the algorithm configured with K=<NUM>, V1=<NUM>, V2=<NUM>, and V3=<NUM>. It should be understood that alternative variable values may be utilized, and the values may vary based on the environment and other factors of the imaging. In this case, four codes are present, including three background codes 1502a-1502c and one foreground code 1502d. The background codes are difficult to see in the image data 1500a, but more visible in the third image data 1500c.

With regard to <FIG>, an LDR detailed output image 1500d generated by the algorithm of <FIG> that shows how four barcodes 1502a-1502d are visible, but only the nearest barcode 1502d is decodable as a result of the other barcodes being unfocused is shown. Even though the algorithm does not improve the decodability, the LDR-detailed image 1500d gathers all of the information of the three sets of simulated input images, thereby significantly enhancing the user experience (e.g., enhanced reliability) and the image appearance.

With regard to <FIG>, sets of image data 1600a-1600c (collectively <NUM>) of the scene of <FIG>, and show that the nearest code (in focus) is decodable only in <FIG> while the other codes (out of focus) are not decodable are shown. In each of the sets of image data 1600a and 1600b, which match the image data 1500a and 1500b, the codes 1602a and 1602b with their respective simulated lighting are decodable. However, the simulated image data 1600c, which matches the image data 1500c, is overexposed such that the code 1602c is not decodable. The decodable codes 1602a and 1602b have decode indicators 1604a and 1604b, while the code 1602c, which is overexposed and not decodable, does not have an indicator. It should be understood that the decodable indicators 1604a-1604c are illustrative, and that any indicator may be utilized to indicate that a machine-readable indicia was successfully decoded.

With regard to <FIG>, image data 1600d representative of an LDR detailed output image generated by the algorithm of <FIG> that shows (i) how the nearest code of <FIG>15C is still decodable, (ii) how the other code of <FIG> are still not decodable and (iii) how the single-shot HDR algorithm does not adversely impact the ability to decode machine-readable indicia is shown. The brightness of the code 1602d may be similar to the brightness of either of the codes 1602a or 1602b, which are both decodable, and includes a decode indicator 1604d. Moreover, as exemplified by the images 1600a-1600d, the single-shot HDR algorithm <NUM> enables improvement of a captured image without compromising decodability of the codes. In particular, the output image 1600d "collects" all of the decodable codes 1602a and 1602b of the simulated input images 1600a-1600c.

It should be understood that the algorithm <NUM> to produce an LDR detailed output image may be applied to any captured image that includes or does not include machine-readable indicia. Moreover, the LDR output image may be used for any purpose. For example, the algorithm <NUM> that may be deployed as a hardware solution and may be deployed on a variety of different electronic devices, such as mobile devices (e.g., smart phones) or non-mobile devices (e.g., surveillance cameras). The algorithm <NUM> may be embodied on any hardware device, including a stand-alone electronic device or incorporated within another electronic device already existing on a system. The algorithm <NUM> may enable the same or similar functionality described herein to be utilized, while supporting relative motion within the captured images.

With regard to <FIG>, the same sets of image data 1700a-1700c of the scene of <FIG> are shown, but the focus is on the farthest codes 1702a, 1702b, and 1702c (collectively <NUM>). Because the focus is on the farther codes <NUM>, the ability to decode those codes are exemplified.

With regard to <FIG>, an LDR detailed output image generated by the algorithm of <FIG> that shows how four barcodes are visible simultaneously, where the nearest barcode is from <FIG> and the other three barcodes are from <FIG> is shown. In this case, it is not possible to read the foreground code 1702d because the code 1702d is out of focus and overexposed. The LDR detailed output image 1700d thus helps improve the ability to decode machine-readable indicia, in this case barcodes, such that an improved user experience results.

With regard to <FIG>, the same sets of image data of the scene of <FIG>, and show that the nearest code (out of focus) is not decodable while the other codes (in focus) are decodable in <FIG> are shown. In this example, in the LDR detailed output image data 1800d of <FIG>, all four codes <NUM> are neither overexposed nor underexposed. However, the code 1802d is not decodable due to being out of focus. As such, this example shows that the single-shot HDR algorithm <NUM> of <FIG> can improve both the image appearance and the decodability of machine-readable indicia.

With regard to <FIG>, an output LDR detailed image 1800d generated by the algorithm <NUM> of <FIG> shows how each of the codes 1802a-1802c that are in focus are decodable. An indicia <NUM> (e.g., "CODE <NUM> #<NUM>") may be displayed on an electronic display overlaying each of the decodable codes 1802a-1802c when decoded by a code reader.

With regard to <FIG>, a flow diagram of an illustrative single-shot HDR process <NUM> that may be used to capture an HDR image and output an LDR image having good contrast in the LDR image data is shown. The process <NUM> may start at step <NUM>, where an image of a scene may be captured. At step <NUM>, image data having M-bits per pixel may be generated, where an image sensor that produces more than <NUM> bits per pixel image data may be used. Multiple sets of simulated image data of the scene may be generated by applying different simulated exposure times to the generated image data. It should be understood that the simulated image data includes the same images, but that the brightness (and/or other parameters) may be adjusted to represent different exposure times. The number of sets of simulated image data may be two or more. At step <NUM>, a processed image derived from the sets of simulated image data may be generated. The processed image may be an LDR image that has improved image qualities, such as providing for the ability to read machine-readable indicia and reducing the lighting flatness that would otherwise exist.

In generating the image data with M-bits per pixel, the image data has more than eight bits per pixel. In generating multiple sets of simulated image data, at least two sets of simulated image data are generated. In generating at least two sets of simulated image data, each of the sets of simulated image data are generated by (i) multiplying the generated image data by different scale factors to generate different sets of scaled image data with P-bits per pixel, wherein P is greater than M, and (ii) saturating the pixel values of the sets of scaled image data by limiting pixel values to a maximum value, the maximum value may be defined by a maximum number of N that defines a number of bits per pixel (N-bits) to produce the sets of simulated image data, wherein N is less than P and equal to <NUM>. In saturating the sets of scaled image data, the pixel values may be limited to a maximum value of <NUM>. A level of detail of each of the sets of simulated image data may be computed by (i) smoothing each of the sets of simulated image data to produce sets of smooth image data, (ii) computing gradient image data of each of these sets of smooth image data to produce sets of gradient image data, and (iii) averaging the sets of gradient image data to produce sets of detailed image data having N-bits per pixel.

The process may further include blending the sets of detailed image data to produce the processed image having N-bits per pixel. Capturing an image of a scene includes capturing a high dynamic range (HDR) image, and producing the processed image includes producing a low dynamic range (LDR) image. Blending the sets of detailed image data may include (i) adding the respective pixels of the sets of detailed image data to produce a summation set of detailed image data, (ii) dividing each of the sets of detailed image data by the summation set of detailed image data to form sets of sub-blended image data, and (iii) smoothing the sets of sub-blended image data to form sets of blended image data. Mixing the sets of blended image data may be performed by (i) multiplying the sets of blended image data by the respective sets of detailed image data to produce sets of weighted image data, and (ii) summing the sets of weighted image data to produce weighted average image data that represents the processed image.

As will be appreciated by one of skill in the art, the steps in the foregoing embodiments may be performed in any order. Words such as "then," "next," etc. are not intended to limit the order of the steps; these words are simply used to guide the reader through the description of the methods. Although process flow diagrams may describe the operations as a sequential process, many of the operations may be performed in parallel or concurrently. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination may correspond to a return of the function to the calling function or the main function.

The various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the embodiments disclosed here may be implemented as electronic hardware, computer software, or combinations of both.

Embodiments implemented in computer software may be implemented in software, firmware, middleware, microcode, hardware description languages, or any combination thereof. A code segment or machine-executable instructions may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to and/or in communication with another code segment or a hardware circuit by passing and/or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc. may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, etc..

The actual software code or specialized control hardware used to implement these systems and methods is not limiting of the invention. Thus, the operation and behavior of the systems and methods were described without reference to the specific software code being understood that software and control hardware can be designed to implement the systems and methods based on the description here.

When implemented in software, the functions may be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of a method or algorithm disclosed here may be embodied in a processor-executable software module which may reside on a computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable media includes both computer storage media and tangible storage media that facilitate transfer of a computer program from one place to another. A non-transitory processor-readable storage media may be any available media that may be accessed by a computer. By way of example, and not limitation, such non-transitory processor-readable media may comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other tangible storage medium that may be used to store desired program code in the form of instructions or data structures and that may be accessed by a computer or processor. Disk and disc, as used here, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Additionally, the operations of a method or algorithm may reside as one or any combination or set of codes and/or instructions on a non-transitory processor-readable medium and/or computer-readable medium, which may be incorporated into a computer program product.

Claim 1:
A computer-implemented method of processing an image, comprising:
capturing an image of a scene;
generating image data (<NUM>) having M-bits per pixel of the image;
generating a plurality of sets (404a, 404b, 404c) of simulated image data of the scene by applying different simulated exposure times to the generated image data; and
generating a processed image (<NUM>) derived from the sets (404a, 404b, 404c) of simulated image data,
wherein:
generating image data (<NUM>) with M-bits per pixel includes generating image data (<NUM>) having more than eight bits per pixel; and
generating a plurality of sets (404a, 404b, 404c) of simulated image data includes generating at least two sets (404a, 404b, 404c) of simulated image data; and
characterized in that generating at least two sets (404a, 404b, 404c) of simulated image data includes generating each of the sets (404a, 404b, 404c) of simulated image data by:
- multiplying the generated image data (<NUM>) by different scale factors (V#i) to generate different sets (<NUM>) of scaled image data with P-bits per pixel, wherein P is greater than M; and
- saturating the pixel values of the sets (<NUM>) of scaled image data by limiting pixel values to a maximum value, the maximum value defined by a maximum number of n that defines a number of bits per pixel (N-bits) to produce the sets (404a, 404b, 404c) of simulated image data, wherein N is less than P and equal to <NUM>, and
wherein the image data (<NUM>) having M-bits per pixel is an HDR image, and the processed image (<NUM>) is a LDR image having <NUM> bits per pixel.