System, method, and computer program for adjusting image contrast using parameterized cumulative distribution functions

A system and method are provided for optimizing histogram cumulative distribution function curves. In use, a first image is received and divided into two or more pixel regions. For at least one of the two or more pixel regions, a first histogram is computed, and based on the first histogram, at least one cumulative distribution function is computed for the at least one of the two or more pixel regions. Next, based on the at least one cumulative distribution function, two or more curve fit coefficients are extracted and interpolated. Further, an interpolated cumulative distribution function is created based on the interpolation and the interpolated cumulative distribution function is applied to the at least one of the two or more pixel regions.

FIELD OF THE INVENTION

The present invention relates to digital image processing, and more particularly to adjusting image contrast based on a parameterized cumulative distribution function.

BACKGROUND

Current digital photographic systems use histogram equalization and adaptive histogram equalization (including contrast-limited techniques) to improve perceived image detail and quality by adjusting contrast within digital images. Because adaptive histogram equalization is computationally intensive, certain approximation techniques are frequently implemented to reduce overall computational effort. One such approximation involves computing an effective cumulative distribution function (CDF) for a given pixel using bilinear interpolation between pre-computed CDFs for fixed regions within the image rather than computing a unique CDF for a region around the pixel. However, many issues can arise from using such techniques. For example, bilinear interpolation can produce visible artifacts along boundaries of the fixed regions in a resulting image. Such artifacts can become significant and degrade image quality when adjacent fixed regions have sufficiently different CDFs.

SUMMARY

A system and method are provided for optimizing histogram cumulative distribution function curves. In use, a first image is received and divided into two or more pixel regions. For at least one of the two or more pixel regions, a first histogram is computed, and based on the first histogram, at least one cumulative distribution function is computed for the at least one of the two or more pixel regions. Next, based on the at least one cumulative distribution function, two or more curve fit coefficients are extracted and interpolated. Further, an interpolated cumulative distribution function is created based on the interpolation and the interpolated cumulative distribution function is applied to the at least one of the two or more pixel regions.

DETAILED DESCRIPTION

FIG. 1illustrates an exemplary method100for applying a parameterized cumulative distribution function to a pixel region, in accordance with one possible embodiment. As shown, a first image is received (see operation102), and the first image is divided into two or more pixel regions (see operation104). Next, for at least one of the two or more pixel regions, a first histogram is computed (see operation106), and based on the first histogram, at least one cumulative distribution function for the at least one of the two or more pixel regions is computed (see operation108).

In the context of the present description, a cumulative distribution function (“CDF”) comprises a normalized accumulation of histogram values from a minimum intensity value for the histogram to a maximum intensity value for the histogram. In another embodiment, any type of histogram characterization function may be used.

Based on the at least one cumulative distribution function, two or more curve fit coefficients are extracted, (see operation110). In one embodiment, the two or more curve fit coefficients (CDF parameters) may include two or more points and two or more corresponding angles. The CDF parameters may be interpolated with respect to a second set of CDF parameters for a second pixel region to create an interpolated cumulative distribution function (see operation112). Further, the interpolated cumulative distribution function is applied to the at least one of the two or more pixel regions (see operation114).

In an embodiment, CDF parameters for the first pixel region may be used to represent a directly computed CDF for first pixel region. In another embodiment, a first set of CDF parameters are computed for the first pixel region and a second set of CDF parameters are computed for the second pixel region; furthermore, an interpolated CDF is computed for a given pixel based on the pixel position within the first pixel region. Any technically feasible technique may be used to interpolate between the first set of CDF parameters and the second set of CDF parameters. In yet another embodiment, CDF parameters are computed for different pixel regions comprising the first image, and an interpolated CDF for a given pixel is computed based on the pixel position, with CDF parameters of surrounding pixel regions contributing to the interpolated CDF according to distance from the pixel.

FIG. 2Aillustrates a method200for computing a cumulative distribution function, in accordance with one embodiment. As an option, the method200may be implemented in the context of any one or more of the embodiments set forth in any previous and/or subsequent figure(s) and/or description thereof. Of course, however, the method200may be implemented in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, an image201A may be divided into two or more pixel regions201B. One specific pixel region201C may be the basis for which a pixel region comprising N by N pixels may be determined (as shown in item201D). Next, a histogram201E is calculated for the pixel region201C, based on, for example, intensity values of pixels within the pixel region201C. Further, a cumulative distribution function201F may be calculated based on the histogram201E. The cumulative distribution function201F may be used to equalize a pixel from the center (or any location) of pixel region201C. In one embodiment, the pixel selected from pixel region201C may be associated with a pre-selected location, and/or may be associated with a pixel of an object in interest. For example, a pixel within a pixel region may include an image of a pen, and an object of interest may include a tip of the pen (e.g. based on focus point, etc.) such that a pixel associated with the tip of the pen used to equalize a pixel from the center of the pixel region selected. While the disclosed technique is described with respect to square pixel regions, any shape of region may be implemented without departing the scope of the present disclosure. For example, an N by M (N not equal to M) rectangular pixel region may be used instead of an N by N square pixel region.

FIG. 2Billustrates a cumulative distribution function and histogram203, in accordance with one embodiment. As an option, the cumulative distribution function and histogram203may be implemented in the context of any one or more of the embodiments set forth in any previous and/or subsequent figure(s) and/or description thereof. Of course, however, the cumulative distribution function and histogram203may be implemented in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, a cumulative distribution function204may be generated from a histogram202. In one embodiment, the cumulative distribution function may include an integral (accumulated sum) of histogram202, with the area of the cumulative distribution function204normalized, for example to one (1.0). Additionally, as has been indicated hereinabove, another histogram characterization function may be used to equalize and/or normalize the distribution of intensity values of the histogram202.

FIG. 2Cillustrates multiple cumulative distribution functions205, in accordance with one embodiment. As an option, the multiple cumulative distribution functions205may be implemented in the context of any one or more of the embodiments set forth in any previous and/or subsequent figure(s) and/or description thereof. Of course, however, the multiple cumulative distribution functions205may be implemented in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, the multiple cumulative distribution functions205may include function206, function208, function210, function212, and/or any number of additional functions. Of course, it is to be appreciated that the distributions functions205may include any number of functions. By construction, cumulative distribution functions may be monotonic (and/or include a non-decreasing distribution). In general, cumulative distribution functions may follow a similar shape (as shown). Consequently, approximating (modeling) a cumulative distribution function as a set of curve-fit coefficients of a curve-fit function may be relatively simple. In one embodiment, any of function206, function208, function210, and/or function212may be used as the basis for determining curve fit coefficients for a parameterized CDF207(shown below). Further, one or more functions may be created based on histogram202, and of which may be the basis for calculating the determining curve fit coefficients.

FIG. 2Dillustrates a parameterized CDF207, in accordance with one embodiment. As an option, the a parameterized CDF207may be implemented in the context of any one or more of the embodiments set forth in any previous and/or subsequent figure(s) and/or description thereof. Of course, however, the a parameterized CDF207may be implemented in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, a cumulative distribution function214may be modeled as a parameterized CDF by a first point216A, first angle218A, second point216B, and second angle218B. In this context, the first point216A, the first angle218A, the second point216B, and the second angle218B comprise control points for an arbitrary spline curve, such a Bézier curve, an exponential function, a NURB, or any other technically feasible parameterized curve element or basis function. Furthermore, any technically feasible curve fit basis function may be implemented (e.g., B-spline, exponential, etc.). Of course, any number of points and angles may be used to create a parameterized CDF and/or interpolated CDF. In one embodiment, an increased number of points and angles may be used for a more accurate parameterized cumulative distribution function, which may require an additional computational effort. In an embodiment, at a minimum, two points and angles may be used for purposes of creating a parameterized cumulative distribution function. In this manner, the parameterized CDF may be used to more efficiently represent a CDF for a pixel region, as only a small number of control point values need be stored rather than a value for each possible intensity quantization level. Furthermore, interpolating parameters among two or more parameterized CDF functions can eliminate bilinear filtering artifacts associated with conventional techniques.

FIG. 2Eillustrates interpolation209based on a cumulative distribution function, in accordance with one embodiment. Such interpolation may be implemented to approximate individual cumulative distribution functions for individual pixels within a given pixel region. As an option, the interpolation209may be implemented in the context of any one or more of the embodiments set forth in any previous and/or subsequent figure(s) and/or description thereof. Of course, however, the interpolation209may be implemented in the context of any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, the interpolation209may be applied to a first image220and a first region222of first image220. For a first pixel224, a pixel region225around the first pixel224defines a first vertical distance226and a second vertical distance228, as well as a first horizontal distance230, and a second horizontal distance232. Using the first vertical distance226, the second vertical distance228, the first horizontal distance230, and the second horizontal distance232, a corresponding interpolated CDF may be applied to the pixel224. In an embodiment, the interpolated CDF is generated by interpolating respective parameters for parameterized CDFs of pixel regions surrounding the first pixel224; and an equalized pixel value for the first pixel224is computed using the interpolated parameters in a parameterized CDF generated for the first pixel224. In another embodiment, equalized pixels are generated for the first pixel224according to parameterized CDFs for surrounding pixel regions the first pixel224; and the equalized pixels are interpolated to generate an equalized pixel value for the first pixel224. Additionally, perFIG. 2D, the adjusted cumulative distribution function may be applied to the pixel224and to additional pixels comprising the first region222of first image220.

For example, an interpolation of weights may be applied to the first region222of first image220based on a parameterized and interpolated CDF (as described herein). A first of multiple interpolations based on a parameterized CDF may be along the horizontal (e.g. such as first horizontal distance230and second horizontal distance232). Additionally, a second of the interpolations based on the parameterized CDF may be along the vertical (e.g. such as first vertical distance226and second vertical distance228). A computation for each pixel may occur whereby a corresponding interpolated CDF is applied to each pixel. Additionally, an interpolation of weights may be applied to pixels surrounding such modified pixel within the region222of first image220such that effects of a given parameterized CDF may be propagated to the surrounding pixels.

In one embodiment, depending on how close the pixel is to a border, a greater weight may be applied (for the interpolation). Additionally, an interpolation map defining parameter weights for respective parameterized CDFs may be constructed and applied, e.g. to the first pixel224. Application of an interpolated CDF for each pixel may be used to equalize (e.g., reassign) an intensity for the first pixel224. In such an embodiment, color and RGB values may be preserved for the first pixel224.

In an embodiment, a set of parameterized CDFs for an image may be represented as an array of CDF parameters. For example, each different parameterized CDF corresponding to a region of the image may be represented as array of parameters stored as elements of a texture map. Each of the parameterized CDFs may be represented as curve-fit coefficients (parameters) for greater storage efficiency relative to storing a conventional CDF.

It should be noted that although the specification may reference a cumulative distribution function (CDF), an interpolated CDF, and/or a parameterized CDF, the techniques disclosed herein may equally apply to any such functions.

Still yet, equalizing a given pixel region may include modifying a contrast over a range of pixels according to pixel position and corresponding interpolated CDFs. Additionally, the cumulative distribution function may include a re-mapping for intensity values for pixel. For example, the intensity of pixel values may be adjusted for first pixel224, and/or may be adjusted for pixels surrounding first pixel224within the region222of the first image220(e.g. based on weights and interpolation). In another embodiment, the intensity of pixel values may be adjusted for a range of pixels, wherein one pixel from each of a plurality of pixel regions is each adjusted.

In another embodiment, the cumulative distribution function may be applied to a subset of an image. For example, it may be determined that an object within the image is of high priority (e.g. a face, a building, etc.) and equalization using interpolated CDFs may be applied directly to the pixel region associated with the high priority object. In one embodiment, the priority ascribed to a particular object may be predetermined, and in some instances, provided by user input (e.g., by selecting a point of interest of object type of interest). Additionally, artificial intelligence may be applied to determine one or more parts (or objects) of interest in an image to extract or use as a basis for the cumulative distribution function.

A pixel region (such as first pixel region222) may comprise multiple pixels. In another embodiment, a pixel region may be determined based on a selected pixel. In a first step, a histogram is computed for a given pixel region. As an example, if an image was divided into twenty pixel regions, twenty histograms would be computed. In a second step, a cumulative distribution function is computed for each pixel region. Using the same example, for twenty pixel regions, twenty parameterized CDFs would be computed from respective cumulative distribution functions. In contrast to conventional methods and systems, a parameterized cumulative distribution function provides for both a more compact representation for storage and may inherently eliminate bilinear interpolation artifacts due to slope matching properties of parameterized curves including, without limitation, Bézier curves and NURBs.

In one embodiment, an interpolated CDF may include interpolated curve-fit coefficients from a two-by-two pixel region of N-by-N pixels each. An individual interpolated CDF may be applied to equalize an individual pixel by generating an interpolated CDF at the pixel location. Alternatively, a pixel may be equalized by four different curve-fit cumulative distribution functions, with the resulting four equalized pixel values interpolated to form a final value for the pixel. Any technically feasible technique may be used to map a pixel value to an equalized pixel value; for example, techniques known in the art may be applied to perform pixel equalization using the presently disclosed technique of generating a curve-fit cumulative distribution function. In an alternative embodiment, parameterized CDF parameters are interpolated using bicubic interpolation.

In one embodiment, use of the adjusted cumulative distribution function (based on the curve fit coefficients) may be used to eliminate artifacts. For example, bilinear interpolation on samples that are over-zoomed may cause artifacts such as subtle horizontal or vertical lines (e.g. shown inFIG. 5). By using the interpolated CDF, however, such artifacts may be eliminated.

In another embodiment, cumulative distribution functions may differ slightly between pixel regions (e.g. shown inFIG. 2C). Notwithstanding such differences (which may be very minor), the cumulative distribution functions may be represented by approximating such cumulative distribution functions via at least two points and at least two angles (shown inFIG. 2D). Of course, any number of points and accompanying angles may be taken. In this manner, rather than recording an entire array, a number of points and angles may be taken for each cumulative distribution function.

As such, use of curve fit coefficients to compute a parameterized CDF may include an ancillary benefit of more efficient processing of data and less memory, which in turn may require less power and preserve battery usage.

Additionally, use of curve fit coefficients to compute a parameterized CDF may also eliminate artifacts along bilinear interpolation boundaries (e.g. shown inFIG. 5). In this manner, a typical cumulative distribution function (which may be represented as an array) may be replaced with adjusted parameterized cumulative distribution function (which may be represented as a function) that can be computed per pixel, per pixel region.

FIG. 3Aillustrates a digital photographic system300, in accordance with one embodiment. As an option, the digital photographic system300may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the digital photographic system300may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, the digital photographic system300may include a processor complex310coupled to a camera module330via an interconnect334. In one embodiment, the processor complex310is coupled to a strobe unit336. The digital photographic system300may also include, without limitation, a display unit312, a set of input/output devices314, non-volatile memory316, volatile memory318, a wireless unit340, and sensor devices342, each coupled to the processor complex310. In one embodiment, a power management subsystem320is configured to generate appropriate power supply voltages for each electrical load element within the digital photographic system300. A battery322may be configured to supply electrical energy to the power management subsystem320. The battery322may implement any technically feasible energy storage system, including primary or rechargeable battery technologies. Of course, in other embodiments, additional or fewer features, units, devices, sensors, or subsystems may be included in the system.

In one embodiment, a strobe unit336may be integrated into the digital photographic system300and configured to provide strobe illumination350during an image sample event performed by the digital photographic system300. In another embodiment, a strobe unit336may be implemented as an independent device from the digital photographic system300and configured to provide strobe illumination350during an image sample event performed by the digital photographic system300. The strobe unit336may comprise one or more LED devices, a gas-discharge illuminator (e.g. a Xenon strobe device, a Xenon flash lamp, etc.), or any other technically feasible illumination device. In certain embodiments, two or more strobe units are configured to synchronously generate strobe illumination in conjunction with sampling an image. In one embodiment, the strobe unit336is controlled through a strobe control signal338to either emit the strobe illumination350or not emit the strobe illumination350. The strobe control signal338may be implemented using any technically feasible signal transmission protocol. The strobe control signal338may indicate a strobe parameter (e.g. strobe intensity, strobe color, strobe time, etc.), for directing the strobe unit336to generate a specified intensity and/or color of the strobe illumination350. The strobe control signal338may be generated by the processor complex310, the camera module330, or by any other technically feasible combination thereof. In one embodiment, the strobe control signal338is generated by a camera interface unit within the processor complex310and transmitted to both the strobe unit336and the camera module330via the interconnect334. In another embodiment, the strobe control signal338is generated by the camera module330and transmitted to the strobe unit336via the interconnect334.

Optical scene information352, which may include at least a portion of the strobe illumination350reflected from objects in the photographic scene, is focused as an optical image onto an image sensor332within the camera module330. The image sensor332generates an electronic representation of the optical image. The electronic representation comprises spatial color intensity information, which may include different color intensity samples (e.g. red, green, and blue light, etc.). In other embodiments, the spatial color intensity information may also include samples for white light. The electronic representation is transmitted to the processor complex310via the interconnect334, which may implement any technically feasible signal transmission protocol.

In one embodiment, input/output devices314may include, without limitation, a capacitive touch input surface, a resistive tablet input surface, one or more buttons, one or more knobs, light-emitting devices, light detecting devices, sound emitting devices, sound detecting devices, or any other technically feasible device for receiving user input and converting the input to electrical signals, or converting electrical signals into a physical signal. In one embodiment, the input/output devices314include a capacitive touch input surface coupled to a display unit312. A touch entry display system may include the display unit312and a capacitive touch input surface, also coupled to processor complex310.

Additionally, in other embodiments, non-volatile (NV) memory316is configured to store data when power is interrupted. In one embodiment, the NV memory316comprises one or more flash memory devices (e.g. ROM, PCM, FeRAM, FRAM, PRAM, MRAM, NRAM, etc.). The NV memory316comprises a non-transitory computer-readable medium, which may be configured to include programming instructions for execution by one or more processing units within the processor complex310. The programming instructions may implement, without limitation, an operating system (OS), UI software modules, image processing and storage software modules, one or more input/output devices314connected to the processor complex310, one or more software modules for sampling an image stack through camera module330, one or more software modules for presenting the image stack or one or more synthetic images generated from the image stack through the display unit312. As an example, in one embodiment, the programming instructions may also implement one or more software modules for merging images or portions of images within the image stack, aligning at least portions of each image within the image stack, or a combination thereof. In another embodiment, the processor complex310may be configured to execute the programming instructions, which may implement one or more software modules operable to create a high dynamic range (HDR) image.

Still yet, in one embodiment, one or more memory devices comprising the NV memory316may be packaged as a module configured to be installed or removed by a user. In one embodiment, volatile memory318comprises dynamic random access memory (DRAM) configured to temporarily store programming instructions, image data such as data associated with an image stack, and the like, accessed during the course of normal operation of the digital photographic system300. Of course, the volatile memory may be used in any manner and in association with any other input/output device314or sensor device342attached to the process complex310.

In one embodiment, sensor devices342may include, without limitation, one or more of an accelerometer to detect motion and/or orientation, an electronic gyroscope to detect motion and/or orientation, a magnetic flux detector to detect orientation, a global positioning system (GPS) module to detect geographic position, or any combination thereof. Of course, other sensors, including but not limited to a motion detection sensor, a proximity sensor, an RGB light sensor, a gesture sensor, a 3-D input image sensor, a pressure sensor, and an indoor position sensor, may be integrated as sensor devices. In one embodiment, the sensor devices may be one example of input/output devices314.

Wireless unit340may include one or more digital radios configured to send and receive digital data. In particular, the wireless unit340may implement wireless standards (e.g. WiFi, Bluetooth, NFC, etc.), and may implement digital cellular telephony standards for data communication (e.g. CDMA, 3G, 4G, LTE, LTE-Advanced, etc.). Of course, any wireless standard or digital cellular telephony standards may be used.

In one embodiment, the digital photographic system300is configured to transmit one or more digital photographs to a network-based (online) or “cloud-based” photographic media service via the wireless unit340. The one or more digital photographs may reside within either the NV memory316or the volatile memory318, or any other memory device associated with the processor complex310. In one embodiment, a user may possess credentials to access an online photographic media service and to transmit one or more digital photographs for storage to, retrieval from, and presentation by the online photographic media service. The credentials may be stored or generated within the digital photographic system300prior to transmission of the digital photographs. The online photographic media service may comprise a social networking service, photograph sharing service, or any other network-based service that provides storage of digital photographs, processing of digital photographs, transmission of digital photographs, sharing of digital photographs, or any combination thereof. In certain embodiments, one or more digital photographs are generated by the online photographic media service based on image data (e.g. image stack, HDR image stack, image package, etc.) transmitted to servers associated with the online photographic media service. In such embodiments, a user may upload one or more source images from the digital photographic system300for processing by the online photographic media service.

In one embodiment, the digital photographic system300comprises at least one instance of a camera module330. In another embodiment, the digital photographic system300comprises a plurality of camera modules330. Such an embodiment may also include at least one strobe unit336configured to illuminate a photographic scene, sampled as multiple views by the plurality of camera modules330. The plurality of camera modules330may be configured to sample a wide angle view (e.g., greater than forty-five degrees of sweep among cameras) to generate a panoramic photograph. In one embodiment, a plurality of camera modules330may be configured to sample two or more narrow angle views (e.g., less than forty-five degrees of sweep among cameras) to generate a stereoscopic photograph. In other embodiments, a plurality of camera modules330may be configured to generate a 3-D image or to otherwise display a depth perspective (e.g. a z-component, etc.) as shown on the display unit312or any other display device.

In one embodiment, a display unit312may be configured to display a two-dimensional array of pixels to form an image for display. The display unit312may comprise a liquid-crystal (LCD) display, a light-emitting diode (LED) display, an organic LED display, or any other technically feasible type of display. In certain embodiments, the display unit312may be able to display a narrower dynamic range of image intensity values than a complete range of intensity values sampled from a photographic scene, such as within a single HDR image or over a set of two or more images comprising a multiple exposure or HDR image stack. In one embodiment, images comprising an image stack may be merged according to any technically feasible HDR blending technique to generate a synthetic image for display within dynamic range constraints of the display unit312. In one embodiment, the limited dynamic range may specify an eight-bit per color channel binary representation of corresponding color intensities. In other embodiments, the limited dynamic range may specify more than eight-bits (e.g., 10 bits, 12 bits, or 14 bits, etc.) per color channel binary representation.

FIG. 3Billustrates a processor complex310within the digital photographic system300ofFIG. 3A, in accordance with one embodiment. As an option, the processor complex310may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the processor complex310may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

As shown, the processor complex310includes a processor subsystem360and may include a memory subsystem362. In one embodiment, processor complex310may comprise a system on a chip (SoC) device that implements processor subsystem360, and memory subsystem362comprises one or more DRAM devices coupled to the processor subsystem360. In another embodiment, the processor complex310may comprise a multi-chip module (MCM) encapsulating the SoC device and the one or more DRAM devices comprising the memory subsystem362.

The processor subsystem360may include, without limitation, one or more central processing unit (CPU) cores370, a memory interface380, input/output interfaces unit384, and a display interface unit382, each coupled to an interconnect374. The one or more CPU cores370may be configured to execute instructions residing within the memory subsystem362, volatile memory318, NV memory316, or any combination thereof. Each of the one or more CPU cores370may be configured to retrieve and store data through interconnect374and the memory interface380. In one embodiment, each of the one or more CPU cores370may include a data cache, and an instruction cache. Additionally, two or more of the CPU cores370may share a data cache, an instruction cache, or any combination thereof. In one embodiment, a cache hierarchy is implemented to provide each CPU core370with a private cache layer, and a shared cache layer.

In some embodiments, processor subsystem360may include one or more graphics processing unit (GPU) cores372. Each GPU core372may comprise a plurality of multi-threaded execution units that may be programmed to implement, without limitation, graphics acceleration functions. In various embodiments, the GPU cores372may be configured to execute multiple thread programs according to well-known standards (e.g. OpenGL™, WebGL™, OpenCL™, CUDA™, etc.), and/or any other programmable rendering graphic standard. In certain embodiments, at least one GPU core372implements at least a portion of a motion estimation function, such as a well-known Harris detector or a well-known Hessian-Laplace detector. Such a motion estimation function may be used at least in part to align images or portions of images within an image stack. For example, in one embodiment, an HDR image may be compiled based on an image stack, where two or more images are first aligned prior to compiling the HDR image.

As shown, the interconnect374is configured to transmit data between and among the memory interface380, the display interface unit382, the input/output interfaces unit384, the CPU cores370, and the GPU cores372. In various embodiments, the interconnect374may implement one or more buses, one or more rings, a cross-bar, a mesh, or any other technically feasible data transmission structure or technique. The memory interface380is configured to couple the memory subsystem362to the interconnect374. The memory interface380may also couple NV memory316, volatile memory318, or any combination thereof to the interconnect374. The display interface unit382may be configured to couple a display unit312to the interconnect374. The display interface unit382may implement certain frame buffer functions (e.g. frame refresh, etc.). Alternatively, in another embodiment, the display unit312may implement certain frame buffer functions (e.g. frame refresh, etc.). The input/output interfaces unit384may be configured to couple various input/output devices to the interconnect374.

In certain embodiments, a camera module330is configured to store exposure parameters for sampling each image associated with an image stack. For example, in one embodiment, when directed to sample a photographic scene, the camera module330may sample a set of images comprising the image stack according to stored exposure parameters. A software module comprising programming instructions executing within a processor complex310may generate and store the exposure parameters prior to directing the camera module330to sample the image stack. In other embodiments, the camera module330may be used to meter an image or an image stack, and the software module comprising programming instructions executing within a processor complex310may generate and store metering parameters prior to directing the camera module330to capture the image. Of course, the camera module330may be used in any manner in combination with the processor complex310.

In one embodiment, exposure parameters associated with images comprising the image stack may be stored within an exposure parameter data structure that includes exposure parameters for one or more images. In another embodiment, a camera interface unit (not shown inFIG. 3B) within the processor complex310may be configured to read exposure parameters from the exposure parameter data structure and to transmit associated exposure parameters to the camera module330in preparation of sampling a photographic scene. After the camera module330is configured according to the exposure parameters, the camera interface may direct the camera module330to sample the photographic scene; the camera module330may then generate a corresponding image stack. The exposure parameter data structure may be stored within the camera interface unit, a memory circuit within the processor complex310, volatile memory318, NV memory316, the camera module330, or within any other technically feasible memory circuit. Further, in another embodiment, a software module executing within processor complex310may generate and store the exposure parameter data structure.

FIG. 3Cillustrates a digital camera302, in accordance with one embodiment. As an option, the digital camera302may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the digital camera302may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the digital camera302may be configured to include a digital photographic system, such as digital photographic system300ofFIG. 3A. As shown, the digital camera302includes a camera module330, which may include optical elements configured to focus optical scene information representing a photographic scene onto an image sensor, which may be configured to convert the optical scene information to an electronic representation of the photographic scene.

Additionally, the digital camera302may include a strobe unit336, and may include a shutter release button315for triggering a photographic sample event, whereby digital camera302samples one or more images comprising the electronic representation. In other embodiments, any other technically feasible shutter release mechanism may trigger the photographic sample event (e.g. such as a timer trigger or remote control trigger, etc.).

FIG. 3Dillustrates a wireless mobile device376, in accordance with one embodiment. As an option, the mobile device376may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the mobile device376may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the mobile device376may be configured to include a digital photographic system (e.g. such as digital photographic system300ofFIG. 3A), which is configured to sample a photographic scene. In various embodiments, a camera module330may include optical elements configured to focus optical scene information representing the photographic scene onto an image sensor, which may be configured to convert the optical scene information to an electronic representation of the photographic scene. Further, a shutter release command may be generated through any technically feasible mechanism, such as a virtual button, which may be activated by a touch gesture on a touch entry display system comprising display unit312, or a physical button, which may be located on any face or surface of the mobile device376. Of course, in other embodiments, any number of other buttons, external inputs/outputs, or digital inputs/outputs may be included on the mobile device376, and which may be used in conjunction with the camera module330.

As shown, in one embodiment, a touch entry display system comprising display unit312is disposed on the opposite side of mobile device376from camera module330. In certain embodiments, the mobile device376includes a user-facing camera module331and may include a user-facing strobe unit (not shown). Of course, in other embodiments, the mobile device376may include any number of user-facing camera modules or rear-facing camera modules, as well as any number of user-facing strobe units or rear-facing strobe units.

In some embodiments, the digital camera302and the mobile device376may each generate and store a synthetic image based on an image stack sampled by camera module330. The image stack may include one or more images sampled under ambient lighting conditions, one or more images sampled under strobe illumination from strobe unit336, or a combination thereof.

FIG. 3Eillustrates camera module330, in accordance with one embodiment. As an option, the camera module330may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the camera module330may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the camera module330may be configured to control strobe unit336through strobe control signal338. As shown, a lens390is configured to focus optical scene information352onto image sensor332to be sampled. In one embodiment, image sensor332advantageously controls detailed timing of the strobe unit336though the strobe control signal338to reduce inter-sample time between an image sampled with the strobe unit336enabled, and an image sampled with the strobe unit336disabled. For example, the image sensor332may enable the strobe unit336to emit strobe illumination350less than one microsecond (or any desired length) after image sensor332completes an exposure time associated with sampling an ambient image and prior to sampling a strobe image.

In other embodiments, the strobe illumination350may be configured based on a desired one or more target points. For example, in one embodiment, the strobe illumination350may light up an object in the foreground, and depending on the length of exposure time, may also light up an object in the background of the image. In one embodiment, once the strobe unit336is enabled, the image sensor332may then immediately begin exposing a strobe image. The image sensor332may thus be able to directly control sampling operations, including enabling and disabling the strobe unit336associated with generating an image stack, which may comprise at least one image sampled with the strobe unit336disabled, and at least one image sampled with the strobe unit336either enabled or disabled. In one embodiment, data comprising the image stack sampled by the image sensor332is transmitted via interconnect334to a camera interface unit386within processor complex310. In some embodiments, the camera module330may include an image sensor controller (e.g., controller333ofFIG. 3G), which may be configured to generate the strobe control signal338in conjunction with controlling operation of the image sensor332.

FIG. 3Fillustrates a camera module330, in accordance with one embodiment. As an option, the camera module330may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the camera module330may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the camera module330may be configured to sample an image based on state information for strobe unit336. The state information may include, without limitation, one or more strobe parameters (e.g. strobe intensity, strobe color, strobe time, etc.), for directing the strobe unit336to generate a specified intensity and/or color of the strobe illumination350. In one embodiment, commands for configuring the state information associated with the strobe unit336may be transmitted through a strobe control signal338, which may be monitored by the camera module330to detect when the strobe unit336is enabled. For example, in one embodiment, the camera module330may detect when the strobe unit336is enabled or disabled within a microsecond or less of the strobe unit336being enabled or disabled by the strobe control signal338. To sample an image requiring strobe illumination, a camera interface unit386may enable the strobe unit336by sending an enable command through the strobe control signal338. In one embodiment, the camera interface unit386may be included as an interface of input/output interfaces384in a processor subsystem360of the processor complex310ofFIG. 3B. The enable command may comprise a signal level transition, a data packet, a register write, or any other technically feasible transmission of a command. The camera module330may sense that the strobe unit336is enabled and then cause image sensor332to sample one or more images requiring strobe illumination while the strobe unit336is enabled. In such an implementation, the image sensor332may be configured to wait for an enable signal destined for the strobe unit336as a trigger signal to begin sampling a new exposure.

In one embodiment, camera interface unit386may transmit exposure parameters and commands to camera module330through interconnect334. In certain embodiments, the camera interface unit386may be configured to directly control strobe unit336by transmitting control commands to the strobe unit336through strobe control signal338. By directly controlling both the camera module330and the strobe unit336, the camera interface unit386may cause the camera module330and the strobe unit336to perform their respective operations in precise time synchronization. In one embodiment, precise time synchronization may be less than five hundred microseconds of event timing error. Additionally, event timing error may be a difference in time from an intended event occurrence to the time of a corresponding actual event occurrence.

In another embodiment, camera interface unit386may be configured to accumulate statistics while receiving image data from camera module330. In particular, the camera interface unit386may accumulate exposure statistics for a given image while receiving image data for the image through interconnect334. Exposure statistics may include, without limitation, one or more of an intensity histogram, a count of over-exposed pixels, a count of under-exposed pixels, an intensity-weighted sum of pixel intensity, or any combination thereof. The camera interface unit386may present the exposure statistics as memory-mapped storage locations within a physical or virtual address space defined by a processor, such as one or more of CPU cores370, within processor complex310. In one embodiment, exposure statistics reside in storage circuits that are mapped into a memory-mapped register space, which may be accessed through the interconnect334. In other embodiments, the exposure statistics are transmitted in conjunction with transmitting pixel data for a captured image. For example, the exposure statistics for a given image may be transmitted as in-line data, following transmission of pixel intensity data for the captured image. Exposure statistics may be calculated, stored, or cached within the camera interface unit386. In other embodiments, an image sensor controller within camera module330may be configured to accumulate the exposure statistics and transmit the exposure statistics to processor complex310, such as by way of camera interface unit386. In one embodiment, the exposure statistics are accumulated within the camera module330and transmitted to the camera interface unit386, either in conjunction with transmitting image data to the camera interface unit386, or separately from transmitting image data.

In one embodiment, camera interface unit386may accumulate color statistics for estimating scene white-balance. Any technically feasible color statistics may be accumulated for estimating white balance, such as a sum of intensities for different color channels comprising red, green, and blue color channels. The sum of color channel intensities may then be used to perform a white-balance color correction on an associated image, according to a white-balance model such as a gray-world white-balance model. In other embodiments, curve-fitting statistics are accumulated for a linear or a quadratic curve fit used for implementing white-balance correction on an image. As with the exposure statistics, the color statistics may be presented as memory-mapped storage locations within processor complex310. In one embodiment, the color statistics may be mapped in a memory-mapped register space, which may be accessed through interconnect334. In other embodiments, the color statistics may be transmitted in conjunction with transmitting pixel data for a captured image. For example, in one embodiment, the color statistics for a given image may be transmitted as in-line data, following transmission of pixel intensity data for the image. Color statistics may be calculated, stored, or cached within the camera interface386. In other embodiments, the image sensor controller within camera module330may be configured to accumulate the color statistics and transmit the color statistics to processor complex310, such as by way of camera interface unit386. In one embodiment, the color statistics may be accumulated within the camera module330and transmitted to the camera interface unit386, either in conjunction with transmitting image data to the camera interface unit386, or separately from transmitting image data.

In one embodiment, camera interface unit386may accumulate spatial color statistics for performing color-matching between or among images, such as between or among an ambient image and one or more images sampled with strobe illumination. As with the exposure statistics, the spatial color statistics may be presented as memory-mapped storage locations within processor complex310. In one embodiment, the spatial color statistics are mapped in a memory-mapped register space. In another embodiment the camera module may be configured to accumulate the spatial color statistics, which may be accessed through interconnect334. In other embodiments, the color statistics may be transmitted in conjunction with transmitting pixel data for a captured image. For example, in one embodiment, the color statistics for a given image may be transmitted as in-line data, following transmission of pixel intensity data for the image. Color statistics may be calculated, stored, or cached within the camera interface386.

In one embodiment, camera module330may transmit strobe control signal338to strobe unit336, enabling the strobe unit336to generate illumination while the camera module330is sampling an image. In another embodiment, camera module330may sample an image illuminated by strobe unit336upon receiving an indication signal from camera interface unit386that the strobe unit336is enabled. In yet another embodiment, camera module330may sample an image illuminated by strobe unit336upon detecting strobe illumination within a photographic scene via a rapid rise in scene illumination. In one embodiment, a rapid rise in scene illumination may include at least a rate of increasing intensity consistent with that of enabling strobe unit336. In still yet another embodiment, camera module330may enable strobe unit336to generate strobe illumination while sampling one image, and disable the strobe unit336while sampling a different image.

FIG. 3Gillustrates camera module330, in accordance with one embodiment. As an option, the camera module330may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the camera module330may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the camera module330may be in communication with an application processor335. The camera module330is shown to include image sensor332in communication with a controller333. Further, the controller333is shown to be in communication with the application processor335.

In one embodiment, the application processor335may reside outside of the camera module330. As shown, the lens390may be configured to focus optical scene information to be sampled onto image sensor332. The optical scene information sampled by the image sensor332may then be communicated from the image sensor332to the controller333for at least one of subsequent processing and communication to the application processor335. In another embodiment, the controller333may control storage of the optical scene information sampled by the image sensor332, or storage of processed optical scene information.

In another embodiment, the controller333may enable a strobe unit to emit strobe illumination for a short time duration (e.g. less than ten milliseconds) after image sensor332completes an exposure time associated with sampling an ambient image. Further, the controller333may be configured to generate strobe control signal338in conjunction with controlling operation of the image sensor332.

In one embodiment, the image sensor332may be a complementary metal oxide semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor. In another embodiment, the controller333and the image sensor332may be packaged together as an integrated system, multi-chip module, multi-chip stack, or integrated circuit. In yet another embodiment, the controller333and the image sensor332may comprise discrete packages. In one embodiment, the controller333may provide circuitry for receiving optical scene information from the image sensor332, processing of the optical scene information, timing of various functionalities, and signaling associated with the application processor335. Further, in another embodiment, the controller333may provide circuitry for control of one or more of exposure, shuttering, white balance, and gain adjustment. Processing of the optical scene information by the circuitry of the controller333may include one or more of gain application, amplification, and analog-to-digital conversion. After processing the optical scene information, the controller333may transmit corresponding digital pixel data, such as to the application processor335.

In one embodiment, the application processor335may be implemented on processor complex310and at least one of volatile memory318and NV memory316, or any other memory device and/or system. The application processor335may be previously configured for processing of received optical scene information or digital pixel data communicated from the camera module330to the application processor335.

FIG. 4illustrates a network service system400, in accordance with one embodiment. As an option, the network service system400may be implemented in the context of the details of any of the Figures disclosed herein. Of course, however, the network service system400may be implemented in any desired environment. Further, the aforementioned definitions may equally apply to the description below.

In one embodiment, the network service system400may be configured to provide network access to a device implementing a digital photographic system. As shown, network service system400includes a wireless mobile device376, a wireless access point472, a data network474, a data center480, and a data center481. The wireless mobile device376may communicate with the wireless access point472via a digital radio link471to send and receive digital data, including data associated with digital images. The wireless mobile device376and the wireless access point472may implement any technically feasible transmission techniques for transmitting digital data via digital radio link471without departing the scope and spirit of the present invention. In certain embodiments, one or more of data centers480,481may be implemented using virtual constructs so that each system and subsystem within a given data center480,481may comprise virtual machines configured to perform data processing and network data transmission tasks. In other implementations, one or more of data centers480,481may be physically distributed over a plurality of physical sites.

The wireless mobile device376may comprise a smart phone configured to include a digital camera, a digital camera configured to include wireless network connectivity, a reality augmentation device, a laptop configured to include a digital camera and wireless network connectivity, or any other technically feasible computing device configured to include a digital photographic system and wireless network connectivity.

In various embodiments, the wireless access point472may be configured to communicate with wireless mobile device376via the digital radio link471and to communicate with the data network474via any technically feasible transmission media, such as any electrical, optical, or radio transmission media. For example, in one embodiment, wireless access point472may communicate with data network474through an optical fiber coupled to the wireless access point472and to a router system or a switch system within the data network474. A network link475, such as a wide area network (WAN) link, may be configured to transmit data between the data network474and the data center480.

In one embodiment, the data network474may include routers, switches, long-haul transmission systems, provisioning systems, authorization systems, and any technically feasible combination of communications and operations subsystems configured to convey data between network endpoints, such as between the wireless access point472and the data center480. In one implementation scenario, wireless mobile device376may comprise one of a plurality of wireless mobile devices configured to communicate with the data center480via one or more wireless access points coupled to the data network474.

Additionally, in various embodiments, the data center480may include, without limitation, a switch/router482and at least one data service system484. The switch/router482may be configured to forward data traffic between and among a network link475, and each data service system484. The switch/router482may implement any technically feasible transmission techniques, such as Ethernet media layer transmission, layer2switching, layer3routing, and the like. The switch/router482may comprise one or more individual systems configured to transmit data between the data service systems484and the data network474.

In one embodiment, the switch/router482may implement session-level load balancing among a plurality of data service systems484. Each data service system484may include at least one computation system488and may also include one or more storage systems486. Each computation system488may comprise one or more processing units, such as a central processing unit, a graphics processing unit, or any combination thereof. A given data service system484may be implemented as a physical system comprising one or more physically distinct systems configured to operate together. Alternatively, a given data service system484may be implemented as a virtual system comprising one or more virtual systems executing on an arbitrary physical system. In certain scenarios, the data network474may be configured to transmit data between the data center480and another data center481, such as through a network link476.

In another embodiment, the network service system400may include any networked mobile devices configured to implement one or more embodiments of the present invention. For example, in some embodiments, a peer-to-peer network, such as an ad-hoc wireless network, may be established between two different wireless mobile devices. In such embodiments, digital image data may be transmitted between the two wireless mobile devices without having to send the digital image data to a data center480.

As shown, item502is the original image, generated using prior art local equalization techniques (CLAHE technique known in the art). Item504emphasizes that the artifacts are found within emphasis506, with the actual border artifacts faintly shown along dotted line border508. To more clearly emphasize such artifacts, mid-tones from the item502were adjusted towards shadows (using levels adjustment). In this manner, the darker hues associated with the sky are shown. Emphasis510A shows faint vertical artifacts, and emphasis510B shows horizontal artifacts, both of which may result from conventional image interpolation algorithms, in particular bilinear interpolation algorithms, including bilinear interpolation used in CLAHE.

FIG. 6illustrates a network architecture600, in accordance with one possible embodiment. As shown, at least one network602is provided. In the context of the present network architecture600, the network602may take any form including, but not limited to a telecommunications network, a local area network (LAN), a wireless network, a wide area network (WAN) such as the Internet, peer-to-peer network, cable network, etc. While only one network is shown, it should be understood that two or more similar or different networks602may be provided.

Coupled to the network602is a plurality of devices. For example, a server computer612and an end user computer608may be coupled to the network602for communication purposes. Such end user computer608may include a desktop computer, lap-top computer, and/or any other type of logic. Still yet, various other devices may be coupled to the network602including a personal digital assistant (PDA) device610, a mobile phone device606, a television604, a camera614, etc.

FIG. 7illustrates an exemplary system700, in accordance with one embodiment. As an option, the system700may be implemented in the context of any of the devices of the network architecture600ofFIG. 6. Of course, the system700may be implemented in any desired environment.

As shown, a system700is provided including at least one central processor702which is connected to a communication bus712. The system700also includes main memory704[e.g. random access memory (RAM), etc.]. The system700also includes a graphics processor708and a display710.

The system700may also include a secondary storage706. The secondary storage706includes, for example, a hard disk drive and/or a removable storage drive, representing a floppy disk drive, a magnetic tape drive, a compact disk drive, etc. The removable storage drive reads from and/or writes to a removable storage unit in a well known manner.

Computer programs, or computer control logic algorithms, may be stored in the main memory704, the secondary storage706, and/or any other memory, for that matter. Such computer programs, when executed, enable the system700to perform various functions (as set forth above, for example). Memory704, storage706and/or any other storage are possible examples of non-transitory computer-readable media.

It is noted that the techniques described herein, in an aspect, are embodied in executable instructions stored in a computer readable medium for use by or in connection with an instruction execution machine, apparatus, or device, such as a computer-based or processor-containing machine, apparatus, or device. It will be appreciated by those skilled in the art that for some embodiments, other types of computer readable media are included which may store data that is accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, random access memory (RAM), read-only memory (ROM), and the like.