Method and apparatus for image optimization in backlit displays

A display apparatus 200, a display controller 222, and a method for optimizing a displayed image for use in an electronic device 100 comprising a display 208 for presenting a visual image, a processor 212 for determining the intensity of a backlight 216 used for illuminating the display 208 and a controller 202 that optimizes the visual image corresponding to the intensity of the backlight 216 are described. As the intensity of the backlight 216 is reduced, the brightness of the pixels 210 is increased to compensate the image when, for example, the backlight intensity is reduced to save power. The method and apparatus are also described for compensating for uneven backlight 216 conditions.

TECHNICAL FIELD

This invention relates in general to electronic devices with displays, and more specifically to a method and apparatus for image optimization in backlit displays.

BACKGROUND

Battery life and the corresponding operating time between battery recharges in electronic devices is a key success factor for acceptance in the marketplace. The power consumed by a display is a critical element in overall power consumption in an electronic device. This is especially true with the trend to larger displays. Moreover, the addition of color displays significantly increases the need for power management in these devices. In most lighting situations, a backlight is required by a color liquid crystal display (“LCD”) to achieve the highest image quality. A color LCD display without sufficient background light is often perceived as washed out and flat. A bright backlight, however, is a significant drain on an electronic device. The level of the backlight can be reduced to improve battery life but can result in reduced readability and clarity.

DETAILED DESCRIPTION

In overview, the present disclosure concerns electronic devices that use LCD displays, particularly color LCD displays. More particularly, various inventive concepts and principles embodied in the methods and apparatus for use in optimizing the viewable image on a color LCD when a backlight is adjusted are discussed. This is of particular interest in portable electronic devices where a user has had to choose between a bright, high contrast image with a short operating time and a harder-to-view image with a longer operating time. The operating time being driven at least in part by backlight power consumption.

As further discussed below various inventive principles are advantageously employed to optimize the displayed image when the backlight levels are adjusted, particularly to prolong battery life. When the displayed image becomes washed out, dim or flat-looking, users of electronic devices will often choose to keep the backlight on and accept the reduced battery life, often to their detriment should the electronic device become unusable later. When incorporated in an electronic device, the instant disclosure provides for compensating the image display to encourage users to operate with lower backlight levels and preserve the battery, correspondingly lengthening the operating time.

The instant disclosure is provided to further explain in an enabling fashion the best modes of making and using various embodiments in accordance with the present invention. The disclosure is further offered to enhance an understanding and appreciation for the inventive principles and advantages thereof, rather than to limit in any manner the invention. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.

It is further understood that the use of relational terms, if any, such as first and second, top and bottom, and the like are used solely to distinguish one from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.

FIG. 1illustrates a typical electronic device100with a color display102. The user is often able to adjust or select a level of backlight via one or more elements of a user interface. In other cases, the level of the backlight is adjusted automatically based on one or more conditions such as inactivity, modes of operation or ambient light level.

Cellular phones, personal digital assistants, organizers, personal games, and portable entertainment systems incorporating displays that would benefit from an embodiment of the present disclosure are in common use and are readily available at retail establishments. In most cases this is a color LCD display but other configurations including grayscale or other color display technologies can be envisioned.

Referring toFIG. 2, a simplified and representative block diagram of a display apparatus200for use in an electronic device is discussed and described. The electronic device may be a wireless communication device, a personal digital assistant or any other of a number of electronic devices that use LCD displays and could benefit from reduced power consumption. A controller202has an input204and an output206. The output206may be a multiplexed set of connections for driving a display208. The display208most often includes individual pixels210forming a pixel array. A processor212having an output214may be used to drive a backlight216for illuminating the display208. The processor212may be coupled to the controller via a connection218for communicating the level or intensity of the backlight216and for accessing pixel data. Depending on the physical configuration of the display apparatus and the specific electronic device, one or more additional backlights220may be employed also driven by the processor212. The processor and controller may be disposed physically or logically in a single device222as depicted.

In operation, a display apparatus200for use in an electronic device100may have a display208for presenting a visual image, a processor212for determining an intensity of a backlight216for illuminating the display208, and a controller202coupled to the display208and the processor212for optimizing the visual image corresponding to an intensity of the backlight216. The level of the backlight216may be selected by the user via some form of the user interface or may be adjusted automatically by the processor228based on another factor. Such factors may include ambient light, time since a key press, remaining battery power, or others. For example, in some conditions the backlight level may need to be at its highest intensity resulting in a sacrifice of some battery life for the user to be able to see the display well. There may be a case when the electronic device100is equipped with a light sensor and could sense that given a dark ambient light condition, such as at night, the backlight can be reduced to a predetermined low level. In another instance, perhaps indoors, the user may want to reduce the level of the backlight to save power and might be willing to sacrifice some quality of the displayed image.

The nature of a color display, especially an LCD display208is that the screen is divided into small segments called pixels210. A pixel210can vary in size depending on the resolution of the display and the type of construction. The hue, saturation, and brightness of each pixel210are most often determined according to red, green and blue (“RGB”) sub-element settings for the pixel210. In some embodiments other color spaces can be used such as cyan, magenta, yellow, however, the principles discussed regarding RGB apply equally. By adjusting the RGB settings for a pixel210not only the hue and saturation, but the value or brightness can be set. The visual image can be optimized by adjusting the red, green and blue settings for each of the pixels210in the array. One such measure of hue, saturation, and brightness is defined by Smith, A. R. in the SIGRAPH 78 paper titled “Color Gamut Transform Pairs.” Mapping between hue, saturation, and value (or brightness, or lightness) can be found in “Computer Graphics” by Foley, et al. Measures of hue and saturation are defined in HSV (Hue Saturation Value), HSB (Hue Saturation Brightness), HLS (Hue Lightness Saturation) color spaces know by practitioners of ordinary skill in the art. Keeping a constant scaling between red, green, and blue settings when making the adjustment allows the value, brightness, lightness, or luminance of a pixel210to be changed without changing the hue and saturation of the pixel210. This can be accomplished by scaling the original settings by the same scale factor or by increasing each value by the same percentage increase. Since the procedures, methods, and apparatus are the same for scaling red, green, and blue settings to adjust the value, brightness, lightness, or luminance, while maintaining hue and saturation, the terms brightness will be used to describe the pixel value, brightness, lightness, or luminance henceforth. The human eye may be considered when working with colors as the eye is more sensitive to changes in brightness than changes in color. Therefore, a strict ratio adjustment may not always be used when changing brightness and maintaining hue and saturation.

Applying this characteristic of an LCD display208, an image can be optimized by increasing the brightness of a pixel210as the backlight216level is decreased. In other words, the visual image may be optimized by adjusting the brightness of the pixel inversely proportionally to the intensity of the backlight216. The integrity of the image is preserved by maintaining the hue and saturation of each pixel of the pixel array.

In one example where a pixel210may already have a relatively high brightness and the backlight216is reduced in intensity, one or more of the red, green, blue settings may require adjustment beyond a maximum setting. For the sake of illustration, let us assume that the red, green and blues settings for a pixel210can range from 0–255. In the exemplary instance, red is at 200, green is at 150 and blue is at 100, resulting in a medium brown hue. A 15% decrease in backlight216intensity (as measured by current driving the light, a light sensor, or other mechanism) is followed by a corresponding increase in pixel210brightness, for example 20%, as may be empirically derived as having the best effect. A 20% increase results in new red, green, and blue values of 240, 180, and 120, respectively, resulting in a lighter shade of brown, providing higher brightness while still maintaining the hue. However, when a 35% decrease in backlight216intensity occurs, and the pixel210value is adjusted up a corresponding 35%, for example, the resulting red value of 270 would exceed the maximum. In an exemplary embodiment, the red value may be set to 255, establishing an increase ratio of (255–200)/200 of 27.5%. The remaining green and blue values are each increased by 27.5% giving final settings for the pixel210(rounded to a whole number) of 255, 191 and 128 respective to red, green and blue. This is still a brown hue and has the highest brightness available while maintaining that hue. That is, the red, green and blue settings are adjusted inversely proportionally to the intensity of the backlight until the maximum setting would exceed a limit value. Then, that setting is set to a maximum value and the percentage increase of that setting is used to increase the remaining settings. The ability to calculate that a pixel has reached a limit value at a given backlight level may be used to adjust the backlight level to point at or near impending brightness saturation, giving a mix of full color intensity and backlight level.

This use of ratio adjustments and limit values may also apply to a dark hue (low red, green, blue settings) when the backlight intensity is increased. A corresponding reduction in a hue or brightness may result in an effective loss of any hue (black) if the color elements are all reduced equally. Again, a limit value may apply so that the luminance is not lowered to a point where contrast is lost and any image presented is simply black. In this case a lower limit, for example 50, may be set. When any value would be reduced below 50 when adjusting to a brighter backlight, the value most below 50 would be set to 50 and the percentage decrease of that setting would be used to decrease the value of the remaining setting.

Referring briefly toFIG. 4, As shown by the line402, representing backlight216intensity across a horizontal section of the display, the light supplied by the backlight diminishes with increased distance from the light source. This can be due simply to the geometry of the placement of the backlight216with respect to the display208or due to the optics used for channeling the light from the backlight216to the display208. It can be seen that given the single light source example using backlight216that the display will be brighter on the left side of the screen than on the right. One way to describe this effect is luminance roll off. The ability to adjust the luminance (brightness) of the display208on a pixel-by-pixel basis allows the designer of the electronic device to compensate for the light intensity difference without more expensive optics or additional backlights220. In the example, the brightness of pixels on the right side of the display208can be adjusted to more closely match those on the left side of the display208. When a second backlight is used, such as220, or when different optics are employed, the lighting pattern on the display may be altered, as shown by the graph line404illustrating the effect of the second backlight. In the two backlight216220case, the pixels in the center of the display208can be adjusted to match those on each edge according to the luminance roll off shown by lines402and404. While the sample inFIG. 4shows a linear luminance roll off, it will be evident to one of ordinary skill in the art that such a change in backlight intensity may be highly non-linear both across the width and the height of the display208and corresponding changes to the image optimization algorithm may be required.

Returning toFIG. 2, the controller202may use any of several methods to calculate a new value for a pixel210in the pixel array when a change in backlight216level occurs. In one exemplary embodiment, the controller may optimize the visual image by adjusting all pixels in the pixel array by a fixed value according to a look up table. As an example to be used strictly for illustrative purposes, a table may describe that for a reduction in backlight216level of about >0% to about 15% an increase in pixel brightness of 8%–10% will be applied. Backlight216reductions from about >15% to about 25% can result in an 18%–20% increase in pixel luminance. Any such adjustment can be made while maintaining hue and saturation while accounting for maximum values as described above. In another embodiment, each pixel210can be adjusted by calculating in real time a new value for that pixel210based on the change in backlight216intensity, and may or may not include compensation for backlight display variation as described above. In yet another embodiment, the pixel array may be broken into portions and a new adjustment value for that portion calculated and applied to the pixels210therein.

In one embodiment, for example when displaying live video, the processor212may be called upon to enhance the image in real time. In another embodiment, for example when display208changes are not as rapid, the controller202may be driven from a front memory while the contents for a next display screen are processed for image enhancement in a back memory. The processor212can read out sections of the back memory, for example rectangular areas of the display, process the image, and rewrite the data to the back memory. When all pixel230values have been updated the processor212can signal the controller202to switch from the front to the back memory to change the displayed image. Essentially, the back memory is made the front memory, the former front memory is available for writing new display data and for image optimization. The front and back display memory are often part of the controller202but may be separate. In one embodiment, access to the display memory is made using OpenGL™ software calls. OpenGL™ is a trademark of Silicon Graphics, Inc.

The components shown inFIG. 2are known and available. The controller202is or may include a digital signal processor or another controller available from Motorola, Inc. or other manufacturers. Similarly, the processor212may be or may include a digital signal processor from Motorola, Inc. or other manufacturers. The processor212and controller202may functionally operate on the same chip and be expressed in software or hardware. Whether implemented in hardware (such as in a programmable logic array) or software (in C++, Java or other computer operable instructions) implementation of the functions described for the processor212and controller202are easily understood and implemented by a practitioner of average experience and capability in the field. In one embodiment the display controller may be an ATI Imageon™ 3200 display controller used with a National Semiconductor LM2791 LED driver, although other combinations can be made at the discretion of one of ordinary skill in the art. Imageon™ is a trademark of ATI Technologies, Inc. The backlights216220may be cold fluorescent lights (CFL), electroluminescent lights (EL), light emitting diode (LED) or other device or combination thereof. They are commodity parts and are available from manufacturers such as Kyocera and component distributors such as Arrow Electronics or Hamilton Avnet. A number of displays208are in production and available from manufacturers such as Kyocera, Hitachi or others.

Similar to the above, a display controller222provides an image optimized to a backlight intensity. The display controller has a first input204for receiving a data to display as the image and a second input228corresponding to a backlight intensity of a display. The display208being driven may be composed of pixels210in a pixel array. The display controller further comprises an output206for controlling one or more of the pixels210of the pixel array. The display controller also has a processor212or controller202for adjusting the brightness of the pixels210of the pixel array in response to changes in one or both of the inputs204228. One of the display controller222inputs, for example228, may correspond to the backlight intensity directly. Alternately, the display controller input228may be data related to, for instance, keyboard activity, and is used by the display controller222for programmatically setting the backlight intensity. Whether via monitoring the backlight intensity or controlling it, the display controller222is aware of the backlight intensity for use in optimizing the image.

The processor212or controller202, as part of the display controller222, may optimize the visual image based on the backlight intensity according to one of a fixed value look up and a real-time calculation.

The processor212or controller202may adjust the value for one of the red, green or blue pixel elements in inverse proportion to the backlight intensity to compensate for the different level of backlight available. The brightness of the pixel210can be maintained by adjusting each of the elements (red, green and blue) of the pixel in the same proportion.

When the adjustment in inverse proportion to the backlight level would cause one of the red, green, or blue element values to exceed a limit value, that setting is set to a maximum value and the percentage increase of that setting is used to increase the remaining settings. In a case where all values or values from several pixels exceed the limit, the value most exceeding the limit is set to the maximum and is used to determine the other adjusted values. This allows maintaining the hue and saturation of the pixel210while affording the maximum adjustment available.

The display controller may be programmed to allow separately calculating the adjustment in inverse proportion to the backlight level so that some portions of the display are adjusted to different levels, have different limits, or use a different proportion than other areas of the display. As described above, this may be used to compensate for displays where the backlighting is not uniform.

The display controller222may be any of a number of controllers, single chip processors or programmable arrays available from manufacturers such as Motorola, Inc. In one embodiment the display controller can an ATI Imageon™ 3200 Graphics controller. The functions described may be implemented in hardware, firmware or software and is easily accomplished by one of average skill in the art. The conversion of a design in one form to another form is known to those of ordinary skill in the art.

Referring toFIG. 3, a method for LCD display compensation when backlight adjusts are made is discussed and described. The method optimizes an image in a display of an electronic device when the backlight intensity changes by first determining300a backlight intensity, or more appropriately, a change in the backlight intensity. In some cases, the backlight intensity may be viewed as a difference from a known state, that is, for example, 50% below a maximum level. In another embodiment, the backlight intensity, and a change thereof, may be expressed in terms corresponding to the energy used to drive the backlight, for example, milliamps of drive current. Once the backlight intensity is determined, a brightness scale factor is calculated302, that is, a factor for adjusting the image according to the intensity of the backlight is determined. In one embodiment, a constant value brightness scale factor may be determined for all pixels210in the display by calculating a percentage change to apply based on a percentage change in the backlight intensity. In another embodiment, a constant value brightness scale factor may be determined for all pixels210according to a predetermined table of values that may be empirically derived for a particular display configuration and may have the benefit of speed over real-time calculation of a scaling factor. A second table of scale factors may also be calculated wherein the table contains an entry for each color component value, from 0 to a maximum. In yet another embodiment, variations in the backlight level at different areas of the screen may be accounted for. After a brightness scaling factor is determined, a second factor, based on the intensity of the backlight in a portion of the screen may be used for adjusting the image in that portion of the display. This is particularly applicable in situations where, due to optics or uneven distribution of the light from the backlight, some areas of the screen are always brighter than others. Adjusting the brightness according to both the change in backlight intensity and the change in observed intensity at a given location allows for creating a perceived uniformity across the display viewing area.

A further consideration in determining the brightness scaling factor is maintaining the hue and saturation of the image. In a display characterized by pixels210composed of red, green and blue primary colors care may be exercised to adjust each of the three elements proportionally to maintain the hue and saturation of the displayed image.

A limit value for the pixel elements or color components is determined304. The limit may be one imposed by the display itself, that is a maximum luminance or brightness supported, or it may be an empirical limit, such as a low value determined to be needed for color discrimination or observed color saturation. The limit may be different for different areas of the screen. As discussed above, the screen may be divided into sections for ease of calculation or to simplify compensation for backlight variations.

The color component values are tested308to see if they exceed a color component limit value. If any of the element or color component values exceed the limit, the yes branch of308is followed. When the magnitude of the one or more of the red, green or blue elements exceeds a limit value, the setting exceeding the limit by the greatest amount can be set310to the maximum value or another predetermined value. The settings for the remaining color component values are increased312by the same percentage increase as the highest original color component value. When determining a brightness value, that is, the magnitude of the red, green and blue pixel settings, in a limit situation, it is desirable to maintain the original hue and saturation of the pixel210but not necessary. Variations from strict proportional adjustments can be accommodated for speed of calculation, rounding errors, or table look up matching. The method continues at316.

When no setting exceeds a limit established for it, or when hue and saturation are not maintained, as when maximum values are reached and proportionality is not maintained, the no branch of308is followed to314. The new values for red, green, and blue settings are applied314to the image at the pixel210to adjust the image using the brightness scale factor. In practice, one embodiment reads display values from the controller202, operates on the image, either in whole or in parts, and then rewrites the optimized data back into the controller202. A test316is performed to determine if all pixels or sections of the display have been adjusted. If not, the no branch of316is followed and the process continues at306. If all processing is complete, the yes branch of316is followed and processing ends318.

In a preferred embodiment, the new values of the magnitude of the red, green and blue elements are calculated306in a relationship that is inversely proportional to the change in intensity of the display. If limit values are not checked, processing continues at314.

The apparatus and method discussed above, and the inventive principles thereof are intended to and will alleviate problems caused by changes in backlight intensity and variations across a display in backlight intensity. By making the display more readable and improving the overall appearance of the displayed image, a user will be more likely to operate the electronic device at a lower backlight level. Thus, the user will reap the benefits of longer battery life with an acceptable, if even perceptible, reduction in image quality of the display.

Further, beyond the direct benefit to the user in terms of longer battery life, the user will benefit from the perception of even backlighting across all areas of the screen, by the compensation of the brightness in pixels where the luminance rolls off. The benefit to the manufacturer is as apparent. The optics for use in backlight diffusion can be less expensive and the types of backlight sources employed may be broadened when the display apparatus200or display controller222is employed to provide uniformity of perceived image quality when variations in backlight luminance exist.

The display described in this illustration is a liquid crystal display but it is obvious to one of ordinary skill in the art that the technique described is applicable to other display types and configurations and for purposes other than those associated with power conservation. It is easily understood that plasma displays, conventional tube monitors and others are used in environments where the brightness is adjusted, for example, when room lighting changes. The ability to adjust the brightness while maintaining hue and possibly color saturation in those situations is advantageous and is a direct application of the methods and apparatus described herein.

One embodiment of implementing the method ofFIG. 3andFIG. 4follows:

Various embodiments of methods and apparatus for optimizing an image in an LCD display have been discussed and described. It is expected that these embodiments or others in accordance with the present invention will have application to many electronic devices that use backlit displays. The disclosure extends to the constituent elements or equipment comprising such electronic devices and specifically the methods employed thereby and therein.