Decimated burn-in compensation with pixel shifting

A flat-panel display device and method to prevent display panel burn-in through a decimated look-up table with pixel shifting in a display or an augmented reality display.

BACKGROUND

Field

Aspects of the disclosure relate in general to displays. Aspects include a method and device to prevent display panel burn-in through a decimated look-up table with pixel shifting in a flat-panel display or an augmented reality display.

Description of the Related Art

Displays are electronic viewing technologies used to enable people to see content, such as still images, moving images, text, or other visual material.

A flat-panel display includes a display panel including a plurality of pixels arranged in a matrix format. The display panel includes a plurality of scan lines formed in a row direction (y-axis) and a plurality of data lines formed in a column direction (x-axis). The plurality of scan lines and the plurality of data lines are arranged to cross each other. Each pixel is driven by a scan signal and a data signal supplied from its corresponding scan line and data line.

Flat-panel displays can be classified as passive matrix type light emitting display devices or active matrix type light emitting display devices. Active matrix panels selectively light every unit pixel. Active matrix panels are used due to their resolution, contrast, and operation speed characteristics.

One type of active matrix display is an active matrix organic light emitting diode (AMOLED) display. The active matrix organic light emitting display produces an image by causing a current to flow to an organic light emitting diode to produce light. The organic light emitting diode is a light-emitting element in a pixel. The driving thin film transistor (TFT) of each pixel causes a current to flow in accordance with the gradation of image data.

Flat-panel displays are used in many portable devices such as laptops and mobile phones.

Screen burn-in, image burn-in or ghost image, is a discoloration of areas on a display caused by cumulative non-uniform use of the pixels.

In organic light emitting diode (OLED) displays the wide variation in luminance degradation will cause noticeable color drift over time where one of the red-green-blue (RGB) colors becomes more prominent due to luminance degradation of the light-emitting pixels.

In the case of liquid crystal displays (LCDs), the mechanics of burn-in are different. For LCDs, burn-in develops in some cases because pixels permanently lose their ability to return to their relaxed state after a continued static use profile. In most typical usage profiles, this image persistence in LCD is only transient.

In desktop computer applications, “screensaver” software actively attempts to stave off screen burn. By ensuring that no pixel or group of pixels was left displaying a static image for extended periods of time, phosphor luminosity was preserved. Modern screensavers can turn off the screen when not in use.

SUMMARY

Embodiments include an electronic display designed to prevent display panel burn-in through a decimated look-up table with pixel shifting in a flat-panel display or an augmented reality display.

In one embodiment, an electronic display comprises a display panel and a decimated burn-in compensator. The decimated burn-in compensator is configured to receive an image frame and output a compensated output frame to the display panel. The decimated burn-in compensator further comprises a pixel shifter, a downsampler, an N×N bin compensating look up table, an interpolator, and a multiplier. The pixel shifter is configured to receive the image frame and to shift the image frame by a predetermined number of pixels resulting in a shifted image frame. The downsampler is configured to receive the image frame and to downsample time image frame into a downsampled image frame. The N×N bin compensating look up table is configured to receive the downsampled image frame and compensate the downsampled image frame into a compensated image frame. The interpolator is configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame. The multiplier is configured to combine the shifted image frame and the interpolated image frame, resulting in the compensated output frame. The display panel is further configured to display the compensated output frame.

In an augmented reality embodiment, image data comes from two sources—a real-world image component captured by a digital camera, and a virtual image component generated by a graphics processing unit (GPU). An augmented reality display comprises a display panel, a graphics processing unit, a camera, a blend unit, and a decimated burn-in compensator. The graphics processing unit is configured to generate a virtual image frame. The camera is configured to capture a real-world image frame. The blend unit is configured to receive the virtual image frame from the graphics processing unit, to receive the real-world image frame from the camera, and to combine the virtual image frame with the real-world image frame to produce a combined image frame. The decimated burn-in compensator is configured to receive the combined image frame and output a compensated output frame to the display panel. The decimated burn-in compensator further comprises a pixel shifter, a downsampler, an N×N bin compensating look up table, an interpolator, and a multiplier. The pixel shifter is configured to receive the combined image frame and to shift the image frame by a predetermined number of pixels resulting in a shifted image frame. The downsampler is configured to receive the image frame and to downsample time image frame into a downsampled image frame. The N×N bin compensating look up table is configured to receive the downsampled image frame and to compensate the downsampled image frame into a compensated image frame. N is an integer greater than one. The interpolator is configured to receive the compensated image frame and interpolate the compensated image frame into an interpolated image frame. The multiplier is configured to combine the shifted image frame and the interpolated image frame resulting in the compensated output frame. The display panel is further configured to display the compensated output frame.

DETAILED DESCRIPTION

One aspect of the disclosure is the realization that a number of burn-in compensation techniques are sub-optimal. While flat-panel display burn-in can be addressed by per-pixel tracking and compensation, the use of per-pixel tracking results in a large storage and reading look-up-table (LUT). The large amount of dynamic read only memory (DRAM) or static read only memory (SRAM) required increases semiconductor die-area and fabrication cost. Moreover, the use of a large amount of DRAM causes excess power consumption. In turn, higher power consumption results in lower battery life in portable devices, such as tablet computers, mobile phones, virtual reality headset or augmented reality glassesm and augmented reality display smartphones, digital “smart” watches, and other digital devices.

An aspect of the disclosure includes the observation that using an N×N pixel block instead of per-pixel tracking reduces compensation memory band-width and power requirements and saves the look-up-table footprint. This solution reduces the memory and die-size requirements—which decreases manufacturing costs.

In order to better appreciate the features and aspects of the present disclosure, further context for the disclosure is provided in the following section by an implementation of a flat-panel display that prevents display panel burn-in through a decimated look-up table with pixel shifting in the flat-panel display according to embodiments of the disclosure. Alternate embodiments show how such a flat-panel display might be implemented for use in an augmented reality display. These embodiments are for explanatory purposes only and other embodiments may be employed in other display devices. For example, embodiments of the disclosure can be used with any display device that compensates and prevents display panel burn-in through a decimated look-up table with pixel shifting in the flat-panel display.

FIG. 1is a block diagram of a flat-panel display100embodiment with a decimated burn-in compensator1000designed to prevent display panel burn-in through a decimated compensating look-up table with pixel shifting in a display panel200in accordance with an embodiment of the present disclosure. Flat-panel display100may be a stand-alone display, or part of: a computer display, television set, notebook computer, tablet computer, mobile phone, smartphone, augmented reality display, digital “smart” watch, or other digital device. Decimated burn-in compensator1000is configured to receive an image frame and output a compensated frame which prevents the display panel burn-in. Essentially, the flat-panel display100receives input from two data streams combined by a multiplier: a pixel-shifted image from the frame buffer, and an image passed through a N×N compensating lookup table.

In this embodiment, a flat-panel display100has a decimated burn-in compensator1000and a display panel200.

The display panel200may be an organic light-emitting diode (OLED) display, such as a passive-matrix (PMOLED) or active-matrix (AMOLED). In other embodiments, the display panel200may be a liquid crystal display (LCD) or micro-light emitting diode (micro-LED) display. The display panel200displays an image received from a decimated burn-in compensator1000.

Decimated burn-in compensator1000uses a combination of pixel shifting and compensating for an N×N pixel block, where N is a decimation factor represented by an integer greater than one. For example, in some embodiments N is two. Accordingly, the decimated burn-in compensator1000comprises a pixel shifter1200and an N×N bin compensating look up table1600. Embodiments of the decimated burn-in compensator1000may also comprise a frame buffer1100, pixel history accumulation1400, a down sampler1500, an interpolator1700, and a multiplier1300.

Frame buffer1100is a portion of random access memory (RAM) containing bitmap image frame data to drive a display panel200. For the purposes of this embodiment, frame buffer1100stores at least one image frame of data. Frame buffer1100may receive an image frame of data from an external graphics card or driver (not shown), and forwards the image frame to pixel shifter1200.

Pixel shifter1200receives the image frame from frame buffer1100and periodically (vertically and/or horizontally) shifts the image by a predetermined number of pixels. In some embodiments, the image frame is shifted in a circle in a defined rhythm and pixel interval. Pixel shifter1200may shift the image frame imperceptibly to a viewer of display panel200.

When pixels within an N×N pixel block have substantially different content history, there is a danger for over-compensating less burnt-in pixels, and under compensating for other pixels. By periodically shifting content, pixel shifter1200smooths out stress level differences within the N×N pixel block.

Images may be shifted by a maximum of 1, 2, 4, 8, 12, or 16 pixels. The image shift may occur in l×1 or 2×2 pixels shift per step. In other embodiments, images may be shifted by 1-16 pixels. The number of pixels shifted may be related to the N×N pixel block decimated compensation. For example, in one embodiment, a 1-pixel shift may be used with an 8×8 pixel block. In effect, pixel shifting distributes burn-in content stress, and is equivalent to applying low pass filtering to content before burn-in stress (BIS). Larger binning results in a smoother gradient of a low pass filter, and smaller error at a sharp image edge. A longer range of image shifting achieves a good smoothing effect.

Pixel history accumulation1400is a computer memory that stores a history of previously displayed image frames. Pixel history accumulation1400may be a Random Access Memory (RAM), flash memory, and the like. In some embodiments, pixel history accumulation1400includes images from frame buffer1100.

Down sampler1500receives a previously displayed image frame from pixel history accumulation1400and decimates the image to reduce its size. The decimation process reduces the image into N×N pixel bins, where N is a decimation factor represented by an integer number greater than or equal to two (2). The reduction in size by N2significantly saves on the size of a look-up table used by the decimated burn-in compensator1000, and reduces the amount of memory used as well.

N×N bin compensating look up table1600allows decimated burn-in compensator1000to apply characteristic curves to an image frame received from down sampler1500. In this application, N×N bin compensating look up table1600assign an output value to every possible N×N input value, which allows correcting color space calculations to performed quickly and thus preventing burn-in. In the N×N bin compensating look up table1600, N is a decimation factor represented by an integer greater than one. It is understood by one skilled in the art that N×N bin compensating look up table1600would facilitate image contrast, brightness changes, grey value spreading, individual gradation tables or enhancing image gamma. Once the image frame is compensated by the N×N compensating lookup table1600, a compensated image is produced and sent to interpolator1700.

Interpolator1700resizes the compensated image received from N×N compensating lookup table1600into an upscaled image that matches the native resolution of display panel200. The upscaled image is forwarded to multiplier1300which acts as a compositing block to combine the upscaled image with the shifted image from pixel shifter1200. The resulting image is output by the decimated burn-in compensator1000to display panel200.

We now turn toFIGS. 2-3, which each depict alternate embodiments of a flat-panel display used in an augmented reality application. Each of the alternate embodiments includes a decimated burn-in compensator1000designed to prevent display panel burn-in. In augmented reality embodiments, image data comes from two sources—a real-world image component captured by a digital camera, and a virtual image component generated by a graphics processing unit (GPU). The two sources are combined by a blend unit, and eventually projected on a display panel. Depending upon the implementation of a particular embodiment, a decimated burn-in compensator may be used to compensate each of the two image data sources, or the combined image data.

It is understood by one skilled in the art that the embodiments depicted inFIGS. 2-3may include one or more image frame buffers, which are not shown.

FIG. 2depicts a flat-panel display2000with a single decimated burn-in compensator1000designed to compensate a combined augmented reality image to prevent burn-in at a display panel200, in accordance with an embodiment of the present disclosure.

A virtual image component is comprised of a plurality of virtual image frames generated by a graphics processing unit2100. Graphics processing unit2100may be an application specific integrated circuit (ASIC) designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer. In other embodiments, the graphics processing unit2100may be embedded on a central processing unit (CPU) die or a motherboard of the flat-panel display2000. Once generated by the graphics processing unit2100, the virtual image frame is forwarded to blend unit2400to be combined with a real-world image frame.

A real-world image component is comprised of a plurality of real-world image frames captures by a camera2200. Camera2200may be any optical capture system with a digital image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductors (CMOS). Once captured by the camera2200, the real-world image frame is forwarded to image signal processor (ISP)2300.

Image signal processor2300is a digital signal processor (DSP) integrated circuit specialized for image processing data from camera2200. In particular, image signal processor2300may perform the Bayer transformation or demosaicing to enable color accuracy, noise reduction, image sharpening or interpolation to size the captured image received from camera2200. Image signal processor2300may implement single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) techniques to enable parallel processing of image data. The resulting imaged processed real-world image frame is forwarded to blend unit2400to be combined with the virtual image frame.

Blend unit2400is a specialized image processing unit to combine a virtual image frame with a real-world image frame. Blend unit2400may treat the virtual image frame and the real-world image frame as two layers to be blended together. In such an embodiment, the virtual image frame is treated as a “top layer” or “active layer” image to be superimposed on top of a real-world “bottom layer” image. The resulting blended image is an augmented reality image forwarded to lens distortion corrector2500.

Lens distortion corrector2500is an image processor used to correct for radial (optical) distortion caused by a lens in the camera2200. In some embodiments, lens distortion corrector2500is implemented as a software or firmware correction that may use the Brown-Conrady distortion model to correct for radial distortion and for tangential image distortion. Once distortion corrected, the corrected image is sent to decimated burn-in compensator1000. The decimated burn-in compensator1000prevents display panel burn-in through a decimated compensating look-up table with pixel shifting, as described inFIG. 1. The resulting image is forwarded on to a pixel pipeline2600.

In the art, pixel pipeline2600is sometimes also referred to as a computer graphics pipeline, rendering pipeline or graphics pipeline. Pixel pipeline2600renders a three-dimensional scene on to a two-dimensional display screen. It is understood by one skilled in the art, that the rendering for this operation depends upon the software and hardware used, and the characteristics of display panel200. Typically, pixel pipeline2600performs a real-time rendering implemented in hardware. The resulting image can then be displayed at display panel200.

In an alternate implementation,FIG. 3depicts a flat-panel display3000used in an augmented reality embodiment, with two decimated burn-in compensators1000a-bdesigned to separately compensate a real-world image component and a virtual image component before the images are combined and displayed at a display panel200.

The virtual image component is comprised of a plurality of virtual image frames generated by a graphics processing unit3100. Graphics processing unit3100may be an application specific integrated circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer. In other embodiments, the graphics processing unit3100may be embedded on a central processing unit die or a motherboard of the flat-panel display200. Once generated by the graphics processing unit3100, the virtual image frame is forwarded to a first decimated burn-in compensator1000a. The first decimated burn-in compensator1000aprevents display panel burn-in through a decimated compensating look-up table with pixel shifting, as described inFIG. 1. The resulting image is forwarded on to blend unit3400to be combined with a real-world image frame.

The real-world image component is comprised of a plurality of real-world image frames captures by a camera3200. Camera3200may be any optical capture system with a digital image sensor, such as a charge-coupled device or complementary metal-oxide-semiconductors. Once captured by the camera3200, the real-world image frame is forwarded to image signal processor3300.

Image signal processor3300is a digital signal processor integrated circuit specialized for image processing data from camera3200. In particular, image signal processor3300may perform demosaicing or the Bayer transformation to enable color accuracy, noise reduction, image sharpening or interpolation to size the captured image received from camera3200. Image signal processor3300may enable parallel processing of image data through single instruction, multiple data (SIMD) or multiple instruction, multiple data (MIMD) techniques. The resulting imaged processed real-world image frame is forwarded to tlens distortion corrector3500.

The lens distortion corrector3500is an image processor used to correct for radial (optical) distortion caused by a lens in the camera3200. In some embodiments, lens distortion corrector2500is implemented as a software or firmware correction that may use the Brown-Conrady distortion model to correct for radial distortion and for tangential image distortion. Once distortion corrected, the corrected image is sent to a second decimated burn-in compensator1000b.

The second decimated burn-in compensator1000bprevents display panel burn-in through a decimated compensating look-up table with pixel shifting, as described inFIG. 1. It is understood by one skilled in the art that the second decimated burn-in compensator1000bdoes not have to shift an image the same number of pixels as the first decimated burn-in compensator1000a, and may use a different N×N bin (designated “M×M,” where M is greater than one) for image compensation. In other embodiments, the first decimated burn-in compensator1000aand the second decimated burn-in compensator1000bmay shift an image the same number of pixels, and may use the same N×N bin compensating look up table1600. The resulting image is forwarded on to blend unit3400to be combined with the virtual image frame.

A specialized image processor, blend unit3400, combines the virtual image frame with the real-world image frame. Blend unit3400may treat the virtual image frame and the real-world image frame as two layers to be blended together. In such an embodiment, the virtual image frame is treated as a “top layer” or “active layer” image to be superimposed on top of a real-world “bottom layer” image. The resulting blended image is an augmented reality image forwarded to a pixel pipeline3600.

In the art, pixel pipeline3600is sometimes also referred to as a computer graphics pipeline, rendering pipeline or graphics pipeline. Pixel pipeline3600renders a three-dimensional scene on to a two-dimensional display screen. It is understood by one skilled in the art, that the rendering for this operation depends upon the software and hardware used, and the characteristics of display panel200. Typically, pixel pipeline3600performs a real-time rendering implemented in hardware. The resulting image can then be displayed at display panel200.

It is understood by those familiar with the art that the system described herein may be implemented in a variety of hardware or firmware solutions.