An electronic device may include an electronic display to display images during frames based on image data. The electronic display may be divided into multiple regions each having multiple pixels. The electronic device may also include a display pipeline to process the image data and output the processed image data to the electronic display. The display pipeline may also determine a history update corresponding to an estimated burn-in aging effect of the pixels based on usage. A first portion of the history update corresponding to pixels in a first region may be determined during a first frame and a second portion of the history update corresponding to pixels in a second region may be determined during a second frame.

BACKGROUND

This disclosure relates to image data processing to identify and compensate for burn-in on an electronic display.

Numerous electronic devices—including televisions, portable phones, computers, wearable devices, vehicle dashboards, virtual-reality glasses, and more—display images on an electronic display. As electronic displays gain increasingly higher resolutions and dynamic ranges, they may also become increasingly more susceptible to image display artifacts due to pixel burn-in. Burn-in is a phenomenon whereby pixels degrade over time owing to the different amount of light that different pixels emit over time. In other words, pixels may age at different rates depending on their relative utilization. For example, pixels used more than others may age more quickly, and thus may gradually emit less light when given the same amount of driving current or voltage values. This may produce undesirable burn-in image artifacts on the electronic display.

SUMMARY

This disclosure relates to identifying and compensating for burn-in and/or aging artifacts on an electronic display. Burn-in is a phenomenon whereby pixels degrade over time owing to various factors, including the different amounts of light that different pixels may emit over time. For example, if certain pixels are used more frequently than others, or used in situations that are more likely cause undue aging, such as high temperature environments, those pixels may exhibit more aging than other pixels. As a result, those pixels may gradually emit less light when given the same driving current or voltage values, effectively becoming darker than the other pixels when given a signal for the same brightness level. As such, without compensation, burn-in artifacts may be visibly perceived due to non-uniform sub-pixel aging. To prevent this sub-pixel aging effect from causing undesirable image artifacts on the electronic display, circuitry and/or software may monitor and/or model the amount of burn-in that is likely to have occurred in the different pixels. Based on the monitored and/or modeled amount of burn-in that is determined to have occurred, the image data may be adjusted before it is sent to the electronic display to reduce or eliminate the appearance of burn-in artifacts on the electronic display.

In one example, circuitry and/or software may monitor or model a burn-in effect that would be likely to occur in the electronic display as a result of the image data that is sent to the electronic display. Additionally or alternatively, the circuitry and/or software may monitor and/or model a burn-in effect that would be likely to occur in the electronic display as a result of the temperature of different parts of the electronic display while the electronic display is operating. For instance, a pixel may age more rapidly by emitting a larger amount of light at a higher temperature and may age more slowly by emitting a smaller amount of light at a lower temperature.

By monitoring and/or modeling the amount of burn-in that has likely taken place in the electronic display, burn-in gain maps may be derived to compensate for the burn-in effects. Namely, the burn-in gain maps may gain down image data that will be sent to the less-aged pixels (which would otherwise appear brighter) without gaining down the image data that will be sent to the pixels with the greatest amount of aging (which would otherwise appear darker). In this way, the pixels of the electronic display that have suffered the greatest amount of aging will appear to be equally as bright as the pixels that have suffered the least amount of aging. As such, perceivable burn-in artifacts on the electronic display may be reduced or eliminated.

In some embodiments, the gain applied to the image data may be determined based on aging relationships between gray level, the average luminance output of the display, and/or the emission duty cycle of each pixel from previously obtained burn-in statistics and/or the current frame to be displayed. The emission duty cycle may be indicative of pulse-width modulation of the emission pulse used for a pixel to obtain a desired brightness. For example, below a threshold brightness, the voltage may be held constant, and the emission pulse-width modulated at a particular duty cycle to obtain darker luminance levels. Moreover, the effect of burn-in on a pixel may differ at different emission duty cycles. Additionally, in some embodiments, the emission duty cycle may change the burn-in aging rate of the pixel and/or the output luminance of the pixel.

Furthermore, the collection of burn-in statistics may be based on the gray level, the emission duty cycle of each pixel, the global brightness of the display, and/or the average brightness of the display. In some embodiments, the burn-in statistics may be downsampled for storage and/or computational efficiency. For example, the burn-in statistics may utilize a dynamic string (e.g., a string of 8 bits) that has a different interpretation depending on the emission duty cycle of the pixel. For example, the write out of the burn-in statistics to memory may represent different levels of burn-in for each pixel depending on the emission duty cycle of each pixel.

Additionally or alternatively, the burn-in statistics may be gathered on all of the display pixels, or a subset of the display pixels, depending on the active region. Moreover, the pixels within the active region may be split into multiple vertical segments and burn-in statistics may be gathered on each vertical segment during different periods of time to reduce the overall statistics gathered while maintaining comprehensive burn-in statistics for the display.

DETAILED DESCRIPTION

By monitoring and/or modeling an amount of burn-in that has likely taken place in the electronic display, burn-in gain maps may be derived to compensate for the burn-in effects. The burn-in gain maps may gain down image data that will be sent to the less-aged pixels (which would otherwise be brighter) without gaining down, or by gaining down less, the image data that will be sent to the pixels with the greatest amount of aging (which would otherwise be darker). In this way, the pixels of the electronic display that are likely to exhibit the greatest amount of aging will appear to be equally as bright as pixels with less aging. In this manner, perceivable burn-in artifacts on the electronic display may be reduced or eliminated.

To help illustrate, one embodiment of an electronic device10that utilizes an electronic display12is shown inFIG. 1. As will be described in more detail below, the electronic device10may be any suitable electronic device, such as a handheld electronic device, a tablet electronic device, a notebook computer, and the like. Thus, it should be noted thatFIG. 1is merely one example of a particular implementation and is intended to illustrate the types of components that may be present in the electronic device10.

In the depicted embodiment, the electronic device10includes the electronic display12, input devices14, input/output (I/O) ports16, a processor core complex18having one or more processors or processor cores, local memory20, a main memory storage device22, a network interface24, a power source26, and image processing circuitry27. The various components described inFIG. 1may include hardware elements (e.g., circuitry), software elements (e.g., a tangible, non-transitory computer-readable medium storing instructions), or a combination of both hardware and software elements. It should be noted that the various depicted components may be combined into fewer components or separated into additional components. For example, the local memory20and the main memory storage device22may be included in a single component. Additionally, the image processing circuitry27(e.g., a graphics processing unit, a display image processing pipeline) may be included in the processor core complex18.

As depicted, the processor core complex18is operably coupled with local memory20and the main memory storage device22. In some embodiments, the local memory20and/or the main memory storage device22may include tangible, non-transitory, computer-readable media that store instructions executable by the processor core complex18and/or data to be processed by the processor core complex18. For example, the local memory20may include random access memory (RAM) and the main memory storage device22may include read only memory (ROM), rewritable non-volatile memory such as flash memory, hard drives, optical discs, and/or the like.

In some embodiments, the processor core complex18may execute instructions stored in local memory20and/or the main memory storage device22to perform operations, such as generating source image data. As such, the processor core complex18may include one or more general purpose microprocessors, one or more application specific processors (ASICs), one or more field programmable logic arrays (FPGAs), or any combination thereof.

As depicted, the processor core complex18is also operably coupled with the network interface24. Using the network interface24, the electronic device10may be communicatively coupled to a network and/or other electronic devices. For example, the network interface24may connect the electronic device10to a personal area network (PAN), such as a Bluetooth network, a local area network (LAN), such as an 802.11x Wi-Fi network, and/or a wide area network (WAN), such as a 4G or LTE cellular network. In this manner, the network interface24may enable the electronic device10to transmit image data to a network and/or receive image data from the network.

Additionally, as depicted, the processor core complex18is operably coupled to the power source26. In some embodiments, the power source26may provide electrical power to operate the processor core complex18and/or other components in the electronic device10. Thus, the power source26may include any suitable source of energy, such as a rechargeable lithium polymer (Li-poly) battery and/or an alternating current (AC) power converter.

Furthermore, as depicted, the processor core complex18is operably coupled with the I/O ports16and the input devices14. In some embodiments, the I/O ports16may enable the electronic device10to interface with various other electronic devices. Additionally, in some embodiments, the input devices14may enable a user to interact with the electronic device10. For example, the input devices14may include buttons, keyboards, mice, trackpads, and the like. Additionally or alternatively, the electronic display12may include touch sensing components that enable user inputs to the electronic device10by detecting occurrence and/or position of an object touching its screen (e.g., surface of the electronic display12).

In addition to enabling user inputs, the electronic display12may facilitate providing visual representations of information by displaying one or more images (e.g., image frames or pictures). For example, the electronic display12may display a graphical user interface (GUI) of an operating system, an application interface, text, a still image, or video content. To facilitate displaying images, the electronic display12may include a display panel with one or more display pixels. Additionally, each display pixel may include one or more sub-pixels, which each control luminance of one color component (e.g., red, blue, or green).

As described above, the electronic display12may display an image by controlling luminance of the sub-pixels based at least in part on corresponding image data (e.g., image pixel image data and/or display pixel image data). In some embodiments, the image data may be received from another electronic device, for example, via the network interface24and/or the I/O ports16. Additionally or alternatively, the image data may be generated by the processor core complex18and/or the image processing circuitry27.

As described above, the electronic device10may be any suitable electronic device. To help illustrate, one example of a suitable electronic device10, specifically a handheld device10A, is shown inFIG. 2. In some embodiments, the handheld device10A may be a portable phone, a media player, a personal data organizer, a handheld game platform, and/or the like. For example, the handheld device10A may be a smart phone, such as any iPhone® model available from Apple Inc.

As depicted, the handheld device10A includes an enclosure28(e.g., housing). In some embodiments, the enclosure28may protect interior components from physical damage and/or shield them from electromagnetic interference. Additionally, as depicted, the enclosure28surrounds the electronic display12. In the depicted embodiment, the electronic display12is displaying a graphical user interface (GUI)30having an array of icons32. By way of example, when an icon32is selected either by an input device14or a touch-sensing component of the electronic display12, an application program may launch.

Furthermore, as depicted, input devices14open through the enclosure28. As described above, the input devices14may enable a user to interact with the handheld device10A. For example, the input devices14may enable the user to activate or deactivate the handheld device10A, navigate a user interface to a home screen, navigate a user interface to a user-configurable application screen, activate a voice-recognition feature, provide volume control, and/or toggle between vibrate and ring modes. As depicted, the I/O ports16also open through the enclosure28. In some embodiments, the I/O ports16may include, for example, an audio jack to connect to external devices.

To further illustrate, another example of a suitable electronic device10, specifically a tablet device10B, is shown inFIG. 3. For illustrative purposes, the tablet device10B may be any iPad® model available from Apple Inc. A further example of a suitable electronic device10, specifically a computer10C, is shown inFIG. 4. For illustrative purposes, the computer10C may be any MacBook® or iMac® model available from Apple Inc. Another example of a suitable electronic device10, specifically a watch10D, is shown inFIG. 5. For illustrative purposes, the watch10D may be any Apple Watch® model available from Apple Inc. As depicted, the tablet device10B, the computer10C, and the watch10D each also includes an electronic display12, input devices14, I/O ports16, and an enclosure28.

As described above, the electronic display12may display images based at least in part on image data received, for example, from the processor core complex18and/or the image processing circuitry27. Additionally, as described above, the image data may be processed before being used to display a corresponding image on the electronic display12. In some embodiments, a display pipeline may process the image data, for example, to identify and/or compensate for burn-in and/or aging artifacts.

To help illustrate, a portion34of the electronic device10including a display pipeline36is shown inFIG. 6. In some embodiments, the display pipeline36may be implemented by circuitry in the electronic device10, circuitry in the electronic display12, or a combination thereof. For example, the display pipeline36may be included in the processor core complex18, the image processing circuitry27, a timing controller (TCON) in the electronic display12, or any combination thereof.

As depicted, the portion34of the electronic device10also includes an image data source38, a display panel40, and a controller42. In some embodiments, the display panel40of the electronic display12may be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic LED (OLED) display, or any other suitable type of display panel40. In some embodiments, the controller42may control operation of the display pipeline36, the image data source38, and/or the display panel40. To facilitate controlling operation, the controller42may include a controller processor44and/or controller memory46. In some embodiments, the controller processor44may be included in the processor core complex18, the image processing circuitry27, a timing controller in the electronic display12, a separate processing module, or any combination thereof and execute instructions stored in the controller memory46. Additionally, in some embodiments, the controller memory46may be included in the local memory20, the main memory storage device22, a separate tangible, non-transitory, computer readable medium, or any combination thereof.

In the depicted embodiment, the display pipeline36is communicatively coupled to the image data source38. In this manner, the display pipeline36may receive source image data48corresponding with an image to be displayed on the electronic display12from the image data source38. The source image data48may indicate target characteristics (e.g., pixel data) corresponding to a desired image using any suitable source format, such as an 8-bit fixed point αRGB format, a 10-bit fixed point αRGB format, a signed 16-bit floating point αRGB format, an 8-bit fixed point YCbCr format, a 10-bit fixed point YCbCr format, a 12-bit fixed point YCbCr format, and/or the like. In some embodiments, the image data source38may be included in the processor core complex18, the image processing circuitry27, or a combination thereof. Furthermore, the source image data48may reside in a linear color space, a gamma-corrected color space, or any other suitable color space. As used herein, pixels or pixel data may refer to a grouping of sub-pixels (e.g., individual color component pixels such as red, green, and blue) or the sub-pixels themselves.

As described above, the display pipeline36may operate to process source image data48received from the image data source38. The display pipeline36may include one or more image data processing blocks (e.g., circuitry, modules, or processing stages) such as the burn-in compensation (BIC)/burn-in statistics (BIS) block50. As should be appreciated, multiple other image data processing blocks may also be incorporated into the display pipeline36, such as a color management block, a dither block, etc. Further, the functions (e.g., operations) performed by the display pipeline36may be divided between various image data processing blocks, and while the term “block” is used herein, there may or may not be a logical separation between the image data processing blocks.

The BIC/BIS block50may compensate for burn-in to reduce or eliminate the visual effects of burn-in, as well as to collect image statistics about the degree to which burn-in is expected to have occurred on the electronic display12. As such, the BIC/BIS block50may receive input pixel values52representative of each of the color components of source image data48and output compensated pixel values54. As stated above, other image data processing blocks may also be utilized in the display pipeline36. As such, the input pixel values52and/or the compensated pixel values54may be processed by other image data processing blocks before and/or after the BIC/BIS block50. Moreover, the resulting display image data56output by the display pipeline36for display on the display panel40may suffer substantially fewer or no burn-in artifacts.

After processing, the display pipeline36may output the display image data56to the display panel40. Based at least in part on the display image data56, the display panel40may apply analog electrical signals to the display pixels of the electronic display12to display one or more corresponding images. In this manner, the display pipeline36may facilitate providing visual representations of information on the electronic display12.

To help illustrate, an example of a process58for operating the display pipeline36is described inFIG. 7. Generally, the process58may include receiving source image data48from the image data source38or from another block of the image data processing blocks (process block60). The display pipeline may also perform burn-in compensation (BIC) and/or collect burn-in statistics (BIS) (process block62), for example, via the BIC/BIS block50. The display pipeline may then output the display image data56, which is compensated for burn-in effects (process block64). In some embodiments, the process58may be implemented based on circuit connections formed in the display pipeline36. Additionally or alternatively, in some embodiments, the process58may be implemented in whole or in part by executing instructions stored in a tangible non-transitory computer-readable medium, such as the controller memory46, using processing circuitry, such as the controller processor44.

As shown inFIG. 8, the BIC/BIS block50may encompass burn-in compensation (BIC) processing74and burn-in statistics (BIS) collection processing76. The BIC processing74may receive the input pixel values52and output the compensated pixel values54adjusted for non-uniform pixel aging of the electronic display12. Additionally, the BIS collection processing76may analyze all or a portion of the compensated pixel values54to generate a burn-in statistics (BIS) history update78indicative of an incremental update representing an increased amount of pixel aging that is estimated to have occurred since a corresponding previous BIS history update78. Although the BIC processing74and the BIS collection processing76are shown as components of the display pipeline36, the BIS history update78may be output for use by the controller42or other data processing hardware or software (e.g., an operating system, application program, or firmware of the electronic device10). The controller42or other software may use the BIS history update78in a compute gain maps block80to generate gain maps82. The gain maps82may be two-dimensional (2D) maps of per-color-component pixel gains. For example, the gain maps82may be programmed into 2D lookup tables (LUTs) in the display pipeline36for use by the BIC processing74.

The controller42or other software (e.g., an operating system, application program, or firmware of the electronic device10) may also include a compute gain parameters block84to generate gain parameters86that may be provided to the display pipeline36for use by the BIC processing74. For example, the gain parameters86may include a normalization factor and a brightness adaptation factor, which may vary depending on the global display brightness, the gray level of the pixel, the emission duty cycle of the pixel, and/or the color component of image data to which the gain parameters86are applied (e.g., red, green, or blue), as discussed further below. As should be appreciated, the gain parameters86discussed herein are non-limiting, and additional parameters may also be included in determining the compensated pixel values54such as floating or fixed reference values and/or parameters representative of the type of electronic display panel40. As such, the gain parameters86may represent any suitable parameters that the BIC processing74may use to appropriately adjust the values of and/or apply the gain maps82to compensate for burn-in.

A closer view of the BIC processing74is shown inFIG. 9. The BIC processing74may include an up-sampling block88, a brightness adaptation block90, and/or an apply gain block92. The up-sampling block88may receive and up-sample the gain maps82to spatially support the resolution of the pixel grid (e.g., the pixels of the display panel40) and provide the per-component pixel gain value to the apply gain block92. The brightness adaptation block90may receive the input pixel values52and generate the brightness adaptation factor based on a global brightness (e.g., an average luminance output, a total luminance output, any suitable luminance measure associated with the entire frame, and/or a brightness setting indicative of or associated with the luminance output) of the display panel40and/or the emission duty cycle of the individual pixels and provide it to the apply gain block92. In some embodiments, the per-component pixel gain values may be indicative of red, green, or blue color components, for example, when the electronic display12has red, green, and blue colored sub-pixels, but may include other color components if the electronic display12has subpixels of other colors (e.g., white subpixels in an RGBW display). Furthermore, the input pixel values52may include location data indicative of the spatial location of the pixel on the electronic display12.

In some embodiments, the up-sampling block88may allow the BIC processing74to use gain maps82that are sized to have a lower resolution than the size of the electronic display12. For example, when the gain maps82have a lower resolution format, the up-sampling block88may up-sample values of the gain maps82(e.g., on a per-pixel or per-region basis). Several example operations of the up-sampling block88will be described further below with reference toFIGS. 11 and 12.

The pixel gain values of the gain map82may have any suitable format and precision. For example, the precision of the pixel gain value may be between 8 and 12 bits per component, and may vary by configuration. In one embodiment, the alignment of the most significant bit (MSb) of a pixel gain value may be configurable through a right-shift parameter, which may vary (e.g., between 0 and 7) based on implementation. For example, a right-shift parameter value of 0 may represent alignment with the first bit after the decimal point. For a right-shift parameter value of 2, the MSb of the gain value may be aligned to the fourth bit after the decimal point, effectively yielding a gain with precision between u0.11 and u0.15 precision, corresponding, for example, to a fetched value with 8 to 12 bits of precision.

The apply gain block92may receive input pixel values52for a given location on the electronic display12, a per-component pixel gain value (e.g., derived from the gain maps82, which may be up-sampled by the up-sampling block88), and/or the brightness adaptation factor. The apply gain block92may apply the per-component pixel gain value to the input pixel values52for each sub-pixel according to the gain parameters86(e.g., the normalization factor and the brightness adaptation factor). In some embodiments, the apply gain block92may generate a compensation value to be applied (e.g., added or multiplied) to an input pixel value52to obtain a compensated pixel value54. For example, the compensation value for a given sub-pixel may be determined based on the per-component pixel gain value from the fetched and/or up-sampled gain maps82, the brightness adaptation factor, and/or the normalization factor. Moreover, in some embodiments, the compensation value may be proportional to the per-component pixel gain value from the fetched and/or up-sampled gain maps82, the brightness adaptation factor, and/or the normalization factor with or without an offset such as the normalization factor. When applied, the brightness adaptation factor may, at least partially, compensate the input pixel values52for the emission duty cycle and/or the brightness of the current frame. Moreover, in some embodiments, the normalization factor may normalize the luminance output of the pixels with respect to one or more of the pixels with the most burn-in with respect to the maximum gain for each color component. The compensation value may be encoded in any suitable way, and, in some embodiments, may be clipped.

As stated above, the brightness adaptation factor may take any suitable form, and may take into account the global brightness setting of the electronic display12and/or the emission duty cycle of the pixel of interest. The emission duty cycle may be indicative of pulse-width modulation of current to the pixel to obtain a desired brightness. For example, above a threshold brightness, the brightness of the pixel may be adjusted by a voltage supplied to the pixel. However, below a threshold brightness, the voltage may be held constant, and the emission pulse-width modulated at a particular duty cycle to obtain luminance levels below the threshold brightness. The effect of burn-in on a pixel may differ at different emission duty cycles. As such, the brightness adaptation factor and/or the normalization factor may employ the emission duty cycle to assist in compensating for burn-in.

In one embodiment, the brightness adaptation block90may scale the input pixel values52by a luminance normalizer and derive the brightness adaptation factor via a lookup table (LUT) based on the scaled (e.g., via the luminance normalizer) pixel values. In some embodiments, the scaling luminance normalizer may be proportional and/or inversely proportional to the emission duty cycle of the pixel and/or the global brightness of the display panel40for the current frame. Moreover, in some embodiments, the luminance normalizer may be proportional to the global brightness normalized by a reference brightness. Moreover, the reference brightness, may be a fixed or floating reference value based on the luminance output of the pixels. As should be appreciated, the brightness adaptation factor may be obtained via a LUT, by computation, or any suitable method accounting for the global brightness setting of the electronic display12and/or the emission duty cycle of the pixel of interest.

In further illustration, an example process94for determining the brightness adaptation factor is described inFIG. 10. The brightness adaptation block90may receive the input pixel values52for each color component of each pixel (process block96). Additionally, the global brightness and/or emission duty cycle may be determined (process block98). The global brightness and/or the emission duty cycle may be used to determine the luminance normalizer (process block100). Further, the input pixel values52may be scaled by the luminance normalizer (process block102), and the scaled pixel values may be used to determine the brightness adaptation factor (process block104), for example, via a lookup table (LUT).

Additionally, in some embodiments, the normalization factor may also be a function of the luminance normalizer. The normalization factor may be calculated on a per-component basis and may take into account a maximum gain across all channels. In other words, the normalization factor may compensate for an estimated pixel burn-in of the most burnt-in pixel with respect to the maximum gain of each color component. For example, in some embodiments, the normalization factor may assign a gain of 1.0 to the pixel(s) determined to have the most burn-in and a gain of less than 1.0 to the pixel(s) that are less likely to exhibit burn-in effects.

The normalization factor may be encoded in any suitable way, and in some cases, the normalization factor may be encoded in the same format as the brightness adaptation factor. As mentioned above, the gain parameters86may include the normalization factor and the brightness adaptation factor. Furthermore, the gain parameters86may be updated and provided to the apply gain block92at any suitable frequency. For example, in some embodiments, the normalization factor and the brightness adaptation factor may be updated every frame or some multiple of frames and/or every time the global brightness settings change. In some scenarios, the normalization factor and/or the brightness adaptation factor may be updated less often (e.g., once every other frame, once every 5 frames, once per second, once per 2 seconds, once per 5 seconds, once per 30 seconds, once per minute, or the like).

FIGS. 11 and 12describe the up-sampling block88to extract the per-component pixel gain value from the gain maps82. The gain maps82may be full resolution per-sub-pixel two-dimensional (2D) gain maps or may be spatially downsampled, for example, to save memory and/or computational resources. When the dimensions of the gain maps82are less than the full resolution of the electronic display12, the up-sampling block may up-sample the gain maps82to obtain the per-component pixel gain values discussed above. In some embodiments, the gain maps82may be stored as a multi-plane frame buffer. For example, when the electronic display12has three color components (e.g., red, green, and blue), the gain maps82may be stored as a 3-plane frame buffer. When the electronic display has some other number of color components (e.g., a 4-component display with red, green, blue, and white sub-pixels, or a 1-component monochrome display with only gray sub-pixels), the gain maps82may be stored with the corresponding number of planes.

Each plane of the gain maps82may be the full spatial resolution of the electronic display12, or may be spatially downsampled by some factor (e.g., downsampled by some factor greater than 1, such as 1.5, 2, 3.5, 5, 7.5, 8, or more). Moreover, the amount of spatial downsampling may vary independently by dimension, and the dimensions of each of the planes of the gain maps82may differ. By way of example, a first color component (e.g., red) plane of the gain maps82may be spatially downsampled by a factor of 2 in both dimensions (e.g., in both x and y dimensions), a second color component (e.g., green) plane of the gain maps82may be spatially downsampled by a factor of 2 in one dimension (e.g., the x dimension) and downsampled by a factor of 4 in the other dimension (e.g., the y dimension), and a third color component (e.g., blue) plane of the gain maps82may be spatially downsampled by a factor of 4 in both dimensions (e.g., in both x and y dimensions). Further, in some examples, planes of the gain maps82may be downsampled to variable extents across the full resolution of the electronic display12.

One example plane of the gain maps82appears inFIG. 11, and represents a downsampled mapping with variably reduced dimensions, and thus has been expanded to show the placement across a total input frame height106and an input frame width108of the electronic display12of the various gain values110. Moreover, the plane of the gain maps82may have gain values110that are spaced unevenly, but as noted above, other planes of gain maps82may be spaced evenly.

Whether the gain values110are spaced evenly or unevenly across the x and y dimensions, the up-sampling block88may perform interpolation to obtain gain values for sub-pixels at (x, y) locations that are between the points of the gain values110. Bilinear interpolation and nearest-neighbor interpolation methods will be discussed below. However, any suitable form of interpolation may be used.

In the example ofFIG. 11, an interpolation region112of the plane of the gain maps82contains the four closest gain values110A,110B,110C, and110D to a current sub-pixel location114when the current interpolation region112the plane of the gain maps82has been downsampled by a factor2in both dimensions in this region. The size of the plane and/or of the interpolation region(s) of the gain maps82may be determined based on the active interpolation region, panel type, interpolation mode, phase and spatial sub-sampling factor for each color component and/or region.

The up-sampling block88may perform spatial interpolation of the fetched plane of the gain maps82. Moreover, in some embodiments, a spatial shift of the plane of the gain maps82, when down-sampled with respect to the pixel grid of the electronic display12, may be supported through a configurable initial interpolation phase in each of the x and y dimensions (e.g., the initial value for sx and/or sy inFIG. 11). In some embodiments, when a plane or an interpolation region of the gain maps82is spatially down-sampled, sufficient gain value data points may be present for the subsequent up-sampling to happen without additional samples at the edges of the plane of the gain maps82. As such, bilinear and/or nearest neighbor interpolation may be supported. Moreover, the up-sampling factor and interpolation method may be configurable separately for each of the color components.

In some cases, planes may be horizontally or vertically sub-sampled due to the panel layout. For example, some electronic displays12may support pixel groupings of less than every component of pixels, such as a GRGB panel with a pair of red and green and pair of blue and green pixels. In an example such as this, each red/blue component may be up-sampled by replication across a gain pair, as illustrated inFIG. 12. In the example ofFIG. 12, an even gain pixel group116includes a red gain118and a green gain120, and an odd gain pixel group122includes a green gain124and a blue gain126. The output gain pair may thus include an even gain pixel group128that includes the red gain118, the green gain120, and the blue gain126, and an odd gain pixel group130that includes the red gain118, the green gain120, and the blue gain126.

As discussed above with reference toFIG. 8, the controller42or other software (e.g., an operating system, application program, or firmware of the electronic device10) may use burn-in statistics (BIS) to generate the gain maps82. The gain maps82are used to lower the maximum brightness for pixels that have not experienced as much aging, and, therefore, match other pixels that have experienced more aging. The gain maps82compensate for non-uniform aging effects and thereby aid in reducing or eliminating perceivable burn-in artifacts on the electronic display12.

Furthermore, the total amount of luminance emitted by a pixel, as well as the environmental conditions (e.g., temperature) during emission, over its lifetime may have a substantial impact on the aging of that pixel. As such, the BIS collection processing76of the BIC/BIS block50may monitor and/or model a burn-in effect that would be likely to occur on the pixels of the electronic display12based on the image data sent to the electronic display12and/or the temperature of the electronic display12. One or both of these factors (e.g., image data and temperature) may be considered by the BIS collection processing76in generating a BIS history update132, as depicted inFIG. 13. The BIS history update132may be provided to the controller42or other data processing hardware or software to keep track of the usage history (e.g., history of luminance output) of the pixels and/or the environmental conditions of the pixel and to generate the gain maps82therefrom. In one embodiment, the BIS collection processing76may determine a luminance aging factor134from a burn-in aging block136or other computational structure and a temperature adaptation factor138from a temperature adaptation block140or other computational structure. The luminance aging factor134and the temperature adaptation factor138may be combined in a multiplier142and downsampled by a downsampling block144to generate the BIS history update132. Additionally, although the BIS history update132is shown as having 8 bits per component (bpc), as should be appreciated, the BIS history update132may utilize any suitable bit depth.

The burn-in aging block136may combine multiple gain parameters86to estimate the impact of burn-in on the pixels and obtain the luminance aging factor134. For example, the burn-in aging block136may determine the luminance aging factor134based on the compensated pixel values54, the emission duty cycle, the global brightness, and/or a measure of the average pixel luminance (APL) of the current frame or previous frame. In one embodiment, the burn-in aging block136may determine the impact of the pixel gray level and the impact of the average pixel luminance and combine the two according to respective weights to determine the net burn-in impact.

Indeed, in one embodiment, the impact of the pixel gray level may be determined based on the agglomeration of the emission duty cycle, the global brightness of the display, the compensated pixel values54per color component, and/or one or more reference brightnesses. For example, the impact of the pixel gray level may be determined by scaling the compensated pixel values54by the global brightness normalized to a reference brightness and/or the inverse of the emission duty cycle. Furthermore, the impact of the pixel gray level may include an exponential factor that may vary per color component. As should be appreciated, the reference brightness, may be fixed or floating and, furthermore, may be based on the luminance output of the pixels. In one embodiment, the reference brightness may change between frames based on the emission duty cycle and the global brightness.

Furthermore, in one embodiment, the impact of the average pixel luminance may be determined based on the agglomeration of the emission duty cycle, the global brightness of the display, the compensated pixel values54per color component, a parameter characterizing the infrared (IR) drop of the display panel40, the average pixel luminance of the current and/or previous frame, and/or a reference average pixel luminance. In some embodiments, the compensated pixel values54may be scaled by the APL. The scaling may be countered by the reference average pixel luminance and/or further scaled by the IR drop parameter, global brightness, and/or emission duty cycle and/or an inverse thereof. Furthermore, the impact of the pixel gray level may include one or more constant offsets and/or an exponential factor that may vary per color component. In some embodiments, it may be desirable to use the average pixel luminance of the previous frame, for example due to timings between computations. However, as should be appreciated, the APL of the current frame may also be used in computing the impact of the average pixel luminance on pixel aging.

In some embodiments, the net burn-in impact may be the product or addition of the impact of the pixel gray level and the impact of the average pixel luminance. As such, the net burn-in impact may be based on the compensated pixel values54, the global brightness of the display panel40, the emission duty cycle of the pixels, the average pixel luminance of the current frame, and/or the average pixel luminance of a previous frame. Furthermore, the net burn-in impact may be used to determine the luminance aging factor134. For example, in some embodiments, the net burn-in impact may be fed into a luminance aging lookup table (LUT)146. The luminance aging LUT146may be independent per color component and, as such, indexed by color component. Any suitable interpolation between the entries of the luminance aging LUT146may be used, such as linear interpolation between LUT entries. The luminance aging LUT146may output the luminance aging factor134, which may be taken into account to model the amount of aging on each of the pixels and/or sub-pixels of the electronic display12.

Non-uniform pixel aging may also be affected by the temperature of the electronic display12while the pixels of the electronic display12are emitting light. Indeed, temperature can vary across the electronic display12due to the presence of components such as the processor core complex18and other heat-producing circuits at various positions behind the electronic display12.

To accurately determine an estimate of the local temperature on the electronic display12, a two-dimensional (2D) grid of temperatures148may be used. An example of such a 2D grid of temperatures148is shown inFIG. 14and will be discussed in greater detail below. Continuing withFIG. 13, a pick tile block150may select a particular region (e.g., tile) of the 2D grid of temperatures148from the (x, y) coordinates of the currently selected pixel. The pick tile block150may also use grid points in the x dimension (grid_points_x), grid points in the y dimension (grid_points_y), grid point steps in the x direction (grid_step_x), and grid point steps in the y direction (grid_step_y). These values may be adjusted, as discussed further below. A current pixel temperature value txymay be selected from the resulting region of the 2D grid of temperatures148via an interpolation block152, which may take into account the (x, y) coordinates of the currently selected sub-pixel and values of a grid step increment in the x dimension (grid_step_x[idx]) and a grid step increment in the y dimension (grid_step_y[idy]). The current pixel temperature value t, may be used by the temperature adaptation block140to produce the temperature adaptation factor138, which indicates an amount of aging of the current pixel is likely to have occurred as a result of the current temperature of the current pixel. Additionally, in some embodiments, the current pixel temperature value txymay be fed into a temperature lookup table (LUT)154to obtain the temperature adaptation factor138.

An example of the two-dimensional (2D) grid of temperatures148appears inFIG. 14. The 2D grid of temperatures148illustrates the placement across a total input frame height156and an input frame width158of the electronic display12of the various current temperature grid values160. The current temperature grid values160may be populated using any suitable measurement (e.g., temperature sensors) or modeling (e.g., an expected temperature value due to the current usage of various electronic components of the electronic device10). An interpolation region162represents a region of the 2D grid of temperatures148that bounds a current spatial location (x, y) of a current pixel. A current pixel temperature value txymay be found at an interpolated point163. The interpolation may take place according to bilinear interpolation, nearest-neighbor interpolation, or any other suitable form of interpolation.

In one example, the two-dimensional (2D) grid of temperatures148may split the frame into separate regions (a region may be represented a rectangular area with a non-edge grid point at the center), or equivalently, 17×17 tiles (a tile may be represented as the rectangular area defined by four neighboring grid points, as shown in the interpolation region162), is defined for the electronic display12. Thus, the 2D grid of temperatures148may be determined according to any suitable experimentation or modeling for the electronic display12. The 2D grid of temperatures148may be defined for an entirety of the electronic display12, as opposed to just the current active region. This may allow the temperature estimation updates to run independently of the BIS/BIC updates. Moreover, the 2D grid of temperatures148may have uneven distributions of temperature grid values160, allowing for higher resolution in areas of the electronic display12that are expected to have greater temperature variation (e.g., due to a larger number of distinct electronic components behind the electronic display12that could independently emit heat at different times due to variable use).

To accommodate for finer resolution at various positions, the 2D grid of temperatures148may be non-uniformly spaced. Two independent multi-entry1D vectors (one for each dimension), grid_points_x and grid_points_y, are described in this disclosure to represent the temperature grid values160. In the example ofFIG. 14, there are 18 temperature grid values160in each dimension. However, any suitable number of temperature grid values160may be used. In addition, while these are shown to be equal in number in both dimensions, some 2D grids of temperatures148may have different numbers of temperature grid values160per dimension. The interpolation region162shows a rectangle of temperature grid values160A,160B,160C, and160D. The temperature grid values160may be represented in any suitable format, such as unsigned 8-bit, unsigned 9-bit, unsigned 10-bit, unsigned 11-bit, unsigned 12-bit, unsigned 13-bit, unsigned 14-bit, unsigned 15-bit, unsigned 16-bit, or the like. A value such as unsigned 13-bit notation may allow be implemented in a display panel40with a dimension of 8191 pixels.

Moreover, each tile (e.g., as shown in the interpolation region162) may start at a temperature grid value160and may end one pixel prior to the next temperature grid value160. Hence, for uniform handling in hardware, in some embodiments, at least one temperature grid value160(e.g., the last one) may be located a minimum of one pixel outside the frame dimension. Not all of the temperature grid values160may be used in all cases. For example, if a whole frame dimension of 512×512 is to be used as a single tile, grid_points_x[0] and grid_points_y[0] may each be programmed to512. Spacing between successive temperature grid values160may include a minimum number of pixels (e.g., 8, 16, 24, 48, and so forth) and some maximum number of pixels (e.g., 512, 1024, 2048, 4096, and so forth). The temperature grid values160may have any suitable format.

Returning again toFIG. 13, the BIS history update132may involve the multiplication or other integration of the luminance aging factor134and the temperature adaptation factor138in conjunction with the emission duty cycle. For example, the multiplier142may combine the luminance aging factor and the temperature adaptation factor138and the emission duty cycle to generate a pre-downsampled history update. The downsampling block144may receive the pre-downsampled history update and generate the BIS history update132. As discussed above, the BIS history update132may be of any suitable format.

The downsampling block144may help reduce the throughput of and usage of resources (e.g., processor bandwidth, memory, etc.) involved in storing and/or utilizing the BIS history update132. For example, the downsampling block may reduce the BIS history update132to an 8-bit string, or other suitable format of suitable bit-depth. In one embodiment, the BIS history update may be written out as three independent planes with the base addresses for each plane being byte aligned (e.g., 128-byte aligned). However, prior to write-out of the BIS history update132(e.g., updating the overall BIS with the BIS history update132), the number of components per pixel may be down-sampled from3to2, for example as illustrated inFIG. 15. Some electronic displays12may support pixel groupings of less than every component of pixels, such as a GRGB panel with a pair of red and green and pair of blue and green pixels. In an example such as this, each pair of pixels may have the red/blue components dropped to form a history update pair. In the example ofFIG. 15, an even history update pixel group164includes a red history update value166, a green history update value168, and a blue history update value170, and an odd history update pixel group172includes a red history update value174, a green history update value176, and a blue history update value178. To down-sample this pair, the output history update pair may, thus, include an even history update pixel group180that includes the red history update value166and the green history update value168, and an odd history update pixel group182that includes the green history update value184and the blue history update value186.

Additionally or alternatively, in one embodiment, the BIS history update132may include a dynamic string format (e.g., 8-bits) to accurately represent a higher bit depth string (e.g., 10-bit, 12-bit, and so on). The dynamic string format may allow for the single string of bits to have multiple different meanings. For example, the dynamic string may represent different amounts of burn-in for a pixel depending on the emission duty cycle of the pixel during the frame. Moreover, in some embodiments, the information about the emission duty cycle of the pixel may be stored within the BIS history update132, for example, as multiplied with the luminance aging factor and the temperature adaptation factor at the multiplier142.

In some embodiments, the BIS history update132may be determined for each frame of input pixel values52sent to the display panel40. In some implementations, however, it may not be practical to sample every frame. For example, resources such as electrical power, processing bandwidth, and/or memory allotment may vary depending on the electronic display12. As such, in some embodiments, the BIS history update132may be determined periodically in time or by frame. For example, the BIS history update132may be determined at a rate of 1 Hz, 10 Hz, 60 Hz, 120 Hz, and so on. Additionally or alternatively, the BIS history update132may be determined once every other frame, every 10thframe, every 60thframe, every 120thframe, etc. Furthermore, the write out rate of the BIS history update132may be dependent upon the refresh rate of the electronic display12, which may also vary depending on the source image data48. As such, the write out rate of the BIS history update132may be determined based on the bandwidth of the electronic device10or the electronic display12, and may be reduced to accommodate the available processing bandwidth.

Additionally or alternatively, in some embodiments, BIS collection may be spread out over multiple frames by determining a BIS history update132for a portion of each frame. For example,FIG. 16illustrates a display panel40divided into four regions. In one embodiment, a BIS history update132may be determined for a first region188during a first frame, a second region190during a second frame, a third region192during a third frame, and a fourth region194during a fourth frame. By spreading out the BIS history updates132over multiple frames, the write out of the BIS history update132may utilize a reduced amount of bandwidth (e.g., data processing or transfer over time). As such, the write out rate of the BIS history update132may be maintained or increased, while still remaining within the bandwidth capabilities of the electronic display12. Furthermore, in some embodiments, the BIS history update132of each region may written out individually or be stored in a buffer until each region has been stored, and the entire buffer may be written out at once.

Moreover, in some embodiments, by spreading out the BIS history updates132over multiple frames and utilizing a reduced amount of bandwidth, a smaller amount of buffer memory may be used to write out the BIS history update132. As such, the buffer size and/or the number of buffers used may be reduced. In one embodiment, a single buffer with a size corresponding to the size of the first region188may hold the BIS history update132for the pixels at pixel locations in the first region188during the first frame. Subsequently, the BIS history update132for the first region188may be written out (e.g., to memory) and the BIS history update132for the second region190may be held in the same memory buffer. As such, a single memory buffer may be reused for BIS history updates132for each region188,190,192,194and have a size large enough to accommodate the BIS history update132for a single entire region. Additionally or alternatively, each region188,190,192,194may have a separate buffer large enough for the corresponding region188,190,192,194.

As should be appreciated, the display panel40may be divided into any suitable number of regions. For example, the number of regions may be determined based on the size (e.g., width and/or height) of the display panel40, a processing speed, and/or a desired bandwidth to remain within. The regions may also be of any suitable shape (e.g., rectangular, polygonal, etc.), and may be of approximately the same size or of different sizes. In one embodiment, the regions may be described as non-overlapping vertical stripes dividing the display panel40.

The use of vertical regions may assist in processing efficiency, for example, in conjunction with the use of raster scan image data storage/transmission. In one embodiment, vertical regions may facilitate a stride196separating memory locations of the horizontal beginning of lines for a particular region (e.g., region190). In other words, the stride196may allow memory locations of other regions (e.g., regions188,192, and194) to be skipped to allow quick access of the region of interest (e.g., region190). The stride196may correspond to the width of the regions and may assist in determining a BIS history update132for each region. For example, the pixel locations may be offset by a factor of the stride196to conveniently identify the pixels of a region of interest. For example, a first line of a region190, beginning at a first memory location195, may be accessed to determine a BIS history update132. Subsequently, a second line of the region190, beginning at a second memory location197, may be accessed by adding the stride196to the first memory location195to continue determining the BIS history update132for the region190without cycling through memory locations of other regions (e.g., regions188,192, and194). Such a process may be repeated for each region188,190,192, and194. The stride196may be of any suitable size (e.g., corresponding to the width of the regions), and, in some embodiments, may be byte aligned (e.g., 128-byte aligned). Furthermore, the stride196may be used to identify the buffer size to retain the BIS history update132for a region188,190,192,194. For example, the buffer size may be based on the stride196multiplied by the height of the frame (e.g., the pixel height of the display panel40).

Additionally, dividing the display panel40into multiple regions may also assist in generating a BIS history update132for pixels in an active region198of the display panel40, while ignoring pixels in a non-active region200(e.g., pixels that are effectively off and/or are desired to be excluded from a BIS history update132), as illustrated inFIG. 17. In some scenarios, the source image data48may not contain input pixel values52for each pixel location of the display panel40. For example, letterboxes or borders may be implemented as non-active regions200of the display panel40such that the pixels in the non-active regions200are off or given a defined value (e.g., a constant value or a value that forms part of a visual texture such as a gradient, which may allow the BIS to be determined based on the known defined value). Additionally, in some embodiments, the display panel40may have a notch202. The notch202may be a portion of the display panel40without pixels, but may still be included in the pixel grid (e.g., having pixel coordinates corresponding to the input pixel values52). As such, due to the constant and/or negligible aging of pixels in the non-active regions200or the lack of physical pixels in the notch202, the BIS corresponding to pixels in the non-active region200or the notch202may be superfluous and, thus, not included in the BIS history update132.

On the other hand, BIS corresponding to pixels in the active region198may be included into the BIS history update132. Additionally, by using a stride196and dividing the display panel40into multiple regions, the active region198may be more flexibly identified and segmented such that the BIS history updates are more efficiently populated with BIS corresponding to pixels of the active region198. As shown by example inFIG. 17, the display panel40may be divided into multiple regions such that the first region188and the fourth region194are non-active regions200and the second region190and the third region192are part of the active region198. As such, the BIS history updates132may be more efficiently gathered based on pixels in the active region198while not gathering BIS for pixel values in non-active regions200or the notch202. Furthermore, in some embodiments, portions204above or below the active region198and within the second region190and the third region192may be included or not included in the BIS history update132depending on implementation. Additionally or alternatively, the display panel40may be divided into multiple regions, and a BIS history update132may be generated for the regions that contain at least a portion of the active region198and no BIS may be calculated for regions that do not contain a portion of the active region198.

FIG. 18is a flow diagram of an example process206for collecting BIS history updates132for the display panel40divided into one or more regions. The process206may include determining the division of the display panel40into multiple regions and determining the stride196associated with the division (process block208). Additionally, in some embodiments, the active region198may be determined (process block210). As should be appreciated, depending on implementation, the active region198may be determined before or after the division of the display panel40into regions. For example, the regions may be determined based in part on the active region198. The regions or portions of regions to be incorporated into a BIS history update132may also be determined (process block212). During a first frame, the BIS history update132for a first region (e.g., region188,190,192, or194) may be determined (process block214). Additionally, during a second frame, subsequent to the first frame, the BIS history update132for a second region (e.g., region188,190,192, or194) may be determined (process block216). The BIS history update132may also be determined for additional regions as desired. The regions may be processed for BIS in any desired order. In one embodiment, the regions incorporated into the BIS may be processed from left to right, relative to the display panel40, for example by processing the first region188, then the second region190, then the third region192, and so on. Furthermore, the frames in which each region's BIS history update132is determined may be immediately subsequent or may have frames in between. Once the BIS history update132for each desired region has been determined, the BIS history updates132may be written out (process block218). As should be appreciated, the write out of the BIS history updates132may be done in bulk (e.g., for all of the entire regions) or individually (e.g., as the BIS history update132is determined for each region).

By compiling and storing the values of the BIS history update132, the controller42or other software may determine a cumulative amount of non-uniform pixel aging across the electronic display12. This may allow the gain maps82to be determined that may counteract the effects of the non-uniform pixel aging. By applying the gains of the gain maps82to the input pixels before they are provided to the electronic display12, burn-in artifacts that might have otherwise appeared on the electronic display12may be reduced or eliminated in advance. Thereby, the burn-in compensation (BIC) and/or burn-in statistics (BIS) of this disclosure may provide a vastly improved user experience while efficiently using resources of the electronic device10.