COLOR ENHANCEMENT ALGORITHM FOR COLOR-DEFICIENT PEOPLE

An electronic device includes a display and processing circuitry that is communicatively coupled to the display. The processing circuitry is configured to receive source image data indicative of color components for a pixel of the source image data. The color components include a maximum color component, a middle color component, and a minimum color component. The processing circuitry is also configured to determine a classification for the pixel based at least in part on the color component and to generate adjusted image data by modifying one or more of the color components based at least in part on the classification.

BACKGROUND

The present disclosure generally relates to image processing, and, more particularly, to techniques for modifying image data to generate content that, when displayed, is perceivable by people with regular vision and those that suffer from color vision deficiency (e.g., colorblindness).

Electronic devices often use one or more electronic displays to present visual representations of information, for example, as text, still images, and/or video based on corresponding image data. Some users may perceive image content different than others. For example, approximately eight percent of men and less than one percent of women suffer from color vision deficiency, which is also known as colorblindness. For users with color vision deficiency, it may be difficult to perceive which colors are present in displayed content or to discern between the colors of the display content.

SUMMARY

The present disclosure generally relates to processing techniques that may be utilized when performing image processing. For example, the techniques described herein may be utilized as part of a process for altering image data to enhance the visibility of images (e.g., content shown on a display) for users with color vision deficiency, such as colorblindness.

In particular, the techniques described herein relate to modifying image data to generate image data that, when displayed as image content, enables users with color vision deficiency to discern, or better discern, between colors in the image content. These techniques may be applied in a user-specific manner so that each user may alter settings for how image data is modified so as to generate image data that best suits the user. For example, image content for a pixel in a display may have color components (e.g., RGB values) that define the amount of red, green, and blue to be displayed at the pixel. Based on which of these colors is the largest color component, which of these colors is the middle color component, and which of these colors is the minimum color component, original image data may be modified to generate image data that, when displayed, better enables a user with color vision deficiency to discern between colors (e.g., compared to an image generated from the original or unmodified image content). For instance, as described below, the middle color component, minimum color component, or both the middle color component and the minimum color component may be modified to enable users with color vision deficiency to better distinguish between colors in image content.

DETAILED DESCRIPTION

The present disclosure describes techniques for displaying content in a manner that is more viewable to users with color vision deficiency, which can also be referred to as colorblindness. In particular, a user may set certain settings, and image data may be modified in a user-specific manner based on the settings. When displayed, color(s) in the displayed content are relatively more discernable to the user, for example, compared to other techniques or algorithms used to adapt image data for those with color vision deficiency.

With the foregoing in mind, an electronic device10(e.g., computing device) that may utilize an electronic display12to display image frames based on image data and/or an image sensor13(e.g., a camera) to capture image data is described inFIG.1. As will be described in more detail below, the electronic device10may be any suitable computing device, electronic device, such as a computer, a mobile phone, a portable media device, a tablet, a television, a virtual-reality headset, a vehicle dashboard, and the like. Thus, it should be noted thatFIG.1is merely one example of a particular implementation and is intended to illustrate the types of components that may be present in the electronic device10.

The electronic device10includes the electronic display12, an image sensor13, one or more input structures14(e.g., input devices), one or more input/output (I/O) ports16, a processor core complex18having one or more processor(s) or processor cores, image pre-processing circuitry, local memory20, a main memory storage device22, a network interface24, a power source26, and image processing circuitry28. The various components described inFIG.1may include hardware elements (e.g., circuitry), software elements (e.g., a tangible, non-transitory computer-readable medium storing instructions), or a combination of both hardware and software elements. It should be noted that the various depicted components may be combined into fewer components or separated into additional components. For example, the local memory20and the main memory storage device22may be included in a single component.

The electronic display12may be any suitable electronic display. For example, the electronic display12may include a self-emissive pixel array having an array of one or more of self-emissive pixels. The electronic display12may include any suitable circuitry to drive the self-emissive pixels, including for example row driver and/or column drivers (e.g., display drivers). Each of the self-emissive pixels may include any suitable light emitting element, such as a LED, one example of which is an OLED. However, any other suitable type of pixel, including non-self-emissive pixels (e.g., liquid crystal as used in liquid crystal displays (LCDs), digital micromirror devices (DMD) used in DMD displays) may also be used.

The processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may execute instructions stored in local memory20and/or the main memory storage device22to perform certain image processing operations. For example, the processor core complex18and the image processing circuitry28may encode image data captured by the image sensor13and/or decode image data for display on the electronic display12. And, as discussed in greater detail below, the processor core complex18and/or image processing circuitry28may modify image data to generate adjusted image data that, when displayed, is more viewable by users with color vision deficiency. As such, the processor core complex18and image processing circuitry28may include one or more general purpose microprocessors, one or more application specific processors (ASICs), one or more field programmable logic arrays (FPGAs), or any combination thereof. Additionally, in some embodiments, the image processing circuitry28may be included (partially or completely) in the processor core complex18.

The local memory20and/or the main memory storage device22may be tangible, non-transitory, computer-readable mediums that store instructions executable by and data to be processed by the processor core complex18and the image pre-processing circuitry. For example, the local memory20may include random access memory (RAM) and the main memory storage device22may include read only memory (ROM), rewritable non-volatile memory such as flash memory, hard drives, optical discs, and the like. By way of example, a computer program product containing the instructions may include an operating system or an application program.

Using the network interface24, the electronic device10may communicatively couple to a network and/or other computing devices. For example, the network interface24may connect the electronic device10to a personal area network (PAN), such as a Bluetooth network, a local area network (LAN), such as an 802.11x Wi-Fi network, and/or a wide area network (WAN), such as a 4G or LTE cellular network. In this manner, the network interface24may enable the electronic device10to transmit encoded image data to a network and/or receive encoded image data from the network for display on the electronic display12.

The processor core complex18is operably coupled with I/O ports16, which may enable the electronic device10to interface with various other electronic devices. For example, a portable storage device may be connected to an I/O port16, thereby enabling the processor core complex18to communicate data with a portable storage device. In this manner, the I/O ports16may enable the electronic device10to output encoded image data to the portable storage device and/or receive encoded image data from the portable storage device.

In addition to enabling user inputs, the electronic display12may include one or more display panels. Each display panel may be a separate display device or one or more display panels may be combined into a same device. The electronic display12may control light emission from the display pixels to present visual representations of information, such as a graphical user interface (GUI) of an operating system, an application interface, a still image, or video content, by displaying frames based on corresponding image data. As depicted, the electronic display12is operably coupled to the processor core complex18and the image processing circuitry28. In this manner, the electronic display12may display frames based on image data generated by the processor core complex18and/or the image processing circuitry28. Additionally or alternatively, the electronic display12may display frames based on image data received via the network interface24, an input device14, an I/O port16, or the like.

The power source26may include any suitable source of energy, such as a rechargeable lithium polymer (Li-poly) battery and/or an alternating current (AC) power converter. Furthermore, as depicted, the processor core complex18is operably coupled with input structures14, which may enable a user to interact with the electronic device10. The input structures14may include buttons, keyboards, mice, trackpads, and/or the like. Additionally or alternatively, the electronic display12may include touch components that enable user inputs to the electronic device10by detecting occurrence and/or position of an object touching its screen (e.g., surface of the electronic display12).

In addition to enabling user inputs, the electronic display12may present visual representations of information by display images (e.g., image frames), such as a graphical user interface (GUI) of an operating system, an application interface, a still image, or video content. As described above, the electronic display12may display an image based on corresponding image data. In some embodiments, the image data may be received from other electronic devices10, for example, via the network interface24and/or the I/O ports16. Additionally or alternatively, the image data may be generated by electronic device10using the image sensor13. In some embodiments, image sensor13may digitally capture visual representations of proximate physical features as image data.

The image data may be encoded (e.g., compressed), for example, by the electronic device10that generated the image data, to reduce number of memory addresses used to store and/or bandwidth used to transmit the image data. Once generated or received, the encoded image data may be stored in local memory20. Accordingly, to a display image corresponding with encoded image data, the processor core complex18or other image data processing circuitry may retrieve encoded image data from local memory20, decode the encoded image data, and instruct the electronic display12to display image frames based on the decoded image data.

As noted above, the electronic device10may be any suitable electronic device. To help illustrate, one example of a handheld device10A is described inFIG.2, which may be a portable phone, a media player, a personal data organizer, a handheld game platform, or any combination of such devices. For example, the handheld device10A may be a smart phone, such as any iPhone® model available from Apple Inc.

The handheld device10A includes an enclosure30(e.g., housing). The enclosure30may protect interior components from physical damage and/or shield them from electromagnetic interference, such as by surrounding the electronic display12. The electronic display12may display a graphical user interface (GUI)32having an array of icons. When an icon34is selected either by an input device14or a touch-sensing component of the electronic display12, an application program may launch.

The input devices14may be accessed through openings in the enclosure30. The input devices14may enable a user to interact with the handheld device10A. For example, the input devices14may enable the user to activate or deactivate the handheld device10A, navigate a user interface to a home screen, navigate a user interface to a user-configurable application screen, activate a voice-recognition feature, provide volume control, and/or toggle between vibrate and ring modes. The I/O ports16may be accessed through openings in the enclosure30and may include, for example, an audio jack to connect to external devices.

Another example of a suitable electronic device10, specifically a tablet device10B, is shown inFIG.3. The tablet device10B may be any IPAD® model available from Apple Inc. A further example of a suitable electronic device10, specifically a computer10C, is shown inFIG.4. For illustrative purposes, the computer10C may be any MACBOOK® or IMAC® model available from Apple Inc. Another example of a suitable electronic device10, specifically a watch10D, is shown inFIG.5. For illustrative purposes, the watch10D may be any APPLE WATCH® model available from Apple Inc. As depicted, the tablet device10B, the computer10C, and the watch10D each also includes an electronic display12, input devices14, I/O ports16, and an enclosure30. The electronic display12may display a GUI32. Here, the GUI32shows a visualization of a clock. When the visualization is selected either by the input device14or a touch-sensing component of the electronic display12, an application program may launch, such as to transition the GUI32to presenting the icons34discussed inFIGS.2and3.

Turning toFIG.6, a computer10E may represent another embodiment of the electronic device10ofFIG.1. The computer10E may be any computer, such as a desktop computer, a server, or a notebook computer, but may also be a standalone media player or video gaming machine. By way of example, the computer10E may be an iMac®, a MacBook®, or other similar device by Apple Inc. of Cupertino, California. It should be noted that the computer10E may also represent a personal computer (PC) by another manufacturer. A similar enclosure36may be provided to protect and enclose internal components of the computer10E, such as the electronic display12. In certain embodiments, a user of the computer10E may interact with the computer10E using various peripheral input devices14, such as the keyboard14A or mouse14B (e.g., input devices14), which may connect to the computer10E.

As described above, image data may modified to cause content that is generated from the modified image data and displayed to be more viewable to users with color vision deficiency (e.g., colorblind). More specifically, the techniques described herein may enable user-specific settings to be set and utilized to modify image data (e.g., color components of image data) to reduce or eliminate zones of confusion that a user with color vision deficiency may experience. A zone of confusion may exist when a user cannot discern what a particular color is. For example, a user with red-green colorblindness may be unable to discern whether particular content is red or green.

Bearing this in mind,FIG.7is a diagram of a color wheel50that includes various regions52(e.g., regions52A-52F) and sections54(e.g., sections54A-54P). Each section54(referring to any of the sections54A-54P) include hues or shades of a color of the section54, which generally become less saturated (e.g., paler) the closer to a center56of the color wheel50one looks. The color wheel50may be referenced below when discussing various aspects of the present disclosure. Furthermore, while the color wheel50has six regions52and the techniques described below are based on having six regions52, in other embodiments, the color wheel50may be subdivided into fewer or more than six regions52, and the techniques described below may accordingly be modified based on the number of regions utilized. For example, in other embodiments, the number of regions52may be two, three, four, five, or an integer between seven and thirty-six, inclusive.

As also illustrated, the color wheel50includes zones of confusion58(e.g., zones of confusion58A-58D). The zones of confusion58are representative of portions of the color wheel50(e.g., two individual hues or two portions of two sections54of the color wheel50) that people with color vision deficiency may experience difficulty discerning or be unable to discern. For example, zones of confusion58A,58B may occur when a particular hue is near a yellow hue (e.g., near section54B). As another example, another zone of confusion58C may exist for hues of green (e.g., near section54F). For these zones of confusion58(i.e., zones of confusion58A-58C), a user may be unable to discern whether a particular hue is closer to a red hue or a green hue. As another example, another zone of confusion58D may exist for those who are unable (or less able) to differentiate between blue-green and purple, which may occur for those with red-green colorblindness (e.g., due to being unable to discern whether a particular hue is a mixture of blue with red or blue with green). Furthermore, zones of confusion may exist for less saturated hues. Indeed, as one progresses closer to the center56of the color wheel50and the hues become less saturated, those with color vision deficiency may be unable to discern between gray hues and blue-green hues as well as between gray hues and blue-red hues.

As noted above, the present disclosure describes techniques for eliminating zones of confusion (e.g., zones of confusion58A-58D), which may thereby enable electronic devices to generate and display content with colors discernable to those with color vision deficiency. Before describing the implementation of such techniques, several examples of images will be discussed to show how images generated using the techniques of the present disclosure compare with images generated using other techniques.

FIG.8Ais an image70A that is known as an Ishihara template. The image70A includes six samples72(e.g., samples72A-72F), each of which is a larger circle that includes 1) a “background” composed of smaller circles of one or more colors and 2) one or more numerals surrounded by the background. For example, in a first sample72A, the background includes red, orange, and yellow hues along with a numeral74A (“7”) that is composed of green hues. A second sample72B includes a background of several brown hues (e.g., tan hues). The second sample72B also includes a numeral76A (“1”) that includes pink and purple hues and a numeral78A “3” that includes pink and red hues. A third sample72C includes a gray background with numerals80A,82A (“1” and “6,” respectively) of an orange hue. A fourth sample72D includes a background with yellow and green hues and a numeral84(“8”) composed of pink and orange hues. A fifth sample72E includes a background made of red and orange hues. The fifth sample72E also includes a numeral86A (“1”) composed of green hues and another numeral88A (“2”) composed of green and yellow hues. The image70A also includes a sixth sample72F that includes a background of green, yellow, and blue hues. The sixth sample72F also includes a numeral90A “9” that is formed by red or orange hues.

FIG.8Bis an image70B that is generated by utilizing a technique called the Dalton algorithm to modify the image70A. The image70B includes samples72G-72L, which respectively correspond to samples72A-72F ofFIG.8A. More specifically, the image70B is generated using the Dalton algorithm for those with protanomaly (also known as protanopia), which is a form of red-green color deficiency in which the individual is missing (or has malfunctioning) long cones. Long cones are photoreceptors in the human eye that are responsible for detecting relatively long wavelengths in the visible color spectrum, which correspond to red. The human eye also includes medium cones and short cones, which are respectively responsible for detecting green and blue wavelengths. A person with protanomaly may be confused between red-green, red-orange, blue-green, and gray. In the Dalton algorithm, pixels are converted from RGB (red, green, blue) components to a LMS (long, medium, short) scale. For protanopia, the long component is replaced by a linear combination of medium and short, while the medium and short components remain the same. While the image70B is generated for those with protanomaly, it should also be noted that the Dalton algorithm may also be used for those with another form of red-green color deficiency, deuteranopia, which occurs when a person is missing or has malfunctioning medium cones. However, in either case the resulting image (e.g., image70B) may cause the colors in an image to become distorted. For instance, compared toFIG.8A, inFIG.8B, the “warmer” colors (e.g., red, orange, and yellow hues) are generally represented in a “colder” manner, such as with hues of blue or green.

FIG.8Cis image70C that is generated by utilizing the techniques of the present disclosure on the image70A ofFIG.8Awith samples72M-72R respectively corresponding to samples72A-72F ofFIG.8A. As can be observed by comparing the image70C to the image70A and the image70B, the colors utilized for the background and numerals (e.g., numerals74C,76C,78C,80C,82C,84C,86C,88C,90C) are relatively more similar to those of the image70A compared to the image70B ofFIG.8B. Accordingly, the techniques described herein may enable hues from original images to be retained or utilize hues similar those in an original image while also providing content in a manner that enables a user with color vision deficiency to discern between the hues included in the image.

FIG.9Ais an original image120A that includes a rainbow122B in the foreground and a canyon and sky with clouds in the background. The original image120A is original image, meaning the original image120A is generated or captured without utilizing an algorithm or technique to adjust colors, for example, to account for viewers with color vision deficiency. As illustrated, the rainbow122A includes a red, orange, yellow, green, blue, indigo, and violet bands.

FIG.9Bis an image120B that is generated by using the Dalton algorithm on the original image120A. As illustrated, in rainbow122B in the image120B, bands that were red, orange, yellow or green ofFIG.9A, are depicted in hues of yellow, while bands that were previously blue, indigo, or violet appear in hues or blue or purple (or blue-purple). Thus, similar toFIG.8Brelate toFIG.8A, using the Dalton algorithm may cause some colors or hues to become completely different colors (e.g., instead of different shades of a color).

FIG.9Cis an image120C generated by using the techniques of the present disclosure on the original image120A ofFIG.9A. In the image120C, rainbow122C includes individual bands for each of the colors (e.g., red, orange, yellow, green, blue, indigo, and violet) of the original image120A using colors that are more similar to those of the original image120A relative to the image120B. Accordingly, the techniques described herein may enable hues from original images to be retained or utilize hues similar those in an original image while also providing content in a manner that enables a user with color vision deficiency to discern between the hues included in the image.

As another example,FIGS.10A-10Cwill be discussed. In particular,FIG.10Ais an original image130A that includes green elements (e.g., leaves, stems) and red elements (e.g., berries or fruits). InFIG.10B, which is an image130B generated by utilizing the Dalton algorithm on the original image130A ofFIG.10A, the formerly red and green elements are both shown in hues of green, which may cause confusion for those with protanomaly or deuteranopia. In other words, a person with certain forms of color vision deficiency may be unable to discern between colors in the image130B or recognize that certain elements of the image130B were originally (e.g., as shown inFIG.10A) different colors than those used inFIG.10B.

In contrast toFIG.10B,FIG.10C, which is an image130C generated by using the techniques of the present disclosure on the original image130A, retains separate green and red hues. Accordingly, viewers with color vision deficiency may be able to discern that different colors (e.g., green and red) are present in an image. Thus, as also noted above, the techniques described herein may enable hues from original images to be retained or utilize hues similar those in an original image while also providing content in a manner that enables a user with color vision deficiency to discern between the hues included in the image.

Keeping the foregoing in mind, an overview of features of the present application will be discussed. As shown inFIGS.8C,9C, and10C(as respectively compared toFIGS.8A,9A, and10A), the techniques of the present application cause the hue (e.g., a moving to a different section54of the color wheel50), saturation (e.g., movement within a section54of the color wheel50towards or away from the center56), or both to be performed (e.g., for each pixel (or a portion of the pixel) of an original image or hue (or portion of the hues) of the original image while also maintaining a similar appearance (e.g., in terms of hues being used) as the original image.

For example, referring toFIG.7, the color wheel50may be divided into the regions52, with each region52representing sixty degrees of the 360 degrees in the color wheel50(because the color wheel50is a circle). Region52A may be referred to as a red-yellow region (representing zero to sixty degrees of the color wheel50), region52B may be referred to as a yellow-green region (representing sixty to 120 degrees of the color wheel50), region52C may be referred to as a green-cyan region (representing 120 to 180 degrees of the color wheel50), region52D may be referred to as a cyan-blue region (representing 180 to 240 degrees of the color wheel50), region52E may be referred to as a blue-violet region (representing 240 to 300 degrees of the color wheel50), and region52F may be referred to as a violet-red region (representing 300 to 360 degrees of the color wheel50). As discussed below, a user may cause settings associated with each of the regions52to be modified, which alters the appearance of image content displayed on the electronic device10. In other words, a user may modify settings to cause the electronic device10to modify image content to be adapted in a manner that enables image content to be provided in a manner that is preferred or best for that specific user.

Keeping this in mind,FIG.11is a front view of a graphical user interface (GUI)140that may be displayed via the electronic display12of the electronic device10. The GUI140may be accessible to the user of the electronic device10, for example, from a settings menu (e.g., in a display settings section of a settings menu or a submenu within the display settings section). As illustrated, the GUI includes an image window142and sliders144(e.g., sliders144A-144C). The image window142may display one or more images that may be modified as a user modifies the positioning of one or more of the sliders144. For example, the image(s) initially include one or more of the images (portions thereof) included inFIGS.8A,9A, and10A. Furthermore, while the sliders144are included inFIG.11, it should be noted that, in other embodiments, another suitable GUI element may be used instead of a slider. That is, other GUI elements that a user many interact with or display (or otherwise indicate) user-selected values (or settings) may be used in lieu of the sliders144. Furthermore, while the discussion below regarding the sliders144generally pertains to altering how image data is modified, it should be noted that the GUI140(or another GUI presented in a setting menu or submenu) may include a slider144or other GUI element with which a user many interact to enable a color vision deficiency mode, which may also be called a “colorblind mode”. A user may make a user input to interact with such a GUI element (e.g., using the input devices14or a touch input in embodiments of the electronic device10in which the electronic display12is a touchscreen) to enable color vision deficiency mode, and, in response to the user input, the processor core complex18, image processing circuitry28, or the processor core complex18and the image processing circuitry28may begin to generate adjusted image data. In other words, the electronic device10may display original or source image data until a user input is made to activate color vision deficiency mode. In response, the color vision deficiency mode may be activated, and the adjusted image data may be generated and displayed. Furthermore, in one embodiment, display of the sliders144, access to the sliders144, or both may be prevented until a user input to activate color vision deficiency mode has been received.

The sliders144may include a first slider144A, a second slider144B, and a third slider144C. The sliders144, or a portion thereof, may be provided for each of the regions52. As discussed below, a user may interact with the sliders144to alter how image data is modified, thereby enabling the user (e.g., a user with color vision deficiency) to cause image content to be altered in a user-specific manner that best enables the user to differentiate between colors in image data presented. The sliders144may be different types of sliders. For example, the first slider144A may be a threshold slider, the second slider144B may a power slider, and the third slider144C may be a minimum color (C_min) slider. Power sliders and threshold sliders may be provided for each of the regions52, while three C_min sliders may be provided. Accordingly, in one embodiment, there may be fifteen sliders144: six threshold sliders, six power sliders and three C_min sliders. In another embodiment, the GUI140may represent one of several GUIs that include sliders. For example, in one embodiment, sliders144specific to one of the regions52(e.g., a threshold slider and a power slider) may be provided in the GUI140along with the image window142. A user may navigate (e.g., using a swiping motion on a device in which the electronic display12is a touchscreen) to a different GUI that also includes an image window (with one or more images, which could be the same as the image(s) provided in the image window142) and additional sliders for another of the regions52. Additionally, one of the GUIs may the C_min sliders. Thus, in such an embodiment, there may be seven GUIs140: one for each of the six regions52and one for the C_min sliders. In another embodiment, all of the sliders144may be presented in the GUI140or accessible via the GUI140(e.g., by scrolling or swiping upwards or downwards within the GUI140).

Before discussing the types of sliders144in more detail, it should be noted that in displays, such as the electronic display12, pixels emit light to cause content to be displayed. Pixels may include subpixels such as red, green, and blue subpixels, which may respectively emit red, green, and blue light at different brightness levels. By utilizing red, green, and blue light at certain brightness levels, each of the hues of the color wheel50may be displayed. For example, image data may include values (e.g., RGB values) indicative of the brightness levels for each of the red, green, and blue subpixels of a given pixel, and a particular hue will be emitted with a particular combination of RGB values. As a more specific example, white light may correspond to RGB value 255, 255, 255, meaning each of the red, green, and blue subpixels emits light at a maximum brightness. Accordingly, the content to be emitted by a pixel may include red, green, and blue components. For a particular set of RGB values, there may be a maximum color component, a middle color component, and a minimum color component. For example, the color of jade green may have an RGB value of 0, 168, 107, in which case green (corresponding to the value of 168) is the maximum color component, blue (corresponding to the value of 107) is the medium color component, and red (corresponding to the value of zero) is the minimum color component. As discussed below, by modifying the position of one or more of the sliders, one or more of the RGB values associated with a pixel (e.g., original image data) may be modified in a user-specific manner, and image content generated and displayed using the modified image data may allow the user (e.g., a person with color vision deficiency) to better differentiate between colors in content provided on the electronic display12.

To help provide more context forFIG.11,FIG.12is a flow diagram of a process160for generating and displaying adjusted image data based on a user's interaction with the GUI140. Generally, the process160includes displaying interface items for color settings (process block162), receiving user input indicative of a selection of color settings (process block164), modifying image data based on the selected color settings (process block166), and displaying adjusted image data (process block168). In some embodiments, the process160may be implemented at least in part based on circuit connections formed (e.g., programmed) in the processor core complex18, the image processing circuitry28, or both (e.g., partially or completely) the processor core complex18and the image processing circuitry28. Additionally or alternatively, the process160may be implemented at least in part by executing instructions stored in a tangible non-transitory computer-readable medium, such as the local memory20, using processing circuitry, such as the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28.

At process block162, the processor core complex18may cause interface items for color settings to be displayed, for example, via the electronic display12. The interface items may include the image window142and the sliders144.

At process block164, the processor core complex18may receive user input indicative of a selection of color settings. For example, the user may adjust the positioning of one of more of the sliders144using one of the input devices14or, in embodiments in which the electronic display12is a touchscreen, an interaction with the touchscreen (e.g., a swiping or sliding motion made using a finger).

At process block166, the processor core complex18, image processing circuitry28, or both the processor core complex and the image processing circuitry28may modify image data based on the selected color settings (e.g., as indicated by the user input received at process block164) to generate adjusted image data. Furthermore, at process block168, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may cause the adjusted image data to be presented, for instance, via the electronic display12. For example, based on the color settings selected by the user of the electronic device10, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may modify image data, including an original image (or images) originally presented in the image window142or any other content (e.g., images, video, user interfaces) shown after the user input provided at process block164, based on the color settings indicated by user input. For instance, color(s) in an image provided in the image window142may be modified in response to the user input. More specifically, as a user causes a slider144to be moved, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine adjusted image data (e.g., new RGB or color values for one or more pixels of the electronic display12) based on the user input to move the slider144, and the adjusted image data may be displayed. As such, the user may view see how moving a particular slider144causes image content to be modified and allow the user to find color settings (which correspond to the placements of the sliders144) that best suit the user.

While modification of image data is discussed in more detail below, the sliders144themselves will first be described. Referring briefly back toFIG.11, adjusting the threshold slider (e.g., first slider144A) causes colors of a region52to be modified to be closer to a boundary of the region. For instance, moving the threshold slider for the yellow-green region (i.e., region52B) to one side (e.g., to the left) may cause an original hue to have more yellow, while moving the slider to the other side (e.g., to the right) may cause the original hue to be transformed to a hue that includes more green. More specifically, the positioning of the first slider144A may adjust where a threshold within the region52B is such that hues falling one side of the threshold are modified to have more of one color (e.g., in the current example, yellow or green) depending on which side of the threshold the hue is located. Furthermore, adjusting the threshold slider may also cause the secondary component (e.g., the medium color component) for colors of a region52to be increased. As an example, if the first slider144A for the cyan-blue region (e.g., region52D) is placed relatively close to blue (e.g., to the left), the green component of the cyan-blue region may be enhanced within the region52D so that blue-green is relatively more differentiable from blue-red or purple compared to the original image data.

Bearing this in mind,FIG.13is a graph180A of the value of a function ƒ (indicated by axis182) across color sectors (indicated by axis184), which includes sections54of the color wheel50. For example, the axis184includes areas186A-186D corresponding to regions52A-52D of the color wheel50. The function ƒ itself is indicated by line188, which is indicative of the color (e.g., for a particular pixel) as modified by the processor core complex18, image processing circuitry28, or combination of the processor core complex18and the image processing circuitry28(e.g., as performed at process block166of the process160ofFIG.12.) In particular, ƒ is defined as:

where Cmaxis the value of the maximum color component, Cmidis the value of the middle color component (which is also indicated as “secondary color” within each area186of the graph180A), and Cminis the value of minimum color component. The maximum, middle, and minimum colors for each area186of the graph180A are indicated below in Table 1. Table 1 also indicates the maximum, middle, and minimum color components associated with regions52E,52F.

The graph180A also includes line190, which is indicative of unmodified color values. In other words, the line190is indicative of the value off for image data that is not modified using the techniques of the present disclosure (e.g., original image data). For example, at a minimum point192of the area186A corresponding to the color red (e.g., RGB value 255, 0, 0), ƒ has a value of zero due to Cmid being zero. At a maximum point194corresponding to yellow (e.g., RGB value 255, 255, 0) that forms the boundary between the areas186A,186B, ƒ has a value of one because Cmidand Cmaxare equal. Accordingly, as the line190transitions from the minimum point192to the maximum point194within the area186A, the amount of the secondary color (i.e., green, which is associated with Cmid) increases.

For users with color vision deficiency, increasing the amount of a secondary color (to an extent and depending on the specific user) may result in a zone of confusion. More specifically, as the amount of the secondary color increases until a certain point (which may vary from user to user) at which ƒ reaches a value relatively closer to one, a viewer may not be able to discern between colors. For example, area196of the graph180A is representative of a zone of confusion a user may experience when viewing unmodified content. In other words, for a certain range of values of the function ƒ, a user may experience difficulty discerning between colors. For instance, at point198on the line190(which is one boundary of the zone of confusion represented by the area196), the value off may be a first value. At point200on the line190, which represents another boundary of the zone of confusion represented by the area196, the value off may be a second value.

By modifying the amount of the secondary color present, the zone of confusion may be reduced relative to using unmodified image data. For example, as represented by area202associated with the line188, when image data is modified using the techniques of the present disclosure, the zone of the confusion may be reduced relative to when unmodified image data is used. In particular, point204, which corresponds to point198(e.g., has a same value off), is positioned further right along the axis184(relative to point198), and point206, which corresponds to point200, is positioned further left along the axis184(relative to point200), signifying that colors (e.g., as indicated by original (i.e., unmodified) image data) may be modified to include less of the secondary color or more of the secondary color depending on the location within an area186(corresponding to a region52of the color wheel50) a point on the line188is. For example, for colors having relatively less of the secondary color (e.g., green, in region52A and area186A), unmodified image data may be modified to use less of the secondary color (e.g., green in area186A) in area208, while colors that have more of the secondary color present (e.g., colors in area210) may be modified to include more of the secondary color. As such, modified colors may resemble colors called for by original image data while also being discernable to users with color vision deficiency.

To help provide more context regarding the threshold slider,FIG.14is provided. In particular,FIG.14is graph180B that includes the axes182,184, lines190, and a line188A. The line188A is similar to the line188ofFIG.13in that the line188A is representative of values of ƒ for adjusted image data. In other words, the line188A is representative of values of ƒ that occur for adjusted image data, with the adjusted image data being adjusted in a different manner than the adjusted image data represented by line188ofFIG.13, for example, due to the sliders144having a different placement (than the placement that would result in adjusted image data represented by the line188).

The graph180B also includes lines220(e.g., lines220A-220D), which are representative of where thresholds are positioned within the areas186(e.g., areas186A-186D). That is, line220A is representative of the placement of the threshold for area186A (which corresponds to region52A of the color wheel50), line220B is representative of the placement of the threshold for area186B (which corresponds to region52B of the color wheel50), line220C is representative of the placement of the threshold for area186C (which corresponds to region52C of the color wheel50), and line220D is representative of the placement of the threshold for area186D (which corresponds to region52D of the color wheel50). Thresholds associated with the regions52E,52F may also be defined. Referring specifically to the line220A, to one side of the line188A (e.g., left of line220A), the line188A has a value off that is lower than the corresponding value off for the same value on the axis184. Thus, for points along the line188A that are to the left of the line220A, the relative amount of the secondary color component (e.g., green) may be reduced relative to the line190. Conversely, for points along the line188A to the right of the line220A, the relative amount of the second color component may be increased. As discussed above, by performing such adjustments when generating adjusted image content, users with color vision deficiency may be better able to discern between colors.

By utilizing the sliders144(e.g., one or more first sliders144A), a user may adjust the placement of the lines220. In turn, the processor core complex18, image processing circuitry28, or both may modify how adjusted image data is generated. For example, if a user were to adjust a threshold slider (e.g., first slider144A) for the region52, the placement of the line220A would have a corresponding adjustment, and the line188A would also be modified. For instance, if the slider144A were moved to increase the amount of red color present, the line220A may be moved to the right so that more colors within the region52A are represented with more red (and less green). Additionally, the line188A would be adjusted so that a larger range of values along the axis184within the area186A on the line188A would have values off that are lower than the corresponding values off along the line190at the same point on the axis184.

Before continuing with the discussion of the sliders144, it should be noted that the examples discussed above with respect to the region52A and area186A are non-exclusive and non-limiting examples. That is, the techniques described above, as well as those described below, with respect to one specific region52A or area (e.g., area186A) may be applied to each region52or area186. In this manner, the techniques provided in the present disclosure may be applied on a region by region basis.

Returning briefly toFIG.11, as noted above, the sliders144may include power sliders such as the second slider144B. Adjusting the power slider may cause a power value (e.g., exponent value) to be modified, thereby causing how adjusted image data is generated to be modified. In other words, when a user adjusts the position of a power slider, one or more exponent values associated with a region52of the color wheel may be modified. Consequently, processing circuitry that generates the adjusted image data (e.g., the processor core complex18, image processing circuitry28, or both) may determine different adjusted image data using a one or more modified power values based on a user inputs with one or more power sliders.

To help demonstrate,FIG.15is provided. In particular,FIG.15is a graph180C that includes the axes182,184and the line190. The graph180C also includes lines188B,188C,188D, which are representative of values off for three different sets of adjusted image data that may be generated based on three different positions of the respective power sliders for the regions52A-52D. As the power slider is adjusted (e.g., by a user input) to increase or decrease the power value (which may or may not be displayed), the adjusted image content may be modified, which is represented by the lines188B-188D. For example, line188C represents a decrease in the power value from line188B, and line188D represents a decrease in the power values from line188C.

Each region52(and, thus, each area186) may have one or more associated power values. Thus, similar to how the portions of the line188A of each area186of the graph180B may be associated with a region-specific threshold value, the portions of the lines188B-D found in each area186of the graph180C may be associated with a different power value. In other words, each region52may have a power value that is selected by a user using a power slider (of the sliders144) for the region52, and the adjusted image data may be determined in a region-specific manner.

Returning briefly toFIG.11, as also noted above, the sliders144may include sliders144may also include minimum color (c_min) sliders, such as the third slider144C. For example, there may be three minimum color sliders: one for red; one for green; and one for blue. As noted above in Table 1, blue is the minimum color for regions52A,52B, red is the minimum color for regions52C,52D, and green is the minimum color for regions52E,52F. As also noted above, one potential way to increase discernibility between zones of confusion is to increase the saturation of colors within a region52. To increase the saturation of colors, the minimum color component may be modified, for instance, as indicated by a user input made using the minimum color sliders.

Bearing this in mind,FIG.16is a graph240illustrating a minimum color adjustment (represented by arrow242) for the color blue, which, as discussed above, is the minimum color in regions52A,52B. Area244A and area244B of the graph240respectively correspond to area52A and area52B of the color wheel50. The graph240also includes line246, axis248that is indicative of color within regions52A,52B, and axis250. The axis250is indicative of the amount of the minimum color in adjusted image data. For example, the closer to line252the line246is, the more reduced the minimum color component (e.g., blue) is. By adjusting the position of the minimum color slider, the amount of the minimum color is modified in the adjusted image content, and the slope of the line246changes. For example, if a user were to adjust the minimum color slider to a minimum position, the line246would be horizontal (e.g., constantly at a value of one on the axis250), and no additional (as desired by a user) adjustment to minimum color would occur. However, moving the minimum color slider to a maximum position, the amount of the minimum color present would be decreased, and the line246would be steeper. Accordingly, users may adjust the minimum color component for each color (e.g., red, green, and blue) using a respective minimum color slider, and processing circuitry (e.g., processor core complex18, image processing circuitry28, or both) may generate adjusted image data based on the respective position of each slider. In this manner, the electronic device10may generate adjusted image data that, when displayed (e.g., via the electronic display12) better enables the user to discern between colors.

Keeping the foregoing in mind,FIG.17is a flow diagram of a process270for generating adjusted image data. The process270may be performed as process block166, portions of the process270may be performed as process block166, or the process270may be performed as part of process block166of the process160discussed above with respect toFIG.12. In some embodiments, the process270may be implemented at least in part based on circuit connections formed (e.g., programmed) in the processor core complex18, the image processing circuitry28, or both (e.g., partially or completely) the processor core complex18and the image processing circuitry28. Additionally or alternatively, the process270may be implemented at least in part by executing instructions stored in a tangible non-transitory computer-readable medium, such as the local memory20, using processing circuitry, such as the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28. The process270generally includes receiving image data (process block272), determining whether the maximum, middle, and minimum color components of image data for a pixel are equal (decision block274), and, when the maximum, middle, and minimum color components of image data are equal, using the received image data as adjusted image data (process block276) and outputting the adjusted image data (process block278). When the maximum, middle, and minimum color components of image data are not equal, the process270includes generating adjusted image data (process block280) and outputting the adjusted image data (process block278). Generating adjusted image data may be performed through a series of operations. For example, adjusted image may be generated by normalizing color components of image data, such as original or source image data (sub-process block282), determining degamma values for the normalized color components (sub-process block284), denormalizing color components of the image data (sub-process block286), calculating hue for a pixel (sub-process block288), classifying image data for a pixel (sub-process block290), modifying image data for the pixel based on the classification of the image data (sub-process block292), performing a luma adjustment (sub-process block294), and performing gamma adjustment (sub-process block296). It should also be noted that, in some embodiments, the process270may include fewer operations than those described below. As such, the process270may be performed using only a portion of the operations provided inFIG.17.

At process block272, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may receive image data. For example, the image data may be considered as source image data or original image data that will be modified during performance of the process270. The source image data may be generated by the electronic device10(e.g., via image sensor13or from memory20or main memory storage device22) or received by the electronic device10(e.g., via I/O ports16or network interface24).

At decision block274, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine whether the maximum, middle, and minimum color components of image data for pixels (e.g., for each pixel for which there is image data) are equal. For instance, as discussed above, image data for a pixel may include red, green, and blue components (e.g., RGB values), one of which is the maximum color component of the pixel, another of which is the middle color component of the pixel, and yet another of which is the minimum color component of the pixel. Accordingly, at decision block274, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine whether the color components for a pixel are equal, and such a determination may be made for each pixel for which there is image data. It should be noted that, in another embodiment, rather than determining whether the maximum, middle, and minimum color components of the image data for pixels are equal, decision block274may be performed by determining whether the middle and minimum color components are equal.

If it is determined that the maximum, middle, and minimum color components are equal, at process block276, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may utilize the received image data as adjusted image data. In other words, when the color components for a pixel are equivalent, the image data (as received at process block272) may be unmodified but used as adjusted image data. Furthermore, at process block278, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the adjusted image data. Thus, when for image data for pixels in which the color components are equivalent, the adjusted image data that is output may be the image data that is received at process block272.

Keeping this in mind,FIG.18is a block diagram of image data processing circuitry300that may be utilized to generate adjusted image data. Accordingly, the image data processing circuitry300may be utilized to perform the process270. The image data processing circuitry300may be included entirely or partially in the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28. For instance, in some embodiments, one of the processor core complex18and the image processing circuitry28may include the image data processing circuitry. In other embodiments, the processor core complex18and the image processing circuitry28may each include the image data processing circuitry. In further embodiments, the processor core complex18and the image processing circuitry28may each include portions of the image data processing circuitry300. Furthermore, while elements of the image data processing circuitry300are described as circuitry, it should be noted that one or more components of the image data processing circuitry300may be implemented as computer-readable instructions that are executed by the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28.

The image data processing circuitry300includes image data modification circuitry302that, as discussed below, may be utilized to generate adjusted image data. The image data processing circuitry300also includes a multiplexer304that may receive original image data (e.g., image data received at process block272) and image data generated by the image data modification circuitry302. The multiplexer304may also receive an input indicative of whether the color components are equal (as indicated by “R==G==B”) and output either the original image data or image data generated by the image data modification circuitry302based on the input. More specifically, when the input is indicative of the color components being equal, the output of the multiplexer304is the original image data, and when the input is not indicative of the color components being equal, the output of the multiplexer304is a the adjusted image data that is generated by the image data modification circuitry302. Accordingly, in instances in which the color components of a pixel are equal, the image data processing circuitry300may output adjusted image data for the pixel that is equivalent to the image data received by the image data processing circuitry300.

Returning toFIG.17, if at decision block274it is determined that the maximum, middle, and minimum color components are not equal, at process block280, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may generate adjusted image data. In other words, when at least one color component is different than another color component, adjusted image data that enables a user with color vision deficiency to better discern colors or hues in image content may be generated. As discussed below with respect to sub-process blocks282,284,286,288,290,292,294,296, generating the adjusted image data may include several operations.

At sub-process block282, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may normalize the color components of image data received at process block272(e.g., original or source image data). For example, color components for each pixel in the electronic display12may be values that may be defined according to one or more scales (e.g., using a range of values from zero to 255 scale or a different scale), and the image data may be normalized so that each value is defined on a different, normalized scale (e.g., a value from zero to one, inclusive).

Keeping this in mind, and referring toFIG.18, the image data processing circuitry300includes a normalization block306that may perform the normalization operation associated with sub-process block282. For example, the normalization block306may include multiplier circuitry that can perform multiplication and shift (e.g., left-shift or right-shift) operations on the color components of image data for each pixel (e.g., in a frame of image content) to generate normalized image data.

Returning toFIG.17, at sub-process block284, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine degamma values for the normalized color components generated at sub-process block282. Gamma correction can be used to control the overall brightness of an image. For example, gamma may define the relationship between a pixel's numerical value and its actual luminance. Thus, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine values that remove gamma correction that may have previously been applied to image data to determine degamma values. The degamma values may include values for each component of the image data for each pixel. For instance, for a given pixel, there may be red, green, and blue degamma values.

Turning briefly toFIG.18, the image data processing circuitry300includes degamma circuitry310, which may be implemented using a look-up table (e.g., alone or in combination with processing circuitry). For example, degamma circuitry310may receive normalized image data (which includes a red, green, and blue component). The look-up table may define a relationship between received values and output values such that for any given values of a component, a particular output value will be selected and output from the degamma circuitry310. Thus, the degamma circuitry310may receive normalized image data and determine (using the look-up table) degamma values for each color component, and output the determined degamma values. In other words, the degamma circuitry310may generate degamma normalized color components. Furthermore, while the degamma circuitry310is described as being implemented partially using a look-up table, in other embodiments, other circuitry may be used.

Continuing withFIG.17and the process270, at sub-process block286, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may denormalize color components of the image data. For example, the degamma values may be multiplied using pixel denormalization circuitry308of the image data processing circuitry300. The pixel denormalization circuitry308may include multiplier circuitry that multiplies the degamma values by a particular value (e.g., 255) to generate values scaled to the same scale as the image data received at process block272of the process. Furthermore, it should be noted that, in some embodiments, denormalization of the color components performed at sub-process block284may be performed later during the operations included in process block278. For example, in one embodiment, the operations associated with sub-process block284may be performed last among the operations associated with process block278.

At sub-process block288, the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may calculate hue for each pixel for which there is image data (e.g., as generated at sub-process block284or sub-process block286). Here, hue may be defined by the red, green, and blue color components generated at one of sub-process blocks282,284,286. Thus, hue may be a particular color that is defined as the red, green, and blue components. It should also be noted that, in some embodiments, the operations associated with sub-process block288may be skipped.

At sub-process block290, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may classify image data for each pixel such that each pixel is classified into one of the regions52of the color wheel50based on the hue for the pixel. More specifically, classification may be performed based which colors form the maximum color (component), middle color (component), and minimum color (component) for the hue. For example, based on which color (e.g., among red, green, and blue) is the maximum color component, which color is the middle color component, and which color is the minimum color component, the pixel for each hue may be classified in accordance with Table 1. For instance, a hue that has a maximum color component of red, a middle color component of blue, and a minimum color component of green may be classified in region52F.

In other embodiments, the pixel for each hue may be classified in accordance with Table 2 below. In Table 2, the letter “R” is used to refer to the red color component, “G” is used to refer to the green color component, and “B” is used to refer to the blue color component. Additionally, Table 2 includes two conditions for each region52. When one of the conditions has been met, a hue for a pixel may be classified as belonging to the region52to which the met condition pertains. Furthermore, the conditions in Table 2 include “&,” which is used to signify a logical AND. Thus, a condition is met when each element of the condition is satisfied. For example, the first condition for region52A is met when: 1) the red component is greater than or equal to the green component; 2) the red component is greater than the blue component; and 3) the green component is greater than the blue component.

Referring briefly toFIG.18, the image data processing circuitry300may include regionalization circuitry312that receives image data (e.g., as generated at sub-process block284, sub-process block286, or sub-process block288). The regionalization circuitry312may determine which the maximum, middle, and minimum color component for each pixel for which there is data and classify each pixel into one of the regions52. As discussed below, image data may be modified based on the classification. In other words, image data may be modified in a region-specific manner as indicated as indicated by one or more user inputs indicative of a placement for one or more of the sliders144.

Returning toFIG.17, at sub-process block292, image processing circuitry28, or both the processor core complex18and the image processing circuitry28may modify image data for the pixel based on the classification of the image data. In particular, modified image data for a pixel may be generated by modifying the middle color component, minimum color component, or both the middle color component and minimum color component of the pixel (e.g., as provided in the image data received at process block272). To help explain how modified image data may be generated,FIGS.18-20are referenced below.

Referring now toFIG.18, the image data processing circuitry300includes a multiplexer314(e.g., a 6 to 1 multiplexer). The multiplexer314may receive regionalization configuration data for each of the regions52as well as an output of the regionalization circuitry312indicative the classification of which region52the hue for a pixel is in. The multiplexer314may then output the regionalization configuration settings associated with the region52indicated by the output of the regionalization circuitry312. The regionalization configuration settings may be stored in the local memory20or other memory that may be included in the electronic device10, such as, but not limited to, registers or cache memory of the processor core complex18, image processing circuitry28, image data processing circuitry300, or any combination thereof. The regionalization configuration settings may include data or values that are used to determine modified image data. For instance, the value of the threshold for a region52(associated with the threshold slider for the region52), the value of the power for the region52(associated with power slider for the region52), and a minimum color adjustment value for the region52(associated with the c_min slider for the region52) may be included in regionalization configuration settings. Furthermore, values derived from the value of the threshold, the value of the power, and the minimum color adjustment value, such as sums, differences, reciprocal values, products, or quotients (or values that utilized any combination thereof) may also be included in the regionalization configuration settings.

Bearing this in mind,FIG.19Ais a flow diagram of a process350A for generating modified image data. The process350A may be performed as sub-process block292of the process270, portions of the process350A may be performed as sub-process block292, or the process350A may be performed as part of sub-process block292of the process270discussed above with respect toFIG.17. The process350A may be performed for each pixel of image data that is to be modified (e.g., as determined as decision block274of process270). The process350A may also be performed on a region-wide basis, meaning the process350A may be performed for each region52of the color wheel50for which there is image data to be modified. In some embodiments, the process350A may be implemented at least in part based on circuit connections formed (e.g., programmed) in the processor core complex18, the image processing circuitry28, or both (e.g., partially or completely) the processor core complex18and the image processing circuitry28. The process350A may also be implemented using image data processing circuitry300as well as the circuitry discussed below with respect toFIG.20. Additionally or alternatively, the process350A may be implemented at least in part by executing instructions stored in a tangible non-transitory computer-readable medium, such as the local memory20, using processing circuitry, such as the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28.

The process350A generally includes performing a middle color component modification (process block352A) to generate a modified middle color component354and performing a minimum color component modification (process block356) to generate a modified minimum color component358. Performing middle color component modification (process block352A) may include determining a threshold for a region52(sub-process block360), determining a value of the function ƒ for a pixel (sub-process block362), and determining whether the value of the function ƒ is greater than, less than, or equal to the threshold (sub-decision block364A). When the value of the function ƒ is greater than the threshold, performing a middle color component modification includes generating a modified middle color component by increasing the middle color component (sub-process block366) and outputting the modified middle color component (sub-process block368). When the value of the function ƒ is greater than the threshold, performing a middle color component modification includes generating a modified middle color component by increasing the middle color component (sub-process block366) and outputting the modified middle color component (sub-process block368). When the value of the function ƒ is less than the threshold, performing a middle color component modification includes generating a modified middle color component by decreasing the middle color component (sub-process block370) and outputting the modified middle color component (sub-process block368). When the value of the function ƒ is equal to the threshold, performing a middle color component modification includes using the middle color component (e.g., as received) as the modified middle color component (sub-process block372) and outputting the modified middle color component (sub-process block368). Furthermore, performing minimum color component modification (process block356) may include determining a minimum color adjustment value for a region (sub-process block374), determining a minimum color factor based on the minimum color adjustment value (sub-process block376), determining a modified minimum color component based on the minimum color factor (sub-process block378), and outputting the modified minimum color component (sub-process block380).

At process block352A, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may perform a middle color component modification on received image data (e.g., as received at process block272of the process270). Middle color component modification may be performed on a region-specific basis, meaning middle color component modification may occur based on into which of the regions52the hue of a pixel has been classified. As noted above, middle color component modification may be performed using several operations, such as those described below with respect to sub-process blocks360,362, sub-decision block364A, and sub-process blocks366,368,370,372.

At sub-process block360, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine a threshold for a region52, such as the region52determined by the regionalization circuitry312. Referring toFIG.18, as discussed above, the multiplexer314may output regional configuration settings for the region52indicated by the regionalization circuitry, and the regionalization configuration settings may include the value of the threshold for a region52, the value of the power for the region52, and the minimum color adjustment value for the region52. Thus, at sub-process block360, the image data modification circuitry302may determine the threshold (as well as the power value and minimum color adjustment value) for the region52, for example, by reading such values from the local memory20or one or more registers that store the values.

The values of the threshold, power, and minimum color adjustment for each region52may have default values, which are provided below in Table 3. However, as noted above, the threshold, power, and minimum color adjustment for each region52is modifiable by the user by interacting with one or more of the sliders144. For example, as a user modifies the position a slider144for the threshold associated with region52A (e.g., a threshold slider), the value of the threshold for the region52A may be modified based on the user's interaction with the slider144. The modified value of the threshold, as opposed to the default value, would be utilized when performing sub-process block360if the value of the threshold has been modified. The default values may be provided in Table 3 could therefore be using for values that have not been modified. Before discussing additional regionalization configuration settings, it should be noted that in one embodiment, the value for each threshold may range from zero to one (inclusive), the value for each power may be an integer ranging from two to four (inclusive), and the value for the minimum color adjustment may range from zero to one (inclusive). In other embodiments, the ranges of values for the value of the respective thresholds of the regions52, the ranges of values of each power, and the ranges of the value of the minimum color adjustment may differ. For example, the value of the power for a region52may range from one to an integer that is greater than four (inclusive), such as eight.

As noted above, the regionalization configuration settings may also include values that are derived using the threshold, power, and minimum color adjustment values. In one embodiment, such values may also be determined at sub-process block360. In another embodiment, such values are predetermined. In either case, such derived values may include an enhancement factor, a reduction factor, and a minimum color adjustment factor. Regionalization setting determination circuitry316, which may be included for each region52, may determine the enhancement factor, the reduction factor, and the minimum color adjustment factor for the regions52. The regionalization setting determination circuitry316as well as determination of the enhancement factor, reduction factor, and minimum color adjustment factor are discussed below with respect to sub-process block376of the process350A as well asFIG.20.

Returning toFIG.19Aand the discussion of the process350A, at sub-process block362, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine a value of the function ƒ for a pixel (e.g., the pixel for which the modified middle color component354is to be generated). As described above, such a value may be determined using Equation 1, in which ƒ is equal to the difference of the values of the medium color component and minimum color component divided by the difference of the values of the maximum color component and the minimum color component.

At sub-decision block364A, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine whether the value of the function ƒ is greater than, less than, or equal to the threshold. Referring briefly toFIG.18, the image data processing circuitry300may include pixel modification circuitry318, which may include adders, multipliers, and other circuitry that can perform mathematical operations on values such as the color components (maximum, middle, and minimum color components) as well as the values of the regionalization configuration settings. The pixel modification circuitry318may also perform the comparison performed at sub-process block364A.

Returning toFIG.19A, when the value of the function ƒ is determined to be greater than the threshold, at sub-process block366, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may generate a modified middle color component by increasing the value of the middle color component. More specifically, the pixel modification circuitry318may modify the middle color according to Equation 2 provided below:

where Cmid,modifiedis the modified middle color component354, Cminis the minimum color component (e.g., as received at process block272of the process270), Cmaxis the maximum color component (e.g., as received at process block272of the process270), T is the value of the threshold for the region52, and Fenhanceis the value of the enhancement factor for the region. The enhancement factor, which may be determined by the regionalization setting determination circuitry316, may be determined as provided below in Equation 3:

where Cmidis the middle color component (e.g., as received at process block272of the process270) and P is the value of the power for the region52.

At sub-process block368, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the modified middle color component354generated at sub-process block366.

However, when the value of the function ƒ is determined to be less than the threshold, at sub-process block370, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may generate a modified middle color component by decreasing the value of the middle color component. More specifically, the pixel modification circuitry318may modify the middle color according to Equation 4 provided below:

where Cmin,modifiedis the modified middle color component354, Cminis the minimum color component (e.g., as received at process block272of the process270), Cmaxis the maximum color component (e.g., as received at process block272of the process270), T is the value of the threshold for the region52, and Freduceis the value of the reduction factor for the region52. The reduction factor, which may be determined by the regionalization setting determination circuitry316, may be determined as provided below in Equation 5:

where Cmidis the middle color component (e.g., as received at process block272of the process270) and P is the value of the power for the region52.

At sub-process block368, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the modified middle color component354generated at sub-process block370.

Furthermore, when the value of the function ƒ is determined to be equal to the threshold, at sub-process block370, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may utilize the middle color component as the modified middle color component. At sub-process block368, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the middle color component as the modified middle color component354.

Continuing with the discussion of the process350A, at process block356, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may perform a minimum color component modification to generate the modified minimum color component358. As discussed below, several operations may performed at process block356.

For example, at sub-process block374, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine a minimum color adjustment value for a region52(e.g., the region52identified at sub-process block290of the process270). As noted above, the minimum color adjustment value may correspond to a setting of a slider144(e.g., a c_min slider) as indicated by a user input or a default value (e.g., when no user input has been made using the slider144). In one embodiment, the minimum color adjustment value may be a value between zero and one, inclusive.

At sub-process block376, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine a minimum color factor based on the minimum color adjustment value. The minimum color factor may be determined according to Equation 6:

where FCminis the minimum color factor, Cmidis the middle color component (e.g., as received at process block272of the process270), Cminis the minimum color component (e.g., as received at process block272of the process270), Cmaxis the maximum color component (e.g., as received at process block272of the process270), and ACminis the minimum color adjustment value.

The minimum color factor may be determined by the regionalization setting determination circuitry316. In some embodiments, the regionalization setting determination circuitry316may be implemented by executing computer-readable instructions (e.g., instructions stored in the local memory20or main memory storage device22) using the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28. In other embodiments, the regionalization setting determination circuitry316may be implemented physically. For example,FIG.20is a block diagram of an embodiment of the regionalization setting determination circuitry316, which may be separate from or included (partially or wholly) within the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28. It should be noted that the input values shown inFIG.20pertain to process350B ofFIG.19B, which, as discussed below, is an alternative process for generating modified image data that may be utilized in lieu of the process350A. As such,FIG.20is discussed in more detail below in relation to the process350B.

Returning toFIG.19Aand the discussion of the process350A, at sub-process block378, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine a modified minimum color component based on the minimum color factor (sub-process block378). More specifically, the pixel modification circuitry318may determine the modified minimum color component in accordance with Equation 7:

where Cmin,modifiedis the modified minimum color component358.

At sub-process block380, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the modified minimum color component358generated by performed the operations discussed above with respect to sub-process blocks374,376,378. Accordingly, the process350A may be performed to generate modified image data that includes the modified middle color component354and modified minimum color component358.

Turning now toFIG.19B, as noted above, modified image data may also be generated in accordance with process350B. In other words, the process350B may be another manner in which modified image data is generated at sub-process block292of the process270. Accordingly, the process350B may be performed as sub-process block292of the process270, portions of the process350B may be performed as sub-process block292, or the process350B may be performed as part of sub-process block292of the process270discussed above with respect toFIG.17. The process350B may be performed for each pixel of image data that is to be modified (e.g., as determined as decision block274of process270). The process350B may also be performed on a region-wide basis, meaning the process350B may be performed for each region52of the color wheel50for which there is image data to be modified. In some embodiments, the process350B may be implemented at least in part based on circuit connections formed (e.g., programmed) in the processor core complex18, the image processing circuitry28, or both (e.g., partially or completely) the processor core complex18and the image processing circuitry28. The process350B may also be implemented using image data processing circuitry300as well as the circuitry discussed below with respect toFIG.20. Additionally or alternatively, the process350B may be implemented at least in part by executing instructions stored in a tangible non-transitory computer-readable medium, such as the local memory20, using processing circuitry, such as the processor core complex18, image processing circuitry28, or both the processor core complex18and the image processing circuitry28.

The process350B is generally similar to the process350A in that the process350B shares several operations in common with the process350A. For example, the process350B also generally includes performing a middle color component modification (process block352B) to generate a modified middle color component354and performing a minimum color component modification (process block356) to generate a modified minimum color component358. In the process350B, minimum color component modification (process block356) may be performed in the same manner as discussed above with respect to the process350A. However, the modification of the middle color component (process block352B) may be performed in a similar yet different manner than process block352A of the process350. For example, process block352B includes sub-process blocks366,368,370, each of which may be performed as discussed above with respect to process block352A of the process350A. However, in the process350B, the decision of when to modify the middle color component is different (e.g., sub-process block364B), and, as can be gleaned from comparingFIG.19AtoFIG.19B, the process350B does not include sub-process blocks360,362,372.

In particular, at sub-process block364B, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may determine whether a difference of the middle color component and the minimum color component is greater than a product of the value of the threshold for a region52(e.g., the region52determined at sub-process block290of the process270) and a difference of the maximum color component and the minimum color component. Such a determination may be made by the pixel modification circuitry318. When the difference of the middle color component and the minimum color component is greater than the product of the value of the threshold for a region52and the difference of the maximum color component and the minimum color component, at sub-process block366, the modified middle color component354may be generated as discussed above with respect to the process350. However, when the difference of the middle color component and the minimum color component is less than or equal to the product of the value of the threshold for a region52and the difference of the maximum color component and the minimum color component, at process block370, the modified middle color component354may be generated as discussed above with respect to the process350. As such, in the illustrated embodiment of process block352B, the middle color component will either be increased or reduced to generate the modified middle color component354. Thus, when performing sub-process block292of the process270(as discussed above with respect to the processes350A,350B), it is possible that only the middle color component and minimum color component may be modified when generating modified image data. In other words, the maximum color component may not be modified.

It should also be noted that, in some embodiments, the processes350A,350B may include fewer operations than those described below. As such, the processes350A,350B may be performed using only a portion of the operations discussed above with respect toFIG.19AandFIG.19B.

Having discussed how modified image data may be generated, the discussion will now return toFIG.20and how the regionalization setting determination circuitry316may generate the minimum color factor (FCminabove and “C_min factor” inFIG.20) used to generate the modified minimum color component358as well as the enhancement factor (Fenhanceabove and “enhancement factor” inFIG.20) and reduction factor (Freduceabove and “reduction factor” inFIG.20) that may be used to generate the modified middle color component354.

For example, the regionalization setting determination circuitry316may include a look-up table400that receives a value equal to a difference between the value of the maximum color component and the minimum color component. The difference may be a value between zero and 255, inclusive. As such, the look-up table400may be a 256-value look-up table. The look-up table may output a value equivalent to the reciprocal of the received value.

The regionalization setting determination circuitry316also includes multiplier402, which may receive the reciprocal value output by the look-up table400. As indicated inFIG.20, the regionalization setting determination circuitry316may also include round and shift circuitry and one's complement circuitry.

In the case of determining the minimum color factor, the multiplier402may output the reciprocal value. In other words, when determining the minimum color factor, the multiplier402may output the received reciprocal value without performing multiplication (or instead by multiplying the reciprocal value by one). The reciprocal value (e.g., as rounded and shifted) may be received by minimum color factor circuitry404, where more operations may take place to generate the minimum color factor. More specifically, the minimum color factor circuitry404may include a multiplier406that receives the reciprocal value and the value of the minimum color adjustment, determines a product of the reciprocal value and the value of the minimum color adjustment, and output the product. The product (or the one's complement of the product) may be multiplied a value equal to the difference between the middle color component and the minimum color component by multiplier408of the minimum color factor circuitry404to generate a second product. This second product (e.g., as rounded and shifted) may be output as the minimum color factor.

In the case of determining the reduction factor and the enhancement factor, the multiplier402may multiply the reciprocal value received from the look-up table400by a value received from a multiplexer410. In particular, the multiplexer receives two values as inputs as well as another value (e.g., a zero or one). The two values received as inputs may be equivalent to the values of the numerators of fractions in Equation 3 and Equation 5. The other value is indicative of whether a difference of the middle color component and the minimum color component is less than or equal to a product of the threshold for the region52(e.g., as determined at sub-process block290) and a difference of the maximum color component and the minimum color component. For example, when the difference of the middle color component and the minimum color component is less than or equal to the product of the threshold for the region52and the difference of the maximum color component and the minimum color component, the multiplexer410may receive a one and output the received input that is equivalent to the value of the numerator of the fraction in Equation 5. When the difference of the middle color component and the minimum color component is greater than the product of the threshold for the region52and the difference of the maximum color component and the minimum color component, the multiplexer410may receive a zero and output the received input that is equivalent to the value of the numerator of the fraction in Equation 3. The product generated by the multiplier402may be output, and middle color modification circuitry412may receive the product (e.g., as rounded and shifted).

The middle color modification circuitry412may perform further operations on the received product and ultimately output the reduction factor or enhancement factor (depending on how the middle color value will be modified). For example, multiplier414may receive the product as well as an output of multiplexer416. The multiplexer416may receive two input values (e.g., the reciprocal of the threshold of the region52and the reciprocal of the difference of one and the threshold of the region52). The multiplexer416may also receive a signal (e.g., a zero or one, with the value being as the value utilized by the multiplexer410to determine which input to select) and selectively choose one of the input values as the output. The multiplier414may determine a second product by multiplying the product received from the multiplier402(e.g., as rounded and shifted) and the value received from the multiplexer416and also output the second product. Depending on whether enhancement factor or the reduction factor is being determined, the second product (e.g., as output or as rounded and shifted or a one's complement of the product as rounded and shifted) may respectively be equal to the term of Equation 3 or Equation 5 that is to be raised to the power P.

Exponent circuitry418of the middle color modification circuitry412may perform further operations on the second product (e.g., as output or as rounded and shifted or a one's complement of the product as rounded and shifted) and output the reduction factor or the enhancement factor (depending on which is being determined). As illustrated, multiplexer420may receive one input that is a rounded and shifted second product as well as another input that is the one's complement of the rounded and shifted second product. The multiplexer420may also receive a (e.g., a zero or one, with the value being as the value utilized by the multiplexer410to determine which input to select) and selectively choose one of the input values as the output. The output of the multiplexer420may be received by multiplier422and squared (i.e., multiplied by itself) by the multiplier422.

Multiplexer424may receive the product generated by the multiplier422(e.g., as rounded and shifted) as an input as well as the output of the multiplexer420as another input. The multiplexer424may also receive a signal (e.g., a one or zero) indicative of whether the power value (i.e., P in Equation 3 or Equation 5, depending on whether the reduction factor or enhancement factor is being determined (with the value of P itself being dependent upon which region52is identified at sub-process block290of the process270)) is equal to four. When the power value is equal to four, the multiplexer424may output the product (e.g., as rounded and shifted) generated by the multiplier422. When the power value is not equal to four, the multiplexer424may output the value received from the multiplexer420.

In this manner, multiplier426may either determine a product equivalent to the output of the multiplexer420raised to the third power or the output of the multiplexer420raised to the fourth power. More specifically, the multiplier426may receive the product (e.g., as rounded and shifted) output from the multiplier422(which is equivalent to the output of the multiplexer420raised to the second power) and the output of the multiplexer424, which is either the output of the multiplexer420(when the power value is three) or the product output by the multiplier422(when the power value is four). The multiplier426may then multiply the received values and output a product that is equivalent either to the output of the multiplexer420raised to the third power or the output of the multiplexer420raised to the fourth power (depending on whether the power value is equal to four).

A multiplexer428may receive the product (e.g., as rounded and shifted) generated by the multiplier422as well as the product (e.g., as rounded and shifted) generated by the multiplier426and selectively output one of the received products based on a control signal (e.g., a zero or one) that is indicative of whether the power value is two. When the power value is two, the multiplexer428may output the product (e.g., as rounded and shifted) generated by the multiplier422. When the power value is not two, the multiplexer428may output the product (e.g., as rounded and shifted) generated by the multiplier426. When determining enhancement factor, the enhancement factor may be equivalent to the one's complement of the output of the multiplexer428, whereas, when determining the reduction factor, the reduction factor may be the output of the multiplexer428. In this manner, the regionalization setting determination circuitry316may determine the values of the minimum color adjustment factor, reduction factor, and enhancement factor.

Returning toFIG.17and the discussion of generating adjusted image data (process block280) in the process270, at sub-process block294, the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may perform a luma adjustment to generate luma-adjusted pixel data. Luma-adjusted pixel data may be generated for each pixel of the electronic display12(e.g., that will be utilized to display image data such as a frame of image content). Referring toFIG.18, luma adjustment circuitry320may receive modified color component values generated by the pixel modification circuitry318(e.g., modified middle color component354, modified minimum color component358, and the maximum color component) as well as pixel data indicative of the color components of the original image data (e.g., as output by degamma circuitry310). Turning now toFIG.21, the luma adjustment circuitry320may include a look-up table440that receives a value for the luma of a modified pixel. The modified pixel may have RGB values based on values generated by degamma circuitry310and pixel modification circuitry318. For example, the maximum color component of the modified pixel may be the maximum color component generated by degamma circuitry310(e.g., a value unmodified by the pixel modification circuitry318), whereas the middle color component and minimum color component of the modified pixel may respectively be the modified middle color component354and modified minimum color component358generated at sub-process block292of the process270. In one embodiment, the value of the luma for the modified pixel may be determined by the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28in accordance with Equation 8:

where Luma is the luma, R is the value of the red component of the modified pixel, G is the value of the green component of the modified pixel, and B is the value of the blue component of the modified pixel. The look-up table440may output a value that is the reciprocal of the luma of the modified pixel.

The luma adjustment circuitry320also includes a multiplier442that may multiply the value output by the look-up table440by the luma of the unmodified pixel. The luma of the unmodified pixel may be determined by the processor core complex18, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28using Equation 8 in which the RGB values generated by the degamma circuitry310are utilized.

The product determined and output by the multiplier442may then be multiplied by multipliers444(referring collectively to multipliers444A,444B,444C). More specifically, multiplier444A may multiply the product generated by the multiplier442and the value of red value (i.e., the R value of the RGB values of the modified pixel), multiplier444B may multiply the product generated by the multiplier442and the value of green value (i.e., the G value of the RGB values of the modified pixel), and multiplier444C may multiply the product generated by the multiplier442and the value of blue value (i.e., the B value of the RGB values of the modified pixel). The values generated by the multipliers444may be rounded and shifted by rounding and shifting circuitry446, and the rounded and shifted values may be output as luma-adjusted pixel data. Thus, the luma-adjusted pixel data may include luma-adjusted RGB values.

At sub-process block296, processor core complex18the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may perform gamma adjustment. For example, referring briefly toFIG.18, engamma look-up table322may receive the luma-adjusted pixel data generated by the luma adjustment circuitry320and may gamma encode the luma-adjusted pixel data by outputting RGB values based on the indicated by the engamma look-up table322. The values output at sub-process block296may be the adjusted image data that is utilized by the electronic display12to display content that has been modified based on user-selected settings (e.g., placement of the sliders144).

Returning toFIG.17, at process block278, the image processing circuitry28, or both the processor core complex18and the image processing circuitry28may output the adjusted image data. For example, the adjusted image data may be provided to the electronic display12, and the electronic display12may generate image content using the adjusted image data (e.g., to perform process block168of the process160). In some embodiments, the operations associated with sub-process block286(e.g., operations associated with denormalizing color components) may be performed immediately before the adjusted image data is output. As such, the image data (e.g., color components) that has been modified (e.g., as modified at one or more of sub-process blocks292,294,296) may be denormalized, and the denormalized color components may be output as the adjusted image data at process block278. The adjusted image data, when displayed, may result in images such as images70C,120C,140B.

Accordingly, the technical effects of the present disclosure include enabling electronic devices to generate image data that is modified (e.g., for users with color vision deficiency) in a user-specific manner to better enable users to discern between colors in displayed image content. Thus, the techniques described herein enable electronic device to generate improved image content.