Patent Description:
Commercial video production typically involves acquiring and editing video sequences. Before being distributed, colors in the video production may be adjusted in a color grading step. Color grading typically involves displaying the video production on a reference display under ideal viewing conditions and adjusting colors and tones in the video production to achieve a desired artistic intent. For example, a person operating color grading apparatus (a >color grader=) may decide to make the mood of a scene gloomy by reducing the brightness of the image. Artistic goals may also be met by making other adjustments, for example, changing the colors of certain displayed objects, changing the overall contrast in the scene, changing the overall brightness of the scene, and the like. The result, when displayed on the reference display, matches the artistic intent of the color grader.

A wide number of alternative display technologies are available commercially. These range from digital cinema displays to television sets which may be based on cathode ray tubes, plasma displays, LED displays, LCD panel displays in which the LCD panel is backlit by various types of backlight systems, and the like. Different displays have different capabilities. For example, some displays may have a broader color gamut than other displays. Some displays may be capable of reproducing a broader range of brightnesses and/or a higher maximum brightness than other displays. The inventors have recognized a need for methods and apparatus capable of transforming image data in a manner which can take advantage of the capabilities of a particular display without compromising the artistic intent represented by the images in the image data.

This need also extends to reproductions of still images by various types of printing, engraving, and other image reproduction technology in which the reproduction technology may have capabilities for reproducing colors and/or tones that are different from those assumed by the image data.

There remains a need for flexible and cost-effective tone and/or gamut mapping apparatus and methods. There is a particular need for such apparatus and methods that may be applied to produce enjoyable images on high quality displays.

<CIT> discloses a data storage format for storing images with extended color gamut that is compatible with limited color gamut imaging devices. The method for storing the images includes identifying regions within the color image that contain extended gamut color values, using a compact notation to store an identification of each extended color gamut region in a second section, gamut-mapping the image data pertaining to the extended gamut regions to a limited gamut color space, storing the gamut-mapped image data together with all image data not in an extended gamut region in a first section of the data storage format, and storing image data for the extended gamut regions in an extended gamut color space in a third section of the data storage format such that the limited color gamut image stored in the first section can be combined with the extended gamut data stored in the third section to produce an extended color gamut digital image.

<CIT>discloses a colour processing apparatus used for colour processing and converting colour data between colour gamuts. The apparatus comprises an inverse collector that performs inverse correction to mapped colour data based on correction region of output gamut. The apparatus further comprises an input image collector that determines whether a lightness value of a pixel of interest falls inside the lightness range corresponding to the correction region of the input gamut. When the lightness value of the pixel of interest falls inside the lightness range, the input image collector corrects the colorimetric value of the pixel of interest.

<CIT> discloses a digital imaging system wherein tone correction is applied using a blending mask to select an appropriate tone correction curve so as a high contrast and high dynamic range image with areas lit by multiple light sources can be rendered properly on a display.

A range of aspects of the invention and example embodiments of the invention are described below and illustrated in the accompanying drawings.

The appended drawings illustrate non-limiting example embodiments of the invention.

Throughout the following description specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense.

<FIG> is a high level block diagram illustrating a tone and gamut mapping apparatus <NUM> according to an embodiment of the present disclosure. Components of apparatus <NUM> (e.g. such as those shown in <FIG>) are located at the display end. Certain other components of apparatus <NUM> (e.g. such as those shown in <FIG>) may be located at a color grading tool, upstream of the display, and may operate with a color grader toolset (e.g. a set of color grading functions provided by the color grading tool) to drive the metadata creation. A color grading tool may comprise, for example, software and/or hardware-based systems operable by a color grader to receive input image data and to adjust colors and/or luminance of pixel data in areas of the input image data to achieve a desired artistic effect for the image.

At the display end, apparatus <NUM> has an input <NUM> at which image data <NUM> is received. Image data <NUM> may comprise video data, data specifying one or more still images or the like. Input <NUM> may comprise a physical input, such as a serial or parallel data bus, a wireless data input, a memory location from which image data <NUM> may be accessed, an interface which receives image data <NUM> from some data communication channel, hardware for retrieving image data <NUM> from a medium, such as a DVD, CD-ROM, flash memory, or other data record, or the like.

Image data <NUM> comprises pixel data <NUM> for pixels in an image to be displayed. Each set of pixel data <NUM> comprises a number of color coordinates, in this example C1, C2, and C3, which indicate the color and tone for a pixel. Pixel data <NUM> also includes either explicitly or implicitly location information, in this example identified by the coordinates X and Y, for a pixel represented by pixel data <NUM>. Location information X, Y may be represented by an offset or other relative location within image data <NUM> of a particular pixel data <NUM>. In the example embodiment shown in <FIG>, location information X and Y are shown explicitly.

Apparatus <NUM> optionally includes a coordinate transform block <NUM> which transforms coordinates C1, C2, and C3 into a color space different from a color space of image data <NUM>. In some embodiments, it is convenient for apparatus <NUM> to work internally in a different color space. The different color space may be, for example, a linear color space (i.e. a color space in which a physical value such as brightness (e.g., lightness or intensity or luminance) is represented linearly). In such embodiments, coordinate transform block <NUM> may perform a translation into a linear color space. In certain embodiments, coordinate transform block <NUM> may perform a translation into IPT, LAB or RGB color space or another color space which is more suitable for tone or gamut mapping than the color space of image data <NUM>.

Block <NUM> receives pixel data <NUM> and performs a mapping of the pixel data <NUM> to mapped pixel data <NUM>=. The mapping may be performed through use of lookup tables, logic circuits which execute mapping algorithms, or the like. The particular mapping performed is specified by mapping parameters <NUM> which influence the operation of mapping logic <NUM>.

Mapped pixel data <NUM>= is output as mapped image data <NUM>=. Output of mapped image data <NUM>= may comprise, for example, placing mapped pixel data <NUM>= into a file or buffer, onto a serial or parallel data bus, transmitting the mapped pixel data <NUM>= supplying the mapped pixel data <NUM>= to a downstream device or circuit or the like.

In the illustrated embodiment, mapping parameters <NUM> include parameters <NUM> set by a configuration mechanism <NUM>. Configuration mechanism <NUM> sets mapping parameters <NUM> according to metadata received from a metadata extraction/reception unit <NUM>. In some embodiments, metadata is embedded in a signal, data structure, or other representation of image data <NUM> and is extracted by metadata extraction/reception unit <NUM> from the image data <NUM>. In other embodiments, metadata is received on a separate channel, in a separate data structure, or the like. In any event, the operation of mapping logic <NUM> is controlled by parameters <NUM> which may be set or selected, at least in part, in response to metadata.

In the illustrated embodiment, parameters <NUM> include general parameters 22A, color parameters 22B, and location parameters 22C. In this embodiment, the mapping performed by mapping logic <NUM> may vary depending upon the color coordinates represented in a pixel data <NUM> being mapped and/or the location information corresponding to the pixel <NUM> being mapped. Example applications of these capabilities are described below.

In an example embodiment, metadata <NUM> supplied to configuration unit <NUM> includes general metadata to be applied to all of or selected portions of video data <NUM>. For example, the general metadata may be specified for selected scenes or frames within video data <NUM>. Configuration unit <NUM> may be configured to establish (for example by selection or calculation) a set of mapping parameters <NUM> to use for mapping the image data <NUM> based upon that general configuration metadata. The general configuration metadata may be specific to a particular display or image reproduction technology within which apparatus <NUM> is embedded or which is downstream from apparatus <NUM>.

Metadata presented to configuration unit <NUM> also includes metadata which specifies particular mapping behaviour for particular colors. According to the present disclosure, the metadata specifies one or more reserved colors which should not be affected by the transformation of block <NUM>. Such reserved colors may be specified in parameters 22B. Mapping logic <NUM> is configured to make transformations in such a manner that reserved colors identified in parameters 22B are not changed by the transformation.

As another example, in some embodiments, the mapping specified by mapping parameters <NUM> may have the capability of mapping at least some colors to transformed colors outside of a gamut represented in image data <NUM>. In some embodiments, colors within a range of colors lying close to the boundary of gamut <NUM> may be subjected to a transformation which is different from a transformation applied to colors that are at color coordinates far from the boundary of the gamut in which image data <NUM> is represented.

As another example, one or more selected colors from image data <NUM> may be associated with corresponding target colors outside of the gamut of image data <NUM>. In such cases, mapping logic <NUM> may be configured to map the selected colors toward the corresponding target colors to a color that is within the display capabilities of the display or other target device.

Mapping logic <NUM> is also configured to perform different transformations depending upon the location information corresponding to pixel data (e.g. on the coordinates X, Y in the illustrated example). For example, metadata received by configuration unit <NUM> may specify the locations of light sources within a current image. Different mappings may be applied depending upon whether or not the location information corresponding to pixel data being processed indicates that the pixel data corresponds to a location which is in a light source or outside of a light source. Different mappings may be applied by adjusting the tone curves. For example, <FIG> illustrates a tone curve for standard video data (without any adjustments for light sources) and a tone curve adjusted for an area identified as a light source (the Aadjusted tone curve@). To ensure seamless adjustment, the adjusted tone curve may be matched to the tone curve for standard video data at the boundary between light source and non-light source areas. For example, as seen in <FIG>, toward the lower input pixel drive values (e.g. corresponding to areas outside of the area identified as the light source), the adjusted tone curve may be the same as the tone curve of the standard video data. At some value between the lower input pixel drive values and the upper input pixel drive values, the adjusted tone curve diverges from the tone curve of the standard video data. The adjusted tone curve may differ from the tone curve of the standard video data for the upper input pixel drive values, which may correspond to the area identified as the light source (e.g. upper input pixel drive values may be mapped to output values higher than for standard video data, as shown in <FIG>). At the upper input drive values, the adjusted tone curve defines the intensity of the pixels in the area identified as the light source. The adjusted tone curve of <FIG> shows how the intensity of pixels in the light source area may be adjusted, for one example embodiment. In other embodiments, the adjusted tone curve may be a different curve than shown in <FIG> (e.g. the adjusted tone curve may have a particular inflection point which may be positioned up or down the curve, for various embodiments). Depending on the desired effect, a particular adjusted tone curve may be applied to an image to affect how the intensity of the light source area of the image is adjusted, relative to other areas of the image.

In some embodiments, different mappings are applied depending upon the type of light source. For example, different mappings may be applied depending upon whether the light source is identified in the metadata as being a reflective light source or an emissive light source. In some embodiments, separate mapping algorithms and/or mapping parameters are provided for each of a plurality of different types of light sources. <FIG> illustrates example tone curves to be applied for areas identified as having emissive and reflective light sources and a tone curve for standard video data (without any adjustments for the light sources).

<FIG> represents an example useful for understanding the invention and illustrates components of apparatus <NUM> that may be located at the color grading tool. At the color grading tool, apparatus <NUM> receives input image data 14A at input 12A and outputs color graded image data 14B (which may include metadata <NUM>). Image data 14A, 14B may comprise video data, data specifying one or more still images or the like. Image data 14A may comprise pixel data 15A for pixels in an image to be displayed. The color grading tool may comprise a color grading processor <NUM>, which receives pixel data 15A and executes one or more color grading functions in response to user input so as to result in color graded pixel data 15B. Color grading functions may be implemented by software and/or hardware. Operation of color grading functions by processor <NUM> may be controlled by a professional color grader. For example, the color grader may decide to intensify certain colors, or alter luminance, saturation and/or hue of pixels in an image, and may provide appropriate user inputs to the color grading tool to cause selected color grading functions of the color grading tool to be applied to the pixel data 15A, so as to achieve the desired artistic effect.

In the illustrated embodiment, color grading processor <NUM> is communicatively coupled to a metadata generator <NUM>. Metadata generator <NUM> may have access to, or may generate, a set of metadata definitions <NUM> which may describe tone/gamut parameters. Metadata generator <NUM> may have access to color graded pixel data 15B. Metadata generator <NUM> may generate metadata <NUM> for transmission with the color graded pixel data 15B. Metadata generation may be based on image characteristics and adjustments to the pixel data by color grading processor <NUM> as communicated by color grading processor <NUM> and/or determined from color graded pixel data 15B. Metadata generator <NUM> may identify parameters of the color graded pixel data to drive metadata creation.

In the illustrated embodiment, metadata generator <NUM> includes a program memory <NUM> for storing software functions, which when executed by a processor <NUM> of metadata generator <NUM>, may identify parameters of the color graded pixel data such as, for example, locations of light sources in the image, colors which are intentionally color graded out of gamut, reserved colors, etc. Metadata generator <NUM> may use information about such parameters to generate metadata <NUM>. Metadata <NUM> generated by metadata generator <NUM> may be combined with or transmitted with color graded pixel data 15B and output as image data 14B to a downstream device. In certain embodiments, components of apparatus <NUM> as shown in <FIG> may receive image data 14B (<FIG>) as image data <NUM>.

<FIG> illustrates a color grading process according to an example useful for understanding the invention. A professional color grader <NUM> starts the process by controlling a reference display to display video data from an external video content source; such video data may have been created through camera capture, CGI (computer generated imagery), etc. In the color grading process, while viewing the displayed video data on the reference display, the color grader <NUM> uses a color grading tool <NUM> to adjust or modify a video frame 78A (or a sequence of frames (i.e. scene)) according to the creator=s artistic intent. Color grading processor <NUM> adjusts video frame 78A according to inputs from color grader <NUM> to produce color graded video frame <NUM>.

In the color grading process, portions of each frame may be modified differently for: particular image areas (e.g. a particular tone mapping may be applied to an image area corresponding to a light source location); special colors (e.g. certain colors or tones/gamut may be mapped according to a color-specific mapping); and/or reserved colors (e.g. adjustment of skin tones, grass tones, and other tones designated as reserved colors in the color grading process may be inhibited during tone/gamut mapping). Within a frame or scene, localized tone or gamut flags may be set on a metadata grid <NUM> defining the location of particular pixels for which the tone/gamut has been adjusted in the color grading process; and/or the location of particular pixels for which tone/gamut has deliberately not been adjusted in the color grading process (e.g. as may be the case for reserved colors). The localized tone or gamut flag may identify information about the tone/gamut adjustment made to the pixel. Color graded video content, localized tone or gamut flags and pixel locations corresponding to each localized tone or gamut flag may be saved to a storage or buffer device <NUM>. Localized tone or gamut flags and the information indicated by such flags may be saved into a frame metadata container <NUM> as accumulated localized metadata.

A codec <NUM> receives the metadata stored in metadata container <NUM>, and color graded content in video frame <NUM> and local tone/gamut adjustments (as stored in storage device <NUM>), and encodes the metadata and color graded content for distribution. Methods for encoding by codec <NUM> may include, for example:
Encoding accumulated localized metadata and placing into metadata at the start of the video picture or scene. The video and frame metadata may be encoded and may follow the accumulated metadata. The frame metadata may be formatted to index the accumulated localized metadata to minimize the amount of data required to transmit the metadata.

$ Ignoring the accumulated metadata and encoding the frame metadata along with the color graded video (this may increase overall bandwidth).

<FIG> illustrates a portion of a video delivery pipeline which begins with color grading stage <NUM> and ends with display tone/gamut mapping stage <NUM>. Other stages preceding or succeeding either of stages <NUM>, <NUM> of the video delivery pipeline are not shown. Components of apparatus <NUM> shown in <FIG> and/or color grading tool shown in <FIG> may be used at the color grading stage <NUM> to generate metadata for tone/gamut mapping purposes at downstream devices. The metadata may be delivered along with video data to a codec stage <NUM> for encoding and/or decoding the video data and metadata. In some embodiments, the metadata may be delivered by a separate communication channel. The video data and metadata are delivered to a downstream device for tone/gamut mapping at stage <NUM>. Components of apparatus <NUM> shown in <FIG> may be used at tone/gamut mapping stage <NUM> to implement the tone/gamut mapping methods described herein.

<FIG> illustrates a method <NUM> which may be applied to mapping image data <NUM> to mapped image data <NUM>=. In block <NUM>, metadata is obtained. The metadata may be extracted from a video stream, a file containing video data, a separate data structure, received on a separate data stream, or the like. Metadata may be obtained prior to and/or during obtaining image data. Block <NUM> may be performed continuously during processing of image data in some embodiments.

In block <NUM>, mapping logic is configured in response to received metadata. The configuration may include, for example, writing values to lookup tables used for mapping color values and/or tone values; recording reserved colors, identifying special color ranges having separate mappings, or modified mappings; identifying special locations having specific mappings or modified mappings; writing parameter values to registers accessed by mapping algorithms; configuring configurable logic implement mapping algorithms or the like. Block <NUM> may be repeated each time updated metadata is received.

Metadata may include metadata that affects the manner in which color/luminance values for pixel data are shown on a display or the manner in which mapping is performed from the transmitted color/luminance values to the mapped color/luminance values (in either the same or a different color space). Metadata may also include metadata that assists in abbreviating metadata or reducing the amount of data required to transmit metadata. For example, in some embodiments, the metadata may define tone/gamut maps for later use in tone/gamut mapping. Subsequent metadata may indicate that for certain pixels, one of the predefined gamut maps is to be used. The predefined maps may be stored or buffered at the display end of a video delivery pipeline in a data structure that may be referenced by the mapping apparatus. The data structure may be, for example, a one-dimensional or two-dimensional data structure.

According to the present disclosure, the metadata defines metadata flags providing information about tone or gamut adjustments made in the color grading process. Information about tone or gamut adjustments identify: colors that have not been adjusted because they are reserved colors; optionally: colors that have been deliberately color graded out-of-gamut; special colors to be mapped according to a color-specific map, etc. The metadata flags and corresponding tone or gamut adjustment information may be stored in a data structure (e.g. a look-up table) that may be stored or buffered at the display end and later referenced by the mapping apparatus (e.g. see <FIG> showing localized tone or gamut flags set at pixels on a metadata grid corresponding to a frame of image data, and a predefined gamut metadata structure). Subsequent metadata transmitted to the display end may indicate that particular localized tone or gamut flags correspond to certain pixels in the pixel data. Such information may in turn be used by the mapping apparatus to look up in the stored metadata structure the tone or gamut adjustments made to those pixels, and to select and apply a tone/gamut mapping method based on such tone or gamut adjustments.

Method <NUM> includes a data processing sequence 25B which performs mappings of pixel data. Sequence 25B may be repeated for each pixel data <NUM> in image data <NUM>. In block <NUM>, pixel data is received. Block <NUM> may, for example, comprise retrieving a next pixel data from a buffer; retrieving pixel data from a memory; receiving pixel data on an input line such as a parallel or serial bus; or the like. In block <NUM> the color coordinates of the pixel data are compared to a set of zero or more reserved colors. If there are no reserved colors then block <NUM> is optional and need not be performed. If the color coordinates of the pixel data being processed correspond to a reserved color, then either the pixel data is passed through unmapped as indicated by path <NUM>, or, the pixel data is passed through block <NUM> which performs mapping of the tone of the pixel data only (for example, block <NUM> may expand a dynamic range of the pixel data or otherwise change a luminance value of the pixel data to suit a downstream application). In certain embodiments, the tone is preserved for pixels that represent particular objects or areas of the image (e.g. pixels representing skin, grass, sky, etc.). Accordingly, reserved colors may correspond to those used to depict such objects. Reserved colors may be set by a color grader at the color grading stage (e.g. the color grader may designate certain tones as skin tones). During mapping of the pixel data, tones that are within a predetermined margin of the reserved colors may be preserved.

In block <NUM>, location information corresponding to the pixel data is compared to a set of zero or more special locations. If no special locations have been specified by metadata, then block <NUM> is optional and need not be performed. Where special locations have been identified then block <NUM> determines whether the location information for the pixel data corresponds to a special location and, if so, makes active a location-specific mapping for the pixel data in block <NUM>.

According to the present disclosure, special locations are be associated with different types of mapping algorithms. For example, different mapping algorithms (or different variations of a mapping algorithm) may be provided for different types of light source. For example, different mapping algorithms may be provided for light emissive objects and for reflective objects. In some embodiments, pixel data <NUM> is mapped according to: a default algorithm; an algorithm for emissive objects; or an algorithm for reflective objects depending upon position-specific metadata for the pixel to which the pixel data <NUM> relates. In such embodiments, block <NUM> may comprise selection from among a plurality of algorithms.

In block <NUM> the location specific mapping is applied to the pixel data to obtain mapped pixel data.

In block <NUM> the color coordinates of the pixel data (which may include a tone or luminance value) are compared to a set of zero or more special color regions. The special color regions may consist of single points in color space or regions in the color space. If no special colors have been specified, then block <NUM> is optional and may be omitted. If the color coordinates of the pixel data correspond to a special region of the color space then in block <NUM> a color-specific map is selected. In block <NUM> the color-specific map is applied to map the pixel data to mapped pixel data.

In block <NUM>, pixel data which does not correspond to a reserved color, a special location, or a special color is mapped according to a general mapping. In block <NUM>, mapped pixel data is output. Block <NUM> may comprise, for example, placing mapped pixel data onto a serial or parallel data bus, sending mapped pixel data across a wired or wireless communication channel, applying mapped pixel data to further image processing displaying or reproduction hardware, writing mapped pixel data to a memory, or the like.

In some embodiments, the metadata includes an image histogram or other image statistics. A mapping algorithm may use this information, for example an average luminance of a current image to control tone and/or gamut mapping. In some embodiments the image statistics are available to data processing sequence 25B before pixel data for the current image is mapped. This can permit mapping of pixel data to be based on statistics for an image even where the mapping is commenced before all data for the image is received. In some embodiments mapping is local (i.e. each set of pixel data <NUM> is mapped without regard to the values of pixel data <NUM> for other pixels in the image).

The image histogram that is provided by metadata for a frame of image data may identify, for example, the number of pixels for each color channel having a luminance value within a luminance range associated with each one of the histogram buckets (e.g. <NUM><NUM>-bit histogram buckets may be provided for each color channel). The image histogram(s) and/or image statistics may be used to analyze the image data. Tone/gamut mapping may be performed based on such image data analysis. For example, in some embodiments, a special tone/gamut mapping algorithm may be performed based on an assessment of brightness or dimness of the overall image, or the significance of bright or dim areas relative to average luminance, or other criteria as extracted from the histogram or image statistics. In some embodiments, tone/gamut mapping may comprise, for example, achieving a balance between bright and dim image areas by: adjusting the tone curve to be applied so that less emphasis is placed on objects in the bright image areas, or adjusting the tone curve to be applied so that more emphasis is placed on objects in dim image areas.

In some embodiments, general metadata includes information specifying a selected mode of operation for a tone/gamut mapping apparatus. This feature may be exploited, for example, to allow a director to exert control over the tone/gamut mapping algorithm(s) applied at a television or other downstream device, if the implementation of the mapping algorithm(s) are standardized for the upstream and downstream devices. In some embodiments the selected mode of operation is identified by a code indicating generally the nature of the mode. For example, some televisions which incorporate tone/gamut mapping apparatus as described herein may support selection among Avivid@, Acinema@, Astandard@ and Apro@ modes. Different televisions may implement supported modes in their own ways.

<FIG> shows apparatus <NUM> according to another example useful for understanding the invention. Apparatus <NUM> comprises a metadata extractor <NUM> that reads metadata <NUM> from image data <NUM> or some other source. A control <NUM> controls a switch <NUM> in response to the detected metadata. For example, control <NUM> may set the position of switch <NUM> in response to a desired mode indicated by metadata <NUM>.

Switch <NUM> may comprise a physical switch or a logical switch. For each position of switch <NUM> a different set <NUM> of mapping algorithms is made available to mapping logic <NUM>. The illustrated embodiment provides sets 56A, 56B and 56C (collectively sets <NUM>). Each set <NUM> comprises, for example, a general algorithm 57A, a plurality of algorithms for different specific types of light sources 57B-<NUM>, 57B-<NUM>, and a plurality of algorithms for specific colors or color regions 57C-<NUM>, 57C-<NUM>.

In some embodiments, televisions or other downstream devices may incorporate tone/gamut mapping apparatus configured to receive specification for a mapping algorithm to be applied to video data <NUM>. The specification may be of certain mapping parameters such as look-up table values, parameters which control some aspect of operation of tone and/or gamut mapping algorithm(s) implemented by the apparatus, or specification of one or more special algorithms to be executed by the apparatus in mapping tone and/or gamut of image data <NUM>. The look-up tables can be set upon initialization, and metadata may be used to index through the look-up tables to select the desired tone/gamut mapping. Initialization may be performed with a fixed set of data (which may include data that is not built in at the receiving end but is supplied by the user) or through the transmission of metadata.

The order of operations in method <NUM> may be altered. For example, determination of whether a piece of pixel data being processed corresponds to a special location may be performed after determining whether the pixel data corresponds to a special region in color space. In a further example embodiment, shown in <FIG>, if a pixel data is identified as corresponding to a special location, then a block 40A is performed to determine whether the pixel data also corresponds to a special point or region in color space. If so, then a map that is both location-specific and color-specific is selected in block 42A and this map is then applied to map the color coordinates for the pixel in block 43A.

In an example application, a video signal has been color graded to be displayed on a "standard color television". However, the video signal will be displayed on a television having a gamut that is broader than that of the standard television and also has a greater dynamic range than the standard television. In this application, it may be desirable to slightly boost the dynamic range of the input video data and then slightly expand the gamut of the input video data in order to take advantage of the capabilities of the display. A default mapping which provides suitable expansion of dynamic range and gamut is provided, for example, in the form of a set of lookup tables. To facilitate this, the image data may be presented or transformed into a color space which includes hue and lightness coordinates.

During color grading, certain colors, such as, for example, a range of colors that appear in flesh of people depicted in the video, are identified as being reserved colors. The metadata identifies these reserved colors so that the mapping will not alter the flesh tones. Additionally, either in the preparation of the video content or at a subsequent location upstream from a mapping apparatus, the video images are processed to locate light sources within the images. Metadata identifying locations of those light sources (and, in some cases, types of the light sources) is generated.

Pixel location information is compared to the locations of light sources. If a pixel data being processed corresponds to the location of a light source, then a different mapping may be applied. In some embodiments, the selected mapping for locations corresponding to light sources expands dynamic range more aggressively than mappings applied to areas that are not identified as corresponding to light sources.

<FIG> illustrates mapping logic <NUM> according to an example useful for understanding the invention. Pixel data <NUM> being mapped is received in a buffer or memory <NUM>. A comparison unit <NUM> compares color coordinates of pixel data <NUM> to a set of reserved color coordinates <NUM> in a buffer or memory <NUM>. A second comparison unit <NUM> compares location information of the pixel data to a set of special locations <NUM> in a buffer or memory <NUM>. Algorithm selection logic <NUM> selects an appropriate mapping algorithm for the pixel data based on outputs of comparison units <NUM> and <NUM>. The illustrated embodiment has several parallel mapping pipelines <NUM> each configured to execute a different mapping algorithm. Algorithm selection logic <NUM> determines which one of pipelines <NUM> will be used to process the pixel data <NUM>. Mapped pixel data <NUM>= is written to a memory or buffer <NUM>.

Selection among pipelines <NUM> may be achieved in any suitable manner. Some examples are:.

As a further alternative, algorithm selection unit <NUM> may be configured to reconfigure a programmable or otherwise configurable logic pipeline on the fly to perform a mapping algorithm corresponding to the outputs of comparison units <NUM> and <NUM>. Comparison units <NUM> and <NUM> may have but are not limited to having two-state outputs. In some embodiments, outputs of one or both comparison units <NUM> and <NUM> can have three or more states.

Some specific ways in which the methods and apparatus according to this invention may be made available include:.

Claim 1:
A method implemented in an image processing apparatus located at a display end for mapping tone, gamut or both tone and gamut of pixel data in image data, the method comprising:
receiving the pixel data for an image;
receiving metadata, the metadata comprising:
information specifying a set of one or more special locations, wherein a special location refers to a particular image area in the pixel data that has been identified in a color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular image area,
information specifying a set of one or more special color regions, wherein a special color region refers to a particular color region in the pixel data that has been identified in the color grading process to be tone, gamut or both tone and gamut mapped according to mapping parameters associated with the particular color region, and
information specifying a set of one or more reserved colors, wherein a reserved color refers to a particular color in the pixel data that has been identified in the color grading process to be unmapped or tone mapped according to mapping parameters associated with the particular color;
mapping the pixel data to mapped pixel data according to an algorithm based at least in part on the mapping parameters determined from the metadata, wherein mapping the pixel data comprises comparing color coordinates for the pixel data to a set of one or more reserved colors, and in response to the color coordinates matching one of the set of reserved colors, inhibiting gamut mapping of the pixel data,
wherein mapping the pixel data comprises comparing location information corresponding to the pixel data to a set of the one or more special locations, and in response to the location information matching one of the set of one or more special locations, performing one or both of tone and gamut mapping the pixel data, and
wherein mapping the pixel data comprises comparing color coordinates of the pixel data to a set of one or more special color regions, and in response to the color coordinates corresponding to one of the set of special color regions, selecting a color-specific map for tone, gamut or both tone and gamut mapping the pixel data, and mapping the pixel data according to the selected color-specific map.