Patent Description:
In vehicle-mounted HUDs, it is a key point to improve the viewability (visual recognizability) of a display image. In vehicle-mounted HUDs, light for projecting a display image is reflected by a combiner to generate a virtual image, and a driver visually recognizes the generated virtual image. In this configuration, background such as a road surface is viewable through the generated virtual image. As the level of brightness in the background changes depending on the time periods or environment, the viewability significantly decreases when the brightness of the display of the HUD is low compared with the brightness of the background.

A configuration is known that an illuminance sensor or the like is used to measure the light quantity of environmental light and the brightness of a display is adjusted according to the measured light quantity of environmental light so as to prevent the viewability of a heads-up display (HUD) from decreasing (see, for example, <CIT>).

However, in known art of adjusting the brightness of the display, there is a problem that the viewability decreases as the precision of the measurement of the brightness of background light is insufficient.

<CIT> discloses a virtual image display device for an automobile.

According to one aspect of the present disclosure, an image display apparatus is provided that achieves viewability regardless of surrounding environment and improves the visual recognizability in comparison to the related art.

A more complete appreciation of exemplary embodiments and the many attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings.

The accompanying drawings are intended to depict exemplary embodiments of the present disclosure and should not be interpreted to limit the scope thereof.

It will be further understood that the terms "includes" and/or "including", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

In describing example embodiments shown in the drawings, specific terminology is employed for the sake of clarity. However, the present disclosure is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have the same structure, operate in a similar manner, and achieve a similar result.

Some embodiments of the present disclosure are described below with reference to the drawings. Note that like reference signs denote like elements in the description of the embodiments for the purpose of simplification.

In the following description of the basic embodiment, the hardware configuration of an on-vehicle HUD <NUM>, which is used in the examples/embodiments of the present disclosure, is described.

<FIG> is a front view of a virtual image displayed in a display area <NUM> over the sight ahead of a vehicle <NUM> viewed by a driver <NUM> through a front windshield <NUM>, in the on-vehicle HUD <NUM> according to the present basic embodiment.

<FIG> is a schematic partially-cutaway side view of the configuration of a car for which the on-vehicle HUD <NUM> according to the present embodiment is provided.

<FIG> is a schematic block diagram of the internal structure of an optical system of the on-vehicle HUD <NUM> illustrated in <FIG>.

In <FIG>, the on-vehicle HUD <NUM> according to the present embodiment is installed, for example, in the dashboard of the vehicle <NUM> that serves as a mobile object. The projection light L, which is the light for projecting an image, that is emitted from the on-vehicle HUD <NUM> disposed in the dashboard is reflected by the front windshield <NUM> that serves as a light transmission member, and is headed for the driver <NUM>. Accordingly, the driver <NUM> can visually recognize a HUD display image such as a navigation image, which will be described later, as a virtual image G. Note that a combiner that serves as a light transmission member may be disposed on the inner wall of the front windshield <NUM>, and the driver <NUM> may visually recognizes a virtual image formed by the projection light L that is reflected by the combiner.

In the upper portion of the front windshield <NUM>, a front camera <NUM> that captures an image in front including an image to be displayed by the HUD <NUM> and its background image through the front windshield <NUM>, and an environmental light sensor <NUM> that detects the lightness or chromaticity of the environmental light around the display image are provided. The term "lightness" herein indicates brightness, illuminance, luminous intensity, total luminous flux, and a value calculated from a result of measuring the degrees of these items.

In the present embodiment, the optical system or the like of the on-vehicle HUD <NUM> is configured such that the distance from the driver <NUM> to a virtual image G becomes equal to or longer than <NUM> meters (m). In the known on-vehicle HUDs, the distance from the driver <NUM> to the virtual image G is about <NUM>. Usually, the driver <NUM> observes a point at infinity ahead of the vehicle, or observes a preceding vehicle a few tens of meters ahead of the vehicle. When the driver <NUM> who is focusing on an object in the distance attempts to visually recognize the virtual image G that is two meters ahead of the vehicle, the crystalline lenses of the eyes need to be moved widely because the focal length greatly varies. In such cases, the time required to adjust the focus of the eyes and focus on the virtual image G becomes longer, and it takes a long time to recognize the detail of the virtual image G. What is worse, the eyes of the driver <NUM> tend to get tired. Moreover, it is difficult for the driver to realize the detail of the virtual image G, and it is difficult to use the virtual image G to appropriately provide information to the driver.

If the distance to the virtual image G is equal to or longer than <NUM> as in the present embodiment, the amount of movement in the crystalline lenses of the eyes is reduced to a less amount of movement than the background art, and the time required to adjust the focus of the eyes and focus on the virtual image G becomes shorter. Accordingly, the driver <NUM> can recognize the detail of the virtual image G at an early stage, and the possible tiredness of the eyes of the driver <NUM> can be reduced. Moreover, it becomes easier for the driver to realize the detail of the virtual image G, and it is easy to use the virtual image G to appropriately provide information to the driver. When the distance to the virtual image G is equal to or greater than <NUM>, the driver can focus on the virtual image G with almost no convergence motion in the eyes. Accordingly, the sense of distance (change in perception distance) or the depth perception (difference in perception distance), which are expected to be brought by motion parallax, can be perceived as desired in absence of the convergence motion of the eyes. As described above, according to the present embodiment, the driver perceive the information as intended in view of the sense of distance or depth perception of an image.

The on-vehicle HUD <NUM> illustrated in <FIG> includes, in the optical system <NUM>, red, green, and blue laser beam sources 201R, <NUM>, and 201B, collimator lenses <NUM>, <NUM>, and <NUM> that are provided for the laser beam sources 201R, <NUM>, and 201B, respectively, and two dichroic mirrors <NUM> and <NUM>. The on-vehicle HUD <NUM> further includes a light quantity adjuster <NUM>, an optical scanner <NUM>, a free-form surface mirror <NUM>, a microlens array <NUM> that serves as a light dispersing member, and a projector mirror <NUM> that serves as a light reflecting member. A light source unit <NUM> according to the present embodiment includes the laser beam sources 201R, <NUM>, and 201B, the collimator lenses <NUM>, <NUM>, and <NUM>, and the dichroic mirrors <NUM> and <NUM>, and these elements are unitized by an optical housing.

Each of the laser beam sources 201R, <NUM>, and 201B may be an LD (semiconductor laser element). The wavelength of the luminous flux that is emitted from the red laser beam source 201R is, for example, <NUM> nanometer (nm). The wavelength of the luminous flux that is emitted from the green laser beam source <NUM> is, for example, <NUM>. The wavelength of the luminous flux that is emitted from the blue laser beam source 201B is, for example, <NUM>.

The on-vehicle HUD <NUM> according to the present embodiment projects the intermediate image formed on the microlens array <NUM> onto the front windshield <NUM> of the vehicle <NUM>, such that the driver <NUM> can visually recognize the magnified intermediate image as a virtual image G. The laser beams of the RGB colors emitted from the laser beam sources 201R, <NUM>, and 201B are approximately collimated by the collimator lenses <NUM>, <NUM>, and <NUM>, and are combined by the two dichroic mirrors <NUM> and <NUM>. The light quantity of combined laser beam is adjusted by the light quantity adjuster <NUM>, and then the adjusted laser beam is two-dimensionally scanned by the mirror of the optical scanner <NUM>. The scanned light L' that is two-dimensionally scanned by the optical scanner <NUM> is reflected by the free-form surface mirror <NUM> so as to correct the distortion, and then is collected and condensed to the microlens array <NUM>. Accordingly, an intermediate image is drawn.

In the present embodiment, the microlens array <NUM> is used as a light dispersing member that individually disperses and emits the luminous flux of each pixel of the intermediate image (i.e., each point of the intermediate image). However, any other light dispersing member may be used. Alternatively, a liquid crystal display (LCD) or a vacuum fluorescent display (VFD) may be used as a method of forming the intermediate image G'. However, in order to display the virtual image G with a wide dimension and high brightness, the laser scanning system is desired as in the present embodiment.

In the systems where an LCD or VFD is used, a non-image area of the display area on which the virtual image G is displayed is slightly irradiated with light, and it is difficult to completely shut such light to the non-image area. For this reason, in the systems where an LCD or VFD is used, the non-image area disturbs the visual recognizability of the sight ahead of the vehicle <NUM>. By contrast, if a laser scanning system is adopted as in the present embodiment, the light that irradiates the non-image area of the display area on which the virtual image G is displayed can be completely shut by switching off the laser beam sources 201R, <NUM>, and 201B. For this reason, if a laser scanning system is adopted as in the present embodiment, the non-image area does not disturb the visual recognizability of the sight ahead of the vehicle <NUM> as the light from the on-vehicle HUD <NUM> that may irradiate the non-image area can be completely shut.

When the degree of warning is to be enhanced by gradually increasing the brightness of the warning image that alerts the driver, the display needs to be controlled such that only the brightness of the warning image gradually increases among the various kinds of images displayed in the display area <NUM>. Again, the laser scanning system is suitable for such cases where the display is controlled such that the brightness of a part of the images displayed in the display area <NUM> is selectively increased. In the systems with the LCD or the VFD, the brightness of the images other than the warning image also increases among the various kinds of images displayed in the display area <NUM>. In such cases, the difference in brightness cannot be increased between the warning image and the other images. Accordingly, the degree of the warning cannot be sufficiently enhanced by gradually increasing the brightness of the warning image.

The optical scanner <NUM> uses a known actuator driver system such as a micro-electromechanical systems (MEMS) to incline the mirror to the main scanning direction and the sub-scanning direction, and two-dimensionally scans (raster-scans) the laser beams incident on the mirror. The mirror is controlled in synchronization with the timing at which the laser beam sources 201R, <NUM>, and 201B emit light. The optical scanner <NUM> may be configured, for example, by a mirror system that includes two mirrors that pivot or rotate around the two axes that are orthogonal to each other.

<FIG> is a schematic block diagram of the internal structure of a control system <NUM> of the on-vehicle HUD <NUM> illustrated in <FIG>, according to the present basic embodiment.

As illustrated in <FIG>, the control system <NUM> of the on-vehicle HUD <NUM> includes a field programmable gate array (FPGA) <NUM>, a central processing unit (CPU) <NUM>, a read-only memory (ROM) <NUM>, a random access memory (RAM) <NUM>, an interface (I/F) <NUM>, a bus line <NUM>, a laser diode (LD) driver <NUM>, and a micro-electromechanical systems (MEMS) controller <NUM>. The FPGA <NUM> uses the LD driver <NUM> to control the operation of the laser beam sources 201R, <NUM>, and 201B of the light source unit <NUM>. Moreover, the FPGA <NUM> uses the MEMS controller <NUM> to controlling the operation of a MEMS 208a of the optical scanner <NUM>. The CPU <NUM> controls the operation of the on-vehicle HUD <NUM>. The ROM <NUM> stores various kinds of programs such as an image processing program that is executed by the CPU <NUM> to control the operation of the on-vehicle HUD <NUM>. The RAM <NUM> is mainly used as a working area in which the CPU <NUM> executes a program. The interface <NUM> allows the on-vehicle HUD <NUM> to communicate with an external controller such as a controller area network (CAN) of the vehicle <NUM>. For example, the interface <NUM> is connected to a vehicle navigation device <NUM>, and various kinds of sensor device <NUM> through the CAN of the vehicle <NUM>.

Moreover, the interface <NUM> is connected to the front camera <NUM> that captures an image in front including an image to be displayed by the on-vehicle HUD <NUM> and its background image through the front windshield <NUM>. Further, the interface <NUM> is connected to the environmental light sensor <NUM> that detects the lightness or chromaticity of the environmental light around the display image are provided. As will be described later in detail, the control system <NUM> performs color correction on a display image according to the examples/ embodiments of the present disclosure (see <FIG>, <FIG>, and <FIG>) to correct the colors of the display image that is displayed by the HUD <NUM>.

<FIG> is a schematic block diagram of the on-vehicle HUD <NUM> illustrated in <FIG> and peripheral equipment, according to the present embodiment.

In the present embodiment, as an information acquisition unit that obtains for-driver information to be provided to a driver via a virtual image G, for example, the vehicle navigation device <NUM>, and the sensor device <NUM> are provided. The on-vehicle HUD <NUM> according to the present embodiment includes the optical system <NUM> that serves as an image-light projection device, and the control system <NUM> that serves as a display controller.

The vehicle navigation device <NUM> according to the present embodiment may be any known vehicle navigation device provided for a vehicle or the like. The vehicle navigation device <NUM> outputs information used for generating a route navigation image to be displayed on a virtual image G, and the information output from the vehicle navigation device <NUM> is input to the control system <NUM>. The information that is output from the vehicle navigation device <NUM> includes, for example, as illustrated in <FIG>, images indicating the number of lanes (traffic lanes) of the road on which the vehicle <NUM> is traveling, the distance to the next point where the direction is to be changed (for example, a right turn, left turn, and a branch point), and the direction to which the path is to be changed next in order. As such information is input from the vehicle navigation device <NUM> to the control system <NUM>, the on-vehicle HUD <NUM> displays navigation images as follows under the control of the control system <NUM>. In particular, navigation images such as a lane indicator image <NUM>, a following-distance presenting image <NUM>, a path indicator image <NUM>, a remaining distance indicator image <NUM>, an intersection or the like name indicator image <NUM>, are displayed on an upper display area A of the display area <NUM>.

In the example image illustrated in <FIG>, images indicating road-specific information (e.g., road name, and speed limit) is displayed on a lower display area B of the display area <NUM>. The road-specific information is also input from the vehicle navigation device <NUM> to the control system <NUM>. The control system <NUM> uses the on-vehicle HUD <NUM> to display the road-specific information such as a road-name display image <NUM>, a speed limit display image <NUM>, and a no-passing zone display image <NUM> on the lower display area B of the display area <NUM>.

The sensor device <NUM> illustrated in <FIG> includes one or two or more sensors that detect various kinds of information such as the behavior of the vehicle <NUM>, the state of the vehicle <NUM>, and the environment around the vehicle <NUM>. The sensor device <NUM> outputs sensing information used for generating an image to be displayed as a virtual image G, and the information output from the sensor device <NUM> is input to the control system <NUM>. For example, in the example image illustrated in <FIG>, a vehicle speed display image <NUM> indicating the vehicle speed of the vehicle <NUM> (i.e., the textual image of "<NUM>/h" in <FIG>) is displayed on the lower display area B of the display area <NUM>. The vehicle-speed information included in the CAN information of the vehicle <NUM> is input from the sensor device <NUM> to the control system <NUM>, and the control system <NUM> controls the on-vehicle HUD <NUM> to display the textual image indicating the vehicle speed on the lower display area B of the display area <NUM>.

In addition to the sensor that detects the vehicle speed of the vehicle <NUM>, the sensor device <NUM> includes, for example, (<NUM>) a laser radar or imaging device that detects the distance from another vehicle, a pedestrian, or construction such as a guard rail and a utility pole, which exist around (ahead of, on the side of, in the rear of) the vehicle <NUM>, and a sensor that detects the external environmental information (e.g., outside air temperature, brightness, and weather) of the vehicle <NUM>, (<NUM>) a sensor that detects the driving action (e.g., braking action, and the degree of acceleration) of the driver <NUM>, (<NUM>) a sensor that senses the amount of fuel remaining in the fuel tank of the vehicle <NUM>, and (<NUM>) a sensor that senses the state of various kinds of vehicle-borne equipment such as an engine and a battery. As such information is detected by the sensor device <NUM> device and sent to the control system <NUM>, the on-vehicle HUD <NUM> can display the information as a virtual image G. Accordingly, the information can be provided to the driver <NUM>.

Next, a virtual image G that is displayed by the on-vehicle HUD <NUM> according to the present embodiment is described. In the present embodiment, for-driver information that the on-vehicle HUD <NUM> provides for the driver <NUM> via a virtual image G may be any information. In the present embodiment, the for-driver information is broadly divided into passive information and active information.

The passive information is the information to be passively recognized by the driver <NUM> at the timing when a prescribed information provision condition is met. Accordingly, the passive information includes the information to be provided to the driver <NUM> at the timing when the on-vehicle HUD <NUM> is configured, and the passive information includes the information whose provision timing has a certain relation with the detail of the information. The passive information includes, for example, security information for driving, and route navigation information. The security information for driving includes, for example, the following distance indicating the distance between the vehicle <NUM> and the preceding vehicle <NUM> (i.e., a following-distance presenting image <NUM> as will be described later), and information including urgent matters for driving (e.g., warning information such as an instruction for urgent action to be taken by a driver, or attention attracting information). The route navigation information indicates a route to a prescribed destination, and such a route is provided to a driver by any known vehicle navigation device. The route navigation information includes, for example, lane information (i.e., the lane indicator image <NUM>) indicating a lane to be taken at an upcoming intersection, and direction-change instruction information indicating a direction change to be made at the next intersection or branch point where the direction is to be changed from the straight-ahead direction. The direction-change instruction information includes, for example, path indicating information (i.e., the path indicator image <NUM>) that indicates the path to be taken at the next intersection or branch point, remaining distance information (i.e., the remaining distance indicator image <NUM>) indicating the distance to the intersection or branch point where the direction change is to be made, and name information of the intersection or branch point (i.e., the intersection or the like name indicator image <NUM>).

The active information is the information to be actively recognized by the driver <NUM> at the timing specified by the driver himself or herself. The active information is to be provided to the driver <NUM> only when he or she wishes. For example, the active information includes information where the timing of its provision has low or no relevance to the detail of the information. As the active information is obtained by the driver <NUM> at the timing when he or she wishes, the active information is usually displayed for a long time or displayed continuously. The active information includes, for example, specific information of the road on which the vehicle <NUM> is traveling, the vehicle-speed information (i.e., the vehicle speed display image <NUM>) of the vehicle <NUM>, and the current-time information. The road-specific information includes, for example, the road-name information (i.e., the road-name display image <NUM>), the regulation information of the road such as speed limit (i.e., the speed limit display image <NUM> and the no-passing zone display image <NUM>), and other kinds of information of the road useful for the driver.

In the present embodiment, the for-driver information, which is broadly divided into the active information and the passive information as described above, is displayed in a corresponding area of the display area <NUM> where a virtual image is displayable. More specifically, in the present embodiment, the display area <NUM> is divided into two display areas in the up-and-down directions. Then, a passive-information image that corresponds to the passive information is mainly displayed in the upper display area A of the obtained three display areas, and an active-information image that corresponds to the active information is mainly displayed in the lower display area B. Note that only some of the active-information image may be displayed upper display area A. In such cases, the active-information image is displayed in such a manner that a higher priority is given to the viewability of the passive-information image displayed in the upper display area A.

In the present embodiment, a stereoscopic image is used as the virtual image G that is displayed in the display area <NUM>. More specifically, perspective images are used as the lane indicator image <NUM> and the following-distance presenting image <NUM> that are displayed in the upper display area A of the display area <NUM>.

More specifically, a perspective image that is drawn by the perspective drawing method such that the length of the five horizontal lines of the following-distance presenting image <NUM> becomes shorter towards the upper side and the following-distance presenting image <NUM> heads for a single vanishing point. In particular, in the present embodiment, the following-distance presenting image <NUM> is displayed such that the vanishing point approximately matches the observation point of the driver <NUM>. Due to this configuration, while the driver <NUM> is driving, he or she can easily perceive the depth of the following-distance presenting image <NUM>. Moreover, in the present embodiment, a perspective image in which the thickness of the horizontal lines becomes thinner towards the upper side and the brightness of the horizontal lines becomes lower towards the upper side is used. Due to this configuration, while the driver <NUM> is driving, he or she can even more easily perceive the depth of the following-distance presenting image <NUM>.

In the description of the basic embodiment, the hardware configuration of the on-vehicle HUD <NUM>, which is used in the examples/embodiments of the present disclosure, is described. In the examples/embodiments described below, hardware elements are used as follows. The first and second examples are included as useful examples for understanding the technical effect of the invention, which is defined by the claims.

In typical related art examples, an illuminance sensor that is provided for the HUD is used to control the level of brightness of the display. In such a configuration, there exist distance between the position at which the illuminance sensor is disposed and the background area. For this reason, when the brightness abruptly changes, for example, when a vehicle passes through the shade of a building or the like, a difference is caused between the actual brightness of the background and the light quantity sensed by the illuminance sensor, and the viewability temporarily decreases. It is known in the art that a camera or the like is used to capture an area ahead of the vehicle to calculate the brightness of the background area and the brightness of the display is adjusted. In such a configuration, due to, for example, variations among individuals in the location of the viewpoint of a driver, a difference is caused between the actual brightness of the background and the calculated brightness of the background. In the present example, the on-vehicle HUD <NUM> is provided that achieves viewability in any environment regardless of the variations among individuals in drivers or surrounding environment and improves the visual recognizability in comparison to the related art.

<FIG> is a chromaticity diagram illustrating a chromaticity coordinate area <NUM> of the background light used in the HUD <NUM> according to the first example of the present disclosure and chromaticity coordinates <NUM> outside the chromaticity coordinate area <NUM>.

<FIG> is a chromaticity diagram illustrating the color conversion performed by the control system <NUM> according to the first example of the present disclosure.

In the first example, changes in the color of environmental light are not considered. In the first example, in particular, color-gamut data of the assumed background of a display image of the HUD <NUM> is stored in advance in the background light information table 253t of the ROM <NUM>. Then, as illustrated in <FIG>, the colors of a display image are corrected such that the color gamut of the display image including sign images consists of chromaticity dots (for example, chromaticity coordinates <NUM> indicated by a star mark) outside the color gamut of the background light (i.e., the chromaticity coordinate area <NUM>). Due to this configuration, regardless of the state of background, a color difference is caused between the background color and the display color, and a person to visually recognize an image such as a driver can distinguish between the background and the display.

The chromaticity coordinate area data of the background light to be stored in the background light information table 253t may be obtained by measuring in advance the chromaticity coordinates of the light reflected from various kinds of objects and summarizing the measured data of the chromaticity coordinates. The chromaticity coordinate area of the background light can be determined by such chromaticity coordinate area data. For example, a sample is irradiated with a standard light source D65 set by the CIE (International Commission on Illumination/Commission International de L'Eclairage) to measure the chromaticity coordinates of the reflection light. Alternatively, for example, as disclosed in "<NPL>," the data that is obtained in the past research where the color gamut of the light reflected from an object is compiled may substitute, as will be described later in detail with reference to <FIG>. In the present example, a predetermined color conversion method, as described later in detail, is used to move the color coordinates outside the chromaticity coordinate area of the background light when the display colors are included in the chromaticity coordinate area of the background light. In regard to the brightness of a display image, for example, the environmental light sensor <NUM> is used to measure the lightness of environmental light, and the brightness of a display image is changed accordingly. Unlike the second example, a unit to measure the color information of environmental light is not necessary in the first example. Accordingly, the configuration can be simplified.

In the present disclosure, lower-cases "xy" indicate chromaticity coordinates, and "XYZ" in uppercase indicate the levels of tristimulus values.

Next, a color conversion method according to the first example for correcting the colors of a display image is described with reference to <FIG>.

Firstly, in <FIG>, the chromaticity coordinates (xw, yw) of a white dot w are determined preparatory to color conversion. In the first example, the light reflected from various kinds of objects is measured in advance, and the color of that illumination is used as a white dot. By way of example, an object is irradiated with a standard light source D65, and the chromaticity coordinates of the reflection light are measured. In the color gamut of the background that is obtained by measurement, chromaticity coordinates (xw, yw)=(<NUM>, <NUM>) of the standard light source D65 are used as the chromaticity coordinates of a white dot. In addition to these chromaticity coordinates, as illustrated in <FIG>, the chromaticity coordinates (xc, yc) of a color c on the display image are obtained. For example, when the display image indicates the information of sRGB (D65), the chromaticity coordinates (Xc, Yc, Z) of the display image are converted into chromaticity coordinates (xc, yc) using the following equations. <MAT> <MAT>.

When the display image is described by a different color space format, color conversion is performed according to that color space format to obtain (xc, yc).

Next, a vector that connects the white dot w and a chromaticity dot c on the display image illustrated in <FIG> is derived. Accordingly, a vector where the white dot w and the chromaticity dot c on the display image are the starting point and the ending point, respectively, is obtained. On the extension of the obtained vector extended from the ending point c, a chromaticity dot c' of a post-color-conversion display image is set. In so doing, a color difference Δ is set between the chromaticity dot c' and the chromaticity coordinate area <NUM> that is the color gamut of the background light. The color difference Δ is, for example, distance on an xy plane and takes on a value of <NUM> that is equal to or greater than a predetermined threshold <NUM>. As described above, color conversion is performed. Alternatively, different thresholds may be set for each area of chromaticity coordinates. In this process, a dot other than the white dot w may be used as a starting point of vector.

<FIG> is a chromaticity diagram illustrating a chromaticity coordinate area of the background light used in the HUD <NUM> according to the first example of the present disclosure, and the data listed in "<NPL>" is plotted in <FIG>.

<FIG> is a chromaticity diagram where an envelope including the plotted area of <FIG> is set to be a chromaticity coordinate area <NUM> of the background light.

As is apparent from <FIG>, it can be estimated that the light reflected from all sorts of objects existing in the nature falls within the chromaticity coordinate area <NUM>. Accordingly, the chromaticity coordinate area <NUM> may be used as the chromaticity coordinate area of the background light in the present example.

<FIG> is a flowchart of color correction processes performed on a display image by the control system <NUM> according to the first example of the present disclosure.

In step S1 of <FIG>, it is determined whether the chromaticity coordinates of a display image are included in a predetermined chromaticity coordinate area of the background light. If "YES" in step S1, the process proceeds to S2. If "NO" in step S1, the process proceeds to S3. In step S2, color conversion is performed such that the chromaticity coordinates of the display image are excluded from the predetermined chromaticity coordinate area of the background light. In step S3, the environmental light sensor <NUM> is used to detect the lightness of environmental light. In step S4, the brightness of the chromaticity coordinates of the display image is corrected based on the detected lightness of environmental light. Then, in step S5, the display image is displayed using the obtained chromaticity coordinates of the display image, and the color correction of display image ends.

According to the present example as described above, color-gamut data of the assumed background of a display image of the HUD <NUM> is stored in advance, and color correction is performed such that the colors of a display image are excluded from the stored color gamut. Due to this configuration, regardless of the state of background, a color difference is caused between the background color and the display color, and a person to visually recognize an image such as a driver can distinguish between the background and the display. Accordingly, the on-vehicle HUD <NUM> can be provided that achieves viewability in any environment regardless of the variations among individuals in drivers.

In the color correction processes of display image illustrated in <FIG>, the processes in steps S3 and S4 may be omitted when necessary. Alternatively, the processes in steps S3 and S4 may be added, when necessary, to the processes in <FIG> as will be described later.

In the color correction processes of display image illustrated in <FIG>, the chromaticity coordinates of the display colors may be determined based on the chromaticity coordinate area information of the background light such that the display colors of the display image are excluded from the predetermined chromaticity coordinate area of the background light.

In the first example as described above, the color correction processes are performed when a display image to be displayed by the HUD <NUM> is sequentially generated. However, no limitation is intended thereby. When a display image to be displayed by the HUD <NUM> is known and sign images for the display image are to be produced, in a similar manner to the first example, the sign images for the display image may be produced such that the chromaticity coordinates of the display image are excluded from the chromaticity coordinate area of the background light.

<FIG> is a chromaticity diagram illustrating the color conversion performed by the control system <NUM> according to the second example of the present disclosure.

In the second example, the chromaticity coordinates of display colors are adjusted according to changes in chromaticity of environmental light, and <FIG> is a chromaticity diagram illustrating how color correction is performed on display colors. The light reflected from the background changes as the chromaticity of environmental light changes. For example, the scenery appears to be reddish at sunset. In the second example, the environmental light sensor <NUM> including a colorimetry sensor that detects the chromaticity of environmental light is used, and preferably, the environmental light sensor <NUM> detects the brightness and chromaticity coordinates. Then, the brightness information and chromaticity information of the environmental light are used to perform chromaticity correction with reference to the chromaticity coordinates of the chromaticity coordinate area of the background light, which is measured and stored in advance in the background light information table 253t. In other words, as illustrated in <FIG>, color conversion is performed such that the chromaticity coordinate area <NUM> of the background light are converted into the chromaticity coordinate area <NUM> of the environmental light, as indicated by an arrow <NUM>. As the result of color conversion, when the colors of a display image overlap with the chromaticity coordinate area <NUM> of the background light (see, for example, chromaticity coordinates C1 in <FIG>), as indicated by an arrow <NUM>, color conversion is performed such that the colors of a display image are excluded from the newly set chromaticity coordinate area <NUM> of the background light.

Next, the correction of the chromaticity coordinate area of background light as a function of environmental light, according to the second example of the present disclosure, is described.

In a similar manner to the first example, the information of the chromaticity coordinates (xw, yw) of a white dot is stored in the ROM <NUM> in advance. From the environmental light sensor <NUM> including a colorimetry sensor that detects the chromaticity of environmental light, tristimulus values (Xe, Ye, Ze) of the environmental light are output. Note that a colorimetry sensor using RGB values may be used instead of the colorimetry sensor as described above. In such cases, RGB values are color-converted into tristimulus values (Xe, Ye, Ze). For example, when the format of RGB values is in sRGB (D65), color conversion is performed as in the following equations.

The chromaticity coordinates of the white dot w and the tristimulus values of environmental light are used to calculate a correction factor in that environment. The procedure for calculating a correction factor is as follows.

The procedure for performing color correction on the points within the chromaticity coordinate area <NUM> of the background light is described as follows. Here, the points within the chromaticity coordinate area <NUM> of the background light may be all the points of the measurement results on a sample, which is measured to obtain the background region. Alternatively, the points within the chromaticity coordinate area <NUM> of the background light may be some of the points that are reduced to a certain extent not affecting the shape of the chromaticity coordinate area of the background light.

Based on the corrected chromaticity coordinates (xb', yb') of points as obtained above, the chromaticity coordinate area of the background light is adjusted again, and color conversion is performed accordingly on a display image including sign images. The color conversion according to the present example is similar to that of the first example. However, in the present example, the chromaticity coordinates of the environmental light that are obtained by the environmental light sensor <NUM>, which serves as a colorimetry sensor, are used for the chromaticity coordinates of the white dot.

<FIG> is a flowchart of color correction processes performed on a display image by the control system <NUM> according to the second example of the present disclosure.

In step S11 of <FIG>, the environmental light sensor <NUM> is used to detect the chromaticity coordinates of environmental light. Subsequently, in step S12, the detected chromaticity coordinates of environmental light are used to color conversion is performed such that the chromaticity coordinate area <NUM> of the background light, which are measured in advance, is converted into the chromaticity coordinate area <NUM> of the background light where the environmental light is taken into consideration. Then, in step S13, it is determined whether the chromaticity coordinates of a display image are included in the color-converted chromaticity coordinate area of the background light. If "YES" in step S13, the process proceeds to S14. If "NO" in step S13, the process proceeds to S15. In step S14, color conversion is performed such that the chromaticity coordinates of the display image are excluded from the predetermined chromaticity coordinate area of the background light. In step S15, the display image is displayed using the obtained chromaticity coordinates of the display image, and the color correction of display image ends.

In the present example as described above, the detected chromaticity coordinates of environmental light are used to perform color conversion such that the chromaticity coordinate area of the background light, which are measured in advance, are converted into the chromaticity coordinate area of the background light where the environmental light is taken into consideration. Accordingly, the colors of a display image can be adjusted according to the changes in background light due to environmental light, and the visibility for a person to visually recognize the image significantly improves.

The color correction according to the second example is applied to the color correction according to the first example, as described above. However, no limitation is intended thereby, and the color correction according to the second example may be applied to the color correction according to a first embodiment as will be described later.

<FIG> is a chromaticity diagram illustrating the color conversion performed by the control system <NUM> according to the first embodiment of the present disclosure.

In the first embodiment, the chromaticity coordinate area <NUM> of the background light is not set in advance unlike the first and second examples, but the two-dimensional distribution data of the background light is used to obtain the information of the chromaticity coordinate area <NUM> (see <FIG>) of the background light. The two-dimensional distribution data of the background light is obtained, for example, by using the front camera <NUM> disposed on a vehicle, or by measuring such two-dimensional distribution with a plurality of photodiodes disposed towards the area in front of the vehicle. Alternatively, the two-dimensional distribution data of background light that is measured by other sorts of means may be used.

While the vehicle is traveling, the front camera <NUM> collects the chromaticity coordinates of an area that serves as the background image of the HUD <NUM>, for example, at regular time intervals to obtain the information of the chromaticity coordinate area <NUM> of the background light, and store the obtained information in the RAM <NUM>. The chromaticity coordinates of the chromaticity coordinate area <NUM> of the background light stores only the data of prescribed length of time (for example, the data of ten minutes). When new data is acquired by measurement, old measured values are deleted, and the chromaticity coordinate area of background light is adjusted. In the first and second examples, the chromaticity coordinate area of background light tends to be too broadly set. However, in the first embodiment, the chromaticity coordinate area of background light is set according to the actual environment on the roads. For this reason, the chromaticity coordinate area of the background light is not too broadly set. Accordingly, a wide range of colors can be used as display colors of a display image.

Next, the color conversion according to the first embodiment of the present disclosure is described.

A method for setting a white dot w according to the first embodiment is different from that of the first and second examples. The images of the area in front of the vehicle, which are captured by the front camera <NUM>, are used to calculate the average color of the images, and the calculated average color is used as the white dot w. In order to derive such an average color, firstly, the sum of all the pixel values of captured images is calculated, and then the obtained sum of the pixel values are divided by the number of pixels n. By so doing, averages Rave, Gave, and Bave are calculated for RGB (red, green, blue) values.

The obtained averages Rave, Gave, and Bave for RGB values are converted into chromaticity coordinates using the following equations, and the chromaticity coordinates of the white dot w are obtained. <MAT> <MAT>.

In this configuration, the environmental light sensor <NUM>, which includes a colorimetry sensor and is separate from the front camera <NUM>, is used to obtain the chromaticity coordinates of environmental light, and the obtained chromaticity coordinates of environmental light are used as the white dot w. Alternatively, the white dot may be set to chromaticity coordinates (x, y)=(<NUM>, <NUM>) of the standard light source D65.

<FIG> is a flowchart of color correction processes performed on a display image by the control system <NUM> according to the first embodiment of the present disclosure.

In step S21 of <FIG>, the front camera <NUM> is used to collect the chromaticity coordinates of chromaticity coordinate area of the background light, for example, at regular time intervals to detect the chromaticity coordinate area of the background light. In step S22, it is determined whether the chromaticity coordinates of a display image are included in the chromaticity coordinate area of the background light. If "YES" in step S22, the process proceeds to S23. If "NO" in step S22, the process proceeds to S24. In step S23, color conversion is performed such that the chromaticity coordinates of the display image are excluded from the predetermined chromaticity coordinate area of the background light. In step S24, the display image is displayed using the obtained chromaticity coordinates of the display image, and the color correction of display image ends.

In the embodiments of the present disclosure as described above, the detected chromaticity coordinates of environmental light are used to perform color conversion such that the chromaticity coordinate area of the background light, which are actually measured, are converted into the chromaticity coordinate area of the background light where the environmental light is taken into consideration. Accordingly, the colors of a display image can be adjusted according to the changes in background light due to environmental light, and the visibility for a person to visually recognize the image significantly improves.

The color correction according to the examples/embodiments of the present disclosure as described above may be performed repeatedly at predetermined time intervals.

The color correction according to the examples/ embodiments of the present disclosure may be applied, for example, only to a certain display image such as warning display.

In the first embodiment, a chromaticity coordinate area of background light is not set in advance. Instead, a vehicle-installed camera is used to obtain the chromaticity coordinate area information of the background light while the vehicle is travelling. In the first and second examples as described above, the chromaticity coordinate area of background light tends to be too broadly set. However, in the first embodiment, the chromaticity coordinate area of background light is set according to the actual environment on the roads. For this reason, the chromaticity coordinate area of the background light is not too broadly set. Accordingly, a wide range of colors can be used as display colors of a display image. In the first embodiment and in accordance with the invention, an area that overlaps with a display area of the HUD is captured by a camera, and the chromaticity coordinate area information of background light is obtained from the captured image of the two-dimensional distribution of color information of the chromaticity coordinate area of background light.

<FIG> is a front view of a color information obtaining area where the color information is obtained by the control system <NUM>, when viewed from a viewpoint of a driver, according to a first embodiment of the present disclosure.

In other words, <FIG> is a front view of a color-information obtaining area <NUM> and a display area <NUM> of the HUD <NUM>, when viewed from a viewpoint of a driver, and the color-information obtaining area <NUM> overlaps with the display area <NUM> in the display area <NUM>.

<FIG> is a side view of the color-information obtaining area <NUM> of the embodiment illustrated in <FIG>, when viewed from the side of a vehicle.

<FIG> is a top view of the color-information obtaining area <NUM> of the embodiment illustrated in <FIG>, when viewed from the upper side of a vehicle.

In <FIG>, the location of the viewpoint of eyes 300E of a driver is at a height of <NUM> meters (m) above a road surface <NUM>, and the angles of view of the HUD <NUM> are <NUM>° in the horizontal direction, <NUM>° in the vertical direction, and <NUM>° in the angle of depression. <FIG> illustrates the color-information obtaining area <NUM> when the front camera <NUM> is at a height of <NUM> centimeters (cm) above the location of the viewpoint of the eyes 300E of the driver inside the front windshield <NUM>. Under these conditions as described above, as illustrated in <FIG>, the color-information obtaining area <NUM> ranges from <NUM> to <NUM> ahead of the viewpoint of the driver. In order to capture the color-information obtaining area <NUM> by the front camera <NUM> that is disposed above the driver, the angle of view of the front camera <NUM> is, for example, -<NUM>° to -<NUM>°. <FIG> merely illustrates an example case, and the specification or installation conditions of the front camera <NUM> are not limited thereby. Note also that the two-dimensional distribution of the chromaticity coordinate area of the background light may be obtained from the information obtained from a vehicle-installed camera such as a stereo camera or a drive recorder.

<FIG> is a top view of the color-information obtaining area <NUM>, when viewed from the upper side of the vehicle under these conditions as described above.

As is apparent from <FIG>, the color-information obtaining area <NUM> is in a shape of trapezoid on the road surface <NUM> as diagonally shaded.

<FIG> are chromaticity diagrams illustrating a method of determining the chromaticity coordinate area of background light, according to the embodiment <FIG>.

The method of determining the chromaticity coordinate area of the background light according to the embodiment illustrated in <FIG> is as follows.

The background color information to be used is obtained by cropping the color-information obtaining area <NUM> from the images captured by the RGB camera <NUM> and putting plots, for example, of the chromaticity per pixel on a chromaticity diagram of x-y plane. In <FIG>, a plurality of plots <NUM> of chromaticity are placed. <FIG> illustrates a result of setting an area <NUM> that contains the plots, using in accordance with the invention a convex hull, from the distribution of the plots <NUM> as illustrated in <FIG>. For example, a gift wrapping algorithm is used for the convex hull to set the area <NUM> that contains the plots. The RGB camera <NUM> is used to capture images at prescribed time intervals (for example, every five seconds), and the data of prescribed length of time (for example, the data of ten minutes) is stored. When new data is acquired by measurement, the oldest measured values are deleted, and the chromaticity coordinate area of background light is adjusted. Due to the steps as described above, the chromaticity coordinate area of background light can be determined.

In <CIT>, it is aimed at letting a user visually recognize a virtual image in a clear manner regardless of the surrounding environment. Moreover, the surrounding environment of a position at which a virtual image is produced is obtained, and the brightness of a light source and the brightness of the color components (RGB) of moving images to be projected are adjusted based on the obtained surrounding environment. However, there is a problem that the viewability decreases as the precision of the measurement of the brightness of background light is insufficient.

In the embodiments of the present disclosure as described above, the HUD <NUM> to be used in cars is described. However, no limitation is intended thereby. The HUD <NUM> according to the embodiments as described above may be applied to other various kinds of vehicles or image display apparatuses such as display devices, or may be used for other various kinds of purposes.

In the embodiments described above, color conversion and correction are performed such that the color difference between the chromaticity of a display image and the chromaticity of a background image becomes equal to or greater than a prescribed threshold. Accordingly, the viewability of the display image improves in comparison to the related art. Such a display image that is to be color-converted and corrected may be limited to displays of warning or danger that warn a driver that, for example, his/her vehicle is approaching a vehicle ahead too close.

Claim 1:
An image display apparatus (<NUM>) comprising:
an image-light projection device (<NUM>) configured to display a display image (G);
a display controller (<NUM>) configured to determine chromaticity coordinates (<NUM>) of a display color of the display image (G); and
a detector (<NUM>) configured to acquire data of background light, wherein the detector is a camera;
characterised by means to determine a chromaticity coordinate area (<NUM>) of the background light in form of a convex hull of a two-dimensional distribution of chromaticity coordinates (<NUM>) of the background light based on the data of the background light acquired by the detector;
wherein the display controller (<NUM>) is configured to determine the chromaticity coordinates (<NUM>) of the display color of the display image (G) based on the data of the two-dimensional distribution of chromaticity coordinates (<NUM>) of the background light such that the display color of the display image (G) is set outside the determined chromaticity coordinate area (<NUM>) of the background light.