IMAGE DISPLAY APPARATUS AND IMAGE DISPLAY METHOD

An image display apparatus includes an imager that captures an image of an area adjacent to a vehicle, a gaze point detector that detects a position of a gaze point of an observer seated on a seat of the vehicle, an image display located on an obstructive object obstructing an outside view of the observer, an image data generator that generates, based on captured image data output from the imager, image data of an image corresponding to the outside view of the observer obstructed by the obstructive object, and a display device that causes the image data to be displayed on the image display. The imager changes a resolution of the imager based on the position of the gaze point.

TECHNICAL FIELD

The present disclosure relates to an image display apparatus and an image display method.

BACKGROUND OF INVENTION

A known technique is described in, for example, Patent Literature 1.

CITATION LIST

Patent Literature

SUMMARY

In an aspect of the present disclosure, an image display apparatus includes an imager that captures an image of an area adjacent to a vehicle, a gaze point detector that detects a position of a gaze point of an observer seated on a seat of the vehicle, an image display located on an obstructive object obstructing an outside view of the observer, an image data generator that generates, based on captured image data output from the imager, image data of an image corresponding to the outside view of the observer obstructed by the obstructive object, and a display device that causes the image data to be displayed on the image display. The imager changes a resolution of the imager based on the position of the gaze point.

In another aspect of the present disclosure, an image display method includes detecting a position of a gaze point of an observer seated on a seat of a vehicle, capturing an image of an area adjacent to the vehicle, generating, based on captured image data obtained in the capturing the image, image data of an image corresponding to an outside view of the observer seated on the seat of the vehicle obstructed by an obstructive object, and displaying the image data on an image display located on the obstructive object. The capturing the image includes changing a resolution of an imager based on the position of the gaze point.

DESCRIPTION OF EMBODIMENTS

The objects, features, and advantages of the present disclosure will become more apparent from the following detailed description and the drawings.

Various transparency techniques and various image display apparatuses using such transparency techniques have been recently developed for projecting an outside image captured by a camera onto a retroreflective screen located on an obstructive object obstructing the field of view of an observer and allowing the observer to perceive a transparent image of the outside as viewed through the obstructive object being transparent (refer to, for example, Patent Literature 1).

Such known image display apparatuses may have difficulty in allowing the observer to perceive a transparent image as connecting with outside scenery or to perceive a high-definition transparent image when the position of a gaze point of the observer changes.

An image display apparatus and an image display method according to one or more embodiments of the present disclosure will now be described with reference to the accompanying drawings. The drawings used herein are schematic and are not drawn to scale relative to the actual size of each component. For ease of explanation, each drawing illustrates the orthogonal XYZ coordinate system with X-direction referring to the width direction of a vehicle, Y-direction to the length direction of the vehicle, and Z-direction to the height direction.

FIG.1is a schematic diagram of an image display apparatus according to one embodiment of the present disclosure.FIG.2is a schematic plan view of a vehicle incorporating the image display apparatus inFIG.1.FIG.3is a schematic side view of the vehicle incorporating the image display apparatus inFIG.1.FIG.4is a schematic cross-sectional view of an example retroreflective screen in the image display apparatus inFIG.1.FIG.5is a schematic plan view illustrating an imaging range of an imager and a projection range of a projector in the image display apparatus inFIG.1.FIGS.6to9are diagrams each illustrating generation and projection of transparent image data performed by the image display apparatus inFIG.1.FIGS.10and11are each a diagram of transparent image data projected on an obstructive object without changing the resolution of the imager in a comparative example.FIG.12is a schematic plan view of a vehicle incorporating an image display apparatus according to a variation of the embodiment of the present disclosure.FIG.13is a schematic side view of the vehicle incorporating the image display apparatus according to the variation of the embodiment of the present disclosure.FIG.14is a schematic plan view of a vehicle incorporating an image display apparatus according to another embodiment of the present disclosure.FIG.15is a flowchart of an image display method according to one embodiment of the present disclosure.

In the present embodiment, an image display apparatus1includes an imager2, a gaze point detector3, an image display4, an image data generator5, and a display device6(refer toFIG.1).

The image display apparatus1is incorporated in a vehicle10as illustrated in, for example,FIGS.2and3.FIGS.2and3illustrate the vehicle10that is a passenger vehicle. In some embodiments, the vehicle10may not be a passenger vehicle and may be an automobile such as a truck, a bus, a motorcycle, or a trolley bus. The vehicle10may not be an automobile and may be a railroad vehicle, an industrial vehicle, a community vehicle, or a fixed-wing aircraft that travels on a runway.

The image display apparatus1includes a controller9. The controller9is connected to the components of the image display apparatus1to control these components. The controller9is implemented by a processor such as an electronic control unit (ECU) as a hardware resource and by a computer-readable program as a software resource. The controller9may include one or more processors. The processors may include a general-purpose processor that reads a specific program and performs a specific function, and a processor dedicated to specific processing. The dedicated processor may include an application-specific integrated circuit (ASIC). The processors may include a programmable logic device (PLD). The PLD may include a field-programmable gate array (FPGA). The controller9may be either a system on a chip (SoC) or a system in a package (SiP) in which one or more processors cooperate with other components. The controller9may include a storage and store, into the storage, various items of information or programs to operate each component of the image display apparatus1. The storage may be, for example, a semiconductor memory. The storage may serve as a storage area temporarily used during the data processing performed by the controller9.

The imager2captures an image of the scenery adjacent to the vehicle10. The imager2outputs captured image data (also referred to as image data) resulting from such image capturing to the image data generator5. The imager2includes an exterior camera21. The exterior camera21may be located on a front end of the vehicle10as illustrated in, for example,FIGS.2and3. The exterior camera21may also be located inside the cabin or inside the engine compartment in the vehicle10. The exterior camera21includes multiple image sensors. The image sensors may be, for example, charge-coupled device (CCD) image sensors or complementary metal-oxide semiconductor (CMOS) image sensors. The exterior camera21may include a lens with a wide angle of view, such as a wide-angle lens or a fisheye lens.

The imager2can change its resolution (more specifically, the resolution of an image captured by the exterior camera21). The imager2may change the resolution based on a control signal output from the controller9. The resolution of the imager2may be changed by increasing or decreasing, of the multiple image sensors, the number of image sensors that receive light of an image of scenery. The resolution of the imager2may be changed by increasing or decreasing, of the multiple image sensors, the number of image sensors that output light reception signals.

The vehicle10includes obstructive objects8that partially obstruct the outside view of an observer7seated on a seat12of the vehicle10. Examples of the obstructive objects8include a dashboard81, side pillars82, and doors83. The exterior camera21captures an image of scenery adjacent to the vehicle10that is not obstructed by the obstructive objects8and perceivable by the observer7and scenery adjacent to the vehicle10that is obstructed by the obstructive objects8and imperceivable by the observer7. In the example described below, the dashboard81is an obstructive object8. The seat12on which the observer7is seated may be the driver's seat of the vehicle10. The observer7may be the driver of the vehicle10. The observer7is hereafter also referred to as a driver7. The seat12is hereafter also referred to as a driver's seat12.

The gaze point detector3detects a gaze point P of the observer7seated on the seat12of the vehicle10. The gaze point P is a point in the three-dimensional space at which the observer7gazes. The gaze point P may be a point at which the line of sight of a left eye (also referred to as a first eye)7L of the observer7intersects with the line of sight of a right eye (also referred to as a second eye)7R of the observer7. The line of sight is a straight line connecting the center of a pupil of the observer7and the center of the corneal curvature of the observer7. The lines of sight are detectable based on, for example, captured image data of images of the left eye7L and the right eye7R captured by a camera. The left eye7L and the right eye7R may be hereafter simply and collectively referred to as eyes7L and7R without being distinguished from each other. The gaze point P may be, for example, a position of an object that attracts attention of the observer7. The gaze point P may be a position of an object at the shortest distance from the vehicle10or a position of an object at the longest distance from the vehicle10.

The gaze point detector3includes an interior camera31. The interior camera31captures an image of the face of the observer7, in particular, the left eye7L and the right eye7R of the observer7.

The interior camera31may be located on the ceiling of the cabin in the vehicle10as illustrated in, for example,FIGS.2and3. The interior camera31may be located on the dashboard81or on one of the side pillars82. The interior camera31may be integral with a room mirror. The interior camera31includes multiple image sensors. The image sensors may be, for example, CCD image sensors or CMOS image sensors. The interior camera31may be a visible light camera or an infrared camera. Multiple interior cameras31may be located at different positions in the cabin.

The gaze point detector3performs an arithmetic operation on captured image data including the coordinates of the eyes7L and7R of the observer7output from the interior camera31and detects the gaze point P of the observer7. The gaze point detector3outputs the coordinates of the detected gaze point P to the image data generator5and to the controller9. The controller9may generate a control signal for changing the resolution of the exterior camera21based on the coordinates of the gaze point P output from the gaze point detector3and output the generated control signal to the imager2.

The gaze point detector3may detect the X-coordinate, the Y-coordinate, and the Z-coordinate of the gaze point P in the three-dimensional space. In some embodiments, the Z-coordinate of the gaze point P is estimated to be substantially the same as the Z-coordinate of the position of a headrest121of the seat12at which the eyes7L and7R of the observer7are estimated to be located. Thus, the gaze point detector3may detect the X-coordinate and the Y-coordinate alone of the gaze point P in a plane that is perpendicular to the height direction (Z-direction) and includes the headrest121of the seat12. This can reduce a processing load of the gaze point detector3. A transparent image to be perceived by the observer7can thus be processed at higher speed, allowing the observer7to perceive a natural transparent image.

The image display4is located on the surface of the obstructive object8facing the observer7seated on the seat12of the vehicle10. The display device6causes image data to be displayed on the image display4. The image display4may be a screen such as a retroreflective screen or a diffuse reflection screen. The image display4is not limited to a screen. The image display4may include an image display device such as a liquid crystal display device, an organic electroluminescent (EL) display device, or a micro-light-emitting diode (μLED) display device. The image display4is hereafter a retroreflective screen40. The display device6being a projector projects image data onto the retroreflective screen40. The retroreflective screen40is retroreflective and reflects all the incident light to travel in a direction reverse to the direction in which the light enters. The retroreflective screen40may include, for example, microbeads or corner cubes.

The retroreflective screen40including microbeads includes a retroreflector41. The retroreflector41includes a reflective film41aand multiple glass beads41battached to the reflective film41a. The reflective film41amay contain metal materials such as aluminum (Al), silver (Ag), gold (Au), chromium (Cr), nickel (Ni), platinum (Pt), or tin (Sn). The glass beads41bmay have a diameter of, for example, about 20 to 100 μm inclusive.

Incident light entering the retroreflective screen40is refracted at the front surface of each glass bead41bopposite to the reflective film41a, reaches the back surface of the glass bead41bon the reflective film41a, and is reflected from the reflective film41a. The light reflected from the reflective film41ais again refracted at the back surface of the glass bead41band travels along a light path separate from the light path of the incident light by a small distance less than or equal to the diameter of the glass bead41band parallel to the light path of the incident light. In this manner, the retroreflection is performed by the retroreflective screen40.

The retroreflective screen40may include a diffuser42. The diffuser42is located between the retroreflector41and the display device6that is a projector. The diffuser42is located proximate to the retroreflector41. The diffuser42may diffuse light flux of image light emitted from an exit pupil in the display device6. The diffuser42may be, for example, a diffractive element.

With the retroreflective screen40without including the diffuser42, image light projected by the projector is retroreflected from the retroreflective screen40and travels toward the projector. The observer7may thus perceive the transparent image with lower luminance. With the retroreflective screen40including the diffuser42, image light retroreflected from the retroreflective screen40can enter the eyes7L and7R of the observer7. The observer7can thus perceive the transparent image with higher luminance.

The diffuser42may be anisotropic and have a difference in a diffusion capability between the width direction of the vehicle (X-direction) and the height direction (Z-direction). The anisotropic diffuser may have a diffusion capability settable based on, for example, the estimated positions of the eyes7L and7R of the observer7(e.g., the position of the headrest121of the seat12) and the position of the display device6that is a projector. This allows image light retroreflected from the retroreflective screen40to effectively enter the eyes7L and7R of the observer7, thus allowing the observer7to perceive a clear transparent image.

The image data generator5generates, based on image data about the scenery adjacent to the vehicle10output from the imager2, image data of an image (also referred to as a transparent image) corresponding to the outside view of the observer7that is obstructed by the obstructive object8.

The image data generator5includes an outside-image memory51storing image data about the scenery adjacent to the vehicle10, a transparent image memory52storing image data (also referred to as transparent image data) to be projected on the retroreflective screen40, and a processor53implementable by, for example, a processor.

Image data of images captured by the exterior camera21is stored and updated in the outside-image memory51. The image data stored and updated in the outside-image memory51includes a range corresponding to an outside view of the observer7that is obstructed by the obstructive object8.

The processor53reads the image data stored in the outside-image memory51and generates, based on the image data, transparent image data of an image corresponding to the outside view of the observer7that is obstructed by the obstructive object8. To generate the transparent image data, the processor53performs an arithmetic operation based on the position and the shape of the obstructive object8. The transparent image memory52stores the image data generated by the processor53. The image data generator5outputs the image data stored in the transparent image memory52to the display device6.

The image data generator5cuts (extracts) a portion of the image data output from the imager2and generates the transparent image data based on the cut (extracted) image data. The transparent image data generated by the image data generator5may be a portion of the image data output from the imager2. The image data generator5may correct, based on, for example, the positional relationship between the positions of the eyes7L and7R and the position of the gaze point P as well as the shape of the obstructive object8, the cut image data to deform the transparent image. This allows the observer7to perceive a less distorted transparent image as connecting with the outside scenery when the obstructive object8includes a curved surface on which the retroreflective screen40is located.

The image data generator5may include an illuminance sensor (not illustrated) that detects the illuminance inside the cabin. The image data generator5may adjust luminance information about the image data based on the illuminance obtained from the illuminance sensor. This allows the observer7to perceive a clear transparent image corresponding to the illuminance inside the cabin.

The display device6causes image data generated by the image data generator5to be displayed on the image display4. When the image display4is the retroreflective screen40, the display device6is a projector60. The projector60projects the image data generated by the image data generator5on the retroreflective screen40. The projector60includes a display panel61and a projection lens62. The projection lens62may be located to have its exit pupil adjacent to the eyes7L and7R of the observer7. When a screen of an image display device such as a liquid crystal display device is used as the image display4, the image display device includes the display device6. This eliminates the projector60.

The display panel61displays image data generated by the image data generator5. The display panel61may be a transmissive display panel or a self-luminous display panel. The transmissive display panel may be a liquid crystal display panel including transmissive liquid crystal display elements and a backlight. The self-luminous display panel may include self-luminous light emitters such as light-emitting diodes, organic light-emitting diodes, or semiconductor laser diodes. Image light emitted from the display surface of the display panel61is projected through the projection lens62onto the retroreflective screen40. The projection lens62may be a single lens or a combination of multiple lenses.

The image light of the transparent image data projected onto the retroreflective screen40by the projector60is retroreflected from the retroreflective screen40and enters the eyes7L and7R of the observer7. This allows the observer7to perceive the transparent image as connecting with the outside scenery.

In the image display apparatus1, the controller9may control the resolution of the imager2based on the position of the gaze point P detected by the gaze point detector3. The controller9may change the resolution of the imager2based on a distance D between the vehicle10and the gaze point P. The distance D between the vehicle10and the gaze point P may be a distance between the exterior camera21and the gaze point P. The distance D between the exterior camera21and the gaze point P may be a distance between the exterior camera21and the gaze point P in the length direction of the vehicle (Y-direction) as viewed in the height direction (Z-direction). In other words, the distance D may be a distance between the exterior camera21and the gaze point P in the length direction of the vehicle (Y-direction) in an XY plane.

Generation of a transparent image performed by the image display apparatus1will now be described.FIG.5illustrates the mutual positional relationship between the gaze point P of the observer7, the exterior camera21, and the projector60as viewed in the height direction (Z-direction). The image display apparatus1has a constant distance L between the projector60and the retroreflective screen40and a constant distance A between the retroreflective screen40and the imager2(exterior camera21). InFIG.5, the distance D between the exterior camera21and the gaze point P is X1 or X2 (X2>X1). The distance D, the distance L, and the distance A are in the length direction of the vehicle (Y-direction) in an XY plane. The image display apparatus1has a constant angle of view Θ of the exterior camera21and a constant throw ratio S of the projector60. An imaging range IR of the imager2and a projection range PR of the projector60are areas in a ZX plane.

With the gaze point P at position P1, or more specifically, with the distance D between the exterior camera21and the gaze point P being a predetermined distance X1, the image display apparatus1has an imaging range IR1of the imager2aligned with a projection range PR1of the projector60. This satisfies Formula 1 below. With the distance D between the exterior camera21and the gaze point P being X1, the image display apparatus1may have the resolution of the imager2matching the resolution of the projector60.

With the distance D between the exterior camera21and the gaze point P being X1, the projection range PR1has a width WPRin the width direction of the vehicle (X-direction) expressed by Formula 2 below, and the imaging range IR1has a width WIRin the width direction of the vehicle (X-direction) expressed by Formula 3 below.

With the distance D between the exterior camera21and the gaze point P being X1, Formula 1 holds, and thus WPR=WIR. With the projection range PR and the imaging range IR1aligned with each other, image data output from the imager2is projected onto the retroreflective screen40, allowing the observer7to perceive a high-definition transparent image as connecting with the outside scenery.

Generation and projection of transparent image data with the distance D between the exterior camera21and the gaze point P being X1 will now be described with reference toFIGS.6to9.FIG.6illustrates the outside view of the observer7seated on a seat of the vehicle10. The image display apparatus1inFIG.6is not in operation. In this state, the obstructive object8(dashboard81) partially obstructs the outside view of the observer7.FIG.7illustrates scenery image captured by the imager2. The image data generator5cuts a portion of the captured image data output from the imager2as illustrated inFIG.8to generate transparent image data to be projected onto the retroreflective screen40. The image data generator5may generate the transparent image data to be projected onto the retroreflective screen40based on, for example, the difference between the outside view of the observer7with the image display apparatus1not in operation (refer toFIG.6) and the scenery image captured by the imager2(refer toFIG.7). The projector60projects the image data generated by the image data generator5on the retroreflective screen40. This allows the observer7to perceive the transparent image as connecting with the outside scenery as illustrated in, for example,FIG.9.

With the gaze point P being at position P2, or more specifically, with the distance D between the exterior camera21and the gaze point P being a distance X2 (>X1), the projection range PR2has a width WPRin the width direction of the vehicle (X-direction) expressed by Formula 4 below, and the imaging range IR2has a width WIRin the width direction of the vehicle (X-direction) expressed by Formula 5 below.

With the distance D between the exterior camera21and the gaze point P being X2, the imaging range IR2of the imager2is larger than the projection range PR2of the projector60. When a portion of image data output from the imager2is cut and projected onto the retroreflective screen40in the same or similar manner as when the distance D between the exterior camera21and the gaze point P is X1, the observer7cannot perceive the transparent image as connecting with the outside scenery (refer toFIG.10). When a smaller portion is cut from the image data output from the imager2, the observer7can perceive a transparent image as connecting with the outside scenery. However, the transparent image generated in this manner has a lower resolution than the projector60, or for example, has the resolution of the projector60multiplied by (WPR/WIR)2. Thus, the observer7cannot perceive a high-definition transparent image (refer toFIG.11). When the resolution of the imager2is constantly set high, the observer7can perceive a high-definition transparent image as connecting with the outside scenery. However, the resolution of the imager2constantly set high increases the processing load of the image data generator5. This causes the observer7to be less likely to perceive a transparent image updated in real time and increases the power consumption of the image display apparatus1.

In the present embodiment, the image display apparatus1changes the resolution of the imager2based on the position of the gaze point P of the observer7. With the imaging range of the imager2aligned with the projection range of the projector60, the image display apparatus1allows the observer7to perceive a high-definition transparent image as connecting with the outside scenery as illustrated inFIG.9. With the imaging range of the imager2larger than the projection range of the projector60, the image display apparatus1changes the resolution of the exterior camera21and cuts, from the image data output from the imager2, a range smaller than the range that is cut when the imaging range of the imager2is aligned with the projection range of the projector60. In this manner, with the imaging range of the imager2being larger than the projection range of the projector60, the image display apparatus1allows the observer7to perceive a high-definition transparent image as connecting with the outside scenery in the same or similar manner as when the imaging range of the imager2is aligned with the projection range of the projector60.

The image display apparatus1may have a higher resolution of the imager2for a longer distance D between the imager2(exterior camera21) and the gaze point P. With the distance D between the exterior camera21and the gaze point P being k×X1 (k is a real number greater than or equal to 1), the image display apparatus1may set the resolution of the exterior camera21to be k times the resolution of the projector60. In this manner, with the distance D between the exterior camera21and the gaze point P different from the distance X1, the image display apparatus1can change the resolution of the exterior camera21based on the distance D to cause the resolution of the imaging range IR corresponding to the projection range PR of the projector60to match or substantially match the resolution of the projector60. The image display apparatus1thus allows the observer7to perceive a high-definition transparent image as connecting with the outside scenery when the position of the gaze point P of the observer7is changed.

The image display apparatus1may dynamically change the resolution of the imager2based on the distance D between the imager2(exterior camera21) and the gaze point P. This allows the resolution of the imager2to be optimized based on the distance D between the imager2(exterior camera21) and the gaze point P, reducing the processing load of the image display apparatus1. A transparent image to be perceived by the observer7can thus be processed at higher speed, allowing the observer7to perceive a natural transparent image. This can also reduce an increase in the power consumption of the image display apparatus1.

As illustrated in, for example,FIG.3, the image display apparatus1may include a seating sensor11. The seating sensor11is located on the driver's seat12of the vehicle10and detects the driver7being seated. The seating sensor11may be a known sensor such as a load sensor or a limit switch. In response to detecting the driver7being seated on the driver's seat12, the seating sensor11outputs a detection signal to the controller9. In response to the detection signal input from the seating sensor11, the controller9causes the gaze point detector3to start detecting the gaze point P of the driver7, and then causes the imager2, the image data generator5, and the projector60to start operating. This eliminates a switching operation performed by the driver7to cause the image display apparatus1to start operating, thus improving the convenience of the driver7.

The image display apparatus1may include no gaze point detector3. As illustrated in, for example,FIGS.12and13, the image display apparatus1including no gaze point detector3may include a ranging unit13. The ranging unit13may include, for example, a light detection and ranging (LiDAR), a millimeter wave radar, an ultrasonic sensor, or a stereo camera. The ranging unit13may be located at, for example, a front end of the vehicle10. The ranging unit13detects a relative position and a relative velocity of a target object located adjacent to the vehicle10with respect to the vehicle10. The target object may be an object that attracts the attention of the observer7among objects located adjacent to the vehicle10. The target object may be an object at the shortest distance from the vehicle10or an object at the longest distance from the vehicle10. The image display apparatus1may change the resolution of the imager2based on the position of the target object. The image display apparatus1including the ranging unit13in place of the gaze point detector3allows the observer7to perceive a high-definition transparent image as connecting with the outside scenery. The image display apparatus1may change the resolution of the imager2based on the distance D1between the vehicle10and a target object measured by the ranging unit13. The distance D1between the vehicle10and the target object may be a distance between the exterior camera21and the target object in the length direction of the vehicle (Y-direction) as viewed in the height direction (Z-direction). The image display apparatus1may have a higher resolution of the imager2for a longer distance D1.

The image display apparatus1may allow the observer7to perceive a transparent image that is a three-dimensional image (refer toFIG.14).

The imager2may capture an image of outside scenery and output first captured image data and second captured image data having parallax between them. The first captured image data and the second captured image data may have parallax corresponding to the eyes7L and7R of the observer7. The exterior camera21may be, for example, a stereo camera or a monocular camera. When the exterior camera21is a monocular camera, the imager2may perform an arithmetic operation on a single piece of captured image data and generate the first captured image data and the second captured image data having parallax between them.

The image data generator5generates image data of a first image to be perceived by the left eye (first eye)7L of the observer7based on the first captured image data output from the imager2. The image data generator5generates image data of a second image to be perceived by the right eye (second eye)7R of the observer7based on the second captured image data output from the imager2. The first image and the second image have parallax between them.

As illustrated in, for example,FIG.14, the projector60includes a first projector63and a second projector64. The first projector63projects the image data of the first image generated by the image data generator5onto the retroreflective screen40. The second projector64projects the image data of the second image generated by the image data generator5onto the retroreflective screen40. The first projector63and the second projector64may be located proximate to the headrest121of the seat12. The first projector63may be located to have its exit pupil adjacent to the left eye7L of the observer7. The second projector64may be located to have its exit pupil adjacent to the right eye7R of the observer7.

The image display apparatus1illustrated inFIG.14allows the observer7to stereoscopically perceive a high-definition transparent image as connecting with the outside scenery. The image display apparatus1may change the resolution of the imager2based on the position of the gaze point P of the observer7. The image display apparatus1may change the resolution of the imager2based on the distance D between the vehicle10and the gaze point P. The image display apparatus1may have a higher resolution of the imager2for a longer distance D.

An image display method according to one embodiment of the present disclosure will now be described.FIG.15is a flowchart of the image display method according to the present embodiment. The controller9performs the processing in the flowchart. The controller9may start the processing in the flowchart in response to a detection signal input from the seating sensor11. The controller9repeatedly performs the processing in the flowchart during the operation of the image display apparatus1.

In the present embodiment, the image display method includes detection S1, imaging S2, generation S3, and display S4.

The detection S1includes detecting the position of the gaze point P of the observer7seated on the seat12of the vehicle10. The controller9controls the gaze point detector3to cause the interior camera31to capture an image of the eyes7L and7R of the observer7and to detect the gaze point P of the observer7based on the captured image data about the eyes7L and7R.

The imaging S2includes capturing an image of scenery adjacent to the vehicle10with the imager2. The controller9controls the imager2to change the resolution based on the position of the gaze point P detected in the detection S1and then to capture an image of the scenery adjacent to the vehicle10. The controller9controls the imager2to change the resolution to cause the resolution of the imaging range IR of the imager2corresponding to the projection range PR of the projector60to match or substantially match the resolution of the projector60. The controller9may have a higher resolution of the imager2for a longer distance D between the vehicle10and the gaze point P.

The generation S3includes generating, based on the captured image data obtained in the imaging S2, image data (transparent image data) of an image corresponding to an outside view of the observer7seated on the seat12of the vehicle10and obstructed by the obstructive object8. The controller9controls the image data generator5to generate the transparent image data based on the captured image data obtained in the imaging S2. The controller9may control the image data generator5to cut a portion of the captured image data obtained in the imaging S2and generate the transparent image data based on the cut captured image data.

The display S4includes displaying the image data generated in the generation S3on the image display4located on the obstructive object8. The controller9controls the display device6to cause the transparent image data generated in the generation S3to be displayed on the image display4and ends the processing in the flowchart. When the image display4is the retroreflective screen40and the display device6is the projector60, the controller9controls the projector60to project the transparent image data onto the retroreflective screen40.

In the present embodiment, the image display method allows the observer7to perceive a high-definition transparent image as connecting with the outside scenery when the position of the gaze point P of the observer7is changed.

The present disclosure may be implemented in the following forms.

In one or more embodiments of the present disclosure, an image display apparatus includes an imager that captures an image of an area adjacent to a vehicle, a gaze point detector that detects a position of a gaze point of an observer seated on a seat of the vehicle, an image display located on an obstructive object obstructing an outside view of the observer, an image data generator that generates, based on captured image data output from the imager, image data of an image corresponding to the outside view of the observer obstructed by the obstructive object, and a display device that causes the image data to be displayed on the image display. The imager changes a resolution of the imager based on the position of the gaze point.

In one or more embodiments of the present disclosure, an image display method includes detecting a position of a gaze point of an observer seated on a seat of a vehicle, capturing an image of an area adjacent to the vehicle, generating, based on captured image data obtained in the capturing the image, image data of an image corresponding to an outside view of the observer seated on the seat of the vehicle obstructed by an obstructive object, and displaying the image data on an image display located on the obstructive object. The capturing the image includes changing a resolution of an imager based on the position of the gaze point.

In the embodiments of the present disclosure, the image display apparatus and the image display method allow the observer to perceive a high-definition transparent image as connecting with the outside scenery when the position of the gaze point of the observer is changed.

Although the embodiments of the present disclosure have been described in detail, the present disclosure is not limited to the embodiments described above, and may be changed or varied in various manners without departing from the spirit and scope of the present disclosure. The components described in the above embodiments may be entirely or partially combined as appropriate unless any contradiction arises.

REFERENCE SIGNS