Vehicle display system and method of controlling vehicle display system

A vehicle display system includes a display device, a position acquiring unit, and a display controller. The display device is configured to provide a display on a window of a vehicle. The position acquiring unit is configured to acquire a position of a target person present outside the vehicle. The display controller is configured to control the display device to invert the display to be provided for the target person, on the basis of the position of the target person.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from Japanese Patent Application No. 2017-050348 filed on Mar. 15, 2017, the entire contents of which are hereby incorporated by reference.

BACKGROUND

The technology relates to a vehicle display system and a method of controlling the vehicle display system.

Japanese Unexamined Patent Application Publication (JP-A) No. 2007-52719 discloses an exemplary vehicle display device that displays a partly negative image.

SUMMARY

An aspect of the technology provides a vehicle display system that includes: a display device configured to provide a display on a window of a vehicle; a position acquiring unit that acquires a position of a target person present outside the vehicle; and a display controller that controls the display device to invert the display to be provided for the target person, on a basis of the position of the target person.

An aspect of the technology provides a method of controlling a vehicle display system. The method includes: acquiring a position of a target person present outside a vehicle, in which the vehicle includes a display device that provides a display on a window; and controlling the display device to invert the display to be provided for the target person, on a basis of the position of the target person.

An aspect of the technology provides a vehicle display system that includes: a display device configured to provide a display on a window of a vehicle; and circuitry configured to acquire a position of a target person present outside the vehicle, and controls the display device to invert the display to be provided for the target person, on a basis of the position of the target person.

DETAILED DESCRIPTION

When providing a display for a target person outside a vehicle, if a driver or other person inside the vehicle does not recognize a content of this display, for example, an appropriate display may not be provided to the target person outside the vehicle. A technique disclosed in JP-A No. 2007-52719 is, however, intended to invert a display provided for a driver to display a negative image and thus not intended to provide a display for a target person outside a vehicle.

It is desirable to provide a vehicle display system and a method of controlling the vehicle display system that both enable a person inside a vehicle to reliably recognize a display to be provided for a target outside the vehicle.

FIG. 1schematically illustrates an example of a configuration of a vehicle display system1000according to one implementation of the technology. The vehicle display system1000may be basically mounted in a vehicle. The vehicle may be, for example but not limited to, an automobile. Referring toFIG. 1, the vehicle display system1000may include vehicle-exterior sensors100, a driver sensor200, seating sensors300, a controller400, a head-up display (HUD) device500, speakers600, and seat vibrators700. In one implementation, the speakers600may serve as a “warning sound generator”.

Each of the vehicle-exterior sensors100may be a stereo camera, a monocular camera, millimeter wave radar, an infrared sensor, or any other sensing device. Each vehicle-exterior sensor100may measure a parameter such as, but not limited to, a position and speed of an object. Non-limiting examples of the object may include a vehicle and a person around the vehicle. In an example implementation, each vehicle-exterior sensor100may be a stereo camera that includes a pair of right and left cameras each having an imaging device such as, but not limited to, a charge-coupled device (CCD) sensor or a complementary metal-oxide semiconductor (CMOS) sensor. Then, the stereo camera may capture an image of an environment outside a vehicle and transmit information on the captured image to the controller400. In an example implementation, the stereo camera may be a color camera disposed on an upper portion of a windshield of the vehicle and be able to acquire color information.

The HUD device500may be a display device that displays information directly in a human visual field. The HUD device500may display a real image on a window such as, but not limited to, the windshield or a rear window of the vehicle. Although a known HUD device typically displays a virtual image, the HUD device500in this implementation may be a display device that displays a real image. The HUD device500may display a real image at a viewing angle of about 360 degrees, thereby allowing both persons inside and outside the vehicle to view the image. In an alternative implementation, however, the HUD device500may display a virtual image.

In one specific but non-limiting implementation, the HUD device500may be a device that includes a self-luminous interlayer film510, as illustrated inFIG. 2. In such an implementation, the self-luminous interlayer film510may be disposed between two sheets of glass520in the window of the vehicle such as, but not limited to, the windshield and the rear window. The self-luminous interlayer film510may contain a luminescence material. When a portion of the self-luminous interlayer film510is irradiated with laser light from a projector530mounted in the vehicle, the irradiated portion may emit light, displaying objects, including characters and images. The objects may have visibility at all angles and may be thus viewable not only from the driver seat but also from the other seats as well as any position outside the vehicle. In an alternative implementation, the HUD device500may have a configuration in which a self-luminous device is disposed at the window of the vehicle. In such an alternative implementation, the HUD device500may have a transparent screen that contains an organic electroluminescence (EL) element, a transmissive liquid crystal device, or any other member that allows for transmissive displaying. In a further alternative implementation, any device other than the HUD device500may be used as the display device. In such an alternative implementation, a large-sized liquid crystal display device, a light-emitting diode (LED) display device, or any display device provided on a member such as, but not limited to, an instrument panel may be used as the display device. In the following, a description is given of one implementation in which the HUD device500provides a display on the window, such as the windshield and the rear window, of the vehicle. In one implementation, the HUD device500may serve as a “display device”. The display device according to any implementation of the technology, however, is not limited to the HUD device500. The term “HUD device” or “display device” according to any implementation of the technology encompasses any and all display devices provided inside and/or outside a vehicle. In an example implementation, the display device may be provided at any external site of the vehicle other than the window, such as a body of the vehicle. In an example implementation, the display device may be provided at any internal site of the vehicle other than the window, such as the instrument panel and a seat. The display devices disposed inside and outside the vehicle may be provided integrally with or separately from each other.

The driver sensor200may be implemented by a camera, a line-of-sight sensor, a motion sensor, or any other sensing device. This driver sensor200may measure movements of the driver's arms and head, a direction of his/her line of sight, and any other parameters related to the driver. In an example implementation where the driver sensor200is a camera, the driver sensor200may acquire the movements of the arms and head, the direction of the line of sight, and any other parameters by subjecting an image captured by the camera to image processing. In another example implementation where the driver sensor200is a line-of-sight sensor, the driver sensor200may detect the line of sight by using a corneal reflection method or any other similar method. The seating sensors300may be provided in the respective seats of the vehicle, and each of the seating sensors300may determine whether a person sits on a corresponding seat. The speakers600may emit a warning sound toward the inside and outside of the vehicle when the HUD device500displays a warning to persons inside and outside of the vehicle. The seat vibrators700may be provided in the respective seats inside the vehicle. In addition, the seat vibrators700may vibrate the seats to give a warning to the driver and occupants when the HUD device500displays a warning inside the vehicle.

The controller400may control the displaying performed by the HUD device500on the basis of the information detected by the vehicle-exterior sensors100, the driver sensor200, and other sensors. The controller400may include an environment information acquiring unit402, an environment state determiner407, a distance determiner410, and a HUD controller414. In one implementation, the environment information acquiring unit402may serve as a “position acquiring unit”. In one implementation, the HUD controller414may serve as a “display controller”. Each of the components of the controller400may be implemented in hardware or in software. In an example implementation where one component of the controller400is implemented in hardware, this component may be circuitry. In another example implementation where one component of the controller400is implemented in software, this component may be a program that causes a central processing unit (CPU) or other processor to perform its function.

The environment information acquiring unit402may acquire information on a distance to a target, namely, a subject in this case. This information may be called distance information. In an example implementation, the environment information acquiring unit402may generate the distance information, on the basis of an amount of shift between corresponding positions in a pair of right and left stereo images and by means of principle of triangulation. The pair of right and left stereo images may be captured by a set of right and left cameras of the stereo camera which constitutes each vehicle-exterior sensor100in one implementation. Together with the acquisition of the distance information, the environment information acquiring unit402may also acquire information on a position of the subject from the information on the images captured by the set of right and left cameras of the stereo camera which constitutes each vehicle-exterior sensor100. This information may be called image information. Further, the environment information acquiring unit402may perform a known grouping process on the distance information generated by the principle of triangulation. Then, the environment information acquiring unit402may obtain data on a three-dimensional object, a lane line, and other objects by comparing the distance information subjected to the grouping process with preset three-dimensional data such as, but not limited to, three-dimensional object data. By performing the processes in this manner, the controller400is able to recognize various entities, including a person, another vehicle, a stop sign, a stop line, and an electronic toll collection (ETC) gate.

On the basis of the information on the distance to a factor generated by the principle of triangulation, the environment information acquiring unit402may also calculate a variation in the distance to the factor and/or a speed relative to the factor. Non-limiting examples of the factor may include a person and another vehicle. The variation in the distance may be determined by integrating distances to the factor in frame images. The frame images may be acquired at regular time intervals. The relative speed may be determined by dividing the distances acquired at regular time intervals by the unit time.

In the above manner, the environment information acquiring unit402may acquire the image information regarding an environment outside the vehicle, from the vehicle-exterior sensors100. In addition, the environment information acquiring unit402may analyze the acquired image information through an image analyzing process. As a result of analyzing the image information, the environment information acquiring unit402may acquire environment information regarding the environment outside the vehicle.

FIG. 3schematically illustrates an example of a manner in which an angle at which a face of a target person outside a vehicle is oriented and other face-related factors are calculated on the basis of the face region10of the target person. On the basis of a face region10of a person900which has been acquired from the image information, the environment state determiner407may monitor whether the face orientation of the person900falls outside a predetermined region. In such an implementation, the face region10of the person900may be defined from the position information on the feature points of the parts, including the eyes, the nose, and the mouth, of the face. In order to estimate the face orientation, as an example, the controller400may use, as a reference, a distance between the eyes as seen from the front and then may compare this reference with a distance between the eyes obtained from the image information. It may be determined that, as the distance between the eyes obtained from the image information becomes smaller than the reference, the face is oriented sideways more largely as seen from the front.

FIG. 3illustrates a detection result20at its lower right. In the detection result20, the vertical axis represents “vertical angle of face orientation”, the horizontal axis represents “horizontal angle of face orientation”, and an X-mark30denotes “detected angles of face orientation”.

The environment state determiner407in the controller400may determine an environment state outside the vehicle on the basis of the environment information acquired by the environment information acquiring unit402. Further, the environment state determiner407may determine a state of a person or a vehicle as a target outside the vehicle. The distance determiner410may determine a distance between the vehicle and the target outside the vehicle, on the basis of the distance information acquired by the environment information acquiring unit402. The environment state determiner407may determine whether the person outside the vehicle is aware of the vehicle. In one implementation, the environment state determiner407may serve as a “determiner”.

In one implementation, the HUD controller414may switch between displays to be performed by the HUD device500, on the basis of a situation around the vehicle, in such a way that persons inside and outside the vehicle are able to realize for which person the display is intended. For example, the HUD controller414may switch between the displays in such a way that the persons inside and outside the vehicle are able to realize for which person the display is intended, by controlling the HUD device500to perform a display based on an animation that involves inversion of the display or by controlling the HUD device500to perform a display based on inversion of positive and negative images, on the basis of the situation around the vehicle.

In one specific but non-limiting implementation, the above situation around the vehicle may contain a distance to the target outside the vehicle. On the basis of the distance to the target, the distance determiner410may determine an urgent level. If the determined urgent level is high, the HUD controller414may control the HUD device500to provide a display for the target outside the vehicle immediately. However, if the urgent level is low, the HUD controller414may control the HUD device500to perform, before providing the display for the target outside the vehicle, a process of displaying the animation in which the display is inverted or a process of displaying the positive and the negative images in the inverted fashion. The driver is thereby expected to recognize a content of the display. Performing the inverted display in this manner makes it possible to help the driver or a person inside the vehicle perceive the content of the display which is intended even for the person outside the vehicle. Hence, if the urgent level is low, the HUD controller414may control the HUD device500to perform the inverted display so that the driver or other person inside the vehicle is able to reliably recognize the content of the display.

The situation around the vehicle may further contain information indicating whether the target is aware of the vehicle. If the target is not aware of the vehicle, the speaker600may make a horn or other sound in order to attract target's attention, after which the HUD controller414may control the HUD device500to provide the display for the target outside the vehicle. This makes it possible to help the target become aware of the display provided for the outside of the vehicle.

FIG. 4schematically illustrates an example of a display, according to one implementation, based on the animation that involves the inversion of the display. In one example illustrated inFIG. 4, the HUD controller414may control the HUD device500to provide a display802of a message saying “START MOVING” on the windshield810of the vehicle upon starting of the vehicle. The HUD controller414may control the HUD device500to invert front and back sides of the display802repeatedly, and may eventually stop inverting the front and the back sides of the display802to provide the display802for the outside of the vehicle. Performing the display based on the animation that involves the inversion of the display in this manner enables a person inside the vehicle to reliably recognize the display802when performing the display to be provided for the outside of the vehicle.

FIG. 5schematically illustrates an example of a display, according to one implementation, based on the inversion of the positive and the negative images. In one example illustrated inFIG. 5, the HUD controller414may control the HUD device500to display, on the windshield810of the vehicle, the display802of an arrow mark indicating a position of a person present outside the vehicle or a moving direction of the vehicle, for example. The HUD controller414may control the HUD device500to invert an image of the display from the positive image to the negative image and vice versa repeatedly, and may eventually stop inverting the positive and the negative images to provide the display802for the outside of the vehicle. Performing the display based on the inversion of the positive and the negative images in this manner enables a person inside the vehicle to reliably recognize the display802when performing the display to be provided for the outside of the vehicle.

When the person outside a vehicle800is aware of the vehicle800, the HUD controller414may control the HUD device500to change a position and size of the display provided for the outside of the vehicle, thereby making the displayed image less conspicuous.FIG. 6schematically illustrates an example of a method of determining whether the person900outside the vehicle800is aware of the vehicle800.FIG. 6illustrates states of the vehicle800and the person900as seen from the top. As illustrated inFIG. 6, the vehicle-exterior sensors100may detect the person900. Then, the environment information acquiring unit402may detect a face orientation of the person900. On the basis of these detection results, the environment state determiner407may determine an arrow mark A1indicating the face orientation which starts from a position W of the person900. If the arrow mark A1extends in a direction toward the vehicle800, the environment state determiner407may determine that the person900is aware of the vehicle800.

Next, a description is given of an example of a process performed by the vehicle display system1000according to one implementation, with reference to a flowchart ofFIG. 7. The process ofFIG. 7may be performed by the components of the controller400and may be repeated during predetermined control periods. In step S10, the environment state determiner407may determine whether a target, e.g., the person900who potentially views a display to be provided by the HUD device500is present outside the vehicle, on the basis of the information acquired from the vehicle-exterior sensors100. When the environment state determiner407determines in step S10that the target is present outside the vehicle (S10: YES), the flow may proceed to step S12in which the environment information acquiring unit402acquires a distance and direction to the person900. Thereafter, in step S14, the distance determiner410may determine whether a distance L from the vehicle800to the person900is equal to or less than a predetermined threshold X. When the distance determiner410determines that the distance L is less than the threshold X (S14: YES), the distance determiner410may determine that the urgent level is high, after which the flow may proceed to step S16. In step S16, the HUD controller414may control the HUD device500to provide the display802for the person900. When the distance determiner410determines that the distance L is equal to or greater than the threshold X (S14: NO), the flow may proceed to step S24. In step S24, the HUD controller414may control the HUD device500to perform the inverted display, e.g., may perform the display based on the animation that involves the inversion of the display as illustrated inFIG. 4or may perform the display based on the inversion of the positive and the negative images as illustrated inFIG. 5, thereby indicating, to the driver, the display802to be provided for the outside of the vehicle. After step S24has been performed, the flow may proceed to step S16, in which the HUD controller414may control the HUD device500to provide the display802for the target.

After step S16has been performed, the flow may proceed to step S18, in which the environment state determiner407may determine whether the person900is aware of the vehicle800. For example, the environment state determiner407may employ the method illustrated inFIG. 6to determine whether the person900is aware of the vehicle800. When the environment state determiner407determines that the person900is aware of the vehicle800(S18: YES), the flow may proceed to step S20. In step S20, the distance determiner410may determine whether the distance to the target increases. In one example implementation, when the distance to the target is equal to or greater than another predetermined threshold that is greater than the threshold X, the distance determiner410may determine that the distance to the target increases. When the distance to the target increases (S20: YES), the flow may proceed to step S22. In step S22, the HUD controller414may control the HUD device500to clear the display, and may terminate the process in this control period (END).

When the environment state determiner407determines that the person900is not aware of the vehicle800in step S18(S18: NO), the flow may proceed to step S26. In step S26, the speakers600may make a horn or other sound in order to make the target become aware of the vehicle. After step S26has been performed, the flow may proceed to step S20.

When the distance to the target does not increase in step S20(S20: NO), the HUD controller414may control the HUD device500to clear the display, and may terminate the process in this control period (END). The HUD controller414may also terminate the process in this control period (END) when the environment state determiner407determines in step S10that the target is not present outside the vehicle (S10: NO).

According to one implementation, as described above, the vehicle display system1000, when providing a display for a person outside a vehicle, inverts the display, thereby enabling a driver or other person inside the vehicle to reliably recognize a content of the display to be provided for the person outside the vehicle. Further, in one implementation, if the urgent level is high, such as when the person is approaching the vehicle, the vehicle display system1000may provide the display for the person outside the vehicle without inverting the display, thereby enabling the person outside the vehicle to reliably recognize a content of the display.

Although some implementations of the technology have been described in the foregoing with reference to the accompanying drawings, the technology is by no means limited to the implementations described above. It should be appreciated that modifications and alterations may be made by persons skilled in the art without departing from the scope as defined by the appended claims. The technology is intended to include such modifications and alterations in so far as they fall within the scope of the appended claims or the equivalents thereof.