Vehicle display system and method of controlling vehicle display system

A vehicle display system includes a display device, an authentication device, and a display controller. The display device is mounted on a vehicle. The authentication device is configured to perform authentication of an intended person present outside the vehicle. The display controller is configured to control, on a basis of a result of the authentication, the display device to perform displaying that is directed to the intended person.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority from Japanese Patent Application No. 2017-050346 filed on Mar. 15, 2017, the entire contents of which are hereby incorporated by reference.

BACKGROUND

The technology relates to a vehicle display system and a method of controlling the vehicle display system.

For example, Japanese Unexamined Patent Application Publication (JP-A) No. 2007-52719 discloses a technique that displays, on an outer surface of a display glass panel, i.e., a window of a vehicle, a message directed to a surrounding of the vehicle.

SUMMARY

An aspect of the technology provides a vehicle display system that includes: a display device mounted on a vehicle; an authentication device configured to perform authentication of an intended person present outside the vehicle; and a display controller configured to control, on a basis of a result of the authentication, the display device to perform displaying that is directed to the intended person.

An aspect of the technology provides a method of controlling a vehicle display system. The method includes: performing authentication of an intended person present outside a vehicle, the vehicle having a display device; and controlling, on a basis of a result of the authentication, the display device to perform displaying that is directed to the intended person.

An aspect of the technology provides a vehicle display system that includes: a display device mounted on a vehicle; and circuitry configured to perform authentication of an intended person present outside the vehicle, and control, on a basis of a result of the authentication, the display device to perform displaying that is directed to the intended person.

DETAILED DESCRIPTION

A technique disclosed in JP-A No. 2007-52719 involves difficulty in determining to whom a message displayed on outer side of a display glass panel is delivered.

It is desirable to provide a vehicle display system and a method of controlling the vehicle display system that are able to perform, on a display device mounted on a vehicle, displaying directed to a particular intended person outside the vehicle.

As described above, a technique disclosed in JP-A No. 2007-52719 involves difficulty in determining to whom a message displayed on outer side of a window of a vehicle is delivered. A further concern is that, in a situation where a driver stops the vehicle and waits inside the vehicle for the arrival of a meeting partner, it may take time for the meeting partner to find the intended vehicle when many vehicles are at a meeting place.

At least one implementation of the technology therefore provides a vehicle display system and a method of controlling the vehicle display system that are able to perform, on a display device mounted on a vehicle, displaying directed to a particular intended person outside the vehicle.

FIG. 1schematically illustrates an example of a configuration of a vehicle display system1000according to one implementation of the technology. The vehicle display system1000may basically relate to a vehicle and a person that uses the vehicle. The vehicle may be, for example but not limited to, an automobile. Referring toFIG. 1, the vehicle display system1000may include a vehicle-exterior sensor100, an operation input device202, a controller400, and a head-up display (HUD) device300that are mounted on a vehicle800. The vehicle display system1000may also include a mobile terminal200, a server,500, and surrounding vehicles600and700. The mobile terminal200may belong to a meeting partner900who is scheduled to meet with an occupant, such as a driver, of the vehicle800. As used herein, the term “occupant” encompasses the driver of the vehicle800.

The vehicle-exterior sensor100may be a stereo camera, a monocular camera, millimeter wave radar, an infrared sensor, or any other sensor usable for the vehicle800. The vehicle-exterior sensor100may measure a factor such as, but not limited to, a position and a speed of an object around the own vehicle800. Non-limiting examples of the object may include a person and a vehicle. In an example implementation where the vehicle-exterior sensor100is the stereo camera, the stereo camera may include a pair of right and left cameras each having an imaging device such as, but not limited to, a charge-coupled device (CCD) sensor and a complementary metal-oxide semiconductor (CMOS) sensor. The stereo camera may perform imaging of an external environment outside the vehicle800, and transmit information on thus-captured images to the controller400. In an example implementation, the stereo camera may be a color camera disposed at an upper part of a windshield of the vehicle800and able to acquire color information.

The operation input device202may be a touch panel and/or any other device that allows for input of information. The operation input device202may receive information on operation performed by the occupant such as, but not limited to, the driver and a fellow passenger. The occupant is able to input information on features of the meeting partner900through operating the operation input device202. The information on the features may be directed to color of clothes, height, and/or any other features that indicate the meeting partner900.

The HUD device300may provide information directly to a human visual field. The HUD device300may display a real image on a window such as, but not limited to, the windshield and a rear window of the vehicle800. Note that an HUD device typically displays a virtual image; however, the HUD device300according to one implementation may provide the real image, making it possible to visually recognize displaying from both inside and outside of the vehicle800owing to a viewing angle of substantially 360 degrees derived from the real image displayed by the HUD device300. In an alternative implementation, however, the HUD device300may display the virtual image.

In one specific but non-limiting implementation, the HUD device300may be a device that uses a self-luminous interlayer film1510as illustrated inFIG. 2. In such an implementation, the self-luminous interlayer film1510may have a configuration in which the self-luminous interlayer film1510is sandwiched between two sheets of glass1520that are disposed on the front and back of the self-luminous interlayer film1510, and may be disposed at the window of the vehicle800such as, but not limited to, the windshield and the rear window. The self-luminous interlayer film1510may contain a material that emits light, and part of the self-luminous interlayer film1510irradiated with laser light may thus emit light upon application of the laser light, thereby displaying characters and images. The laser light may be applied from a projector530provided inside the vehicle800. Such an object thus displayed may have a visibility from all angles and may be thus visually recognizable not only from a driver's seat but also from any seat besides the driver's seat and any location outside the vehicle800.

In an alternative implementation, the HUD device300may have a configuration in which a self-luminous device is disposed at the window of the vehicle800. In such an alternative implementation, the HUD device300may utilize a transparent screen based on an organic electroluminescence (EL) element, may utilize a transmissive liquid crystal device, or may utilize any other member that allows for transmissive displaying. In a further alternative implementation, any device other than the HUD device300may be used as a display device. In such an alternative implementation, a large-sized liquid crystal display device, a light-emitting diode (LED) display device, or any display device provided on a member such as, but not limited to, an instrument panel may be used as the display device. In the following, a description is given of one implementation in which the HUD device300performs displaying on the window, such as the windshield and the rear window, of the vehicle800. In one implementation, the HUD device300may serve as a “display device”. The display device according to any implementation of the technology, however, is not limited to the HUD device300. The term “HUD device” or “display device” according to any implementation of the technology encompasses any and all display devices provided inside and/or outside a vehicle. In an example implementation, the display device may be provided at any location outside the vehicle800other than the window, such as a body of the vehicle800. In an example implementation, the display device may be provided at any location inside the vehicle800other than the window, such as the instrument panel and a seat. The display device located inside the vehicle800and the display device located outside the vehicle800may be provided integrally, or may be provided separately. Accordingly, the wording “mounted on” the vehicle as used herein and its variants are intended to encompass both an implementation in which the display device is located inside the vehicle800and an implementation in which the display device is located outside the vehicle800.

The controller400may be provided in the vehicle800, and may include an environment information acquiring unit402, an environment state determiner404, an authentication device410, an HUD controller420, a communicator430, a positional information receiver440, and a storage450. In one implementation, the environment information acquiring unit402may serve as an “image information acquiring unit”. In one implementation, the environment state determiner404may serve as a “determiner”. In one implementation, the authentication device410may serve as an “authentication device”. In one implementation, the HUD controller420may serve as a “display controller”. The HUD controller420may control the HUD device300. The communicator430may perform communication with the mobile terminal200. The positional information receiver440may receive positional information, and may be, for example but not limited to, a global positioning system (GPS). The storage450may store information such as, but not limited to, registration information and registration image information. The registration information may be directed to authentication of identification information of the mobile terminal200, and registration image information may be directed to authentication of image information on the meeting partner900. The mobile terminal200may include a communicator210, a positional information receiver220, and a display230. The communicator210may perform communication with the communicator430of the controller400. The positional information receiver220may receive positional information, and may be, for example but not limited to, the GPS. The server500may be provided on a cloud without limitation, and may include a communicator510and a database520. The communicator510may perform communication with one or more of the communicator430provided in the controller400, the surrounding vehicle600, and the surrounding vehicle700. Note that one or more of elements of the controller400may be configured by hardware such as, but not limited to, a circuit. In an alternative implementation, one or more of the elements of the controller400may be configured by a processor and software such as, but not limited to, a program, that causes the processor to function. The processor may be, for example but not limited to, a central processing unit (CPU).

The environment information acquiring unit402may acquire information on a distance relative to a target object, i.e., distance information. In an example implementation, the environment information acquiring unit402may generate the distance information on the basis of an amount of shift between corresponding positions in a pair of right and left stereo images and by means of principle of triangulation. The pair of right and left stereo images may be acquired as a result of the imaging performed by a set of right and left cameras of the stereo camera that configures the vehicle-exterior sensor100in one implementation. Together with the acquisition of the distance information, the environment information acquiring unit402may also acquire positional information of the object, from image information obtained by the imaging performed by the set of right and left cameras of the stereo camera that configures the vehicle-exterior sensor100. Further, the environment information acquiring unit402may perform a known grouping process on the distance information thus generated by the principle of triangulation to detect, in the form of data, factors such as, but not limited to, a three-dimensional object and a lane line. The environment information acquiring unit402may perform the detection of the factors by comparing the distance information having been subjected to the grouping process with preset three-dimensional data such as, but not limited to, three-dimensional object data. By the processes performed by the environment information acquiring unit402, the controller400is also able to recognize factors such as, but not limited to, a person, another vehicle, a stop sign, a stop line, and an electronic toll collection (ETC) gate.

The environment information acquiring unit402may also calculate one or more of an amount of variation in distance relative to a factor and a speed relative to the factor, on the basis of the distance information relative to the factor generated by the principle of triangulation. Non-limiting examples of the factor may include a person and another vehicle. The amount of variation in distance may be determined by integrating a distance between frame images. The frame image may be acquired per unit time. The relative speed may be determined by dividing the distance acquired per unit time by the unit time.

In an example implementation where the vehicle-exterior sensor100is a camera, the image captured by the vehicle-exterior sensor100may be supplied to the controller400. The environment information acquiring unit402, serving as the image information acquiring unit in one implementation, may acquire a face region of the meeting partner900on the basis of the received image, and may acquire an orientation of the face of the meeting partner900on the basis of the face region. The environment information acquiring unit402may acquire the face region by means of an image processing technique such as, but not limited to, an edge detection and a face detection. The environment state determiner404may determine whether the meeting partner900notices the vehicle800on the basis of the orientation of the face.

FIG. 3schematically illustrates an example where calculation of factors, including an angle at which the face is oriented, is performed on the basis of a detected face region10of the meeting partner900. The environment state determiner404may determine whether the meeting partner900notices the vehicle800, by performing monitoring as to whether the orientation of the face of the meeting partner900is off a predetermined region on the basis of the face region10of the meeting partner900obtained from the image information. The face region10of the meeting partner900may be set on the basis of the positional information on feature points of respective sites of the face. Non-limiting examples of the sites of the face may include eyes, nose, and mouth.

FIG. 3also illustrates, at the bottom right, an example of a detection result20obtained by the environment information acquiring unit402. The detection result20may represent an “angle of face in vertical direction” on the vertical axis and an “angle of face in horizontal direction” on the horizontal axis. An “x” mark30denotes the detected orientation of the face.

Hence, the environment information acquiring unit402may acquire the image information that is obtained from the vehicle-exterior sensor100and directed to an environment outside the vehicle800. Further, the environment information acquiring unit402may perform an analysis of the thus-acquired image information on the basis of an image analyzing process. By performing the analysis of the image information, the environment information acquiring unit402may acquire environment information directed to the environment outside the vehicle800.

Incidentally, in an example situation where the occupant, such as the driver, stops the vehicle800and waits inside the vehicle800for the arrival of the meeting partner900, it may take time for the meeting partner900to find the intended vehicle800. In particular, it takes a long time for the meeting partner900to find the intended vehicle800especially when, for example, the meeting partner900sees the vehicle800belonging to the occupant for the first time, or the meeting partner900does not remember the vehicle800. It also takes a long time for the meeting partner900to find the intended vehicle800when there are many stopped vehicles, such as when the occupant meets the meeting partner900in front of a station, or any other location where many vehicles are present. Further, when the vehicle800is colored in white, black, etc., and a large-number of vehicles are present that are same in color as the vehicle800, it takes a long time for the meeting partner900to find the intended vehicle800as well.

In an example implementation, authentication is performed of a person outside the vehicle800, i.e., the meeting partner900, and the HUD device300is caused to perform displaying outside the vehicle800in accordance with a result of the authentication. In one specific but non-limiting implementation, when the authentication is successful as a result of the authentication and a person outside the vehicle800corresponds to a person registered in advance, a particular message directed to the person may be displayed outside the vehicle800. Thus, the meeting partner900is able to find the intended vehicle800easily and in a short time by seeing a window of the vehicle800.

The authentication of the meeting partner900is performed by the authentication device410of the controller400. The authentication device410may perform the authentication of the meeting partner900by comparing the identification information transmitted from the mobile terminal200belonging to the meeting partner900with the registration information registered in advance in the controller400. The authentication device410may perform the authentication of the meeting partner900by comparing the image information acquired by the vehicle-exterior sensor100with the registration image information registered in advance in the controller400. The authentication device410may determine, when the authentication is successful, whether the mobile terminal200belonging to the meeting partner900is near the own vehicle800.

FIGS. 4 to 6each schematically illustrate a state in which an image540is displayed on a windshield810of the vehicle800by means of the HUD device300, when the authentication performed by the authentication device410is successful. InFIGS. 4 to 6, a left part of each drawing of the vehicle800illustrates an example case where the image540is seen from the outside of the vehicle800, whereas a right part of each drawing of the vehicle800illustrates an example case where the image540is seen from the inside of the vehicle800.

Referring toFIG. 4, when the authentication is successful, the image540may be displayed at a position easily seen by the meeting partner900. Thus, the meeting partner900is able to find the intended vehicle800easily and in a short time by seeing the image540as illustrated inFIG. 5.

In an example implementation, the nickname such as “Taro” may be displayed as illustrated inFIG. 4when a permission to display the nickname is given in advance by the meeting partner900. The family name, or any other personally identifiable information, may be displayed alternatively when a permission is given in advance by the meeting partner900. In an alternative implementation, any image that makes sense only between the meeting partner900and the occupant such as the driver may be displayed on the basis of a prior arrangement made between the meeting partner900and the occupant.

FIG. 6schematically illustrates an example case where a priority is placed on driver's visibility by changing a factor of the image540such as a position and a size of the image540. In an example implementation illustrated inFIG. 6, the image540may be displayed small at an end of the windshield810, making it possible to increase the visibility when the driver sees the front while being seated.

FIG. 7schematically illustrates an example of the authentication that utilizes the surrounding vehicles600and700. The surrounding vehicles600and700each may have a configuration similar to that of the vehicle800, and may have the vehicle-exterior sensor100. The image information acquired by the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700may be transmitted to the server500. The image information transmitted to the server500may be transmitted from the server500to the controller400of the vehicle800. The environment information acquiring unit402, serving as the image information acquiring unit in one implementation, may acquire the image information transmitted to the controller400. The authentication device410of the controller400may perform the authentication of the meeting partner900by comparing the image information acquired by the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700with the registration image information registered in advance in the controller400. Performing the authentication that utilizes the surrounding vehicles600and700makes it possible to ensure that the authentication of the meeting partner900is performed even when the vehicle800is unable to perform the authentication of the meeting partner900directly.

For example, the vehicle-exterior sensor100of the vehicle800may sometimes not be able to acquire the image information of the meeting partner900in an example case where the meeting partner900is relatively distant from the vehicle800, or where the meeting partner900is at a blind spot of the vehicle-exterior sensor100of the vehicle800. In such circumstances, however, it is still possible to perform the authentication on the basis of the image information acquired by the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700.

In an example implementation, the image540may be made less conspicuous by changing a position, a size, and/or any other factor, of the displaying performed outside the vehicle800when the meeting partner900notices the intended vehicle800.FIG. 8schematically illustrates an example of a method of determining whether the meeting partner900notices the vehicle800, where both the vehicle800and the meeting partner900are seen from above. Referring toFIG. 8, detecting the meeting partner900by the vehicle-exterior sensor100and detecting the orientation of the face of the meeting partner900define an arrow A1that represents the orientation of the face with a position W of the meeting partner900being a starting point. Hence, the environment state determiner404may determine that the meeting partner900notices the vehicle800when the arrow A1points toward the vehicle800.

A description is given next, with reference to a flowchart illustrated inFIGS. 9A and 9B, of an example of a process performed in the vehicle display system1000according to one implementation. First, in step S10, a factor such as the meeting partner900and a meeting place may be set. In an example implementation, the occupant may input the information on the features of the meeting partner900through operating the operation input device202. The information on the features of the meeting partner900may be once stored in the storage450.

Thereafter, in step S12, the communicator430of the controller400may communicate with the mobile terminal200belonging to the meeting partner900, and the authentication device410may perform the authentication on the basis of the identification information of the mobile terminal200. Further, in step S12, the positional information of the mobile terminal200may be acquired, and the positional information of the mobile terminal200and the positional information of the vehicle800that is obtained by the positional information receiver440may be compared with each other. By performing the comparison, a determination may be made in step S12as to whether the mobile terminal200belonging to the meeting partner900is near the own vehicle800.

The flow may proceed to step S14when the authentication is successful and the mobile terminal200belonging to the meeting partner900is determined as being near the own vehicle800(S12: YES). In step S14, a determination may be made as to whether the permission to display the personally identifiable information, such as the family name and the nickname, has been given by the meeting partner900. The flow may proceed to step S16when the permission has been given (S14: YES). In an example implementation, information on whether the permission to display the personally identifiable information, such as the family name and the nickname, has been given may be stored in advance in the storage450. In an alternative implementation, the information on whether the permission has been given may be inputted by the occupant in step S10. In step S16, the displaying may be performed outside the vehicle800. The displaying may contain the personally identifiable information, such as the family name and the nickname, of the occupant of the vehicle800, the meeting partner900, or both.

The flow may proceed to step S18when the permission to display the personally identifiable information, such as the family name and the nickname, has not been given by the meeting partner900in step S14(S14: NO). In step S18, displaying linked with the mobile terminal200of the meeting partner900may be performed outside the vehicle800. In one specific but non-limiting implementation, in step S18, the HUD device300may display a predetermined mark, and information on the mark may be transmitted to the mobile terminal200to cause the display230of the mobile terminal200to display the same mark. This allows the meeting partner900to compare the mark displayed on the mobile terminal200with the mark displayed by the HUD device300, and to thereby find the intended vehicle800having the corresponding mark.

The flow may proceed to step S20after step S16or S18. In step S20, a determination may be made as to whether the meeting partner900notices the displaying performed outside the vehicle800. The determination in step S20may be made on the basis of the non-limiting method described above with reference toFIG. 8. The flow may proceed to step S22when the meeting partner900notices the displaying performed outside the vehicle800(S20: YES). In step S22, the displaying performed outside the vehicle800may be made less conspicuous by changing a factor such as the position and the size of the displaying. Thereafter, in step S24, the displaying performed outside the vehicle800may be cleared after the displaying is performed for “t3” seconds. The process may end after step S24is performed (END).

When a determination is made in step S20that the meeting partner900does not notice the displaying performed outside the vehicle800(S20: NO), the flow may proceed to step S25. In step S25, the communication may be performed with the mobile terminal200of the meeting partner900to provide the mobile terminal200with a notification. In an example implementation, a notification may be sent to the mobile terminal200that prompts the meeting partner900to see a window of the stopped vehicle800. After step S25, the flow may proceed to step S22in which the displaying performed outside the vehicle800may be made less conspicuous by changing a factor such as the position and the size of the displaying.

The flow may proceed to step S26when the authentication in step S12is not successful (S12: NO). In step S26, the authentication device410may compare the image information of the meeting partner900obtained from the information that is derived from the vehicle-exterior sensor100with the registration image information registered in advance. By performing the comparison, the authentication may be performed in step S26as to whether a person corresponding to the features of the meeting partner900is around the vehicle800. The flow may proceed to step S28when the person corresponding to the features of the meeting partner900is around the vehicle800(S26: YES). The flow may proceed to step S30when the person corresponding to the features of the meeting partner900is not around the vehicle800(S26: NO).

In step S30, a determination may be made as to whether there is a person corresponding to the features of the meeting partner900, on the basis of the image information obtained from the information that is derived from the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700. The flow may proceed to step S28when there is the person corresponding to the features of the meeting partner900on the basis of the image information obtained from the information derived from the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700(S30: YES). The process may end (END) when there is no person corresponding to the features of the meeting partner900on the basis of the image information obtained from the information derived from the vehicle-exterior sensor100of one or both of the surrounding vehicles600and700(S30: NO).

In step S28, in consideration of a fact that the person corresponding to the features of the meeting partner900is around the vehicle800on the basis of the image information obtained from the own vehicle800or from one or both of the surrounding vehicles600and700, a determination may be made as to whether the permission to display the personally identifiable information, such as the family name and the nickname, has been given by the meeting partner900. The flow may proceed to step S32when the permission has been given (S28: YES). In step S32, the displaying may be performed outside the vehicle800which contains the personally identifiable information, such as the family name and the nickname, of the occupant of the vehicle800, the meeting partner900, or both.

The flow may proceed to step S34when the permission to display the personally identifiable information, such as the family name and the nickname, has not been given by the meeting partner900in step S28(S28: NO). In step S34, an image, such as a mark, that makes sense between the meeting partner900and the occupant of the vehicle800may be displayed outside the vehicle800.

The flow may proceed to step S36after step S32or S34. In step S36, a determination may be made as to whether the meeting partner900notices the displaying performed outside the vehicle800. The determination in step S36may be made on the basis of the non-limiting method described above with reference toFIG. 8. The flow may proceed to step S38when the meeting partner900notices the displaying performed outside the vehicle800(S36: YES). In step S38, the displaying performed outside the vehicle800may be made less conspicuous by changing a factor such as the position and the size of the displaying.

When a determination is made in step S36that the meeting partner900does not notice the displaying performed outside the vehicle800(S36: NO), the flow may proceed to step S40. In step S40, the displaying performed outside the vehicle800may be continued for “t2” seconds, following which the displaying may be made less conspicuous by changing a factor such as the position and the size of the displaying. The flow may proceed to step S24after step S38or S40. In step S24, the displaying performed outside the vehicle800may be cleared after the displaying is performed for “t3” seconds. The process may end after step S24is performed (END).

According to one implementation described above, it is possible to perform, by a display device mounted on the vehicle800, the displaying directed to a particular intended person outside the vehicle800. Specifically, in one implementation, the authentication is performed of the meeting partner900outside the vehicle800as a non-limiting example of a particular intended person. Further, the displaying is performed by the HUD device300as a non-limiting example of the display device, when the authentication is successful. Hence, it is possible to ensure that the meeting partner900is able to find the intended vehicle800in a short time.

Although some implementations of the technology have been described in the foregoing with reference to the accompanying drawings, the technology is by no means limited to the implementations described above. It should be appreciated that modifications and alterations may be made by persons skilled in the art without departing from the scope as defined by the appended claims. The technology is intended to include such modifications and alterations in so far as they fall within the scope of the appended claims or the equivalents thereof.