Patent ID: 12246598

DETAILED DESCRIPTION

Explanation follows regarding a vehicle12to which a vehicle display control device10according to an exemplary embodiment has been applied, with reference toFIG.1toFIG.7.

As illustrated inFIG.1, an instrument panel14is provided at a front part inside a vehicle cabin of the vehicle12. The instrument panel14extends in the vehicle width direction, and a steering wheel16is provided at a vehicle right side of the instrument panel14. Namely, in the present exemplary embodiment, as an example, a right hand drive vehicle is configured with a steering wheel16provided on the right side, and a driver's seat28is installed on the right side of the vehicle.

A windshield glass18is provided at a front end part of the instrument panel14. The windshield glass18extends in the vehicle vertical direction and the vehicle width direction to partition the vehicle cabin interior from an exterior of the vehicle cabin.

In front of the driver's seat28, the windshield glass18is provided with a head-up display area26(hereafter simply referred to as the “display area26”). The display area26is configured by a projection plane that is projected by a head-up display device48(seeFIG.2) serving as a vehicle display device. More specifically, the head-up display device48is provided at a vehicle front side of the instrument panel14, and an image is projected onto the display area26of the windshield glass18from the head-up display device48. Namely, the display area26is a part of the windshield glass18serving as a projection plane of the head-up display device48.

Note that the vehicle12is provided with a vehicle display control device10. The vehicle display control device10of the present exemplary embodiment is configured by, for example, one or more electronic control units (ECUs).

Hardware Configuration of the Vehicle Display Control Device10

FIG.2is a block diagram illustrating a hardware configuration of the vehicle display control device10. As illustrated inFIG.2, the vehicle display control device10includes a central processing unit (CPU; serving as a processor)30, read only memory (ROM)32, random access memory (RAM)34, storage36, a communication interface (communication I/F)38, and an input/output interface (input/output I/F)40. These respective configurations are communicably connected to each other via an internal bus42.

The CPU30is a central processing unit that executes various programs and controls various components. Namely, the CPU30reads a program from the ROM32or the storage36, and executes the program using the RAM34as a workspace. The CPU30controls the respective configurations described above and performs a variety of computation processing in accordance with programs stored in the ROM32or the storage36.

The ROM32stores various programs and various data. The RAM34serves as a workspace to temporarily store programs and data. The storage36is configured by a hard disk drive (HDD) or a solid state drive (SSD), and stores various programs including an operating system, as well as various data.

The communication I/F38is an interface for the vehicle display control device10to communicate with an external server and other devices and, for example, a protocol such as a controller area network (CAN), Ethernet (registered trademark), long term evolution (LTE), a fiber distributed data interface (FDDI), or Wi-Fi (registered trademark) is used.

The input/output I/F40is electrically connected to an image capturing device44, a navigation device46, the head-up display device48, and an object detection sensor50.

The image capturing device44is configured by, for example, an onboard camera, and captures the surroundings (forward, rearward, and sides) of the vehicle12. The navigation device46includes a non-illustrated global positioning system (GPS) sensor and map data, and structures a known navigation system. The navigation device46identifies a current position of the vehicle12based on a signal from a global positioning system (GPS) sensor, and, based on map data, sets a travel path from the current position of the vehicle12to a destination set by a user.

The head-up display device48causes display of predetermined information at the display area26of the windshield glass18. The information displayed by the head-up display device48includes a superimposed image. The superimposed image is an AR image. The AR image is an image that is drawn using augmented reality technology. Namely, the view ahead of the vehicle12is visible at the display area26through the windshield glass18, and a superimposed image is displayed superimposed on this view. The information displayed as the superimposed image includes path information relating to the travel path set by the navigation device46and information relating to a driving system during operation. In the present exemplary embodiment, in particular, a superimposed image indicating the position of an object that is present ahead of the vehicle12is displayed at the display area26.

The object detection sensor50detects an object present in the periphery of the vehicle12. The object detection sensor50can be configured by, for example, one or a combination of well-known sensors typified by an onboard camera, an ultrasonic sensor, a millimeter-wave radar, a LIDAR sensor, or the like.

Functional Configuration of the Vehicle Display Control Device10

The vehicle display control device10implements various functions using the above-described hardware resources. Explanation follows regarding functional configurations implemented by the vehicle display control device10, with reference toFIG.3.

As illustrated inFIG.3, the functional configurations of the vehicle display control device10includes a recognition section60, a distance calculation section62, an image generating section64, and a display control section66. The respective functional configurations are implemented by the CPU30reading and executing programs stored in the ROM32or the storage36.

The recognition section60recognizes an object that is present ahead of the vehicle12based on peripheral information of the vehicle12. As an example, the recognition section60acquires peripheral information based on an image capturing ahead of the vehicle12, which is captured using the image capturing device44, and recognizes an object that is present ahead of the vehicle12. The recognition section60recognizes, for example, a vehicle such as a four-wheel vehicle or a two-wheel vehicle, a bicycle, a pedestrian, or the like, as an object. In the present exemplary embodiment, the recognition section60recognizes an object that is present in a travel lane of the vehicle12.

Moreover, the recognition section60recognizes a travel lane of the vehicle12. More specifically, the recognition section60recognizes a travel lane from a boundary such as a partition line, asphalt, grass, soil, or a curb, based on an image capturing ahead of the vehicle12, which is captured using the image capturing device44.

The distance calculation section62calculates a distance between the vehicle12and an object ahead of the vehicle12. As an example, the distance calculation section62calculates a distance between the vehicle12and an object ahead of the vehicle12based on a signal received from the object detection sensor50.

The image generating section64generates a superimposed image that is displayed at the display area26. The superimposed image is displayed so as to be superimposed on the object recognized by the recognition section60in a case in which the object is viewed through the display area26. The superimposed image is displayed so as to indicate, to the user, the position of the object that is present ahead of the vehicle12. This superimposed image is displayed during operation of a driving system typified by a known advanced driving assistant system (ADAS), such as a stop system or a start system for avoiding contact between a detected object and the vehicle12, an adaptive cruise control (ACC) system, a lane tracing assistance (LTA) system, or a lane change assistance (LCA) system.

Note that the superimposed image is configured by two respective images, displayed so as to be spaced apart from each other in the vehicle width direction, at each side of an object ahead of the vehicle. The respective images are designed to have lengths in the vehicle vertical direction. For example, as illustrated inFIG.4, the superimposed image80is configured by an image in which respective linear images having lengths in the vehicle vertical direction are displayed so as to be spaced apart from each other in the vehicle width direction at each side of an object (vehicle ahead V). Note that the design of the superimposed image80is merely an example, and various embodiments can be implemented. The superimposed image according to the present disclosure is not limited to a pair of linear images, as long as it is an embodiment in which respective superimposed images have lengths in the vehicle vertical direction at each side of the object in the vehicle width direction, as in the illustrated example. Note that the case referred to herein as “having lengths in the vehicle vertical direction” is a broad concept including a case in which the superimposed images have lengths in a direction inclined with respect to an actual vehicle vertical direction.

In a case in which plural objects are present ahead of the vehicle12, the image generating section64recognizes the plural objects as one superimposition target and causes display of the superimposed images according to a predetermined condition. Namely, as illustrated inFIG.5, respective superimposed images having lengths in the vehicle vertical direction are displayed spaced apart from each other at each side of plural objects (two-wheel vehicles B1and B2) in the vehicle width direction. More specifically, in a case in which plural objects are present ahead of the vehicle12in a range in which a distance L to the vehicle12is less than a first distance L1, the image generating section64recognizes the plural objects as a single superimposition target. The image generating section64then generates respective superimposed images displayed at each side of the plural objects in the vehicle width direction. Moreover, after displaying the respective superimposed images at each side of the plural objects, in a case in which at least one of the plural objects moves in a direction away from the vehicle12, under a condition in which the distance L between the object and the vehicle12is equal to or greater than the first distance L1and less than a second distance L2, the image generating section64continuously recognizes the plural objects as a single superimposition target. For example, the first distance L1is set to 30 m, and the second distance L2is set to 50 m.

The display control section66causes display of the superimposed images generated by the image generating section64at the display area26so as to be superimposed on a view ahead of the vehicle. Namely, the display control section66causes display of the respective superimposed images, having lengths in the vehicle vertical direction, at each side of the object, recognized by the recognition section60, in the vehicle width direction, the superimposed images being spaced apart from each other in the vehicle width direction. The superimposed images having lengths in the vehicle vertical direction are images that are long in the depth direction of the display area26when viewed from a user seated in the driver's seat28. Accordingly, the superimposed images having lengths in the vehicle vertical direction are a design that makes it difficult to perceive a shift in the display position of the superimposed images as compared to a case in which an image that is long in the vehicle width direction is displayed superimposed on an object, and also enables the rendering performance of perspective to be improved.

The display control section66changes the distance between the respective superimposed images displayed at each side of the object in the vehicle width direction in accordance with the distance between the host vehicle and the object. More specifically, the greater the distance between the host vehicle and the object, the smaller the distance between the respective superimposed images displayed at each side of the object in the vehicle width direction. This enables the superimposed images to be displayed in accordance with the size of the object that decreases as the distance from the host vehicle increases.

Moreover, the display control section66causes display of the lengths of the superimposed images in the vehicle vertical direction to be constant regardless of the distance between the host vehicle and the object. Accordingly, when an object becomes further away from the host vehicle, the distance between the respective images displayed at each side of the object in the vehicle width direction becomes smaller as a whole in the superimposed image, while keeping the length in the vehicle vertical direction constant enables display consistency to be maintained.

Explanation follows regarding an example, with reference toFIG.4AtoFIG.4C.FIG.4AtoFIG.4Cillustrate a display area26provided at the windshield glass18. As illustrated in these figures, a vehicle ahead V that is detected as an object present ahead of the vehicle12is visible through the windshield glass18through the display area26. Further, the superimposed image80is displayed at the display area26so as to be superimposed on the vehicle ahead V. The superimposed image80is displayed so that images80A and80B, which have lengths in the vehicle vertical direction, are arranged spaced apart from each other at each side of the vehicle ahead V in the vehicle width direction. As illustrated inFIG.4AtoFIG.4C, the superimposed image80changes so as to decrease the distance between the images80A and80B in the vehicle width direction as the vehicle ahead V becomes further away. On the other hand, regardless of the distance between the vehicle12and the vehicle ahead V, the lengths of the images80A and80B in the vehicle vertical direction are displayed with a constant length.

As illustrated inFIG.5AtoFIG.5D, in a case in which plural objects are present ahead of the vehicle12, the display control section66recognizes the plural objects as a single superimposition target according to a predetermined condition, and causes display of the superimposed image80. For example, inFIG.5A, a single two-wheel vehicle B1is present in a range in which a distance L to the vehicle12is less than the first distance L1(=30 m) in a travel lane70of the vehicle12. Further, a single two-wheel vehicle B2is present in a range in which a distance L to the vehicle12is equal to or greater than the first distance L1and less than the second distance L2(=50 m). In this case, the display control section66recognizes only the one two-wheel vehicle B1as a superimposition target, and causes display of the respective superimposed images80at each side of the two-wheel vehicle B1in the vehicle width direction. Moreover, as illustrated inFIG.5B, in a case in which the two-wheel vehicle B2approaches the vehicle12and the two two-wheel vehicles B1and B2are present in a range in which the distance L to the vehicle12is less than the first distance L1, the two two-wheel vehicles B1and B2are recognized as a single superimposition target. As a result, the display control section66causes display of the respective superimposed images80at each side of the two two-wheel vehicles B1and B2in the vehicle width direction. Then, when the single two-wheel vehicle B2become further away from the vehicle12by the first distance L1or more, as illustrated inFIG.5C, in a case in which the two two-wheel vehicles B1and B2are present in a range that is equal to or greater than the first distance L1and that is less than the second distance L2from the vehicle12, the respective superimposed images80are displayed at each side of the two two-wheel vehicles B1and B2in the vehicle width direction. As illustrated inFIG.5D, after the at least one two-wheel vehicle B2become further away from the vehicle12by the second distance L2or greater, the two-wheel vehicle B1is recognized as a superimposition target again, and the respective superimposed image80are displayed at each side of the two-wheeled vehicle B1in the vehicle width direction. As described above, in a case in which plural objects are detected ahead of the vehicle12, hysteresis is imparted to the display of the superimposed image, and the complexity felt by the user can be reduced by frequently changing the superimposition target.

Operation of Display Processing

Explanation follows regarding an example of operation of display processing executed by the vehicle display control device10, with reference to the flowchart illustrated inFIG.6. The display processing is executed by the CPU30reading a program from the ROM32or the storage36and loading the program in the RAM34.

As illustrated inFIG.6, at step S10, the CPU30acquires peripheral information for the vehicle12. More specifically, the image capturing device44acquires an image capturing ahead of the vehicle12.

At step S11, based on a signal from the object detection sensor50, the CPU30determines whether or not an object is present ahead of the vehicle12. In a case in which an object is present, the determination of step S11is affirmative, and the CPU30proceeds to the processing of step S12. On the other hand, in a case in which no object is present, the determination at step S11is negative, and the CPU30ends the display processing.

At step S12, the CPU30executes image generation processing to generate a superimposed image. The image generation processing is described below.

At step S13, the CPU30outputs the image generated by the image generation processing. More specifically, the CPU30transmits the generated image data to the head-up display device48, and the transmitted image data is output by the head-up display device48so as to be displayed at the display area26.

At step S14, the CPU30determines whether or not an object is no longer present ahead of the vehicle12. A case in which an object is no longer present, also includes a case in which the object detection sensor50no longer detects an object following a stop of the corresponding driving system. In a case in which an object is no longer present ahead of the vehicle12, the determination at step S14is affirmative, and the CPU30ends the display processing. In a case in which an object is present ahead of the vehicle12, the CPU30returns to the processing of step S12.

Image Generation Processing

Next, explanation follows regarding an example of operation of the image generation processing, with reference toFIG.7. As illustrated in this figure, at step S20, the CPU30determines whether or not plural objects are present ahead of the vehicle12. In a case in which plural objects are present, the determination at step S20is affirmative, and the processing proceeds to step S23which is described below. On the other hand, in a case in which plural objects are not present, namely, in a case in which one object (singular object) is present, the determination of step S20is negative, and the CPU30proceeds to the processing of step S21.

At step S21, the CPU30calculates the distance L between the vehicle12and the object ahead.

At step S22, the CPU30generates respective superimposed images arranged at each side of a singular object, in the vehicle width direction, which are present ahead of the vehicle12. More specifically, as illustrated inFIG.4AandFIG.5A, the CPU30generates a superimposed image80in which respective superimposed images80A and80B having lengths in the vehicle vertical direction are spaced apart from each other at each side of a single object (V, B1), ahead of the vehicle12, in the vehicle width direction.

The superimposed image80generated at step S22is as illustrated inFIG.4AtoFIG.4C. The distance between the respective superimposed images80A and80B, which are displayed at each side of the singular object in the vehicle width direction, changes in accordance with the distance between the vehicle12and the singular object (V). On the other hand, the superimposed image80is displayed with a length of the superimposed image80(80A,80B) in the vehicle vertical direction constant regardless of the distance L between the vehicle12and the singular object. Note that as an example, the length of the superimposed image80(80A,80B) in the vehicle vertical direction is set with reference to at least one of the distance L between the vehicle12and the object, or the size of the object.

Next, explanation follows regarding a case in which it is determined at step S20that plural objects are present ahead of the vehicle12, and the processing proceeds to step S23. At step S23, the CPU30determines whether or not plural objects are present in a range in which the distance L to the vehicle12is less than the first distance L1(=30 m). In a case in which plural objects are present in a range that is less than the first distance L1, the determination of step S23is affirmative, and the CPU30proceeds to the processing of step S24. On the other hand, in a case in which the determination at step S23is negative, the processing proceeds to step S21. In this case, after the processing of the following step S22, as illustrated inFIG.5A, a superimposed image80is generated which is superimposed on an object (two-wheel vehicle B1) closest to the vehicle12.

At step S24, the CPU30recognizes plural objects as a single superimposition target, and calculates the distance between the vehicle12and the plural objects. For example, from among the plural objects, an object with a distance L that is closest to the vehicle12is calculated.

At step S25, as illustrated inFIG.5B, the CPU30generates respective superimposed images80arranged at each side of plural objects (two two-wheel vehicles B1and B2) which are present ahead of the vehicle12.

At step S26, the CPU30determines whether or not the plural objects that generated the superimposed image80at step S25are present in a range in which the distance L to the vehicle12is equal to or greater than the first distance L1and less than the second distance L2(=50 m). In a case in which the plural objects are present within a range that is equal to or greater than the first distance L1and less than the second distance L2from the vehicle12, the determination at step S26is affirmative, and the CPU30proceeds to the processing at step S13of the display processing. Accordingly, as illustrated inFIG.5C, even in a case in which a single two-wheel vehicle B2serving as an object ahead of the vehicle has become further away from the vehicle12by a first distance L1or greater, display of the respective superimposed images80arranged at each side of the two two-wheel vehicles B1and B2in the vehicle width direction continues at the display area26.

On the other hand, in a case in which the plural objects are not present within the range equal to or greater than the first distance L1and less than the second distance L2from the vehicle12at step S26, the determination at step S26is negative, and the CPU30proceeds to the processing at step S21. Accordingly, as illustrated inFIG.5D, the CPU30recognizes only one two-wheel vehicle B1that is closest to the vehicle12as a superimposition target, and generates respective superimposed images80arranged at each side of the two-wheel vehicle B1in the vehicle width direction.

As described above, the vehicle display control device10according to the present exemplary embodiment causes display of respective superimposed images having lengths in the vehicle vertical direction, at each side in the vehicle width direction of an object that is present ahead of the vehicle12, so as to be superimposed on a view ahead of the vehicle12(host vehicle), the superimposed images being spaced apart from each other in the vehicle width direction. As a result, as illustrated inFIG.4, when the distance between the vehicle12and the object changes, the user hardly perceives a shift in the display position of the superimposed image80with respect to the object ahead of the vehicle12. Moreover, since the superimposed images80having lengths in the vehicle vertical direction are long in the depth direction of the display image, the user is less likely to perceive limitations in the rendering performance of perspective. As a result, it is difficult to give a visual effect such as the superimposed image80appearing to float at a position ahead of the object in a case in which the object is distant from the vehicle12. In this manner, the vehicle display control device10can reduce the sense of discomfort of the user in a case in which the superimposed image is displayed so as to be superimposed on an object located ahead of the host vehicle.

Moreover, in the present exemplary embodiment, the distance between the respective superimposed images80displayed at each side of the object in the vehicle width direction is changed in accordance with the distance between the host vehicle and the object. As illustrated inFIG.4, the greater the distance to the object ahead of the vehicle12, the smaller the object that is visible from the position of the vehicle12, and therefore, the distance between the respective superimposed images80(80A,80B) displayed at each side of the object in the vehicle width direction is changed so as to be smaller. This enables the user to easily intuitively perceive a change in the distance between the vehicle12and the object, enabling the rendering performance of perspective to be improved.

Moreover, in the present exemplary embodiment, as illustrated inFIG.4, regardless of the distance between the vehicle12and the object, the length of the superimposed image80in the vehicle vertical direction is displayed with a constant length. This enables the consistency of display to be maintained, enabling the user to easily understand an object ahead of the vehicle12based on the display of the superimposed image80.

Although explanation has been given regarding the vehicle display control device10according to the exemplary embodiments, obviously various embodiments may be implemented within a range not departing from the gist of the present disclosure. For example, although the superimposed image is generated for an object that is present in the travel lane70of the vehicle12in the above-described exemplary embodiments, there is no limitation thereto, and the superimposed image80may be displayed for an object that is present outside the travel lane70or an object detected at a location where the travel lane70is not present.

Further, although an example in which an image is displayed at the display area26of the head-up display device48has been described, there is no limitation thereto. For example, a superimposed image may be displayed at a display or the like that is provided at the instrument panel. Namely, the superimposed image is displayed so as to be superimposed on an image capturing ahead of the vehicle12, and is output to the display. Moreover, the display area may be a display of an external device or the like. For example, in a case in which the display may be a display of an operation device that is operated by an operator who remotely drives the vehicle12from a remote location, an image capturing ahead of the vehicle12is received by the operation device that is connected via a network, and a superimposed image is displayed on the received image. Accordingly, the user in the above-described exemplary embodiments may be an occupant inside a vehicle cabin or an operator at a remote location.

Moreover, in the above-described exemplary embodiments, any of various types of processors other than the CPU30may execute the processing that the CPU30executes by reading a program. Examples of such processors include a Programmable Logic Device (PLD) in which the circuit configuration can be modified post-manufacture, such as a Field-Programmable Gate Array (FPGA), or a specialized electric circuit that is a processor with a specifically-designed circuit configuration for executing specific processing, such as an Application Specific Integrated Circuit (ASIC). Further, the display processing and the image generation processing may be executed by one of these various types of processors, or may be executed by combining two or more of the same type or different types of processors, for example, the above-described processing may be executed by plural FPGAs, or a combination of a CPU and an FPGA, or the like. Moreover, a hardware configuration of the various processors is specifically formed as an electric circuit combining circuit elements such as semiconductor elements.

In addition, although various data is stored in the storage36in the above-described exemplary embodiments, there is no limitation thereto. For example, a non-transitory storage medium such as a compact disc (CD), a digital versatile disc (DVD), or universal serial bus (USB) memory may act as a storage section. In this case, various programs, data, and the like are stored in these storage media.