Patent Description:
In the vehicle control device disclosed in Japanese Patent Application Laid-Open (<CIT>, in a case in which it is judged that an autonomous vehicle cannot merge while autonomous driving is continued, the driver of the own vehicle is given advance notice that he/she must carry out manual operation in order to merge the own vehicle from the traveling lane into the lane to be merged into (merging destination lane). This warning is carried out by the outputting of an image onto a display or by voice output from a speaker. Documents <CIT>, <CIT> and <CIT> are also known and disclose systems for control of an autonomous vehicle including lane merging systems. Also, document <CIT> discloses a safe warning system for automatic driving takeover.

The above-described related art does not disclose how to concretely give warning of the necessity of driver intervention into the driving operation, by output of an image onto a display or voice output from a speaker.

In view of the above-described circumstances, an object of the present invention is to provide an onboard display control device, an onboard display device, a display control method and a display control program in which, in a case in which merging by autonomous driving is attempted, it is easy for the vehicle occupant to understand the need to intervene in the driving operation.

An onboard display control device of a first aspect of the present invention is defined in claim <NUM>.

In the first aspect, the onboard display control device that is installed in a vehicle configured to enable autonomous driving has the information acquisition section and the display control section. In a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, the information acquisition section acquires information regarding a point at which the attempt will be aborted, before the vehicle arrives at that point. The display control section displays the information regarding the point that is acquired by the information acquisition section, before the arrival at the point, at a display portion that is provided within the vehicle cabin of the vehicle. Due thereto, the occupant of the vehicle can visually grasp up to when the attempt to merge in autonomous driving will continue. As a result, the vehicle occupant can easily understand, in advance, the necessity of intervening in the driving operation.

In an onboard display control device of a second aspect of the present invention, in the first aspect, the information acquisition section is configured to acquire information regarding the traveling lane, and, on the basis of the information acquired by the information acquisition section, the display control section displays, at the display portion, an image in which an image of the traveling lane and an image of the point are combined.

In the second aspect, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, the information acquisition section acquires information regarding the point at which the attempt will be aborted, and information regarding the traveling lane. On the basis of the information acquired by the information acquisition section, the display control section displays, at the display portion, an image in which an image of the traveling lane and an image of the point are combined. Due thereto, the occupant of the vehicle can more easily visually grasp up to when the attempt to merge in autonomous driving will continue.

In an onboard display control device of a third aspect of the present invention, in the second aspect, the display control section displays, at the display portion, an image of a fixed segment that is established in the traveling lane and that has the point as a starting point.

In the third aspect, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, an image, which combines an image of the traveling lane and an image of a fixed segment that is established in the traveling lane and that has the point as a starting point at which the attempt will be aborted, is displayed at the display portion. Due thereto, the occupant of the vehicle can visually recognize this point easily as compared with a case in which, for example, only the point is displayed at the display portion.

In an onboard display control device of a fourth aspect of the present invention, in the third aspect, the fixed segment includes a driving hand-over request segment in which at least one of steering or acceleration of the vehicle is controlled by the vehicle, and in which an occupant of the vehicle is urged to intervene in driving operation.

In the fourth aspect, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, an image of a fixed segment, which is established in the traveling lane and that has the point as a starting point at which the attempt will be aborted, is displayed at the display portion. This fixed segment includes a driving hand-over request segment in which at least one of steering or acceleration of the vehicle is controlled by the vehicle, and in which an occupant of the vehicle is urged to intervene in the driving operation. Because this driving hand-over request segment is displayed at the display portion in advance, for example, it is easy for the occupant of the vehicle to make, in advance, preparations to intervene in the driving operation.

In an onboard display control device of a fifth aspect of the present invention, in the third or fourth aspect, the fixed segment includes a braking segment in which the vehicle is automatically stopped in a case in which an occupant of the vehicle does not intervene in driving operation.

In the fifth aspect, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, an image of a fixed segment, which is established in the traveling lane and that has the point as a starting point at which the attempt will be aborted, is displayed at the display portion. This fixed segment includes a braking segment in which the vehicle is automatically stopped in a case in which an occupant of the vehicle does not intervene in the driving operation. Due thereto, in a case in which the occupant of the vehicle does not intervene in the driving operation, the point at which braking of the vehicle will start, and the point at which automatic stoppage will occur, and the like can be known in advance.

In an onboard display control device of a sixth aspect of the present invention, in any one of the third aspect through the fifth aspect, the display control section displays the fixed segment as a belt-shaped image at the display portion.

In the sixth aspect, the display control section displays, at the display portion, an image in which an image of the traveling lane and a belt-shaped image are combined. The fixed segment that is shown as this belt-shaped image is a segment that has the point as a starting point at which the attempt to merge in autonomous driving will be aborted. Because the fixed segment is displayed as a belt-shaped image in this way, it is easy for the occupant of the vehicle to visually recognize the point as compared with a case in which, for example, the point is displayed at the display portion as a dot-shaped or a linear image.

In an onboard display control device of a seventh aspect of the present invention, in any one of the third aspect through the sixth aspect, the display control section displays, at the display portion, an image of the fixed segment in forms that differ for plural stages in accordance with a deferment time until the vehicle is stopped automatically in a case in which an occupant of the vehicle does not intervene in driving operation.

In the seventh aspect, the image of the fixed segment, which the display control section displays at the display portion, is displayed in forms that differ for plural stages in accordance with a deferment time until the vehicle is stopped automatically in a case in which the occupant of the vehicle does not intervene in the driving operation. Due thereto, the occupant of the vehicle can easily recognize the deferment time until automatic stoppage.

In an onboard display control device of an eighth aspect of the present invention, in any one of the third aspect through the seventh aspect, the display control section displays, at the display portion, an image of the fixed segment in forms that differ for plural stages in correspondence with a change in control of the autonomous driving that is carried out in a case in which an occupant of the vehicle does not intervene in driving operation.

In the eighth aspect, the image of the fixed segment, which the display control section displays at the display portion, is displayed in forms that differ for plural stages in correspondence with the change in the control of the autonomous driving that is carried out in a case in which the occupant of the vehicle does not intervene in the driving operation. Due thereto, in a case in which the occupant of the vehicle does not intervene in the driving operation, the occupant of the vehicle can easily recognize in advance that control of the autonomous driving will change.

In an onboard display control device of a ninth aspect of the present invention, in any one of the first aspect through the eighth aspect, in a case in which the vehicle arrives at the point, the display control section displays, at the display portion, a driving hand-over request image for urging an occupant of the vehicle to intervene in driving operation.

In the ninth aspect, in a case in which the vehicle arrives at the point at which the attempt to merge in autonomous driving is aborted, the display control section displays, at the display portion, the driving hand-over request image that is for urging an occupant of the vehicle to intervene in the driving operation. Due thereto, it is even easier for the occupant of the vehicle to recognize the necessity of intervening in the driving operation.

In an onboard display control device of a tenth aspect of the present invention, in any one of the first aspect through the ninth aspect, the display portion includes a head-up display that displays an image so as to overlap a front view that an occupant of the vehicle sees through a windshield.

In the tenth aspect, in a case in which the vehicle attempts to merge in autonomous driving, information regarding the point at which the attempt will be aborted can be displayed so as to overlap the front view that the occupant of the vehicle sees through the windshield. Due thereto, the occupant of the vehicle can recognize the information of this point, while viewing the front view through the windshield.

In an onboard display control device of an eleventh aspect of the present invention, in any one of the first aspect through the tenth aspect, the display portion includes a meter display that is provided at an instrument panel of the vehicle.

In the eleventh aspect, in a case in which the vehicle attempts to merge in autonomous driving, information regarding the point at which the attempt will be aborted is displayed on the meter display that is provided at the instrument panel. Due thereto, the occupant of the vehicle can confirm the information regarding the point in the meter display, even in a situation in which it is difficult to see the image displayed on the head-up display due to the effects of sunlight or the like for example.

In an onboard display control device of a twelfth aspect of the present invention, in any one of the first aspect through the eleventh aspect, the display control section displays the vehicle and a periphery thereof at the display portion as an image that models a state in which the vehicle and the periphery thereof are seen from an obliquely rear upper side of the vehicle.

In accordance with the twelfth aspect, the vehicle and the periphery thereof are displayed, at the display portion provided at the vehicle, as an image that models a state in which the vehicle and the periphery thereof are viewed from an obliquely rear upper side of the vehicle, and information regarding the point at which the attempt to merge during autonomous driving will be aborted is displayed at the display portion provided at the vehicle. As a result, the occupant of the vehicle can, while looking at an image of the own vehicle from a bird's eye view, confirm up to when the attempt to merge in autonomous driving will continue.

An onboard display device of a thirteenth aspect of the present invention is an onboard display device that is installed in a vehicle configured to enable autonomous driving, the device including: a display portion that is provided within a vehicle cabin of the vehicle; and the onboard display control device of any one of the first aspect through the twelfth aspect.

In the thirteenth aspect, the onboard display device that is installed in a vehicle configured to enable autonomous driving has a display portion that is provided within a vehicle cabin of the vehicle, and an onboard display control device. Because this onboard display control device is the device of any one of the first aspect through the twelfth aspect, the operation and effects that are described above are obtained.

A display control method of a fourteenth aspect of the present invention is defined in claim <NUM>.

In the fourteenth aspect, the display control method that is executed by a processor that is installed in a vehicle configured to enable autonomous driving has the acquiring step and the displaying step. In the acquiring step, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, information regarding a point at which the attempt will be aborted is acquired before the vehicle arrives at the point. In the displaying step, the information regarding the point that is acquired in the acquiring step is displayed before the arrival at the point, at a display portion that is provided within the vehicle cabin of the vehicle. Due thereto, the occupant of the vehicle can visually grasp up to when the attempt to merge in autonomous driving will continue. As a result, the vehicle occupant can easily understand in advance the necessity of intervening in the driving operation.

A display control program of a fifteenth aspect of the present invention is defined in claim <NUM>.

In the fifteenth aspect, the display control program, which is executed by a processor that is installed in a vehicle configured to enable autonomous driving, has the acquiring step and the displaying step. In the acquiring step, in a case in which the vehicle attempts to merge from a traveling lane into a merging destination lane during autonomous driving, information regarding a point at which the attempt will be aborted is acquired before the vehicle arrives at the point. In the displaying step, the information regarding the point that is acquired in the acquiring step is displayed before the arrival at the point, at a display portion that is provided within the vehicle cabin of the vehicle. Due thereto, the occupant of the vehicle can visually grasp up to when the attempt to merge in autonomous driving will continue. As a result, the vehicle occupant can easily understand in advance the necessity of intervening in the driving operation.

As described above, in the onboard display control device, the onboard display device, the display control method and the display control program relating to the present invention, in a case in which merging in autonomous driving is attempted, it is easy for a vehicle occupant to understand the need to intervene in the driving operation.

An onboard display device <NUM> relating to a first embodiment of the present invention is described hereinafter with reference to <FIG>. This onboard display device <NUM> is installed in a vehicle <NUM> that is shown in <FIG>. The vehicle <NUM> is structured so as to be able to switch between autonomous driving and manual driving. Hereinafter, the vehicle <NUM> will be called the "own vehicle <NUM>".

As shown in <FIG>, an instrument panel <NUM> is provided at the front portion of the vehicle cabin interior of the own vehicle <NUM>. The instrument panel <NUM> extends in the vehicle transverse direction, and a steering wheel <NUM> is provided at the vehicle right side of the instrument panel <NUM>. Namely, in the present embodiment, as an example, the own vehicle <NUM> is a right-hand-drive vehicle in which the steering wheel <NUM> is provided at the right side, and an unillustrated driver's seat is disposed at the right side within the vehicle cabin.

The instrument panel <NUM> is disposed at the vehicle lower side of a windshield glass <NUM>. The windshield glass <NUM> is inclined so as to slope downward toward the vehicle front side, and divides the vehicle cabin interior and the vehicle cabin exterior. This windshield glass <NUM> corresponds to the "windshield" in the present disclosure.

The vehicle right side end portion of the windshield glass <NUM> is fixed to a front pillar <NUM> that is at the vehicle right side. The front pillar <NUM> extends in the vehicle vertical direction, and the windshield glass <NUM> is fixed to the vehicle transverse direction inner side end portion of the front pillar <NUM>. Further, a front side glass <NUM> is disposed at the vehicle rear side of the front pillar <NUM>. Note that the vehicle left side end portion of the windshield glass <NUM> is fixed to an unillustrated front pillar that is at the vehicle left side.

A meter display <NUM> that is the first display portion is provided at the upper portion of the instrument panel <NUM> at the vehicle front side of the driver's seat. The meter display <NUM> is provided at a position that is within the visual field in the state in which the sightline of the vehicle occupant who is in the driver's seat of the own vehicle <NUM> (hereinafter, simply called "occupant of the own vehicle <NUM>") faces toward the vehicle front side.

Further, a head-up display (refer to <FIG>; not shown in <FIG>) <NUM> that is the second display portion is provided at the upper portion of the instrument panel <NUM> at the vehicle front side of the driver's seat. This head-up display (called "HUD" hereinafter) <NUM> is structured such that an image can be projected onto a display region 26A that is set at the lower portion of the windshield glass <NUM> at the vehicle front side of the driver's seat. The HUD <NUM> can display an image so as to be superposed on the front view that the occupant of the own vehicle <NUM> sees through the windshield glass <NUM>.

The structure of an onboard system <NUM> that is installed in the above-described own vehicle <NUM> is shown in a block drawing in <FIG>. The onboard system <NUM> has a communication bus <NUM>. A peripheral situation acquiring device group <NUM>, a vehicle state detecting sensor group <NUM>, an autonomous driving ECU (Electronic Control Unit) <NUM>, and a display control ECU <NUM> are respectively connected to the communication bus <NUM>. The display control ECU <NUM>, together with the meter display <NUM> and the HUD <NUM>, structure the onboard display device <NUM>. Note that <FIG> shows only a portion of the onboard system <NUM>.

The peripheral situation acquiring device group <NUM> includes a GPS (Global Positioning System) device <NUM>, an onboard communicator <NUM>, a navigation system <NUM>, a radar device <NUM>, a camera <NUM> and the like, as devices that acquire information expressing what type of situation the peripheral environment of the own vehicle <NUM> is.

The GPS device <NUM> receives GPS signals from plural GPS satellites, and measures the position of the own vehicle <NUM>. The accuracy of measurement of the GPS device <NUM> increases as the number of GPS signals that can be received increases. The onboard communicator <NUM> is a communication device that carries out at least one of vehicle-to-vehicle communication with other vehicles and road-to-vehicle communication with roadside devices. The navigation system <NUM> includes a map information storing section 40A that stores map information. On the basis of the position information obtained from the GPS device <NUM> and the map information that is stored in the map information storing section 40A, the navigation system <NUM> carries out processes such as displaying the position of the own vehicle <NUM> on a map, and guiding the own vehicle <NUM> along the route to the destination.

The radar device <NUM> includes plural radar devices that have respectively different detection ranges, and detects peripheral objects, such as pedestrians and other vehicles and the like that exist at the periphery of the own vehicle <NUM>, as point group information, and acquires the relative positions and the relative speeds of the own vehicle <NUM> and the detected peripheral objects. The radar device <NUM> incorporates therein a processing device that processes the results of detecting peripheral objects. On the basis of changes in the relative positions and the relative speeds of the individual peripheral objects that are included in the detection results of the most recent several times, and the like, this processing device excludes, from objects of monitoring, noise, roadside objects such guardrails and the like, and the like, and tracks specific objects such as pedestrians, other vehicles and the like as objects of monitoring. Then, the radar device <NUM> outputs information such as the relative positions and the relative speeds with respect to the individual objects of monitoring, and the like. The camera <NUM> captures images of the periphery of the own vehicle <NUM> by plural cameras, and outputs the captured images.

The vehicle state detecting sensor group <NUM> includes, as plural sensors that acquire the state of the own vehicle <NUM>, a vehicle speed sensor <NUM> that detects the vehicle speed of the own vehicle <NUM>, an acceleration sensor <NUM> that detects the acceleration that is applied to the own vehicle <NUM>, a steering sensor <NUM> that detects the steering angle of the own vehicle <NUM>, an accelerator sensor <NUM> that detects the amount of operation of the accelerator pedal of the own vehicle <NUM>, and a brake sensor <NUM> that detects the amount of operation of the brake pedal of the own vehicle <NUM>.

A throttle ACT <NUM> that changes the throttle opening of the own vehicle <NUM>, and a brake ACT <NUM> that changes the braking force that the braking device of the own vehicle <NUM> generates, are connected to the autonomous driving ECU <NUM>. Further, a steering ACT <NUM> that changes the amount of steering by the steering device of the own vehicle <NUM> is connected to the autonomous driving ECU <NUM>.

The autonomous driving ECU <NUM> includes a CPU (Central Processing Unit), memories such as a RAM (Random Access Memory), a ROM (Read Only Memory) and the like, a non-volatile storage such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) or the like, and a communication I/F (Inter Face). Due to the CPU executing an autonomous driving program, in a case in which an autonomous driving mode is selected, the autonomous driving ECU <NUM> carries out autonomous driving processing that causes the own vehicle <NUM> to travel autonomously without being accompanied by driving operation of the occupant of the own vehicle <NUM>. The autonomous driving processing is processing that, on the basis of information obtained from the peripheral situation acquiring device group <NUM> and the vehicle state detecting sensor group <NUM>, judges the situations of the own vehicle <NUM> and the periphery thereof, and controls the throttle ACT <NUM>, the brake ACT <NUM> and the steering ACT <NUM>.

In the present embodiment, the level of the autonomous driving that the autonomous driving ECU <NUM> carries out is a level <NUM> or a level <NUM> for example. In level <NUM> and level <NUM> autonomous driving, for example, in cases of deviating from a controllable range, or in advance consideration of operations that are improper due to misdetection, lack of detection, or malfunctioning of the sensors, or the like, the driver monitors the autonomous driving by the autonomous driving ECU <NUM>, and intervention of the driver is requested as needed. Note that the level of autonomous driving that the autonomous driving ECU <NUM> carries out may be a level <NUM> or a level <NUM>. In level <NUM> or level <NUM> autonomous driving, intervention by the driver is unnecessary.

Further, in the present embodiment, the autonomous driving that the autonomous driving ECU <NUM> carries out includes, for example, constant speed traveling at which the own vehicle <NUM> is made to travel in the same traveling lane at a constant speed, traveling while following in which the own vehicle <NUM> is made to follow another vehicle that is the closest to the own vehicle <NUM> and that exists in the same traveling lane within a predetermined distance (e.g., within <NUM>) in front of the own vehicle <NUM>, lane changing in which the own vehicle <NUM> is made to change lanes from the lane in which the own vehicle <NUM> is currently traveling (the own lane) to an adjacent lane, traveling while branching-off in which, at a branch in the road, the own vehicle <NUM> is made to branch-off from the traveling lane into the lane that branches-off, traveling while merging in which, at the merge region of a road, the own vehicle <NUM> is made to merge from the traveling lane into the lane to be merged into (merging destination lane), and the like.

In a case in which the above-described traveling while merging is to be carried out, the autonomous driving ECU <NUM> judges whether or not merging is possible with the autonomous driving being continued, on the basis of information such as the road shape of the traveling lane, the situations of other vehicles that are in merging destination lane, and the like that are acquired by the peripheral situation acquiring device group <NUM>. In this case, for example, the autonomous driving ECU <NUM> acquires information such as the inter-vehicle distance between other vehicles that exist in the merging destination lane, the relative speeds between the own vehicle <NUM> and the other vehicles that exist in the merging destination lane, the final point of the traveling lane that the own vehicle <NUM> is traveling in, and the like, and carries out the aforementioned judgment by combining these information. As a result, in a case in which it is judged that the own vehicle <NUM> cannot merge with the autonomous driving being continued, the autonomous driving ECU <NUM>, by using the onboard display device <NUM>, requests that the occupant of the own vehicle <NUM> intervene in the driving operation. Due to this request, when the occupant of the own vehicle <NUM> operates at least any of the steering wheel, the accelerator pedal and the brake pedal of the own vehicle <NUM>, that operation is detected by at least any of the steering sensor <NUM>, the accelerator sensor <NUM> and the brake sensor <NUM>. Due thereto, the autonomous driving ECU <NUM> carries out switching from the autonomous driving mode to the manual driving mode. The structure of the onboard display device <NUM> is described in detail hereinafter.

The onboard display device <NUM> is structured by the aforementioned meter display <NUM> and HUD <NUM>, and the display control ECU <NUM>. The display control ECU <NUM> corresponds to the "onboard display control device" in the present disclosure. The display control ECU <NUM> includes a CPU (Central Processing Unit: processor) <NUM>, a memory <NUM> such as a ROM (Read Only Memory) and a RAM (Random Access Memory) and the like, a non-volatile storage <NUM> such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) or the like, and a communication I/F (Inter Face) <NUM>. The CPU <NUM>, the memory <NUM>, the storage <NUM> and the communication I/F <NUM> are connected so as to be able to communicate with one another via an internal bus <NUM>. A display control program <NUM> is stored in the storage <NUM>. This display control program <NUM> is a program for executing the display control method relating to the present disclosure. At the display control ECU <NUM>, display control processing that is described later is carried out by the display control program <NUM> being read-out from the storage <NUM> and expanded in the memory <NUM>, and the display control program <NUM> that has been expanded in the memory <NUM> being executed by the CPU <NUM>. The HUD <NUM> and the meter display <NUM> are electrically connected to the display control ECU <NUM>.

The display control ECU <NUM> realizes various functions by using the above-described hardware resources. Some of the functional structures that are realized by the display control ECU <NUM> are shown in a block drawing in <FIG>. As shown in <FIG>, the display control ECU <NUM> includes, as the functional structures thereof, an information acquisition section <NUM> and a display control section <NUM>. These functional structures are realized by the CPU <NUM> reading-out and executing the display control program <NUM> that is stored in the storage <NUM>.

The information acquisition section <NUM> acquires, from the autonomous driving ECU <NUM>, information of the lane and information of peripheral objects and the like that are recognized by the autonomous driving ECU <NUM>. The information of the lane includes information regarding the traveling lane that is the lane in which the own vehicle <NUM> is currently traveling (information such as whether the lane is straight or curves or the like), information relating to the adjacent lanes that are adjacent at the left and right of the own lane (the absence/presence of adjacent lanes, and the like), information of a branching-off lane that branches-off from the traveling lane, information of a merging destination lane that the traveling lane merges into, and the like. In a case in which the own vehicle <NUM> attempts to merge from the traveling lane into a merging destination lane during autonomous driving, this information that the information acquisition section <NUM> acquires includes information of a first point (refer to P1 shown in <FIG>) that is the point at which the attempt is to be aborted. The information acquisition section <NUM> acquires information of the first point before the own vehicle <NUM> arrives at the first point.

On the basis of the information acquired by the information acquisition section <NUM>, the display control section <NUM> controls the display of the HUD <NUM> and the meter display <NUM>. As an example, the display control section <NUM> can make the HUD <NUM> and the meter display <NUM> display usual images that are shown in <FIG> and <FIG>, images for merging that are shown in <FIG> and <FIG>, and driving hand-over request images that are shown in <FIG> and <FIG>. Note that, in the examples that are shown in <FIG>, the current speed of the own vehicle <NUM> and the set speed of the cruise control are displayed in the lower portion of the display region 26A of the HUD <NUM> and in the upper portion of the meter display <NUM>. Further, in the example shown in <FIG>, planned operations M1 through M3 of the autonomous driving are displayed in the right side portion of the meter display <NUM>.

In <FIG>, a first-person one-lane display image G1, which is an example of a usual image, is displayed on the HUD <NUM>. This first-person one-lane display image G1 is an image that models the front view that the occupant of the own vehicle <NUM> sees through the windshield glass <NUM>. The first-person one-lane display image G1 is displayed in cases in which, for example, the autonomous driving ECU <NUM> causes the own vehicle <NUM> to carry out constant speed traveling or traveling while following. An image RV1 of the traveling lane in which the own vehicle <NUM> is currently traveling, images LV of the borderlines of the traveling lane, an image V2 of a peripheral object that exists in front of the own vehicle <NUM> (another vehicle in the example shown in <FIG>) are included as objects of display in the first-person one-lane display image G1. Images of adjacent lanes that are adjacent at the left and the right of the traveling lane are excluded from objects of display in this first-person one-lane display image G1. The first-person one-lane display image G1 is an image that minimizes difficulty by, on the one hand, displaying the image RV1 of the traveling lane of the own vehicle <NUM> as large as possible, and, on the other hand, omitting display of information that has little importance to the vehicle occupant understanding the state of the autonomous driving.

In <FIG>, a third-person multiple lane display image G2 that is an example of a usual image is displayed on the meter display <NUM>. This third-person multiple lane display image G2 is an image in which the own vehicle <NUM> and the periphery thereof are shown in a state of being seen from an obliquely rear upper side (a bird's eye view) of the own vehicle <NUM>. This third-person multiple lane display image G2 is displayed in cases in which, for example, the autonomous driving ECU <NUM> causes the own vehicle <NUM> to carry out constant speed traveling or traveling while following. Images RV2, RV3 of the adjacent lanes that are adjacent at the left and right of the traveling lane are included as objects of display in the third-person multiple lane display image G2. As an example, a maximum of three lanes can be displayed in the third-person multiple lane display image G2. However, in order to make it easy to understand the transitional state that accompanies branching-off or merging, or the like, the number of lanes that are displayed is not limited to a maximum of three lanes, and the number of lanes that are displayed is changed appropriately. The image RV1 of the traveling lane, the images RV2, RV3 of the adjacent lanes, the images LV of the borderlines of these traveling lanes, the image V2 of a peripheral object that exists in the traveling lane in front of the own vehicle <NUM> (another vehicle in the example shown in <FIG>), and an image V3 of a peripheral object that exists in an adjacent lane (another vehicle in the example shown in <FIG>) are included as objects of display in the third-person multiple lane display image G2.

In <FIG>, a third-person merge region display image G3 that is an example of an image for merging is displayed on the HUD <NUM>. This third-person merge region display image G3 is an image in which the own vehicle <NUM> and the periphery thereof are shown in a state of being seen from an obliquely rear upper side (a bird's eye view) of the own vehicle <NUM>. The third-person merge region display image G3 is displayed in cases in which the autonomous driving ECU <NUM> causes the own vehicle <NUM> to travel while merging. As an example, an image RV4 of the traveling lane in which the own vehicle <NUM> is traveling (the merge lane), an image RV5 of the lane that the own vehicle <NUM> will merge into, and an image RV6 of the adjacent lane that is adjacent to the merging destination lane, are included as objects of display in the third-person merge region display image G3. Note that, although not illustrated in <FIG>, images of the other vehicles that exist in the traveling lane and in the merging destination lane also are included as objects of display in the third-person merge region display image G3.

In <FIG>, a third-person merge region display image G4 that is an example of an image for merging is displayed on the meter display <NUM>. This third-person merge region display image G4 is basically similar to the third-person merge region display image G3 that is displayed on the HUD <NUM>, and is displayed in cases in which the autonomous driving ECU <NUM> causes the own vehicle <NUM> to travel while merging.

In <FIG>, a hands-on image G5 that is an example of a driving hand-over request image is displayed on the HUD <NUM>. An image showing a state in which the steering wheel is being grasped by both hands, and an image of words such as "take over driving" and "autonomous driving is ending", are included in the hands-on image G5. This hands-on image G5 is an image that urges the occupant of the own vehicle <NUM> to intervene in the driving operation, and is displayed, for example, in cases in which the autonomous driving ECU <NUM> aborts the attempt to merge in autonomous driving.

In <FIG>, a hands-on image G6 that is an example of a driving hand-over request image is displayed on the meter display <NUM>. The hands-on image G6 is basically similar to the hands-on image G5 that is displayed on the HUD <NUM>, and is displayed, for example, in cases in which the autonomous driving ECU <NUM> aborts the attempt to merge in autonomous driving.

Here, in the onboard display device <NUM> of the present exemplary embodiment, at a merge region of a road shown in <FIG> for example, in a case in which the own vehicle <NUM> attempts to merge from traveling lane R4 into merging destination lane R5 during autonomous driving, information of the first point P1 at which the attempt will be aborted is acquired by the information acquisition section <NUM>. Note that, in <FIG>, L is the border between the traveling lane R4 and the traveling lane R5.

The display control section <NUM> displays the information of the first point P1 that was acquired by the information acquisition section <NUM>, on the HUD <NUM> and the meter display <NUM> before the own vehicle <NUM> arrives at the first point P1. Specifically, as shown in <FIG> and <FIG>, the display control section <NUM> displays the third-person merge region display images G3, G4 on the HUD <NUM> and the meter display <NUM>. The image RV4 of the traveling lane R4, the image RV5 of the merging destination lane R5, the image RV6 of adjacent lane R6 that is adjacent to the merging destination lane R5, and the images LV of the borderlines L of the respective lanes R4, R5, R6 are displayed in these third-person merge region display images G3, G4. Moreover, the display control section <NUM> combines image SV, which is of a fixed segment S (see <FIG>) that has the first point P1 as a starting point and that is established in the traveling lane R4, with the third-person merge region display images G3, G4, and displays the combined image.

In the example shown in <FIG> and <FIG>, the aforementioned fixed segment S is displayed, as the image SV that is belt-shaped, on the HUD <NUM> and the meter display <NUM>. In the present embodiment, this fixed segment S is structured by driving hand-over request segment S1 and braking segment S2 that are shown in <FIG>. The driving hand-over request segment S1 is a segment in which at least either of steering and acceleration of the own vehicle <NUM> is controlled by the autonomous driving ECU <NUM>, and the occupant of the own vehicle <NUM> is urged to intervene in the driving operation. This driving hand-over request segment S1 extends from its starting point that is the first point P1 toward a final point P3 side of the image RV4 of the traveling lane R4. The final point of the driving hand-over request segment S1 is second point P2 that is the starting point of the braking segment S2. The braking segment S2 is a segment in which, in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation, the autonomous driving ECU <NUM> causes the own vehicle <NUM> to brake in order to automatically stop the own vehicle <NUM>. This braking segment S2 extends from its starting point that is the second point P2 to the final point P3 of the image RV4 of the traveling lane R4.

The position information of the final point P3 of the traveling lane RV4 is stored in the map information storing section 40A. Central point sequence CPS (see <FIG>) of the traveling lane RV4 is stored in the map information storing section 40A. The above-described first point P1, second point P2 and final point P3 are set on this central point sequence CPS. The autonomous driving ECU <NUM> sets the driving hand-over request segment S1 and the braking segment S2, which are determined by the first point P1, the second point P2 and the final point P3, on the basis of the position information of the final point P3. Specifically, for example, the driving hand-over request segment S1 and the braking segment S2 are established as segments that the subject vehicle <NUM> will pass through in a time period of several seconds (e.g., a four-second time period) at the current vehicle speed, respectively.

The display control section <NUM> displays the driving hand-over request segment S1 and the braking segment S2, which are set as described above, as images SV1, SV2 of different forms (here, different colors) on the HUD <NUM> and the meter display <NUM>. In the present embodiment, the driving hand-over request segment S1 is displayed as the image SV1 that is yellow and belt-shaped, and the braking segment S2 is displayed as the image SV2 that is orange and belt-shaped, on the HUD <NUM> and the meter display <NUM> respectively. Note that, in <FIG> and <FIG>, dots are applied instead of colors to the images SV1, SV2.

Namely, in the present embodiment, the image SV of the fixed segment S whose starting point is the first point P1 is displayed on the HUD <NUM> and the meter display <NUM> in forms (colors) that differ in two stages that are the image SV1 of the driving hand-over request segment S1 and the image SV2 of the braking segment S2. The respective segments S1, S2 (the respective images SV1, SV2) correspond to the deferment time until the own vehicle <NUM> is automatically stopped (emergency-stopped) in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation.

Further, the respective segments S1, S2 correspond to the change in the control of the autonomous driving that is carried out in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation. Concretely, in the driving hand-over request segment S1, the autonomous driving ECU <NUM> requests the occupant of the own vehicle <NUM> to intervene in the driving operation, by using the onboard display device <NUM>. In this case, when the display control section <NUM> senses, on the basis of information from the autonomous driving ECU <NUM>, that the own vehicle <NUM> has arrived at the first point P1, the display control section <NUM> displays the hands-on images G5, G6 that are shown in <FIG> and <FIG> on the HUD <NUM> and the meter display <NUM>. On the other hand, in the braking segment S2, the autonomous driving ECU <NUM> carries out braking in order to automatically stop the own vehicle <NUM>. In the driving hand-over request segment S1 and the braking segment S2, when the occupant of the own vehicle <NUM> operates at least any of the steering wheel, the accelerator pedal and the brake pedal of the own vehicle <NUM>, the autonomous driving ECU <NUM> switches from the autonomous driving mode to the manual driving mode.

An example of the flow of display control processing that is executed by the display control ECU <NUM> is shown in a flowchart in <FIG>. This display control processing is executed due to the CPU <NUM> reading-out the display control program <NUM> from the storage <NUM>, and expanding and executing the program in the memory <NUM>. This display control processing is executed during autonomous driving of the own vehicle <NUM> by the autonomous driving ECU <NUM>.

In this display control processing, first, in step S1, by the function of the information acquisition section <NUM>, the CPU <NUM> judges whether or not the autonomous driving ECU <NUM> is starting to cause the own vehicle <NUM> to travel while merging. If this judgment is negative, the routine moves on to step S2, and, if this judgement is affirmative, the routine moves on to step S3.

In the case of moving on to step S2, by the function of the display control section <NUM>, the CPU <NUM> displays the first-person one lane display image G1 and the third-person multiple lane display image G2, which are usual images respectively, on the HUD <NUM> and the meter display <NUM>. When the processing of this step S2 is completed, the routine returns to step S1.

In the case of moving on to step S3, by the function of the information acquisition section <NUM>, the CPU <NUM> acquires information of the periphery of the own vehicle <NUM>. Information of the first point, at which the attempt to merge in autonomous driving is aborted, also is included in this information. This information is acquired by the information acquisition section <NUM> before the own vehicle <NUM> arrives at the first point P1. This step S3 corresponds to the "acquiring step" in the present invention. When the processing of step S3 is completed, the routine moves on to next step S4.

In step S4, by the function of the display control section <NUM>, the CPU <NUM> displays the images G3, G4 for merging on the HUD <NUM> and the meter display <NUM> respectively, and combines the image SV of the fixed segment S, whose starting point is the first point P1, with the images G3, G4 for merging, and displays the combined image. This display is carried out before the own vehicle <NUM> arrives at the first point P1. This step S4 corresponds to the "displaying step" of the present invention. When the processing of step S4 is completed, the routine moves on to step S5.

In step S5, on the basis of the information acquired by the function of the information acquisition section <NUM>, the CPU <NUM> judges whether or not the own vehicle <NUM> has arrived at the first point P1. If this judgment is negative, the routine returns to above-described step S4, and, if this judgment is affirmative, the routine moves on to next step S6.

In the case of moving on to step S6, by the function of the display control section <NUM>, the CPU <NUM> displays the hands-on images G5, G6 on the HUD <NUM> and the meter display <NUM> respectively. When the processing of step S6 is completed, the routine moves on to next step S7.

In the case of moving on to step S7, the CPU <NUM> judges whether or not the own vehicle <NUM> has completed merging from the traveling lane into the merging destination lane. If this judgment is negative, the routine returns to step S6, and, if this judgment is affirmative, the routine returns to step S1.

Operation and effects of the present embodiment are described next.

As described above, in the present embodiment, in a case in which the own vehicle <NUM> attempts merging from the traveling lane R4 into the merging destination lane R5 during autonomous driving, the information acquisition section <NUM> acquires information of the first point, which is where this attempt will be aborted, before the own vehicle <NUM> arrives at the first point P1. The display control section <NUM> displays the information of the first point P1, which was acquired by the information acquisition section, on the HUD <NUM> and the meter display <NUM> before the own vehicle <NUM> arrives at the first point P1. Due thereto, the occupant of the own vehicle <NUM> can visually understand up to when the attempt to merge by autonomous driving will be continued. As a result, the vehicle occupant can easily understand in advance the necessity of intervening in the driving operation.

Further, in the present embodiment, an image, in which the image RV4 of the traveling lane R4 in which the own vehicle <NUM> is traveling and image PV1 of the first point P1 are combined, is displayed on the HUD <NUM> and the meter display <NUM>. Due thereto, the occupant of the own vehicle <NUM> can more easily visually understand up to when the attempt to merge by autonomous driving will continue.

In the present embodiment, the image SV of the fixed segment S, which is set on the traveling lane RV4 and whose starting point is the first point P1, and the image RV4 of the traveling lane R4 are combined and displayed on the HUD <NUM> and the meter display <NUM>. Due thereto, the occupant of the own vehicle <NUM> can visually confirm the first point P1 easily as compared with a case in which, for example, only the image PV1 of the first point P1 is displayed on the HUD <NUM> and the meter display <NUM>.

In the present embodiment, the fixed segment S that is displayed on the HUD <NUM> and the meter display <NUM> includes the driving hand-over request segment S1 in which at least either of steering and acceleration of the own vehicle <NUM> is controlled by the autonomous driving ECU <NUM>, and in which the occupant of the own vehicle <NUM> is urged to intervene in the driving operation. Because this driving hand-over request segment S1 is displayed in advance on the HUD <NUM> and the meter display <NUM>, for example, it is easy for the occupant of the own vehicle <NUM> to make, in advance, preparations to intervene in the driving operation.

Further, in the present embodiment, the fixed segment S that is displayed on the HUD <NUM> and the meter display <NUM> includes the braking segment S2 in which the own vehicle <NUM> is automatically stopped in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation. Due thereto, in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation, the occupant of the vehicle can know in advance of the point at which braking of the own vehicle <NUM> will start, the point at which automatic stoppage will occur, and the like.

In the present embodiment, the fixed segment S is displayed on the HUD <NUM> and the meter display <NUM> as the belt-shaped image SV whose starting point is the first point P1. Due thereto, it is easy for the occupant of the own vehicle <NUM> to visually confirm the first point P1 as compared with a case in which, for example, the first point P1 is displayed as a dot-shaped or a linear image.

Further, in the present embodiment, the image SV of the fixed segment S is displayed in forms (colors) that differ in two stages that are the driving hand-over request segment S1 and the braking segment S2, in accordance with the deferment time until the own vehicle <NUM> is automatically stopped in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation. Due thereto, it is easy for the occupant of the own vehicle <NUM> to recognize the deferment time until automatic stoppage.

In the present embodiment, the image SV of the fixed segment S is displayed in forms (colors) that differ in two stages that are the driving hand-over request segment S1 and the braking segment S2, in correspondence with the change in the control of the autonomous driving that is carried out in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation. Due thereto, in a case in which the occupant of the own vehicle <NUM> does not intervene in the driving operation, it is easy for the occupant of the own vehicle <NUM> to understand in advance that control of the autonomous driving will change.

In the present exemplary embodiment, in a case in which the own vehicle <NUM> arrives at the first point P1 at which an attempt to merge in autonomous driving is aborted, the hands-on images G5, G6 for urging the occupant of the own vehicle <NUM> to intervene in the driving operation are displayed on the HUD <NUM> and the meter display <NUM>. Due thereto, it is even easier for the occupant of the own vehicle <NUM> to recognize the necessity of intervening in the driving operation.

In the present embodiment, in a case in which the own vehicle <NUM> attempts merging in autonomous driving, information of the first point P1 (here, the image PV1) is displayed so as to be superposed on the front view that the occupant of the own vehicle <NUM> sees through the windshield glass <NUM>. Due thereto, the occupant of the own vehicle <NUM> can confirm the first point P1 while looking at the front view through the windshield glass <NUM>.

Further, in the present embodiment, the image of the first point P1 is displayed also on the meter display <NUM> that is provided at the instrument panel <NUM>. Due thereto, the occupant of the own vehicle <NUM> can confirm the first point P1 in the meter display <NUM>, even in a situation in which it is difficult to see the image displayed on the HUD <NUM> due to the effects of sunlight or the like for example.

Further, in the present embodiment, at the time when the own vehicle <NUM> is traveling while merging, the third-person images G3, G4 for merging are displayed on the HUD <NUM> and the meter display <NUM>. In these third-person images G3, G4 for merging, the own vehicle <NUM> and the periphery thereof are displayed as images that model a state in which the own vehicle <NUM> and the periphery thereof are seen from an obliquely rear upper side of the own vehicle <NUM>, and the image PV1 of the first point P1 at which an attempt to merge in autonomous driving is aborted is displayed. Due thereto, the occupant of the own vehicle <NUM> can, while looking at an image of the own vehicle <NUM> from a bird's eye view, confirm up to when the attempt to merge in autonomous driving will continue.

Other embodiments of the present invention are described next. Note that structures and operations that are basically similar to those of the first embodiment are denoted by the same reference numerals as in the first embodiment, and description thereof is omitted.

An image for merging of the HUD <NUM> in the second embodiment is shown in <FIG>. An image for merging of the meter display <NUM> in the second embodiment is shown in <FIG>. In this embodiment, in a case in which the own vehicle <NUM> attempts to merge from the traveling lane R4 into the merging destination lane R5 during autonomous driving, the information of the first point P1 at which the attempt will be aborted is displayed in the HUD <NUM> and in the meter display <NUM> by bar graphs B. Right ends B3 of the respective bar graphs B show the final point P3 of the traveling lane R4. Position B1 that indicates the first point P1, and position B2 that indicates the second point P2, are set in the intermediate portions of the respective bar graphs B. In the bar graphs B, as the own vehicle <NUM> advances in the traveling lane R4 toward the final point P3 side, a bar-like display extends from left end B0 toward the right end B3. For example, this bar-like display is displayed in white in the segment from B0 to B1, and is displayed in yellow in the segment from B1 to B2, and is displayed in orange in the segment from B2 to B3. In the segment from B0 to B1 that is displayed in white, the attempt to merge in autonomous driving is continued.

In this embodiment, structures other than those described above are similar to those of the first embodiment. In this embodiment as well, the occupant of the own vehicle <NUM> can, by the bar graphs B, visually comprehend up to when the attempt to merge in autonomous driving will continue. Therefore, the vehicle occupant can easily understand in advance the necessity of intervening in the driving operation.

An image for merging of the HUD <NUM> in the third embodiment is shown in <FIG>. An image for merging of the meter display <NUM> in the third embodiment is shown in <FIG>. In this embodiment, in a case in which the own vehicle <NUM> attempts to merge from the traveling lane R4 into the merging destination lane R5 during autonomous driving, the first point P1 at which the attempt will be aborted, and the second point P2 that is the starting point of the braking segment S2, are displayed on the HUD <NUM> and the meter display <NUM> as linear images PV1, PV2.

In this embodiment, structures other than those described above are similar to those of the first embodiment. In this embodiment as well, the occupant of the own vehicle <NUM> can, by the linear image PV1, visually comprehend up to when the attempt to merge in autonomous driving will continue. Therefore, the vehicle occupant can easily understand in advance the necessity of intervening in the driving operation.

Although the present disclosure has been described above by illustration of several embodiments, the present disclosure can, of course, be implemented in various forms within the scope defined by the claims. For example, in the above-described embodiments, the meter display <NUM> is provided at the vehicle front side of the driver's seat at the instrument panel <NUM>, but the present disclosure is not limited to this. The meter display <NUM> may be a center display that is provided at the vehicle transverse direction central portion of the instrument panel <NUM>.

Further, although the above-described embodiments are structured such that the HUD <NUM>, which projects images onto the windshield glass <NUM> that is the windshield, is included in the display portion, the present disclosure is not limited to this. For example, the display portion may be a portion that can project images onto a projection surface that is other than the windshield.

Note that any of various types of processors other than the CPU <NUM> may execute the display control processing that is executed by the CPU <NUM> reading-in software (a program) in the above-described embodiments. Examples of processors in this case include PLDs (Programmable Logic Devices) whose circuit structure can be changed after production such as FPGAs (Field-Programmable Gate Arrays) and the like, and dedicated electrical circuits that are processors having circuit structures that are designed for the sole purpose of executing specific processings such as ASICs (Application Specific Integrated Circuits) and the like, and the like. Further, the display processing may be executed by one of these various types of processors, or may be executed by a combination of two or more of the same type or different types of processors (e.g., plural FPGAs, or a combination of a CPU and an FPGA, or the like). Further, the hardware structures of these various types of processors are, more concretely, electrical circuits that combine circuit elements such as semiconductor elements and the like.

Claim 1:
An onboard display control device (<NUM>) that is to be installed in a vehicle (<NUM>) configured to enable autonomous driving, the device (<NUM>) comprising:
an information acquisition section (<NUM>) that is configured to, in a case in which the vehicle (<NUM>) attempts to merge from a traveling lane (R4) into a merging destination lane (R5) during autonomous driving, acquire information regarding a point (P1) at which the attempt will be aborted, before the vehicle (<NUM>) arrives at the point (P1); and
a display control section (<NUM>) that is configured to display at a display portion (<NUM>, <NUM>) that is to be provided within a vehicle cabin of the vehicle (<NUM>), characterized in that the display control section (<NUM>) is configured to display the point (P1) at which the attempt will be aborted, before arrival at the point (P1).