VEHICLE CONTROL DEVICE, STORAGE MEDIUM STORING COMPUTER PROGRAM FOR CONTROLLING VEHICLE AND METHOD FOR CONTROLLING VEHICLE

A vehicle control device has a processor configured to estimate a direction of a source of a warning sound generated by another vehicle relative to the host vehicle based on an acoustic signal acquired by an acoustical sensor, and decide to drive the host vehicle in a direction perpendicular to a current traveling direction of the host vehicle and away from the source of the warning sound, when the direction of the source of the warning sound relative to the host vehicle is estimated.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to Japanese Patent Application No. 2022-168349 filed on Oct. 20, 2022, the entire contents of which are herein incorporated by reference.

FIELD

The present disclosure relates to a vehicle control device, a storage medium storing a computer program for controlling a vehicle, and a method for controlling a vehicle.

BACKGROUND

An automatic control system mounted on a vehicle has, for example, an automatic driving mode in which the automatic control system is the primarily controller of the operation of the vehicle, and a manual driving mode in which the driver is the primarily controller of the operation of the vehicle (see Japanese Unexamined Patent Publication JP 2019-167116 JP, for example).

SUMMARY

In the automatic driving mode, the automatic control system generates a driving plan of the vehicle while detecting the surrounding environment of the vehicle using sensors such as an image sensor or a LiDAR sensor, etc. mounted on the vehicle.

When the position of another vehicle relative to a host vehicle is in a position where it is difficult to detect another vehicle for the sensor (blind spot), the sensor may not be able to detect another vehicle accurately. For example, the right rear and left rear of the host vehicle may be a position that is difficult to detect another vehicle for the sensor.

Thus, if the sensor used to generate the driving plan fails to accurately detect the surrounding environment of the host vehicle, a safe driving plan of the host vehicle can not be generated. Therefore, another vehicle and the host vehicle may close to each.

Therefore, even when a sensor used to generate the driving plan such as an imaging sensor or a LiDAR sensor cannot accurately detect the surrounding environment of the host vehicle, it is required to safely drive the host vehicle.

For example, there is an acoustical sensor to detect ambient sounds. The sound has a characteristic (diffraction) that travels around even though there is an obstacle. Therefore, the acoustical sensor can detect the sound emitted by another vehicle no matter what the position another vehicle. The acoustical sensor is usually not used to generate a driving plan for the vehicle.

Accordingly, it is an object of the present disclosure to provide a vehicle control device which is capable of driving the host vehicle so as to avoid the host vehicle approaching another vehicle when a warning sound of another vehicle is detected.(1) According to one embodiment, the present disclosure provides a vehicle control device. The vehicle control device has a processor configured to estimate a direction of a source of a warning sound generated by another vehicle relative to the host vehicle based on an acoustic signal acquired by an acoustical sensor, and decide to drive the host vehicle in a direction perpendicular to a current traveling direction of the host vehicle and away from the source of the warning sound, when the direction of the source of the warning sound relative to the host vehicle is estimated.(2) In the vehicle control device of embodiment (1), the processor is further configured to determine whether the reliability of detecting an environment around the host vehicle by another sensor other than the acoustical sensor is at or below a predetermined reference reliability, and decide to drive the host vehicle in the direction perpendicular to the current traveling direction of the host vehicle and away from the source of the warning sound, when the reliability is at or below the predetermined reference reliability and the direction of the source of the warning sound relative to the host vehicle is estimated.(3) In the vehicle control device of embodiment (1) or (2), the processor is further configured to cause the host vehicle to travel back to the travelling lane when the host vehicle is starting to move from the travelling lane to the adjacent lane and the direction of the source of the warning sound relative to the host vehicle coincides with the direction of the moving direction of the host vehicle.(4) In the vehicle control device of any of embodiments (1) to (3), the processor is further configured to decide to drive the host vehicle in the central of the traveling lane in which the host vehicle is traveling, when the direction of the source of the warning sound relative to the host vehicle is not estimated, and decide to drive the host vehicle off the central of the traveling lane in which the host vehicle is traveling on the opposite side from the direction of the source of the warning sound relative to the host vehicle.(5) According to another embodiment there is provided a computer-readable, non-transitory storage medium which stores a computer program for controlling vehicle. The computer program for controlling vehicle includes estimating a direction of a source of a warning sound generated by another vehicle relative to the host vehicle based on an acoustic signal acquired by an acoustical sensor; and deciding to drive the host vehicle in a direction perpendicular to a current traveling direction of the host vehicle and away from the source of the warning sound, when the direction of the warning sound relative to the host vehicle is estimated.(6) According to yet another embodiment there is provided a method for controlling vehicle. The method for controlling vehicle includes estimating a direction of a source of a warning sound generated by another vehicle relative to the host vehicle based on an acoustic signal acquired by an acoustical sensor; and deciding to drive the host vehicle in a direction perpendicular to a current traveling direction of the host vehicle and away from the source of the warning sound, when the direction of the warning sound relative to the host vehicle is estimated.

The vehicle control device of this disclosure can move the host vehicle so as to avoid the host vehicle approaching another vehicle when a warning sound of another vehicle is detected.

The object and advantages of the present disclosure will be realized and attained by the elements and combinations particularly indicated in the claims. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory, and are not restrictive of the present disclosure as claimed.

DESCRIPTION OF EMBODIMENTS

FIG.1AandFIG.1Bare diagrams illustrating in overview the operation of a drive planning device15of the present embodiment.FIG.1Ashows a situation in which another vehicle emitting a warning sound.FIG.1Bshows a situation in which the host vehicle avoids approaching another vehicle.

The operation of the drive planning device15disclosed herein will now be described in overview with reference toFIG.1AandFIG.1B. The drive planning device15is an example of the vehicle control device.

As shown inFIG.1A, the vehicle10is traveling on a traffic lane51of a road50having traffic lanes51,52. The traffic lane51is divided by a lane marking line53and a lane marking line54. The traffic lane52is divided by the lane marking line54and a lane marking line55. The traffic lane51and traffic lane52are divided by the lane marking line (lane boundary line)54.

The vehicle10has the drive planning device15. The drive planning device15uses sensors, such as cameras, mounted on the vehicle10to generate a driving plan of the vehicle10while detecting the environment (road features such as the lane marking lines and other vehicles, etc.) surrounding the vehicle10. The driving plan is represented as a combination of a target location of the vehicle10and a target vehicle speed at the target location, at each time from the current time until the predetermined time.

As shown inFIG.1A, a vehicle30traveling in the traffic lane52is approaching the vehicle10from the right rear of the vehicle10. However, the vehicle30is not detected by the vehicle10, since the position of the vehicle30relative to the vehicle10is in the blind spot of the sensor mounted on the vehicle10. The drive planning device15generates a driving plan in which the vehicle10moves from the traffic lane51to the adjacent traffic lane52. The vehicle10then starts moving from the traffic lane51to the adjacent traffic lane52.

As shown inFIG.1A, the vehicle30emitted a warning sound to the vehicle10that is about to move in front of the vehicle30. The drive planning device15of the vehicle10estimates a direction of a source of the warning sound generated by the vehicle30relative to the vehicle10based on an acoustic signal acquired from an acoustical sensor3.

For example, the acoustical sensor3inputs the left or right sounds relative to the traveling direction of the vehicle10to generate the left and right acoustic signal, respectively. The drive planning device15estimates the direction of the source of the warning sound whether the left or right relative to the traveling direction of the vehicle10, based on the acoustic signals acquired by the acoustical sensor3, when the warning sound emitted by the vehicle30is detected.

In the embodiment shown inFIG.1A, the drive planning device15estimates that the source of the warning sound of the vehicle30is on the left relative to the traveling direction of the vehicle10. The drive planning device15decides to move the vehicle10away from the source of the warning sound, based on the direction of the source of the warning sound.

As shown inFIG.1B, the vehicle10cancels the lane change and returns to the traffic lane51in which the vehicle10has been traveling until then. The vehicle30travels along the traffic lane52without approaching the vehicle10too closely and passes the vehicle10.

As described above, the drive planning device15can move the vehicle10so as to avoid the vehicle10approaching the vehicle30when the warning sound of the vehicle30is detected. This allows the vehicle10to travel safely, even when a sensor used to generate the driving plan cannot accurately detect the environment around the vehicle10.

FIG.2is a general schematic drawing of the vehicle in which the vehicle control system1having the drive planning device15of the present embodiment is mounted. The vehicle10has a front camera2aand a rear camera2b, the acoustical sensor3, a positioning information receiver4, a navigation device5, a user interface (UI)6, a map information storage device11, a location estimating device12, an object detector13, a travelling lane planning device14, the drive planning device15and a vehicle control device16, and the like. In addition, the vehicle10may have a ranging sensor (not shown) for measuring the distance of objects around the vehicle10, such as a LiDAR sensor. The vehicle control system1has at least the acoustical sensor3and the drive planning device15.

The front camera2aand the rear camera2b, the acoustical sensor3, the positioning information receiver4, the navigation device5, UI6, the map information storage device11, the location estimating device12, the object detector13, the travelling lane planning device14, the drive planning device15and the vehicle control device16, are communicatively connected via an in-vehicle network17conforming to standards such as a controller area network. The front camera2aand the rear camera2bare an exemplary imaging units mounted in the vehicle10. The front camera2ais mounted to the vehicle10directed toward the front of the vehicle10. The front camera2acaptures, for example, a camera image in which the environment of a predetermined region ahead of the vehicle10is shown, at a predetermined cycle. The rear camera2bis mounted to the vehicle10directed toward the back of the vehicle10. The rear camera2bcaptures, for example, a camera image in which the environment of a predetermined region back of the vehicle10is shown, at a predetermined cycle. The camera image can show the road in the predetermined region ahead and back of the vehicle10, and road features such as lane marking lines on the road surface and the other vehicles. Each of the front camera2aand the rear camera2bhas a 2D detector composed of an array of photoelectric conversion elements with visible light sensitivity, such as a CCD or C-MOS, and an imaging optical system that forms an image of the captured region on the 2D detector.

Each time a camera image is captured, each of the front camera2aand the rear camera2boutput the camera image and the camera image captured time at which the camera image was captured, through the in-vehicle network17to the location estimating device12and object detector13. The camera image is also used for processing at the location estimating device12to estimate the location of the vehicle10. At the object detector13, the camera image is used for processing to detect other objects surrounding the vehicle10.

The acoustical sensor3inputs the sound around the vehicle10to generate the acoustic signal. The acoustical sensor3outputs the acoustic signal to the drive planning device15, etc. via the in-vehicle network17. As the acoustical sensor3, for example, a stereo microphone can be used. Sound has a characteristic (diffraction) whereby it travels around any obstacle. Therefore, the acoustical sensor3, compared to the front camera2aand the rear camera2bor a LiDAR sensor, there are fewer blind spots where it is difficult to detect another vehicle. In some embodiments, the acoustical sensor3is at a location that facilitates detection of ambient noise around the vehicle10.

In some embodiments, the acoustical sensor3is arranged in the vehicle10so as to acquire the left and right sounds relative to the traveling direction of the vehicle10. The acoustical sensor3inputs the left and right and right sounds for the traveling direction of the vehicle10to generate the left and right acoustic signals, respectively. In some embodiments, the acoustical sensor3is disposed in the vehicle compartment from the viewpoint of protecting the acoustical sensor3from an external environment.

The positioning information receiver4outputs positioning information that represents the current location of the vehicle10. The positioning information receiver4may be a GNSS receiver, for example. The positioning information receiver4outputs positioning information and the positioning information acquisition time at which the positioning information has been acquired, to the navigation device5and map information storage device11, etc., each time the positioning information is acquired at a predetermined receiving cycle.

Based on the navigation map information, the destination location of the vehicle10input through the UI6, and positioning information representing the current location of the vehicle10input from the positioning information receiver4, the navigation device5generates a navigation route from the current location to the destination location of the vehicle10. The navigation route includes information about positions such as right turn, left turn, confluence, branching, etc. When the destination location has been newly set or the current location of the vehicle10has exited the navigation route, the navigation device5generates a new navigation route for the vehicle10. Every time a navigation route is generated, the navigation device5outputs the navigation route to the location estimating device12and the traveling lane planning device14, etc., via the in-vehicle network17.

The UI6is an example of the notification unit. The UI6, controlled by the navigation device5, the drive planning device15and the vehicle control device16, etc., notifies the driver of the vehicle10traveling information. The traveling information of the vehicle10includes information relating to the current location of the vehicle and the current and future route of the vehicle, such as the navigation route. The UI6has a display device6asuch as a liquid crystal display or touch panel, for display of the traveling information. The UI6may also have an acoustic output device (not shown) to notify the driver of traveling information. The UI6also generates an operation signal in response to operation of the vehicle10by the driver. The operation information may be, for example, a destination location, transit points, vehicle speed or other control information of the vehicle10. The UI6also has a touch panel or operating button, for example, as an input device for inputting operation information from the driver to the vehicle10. The UI6outputs the input operation information to the navigation device5and the vehicle control device16, etc., via the in-vehicle network17.

The map information storage device11stores wide-area map information for a relatively wide area (an area of 10 km2to 30 km2, for example) that includes the current location of the vehicle10. In some embodiments, the map information has high precision map information including three-dimensional information for the road surface, information on the types and locations of structures and road features such as road lane marking lines, and the legal speed limit for the road.

The map information storage device11receives the wide-area map information from an external server via a base station, by wireless communication through a wireless communication device (not shown) mounted in the vehicle10, in relation to the current location of the vehicle10, and stores it in the storage device. Each time positioning information is input from the positioning information receiver4, the map information storage device11refers to the stored wide-area map information and outputs map information for a relatively narrow area including the current location represented by the positioning information (for example, an area of 100 m2to 10 km2), through the in-vehicle network17to the location estimating device12, object detector13, traveling lane planning device14, drive planning device15and vehicle control device16, etc.

The location estimating device12estimates the location of the vehicle10at the camera image captured time, based on the road features surrounding the vehicle10represented in the camera image taken by the front camera2a. For example, the location estimating device12compares lane marking lines identified in the camera image with lane marking lines represented in the map information input from the map information storage device11, and determines the estimated location and estimated declination of the vehicle10at the camera image capture time.

The location estimating device12has a classifier trained to input camera images and identify lane marking lines. The location estimating device12inputs the camera image to the classifier to identify the area of the lane marking line represented in the camera image and determine the reliability (confidence) of the lane marking line identified by the classifier (e.g., real number between 0 and 1).

The location estimating device12estimates the road traveling lane where the vehicle10is located, based on the lane marking lines represented in the map information and on the estimated location and estimated declination of the vehicle10. Each time the estimated location, estimated declination and traveling lane of the vehicle10are determined at the camera image captured time, the location estimating device12outputs the estimated information to the object detector13, traveling lane planning device14, drive planning device15and vehicle control device16, etc. The estimated information may also include areas of the lane marking line represented in the camera image and the reliability of the lane marking line identified by the classifier.

Further, the location estimating device12calculates information representing the position of the nearest lane marking line relative to the vehicle10. When two lane marking lines on the left and right sides are detected relative to the traveling direction of the vehicle10, the location estimating device12outputs information representing the respective position of the two lane marking lines relative to the vehicle10to the drive planning device15. Further, when only one of the left and right lanes marking lines is detected relative to the traveling direction of the vehicle10, the location estimating device12outputs information representing the position of the detected one lane marking line to the drive planning device15. Further, when the lane marking line is not detected within a predetermined area from the vehicle10, the location estimating device12outputs information representing that the position of the lane marking line is not detected to the drive planning device15.

The object detector13detects other objects around the vehicle10as well as their types, based on the camera image captured by the front camera2aand the rear camera2b.

The object detector13has a classifier trained to input camera images and detect other objects. The location estimating device12inputs the camera image to the classifier to identify the area of the other object represented in the camera image and reliability of the other object identified by the classifier (e.g., real number between 0 and 1).

The other objects also include other vehicles traveling around the vehicle10. The object detector13also tracks the other object to be detected and calculates the trajectory of the other object being tracked. In addition, the object detector13identifies the traveling lane in which the other object is traveling, based on the lane marking lines represented in the map information and the location of the object. The object detector13outputs object detection information which includes information representing the type of other objects that were detected, information indicating their locations, and also information indicating their traveling lanes, to the traveling lane planning device14, drive planning device15and vehicle control device16, etc. via the in-vehicle network17. The object detection information may include the reliability of the other object identified by the classifier.

The object detector13may not be able to accurately detect another vehicle based on the camera images captured by the front camera2aand the rear camera2bwhen the position of another vehicle relative to the vehicle10is one where it is difficult to detect another vehicle. For example, the right rear and left rear of the vehicle10may be a position at which it is difficult for the object detector13to detect another vehicle.

At a traveling lane-planning generation time set in a predetermined cycle, the traveling lane planning device14selects a traffic lane on the road on which the vehicle10is traveling, within the nearest driving zone (for example, 10 km) selected from the navigation route, based on the map information, the navigation route and surrounding environment information and the current location of the vehicle10, and generates a traveling lane plan representing the scheduled traveling lane for traveling of the vehicle10. For example, the traveling lane planning device14generates a traveling lane plan for the vehicle10to travel on a traffic lane other than a passing traffic lane. Each time a traveling lane plan is generated, the traveling lane planning device14outputs the traveling lane plan to the drive planning device15.

The drive planning device15carries out planning processing, estimating processing, deciding processing, and assessment processing. For this purpose, the drive planning device15has a communication interface (IF)21, a memory22and a processor23. The communication interface21, memory22and processor23are connected via signal wires24. The communication interface21has an interface circuit to connect the drive planning device15with the in-vehicle network17. The drive planning device15is an exemplary the vehicle control device.

The memory22is an example of a storage unit, and it has a volatile semiconductor memory and a non-volatile semiconductor memory, for example. The memory22stores application computer programs and various data to be used for information processing carried out by the processor23.

All or some of the functions of the drive planning device15are functional modules implemented, for example, by a computer program executed by the processor23. The processor23has a planning unit231, an estimating unit232, a deciding unit233, and an assessment unit234. Alternatively, the functional module of the processor23may be a dedicated arithmetic circuit provided in the processor23. The processor23includes one or more CPUs (Central Processing Unit) and its peripheral circuitries. The processor23may further include other operational circuitry, such as a logic unit, a numerical unit, or a graphic processing unit.

At a driving plan generation time set with a predetermined cycle, the planning unit231carries out driving plan processing in which it generates a driving plan representing the scheduled traveling trajectory of the vehicle10up until a predetermined time (for example, 5 seconds), based on the traveling lane plan, the map information, the current location of the vehicle10, the surrounding environment information and the vehicle status information. The driving plan is represented as a combination of the target location of the vehicle10and the target vehicle speed at the target location, at each time from the current time until the predetermined time. In some embodiments, the cycle in which the driving plan is generated is shorter than the cycle in which the traveling lane plan is generated. The drive planning device15generates a driving plan to maintain a spacing of at least a predetermined distance between the vehicle10and other vehicles.

The planning unit231generates the driving plan based on the camera images captured by the front camera2aand the rear camera2b. On the other hand, the acoustical sensor3is used to detect the warning sound emitted by another vehicle but is not usually used to generate the driving plan of the vehicle10.

The drive planning device15outputs the driving plan to the vehicle control device16for each driving plan generated. Other operations of the drive planning device15will be described later.

The vehicle control device16controls each unit of the vehicle10based on the current location of the vehicle10and the vehicle speed and yaw rate, as well as on the driving plan generated by the drive planning device15. For example, the vehicle control device16determines the steering angle, acceleration and angular acceleration of the vehicle10according to the driving plan and the speed and yaw rate of the vehicle10, and sets the amount of steering, and the accelerator or brake level so as to match that steering angle, accelerator level and angular acceleration. The vehicle control device16also outputs a control signal corresponding to a set steering amount, to an actuator (not shown) that controls the steering wheel for the vehicle10, via the in-vehicle network17. The vehicle control device16also outputs a control signal corresponding to set accelerator level, to a drive unit (engine or motor) via the in-vehicle network17. Alternatively, the vehicle control device16may output a control signal corresponding to a set brake level to the brake (not shown) of the vehicle10, via the in-vehicle network17.

The map information storage device11, location estimating device12, object detector13, travelling lane planning device14, drive planning device15, and vehicle control device16, for example, an electronic control unit (ECU). ForFIG.2, the map information storage device11, location estimating device12, object detector13, traveling lane planning device14, drive planning device15, and vehicle control device16, were explained as separate devices, but all or some of them may be constructed in a single device.

FIG.3is an example of an operation flow chart for vehicle control processing for the drive planning device15according to the present embodiment. The vehicle control processing by the drive planning device15will now be explained with reference toFIG.3. The drive planning device15carries out the vehicle control processing according to the operation flow chart shown inFIG.3, at a vehicle control time having a predetermined cycle. The period at which the vehicle control processing is carried out may be, for example, 1 to 5 seconds.

First, the estimating unit232determines whether a direction of a source of a warning sound generated by another vehicle relative to the vehicle10based on an acoustic signal acquired by the acoustical sensor3is estimated (step S101). The estimating unit232has a classifier trained to input the left and right the acoustic signal acquired by the acoustical sensor3to classify the warning sound generated by another vehicle and to identify the direction of the source of the warning sound relative to the vehicle10. The classifier identifies whether the direction of the source of the warning sound is forward, right, left, or backward direction to the travelling direction of the vehicle10. The estimating unit232inputs the acoustic signal acquired from the acoustical sensor3into the classifier to estimate the direction of the source of the warning sound relative to the vehicle10upon detection of the warning sound generated by another vehicle.

The classifier is, for example, a convolutional neural network (CNN) having a plurality of layers connected in series from an input side to an output side. The acoustic signal including the warning sound of vehicle is input to the CNN as the teacher data and the CNN is trained. The CNN performs as a classifier by identifying the warning sound of the vehicle and the direction of the source of the warning sound. Another machine learning model may be used as the classifier.

Further, the estimating unit232compares the frequency, the amplitude and waveform of the acoustic signal acquired by the acoustical sensor3with the frequency, the amplitude and waveform of the reference warning sound and obtains the degree of similarity between them. The estimating unit232determines that a warning sound is detected when the degree of similarity is at or above a predetermined degree of similarity. When the warning sound is detected, the estimating unit232may compare the amplitudes of the left and right acoustic signals acquired by the acoustical sensor3and estimate the direction of the source of the warning sound relative to the vehicle10as one of the sides indicating a greater amplitude. When the amplitudes of the left and right acoustic signals acquired by the acoustical sensor3are the same, the estimating unit232estimates that the direction of the source of the warning sound relative to the vehicle10as the traveling direction of the vehicle10or a direction opposite to the traveling direction of the vehicle10.

When the direction of the source of the warning sound relative to the vehicle10is estimated (step S101—Yes), the deciding unit233decides to drive the vehicle10in a direction perpendicular to the current traveling direction of the vehicle10and away from the source of the warning sound (step S102). The deciding unit233decides the direction away from the source of the warning sound based on the direction of the source of the warning sound. The deciding unit233notifies the planning unit231of a request to drive the vehicle10in a direction perpendicular to the current traveling direction of the vehicle10and away from the source of the warning sound.

The deciding unit233decides to move the vehicle10to the left relative to the traveling direction when the source of the warning sound is on the right relative to the traveling direction of the vehicle10. The deciding unit233decides to move the vehicle10to the right relative to the traveling direction when the source of the warning sound is on the left relative to the traveling direction of the vehicle10. The deciding unit233decides to decelerate the vehicle10and move the vehicle10to the right or left relative to the traveling direction when the source of the warning sound is forward relative to the traveling direction of the vehicle10. The deciding unit233decides to accelerate the vehicle10and move the vehicle10to the right or left relative to the traveling direction when the source of the warning sound is backward relative to the traveling direction of the vehicle10.

Moving the vehicle10to the right or left relative to the traveling direction is an example of moving in the direction perpendicular to the current traveling direction of the vehicle10.

The planning unit231generates a driving plan that moves the vehicle10in a direction perpendicular to the current traveling direction of the vehicle10and away from the source of the warning sound (step S103), and the series of processing steps is complete. The vehicle control device16drives the vehicle10in a direction perpendicular to the current traveling direction of the vehicle10and away from the source of the warning sound based on the driving plan.

On the other hand, when the direction of the source of the warning sound relative to the vehicle10is not estimated (step S101—No), the planning unit231generates a driving plan based on the environment, etc. around the vehicle10detected using the front camera2aand the rear camera2b, and the series of processing steps is complete.

Further, the vehicle control processing by the drive planning device15will now be explained with reference toFIG.1andFIG.4toFIG.7.

In the embodiment shown inFIG.1A, the vehicle10is traveling in the traffic lane51of the road50having the traffic lanes51,52. The vehicle30traveling in the traffic lane52is approaching vehicle10from the right rear of vehicle10. However, the vehicle30is not detected by the vehicle10, since the position of the vehicle30relative to the vehicle10is in the blind spot of the rear camera2bmounted on the vehicle10. The drive planning device15generates a driving plan in which the vehicle10moves from the traffic lane51to the adjacent traffic lane52. The vehicle10then started moving from the traffic lane51to the adjacent traffic lane52.

As shown inFIG.1A, the vehicle30emitted a warning sound to the vehicle10that is about to move in front of the vehicle30. The drive planning device15of the vehicle10detects the warning sound generated by the vehicle30and estimates the direction of the source relative to the vehicle10, based on the acoustic signal acquired from the acoustical sensor3.

The drive planning device15of the vehicle10detects the warning sound generated by the vehicle30based on the acoustic signal acquired from the acoustical sensor3. The drive planning device15also estimates that the source of the warning sound generated by the vehicle30is on the left relative to the traveling direction of the vehicle10.

The drive planning device15determines that the vehicle10is starting to move from the traffic lane51to the adjacent lane52and the direction of the source of the warning sound relative to the vehicle10coincides with the direction of the moving direction of the vehicle10.

The drive planning device15decides to cancel the vehicle10moving between the traffic lanes and cause the vehicle10to return to the traffic lane51. The drive planning device15cancels the moving of the vehicle10between the traffic lanes and generates a driving plan so as to travel in the traffic lane51. The drive planning device15may cause the vehicle10to move away from the source of the warning sound by moving the vehicle10back to the traffic lane51.

As shown inFIG.1B, the vehicle10cancels the lane change and returns to the traffic lane51in which the vehicle10was traveling until then. The vehicle30travels along the traffic lane52without approaching the vehicle10too much and passes by the vehicle10.

According to the drive planning device15, the vehicle10can drive safely even when the rear camera2bused to generate the driving plan cannot accurately detect the environment of the vehicle10.

In the embodiment shown inFIG.4A, the vehicle10is traveling in the traffic lane51of the road50having the traffic lanes51,52. The drive planning device15has detected the left and right lane making lines53,54which mark the traffic lane51in which the vehicle10travels.

The drive planning device15generates a driving plan such that the vehicle10drives in the widthwise central of the traffic lane51based on information representing the respective positions of the two lane marking lines53,54relative to the vehicle10.FIG.4Ashows a centerline51aindicating the widthwise central of the traffic lane51.

The vehicle30traveling in the traffic lane52is approaching the vehicle10from the right rear of the vehicle10. As shown inFIG.4A, the vehicle30emits a warning sound to the vehicle10.

As a reason why the vehicle30emitted the warning sound, it is considered that the vehicle10was traveling closer to the traffic lane52within the traffic lane51because the locations of the lane marking lines53,54detected by the vehicle10were incorrect.

The drive planning device15detected the warning sound generated by the vehicle30based on the acoustic signal acquired by the acoustical sensor3. The drive planning device15also estimates that the source of the warning sound generated by the vehicle30is on the right relative to the travelling direction of the vehicle10.

The drive planning device15decides to move the vehicle10off the widthwise central of the traffic lane51in which the vehicle10is traveling, opposite the direction of the source of the warning sound.

The drive planning device15generates a driving plan so that the vehicle10moves off the centerline51aof the traffic lane51, opposite the direction of the source of the warning sound.

As shown inFIG.4B, the vehicle10travels along the lane51off the widthwise central of the traffic lane51, opposite the direction of the source of the warning sound and away from the vehicle30which emitted the warning sound. The vehicle30travels along the traffic lane52without approaching the vehicle10too much and passes by the vehicle10.

As a reason why the locations of the lane marking lines53,54relative to the vehicle10were incorrect, it is considered that the lane marking lines53,54were not accurately detected from the camera image captured by the front camera2a. According to the drive planning device15, the vehicle10can drive safely even when the sensor used to generate the driving plan cannot accurately detect the environment of the vehicle10.

In the embodiment shown inFIG.5A, the vehicle10is traveling in the traffic lane51of the road50having the traffic lanes51,52. The vehicle10has detected only the right lane making line54which marks the traffic lane51in which the vehicle10travels.

The drive planning device15decides to drive the vehicle10in the widthwise central of the traffic lane51based on information representing the position of the lane marking line54relative to the vehicle10. The drive planning device15generates a driving plan such that the vehicle10travels a predetermined distance L1away from the lane marking line54. The distance L1may be equal to half the width (e.g., 3.6 m) of a typical lane. InFIG.5A, a line56is shown indicating a location away from the lane marking line54by a predetermined distance L1.

The vehicle30traveling in the traffic lane52is approaching the vehicle10from the right rear of the vehicle10. As shown inFIG.5A, the vehicle30emits a warning sound to the vehicle10.

As a reason why the vehicle30emitted the warning sound, it is considered that the vehicle10was traveling closer to the traffic lane52within the traffic lane51because the location of the lane marking line54detected by the vehicle10is incorrect.

The drive planning device15detected the warning sound generated by the vehicle30based on the acoustic signal acquired by the acoustical sensor3. The drive planning device15also estimates that the source of the warning sound generated by the vehicle30is on the right relative to the travelling direction of the vehicle10.

The drive planning device15decides to cause the vehicle10to drive at a location further shifted from the location which is apart from the lane marking line54by the predetermined distance L1, opposite the direction of the source of the warning sound relative to the vehicle10.

The drive planning device15generates a driving plan such that the vehicle10drives at the location further shifted from the location which is apart from the lane marking line54by the predetermined distance L1, opposite the direction of the source of the warning sound relative to the vehicle10.

As shown inFIG.5B, the vehicle10travels along the lane51at a location further shifted from the location which is apart from the lane marking line54by the predetermined distance L1, opposite the direction of the source of the warning sound relative to the vehicle10, and away from the vehicle30which emitted the warning sound. The vehicle30travels along the traffic lane52without approaching the vehicle10too much and passes by the vehicle10.

As a reason why the location of the lane marking line54relative to the vehicle10was incorrect, it is considered that the lane marking line54was not accurately detected from the camera image captured by the front camera2a. According to the drive planning device15, the vehicle10can drive safely even when the sensor used to generate the driving plan cannot accurately detect the environment of the vehicle10.

In the embodiment shown inFIG.6A, the vehicle10is traveling in the traffic lane51of the road50having the traffic lanes51,52. The vehicle10has detected only the left lane making line53which marks the traffic lane51in which the vehicle10travels.

The drive planning device15decides to drive the vehicle10in the widthwise central of the traffic lane51based on information representing the position of the lane marking line53relative to the vehicle10. The drive planning device15generates a driving plan such that the vehicle10travels a predetermined distance L2away from the lane marking line53. The distance L2may be equal to half the width (e.g., 3.6 m) of a typical lane. InFIG.6A, a line57is shown indicating a location away from the lane marking line53by a predetermined distance L2.

The vehicle30traveling in the traffic lane52is approaching the vehicle10from the right rear of the vehicle10. As shown inFIG.6A, the vehicle30emits a warning sound to the vehicle10.

As a reason why the vehicle30emitted the warning sound, it is considered that the vehicle10was traveling closer to the traffic lane52within the traffic lane51because the location of the lane marking line53detected by the vehicle10is incorrect.

The drive planning device15detected the warning sound generated by the vehicle30based on the acoustic signal acquired by the acoustical sensor3. The drive planning device15also estimates that the source of the warning sound generated by the vehicle30is on the right relative to the travelling direction of the vehicle10.

The drive planning device15decides to cause the vehicle10to drive at a location further shifted from the location which is apart from the lane marking line53by the predetermined distance L2, opposite the direction of the source of the warning sound relative to the vehicle10.

The drive planning device15generates a driving plan such that the vehicle10drives at a location further shifted from the location which is apart from the lane marking line53by the predetermined distance L2, opposite the direction of the source of the warning sound relative to the vehicle10.

As shown inFIG.6B, the vehicle10travels along the lane51at a location further shifted from the location which is apart from the lane marking line53by the predetermined distance L2, opposite the direction of the source of the warning sound relative to the vehicle10, and away from the vehicle30which emitted the warning sound. The vehicle30travels along the traffic lane52without approaching the vehicle10too much and passes by the vehicle10.

As a reason why the location of the lane marking line53relative to the vehicle10was incorrect, it is considered that the lane marking line53was not accurately detected from the camera image captured by the front camera2a. According to the drive planning device15, the vehicle10can drive safely even when the sensor used to generate the driving plan cannot accurately detect the environment of the vehicle10.

In the embodiment shown inFIG.7A, the vehicle10is traveling in the traffic lane51of the road50having the traffic lanes51,52. The vehicle10has not detected any lane making lines which mark the traffic lane51in which the vehicle10travels.

The drive planning device15decides to cause the vehicle10to travel in the widthwise central of the traffic lane51in which the vehicle10is traveling, based on the current location of the vehicle10and the map information. The drive planning device15estimates that the lane in which the vehicle10is traveling is the traffic lane51and generates a driving plan to travel in the widthwise central of the traffic lane51in the map information. InFIG.7A, a centerline58is shown indicating the widthwise central of the traffic lane51.

The vehicle30traveling in the traffic lane52is approaching the vehicle10from the right rear of the vehicle10. As shown inFIG.7A, the vehicle30emits a warning sound to the vehicle10.

As a reason why the vehicle30emitted the warning sound, it is considered that the vehicle10was traveling closer to the traffic lane52within the traffic lane51because the actual location of the traffic lane51is different from the estimated location from the map information.

The drive planning device15detected the warning sound generated by the vehicle30based on the acoustic signal acquired by the acoustical sensor3. The drive planning device15also estimates that the source of the warning sound generated by the vehicle30is on the right relative to the travelling direction of the vehicle10.

The drive planning device15decides to drive the vehicle10at a location further shifted from the widthwise central of the traffic lane51in the map information, opposite the direction of the source of the warning sound relative to the vehicle10.

The drive planning device15generates a driving plan such that the vehicle10drives at a location further shifted from the widthwise central of the traffic lane51in the map information, opposite the direction of the source of the warning sound relative to the vehicle10.

As shown inFIG.7B, the vehicle10travels along the lane51at a location further shifted from the widthwise central of the traffic lane51in the map information, opposite the direction of the source of the warning sound relative to the vehicle10, and away from the vehicle30which emitted the warning sound. The vehicle30travels along the traffic lane52without approaching the vehicle10too much and passes by the vehicle10.

According to the drive planning device15, the vehicle10can drive safely even when the front camera2aused to generate the driving plan cannot accurately detect the environment of the vehicle10.

As described above, the drive planning device can drive the host vehicle so as to avoid the host vehicle approaching another vehicle when the warning sound of another vehicle is detected. This allows the host vehicle to travel safely even when a sensor used to generate the driving plan cannot accurately detect the environment around the host vehicle.

Next, a modified embodiment of the drive planning device15of the present embodiment described above will be explained below with reference toFIG.8.FIG.8is another example of an operation flow chart for vehicle control processing for the drive planning device15according to the present embodiment.

In the vehicle control processing of this modified embodiment, a step S201is added, which is different from the above-described embodiment. The processing of step S202to step S204is similar to the step S101to step S103described above.

In the vehicle control processing of this modified embodiment, first, the assessment unit234determines whether the reliability of detecting an environment around the vehicle10by another sensor other than the acoustical sensor3is at or below a predetermined reference reliability (step S201). The front camera2aand the rear camera2bare exemplary sensors other than the acoustical sensor3.

The assessment unit234determines that the reliability is decreased when a state where the reliability of the lane marking line represented in the camera image is at or below a first reference reliability (for example, 0.3 to 0.6) continues for a first period.

In addition, the assessment unit234determines that the reliability is decreased when a state where the reliability of type of an object is at or below a second reference reliability (for example, 0.3 to 0.6) continues for a second period.

For example, the camera image becomes blurred when raindrops adhere to the light receiving portion of the front camera2aor the rear camera2b(e.g., lenses, etc.). In addition, the presence of the raindrops between the front camera2aor the rear camera2b, and the road features, etc. represented in the camera image becomes blurred. When the camera image becomes blurred, it is difficult to detect the road features such as lane marking line from the camera image. In addition, when the front camera2aor the rear camera2bfails, the assessment unit234may determine that the reliability of detecting the environment around the vehicle10by the front camera2aor the rear camera2bis decreased.

Further, when a LiDAR sensor is mounted as a sensor, and the raindrops adhere to the portion for emitting and receiving the laser, or the emitted laser is scattered by the raindrop, measuring the distance between the vehicle10and other objects may not be performed accurately since the laser is not normally received.

When it is determined that the reliability of the sensor is decreased (step S201—Yes), the estimating unit232determines whether a direction of a source of a warning sound generated by another vehicle relative to the vehicle10based on an acoustic signal acquired from the acoustical sensor3is estimated (step S201).

On the other hand, when it is determined that the reliability of the sensor is not decreased (step S201—No), the planning unit231generates a driving plan based on the environment, etc. around the vehicle10detected using the front camera2aand the rear camera2b, and the series of processing steps is complete. The processing in the other steps is the same as in the above-described embodiment.

Another vehicle may not emit the warning sound only when another vehicle is about to approach the vehicle10. Causing the vehicle10to avoid approaching another vehicle each time another vehicle emits the warning sound result in extra operation.

Therefore, in this modified embodiment, the drive planning device15causes the vehicle10to avoid approaching another vehicle when the reliability of the front camera2aor the rear camera2bin detecting the environment around the vehicle10is decreased and the warning sound of another vehicle is detected.

When the reliability of the front camera2aor the rear camera2bin detecting the environment around the vehicle10is normal, a safe driving plan may be generated based on the information detected by the front camera2aor the rear camera2b.

On the other hand, when the reliability of the front camera2aor the rear camera2bin detecting the environment around the vehicle10is decreased, the drive planning device15can generate a safe driving plan by detecting the warning sound of another vehicle.

The vehicle control device, a computer program for controlling vehicle, and a method for controlling vehicle according to the embodiments described in the present disclosure may incorporate appropriate modifications that still fall within the gist of the disclosure. Moreover, the technical scope of the disclosure is not limited to these embodiments, and includes the present disclosure and its equivalents as laid out in the Claims.