Patent ID: 12258006

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments will be described with reference to the drawings. In the following description, the same or equivalent elements are given the same reference numerals, and overlapping description will not be repeated.

Configurations of Vehicle and Travel Control Device

FIG.1is a functional block diagram of an example of a vehicle including an object detection device according to an embodiment. As illustrated inFIG.1, an object detection device1is mounted on a vehicle2(an example of a moving object) such as a bus, a taxi, or a general passenger car. The vehicle2may travel through a driver's operation, may travel through autonomous driving, or may travel through remote control.

The vehicle2includes an external sensor3, an internal sensor4, a global positioning system (GPS) receiver5, an electronic control unit (ECU)6, a human machine interface (HMI)7, and an actuator8.

The external sensor3is a detector that detects information regarding an external environment of the vehicle2. The external environment is a position of an object in the surroundings of the vehicle2, a situation of the object, or the like. Detection results from the external sensor3include a position, a shape, a color, and the like of an object in front of a roadway on which the vehicle2is traveling. Objects include vehicles, pedestrians, traffic signals, road paint, and the like. The external sensor3is, for example, a camera.

The camera is an imaging device that images an external situation of the vehicle2. The camera is provided behind a front windshield of the vehicle2as an example. The camera acquires imaging information regarding the external situation of the vehicle2. The camera may be a monocular camera or a stereo camera. The stereo camera has two imaging units disposed to reproduce binocular parallax. The imaging information of the stereo camera also includes information in a depth direction.

The external sensor3is not limited to the camera, and may be a radar sensor or the like. The radar sensor is a detector that detects an object in the surroundings of the vehicle2by using electric waves (for example, millimeter waves) or light. The radar sensor includes, for example, millimeter-wave radar or laser imaging detection and ranging (LIDAR). The radar sensor transmits electric waves or light to the surroundings of the vehicle2and detects an object by receiving the electric waves or the light reflected by the object.

The internal sensor4is a detector that detects a traveling state of the vehicle2. The internal sensor4may include a steering angle sensor, a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. The steering angle sensor is a detector that detects an amount of rotation of a steering shaft of the vehicle2. The vehicle speed sensor is a detector that detects a speed of the vehicle2. As the vehicle speed sensor, for example, a wheel speed sensor that is provided at a wheel of the vehicle2or a drive shaft that rotates integrally with the wheel and detects a rotation speed of the wheel is used. The acceleration sensor is a detector that detects an acceleration of the vehicle2. The acceleration sensor may include a front-rear acceleration sensor that detects an acceleration in the front-rear direction of the vehicle2and a lateral acceleration sensor that detects an acceleration of the vehicle2. The yaw rate sensor is a detector that detects a yaw rate (rotational angular velocity) about a vertical axis of the centroid of the vehicle2. As the yaw rate sensor, for example, a gyro sensor may be used.

The GPS receiver5measures a position of the vehicle2(for example, latitude and longitude of the vehicle2) by receiving signals from three or more GPS satellites. The ECU6may acquire position information of the vehicle2by using the detection results from the external sensor3and map information.

The ECU6controls an operation of the vehicle2. The ECU6is an electronic control unit having a central processing unit (CPU), a read only memory (ROM), a random access memory (RAM), a Controller Area Network (CAN) communication circuit, and the like. The ECU6is connected to a network that performs communication by using, for example, the CAN communication circuit, and is communicatively connected to the above constituent elements of the vehicle2. For example, the ECU6operates the CAN communication circuit to input and output data on the basis of a signal output by the CPU, stores the data in the RAM, loads a program stored in the ROM to the RAM, and realizes functions that will be described later by executing the program loaded in the RAM. The ECU6may be configured with a plurality of electronic control units.

The HMI7is an interface between an occupant (including a driver) of the vehicle2or a person present outside the vehicle and a system realized by the ECU6. The HMI7includes an in-vehicle HMI71and an outward notification unit72. The in-vehicle HMI71is an interface for an occupant of the vehicle2, and includes, for example, a touch display capable of displaying information and receiving an operation input of the occupant. The outward notification unit72is an interface for a person present outside the vehicle such as a pedestrian, and is, for example, a display or a road surface projector provided on the exterior of the vehicle2. The HMI7is not limited to an interface for displaying an image or a video, and may be an interface for outputting sound.

The actuator8is a device that executes travel control of the vehicle2. The actuator8includes at least an engine actuator, a brake actuator, and a steering actuator. The engine actuator controls a drive force of the vehicle2by changing an amount of air supplied to an engine (for example, changing a throttle opening degree) according to a driving operation or a control signal from the ECU6. When the vehicle2is a hybrid vehicle or an electric vehicle, the engine actuator controls a drive force of a motor as a power source.

The brake actuator controls the brake system according to a control signal from the ECU6and controls a braking force applied to the wheels of the vehicle2. As the brake system, for example, a hydraulic brake system may be used. When the vehicle2is provided with a regenerative braking system, the brake actuator may control both the hydraulic braking system and the regenerative braking system. The steering actuator controls the drive of an assist motor that controls the steering torque in an electric power steering system according to a control signal from the ECU6. As a result, the steering actuator controls the steering torque of the vehicle2.

Outline of Blind Spot

Prior to description of each function of the ECU6, a blind spot that is a target determined by the ECU6will be described.FIG.2is a tree diagram for describing definition of a blind spot. As illustrated inFIG.2, a blind spot9is divided into a dynamic blind spot91and a static blind spot92. The static blind spot92is divided into an open blind spot93and a closed blind spot94. The dynamic blind spot91is divided into an open blind spot93. A target determined by the ECU6is the closed blind spot94included in the blind spot9.

First, the blind spot9will be described.FIG.3Ais a top view for describing the inside of a visual field and a blind spot, andFIG.3Bis a side view for describing the inside of the visual field and the blind spot. In the example illustrated inFIGS.3A and3B, the vehicle2includes a single external sensor3. As illustrated inFIGS.3A and3B, the external sensor3can observe a visual field region10that is a region in a visual field. The visual field region10is, for example, fan-shaped in a top view. The visual field region10is separated by a boundary line L indicating an observable limit distance. For example, when the external sensor3is LIDAR, there is a distance at which light does not reach even in the visual field (a distance at which reflected light does not sufficiently return). Alternatively, when the external sensor3is a camera, a resolution becomes low at a certain distance or more, and thus an object cannot be detected and identified. A distance farther than such a distance is defined as the blind spot9. The boundary line L in the figure may be an arcuate line centered on the external sensor3instead of a straight line. When there is no three-dimensional object around the vehicle2, the outside of the visual field is the blind spot9, and inside the visual field is not the blind spot9.

Next, a case where a three-dimensional object is present in the visual field will be described.FIG.3Cis a top view for describing a dynamic blind spot and a static blind spot, andFIG.3Dis a side view for describing a dynamic blind spot and a static blind spot. When another vehicle is present in the blind spot9outside the visual field, the other vehicle cannot be observed by the external sensor3. Thus, the blind spot9outside the visual field is the blind spot9regardless of the presence of a three-dimensional object. On the other hand, as illustrated inFIG.3CandFIG.3D, when another vehicle2A is present in a region other than the blind spot9in the visual field, the external sensor3cannot observe a region deeper than the other vehicle2A. Therefore, a new blind spot9occurs. As described above, the blind spot9in the visual field caused by an external factor is defined as the dynamic blind spot91. The external factor is not limited to a dynamic object. The external factor may be, for example, a wall at an intersection. When a positional relationship between the vehicle2and the wall changes, a blind spot region is deformed, but the wall itself does not change. Therefore, even a stationary object may be an external factor that causes the dynamic blind spot91.

On the other hand, the blind spot9that is fixedly present to the external sensor3to be present around the visual field region10is defined as the static blind spot92. The static blind spot92, that is, the blind spot9that is fixedly present to the external sensor3may be generated not only by the periphery of the visual field region10but also by a method in which the external sensor3is attached. For example, the external sensor3is provided with a cover to prevent raindrops or stepping stones, or to shape the appearance design. The cover is present in the visual field of the external sensor3and may generate a fixed blind spot. Alternatively, since a vehicle body is reflected in the visual field of the external sensor3, a region behind the reflected region may be a fixed blind spot.FIG.4Ais a side view for describing details of the static blind spot. As illustrated inFIG.4A, when the vehicle body of the vehicle2is reflected in the visual field of the external sensor3, a region behind the reflected region becomes a static blind spot92A that is a fixed blind spot. In the present embodiment, the blind spot generated by an attachment method of the external sensor3is also handled as the static blind spot92.

The above definition can be extended not only to a single external sensor3but also to a sensor set (a sensor group configured with a plurality of sensors and/or a plurality of types of sensors).FIG.4Bis a top view for describing blind spots of a plurality of external sensors. In the figure, the vehicle2is not illustrated. In the example illustrated inFIG.4B, the vehicle2includes a narrow-angle camera31and a wide-angle camera32. The static blind spot92in the sensor set is a fixed blind spot in the sensor set. In the example illustrated inFIG.4B, a visual field region10of the narrow-angle camera31and a visual field region10A of the wide-angle camera32do not completely overlap. In this case, the static blind spot92is the region where both the narrow-angle camera31and the wide-angle camera32are blind spots. That is, a static blind spot common to both cameras is the static blind spot92in the sensor set. Also, the definition of dynamic blind spot91is extended to a blind spot in the visual field of the sensor set caused by external factors.

Open Blind Spot and Closed Blind Spot

As illustrated inFIG.2, the static blind spot92is classified into the open blind spot93and the closed blind spot94.FIG.5Ais a top view for describing an example of the open blind spot. As illustrated inFIG.5A, when a pedestrian moves from a position H1in the visual field (visual field region10) to a position H3that is a static blind spot, the external sensor3can catch the pedestrian until just before entering the blind spot. Thus, the vehicle2can detect that the pedestrian has entered the static blind spot and the pedestrian is present in the static blind spot. However, when the pedestrian moves from a position H2that is a static blind spot to the position H3, the external sensor3cannot detect the presence of the pedestrian. As described above, “a static blind spot that a dynamic object farther than the observable distance can enter without passing through the inside of the visual field” is classified as an “open blind spot” (reference numeral93in the figure). As illustrated inFIG.2, the dynamic blind spot91is necessarily classified as the open blind spot93. Thus, “a static blind spot that a dynamic object farther than the observable distance can enter without passing through the inside of the visual field” and “a blind spot in the visual field caused by external factors (that is, a dynamic blind spot)” are defined as “open blind spots”.

Next, the case of the sensor set will be described.FIG.5Bis a top view for describing a static blind spot of an ultra-wide-angle sensor. As illustrated inFIG.5B, the ultra-wide-angle sensor has a partially missing circular visual field region10, and a region around the visual field region is an open blind spot93. An example in which these ultra-wide-angle sensors are provided at the front part of and the rear part of the vehicle2is illustrated inFIGS.5C and5D.FIG.5Cis a top view for describing an example of a closed blind spot, andFIG.5Dis a side view for describing an example of a closed blind spot. As illustrated inFIG.5C, the vehicle2is provided with ultra-wide-angle sensors33and34at the front part and the rear part. In this case, the entire circumference of the vehicle2is surrounded by the visual field region10of the sensor set including the ultra-wide-angle sensors33and34. As illustrated inFIG.5D, a road surface (ground) is in the visual field of the ultra-wide-angle camera. In such a case, when a “dynamic object (a pedestrian present at a position H4in the figure) that is farther than the observable distance” moves, the pedestrian is necessarily in the visual field in order to enter a static blind spot in the surroundings of the vehicle2. Such a “static blind spot that a dynamic object farther than the observable distance cannot enter without passing through the inside of the visual field” is defined as a “closed blind spot” (reference numeral94in the figure).

InFIG.5D, the road surface (ground) is in the visual field of the ultra-wide-angle camera, but the fact that the road surface (ground) is in the visual field of the ultra-wide-angle camera is not a mandatory requirement to define that a blue light source is the closed blind spot94. In order to define the closed blind spot94, a predetermined ground height may be in the visual field. The predetermined ground height is appropriately set. For example, when an autonomous driving system is mounted on the vehicle2, a ground height that the autonomous driving system can handle as a safe blind spot may be set as the predetermined ground height. The closed blind spot may be defined not only in a space below the vehicle2but also in a space above the vehicle2. The vehicle2is assumed to have a ceiling at a position of the vehicle height (a height including an upper structure such as a sensor) when there is a pier or a tunnel with a height limit, for example, as in a case where there is no blind spot below the ground, it is assumed that there is no blind spot above the ceiling. With this configuration, it is possible to classify whether the blind spot present in the upper part of the vehicle2is the open blind spot93or the closed blind spot94.

Next, an aspect of a change of the closed blind spot will be described.FIG.6Ais a top view for describing examples of the inside of a visual field and a closed blind spot,FIG.6Bis a top view for describing an example of the occurrence of an open blind spot, andFIG.6Cis a top view for describing an example of a change from a closed blind spot to an open blind spot. In the figures, the vehicle2is not illustrated, and only the sensor set30of the external sensor3is illustrated. As illustrated inFIG.6A, the sensor set30has the visual field region10and the closed blind spot94. It is assumed that a pedestrian H enters the visual field region10from a position outside the visual field region10.

As illustrated inFIG.6B, when the pedestrian H has entered the visual field region10, the open blind spot93that is the dynamic blind spot91occurs in a region behind the pedestrian H when viewed from the sensor set30. Then, as illustrated inFIG.6C, it is assumed that the pedestrian H has further moved and reached the closed blind spot94. When the open blind spot93and the closed blind spot94overlap, that is, when the open blind spot93and the closed blind spot94are connected (refer to a connection point X in the figure), a state occurs in which “a dynamic object farther than the observable distance can enter a region that is the closed blind spot94without passing through the visual field”. When the open blind spot93and the closed blind spot94are connected as described above, the definition of the closed blind spot94breaks down, and the definition of the open blind spot93is established.FIG.7is a top view for describing examples of the inside of a visual field and a closed blind spot. In the example illustrated inFIG.7, the sensor set30includes sensors31to33. Others are the same as inFIG.6A. Even in such a case, the open blind spot93that is the dynamic blind spot91occurs in a region behind the pedestrian H when viewed from the sensor set30. Then, when the open blind spot93and the closed blind spot94are connected, a state occurs in which “a dynamic object farther than the observable distance can enter a region that is the closed blind spot94without passing through the visual field”. For example, there is concern that an object hidden in the open blind spot93that is the dynamic blind spot91may enter the closed blind spot94without being detected by the sensor set30. The pedestrian H is an example of an obstacle. An obstacle (an example of an object) may be a dynamic obstacle such as the pedestrian H or a static obstacle such as a pillar. As described above, the closed blind spot94may change to the open blind spot93due to an obstacle.

Next, another aspect of the change in a closed blind spot will be described.FIG.8Ais a top view for describing an example of a change in a closed blind spot. In the figures, the vehicle2is not illustrated, and only the sensor set30of the external sensor3is illustrated. As illustrated inFIG.8A, the sensor set30has the visual field region10and the closed blind spot94. A wall W is present in the visual field region10and the closed blind spot94of the sensor set30. In this case, a region behind the wall W when viewed from the sensor set30is the open blind spot93. Since the open blind spot93and the closed blind spot94closed are shielded by the wall W, and both ends of the wall W are present in the visual field region10, the state in which “a dynamic object farther than the observable distance cannot enter the closed blind spot94without passing through the visual field” is maintained. That is, the wall W illustrated inFIG.8Ais an obstacle that does not connect the open blind spot93to the closed blind spot94, and changes a part of the closed blind spot94to the open blind spot93A. As described above, an obstacle that does not connect the open blind spot93to the closed blind spot94has an effect of narrowing a region of the closed blind spot94.

FIG.8Bis a top view for describing another example of the change in a closed blind spot. In the figures, the vehicle2is not illustrated, and only the sensor set30of the external sensor3is illustrated. As illustrated inFIG.8B, the sensor set30has the visual field region10and the closed blind spot94. A wall W is present in the visual field region10and the closed blind spot94of the sensor set30. Here, in the example illustrated inFIG.8B, one end of the wall W is present in the closed blind spot94. In such a case, the open blind spot93and the closed blind spot94are connectable (refer to a connection point X in the figure). That is, the closed blind spot94is changed to “a blind spot that a dynamic object farther than the observable distance can enter without passing through the visual field”. Consequently, the closed blind spot94in the figure is changed to the open blind spot93.

Next, an aspect in which an open blind spot is changed to a closed blind spot will be described.FIG.8Cis a top view for describing an example of a change from an open blind spot to a closed blind spot. In the figure, the vehicle2is not illustrated. The external sensor3illustrated inFIG.8Cis, as an example, the ultra-wide-angle sensor illustrated inFIG.5B. As illustrated inFIG.8C, the external sensor3is brought close to the wall W that is an obstacle. Consequently, the open blind spot93is confined by the visual field region10and the wall W, and is changed to the closed blind spot94. In other words, the obstacle defines a region that “a dynamic object farther than the observable distance cannot enter without passing through the visual field”. As described above, the confined open blind spot93may be changed to the closed blind spot94.

Each Function of ECU

The object detection device1illustrated inFIG.1detects the presence or absence of an object in a blind spot region by using the above property of the closed blind spot94. The ECU6of the object detection device1has, as functional constituents, a vehicle information acquisition unit11, an obstacle recognition unit12, a blind spot information DB13, an entry/exit status acquisition unit14(an example of an acquisition unit), a determination unit15, a moving object controller16, a number-of-persons acquisition unit17, and a notification controller18. The object detection device1includes the entry/exit status acquisition unit14and the determination unit15in order to realize an object detection function, and includes the vehicle information acquisition unit11, the obstacle recognition unit12, the blind spot information DB13, the moving object controller16, the number-of-persons acquisition unit17, and the notification controller18in order to realize an auxiliary or additional function of the object detection function.

The vehicle information acquisition unit11acquires information regarding a state of the vehicle2. As an example, the vehicle information acquisition unit11acquires position information of the vehicle2acquired by the GPS receiver5, an orientation of the vehicle2detected by the internal sensor4, and the like. The vehicle information acquisition unit11may acquire speed information of the vehicle2for interpolation (interpolation) of position information. For example, when positions of the vehicle2at respective times t1, t2, and t3(t1<t2<t3) are set to p1, p2, and p3, it is assumed that the positions of vehicle2at times t1and t3are acquired. In this case, the vehicle information acquisition unit11calculates the position of the vehicle2at time t2as p2=p1+(p3−p1)*(t2−t1)/(t3−t1). Alternatively, when speeds v1and v3of the vehicle2at the times t1and t3are acquired, the vehicle information acquisition unit11may calculate the position of the vehicle2at the time t2as p2=p1+(v1+v3)/2*(t2−t1).

The obstacle recognition unit12recognizes an obstacle. The obstacle is an object present around the vehicle2, and includes a stationary object such as a road structure or a plant, or a dynamic object such as another vehicle. The obstacle recognition unit12recognizes a position, a speed, an attribute, and the like of the obstacle on the basis of, for example, a detection result from the external sensor3. The attribute includes a pedestrian, a motorcycle, a vehicle, a stationary object, and the like.

The blind spot information DB13stores information regarding the closed blind spot94. The information regarding the closed blind spot94is three-dimensional information indicating a three-dimensional structure (a position and a three-dimensional shape) of the closed blind spot94. As described above, the closed blind spot94is “the static blind spot92that a dynamic object farther than the observable distance cannot enter without passing through the visual field”, and the static blind spot92is a “blind spot9that is fixedly present in the external sensor3”. Thus, when the external sensor3is fixed to the vehicle2and thus an angle of view thereof is fixed, a position and a three-dimensional shape of the closed blind spot94are fixed.

The information regarding the closed blind spot94may be dynamically updated. For example, when a visual field of the external sensor3changes dynamically, the information regarding the closed blind spot94is updated according to the change in the visual field. An example of the case where the visual field of the external sensor3changes dynamically is a case where the external sensor3is attached to a tip of a robot arm, a case where a gaze direction and a visual field angle can be controlled by using an optical member such as a mirror, and the like. In this case, the blind spot information DB13may store time information and a position and a three-dimensional shape of the closed blind spot94. The blind spot information DB13may record that the open blind spot93has been changed to the closed blind spot94, or that the closed blind spot94has been changed to the open blind spot93.

Alternatively, when the visual field of the external sensor3is expanded on the basis of information regarding another person, the information regarding the closed blind spot94may be updated according to the change in the visual field. For example, by acquiring information from an infrastructure camera fixed in a parking lot or sensors of surrounding other vehicles in a wireless manner or the like, the three-dimensional shape of the closed blind spot94may be reduced, the open blind spot93may be changed to the closed blind spot94, or the information regarding the closed blind spot94may be deleted.

Alternatively, when the blind spot shape is changed due to an environment, the information regarding the closed blind spot94may be updated. For example, due to backlight, some information regarding a visual field angle may not be acquired, resulting in a blind spot. Alternatively, the LIDAR may not be able to obtain information because the light is diffused when thick fog is generated. When a shape of the blind spot is changed due to such environmental factors, a three-dimensional structure thereof may be stored with time.

Alternatively, as illustrated inFIG.6CandFIGS.8A to8C, when the closed blind spot94is changed to the open blind spot93, or the open blind spot93is changed to the closed blind spot94, the information regarding the closed blind spot94may be updated. The three-dimensional structure of the closed blind spot94changed by the obstacle may be stored with time.

The entry/exit status acquisition unit14acquires an entry/exit status of an obstacle into/from the closed blind spot94of the external sensor3on the basis of the detection result from the external sensor3. The entry/exit status of an obstacle into/from the closed blind spot94is entry and exit of the obstacle into/from the closed blind spot94. The entry/exit status acquisition unit14acquires an entry/exit status of an obstacle on the basis of a change in the position of the obstacle (detection result from the external sensor3) acquired by the obstacle recognition unit12and the information regarding the closed blind spot94acquired from the blind spot information DB13. For example, when the obstacle moves toward the closed blind spot94and overlaps with the closed blind spot94, and the obstacle cannot be detected by the external sensor3, the entry/exit status acquisition unit14determines that the obstacle has entered the closed blind spot94. Alternatively, for example, when an obstacle appearing from the closed blind spot94is detected by the external sensor3, the entry/exit status acquisition unit14determines that the obstacle has exited from the closed blind spot94.

The determination unit15determines the presence or absence of an obstacle in the closed blind spot94on the basis of the entry/exit status acquired by the entry/exit status acquisition unit14. As described above, the closed blind spot94always passes through the visual field of the external sensor3when the obstacle approaches from a distance farther than the observable distance.FIG.9Ais a top view for describing examples of the inside of a visual field and a closed blind spot, andFIG.9Bis a top view for describing time-series transition of the inside of a visual field and a closed blind spot. As illustrated inFIG.9A, in the stopped vehicle2before starting, the closed blind spot94is present in the vicinity of the vehicle2, and it is not clear whether a dynamic object (a human, a motorcycle, or the like) is present in the closed blind spot94. In contrast, inFIG.9B, the traveling vehicle2is illustrated. In the figure, blind spots at two different times overlap, and the past blind spot is illustrated by a dashed line. The closed blind spot94includes a region94A that is a closed blind spot over two different times and a region94B that was in the visual field in the past and is now included in the closed blind spot. In the region94A, it is not clear whether or not a dynamic object is present. However, the region94B has already been observed in the past, and that region has only become a blind spot. Thus, the presence or absence of a dynamic object in past observation can be directly applied to the current state. In the example illustrated inFIG.9B, only two different times are handled, but more times may be used. Consequently, the region94A is not present, and all the closed blind spots94are regions where past observation information can be used like the region94B. When it is observed that “there is no dynamic object in the closed blind spot94” by using the time-series detection results by the external sensor3, even if the vehicle2is stopped after that (unless a dynamic object enters the blind spot94via the visual field), there is no dynamic object in the closed blind spot94, and the vehicle2can safely move forward.

Even when the above time-series detection results are used, it is not clear whether a dynamic object (a human, a motorcycle, or the like) is present in the closed blind spot94immediately after the formation of the closed blind spot94, such as when the autonomous driving system starts. When the vehicle2is stopped and information indicating that there is no obstacle in the closed blind spot94cannot be acquired, the moving object controller16moves the vehicle2such that a position corresponding to the closed blind spot in which the information indicating that there is no obstacle cannot be acquired falls within the visual field region10of the external sensor3.FIG.10Ais a diagram for describing an example of a closed blind spot, andFIG.10Bis a diagram for describing an operation for confirming that no obstacle is present in the closed blind spot. For example, as illustrated inFIG.10A, it is assumed that the closed blind spot94is present at a location C in front of the vehicle2, and the presence or absence of a dynamic object is not clear. In this case, as illustrated inFIG.10B, the moving object controller16operates the actuator8to retract the vehicle2such that the location C falls within the visual field. By retreating the vehicle2, the closed blind spot94can be superimposed on the location B where it has already been confirmed that an object is not present. Through such an operation, it is possible to actively create the closed blind spot94in which a dynamic object is not present.

The determination unit15acquires information indicating that there is no obstacle in the closed blind spot94, and if the entry of an obstacle into the closed blind spot94is not detected after the information is acquired, the determination unit15may determine that there is no obstacle in the closed blind spot94. The information indicating that there is no obstacle in the closed blind spot94may be acquired from a detection result detected by another in-vehicle sensor or from an external infrastructure. The determination unit15may acquire in advance the number of obstacles present in the closed blind spot94, instead of the information indicating that no obstacle is present in the closed blind spot94. Even in this case, the determination unit15may determine the presence or absence of an obstacle on the basis of the number of obstacles that are initially present, the number of obstacles that have entered the closed blind spot94, and the number of obstacles that have exited from the closed blind spot94.

As an example of using another in-vehicle sensor, for example, a vehicle cabin camera may be used. It is assumed that a person enters the closed blind spot94when an occupant gets on and off the vehicle2. In such a case, the number-of-persons acquisition unit17acquires the number of persons who have got on the vehicle2from the closed blind spot94and the number of persons who have got off the vehicle2to the closed blind spot94. Specifically, the number of persons getting off the vehicle is counted by the vehicle cabin camera, and a difference from the number of persons leaving the closed blind spot94is collated. Consequently, it is possible to determine whether or not a person remains in the closed blind spot94. Alternatively, if a “change in vehicle weight before and after an occupant gets on and off” and a “change in weight of a person at a bus stop” are the same as each other by using a vehicle weight sensor and an installed gravimeter (for example, a gravimeter installed at the bus stop), it may also be determined that there is no person in the closed blind spot94. Alternatively, when a door of the vehicle2is opened or closed, “a state may occur in which it is not clear whether or not there is a person in the closed blind spot94(an unknown state or an initial state)”. An occupant may visually confirm that there is no obstacle in the closed blind spot94, and then the vehicle2may start.

When a plurality of closed blind spots94are present around the vehicle2, the determination unit15may determine the presence or absence of the above object on each of the closed blind spots94. FIG.10C is a diagram for describing a plurality of closed blind spots. InFIG.10C, the ultra-wide-angle sensor illustrated inFIG.5Band two side sensors are provided in the vehicle2. The ultra-wide-angle sensor can detect an object present in the visual field region10A (boundary line LA), and there is a closed blind spot in the vicinity of the vehicle2. The closed blind spot is divided into two blind spots by a left sensor (boundary line LB) in the visual field region10B and a right sensor (boundary line LC) in the visual field region10C. In the example in the figure, there is a closed blind spot94A behind the vehicle2, and there is a closed blind spot94B in front of the vehicle2. In this case, the determination unit15determines the presence or absence of an object for each of the closed blind spots94A and94B. Consequently, it is possible to determine, for example, that an object is present in the closed blind spot94B and that an object is not present in the closed blind spot94A. Therefore, even if an object is present in a blind spot, it is possible to move the vehicle2such that the vehicle2does not come into contact with the object.

The determination unit15outputs the determination result to the autonomous driving system or the like. When it is determined that there is no dynamic object in the closed blind spot94, the autonomous driving system determines that the vehicle2can advance. When it is determined that a dynamic object is present in the closed blind spot94, the autonomous driving system takes measures such as not starting before the dynamic object goes out of the closed blind spot94.

The notification controller18notifies the occupant of the vehicle2of the information regarding the closed blind spot94of the external sensor3, and notifies the outside of the vehicle2of the information regarding the closed blind spot94of the external sensor3.FIG.11Aillustrates an example of a screen displayed by the object detection device. As illustrated inFIG.11A, the vehicle2includes an in-vehicle HMI71that displays information toward the vehicle cabin. The notification controller18displays the information regarding the closed blind spot94of the external sensor3on the in-vehicle HMI71.FIG.11Bis a top view for describing an outward notification of the object detection device. As illustrated inFIG.11B, the vehicle2includes displays721and722that display information toward the outside of the vehicle. For example, when it is determined that a dynamic object is present in the closed blind spot94A, the notification controller18displays the information regarding the closed blind spot94of the external sensor3on the display722corresponding to the closed blind spot94A. The notification controller18may display information indicating that a dynamic object is in the blind spot of the vehicle2, or may output information for prompting a dynamic object to evacuate from the blind spot. Alternatively, the notification controller18may acquire the shape information of the closed blind spot94from the blind spot information DB13, and project light onto a road surface such that the shape of the closed blind spot94can be visually recognized on the basis of the acquired shape information. Then, the notification controller18may output information for prompting the dynamic object to evacuate from the blind spot while projecting the closed blind spot94.

Operation of Object Detection Device

FIG.12is a flowchart for describing an operation of the object detection device. The flowchart ofFIG.12is executed by the object detection device1at a timing when a start button of the object detection function provided in the vehicle2is turned on, for example.

As illustrated inFIG.12, the entry/exit status acquisition unit14of the object detection device1acquires information regarding the closed blind spot94from the blind spot information DB13as a blind spot information acquisition process (step S10). Subsequently, the vehicle information acquisition unit11of the object detection device1acquires a position of the vehicle2as a vehicle information acquisition process (step S12). Subsequently, the obstacle recognition unit12of the object detection device1recognizes an obstacle as an obstacle recognition process (step S14). Finally, the entry/exit status acquisition unit14and the determination unit15of the object detection device1acquire an entry/exit status of an object into/from the closed blind spot94of the external sensor3and determine the presence or absence of an object in the closed blind spot94on the basis of the entry/exit status as a determination process (step S16). When the determination process (step S16) is finished, the flowchart ofFIG.12ends. When the flowchart ends, the process is executed again from step S10until an end condition is satisfied. The end condition is, for example, when an end button of the object detection function is turned on. The execution order of steps S10to S14is not particularly limited, and steps S10to S14may be performed simultaneously.

SUMMARY OF EMBODIMENTS

According to the object detection device1, the entry/exit status of an obstacle into/from the closed blind spot94of the external sensor3is acquired by the entry/exit status acquisition unit14. The presence or absence of an obstacle in the closed blind spot94is determined by the determination unit15on the basis of the entry/exit status. The closed blind spot94of the external sensor3is, for example, a blind spot that cannot be entered without passing through the visual field of the external sensor3. That is, when the external sensor3detects the entry/exit of an obstacle into/from the closed blind spot94of the external sensor3, the number of obstacles present in the closed blind spot94of the external sensor3increases or decreases. The object detection device1can determine the presence or absence of an object in the blind spot region without estimating a movement position of an obstacle present in the blind spot by ascertaining the entry/exit status of an obstacle into/from the closed blind spot94of the external sensor3. That is, the object detection device1can determine the presence or absence of an obstacle in a target region without directly observing the target region that is a blind spot.

According to the object detection device1, it is ascertained in advance that the number of obstacles present in the closed blind spot94of the external sensor3is 0, and it is detected that the number of obstacles present in the closed blind spot94of the external sensor3does not increase. Therefore, it can be determined that there is no obstacle in the blind spot. According to the object detection device1, the number of obstacles present in the closed blind spot94of the external sensor3is ascertained in advance, and an increase or decrease in the number of obstacles present in the closed blind spot94of the external sensor3is detected. Therefore, the presence or absence of obstacles in the blind spot can be determined.

According to the object detection device1, the number of persons present in the closed blind spot94of the external sensor3is ascertained in advance by the number-of-persons acquisition unit17, and an increase or a decrease in the number of persons present in the closed blind spot94of the external sensor3is detected. Therefore, the presence or absence of obstacles in the blind spot can be determined.

According to the object detection device1, when the vehicle2is stopped and the information indicating that there is no obstacle in the closed blind spot94cannot be acquired, the vehicle2is moved such that a position corresponding to the closed blind spot94in which the information indicating that there is no obstacle cannot be acquired falls within the visual field of the external sensor3. Consequently, the object detection device1can actively ascertain the information indicating that there is no obstacle in the closed blind spot94.

According to the object detection device1, a driver is notified of the information regarding the closed blind spot94of the external sensor3. According to the object detection device1, the information regarding the closed blind spot94of the external sensor3is reported toward the closed blind spot94of the external sensor3outside the vehicle.

Although various exemplary embodiments have been described above, various omissions, substitutions, and changes may be made without being limited to the above exemplary embodiments.