VEHICULAR OBJECT IDENTIFICATION SYSTEM

A vehicular object identification system includes a distance sensor and a processing device. The distance sensor scans a single beam in the horizontal direction so as to measure the distances to points on the surface of an object OBJ. The processing device includes a classifier that is capable of identifying the kind of the object OBJ based on point cloud data PCD that corresponds to the single scan line acquired by the distance sensor. The classifier is implemented based on a learned model generated by machine learning. The machine learning is executed using multiple items of point cloud data that correspond to multiple scan lines acquired by measuring a predetermined object by means of a LiDAR that supports the multiple scan lines in the vertical direction.

BACKGROUND

1. Technical Field

The present disclosure relates to an object identification system.

2. Description of the Related Art

Candidates of vehicle sensors include Light Detection and Ranging, Laser Imaging Detection and Ranging (LiDAR), cameras, millimeter-wave radars, ultrasonic sonars, and so forth. In particular, LiDAR has advantages as compared with other sensors. Examples of such advantages include: (i) an advantage of being capable of identifying an object based on point group data; (ii) an advantage in employing active sensing of providing high-precision detection even in bad weather conditions; (iii) an advantage of providing wide-range measurement; etc. Accordingly, LiDAR is anticipated to become mainstream in vehicle sensing systems.

The precision of object identification based on the point group data generated by the LiDAR increases according to an increase in the resolution of the point group data. However, this involves a drastic increase in calculation costs. In consideration of a case in which the LiDAR is mounted on a vehicle, in some cases, it may be necessary to mount a low-cost, low-end processing device. In this case, such an arrangement naturally requires the number of scan lines to be reduced.

SUMMARY

The present disclosure has been made in view of such a situation.

An embodiment of the present disclosure relates to a vehicular object identification system. The vehicular object identification system includes: a distance sensor structured to scan a single beam in the horizontal direction so as to measure the distances to points on the surface of an object; and a processing device including a classifier structured to be capable of identifying the kind of the object based on point cloud data that corresponds to a single scan line acquired by the distance sensor. The classifier is implemented based on a learned model generated by machine learning. The machine learning is executed using multiple items of point cloud data that correspond to multiple scan lines obtained by measuring a predetermined object by means of a LiDAR (Light Detection and Ranging) including the multiple scan lines in the vertical direction.

DETAILED DESCRIPTION

Overview of the Embodiments

An embodiment disclosed in the present specification relates to a vehicular object identification system. The vehicular object identification system includes: a distance sensor structured to scan a single beam in the horizontal direction so as to measure the distances to points on the surface of an object; and a processing device including a classifier structured to be capable of identifying the kind of the object based on point cloud data that corresponds to a single scan line acquired by the distance sensor. The classifier is implemented based on a learned model generated by machine learning. The machine learning is executed using multiple items of point cloud data that correspond to multiple scan lines obtained by measuring a predetermined object by means of a LiDAR (Light Detection and Ranging) including the multiple scan lines in the vertical direction.

The object identification system allows the kind of an object to be judged using a single scan line. In a case in which the same distance sensor as that to be used in the object identification system is used in learning, in a case in which there is a difference between the height at which the distance sensor is used in learning and the height at which the distance sensor is mounted on the vehicle, such an arrangement has the potential to cause degradation in the object recognition rate. In order to solve this problem, with an arrangement in which training data is acquired while changing the height at which the distance sensor is set for training data acquisition, such an arrangement has a problem of an increased cost for data acquisition. In order to solve such a problem, a LiDAR that supports multiple scan lines and which differs from the distance sensor mounted on the vehicle is used in learning. Specifically, the multiple scan lines are each associated with the single scan line of the distance sensor so as to provide the training data, thereby providing improvement in the efficiency of data acquisition. In addition, the point cloud data that corresponds to the scan lines arranged at different heights is employed as the training data. This allows an object to be identified independent of the height of an emitted beam from the distance sensor.

Also, the distance sensor may include: a light source; a scanning device including a motor and a mirror attached to the motor and structured to reflect emitted light of the light source, in which the scanning device is structured such that probe light, which is light reflected by the mirror, can be scanned according to the rotation of the motor; a photosensor structured to detect return light, which is the probe light reflected from a point on an object; and a processor structured to detect the distance to the point on the object based on the output of the photosensor. With such a distance sensor, the scanning device is configured as a combination of a commonplace motor and mirrors arranged in a fan structure. This provides the distance sensor with a lower cost.

Embodiments

Description will be made below regarding the present disclosure based on preferred embodiments with reference to the drawings. The same or similar components, members, and processes are denoted by the same reference numerals, and redundant description thereof will be omitted as appropriate. The embodiments have been described for exemplary purposes only, and are by no means intended to restrict the present invention. Also, it is not necessarily essential for the present invention that all the features or a combination thereof be provided as described in the embodiments.

FIG. 1is a block diagram showing an object identification system10according to an embodiment. The object identification system10is mounted on a vehicle such as an automobile, motorcycle, or the like. The object identification system10judges the kind (category) of an object OBJ that is present in the surroundings of the vehicle.

The object identification system10mainly includes a distance sensor20and a processing device40. The distance sensor20scans a single beam in the horizontal direction so as to measure the distances to points P on the surface of the object OBJ. The distance sensor20generates a single item of point group data PCD that corresponds to a single scan line SL.

Each item of point cloud data PCD includes the distance information to multiple sampling points P along the scan line SL. The distance sensor20is not restricted in particular. However, in a case in which there is a need to identify an object with small irregularities, such as a pedestrian, with high precision, a LiDAR is preferably employed. It should be noted that typical LiDARs support multiple scan lines in the vertical direction. In contrast, the object identification system10according to the present embodiment supports only a single scan line.

The processing device40includes a classifier42that is capable of classifying the kind of the object OBJ based on a single item of point cloud data PCD that corresponds to a single scan line SL acquired by the distance sensor20. The classifier42is structured using machine learning as described later. The data format of the point group data PCD is not restricted in particular. The data format of the point cloud data PCD may be a rectangular coordinate system or a polar coordinate system.

The processing device40outputs output data OUT that indicates the kind of the object OBJ. Also, the output data OUT may indicate the probability with which the object OBJ included in the point cloud data PCD matches each of multiple categories. It should be noted that the present invention is not restricted to such an arrangement. Examples of such kinds (categories) of the object include a pedestrian, bicycle, automobile, utility pole, etc. Regarding a pedestrian, a pedestrian as viewed from the front, a pedestrian as viewed from the rear, and a pedestrian as viewed from the side may be classified and defined as the same kind of object. The same can be said of an automobile and a bicycle. In the present embodiment, this definition is employed.

The processing device40may be provided as a combination of a processor (hardware component) such as a Central Processing Unit (CPU), Graphics Processing Unit (GPU), microcontroller, or the like, and a software program to be executed by the processor (hardware component). The processing device40may be configured as a combination of multiple processors.

FIG. 2is a block diagram showing an example configuration of the classifier42. The classifier42may be configured employing a neural network NN. The neural network NN is configured including an input layer50, three intermediate layers (hidden layers)52, and an output layer54.

The number of units of the input layer50is determined according to the number of sample points for each line, which is designed to be 5,200. There are three intermediate layers with the number of units designed to be 200, 100, and 50, respectively. In the intermediate layers52, affine transformation and transformation using a sigmoid function are performed. In the output layer54, probability calculation is performed using affine transformation and a softmax function.

The output layer54may be designed to support multiple categories (e.g., four categories: pedestrian (Human), automobile (Car), bicycle (Bicycle), and utility pole (Pole)). In this case, the output data OUT may include four items of data, i.e., Human, Car, Bicycle, and Pole, each indicating the probability that the object OBJ matches the corresponding category.

As the preprocessing for the neural network NN, extraction, shifting, and normalization are preferably performed.

Extraction is processing for removing the background so as to extract the object OBJ. Shifting is data shifting processing for shifting the object such that it is positioned at the center. Normalization is processing for dividing the distance data by a predetermined value. For example, as the predetermined value, the distance (reference distance) between the distance sensor20and a predetermined portion of the object OBJ at the time of the learning may be employed. This processing normalizes the value of the point cloud data such that it becomes a value in the vicinity of1.

The above is the basic configuration of the object identification system10. With the object identification system10, the kind of the object OBJ can be judged using a single scan line. As the number of scan lines becomes larger, the amount of calculation performed by the processing device becomes enormous. Such an arrangement requires a high-speed processor. With the present embodiment, this arrangement requires processing for only a single scan line of point cloud data, thereby allowing the amount of calculation to be reduced. This means that the processing device40can be configured as a low-cost microcontroller. This allows the object identification system10to be provided with a lower cost.

Regarding Learning

Next, description will be made regarding learning of the classifier42. In a case in which the same sensor as that employed in the object identification system10, i.e., the distance sensor20, is used in the learning of the classifier42, and in a case in which there is a difference in height between the distance sensor used in the learning and the distance sensor when it is mounted on the vehicle, such an arrangement has the potential to cause a problem of degradation in the object recognition rate.

In order to solve this problem, an approach is conceivable in which, in the learning, training data (which is also referred to as “learning data”) is acquired while changing the height (or elevation/depression angle) of the distance sensor so as to change the height of the scan line. However, such an approach has a problem of an increased cost of data acquisition.

In order to solve such a problem, with the present embodiment, training data is acquired by means of a LiDAR that supports multiple scan lines, which differs from the distance sensor20mounted on the vehicle.FIG. 3is a block diagram showing a learning system according to an embodiment.

A learning system70includes a LiDAR72and a computer74. In the learning, a LiDAR (Light Detection and Ranging)72that supports multiple scan lines SL1through SLNin the vertical direction is used.FIG. 3shows an example in which there is a pedestrian (human) as the object OBJ. The LiDAR72generates multiple items of point cloud data PCD1through PCDNthat correspond to the multiple scan lines. For example, the number of lines N supported by the LiDAR to be employed for the learning is preferably on the order of eight.

FIG. 3shows an example in which the pedestrian and the LiDAR72are positioned such that they face each other. However, it is preferable to acquire multiple items of point cloud data PCD1through PCDNfor the pedestrian from multiple different directions while changing the orientation of the pedestrian.

The multiple items of point cloud data PCD1through PCDNare input to the computer74. The computer74performs machine learning with the multiple items of point cloud data PCD1through PCDNas the training data so as to allow a given object (a pedestrian in this example) to be identified. With this, the object identification system10shown inFIG. 1is capable of judging the kind of the object based on the point cloud data that corresponds to a single scan line.

It should be noted that all the items of point cloud data PCD1through PCDNthat correspond to the multiple scan lines SL1through SLNare not necessarily required to be used as the training data. Also, only a part of the point cloud data, which corresponds to the multiple scan lines except for both ends of the scan lines (or except for the top scan line or bottom scan line) may be employed as the training data.

In a case in which the object identification system10is to be designed to be capable of identifying multiple kinds of objects, multiple sets of point cloud data may preferably be acquired by means of the LiDAR72while changing the kind of the object OBJ.

Finally, the classifier42of the object identification system10is implemented based on a learned model (trained model) generated by machine learning.

FIGS. 4A through 4Dare diagrams showing multiple items of point cloud data PCD1through PCD8respectively acquired by means of the distance sensor20for a pedestrian, bicycle, automobile, and utility pole. Upon receiving the point cloud data PCDithat corresponds to any one from among the scan lines SLi(i=1 through 8), the classifier42implemented based on the learned model generated by the machine learning described above is capable of judging which one from among the multiple categories matches the point cloud data PCDiwith a high probability.

FIG. 5is a flowchart showing the learning by the learning system70. At least one predetermined object is measured using the LiDAR72that supports multipole scan lines in the vertical direction, and that differs from the distance sensor20(S100). With this, multiple items of point cloud data PCD1through PCDNthat correspond to the multiple scan lines are generated for each object.

Subsequently, machine learning is performed with each of the multiple items of point cloud data PCD1through PCDNas training data so as to allow a given object to be identified (S102). Subsequently, the classifier42is implemented based on a learned model generated by machine learning (S104).

The above is a description of the learning system70and the learning method. With the learning system70or the learning method, the multiple scan lines SL1through SLNof the LiDAR72are each associated with a single scan line of the distance sensor. This provides efficient data acquisition.

Furthermore, learning is performed using point cloud data acquired by scan lines having different heights. This enables object recognition independent of the height of the emitted beam of the distance sensor20. That is to say, this means that the restriction on the height at which the distance sensor20is mounted on a vehicle is relaxed. Furthermore, this means that such an arrangement provides improved tolerance for pitching of a vehicle while traveling.

Next, description will be made regarding an example configuration of the distance sensor20.FIG. 6is a block diagram showing a distance sensor100according to an embodiment. The distance sensor100includes a light source110, a scanning device120, a photosensor130, and a processor140. The light source110emits light L1having an infrared spectrum, for example. The emitted light L1of the light source110may be modulated with respect to time.

The scanning device120includes a motor122and one or multiple mirrors (which will be also referred to as “blades”)126. The mirrors126are configured to have a fan structure. The mirrors126are attached to a rotational shaft124of the motor122such that they reflect the emitted light L1of the light source110. The emission angle (which will also be referred to as a “scan angle”) θ of probe light L2, which is light reflected from the mirrors126, changes according to the positions of the mirrors126(i.e., rotational angle ϕ of the motor). Accordingly, by rotationally driving the motor122, the probe light L2can be scanned in the θ direction ranging between θMINand θMAX. It should be noted that, in a case in which the number of mirrors126thus provided is two, one half-rotation of the motor122(mechanical angle of 180 degrees) corresponds to a single scan. Accordingly, the probe light L2is scanned twice every time the motor122is rotated once. It should be noted that the number of the mirrors126is not restricted in particular.

The rotational angle ϕ of the motor122can be detected by means of a position detection mechanism such as a Hall sensor, optical encoder, or the like. Accordingly, the scan angle θ at each time point can be obtained based on the rotational angle ϕ. In a case in which a stepping motor is employed as the motor122, the rotational angle ϕ can be controlled by an open-loop control operation, thereby allowing the position detection mechanism to be omitted.

The photosensor130detects return light L3which is the probe light L2reflected at a point P on an object OBJ. The processor140detects the distance to the point P on the object OBJ based on the output of the photosensor130. The distance detection method or algorithm is not restricted in particular. Rather, known techniques may be employed. For example, the delay time from the emission of the probe light L2to the reception of the return light by means of the photosensor130, i.e., the time of flight (TOF), may be measured so as to acquire the distance.

The above is the basic configuration of the distance sensor100. Next, description will be made regarding the operation thereof. The motor122is rotationally driven so as to change the scan angle θ of the probe light L2in the order of θ1, θ2, . . . . In this operation, the distance rito the point Pion the surface of the object OBJ is measured at each scan angle θ1(i=1, 2, . . . ). With this, data pairs (point cloud data) each configured as a pair of the scan angle θ1and the corresponding distance ri, can be acquired.

With such a distance sensor100, the scanning device120can be configured as a combination of the motor122configured as a commonplace motor and the mirrors126arranged in a fan structure. This provides the distance sensor100with a lower cost.

FIG. 7is a block diagram showing an automobile provided with the object identification system10. An automobile300is provided with headlamps302L and302R. At least one from among the headlamps302L and302R is provided with the object identification system10as a built-in component. Each headlamp302is positioned at a frontmost end of the vehicle body, which is most advantageous as a position where the distance sensor100is to be installed for detecting an object in the vicinity.

FIG. 8is a block diagram showing an automotive lamp200including the object detection system400. The automotive lamp200forms a lamp system310together with an in-vehicle ECU304. The automotive lamp200includes a light source202, a lighting circuit204, and an optical system206.

Furthermore, the automotive lamp200is provided with the object detection system400. The object detection system400corresponds to the above-described object identification system10, and includes the distance sensor100and a processing device410. The distance sensor100corresponds to the distance sensor20shown inFIG. 2. The processing device410judges the presence or absence and the kind of an object OBJ in front of the vehicle based on point cloud data acquired by the distance sensor100. The processing device410may include an identifying device obtained by machine learning. The processing device410corresponds to the processing device40shown inFIG. 2.

Also, the information with respect to the object OBJ detected by the processing device410may be used to support the light distribution control operation of the automotive lamp200. Specifically, a lamp ECU208generates a suitable light distribution pattern based on the information with respect to the kind of the object OBJ and the position thereof generated by the processing device410. The lighting circuit204and the optical system206operate so as to provide the light distribution pattern generated by the lamp ECU208.

Also, the information with respect to the object OBJ detected by the processing device410may be transmitted to the in-vehicle ECU304. The in-vehicle ECU may support autonomous driving based on the information thus transmitted.

Description has been made above regarding the present invention with reference to the embodiments. The above-described embodiments have been described for exemplary purposes only, and are by no means intended to be interpreted restrictively. Rather, it can be readily conceived by those skilled in this art that various modifications may be made by making various combinations of the aforementioned components or processes, which are also encompassed in the technical scope of the present invention. Description will be made below regarding such modifications.

In an embodiment, the object may be defined as a different kind (category) for each orientation as viewed from the user's vehicle. That is to say, the same object is identified as a different kind according to the orientation thereof, e.g., whether or not the object is positioned with a face-to-face orientation with respect to the user's vehicle. This is because such identification is advantageous in estimating the object OBJ moving direction.

The processing device40may be configured of only a hardware component using an FPGA or the like.

Description has been made regarding the present invention with reference to the embodiments using specific terms. However, the above-described embodiments show only an aspect of the mechanisms and applications of the present invention. Rather, various modifications and various changes in the layout can be made without departing from the spirit and scope of the present invention defined in appended claims.