Patent ID: 12248074

DESCRIPTION OF EMBODIMENTS

Examples of embodiments will be described in detail below with reference to the accompanying drawings. In each drawing using the description hereinafter, a scale is appropriately changed in order to show each of members in a recognizable size.

First Embodiment

FIG.1is a top view showing a vehicle1to which a LiDAR sensor unit10according to a first embodiment of the present invention is attached. As shown inFIG.1, the LiDAR sensor unit10is attached to a right front portion, a left front portion, a right rear portion, and a left rear portion of the vehicle1.

FIG.2is a schematic view showing the LiDAR sensor unit10attached to the left front portion of the vehicle1. In the present embodiment, the LiDAR sensor unit10is provided together with a lamp unit30configured to illuminate surroundings of the vehicle1. The lamp unit30can be a head lamp, a rear combination lamp, a daytime running lamp, a fog lamp, a clearance lamp, or a stop lamp. The LiDAR sensor unit10includes a housing11and an outer cover12. The lamp unit30and a LiDAR sensor20are provided inside a space formed by the housing11and the outer cover12.

FIG.3is a schematic view showing the LiDAR sensor20. As shown inFIG.3, the LiDAR sensor20includes a light emitting unit41, a light receiving unit42, a MEMS mirror43, and a converging lens44.

Light emitted from the light emitting unit41is reflected by the MEMS mirror43, and is emitted to an outside of the vehicle1via the converging lens44. The light (hereinafter referred to as return light) that is emitted from the light emitting unit41and is reflected by an object is reflected by the MEMS mirror43via the converging lens44, and is incident on the light receiving unit42. The light emitting unit41may emit visible light or emit invisible light such as infrared light or ultraviolet light. The LiDAR sensor20is configured to emit light to the object, and to acquire a distance to the object based on time until the return light is detected. The MEMS mirror43is configured to freely change a direction in which the light reflected by the MEMS mirror43is emitted. The LiDAR sensor20is configured to acquire information in a wide range by changing the emission direction of the reflected light by the MEMS mirror43.

FIG.4is a block diagram showing the LiDAR sensor20. As shown inFIG.4, the LiDAR sensor20includes the light emitting unit41, a light source control unit45, the MEMS mirror43, a mirror control unit46, the light receiving unit42, a signal processing unit47, and a memory48. The light source control unit45is configured to control an operation of the light emitting unit41. The mirror control unit46is configured to control an operation of the MEMS mirror43. The memory48is a rewritable recording unit. The signal processing unit47is configured to process a signal output from the light receiving unit42and to output the processed signal to a vehicle control unit3. The vehicle control unit3is configured to control an operation of the vehicle1.

The vehicle control unit3is configured to control operations of an engine, a brake device, and a steering device according to a signal output in response to an output from the LiDAR sensor20or another sensor, an operation of a steering wheel of a driver, an operation of an accelerator pedal of the driver, or an operation of a brake pedal of the driver. The vehicle control unit3is configured to perform automatic driving of the vehicle1. Alternatively, the vehicle control unit3is configured to support driving of the driver.

The signal processing unit47includes a processor and a memory. Examples of the processor include a CPU, an MPU, and a GPU. The processor may include a plurality of processor cores. Examples of the memory include a ROM and a RAM. In the ROM, a program for executing the above-described processing may be stored. The program may include an artificial intelligence program. Examples of the artificial intelligence program include a learned neural network based on deep learning. The processor is configured to specify at least a part of programs stored in the ROM, to load the programs on the RAM, and to execute the processing in cooperation with the RAM. The signal processing unit47may be implemented by a dedicated integrated circuit such as a microcontroller, an FPGA, or an ASIC.

A detection principle of the LiDAR sensor20will be briefly described with reference toFIG.5.

The signal processing unit47acquires information indicating how much time is taken for light emitted in which direction to hit the object and be reflected. Based on this, the signal processing unit47outputs direction information and a distance to the object in the direction.

Generally, the LiDAR sensor20is configured to output data of an azimuth angle θ [°], an elevation angle φ [°], and a distance d [m].

For example, the light source control unit45is configured to control the light emitting unit41such that the light emitting unit41emits light at predetermined time intervals. The mirror control unit46is configured to set 100 detection points in a vertical direction and 360 detection points in a horizontal direction in a certain detection range. The mirror control unit46is configured to control the MEMS mirror43such that the MEMS mirror43sequentially reflects the light incident on the MEMS mirror43from the light emitting unit41toward each detection point.

The light receiving unit42is configured to detect the return light that hits the object and is reflected by the object at each measurement point, and to output to the signal processing unit47a fact that the return light is detected. Since the MEMS mirror43sequentially reflects the light emitted from the light emitting unit41toward each measurement point, the return light sequentially detected by the light receiving unit42can be treated as light reflected from each measurement point. For example, when 36000 detection points are set, return light detected at a first time and return light detected at a 36001th time can be treated as light returning from the same direction. Alternatively, in a case in which 0.01 seconds are taken to emit light toward all detection points of 360000 detection points, light emitted after 0.01 seconds from a certain time and light emitted after 0.02 seconds from the certain time can be treated as light emitted in the same direction.

The signal processing unit47measures, for each measurement point, time from a time when the light emitting unit41emits light to a time when the return light is detected. The signal processing unit47calculates the distance to the object based on the measured time. In this way, the signal processing unit47outputs the distance in association with the direction of each detection point. The signal processing unit47outputs data such as (θ, φ, d) in order to indicate that a distance in a certain direction (θ, φ) is d.

As shown inFIG.2, the outer cover12is provided in a direction in which the light is emitted from the light emitting unit41of the LiDAR sensor20and in a direction in which the light is incident on the light receiving unit42. The outer cover12is often formed of a curved surface due to design restrictions of the vehicle1. In particular, as shown inFIG.1, when the LiDAR sensor20is provided at a corner portion of the vehicle1, such as the right front portion, the left front portion, the right rear portion, and the left rear portion of the vehicle1, a portion12awhere a curvature of the curved surface of the outer cover12locally increases is present as shown inFIG.2.

When the light of the LiDAR sensor20passes through the portion12awhere the curvature locally increases in this way, the light is refracted. Therefore, the LiDAR sensor20measures a distance in a direction deviated from the originally assumed direction. Therefore, in the LiDAR sensor20according to the present embodiment, a direction of large refraction is specified and recorded in the memory48before a product is shipped, and the LiDAR sensor20does not output data of the direction in the product after the shipment. This processing will be described in detail.

FIG.6shows a state in which the LiDAR sensor20is operated toward a standard screen S in a state in which the outer cover12is not attached to the LiDAR sensor unit10.

FIG.7a visualization of an output of the LiDAR sensor20that is output in the state shown inFIG.6.FIGS.7and9are schematic views showing the output of the LiDAR sensor20. A large number of output points are actually present. However,FIGS.7and9are shown with the number of points smaller than the number of the actual output points for convenience of drawing.

The standard screen S is a flat screen having a predetermined size. InFIG.6, the LiDAR sensor20is operated by separating the standard screen S from the LiDAR sensor20by a predetermined distance. Since the standard screen S is flat, a point group of the measurement points in the output of the LiDAR sensor20also forms a flat surface. The signal processing unit47records, in the rewritable memory48, an output when the standard screen S is sensed in a state in which the outer cover12is not attached to the LiDAR sensor unit10.

FIG.8shows a state in which the LiDAR sensor20is operated toward the standard screen S in a state in which the outer cover12is attached to the LiDAR sensor unit10.FIG.9a visualization of an output of the LiDAR sensor20that is output in the state shown in FIG.8.

When the LiDAR sensor20is operated in a state in which the outer cover12is attached, as shown inFIG.8, the light is refracted at the portion12awhere the curvature is locally large in the outer cover12. InFIG.8, a trajectory of the passing light in the case in which the outer cover12is not provided is shown by a broken line. InFIG.8, a state of the refraction of the light is shown in a deformed manner.

As shown inFIG.6, coordinates at which the light emitted in a direction (θ1, φ1) toward the portion12ahaving the large curvature reaches the standard screen S are expressed as P1=(θ1, φ1, d1). That is, in the direction (θ1, φ1), a distance to the standard screen S is d1.

As shown inFIG.8, coordinates at which the light emitted in the direction (θ1, φ1) toward the portion12ahaving the large curvature and refracted by the portion12areaches the standard screen S are expressed as P2=(θ1, φ1, d2).

That is, actually, if the outer cover12is attached even though the distance to the standard screen S in the direction (θ1, φ1) is d1, the LiDAR sensor20erroneously recognizes the distance to the standard screen S in the direction (θ1, φ1) as d2. Therefore, when data output from the LiDAR sensor20as it is visualized, as shown inFIG.9, the visualized data is different from data inFIG.7.

Therefore, in the present embodiment, the output when the standard screen S is sensed in the state in which the outer cover12is attached to the LiDAR sensor unit10is compared with the output when the standard screen S is sensed in the state in which the outer cover12is not attached to the LiDAR sensor unit10. When a difference of distances in the same direction is equal to or greater than a threshold value, the LiDAR sensor20does not output the distances in the direction.

That is, in examples shown inFIGS.6to9, for the direction (θ1, φ1), when an absolute value Δd=|d2−d1| of a difference between the distance d1when the outer cover12is not attached to the LiDAR sensor unit10and the distance d2when the outer cover12is attached to the LiDAR sensor unit10is larger than a threshold value da (Δd>da), the direction is recorded in the memory48as a non-output direction.

In the shown example, the LiDAR sensor20calculates a length of an optical path along which the refracted light travels as the distance d2. That is, the distance d2is a sum of a distance d0from the LiDAR sensor20to the portion12aand a distance d2′ from the portion12ato a point P2. When a distance from the portion12ato a point P1is d1′. Δd=d2−d1=(d0+d2′)−(d0+d1′)=d2′−d1′. When d2′−d1′ is larger than the predetermined value da, the signal processing unit47records the direction (θ1, φ1) in the memory48as the non-output direction.

Further, when the LiDAR sensor20is normally used, the signal processing unit47reads out the non-output direction from the memory48and does not output data for a direction corresponding to the non-output direction as shown inFIG.10.FIG.10visualizes and shows output data output from the signal processing unit47to the vehicle control unit3when the inspection screen S is detected. As shown inFIG.10, even if a part of the data output from the LiDAR sensor20is missing, an entire image can be grasped.

In general, a large number of measurement points (measurement directions) of the LiDAR sensor20are present. Therefore, even if measurement points affected by the refraction of the outer cover12are ignored, a good resolution is easily maintained. Rather, according to the present embodiment, since it is not necessary to correct data due to refraction of the outer cover12or execute exceptional processing, a processing load of the signal processing unit47is reduced.

The signal processing unit47may determine whether a ratio n/m of the number (n) of non-detection directions to the number (m) of all the measurement points is equal to or greater than 0.1. When the ratio is equal to or greater than 0.1, the signal processing unit47may output an error signal to the vehicle control unit3and may output data for the direction corresponding to the non-output direction.

In the above-described embodiment, an example has been described in which the measurement points before and after the attachment of the outer cover12are treated.

However, the present invention is not limited thereto. In the LiDAR sensor unit10, an inner lens may be provided inside a space formed by the housing11and the outer cover12. Even before and after the attachment of the inner lens, a direction to be measured may change due to refraction of the inner lens. The present invention can also be applied to this case.

That is, the output when the standard screen S is sensed in the state in which the inner lens is attached to the LiDAR sensor unit10is compared with the output when the standard screen S is sensed in the state in which the inner lens is not attached to the LiDAR sensor unit10. When a difference of distances in the same direction is equal to or greater than a threshold value, the LiDAR sensor20may not output the distances in the direction.

In the present embodiment, the output when the standard screen S is sensed in the state in which the outer cover12is not attached to the LiDAR sensor unit10and the output when the standard screen S is sensed in the state in which the outer cover12is attached to the LiDAR sensor unit10are recorded in the rewritable memory48.

The outer cover12may be scratched or damaged by a flying stone or the like. As described above, refraction may occur at a portion where a scratch is formed or the light may be scattered, and the distance may not be measured properly. Even if the scratch is small and inconspicuous, if the scratch is located on an extension line of the measurement direction, the LiDAR sensor20is affected by the scratch. However, it is not realistic to replace the outer cover12since the small scratch is formed.

Therefore, in the present embodiment, it is possible to periodically update information recorded in the memory48. For example, in a periodic inspection of the vehicle or the like, an inspection using the above-described standard screen S is performed. In the inspection, the output when the standard screen S is sensed in the state in which the outer cover12is not attached to the LiDAR sensor unit10and the output when the standard screen S is sensed in the state in which the outer cover12is attached to the LiDAR sensor unit10can be acquired and rewritten periodically. Accordingly, it is possible not to output the data for the direction in which the distance cannot be measured due to the scratch, and it is possible to avoid a defect caused by the scratch.

Even when the outer cover12is replaced, by updating the information recorded in the memory48, it is possible not to output data in a direction in which a degree of refraction is large according to a shape of the outer cover12after the replacement.

An electronic circuit that is configured to implement a function of specifying the non-output direction and not outputting the distance in the non-output direction to the vehicle control unit3and that is described with reference toFIGS.6to10may be provided separately from an electronic circuit configured to implement a function of detecting the incidence of the light on the light receiving unit and calculating the distance of the object. That is, the electronic circuit incorporated in the LiDAR sensor20is configured to implement the function of detecting the incidence of the light on the light receiving unit and calculating the distance of the object, and the electronic circuit configured to implement the function of specifying the non-output direction and not outputting the distance in the non-output direction to the vehicle control unit3may be provided in a middle of a communication path between the LiDAR sensor20and the vehicle control unit3.

Second Embodiment

A LiDAR sensor can measure a precise distance. The LiDAR sensor can acquire a large amount of measurement data at an extremely short time interval. However, as described above, since the distance can be measured with an extremely high resolution and the measurement data can be acquired at the extremely short time interval, the LiDAR sensor is greatly affected by a fluctuation due to a vibration of the vehicle, sound, heat, wind, mere noise, or the like.

In order to be used for the automatic driving of the vehicle, an output of the LiDAR sensor including fine distance information such as 0.1 mm or less is unnecessary.

Therefore, in a second embodiment of the present invention to be described next, the LiDAR sensor unit10suitable for a vehicle that can be automatically driven is provided.

The LiDAR sensor unit10according to the second embodiment has the same structure and detection principle as the LiDAR sensor unit10described with reference toFIGS.1to5in the above-described first embodiment. Therefore, the description of the structure and the detection principle of the LiDAR sensor unit10according to the second embodiment will be omitted.

FIG.11is a flowchart showing processing executed by the signal processing unit47.

As shown inFIG.11, the signal processing unit47calculates that a distance to an object in the direction (θ, φ) at the time t is D (step S01). That is, raw data at the time t is (θ, φ, D). The signal processing unit47records the raw data (θ, φ, D) at the time tin the memory48.

Next, the signal processing unit47reads out last 10 distances for the certain direction (θ, φ) (step S02). The signal processing unit47extracts a minimum value dmin and a maximum value dmax from these values. The signal processing unit47calculates an average value dmean of the last 10 distances for the certain direction (θ, φ) (step S03). The signal processing unit47records dmin, dmax, and dmean in the memory48.

Next, the signal processing unit47calculates, for the certain direction (θ, φ), a difference Δd between an immediately preceding distance D [t−1] and a distance D [t] measured this time (step S04). The signal processing unit47calculates Δd=D [t]−D [t−1]. The signal processing unit47records the difference Δd in the memory48.

The signal processing unit47reads out the minimum value dmin and the maximum value dmax from the memory48and multiplies a difference between the minimum value dmin and the maximum value dmax by a coefficient of 0.5 to calculate a fluctuation threshold value dth (step S05). The signal processing unit47records the fluctuation threshold value dth in the memory48.

The signal processing unit47reads out the difference Δd and the fluctuation threshold value dth from the memory48, and compares absolute values of the difference Δd and the fluctuation threshold value dth (step S06).

When the absolute value of the difference Δd is larger than the absolute value of the fluctuation threshold value dth (step S06: No), the signal processing unit47outputs the raw data D [t] as the distance in the certain direction (θ, φ). That is, the signal processing unit47outputs (θ, q, D [t]) to the vehicle control unit3as measurement data at the time t (step S07).

When the absolute value of the difference Δd is smaller than the absolute value of the fluctuation threshold value dth (step S06: Yes), the signal processing unit47outputs the average value dmean as the distance in the certain direction (θ, φ. That is, the signal processing unit47outputs (θ, φ, dmean) to the vehicle control unit3as measurement data at the time t (step S08).

FIG.12is a graph plotting the measurement data and the raw data that are obtained in this way. InFIG.12, for the certain direction (θ, φ), the distance D of the raw data output from the LiDAR sensor20is indicated by X, and the distance d of data (referred to as output data) that is processed by the signal processing unit47and is output to the vehicle control unit3is indicated by a square. A portion where the distance of the raw data and the distance of the output data overlap each other is represented by the distance (the square) of the output data.

As shown inFIG.12, the fluctuation of the output data represented by the square is reduced as compared with the raw data represented by x.

Incidentally, the present inventor has noticed that the output of the LiDAR sensor is likely to fluctuate finely. However, even if the measurement data that fluctuates finely is output to the vehicle control unit3as it is, the vehicle control unit3does not control the vehicle1using a value that fluctuates finely in this way.

In order to reduce this fine fluctuation, it is conceivable that a certain threshold value is set and a fluctuation within the threshold value is not output to the vehicle control unit3.

However, the present inventor has noticed that, in the output of the LiDAR sensor, the threshold value suitable for a plurality of factors such as a temperature, humidity, or a traveling state changes. That is, even if a certain threshold value is an appropriate threshold value in a certain situation, too many fluctuations beyond the threshold value occur and the threshold value becomes meaningless in another situation. Alternatively, too many fluctuations below the threshold value occur and only certain distance data is output in another situation.

Therefore, the present inventor has completed the present invention in which the distance data in a predetermined period is always accumulated, the threshold value is determined based on a maximum value and a minimum value during the predetermined period, and the measurement data is output to the vehicle control unit3with the distance within the threshold value that fluctuates depending on the time as a constant distance. According to this aspect, since the fluctuation threshold value dth suitable for the situation is appropriately set, it is possible to smooth the fluctuation of the distance data that fluctuates due to various factors with the appropriate fluctuation threshold value dth.

In the above-described embodiment, the reading of the nearest 10 distances in the certain direction has been described. However, the number of distances to be read out is not limited to 10. Any number of pieces of data may be read out. As the number of pieces of data to be read is set to be large, the data output to the vehicle control unit3is less likely to fluctuate. This coefficient may be fluctuated according to a driving situation or the like according to a signal or the like output from the vehicle control unit3. For example, when it is assumed that there is little change in a state of the outside of the vehicle, such as during traveling on a highway, a large number of pieces of data may be read out.

In the above-described embodiment, it has been described that the coefficient of 0.5 is multiplied by (dmax−dmin) in order to calculate the fluctuation threshold value. However, the coefficient to be multiplied by (dmax−dmin) can be set freely. The coefficient can be set to a value of 0 to 1. When the coefficient is increased, it is possible to reduce the fluctuation in the distance output from the LiDAR sensor unit to the vehicle control unit3. When the coefficient is reduced, the distance output from the LiDAR sensor unit to the vehicle control unit3is easily reproduced faithfully to the raw data output from the LiDAR sensor.

This coefficient may be fluctuated according to the driving situation or the like according to the signal or the like output from the vehicle control unit3. For example, when it is assumed that there is little change in the state of the outside of the vehicle, such as during traveling on a highway, a large coefficient may be set.

In the above-described embodiment, an example has been described in which the average value dmean at last 10 points is set as the distance of the output data when the fluctuation value is within the fluctuation threshold value (step S06: Yes). However, the present invention is not limited thereto. Instead of the average value dmean, a distance of any of the last 10 points may be output, or the minimum value dmin or the maximum value dmax may be output.

Third Embodiment

A LiDAR sensor can measure a distance at a large number of measurement points, and can accurately grasp a surrounding situation. However, since the LiDAR sensor outputs a large number of measurement points in short time, a large burden is generated on a processing apparatus configured to process the measurement points. In a control of a vehicle in automatic driving or manual driving, some action is required when a situation changes.

Therefore, in a third embodiment of the present invention, a LiDAR sensor unit having an output suitable for the control of the vehicle is provided.

Next, a LiDAR sensor unit110according to the third embodiment of the present invention will be described.

The LiDAR sensor unit110according to the third embodiment also has the same structure as the LiDAR sensor unit10described with reference toFIGS.1to3in the above-described first embodiment. A detection principle of the LiDAR sensor unit110according to the third embodiment is also the same as the detection principle described with reference toFIG.5. Therefore, the description of the structure and the detection principle of the LiDAR sensor unit110according to the third embodiment will be omitted.

A functional block of the LiDAR sensor unit110according to the third embodiment is different from a functional block of the LiDAR sensor unit10according to the first embodiment.FIG.13is a block diagram showing a LiDAR sensor120according to the third embodiment of the present invention. As shown inFIG.13, the LiDAR sensor120includes a light emitting unit141, a light source control unit145, a MEMS mirror143, a mirror control unit146, a light receiving unit142, a signal processing unit147, a memory148, and an image forming unit149. The light source control unit145is configured to control an operation of the light emitting unit141. The mirror control unit146is configured to control an operation of the MEMS mirror143. The memory148is a rewritable recording unit. The signal processing unit147is configured to process a signal output from the light receiving unit142and to output the processed signal to the image forming unit149. The vehicle control unit103is configured to control an operation of a vehicle101. The image forming unit149is configured to form an image based on the signal output from the signal processing unit147and to output the image to the vehicle control unit103.

FIG.14is a flowchart showing processing executed by the image forming unit149.FIG.15shows a state of a front of a vehicle at the time t−1 before predetermined time from the certain time t. A point group inFIG.15shows a visualization of measurement data of the LiDAR sensor120as it is. As shown inFIG.15, at the time t−1, a tree T is present on a left side of a front of the vehicle101. At the time t−1, the LiDAR sensor120measures the distance d for all measurement points. The image forming unit149acquires measurement data P [t−1]=(θ[t−1], φ [t−1], d [t−1]) at the time t−1 (step S101).

FIG.16shows a state of the front of the vehicle at the certain time t. A point group inFIG.16shows a visualization of measurement data of the LiDAR sensor120as it is. At the time t, the tree T is present on the left side of the front of the vehicle101, and a pedestrian is present on a right side of the front of the vehicle101. That is, when time changes from the time t−1 to the time t, the pedestrian enters the front of the vehicle.

At the time t, the LiDAR sensor120also measures the distance for all the measurement points. The image forming unit149acquires measurement data P [t]=(θ [t], φ [t], d [t]) at the time t (step S102).

The image forming unit149calculates the fluctuation Δd=d [t]−d [t−1] of the distance in the same direction (θ, φ) at the time t−1 and the time t (step S103).

The image forming unit149forms an image in which the output of the LiDAR sensor120at the time t is visualized. At this time, as shown inFIG.17, the image forming unit149emphasizes a portion changed from the time t−1 to the time t to form an image.

First, the image forming unit149determines whether Δd is greater than the predetermined threshold value dth for all the directions (step S104).

For a direction in which Δd is smaller than the predetermined threshold value dth (step S104: Yes), the image forming unit149visualizes the distance d [t] at the time t based on a normal method (step S105). For example, the distance d [t] is drawn with a black dot on a white background for the directions θ and φ.

When Δd is equal to or greater than the predetermined threshold value dth (step S104: No), the image forming unit149visualizes the distance d [t] at the time t based on a method different from the normal method (step S106). For example, the distance d [t] is drawn with a red circle on the white background for the directions θ and φ.

In this way, the image forming unit149forms the image shown inFIG.17and outputs the image to the vehicle control unit103. The image forming unit149forms an image in any format such as BMP, PNG, GIF, JPEG, or TIFF.

That is, the image forming unit149compares the first measurement data P [t−1] acquired from the LiDAR sensor120at the first time t−1 with the second measurement data P [t] acquired from the LiDAR sensor120at the second time t after predetermined time has elapsed from the first time t−1, and generates an image in which the second measurement data P [t] having the same direction and a difference in distance being equal to or greater than the threshold value is represented in a mode different from that of the second measurement data P [t] having the same direction and a difference in distance being less than the threshold value.

The vehicle control unit103may acquire an image from the image forming unit149and may display the image on a display provided in a vehicle interior.

In this way, according to the LiDAR sensor unit110in the present embodiment, a measurement point that is different from an immediately preceding situation is displayed separately from a point that is not changed from the immediately preceding situation. Since the measurement point different from the immediately preceding situation can be instantly grasped, the vehicle control unit103can instantly grasp the point different from the immediately preceding situation and can quickly perform a necessary operation. For example, in the present embodiment, the vehicle control unit103can instantly grasp the appearance of the pedestrian different from the immediately preceding situation, and can stop the vehicle101.

The image forming unit149may be integrated with the signal processing unit147or may be separated from the signal processing unit147. The image forming unit149may be provided in a middle of a communication path between the signal processing unit147and the vehicle control unit103.

Fourth Embodiment

Since a LiDAR sensor can acquire information on surroundings of a vehicle, it is conceivable to use the LiDAR sensor as a security sensor configured to monitor a suspicious person trying to open a door while the vehicle is parked. However, a viewpoint is present that a LiDAR consumes a large amount of power and is difficult to be used as a security sensor.

Therefore, a fourth embodiment of the present invention provides a power saving vehicle security system using the LiDAR sensor.

FIG.18is a top view showing the vehicle1to which a vehicle security system202according to the present embodiment is attached. The vehicle security system202includes a plurality of LiDAR sensor units210. As shown inFIG.18, the LiDAR sensor unit210is attached to a right front portion, a left front portion, a right rear portion, and a left rear portion of the vehicle1.

Since the LiDAR sensor unit210used in the present embodiment is the same as the LiDAR sensor unit10described with reference toFIGS.2and3in the above-described first embodiment, the description thereof will be omitted.

FIG.19is a block diagram showing the vehicle security system202. As shown inFIG.19, the vehicle security system202includes a LiDAR sensor unit210, a lamp unit230, an image forming unit251, an image comparing unit252, and a lamp control unit253(a control unit). As shown inFIG.19, the LiDAR sensor unit210includes a light emitting unit241, a light source control unit245, a MEMS mirror243, a mirror control unit246, a light receiving unit242, and a signal processing unit247. The light source control unit245is configured to control an operation of the light emitting unit241. The mirror control unit246is configured to control an operation of the MEMS mirror243. The lamp control unit253is configured to control turning on and off of the lamp unit230. The signal processing unit247is configured to process a signal output from the light receiving unit242and to output the processed signal to the vehicle control unit203. The vehicle control unit203is configured to control an operation of a vehicle201.

FIG.20is a flowchart showing processing executed by the vehicle security system202.FIG.21shows a state of surroundings of the parked vehicle201at the time t−1 before predetermined time from the certain time t. At the time t−1, the vehicle201is parked in a parking lot.FIG.22is an image I [t−1] formed by the image forming unit251based on measurement data acquired by the LiDAR sensor220in the state shown inFIG.21.

At the time t−1, the LiDAR sensor220measures a distance for all measurement points. The image forming unit251acquires the measurement data P [t−1]=(θ [t−1], φ [t−1], d [t−1]) at the time t−1, and forms the image I [t−1] shown inFIG.22(step S201).

FIG.23shows a state of surroundings of the parked vehicle201at the time t. At the time t, a pedestrian A approaches the parked vehicle201from upper right inFIG.23.FIG.24is an image I [t] formed by the image forming unit251based on measurement data acquired by the LiDAR sensor220in the state shown inFIG.23.

At the time t, the LiDAR sensor220measures a distance for all the measurement points. The image forming unit251acquires the measurement data P [t]=(θ [t], φ [t], d [t]) at the time t, and forms the image I [t] shown inFIG.24(step S202).

The image comparing unit252compares the image I [t−1] acquired at the time t−1 with the image I [t] acquired at the time t. The image comparing unit252determines whether a difference equal to or greater than a predetermined value is generated between the two images I [t−1] and I [t] (step S203). For example, when the number of pixels having a difference between the two images [t−1] and I [t] is equal to or greater than a predetermined value, the image comparing unit252determines that the difference equal to or greater than the predetermined value is generated. For example, when the number of pixels having a difference is 3% or more with respect to the total number of pixels constituting the image, the image comparing unit252determines that the difference equal to or greater than the predetermined value is generated between the two images I [t−1] and I [t]. A ratio of the number of pixels having a difference to all the pixels to be set to a threshold value may be changed stepwisely by a user, such as 5% or 10%.

When no difference equal to or greater than the predetermined value is present between the two images I [t−1] and I [t] (step S203: No), the vehicle security system202executes step S201again after a first period elapses (step S204).

When the difference equal to or greater than the predetermined value is generated between the two images I [t−1] and I [t] (step S203: Yes), the image comparing unit252performs alarm processing (step S205).

In the present embodiment, the image comparing unit252performs the following alarm processing. First, the image comparing unit252identifies in which direction the difference is generated when viewed from the host vehicle201. Further, the image comparing unit252outputs the identified direction to the light source control unit245and the mirror control unit246of the LiDAR sensor220. In the shown example, the direction to be identified is only an upper right position direction of the host vehicle201. However, when the difference is generated in a plurality of directions, the plurality of directions may be specified and output to the LiDAR sensor220. Further, the lamp control unit turns on the lamp unit230so as to illuminate the identified direction. The image comparison unit252may output a signal notifying an abnormality to a mobile phone owned by the user.

Further, the image comparing unit252determines whether an alarm release signal has been acquired (step S206). For example, the image comparing unit252is able to acquire a predetermined alarm release signal from the vehicle control unit203or the mobile phone of the user.

When the image comparing unit252acquires the alarm release signal within a predetermined period (step S206: Yes), the image comparing unit252releases the alarm (step S207) and ends the processing.

When the image comparison unit252does not acquire the alarm release signal within the predetermined period (step S206: No), the vehicle security system202executes step S201after a second predetermined period elapses (step S208). Here, the second predetermined period is set to be shorter than the first predetermined period. For example, the first predetermined period is 30 seconds, and the second predetermined period is 0.5 seconds. Alternatively, the first predetermined period is 60 seconds, and the second predetermined period is one second. That is, the vehicle security system202repeats steps S201to S203at a cycle of approximately 30 seconds.

Measurement data acquired at a first predetermined time interval is referred to as first measurement data. Measurement data acquired at a second predetermined time interval is referred to as second measurement data. In other words, in the above-described embodiment, the second measurement data is acquired in the second period shorter than the first period when a fluctuation greater than the predetermined value occurs during the measurement of the first measurement data.

Since the LiDAR sensor220can acquire a shape of the object with high accuracy, the LiDAR sensor220is suitable for a security system. In many cases, the LiDAR sensor220is mounted on the vehicle201that can be automatically driven. The LiDAR sensor220mounted on the vehicle201that can be automatically driven operates while the vehicle201travels, but is not used while the vehicle201is parked. The vehicle security system202using the LiDAR sensor220can use the LiDAR sensor220that is not used while the vehicle201is parked, and it is not necessary to use a separate sensor only for the vehicle security system, which is rational.

Further, a vehicle security sensor2according to the present embodiment is configured to acquire the second measurement data in the second cycle shorter than the first cycle when the fluctuation greater than the predetermined value occurs during the measurement of the first measurement data. That is, in a normal state, the LiDAR sensor220is operated in the first cycle so as to reduce power consumption, and in a case in which any state changes and an abnormality is suspected, the LiDAR sensor220can be operated in the second cycle to perform investigating in detail. Accordingly, it is possible to acquire highly accurate information when necessary while power consumption is reduced.

In the present embodiment, the light source control unit245of the vehicle security system202may operate the LiDAR sensor220while the vehicle201is parked at a resolution lower than a resolution of sensing performed at a time of traveling, or may operate the LiDAR sensor220while the vehicle201is parked at a cycle longer than a cycle of the sensing performed at the time of traveling.

When the LiDAR sensor220is used for security purposes while the vehicle201is parked, a resolution lower than the resolution (the number of measurement points) required at the time of traveling is sufficient. When the LiDAR sensor220is used for the security purposes while the vehicle201is parked, a cycle longer than a cycle (a scan cycle) obtained at the time of traveling is sufficient. By operating the LiDAR sensor220with a low resolution or a long cycle, the power consumed by the LiDAR sensor220can be reduced.

When the image comparison unit252specifies the direction in which the difference is generated in the present embodiment, the image comparing unit252divides the image I [t] acquired at the time t into 12 regions in a circumferential direction around the host vehicle201, and specifies the divided regions. Instead of specifying the direction, the image comparing unit252may specify which region the pixel having the difference belongs to, and may output the region to which the pixel belongs to the lamp control unit253. In this case, the lamp control unit253may turn on the lamp unit230capable of emitting light toward the input region.

In the above-described embodiment, the alarm processing is performed immediately when the difference equal to or greater than the predetermined value is generated between the image [t−1] and the image I [t]. However, when the difference equal to or greater than the predetermined value is generated, it may be determined whether the difference matches, for example, a pattern registered as an action pattern of a suspicious person, and the alarm processing may be performed when the difference matches the pattern.

The alarm processing may be a known method of notifying an owner of the vehicle201such as emitting light and emitting a sound.

In the present embodiment, the vehicle security system202may be capable of identifying the owner of the vehicle201or a registered person (referred to as a user) as an unregistered person. The vehicle security system202may include a user information recording unit254. The user information recording unit254records a specific shape associated with the user. After determining that a difference is present between the image I [t−1] at the time t−1 and the image I [t] at the time t, the image comparing unit252determines whether the difference in the image I [t] at the time t matches the shape recorded in the user information recording unit254. If the difference matches the shape, the lamp control unit253turns on the lamp unit230based on a method different from that in a case in which the difference does not match the shape.

For example, if the difference matches the shape recorded in the user information recording portion254, the lamp unit230may be constantly on toward the corresponding direction, and if the difference does not match the shape recorded in the user information recording portion254, the lamp unit230may be blinked toward the corresponding region. For example, the user information recording unit254can record a shape of a face of the user, a shape of a key holder owned by the user, and the like.

The lamp control unit253differentiating a lighting mode of the lamp unit230depending on whether the shapes match or do not match includes differentiating lighting and blinking of the lamp unit230, differentiating blinking of the lamp unit230, differentiating lighting colors of the lamp unit230, and differentiating brightness of the lamp unit230. When the lamp unit230includes a plurality of light sources, changing the lighting mode of the lamp unit230includes changing the number and a shape of the light sources to be turned on.

The image forming unit251, the image comparing unit252, and the lamp control unit253may be integrated with the signal processing unit247or may be separated from the signal processing unit247. The image forming unit251, the image comparing unit252, and the lamp control unit253may be provided in a middle of a communication path between the signal processing unit247and the vehicle control unit203.

In the above-described embodiment, when the difference is present between the image at the time t−1 and the image at the time t, the image comparing unit252outputs the direction in which the difference is present to the lamp control unit253, and the lamp control unit253emits light toward the corresponding direction. However, the present invention is not limited thereto. In a case in which the LiDAR sensor unit210is provided with a directional speaker, when the difference is present between the image at the time t−1 and the image at the time t, the image comparing unit252may output the direction in which the difference is present to a sound source control unit, and the sound source control unit may cause the directional speaker to emit a sound toward the corresponding direction.

Further, in the above-described embodiment, the LiDAR sensor220provided in the right front portion, the left front portion, the right rear portion, and the left rear portion of the vehicle201has been described. However, the present invention can be applied to all the LiDAR sensors220regardless of a mounting position of the vehicle201.

Further, in the above-described embodiment, an example has been described in which the LiDAR sensor220is provided together with the lamp unit230inside the outer cover12and the housing11. The outer cover12and the housing11are common. However, the present invention is not limited thereto. The LiDAR sensor220may be provided independently of the lamp unit230. The LiDAR sensor220may be provided inside the outer cover12and the housing11that are common to other sensors such as a camera and a millimeter wave radar.

The above-described embodiments are merely examples for facilitating an understanding of the present invention. The configurations according to the above-described embodiments can be appropriately modified and improved without departing from the spirit of the present invention.

In the above-described embodiment, the LiDAR sensor unit provided in the right front portion, the left front portion, the right rear portion, and the left rear portion of the vehicle1has been described. However, the present invention can be applied to all the LiDAR sensor units regardless of a mounting position of the vehicle.

Further, in the above-described embodiment, an example has been described in which the LiDAR sensor is provided together with the lamp unit inside the outer cover and the housing.

The outer cover and the housing are common. However, the present invention is not limited thereto. The LiDAR sensor may be provided independently of the lamp unit. Alternatively, the LiDAR sensor may be provided inside the outer cover and the housing that are common to other sensors such as a camera and a millimeter wave radar.

The present application is based on a Japanese patent application (Patent Application No. 2019-002788) filed on Jan. 10, 2019, a Japanese patent application (Patent Application No. 2019-002789) filed on Jan. 10, 2019, a Japanese patent application (Patent Application No. 2019-002790) filed on Jan. 10, 2019, and a Japanese patent application (Patent Application No. 2019-002791) filed on Jan. 10, 2019, the contents of which are incorporated herein by reference.

INDUSTRIAL APPLICABILITY

The present invention provides a LiDAR sensor unit in which, even if a lens element is provided, the lens element is less likely to affect measurement data.

REFERENCE SIGNS LIST

1vehicle3vehicle control unit10LiDAR sensor unit11housing12outer cover (lens element)20LiDAR sensor30lamp unit41light emitting unit42light receiving unit43MEMS mirror44converging lens45light source control unit46mirror control unit47signal processing unit (processing unit)48memory (recording unit)101vehicle103vehicle control unit110LiDAR sensor unit111housing112outer cover120LiDAR sensor141light emitting unit142light receiving unit143MEMS mirror144converging lens145light source control unit146mirror control unit147signal processing unit (processing unit)148memory149image forming unit201vehicle202vehicle security system203vehicle control unit210LiDAR sensor unit220LiDAR sensor230lamp unit (light source)241light emitting unit242light receiving unit243MEMS mirror244converging lens245light source control unit246mirror control unit247signal processing unit251image forming unit252image comparing unit253lamp control unit (control unit)254user information recording unitS standard screenθ direction angleφ elevation angled distancedmax maximum valuedmin minimum valuedmean average valuedth fluctuation threshold valueΔd differencedth threshold valueΔd fluctuationA pedestrian