Patent Description:
<CIT> (<CIT>) discloses a signal recognition system. The signal recognition system detects a target traffic light around the vehicle based on an image captured by an in-vehicle camera, and acquires signal detection information. The signal detection information indicates at least the appearance of each of a plurality of detected portions of the target traffic light. Light pattern information indicates the relative positional relationship between or among a plurality of light portions of a traffic light and the appearance of each of the light portions when each light portion is on. The signal recognition system recognizes the lighting state of the target traffic light by comparing the signal detection information and the light pattern information. <CIT> discloses a vehicle headlight for generating multiple light distributions, a method for detecting light signals and a driver assistance device. <CIT> discloses a method and apparatus for controlling headlights of autonomous vehicle. <CIT> discloses a method and apparatus for controlling a lighting system of a vehicle.

A situation in which a vehicle recognizes a traffic light using an in-vehicle camera will be considered. There is a possibility that the accuracy of traffic light recognition may decrease in a dark environment such as at night or in the evening.

For example, there is not only the light of signal indications but also various kinds of light at night. Even if red, yellow, or green light is recognized, it is not always the light of a signal indication. In order to determine whether the recognized light is the light of a signal indication, it is necessary to recognize whether there is a traffic light at the position of the light. One possible way to recognize the presence of a traffic light is to recognize a traffic light housing. However, objects that are not emitting light are hard to see at night, and traffic light housings are also hard to see at night. Accordingly, the probability of correctly recognizing the presence of a traffic light decreases. That is, the accuracy of traffic light recognition decreases at night.

The present invention provides a technique capable of improving the accuracy of traffic light recognition in a dark environment as set out in the appended claims.

A first aspect relates to a vehicle control method comprising: recognizing a signal candidate position around a vehicle by using a sensor mounted on the vehicle, the signal candidate position being a position at which a traffic light is possibly present, the vehicle including a light configured in such a manner that a light distribution state of the light changes; and performing a first light control process of controlling the light in response to recognition of the signal candidate position, the first light control process including at least one of: increasing an intensity of light emitted to the signal candidate position to a value higher than before the recognition of the signal candidate position; or making the intensity of the light emitted to the signal candidate position higher than an intensity of light emitted to an area other than the signal candidate position.

A second aspect relates to a vehicle control system comprising: one or more processors configured to perform, a process of recognizing a signal candidate position around a vehicle by using a sensor mounted on the vehicle, the signal candidate position being a position at which a traffic light is possibly present, the vehicle including a light configured in such a manner that a light distribution state of the light changes; and a first light control process of controlling the light in response to recognition of the signal candidate position, the first light control process including at least one of: increasing an intensity of light emitted to the signal candidate position to a value higher than before the recognition of the signal candidate position; or making the intensity of the light emitted to the signal candidate position higher than an intensity of light emitted to an area other than the signal candidate position.

A third aspect relates to a non-transitory storage medium storing instructions that are executable by one or more processors and that cause the one or more processors to perform functions comprising: recognizing a signal candidate position around a vehicle by using a sensor mounted on the vehicle, the signal candidate position being a position at which a traffic light is possibly present, the vehicle including a light configured in such a manner that a light distribution state of the light changes; and performing a first light control process of controlling the light in response to recognition of the signal candidate position, the first light control process including at least one of: increasing an intensity of light emitted to the signal candidate position to a value higher than before the recognition of the signal candidate position; or making the intensity of the light emitted to the signal candidate position higher than an intensity of light emitted to an area other than the signal candidate position.

According to the present invention, a signal candidate position around the vehicle where there is a possibility that a traffic light is present is recognized. The first light control process is performed in response to the recognition of the signal candidate position. The first light control process includes either or both of: increasing the intensity of the light emitted to the signal candidate position to a value higher than before the recognition of the signal candidate position; and making the intensity of the light emitted to the signal candidate position higher than the intensity of the light emitted to the area other than the signal candidate position. The first light control process makes the signal candidate position more visible even in a dark environment, and increases the probability of recognizing a traffic light in a dark environment. That is, the accuracy of traffic light recognition is improved.

An embodiment of the present invention will be described with reference to the accompanying drawings.

<FIG> is a conceptual diagram illustrating an overview of a vehicle control system <NUM> applied to a vehicle <NUM> according to the present embodiment. The vehicle control system <NUM> controls the vehicle <NUM>. The vehicle control system <NUM> is typically mounted on the vehicle <NUM>. Alternatively, at least a part of the vehicle control system <NUM> may be included in a remote system external to the vehicle <NUM> to remotely control the vehicle <NUM>. That is, the vehicle control system <NUM> may be dispersedly located in the vehicle <NUM> and the remote system.

The vehicle <NUM> may be an autonomous driving vehicle. In that case, the vehicle control system <NUM> performs autonomous driving control of the vehicle <NUM>. For example, the autonomous driving level is assumed to be such a level that the driver does not necessarily have to focus <NUM>% on driving. The vehicle <NUM> may be a driverless autonomous driving vehicle.

A situation in which the vehicle control system <NUM> recognizes a traffic light around the vehicle <NUM> will be considered. For example, the vehicle control system <NUM> recognizes a traffic light ahead of the vehicle <NUM> and its signal indication (red, green, yellow, arrow, etc.), and performs autonomous driving control based on the recognition result. As another example, the vehicle control system <NUM> may notify the driver of the recognition result of the traffic light or signal indication ahead of the vehicle <NUM> to assist the driver in driving. As still another example, the vehicle control system <NUM> may send the recognition result of the traffic light or signal indication to an external management device (e.g., map management device). In any case, it is desired to accurately recognize a traffic light.

The vehicle control system <NUM> recognizes a traffic light around the vehicle <NUM> by using a camera C mounted on the vehicle <NUM>. More specifically, the camera C acquires an image IMG showing the surroundings of the vehicle <NUM>. The vehicle control system <NUM> recognizes a traffic light around the vehicle <NUM> based on the image IMG obtained by the camera C. For example, the vehicle control system <NUM> recognizes a traffic light in the image IMG by using image recognition artificial intelligence (AI) obtained by machine learning.

However, there is a possibility that the accuracy of the traffic light recognition based on the image IMG may decrease in a dark environment such as at night or in the evening. <FIG> is a conceptual diagram illustrating this problem.

For example, there is not only the light of signal indications but also various kinds of light at night. Even if red, yellow, or green light is recognized, it is not always the light of a signal indication. For example, there is red light from taillights of a preceding vehicle. Another example is yellow light from a blinker of a preceding vehicle. Yet another example is green or yellow light from a rooflight (top light) of a taxi. In a situation where there is light similar to the light of a signal indication as described above, it is not preferable to determine that the recognized light is the light of a signal indication based only on the light recognition result. For example, during autonomous driving control, it may turn out that there is actually no red signal after the vehicle control system <NUM> determined that red light was a red traffic signal and automatically slowed down the vehicle <NUM>. This means a decrease in accuracy of autonomous driving control, which is not preferable.

In order to determine whether the recognized light is the light of a signal indication, it is necessary to recognize whether there is a traffic light at the position of the light. One possible way to recognize the presence of a traffic light is to recognize a traffic light housing. Traffic light housings are relatively easy to recognize during the day. However, objects that are not emitting light are hard to see at night, and traffic light housings are also hard to see at night. Accordingly, the probability of correctly recognizing (detecting) the presence of a traffic light decreases. That is, the accuracy of traffic light recognition decreases at night.

As another example, an arrow signal that accompanies a main signal will be considered. There are cases where a main signal indication is "red (prohibit from proceeding)" but an arrow signal indication is "green (permit to proceed). " However, since the amount of light of an arrow signal is smaller than that of a main signal, the arrow signal indication may not be accurately recognized until the vehicle <NUM> gets somewhat close to the traffic light. Automatically slowing down the vehicle <NUM> upon mere recognition of red light when an arrow signal indication is not recognized leads to a decrease in accuracy of autonomous driving control. In order to prevent such hasty autonomous driving control, it is desirable to first recognize whether there is an arrow signal. However, arrow signal housings are also hard to see at night. Accordingly, the probability of correctly recognizing the presence of an arrow signal decreases. That is, the accuracy of traffic light recognition decreases at night.

There is a similar problem with "shielded signals" that their indications cannot be seen until the vehicle gets somewhat close.

The above problem becomes more pronounced as the distance to the traffic light is greater. On the other hand, in order to implement deliberate autonomous driving control, it is also required to accurately recognize a traffic light from as far as about <NUM> away. It is preferable to improve the accuracy of traffic light recognition as much as possible even at positions far from the traffic light.

Improving the accuracy of traffic light recognition that is based on the image IMG is also useful when the vehicle <NUM> is remotely assisted. When the vehicle <NUM> is remotely assisted, the image IMG obtained by the camera C is sent to a remote operator terminal of a remote operator. The remote operator terminal is equipped with a display device, and displays the received image IMG on the display device. The remote operator checks the surroundings of the vehicle <NUM> by looking at the image IMG displayed on the display device, and remotely assists the vehicle <NUM> in operation. Remote assistance by the remote operator includes, for example, recognition assistance, decision assistance, and remote driving. In any case, when a traffic light in the image IMG is less visible, the accuracy of remote assistance by the remote operator may decrease. Therefore, making a traffic light in the image IMG more visible is also preferable in view of the accuracy of remote assistance.

In view of the above, the present embodiment proposes a technique capable of improving the accuracy of traffic light recognition in a dark environment such as at night or in the evening. Specifically, the vehicle control system <NUM> according to the present embodiment uses a light <NUM> mounted on the vehicle <NUM> in order to improve the accuracy of traffic light recognition.

<FIG> is a conceptual diagram illustrating the light <NUM> mounted on the vehicle <NUM>. The light <NUM> emits light to the outside of the vehicle <NUM>. The light distribution state of the light <NUM> can be changed. That is, the light <NUM> is configured so that its irradiation range and radiation intensity can be changed as desired.

More specifically, the set maximum irradiation range of the light <NUM> is wide, and the irradiation range can be set as desired within the maximum irradiation range. That is, the light <NUM> is configured to be able to selectively illuminate only a part of the maximum irradiation range. For example, the light <NUM> includes a plurality of light sources. For example, when the light sources are light-emitting diodes (LEDs), the light <NUM> includes an LED array composed of a plurality of LEDs. The light sources can be independently controlled on and off. By independently controlling the light sources on and off, the irradiation range can be set as desired within the maximum irradiation range. As shown in <FIG>, the irradiation range can be changed in both the horizontal direction (XY direction) and the vertical direction (Z direction).

The radiation intensity (output power) of each of the light sources can be changed as desired. In order to reduce power consumption, the default value of the radiation intensity of each light source is typically set to a value less than the maximum radiation intensity. However, it is possible to temporarily increase or decrease the radiation intensity of each light source as needed.

For example, the light <NUM> is incorporated in a headlight of the vehicle <NUM>. Particularly, the light <NUM> in this example is a high beam (a low beam is provided separately). A high beam can illuminate positions as high as traffic lights. A high beam can also illuminate at least up to about <NUM> ahead of the vehicle <NUM>. For example, the light <NUM> can be an adaptive high-beam system (AHS) mounted on recent vehicles.

As shown in <FIG>, the vehicle <NUM> may include a right light 50R and a left light <NUM>. It is also possible to "collect light" by superimposing the light emitted from the right light 50R and the light emitted from the left light <NUM>. The radiation intensity in a part of the irradiation range can be increased by collecting light.

The vehicle control system <NUM> according to the present embodiment performs a "light control process" of controlling the light <NUM> mounted on the vehicle <NUM>. Particularly, the vehicle control system <NUM> performs the light control process such that the accuracy of traffic light recognition in a dark environment is improved. The light control process that is performed by the vehicle control system <NUM> according to the present embodiment will be described in more detail below.

First, a "signal candidate position SC" used in the light control process will be described with reference to <FIG>. The signal candidate position SC refers to a position around the vehicle <NUM> where there may be a traffic light. As used herein, the term "position" is a concept including "area" and "space. " The "position" may be a position in real space or a position in the image IMG. The size of the signal candidate position SC is finite. The size of a single signal candidate position SC in the image IMG is smaller than the size of the entire image IMG.

The vehicle control system <NUM> recognizes (acquires) a signal candidate position SC around the vehicle <NUM> by using a sensor group <NUM> mounted on the vehicle <NUM>.

For example, the sensor group <NUM> includes a recognition sensor <NUM> that recognizes the surroundings of the vehicle <NUM>. Examples of the recognition sensor <NUM> include the camera C and a laser imaging detection and ranging (LIDAR). The vehicle control system <NUM> recognizes a signal candidate position SC based on the recognition result from the recognition sensor <NUM>. At this time, evidence is not necessarily required that there is a traffic light at the signal candidate position SC. When light, an object, etc. associated with a traffic light is recognized, its recognized position may be acquired as a signal candidate position SC.

As another example, the sensor group <NUM> includes a position sensor <NUM> that acquires the position of the vehicle <NUM>. An example of the position sensor <NUM> is a Global Positioning System (GPS) sensor. Traffic lights are likely to be installed at intersections. Therefore, the vehicle control system <NUM> may acquire the position of an intersection around the vehicle <NUM> based on position information of the vehicle <NUM> and map information, and recognize the position of the intersection as a signal candidate position SC.

There are also various other methods to recognize a signal candidate position SC. Various examples of the method for recognizing a signal candidate position SC will be described later.

The vehicle control system <NUM> recognizes (acquires) one or more signal candidate positions SC around the vehicle <NUM>. There are cases where a plurality of signal candidate positions SC is recognized at the same time. The vehicle control system <NUM> performs the light control process in response to recognition of the signal candidate position(s) SC. Hereinafter, various examples of the light control process will be described.

<FIG> is a conceptual diagram illustrating a "brightening process" that is an example of the light control process. The brightening process is a process of increasing the intensity of the light (output of the light <NUM>) emitted to a signal candidate position SC in response to a certain trigger. In this example, the trigger is recognition of a signal candidate position SC. That is, in response to recognition of a signal candidate position SC, the vehicle control system <NUM> increases the intensity of the light emitted to the signal candidate position SC to a value higher than before the recognition of the signal candidate position SC.

In the example shown in <FIG>, the intensity of the light emitted to the signal candidate position SC increases at time ts. Time ts is when the signal candidate position SC is recognized, or immediately after the recognition of the signal candidate position SC. The intensity of the light emitted to the area other than the signal candidate position SC remains unchanged. Typically, the intensity of the light emitted to the signal candidate positions SC becomes higher than that of the light emitted to the area other than the signal candidate positions SC after time ts.

For example, the light <NUM> (high beam) is off before time ts. Thereafter, in response to recognition of a signal candidate position SC, the vehicle control system <NUM> partially turns on the light <NUM> so as to illuminate only the recognized signal candidate position SC. That is, the vehicle control system <NUM> selectively illuminates the signal candidate position SC without illuminating the area other than the signal candidate position SC. Such a light control process can also be called "selective lighting.

As another example, the light <NUM> may be on before time ts. Typically, the radiation intensity before time ts is the default value. Thereafter, in response to recognition of a signal candidate position SC, the vehicle control system <NUM> may increase the radiation intensity for the recognized signal candidate position SC to a value higher than the default value.

If an object such as a traffic light is irradiated more intensely than necessary, the object may rather become less visible due to factors such as reflected light and flare. Therefore, the radiation intensity for the signal candidate position SC does not necessarily have to be increased to the maximum radiation intensity. An appropriate target radiation intensity may be calculated in advance through simulations, experiments, etc. In that case, the vehicle control system <NUM> performs the light control process so that the radiation intensity for the signal candidate position SC becomes the appropriate target radiation intensity.

The brightening process described above provides the following effects.

First, the intensity of the light emitted to the signal candidate position SC is increased, so that the brightness (illuminance) of the signal candidate position SC increases. Therefore, when there is actually a traffic light at the signal candidate position SC, the brightness of the housing of that traffic light also increases, and the housing of that traffic light becomes more visible. Therefore, it becomes easier to recognize (detect) the presence of a traffic light based on the image IMG even in a dark environment. As a result, the probability of recognizing (detecting) a traffic light based on the image IMG increases. That is, the accuracy of traffic light recognition is improved.

A high beam can illuminate at least up to about <NUM> ahead of the vehicle <NUM>. Therefore, even when the vehicle <NUM> is about <NUM> away from a traffic light, the traffic light is brightened, so that visibility of the traffic light can be improved. That is, it is possible to increase the accuracy of traffic light recognition even at positions far from the traffic light. This is preferable in terms of implementing deliberate autonomous driving control.

Moreover, when the light emitted to the signal candidate position SC becomes more intense than the light emitted to the area other than the signal candidate position SC, the contrast of the image IMG increases. The signal candidate position SC can be made more visible by removing noise light at positions other than the signal candidate position SC from the image IMG through a filter etc. This also contributes to an increase in probability of recognizing (detecting) a traffic light based on the image IMG. Since the noise light is removed, the noise light is less likely to be misrecognized as a traffic light. That is, the accuracy of traffic light recognition is improved.

Improvement in accuracy of traffic light recognition contributes to improvement in accuracy of autonomous driving control that is based on the result of traffic light recognition.

Brightening the signal candidate position SC is also useful for remote assistance by the remote operator. In the remote assistance, the image IMG obtained by the camera C is sent to the remote operator terminal of the remote operator. The remote operator checks the surroundings of the vehicle <NUM> by looking at the image IMG. At this time, if the signal candidate position SC in the image IMG is bright, the remote operator can easily recognize whether there is a traffic light. Moreover, the remote operator can easily recognize whether there is an arrow signal or a shielded signal, even when a signal indication is not visible. These are preferable in terms of the accuracy of remote assistance.

<FIG> is a conceptual diagram illustrating a "dimming process" that is another example of the light control process. The dimming process is a process of reducing the intensity of the light (output of the light <NUM>) emitted to the area other than a signal candidate position SC in response to a certain trigger. In this example, the trigger is recognition of a signal candidate position SC. That is, in response to recognition of a signal candidate position SC, the vehicle control system <NUM> reduces the intensity of the light emitted to the area other than the signal candidate position SC to a value lower than before the recognition of the signal candidate position SC.

In the example shown in <FIG>, the light <NUM> (high beam) is on before time ts. Typically, the radiation intensity before time ts is the default value. The intensity of the light emitted to the area other than the signal candidate position SC then decreases at time ts. The light <NUM> may be partially turned off so that the area other than the signal candidate position SC will not be illuminated. The intensity of the light emitted to the signal candidate position SC remains unchanged. As a result, as in the example shown in <FIG>, the light emitted to the signal candidate position SC becomes more intense than the light emitted to the area other than the signal candidate position SC.

This dimming process provides the following effects.

Since the light emitted to the signal candidate position SC becomes more intense than the light emitted to the area other than the signal candidate position SC, the contrast of the image IMG increases. The signal candidate position SC can be made more visible by removing noise light at positions other than the signal candidate position SC from the image IMG through a filter etc. This also contributes to an increase in probability of recognizing (detecting) a traffic light based on the image IMG. Since the noise light is removed, the noise light is less likely to be misrecognized as a traffic light. That is, the accuracy of traffic light recognition is improved.

Reflectors are often attached to roads or roadside structures. Reflected light from such reflectors becomes noise light for signal recognition. Dimming the light emitted to the area other than the signal candidate position SC can reduce generation of noise light. The accuracy of traffic light recognition is thus improved.

Moreover, the light emitted to the area other than the signal candidate position SC becomes less intense, and the brightness (illuminance) of the positions other than the signal candidate position SC decreases. Therefore, objects at positions other than the signal candidate position SC become less visible. As a result, the probability of misrecognizing an object other than a traffic light as a traffic light is reduced. This also contributes to improvement in accuracy of traffic light recognition.

Moreover, partially dimming the emitted light can reduce power consumption.

<FIG> shows a combination of the above "brightening process" and the "dimming process. " In response to recognition of a signal candidate position SC, the vehicle control system <NUM> increases the intensity of the light emitted to the signal candidate position SC to a value higher than before the recognition of the signal candidate position SC, and reduces the intensity of the light emitted to the area other than the signal candidate position SC to a value lower than before the recognition of the signal candidate position SC. This combination provides both the effects of the brightening process and the effects of the dimming process.

<FIG> is a conceptual diagram illustrating an "attenuation process" following the brightening process. As described above, the brightening process increases the intensity of the light emitted to the signal candidate position SC and thus increases the illuminance of the signal candidate position SC. Thereafter, the vehicle <NUM> travels and approaches the signal candidate position SC. When the intensity of the light output from the light <NUM> of the vehicle <NUM> does not change, the illuminance of the signal candidate position SC increases as the vehicle <NUM> approaches the signal candidate position SC. However, if an object such as a traffic light is irradiated more intensely than necessary, the object may rather become less visible due to factors such as reflected light and flare.

Therefore, in order to prevent the signal candidate position SC from being irradiated more intensely than necessary, the vehicle control system <NUM> may perform the attenuation process following the brightening process. Specifically, after the brightening process, the vehicle control system <NUM> attenuates the light emitted to the signal candidate position SC as the vehicle <NUM> travels.

The attenuation rate of the emitted light intensity in the attenuation process may be dynamically set according to the speed of the vehicle <NUM>. In this case, the attenuation rate of the emitted light intensity is set to increase as the speed of the vehicle <NUM> increases.

The emitted light intensity in the attenuation process may be set according to the distance from the vehicle <NUM> to the signal candidate position SC. In this case, the emitted light intensity is set to decrease as the distance from the vehicle <NUM> to the signal candidate position SC decreases.

The emitted light intensity of the light <NUM> on the output side may be set so that the illuminance of the signal candidate position SC on the light-receiving side has a constant value during the attenuation process. The constant value is the illuminance that allows appropriate signal recognition.

By the attenuation process described above, the signal candidate position SC is less likely to be irradiated more intensely than necessary after the brightening process. As a result, the possibility of an object such as a traffic light becoming less visible is reduced. This also contributes to improvement in accuracy of traffic light recognition.

As described above, according to the present embodiment, a signal candidate position SC around the vehicle <NUM> where there may be a traffic light is recognized. The light control process described above is performed in response to the recognition of the signal candidate position SC. The light control process includes either or both of: increasing the intensity of the light emitted to the signal candidate position SC to a value higher than before the recognition of the signal candidate position SC; and making the intensity of the light emitted to the signal candidate position SC higher than that of the light emitted to the area other than the signal candidate position SC. This light control process makes the signal candidate position more visible even in a dark environment, and increases the probability of recognizing (detecting) a traffic light in a dark environment. That is, the accuracy of traffic light recognition is improved.

As described above, the light control process is performed in response to recognition of a signal candidate position SC. This light control process is hereinafter referred to as "first light control process" for convenience. The first light control process increases the accuracy of traffic light recognition that is based on the image IMG. As a result, in some cases, it turns out that there is actually a traffic light at the signal candidate position SC, or that the possibility that there is a traffic light at the signal candidate position SC is very high. In other cases, it turns out that there is no traffic light at the signal candidate position SC, or that the possibility that there is a traffic light at the signal candidate position SC is very low. It is therefore possible to further narrow down the signal candidate position(s) SC after the first light control process. In other words, it is possible to screen the signal candidate position(s) SC.

<FIG> is a conceptual diagram illustrating the screening process. The image IMG obtained by the camera C includes one or more signal candidate positions SC. After the first light control process, the vehicle control system <NUM> further performs signal recognition based on the image IMG to narrow down the signal candidate position(s) SC. At this time, the vehicle control system <NUM> may narrow down the "number" of the signal candidate positions SC, or may narrow down the "size" of the signal candidate position SC.

For example, when the vehicle control system <NUM> determines that there is no traffic light at a certain signal candidate position SC or that the possibility that there is a traffic light at a certain signal candidate position SC is less than a threshold, the vehicle control system <NUM> excludes that signal candidate position SC from signal candidate positions SC for the subsequent processes. As another example, when the vehicle control system <NUM> determines that there is no traffic light in at least a part of the range of a certain signal candidate position SC, or that the possibility that there is a traffic light in at least a part of the range of a certain signal candidate position SC is less than a threshold, the vehicle control system <NUM> excludes this part of the range of the signal candidate position SC from the signal candidate position SC.

The position excluded from the signal candidate position(s) SC by the screening process is hereinafter referred to as "excluded position EX. " The possibility that there is a traffic light at the excluded position EX is low. On the other hand, the possibility that there is a traffic light at the signal candidate position(s) SC after the screening process is high.

By narrowing down the signal candidate position(s) SC to the signal candidate position(s) SC where a traffic light is highly likely to present, a traffic light can be more accurately and more quickly recognized. That is, the screening process can increase both the accuracy of traffic light recognition and the speed of traffic light recognition.

The vehicle control system <NUM> further performs the light control process following the screening process. The light control process performed after the screening process is hereinafter referred to as "second light control process" for convenience. The second light control process is triggered by execution of the screening process.

For example, there is no need to keep illuminating the excluded position EX excluded from the signal candidate position(s) SC. Continuing to illuminate the excluded position EX may cause generation of unnecessary noise light and misrecognition of an object other than a traffic light as a traffic light. Therefore, in the second light control process, the vehicle control system <NUM> performs a dimming process (see <FIG> and <FIG>) on the excluded position EX. That is, the vehicle control system <NUM> reduces the intensity of the light emitted to the excluded position EX to a value lower than before the screening process. This dimming process provides the effects described above, and further improves the accuracy of traffic light recognition.

As another example, in the second light control process, the vehicle control system <NUM> may further perform a brightening process (see <FIG> and <FIG>) on the signal candidate position(s) SC. That is, the vehicle control system <NUM> may increase the intensity of the light emitted to the signal candidate position SC to a value even higher than before the screening process. At this time, the vehicle control system <NUM> may "collect light" using the right light 50R and the left light <NUM> shown in <FIG> to increase the intensity of the light emitted to the signal candidate position SC. This brightening process provides the effects described above, and further improves the accuracy of traffic light recognition.

The vehicle control system <NUM> may further perform an attenuation process (see <FIG>) following the brightening process. By this attenuation process, the signal candidate position(s) SC is less likely to be irradiated more intensely than necessary after the brightening process. As a result, the possibility of an object such as a traffic light becoming less visible is reduced. This also contributes to improvement in accuracy of traffic light recognition.

<FIG> is a conceptual diagram illustrating an example of a traffic light that is an irradiation target in the present embodiment. A first lane L1 is a lane in which the vehicle <NUM> is traveling. A first traffic light SG1 is a traffic light for the first lane L1. Particularly, the first traffic light SG1 is located ahead of the vehicle <NUM>. The vehicle <NUM> travels according to the signal indication of the first traffic light SG1. At least the first traffic light SG1 is an irradiation target. The signal candidate position SC includes a position where there may be the first traffic light SG1.

<FIG> is a conceptual diagram illustrating another example of traffic lights that are irradiation targets according to the present embodiment. A second lane L2 is a lane that crosses the first lane L1. There are other vehicles <NUM> in the second lane L2. Second traffic lights SG2 are traffic lights for the second lane L2. The other vehicles <NUM> travel according to the signal indications of the second traffic lights SG2. The second traffic lights SG2 may also be irradiation targets. That is, the signal candidate positions SC may include not only a position where there may be the first traffic light SG1, but also a position where there may be the second traffic light SG2. The vehicle control system <NUM> of the vehicle <NUM> in the first lane L1 also illuminates the second traffic lights SG2 for the second lane L2. This makes it easier for the other vehicles <NUM> in the second lane L2 to recognize the second traffic lights SG2. That is, the vehicle control system <NUM> of the vehicle <NUM> can assist the other vehicles <NUM> in signal recognition.

<FIG> is a block diagram showing an example of the configuration of the vehicle control system <NUM> according to the present embodiment. The vehicle control system <NUM> includes the sensor group <NUM>, a driving device <NUM>, a communication device <NUM>, the light <NUM>, and a control device <NUM>.

The sensor group <NUM> is mounted on the vehicle <NUM>. The sensor group <NUM> includes the recognition sensor <NUM>, a vehicle state sensor <NUM>, the position sensor <NUM>, etc..

The recognition sensor <NUM> recognizes (detects) the surroundings of the vehicle <NUM>. The recognition sensor <NUM> includes the camera C. The recognition sensor <NUM> may include a laser imaging detection and ranging (LIDAR), a radar, etc..

The vehicle state sensor <NUM> detects the state of the vehicle <NUM>. The vehicle state sensor <NUM> includes, for example, a speed sensor, an acceleration sensor, a yaw rate sensor, and a steering angle sensor.

The position sensor <NUM> detects the position and orientation of the vehicle <NUM>. An example of the position sensor <NUM> is a Global Positioning System (GPS) sensor.

The driving device <NUM> includes a steering device, a drive device, and a braking device. The steering device steers wheels. For example, the steering device includes an electric power steering (EPS) system. The drive device is a power source that generates a driving force. Examples of the drive device include an engine, an electric motor, and in-wheel motors. The braking device generates a braking force.

The communication device <NUM> communicates with the outside of the vehicle <NUM>. For example, the communication device <NUM> communicates with a management device external to the vehicle <NUM>. Examples of the management device include a map management device that manages map information and an autonomous driving management device that manages autonomous driving of the vehicle <NUM>. As still another example, the communication device <NUM> may communicate with the remote operator terminal that provides remote assistance of the vehicle <NUM>.

The light <NUM> is mounted on the vehicle <NUM>, and emits light to the outside of the vehicle <NUM>. The light distribution state of the light <NUM> can be changed. That is, the light <NUM> is configured so that its irradiation range and radiation intensity can be changed as desired. For example, the light <NUM> includes a plurality of light sources. When the light sources are LEDs, the light <NUM> includes an LED array composed of a plurality of LEDs. The light sources can be independently controlled.

The control device <NUM> controls the vehicle <NUM>. The control device <NUM> includes one or more processors <NUM> (hereinafter simply referred to as "processor <NUM>") and one or more storage devices <NUM> (hereinafter simply referred to as "storage device <NUM>"). The processor <NUM> performs various processes. For example, the processor <NUM> includes a central processing unit (CPU). The storage device <NUM> stores various kinds of information. Examples of the storage device <NUM> include a volatile memory, a nonvolatile memory, a hard disk drive (HDD), and a solid state drive (SSD). The control device <NUM> may include one or more electronic control units (ECUs). A part of the control device <NUM> may be an information processing device external to the vehicle <NUM>. In that case, the part of the control device <NUM> communicates with the vehicle <NUM> to remotely control the vehicle <NUM>.

A vehicle control program PROG is a computer program for controlling the vehicle <NUM>. The various processes that are performed by the control device <NUM> are implemented by the processor <NUM> executing the vehicle control program PROG. The vehicle control program PROG is stored in the storage device <NUM>. Alternatively, the vehicle control program PROG may be recorded on a computer-readable recording medium.

The control device <NUM> acquires driving environment information <NUM> by using the sensor group <NUM>. The driving environment information <NUM> indicates the driving environment of the vehicle <NUM>. The driving environment information <NUM> is stored in the storage device <NUM>.

<FIG> is a block diagram showing an example of the driving environment information <NUM>. The driving environment information <NUM> includes surroundings information <NUM>, vehicle state information <NUM>, vehicle position information <NUM>, and map information <NUM>.

The surroundings information <NUM> is information indicating the surroundings of the vehicle <NUM>. The control device <NUM> recognizes the surroundings of the vehicle <NUM> using the recognition sensor <NUM>, and acquires the surroundings information <NUM>. For example, the surroundings information <NUM> includes the image IMG captured by the camera C. As another example, the surroundings information <NUM> includes point group information obtained by the LIDAR.

The surroundings information <NUM> further includes object information <NUM> on an object around the vehicle <NUM>. Examples of the object around the vehicle <NUM> include pedestrians, bicycles, motorcycles, other vehicles (e.g., preceding vehicles and parked vehicles), white lines, traffic lights, structures (e.g., utility poles and pedestrian bridges), traffic signs, and obstacles. The object information <NUM> indicates the relative position and relative speed of the object with respect to the vehicle <NUM>. For example, the object can be identified and the relative position of the object can be calculated by analyzing the image IMG obtained by the camera C. For example, the control device <NUM> identifies the object in the image IMG, such as a traffic light, by using the image recognition AI obtained by machine learning. Alternatively, the object can be identified and the relative position and relative speed of the object can be acquired based on the point group information obtained by the LIDAR.

The vehicle state information <NUM> is information indicating the state of the vehicle <NUM>, and includes, for example, vehicle speed, acceleration, yaw rate, and steering angle. The control device <NUM> acquires the vehicle state information <NUM> from the vehicle state sensor <NUM>. The vehicle state information <NUM> may indicate the driving state (manual driving or autonomous driving) of the vehicle <NUM>.

The vehicle position information <NUM> is information indicating the current position of the vehicle <NUM>. The control device <NUM> acquires the vehicle position information <NUM> from the detection result from the position sensor <NUM>. The control device <NUM> may acquire accurate vehicle position information <NUM> by a well-known localization process using the object information <NUM> and the map information <NUM>.

The map information <NUM> includes a common navigation map. The map information <NUM> may indicate lane arrangements and road shapes. The map information <NUM> may include position information of structures, traffic lights, and traffic signs, etc. The control device <NUM> acquires the map information <NUM> of a necessary area from a map database. The map database may be stored in the storage device <NUM>, or may be stored in the map management device external to the vehicle <NUM>. In the latter case, the control device <NUM> communicates with the map management device via the communication device <NUM> to acquire necessary map information <NUM>. The map information <NUM> is stored in the storage device <NUM>.

The control device <NUM> performs "vehicle driving control" for controlling driving of the vehicle <NUM>. The vehicle driving control includes steering control, acceleration control, and deceleration control. The control device <NUM> performs the vehicle driving control by controlling the driving device <NUM>. Specifically, the control device <NUM> performs the steering control by controlling the steering device. The control device <NUM> performs the acceleration control by controlling the drive device. The control device <NUM> performs the deceleration control by controlling the braking device.

The control device <NUM> performs autonomous driving control based on the driving environment information <NUM>. More specifically, the control device <NUM> generates a travel plan for the vehicle <NUM> based on the driving environment information <NUM>. The travel plan includes keeping the current lane, changing lanes, making a right or left turn, avoiding an obstacle, etc. The control device <NUM> also generates a target trajectory that is necessary for the vehicle <NUM> to travel according to the travel plan, based on the driving environment information <NUM>. The target trajectory includes a target position and a target speed. The control device <NUM> then performs the vehicle driving control so that the vehicle <NUM> follows the target trajectory.

The control device <NUM> communicates with the outside of the vehicle <NUM> via the communication device <NUM>. For example, the control device <NUM> communicates with a management device external to the vehicle <NUM> via the communication device <NUM>. Examples of the management device include a map management device that manages map information and an autonomous driving management device that manages autonomous driving of the vehicle <NUM>. The control device <NUM> may perform vehicle-to-vehicle communication with other vehicle via the communication device <NUM>.

When the vehicle <NUM> is an object to be remotely assisted by the remote operator, the control device <NUM> communicates with the remote operator terminal of the remote operator via the communication device <NUM>. Remote assistance by the remote operator includes, for example, recognition assistance, decision assistance, and remote driving. When the control device <NUM> determines that remote assistance by the remote operator is necessary, the control device <NUM> sends a request for assistance to the remote operator terminal. The control device <NUM> also sends at least a part of the driving environment information <NUM> to the remote operator terminal. Particularly, the control device <NUM> sends the image IMG obtained by the camera C to the remote operator terminal. The remote operator terminal displays the received image IMG on the display device. The remote operator checks the surroundings of the vehicle <NUM> by looking at the image IMG displayed on the display device, and remotely assists the vehicle <NUM> in operation. The control device <NUM> receives instruction information indicating instructions of the remote operator from the remote operator terminal. The control device <NUM> then performs the vehicle driving control according to the instruction information.

The control device <NUM> performs the light control process of controlling the light <NUM>. As described above, the light distribution state of the light <NUM> can be changed. That is, the light <NUM> is configured so that its irradiation range and radiation intensity can be changed as desired. The control device <NUM> can change the irradiation range and radiation intensity of the light <NUM> by controlling the light <NUM>.

The control device <NUM> according to the present embodiment performs the light control process so as to improve the accuracy of traffic light recognition in a dark environment. <FIG> is a flowchart of a process related to such a light control process.

In step S110, the control device <NUM> recognizes a signal candidate position SC around the vehicle <NUM> where a traffic light may be present (see <FIG>). The sensor group <NUM> is used for this recognition of a signal candidate position SC. Various examples of the method for recognizing a signal candidate position SC will be described below. Evidence is not necessarily required that there is a traffic light at the signal candidate position SC.

The control device <NUM> acquires the image IMG obtained by the camera C. The control device <NUM> analyzes the image IMG and tentatively recognizes a traffic light in the image IMG. For example, the image recognition AI obtained by machine learning is used. A set of traffic lights at an intersection may be tentatively recognized. The control device <NUM> then sets a signal candidate position SC to the position (fixed range) around the tentatively recognized traffic light. The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated based on the image IMG.

The control device <NUM> acquires the image IMG obtained by the camera C. The control device <NUM> analyzes the image IMG and recognizes a light source of a signal indication color (green, yellow, red) in the image IMG. The control device <NUM> then sets a signal candidate position SC to the position (fixed range) around the light source of the signal indication color. The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated based on the image IMG.

When a traffic light ahead of the vehicle <NUM> is red, there is a possibility that a preceding vehicle is stopped at the traffic light. Taillights of a stopped preceding vehicle are red. The control device <NUM> analyzes the image IMG and recognizes red lights of a preceding vehicle in the image IMG. The control device <NUM> then sets a signal candidate position SC to the position (fixed range) above the recognized red lights. The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated based on the image IMG.

The possibility that traffic lights are installed at intersections is high. Therefore, the control device <NUM> analyzes the image IMG and recognizes an intersection in the image IMG. Alternatively, the control device <NUM> may acquire the position of an intersection around the vehicle <NUM>, based on the vehicle position information <NUM> obtained by the position sensor <NUM> and the map information <NUM>. The control device <NUM> then sets a signal candidate position SC to the position (fixed range) above the intersection. The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated based on the image IMG, or is calculated based on the vehicle position information <NUM> and the map information <NUM>.

Traffic lights are sometimes installed on structures such as utility poles and pedestrian bridges. Structures related to such traffic lights are hereinafter referred to as "traffic light-related structures. " The control device <NUM> recognizes a traffic light-related structure around the vehicle <NUM>, based on the recognition result from the recognition sensor <NUM> (e.g., camera, LIDAR). That is, the control device <NUM> recognizes a traffic light-related structure based on the surroundings information <NUM> (image IMG, object information <NUM>). As another example, the map information <NUM> in which the positions of structures are registered may be prepared. In that case, the control device <NUM> can acquire the position of a traffic light-related structure around the vehicle <NUM>, based on the vehicle position information <NUM> obtained by the position sensor <NUM> and the map information <NUM>. The control device <NUM> then sets a signal candidate position SC to the position (fixed range) around the traffic light-related structure. The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated based on the object information <NUM>, or is calculated based on the vehicle position information <NUM> and the map information <NUM>.

In step S120 after step S110, the control device <NUM> performs the "first light control process" of controlling the light <NUM> based on the signal candidate position SC. The first light control process is as described in Section <NUM> above, and includes either or both of the "brightening process" and the "dimming process" (see <FIG>). The first light control process may include the "attenuation process" following the brightening process (see <FIG>).

The relative position of the signal candidate position SC with respect to the vehicle <NUM> is calculated as described above. The installation position of the light <NUM> on the vehicle <NUM>, the installation direction of the light <NUM>, and the optical axis direction of each light source included in the light <NUM> are known information. Therefore, the control device <NUM> can control the light <NUM> so that the signal candidate position SC is selectively illuminated.

In step S130 after step S120, the control device <NUM> performs the "screening process" of narrowing down the signal candidate position(s) SC (see Section <NUM> above). For example, the control device <NUM> recognizes a traffic light in the image IMG using the image recognition AI. The accuracy of traffic light recognition has been improved as a result of the first light control process in step S <NUM>. Therefore, in this step S130, the control device <NUM> can recognize a traffic light in the image IMG with relatively high accuracy.

When the control device <NUM> determines that there is no traffic light at a certain signal candidate position SC or that the possibility that there is a traffic light at a certain signal candidate position SC is less than the threshold, the control device <NUM> excludes that signal candidate position SC from signal candidate positions SC for the subsequent processes. As another example, when the control device <NUM> determines that there is no traffic light in at least a part of the range of a certain signal candidate position SC, or that the possibility that there is a traffic light in at least a part of the range of a certain signal candidate position SC is less than the threshold, the control device <NUM> excludes this part of the range of the signal candidate position SC from the signal candidate position SC. The position excluded from the signal candidate position(s) SC is an excluded position EX (see <FIG>).

In step S140 after step S130, the control device <NUM> performs the "second light control process" of controlling the light <NUM> based on the signal candidate position(s) SC or the excluded position EX. The second light control process is as described in Section <NUM> above, and includes either or both of the "brightening process" and the "dimming process. " The second light control process may include the "attenuation process" following the brightening process.

<FIG> and <FIG> are conceptual diagrams illustrating switching of irradiation targets at an intersection. It is herein assumed that a plurality of traffic lights is installed in succession at one intersection ahead of the vehicle <NUM>. The traffic lights include a first traffic light SGa and a second traffic light SGb. The second traffic light SGb is farther away from the vehicle <NUM> than the first traffic light SGa. That is, the first traffic light SGa is located closer to the vehicle <NUM> and the second traffic light SGb is located farther away from the vehicle <NUM>, as viewed from the vehicle <NUM>.

The image IMG obtained by the camera C includes both the first traffic light SGa and the second traffic light SGb. The control device <NUM> recognizes both the first traffic light SGa and the second traffic light SGb based on the image IMG. However, the control device <NUM> does not set irradiation targets of the light <NUM> to both the first traffic light SGa and the second traffic light SGb at the same time. The control device <NUM> switches irradiation targets of the light <NUM>.

More specifically, just being able to recognize the signal indication of the first traffic light SGa is enough when the vehicle <NUM> is relatively far from the intersection as shown in <FIG>. There is no need to recognize the signal indication of the second traffic light SGb. Therefore, the control device <NUM> sets the irradiation target to the first traffic light SGa and excludes the second traffic light SGb from the irradiation target. That is, the control device <NUM> controls the light <NUM> so that the first traffic light SGa is illuminated and the second traffic light SGb is not illuminated. This reduces unnecessary reflected light (noise light) from the second traffic light SGb. Power consumption is also reduced.

The irradiation target is then switched from the first traffic light SGa to the second traffic light SGb when the vehicle <NUM> gets somewhat close to the intersection as shown in <FIG>. That is, the control device <NUM> controls the light <NUM> so that the second traffic light SGb is illuminated and the first traffic light SGa is not illuminated. This reduces unnecessary reflected light (noise light) from the first traffic light SGa. Power consumption is also reduced.

<FIG> is a flowchart of a process related to switching of irradiation targets.

In step S200, the control device <NUM> recognizes a traffic light ahead of the vehicle <NUM> by using the camera C. When a plurality of traffic lights installed at the same intersection is recognized (step S200; YES), the process proceeds to step S210.

In step S210, the control device <NUM> sets the irradiation target to the first traffic light SGa closer to the vehicle <NUM>, and excludes the second traffic light SGb farther away from the vehicle <NUM> from the irradiation target. That is, the control device <NUM> controls the light <NUM> so that the first traffic light SGa is illuminated and the second traffic light SGb is not illuminated.

In step S220, the control device <NUM> estimates the time required for the vehicle <NUM> to reach the position of the first traffic light SGa. The position of a stop line (see <FIG>) near the first traffic light SGa may be used instead of the position of the first traffic light SGa. The distance from the vehicle <NUM> to the first traffic light SGa or the stop line can be obtained from the object information <NUM>. The speed and acceleration of the vehicle <NUM> are obtained from the vehicle state information <NUM>. Therefore, the control device <NUM> can calculate the estimated time required for the vehicle <NUM> to reach the position of the first traffic light SGa, based on the object information <NUM> and the vehicle state information <NUM>.

In step S230, the control device <NUM> determines whether a switching condition is satisfied. The switching condition includes at least that "the estimated time is less than a threshold. " The switching condition may further include that "the vehicle <NUM> proceeds straight through the intersection. " Whether the vehicle <NUM> proceeds straight through the intersection can be determined based on, for example, the target trajectory of the vehicle <NUM> in autonomous driving control. When the estimated time is equal to or more than the threshold, the switching condition is not satisfied (step S230; NO). In this case, the process returns to step S210. When the switching condition is satisfied (step S230; YES), the process proceeds to step S240.

Claim 1:
A vehicle control method comprising:
recognizing a signal candidate position (SC) around a vehicle (<NUM>) by using a sensor (<NUM>) mounted on the vehicle (<NUM>), the signal candidate position (SC) being a position at which a traffic light is possibly present, the vehicle (<NUM>) including a light (<NUM>) configured in such a manner that a light distribution state of the light (<NUM>) changes;
performing a first light control process of controlling the light (<NUM>) in response to recognition of the signal candidate position (SC), the first light control process including at least one of:
increasing an intensity of light emitted to the signal candidate position (SC) to a value higher than before the recognition of the signal candidate position (SC); or
making the intensity of the light emitted to the signal candidate position (SC) higher than an intensity of light emitted to an area other than the signal candidate position (SC);
whereby the method further comprises:
acquiring an image (IMG) of surroundings of the vehicle (<NUM>) by using a camera (C) mounted on the vehicle (<NUM>);
performing, after the first light control process, a screening process of narrowing down the signal candidate position (SC) based on the image (IMG);
performing a second light control process of controlling the light (<NUM>) after the screening process;
wherein the method is characterized in that
an excluded position (EX) is a position excluded from the signal candidate position (SC) by the screening process; and
the second light control process includes reducing an intensity of light emitted to the excluded position (EX) to a value lower than before the screening process.