Patent Description:
<CIT> (<CIT>) discloses a technology relating to remote control on an autonomous driving vehicle. According to this technology, when autonomous driving is difficult, communication is performed between the vehicle and a remote control management facility, and the vehicle is remotely driven by a remote operator. <CIT> discloses systems and methods for determining path confidence for unmanned vehicles, and <CIT> discloses a system, vehicle, network, component, apparatuses, methods and computer programs for a transportation vehicle and a network component. <CIT> shows a remote control request system according to the preamble of claim <NUM>.

When autonomous driving is difficult, it is not always necessary to switch to remote driving. The autonomous driving may be continued by removing the cause of the difficulty. In addition, the number of remote operators who can handle remote driving is limited. Therefore, when the remote driving is always requested every time the autonomous driving is difficult, a great burden is placed on the remote operator.

The present disclosure provides a technology capable of reducing the burden on the remote operator while continuing the operation of the autonomous driving vehicle when the autonomous driving of the autonomous driving vehicle is difficult.

A first aspect of the present disclosure relates to a remote control request system including a remote control request device. The remote control request device includes a memory storing a program, and a processor connected to the memory. The processor is configured to request a remote operator to perform remote control on an autonomous driving vehicle when the autonomous driving vehicle currently has or is expected to have difficulty in continuing autonomous driving. The remote control has or includes remote assistance in which the remote operator makes at least a part of determinations for the autonomous driving, and remote driving in which the remote operator performs at least one of a steering operation and an acceleration or deceleration operation of the autonomous driving vehicle. The processor is configured to, during execution of the program, request the remote assistance inside an autonomous driving domain in which the autonomous driving is continuable by receiving the remote assistance (that is, inside the autonomous driving domain, the remote assistance makes it possible for the vehicle to be driven in autonomous driving mode), and request the remote driving outside the autonomous driving domain.

In some embodiments, making at least a part of determinations for the autonomous driving consists in making at least (and in some cases, in making only) sporadic determinations for the autonomous driving.

In the first aspect, the processor may be configured to request the remote assistance without requesting the remote driving inside the autonomous driving domain and a remote driving domain in which the remote driving is performable. That is, in an area where both the remote assistance and the remote driving are performable, the remote assistance may have priority over the remote driving.

In the first aspect, the processor may be configured to determine whether the autonomous driving vehicle has difficulty in continuing the autonomous driving up to a predetermined period ahead or a predetermined distance ahead based on a route generated at least up to the predetermined period ahead or the predetermined distance ahead.

In the first aspect, the autonomous driving domain may include a preset domain.

In the first aspect, the autonomous driving domain may include a domain that changes with time.

In the first aspect, the processor may be configured to execute inside or outside determination on the autonomous driving domain (that is, to determine whether the vehicle is inside or outside the autonomous driving domain) based on external information and internal information acquired by the autonomous driving vehicle.

In the first aspect, the processor may be configured to request the remote driving outside the autonomous driving domain and inside a remote driving domain in which the remote driving is performable, and stop the autonomous driving vehicle or change a trajectory of the autonomous driving vehicle outside the autonomous driving domain and the remote driving domain.

In the first aspect, the remote driving domain may include a preset domain.

In the first aspect, the remote driving domain may include a domain that changes with time.

In the first aspect, the processor may be configured to execute inside or outside determination on the remote driving domain based on external information and internal information acquired by the autonomous driving vehicle.

In the first aspect, the remote control request system may include a server configured to connect to the remote control request device via a communication network. The remote control request device may be mounted on the autonomous driving vehicle. The remote control request device may be configured to cooperate with the server to request the remote assistance inside the autonomous driving domain in which the autonomous driving is continuable by receiving the remote assistance. The remote control request device may be configured to cooperate with the server to request the remote driving outside the autonomous driving domain.

In the first aspect, the remote control request device may be configured to transmit, to the server, first information held by the autonomous driving vehicle for inside or outside determination on the autonomous driving domain when the autonomous driving vehicle currently has or is expected to have difficulty in continuing the autonomous driving. The server may be configured to execute the inside or outside determination on the autonomous driving domain based on the first information transmitted from the remote control request device for the inside or outside determination on the autonomous driving domain and second information that affects the autonomous driving and is acquired by the server, and determine whether to request the remote assistance or the remote driving.

A second aspect of the present disclosure relates to a remote control request method as defined in claim <NUM>. The remote control request method includes requesting, by a computer, a remote operator to perform remote control on an autonomous driving vehicle when the autonomous driving vehicle currently has or is expected to have difficulty in continuing autonomous driving, requesting remote assistance in which the remote operator makes at least a part of determinations for the autonomous driving inside an autonomous driving domain, the autonomous driving being continuable by receiving the remote assistance in the autonomous driving domain, and requesting, outside the autonomous driving domain, remote driving in which the remote operator performs at least one of a steering operation and an acceleration or deceleration operation of the autonomous driving vehicle. The remote control has or includes the remote assistance and the remote driving.

A third aspect of the present disclosure relates to a non-transitory storage medium storing a program that is executable by one or more processors and that causes the one or more processors to perform functions, as defined in claim <NUM>. The functions include requesting a remote operator to perform remote control on an autonomous driving vehicle when the autonomous driving vehicle currently has or is expected to have difficulty in continuing autonomous driving, requesting remote assistance in which the remote operator makes at least a part of determinations for the autonomous driving inside an autonomous driving domain, the autonomous driving being continuable by receiving the remote assistance in the autonomous driving domain, and requesting, outside the autonomous driving domain, remote driving in which the remote operator performs at least one of a steering operation and an acceleration or deceleration operation of the autonomous driving vehicle. The remote control has or includes the remote assistance and the remote driving.

According to the first aspect, the second aspect, and the third aspect of the present disclosure, when the autonomous driving vehicle currently has or is expected to have difficulty in continuing the autonomous driving, the remote operator is requested to perform the remote assistance inside the autonomous driving domain in which the autonomous driving is continuable by receiving the remote assistance. According to the remote assistance, the remote operator makes at least a part of the determinations for the autonomous driving. Therefore, it is possible to continue the operation of the autonomous driving vehicle by the autonomous driving. The remote operator is requested to perform the remote driving outside the autonomous driving domain. According to the remote driving, the remote operator performs at least one of the steering operation and the acceleration or deceleration operation of the autonomous driving vehicle. Therefore, it is possible to continue the operation of the autonomous driving vehicle even in a situation in which the autonomous driving is difficult to continue. By selectively using the remote assistance and the remote driving as appropriate depending on situations when the autonomous driving vehicle has difficulty in the autonomous driving, the burden on the remote operator can be reduced while continuing the operation of the autonomous driving vehicle.

Embodiments of the present disclosure will be described with reference to the drawings. Numerical values for each element that are mentioned in the following embodiments, such as the number, quantity, amount, and range of each element, are not intended to limit the technical ideas of the present disclosure to the mentioned numerical values unless otherwise specified or unless it is theoretically obvious that the technical ideas of the present disclosure are limited to the mentioned numerical values. Structures etc. that are described in the following embodiments are not necessarily essential to the technical ideas of the present disclosure unless otherwise specified or unless it is theoretically obvious that the structures etc. are essential to the technical ideas of the present disclosure.

<FIG> is a diagram schematically showing the configuration of a remote control system common to all the embodiments described later. A remote control system <NUM> is a system for remotely controlling autonomous driving vehicles <NUM> from a remote control center <NUM>. Examples of autonomous driving levels of the autonomous driving vehicles <NUM> include Level <NUM>, Level <NUM>, and Level <NUM>. The autonomous driving vehicle <NUM> is hereinafter referred to simply as "vehicle <NUM>".

Remote control in the present disclosure includes remote assistance and remote driving. The remote assistance and the remote driving are performed when the vehicle <NUM> currently has or is expected to have difficulty in continuing autonomous driving. Examples of the case of difficulty in continuing autonomous driving include the following cases.

In the above cases, the vehicle <NUM> may continue to stop on the spot or may fail to reach a destination. Processes performed to avoid such situations are the remote assistance and the remote driving, which are performed by remote operators <NUM> and <NUM>. The remote operator <NUM> who performs the remote assistance is hereinafter referred to as "remote assistant <NUM>", and the remote operator <NUM> who performs the remote driving is hereinafter referred to as "remote driver <NUM>".

In the remote assistance, the remote assistant <NUM> makes at least a part of determinations for the autonomous driving of the vehicle <NUM>. Basic calculations for recognition, determination, and operation required for driving are performed in the vehicle <NUM>. The remote assistant <NUM> assists the vehicle <NUM> by determining an action to be taken by the vehicle <NUM> based on information transmitted from the vehicle <NUM>. Details of the remote assistance provided by the remote assistant <NUM> to the vehicle <NUM> include a travel of the vehicle <NUM> and a stop of the vehicle <NUM>. The details of the remote assistance may also include offset avoidance for an obstacle ahead, overtaking of a preceding vehicle, and emergency evacuation.

The "at least a part of determinations" made by the remote assistant <NUM> is not limited to at least one determination on a plurality of determination factors. For example, a determination result may be obtained as a result of taking a plurality of steps below.

In the remote driving, the remote driver <NUM> drives the vehicle <NUM>, specifically, performs at least a part of a steering operation or an acceleration or deceleration operation. In the remote driving, the recognition, determination, and operation required for driving are performed by the remote driver <NUM>. The remote driver <NUM> drives the vehicle <NUM> from a remote location in the same manner as in a driver's seat of the vehicle <NUM>. In the remote driving, the remote driver <NUM> need not always perform all the recognition, determination, and operation. At least a part of the recognition, determination, and operation may be assisted by functions of the vehicle <NUM>.

A server <NUM>, a remote assistance terminal <NUM>, and a remote driving terminal <NUM> are installed in the remote control center <NUM>. The vehicle <NUM> is connected to the server <NUM> via a communication network <NUM> including the fourth-generation mobile communication system (<NUM>) and the fifth-generation mobile communication system (<NUM>). The number of vehicles <NUM> communicable with the server <NUM> is one or more, preferably plural.

The remote assistance terminal <NUM> is an operation terminal to be operated by the remote assistant <NUM> for remote assistance. The remote driving terminal <NUM> is an operation terminal to be operated by the remote driver <NUM> for remote driving. At least one remote assistance terminal <NUM>, preferably a plurality of remote assistance terminals <NUM>, is provided. Similarly, at least one remote driving terminal <NUM>, preferably a plurality of remote driving terminals <NUM>, is provided. The remote control center <NUM> has as many remote assistants <NUM> as the remote assistance terminals <NUM>. The remote control center <NUM> also has as many remote drivers <NUM> as the remote driving terminals <NUM>. The remote operator <NUM> and the remote operator <NUM> may be different persons or may be the same person. The remote assistance terminal <NUM> and the remote driving terminal <NUM> may be different terminals or may be the same terminal capable of switching the remote assistance function and the remote driving function.

Each remote assistance terminal <NUM> and each remote driving terminal <NUM> are connected to the server <NUM> via a communication network including a local area network (LAN) and the Internet. The remote control center <NUM> need not be a real facility. The remote control center <NUM> is herein a system in which the remote assistance terminal <NUM> and the remote driving terminal <NUM> are connected to the server <NUM> via the communication network. Therefore, the server <NUM> may be installed on a cloud and the remote assistance terminal <NUM> and the remote driving terminal <NUM> may be installed in satellite offices at various places or in homes of the remote operators. The remote assistance terminal <NUM> and the remote driving terminal <NUM> may be connected to different servers. The remote assistance terminal <NUM> and the remote driving terminal <NUM> may be installed at different places.

<FIG> is a block diagram showing an example of the configuration of the vehicle <NUM>. The vehicle <NUM> includes an in-vehicle computer <NUM>. The in-vehicle computer <NUM> is an aggregate of a plurality of electronic control units (ECUs) mounted on the vehicle <NUM>. The vehicle <NUM> includes an external sensor <NUM>, an internal sensor <NUM>, an actuator <NUM>, and a communication device <NUM>. These devices are connected to the in-vehicle computer <NUM> by using an in-vehicle network such as a controller area network (CAN).

The in-vehicle computer <NUM> includes one or more processors 21a (hereinafter referred to simply as "processor 21a") and one or more memories 21b (hereinafter referred to simply as "memory 21b") coupled to the processors 21a. The memory 21b stores one or more programs 21c (hereinafter referred to simply as "program 21c") that can be executed by the processor 21a and various types of information related to the program 21c.

When the processor 21a executes the program 21c, various processes are implemented by the processor 21a. The program 21c includes, for example, a program for implementing autonomous driving, a program for implementing remote assistance, and a program for implementing remote driving. The program 21c includes a remote control request program that causes the in-vehicle computer <NUM> to function as a remote control request device described later. The memory 21b includes a main storage device and an auxiliary storage device. The program 21c can be stored in the main storage device or in a computer-readable recording medium serving as the auxiliary storage device. The auxiliary storage device may store a map database that manages map information for autonomous driving.

The external sensor <NUM> includes a camera that images a view around the vehicle <NUM>, in particular, ahead of the vehicle <NUM>. The camera may be a monocular camera or a stereo camera. A plurality of cameras may be provided to image views on the sides of and behind the vehicle <NUM> in addition to the view ahead of the vehicle <NUM>. The camera may be shared between the autonomous driving and the remote control. Alternatively, cameras may be provided separately for the autonomous driving and for the remote control.

The external sensor <NUM> includes a recognition sensor other than a camera. The recognition sensor recognizes a situation around the vehicle <NUM>. Examples of the recognition sensor other than the camera include a laser imaging, detection, and ranging (LiDAR) sensor and a millimeter wave radar. The external sensor <NUM> includes a position sensor that detects a position and an orientation of the vehicle <NUM>. Examples of the position sensor include a Global Positioning System (GPS) sensor. Information obtained by the external sensor <NUM> is transmitted to the in-vehicle computer <NUM>. The external sensor <NUM> may include a microphone that collects sounds around the vehicle <NUM>.

The internal sensor <NUM> includes a condition sensor that acquires information about motion of the vehicle <NUM>. Examples of the condition sensor include a wheel speed sensor, an acceleration sensor, an angular velocity sensor, and a steering angle sensor. The acceleration sensor and the angular velocity sensor may be an inertial measurement unit (IMU). Information obtained by the internal sensor <NUM> is transmitted to the in-vehicle computer <NUM>. The information obtained by the internal sensor <NUM> is hereinafter referred to as "internal information", and the information obtained by the external sensor <NUM> is hereinafter referred to as "external information".

The actuator <NUM> includes a steering device that steers the vehicle <NUM>, a drive device that drives the vehicle <NUM>, and a braking device that brakes the vehicle <NUM>. Examples of the steering device include a power steering system, a steer-by-wire steering system, and a rear wheel steering system. Examples of the drive device include an engine, a battery electric vehicle (BEV) system, and a hybrid system. Examples of the braking device include a hydraulic brake and a regenerative brake. The actuator <NUM> operates by a control signal transmitted from the in-vehicle computer <NUM>.

The communication device <NUM> controls wireless communication with the outside of the vehicle <NUM>. The communication device <NUM> communicates with the server <NUM> via the communication network <NUM>. Information processed by the in-vehicle computer <NUM> is transmitted to the server <NUM> by using the communication device <NUM>. Information processed by the server <NUM> is taken into the in-vehicle computer <NUM> by using the communication device <NUM>. When vehicle-to-vehicle communication with other vehicles or road-to-vehicle communication with infrastructure facilities is required for autonomous driving, communication with these external devices is also performed by the communication device <NUM>.

<FIG> is a block diagram showing an example of the configuration of the remote control center <NUM>. The remote control center <NUM> includes the server <NUM>. The server <NUM> is a single computer or an aggregate of a plurality of computers connected via a communication network. The remote control center <NUM> includes the remote assistance terminal <NUM>, the remote driving terminal <NUM>, and a communication device <NUM>. These devices are connected to the server <NUM> by using a communication network. As described above, a plurality of the remote assistance terminals <NUM> and a plurality of the remote driving terminals <NUM> may be connected to the server <NUM>.

The server <NUM> includes one or more processors 32a (hereinafter referred to simply as "processor 32a") and one or more memories 32b (hereinafter referred to simply as "memory 32b") coupled to the processors 32a. The memory 32b stores one or more programs 32c (hereinafter referred to simply as "program 32c") that can be executed by the processor 32a and various types of information related to the program 32c.

When the processor 32a executes the program 32c, various processes are implemented by the processor 32a. The program 32c includes, for example, a program for implementing remote assistance and a program for implementing remote driving. The memory 32b includes a main storage device and an auxiliary storage device. The program 32c can be stored in the main storage device or in a computer-readable recording medium serving as the auxiliary storage device. The auxiliary storage device may store a map database that manages map information for autonomous driving. The map database may be stored in at least one of the server <NUM> and the in-vehicle computer <NUM>.

The remote assistance terminal <NUM> includes an information output unit 34a. The information output unit 34a is a device that outputs information necessary for remote assistance for the vehicle <NUM> to the remote assistant <NUM>. The information output unit 34a includes a display that outputs an image. The display displays, for example, an image ahead of the vehicle <NUM> that is captured by the camera of the vehicle <NUM>. The display may have a plurality of display screens to display images of views on the sides of and/or behind the vehicle <NUM>. The information output unit 34a may include a loudspeaker that outputs sound. The loudspeaker may output, for example, a sound or voice for information about the start or end of remote assistance. When the vehicle <NUM> includes a microphone, the loudspeaker may inform the remote assistant <NUM> about a situation around the vehicle <NUM> that is grasped by collecting sounds by the microphone.

The remote assistance terminal <NUM> includes an operation input unit 34b. The operation input unit 34b is a device for inputting an operation for remote assistance by the remote assistant <NUM>. Specific examples of the input device include buttons, a lever, and a touch panel. For example, the vehicle <NUM> may be assisted in traveling, stopping, or laterally moving depending on a tilting direction of the lever. Examples of the lateral movement include offset avoidance for an obstacle ahead, lane change, and overtaking of a preceding vehicle.

The remote driving terminal <NUM> includes an information output unit 36a. The information output unit 36a is a device that outputs information necessary for remote driving of the vehicle <NUM> to the remote driver <NUM>. The information output unit 36a includes a display that outputs an image. The display displays, for example, an image ahead of the vehicle <NUM> that is captured by the camera of the vehicle <NUM>. As the display method, for example, the display may display the same view as the forward view from the driver's seat of the vehicle <NUM>. The display may have a plurality of display screens to display images of views on the sides of and/or behind the vehicle <NUM>. The information output unit 36a may include a loudspeaker that outputs sound. The loudspeaker may output, for example, a sound or voice for information about the start or end of remote driving. When the vehicle <NUM> includes a microphone, the loudspeaker may inform the remote driver <NUM> about a situation around the vehicle <NUM> that is grasped by collecting sounds by the microphone.

The remote driving terminal <NUM> includes an operation input unit 36b. The operation input unit 36b is a device for inputting an operation for remote driving by the remote driver <NUM>. To simulate operations to be required when actually driving the vehicle <NUM>, the operation input unit 36b includes a steering wheel for steering operation, an accelerator pedal for acceleration operation, and a brake pedal for deceleration operation. When the vehicle <NUM> includes a transmission, the operation input unit 36b may also include a lever or switch for the transmission. In addition, the operation input unit 36b includes devices for inputting operations necessary for safe driving, such as an operation lever for operating a turn signal of the vehicle <NUM> and an operation lever for operating a wiper.

The communication device <NUM> controls communication with the outside of the remote control center <NUM>. The communication device <NUM> communicates with one or more vehicles <NUM> via the communication network <NUM>. Information processed by the server <NUM> is transmitted to the vehicle <NUM> by using the communication device <NUM>. Information processed by the vehicle <NUM> is taken into the server <NUM> by using the communication device <NUM>.

When requesting remote assistance and remote driving, the remote control request device may use a communication device to communicate with the autonomous driving vehicle. For this reason, depending on communication environment, there may be "good communication status where the remote assistance and the remote driving can be used", "limited communication status where only the remote assistance can be used", and "bad communication status so that the remote assistance and the remote control cannot be used". The remote assistance may be a system that controls the autonomous driving vehicle by using an automatic driving system. If the driving system is a system that depends on LiDAR, it may be out of the remote assistance in a situation where light emitted from LiDAR is diffused by rain or fog. One of features of the remote assistance may be that "the vehicle travels along a course. It is not possible to instruct driving that deviates from the course by the remote assistance.

The autonomous driving of the vehicle <NUM> can be implemented by satisfying road conditions, geographical conditions, environmental conditions, hardware conditions, and various other conditions. A domain in which all the conditions for implementing the autonomous driving are satisfied is hereinafter referred to as "autonomous driving domain". When performing the autonomous driving, it is necessary to make determination as to whether the vehicle <NUM> is in the autonomous driving domain, that is, inside or outside determination on the autonomous driving domain.

The autonomous driving domain in the present disclosure is designed as a framework at a level equal to or higher than that of an operational design domain (ODD), which means driving environment conditions on which the autonomous driving system operates. For example, the general ODD framework is limited to expression of only the driving environment conditions, and the outside of the ODD means that "there is a cause outside the vehicle and the autonomous driving system does not operate normally". In the framework of the autonomous driving domain in the present disclosure, the outside of the autonomous driving domain may include such a situation that "there is a cause in the vehicle and the autonomous driving system does not operate normally". All the driving environment conditions that define the general ODD are parameters that can be expressed on a map. Therefore, the ODD can be considered to express an internal/external relationship with a vehicle of interest when grasped in terms of a positional relationship. The autonomous driving domain in the present disclosure may further include an internal/external relationship with the vehicle of interest when grasped on a time axis.

Both the remote assistance and the remote driving are remote control to be performed when the vehicle <NUM> currently has or is expected to have difficulty in continuing the autonomous driving. The remote assistance is an operation in which the remote assistant <NUM> makes a part of the determination for the autonomous driving to continue the autonomous driving. Therefore, it is a prerequisite for the remote assistance that the vehicle <NUM> is in the autonomous driving domain.

When the vehicle <NUM> is outside the autonomous driving domain, the autonomous driving cannot be performed and the remote assistance cannot be performed. In such a case, the remote driving is used as means for operating the vehicle <NUM>. The remote driving can be implemented by satisfying road conditions, geographical conditions, environmental conditions, hardware conditions, and various other conditions. A domain in which all the conditions for implementing the remote driving are satisfied is hereinafter referred to as "remote driving domain". When performing the remote driving, it is necessary to make determination as to whether the vehicle <NUM> is in the remote driving domain, that is, inside or outside determination on the remote driving domain. The remote driving domain in the present disclosure is designed as a framework at a level equal to or higher than that of the ODD similarly to the autonomous driving domain.

Hereinafter, the inside or outside determination on the autonomous driving domain and the inside or outside determination on the remote driving domain will be described with reference to <FIG>. For convenience of explanation using the figure, an autonomous driving domain ADD and a remote driving domain RDD are represented on a map together with a route RT on which the vehicle <NUM> travels.

According to the example shown in <FIG>, the vehicle <NUM> is in the autonomous driving domain ADD at its current position. When the vehicle <NUM> travels along the route RT, the vehicle <NUM> at a point P1 is expected to enter the remote driving domain RDD while remaining in the autonomous driving domain ADD. At a point P2, the vehicle <NUM> is expected to exit the autonomous driving domain ADD but remain in the remote driving domain RDD. At a point P3, the vehicle <NUM> is expected to exit the remote driving domain RDD but enter the autonomous driving domain ADD again.

As described above, in the inside or outside determination on the autonomous driving domain ADD and the remote driving domain RDD, determination is made on an internal/external relationship between each of the autonomous driving domain ADD and the remote driving domain RDD and not only the current vehicle <NUM> but also the vehicle <NUM> in the future. In the determination on the internal/external relationship in the future, determination is made on an internal/external relationship between the vehicle <NUM> and each of the autonomous driving domain ADD and the remote driving domain RDD in terms of a positional relationship, and an internal/external relationship between the vehicle <NUM> and each of the autonomous driving domain ADD and the remote driving domain RDD in terms of a time axis. That is, determination is made as to whether the vehicle <NUM> is inside or outside the autonomous driving domain ADD at a future point or time and whether the vehicle <NUM> is inside or outside the remote driving domain RDD at a future point or time.

Hereinafter, description is given about examples of conditions for the autonomous driving domain ADD and the remote driving domain RDD and examples of criteria for the inside or outside determination.

The upper limit vehicle speed of the vehicle <NUM> needs to cover the speed limit of a road where the vehicle <NUM> travels. For example, when the remote driving only supports a speed up to an upper limit of <NUM>/h, the remote driving service cannot be provided on expressways. In this case, determination can be made that the expressways are outside the remote driving domain RDD.

A lane change function is required to move to a destination that requires lane switching. For example, silent communication between drivers is important to get into a space between vehicles in a congested environment. However, such communication is difficult in the autonomous driving. Therefore, determination can be made that such an environment is outside the autonomous driving domain ADD.

When the autonomous driving is performed with the aid of road markings, determination can be made that, for example, an unpaved riverside is outside the autonomous driving domain ADD.

Communication with the server <NUM> is required to implement the remote driving. Therefore, determination can be made that a mountainous area beyond the reach of radio waves is outside the remote driving domain RDD.

In a land with no map created and a land with a large deviation from a map due to construction work, the autonomous driving service for autonomously driving the vehicle <NUM> with the aid of map information cannot be provided. That is, determination can be made that the land with no available map information is outside the autonomous driving domain ADD.

The autonomous driving is executed based on distance measurement information obtained by the LiDAR sensor, but light for use in LiDAR is scattered under rain or fog. Therefore, determination can be made that the rainy or foggy environment is outside the autonomous driving domain ADD.

When the vehicle <NUM> includes a low-sensitivity camera, an external image cannot be captured in a tunnel or at night. When such a camera image is used for the remote driving, determination can be made that the area in the tunnel or the nighttime is outside the remote driving domain RDD.

When the hardware used only for the autonomous driving is performing a desired operation or returning a desired output, determination can be made that the hardware is in the autonomous driving domain ADD. For example, when the LiDAR sensor is the hardware used only for the autonomous driving and is performing a desired operation or returning a desired output, determination can be made that the LiDAR sensor is in the autonomous driving domain ADD. When the hardware used only for the remote driving is performing a desired operation or returning a desired output, determination can be made that the hardware is in the remote driving domain RDD. Examples of the hardware herein include a calculation device and a sensor.

It is conceivable to predict the sensor operation at a future location and time based on map information, weather information, or the like and estimate the autonomous driving domain ADD and the remote driving domain RDD from the present to the future. For example, rain adversely affects the use of the LiDAR sensor. Therefore, when a location where a heavy shower is occurring is known based on the weather information, determination can be made that the time to reach that location is outside the autonomous driving domain ADD.

When the camera used for the remote driving is susceptible to backlight and the location and time when a signal cannot be recognized due to backlight can be predicted based on the direction of the vehicle <NUM> and the direction of the sun, determination can be made that the location and the time are outside the remote driving domain RDD.

When the autonomous driving is performed by using a sensor mounted at a low position where the sensor is affected by mud splash, determination can be made that rainy weather is outside the autonomous driving domain ADD. When the autonomous driving is performed by using a sensor mounted on a top position of the vehicle <NUM>, determination can be made that a travel in a land with deciduous trees is outside the autonomous driving domain ADD.

When a GPS antenna is used, a location unfavorable to the GPS can be predicted from the number of satellites based on the point and time. When the autonomous driving is performed by using the GPS, determination can be made that the predicted location unfavorable to the GPS is outside the autonomous driving domain ADD.

Most software products used in the autonomous or remote driving are independent and indispensable. The software may be frozen even if the software is operating. Various symptoms may occur. There are many symptoms such as failure in an appropriate output due to a huge amount of calculation, a decrease in a calculation speed due to thermal runaway, and failure in recovery from an inappropriate error due to a bug. Therefore, it is necessary to periodically communicate with the software to check whether a desired output is made, thereby checking the health of the software. For example, when a route planning module can calculate a route to be followed in the future, determination can be made that the route planning module is in the autonomous driving domain ADD.

In the autonomous driving, self-position estimation is performed. In the self-position estimation, the likelihood, that is, reliability of an estimation result can be calculated based on variance or distribution. When the reliability is equal to or higher than a threshold, determination can be made that the reliability is in the autonomous driving domain ADD.

When each municipality has its own regulations, functions that do not meet those regulations cannot be used. For example, when the use of remote driving is prohibited in a certain municipality, the inside of the municipality is outside the remote driving domain RDD.

As in the above condition examples, the autonomous driving domain ADD includes preset domains and also includes domains that change with time. The inside or outside determination on the autonomous driving domain ADD is executed based on the external information and the internal information acquired by the external sensor <NUM> and the internal sensor <NUM>. Similarly, the remote driving domain RDD includes preset domains and also includes domains that change with time. The inside or outside determination on the remote driving domain RDD is executed based on the external information and the internal information acquired by the external sensor <NUM> and the internal sensor <NUM>.

<FIG> and <FIG> are conceptual diagrams for explaining an outline of a remote control request method according to a first embodiment of the present disclosure. Hereinafter, the remote control request method according to the first embodiment will be described with reference to these figures.

In the example shown in <FIG>, the vehicle <NUM> is currently in the autonomous driving domain ADD and is autonomously driven along a route RT generated up to a predetermined period ahead. When the vehicle <NUM> moves along the route RT, the vehicle <NUM> enters an area within the autonomous driving domain ADD and within the remote driving domain RDD. The remote driving is possible in this area, but the autonomous driving is prioritized. Since the remote driver <NUM> is required to have the same high skill as in a case of actually driving the vehicle <NUM>, reduction of a burden on the remote driver <NUM> is one of the reasons for giving priority to the autonomous driving.

It is assumed that the vehicle <NUM> has difficulty in continuing the autonomous driving at a time T1. Alternatively, the vehicle <NUM> is expected to have difficulty in continuing the autonomous driving. Since the vehicle <NUM> is in the autonomous driving domain ADD at the time T1, the autonomous driving can be continued by receiving remote assistance. As an example, it may be assumed that the vehicle <NUM> turns right at an intersection where there is no right turn instruction by a traffic light. In this case, the vehicle <NUM> can turn right at the intersection and continue the autonomous driving when a person makes a right turn determination on behalf of the vehicle <NUM>.

In the first embodiment, the vehicle <NUM> makes the inside or outside determination on the autonomous driving domain ADD and the inside or outside determination on the remote driving domain RDD. At the time T1, the vehicle <NUM> transmits a remote assistance request RQS to the remote assistant <NUM>. The remote assistant <NUM> who has received the remote assistance request RQS operates the remote assistance terminal <NUM> based on information transmitted from the vehicle <NUM>, such as a camera image. An operation signal RMS for remotely assisting the vehicle <NUM> is transmitted from the remote assistance terminal <NUM> to the vehicle <NUM>. By being remotely assisted by the operation signal RMS, the vehicle <NUM> can continue the autonomous driving again.

When the vehicle <NUM> moves along the route RT, the vehicle <NUM> exits the autonomous driving domain ADD at a time T2. By exiting the autonomous driving domain ADD, the vehicle <NUM> has difficulty in continuing the autonomous driving. In this case, the vehicle <NUM> exits the autonomous driving domain ADD but is in the remote driving domain RDD. Therefore, the operation of the vehicle <NUM> can be continued by switching the autonomous driving to the remote driving. At the time T2, the vehicle <NUM> transmits a remote driving request RQD to the remote driver <NUM>. The remote driver <NUM> who has received the remote driving request RQD operates the remote driving terminal <NUM> based on information transmitted from the vehicle <NUM>, such as a camera image. An operation signal RMD for remotely driving the vehicle <NUM> is transmitted from the remote driving terminal <NUM> to the vehicle <NUM>. The remote driving by the operation signal RMD is continued until the vehicle <NUM> enters the autonomous driving domain ADD again while remaining in the remote driving domain RDD.

When the vehicle <NUM> moves along the route RT, the vehicle <NUM> enters the autonomous driving domain ADD again at a time T3. By entering the autonomous driving domain ADD, the vehicle <NUM> can resume the autonomous driving. In this case, the remote driving is stopped and switched to the autonomous driving of the vehicle <NUM>. In such an area where the autonomous driving is possible, the burden on the remote driver <NUM> can be reduced by giving priority to the autonomous driving over the remote driving.

In the example shown in <FIG>, when the vehicle <NUM> that has started the remote driving at the time T2 moves along the route RT, the vehicle <NUM> exits the remote driving domain RDD at a time T4. When the vehicle <NUM> exits the remote driving domain RDD, the remote driver <NUM> cannot continue the remote driving. The remote driving cannot be switched to the autonomous driving because the vehicle <NUM> is not in the autonomous driving domain ADD at the time T4. In this case, the vehicle <NUM> decelerates and stops on the spot to ensure safety. Alternatively, the trajectory of the vehicle <NUM> may be changed to stop the vehicle <NUM> at a safe place such as a road shoulder.

At the stage of determination of the route RT, selection may be made for a route in which the autonomous driving domain ADD or the remote driving domain RDD continues to the destination, that is, a route in which the autonomous driving or the remote driving continues to the destination. When there is a plurality of candidates for the route RT, selection may be made for a route in which the autonomous driving domain ADD continues for a longer period. Since the time when the autonomous driving domain ADD and the remote driving domain RDD are switched (for example, the time T2) can be calculated, selection may be made for a route in which the time of switching agrees with times when the remote assistant <NUM> and the remote driver <NUM> can operate. For example, it is assumed that the time when the remote assistant <NUM> can operate (end time of work) is up to a time T5 and the time when the remote driver <NUM> can operate (start time of work) is after the time T5. In this case, a route in which the autonomous driving domain ADD is switched to the remote driving domain RDD at the time T5 or a route in which both the autonomous driving domain ADD and the remote driving domain RDD are present at the time T5 may be selected from among the candidates for the route RT.

The remote control request method can be implemented by a remote control system according to the first embodiment with a configuration shown in <FIG>. In <FIG>, functions of the in-vehicle computer <NUM> and functions of the server <NUM> are represented by blocks. Hereinafter, the remote control system according to the first embodiment will be described focusing on the functions of the in-vehicle computer <NUM> and the server <NUM>. Description will be omitted or simplified for the components and functions already described above. <FIG> illustrates a LiDAR sensor 22a, a camera 22b, a millimeter wave sensor 22c, and a GPS receiver 22d as the external sensor <NUM>.

The in-vehicle computer <NUM> includes an autonomous driving ECU <NUM>, a remote assistance ECU <NUM>, a remote driving ECU <NUM>, a remote control request determination unit <NUM>, and a remote function selection unit <NUM>. The remote control request determination unit <NUM> and the remote function selection unit <NUM> may be independent ECUs or may be functions of one ECU. Signals including external information and internal information are input to the autonomous driving ECU <NUM> and the remote driving ECU <NUM> from the LiDAR sensor 22a, the camera 22b, the millimeter wave sensor 22c, the GPS receiver 22d, and the internal sensor <NUM>. All the signals need not be input to both the ECUs <NUM> and <NUM>. For example, the input of the signal from the LiDAR sensor 22a to the remote driving ECU <NUM> may be omitted.

The autonomous driving ECU <NUM> includes a vehicle position recognition unit 210a, a surrounding situation recognition unit 210b, a travel planning unit 210c, and an autonomous driving domain investigation unit 210d. These units are implemented as functions of the autonomous driving ECU <NUM> when a program stored in the memory of the autonomous driving ECU <NUM> is executed by the processor.

The vehicle position recognition unit 210a recognizes the position of the vehicle <NUM> on a map based on position information of the vehicle <NUM> that is received by the GPS receiver 22d, information on motion of the vehicle <NUM> that is detected by the internal sensor <NUM>, and the map information obtained from the map database. The vehicle position recognition unit 210a can estimate the position of the vehicle <NUM> based on a relative position of a feature detected by the LiDAR sensor 22a, the camera 22b, or the millimeter wave sensor 22c with respect to the vehicle <NUM>, the information on the motion of the vehicle <NUM> that is detected by the internal sensor <NUM>, and a position of the detected feature on the map.

The surrounding situation recognition unit 210b recognizes an object around the vehicle <NUM> and determines the position and type of the object by using a technique such as pattern matching or deep learning for information received from the LiDAR sensor 22a, the camera 22b, or the millimeter wave sensor 22c. Examples of the target object recognized by the surrounding situation recognition unit 210b include a moving object such as a vehicle, a motorcycle, a bicycle, and a pedestrian, and a stationary object. The surrounding situation recognition unit 210b outputs, as a target, the object whose position and type have been determined.

The travel planning unit 210c creates a travel plan for the vehicle <NUM> based on, for example, the route RT recorded in the map database and target information obtained by the surrounding situation recognition unit 210b. The travel plan is created so that the vehicle <NUM> appropriately travels on the route RT in light of criteria such as safety, legal compliance, and travel efficiency. The travel planning unit 210c generates a target trajectory based on the created travel plan. The target trajectory includes a set of target positions of the vehicle <NUM> in a coordinate system fixed to the vehicle <NUM> and target speeds at individual target points. The autonomous driving ECU <NUM> calculates an actuator control amount for causing the vehicle <NUM> to follow the target trajectory generated by the travel planning unit 210c.

The autonomous driving domain investigation unit 210d executes the inside or outside determination on the autonomous driving domain ADD in accordance with predetermined determination criteria as in the criterion examples in the condition examples described above. The determination result of the inside or outside determination on the autonomous driving domain ADD is input to the remote control request determination unit <NUM> in the subsequent stage.

The remote assistance ECU <NUM> includes a remote assistance necessity determination unit 211a. The remote assistance necessity determination unit 211a is implemented as a function of the remote assistance ECU <NUM> when a program stored in the memory of the remote assistance ECU <NUM> is executed by the processor. The remote assistance ECU <NUM> communicates with the autonomous driving ECU <NUM> and acquires, from the autonomous driving ECU <NUM>, information necessary for determining the necessity of the remote assistance.

The remote assistance necessity determination unit 211a determines, based on the information from the autonomous driving ECU, whether the vehicle <NUM> has difficulty in continuing the autonomous driving, or whether the vehicle <NUM> is expected to have difficulty in continuing the autonomous driving. Next, the remote assistance necessity determination unit 211a determines whether to request the remote assistance based on a determination result about the difficulty in continuing the autonomous driving and the determination result of the inside or outside determination on the autonomous driving domain ADD from the autonomous driving ECU. The determination result from the remote assistance necessity determination unit 211a is input to the remote control request determination unit <NUM> in the subsequent stage.

The remote driving ECU <NUM> includes a vehicle position recognition unit 212a and a remote driving domain investigation unit 212b. These units are implemented as functions of the remote driving ECU <NUM> when a program stored in the memory of the remote driving ECU <NUM> is executed by the processor.

The vehicle position recognition unit 212a has the same function as that of the vehicle position recognition unit 210a of the autonomous driving ECU <NUM>. That is, the vehicle position recognition unit 212a recognizes the position of the vehicle <NUM> on the map. In the example shown in <FIG>, the autonomous driving ECU <NUM> and the remote driving ECU <NUM> separately recognize the position of the vehicle <NUM>, but another independent ECU may recognize the position of the vehicle <NUM> and the autonomous driving ECU <NUM> and the remote driving ECU <NUM> may acquire the recognition result.

The remote driving domain investigation unit 212b executes the inside or outside determination on the remote driving domain RDD in accordance with predetermined determination criteria as in the criterion examples in the condition examples described above. The determination result of the inside or outside determination on the remote driving domain RDD is input to the remote control request determination unit <NUM> in the subsequent stage.

The remote control request determination unit <NUM> determines whether to request the remote control based on the determination result of the inside or outside determination on the autonomous driving domain ADD, the determination result about the necessity of the remote assistance, and the determination result of the inside or outside determination on the remote driving domain RDD. When requesting the remote control, the remote control request determination unit <NUM> determines which of the remote assistance and the remote driving to request. The determination result from the remote control request determination unit <NUM> is input to the remote function selection unit <NUM> and the communication device <NUM> in the subsequent stage.

The remote function selection unit <NUM> transmits a control signal to the actuator <NUM> based on the determination result from the remote control request determination unit <NUM>. Specifically, when the autonomous driving is continued without the remote control, the actuator control amount calculated by the autonomous driving ECU <NUM> is transmitted to the actuator <NUM> as the control signal. When the remote control is selected, the actuator control amount calculated by the autonomous driving ECU <NUM> and a remote assistance signal (for example, a Go signal or a No-Go signal) transmitted from the remote control center <NUM> are transmitted to the actuator <NUM> as the control signal. When the remote driving is selected, a remote driving signal transmitted from the remote control center <NUM> (for example, a signal including an actuator control amount for the remote driving) is transmitted to the actuator <NUM> as the control signal.

The determination result input from the remote control request determination unit <NUM> to the communication device <NUM> is transmitted from the communication device <NUM> to the communication device <NUM> of the remote control center <NUM>. The determination result is transmitted only when determination is made to request any remote control out of the remote assistance and the remote driving. When the determination result is transmitted, the request for the remote assistance or the remote driving from the in-vehicle computer <NUM> to the remote control center <NUM> is completed. As described above, the in-vehicle computer <NUM> according to the first embodiment functions as the remote control request device that requests the remote assistant <NUM> or the remote driver <NUM> to remotely control the vehicle <NUM>.

The determination result received by the communication device <NUM> from the remote control request determination unit <NUM> is input to the server <NUM>. The server <NUM> includes an operator assignment unit <NUM>. The operator assignment unit <NUM> is implemented as a function of the server <NUM> when a program stored in the memory of the server <NUM> is executed by the processor.

The operator assignment unit <NUM> assigns the remote control request from the vehicle <NUM> to an appropriate remote operator based on the determination result from the remote control request determination unit <NUM>. When the remote control request is a remote assistance request, the request is transmitted to the remote assistance terminal <NUM>. The operator assignment unit <NUM> transmits, to the remote assistance terminal <NUM>, information necessary for the remote assistance that is acquired from the vehicle <NUM>. A remote assistance signal input by the remote assistant <NUM> is transmitted from the remote assistance terminal <NUM> to the operator assignment unit <NUM>.

When the remote control request is a remote driving request, the request is transmitted to the remote driving terminal <NUM>. The operator assignment unit <NUM> transmits, to the remote driving terminal <NUM>, information necessary for the remote driving that is acquired from the vehicle <NUM>. A remote driving signal input by the remote driver <NUM> is transmitted from the remote driving terminal <NUM> to the operator assignment unit <NUM>.

The signal transmitted from the remote assistance terminal <NUM> or the remote driving terminal <NUM> to the operator assignment unit <NUM> is transmitted from the operator assignment unit <NUM> to the vehicle <NUM> via the communication device <NUM>. The remote assistance signal or the remote driving signal transmitted from the remote control center <NUM> to the vehicle <NUM> is transmitted from the communication device <NUM> to the remote function selection unit <NUM> of the in-vehicle computer <NUM>.

Next, a remote control request method according to a second embodiment of the present disclosure will be described with reference to <FIG>. In the remote control request method according to the first embodiment, when the vehicle <NUM> currently has or is expected to have difficulty in continuing the autonomous driving, the in-vehicle computer <NUM> of the vehicle <NUM> determines which of the remote assistance and the remote driving to request. In the remote control request method according to the second embodiment, the server <NUM> executes this determination.

In the example shown in <FIG>, when the vehicle <NUM> has difficulty in continuing the autonomous driving at the time T1, information INF necessary for determining the remote control method is transmitted from the vehicle <NUM> to the server <NUM>. The information INF transmitted to the server <NUM> includes an initial determination result of the inside or outside determination on the autonomous driving domain ADD and an initial determination result of the inside or outside determination on the remote driving domain RDD from the in-vehicle computer <NUM>.

The server <NUM> determines whether to request the remote control based on the information transmitted from the vehicle <NUM>. When requesting the remote control, the server <NUM> determines which of the remote assistance and the remote driving to request. The server <NUM> can acquire information that the vehicle <NUM> does not have, in particular, information that affects the autonomous driving and information that affects the remote driving. The server <NUM> executes inside or outside determination on the autonomous driving domain ADD and inside or outside determination on the remote driving domain RDD and determines which of the remote assistance and the remote driving to request based on the information acquired from the vehicle <NUM> and the information independently acquired by the server <NUM>.

At the time T1, the server <NUM> transmits the remote assistance request RQS to the remote assistant <NUM>. The remote assistant <NUM> who has received the remote assistance request RQS operates the remote assistance terminal <NUM> based on information transmitted from the vehicle <NUM>, such as a camera image. The operation signal RMS for remotely assisting the vehicle <NUM> is transmitted from the remote assistance terminal <NUM> to the vehicle <NUM>. By being remotely assisted by the operation signal RMS, the vehicle <NUM> can continue the autonomous driving again.

When the vehicle <NUM> moves along the route RT, the vehicle <NUM> exits the autonomous driving domain ADD at the time T2. By exiting the autonomous driving domain ADD, the vehicle <NUM> has difficulty in continuing the autonomous driving. In this case, the vehicle <NUM> retransmits the information INF necessary for determining the remote control method to the server <NUM>. Similarly to the time T1, the server <NUM> executes the inside or outside determination on the autonomous driving domain ADD and the inside or outside determination on the remote driving domain RDD and determines which of the remote assistance and the remote driving to request based on the information acquired from the vehicle <NUM> and the information independently acquired by the server <NUM>.

At the time T2, the server <NUM> transmits the remote driving request RQD to the remote driver <NUM>. The remote driver <NUM> who has received the remote driving request RQD operates the remote driving terminal <NUM> based on information transmitted from the vehicle <NUM>, such as a camera image. The operation signal RMD for remotely driving the vehicle <NUM> is transmitted from the remote driving terminal <NUM> to the vehicle <NUM>. The remote driving by the operation signal RMD is continued until the vehicle <NUM> enters the autonomous driving domain ADD again while remaining in the remote driving domain RDD.

The remote control request method can be implemented by a remote control system according to the second embodiment with a configuration shown in <FIG>. In <FIG>, functions of the in-vehicle computer <NUM> and functions of the server <NUM> are represented by blocks. Hereinafter, the remote control system according to the second embodiment will be described focusing on the functions of the in-vehicle computer <NUM> and the server <NUM>. Description will be omitted or simplified for the components and functions already described above.

The main difference between the remote control system according to the first embodiment and the remote control system according to the second embodiment is that the function of the remote control request determination unit is transferred from the in-vehicle computer <NUM> to the server <NUM>. Therefore, the determination result of the inside or outside determination on the autonomous driving domain ADD, the determination result about the necessity of the remote assistance, and the determination result of the inside or outside determination on the remote driving domain RDD that are input to the remote control request determination unit in the in-vehicle computer <NUM> are transmitted to the server <NUM> of the remote control center <NUM> via the communication device <NUM>.

The determination results transmitted from the in-vehicle computer <NUM> to the server <NUM> are used as initial determination results, and the final determination is made by the server <NUM>. In the remote control system according to the second embodiment, the server <NUM> includes the operator assignment unit <NUM>, a remote control request determination unit <NUM>, a traffic information database <NUM>, and an operator information database <NUM>. The operator assignment unit <NUM> and the remote control request determination unit <NUM> are implemented as functions of the server <NUM> when a program stored in the memory of the server <NUM> is executed by the processor. The traffic information database <NUM> and the operator information database <NUM> are stored in the auxiliary storage device of the server <NUM>.

The remote control request determination unit <NUM> comprehensively determines whether to request the remote control based on the initial determination results transmitted from the in-vehicle computer <NUM> and the information held only by the server <NUM>. The information held only by the server <NUM> includes traffic information registered in the traffic information database <NUM> and operator information registered in the operator information database <NUM>. When requesting the remote control, the remote control request determination unit <NUM> comprehensively determines which of the remote assistance and the remote driving to request based on the information acquired from the in-vehicle computer <NUM> and the information held only by the server <NUM>. The determination result from the remote control request determination unit <NUM> is input to the operator assignment unit <NUM> in the subsequent stage.

Hereinafter, description is given about an example of the determination as to which of the remote assistance and the remote driving to request based on the information that is not held by the vehicle <NUM>.

By using detailed accident information, it is possible to determine whether the vehicle can take appropriate action by the remote assistance. Examples of the detailed accident information include information indicating an accident that obstructs a road, information indicating that only a specific lane is closed, and information indicating that the accident has already been handled. The vehicle <NUM> can observe a vehicle stopped on a road shoulder, but cannot observe that "a large number of metal pieces are scattered on the road". To obtain such information, it is necessary to utilize police information and the like.

By using communication status information, it is possible to determine which of the remote assistance and the remote driving to request. For example, it is difficult to use the remote driving in an environment where communication may be unstable because of busy pedestrian traffic.

Determination may be made as to which of the remote assistance and the remote driving to request based on the availability of the remote operators. For example, when the remote assistant <NUM> has a tight schedule and the remote driver <NUM> has extra time in the schedule, the remote driving is adopted even in a situation in which the remote assistance may suffice.

It is desirable for the autonomous driving that the behavior of other vehicles be easy to predict. That is, for the vehicle <NUM> that performs the autonomous driving, it is desirable that a vehicle whose behavior can be estimated by applying the behavior to a model or a vehicle whose behavior can be acquired in advance by vehicle-to-vehicle communication be present around the vehicle <NUM>, rather than a vehicle that behaves unexpectedly. Therefore, when the ratio of vehicles whose behavior is easy to predict to the surrounding vehicles is equal to or higher than a threshold, the remote assistance may be adopted with priority.

As described above, in the second embodiment, the in-vehicle computer <NUM> and the server <NUM> cooperate to make the inside or outside determination on the autonomous driving domain ADD and the inside or outside determination on the remote driving domain RDD and determine which of the remote assistance and the remote driving to request. That is, the in-vehicle computer <NUM> and the server <NUM> according to the second embodiment function as a remote control request system that requests the remote assistant <NUM> or the remote driver <NUM> to remotely control the vehicle <NUM>.

Next, a remote control request method according to a third embodiment of the present disclosure will be described with reference to <FIG>. In the third embodiment, when the vehicle <NUM> currently has or is expected to have difficulty in continuing the autonomous driving, the inside or outside determination on the autonomous driving domain and the inside or outside determination on the remote driving domain are made based on a target trajectory.

<FIG> depicts a situation in which the vehicle <NUM> performing the autonomous driving stops behind a preceding vehicle <NUM> and then restarts. When the preceding vehicle <NUM> stops, it is unknown whether the preceding vehicle <NUM> restarts or is parked. In such a case, it is difficult to continue the autonomous driving. To continue the operation of the vehicle <NUM>, it is necessary to request the remote operator to perform the remote control.

In the third embodiment, a trajectory TJRA of the vehicle <NUM> when the remote assistance is selected and a trajectory TJRDK of the vehicle when the remote driving is selected are calculated. The trajectory TJRA is calculated from the maximum steered speed, a steered angle, and the minimum vehicle speed when performing the remote assistance. Similarly, the trajectory TJRDK is calculated from the maximum steered speed, a steered angle, and the minimum vehicle speed when performing the remote driving.

Next, determination is made about contact between each of the calculated trajectories TJRA and TJRDK and the preceding vehicle <NUM>. When the trajectory TJRA comes into contact with the preceding vehicle <NUM>, determination can be made that the vehicle <NUM> is outside the autonomous driving domain. When the trajectory TJRDK comes into contact with the preceding vehicle <NUM>, determination can be made that the vehicle <NUM> is outside the remote driving domain. In the example shown in <FIG>, the trajectory TJRA is in contact with the preceding vehicle <NUM>, but the trajectory TJRDK is not in contact with the preceding vehicle <NUM>. Therefore, the remote driving is selected as the remote control to be requested. When neither of the trajectories TJRA and TJRDK is in contact with the preceding vehicle <NUM>, the remote assistance request has priority based on a difference in burden between the remote assistant <NUM> and the remote driver <NUM>.

<FIG> depicts a situation in which the vehicle <NUM> performing the autonomous driving joins to a main lane from a joining lane. To join to the main lane so as not to come into contact with another vehicle <NUM> traveling on the main lane, it is necessary to recognize the speed of the other vehicle <NUM> and adjust the timing to join. It is difficult to join to the main lane from the joining lane by the autonomous driving while the other vehicle <NUM> is traveling on the main lane. Therefore, the remote operator is requested to perform the remote control.

In the third embodiment, the trajectory TJRA of the vehicle <NUM> when the remote assistance is selected and the trajectory TJRDK of the vehicle when the remote driving is selected are calculated. The trajectory TJRA is calculated from the maximum speed and the maximum acceleration when performing the remote assistance. Similarly, the trajectory TJRDK is calculated from the maximum speed and the maximum acceleration when performing the remote driving. In <FIG>, circles on the trajectories TJRA and TJRDK indicate positions of the vehicle <NUM> at respective times. From differences in the intervals of the circles, it can be understood that the trajectory TJRDK has a higher acceleration at the time of joining than the trajectory TJRA.

Next, the time to collision (TTC) is calculated between each of the calculated trajectories TJRA and TJRDK and the other vehicle <NUM>. When the TTC between the trajectory TJRA and the other vehicle <NUM> is equal to or smaller than a predetermined threshold, determination can be made that the vehicle <NUM> is outside the autonomous driving domain. When the TTC between the trajectory TJRDK and the other vehicle <NUM> is equal to or smaller than the threshold, determination can be made that the vehicle <NUM> is outside the remote driving domain. When the TTC between each of the two trajectories TJRA and TJRDK and the other vehicle <NUM> is larger than the threshold, the remote control that provides a trajectory with a larger TTC is selected. In the example shown in <FIG>, the remote driving is selected as the remote control to be requested.

<FIG> depicts a situation in which the vehicle <NUM> performing the autonomous driving decelerates and stops behind an obstacle <NUM>. When the vehicle <NUM> has difficulty in stopping by the autonomous driving, the remote operator is requested to perform the remote control.

In the third embodiment, the trajectory TJRA of the vehicle <NUM> when the remote assistance is selected and the trajectory TJRDK of the vehicle when the remote driving is selected are calculated. The trajectory TJRA is calculated from the maximum deceleration when performing the remote assistance. Similarly, the trajectory TJRDK is calculated from the maximum deceleration when performing the remote driving. In <FIG>, circles on the trajectories TJRA and TJRDK indicate positions of the vehicle <NUM> at respective times. From differences in the intervals of the circles, it can be understood that the trajectory TJRDK has a higher deceleration at the time of stopping than the trajectory TJRA.

Next, the TTC is calculated between each of the calculated trajectories TJRA and TJRDK and the obstacle <NUM>. When the TTC between the trajectory TJRA and the obstacle <NUM> is equal to or smaller than a predetermined threshold, determination can be made that the vehicle <NUM> is outside the autonomous driving domain. When the TTC between the trajectory TJRDK and the obstacle <NUM> is equal to or smaller than the threshold, determination can be made that the vehicle <NUM> is outside the remote driving domain. When the TTC between each of the two trajectories TJRA and TJRDK and the obstacle <NUM> is larger than the threshold, the remote control that provides a trajectory with a larger TTC is selected. In the example shown in <FIG>, the remote driving is selected as the remote control to be requested.

Claim 1:
A remote control request system (<NUM>) comprising a remote control request device (<NUM>) including
a memory (21b) storing a program (21c), and
a processor (21a) connected to the memory (21b), wherein:
the processor (21a) is configured to request a remote operator (<NUM>,<NUM>) to perform remote control on an autonomous driving vehicle (<NUM>) when the autonomous driving vehicle (<NUM>) currently has or is expected to have difficulty in continuing autonomous driving;
wherein the remote control includes
remote assistance in which the remote operator (<NUM>) makes at least a part of determinations for the autonomous driving, and
remote driving in which the remote operator (<NUM>) performs at least one of a steering operation and an acceleration or deceleration operation of the autonomous driving vehicle (<NUM>);
characterized in that
the processor (21a) is configured to, during execution of the program (21c),
request the remote assistance inside an autonomous driving domain (ADD) in which the autonomous driving is continuable by receiving the remote assistance,
request the remote driving outside the autonomous driving domain (ADD), and during execution of the program (21c), calculate a trajectory (TJRA) of the autonomous driving vehicle (<NUM>) when the remote assistance is selected, from a maximum steered speed, a steered angle and a minimum vehicle speed when performing the remote assistance, and calculate a trajectory (TJRDK) of the autonomous driving vehicle (<NUM>) when the remote driving is selected, from a maximum steered speed, a steered angle and a minimum vehicle speed when performing the remote driving, and determine whether the autonomous driving vehicle (<NUM>) is in the autonomous driving domain (ADD) based on the calculated trajectories (TJRA, TJRDK).