Patent Description:
The convergence of information and communication technology and the automotive industry has made rapid progress in the smartification of vehicles. Vehicles are evolving from simple mechanical devices into smart cars due to such smartification. Smart cars are also called connected cars and may refer to vehicles capable of accessing a network and having a physical system for interconnecting a vehicle and an external network through wireless communication.

Driving control using sensors of a vehicle has been attracting much attention as a key technology of smart cars. Various sensors may be mounted in a smart car, and the smart car may be driven based on information received from the sensors, to provide convenience and safety to a user.

<CIT> describes a method including receiving data associated with a sensor measurement of a perceived object, determining a label associated with the perceived object based on an initial calibration, retrieving log file data associated with the label, determining a calibration parameter associated with the sensor measurement based on the retrieved log file data, and storing the calibration parameter in association with a sensor associated with the sensor measurement. <CIT> describes a vehicle sensor calibration system that can detect an SDV on a turntable surrounded by a plurality of fiducial targets, and rotate the turntable using a control mechanism to provide the sensor system of the SDV with a sensor view of the plurality of fiducial targets. <CIT> describes an online system that builds a high definition (HD) map for a geographical region based on sensor data captured by a plurality of autonomous vehicles driving through a geographical region.

To increase safety of an autonomous vehicle, sensor errors occurring as time passes need to be minimized by periodically calibrating various sensors used for autonomous driving.

Accordingly, an aspect of the disclosure is to provide a method capable of increasing accuracy and efficiency of calibrating a plurality of sensors used for autonomous driving of a vehicle.

Provided are a vehicle driving control apparatus according to claim <NUM>, a calibration method according to claim <NUM> and a vehicle driving control apparatus according to claim <NUM>.

In accordance with an aspect of the disclosure, a vehicle driving control apparatus for controlling driving of a vehicle based on information received from a plurality of sensors is provided. The vehicle driving control apparatus includes at least one processor configured to determine a driving route to include a predetermined area of road, upon determining that calibration of a first sensor is required, and a memory configured to store values measured by the plurality of sensors while the vehicle is being driven along the driving route, wherein the at least one processor is further configured to calibrate the first sensor based on the stored measured values and information about the predetermined area of road.

In accordance with another aspect of the disclosure, a calibration method performed by a vehicle driving control apparatus based on information received from a plurality of sensors is provided. The calibration method includes determining that calibration of a first sensor is required, determining a driving route to include a predetermined area of road, storing values measured by the plurality of sensors while a vehicle is being driven along the driving route, and calibrating the first sensor based on the stored measured values and information about the predetermined area of road.

In accordance with the invention a vehicle driving control apparatus for controlling driving of a vehicle based on information received from a plurality of sensors is provided. The vehicle driving control apparatus includes a communicator configured to transmit a signal indicating that calibration of a first sensor is required, and to receive a driving route determined to include a predetermined area of road, in response to the transmitted signal, a memory configured to store values measured by the plurality of sensors while the vehicle is being driven along the driving route, and at least one processor configured to calibrate the first sensor based on the stored measured values and information about the predetermined area of road.

In accordance with the invention a calibration method performed by a vehicle driving control apparatus based on information received from a plurality of sensors is provided. The calibration method includes transmitting a signal indicating that calibration of a first sensor is required, receiving a driving route determined to include a predetermined area of road, in response to the transmitted signal, storing values measured by the plurality of sensors while a vehicle is being driven along the driving route, and calibrating the first sensor based on the stored measured values and information about the predetermined area of road.

In accordance with the invention a navigation server for providing a driving route to a vehicle driving control apparatus is provided. The navigation server includes a communicator configured to receive, from the vehicle driving control apparatus, a signal indicating that calibration of a first sensor mounted in a vehicle is required, and at least one processor configured to determine a driving route to include a predetermined area of road used to calibrate the first sensor, based on map data including information about an area of road used to calibrate each of a plurality of sensors, wherein the communicator is further configured to transmit, to the vehicle driving control apparatus, the driving route and information about the predetermined area of road.

In accordance with another aspect of the disclosure, a method, performed by a navigation server, of providing a driving route to a vehicle driving control apparatus is provided. The method includes receiving, from the vehicle driving control apparatus, a signal indicating that calibration of a first sensor mounted in a vehicle is required, determining a driving route to include a predetermined area of road used to calibrate the first sensor, based on map data including information about an area of road used to calibrate each of a plurality of sensors, and transmitting, to the vehicle driving control apparatus, the driving route and information about the predetermined area of road.

Some embodiments of the disclosure may be described in terms of functional blocks and various processing operations. Some or all of the functional blocks may be implemented by any number of hardware and/or software elements configured to perform certain functions. For example, the functional blocks of the disclosure may be implemented by one or more microprocessors or circuit elements for certain functions. As another example, the functional blocks of the disclosure may be implemented using any programming or scripting language. The functional blocks may be implemented using various algorithms executed by one or more processors. Furthermore, the disclosure might employ known technologies for electronic settings, signal processing, and/or data processing.

In addition, connection lines or connection members between elements shown in the drawings illustrate functional connections and/or physical or circuit connections. Connections between elements may be represented by replaceable or additional various functional connections, physical connections, or circuit connections in an actual apparatus.

Reference will now be made in detail to embodiments of the disclosure, examples of which are illustrated in the accompanying drawings.

<FIG> is a diagram for describing an autonomous vehicle according to an embodiment of the disclosure.

Referring to <FIG>, an autonomous vehicle <NUM> is a vehicle which is automatically driven by determining a driving environment without a driver's control. As an example, the autonomous vehicle <NUM> may be automatically driven by analyzing information received from various sensors such as a camera, a radar, a lidar, a global positioning system (GPS), and an inertial measurement unit (IMU). The autonomous vehicle <NUM> may include a vehicle-to-everything (V2X) module to communicate with various entities capable of affecting the autonomous vehicle <NUM> (e.g., a network, another vehicle, a pedestrian, a device, a grid, or the like).

The autonomous vehicle <NUM> may be controlled by a steering control unit and/or a speed control unit based on the information received from the various sensors. The various sensors used for autonomous driving may have errors as time passes due to various reasons such as an external impact, an external environment variation (e.g., a tire pressure variation while the autonomous vehicle <NUM> is being driven), or the like. Therefore, to improve the safety of the autonomous vehicle <NUM>, the errors of the sensors serving as the basis of autonomous driving need to be minimized by periodically calibrating the sensors. Calibration refers to a technique for calculating correction values for converting sensing values of sensors into vehicle reference values through comparison between a sensor reference coordinate system and a vehicle reference coordinate system. The sensor reference coordinate system may consist of an origin of sensing values and coordinate axes. The vehicle reference coordinate system may consist of a data origin and coordinate axes for controlling a steering direction and a speed of the autonomous vehicle <NUM>.

In general, a driver locates an autonomous vehicle at a specific place for calibration, and calibration is performed based on ground truth obtained by the autonomous vehicle stopped at the place. For example, the autonomous vehicle (or a separate calibrator or operator) may determine an error of each sensor and calibrate the sensor based on the obtained ground truth. However, performing calibration only at the specific place may cause inconvenience.

According to the disclosure, a method of increasing accuracy and efficiency of calibration by periodically performing calibration based on sensing values received from various sensors while an autonomous vehicle is being driven may be provided.

<FIG> is a diagram for describing an autonomous driving system according to an embodiment of the disclosure.

Referring to <FIG>, the autonomous driving system may include the autonomous vehicle <NUM> supporting the autonomous driving system, and a vehicle driving control apparatus <NUM> for controlling driving of the autonomous vehicle <NUM>. The vehicle driving control apparatus <NUM> may be a device separate from and mounted in the autonomous vehicle <NUM>, a device included in the autonomous vehicle <NUM>, or at least a part of an electronic device for controlling the autonomous vehicle <NUM>. The vehicle driving control apparatus <NUM> may include at least parts of the steering control unit and the speed control unit illustrated in <FIG>.

According to an embodiment of the disclosure, the autonomous vehicle <NUM> supporting autonomous driving may have mounted therein various sensors such as an IMU <NUM> for measuring a speed and a direction of the autonomous vehicle <NUM>, gravity, and an acceleration, a radar <NUM> for recognizing an environment based on a reflection pattern of an irradiated electromagnetic pulse, a lidar <NUM> for recognizing a surrounding environment based on a reflection pattern of an irradiated low-power laser beam, and a camera <NUM> for capturing an image. The vehicle driving control apparatus <NUM> may receive sensing data from the sensors mounted in the autonomous vehicle <NUM> and control the autonomous vehicle <NUM> based on the received sensing data.

The vehicle driving control apparatus <NUM> may store map data including precise road information of a region through which the autonomous vehicle <NUM> is to be driven. The vehicle driving control apparatus <NUM> may control the speed and/or the direction of the autonomous vehicle <NUM> by analyzing the road information and information input from the sensors every moment, by driving an artificial intelligence (AI) program by using a high-speed graphics processing unit (GPU).

The vehicle driving control apparatus <NUM> may calibrate at least one sensor based on sensing data measured by the sensors while the autonomous vehicle <NUM> is being driven. A calibration method performed by the vehicle driving control apparatus <NUM> according to an embodiment of the disclosure will now be described with reference to <FIG>.

<FIG> is a flowchart of a calibration method performed by a vehicle driving control apparatus, according to an embodiment of the disclosure.

Referring to <FIG>, to control driving of the vehicle based on information received from a plurality of sensors, the vehicle driving control apparatus <NUM> may determine that calibration of at least one sensor is required in operation S310. For example, the vehicle driving control apparatus <NUM> may determine that calibration of a first sensor among the plurality of sensors is required. The plurality of sensors may be mounted in the vehicle. The plurality of sensors may include at least one of an IMU, a camera, a radar, a lidar, a GPS, or the like.

The vehicle driving control apparatus <NUM> may determine that calibration of a certain sensor is required when sensing data received from the certain sensor indicates a value different from those of sensing data received from the other sensors. For example, the vehicle driving control apparatus <NUM> may determine whether calibration of the first sensor is required by comparing a value measured by the first sensor with values measured by the other sensors included in the plurality of sensors.

As an example, the vehicle driving control apparatus <NUM> may obtain a difference between three-dimensional (3D) point cloud data obtained from the lidar, and 3D point cloud data obtained from an image captured by the camera, by comparing the two data. The vehicle driving control apparatus <NUM> may obtain 3D coordinate values of points by applying an image feature extraction algorithm such as oriented FAST and rotated BRIEF (ORB) or scale-invariant feature transform (SIFT) to the image captured by the camera.

As another example, the vehicle driving control apparatus <NUM> may estimate location and position variations of the vehicle based on variations of the 3D coordinate values according to time. In addition, the vehicle driving control apparatus <NUM> may estimate location and position variations of the vehicle based on deceleration information and/or acceleration information obtained from the IMU. The vehicle driving control apparatus <NUM> may determine a difference between values sensed by different sensors by comparing the estimated location and position variations of the vehicle.

The vehicle driving control apparatus <NUM> may determine that calibration of the first sensor is required when a difference between the value measured by the first sensor and the values measured by the other sensors is equal to or greater than a threshold value.

Alternatively, the vehicle driving control apparatus <NUM> may determine whether calibration of the sensor is required by reconfiguring information about a road through which the vehicle is driven based on the sensing data received from the plurality of sensors and comparing the reconfigured information with previously known information about the road. The vehicle driving control apparatus <NUM> may determine whether calibration of the sensor is required by determining whether an error between the reconfigured information and the previously known information exceeds a threshold range.

Otherwise, the vehicle driving control apparatus <NUM> may previously store information about an error probability of each sensor and determine that calibration is required for a sensor, the error probability of which is calculated to be equal to or greater than a threshold value as time passes.

Alternatively, the vehicle driving control apparatus <NUM> may previously store a calibration cycle for each sensor. The vehicle driving control apparatus <NUM> may determine that calibration of a certain sensor is required when a certain time has passed after the sensor was calibrated.

In operation S320, the vehicle driving control apparatus <NUM> may determine a driving route to include a predetermined area of road.

The vehicle driving control apparatus <NUM> may previously store map data including information about an area of road used to calibrate each of the plurality of sensors. For example, the map data may include location information of the area of road used to calibrate each of the plurality of sensors and information about a shape of the area of road. The information about the area of road used to calibrate each sensor (hereinafter also referred to as a calibration area) may be tagged on the map data.

An area of road at which accurate information for <NUM> degrees of freedom (DoF) (e.g., x, y, z, roll, pitch, and yaw) calibration of each sensor can be acquired may be indicted as the calibration area on the map data. For example, an area of road at which accurate information for calibration of a sensor for sensing a roll about the X axis or a yaw about the Z axis of the vehicle can be acquired may be a roundabout. An area of road at which accurate information for calibration of a sensor for sensing a pitch about the Y axis of the vehicle can be acquired may be a straight flat road. An area of road at which accurate information for calibration of a depth sensor can be acquired may be an area of road at which a large number of corners of road facilities such as traffic signs are detected.

The vehicle driving control apparatus <NUM> may determine the driving route to include a predetermined area of road used to calibrate the first sensor (hereinafter also referred to as a calibration area of the first sensor), based on the previously stored map data. For example, the vehicle driving control apparatus <NUM> may find the calibration area of the first sensor in the map data. The vehicle driving control apparatus <NUM> may determine an area around a current location of the vehicle or an area around a driving route to a destination in the map data, as a search area. The vehicle driving control apparatus <NUM> may find the calibration area of the first sensor included in the determined search area. The vehicle driving control apparatus <NUM> may determine the driving route of the vehicle to include the calibration area of the first sensor.

<FIG> is a diagram for describing an example of an area of road used to calibrate a sensor, according to an embodiment of the disclosure.

Referring to <FIG>, the calibration area may include at least one of a roundabout <NUM> or a straight flat road <NUM>. Upon determining in operation S310 that calibration of a sensor for sensing a roll about the X axis or a yaw about the Z axis of the vehicle is required, the vehicle driving control apparatus <NUM> may determine the driving route to include a roundabout. Alternatively, upon determining in operation S310 that calibration of a sensor for sensing a pitch about the Y axis of the vehicle is required, the vehicle driving control apparatus <NUM> may determine the driving route to include a straight flat road. Otherwise, upon determining in operation S310 that calibration of a depth sensor such as a lidar is required, the vehicle driving control apparatus <NUM> may determine the driving route to include an area of road at which a large number of corners of road facilities such as traffic signs are detected.

The vehicle driving control apparatus <NUM> may determine the driving route in such a manner that the vehicle is driven by a distance greater than a driving distance along the shortest route to the destination by a threshold value or less. The vehicle driving control apparatus <NUM> may control the vehicle to be driven through an area of road for calibration within a threshold range to prevent an excessively long detour for calibration.

The vehicle driving control apparatus <NUM> may control the vehicle to be driven along the determined driving route only when a difference between a driving distance of the vehicle along the shortest route to the destination (hereinafter also referred to as a shortest driving distance) and a driving distance of the vehicle along the driving route determined to include the calibration area (hereinafter also referred to as a driving distance for calibration) is equal to or less than a threshold value. The vehicle driving control apparatus <NUM> may control the vehicle to ignore the determined driving route and be driven along the shortest route when the difference between the shortest driving distance and the driving distance for calibration is greater than the threshold value.

<FIG> is a diagram for describing a driving route determined to include an area of road used to calibrate a sensor, according to an embodiment of the disclosure.

Referring to <FIG>, the vehicle driving control apparatus <NUM> may determine a driving route <NUM> to include calibration areas <NUM> and <NUM> between a current location of the autonomous vehicle <NUM> and a destination <NUM>. The vehicle driving control apparatus <NUM> may control the autonomous vehicle <NUM> to be driven along the driving route <NUM> when a driving distance of the autonomous vehicle <NUM> along the driving route <NUM> determined to include the calibration areas <NUM> and <NUM> is acceptable compared to a driving distance of the autonomous vehicle <NUM> along a shortest route <NUM> to the destination <NUM>.

In operation S330, the vehicle driving control apparatus <NUM> may store values measured by the plurality of sensors while the vehicle is being driven along the driving route.

The vehicle driving control apparatus <NUM> may control the vehicle to be driven along the driving route determined in operation S320. In addition, the vehicle driving control apparatus <NUM> may receive the values measured by the plurality of sensors while the vehicle is being driven and use the values to control the vehicle.

The vehicle driving control apparatus <NUM> may store the values periodically measured by the plurality of sensors. For example, the vehicle driving control apparatus <NUM> may store data about a location of the vehicle, a shape of an area of road through which the vehicle is driven, or a position of the vehicle in the form of a point cloud by using the GPS, the IMU, the lidar, the radar, or the camera. The point cloud refers to a set of an enormous number of points and may be expressed as a vector including a location coordinate to indicate a point in three dimensions.

In addition, the vehicle driving control apparatus <NUM> may control the vehicle to be repeatedly driven through the calibration area, with permission of a user. The vehicle driving control apparatus <NUM> may provide, to the user, a user interface (UI) for requesting permission of the user for repeated driving for calibration. The vehicle driving control apparatus <NUM> may receive an input of the user in response to the UI. The vehicle driving control apparatus <NUM> may determine the driving route in such a manner that the vehicle is repeatedly driven through the calibration area, based on the input of the user.

<FIG> illustrates an example of a UI provided by a vehicle driving control apparatus to request permission of a user for repeated driving for calibration, according to an embodiment of the disclosure.

Referring to <FIG>, the vehicle driving control apparatus <NUM> may include a touchscreen <NUM> capable of displaying an image and receiving a touch input. The vehicle driving control apparatus <NUM> may display, on the touchscreen <NUM>, a graphical user interface (GUI) <NUM> for requesting permission of the user for repeated driving for calibration. The vehicle driving control apparatus <NUM> may control the vehicle to be repeatedly driven through a calibration area, based on a touch input of the user on a button <NUM> for permitting repeated driving.

For example, the vehicle driving control apparatus <NUM> may request permission of the user for repeated driving through the calibration area, when calibration has not been performed for a long time or upon determining that information collected by the sensors while the vehicle is being driven through the calibration area is inaccurate. For example, the vehicle driving control apparatus <NUM> may perform additional calibration by controlling the vehicle to be repeatedly driven as needed through a calibration area, repeated driving through which does not waste much time, e.g., a roundabout, with permission of the user. The vehicle driving control apparatus <NUM> may increase sensor calibration accuracy by controlling the vehicle to be repeatedly driven through the calibration area.

In operation S340, the vehicle driving control apparatus <NUM> may calibrate the sensor based on the stored measured values and information about the predetermined area of road.

The vehicle driving control apparatus <NUM> may collect driving information from the plurality of sensors while the vehicle is being driven along the driving route that has been changed to include the calibration area. The vehicle driving control apparatus <NUM> may calibrate the sensor when a reliability of calibration based on the collected driving information is equal to or greater than a threshold value. The vehicle driving control apparatus <NUM> may calibrate the sensor based on the measured values stored while the vehicle is being driven or after driving of the vehicle is terminated.

For example, the vehicle driving control apparatus <NUM> may calibrate the first sensor, which is determined to require calibration in operation S310, based on the stored measured values and information about the predetermined area of road. The vehicle driving control apparatus <NUM> may collect sensing data suitable for characteristics of the calibration area and perform calibration. The vehicle driving control apparatus <NUM> may calibrate the sensor requiring calibration based on a driving history through the calibration area by using only data of the sensor.

The vehicle driving control apparatus <NUM> may obtain, from the previously stored map data, information about a shape of the predetermined area of road as the information about the predetermined area of road. The vehicle driving control apparatus <NUM> may reconfigure the shape of the predetermined area of road based on the stored measured values. The vehicle driving control apparatus <NUM> may calibrate the first sensor by comparing the reconfigured shape of the predetermined area of road with the shape of the predetermined area of road obtained from the map data.

As an example, when the vehicle is driven along a driving route including a roundabout, the vehicle driving control apparatus <NUM> may determine whether the shape of the area of road, reconfigured based on the values measured by the sensors, is a closed loop shape. The vehicle driving control apparatus <NUM> may calibrate the sensor in such a manner that the shape of the area of road reconfigured based on the values measured by the sensors is a closed loop shape.

As another example, when the vehicle is driven along a driving route including a straight flat road, the vehicle driving control apparatus <NUM> may determine whether the shape of the area of road reconfigured based on the values measured by the sensors indicates a curved or inclined road. The vehicle driving control apparatus <NUM> may calibrate the sensor in such a manner that the shape of the area of road reconfigured based on the values measured by the sensors indicates a straight and flat road.

Alternatively, the vehicle driving control apparatus <NUM> may obtain information about a location of the predetermined area of road as the information about the predetermined area of road. The vehicle driving control apparatus <NUM> may calibrate the first sensor in such a manner that sensing data received from the first sensor indicates the same value as sensing data received from the other sensors, based on data measured while the vehicle is being driven through the predetermined area of road.

Upon determining that calibration of two or more sensors is required, the vehicle driving control apparatus <NUM> may apply various correction values to the sensors determined to require calibration. The vehicle driving control apparatus <NUM> may find an optimally matching correction value within a preset error range from map data on which results obtained by applying various correction values to sensors are previously stored and calibrate the sensors by using the found correction value.

In addition, the vehicle driving control apparatus <NUM> may control the vehicle to be additionally autonomously driven for calibration during a driver's absence, with permission of the user.

The vehicle driving control apparatus <NUM> may provide, to the user, a UI for requesting permission of the user for additional driving for calibration. The vehicle driving control apparatus <NUM> may receive an input of the user in response to the UI. The vehicle driving control apparatus <NUM> may control the vehicle to be additionally driven for calibration after driving intended by the user is terminated. The vehicle driving control apparatus <NUM> may store values additionally measured by the plurality of sensors while the vehicle is being additionally driven. The vehicle driving control apparatus <NUM> may calibrate the first sensor based on the measured values stored while the vehicle is being driven along the determined driving route, the information about the predetermined area of road, and the additionally measured values.

<FIG> illustrates an example of a UI provided by a vehicle driving control apparatus to request permission of a user for additional driving for calibration, according to an embodiment of the disclosure.

Referring to <FIG>, the vehicle driving control apparatus <NUM> may display, on a touchscreen <NUM>, a GUI <NUM> for requesting permission of the user for additional driving for calibration. The vehicle driving control apparatus <NUM> may control the vehicle to be additionally driven for calibration after driving intended by the user is terminated, based on a touch input of the user on a button <NUM> for permitting additional driving.

The vehicle driving control apparatus <NUM> may increase sensor calibration accuracy by additionally collecting sensing data through additional driving.

The vehicle driving control apparatus <NUM> may update the previously stored map data based on the values measured by the plurality of sensors while the vehicle is being driven. The vehicle driving control apparatus <NUM> may continuously collect the sensing data or the map data reconfigured based on the sensing data and update the stored map data based on the collected information.

The vehicle driving control apparatus <NUM> may delete an indication of a calibration area where sensor calibration characteristics are lost. For example, the vehicle driving control apparatus <NUM> may delete an indication of a calibration area where lane information is lost. Alternatively, the vehicle driving control apparatus <NUM> may add an indication of a calibration area where clear sensor calibration characteristics are newly determined.

<FIG> is a detailed flowchart of a calibration method performed by a vehicle driving control apparatus, according to an embodiment of the disclosure.

Referring to <FIG>, operations S410 and S420 correspond to operation S310 of <FIG>, operation S430 of <FIG> corresponds to operation S320 of <FIG>, operations S440 and S450 of <FIG> correspond to operation S330 of <FIG>, and operation S460 of <FIG> corresponds to operation S340 of <FIG>. The above description provided in relation to <FIG> may also be applied to corresponding operations of <FIG>. A repeated description will not be provided herein.

In operation S410, the vehicle driving control apparatus <NUM> may receive sensing data from a plurality of sensors mounted in the vehicle. The vehicle driving control apparatus <NUM> may periodically receive the sensing data from the plurality of sensors, and control driving of the vehicle based on the received sensing data.

In operation S420, the vehicle driving control apparatus <NUM> may determine whether calibration of a sensor is required. The vehicle driving control apparatus <NUM> may determine the sensor requiring calibration based on a driving history. The vehicle driving control apparatus <NUM> may determine the sensor requiring calibration by comparing the sensing data received from the sensors.

In operation S430, upon determining in operation S420 that calibration of at least one sensor is required, the vehicle driving control apparatus <NUM> may determine a driving route to include a predetermined area of road (e.g., a predetermined calibration area) with respect to the at least one sensor. In operation S440, the vehicle driving control apparatus <NUM> may control the vehicle to be driven along the driving route determined in operation S430. In operation S450, the vehicle driving control apparatus <NUM> may store values measured by the plurality of sensors while the vehicle is being driven. In operation S460, the vehicle driving control apparatus <NUM> may calibrate the sensor based on the stored measured values and information about a predetermined area of road. For example, the vehicle driving control apparatus <NUM> may calibrate the at least one sensor based on data measured while the vehicle is being driven through the calibration area.

On the contrary, in operation S470, upon determining in operation S420 that no sensor requires calibration, the vehicle driving control apparatus <NUM> may determine an optimal driving route to a destination. In operation S480, the vehicle driving control apparatus <NUM> may control the vehicle to be driven along the driving route determined in operation S470.

Therefore, the vehicle driving control apparatus <NUM> may minimize errors of sensors by continuously calibrating the sensors while the vehicle is being driven. Furthermore, the vehicle driving control apparatus <NUM> may increase calibration efficiency and minimize inconvenience of a user by performing calibration based on map data without inconveniently visiting a specific place for calibration.

In addition, the vehicle driving control apparatus <NUM> may improve safety of the autonomous vehicle by controlling the vehicle to be repeatedly driven through a calibration area until accuracy of calibration reaches a desired level. For example, when the vehicle is controlled, with permission of the user, to be repeatedly driven through a driving route including both a roundabout and a straight flat road like the driving route <NUM> of <FIG>, the vehicle driving control apparatus <NUM> may maximize calibration accuracy.

In addition, the vehicle driving control apparatus <NUM> may provide convenience to the user by autonomously performing calibration without a command of the user by controlling the vehicle to be additionally driven through a calibration area even after driving intended by the user is terminated.

As described above in relation to <FIG>, the vehicle driving control apparatus <NUM> may determine a driving route to include a calibration area, by storing map data. However, embodiments of the disclosure are not limited to the above description. The vehicle driving control apparatus <NUM> may be connected to a navigation server and receive, from the navigation server, a driving route including a calibration area.

Referring to <FIG>, the autonomous driving system may include the autonomous vehicle <NUM> supporting the autonomous driving system and the vehicle driving control apparatus <NUM> for controlling driving of the autonomous vehicle <NUM>. The autonomous vehicle <NUM> may have mounted therein various sensors such as an such as the IMU <NUM>, the radar <NUM>, the lidar <NUM>, and the camera <NUM>. The vehicle driving control apparatus <NUM> may be connected to a navigation server <NUM> in a wireless manner.

The navigation server <NUM> may store map data for determining a driving route of the autonomous vehicle <NUM>. The navigation server <NUM> may dynamically update the driving route while the autonomous vehicle <NUM> is being driven. For example, the navigation server <NUM> may receive, from the vehicle driving control apparatus <NUM>, data collected by a GPS module mounted in the autonomous vehicle <NUM> and use the received data to determine the driving route of the autonomous vehicle <NUM>.

The vehicle driving control apparatus <NUM> may transmit, to the navigation server <NUM>, information about a sensor requiring calibration. The navigation server <NUM> may determine the driving route to include a predetermined area of road used to calibrate the sensor and transmit the determined driving route to the vehicle driving control apparatus <NUM>. The vehicle driving control apparatus <NUM> may calibrate at least one sensor based on sensing data measured by sensors while the autonomous vehicle <NUM> is being driven.

The description provided above in relation to <FIG> may also be applied to a method, performed by the vehicle driving control apparatus <NUM>, of controlling the autonomous vehicle <NUM> and calibrating sensors. A repeated description will not be provided herein.

A calibration method performed by the vehicle driving control apparatus <NUM> will now be described with reference to <FIG>.

<FIG> is a flowchart of a calibration method performed by a vehicle driving control apparatus connected to a navigation server, according to an embodiment of the disclosure.

Referring to <FIG>, the vehicle driving control apparatus <NUM> may control the vehicle based on information received from a plurality of sensors, and receive a driving route of the vehicle from the navigation server <NUM> connected thereto in a wireless manner.

Operation S1010 of <FIG> corresponds to operation S310 of <FIG>, operations S1050 and S1060 of <FIG> correspond to operation S330 of <FIG>, and operation S1070 of <FIG> corresponds to operation S340 of <FIG>. The above description provided above in relation to <FIG> may also be applied to corresponding operations of <FIG>. A repeated description will not be provided herein.

In operation S1010, the vehicle driving control apparatus <NUM> may determine that calibration of a sensor is required.

The vehicle driving control apparatus <NUM> may receive sensing data from a plurality of sensors mounted in the vehicle. The vehicle driving control apparatus <NUM> may determine the sensor requiring calibration by comparing the sensing data received from the sensors. For example, the vehicle driving control apparatus <NUM> may determine that calibration of a first sensor among the plurality of sensors is required.

In operation S1020, the vehicle driving control apparatus <NUM> may transmit, to the navigation server <NUM>, a signal indicating that calibration is required. For example, the vehicle driving control apparatus <NUM> may transmit, to the navigation server <NUM>, information about the sensor requiring calibration. Upon determining that calibration of the first sensor is required, the vehicle driving control apparatus <NUM> may transmit, to the navigation server <NUM>, information about the first sensor as a signal indicating that calibration of the first sensor is required.

When the signal indicating that calibration of the sensor mounted in the vehicle is required is received from the vehicle driving control apparatus <NUM>, the navigation server <NUM> may determine a driving route of the vehicle based on the received signal.

In operation S1030, the navigation server <NUM> may determine a driving route to include a predetermined area of road used to calibrate the sensor, based on map data including information about an area of road used to calibrate each of the plurality of sensors. For example, the navigation server <NUM> may receive the information about the first sensor from the vehicle driving control apparatus <NUM> and determine the driving route to include a calibration area of the first sensor.

According to an embodiment of the disclosure, the area of road used to calibrate each of the plurality of sensors may include at least one of a roundabout or a straight flat road, and information about the area of road used to calibrate each of the plurality of sensors may include location information of the area of road and information about a shape of the area of road.

For example, to determine the driving route of the vehicle, when calibration of a sensor for sensing a roll about the X axis or a yaw about the Z axis of the vehicle is required, the navigation server <NUM> may determine the driving route to include a roundabout. When calibration of a sensor for sensing a pitch about the Y axis of the vehicle is required, the navigation server <NUM> may determine the driving route to include a straight flat road.

To determine the driving route from a current location of the vehicle to a destination, the navigation server <NUM> may determine the driving route in such a manner that the vehicle is driven by a distance greater than a driving distance along the shortest route by a threshold value or less. The navigation server <NUM> may control the vehicle to be driven through an area of road for calibration within a threshold range to prevent an excessively long detour for calibration.

The navigation server <NUM> may transmit, to the vehicle driving control apparatus <NUM>, the determined driving route and information about the predetermined area of road used to calibrate the sensor.

In operation S1040, the vehicle driving control apparatus <NUM> may receive the driving route determined to include the predetermined area of road, from the navigation server <NUM> in response to the signal transmitted in operation S1020. The information about the predetermined area of road may include location information of a calibration area found by the navigation server <NUM> based on the signal transmitted in operation S1020, or information about a shape of the calibration area.

In operation S1050, the vehicle driving control apparatus <NUM> may control the vehicle to be driven along the driving route provided by the navigation server <NUM>.

To control the vehicle to be driven along the driving route provided by the navigation server <NUM>, the vehicle driving control apparatus <NUM> may control the vehicle to be repeatedly driven through the calibration area, with permission of a user.

In operation S1060, the vehicle driving control apparatus <NUM> may store values measured by the plurality of sensors while the vehicle is being driven along the driving route. In operation S1070, the vehicle driving control apparatus <NUM> may calibrate the sensor based on the stored measured values and the information about the predetermined area of road. For example, the vehicle driving control apparatus <NUM> may calibrate the first sensor determined to require calibration.

The navigation server <NUM> may receive, from the vehicle driving control apparatus <NUM>, the values measured by the sensors while the vehicle is being driven along the driving route. The navigation server <NUM> may update the map data based on the received measured values.

In addition, the vehicle driving control apparatus <NUM> may control the vehicle to be additionally autonomously driven for calibration during a driver's absence, with permission of the user. The vehicle driving control apparatus <NUM> may calibrate the sensor based on the measured values stored while the vehicle is being driven along the determined driving route, and values measured while the vehicle is being additionally driven.

<FIG> is a block diagram of a vehicle driving control apparatus according to an embodiment of the disclosure.

Referring to <FIG> and <FIG>, the vehicle driving control apparatus <NUM> may include a processor <NUM> and a memory <NUM>. However, the vehicle driving control apparatus <NUM> may include a larger number of elements compared to those illustrated in <FIG>. For example, as illustrated in <FIG>, the vehicle driving control apparatus <NUM> may further include a communicator <NUM>.

Although the vehicle driving control apparatus <NUM> includes a single processor in <FIG> and <FIG>, embodiments of the disclosure are not limited thereto and the vehicle driving control apparatus <NUM> may include a plurality of processors. At least some operations and functions of the processor <NUM> described below may also be performed by a plurality of processors. The vehicle driving control apparatus <NUM> illustrated in <FIG> and <FIG> may perform the methods described above in relation to <FIG>, <FIG>, and <FIG>, and a repeated description will not be provided herein.

The processor <NUM> controls overall operations of the vehicle driving control apparatus <NUM>. For example, the processor <NUM> may control the other elements included in the vehicle driving control apparatus <NUM> (e.g., the memory <NUM>), by executing a program stored in the vehicle driving control apparatus <NUM>.

According to an embodiment of the disclosure, the processor <NUM> may control driving of the vehicle and calibrate at least one sensor based on information received from a plurality of sensors.

The processor <NUM> may determine the sensor requiring calibration by receiving sensing data from the plurality of sensors and comparing the sensing data received from the sensors. For example, upon determining that calibration of a first sensor is required, the processor <NUM> may determine a driving route to include a predetermined area of road used to calibrate the first sensor.

The processor <NUM> may store, in the memory <NUM>, values measured by the plurality of sensors while the vehicle is being driven along the driving route. The processor <NUM> may calibrate the first sensor based on the stored measured values and information about the predetermined area of road.

According to an embodiment of the disclosure, the memory <NUM> may store a program for processing and control operations of the processor <NUM>, and store data input to or to be output from the vehicle driving control apparatus <NUM>.

The memory <NUM> may store map data including information about an area of road used to calibrate each of the plurality of sensors, and values measured by the plurality of sensors while the vehicle is being driven. The memory <NUM> may store at least one of location information of a calibration area or information about a shape of the calibration area, as information about the calibration area.

The processor <NUM> may determine the driving route to include the predetermined area of road used to calibrate the sensor determined to require calibration, based on the map data.

For example, when calibration of a sensor for sensing a roll about the X axis or a yaw about the Z axis of the vehicle is required, the processor <NUM> may determine the driving route to include a roundabout. When calibration of a sensor for sensing a pitch about the Y axis of the vehicle is required, the processor <NUM> may determine the driving route to include a straight flat road.

The processor <NUM> may obtain the information about the shape of the calibration area from the memory <NUM>. The processor <NUM> may reconfigure the shape of the calibration area based on the values measured by the plurality of sensors while the vehicle is being driven along the driving route. The processor <NUM> may calibrate the first sensor by comparing the reconfigured shape of the calibration area with the shape of the calibration area obtained from the memory <NUM>.

The processor <NUM> may update the map data based on the values measured by the plurality of sensors while the vehicle is being driven.

The vehicle driving control apparatus <NUM> may further include the communicator <NUM> capable of exchanging data with the navigation server <NUM>.

Upon determining that calibration of the first sensor is required, the communicator <NUM> may transmit, to the navigation server <NUM>, a signal indicating that calibration of the first sensor is required. The communicator <NUM> may receive a driving route determined to include a predetermined area of road from the navigation server <NUM> in response to the transmitted signal.

<FIG> is a detailed block diagram of a vehicle driving control apparatus according to an embodiment of the disclosure.

Referring to <FIG>, the vehicle driving control apparatus <NUM> may include a sensor <NUM>, the processor <NUM>, the communicator <NUM>, a driving mechanism <NUM>, an outputter <NUM>, the memory <NUM>, and an inputter <NUM>.

Although the sensor <NUM> of the vehicle driving control apparatus <NUM> includes a plurality of sensors in <FIG>, embodiments of the disclosure are not limited thereto. The sensor <NUM> may be separated from the vehicle driving control apparatus <NUM>.

Not all elements illustrated in <FIG> are essential elements of the vehicle driving control apparatus <NUM>. The vehicle driving control apparatus <NUM> may include a larger or smaller number of elements compared to those illustrated in <FIG>. For example, the vehicle driving control apparatus <NUM> may include the processor <NUM> and the memory <NUM> as illustrated in <FIG>, or include the processor <NUM>, the memory <NUM>, and the communicator <NUM> as illustrated in <FIG>.

As described above, the vehicle driving control apparatus <NUM> may be a device separate from and mounted in the vehicle, a device included in the vehicle, or at least a part of an electronic device for controlling the vehicle.

The sensor <NUM> may include a plurality of sensors configured to sense information about a surrounding environment of the vehicle. For example, the sensor <NUM> may include at least one of a location sensor <NUM> (e.g., a global positioning system (GPS), a differential GPS (DGPS), an inertial navigation system (INS), or the like), an inertial measurement unit (IMU) <NUM>, a lidar sensor <NUM>, a radar sensor <NUM>, an image sensor <NUM> (e.g., a camera, stereo camera, a mono camera, a wide angle camera, an around view camera, a three-dimensional (3D) vision sensor, or the like), an ultrasonic sensor <NUM>, an infrared sensor <NUM>, a distance sensor <NUM>, a temperature/humidity sensor <NUM>, a red, green, and blue (RGB) sensor <NUM>, or a motion sensor <NUM>, but is not limited thereto. For example, the sensor <NUM> may also include a barometric pressure sensor, a dust sensor, or the like.

The motion sensor <NUM> may be used to sense motion of the vehicle, and include at least one of, for example, a magnetic sensor <NUM>, an acceleration sensor <NUM>, or a gyroscope sensor <NUM>, but is not limited thereto.

The image sensor <NUM> may include a plurality of cameras provided at a plurality of locations inside or outside the vehicle. For example, three cameras may be provided at a front part of the vehicle, one camera may be provided at a rear part of the vehicle, two cameras may be provided at a left part of the vehicle, and two cameras may be provided at a right part of the vehicle, but the numbers and the locations of the cameras are not limited thereto.

The sensor <NUM> may be configured in the form of a combination of the image sensor <NUM> and the radar sensor <NUM>, or a combination of the image sensor <NUM> and the lidar sensor <NUM>. Functions of the sensors may be intuitively inferred from their names by one of ordinary skill in the art, and thus detailed descriptions thereof will not be provided herein.

The processor <NUM> may control driving of the vehicle based on information received from the plurality of sensors included in the sensor <NUM>. The processor <NUM> may control overall operations of the vehicle driving control apparatus <NUM>. The processor <NUM> may control the sensor <NUM>, the communicator <NUM>, the outputter <NUM>, the memory <NUM>, and the inputter <NUM> by executing programs stored in the memory <NUM>.

According to an embodiment of the disclosure, the processor <NUM> may include an artificial intelligence (AI) processor and, in this case, the processor <NUM> may determine a sensor requiring calibration, determine a driving route, or calibrate the sensor by using a trained network model of an Al system.

The Al processor may be produced in the form of a dedicated hardware chip for Al, or may be produced as a part of an existing general-use processor (e.g., a central processing unit (CPU) or an application processor) or a dedicated graphic processor (e.g., a graphics processing unit (GPU)) and be mounted in the vehicle driving control apparatus <NUM>.

According to an embodiment of the disclosure, upon determining that calibration of a sensor is required, the processor <NUM> may determine a driving route to include a calibration area, and calibrate the sensor based on values measured by the plurality of sensors while the vehicle is being driven along the driving route, and information about the calibration area.

The communicator <NUM> may include one or more elements for communication between the vehicle and an external entity. For example, the communicator <NUM> may include a short-range wireless communicator <NUM>, a mobile communicator <NUM>, and a broadcast receiver <NUM>, but is not limited thereto.

The short-range wireless communicator <NUM> may include, for example, a Bluetooth communicator, a Bluetooth low energy (BLE) communicator, a near field communicator, a wireless local area network (WLAN) (or Wi-Fi) communicator, a Zigbee communicator, an infrared data association (IrDA) communicator, a Wi-Fi direct (WFD) communicator, an ultra-wideband (UWB) communicator, an Ant+ communicator, and a microwave (µWave) communicator, but is not limited thereto.

The communicator <NUM> may transmit, to the navigation server <NUM>, a signal indicating that calibration of a sensor is required and receive a driving route determined to include a calibration area from the navigation server <NUM> in response to the transmitted signal.

The driving mechanism <NUM> may include elements used to drive the vehicle and operate devices in the vehicle. The driving mechanism <NUM> may include at least one of a power supply <NUM>, a propulsion system <NUM>, a running gear system <NUM>, or peripheral devices <NUM>, but is not limited thereto.

The peripheral devices <NUM> may include a navigation system, lamps, direction indicators, windshield wipers, internal lighting, a heater, and an air-conditioner. The navigation system may be a system configured to determine a driving route of the vehicle. The navigation system may be configured to dynamically update the driving route while the vehicle is being driven. For example, the navigation system may use data collected by a GPS module to determine the driving route of the vehicle.

The outputter <NUM> may be used to output an audio signal, a video signal, or a vibration signal, and include, for example, a display <NUM>, a sound outputter <NUM>, and a vibration motor <NUM>. The outputter <NUM> may output information processed by the vehicle driving control apparatus <NUM>. For example, the outputter <NUM> may provide a UI for requesting permission of a user for repeated driving for calibration, or a UI for requesting permission of the user for additional driving for calibration.

The display <NUM> may display information processed by the vehicle driving control apparatus <NUM>. For example, the display <NUM> may display a map including the driving route, display locations of external vehicles, display blind spots of drivers of the external vehicles, or display a current speed, a remaining amount of fuel, or information for guiding the driving route of the vehicle, but is not limited thereto.

When the display <NUM> and a touchpad are layered to configure a touchscreen, the display <NUM> may be used not only as an output device but also as an input device. The vehicle driving control apparatus <NUM> may include two or more displays <NUM> depending on implementation of the vehicle driving control apparatus <NUM>.

The sound outputter <NUM> may output audio data received from the communicator <NUM> or stored in the memory <NUM>. The sound outputter <NUM> may output a sound signal related to a function performed by the vehicle. For example, the sound outputter <NUM> may output a voice message for guiding the driving route of the vehicle, or a voice message for requesting permission of the user for repeated driving (or additional driving). The sound outputter <NUM> may include, for example, a speaker and a buzzer.

The vibration motor <NUM> may output a vibration signal. For example, the vibration motor <NUM> may output a vibration signal corresponding to output of audio or video data (e.g., a warning message).

The memory <NUM> may store programs for processing and control operations of the processor <NUM>, and store input/output data. The memory <NUM> may include at least one of flash memory, a hard disk, a multimedia card micro, a memory card (e.g., a secure digital (SD) or extreme digital (XD) card), random access memory (RAM), static RAM (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), magnetic memory, a magnetic disc, or an optical disc. The vehicle driving control apparatus <NUM> may use a web storage or a cloud server which performs a storage function on the Internet.

The inputter <NUM> refers to a means used by the user to input data for controlling the vehicle or the vehicle driving control apparatus <NUM>. For example, the inputter <NUM> may include a keypad, a dome switch, a touchpad (e.g., a capacitive overlay, a resistive overlay, an infrared beam, a surface acoustic wave, an integral strain gauge, or a piezoelectric touchpad), a jog wheel, or a jog switch, but is not limited thereto.

The inputter <NUM> may receive an input of the user in response to the UI for requesting permission of the user for repeated driving (or additional driving) for calibration.

<FIG> is a block diagram of a navigation server according to an embodiment of the disclosure.

Referring to <FIG>, the navigation server <NUM> may include a communicator <NUM>, a processor <NUM>, and a memory <NUM>. Although the navigation server <NUM> includes a single processor in <FIG>, embodiments of the disclosure are not limited thereto and the navigation server <NUM> may include a plurality of processors. At least some of operations and functions of the processor <NUM> described below may also be performed by a plurality of processors. The navigation server <NUM> illustrated in <FIG> may perform the method described above in relation to <FIG>, and a repeated description will not be provided herein.

The processor <NUM> controls overall operations of the navigation server <NUM>. For example, the processor <NUM> may control other elements included in the navigation server <NUM> (e.g., the communicator <NUM> and the memory <NUM>), by executing a program stored in the navigation server <NUM>.

The communicator <NUM> may receive, from the vehicle driving control apparatus <NUM>, a signal indicating that calibration of a sensor mounted in the vehicle is required. For example, the communicator <NUM> may receive a signal indicating that calibration of a first sensor is required.

The processor <NUM> may determine a driving route to include a predetermined area of road used to calibrate the first sensor, based on map data including information about an area of road used to calibrate each of a plurality of sensors.

The processor <NUM> may determine the driving route including a calibration area in such a manner that the vehicle is driven by a distance greater than a driving distance along the shortest route to a destination by a threshold value or less.

The communicator <NUM> may transmit the driving route and information about the predetermined area of road to the vehicle driving control apparatus <NUM>.

The memory <NUM> may store the map data including the information about the area of road used to calibrate each of the plurality of sensors. The area of road used to calibrate each of the plurality of sensors may include at least one of a roundabout or a straight flat road. The information about the area of road used to calibrate each of the plurality of sensors may include at least one of location information of the area of road or information about a shape of the area of road.

The communicator <NUM> may receive, from the vehicle driving control apparatus <NUM>, values measured by the sensors mounted in the vehicle while the vehicle is being driven along the driving route. The processor <NUM> may update the map data based on the received measured values.

The afore-described embodiments of the disclosure may be implemented by a software program including instructions stored in a computer-readable storage medium.

The computer is a device capable of fetching instructions stored in the storage medium and operating according to the embodiments of the disclosure based on the fetched instructions. The computer may include an image transmission device and an image reception device according to the embodiments of the disclosure.

The computer-readable storage medium may be provided in the form of a non-transitory storage medium. When the storage medium is 'non-transitory', it means that the storage medium does not include signals and is tangible, and it does not limit that data is semi-permanently or temporarily stored in the storage medium.

An electronic device or a method according to the embodiments of the disclosure may be included and provided in a computer program product. The computer program product may be traded between a seller and a buyer.

The computer program product may include a software program, and a computer-readable storage medium storing the software program. For example, the computer program product may include a product (e.g., a downloadable application) that is electronically distributed in the form of a software program by a manufacturer of the electronic device or an electronic market (e.g., Google Play or App Store). For electronic distribution, at least a part of the software program may be stored in a storage medium or be temporarily created. In this case, the storage medium may be a server of the manufacturer, a server of the electronic market, or a storage medium of a relay server that temporarily stores the software program.

The computer program product may include a storage medium of a server or a storage medium of a device (e.g., a vehicle driving control apparatus or a navigation server) in a system including the server and the device. Otherwise, when a third device (e.g., a smartphone) connected to the server or the device is present, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include a software program itself transmitted from the server to the device or the third device, or from the third device to the device.

In this case, one of the server, the device, and the third device may execute the computer program product to perform the method according to the embodiments of the disclosure. Alternatively, two or more of the server, the device, and the third device may execute the computer program product to perform the method according to the embodiments of the disclosure in a distributed fashion.

For example, the server (e.g., a cloud server or an Al server) may execute the computer program product stored in the server, to control the device connected to the server, to perform the method according to the embodiments of the disclosure.

As another example, the third device may execute the computer program product to control the device connected to the third device, to perform the method according to the embodiments of the disclosure.

When the third device executes the computer program product, the third device may download the computer program product from the server and execute the downloaded computer program product. Alternatively, the third device may execute the computer program product provided in a preloaded state, to perform the method according to the embodiments of the disclosure.

Claim 1:
A vehicle driving control apparatus (<NUM>) for controlling driving of a vehicle (<NUM>) based on information received from a plurality of sensors (<NUM>), the vehicle driving control apparatus (<NUM>) characterized by comprising:
at least one processor (<NUM>) configured to determine a driving route to comprise a predetermined area of road, upon determining that calibration of a first sensor (<NUM>) is required; and
a memory (<NUM>) configured to store values measured by the plurality of sensors (<NUM>) while the vehicle (<NUM>) is being driven along the driving route,
wherein the at least one processor (<NUM>) is further configured to calibrate the first sensor (<NUM>) based on the stored measured values and information about the predetermined area of road.