Patent ID: 12202334

DETAILED DESCRIPTION

Hereinafter, some forms of the present disclosure will be described in detail with reference to the exemplary drawings. In adding the reference numerals to the components of each drawing, it should be noted that the identical or equivalent component is designated by the identical numeral even when they are displayed on other drawings. Further, in describing some forms of the present disclosure, a detailed description of the related known configuration or function will be omitted when it is determined that it interferes with the understanding of some forms of the present disclosure.

In describing some forms of the present disclosure, terms such as first, second, A, B, (a), (b), and the like may be used. These terms are merely intended to distinguish the components from other components, and the terms do not limit the nature, order or sequence of the components. Unless otherwise defined, all terms including technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that tams, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

In the present specification, a foremost vehicle of a string of platooning vehicles is referred to as a leading vehicle LV, a vehicle following the leading vehicle LV is referred to as a following vehicle FV, a vehicle right in front of a vehicle is referred to as a preceding vehicle, and the leading vehicle LV and the following vehicle FV are collectively referred to as platooning vehicles.

The present disclosure relates to a technology in which the following vehicle outputs an image of a region ahead recorded by the leading vehicle during the platooning when a recording location of the image of the region ahead and a vehicle location are the same rather than outputs the image as it is to synchronize a behavior of the vehicle with a movement of the image.

FIG.1is a block diagram illustrating a vehicle control device in some forms of the present disclosure.

Referring toFIG.1, a vehicle control device100may be mounted on the vehicle to control the platooning. The vehicle control device100may be implemented as at least one electric control unit (ECU).

The vehicle control device100may include a communication device110, a camera120, a detection device130, a positioning device140, storage150, a user input device160, a display170, a travel controller180, and a processor190.

The communication device110may support wireless communication between the vehicle and an external device (e.g., an ECU mounted on another vehicle in the platoon, a mobile terminal, a server, or the like). The communication device110may use a vehicle to everything (V2X) communication technology such as a vehicle to vehicle (V2V) communication and/or a vehicle to infrastructure (V2I) communication. For example, the communication device110may use at least one of communication technologies such as a wireless Internet (e.g., a Wi-Fi), a short-range communication (e.g., a Bluetooth, a ZigBee, and an infrared communication), a mobile communication, and/or the like to transmit and receive a V2V communication message.

The communication device110may support the vehicle control device100to communicate with the electric control units (ECUs) in the vehicle connected through an in-vehicle network (IVN). As an in-vehicle communication, a controller area network (CAN) communication, a media oriented systems transport (MOST) communication, a local interconnect network (LIN) communication, an X-by-Wire (a Flexray) communication, or the like may be used. The communication device110may include a communication processor, a communication circuit, an antenna, and/or a transceiver.

The camera120may be mounted on the vehicle to capture the image of the region ahead. The image of the region ahead may include at least one frame captured for a predetermined time. The camera120may include at least one of sensors such as a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, a charge priming device (CPD) image sensor, and/or a charge injection device (CID) image sensor. The camera120may include at least one of lenses such as a standard lens, an ultra wide angle lens, a wide angle lens, a zoom lens, a macro lens, a telephoto lens, a fisheye lens, a semi-fisheye lens, and/or the like. The camera120may include an image processor for performing image processing such as noise elimination, color reproduction, file compression, image quality adjustment, saturation adjustment, and the like on an image acquired through the image sensor.

The detection device130may sense a front vehicle and obtain front vehicle information using sensors mounted on the vehicle. The detection device130may sense a behavior of the front vehicle using at least one of sensors such as an advanced driver assistance system (ADDS) sensor, a light detection and ranging (LiDAR), a radio detecting and ranging (RADAR), an image sensor, an ultrasonic sensor, and/or the like. The front vehicle information may include a speed, an acceleration, and/or a distance of the front vehicle.

The detection device130may use the sensors (e.g., a speed sensor, an acceleration sensor, and/or an inertial sensor) and/or other ECUs to detect state information, behavior information, and/or travel environment information of the vehicle.

The positioning device140may measure a current location of the vehicle. The positioning device140may measure the vehicle location using at least one of positioning technologies such as a global positioning system (GPS), a dead reckoning (DR), a differential GPS (DGPS), a carrier phase differential GPS (CDGPS), and/or the like. When using the GPS, the positioning device140may calculate the current location of the vehicle (the vehicle location) using triangulation.

The storage150may be a non-transitory storage medium that stores instructions executed by the processor190. The storage150may store map information (map data), the front vehicle information, and the like, and may store the communication message received through the communication device110. Further, the storage150may store the image of the region ahead captured by the camera120. The storage150may be implemented as at least one of storage media (recording media) of a flash memory, a hard disk, a secure digital card (SD card), a random access memory (RAM), a static random access memory (SRAM), a read only memory (ROM), a programmable read only memory (PROM), an electrically erasable and programmable ROM (EEPROM), an erasable and programmable ROM (EPROM), a register, a removable disk, and/or web storage.

The user input device160may generate data resulted from a manipulation of a user. For example, the user input device160may generate data for turning on or off a platooning function in response to a user input. The user input device160may include at least one of input devices such as a button, a switch, a touch pad, and/or a touch screen. The input device is disposed on a steering wheel, a dashboard, a center fascia, and/or a door trim.

The display170may output a progress status and/or a result based on an operation of the processor190. The display170may output (display) the image of the region ahead provided from the leading vehicle in response to an instruction of the processor190. The display170may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT-LCD), an organic light-emitting diode (OLED) display, a flexible display, a three-dimensional display (3D display), a transparent display, a head-up display (HUD), a touch screen, and/or a cluster.

The display170may include an audio output device such as a speaker capable of outputting audio data. For example, the display170may display start or end of the platooning, and may also output an audio signal through the speaker.

The travel controller180, which controls steering, braking, suspension, and/or driving of the vehicle, may include a driving device (a power transmission device), a steering device, a suspension device, a braking device, and the like. The driving device, which controls the driving of the vehicle, may transmit power generated from a power source (e.g., an engine or a motor) to a wheel. The driving device may be implemented as a traction control system (TCS) and/or an all wheel drive system (AWD). The steering device may change a traveling direction of the vehicle during the travel. The steering device may be implemented as a four wheel steering system (4 WS), an electric power steering (EPS), an active front steering (AFS), and/or a steer by wire (SBW). The suspension device may reduce a vibration and a shock generated on a road surface by connecting a vehicle body and an axle with each other, and maintain a posture of the vehicle. The suspension device may be composed of a spring that mitigates the shock transmitted from the road surface, a damper that suppresses a free vibration, and a stabilizer bar that suppresses a roll movement to improve travel stability of the vehicle. In addition, the suspension device may actively change a height of the vehicle body as well as a damping force of the damper and rigidity of the stabilizer bar based on a travel environment. The braking device may decelerate or stop the vehicle. The braking device may monitor a vehicle state during the travel in real time and control the braking based on the vehicle state. The braking device may include an anti-lock braking system (ABS), an electronic stability control (ESC), and/or an electronic parking brake (EPB) system.

The processor190may control overall operations of the vehicle control device100. The processor190may include at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a microcontroller, and/or a microprocessor.

The processor190may form the platoon with at least one other vehicle through a join process for the platooning of the vehicle. Because the process of forming the platoon may be implemented using at least one of known technologies, a detailed description thereof will be omitted. The processor190may cause the vehicle to start the platooning when the platoon formation is completed.

When the vehicle is the leading vehicle in the platoon, the processor190may capture the image of the region ahead using the camera120mounted on the vehicle. The processor190may obtain the current location of the vehicle (the vehicle location) using the positioning device140when capturing the image of the region ahead. The processor190may insert or tag a vehicle location at an image capturing time point in the captured image of the region ahead. The processor190may insert the vehicle location of the capturing time point for each frame of the image of the region ahead when capturing the image of the region ahead.

In addition, the processor190may obtain feature information centering on a side portion of a screen of the camera120through the camera, the lidar, and/or the radar when capturing the image of the region ahead. The processor190may obtain feature information of a region peripheral to a location corresponding to a side portion of the image frame using the camera120and/or the detection device130. The processor190may insert the feature information into the captured image frame. When obtaining the feature information, because a center of the screen of the camera120is covered by the front vehicle (the preceding vehicle), the feature information of the peripheral region centering on the side portion of the screen is obtained.

The processor190may determine whether an emergency situation has occurred in consideration of the vehicle state, the travel environment, and/or the will of the driver obtained by the detection device130. The processor190may determine a platooning state as an unstable state when a risk of the emergency situation (e.g., sudden braking and/or a falling object ahead) occurred does not satisfy a platoon releasing condition but requires an active response. In addition, the processor190may determine the platooning state as an unstable state when a communication performance is within an allowable range, but is lowered by a value equal to or greater than a predetermined reference value. The processor190may determine the platooning state as a stable state when the communication performance is within the allowable range and no emergency situation has occurred. The allowable range, which is a communication performance range required to maintain the platooning, may be set in advance by a system designer.

The processor190may monitor a transmission performance of the communication device110in real time. The processor190may transmit (deliver) the image of the region ahead, platooning state information, and/or transmission performance information to the following vehicle. The image of the region ahead may include information on the vehicle location of the capturing time point and/or the feature information of the peripheral region.

The processor190may receive the image of the region ahead from the leading vehicle when the vehicle is the following vehicle in the platoon. When receiving the image of the region ahead, the processor190may receive a leading vehicle location at the time point at which the image of the region ahead is captured, the platooning state information, and/or the transmission performance information together. The processor190may monitor a reception performance of the communication device110in real time when receiving the image of the region ahead.

The processor190may obtain vehicle location information (following vehicle location information) through the detection device130and/or the positioning device140.

When the platooning state is stable, the processor190may synchronize the image of the region ahead to a following vehicle location based on preceding vehicle location information included in the image of the region ahead. The processor190may output (display) the synchronized image of the region ahead on the display170. In addition, the processor190may also display the platooning state information.

When the platooning state is unstable, the processor190may output the received image on the display170without performing the synchronization processing for the received image of the region ahead. The processor190may also output a warning sound and/or a warning message indicating the emergency situation.

FIG.2is a flowchart illustrating an operation process of a leading vehicle in some forms of the present disclosure.

When the platoon formation is completed, the leading vehicle may start the platooning (S110).

When the platooning starts, the leading vehicle may monitor the platooning state using the camera120and/or the detection device130while capturing the image of the region ahead using the camera120(S120). The leading vehicle may detect the leading vehicle location of the image capturing time point using the positioning device140when capturing the image of the region ahead. The leading vehicle may insert the leading vehicle location into the captured image of the region ahead.

The leading vehicle may transmit information on the image of the region ahead and the platooning state information to the following vehicle (S130). The leading vehicle may check the transmission performance of the communication device110and transmit the transmission performance together with the information on the image of the region ahead and the platooning state information.

The leading vehicle may determine whether the platooning is ended after transmitting the information on the image of the region ahead and the platooning state information to the following vehicle (S140). The leading vehicle may continuously capture the image of the region ahead until the platooning is ended and transmit the image of the region ahead to the following vehicle along with the platooning state information.

FIG.3is a flowchart illustrating an operation process of a following vehicle in some forms of the present disclosure.

The following vehicle may start the platooning together with the leading vehicle after the platoon formation (S210).

The following vehicle may receive the information on the image of the region ahead and the platooning state information from the leading vehicle (S220). The processor190of the following vehicle may receive the information on the image of the region ahead and the platooning state information transmitted from the leading vehicle through the communication device110. The processor190may monitor the reception performance of the communication device110in real time.

The following vehicle may determine whether the platooning state is the stable state based on the platooning state information (S230).

When the platooning state is the stable state, the following vehicle may perform the synchronization processing on the information on the image of the region ahead based on the vehicle location information (S240). The processor190may perform the synchronization processing on the image of the region ahead such that the image of the region ahead is able to be displayed by the following vehicle at a time point at which the following vehicle location measured by the positioning device140and a capturing time point location included in the information on the image of the region ahead, that is, a preceding vehicle location match.

The following vehicle may output the synchronized image of the region ahead on the display170(S250). The processor190may display the image of the region ahead on the display170when the following vehicle location and the capturing time point location of the image of the region ahead match.

The following vehicle may determine whether the platooning is ended while outputting the synchronized image of the region ahead (S260). The following vehicle may repeatedly perform the operations after S220until the platooning is ended.

When the platooning state is not the stable state in S230, the following vehicle may immediately output the received image of the region ahead on the display170without the synchronization processing (S270). When the platooning state is identified as the unstable state, the processor190outputs the image of the region ahead provided from the leading vehicle as it is on the display170, so that the user of the following vehicle is able to identify the image of the region ahead.

FIG.4is a diagram for illustrating a synchronizing method in some forms of the present disclosure.

As shown inFIG.4, in a situation in which a following vehicle FV1is platooning along the leading vehicle LV, the image of the region ahead must be played after the following vehicle FV1moves by a distance of a sum of a length d1of the leading vehicle LV and a distance d2between the leading vehicle LV and the following vehicle FV1to allow the image of the region ahead displayed on the following vehicle FV1to show the same behavior at the same angle of view as on the leading vehicle LV. To this end, the capturing time point location of the image of the region ahead and a location of the following vehicle FV1may be synchronized with each other. As a synchronization processing method, a method using the sensor, a method using the GPS, and/or a method using the feature information as follows may be used.

#Synchronization Method Using Sensor

The leading vehicle LV may transmit the captured image of the region ahead to the following vehicle FV1. The following vehicle FV1may measure an inter-vehicle distance with the leading vehicle LV using the sensor at a time point at which the image of the region ahead is received. The following vehicle FV1may calculate the capturing time location of the image of the region ahead (a longitudinal location) by adding a vehicle length of the leading vehicle LV to the inter-vehicle distance based on internal logic and taking into account a communication delay (a transmission delay and/or a reception delay). The image of the region ahead may be output on the display170when the following vehicle FV1reaches the capturing time point location by calculating a moved distance of the vehicle using a vehicle speed of the following vehicle FV1.

For example, when the vehicle length of the leading vehicle LV is 15 m, the distance between the leading vehicle LV and the following vehicle FV1is 15 m, the communication delay and the speed of the following vehicle FV1are 20 ms and 20 mps, respectively, the following vehicle FV1may calculate a display delay time of the image of the region ahead of 1.48 seconds (=(15+15)/20−0.02). That is, the following vehicle FV1displays the image of the region ahead after the delay of 1.48 seconds after receiving the image of the region ahead from the leading vehicle LV, so that the capturing location of the image of the region ahead and the location of the following vehicle FV1may be synchronized with each other.

#Synchronization Method Using GPS

The leading vehicle LV may obtain leading vehicle location information of the image capturing time point (the capturing time point location) using the GPS when capturing the image of the region ahead and insert the leading vehicle location information of the image capturing time point into the image frame. The leading vehicle LV may transmit the image of the region ahead including the capturing time point location (an image capturing location) information to following vehicles FV1and FV2. When receiving the image of the region ahead provided from the leading vehicle LV, the following vehicle FV1or FV2compares a current location thereof being tracked in real time with the image capturing location included in the image of the region ahead. The following vehicle FV1or FV2may display the image of the region ahead at a time point at which the two locations match.

#Synchronization Method Using Feature Information

The leading vehicle LV may detect feature information of a peripheral region centering on a front-side portion using the sensors such as the camera and/or the lidar when capturing the image of the region ahead, insert the feature information of the peripheral region of the front-side portion into the image frame, and then transmit the frame to the following vehicle FV1or FV2. The following vehicle FV1or FV2may detect the feature information of the peripheral region of the front-side portion using the sensors such as the camera and/or the lidar, compare the feature information of the peripheral region of the front-side portion with the feature information of the peripheral region included in the image of the region ahead, and display the image of the region ahead at a time point at which the feature information of the two peripheral regions match.

FIG.5is a block diagram illustrating a computing system that executes a method for displaying platooning information of a vehicle control device in some forms of the present disclosure.

Referring toFIG.5, a computing system1000may include at least one processor1100, a memory1300, a user interface input device1400, a user interface output device1500, storage1600, and a network interface1700, which are connected with each other via a bus1200.

The processor1100may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory1300and/or the storage1600. The memory1300and the storage1600may include various types of volatile or non-volatile storage media. For example, the memory1300may include a read only memory (ROM)1310and a random access memory (RAM)1320.

Thus, the operations of the method or the algorithm described in some forms of the present disclosure may be embodied directly in hardware or a software module executed by the processor1100, or in a combination thereof. The software module may reside on a storage medium (that is, the memory1300and/or the storage1600) such as a RAM, a flash memory, a ROM, an EPROM, an EEPROM, a register, a hard disk, a removable disk, and a CD-ROM. The exemplary storage medium may be coupled to the processor1100, and the processor1100may read information out of the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with the processor1100. The processor1100and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside within a user terminal. In another case, the processor1100and the storage medium may reside in the user terminal as separate components.

Hereinabove, although the present disclosure has been described with reference to exemplary forms and the accompanying drawings, the present disclosure is not limited thereto, but may be variously modified and altered by those skilled in the art to which the present disclosure pertains without departing from the spirit and scope of the present disclosure claimed in the following claims. Therefore, the exemplary forms of the present disclosure are provided to explain the spirit and scope of the present disclosure, but not to limit them, so that the spirit and scope of the present disclosure is not limited by the forms of the present disclosure. The scope of the present disclosure should be construed on the basis of the accompanying claims, and all the technical ideas within the scope equivalent to the claims should be included in the scope of the present disclosure.

According to the present disclosure, the image of the region ahead provided from the leading vehicle is received during the platooning and displayed in synchronization with the vehicle location, so that the cognitive dissonance felt by the user of the following vehicle may be reduced.

In addition, according to the present disclosure, the image of the region ahead of the leading vehicle is captured during the platooning and shared with the following vehicle, so that the user (the driver) of the following vehicle may predict and prepare for the behavior of the vehicle through the shared image of the region ahead.

Hereinabove, although the present disclosure has been described with reference to exemplary forms and the accompanying drawings, the present disclosure is not limited thereto, but may be variously modified and altered by those skilled in the art to which the present disclosure pertains without departing from the spirit and scope of the present disclosure claimed in the following claims.