Vehicle control device

A vehicle control device is disclosed. The vehicle control device includes: an interface unit that is connected to a camera for capturing a vehicle passenger and to a plurality of devices installed inside the vehicle; and a processor that receives an image captured by the camera through the interface unit and exchanges information with the plurality of devices through the interface unit, wherein the processor detects the position of the passenger from the image acquired by the camera, configures a space adjacent to the passenger as a virtual control space, matches a device corresponding to the position of the passenger, among the plurality of devices, to the virtual control device, and, upon detecting a preset gesture through the virtual control space, controls the device based on the preset gesture.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage application under 35 U.S.C. § 371 of International Application No. PCT/KR2019/003072, filed on Mar. 15, 2019, the disclosure of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The present invention relates to a method for controlling an autonomous vehicle.

BACKGROUND ART

A vehicle may be classified as an internal combustion engine vehicle, an external combustion engine vehicle, a gas turbine vehicle, or an electric vehicle depending on the type of motor used.

In recent years, smart vehicles have been actively developed for the safety or convenience of drivers, pedestrians, etc., and active research is ongoing on sensors mounted on smart vehicles. Cameras, infrared sensors, radar, GPS, Lidar, gyroscopes, etc. are being used in smart vehicles, among which cameras serve to substitute for human eyes.

Due to the development of various types of sensors and electronic equipment, vehicles with functions for assisting a passenger in driving and improving the safety and convenience of driving are attracting attention.

Notably, remote control of a device inside a vehicle by a driver's gesture is becoming an issue.

DISCLOSURE

Technical Problem

An aspect of the present invention is to provide a vehicle control device equipped in a vehicle.

Another aspect of the present invention is to provide a vehicle control device that forms a virtual control space to allow a driver to control a device inside a vehicle by gesture while keeping their eyes ahead.

Another aspect of the present invention is to provide a vehicle control device that varies a target device based on a passenger twisting the upper body.

Another aspect of the present invention is to provide a vehicle control device that remotely controls a device inside a vehicle by detecting a passenger's gesture.

Technical Solution

An exemplary embodiment of the present invention provides a vehicle control device including: an interface unit that is connected to a camera for capturing a vehicle passenger and to a plurality of devices installed inside the vehicle; and a processor that receives an image captured by the camera through the interface unit and exchanges information with the plurality of devices through the interface unit, wherein the processor detects the position of the passenger from the image acquired by the camera, configures a space adjacent to the passenger as a virtual control space, matches a device corresponding to the position of the passenger, among the plurality of devices, to the virtual control device, and, upon detecting a preset gesture through the virtual control space, controls the device based on the preset gesture.

The passenger may be positioned in a driver's seat, and a plurality of devices corresponding to the position of the passenger may be provided in front of the passenger, wherein the interface unit may be connected to a voice recognition unit installed inside the vehicle, and the processor may receive the passenger's voice recognized by the voice recognition unit through the interface unit, select one device corresponding to the position of the passenger from among the plurality of devices based on the passenger's voice, and match the selected device to the virtual control space.

The interface unit may be connected to a button provided inside the vehicle, and the processor may activate the voice recognition unit upon receiving an input into the button through the interface unit.

If the preset gesture is not detected from the virtual control space for a preset period of time after controlling the device based on the preset gesture, the processor may undo the matching of the selected device and the virtual control space.

Upon detecting a preset gesture through the virtual control space and detecting that the passenger's body is partially outside the virtual control space, the processor may control the device to correspond to the detected gesture and undo the matching of the selected device and the virtual control space.

The vehicle may include an assistant driver's seat spaced apart from the driver's seat, and the virtual control space may be positioned between the driver's seat and the assistant driver's seat.

The passenger may include a first passenger and a second passenger, and, if a device corresponding to the position of the first passenger and a device corresponding to the position of the second passenger are the same, and the first passenger is matched with the device earlier than the second passenger, the processor may give priority to the first passenger's gesture and control the device according to the priority.

The processor may configure the virtual control space in front of the first passenger.

The interface unit may be connected to a voice recognition unit that is installed inside the vehicle and detects the direction of a voice, and the processor may acquire the first passenger's voice and the first passenger's utterance position, sense the second passenger's voice based on the first passenger's utterance position, and perform noise cancelling on the second passenger's voice.

The device may be positioned in front of the passenger, and the virtual control space may be provided at a predetermined distance in front of the passenger and include a width corresponding to the shoulder width of the passenger.

The device may be positioned in front the passenger, the interface may be connected to a voice recognition unit installed inside the vehicle, and the processor may acquire the passenger's voice detected by the voice recognition unit through the interface unit and control the device based on the passenger's voice.

The device may be a first device, the virtual control space may be a first virtual control space, and the processor may detect a twist of the passenger's upper body from the image acquired by the camera, configure a space in front of the passenger as a second virtual control space based on the passenger twisting the upper body, match a second device corresponding to the passenger twisting the upper body, among the plurality of devices, to the second virtual control space, and, upon detecting a preset gesture from the second virtual control space, control the device based on the preset gesture.

The adjacent space may be at least one of a space positioned between the passenger and the device, a space positioned in front of the passenger, and a space positioned to one side of the passenger.

The processor may detect the passenger's field of view through the camera, pick out a plurality of devices positioned in the field of view, form a plurality of virtual control spaces respectively corresponding to the plurality of picked-out devices, match the plurality of picked-out devices and the plurality of virtual control devices, respectively, and, upon detecting the preset gesture through the plurality of virtual control spaces, control the device matched to the virtual control space where the preset gesture is detected.

The plurality of virtual control spaces may be spaced apart from one another, be positioned between the passenger and the plurality of picked-out devices, and surround the passenger.

Another exemplary embodiment of the present invention provides a vehicle control method including: detecting the position of a passenger from an image acquired by a camera for capturing a vehicle passenger; configuring a space adjacent to the passenger as a virtual control space; matching a device corresponding to the position of the passenger, among the plurality of devices, to the virtual control device, and; upon detecting a preset gesture through the virtual control space, controlling the device based on the preset gesture.

The passenger may be positioned in a driver's seat, and a plurality of devices corresponding to the position of the passenger may be provided, wherein the vehicle control method further includes: receiving the passenger's voice recognized by a voice recognition unit installed inside the vehicle; selecting one device corresponding to the position of the passenger from among the plurality of devices based on the passenger's voice; and matching the selected device to the virtual control space.

The vehicle control method may further include, upon receiving an input into a button provided inside the vehicle, activating the voice recognition unit.

The vehicle control method may further include, if the preset gesture is not detected from the virtual control space for a preset period of time after controlling the device based on the preset gesture, undoing the matching of the selected device and the virtual control space.

The vehicle control method may further include, upon detecting a preset gesture through the virtual control space and detecting that the passenger's body is partially outside the virtual control space, controlling the device to correspond to the detected gesture and undoing the matching of the selected device and the virtual control space.

The vehicle control method may further include, if the passenger includes a first passenger and a second passenger, a device corresponding to the position of the first passenger and a device corresponding to the position of the second passenger are the same, and the first passenger is matched with the device earlier than the second passenger, giving priority to the first passenger's gesture and controlling the device according to the priority.

The vehicle control method may further include: acquiring the first passenger's voice and the direction of the voice by a voice recognition unit installed inside the vehicle to detect the direction of voice; and performing noise cancelling on the second passenger's voice based on the direction of the voice.

The device may be a first device, the virtual control space may be a first virtual control space, and the vehicle control method may further include: detecting a twist of the passenger's upper body from the image acquired by the camera; configuring a space in front of the passenger as a second virtual control space based on the passenger twisting the upper body; matching a second device corresponding to the passenger twisting the upper body, among the plurality of devices, to the second virtual control space; and, upon detecting a preset gesture from the second virtual control space, controlling the device based on the preset gesture.

Advantageous Effects

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that forms a virtual control space to allow a driver to control a device inside a vehicle by gesture while keeping their eyes ahead.

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that varies a target device based on a passenger twisting the upper body.

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that remotely controls a device inside a vehicle by detecting a passenger's gesture.

MODE FOR INVENTION

A vehicle as described in this specification may include a car and a motorcycle. Hereinafter, a car will be as an example of a vehicle.

A vehicle as described in this specification may include all of an internal combustion engine vehicle including an engine as a power source, a hybrid vehicle including both an engine and an electric motor as a power source, and an electric vehicle including an electric motor as a power source.

In some implementations, the left of a vehicle means the left of the vehicle in the direction of travel and the right of the vehicle means the right of the vehicle in the direction of travel.

In some implementations, a left hand drive (LHD) vehicle will be assumed unless otherwise stated.

Hereinafter, a user, a driver, a passenger, and a fellow passenger may be used interchangeably depending on the embodiment.

In the following description, the vehicle control device400is a separate device provided in the vehicle700, and may exchange necessary information with the vehicle700through data communication. The vehicle control device400may include at least some of the units of the vehicle700. The vehicle control device400may be referred to as a control device400, a driving assistance device400, a vehicle driving assistance device400, or an assistance device400.

Alternatively, at least some of the units of the vehicle control device400may be a unit(s) of the vehicle700or of another device mounted in the vehicle700. Such external units may be understood as being included in the vehicle control device400by transmitting and receiving data through an interface unit of the vehicle control device400.

FIG. 1shows an exterior appearance of a vehicle700equipped with a vehicle control device400according to an exemplary embodiment of the present invention. Referring toFIG. 1, the vehicle700may include wheels13FL and12RL rotating by a power source.

FIG. 2is an internal block diagram illustrating one example of the vehicle700. The vehicle may include a communication unit710, an input unit720, a sensing unit760, an output unit740, a vehicle drive unit750, a memory730, an interface unit780, a controller770, a power source unit790, a control device400, and an AVN apparatus300. The communication unit710may include one or more modules to enable the wireless communication between the vehicle and the mobile terminal600, between the vehicle and an external server500, or between the vehicle and another vehicle510. In addition, the communication unit710may include one or more modules to connect the vehicle to one or more networks.

The communication unit710may include a broadcast receiving module711, a wireless Internet module712, a short-range communication module713, a location information module714, and an optical communication module715.

The broadcast receiving module711is configured to receive a broadcast signal or broadcast associated information from an external broadcast managing server via a broadcast channel. Here, broadcast includes radio broadcast or TV broadcast.

The wireless Internet module712is a module for wireless Internet access. The wireless Internet module712may be internally or externally coupled to the vehicle700. The wireless Internet module712may transmit or receive wireless signals via communication networks according to wireless Internet technologies.

Examples of such wireless Internet technologies include Wireless LAN (WLAN), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), Worldwide Interoperability for Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), and LTE-A (Long Term Evolution-Advanced). The wireless Internet module712may transmit and receive data according to one or more of such wireless Internet technologies, and other Internet technologies as well. For example, the wireless Internet module712may exchange data with the external server500in a wireless manner. The wireless Internet module712may receive weather information and road traffic state information (e.g., Transport Protocol Expert Group (TPEG) information) from the external server500.

The short-range communication module713may assist short-range communication using at least one selected from among Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-WideBand (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), and the like.

The short-range communication module713forms wireless area networks to perform the short-range communication between the vehicle700and at least one external device. For example, the short-range communication module713may exchange data with the mobile terminal600in a wireless manner. The short-range communication module713may receive weather information and road traffic state information (e.g., Transport Protocol Expert Group (TPEG) information) from the mobile terminal600. When the user boards the vehicle700, the mobile terminal600of the user and the vehicle700may pair with each other automatically or as the user executes a pairing application.

The location information module714is a module to acquire a location of the vehicle700. A representative example of the location information module714includes a Global Position System (GPS) module. For example, when the vehicle utilizes a GPS module, a location of the vehicle may be acquired using signals transmitted from GPS satellites.

The optical communication module715may include a light emitting unit and a light receiving unit.

The light receiving unit may convert light into electrical signals to receive information. The light receiving unit may include Photo Diodes (PDs) to receive light. The photo diodes may convert light into electrical signals. For example, the light receiving unit may receive information regarding a preceding vehicle via light emitted from a light source included in the preceding vehicle.

The light emitting unit may include at least one light emitting element to convert electrical signals into light. Here, the light emitting element may be a Light Emitting Diode (LED). The light emitting unit converts electrical signals into light to thereby emit the light. For example, the light emitting unit may externally emit light via flickering of the light emitting element corresponding to a prescribed frequency. In some embodiments, the light emitting unit may include an array of a plurality of light emitting elements. In some embodiments, the light emitting unit may be integrated with a lamp provided in the vehicle700. For example, the light emitting unit may be at least one selected from among a headlight, a taillight, a brake light, a turn signal light, and a sidelight. For example, the optical communication module715may exchange data with another vehicle510via optical communication.

The input unit720may include a driving operation unit721, a camera722, a microphone723, and a user input unit724.

The driving operation unit721is configured to receive user input for the driving of the vehicle. The driving operation unit721may include a steering input unit721a, a shift input unit721b, an acceleration input unit721c, and a brake input unit721d.

The camera722may include an image sensor and an image processing module. The camera722may process a still image or a moving image acquired by the image sensor (e.g., a CMOS or a CCD). The image processing module may extract required information by processing a still image or a moving image acquired via the image sensor and, then, may transmit the extracted information to the controller770. Meanwhile, the vehicle700may include the camera722to capture a forward image or a surround-view image of the vehicle and a monitoring unit725to capture an image of the interior of the vehicle.

The monitoring unit725may capture an image of a passenger. The monitoring unit725may capture an image of biometrics of the passenger.

The microphone723may process external sound signals into electrical data. The processed data may be utilized in various ways according to a function that the vehicle700is performing. The microphone723may convert a user voice command into electrical data. The converted electrical data may be transmitted to the controller770.

Meanwhile, in some embodiments, the camera722or the microphone723may be components of the sensing unit760, other than components of the input unit720.

The user input unit724is configured to receive information from the user. When information is input via the user input unit724, the controller770may control the operation of the vehicle700to correspond to the input information. The user input unit724may include a touch input unit or a mechanical input unit. In some embodiments, the user input unit724may be located in a region of the steering wheel. In this case, the driver may operate the user input unit724with the fingers while gripping the steering wheel.

The sensing unit760is configured to sense signals associated with, for example, the traveling of the vehicle700. To this end, the sensing unit760may include a collision sensor, a wheel sensor, a speed sensor, gradient sensor, a weight sensor, a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward/backward movement sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor based on the rotation of a steering wheel, a vehicle interior temperature sensor, a vehicle interior humidity sensor, an ultrasonic sensor, an infrared sensor, a radar, and Lidar.

As such, the sensing unit760may acquire sensing signals with regard to, for example, vehicle collision information, vehicle traveling direction information, vehicle location information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward/backward movement information, battery information, fuel information, tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, and steering wheel rotation angle information.

Meanwhile, the sensing unit760may further include, for example, an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an Air Flow-rate Sensor (AFS), an Air Temperature Sensor (ATS), a Water Temperature Sensor (WTS), a Throttle Position Sensor (TPS), a Top Dead Center (TDC) sensor, and a Crank Angle Sensor (CAS).

The sensing unit760may include a biometric information sensing unit. The biometric information sensing unit is configured to sense and acquire biometric information of the passenger. The biometric information may include fingerprint information, iris-scan information, retina-scan information, hand geometry information, facial recognition information, and voice recognition information. The biometric information sensing unit may include a sensor to sense biometric information of the passenger. Here, the monitoring unit725and the microphone723may operate as sensors. The biometric information sensing unit may acquire hand geometry information and facial recognition information via the monitoring unit725.

The output unit740is configured to output information processed in the controller770. The output unit740may include a display unit741, a sound output unit742, and a haptic output unit743.

The display unit741may include at least one selected from among a liquid crystal display (LCD), a thin film transistor liquid-crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display.

The display unit741may configure an inter-layer structure with a touch sensor, or may be integrally formed with the touch sensor to implement a touchscreen. The touchscreen may function as the user input unit724which provides an input interface between the vehicle700and the user and also function to provide an output interface between the vehicle700and the user. In this case, the display unit741may include a touch sensor which senses a touch to the display unit741so as to receive a control command in a touch manner.

When a touch is input to the display unit741as described above, the touch sensor may sense the touch and the controller770may generate a control command corresponding to the touch. Content input in a touch manner may be characters or numbers, or may be, for example, instructions in various modes or menu items that may be designated.

The touch sensor and the proximity sensor may be implemented individually, or in combination, to sense various types of touches. Such touches include a short (or tap) touch, a long touch, a multi-touch, a drag touch, a flick touch, a pinch-in touch, a pinch-out touch, a swipe touch, a hovering touch, and the like. Hereinafter, a touch or a touch input may generally refer to various types of touches mentioned above.

Meanwhile, the display unit741may include a cluster to allow the driver to check vehicle state information or vehicle traveling information while driving the vehicle. The cluster may be located on a dashboard. In this case, the driver may check information displayed on the cluster while looking forward.

Meanwhile, in some embodiments, the display unit741may be implemented as a Head Up display (HUD). When the display unit741is implemented as a HUD, information may be output via a transparent display provided at the windshield. Alternatively, the display unit741may include a projector module to output information via an image projected to the windshield.

The sound output unit742is configured to convert electrical signals from the controller770into audio signals and to output the audio signals. To this end, the sound output unit742may include, for example, a speaker. The sound output unit742may output sound corresponding to the operation of the user input unit724.

The haptic output unit743is configured to generate tactile output. For example, the haptic output unit743may operate to vibrate a steering wheel, a safety belt, or a seat so as to allow the user to recognize an output thereof.

The vehicle drive unit750may control the operation of various devices of the vehicle. The vehicle drive unit750may include at least one of a power source drive unit751, a steering drive unit752, a brake drive unit753, a lamp drive unit754, an air conditioner drive unit755, a window drive unit756, an airbag drive unit757, a sunroof drive unit758, and a suspension drive unit759.

The power source drive unit751may perform electronic control for a power source inside the vehicle700. For example, in the case where a fossil fuel based engine (not illustrated) is a power source, the power source drive unit751may perform electronic control for the engine. As such, the power source drive unit751may control, for example, an output torque of the engine. In the case where the power source drive unit751is the engine, the power source drive unit751may control the speed of the vehicle by controlling the output torque of the engine under the control of the controller770. In another example, when an electric motor (not illustrated) is a power source, the power source drive unit751may perform control for the motor. As such, the power source drive unit751may control, for example, the RPM and torque of the motor.

The steering drive unit752may include a steering apparatus. As such, the direction of travel of the vehicle may be changed.

The brake drive unit753may perform electronic control of a brake apparatus (not illustrated) inside the vehicle700. For example, the brake drive unit753may reduce the speed of the vehicle700by controlling the operation of brakes located at wheels. In another example, the brake drive unit753may adjust the direction of travel of the vehicle700leftward or rightward by differentiating the operation of respective brakes located at left and right wheels.

The lamp drive unit754may turn at least one lamp arranged inside and outside the vehicle700on or off. The lamp drive unit754may include a lighting apparatus. In addition, the lamp drive unit754may control, for example, the intensity and direction of light of each lamp included in the lighting apparatus. For example, the lamp drive unit754may perform control for a turn signal lamp, a headlamp or a brake lamp.

The air conditioner drive unit755may perform the electronic control of an air conditioner (not illustrated) inside the vehicle700. For example, when the interior temperature of the vehicle700is high, the air conditioner drive unit755may operate the air conditioner to supply cold air to the interior of the vehicle700.

The window drive unit756may perform the electronic control of a window apparatus inside the vehicle700. For example, the window drive unit756may control the opening or closing of left and right windows of the vehicle700.

The airbag drive unit757may perform the electronic control of an airbag apparatus inside the vehicle700. For example, the airbag drive unit757may control an airbag to be deployed in a dangerous situation.

The sunroof drive unit758may perform electronic control of a sunroof apparatus inside the vehicle700. For example, the sunroof drive unit758may control the opening or closing of a sunroof.

The suspension drive unit759may perform electronic control on a suspension apparatus (not shown) inside the vehicle700. For example, when the road surface is uneven, the suspension drive unit759may control the suspension apparatus to reduce vibration of the vehicle700.

The memory730is electrically connected to the controller770. The memory730may store basic data for each unit, control data for the operation control of the unit, and input/output data. The memory730may be various hardware storage devices such as, for example, a ROM, a RAM, an EPROM, a flash drive, and a hard drive. The memory730may store various data for the overall operation of the vehicle700such as, for example programs for the processing or control of the controller770.

The interface unit780may serve as a passage for various kinds of external devices that are connected to the vehicle700. For example, the interface unit780may have a port that is connectable to the mobile terminal600and may be connected to the mobile terminal600via the port. In this case, the interface unit780may exchange data with the mobile terminal600.

Meanwhile, the interface unit780may serve as a passage for the supply of electrical energy to the connected mobile terminal600. When the mobile terminal600is electrically connected to the interface unit780, the interface unit780supplies electrical energy from the power source unit790to the mobile terminal600under the control of the controller770.

The controller770may control the overall operation of each unit inside the vehicle700. The controller770may be referred to as an Electronic Control Unit (ECU).

The controller770may execute a function corresponding to an execution signal delivered from the control device100.

The controller770may be implemented in a hardware manner using at least one selected from among Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and electric units for the implementation of other functions.

The power source unit790may supply power required to operate the respective components under the control of the controller770. In particular, the power source unit790may receive power from, for example, a battery (not illustrated) inside the vehicle700.

The AVN apparatus300may exchange data with the controller770. The controller770may receive navigation information from the AVN apparatus300or a separate navigation apparatus (not illustrated). Here, the navigation information may include set destination information, destination based routing information, and map information or vehicle location information related to vehicle traveling.

The vehicle control device400may be understood as included in the vehicle700.

FIG. 3is an internal block diagram illustrating one example of the vehicle control device400according to an exemplary embodiment of the present invention. Referring toFIG. 3, the vehicle control device400may include an input unit410, a communication unit420, an interface430, a memory440a camera460, a sensor unit450, a processor470, an audio output unit485, and a power supply unit490. The vehicle control device400may include additional components in addition to the above-described components, or some of the above-described components may be omitted. Here, units of the same names, among the units included in the vehicle control device400and the units included in the vehicle700may be included in the vehicle700or the vehicle control device400.

The vehicle control device400may include the input unit410for receiving user input. For example, a user may input settings for a sliding function provided by the vehicle control device400through the input unit410, or may input execution of power on/off of the vehicle control device400.

The input unit410may include at least one of a gesture input unit (e.g., an optical sensor, etc.) for sensing a user gesture, a touch input unit (e.g., a touch sensor, a touch key, a push key (mechanical key), etc.) for sensing touch, and a microphone for sensing voice input and receive user input.

Next, the vehicle control device400may include the communication unit420for communicating with another vehicle510, a terminal600and a server500. The communication unit420may be referred to as a wireless communication unit420.

The vehicle control device400may receive communication information including at least one of navigation information, another vehicle510's traveling information, and traffic information through the communication unit420. The vehicle control device400may send information about the vehicle700through the communication unit420.

The communication unit420may receive at least one of position information, weather information and road traffic condition information (e.g., transport protocol experts group (TPEG), etc.) from the mobile terminal600and/or the server500.

The communication unit420may receive traffic information from the server500having an intelligent traffic system (ITS). Here, the traffic information may include traffic signal information, lane information, vehicle surrounding information or position information.

In addition, the communication unit420may receive navigation information from the server500and/or the mobile terminal600. Here, the navigation information may include at least one of map information related to vehicle driving, lane information, vehicle position information, set destination information and route information according to the destination.

For example, the communication unit420may receive the real-time position of the vehicle as the navigation information. In detail, the communication unit420may include a global positioning system (GPS) module and/or a Wi-Fi (Wireless Fidelity) module and acquire the position of the vehicle.

In addition, the communication unit420may receive driving information of another vehicle510from the another vehicle510and transmit information on this vehicle, thereby sharing driving information between vehicles. Here, the shared driving information may include vehicle traveling direction information, position information, vehicle speed information, acceleration information, moving route information, forward/reverse information, adjacent vehicle information and turn signal information.

In addition, when a user rides in the vehicle, the mobile terminal600of the user and the vehicle control device400may pair with each other automatically or by executing a user application.

The communication unit420may exchange data with the another vehicle510, the mobile terminal600or the server500in a wireless manner. The communication unit420may perform wireless communication using a wireless data communication method. As the wireless data communication method, technical standards or communication methods for mobile communications (for example, Global System for Mobile Communication (GSM), Code Division Multiple Access (CDMA), CDMA2000 (Code Division Multiple Access 2000), EV-DO (Evolution-Data Optimized), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), HSUPA (High Speed Uplink Packet Access), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), and the like) may be used.

The communication unit420is configured to facilitate wireless Internet technology. Examples of such wireless Internet technology include Wireless LAN (WLAN), Wireless Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), Worldwide Interoperability for Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA), HSUPA (High Speed Uplink Packet Access), Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), and the like.

The communication unit420is configured to facilitate short-range communication. For example, short-range communication may be supported using at least one of Bluetooth™ Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Wireless USB (Wireless Universal Serial Bus), and the like.

In addition, the vehicle control device400may pair with the mobile terminal located inside the vehicle using a short-range communication method and wirelessly exchange data with the other vehicle510or the server500using a long-distance wireless communication module of the mobile terminal.

Next, the vehicle control device400may include the interface430for receiving data of the vehicle700and transmitting a signal processed or generated by the processor470.

The vehicle control device400may receive at least one of driving information of another vehicle, navigation information and sensor information via the interface430.

To this end, the interface430may perform data communication with at least one of the controller770of the vehicle, an audio-video-navigation (AVN) apparatus300, and the sensing unit760using a wired or wireless communication method. The interface430may receive navigation information by data communication with the controller770, the AVN apparatus300and/or a separate navigation apparatus. In addition, the interface430may receive sensor information from the controller770or the sensing unit760.

Here, the sensor information may include at least one of vehicle traveling direction information, vehicle position information, vehicle speed information, acceleration information, vehicle tilt information, forward/reverse information, fuel information, information on a distance from a preceding/rear vehicle, information on a distance between a vehicle and a lane and turn signal information, etc.

The sensor information may be acquired from a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward/reverse sensor, a wheel sensor, a vehicle speed sensor, a vehicle tilt sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor using handle rotation, a vehicle inside temperature sensor, a vehicle inside humidity sensor, a door sensor, etc. The position module may include a GPS module for receiving GPS information.

The interface430may receive user input via the user input unit410of the vehicle. The interface430may receive user input from the input unit of the vehicle or via the controller770. That is, when the input unit is provided in the vehicle, user input may be received via the interface430.

In addition, the interface430may receive traffic information acquired from the server. The server500may be located at a traffic control surveillance center for controlling traffic. For example, when traffic information is received from the server500via the communication unit420of the vehicle, the interface430may receive traffic information from the controller770.

Next, the memory440may store a variety of data for overall operation of the vehicle control device400, such as a program for processing or control of the processor470.

In addition, the memory440may store data and commands for operation of the vehicle control device400and a plurality of application programs or applications executed in the vehicle control device400. At least some of such application programs may be downloaded from an external server through wireless communication. At least some of such application programs may be installed in the vehicle control device400upon release. Such application programs may be stored in the memory440, and may be executed to perform operation (or function) of the vehicle control device400by the processor470.

The memory440may store data for checking an object included in an image. For example, the memory440may store data for checking a predetermined object using a predetermined algorithm when the predetermined object is detected from an image of the vicinity of the vehicle acquired through the camera460. For example, the memory440may store data for checking a predetermined object such as a passenger, garbage, or a lost item from an image acquired through the camera460.

The memory440may be implemented in a hardware manner using at least one selected from among a flash memory, a hard disk, a solid state drive (SSD), a silicon disk drive (SDD), a micro multimedia card, a card type memory (e.g., an SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk and an optical disc.

In addition, the vehicle control device400may operate in association with a web storage for performing a storage function of the memory440over the Internet.

The sensor unit450may acquire information on the internal state of the vehicle700. The sensor unit450may sense an object or thing within the vehicle700. Alternatively, the camera460may capture a passenger in the vehicle700.

The camera460may capture the inside of the vehicle700. The camera460may capture the surroundings of the vehicle control device400. Alternatively, the camera460may capture the surroundings of a seat where the vehicle control device400is installed. Alternatively, the camera460may capture a passenger in the seat.

A display unit180may commonly refer to a display device installed inside the vehicle. The display unit180may be referred to as a display180or a display device180.

The processor470may be referred to as a control unit470or a controller470. The processor470may detect an object in the vicinity of the vehicle control device400through the camera460. Alternatively, the processor470may detect a passenger, the passenger's gaze, or the passenger's movement through the camera460. The processor470may control a voice recognition unit481.

FIG. 4shows the inside of the vehicle700equipped with the vehicle control device400according to an exemplary embodiment of the present invention. Referring toFIG. 4, a camera may be installed inside the vehicle700. The camera may capture the interior space of the vehicle700. The camera460may capture a driver's seat. Alternatively, the camera460may capture a driver900sitting in the driver's seat S1. Alternatively, the camera460may capture the vicinity of the driver's seat S1, the vicinity of a control board200, and the vicinity of an assistant driver's seat S2. The camera460may acquire a captured image of the driver900at regular time intervals. The voice recognition unit481may recognize voice inside the vehicle700.

The control board200may be installed inside the vehicle700. The control board200may be positioned to one side of the driver's seat S1. Alternatively, the control board200may be positioned between the driver's seat S1and the assistant driver's seat S2. The control board200may be referred to as a center console200or a console200.

The display180may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, a 3D display, and an e-ink display. The display180may be referred to as a display unit180.

The display180may configure an inter-layer structure with a touch sensor, or may be integrally formed with the touch sensor to implement a touchscreen. The touchscreen may function as the user input unit724which provides an input interface between the vehicle700and the user and also function to provide an output interface between the vehicle700and the user. In this case, the display180may include a touch sensor which senses a touch to the display180so as to receive a control command in a touch manner.

When a touch is input to the display180as described above, the touch sensor may sense the touch and the processor470may execute a control command corresponding to the touch. Content input in a touch manner may be characters or numbers, or may be, for example, instructions in various modes or menu items that may be designated.

The display180may commonly refer to a cluster180cinstalled inside the vehicle700, a center display180b, a head-up display180dformed on a front window W of the vehicle700, and a display180einstalled to face a back seat.

FIGS. 5 to 9are views showing an embodiment in which a device inside the vehicle700is controlled with respect to the driver's seat.

Referring toFIG. 5, the processor470may detect the position of a passenger from an image acquired by the camera460(S510). The processor470may detect a passenger sitting in the driver's seat.

The processor470may configure a space adjacent to the passenger as a virtual control space (S520). The virtual control space may be a space inside the vehicle700. The virtual control space may be referred to as a control space, a space, a gesture box, a control box, or a control area. The virtual control space may be formed over the control board200. For example, the virtual control space may be in the shape of a cuboid. The virtual control space may be positioned to the right side of the driver901. The virtual control space may be adjacent to the right hand of the driver901.

The processor470may match a device corresponding to the position of the passenger, among a plurality of devices inside the vehicle700, to the virtual control space (S530). The device corresponding to the position of the driver901may be a device positioned in front of the driver901. For example, the device corresponding to the position of the driver901may be a device, such as an air conditioner, an audio device, or a media device, that controls a function of the vehicle700. If a device and the virtual control space are matched, the processor470may determine that an input detected through the virtual control space is an input on the device. The matched device may be referred to as an area of interest.

The processor470may determine whether a preset gesture is detected from the virtual control space (S540). The processor470may control the device based on the preset gesture (S550).

For example, the preset gesture may involve putting a hand up, and the processor470may increase the volume of the audio based on this gesture. As another example, the preset gesture may involve putting a hand down, and the processor470may lower the volume of the audio based on this gesture.

As another example, the preset gesture may involve moving a hand to the right, and the processor470may turn up the air conditioner based on this gesture. As another example, the preset gesture may involve moving a hand to the left, and the processor470may turn down the air conditioner based on this gesture.

For example, the preset gesture may involve putting a hand up, and the processor may increase the set temperature of the air conditioner based on this gesture. As another example, the preset gesture may involve putting a hand down, and the processor470may lower the set temperature of the air conditioner based on this gesture.

As another example, the preset gesture may involve moving a hand to the left or right, and the processor470may switch between channels on the radio.

Referring toFIG. 6, the processor470may configure a space adjacent to a passenger as a virtual control space (S610). The processor470may receive the passenger's voice from the voice recognition unit481(S620). The processor470may select a device corresponding to the position of the passenger from among a plurality of devices inside the vehicle700based on the passenger's voice (S630). The processor470may match the selected device to the virtual control space (S640).

A plurality of devices may be provided inside the vehicle700, corresponding to the position of the driver901, and it may be necessary to select a device from among the plurality of devices, in order to match the virtual control device and the device. In this case, the passenger's voice is for selecting a device from among the plurality of devices. For example, the radio may be matched to the virtual control space based on a voice saying “radio volume” or “radio channel”. As another example, the air conditioner may match the air conditioner to the virtual control space based on a voice saying “air conditioning”, “air conditioner”, or “heater”. As another example, the media device may be matched to the virtual control space based on “media volume” or “media channel”.

The driver901may activate the voice recognition unit481by pressing a button provided inside the vehicle700. For example, the button may be provided on the steering wheel.

Referring toFIG. 7, the processor470may match a device corresponding to the position of a passenger to a virtual control space (S710). The processor470may control the device based on a preset gesture (S720). The processor470may determine whether the preset gesture is detected through the virtual control space within a preset period of time (S730). If the preset gesture is not detected through the virtual control space within a preset period time, the processor470may undo the matching of the device and the virtual control space (S740). As another example, if no preset gesture is detected through the virtual control space within a preset period of time, the processor470may undo the matching of the device to the virtual control space.

Referring toFIG. 8, the processor470may match a device corresponding to the position of a passenger to a virtual control space (S810). The passenger may be the driver901. The processor470may detect a preset gesture through the virtual control space (S820). The processor470may control the device based on the preset gesture (S830). The processor470may determine whether the passenger's gesture is made outside the virtual control space (S840). If it is determined that the passenger's gesture is made outside the virtual control space, the processor470may undo the matching of the device and the virtual control space and the virtual control space (S850).

Device control through the virtual control space may be done only one time. For example, if the driver901increases the audio volume by a gesture of putting a hand up, the matching of the audio and the virtual control space may be undone. Even if the driver901accesses the virtual control space later, the audio may not be controlled. As such, the driver901may control a device in the vehicle700while looking ahead when driving the vehicle700. Also, the driver901may not be confused or misled since device control through the virtual control space is done only one time.

Referring toFIG. 9, the vehicle700may be equipped with displays180a,180b,180c,180d,180e, and180fas in-vehicle devices. The displays180eand180fmay display an image toward a passenger sitting in a back seat. The passenger901sitting in the driver's seat S1may control the display180adisplaying AVN by entering a gesture901ainto the virtual control space901bwhich is formed over the control board200.

FIGS. 10 to 15are views showing an embodiment in which a device inside the vehicle700is controlled with respect to a passenger seat, not the driver's seat.

Referring toFIG. 10, the processor470may determine whether a device corresponding to the position of a first passenger and a device corresponding to the position of a second passenger are the same (S1010). As for a back seat of the vehicle700, a plurality of passengers may sit in it close to one another. As such, the same device may be positioned in front of the first and second passengers adjacent to each other, and this same device may correspond to both the position of the first passenger and the position of the second passenger.

The processor470may determine which passenger has first accessed the position corresponding to the device (S1020). The processor470may determine which of the first and second passengers has been first matched with the device.

The processor470may configure a virtual control space in front of the passenger who has first accessed (S1030). For example, if the first passenger is matched with the device earlier than the second passenger, the processor470may configure the virtual control space close to the first passenger.

The processor470may match the virtual control space and the device (S1040). For example, if the first passenger is matched with the device earlier than the second passenger, the processor470may configure the virtual control space close to the first passenger and match the virtual control space to the device.

Referring toFIG. 11, the processor470may configure a virtual control space (S1110). The processor470may match the virtual control space and a device (S1120). The processor470may determine if any gesture(s) are detected from a plurality of passengers through the virtual control space (S1130). Upon detecting gestures from the plurality of passengers through the virtual control space, the processor470may assign priority levels to the plurality of passengers (S1140). The processor may control the device according to the priority (S1150).

For example, the processor470may assign priority levels to the passengers in the order they enter a gesture, and may control the device sequentially according to the priority based on a plurality of gestures. As another example, the processor470may give priority to a passenger whom the virtual control space is configured with respect to. As another example, the processor470may control the device based only on a top priority passenger's gesture. As another example, the processor470may assign priority levels to multiple gestures in the order the gestures are entered, and control the device sequentially in that order. As another example, the processor470may assign priority levels to multiple gestures in the order the gestures are entered, and control the device based only on a top priority gesture.

Referring toFIG. 12, the processor470may match a device corresponding to the position of a passenger to a virtual control space (S1210). The processor470may acquire the passenger's voice through the voice recognition unit481(S1220). The processor470may control the device based on the passenger's voice (S1230).

For example, the passenger may control the display by uttering a voice command while controlling the display through the virtual control space. For example, the voice command may be a command for moving to specific content on the display. For example, if the passenger utters “navigation” while the display is showing music playback and music playback control images, the display may show a navigation image.

Referring toFIG. 13, the processor470may match a device corresponding to the position of a first passenger to a virtual control space (S1310). The processor470may acquire the first passenger's voice and the first passenger's utterance position through the voice recognition unit481(S1320). The voice recognition unit481may detect the direction of a recognized sound. For example, the voice recognition unit481may be a directional microphone or a directional speaker that detects the direction of sound. The processor470may detect the position of the first passenger through the voice recognition unit481.

The processor470may detect a second passenger's voice based on the first passenger's utterance position (S1330). The processor470may distinguish between sounds detected from the first passenger's utterance position and sounds detected from other utterance positions.

The processor470may perform noise cancelling on the second passenger's voice (S1340). Noise cancelling can also be referred to as noise reduction or active noise reduction. The processor470may detect the first passenger's voice with a high resolution by performing noise cancelling on other passengers' utterances.

Referring toFIG. 14, a second seat S2may correspond to the display180b, and a second passenger902sitting in the second seat S2may correspond to the display180bpositioned in front of the second passenger902and control the display180bby entering a gesture902ainto a virtual control space902bformed in front of the second passenger902.

A third seat S3may correspond to the display180e, and a third passenger903sitting in the third seat S3may correspond to the display180epositioned in front of the third passenger903and control the display180eby entering a gesture903ainto a virtual control space903bformed in front of the third passenger903.

A fourth seat S4may correspond to the display180a, and a fourth passenger904sitting in the fourth seat S4may correspond to the display180apositioned in front of the fourth passenger904and control the display180aby entering a gesture904ainto a virtual control space904bformed in front of the fourth passenger904.

A fifth seat S5may correspond to the display180f, and a fifth passenger905sitting in the fifth seat S5may correspond to the display180fpositioned in front of the fifth passenger905and control the display180fby entering a gesture905ainto a virtual control space905bformed in front of the fifth passenger905.

Referring toFIG. 15, a passenger may control a display in front of the passenger through a virtual control space. The virtual control space may be provided at a distance L in front of the passenger. For example, the distance L may be 200 mm. The lateral length b of the virtual control space is almost the same as the shoulder width of the passenger. The processor470may detect the shoulder width of the passenger, and the virtual control space may have a lateral length which is almost the shoulder width of the passenger.

If the passenger moves his or her fingertip a length c to the left or right in the virtual control space, the processor470may determine that it has moved a lateral length x on the display by taking into account the ratio of the lateral length a of the display to the lateral length b of the virtual control space. That is, the processor470may calculate the length x by using a proportional expression: length a:length b=length c:length x.

For example, upon detecting a lateral movement of the passenger's fingertip, the processor470may make a change to the content on the display. Also, upon detecting a vertical movement of the passenger's fingertip, the processor470may control the display in such a way that the corresponding content is selected on the display. A vertical movement of the passenger's fingertip may be referred to as an air mouse.

FIGS. 16 and 17are views showing an embodiment in which a device to be controlled from a passenger seat is changed.

Referring toFIG. 16, the processor470may match a first device corresponding to the position of a passenger to a first virtual control space (S1610). The processor470may detect a twist of the passenger's upper body (S1620). If the passenger is not twisting the upper body, this means that the passenger's back is in full contact with the seat and looking ahead. The processor470may detect a twist of the passenger's upper body through the camera460. The twist of the passenger's upper body may be measured with respect to the y axis.

The processor470may configure a second virtual control space based on the passenger twisting the upper body (S1630). The second virtual control space may be formed in front of the passenger with respect to the direction of the passenger's gaze. Alternatively, the second virtual control space may be twisted in shape as much as the passenger's upper body is.

The processor470may match a second device corresponding to the passenger twisting the upper body to the second virtual control space (S1640). When the second virtual control space and the second device are matched, the processor470may undo the matching of the first virtual control space and the first device.

The processor470may determine whether a preset gesture is detected from the second virtual control space (S1650).

The processor470may control the second device based on the preset gesture (S1660). The processor470may not undo the matching of the first virtual control space and the first device even if the second virtual control space and the second device are matched, and, if there is a gesture input in an area where the second virtual control space and the first virtual control space overlap, while the passenger's upper body is twisted, the processor470may control the second device. If the passenger untwists the upper body, the processor470may undo the matching of the second virtual control space and the second device.

Alternatively, the processor470may configure the second virtual control space match the second virtual control space and the second device, based on a change in the direction of the passenger's gaze, instead of a twist of the passenger's upper body. The processor470may detect the direction of the passenger's gaze through the camera460and detect a change in the direction of the passenger's gaze.

Referring (a) ofFIG. 17, the processor470may match a first virtual control space900b1and a display180g, and control the display180gbased on a gesture900a1from the passenger900.

Referring to (b) ofFIG. 17, the passenger900's upper body may be twisted at an angle toward a reference line D2from a reference line D1. Alternatively, the passenger900's upper body may be twisted at an angle A from the reference line D1. If the passenger900's upper body is twisted at the angle A, the passenger900may correspond to a display180h. The display180hmay be positioned in the direction the passenger900's upper body faces or in the direction of the passenger900's gaze.

The processor470may match the second virtual control space900b2and the display180h, and control the display180hbased on a gesture900a2from the passenger900.

FIG. 18is a view showing an embodiment in which a plurality of passengers control displays inside the vehicle700.

Referring toFIG. 18, the vehicle700may include a forward-facing seat S7and a backward-facing seat S6. The vehicle700may be driving autonomously. When the vehicle700is driving autonomously, there may be no driver in the vehicle700. Passengers911,912, and913may sit facing backward, and passengers914,915, and916may sit facing forward. The inside of the vehicle700may be surrounded by displays180LF,180LB,180LL, and180LR.

The passengers914,915, and916sitting facing forward may correspond to the front display180LF. The processor470may form virtual control spaces914b,915b, and916bin front of the passengers914,915, and916sitting facing forward, respectively for the passengers914,915, and916. The passengers914,915, and916sitting facing forward may control the areas the respective passengers914,915, and916are facing on the front display180LF by gestures914a,915a, and916a. Alternatively, the processor470may match the areas on the front display180LF the respective passengers914,915, and916are facing to the virtual control spaces914b,915b, and916bof the respective passengers914,915, and916, and, upon detecting the gestures914a,915a, and916afrom the virtual control spaces914b,915b, and916bof the respective passengers914,915, and916, the processor470may control the areas on the front display180LF the respective passengers914,915, and916are facing, based on the gestures914a,915a, and916a.

The passengers911and912sitting facing backward may correspond to the back display180LB. The processor470may form virtual control spaces911band912bin front of the passengers911and912sitting facing backward, respectively for the passengers911and912. The passengers911and912sitting facing backward may control the areas the respective passengers911and912are facing on the back display180LB by gestures911aand912a. Alternatively, the processor470may match the areas on the back display180LB the respective passengers911and912are facing to the virtual control spaces911band912bof the respective passengers911and912, and, upon detecting the gestures911aand912afrom the virtual control spaces911band912bof the respective passengers911and912, the processor470may control the areas on the back display180LB the respective passengers911and912are facing, based on the gestures911aand912a.

The passenger913may twist the upper body at an angle toward a reference line D3from a reference line D1. Alternatively, the passenger913may twist the upper body at an angle E from the reference line D1. The processor470may form a virtual control space913bwith respect to the passenger913twisting the upper body, and match the virtual control space913band the right display180LR. The processor470may control the right display180LR based on a gesture913afrom the passenger913detected through the virtual control space913b.

FIGS. 19 and 20are views showing an embodiment in which a device inside the vehicle700is controlled based on the movement of a passenger917.

Referring toFIG. 19, the processor470may detect the passenger's field of view through the camera460. The processor470may match a plurality of devices positioned in the passenger's field of view with the passenger, and form different virtual control spaces for the plurality of devices corresponding to the passenger.

The processor470may form a plurality of virtual control spaces. The plurality of virtual control spaces may be disposed in such a fashion as to surround the passenger. The plurality of virtual control spaces may be formed between the devices and the passenger. The plurality of virtual control spaces may not overlap. The plurality of virtual control spaces may be adjacent to one another.

The processor470may match the plurality of devices corresponding to the passenger and the plurality of virtual control spaces, respectively.

For example, the processor470may detect the passenger917's field of view through the camera460, and pick out the displays180a,180e, and180fpositioned in the passenger917's field of view. The processor470may form a virtual control space917cand match the virtual control space917cto the display180e. The processor470may form a virtual control space917band match the virtual control space917bto the display180a. The processor470may form a virtual control space917dand match the virtual control space917dto the display180f.

The passenger may enter a gesture917awhile moving between the virtual control spaces917b,917c, and917d. Upon detecting a gesture917afrom the passenger917through the virtual control spaces917b,917c, and917d, the processor470may inform the passenger917that the displays180a,180e, and180f, which are matched to the virtual control spaces917b,917c, and917dwhere the gesture917ais detected, will be selected or controlled. The processor470may inform the passenger917or give feedback to the passenger917that the displays180a,180e, and180fwill be selected or controlled, by emitting a sound through the audio output unit485or showing an image through the display180.

Referring toFIG. 20, a passenger may move their position. The processor may detect a change in the passenger's position. The processor may detect a change in the passenger's field of view based on the change in the passenger's position. The processor may match a plurality of devices positioned in the passenger's changed field of view with the passenger and form different virtual control spaces for controlling the plurality of devices corresponding to the passenger. The plurality of virtual control spaces may be added or deleted depending on whether the passenger's position has been moved or not. Also, the plurality of virtual control spaces may be rearranged depending on whether the passenger's position has been moved or not.

For example, the passenger may move to a fifth position S5. The processor470may detect the passenger917's field of view through the camera460and pick out the displays180a,180e, and180fpositioned in the passenger917's field of view.

The processor470may detect the display180ethat has disappeared from the passenger917's field of view and the display180ethat has been added into the passenger917's field of view, due to the movement of the passenger917. The processor470may delete the virtual control space917cmatched to the display180ethat has disappeared from the passenger917's field of view. The processor470may match the display180eadded into the passenger917's field of view with the passenger917and create an additional virtual control space917eand match it to the added display180e. The plurality of virtual control spaces917b,917d, and917emay be rearranged with respect to the position to which the passenger917has moved.

The passenger917may enter a gesture917awhile moving between the virtual control spaces917b,917d, and917e. Upon detecting a gesture917afrom the passenger917through the virtual control spaces917b,917d, and917e, the processor470may inform the passenger917that the displays180a,180e, and180f, which are matched to the virtual control spaces917b,917d, and917ewhere the gesture917is detected, will be selected or controlled. The processor470may inform the passenger917or give feedback to the passenger917that the displays180a,180e, and180fwill be selected or controlled, by emitting a sound through the audio output unit485or showing an image through the display180.

An exemplary embodiment of the present invention provides a vehicle control device including: an interface unit that is connected to a camera for capturing a vehicle passenger and to a plurality of devices installed inside the vehicle; and a processor that receives an image captured by the camera through the interface unit and exchanges information with the plurality of devices through the interface unit, wherein the processor detects the position of the passenger from the image acquired by the camera, configures a space adjacent to the passenger as a virtual control space, matches a device corresponding to the position of the passenger, among the plurality of devices, to the virtual control device, and, upon detecting a preset gesture through the virtual control space, controls the device based on the preset gesture.

The passenger may be positioned in a driver's seat, and a plurality of devices corresponding to the position of the passenger may be provided in front of the passenger, wherein the interface unit may be connected to a voice recognition unit installed inside the vehicle, and the processor may receive the passenger's voice recognized by the voice recognition unit through the interface unit, select one device corresponding to the position of the passenger from among the plurality of devices based on the passenger's voice, and match the selected device to the virtual control space.

The interface unit may be connected to a button provided inside the vehicle, and the processor may activate the voice recognition unit upon receiving an input into the button through the interface unit.

If the preset gesture is not detected from the virtual control space for a preset period of time after controlling the device based on the preset gesture, the processor may undo the matching of the selected device and the virtual control space.

Upon detecting a preset gesture through the virtual control space and detecting that the passenger's body is partially outside the virtual control space, the processor may control the device to correspond to the detected gesture and undo the matching of the selected device and the virtual control space.

The vehicle may include an assistant driver's seat spaced apart from the driver's seat, and the virtual control space may be positioned between the driver's seat and the assistant driver's seat.

The passenger may include a first passenger and a second passenger, and, if a device corresponding to the position of the first passenger and a device corresponding to the position of the second passenger are the same, and the first passenger is matched with the device earlier than the second passenger, the processor may give priority to the first passenger's gesture and control the device according to the priority.

The processor may configure the virtual control space in front of the first passenger.

The interface unit may be connected to a voice recognition unit that is installed inside the vehicle and detects the direction of a voice, and the processor may acquire the first passenger's voice and the first passenger's utterance position, sense the second passenger's voice based on the first passenger's utterance position, and perform noise cancelling on the second passenger's voice.

The device may be positioned in front of the passenger, and the virtual control space may be provided at a predetermined distance in front of the passenger and include a width corresponding to the shoulder width of the passenger.

The device may be positioned in front the passenger, the interface may be connected to a voice recognition unit installed inside the vehicle, and the processor may acquire the passenger's voice detected by the voice recognition unit through the interface unit and control the device based on the passenger's voice.

The device may be a first device, the virtual control space may be a first virtual control space, and the processor may detect a twist of the passenger's upper body from the image acquired by the camera, configure a space in front of the passenger as a second virtual control space based on the passenger twisting the upper body, match a second device corresponding to the passenger twisting the upper body, among the plurality of devices, to the second virtual control space, and, upon detecting a preset gesture from the second virtual control space, control the device based on the preset gesture.

The adjacent space may be at least one of a space positioned between the passenger and the device, a space positioned in front of the passenger, and a space positioned to one side of the passenger.

The processor may detect the passenger's field of view through the camera, pick out a plurality of devices positioned in the field of view, form a plurality of virtual control spaces respectively corresponding to the plurality of picked-out devices, match the plurality of picked-out devices and the plurality of virtual control devices, respectively, and, upon detecting the preset gesture through the plurality of virtual control spaces, control the device matched to the virtual control space where the preset gesture is detected.

The plurality of virtual control spaces may be spaced apart from one another, be positioned between the passenger and the plurality of picked-out devices, and surround the passenger.

Another exemplary embodiment of the present invention provides a vehicle control method including: detecting the position of a passenger from an image acquired by a camera for capturing a vehicle passenger; configuring a space adjacent to the passenger as a virtual control space; matching a device corresponding to the position of the passenger, among the plurality of devices, to the virtual control device, and; upon detecting a preset gesture through the virtual control space, controlling the device based on the preset gesture.

The passenger may be positioned in a driver's seat, and a plurality of devices corresponding to the position of the passenger may be provided, wherein the vehicle control method further includes: receiving the passenger's voice recognized by a voice recognition unit installed inside the vehicle; selecting one device corresponding to the position of the passenger from among the plurality of devices based on the passenger's voice; and matching the selected device to the virtual control space.

The vehicle control method may further include, upon receiving an input into a button provided inside the vehicle, activating the voice recognition unit.

The vehicle control method may further include, if the preset gesture is not detected from the virtual control space for a preset period of time after controlling the device based on the preset gesture, undoing the matching of the selected device and the virtual control space.

The vehicle control method may further include, upon detecting a preset gesture through the virtual control space and detecting that the passenger's body is partially outside the virtual control space, controlling the device to correspond to the detected gesture and undoing the matching of the selected device and the virtual control space.

The vehicle control method may further include, if the passenger includes a first passenger and a second passenger, a device corresponding to the position of the first passenger and a device corresponding to the position of the second passenger are the same, and the first passenger is matched with the device earlier than the second passenger, giving priority to the first passenger's gesture and controlling the device according to the priority.

The vehicle control method may further include: acquiring the first passenger's voice and the direction of the voice by a voice recognition unit installed inside the vehicle to detect the direction of voice; and performing noise cancelling on the second passenger's voice based on the direction of the voice.

The device may be a first device, the virtual control space may be a first virtual control space, and the vehicle control method may further include: detecting a twist of the passenger's upper body from the image acquired by the camera; configuring a space in front of the passenger as a second virtual control space based on the passenger twisting the upper body; matching a second device corresponding to the passenger twisting the upper body, among the plurality of devices, to the second virtual control space; and, upon detecting a preset gesture from the second virtual control space, controlling the device based on the preset gesture.

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that forms a virtual control space to allow a driver to control a device inside a vehicle by gesture while keeping their eyes ahead.

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that varies a target device based on a passenger twisting the upper body.

According to at least one exemplary embodiment of the present invention, it is possible to provide a vehicle control device that remotely controls a device inside a vehicle by detecting a passenger's gesture.

The vehicle control device according to the foregoing exemplary embodiment may enhance passengers' convenience. The vehicle control device according to the foregoing exemplary embodiment may be used during autonomous driving or semi-autonomous driving of a vehicle.

The features, structures, and effects described in the above embodiments are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Moreover, the features, structures, and effects described in the embodiments may also be combined or modified to be carried out in other embodiments by those skilled in the art to which the embodiments pertain. Thus, the contents related to the combination and modification shall be construed to be included in the scope of the present invention.

Further, although the embodiments have been mainly described until now, they are just exemplary and do not limit the present invention. Thus, those skilled in the art to which the present invention pertains will know that various modifications and applications which have not been exemplified may be carried out within a range which does not deviate from the essential characteristics of the embodiments. For example, the constituent elements described in detail in the exemplary embodiments can be modified to be carried out. Further, the differences related to such modifications and applications shall be construed to be included in the scope of the present invention specified in the attached claims.