Device and method for recognizing motion in vehicle

A device for recognizing a motion in a vehicle according to an embodiment of the present disclosure may include a camera for acquiring a user image, and a controller that divides the user image into a first region, a second region, and a third region in which the first region and the second region overlap each other, and recognizes a motion of a user occurring in at least one of the first region, the second region, or the third region.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims the benefit of priority to Korean Patent Application No. 10-2019-0096900, filed on Aug. 8, 2019, the entire contents of which are incorporated herein by reference.

FIELD

The present disclosure relates to a device and a method for recognizing a motion in a vehicle.

BACKGROUND

A device control technology using motion recognition has recently been introduced into a vehicle. Some vehicle manufacturers have even made mass production of a vehicle equipped with the motion recognition technology using a camera. Users may use the motion recognition technology to more conveniently control various devices in the vehicle.

However, a recognition rate of the motion recognition technology is still low, and misrecognition occurs when motions generated from a plurality of users in the vehicle are recognized.

SUMMARY

The present disclosure has been made to address the above-mentioned problems occurring in the prior art while advantages achieved by the prior art are maintained intact.

An aspect of the present disclosure provides a device and a method for recognizing a motion in a vehicle that may recognize motions generated from a plurality of users in the vehicle.

Another aspect of the present disclosure provides a device and a method for recognizing a motion in a vehicle that may improve a recognition rate of a motion generated from a user in the vehicle.

In one aspect of the present disclosure, a device for recognizing a motion in a vehicle is provided. One form of a device includes a camera configured to acquire a user image, and a controller that is configured to divide the user image into a first region, a second region, and a third region in which the first region and the second region overlap each other, and to recognize a motion of a user occurring in at least one of the first region, the second region, or the third region.

In some implementations, the first region may include a first main region and a first sub region, and the second region includes a second main region and a second sub region.

In some implementations, the controller may assign a higher priority to a motion of the user recognized in the first main region than a motion of the user recognized in the first sub region, and assign a higher priority to a motion of the user recognized in the second main region than a motion of the user recognized in the second sub region.

In some implementations, the controller may generate a control command corresponding to the recognized motion of the user based on the priority.

In some implementations, the controller may generate a corresponding control command when the motion of the user occurring in the first region is recognized as a motion associated with control of driving of the vehicle, and not generate the control command when the motion of the user occurring in the second region is recognized as the motion associated with the control of the driving of the vehicle.

In some implementations, the controller may generate a corresponding control command when the motion of the user occurring in the second region is recognized as a motion associated with control of an infotainment system while the vehicle is traveling, and not generate the control command when the motion of the user occurring in the first region is recognized as the motion associated with the control of the infotainment system.

In some implementations, the controller may determine the motion of the user as a motion of a user in the first region or the second region when the motion of the user occurring in the third region is recognized, not generate a control command when the motion of the user is recognized as a motion associated with control of an infotainment system of the user in the first region, and generate the control command when the motion of the user is recognized as a motion associated with control of a navigation of the user in the second region.

In some implementations, the controller may determine priorities of the motion of the user recognized in the first region and the motion of the user recognized in the second region based on a traveling environment of the vehicle.

In some implementations, the controller may recognize at least one of a first motion of the user occurring in the first region, a second motion of the user occurring in the second region, or a third motion of the user occurring in the third region, and generate a corresponding control command when at least two of the first motion, the second motion, or the third motion correspond to a preset motion combination.

In some implementations, the controller may set a size or width of the first region, the second region, or the third region based on a frequency or pattern of the motion of the user occurring in at least one of the first region, the second region, or the third region.

In some implementations, the user may include a first user in a driver's seat and a second user in a passenger seat, wherein the first region may be defined as a region including the driver's seat, and wherein the second region may be defined as a region including the passenger seat.

In some implementations, the controller may recognize the motion of the user occurring in the third region as the motion of the user occurring in the first region or the second region based on a direction of entry of the motion of the user into the third region or a pre-stored user image.

In some implementations, the camera may include a first camera for capturing a user image of the first user, and a second camera for capturing a user image of the second user.

In some implementations, the controller may recognize the motion of the user occurring in at least one of the first region, the second region, or the third region based on priorities preset to the first region and the second region.

In another aspect of the present disclosure, a method for recognizing a motion in a vehicle is disclosed. In one form, a method includes acquiring, with a camera, a user image; dividing, with a controller, the user image into a first region, a second region, and a third region in which the first region and the second region overlap each other; and recognizing, with the controller, a motion of a user occurring in at least one of the first region, the second region, or the third region.

In some implementations, the first region may include a first main region and a first sub region and the second region may include a second main region and a second sub region.

In some implementations, the recognizing of the motion of the user occurring in at least one of the first region, the second region, or the third region may include assigning a higher priority to a motion of the user recognized in the first main region than a motion of the user recognized in the first sub region, and assigning a higher priority to a motion of the user recognized in the second main region than a motion of the user recognized in the second sub region.

In some implementations, recognizing of the motion of the user occurring in at least one of the first region, the second region, or the third region may include generating a corresponding control command when the motion of the user occurring in the first region is recognized as a motion associated with control of driving of the vehicle, and not generating the control command when the motion of the user occurring in the second region is recognized as the motion associated with the control of the driving of the vehicle.

In some implementations, recognizing the motion of the user occurring in at least one of the first region, the second region, or the third region may include generating a corresponding control command when the motion of the user occurring in the second region is recognized as a motion associated with control of an infotainment system while the vehicle is traveling, and not generating the control command when the motion of the user occurring in the first region is recognized as the motion associated with the control of the infotainment system.

In some implementations, recognizing the motion of the user occurring in at least one of the first region, the second region, or the third region may include recognizing the motion of the user occurring in at least one of the first region, the second region, or the third region based on priorities preset to the first region and the second region.

DETAILED DESCRIPTION

FIG. 1is a block diagram illustrating one form of a motion recognition device of a vehicle.FIG. 2illustrates an operation of a motion recognition device of a vehicle in a first region, a second region, and a third region of a user image.

Referring toFIG. 1, a motion recognition device100of a vehicle may include a camera110and a controller120.

The camera110may capture a user in the vehicle to acquire a user image. The user may include a first user in a driver's seat and a second user in a passenger seat. For example, the camera110may be disposed on a dashboard in the vehicle to capture the first user and the second user.

The controller120may use the user image to recognize a motion of the user. To this end, the controller120may divide the user image into a plurality of regions. For example, the controller120may divide the user image into a first region, a second region, and a third region. In this connection, the third region may be defined as a region in which the first region and the second region overlap each other. For example, the first region, which is a region including the driver's seat, may be defined as a polygonal, circular, elliptical region defined around the first user. The second region, which is a region including the passenger seat, may be defined as a polygonal, circular, or elliptical region defined around the second user.

The controller120may recognize a motion of the user occurring in at least one of the first region, the second region, or the third region. Thus, the controller120may recognize motions generated from a plurality of users in the vehicle. The controller120may recognize the user motion based on various well-known motion recognition algorithms.

Accordingly, the controller120may generate a control command based on the recognized user motion and may control various devices in the vehicle or operations of the devices using the control command.

Hereinafter, an operation of the controller120will be described in detail with reference toFIG. 2.

Referring toFIG. 2, an example of the user image acquired using the camera110is shown. As described above, the controller120may divide a user image200into a first region210, a second region220, and a third region230.

The first region210may include a first main region211and a first sub region212and the second region220may include a second main region221and a second sub region222. For example, the first main region211may be defined as a rectangular region defined around a face of the first user, and the first sub region212may be defined as the remaining region other than the first main region211in the first region210. For example, the second main region221may be defined as a rectangular region defined around a face of the second user, and the second sub region222may be defined as the remaining region other than the second main region221in the second region220.

The controller120may assign a higher priority to a motion of the user recognized in the first main region211than a motion of the user recognized in the first sub region212(e.g., the motion of the first user). The controller120may assign a higher priority to a motion of the user recognized in the second main region221than a motion of the user recognized in the second sub region222.

For example, when a motion of lowering a volume of an audio device (e.g., infotainment system) in the vehicle is recognized from the first user in the first sub region212and a motion of increasing the volume of the audio device in the vehicle is recognized from the first user in the first main region211, the controller120may generate the control command based on the motion of increasing the volume of the audio device of the vehicle recognized in the first main region211.

When the motion of the user occurring in the first region210is recognized as a motion associated with control of driving of the vehicle, the controller120may generate a control command corresponding thereto. Further, when the motion of the user occurring in the second region220is recognized as a motion related to the motion associated with the control of the driving of the vehicle, the controller120may not generate the control command. That is, the controller120may recognize the motion of the first user prior to the motion of the second user with respect to the motion associated with the control of the driving of the vehicle. This is to support safe driving of the vehicle by ensuring a priority of the first user, that is, a driver, with respect to the control of the driving of the vehicle. For example, the motion associated with the control of the driving of the vehicle may include, but is not limited to, shift control of the vehicle, speed control, control of operations of driving assistance devices, and the like.

For example, when a shift control motion (e.g., a motion of changing a gear stage) is recognized in the first region210, the controller120may generate a control command corresponding thereto. However, when the shift control motion is recognized in the second region220, the controller120may not generate a control command corresponding thereto.

While the vehicle is traveling, when the motion of the user occurring in the second region220is recognized as a motion associated with navigation control (e.g., control related to the infotainment system of the vehicle), the controller120may generate a corresponding control command. Further, when the motion of the user occurring in the first region210is recognized as the motion associated with the navigation control, the controller120may not generate the control command. That is, when a user motion related to a control command (e.g., a motion for inputting a navigation destination while the vehicle is traveling) that cannot be manipulated by the first user is recognized in the first region210, the controller120may not generate a control command. In one example, the user motion associated with the control command that cannot be manipulated by the first user may further include a user motion for Bluetooth connection in addition to the motion for the destination input described above.

The controller120may determine priorities of the motion of the user recognized in the first region210and the motion of the user recognized in the second region220based on a traveling environment of the vehicle.

For example, when a speed of the vehicle is equal to or above a certain speed (e.g., 100 km/h), the controller120may recognize the motion of the user recognized in the second region220prior to the motion of the user recognized in the first region210for safety. In addition, for example, when the driving assistance device of the vehicle is in operation, the controller120may recognize the motion of the user recognized in the first region210prior to the motion of the user recognized in the second region220even when the speed of the vehicle is equal to or greater than a certain speed.

The controller120may recognize a first motion of the user occurring in the first region210, a second motion of the user occurring in the second region220, and a third motion of the user occurring in the third region230, and, when at least two of the first motion, the second motion, or the third motion correspond to a preset motion combination, the controller120may generate a control command corresponding thereto. For example, the preset motion combination may include various combinations, such as a case in which the first motion is ‘◯’ and the second motion is ‘Δ’. When the first motion and the second motion correspond to the preset motion combination, the controller120may improve a convenience of the user and generate interest by generating a control command for executing various functions, such as playing music, outputting a user setting screen, and the like.

When the motion of the user is occurring in the third region230, the controller120may recognize the motion of the user occurring in the third region230as the motion of the user occurring in the first region210or the second region220based on a direction of entry of the user's motion into the third region230or a matching result with a pre-stored user image (e.g., identifying whom the user's hand belongs to by analyzing an image). For example, the controller120may recognize the motion generated by the hand of the user entering the third region230from the first region210as the motion of the first user, and, therefore, recognize the motion generated by the hand of the user as the motion of the user occurring in the first region210.

For example, when the motion of the user occurring in the third region230is recognized, the controller120may determine the motion of the user as the motion of the user seated in the first region210or the second region220. When the motion of the user is recognized as a motion associated with the navigation control of the user seated in the first region210, the controller120may not generate a control command. Further, when the motion of the user is recognized as a motion associated with the navigation control of the user seated in the second region220, the controller120may generate the control command.

The controller120may recognize the motion of the user occurring in at least one of the first region210, the second region220, or the third region230based on priorities preset on the first region210and the second region220. To this end, the controller120may assign priorities to the first region210and the second region220for a region in which the motion of the user corresponding to the control command for executing a specific function of the vehicle is recognized. For example, the controller120may assign a higher priority to the first region210than the second region220in a case in which the motion of the user corresponding to the control command for operating the infotainment system of the vehicle is recognized.

FIGS. 3 to 5are diagrams for illustrating a change in a third region.

Referring toFIGS. 3 to 5, the controller120may determine a size or width of the first region210, the second region220, or the third region230based on a frequency or pattern of a motion of the user occurring in at least one of the first region210, the second region220, or the third region230. For example, the controller120may obtain the frequency or pattern information of the user's motion through a user's input or communication with other devices, or accumulate and analyze a user image acquired for a predetermined period of time via the camera110to generate the frequency or pattern information of the user's motion.

For example, referring toFIG. 3, when a frequency of the motion of the user occurring in the first region210is higher than a frequency of the motion of the user occurring in the second region220, the controller120may determine the first region210to be wider than the second region220.

For example, referring toFIG. 4, when a distribution of positions where the motions of the user occurring in the first region210and the second region220is sporadic (that is, when the pattern of the motion of the user is sporadic), the controller120may enlarge the size or width of the first region210and the second region220.

In addition, for example, when a frequency of the motion of the user occurring in the third region230is higher than the frequencies of the motions occurring in the first region210and the second region220, the controller120may set the third region230to be the widest. This may be achieved by enlarging a region where the first region210and the second region220overlap.

As described above, the controller120may improve a motion recognition rate by adaptively determining the sizes or widths of the first region210, the second region220, and the third region230.

FIG. 6is a block diagram illustrating another form of a camera of a motion recognition device of a vehicle.

Referring toFIG. 6, the camera110of the motion recognition device of the vehicle may include a first camera111and a second camera112.

The first camera111may acquire a user image of the first user (e.g., driver).

The second camera112may acquire a user image of the second user (e.g., passenger sitting in the passenger seat).

As such, the camera110of the motion recognition device of the vehicle may include the first camera111and the second camera112to acquire the user image for each user, thereby more improving the motion recognition rate.

FIG. 7is a block diagram illustrating another form of a motion recognition device of a vehicle.

Referring toFIG. 7, a motion recognition device300of the vehicle may include a camera310, a voice recognition device320, and a controller330. In this connection, the camera310may be substantially the same as the camera110described with reference toFIGS. 1 and 6, and thus redundant description thereof will be omitted.

The voice recognition device320may recognize an input voice of the user. For example, the voice recognition device320may identify the user based on the input voice of the user, and may analyze an emotional state of the user, a relationship between the users, a gender of the user, and the like.

The controller330may set the priorities regarding the recognition of the motion of the user in the first region210and the second region220based on the user recognition result, the emotional state of the user, the relationship between the users, the gender of the user, and the like delivered from the voice recognition device320. For example, when the first user is older than the second user or takes control of the conversation, the controller330may assign a higher priority to the first region210than the second region220. Further, for example, when the second user is female and the first user is male, the controller330may assign a higher priority to the second region220than the first region210.

The controller330may assign the priorities regarding the recognition of the motion of the user in the first region210and the second region220based on the user image transmitted from the camera310. For example, when a first user motion recognition is more urgent in an analysis result of facial expression or the like of the user shown in the user image, the controller330may assign a higher priority to the first region210than the second region220.

Therefore, according to the motion recognition device300of the vehicle according to another embodiment of the present disclosure, the user motion recognition may be achieved adaptively for various situations, and thus convenience of the user may be improved and the motion recognition rate may also be improved.

FIG. 8is a flowchart illustrating a method for recognizing a motion in a vehicle according to an embodiment of the present disclosure.

Referring toFIG. 8, the method for recognizing the motion in the vehicle according to an embodiment of the present disclosure may include acquiring the user image (S110), dividing the user image into the first region, the second region, and the third region in which the first region and the second region overlap each other (S120), and recognizing the motion of the user occurring in at least one of the first region, the second region, or the third region (S130).

Hereinafter, the operations S110to S130described above will be described in detail with reference toFIGS. 1 and 2.

In S110, the camera110may capture the user in the vehicle to acquire the user image. The user may include the first user in the driver's seat and the second user in the passenger seat.

In S120, the controller120may divide the user image into the plurality of regions. For example, the controller120may divide the user image into the first region, the second region, and the third region. In this connection, the third region may be defined as the region in which the first region and the second region overlap each other. For example, the first region, which is the region including the driver's seat, may be defined as the polygonal, circular, elliptical region defined around the first user. The second region, which is the region including the passenger seat, may be defined as the polygonal, circular, or elliptical region defined around the second user.

In S130, the controller120may recognize the motion of the user occurring in at least one of the first region, the second region, or the third region. Thus, the controller120may recognize the motion generated from the plurality of users in the vehicle. The controller120may recognize the user motion based on the various well-known motion recognition algorithms. The controller120may generate the control command based on the recognized user motion, and control the various devices in the vehicle and the operations of the devices using the control command.

In one example, the first region210may include the first main region211and the first sub region212and the second region220may include the second main region221and the second sub region222.

The controller120may assign the higher priority to the motion of the user recognized in the first main region211than the motion of the user recognized in the first sub region212(e.g., the motion of the first user). The controller120may assign the higher priority to the motion of the user recognized in the second main region221than the motion of the user recognized in the second sub region222.

When the motion of the user occurring in the first region210is recognized as the motion associated with the control of the driving of the vehicle, the controller120may generate the control command corresponding thereto. Further, when the motion of the user occurring in the second region220is recognized as the motion related to the motion associated with the control of the driving of the vehicle, the controller120may not generate the control command. That is, the controller120may recognize the motion of the first user prior to the motion of the second user with respect to the motion associated with the control of the driving of the vehicle. This is to support the safe driving of the vehicle by ensuring the priority of the first user, that is, the driver, with respect to the control of the driving of the vehicle.

While the vehicle is traveling, when the motion of the user occurring in the second region220is recognized as the motion associated with navigation control, the controller120may generate the corresponding control command. Further, when the motion of the user occurring in the first region210is recognized as the motion associated with the navigation control, the controller120may not generate the control command. That is, when the user motion related to the control command (e.g., the motion for inputting the navigation destination while the vehicle is traveling) that cannot be manipulated by the first user is recognized in the first region210, the controller120may not generate the control command.

The controller120may determine the priorities of the motion of the user recognized in the first region210and the motion of the user recognized in the second region220based on the traveling environment of the vehicle.

The controller120may recognize the first motion of the user occurring in the first region210and the second motion of the user occurring in the second region220, and, when the first motion and the second motion correspond to the preset motion combination, the controller120may generate the control command corresponding thereto. For example, the preset motion combination may include the various combinations, such as the case in which the first motion is ‘◯’ and the second motion is ‘Δ’. When the first motion and the second motion correspond to the preset motion combination, the controller120may improve the convenience of the user and generate the interest by generating the control command for executing the various functions, such as playing music, outputting the user setting screen, and the like.

When the motion of the user is occurring in the third region230, the controller120may recognize the motion of the user occurring in the third region230as the motion of the user occurring in the first region210or the second region220based on the direction of the entry of the user's motion into the third region230or the matching result with the pre-stored user image (e.g., identifying whom the user's hand belongs to by analyzing the image).

The controller120may recognize the motion of the user occurring in at least one of the first region210, the second region220, or the third region230based on the priorities preset on the first region210and the second region220.

FIG. 9illustrates a computing system for executing one form of a method for recognizing a motion in the vehicle.

Referring toFIG. 9, forms of the method for recognizing the motion in the vehicle described above may also be implemented via the computing system. A computing system1000may include at least one processor1100, a memory1300, a user interface input device1400, a user interface output device1500, storage1600, and a network interface1700connected via a bus1200.

The processor1100may be a central processing unit (CPU) or a semiconductor device that performs processing on instructions stored in the memory1300and/or the storage1600. The memory1300and the storage1600may include various types of volatile or non-volatile storage media. For example, the memory1300may include a ROM (Read Only Memory) and a RAM (Random Access Memory).

Thus, the operations of the method or the algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware or a software module executed by the processor1100, or in a combination thereof. The software module may reside on a storage medium (that is, the memory1300and/or the storage1600) such as a RAM, a flash memory, a ROM, an EPROM, an EEPROM, a register, a hard disk, a removable disk, a CD-ROM. The exemplary storage medium is coupled to the processor1100, which may read information from, and write information to, the storage medium. In another method, the storage medium may be integral with the processor1100. The processor and the storage medium may reside within an application specific integrated circuit (ASIC). The ASIC may reside within the user terminal. In another method, the processor and the storage medium may reside as individual components in the user terminal.

The description above is merely illustrative of the technical idea of the present disclosure, and various modifications and changes may be made by those skilled in the art without departing from the essential characteristics of the present disclosure.

Therefore, the embodiments and implementations disclosed in the present disclosure are not intended to limit the technical idea of the present disclosure but to illustrate the present disclosure, and the scope of the technical idea of the present disclosure is not limited by the embodiments. The scope of the present disclosure should be construed as being covered by the scope of the appended claims, and all technical ideas falling within the scope of the claims should be construed as being included in the scope of the present disclosure.

According to the device and the method for recognizing the motion in the vehicle according to an embodiment of the present disclosure, the motions generated from the plurality of users in the vehicle may be recognized.

According to the device and the method for recognizing the motion in the vehicle according to an embodiment of the present disclosure, the recognition rate of the motion generated from the user in the vehicle may be improved.