GESTURE RECOGNITION METHOD AND APPARATUS, DEVICE, AND MEDIUM

The present application discloses a gesture recognition method and apparatus, a device and a medium. The gesture recognition method includes: taking a first image in the image sequence as a target image; determining at least two lines corresponding to fingers in the target image; determining valid lines from the at least two lines; under a condition that a first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis are less than a first angle threshold, determining a subsequent image of the target image as the target image; recognizing, according to coordinate values of key points on the valid lines in the first image and a previous image of the target image, a gesture corresponding to changing from the first image to the previous image.

TECHNICAL FIELD

The present application relates to a technical field of artificial intelligence, and in particular, to a gesture recognition method and apparatus, a device, and a medium.

BACKGROUND

With the development of artificial intelligence interaction technology, gesture control technology is more and more used in various fields, such as vehicle systems, smart homes, Virtual Reality (VR) interactions, smart phones, and the like. Here, gesture recognition is the basis of gesture control.

In the related art, gestures are mainly recognized through a special gesture recognition model. However, gesture recognition through a gesture recognition model is computationally intensive and inefficient.

SUMMARY

The embodiments of the present application provide a gesture recognition method and apparatus, a device, and a medium.

In a first aspect, the embodiments of the present application provide a gesture recognition method, including:acquiring an image sequence including a human hand, and taking a first image in the image sequence as a target image;determining at least two lines corresponding to fingers in the target image;determining valid lines from the at least two lines;under a condition that the first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis of the target image are less than a first angle threshold, determining a subsequent image of the target image as the target image, and returning to the step of determining at least two lines corresponding to the fingers in the target image until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold;determining, according to coordinate values of key points on the valid lines in the first image and a second image, a relative moving distance of the human hand in a direction of the first coordinate axis, wherein the second image is a previous image of the target image;recognizing, according to the relative moving distance, a gesture corresponding to changing from the first image to the second image.

In a second aspect, the embodiments of the present application provide a gesture recognition apparatus, including:an acquisition module configured to acquire an image sequence including a human hand, and take a first image in the image sequence as a target image;a first determination module configured to determine at least two lines corresponding to fingers in the target image;a second determination module configured to determine valid lines from the at least two lines;a third determination module configured to determine, under a condition that the first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis of the target image are less than a first angle threshold, a subsequent image of the target image as the target image, and trigger the first determination module until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold;a fourth determination module configured to determine, according to coordinate values of key points on the valid lines in the first image and a second image, a relative moving distance of the human hand in a direction of the first coordinate axis, wherein the second image is a previous image of the target image;a recognition module configured to recognize, according to the relative moving distance, a gesture corresponding to changing from the first image to the second image.

In a third aspect, the embodiments of the present application provide an electronic device including: a processor; a memory; and programs or instructions stored on the memory and executable by the processor, wherein the programs or instructions, when executed by the processor, implement steps of the method according to the first aspect.

In a fourth aspect, the embodiments of the present application provide a readable storage medium having programs or instructions stored thereon, wherein the programs or instructions, when executed by a processor, implement steps of the method according to the first aspect.

In a fifth aspect, the embodiments of the present application provide a chip including a processor and a communication interface coupled to the processor, wherein the processor executes programs or instructions to implement steps of the method according to the first aspect.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present application will be clearly described below in combination with the drawings in the embodiments of the present application, and obviously, the described embodiments are part of the embodiments of the present application rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those ordinary skilled in the art fall within the protection scope of the present application.

The terms “first”, “second” and the like in the specification and the claim of the present application are used to distinguish similar objects and not to describe a particular order or sequence. It should be understood that the data described in this way can be interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. Also, the objects distinguished by the terms “first”, “second” and the like are usually belong to one class, and the number of the objects is not limited, for example, the first object may include one or more objects. Furthermore, in the specification and the claim, the statement “and/or” indicates at least one of the connected objects, and the character “/” generally indicates that the associated objects are in an “or” relationship.

The gesture recognition method and apparatus, the device and the medium provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios thereof with reference to the accompanying drawings.

FIG.1is a flow schematic diagram of a gesture recognition method provided by an embodiment of the present application. As shown inFIG.1, the gesture recognition method may include:S101: acquiring an image sequence including a human hand, and taking a first image in the image sequence as a target image;S102: determining at least two lines corresponding to fingers in the target image;S103: determining valid lines from the at least two lines; under a condition that the first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis of the target image are less than a first angle threshold, executing S104, and under a condition that the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold, executing S105;S104: determining a subsequent image of the target image as the target image, and continuing to execute S102;S105: determining, according to coordinate values of key points on the valid lines in the first image and a second image, a relative moving distance of the human hand in a direction of the first coordinate axis, wherein the second image is a previous image of the target image;S106: recognizing, according to the relative moving distance, a gesture corresponding to changing from the first image to the second image.

In some possible implementations of the embodiments of the present application, the subsequent image of the target image refers to an image that is located after and is adjacent to the target image; the previous image of the target image refers to an image that is located before and is adjacent to the target image.

The specific implementations of the above steps will be described in detail below.

In the embodiments of the present application, with regard to the image sequence including the human hand, the first image in the image sequence is determined as the target image; at least two lines corresponding to the fingers in the target image are determined; the valid lines from the at least two lines are determined; under a condition that the first number of the valid lines is greater than the first number threshold and the first angles each between a corresponding one of all of the valid lines and the first coordinate axis of the target image are less than the first angle threshold, the subsequent image of the target image is determined as the target image, and the method is returned to the step of determining at least two lines corresponding to the fingers in the target image until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold; according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image, the gesture corresponding to changing from the first image to the previous image of the target image is recognized. Since the operation amount of recognizing the gesture according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image is smaller than the operation amount of recognizing the gesture by the gesture recognition model, the gesture recognition efficiency can be improved.

In some possible implementations of the embodiments of the present application, in S101, the human hand can be tracked by a camera, and image acquisition may be performed for the human hand, thereby obtaining the image sequence including the human hand.

In some possible implementations of the embodiments of the present application, in S101, the first image may be a first image with the existence of the human hand. Specifically, the human hand detection can be performed on the images in the image sequence successively, and the detected first image with the existence of the human hand can be taken as the target image.

The embodiments of the present application does not define the method used in the human hand detection, and any available method can be applied to the embodiments of the present application.

In some possible implementations of the embodiments of the present application, in S102, the target image may be input into a human hand key point identification model to obtain the key points of the human hand in the target image. Here, the human hand includes 21 key points, wherein the 21 key points include a wrist key point, key points corresponding to fingertips of fingers, and key points corresponding to joints of fingers. As shown inFIG.2,FIG.2is a schematic diagram of key points of a human hand provided by an embodiment of the present application.

A line composed of key points 1-4 is taken as a thumb line l0, a line composed of key points 5-8 is taken as an index finger line l1, a line composed of key points 9-12 is taken as a middle finger line l2, a line composed of key points 13-16 is taken as a ring finger line l3, and a line composed of key points 17-20 is taken as a little finger line l4. Key point 0 is the wrist key point.

In some possible implementations of the embodiments of the present application, S103may include: calculating, for each first target line of the at least two lines, second angles each between the first target line and another line of the at least two lines other than the first target line; determining, under a condition that all of the second angles are greater than a second angle threshold, the first target line as an invalid line; determining lines from the at least two lines other than the invalid line as the valid lines.

Illustratively, taking the little finger line l4 in the image P1 as an example, the angles each between the little finger line l4 and the thumb line l0, the index finger line l1, the middle finger line l2, and the ring finger line l3 are calculated.

It is assumed that the angles each between the little finger line l4 and the thumb line l0, the index finger line l1, the middle finger line l2, and the ring finger line l3 are greater than the second angle threshold, the little finger line l4 is determined as the invalid line.

Similarly, it can be determined whether the thumb line l0, the index finger line l1, the middle finger line l2, and the ring finger line l3 are invalid lines. When the invalid lines are determined, lines other than the invalid lines among the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 may be determined as the valid lines.

The embodiments of the present application do not define the method used for determining the first angle between the line and the first coordinate axis of the target image, and any available method can be applied to the embodiments of the present application. For example, with regard to the index finger line l1, a vector of the key point 5 and the key point 8 in the index finger line l1 is calculated, and then a cosine value of an angle between the vector and the first coordinate axis is calculated. Further, the angle between the vector and the first coordinate axis is determined according to the cosine value, and this angle is used as the angle between the index finger line l1 and the first coordinate axis.

In some possible implementations of the embodiments of the present application, S105may include: calculating first differences between coordinate components on the first coordinate axis of the coordinate values of the key points on the valid lines in the first image and the second image and a coordinate component of a target key point on the first coordinate axis, wherein the target key point is a wrist key point; calculating second differences between coordinate components on a second coordinate axis of the coordinate values of the key points on the valid lines in the second image and a coordinate component of the target key point on the second coordinate axis; determining the relative moving distance according to the first differences and the second differences.

Illustratively, taking the first image P1 and the second image P2 as an example, wherein the first coordinate axis is the X axis of the image, and the second coordinate axis is the Y axis of the image. The X axis is a transverse axis, and the Y axis is a longitudinal axis.

It is assumed that the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 are all valid lines.

With regard to the first image P1, the first differences between the X axis components of the four key points 1-4 on the thumb line l0 and the X axis component of the wrist key point 0 are calculated respectively, and the first differences between the X axis components of the key points 1-4 and the X axis component of the wrist key point 0 are dx1-0, dx2-0, dx3-0and dx4-0, respectively.

Similarly, the first differences between the X axis components of the key points on the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the first image P1 and the X axis component of the wrist key point 0 can also be calculated, and the first differences between the X axis components of the key points on the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 and the X axis component of the wrist key point 0 can also be calculated.

Similarly, the second differences between the Y axis components of the key points on the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3 and the little finger line l4 in the second image P2 and the Y axis component of the wrist key point 0 can also be calculated.

In some possible implementations of the embodiments of the present application, determining the relative moving distance according to the first differences and the second differences may include: calculating, for each first valid line in the first image and the second image, a first average value of the first differences corresponding to the first valid line; calculating, for each second valid line in the second image, a second average value of the second differences corresponding to the second valid line; determining the relative moving distance according to the first average value and the second average value.

Illustratively, taking the first image P1 and the second image P2 as an example, wherein the first coordinate axis is the X axis of the image, and the second coordinate axis is the Y axis of the image. The X axis is a transverse axis, and the Y axis is a longitudinal axis.

It is assumed that the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 are all valid lines.

With regard to the first image P1, the first differences between the X axis components of the four key points 1-4 on the thumb line l0 and the X axis component of the wrist key point 0 are calculated respectively, and the first differences between the X axis components of the key points 1-4 and the X axis component of the wrist key point 0 are dx1-0, dx2-0, dx3-0and dx4-0, respectively.

Then, the first average value of the first differences corresponding to the thumb line l0 in the first image P1 on the X-axis may be: DXP1-l0=(dx1-0+dx2-0+dx3-0+dx4-0)/4.

Similarly, the first average values of the first differences corresponding to the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the first image P1 on the X-axis, and the first average values of the first differences corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 on the X-axis can be calculated (i.e., DXP1-l1, DXP1-l2, DXP1-l3, DXP1-l4, DXP2-l0, DXP2-l1, DXP2-l2, DXP2-l3and DXP2-l4, respectively).

Similarly, the second average values of the second differences corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 on the Y-axis can also be calculated (i.e., DYP2-l0, DYP2-l1, DYP2-l2, DYP2-l3and DYP2-l4, respectively).

According to the first average values corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the first image P1 on the X-axis, and the first average values corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 on the X-axis (i.e., DXP1-l0, DXP1-l1, DXP1-l2, DXP1-l3, DXP1-l4, DXP2-l0, DXP2-l1, DXP2-l2, DXP2-l3and DXP2-l4, respectively), as well as the second average values corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 on the Y-axis (i.e., DYP2-l0, DYP2-l1, DYP2-l2, DYP2-l3and DYP2-l4, respectively), the relative moving distance may be determined.

In some possible implementations of the embodiments of the present application, determining the relative moving distance according to the first average value and the second average value may include: calculating a third average value of third differences between first average values corresponding to the second image and first average values corresponding to the first image; calculating a fourth average value of absolute values of fourth differences of second average values of every two adjacent valid lines in the second image; determining the relative moving distance according to the third average value and the fourth average value.

Illustratively, taking the first image P1 and the second image P2 as an example, wherein the first coordinate axis is the X axis of the image, and the second coordinate axis is the Y axis of the image.

The third average value DX of the third differences between the first average values corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the second image P2 on the X-axis (i.e., DXP2-l0, DXP2-l1, DXP2-l2, DXP2-l3and DXP2-l4, respectively) and the first average values corresponding to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3, and the little finger line l4 in the first image P1 on the X-axis (i.e., DXP1-l0, DXP1-l1, DXP1-l2, DXP1-l3, and DXP1-l4, respectively) may be calculated.

The fourth average value YP2 of the absolute values of the fourth differences of the second average values of every two adjacent valid lines in the second image P2 may be calculated. It will be appreciated that the thumb is adjacent to the index finger, the index finger is adjacent to the middle finger, the middle finger is adjacent to the ring finger, and the ring finger is adjacent to the little finger. The difference between the second average values corresponding to the thumb line and the index finger line may be: DYP2-l0−DYP2-l1; the difference between the second average values corresponding to the index finger line and the middle finger line may be: DYP2-l1−DYP2-l2; the difference between the second average values corresponding to the middle finger line and the ring finger line may be: DYP2-l2−DYP2-l3; the difference between the second average values corresponding to the ring finger line and the little finger line may be: DYP2-l3-DYP2-l4.

The relative moving distance may be determined based on the third average value DX and the fourth average value YP2.

In some possible implementations of the embodiments of the present application, determining the relative moving distance according to the third average value and the fourth average value includes: determining a quotient of the third average value and the fourth average value as the relative moving distance.

Illustratively, taking the first image P1 and the second image P2 as an example, wherein the first coordinate axis is the X axis of the image, and the second coordinate axis is the Y axis of the image.

The process of determining the gesture corresponding to the two images when the first coordinate axis is the Y axis and the second coordinate axis is the X axis may be similar to the process of determining the gesture corresponding to the two images when the first coordinate axis is the X axis and the second coordinate axis is the Y axis. For further details, reference can be made to the above-mentioned process of determining the gesture corresponding to the two images when the first coordinate axis is the X axis and the second coordinate axis is the Y axis. The embodiments of the present application will not be described in detail with respect to this.

In some possible implementations of the embodiments of the present application, the first coordinate axis is an X axis of the image, and the second coordinate axis is a Y axis of the image. Recognizing, according to the relative moving distance, the gesture corresponding to changing from the first image to the second image may include: under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognizing the gesture as a right-dial gesture; under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognizing the gesture as a left-dial gesture.

In some possible implementations of the embodiments of the present application, the first coordinate axis is a Y axis of the image, and the second coordinate axis is an X axis of the image. Recognizing, according to the relative moving distance, the gesture corresponding to changing from the first image to the second image may include: under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognizing the gesture as a down-dial gesture; under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognizing the gesture as an up-dial gesture.

In some possible implementations of the embodiments of the present application, the number of the valid lines in the two images may be determined to be different. For example, the valid lines in the image P1 may be determined to be the index finger line l1, the middle finger line l2, the ring finger line l3 and the little finger line l4, and the valid lines in the image P2 may be determined to be the middle finger line l2, the ring finger line l3 and the little finger line l4, then the index finger line l1 in the image P1 can be ignored, and the relative moving distance of the human hand in the direction of the first coordinate axis can be determined only according to the middle finger lines l2, the ring finger lines l3 and the little finger lines l4 in the images P1 and P2.

Here, the process of determining the relative moving distance of the human hand in the direction of the first coordinate axis according to the middle finger line l2, the ring finger line l3 and the little finger line l4 is similar to the process of determining the relative moving distance of the human hand in the direction of the first coordinate axis according to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3 and the little finger line l4 in the above-mentioned embodiments. Specifically, reference can be made to the process of determining the relative moving distance of the human hand in the direction of the first coordinate axis according to the thumb line l0, the index finger line l1, the middle finger line l2, the ring finger line l3 and the little finger line l4. The embodiments of the present application will not be described in detail with respect to this.

It should be noted that for the gesture recognition method provided in the embodiments of the present application, the executing subject may be a gesture recognition apparatus, or a control module in the gesture recognition apparatus for executing the gesture recognition method. In the embodiments of the present application, the gesture recognition apparatus provided by the embodiment of the present application is illustrated by performing the gesture recognition method.

FIG.3is a schematic structural diagram of a gesture recognition apparatus provided by an embodiment of the present application. The gesture recognition apparatus300may include:an acquisition module301configured to acquire an image sequence including a human hand, and take a first image in the image sequence as a target image;a first determination module302configured to determine at least two lines corresponding to fingers in the target image;a second determination module303configured to determine valid lines from the at least two lines;a third determination module304configured to determine, under a condition that the first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis of the target image are less than a first angle threshold, a subsequent image of the target image as the target image, and trigger the first determination module302until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold;a fourth determination module305configured to determine, according to coordinate values of key points on the valid lines in the first image and a second image, a relative moving distance of the human hand in a direction of the first coordinate axis, wherein the second image is a previous image of the target image;a recognition module306configured to recognize, according to the relative moving distance, a gesture corresponding to changing from the first image to the second image.

In the embodiments of the present application, with regard to the image sequence including the human hand, the first image in the image sequence is determined as the target image; at least two lines corresponding to the fingers in the target image are determined; the valid lines from the at least two lines are determined; under a condition that the first number of the valid lines is greater than the first number threshold and the first angles each between a corresponding one of all of the valid lines and the first coordinate axis of the target image are less than the first angle threshold, the subsequent image of the target image is determined as the target image, and the method is returned to the step of determining at least two lines corresponding to the fingers in the target image until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold; according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image, the gesture corresponding to changing from the first image to the previous image of the target image is recognized. Since the operation amount of recognizing the gesture according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image is smaller than the operation amount of recognizing the gesture by the gesture recognition model, the gesture recognition efficiency can be improved.

In some possible implementations of the embodiments of the present application, the second determination module303includes:a first calculation sub-module configured to calculate, for each first target line of the at least two lines, second angles each between the first target line and another line of the at least two lines other than the first target line;a first determination sub-module configured to determine, under a condition that all of the second angles are greater than a second angle threshold, the first target line as an invalid line;a second determination sub-module configured to determine lines from the at least two lines other than the invalid line as the valid lines.

In some possible implementations of the embodiments of the present application, the fourth determination module305includes:a second calculation sub-module configured to calculate first differences between coordinate components on the first coordinate axis of the coordinate values of the key points on the valid lines in the first image and the second image and a coordinate component of a target key point on the first coordinate axis, wherein the target key point is a wrist key point;a third calculation sub-module configured to calculate second differences between coordinate components on a second coordinate axis of the coordinate values of the key points on the valid lines in the second image and a coordinate component of the target key point on the second coordinate axis;a third determination sub-module configured to determine the relative moving distance according to the first differences and the second differences.

In some possible implementations of the embodiments of the present application, the third determination sub-module includes:a first calculation unit configured to calculate, for each first valid line in the first image and the second image, a first average value of the first differences corresponding to the first valid line;a second calculation unit configured to calculate, for each second valid line in the second image, a second average value of the second differences corresponding to the second valid line;a determination unit configured to determine the relative moving distance according to the first average value and the second average value.

In some possible implementations of the embodiments of the present application, the determination unit includes:a first calculation sub-unit configured to calculate a third average value of third differences between first average values corresponding to the second image and first average values corresponding to the first image;a second calculation sub-unit configured to calculate a fourth average value of absolute values of fourth differences of second average values of every two adjacent valid lines in the second image;a determination sub-unit configured to determine the relative moving distance according to the third average value and the fourth average value.

In some possible implementations of the embodiments of the present application, the determination sub-unit is specifically configured to:determine a quotient of the third average value and the fourth average value as the relative moving distance.

In some possible implementations of the embodiments of the present application, the first coordinate axis is an X axis of the image, and the second coordinate axis is a Y axis of the image;the recognition module306is specifically configured to:under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognize the gesture as a right-dial gesture;under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognize the gesture as a left-dial gesture.

In some possible implementations of the embodiments of the present application, the first coordinate axis is a Y axis of the image, and the second coordinate axis is an X axis of the image;the recognition module306is specifically configured to:under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognize the gesture as a down-dial gesture;under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognize the gesture as an up-dial gesture.

The gesture recognition apparatus in the embodiments of the present application may be an apparatus, or may be a component, an integrated circuit, or a chip in a terminal. The apparatus may be a mobile electronic device or a non-mobile electronic device. By way of example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a vehicle-mounted electronic device, a wearable device, a ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), and the like; the non-mobile electronic device may be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-help machine, and the like; and the embodiments of the present application are not particularly limited.

The gesture recognition apparatus in the embodiments of the present application may be an apparatus having an operating system. The operating system may be an Android operating system, may be an iOS operating system, and may also be other possible operating systems, and the embodiments of the present application are not particularly limited.

The gesture recognition apparatus provided by the embodiments of the present application can implement various processes in the gesture recognition method embodiments ofFIGS.1and2, and in order to avoid repetition, the description thereof will not be repeated here.

Optionally, as shown inFIG.4, the embodiments of the present application also provide an electronic device400, including: a processor401; a memory402; and programs or instructions stored on the memory402and executable by the processor401, wherein the programs or instructions, when executed by the processor401, implement the various processes of the above gesture recognition method embodiments, and can achieve the same technical effect. In order to avoid repetition, the description thereof will not be repeated here.

It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic devices and the non-mobile electronic devices as described above.

In some possible implementations of the embodiments of the present application, the processor401may include a central processing unit (CPU), or an Application Specific Integrated Circuit (ASIC), or may be configured as one or more integrated circuits implementing the embodiments of the present application.

In some possible implementations of the embodiments of the present application, the memory402may include a Read-Only Memory (ROM), a random Access Memory (RAM), a magnetic disk storage media device, an optical storage media device, a flash memory device, and an electrical, optical, or other physical/tangible memory storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g. memory devices) encoded with software including computer-executable instructions, and the software, when executed (e.g. by one or more processors), is operable to perform the operations described with reference to the gesture recognition method according to the embodiments of the present application.

FIG.5is a schematic hardware structural diagram of an electronic device implementing embodiments of the present application.

The electronic device500includes, but is not limited to: a radio frequency unit501, a network module502, an audio output unit503, an input unit504, a sensor505, a display unit506, a user input unit507, an interface unit508, a memory509, and a processor510.

As will be appreciated by those skilled in the art, the electronic device500may also include a power source (e.g. a battery) for powering the various components. The power source may be logically connected to the processor510through a power management system, which may implement functions of managing charge, discharge, and power consumption. The structure of the electronic device shown inFIG.5is not to be construed as limiting the electronic device, and the electronic device may include more or fewer components than shown, or some components may be combined, or the components may be in a different arrangement, which will not be described in detail herein.

Here, the processor510is configured to: take a first image in an acquired image sequence as a target image; determine at least two lines corresponding to fingers in the target image; determine valid lines from the at least two lines; under a condition that the first number of the valid lines is greater than a first number threshold and first angles each between a corresponding one of all of the valid lines and a first coordinate axis of the target image are less than a first angle threshold, determine a subsequent image of the target image as the target image, and under a condition that the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold, determine, according to coordinate values of key points on the valid lines in the first image and a second image, a relative moving distance of the human hand in a direction of the first coordinate axis, wherein the second image is a previous image of the target image; recognize, according to the relative moving distance, a gesture corresponding to changing from the first image to the second image.

In the embodiments of the present application, with regard to the image sequence including the human hand, the first image in the image sequence is determined as the target image; at least two lines corresponding to the fingers in the target image are determined; the valid lines from the at least two lines are determined; under a condition that the first number of the valid lines is greater than the first number threshold and the first angles each between a corresponding one of all of the valid lines and the first coordinate axis of the target image are less than the first angle threshold, the subsequent image of the target image is determined as the target image, and the method is returned to the step of determining at least two lines corresponding to the fingers in the target image until the first number is not greater than the first number threshold or one or more of the first angles are not less than the first angle threshold; according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image, the gesture corresponding to changing from the first image to the previous image of the target image is recognized. Since the operation amount of recognizing the gesture according to the coordinate values of the key points on the valid lines in the first image and the previous image of the target image is smaller than the operation amount of recognizing the gesture by the gesture recognition model, the gesture recognition efficiency can be improved.

In some possible implementations of the embodiments of the present application, the processor510is specifically configured to:calculate, for each first target line of the at least two lines, second angles each between the first target line and another line of the at least two lines other than the first target line;determine, under a condition that all of the second angles are greater than a second angle threshold, the first target line as an invalid line;determine lines from the at least two lines other than the invalid line as the valid lines.

In some possible implementations of the embodiments of the present application, the processor510is specifically configured to:calculate first differences between coordinate components on the first coordinate axis of the coordinate values of the key points on the valid lines in the first image and the second image and a coordinate component of a target key point on the first coordinate axis, wherein the target key point is a wrist key point;calculate second differences between coordinate components on a second coordinate axis of the coordinate values of the key points on the valid lines in the second image and a coordinate component of the target key point on the second coordinate axis;determine the relative moving distance according to the first differences and the second differences.

In some possible implementations of the embodiments of the present application, the processor510is specifically configured to:calculate, for each first valid line in the first image and the second image, a first average value of the first differences corresponding to the first valid line;calculate, for each second valid line in the second image, a second average value of the second differences corresponding to the second valid line;determine the relative moving distance according to the first average value and the second average value.

In some possible implementations of the embodiments of the present application, the processor510is specifically configured to:calculate a third average value of third differences between first average values corresponding to the second image and first average values corresponding to the first image;calculate a fourth average value of absolute values of fourth differences of second average values of every two adjacent valid lines in the second image;determine the relative moving distance according to the third average value and the fourth average value.

In some possible implementations of the embodiments of the present application, the processor510is specifically configured to:determine a quotient of the third average value and the fourth average value as the relative moving distance.

In some possible implementations of the embodiments of the present application, the first coordinate axis is an X axis of the image, and the second coordinate axis is a Y axis of the image; the processor510is specifically configured to:under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognize the gesture as a right-dial gesture;under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognize the gesture as a left-dial gesture.

In some possible implementations of the embodiments of the present application, the first coordinate axis is a Y axis of the image, and the second coordinate axis is an X axis of the image; the processor510is specifically configured to:under a condition that an absolute value of the relative moving distance is greater than a preset distance threshold and the relative moving distance is greater than 0, recognize the gesture as a down-dial gesture;under a condition that the absolute value of the relative moving distance is greater than the preset distance threshold and the relative moving distance is less than 0, recognize the gesture as an up-dial gesture.

It should be appreciated that in embodiments of the present application, the input unit504may include a Graphics Processing Unit (GPU)5041and a microphone5042, and the Graphics Processing Unit5041may process image data for still pictures or videos obtained by an image capture device, such as a camera, in either a video capture mode or an image capture mode. The display unit506may include a display panel5061, and the display panel5061may be configured in the form of a liquid crystal display, an organic light emitting diode, and the like. The user input unit507includes a touch panel5071and other input devices5072. The touch panel5071is also known as a touch screen. The touch panel5071may include two parts, a touch detection device and a touch controller. Other input devices5072may include, but are not limited to, a physical keyboard, function keys (e.g. volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be described in detail herein. The memory509may be used to store software programs and various data including, but not limited to, applications and operating systems. The processor510may integrate an application processor, which primarily handles operating systems, user interfaces, applications, and the like, and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may also not be integrated into the processor510.

The embodiments of the present application also provide a readable storage medium having programs or instructions stored thereon, wherein the programs or instructions, when executed by a processor, implement the various processes of the above gesture recognition method embodiments. Further, the same technical effect can be achieved, and in order to avoid repetition, the description thereof will not be repeated.

Here, the processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, and examples of the computer-readable storage medium include a non-transitory computer-readable storage medium, such as ROM, RAM, magnetic or optical disks, and the like.

The embodiments of the present application also provide a chip including a processor and a communication interface coupled to the processor, wherein the processor executes programs or instructions to implement various processes of the above gesture recognition method embodiments. Further, the same technical effect can be achieved, and in order to avoid repetition, the description thereof will not be repeated.

It should be understood that the chip mentioned in embodiments of the present application may also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip, and the like.

It should be noted that, in this document, the terms “comprising”, “including” or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes other elements that are not explicitly listed but inherent to such a process, method, article or device. Without further limitation, an element defined by the term “comprising . . . ” does not preclude presence of additional elements in a process, method, article or device that includes the element. Furthermore, it should be noted that the scope of the methods and apparatus in the embodiments of the present application is not limited to the order of performing the functions shown or discussed, and may include performing the functions in a substantially simultaneous manner or in a reverse order depending on the functionality involved. For example, the methods described may be performed in a different order than described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

From the description of the embodiments given above, it will be clear to a person skilled in the art that the method of the embodiments described above can be implemented by means of software plus a necessary general purpose hardware platform, but of course also by means of hardware, the former being in many cases a better embodiment. Based on this understanding, the technical solution of the present application essentially or contributing to the relevant technology can be embodied in the form of a computer software product which is stored in a storage medium (such as a ROM/RAM, a magnetic diskette, an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in various embodiments of the present application.

Although the embodiments of the present application have been described above with reference to the accompanying drawings, the present application is not limited to the above-mentioned specific embodiments, which are merely illustrative and not restrictive. Those skilled in the art, with the inspiration from the present application, can make many changes without departing from the protection scope of the present application and the appended claims.