Method, module and system of gesture recognition

A gesture recognition method includes performing a binarization process on an image to obtain a binarized image, wherein the binarized image includes a plurality of foreground pixels and a plurality of background pixels; determining whether the plurality of foreground pixels surrounds at least a first background pixel; and determining a gesture complying with a predefined gesture in response to determination that the plurality of foreground pixels surrounds the at least a first background pixel.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a gesture recognition method, a gesture recognition module and a gesture recognition system, and more particularly, to a gesture recognition method, a gesture recognition module and a gesture recognition system capable of recognizing a pinch gesture.

2. Description of the Prior Art

Gesture recognition technology may provide good human-computer interaction and improve the convenience of operation, which has been widely recognized in the industry. Currently, the lens that may recognize gestures may be divided into two types: long-range lens and short-range lens. The long-range lens may be disposed in front of a user, such as a TV or a game console, and the short-range lens may be disposed on a wearable device on the user. Although the two types of lenses can recognize the gesture, the distance and position are different. The gesture information that can be obtained is of course different from the usage habits, resulting in different gestures for these two distances.

With the development of Virtual Reality (VR) devices or Augmented Reality (AR), short-range lenses may be installed on a VR device or an AR device. The VR device or the AR devices may be operated by gesture. Currently, a specific gesture represents a specific effect, which achieves an effect as shortcuts. For example, the gesture of pinching using the thumb and the index finger is easy for the user to achieve, which may be used to instruct the VR device or the AR device to perform a specific operation.

However, recognizing the pinch gesture effectively has not been developed. Therefore, it is necessary to improve the prior art.

SUMMARY OF THE INVENTION

The present invention provides a gesture recognition method, a gesture recognition module and a gesture recognition system capable of recognizing a pinch gesture, to improve over disadvantages of the prior art.

An embodiment of the present invention discloses a gesture recognition method, applied in a gesture recognition module. The gesture recognition method is configured to determine a gesture to operate an electronic device correspondingly. The gesture recognition comprises providing an image capturing device, wherein the image capturing device is configured to sense the gesture to generate an image; performing a binarization process on the image, to obtain a binarized image, wherein the binarized image comprises a plurality of foreground pixels and a plurality of background pixels, the plurality of foreground pixels is corresponding to a foreground pixel value, and the plurality of background pixels is corresponding to a background pixel value; determining whether the plurality of foreground pixels within the binarized image surrounds at least a first background pixel, wherein the first background pixel is one background pixel of the plurality of background pixels; and determining the gesture complying with a predefined gesture in response to determination that the plurality of foreground pixels surrounds the at least a first background pixel, to operate the electronic device correspondingly.

An embodiment of the present invention further discloses a gesture recognition module, configured to operate an electronic device. The gesture recognition module comprises an image capturing device, configured to sense a gesture and generate an image; an image processing unit, performing a binarization process on the image, to obtain a binarized image, wherein the binarized image comprises a plurality of foreground pixels and a plurality of background pixels, the plurality of foreground pixels is corresponding to a foreground pixel value, and the plurality of background pixels is corresponding to a background pixel value; a determining unit, configured to perform the following steps: determining whether the plurality of foreground pixels within the binarized image surrounds at least a first background pixel, wherein the first background pixel is one background pixel of the plurality of background pixels; and determining the gesture complying with a predefined gesture in response to determination that the plurality of foreground pixels surrounds the at least a first background pixel, to operate the electronic device correspondingly.

An embodiment of the present invention further discloses a gesture recognition system, comprising an electronic device; and a gesture recognition module, electrically coupled to the electronic device and configured to operate the electronic device, comprising: an image capturing device, configured to sense a gesture and generate an image; an image processing unit, performing a binarization process on the image, to obtain a binarized image, wherein the binarized image comprises a plurality of foreground pixels and a plurality of background pixels, the plurality of foreground pixels is corresponding to a foreground pixel value, and the plurality of background pixels is corresponding to a background pixel value; and a determining unit, configured to perform the following steps: determining whether the plurality of foreground pixels within the binarized image surrounds at least a first background pixel, wherein the first background pixel is one background pixel of the plurality of background pixels; and determining the gesture complying with a predefined gesture in response to determination that the plurality of foreground pixels surrounds the at least a first background pixel, to operate the electronic device correspondingly.

DETAILED DESCRIPTION

FIG. 1is a function block diagram of a gesture recognition system1according to an embodiment of the present invention.FIG. 2is a schematic diagram of an appearance of a gesture recognition module10disposed within an electronic device12according to an embodiment of the present invention. The gesture recognition system1comprises the gesture recognition module10and the electronic device12. The gesture recognition module10may be a part of the electronic device12and be electronically connected to an internal component of the electronic device12, or the gesture recognition module10may be an external module of the electronic device12and be electronically coupled to an internal component of the electronic device12. The gesture recognition module10and the electronic device12may form the gesture recognition system1. The electronic device12, for example, may be a VR (Virtual Reality) device, an AR (Augmented Reality) device, or a MR (Mixed Reality) device, i.e., a device which may compute, display or output images. The gesture recognition module10may be disposed in front of the electronic device12. When the user wears the electronic device12, the gesture recognition module10may recognize the gesture of the user, and the electronic device12may perform related computation or operation in response to the gesture recognized by the gesture recognition module10.

The gesture recognition module10comprises an image capturing device11, an image processing unit13and a determining unit18. In an embodiment of the present invention, the image capturing device11may capture a gesture image of the user in front of the image capturing device11in an optical way. The image capturing device11may comprise a light-emitting element14and a light-sensing unit16. In another embodiment of the present invention, the image capturing device11may be an RGB camera, a depth camera or other optical camera which can obtain optical images. The image capturing device11may even be a sensing device which can obtain the gesture image using, for example, ultrasound or radar device. The light-emitting element14is configured to emit an incident light, and may be a light-emitting element emitting an invisible light, e.g., an IR LED (Infrared Light Emitting Diode). The light-sensing unit16is configured to receive a reflected light corresponding to the incident light emitted by the light-emitting element14, and the light-sensing unit16may comprise a lens and a light-sensing element. The light-sensing element may be a light-sensing element for the invisible light, e.g., the light-sensing element may be an IR photo diode. In another embodiment of the present invention, the light-sensing element may be a CCD (charge coupled device), a CMOS (complementary metal-oxide semiconductor) device or other device, which is not limited thereto. In the embodiments of the present invention, the light-sensing unit16may generate an image IMG according to the reflected light (e.g., IR) corresponding to the incident light emitted by the light-emitting element14. The image IMG may be a gray level image, or may have been processed to be a gray level image. After the image processing unit13receives the image IMG, the image processing unit13performs a binarization process to generate a binarized image BG, for distinguishing a foreground, probably comprising a gesture, from a background, probably comprising no gesture. The binarization process may be realized by software program, hardware device or firmware. In another embodiment of the present invention, if the image obtained by the image capturing device (stated in the above) is not a gray level image but an RGB image (or other image), then the image processing unit13performs operation converting a received image into a gray level image. Moreover, the image processing unit13is not limited to performing the gray level image, the image processing unit13may perform other image process operations such as an erosion operation or a dilation operation. The image processing unit13may be independent of the determining unit18or be integrated into the determining unit18.

The determining unit18receives the binarized image BG, and determines whether a user gesture is a pinch gesture according to the binarized image BG. After the determining unit18determines that the user gesture is the pinch gesture, the determining unit18may generate a recognition result signal S_P corresponding to the pinch gesture to the electronic device12, and the electronic device12may perform the computation or operation corresponding to the pinch gesture according to the user gesture recognized by the gesture recognition module10. The determining unit18may comprise a processor, a memory and an algorithm software program code. The processor may be a CPU (central processing unit), an AP (application processor), a general purpose (or specific purpose) programmable microprocessor, DSP (digital signal processor, DSP), ISP (image signal processor), GPU (graphics processing unit) or other devices, IC (integrated circuit) and their combinations. The memory, either fixed or movable, may be a RAM (random access memory), a ROM (read-only memory), a flash memory, a hard driver or other memory device, IC and their combinations. The algorithm software program code is loaded to the memory and configured to perform the processed stated in the below. The determining unit18of the gesture recognition module10of the present invention may be independent of the electronic device12, or may be included in the electronic device12, or a part of which is independent of the electronic device12and the other part of which is included in the electronic device12.

Operations of the determining unit18determining the user gesture according to the image IMG may be referred toFIG. 3.FIG. 3is a schematic diagram of a process30according to an embodiment of the present invention. The process30may be performed by the image processing unit13and the determining unit18, which comprises the following steps:

Step302: Perform a binarization process on the image IMG, to obtain a binarized image BN, wherein the binarized image BN comprises a plurality of foreground pixels PXFGand a plurality of background pixels PXBG, the plurality of foreground pixels PXFGis corresponding to a foreground pixel value, and the plurality of background pixels PXBGis corresponding to a background pixel value.

Step304: Determine whether the plurality of foreground pixels PXFGwithin the binarized image BN surrounds at least a first background pixel PXBG1of the plurality of background pixels PXBG. If yes, go to Step306; otherwise, go to Step308.

Step306: Determine the gesture as the pinch gesture, and generate the recognition result signal S_P to the electronic device to operate the electronic device correspondingly.

In Step302, the binarized image BN, generated by the image processing unit13by performing the binarization process on the image IMG, comprises either white pixels or black pixels. White pixels represent the pixels of the light-sensing unit16at which the received light (e.g., IR) is strong, and the white pixels are regarded as foreground pixels. Black pixels represent the pixels of the light-sensing unit16at which the received light is weak, and the black pixels are regarded as background pixels. Therefore, the binarized image BN comprises the plurality of foreground pixels PXFGand the plurality of background pixels PXBG. The plurality of foreground pixels PXFGis corresponding to a foreground pixel value, and the plurality of background pixels PXBGis corresponding to a background pixel value. In an embodiment, under a condition of the pixel value of the image being represented by 8 bits, the background pixel value is 0 (presenting as black) and the foreground pixel value is 255 (presenting as white).

In Step304, the determining unit18determines whether the plurality of foreground pixels PXFGwithin the binarized image BN surrounds the first background pixel PXBG1, wherein the first background pixel PXBG1is one (background pixel) of the plurality of background pixels PXBG. If the background pixel PXBG1surrounded by the foreground pixels PXFGexists in the binarized image BN, the determining unit18determines that the pinch gesture is in the image IMG (Step306). Otherwise, if there is no background pixel PXBG1surrounded by the foreground pixels PXFGin the binarized image BN, the determining unit18determines that there is no pinch gesture in the image IMG.

According to the process30, the determining unit18may determine whether the pinch gesture is in the image IMG. For example, please refer toFIG. 4andFIG. 5.FIG. 4andFIG. 5are schematic diagrams of a binarized image BN4and a binarized image BN5, respectively. The binarized images BN4and BN5may comprise the foreground pixels PXFGand the background pixels PXBG. InFIG. 4, since there is no background pixel PXBGwhich is surrounded by the foreground pixels PXFG, the determining unit18would determine that there is no pinch gesture is in the binarized image BN4, after performing the process30. In comparison, inFIG. 5, a part of background pixels PXBG, i.e., the background pixels PXBG′, are surrounded by the foreground pixels PXFG, and another part of background pixels PXBG, i.e., the background pixels PXBG″ are not surrounded by the foreground pixels PXFG. It means, the background pixels PXBGin the binarized image BN5comprises the background pixels PXBG′ which is surrounded by the foreground pixels PXFGand the background pixels PXBG″ which is not surrounded by the foreground pixels PXFG. The foreground pixels PXFGsurrounding the background pixel PXBG′ represents that the background pixels PXBG′ are within a closed region formed by the foreground pixels PXFG. Notably, the background pixels PXBG′ and the background pixels PXBG″ are not connected with each other. Therefore, the determining unit18would determine the binarized image BN5includes the pinch gesture, after performing the process30in this case.

Details of Step304may be referred toFIG. 6.FIG. 6is a schematic diagram of a process60according to an embodiment of the present invention. The process60is an embodiment of details of Step304, which may be executed by the determining unit18. The process60comprises the following steps:

Step602: Obtain a plurality of outmost pixels PXOMin the binarized image BN.

Step604: Obtain a plurality of second background pixels PXBG2in the binarized image BN, wherein the plurality of second background pixels PXBG2is the plurality of background pixels PXBGin the binarized image BN which is connected with the plurality of outmost pixels PXOM.

Step606: Determine whether a third background pixel PXBG3, which is neither one of the plurality of outmost pixels PXOMnor one of the plurality of second background pixels PXBG2, exists among the plurality of background pixels PXBG. If yes, go to Step608; otherwise, go to Step610.

Step608: Determine the plurality of foreground pixels PXFGsurrounds the first background pixel PXBG1.

In Step602, the determining unit18obtains the plurality of outmost pixels PXOM. For example, the determining unit18may obtain the leftmost pixels, the upmost pixels, the rightmost pixels and the downmost pixels in the binarized image BN as the plurality of outmost pixels PXOM. Please refer toFIG. 7.FIG. 7is a schematic diagram of the plurality of outmost pixels PXOMin the binarized image BN. InFIG. 7, the slash squares represent the outmost pixels PXOMin the binarized image BN, and the white squares represent a plurality of interior pixels PXINin the binarized image BN. The numbers in the slash squares represent label values of the outmost pixels PXOM(which will be described later on). The numbers in the white squares represent pixel values of the interior pixels PXIN. As can be seen fromFIG. 7, the outmost pixels PXOMis on the periphery (four sides) of the binarized image BN. In addition, the binarized image BN is composed of the plurality of outmost pixels PXOMand the plurality of interior pixels PXIN, which means that a pixel within the binarized image BN is either the outmost pixel PXOM(one of the outmost pixels PXOM) or the interior pixel PXIN(one of the interior pixel PXIN).

In Step604, the determining unit18obtains the plurality of second background pixels PXBG2which is connected with the plurality of outmost pixels PXOM. The plurality of second background pixels PXBG2connected with the plurality of outmost pixels PXOMcomprises the second background pixels PXBG2_1, PXBG2_2, . . . , PXBG2_n-l, PXBG2_n. For example, the second background pixel PXBG2_1is directly adjacent to the outmost pixels PXOM, the second background pixel PXBG2_2is directly adjacent to the second background pixel PXBG2_1(which represents that the second background pixel PXBG2_2is indirectly adjacent to at least an outmost pixel PXOMthrough the second background pixel PXBG2,1), and so on and so forth, the second background pixel PXBG2,nis directly adjacent to the second background pixel PXBG2,n-1(which represents that the second background pixel PXBG2,nis indirectly adjacent to at least an outmost pixel PXOMthrough the second background pixel PXBG2,1-PXBG2,n-1). In this case, the plurality of second background pixels PXBG2composed of the second background pixel PXBG2,1-PXBG2,nis directly or indirectly connected with the plurality of outmost pixels PXOM. In short, “the plurality of second background pixels PXBG2being directly or indirectly connected with the plurality of outmost pixels PXOM” represents that “one second background pixel PXBG2is directly adjacent to one of the plurality of outmost pixels PXOM” or “one second background pixel PXBG2is indirectly adjacent to one of the plurality of outmost pixels PXOMthrough other second background pixel(s) PXBG2”.

In Step606, the determining unit18determines whether the third background pixel PXBG3, which is neither one of the outmost pixels PXOMnor one of the second background pixel PXBG2, exists among the plurality of background pixels PXBG. In other words, the determining unit18determines whether an background pixel, other than the plurality of outmost pixels PXOMand the plurality of second background pixels PXBG2, exists among the plurality of background pixels PXBGin the binarized image BN (if yes, the background pixel is named as the third background pixel PXBG3). If the third background pixel PXBG3, which is neither directly nor indirectly connected with the plurality of outmost pixels PXOM, exists in the binarized image BN, it represents that the third background pixel PXBG3is surrounded by the plurality of foreground pixels PXFGand is not connected with any of the second background pixel PXBG2(at this time, the third background pixel PXBG3is the first background pixel PXBG1stated in Step304), and the determining unit18would determines that there is at least a first background pixel PXBG1which is surrounded by the plurality of foreground pixels PXFG(Step608).

Details of Step604are not limited. For the ease of the computation, a method of tag/class can be used, for example, to label all of the background pixel PXBGas (different) label/tag value(s), and label the second background pixel PXBG2(i.e., the background pixel PXBGwhich are connected with the plurality of outmost pixels PXOM) as a same label value. Finally, the determining unit18may check whether a background pixel with a different label value, which is different from the label value of the outmost pixels PXOM, exists. If yes, the determining unit18may determine that the third background pixel PXBG3exists.

Specifically,FIG. 8is a schematic diagram of a process80according to an embodiment of the present invention. The process80is an embodiment of details of Step604, which may be executed by the determining unit18. The process80comprises the following steps:

Step802: Label a plurality of outmost labels corresponding to the plurality of outmost pixels PXOMas a first label value TG1, wherein the plurality of foreground pixels PXFGare labeled as a null label TGN.

Step804: According to a scanning order, sequentially and individually determine whether a plurality of interior pixel values of the plurality of interior pixels PXINin the binarized image BN is the background pixel value, and sequentially and individually determine a plurality of internal labels corresponding to the plurality of interior pixels PXIN, wherein the plurality of interior pixels PXINin the binarized image BN is not the plurality of outmost pixels PXOM.

Step806: According to the plurality of internal labels of the plurality of interior pixels PXIN, obtain the plurality of second background pixels PXBG2connected with the plurality of outmost pixels PXOM.

Before performing the process80, all pixels in the binarized image BN are defaulted to have the null label TGN. After the process80is executed, all of the background pixels and the outmost pixels in the binarized image BN would be labeled as specific label value(s), other than the null label TGN, and only the foreground pixels are remained as the null label TGN.

In Step802, the determining unit18labels the plurality of outmost labels corresponding to the plurality of outmost pixels PXOMas the first label value TG1, and labels the plurality of foreground pixels PXFGas the null label TGN. In an embodiment, the first label value TG1 may be 1. Please again refer toFIG. 7. InFIG. 7, the determining unit18labels the plurality of outmost pixels PXOMas 1, In addition, the interior pixel values of the interior pixels PXINinFIG. 7may be either 0 (the background pixel value) or 255 (the foreground pixel value).

In Step804, the determining unit18determines whether each interior pixel PXIN_k(k=1, . . . ,K) in the binarized image BN is the background pixel, and determine an interior label corresponding to the interior pixel PXIN_k, according to the scanning order. An index k may represent the scanning order, and K is a number of the interior pixels PXIN. In an embodiment, the determining unit18may execute a process90first and then execute a process A0, to accomplish Step804.FIG. 9is a schematic diagram of the process90according to an embodiment of the present invention. As shown inFIG. 9, the process90comprises the following steps:

Step902: Determine whether the interior pixel value of the interior pixel PXIN_kis the background pixel value. If yes, go to Step904; otherwise, go to Step912.

Step904: Determine whether a first neighboring pixel PXNB1_kto which the interior pixel PXIN_kis subsequent is with a first neighboring label value TGNB1_k. If yes, go to Step906; otherwise, go to Step905.

Step905: Determine whether a second neighboring pixel PXNB2_kprevious to the interior pixel PXIN_kis with a second neighboring label value TGNB2_k. If yes, go to Step908; otherwise, go to Step910.

Step906: Label a first internal label TGIN_kcorresponding to the interior pixel PXIN_kas the first neighboring label value TGNB1_k.

Step908: Label the first internal label TGIN_kcorresponding to the interior pixel PXIN_kas the second neighboring label value TGNB2_k.

Step910: Label the first internal label TGIN_kas a second label value TG2.

Step912: Determine whether k is equal to K. If yes, go to Step916; otherwise, go to Step914.

In the process90, the scanning order may be an order of raster scan or an order of circular scan, which is not limited thereto. The order of raster scan and the order of circular scan (from outward to inward) are illustrated inFIG. 11. InFIG. 11, each square represents a pixel, and the number within the square represents the index k of the scanning order. According to different scanning orders, the index k of the scanning order may be distributed differently.

In Step900, all of the interior pixel PXIN_k(k=1, . . . , K) are defaulted to be with the null label TGN.

In Step902, the determining unit18determines whether the interior pixel PXIN_kis the background pixel. If the interior pixel PXIN_kis the foreground pixel (k<K), the determining unit18would go to Step912and914, to maintain the foreground pixels (the interior pixel PXIN_k) to be with the null label TGN, and determines whether the next interior pixel PXIN_k+1is the background pixel. If the interior pixel PXIN_kis the background pixel, the determining unit18would execute some steps of Steps904-910, to label the first internal label TGIN_kto the interior pixel PXIN_k, wherein the first internal label TGIN_kmay be one of the label values TG1, TG2 and the neighboring label values TGNB1_k, TGNB2_k.

In Step904, the determining unit18determines whether the first neighboring pixel PXNB1_kis labeled as the first neighboring label value TGNB1_k, which is equivalent to determining whether the first neighboring pixel PXNB1_kis the background pixel. The first neighboring pixel PXNB1_kis directly adjacent to the interior pixel PXIN_kin a first direction, and according to the scanning order, the first neighboring pixel PXNB1_kis previous to the interior pixel PXIN_k. That is, the first neighboring pixel PXNB1_kmay be another interior pixel PXIN_k-k′(k′=1, . . . , k−1) which is previous to the interior pixel PXIN_k. In addition, the first direction may be parallel to one of a horizontal direction and a vertical direction. Take the raster scan from left to right as an example, the first neighboring pixel PXNB1_kto which the interior pixel PXIN_kis subsequent may be the pixel on the left of the interior pixel PXIN_k(where the first direction may be in a leftward direction). The first neighboring pixel PXNB1_kmay also be the outmost pixels PXOM. For circular scan, the first neighboring pixel PXNB1_kmay be the interior pixel PXIN_k-1corresponding to the previous iteration.

If the first neighboring pixel PXNB1_kto which the interior pixel PXIN_kis subsequent has already been labeled as the first neighboring label value TGNB1_k, in Step906, the interior pixel PXIN_kwould inherit the first neighboring label value TGNB1_kof the first neighboring pixel PXNB1_k. If the first neighboring pixel PXNB1_kto which the interior pixel PXIN_kis subsequent is the outmost pixels PXOM, the first neighboring label value TGNB1_kwould be the first label value TG1.

If the first neighboring pixel PXNB1_kto which the interior pixel PXIN_kis subsequent is the foreground pixel (representing that the first neighboring pixel PXNB1_kis labeled as the null label TGN), in Step905, the determining unit18further determines whether the second neighboring pixel PXNB2_kis the background pixel, which is to determine whether the second neighboring pixel PXNB2_khas been labeled as the second neighboring label value TGNB2_k. The second neighboring pixel PXNB2_kis the pixel previous to the interior pixel PXIN_kaccording to the scanning order, which may be directly adjacent to the interior pixel PXIN_kin a second direction. The second direction may be parallel to one of the horizontal direction and the vertical direction, and the second direction is perpendicular to the first direction. For the raster scan, the second neighboring pixel PXNB2_kmay be on the up of the interior pixel PXIN_kand directly adjacent to the interior pixel PXIN_k(where the second direction may be in an upward direction), which may also be the outmost pixels PXOM. For the circular scan from outward to inward, take “the interior pixel PXIN_klocated at the right of the binarized image BN and the interior pixel PXIN_k-1corresponding to the previous iteration located at the up of the interior pixel PXIN_kcorresponding to the current iteration” as an example, the first neighboring pixel PXNB1_kis the interior pixel PXIN_k-1located at the up of the interior pixel PXIN_k, and the second neighboring pixel PXNB2_kis another pixel at the right of the interior pixel PXIN_k.

If the second neighboring pixel PXNB2_khas been labeled as the second neighboring label value TGNB2_k, in Step908, the interior pixel PXIN_kwould inherit the second neighboring label value TGNB2_kof the second neighboring pixel PXNB2_k. If the second neighboring pixel PXNB2_kis the outmost pixels PXOM, the second neighboring label value TGNB2_kwould be the first label value TG1.

If the first neighboring pixel PXNB1_kand the second neighboring pixel PXNB2_kare the foreground pixels (which represents that the first neighboring pixel PXNB1_kand the second neighboring pixel PXNB2_kare labeled as the null label TGN), in Step910, the determining unit18labels the interior pixel PXIN_kas the second label value TG2. That is, the determining unit18determines that the first internal label TGIN_kof the first interior pixel PXIN_kis the second label value TG2, where the second label value TG2 is different from the first label value TG1. In an embodiment, suppose that the first label value TG1 is equal to 1, the second label value TG2 is set to be equal to 2 when the determining unit18executes the 1stiteration of the process90, the second label value TG2 is set to be equal to 3 when the determining unit18executes the 2nditeration of the process90, (and so on and so forth) and the second label value TG2 is set to be equal to (n+1) when the determining unit18executes the nthiteration of the process90.

For clearly illustrating the process90, please refer toFIGS. 12-14.FIG. 12andFIG. 13illustrate intermediate results of performing the process90on binarized images BN12and BN13, respectively.FIG. 14illustrates an intermediate result and a final result of performing the processes90and A0on a binarized image BN14. In the embodiments ofFIGS. 12-14, the horizontal raster scan from left to right is adopted. The numbers in the white squares inFIGS. 12-14represent the pixel values after the binarization process, and the numbers in the slash squares represent the label values.

In the embodiment illustrated inFIG. 12, the determining unit18is to perform the determining operations of Step904-910on the (2,2)-th interior pixel (denoted as PXIN_(2,2)) in the binarized image BN12, as shown in the subfigure12a. Since the pixel value of the interior pixel PXIN_(2,2)is 0, which is the background pixel (Step902), and the pixel PXIN_(1,2)adjacent to and on the left of the interior pixel PXIN_(2,2)is with the label value as 1 (Step904), the interior pixel PXIN_(2,2)would inherit the label value 1 of the pixel PXIN_(1,2)(Step906), as shown in the subfigure12b, and proceeds to determine the pixel PXIN_(3,2), as shown in the subfigure12c.

In the embodiment illustrated inFIG. 13a, the determining unit18is to perform the determining operation of Step904-910on the (2,2)-th interior pixel in the binarized image BN13, as show in the subfigure13. Since the pixel value of the interior pixel PXIN_(2,2)is 255, which is the foreground pixels, the determining unit18remains/keeps the null label for the interior pixel PXIN_(2,2), and proceeds to the determining operation of Step904-910on the next interior pixel PXIN_(3,2), as shown in the subfigure13b(corresponding to the path from Step902to Steps912and914inFIG. 9). Although the interior pixel PXIN_(2,2)adjacent to and on the left of the interior pixel PXIN_(3,2)(at this time, the interior pixel PXIN_(2,2)is corresponding to the first neighboring pixel adjacent to the interior pixel PXIN_(3,2)) has the null label (corresponding to the path from Step904to Step905), the pixel PXIN_(1,3)on the up of and adjacent to the interior pixel PXIN_(3,2)(at this time, the interior pixel PXIN_(1,3)is corresponding to the second neighboring pixel adjacent to the interior pixel PXIN_(3,2)) has the label value as 1 (Step905). Hence, the interior pixel PXIN_(3,2)would inherit the label value 1 of the pixel PXIN_(3,1)(Step908), as shown in the subfigure13c(the label value 1 is corresponding to the first label value TG1).

In the embodiment illustrated inFIG. 14, the determining unit18is to perform the determining operation of Step904-910on the (3,3)-th interior pixel (denoted as PXIN_(3,3)) in the binarized image BN14, as shown in the subfigure14a. Since the pixel PXIN_(2,3)adjacent to PXIN_(3,3)on the left (corresponding to the first neighboring pixel) and the pixel PXIN_(3,2)adjacent to PXIN_(3,3)on the up (corresponding to the second neighboring pixel) are both foreground pixels, the determining unit18executes Step910at the 1sttime, where the determining unit18labels the interior pixel PXIN_(3,3)as 2, as shown in the subfigure14b, where the label value 2 is corresponding to the second label value TG2). Similar, when the determining the interior pixel PXIN_(5,3), the determining unit18executes Step910at the 2ndtime, where the determining unit18labels the interior pixel PXIN_(5,3)as 3, as shown in the subfigure14c, where the label value 3 is also corresponding to the second label value TG2. In addition, the subfigure14dillustrates the final result of the determining unit18performing the process90.

By the process90, by keeping the null label for the foreground pixels, the determining unit18may preliminarily exclude the foreground pixels. Furthermore, the determining unit18may label the background pixels as specific label values. After that, by the process A0, the determining unit18may determine all of the background pixels which are connected with the outmost pixels PXOM, i.e., the plurality of second background pixels PXBG2stated in Step604.

FIG. 10is a schematic diagram of the process A0according to an embodiment of the present invention. The determining unit18may perform the process A0on each of the internal background pixels PXBG_INin the binarized image BN. The internal background pixel PXBG_INrepresent that “the pixel PXBG_INis an interior pixel of the plurality of interior pixels PXIN” and “the pixel PXBG_INis a background pixel of the plurality of background pixels PXBG”. As shown inFIG. 10, the process A0comprises the following steps:

Step A02: Determine whether an internal background pixel PXBG_IN_mis connected with the plurality of outmost pixels PXOMin the binarized image BN. If yes, go to Step A04; otherwise, go to Step A06.

Step A04: Label the internal background label TGBG_IN_mcorresponding to the internal background pixel PXBG_IN_mto be the first label value TG1.

In Step A02, the internal background pixel PXBG_IN_mbeing connected with the plurality of outmost pixels PXOMrepresents that “the internal background pixel PXBG_IN_mis directly adjacent to the outmost pixels PXOM” or “the internal background pixel PXBG_IN_mis indirectly adjacent to the outmost pixels PXOMthrough other internal background pixel PXBG_IN_m′”.

If the internal background pixel PXBG_IN_mis connected with the plurality of outmost pixels PXOM, in Step A04, the determining unit18would change the internal background label TGBG,IN,mcorresponding to the internal background pixel PXBG_IN_m, for all m, as the first label value TG1, which is the same as the one with the outmost pixels PXOM.

After performing the process A0on every internal background pixel PXBG_INin the binarized image BN, the internal background pixels PXBG_INwhich are connected with the outmost pixels PXOMall have the first label value TG1, as the outmost pixels PXOMdo.

Please again refer toFIG. 14. As can be seen from the subfigure14d, after the process90is performed, the label values of the interior pixels PXIN_(2,5)-PXIN_(5,5)in the binarized image BN14are all 1, the label values of the interior pixels PXIN_(3,3), PXIN_(3,4)-PXIN_(5,4)in the binarized image BN14are all 2, and the label value of the interior pixel PXIN_(5,3)in the binarized image BN14is 3.

The interior pixels PXIN_(2,5)-PXIN_(5,5)are directly adjacent to the outmost pixels PXOM, the interior pixels PXIN_(3,4)-PXIN_(5,4)are indirectly adjacent to the outmost pixels PXOMthrough the interior pixels PXIN_(3,5)-PXIN_(5,5), the interior pixels PXIN_(3,3), PXIN_(5,3)are indirectly adjacent to the outmost pixels PXOMthrough the interior pixels PXIN_(3,4)-PXIN_(5,4), PXIN_(3,5)-PXIN_(5,5). Therefore, the interior pixels PXIN_(3,3), PXIN_(5,5), PXIN_(3,4)-PXIN_(5,4)are connected with the plurality of outmost pixels PXOM. According to Step A04, the determining unit18would change the label values of the interior pixels PXIN_(3,3), PXIN_(6,3), PXIN_(3, 4)-PXIN_(5,4)as the label value 1, which is the same as the label value of the outmost pixels PXOM, as shown in the subfigure14e. In addition, the interior pixels PXIN_(3,3), PXIN_(5,3), PXIN_(3,4)-XIN_(5,4)having the label value 1 in the binarized image BN14are the plurality of second background pixels PXBG2stated in Step806. As can be seen from the current embodiment (the subfigure14e), the foreground pixels (with the null label) do not surround any of the background pixels (with the label value 1). Therefore, by Steps304-306of the process30ofFIG. 3, it is determined that there is no pinch gesture in the image.

Please refer toFIG. 15.FIG. 15illustrates results of performing the processes90and A0on another binarized image BN15, where the subfigure15aillustrates a schematic diagram of the binarized image BN15before the determining unit18performs the process90, the subfigure15billustrates a schematic diagram of a result of the determining unit18performing the process90on the binarized image BN15, and the subfigure15cillustrates a schematic diagram of a result of the determining unit18performing the process A0on every internal background pixel PXBG_IN. Similarly, the numbers in the white squares inFIG. 15represent the pixel values, and the numbers in the slash squares represent the label values. InFIG. 15, the horizontal raster scan from left to right is also adopted.

As can be seen from the subfigure15b, according to the process90, the interior pixels PXIN_(2,5)-PXIN_(5,5)of the binarized image BN15are with the label value 1, the interior pixel PXIN_(3,3)of the binarized image BN15is with the label value 2, and the interior pixels PXIN_(5,3), PXIN_(5,4)of the binarized image BN15is with the label value 3.

Since the interior pixels PXIN_(5,3), PXIN_(5,4)are directly adjacent to the outmost pixels PXOM, according to Step A04, the determining unit18changes the label values of the interior pixels PXIN_(5,3), PXIN_(5,4)as the label value 1, the sane as which of the outmost pixels PXOM, as shown in the subfigure15c. The interior pixels PXIN_(5,3), PXIN_(5,4), PXIN_(2,5)-PXIN_(5,5)being with the label value 1 represents that they are connected with the outmost pixels PXOM. Also, the interior pixels PXIN_(5,3), PXIN_(5,4), PXIN_(2,5)-PXIN_(5,5)in the binarized image BN15having the label value 1 are the plurality of second background pixels PXBG2stated in Step806. On the other hand, the interior pixel PXIN_(3,3)is not connected with the interior pixel having the label value 1. Thus, after the process A0is performed, the label of the interior pixel PXIN_(3,3)is still the label value 2, as shown in the subfigure15c. As can be seen from the current embodiment, the foreground pixels (with the null label) do surround one background pixel (i.e., the interior pixel PXIN_(3,3)with the label value 2). According to Steps304-306of the process30inFIG. 3, it is determined that there is a pinch gesture in the image.

The processes80,90and A0are details of the determining unit18obtaining the plurality of second background pixels PXBG2using the tag/class method. Specifically, after performing the process A0on every internal background pixel PXBG_IN, the plurality of internal background pixels having the first label value TG1 in the binarized image BN are the plurality of second background pixels PXBG2stated in Steps604and806. Take the subfigures14eand15cas examples, the plurality of internal background pixels having the label value 1 the binarized image BN14and BN15are the plurality of second background pixels PXBG2stated in Steps604and806.

In addition, in Step606, the determining unit18determines whether the third background pixel PXBG3which is neither the outmost pixel PXOMnor the second background pixel PXBG2, exists among the plurality of background pixels PXBG. The determining unit18may determine whether the third background pixel PXBG3exists in the binarized image BN according to a result R of performing the process A0on every internal background pixel PXBG_IN. For example, the subfigure14eor the subfigure15care one kind of the result R. In short, the determining unit18may determine whether a first internal background pixel PXBG_IN_*which does not have the first label value TG1 exists among the plurality of interior pixels PXINaccording to the result R. If the first internal background pixel PXBG_IN_*not having the first label value TG1 exists among the plurality of interior pixels PXIN, the determining unit18determines that the first internal background pixel PXBG_IN_*is the third background pixel PXBG3stated in Step606.

Take the subfigure14eas an example, there is no background pixel not having the label value 1 (corresponding to the first label value TG1) in the binarized image BN14, which means that all of the background pixel in the binarized image BN14have the label value 1, a consequence of the determining unit18executing Step606according to the subfigure14eis “negative”, and the determining unit18may determine that there is no pinch gesture in the binarized image BN14.

Take the subfigure15cas an example, since the label of the interior pixel PXIN_(3,3)is still the label value 2 after the process A0is performed, which means that the interior pixel PXIN_(3,3)is the first internal background pixel PXBG_IN_*, and is also the third background pixel PXBG3stated in Step606, the consequence of the determining unit18executing Step606according to the subfigure15cand executing Step304according to the subfigure15cis “positive”. Therefore, according to the subfigure15c, the determining unit18may determine that the plurality of foreground pixels PXFGsurrounds the first background pixel PXBG1in the binarized image BN15in Step608, and determine that the gesture is the pinch gesture in Step306. Moreover, the interior pixel PXIN_(3,3)is the third background pixel PXBG3stated in Step606, and the interior pixel PXIN_(3,3)is also the first background pixel PXBG1surrounded by the plurality of foreground pixels PXFGstated in Steps608and304.

Notably, the embodiments stated in the above are utilized for illustrating the concept of the present invention. Those skilled in the art may make modifications and alterations accordingly, and not limited herein. For example, the plurality of outmost pixels PXOMmay not be the pixels at the four sides of the binarized image BN. The plurality of outmost pixels PXOMmay be the pixels on at least one side of the leftmost side, the upmost side, the rightmost side and the downmost side, which is also within the scope of the present invention.

In addition, the embodiments of the present invention stated in the above focus on determining whether the gesture is the pinch gesture, which is not limited in the present invention. The present invention may be applied to all gesture recognition method which determines whether the foreground pixels surround a part of the background pixels. The manufacturer or the user may specify a predefined gesture when it is determined that the foreground pixels surround a part of the background pixels. The predefined gesture includes but not limited to be the pinch gesture of the embodiments of the present invention, a circle gesture, a correct gesture, an okay gesture, a confirmation gesture, a lock-on gesture, an aiming gesture, a focusing gesture or other gesture which is synonymous or similar to the functional language stated in the above.

In summary, the present invention utilizes determining whether the background pixel is surrounded by the foreground pixels to determine whether the gesture is the pinch gesture; utilizes obtaining the plurality of second background pixels connected with the outmost pixels and determine whether there is the third background pixel which is not the second background pixel to determine whether the background pixel is surrounded by the foreground pixels; utilizes the tag/class method to obtain the plurality of second background pixels connected with the outmost pixels. Therefore, the present invention can determine the pinch gesture fast and efficiently.