Abstract:
A sign language recognition method includes a depth-sensing camera capturing an image of a gesture of a signer and gathering data about distances between a number of points on the signer and the depth-sensing camera, building a three dimension (3D) model of the gesture, comparing the 3D model of the gesture with a number of 3D models of different gestures to find out the representations of the 3D model of the gesture, and displaying or vocalizing the representations of the 3D model of the gesture.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    Relevant subject matter is disclosed in a pending U.S. patent application (application Ser. No. 12/852,512, filed on Aug. 8, 2010) having the same title and assigned to the same assignee as named herein. 
     
    
     BACKGROUND 
       [0002]    1. Technical Field 
         [0003]    The present disclosure relates to a sign language recognition system and a sign language recognition method. 
         [0004]    2. Description of Related Art 
         [0005]    Hearing impaired people communicate with other people with sign languages. However, people who do not know sign language find it difficult to communicate with the hearing impaired people. In addition, different countries have different sign languages, which makes communication problematic. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0006]    Many aspects of the embodiments can be better understood with reference to the following drawings. The components in the drawings are not necessarily drawn to scale, the emphasis instead being placed upon clearly illustrating the principles of the present embodiments. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views. 
           [0007]      FIG. 1  is a block diagram of an exemplary embodiment of a sign language recognition system. 
           [0008]      FIG. 2  is a schematic view of the sign language recognition system of  FIG. 1 . 
           [0009]      FIG. 3  is another schematic view of the sign language recognition system of  FIG. 1 . 
           [0010]      FIG. 4  is a flowchart of an exemplary embodiment of a sign language recognition method. 
       
    
    
     DETAILED DESCRIPTION 
       [0011]    The disclosure, including the accompanying drawings, is illustrated by way of example and not by way of limitation. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean at least one. 
         [0012]    Referring to  FIG. 1 , an exemplary embodiment of a sign language recognition system  1  includes a depth-sensing camera  10 , a storage unit  12 , a processing unit  15 , a first output unit  16 , and a second output unit  18 . In the embodiment, the first output unit  16  is a screen, and the second output unit  18  is a speaker or an earphone. Hereinafter the term signer is used for the person who uses sign language to communicate. 
         [0013]    In the embodiment, the depth-sensing camera  10  is a time of flight (TOF) camera. The TOF camera captures an image of the gesture of a signer in the line of sight of the TOF camera, and gathers data as to the distance(s) between a plurality of points on the signer and the TOF camera (which is called distance data hereinafter). The processing unit  15  and the storage unit  12  process the images and the distance data obtained by the depth-sensing camera  10 , for obtaining signs from the gestures of the signer. 
         [0014]    The TOF camera is a camera system that creates distance data between a plurality of points and the TOF camera. When the TOF camera shoots a scene in the line of sight of the TOF camera, the TOF camera sends radio frequency (RF) signals. The RF signals are reflected back to the TOF camera when the RF signals meet an object, such as the signer in the scene. As a result, the distance data can be obtained according to time differences between sending and receiving the RF signals of the TOF camera. 
         [0015]    The storage unit  12  includes a three dimensional (3D) model building module  120 , a sign language system setting module  122 , a sign language identification module  123 , a recognition module  125 , a voice conversion module  126 , and a gesture storing module  128 . The 3D model building module  120 , the sign language system setting module  122 , the sign language identification module  123 , the recognition module  125 , and the voice conversion module  126  may include one or more computerized instructions executed by the processing unit  15 . 
         [0016]    The gesture storing module  128  stores 3D models of different types of gestures and representations for each 3D model. Each type of gestures includes a plurality of gestures. In the embodiment, the gesture storing module  128  stores 3D models of two types of gestures. A first type of gestures corresponds to Chinese Sign Language. A second type of gestures corresponds to American Sign Language. In other embodiments, the gesture storing module  128  may store 3D models of more than two types of gestures or just one type of gestures. 
         [0017]    The 3D model building module  120  builds a 3D model of the gesture of the signer according to the image captured by the depth-sensing camera  10  and the data about distances between a plurality of points in the scene and the depth-sensing camera  10 . In the embodiment, according to the data regarding distances between a plurality of points in the scene and the depth-sensing camera  10 , the points in the scene have coordinates relative to the depth-sensing camera  10 . The 3D model building module  120  can obtain a 3D mathematical model according to the coordinates of the points and the image. The 3D mathematical model can be regarded as the 3D model of the gesture of the signer. 
         [0018]    The sign language system setting module  122  sets a work mode of the sign language recognition system  1 . Work mode hereinafter is referring to which language of sign that the signer is using. It can be understood that in the embodiment, the work modes of the sign language recognition system  1  includes a first mode corresponding to the first type of gesture, and a second mode corresponding to the second type of gesture. In the embodiment, receivers can use two buttons to manually set the work mode of the sign language recognition system  1 . 
         [0019]    The sign language system identification module  123  automatically sets the work mode of the sign language recognition system  1  when the receivers do not manually set the work mode of the sign language recognition system  1 . Automatically set the work mode of the sign language recognition system  1  by the sign language identification module  123  will be described as follows. 
         [0020]    The sign language identification module  123  compares the 3D model of the gesture obtained by the 3D model building module  120  with the plurality of 3D models of different types of gestures, to determine which type the 3D model of the gesture obtained belongs to. If the 3D model of the gesture obtained by the 3D model building module  120  belongs to the first type of gesture, the sign language identification module  123  sets the work mode of the sign language recognition system  1  as the first work mode. Moreover, if a 3D model of the gesture obtained by the 3D model building module  120  belongs to both the first and second types of gesture, the sign language identification module  123  may compare the 3D model of the next gesture obtained by the 3D model building module  120  with the plurality of 3D models of different types of gestures, until it is determined which one type the gesture belongs to. 
         [0021]    The recognition module  125  compares the 3D model of the gesture obtained by the 3D model building module  120  with the plurality of 3D models of gestures, corresponding to the work mode of the sign language recognition system  1 , to find out what the representations of the gesture are. The first output unit  16  displays the representations obtained by the recognition module  125 . 
         [0022]    The voice conversion module  126  converts the 3D model of the gesture obtained by the 3D model building module  120  into audible sounds. The second output unit  18  plays the representations of the 3D model of the gesture obtained by the 3D model building module  120 . 
         [0023]    As shown in  FIG. 2 , the sign language recognition system  1  may be embedded within a mobile telephone  100 . The depth-sensing camera  10  mounts on a surface of the body of the mobile telephone  100 . The first output unit  16  is a screen  46  of the mobile telephone  100 . The second output unit  18  is an earphone  48  of the mobile telephone  100 . Furthermore, the sign language recognition system  1  may take the form of glasses  200  worn by the receiver as shown in  FIG. 3 . The depth-sensing camera  10  can be mounted on the nose bridge of the glasses  200 . The first output unit  16  is two glasses  46  of the glasses  200 . The second output unit  18  is an earphone  48  which is connected to the bridge of the glasses  200 . 
         [0024]    Referring to  FIG. 4 , an exemplary embodiment of a sign language recognition method is as follows. Depending on the embodiment, certain of the steps described may be removed, others may be added, and the sequence of steps may be altered. It is also to be understood that the description and the claims drawn to a method may include some indication in reference to certain steps. However, the indication used is only to be viewed for identification purposes and not as a suggestion as to an order for the steps. 
         [0025]    In step S 1 , the receiver determines whether the receiver needs to manually set the work mode of the sign language recognition system  1 . If the receiver needs to manually set the work mode of the sign language recognition system  1 , the process flows to step S 2 . If the receiver does not need to manually set the work mode of the sign language recognition system  1 , the process flows to step S 3 . 
         [0026]    In step S 2 , the receiver manually sets the work mode of the sign language recognition system  1 , then the process flows to step S 3 . 
         [0027]    In step S 3 , the depth-sensing camera  10  captures an image of a gesture of the signer, and gathers data as to the distance(s) between a plurality of points on the signer and the depth-sensing camera  10 . 
         [0028]    In step S 4 , the 3D model building module  120  builds a 3D model of the gesture of the signer according to the image captured by the depth-sensing camera  10  and the data about distances between a plurality of points in the scene and the depth-sensing camera  10 . 
         [0029]    In step S 5 , the recognition module  125  determines whether the work mode is set. If the work mode is not set, the process flows to step S 6 . If the work mode is set, the process flows to step S 7 . 
         [0030]    In step S 6 , the sign language identification module  123  compares the 3D model of the gesture obtained by the 3D model building module  120  with the plurality of 3D models of different types of gestures, to determine which type the gesture of the signer belongs to, and sets the work mode accordingly, then the process flows to step S 7 . For example, if the 3D module of the gesture obtained by the 3D model building module  120  belongs to the first type of gestures, the sign language identification module  123  sets the work mode of the sign language recognition system  1  as the first work mode. Moreover, if a 3D model of the gesture obtained by the 3D model building module  120  belongs to both the first and second types of gestures, the sign language identification module  123  may compare the 3D model of the next gesture obtained by the 3D model building module  120  with the plurality of 3D models of different types of gestures, until a determination is made which type the gesture belongs to. 
         [0031]    In step S 7 , the recognition module  125  compares the 3D model obtained by the 3D model building module  120  with the plurality of 3D models, corresponding to the work mode of the sign language recognition system  1 , to recognize what meanings are associated with the 3D model of the gesture obtained by the 3D model building module  120 . 
         [0032]    In step S 8 , the screen  16  displays the meanings obtained by the recognition module  125 , the voice conversion module  126  converts the meanings of the 3D model of the gesture obtained by the 3D model building module  120  into audible sounds, and the speaker  18  plays the sounds of the 3D model of the gesture obtained by the 3D model building module  120 . 
         [0033]    The foregoing description of the embodiments of the disclosure has been presented only for the purposes of illustration and description and is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Many modifications and variations are possible in light of the above everything. The embodiments were chosen and described in order to explain the principles of the disclosure and their practical application so as to enable others of ordinary skill in the art to utilize the disclosure and various embodiments and with various modifications as are suited to the particular use contemplated. Alternative embodiments will become apparent to those of ordinary skills in the art to which the present disclosure pertains without departing from its spirit and scope. Accordingly, the scope of the present disclosure is defined by the appended claims rather than the foregoing description and the exemplary embodiments described therein.