Abstract:
An apparatus and method provide logic for providing gestural control. In one implementation, an apparatus includes a receiving unit configured to receive a first spatial position associated with a first portion of a human body, and a second spatial position associated with a second portion of the human body. An identification unit is configured to identify a group of objects based on at least the first spatial position, and a selection unit is configured to select an object of the identified group based on the second spatial position.

Description:
TECHNICAL FIELD 
       [0001]    The disclosed exemplary embodiments relate to an information processing apparatus and method and a program. In particular, the disclosed exemplary embodiments relate to an information processing apparatus and method and a program that can achieve a robust user interface employing a gesture. 
       BACKGROUND ART 
       [0002]    In recent years, in the area of information selection user interface (UI), research on a UI employing a noncontact gesture using part of a body, for example, a hand or finger, instead of information selection through an information input apparatus, such as a remote controller or keyboard, has become increasingly active. 
         [0003]    Examples of a proposed technique of selecting information employing a gesture include a pointing operation of detecting movement of a portion of a body, such as a hand or fingertip, and linking the amount of the movement with an on-screen cursor position and a technique of direct association between the shape of a hand or pose and information. At this time, many information selection operations are achieved by combination of information selection using a pointing operation and a determination operation using information on, for example, the shape of a hand or pose. 
         [0004]    More specially, one of the pointing operations most frequently used in information selection operation is the one recognizing the position of a hand. This is intuitive and significantly readily understandable because information is selected by movement of a hand. (See, for example, Horo, at al., “Realtime Pointing Gesture Recognition Using Volume Intersection,” The Japan Society of Mechanical Engineers, Robotics and Mechatronics Conference, 2006.) 
         [0005]    However, with the technique of recognizing the position of a hand, depending on the position of the hand of a human body being a target of estimation, determination whether it is a left or right hand may be difficult. For example, for inexpensive hand detection using a still image, the detection recognizing a hand by matching between detection of a skin color region and the shape of the hand, overlapping of right and left hands may be indistinguishable from each other. Thus, a technique of distinguishing by recognizing a depth using a range sensor, such as an infrared sensor, is proposed. (See, for example, Akahori, et al., “Interface of Home Appliances Terminal on User&#39;s Gesture,” ITX2001, 2001 Non Cited Literature 2.) In addition, a recognition technique having constraints, for example, that it is disabled when right and left hands are used at the same time, that it is disabled when right and left hands are crossed, and that movement is recognizable only when a hand exists in a predetermined region is also proposed (see Non Cited Literature 3). 
       CITATION LIST 
     Non Patent Literature 
       [0006]    NPL 1: Horo, Okada, Inamura, and Inaba, “Realtime Pointing Gesture Recognition Using Volume Intersection,” The Japan Society of Mechanical Engineers, Robotics and Mechatronics Conference, 2006 
         [0007]    NPL 2: Akahori and Imai, “Interface of Home Appliances Terminal on User&#39;s Gesture,” ITX2001, 2001 
         [0008]    NPL 3: Nakamura, Takahashi, and Tanaka, “Hands-Popie: A Japanese Input System Which Utilizes the Movement of Both Hands,” WISS, 2006 
       SUMMARY OF INVENTION 
     Technical Problem 
       [0009]    However, for the technique of Non Cited Literature 1, for example, if a user selects the input symbol  1  by a pointing operation from a large area of options, such as a keyboard displayed on a screen, because it is necessary to largely move a hand or finger while keeping the hand at a raised state, the user tends to be easily tired. Even when a small area of options is used, if the screen of an apparatus for displaying selection information is large, because the amount of movement of a hand or finger is also large, the user tends to be easily tired as well. 
         [0010]    In the case of Non Cited Literatures 2 and 3, it is difficult to distinguish between right and left hands when the hands overlap each other. Even when the depth is recognizable using a range sensor, such as an infrared sensor, if the hands at substantially the same distance from the sensor are crossed, there is a high probability that the hands are not distinguishable. 
         [0011]    Therefore, a technique illustrated in Cited Literature 3 is proposed. Even with this, because there are constraints, for example, that right and left hands are not allowed to be used at the same time, that right and left hands are not allowed to be crossed, and that movement is recognizable only when a hand exists in a predetermined region, a pointing operation is restricted. 
         [0012]    And, it is said that the human spatial perception feature leads to differences between an actual space and a perceived space at a remote site, and this is a problem in pointing on a large screen (see, for example, Shintani, at al., “Evaluation of a Pointing Interface for a Large Screen with Image Features,” Human Interface Symposium, 2009). 
         [0013]    The disclosed exemplary embodiments enable a very robust user interface even using an information selection operation employing a simple gesture. 
       Solution to Problem 
       [0014]    Consistent with an exemplary embodiment, an apparatus includes a receiving unit configured to receive a first spatial position associated with a first portion of a human body, and a second spatial position associated with a second portion of the human body. An identification unit is configured to identify a group of objects based on at least the first spatial position, and a selection unit is configured to select an object of the identified group based on the second spatial position. 
         [0015]    Consistent with an additional exemplary embodiment, a computer-implemented method provides gestural control of an interface. The method includes receiving a first spatial position associated with a first portion of the human body, and a second spatial position associated with a second portion of the human body. A group of objects is identified based on at least the first spatial position. The method includes selecting, using a processor, an object of the identified group based on at least the second spatial position. 
         [0016]    Consistent with a further exemplary embodiment, a non-transitory, computer-readable storage medium stores a program that, when executed by a processor, causes the processor to perform a method for gestural control of an interface. The method includes receiving a first spatial position associated with a first portion of the human body, and a second spatial position associated with a second portion of the human body. A group of objects is identified based on at least the first spatial position. The method includes selecting, using a processor, an object of the identified group based on at least the second spatial position. 
       Advantageous Effect of Invention 
       [0017]    According to the disclosed exemplary embodiments, a robust user interface employing a gesture can be achieved. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0018]      FIG. 1  is a block diagram that illustrates a configuration of an information input apparatus, according to an exemplary embodiment. 
           [0019]      FIG. 2  illustrates a configuration example of a human body pose estimation unit. 
           [0020]      FIG. 3  is a flowchart for describing an information input process. 
           [0021]      FIG. 4  is a flowchart for describing a human body pose estimation process. 
           [0022]      FIG. 5  is a flowchart for describing a pose recognition process. 
           [0023]      FIG. 6  is an illustration for describing the pose recognition process. 
           [0024]      FIG. 7  is an illustration for describing the pose recognition process. 
           [0025]      FIG. 8  is an illustration for describing the pose recognition process. 
           [0026]      FIG. 9  is a flowchart for describing a gesture recognition process. 
           [0027]      FIG. 10  is a flowchart for describing an information selection process. 
           [0028]      FIG. 11  is an illustration for describing the information selection process. 
           [0029]      FIG. 12  is an illustration for describing the information selection process. 
           [0030]      FIG. 13  is an illustration for describing the information selection process. 
           [0031]      FIG. 14  is an illustration for describing the information selection process. 
           [0032]      FIG. 15  is an illustration for describing the information selection process. 
           [0033]      FIG. 16  is an illustration for describing the information selection process. 
           [0034]      FIG. 17  illustrates a configuration example of a general-purpose personal computer. 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
     Configuration Example of Information Input Apparatus 
       [0035]      FIG. 1  illustrates a configuration example of an embodiment of hardware of an information input apparatus, according to an exemplary embodiment. An information input apparatus  11  in  FIG. 1  recognizes an input operation in response to an action (gesture) of the human body of a user and displays a corresponding processing result. 
         [0036]    The information input apparatus  11  includes a noncontact capture unit  31 , an information selection control unit  32 , an information option database  33 , an information device system control unit  34 , an information display control unit  35 , and a display unit  36 . 
         [0037]    The noncontact capture unit  31  obtains an image that contains a human body of a user, generates a pose command corresponding to a pose of the human body of the user in the obtained image or a gesture command corresponding to a gesture being chronological poses, and supplies it to the information selection control unit  32 . That is, the noncontact capture unit  31  recognizes a pose or a gesture in a noncontact state with respect to a human body of a user, generates a corresponding pose command or gesture command, and supplies it to the information selection control unit  32 . 
         [0038]    More specifically, the noncontact capture unit  31  includes an imaging unit  51 , a human body pose estimation unit  52 , a pose storage database  53 , a pose recognition unit  54 , a classified pose storage database  55 , a gesture recognition unit  56 , a pose history data buffer  57 , and a gesture storage database  58 . 
         [0039]    The imaging unit  51  includes an imaging element, such as a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS), is controlled by the information selection control unit  32 , obtains an image that contains a human body of a user, and supplies the obtained image to the human body pose estimation unit  52 . 
         [0040]    The human body pose estimation unit  52  recognizes a pose of a human body on a frame-by-frame basis on the basis of an image that contains the human body of a user supplied from the imaging unit  51 , and supplies pose information associated with the recognized pose to the pose recognition unit  54  and the gesture recognition unit  56 . More specifically, the human body pose estimation unit  52  extracts a plurality of features indicating a pose of a human body from information on an image obtained by the imaging unit  51 . Then, the human body pose estimation unit  52  estimates information on coordinates and an angle of a joint of the human body in a three-dimensional space for each pose using the sum of products of elements of a vector of the plurality of extracted features and a vector of coefficients registered in the pose storage database  53  obtained by learning based on a vector of a plurality of features for each pose, and determines pose information having these as a parameter. Note that the details of the human body pose estimation unit  52  are described below with reference to  FIG. 2 . 
         [0041]    The pose recognition unit  54  searches pose commands associated with previously classified poses registered in the classified pose storage database  55  together with pose information, on the basis of pose information having information on the coordinates and an angle of a joint of a human body as a parameter. Then, the pose recognition unit  54  recognizes a pose registered in association with the pose information searched for as the pose of the human body of the user and supplies a pose command associated with that pose registered together with the pose information to the information selection control unit  32 . 
         [0042]    The gesture recognition unit  56  sequentially accumulates pose information supplied from the human body pose estimation unit  52  on a frame-by-frame basis for a predetermined period of time in the pose history data buffer  57 . Then, the gesture recognition unit  56  searches chronological pose information associated with previously classified gestures registered in the gesture storage database  58  for a corresponding gesture. The gesture recognition unit  56  recognizes a gesture associated with the chronological pose information searched for as the gesture made by the human body whose image has been obtained. The gesture recognition unit  56  reads a gesture command registered in association with the recognized gesture from the gesture storage database  58 , and supplies it to the information selection control unit  32 . 
         [0043]    In the information option database  33 , information being an option associated with a pose command or gesture command supplied from the noncontact capture unit  31  is registered. The information selection control unit  32  selects information being an option from the information option database  33  on the basis of a pose command or gesture command supplied from the noncontact capture unit  31 , and supplies it to the information display control unit  35 . 
         [0044]    The information device system control unit  34  causes an information device functioning as a system (not illustrated) or a stand-alone information device to perform various kinds of processing on the basis of information being an option supplied from the information selection control unit  32 . 
         [0045]    The information display control unit  35  causes the display unit  36  including, for example, a liquid crystal display (LCD) to display information corresponding to information selected as an option. 
         [0046]    Configuration Example of Human Body Pose Estimation Unit 
         [0047]    Next, a detailed configuration example of the human body pose estimation unit  52  is described with reference to  FIG. 2 . 
         [0048]    The human body pose estimation unit  52  includes a face detection unit  71 , a silhouette extraction unit  72 , a normalization process region extraction unit  73 , a feature extraction unit  74 , a pose estimation unit  75 , and a correction unit  76 . The face detection unit  71  detects a face image from an image supplied from the imaging unit  51 , identifies a size and position of the detected face image, and supplies them to the silhouette extraction unit  72 , together with the image supplied from the imaging unit  51 . The silhouette extraction unit  72  extracts a silhouette forming a human body from the obtained image on the basis of the obtained image and information indicating the size and position of the face image supplied from the face detection unit  71 , and supplies it to the normalization process region extraction unit  73  together with the information about the face image and the obtained image. 
         [0049]    The normalization process region extraction unit  73  extracts a region for use in estimation of pose information for a human body as a normalization process region from an obtained image using the obtained image, information indicating the position and size of a face image, and silhouette information and supplies it to the feature extraction unit  74  together with image information. The feature extraction unit  74  extracts a plurality of features, for example, edges, an edge strength, and an edge direction, from the obtained image, in addition to the position and size of the face image and the silhouette information, and supplies a vector having the plurality of features as elements to the pose estimation unit  75 . 
         [0050]    The pose estimation unit  75  reads a vector of a plurality of coefficients from the pose storage database  53  on the basis of information on a vector having a plurality of features as elements supplied from the feature extraction unit  74 . Note that in the following description, a vector having a plurality of features as elements is referred to as a feature vector. Further, a vector of a plurality of coefficients registered in the pose storage database  53  in association with a feature vector is referred to as a coefficient vector. That is, in the pose storage database  53 , a coefficient vector (a set of coefficients) previously determined in association with a feature vector for each pose by learning is stored. The pose estimation unit  75  determines pose information using the sum of products of a read coefficient vector and a feature vector, and supplies it to the correction unit  76 . That is, pose information determined here is information indicating the coordinate positions of a plurality of joints set as a human body and an angle of the joints. 
         [0051]    The correction unit  76  corrects pose information determined by the pose estimation unit  75  on the basis of constraint determined using the size of an image of a face of a human body, such as the length of an arm or leg, and supplies the corrected pose information to the pose recognition unit  54  and the gesture recognition unit  56 . 
       About Information Input Process 
       [0052]    Next, an information input process is described with reference to the flowchart of  FIG. 3 . 
         [0053]    In step S 11 , the imaging unit  51  of the noncontact capture unit  31  obtains an image of a region that contains a person being a user, and supplies the obtained image to the human body pose estimation unit  52 . 
         [0054]    In step S 12 , the human body pose estimation unit  52  performs a human body pose estimation process, estimates a human body pose, and supplies it as pose information to the pose recognition unit  54  and the gesture recognition unit  56 . 
       Human Body Pose Estimation Process 
       [0055]    Here, a human body pose estimation process is described with reference to the flowchart of  FIG. 4 . 
         [0056]    In step S 31 , the face detection unit  71  determines information on the position and size of an obtained image of a face of a person being a user on the basis of an obtained image supplied from the imaging unit  51 , and supplies the determined information on the face image and the obtained image to the silhouette extraction unit  72 . More specifically, the face detection unit  71  determines whether a person being a user is present in an image. When the person is present in the image, the face detection unit  71  detects the position and size of the face image. At this time, when a plurality of face images is present, the face detection unit  71  determines information for identifying the plurality of face images and the position and size of each of the face images. The face detection unit  71  determines the position and size of a face image by, for example, a method employing a black and white rectangular pattern called Haar pattern. For example, a method of detecting a face image using a Haar pattern leverages the fact that the eye and mouse are darker than other regions, and represents a technique of representing lightness of a face in combination of specific patterns called Haar patterns and detecting a face image depending on the arrangement, coordinates, sizes, and number of these patterns. 
         [0057]    In step S 32 , the silhouette extraction unit  72  extracts only a foreground region as a silhouette by measuring a difference from a previously registered background region, and separating the foreground region from the background region in a similar way to detection of a face image, e.g., a so-called background subtraction technique. Then, the silhouette extraction unit  72  supplies the extracted silhouette, information on the face image, and obtained image to the normalization process region extraction unit  73 . Note that the silhouette extraction unit  72  may also extract a silhouette by a method other than the background subtraction technique. For example, it may also employ other general algorithms, such as a motion difference technique using a region having predetermined motion or more as a foreground region. 
         [0058]    In step S 33 , the normalization process region extraction unit  73  sets a normalization process region (that is, a process region for pose estimation) using information on the position and size of a face image being a result of face image detection. The normalization process region extraction unit  73  generates a normalization process region composed of only a foreground region part forming a human body from which information on a background region is removed in accordance with the silhouette of a target human body extracted by the silhouette extraction unit  72 , and outputs it to the feature extraction unit  74 . With this normalization process region, the pose of a human body can be estimated without consideration of the positional relationship between the human body and the imaging unit  51 . 
         [0059]    In step S 34 , the feature extraction unit  74  extracts features, such as edges within a normalization process region, edge strength, and edge direction, and forms a feature vector made up of a plurality of features, in addition to the position and size of a face image and silhouette information, to the pose estimation unit  75 . 
         [0060]    In step S 35 , the pose estimation unit  75  reads a coefficient vector (that is, a set of coefficients) previously determined by learning and associated with a supplied feature vector and pose from the pose storage database  53 . Then, the pose estimation unit  75  determines pose information including the position and angle of each joint in three-dimensional coordinates by the sum of products of elements of the feature vector and the coefficient vector, and supplies it to the correction unit  76 . 
         [0061]    In step S 36 , the correction unit  76  corrects pose information including the position and angle of each joint on the basis of constraint, such as the position and size of a face image of a human body and the length of an arm or leg of the human body. In step S 37 , the correction unit  76  supplies the corrected pose information to the pose recognition unit  54  and the gesture recognition unit  56 . 
         [0062]    Here, a coefficient vector stored in the pose storage database  53  by learning based on a feature vector is described. 
         [0063]    As described above, the pose storage database  53  prepares a plurality of groups of feature vectors obtained from image information for necessary poses and coordinates of a joint in a three-dimensional space that corresponds to the poses, and stores a coefficient vector obtained by learning using these correlations. That is, the pose storage database  53  determines a correlation between a feature vector of the whole of the upper half of the body obtained from an image subjected to a normalization process and coordinates of the position of a joint of the human body in a three-dimensional space, and estimates a pose of the human body enables various poses, for example, crossing of right and left hands to be recognized. 
         [0064]    Various algorithms can be used to determine the coefficient vector. Here, multiple regression analysis is described as an example. A relation between (i) a feature vector x epsilon R_m (epsilon: contained as an element) obtained by conversion of image information, and (ii) a pose information vector X epsilon R_d of elements forming pose information including coordinates of the position of a joint of a human body in a three-dimensional space and the angle of the joint may be expressed in a multiple regression equation using the following expression. 
         [0000]      Expression 1 
         [0000]      γ≡χβ+ε  (1)
 
         [0065]    Here, m denotes a dimension of a used feature, and d denotes a dimension of a coordinate vector of the position of a joint of a human body in a three-dimensional space. Epsilon is called residual vector and represents a difference between the coordinates of the position of a joint of a human body in a three-dimensional space used in learning and predicted three-dimensional positional coordinates determined by multiple regression analysis. Here, to represent an upper half of a body, positional coordinates (x, y, z) in a three-dimensional space of eight joints in total of a waist, a head, and both shoulders, elbows, and wrists are estimated. A calling side can obtain a predicted value of coordinates of the position of a joint of a human body in a three-dimensional space by multiplying together an obtained feature vector and a partial regression coefficient vector beta_(m*d) obtained by learning. The pose storage database  53  stores elements of a partial regression coefficient vector beta_(m*d) (coefficient set) as a coefficient vector described above. 
         [0066]    As a technique of determining a coefficient vector beta using a learning data set described above, multiple regression analysis called ridge regression can be used, for example. Typical multiple regression analysis uses the least squares method to determine a partial regression coefficient vector beta_(m*d), so as to obtain the minimum square of the difference between a predicted value and a true value (for example, coordinates of the position of a joint of a human body in a three-dimensional space and an angle of a joint in learning data) in accordance with an evaluation function expressed using the following expression. 
         [0000]      Expression 2 
         [0000]      min[|γ−χβ∥ 2 ]  (2)
 
         [0067]    For ridge regression, a term containing an optional parameter lambda is added to an evaluation function in the least squares method, and a partial regression coefficient vector beta_(m*d) at which the following expression has the minimum value is determined. 
         [0000]      Expression 3 
         [0000]      min[|γ−χβ∥ 2 −λ∥β∥ 2 ]  (3)
 
         [0068]    Here, lambda is a parameter for controlling “goodness” of fit of a model obtained by a multiple regression equation and learning data. It is known that, in not only multiple regression analysis but also using other learning algorithms, an issue called overfitting should be sufficiently considered. Overfitting is low generalization performance learning that supports learning data, but cannot fit unknown data. A term that contains a parameter lambda appearing ridge regression is a parameter for controlling goodness of fit to learning data, and is effective for controlling overfitting. When a parameter lambda is small, the goodness of fit to learning data is high, but that to unknown data is low. In contrast, when a parameter lambda is large, the goodness of fit to learning data is low, but that to unknown data is high. A parameter lambda is adjusted so as to achieve a pose storage database with higher generalization performance. 
         [0069]    Note that the coordinates of the position of a joint in a three-dimensional space can be determined as coordinates calculated when the position of the center of a waist is the origin point. Even when each coordinate position and angle can be determined using the sum of products of elements of a coefficient vector beta determined by multiple regression analysis and a feature vector, an error may occur in the relationship between lengths of parts of a human body, such as an arm and leg, in learning. Therefore, the correction unit  76  corrects pose information under constraint based on the relationship between lengths of parts (e.g., arm and leg). 
         [0070]    With the foregoing human body pose estimation process, information on the coordinates of the position of each joint of a human body of a user in a three-dimensional space and its angle is determined as pose information (that is, a pose information vector) and supplied to the pose recognition unit  54  and the gesture recognition unit  56 . 
         [0071]    Here, the description returns to the flowchart of  FIG. 3 . 
         [0072]    When pose information for a human body is determined in the processing of step S 12 , the pose recognition unit  54  performs a pose recognition process and recognizes a pose by comparing it with pose information for each pose previously registered in the classified pose storage database  55  on the basis of the pose information in step S 13 . Then, the pose recognition unit  54  reads a pose command associated with the recognized pose registered in the classified pose storage database  55 , and supplies it to the information selection control unit  32 . 
       Pose Recognition Process 
       [0073]    Here, the pose recognition process is described with reference to the flowchart of  FIG. 5 . 
         [0074]    In step S 51 , the pose recognition unit  54  obtains pose information including information on the coordinates of the position of each joint of a human body of a user in a three-dimensional space and information on its angle supplied from the human body pose estimation unit  52 . 
         [0075]    In step S 52 , the pose recognition unit  54  reads unprocessed pose information among pose information registered in the classified pose storage database  55 , and sets it at pose information being a process object. 
         [0076]    In step S 53 , the pose recognition unit  54  compares pose information being a process object and pose information supplied from the human body pose estimation unit  52  to determine its difference. More specifically, the pose recognition unit  54  determines the gap in the angle of a part linking two continuous joints on the basis of information on the coordinates of the position of the joints contained in the pose information being the process object and the obtained pose information, and determines it as the difference. For example, when a left forearm linking a left elbow and a left wrist joint is an example of a part, a difference theta is determined as illustrated in  FIG. 6 . That is, the difference theta illustrated in  FIG. 6  is the angle formed between a vector V 1  (a 1 , a 2 , a 3 ), whose origin point is a superior joint, that is, the left elbow joint and that is directed from the left elbow to the wrist based on previously registered pose information being the process object, and a vector V 2  (b 1 , b 2 , b 3 ) based on the pose information estimated by the human body pose estimation unit  52 . The difference theta can be determined by calculation of the following expression (4). 
         [0000]    
       
         
           
             
               
                 
                   Expression 
                    
                   
                       
                   
                    
                   4 
                 
               
               
                 
                     
                 
               
             
             
               
                 
                   θ 
                   = 
                   
                     
                       cos 
                       
                         - 
                         1 
                       
                     
                     ( 
                     
                       
                         
                           
                             a 
                             1 
                           
                            
                           
                             b 
                             1 
                           
                         
                         + 
                         
                           
                             a 
                             2 
                           
                            
                           
                             b 
                             2 
                           
                         
                         + 
                         
                           
                             a 
                             3 
                           
                            
                           
                             b 
                             3 
                           
                         
                       
                       
                         
                           
                             
                               a 
                               1 
                               2 
                             
                             + 
                             
                               a 
                               2 
                               2 
                             
                             + 
                             
                               a 
                               3 
                               2 
                             
                           
                         
                         · 
                         
                           
                             
                               b 
                               1 
                               2 
                             
                             + 
                             
                               b 
                               2 
                               2 
                             
                             + 
                             
                               b 
                               3 
                               2 
                             
                           
                         
                       
                     
                     ) 
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0077]    In this way, the pose recognition unit  54  determines the difference theta in angle for each of all joints obtained from pose information by calculation. 
         [0078]    In step S 54 , the pose recognition unit  54  determines whether all of the determined differences theta fall within tolerance thetath. When it is determined in step S 54  that all of the differences theta fall within tolerance thetath, the process proceeds to step S 55 . 
         [0079]    In step S 55 , the pose recognition unit  54  determines that it is highly likely that pose information supplied from the human body pose estimation unit  52  matches the pose classified as the pose information being the process object, and stores the pose information being the process object and information on the pose classified as that pose information as a candidate. 
         [0080]    On the other hand, when it is determined in step S 54  that not all of the differences theta is within tolerance thetath, it is determined that the supplied information does not match the pose corresponding to the pose information being the process object, the processing of step S 55  is skipped, and the process proceeds to step S 56 . 
         [0081]    In step S 56 , the pose recognition unit  54  determines whether there is unprocessed pose information in the classified pose storage database  55 . When it is determined that there is unprocessed pose information, the process returns to step S 52 . That is, until it is determined that there is no unprocessed pose information, the processing from step S 52  to step S 56  is repeated. Then, when it is determined in step S 56  that there is no unprocessed pose information, the process proceeds step S 57 . 
         [0082]    In step S 57 , the pose recognition unit  54  determines whether pose information for the pose corresponding to a candidate is stored. In step S 57 , for example, when it is stored, the process proceeds to step S 58 . 
         [0083]    In step S 58 , the pose recognition unit  54  reads a pose command registered in the classified pose storage database  55  together with pose information in association with the pose having the smallest sum of the differences theta among poses being candidates, and supplies it to the information selection control unit  32 . 
         [0084]    On the other hand, when it is determined in step S 57  that pose information corresponding to the pose being a candidate has not been stored, the pose recognition unit  54  supplies a pose command indicating an unclassified pose to the information selection control unit  32  in step S 59 . 
         [0085]    With the above processes, when pose information associated with a previously classified pose is supplied, and an associated pose command is supplied to the information selection control unit  32 . Because of this, as a previously classified pose, for example, as indicated in sequence from above in the left part in  FIG. 7 , poses in which the palm of the left arm LH of a human body of a user (that is, a reference point disposed along a first portion of the human body) points in the left direction (e.g., pose  201 ), points in the downward direction (e.g., pose  202 ), points in the right direction (e.g., pose  203 ), and points in the upward direction (e.g., pose  204 ) into the page with respect to the left elbow can be identified and recognized. And, as indicated in the right part in  FIG. 7 , poses in which the palm of the right arm RH (that is, a second reference point disposed along a second portion of the human body) points to regions  211  to  215  imaginarily arranged in front of the person in sequence from the right of the page can be identified and recognized. 
         [0086]    Additionally, recognizable poses may be ones other than the poses illustrated in  FIG. 7 . For example, as illustrated in  FIG. 8 , from above, a pose in which the left arm LH 1  is at the upper left position in the page and the right arm RH 1  is at the lower right position in the page, a pose in which the left arm LH 2  and the right arm RH 2  are at the upper right position in the page, a pose in which the left arm LH 3  and the right arm RH 3  are in the horizontal direction, and a pose in which the left arm LH 1  and the right arm RH 1  are crossed can also be identified and recognized. 
         [0087]    That is, for example, identification using only the position of the palm (that is, the first spatial position and/or the second spatial position) may cause an error to occur in recognition because a positional relationship from the body is unclear. However, because recognition is performed as a pose of a human body, both arms can be accurately recognized, and the occurrence of false recognition can be suppressed. And, because of recognition as a pose, for example, even if both arms are crossed, the respective palms can be identified, the occurrence of false recognition can be reduced, and more complex poses can also be registered as poses that can be identified. Additionally, as long as only movement of the right side of the body or that of the left side of the body is registered, poses of the right and left arms can be recognized in combination, and therefore, the amount of pose information registered can be reduced, while at the same time many complex poses can be identified and recognized. 
         [0088]    Here, the description returns to the flowchart of  FIG. 3 . 
         [0089]    When in step S 13  the pose recognition process is performed, the pose of a human body of a user is identified, and a pose command is output, the process proceeds to step S 14 . In step S 14 , the gesture recognition unit  56  performs a gesture recognition process, makes a comparison with gesture information registered in the gesture storage database  58  on the basis of pose information sequentially supplied from the human body pose estimation unit  52 , and recognizes the gesture. Then, the gesture recognition unit  56  supplies a gesture command associated with the recognized gesture registered in the classified pose storage database  55  to the information selection control unit  32 . 
       Gesture Recognition Process 
       [0090]    Here, the gesture recognition process is described with reference to the flowchart of  FIG. 9 . 
         [0091]    In step S 71 , the gesture recognition unit  56  stores pose information supplied from the human body pose estimation unit  52  as a history for only a predetermined period of time in the pose history data buffer  57 . At this time, the gesture recognition unit  56  overwrites pose information of the oldest frame with pose information of the newest frame, and chronologically stores the pose information for the predetermined period of time in association with the history of frames. 
         [0092]    In step S 72 , the gesture recognition unit  56  reads pose information for a predetermined period of time chronologically stored in the pose history data buffer  57  as a history as gesture information. 
         [0093]    In step S 73 , the gesture recognition unit  56  reads unprocessed gesture information (that is, the first spatial position and/or the second spatial position) as gesture information being a process object among gesture information registered in the gesture storage database  58  in association with previously registered gestures. Note that chronological pose information corresponding to previously registered gestures is registered as gesture information in the gesture storage database  58 . In the gesture storage database  58 , gesture commands are registered in association with respective gestures. 
         [0094]    In step S 74 , the gesture recognition unit  56  compares gesture information being a process object and gesture information read from the pose history data buffer  57  by pattern matching. More specifically, for example, the gesture recognition unit  56  compares gesture information being a process object and gesture information read from the pose history data buffer  57  using continuous dynamic programming (DP). For example, continuous DP is an algorithm that permits extension and contraction of a time axis of chronological data being an input, and that performs pattern matching between previously registered chronological data, and its feature is that previous learning is not necessary. 
         [0095]    In step S 75 , the gesture recognition unit  56  determines by pattern matching whether gesture information being a process object and gesture information read from the pose history data buffer  57  match with each other. In step S 75 , for example, when it is determined that the gesture information being the process object and the gesture information read from the pose history data buffer  57  match with each other, the process proceeds to step S 76 . 
         [0096]    In step S 76 , the gesture recognition unit  56  stores a gesture corresponding to the gesture information being the process object as a candidate. 
         [0097]    On the other hand, when it is determined that the gesture information being the process object and the gesture information read from the pose history data buffer  57  do not match with each other, the processing of step S 76  is skipped. 
         [0098]    In step S 77 , the gesture recognition unit  56  determines whether unprocessed information is registered in the gesture storage database  58 . In step S 77 , for example, when unprocessed gesture information is registered, the process returns to step S 73 . That is, until unprocessed gesture information becomes nonexistent, the processing from step S 73  to step S 77  is repeated. Then, when it is determined in step  77  that there is no unprocessed gesture information, the process proceeds to step S 78 . 
         [0099]    In step S 78 , the gesture recognition unit  56  determines whether a gesture as a candidate is stored. When it is determined in step S 78  that a gesture being a candidate is stored, the process proceeds to step S 79 . 
         [0100]    In step S 79 , the gesture recognition unit  56  recognizes the most matched gesture as being made by a human body of a user among gestures stored as candidates by pattern matching. Then, the gesture recognition unit  56  supplies a gesture command (that is, a first command and/or a second command) associated with the recognized gesture (that is, a corresponding first gesture or a second gesture) stored in the gesture storage database  58  to the information selection control unit  32 . 
         [0101]    On the other hand, in step S 78 , when no gesture being a candidate is stored, it is determined that no registered gesture is made. In step S 80 , the gesture recognition unit  56  supplies a gesture command indicating that unregistered gesture (that is, a generic command) is made to the information selection control unit  32 . 
         [0102]    That is, with the above process, for example, it is determined that gesture information including chronological pose information read from the pose history data buffer  57  is recognized as corresponding to a gesture in which the palm sequentially moves from state where the left arm LH points upward from the left elbow, as illustrated in the lowermost left row in  FIG. 7 , to a state as indicated by an arrow  201  in the lowermost left row in  FIG. 7 , where the palm points in the upper left direction in the page. In this case, a gesture in which the left arm moves counterclockwise in the second quadrant in a substantially circular form indicated by the dotted lines in  FIG. 7  is recognized, and its corresponding gesture command is output. 
         [0103]    Similarly, a gesture in which the palm sequentially moves from a state where the left arm LH points in the leftward direction in the page from the left elbow, as illustrated in the uppermost left row in  FIG. 7 , to a state where it points in the downward direction in the page, as indicated by an arrow  202  in the left second row in  FIG. 7 , is recognized. In this case, a gesture in which the left arm moves counterclockwise in the third quadrant in the page in a substantially circular form indicated by the dotted lines in  FIG. 7  is recognized, and its corresponding gesture command is output. 
         [0104]    And, a gesture in which the palm sequentially moves from a state where the left arm LH points in the downward direction in the page from the left elbow, as illustrated in the left second row in  FIG. 7 , to a state where it points in the rightward direction in the page, as indicated by an arrow  203  in the left third row in  FIG. 7 , is recognized. In this case, a gesture in which the left arm moves counterclockwise in the fourth quadrant in the page in a substantially circular form indicated by the dotted lines in  FIG. 7  is recognized, and its corresponding gesture command is output. 
         [0105]    Then, a gesture in which the palm sequentially moves from a state where the left arm LH points in the rightward direction in the page from the left elbow as illustrated in the left third row in  FIG. 7  to a state where it points in the upward direction in the page as indicated by an arrow  204  in the lowermost left row in  FIG. 7  is recognized. In this case, a gesture in which the left arm moves counterclockwise in the first quadrant in the page in a substantially circular form indicated by the dotted lines in  FIG. 7  is recognized, and its corresponding gesture command is output. 
         [0106]    Additionally, in the right part in  FIG. 7 , as illustrated in sequence from above, sequential movement of the right palm from the imaginarily set regions  211  to  215  is recognized. In this case, a gesture in which the right arm moves horizontally in the leftward direction in the page is recognized, and its corresponding gesture command is output. 
         [0107]    Similarly, in the right part in  FIG. 7 , as illustrated in sequence from below, sequential movement of the right palm from the imaginarily set regions  215  to  211  is recognized. In this case, a gesture in which the right arm moves horizontally in the rightward direction in the page is recognized, and its corresponding gesture command is output. 
         [0108]    In this way, because a gesture is recognized on the basis of pose information chronologically recognized, false recognition, such as a failure to determine whether the movement is made by the right arm or the left arm, would occur if a gesture is recognized simply on the basis of the path of movement of a palm can be suppressed. As a result, false recognition of a gesture can be suppressed, and gestures can be appropriately recognized. 
         [0109]    Note that although a gesture of rotating the palm in a substantially circular form in units of 90 degrees is described as an example of gesture to be recognized, rotation other than this described example may be used. For example, a substantially oval form, substantially rhombic form, substantially square form, or substantially rectangular form may be used, and clockwise movement may be used. The unit of rotation is not limited to 90 degrees, and other angles may also be used. 
         [0110]    Here, the description returns to the flowchart of  FIG. 3 . 
         [0111]    When a gesture is recognized by the gesture recognition process in step S 14  and a gesture command associated with the recognized gesture is supplied to the information selection control unit  32 , the process proceeds to step S 15 . 
         [0112]    In step S 15 , the information selection control unit  32  performs an information selection process, selects information being an option registered in the information option database  33  in association with a pose command or a gesture command. The information selection control unit  32  supplies the information it to the information device system control unit  34 , which causes various processes to be performed, supplies the information to the information display control unit  35 , and displays the selected information on the display unit  36 . 
         [0113]    Additionally, in step S 16 , the information selection control unit  32  determines whether completion of the process is indicated by a pose command or a gesture command. When it is determined that completion is not indicated, the process returns to step S 11 . That is, when completion of the process is not indicated, the processing of step S 11  to step S 16  is repeated. Then, when it is determined in step S 16  that completion of the process is indicated, the process ends. 
       Information Selection Process 
       [0114]    Here, the information selection process is described with reference to the flowchart of  FIG. 10 . Note that although a process of selecting one of kana characters (the Japanese syllabaries) as information is described here as an example, other information may be selected. At this time, an example in which a character is selected by selecting one of consonants (containing a voiced sound mark regarded as a consonant) moved by one character every time the palm is rotated by the left arm by 90 degrees, as illustrated in the left part in  FIG. 7 , and selecting a vowel by the right palm pointing to one of the regions  211  to  215  horizontally arranged is described. In this description, kana characters are expressed by romaji (a system of Romanized spelling used to transliterate Japanese). A consonant used in this description indicates the first character in a column in which a group of characters is arranged (that is, a group of objects), and a vowel used in this description indicates a character specified in the group of characters in the column of a selected consonant (that is, an object within the group of objects). 
         [0115]    In step S 101 , the information selection control unit  32  determines whether a pose command supplied from the pose recognition unit  54  or a gesture command supplied from the gesture recognition unit  56  is a pose command or a gesture command indicating a start. For example, if a gesture of rotating the palm by the left arm by 360 degrees is a gesture indicating a start, when such a gesture of rotating the palm by the left arm by 360 degrees is recognized, it is determined that a gesture indicating a start is recognized, and the process proceeds to step S 102 . 
         [0116]    In step S 102 , the information selection control unit  32  sets a currently selected consonant and vowel at “A” in the “A” column for initialization. On the other hand, when it is determined in step S 101  that the gesture is not a gesture indicating a start, the process proceeds to step S 103 . 
         [0117]    In step S 103 , the information selection control unit  32  determines whether a gesture recognized by a gesture command is a gesture of rotating the left arm counterclockwise by 90 degrees. When it is determined in step S 103  that a gesture recognized by a gesture command is a gesture of rotating the left arm counterclockwise by 90 degrees, the process proceeds to step S 104 . 
         [0118]    In step S 104 , the information selection control unit  32  reads information being an option registered in the information option database  33 , recognizes a consonant moved clockwise to its adjacent one from a current consonant, and supplies the result of recognition to the information device system control unit  34 , and the information display control unit  35 . 
         [0119]    That is, for example, as illustrated in the left part or right part in  FIG. 11 , as a consonant, “A,” “KA,” “SA,” “TA,” “NA,” “HA,” “MA,” “YA,” “RA,” “WA,” or “voiced sound mark” (resembling double quotes) is selected (that is, a group of objects is identified). In such a case, as indicated by a selection position  251  in a state P 1  in the uppermost row in  FIG. 12 , when the “A” column is selected as the current consonant, if a gesture of rotating the palm by 90 degrees counterclockwise from the left arm LH 11  to the left arm L 12  as indicated by an arrow  261  in a state P 2  in the second row in  FIG. 12  is made, the “KA” column adjacent clockwise is selected as indicated by a selection position  262  in P 2  in the second row in  FIG. 12 . 
         [0120]    In step S 105 , the information display control unit  35  displays information indicating a recognized consonant being adjacent clockwise moved from the current consonant on the display unit  36 . That is, for example, in the initial state, for example, as illustrated in a display field  252  in the uppermost state P 1  in  FIG. 12 , the information display control unit  35  displays “A” in the “A” column being the default initial position to display information indicating the currently selected consonant on the display unit  36 . Then, here, by rotation of the palm by the left arm LH 11  counterclockwise by 90 degrees, the information display control unit  35  largely displays “KA” as illustrated in a display field  263  in the second row in  FIG. 12  on the basis of information supplied from the information selection control unit  32  so as to indicate that the currently selected consonant is switched to “KA.” Note that at this time in the display field  263 , for example, “KA” is displayed as the center and only its adjacent “WA,” “voiced sound mark”, and “A” in the counterclockwise direction and its adjacent “SA,” “TA,” and “NA” in the clockwise direction are displayed. This enables possibility of selection of a consonant before or after the currently selected consonant to be easily recognizable. 
         [0121]    Similarly, from this state, when, as indicated in a state P 3  in the third row in  FIG. 12 , the left arm further moves from the left arm LH 12  to the left arm LH 13  by 90 degrees and the palm further moves counterclockwise, with the processing of steps S 103  and S 104 , as indicated by a selection position  272 , “SA,” which is clockwise adjacent to the “KA” column, is selected. Then, with the processing of step S 105 , the information display control unit  35  largely displays “SA” so as to indicate that the currently selected consonant is switched to the “SA” column, as illustrated in a display field  273  in the state P 3  in the third row in  FIG. 12 . 
         [0122]    On the other hand, it is determined in step S 103  that it is not a gesture of counter-clockwise 90-degree rotation, the process proceeds to step S 106 . 
         [0123]    In step S 106 , the information selection control unit  32  determines whether a gesture recognized by a gesture command is a gesture of rotating the left arm by 90 degrees clockwise. When it is determined in step S 106  that a gesture recognized by a gesture command is a gesture of rotating the left arm by 90 degrees clockwise, for example, the process proceeds to step S 107 . 
         [0124]    In step S 107 , the information selection control unit  32  reads information being an option registered in the information option database  33 , recognizes a consonant moved to the counterclockwise adjacent position with respect to the current vowel, and supplies the result of recognition to the information device system control unit  34  and the information display control unit  35 . 
         [0125]    In step S 108 , the information display control unit  35  displays information indicating the recognized consonant moved to the counterclockwise adjacent position for the current consonant on the display unit  36 . 
         [0126]    That is, this is opposite to the process of rotation of the palm clockwise in the above-described steps S 103  to S 105 . That is, for example, when the palm further moves clockwise by 180 degrees together with movement from the left arm LH 13  to the left arm LH 11  from the state P 3  in the third row in  FIG. 12 , as illustrated in an arrow  281  in the state P 4  in the fourth row, with the processing of steps S 107  and S 108 , as indicated by a selection position  282 , when the palm moves clockwise by 90 degrees, the adjacent “KA” is selected, and when the palm further moves clockwise by 90 degrees, “A” is selected. Then, with the processing of step S 108 , the information display control unit  35  largely displays “A” so as to indicate that the currently selected consonant is switched from “SA” to “A”, as illustrated in a display field  283  in the state P 4  in the fourth row in  FIG. 12 . 
         [0127]    On the other hand, when it is determined in step S 106  that it is not a gesture of clockwise 90-degree rotation, the process proceeds to step S 109 . 
         [0128]    In step S 109 , the information selection control unit  32  determines whether a pose command supplied from the pose recognition unit  54  or a gesture command supplied from the gesture recognition unit  56  is a pose command or a gesture command for selecting a vowel (that is, an object of an identified group of objects). For example, when the palm selects one of the regions  211  to  215  imaginarily arranged in front of the person by the right arm, as illustrated in  FIG. 7 , in the case of a pose that identifies a vowel by that region, a pose command indicating a pose in which the palm points to one of the regions  211  to  215  by the right arm is recognized, it is determined that a gesture indicating that the vowel is identified (that is, the object is identified), and the process proceeds to step S 110 . 
         [0129]    In step S 110 , the information selection control unit  32  reads information being an option registered in the information option database  33 , recognizes a vowel corresponding to the position of the right palm recognized as the pose, and supplies the result of recognition to the information device system control unit  34  and the information display control unit  35 . 
         [0130]    That is, for example, when the “TA” column is selected as a consonant, if a pose command indicating a pose in which the palm points to the region  211  imaginarily set in front of the person by the right arm RH 31  is recognized, as illustrated in the uppermost row in  FIG. 13 , “TA” is selected as a vowel, as indicated by a selection position  311 . Similarly, as illustrated in the second row in  FIG. 13 , if a pose command indicating a pose in which the palm points to the region  212  imaginarily set in front of the person by the right arm RH 32  is recognized, as illustrated in the second row in  FIG. 13 , “TI” is selected as a vowel. As illustrated in the third to fifth rows in  FIG. 13 , if pose commands indicating poses in which the palm points to the regions  213  to  215  imaginarily set in front of the person by the right arms RH 33  to RH 35  are recognized, “TU,” “TE,” and “TO” are selected as their respective vowels. 
         [0131]    In step S 111 , the information display control unit  35  displays a character corresponding to a vowel recognized to be selected on the display unit  36 . That is, for example, a character corresponding to the vowel selected so as to correspond to each of display positions  311  to  315  in the left part in  FIG. 13  is displayed. 
         [0132]    On the other hand, it is determined in step S 109  that it is not a gesture for identifying a consonant, the process proceeds to step S 112 . 
         [0133]    In step S 112 , the information selection control unit  32  determines whether a pose command supplied from the pose recognition unit  54  or a gesture command supplied from the gesture recognition unit  56  is a pose command or gesture command for selecting determination. For example, if it is a gesture in which the palm continuously moves through the regions  211  to  215  imaginarily arranged in front of the person and selects one or a gesture in which the palm continuously moves through the regions  215  to  211 , as illustrated in  FIG. 7 , it is determined that a gesture indicating determination is recognized, and the process proceeds to step S 113 . 
         [0134]    In step S 113 , the information selection control unit  32  recognizes a character having the currently selected consonant and a determined vowel and supplies the recognition to the information device system control unit  34  and the information display control unit  35 . 
         [0135]    In step S 114 , the information display control unit  35  displays the selected character such that it is determined on the basis of information supplied from the information selection control unit  32  on the display unit  36 . 
         [0136]    And, when it is determined in step S 112  that it is a gesture indicating determination, the process proceeds to step S 115 . 
         [0137]    In step S 115 , the information selection control unit  32  determines whether a pose command supplied from the pose recognition unit  54  or a gesture command supplied from the gesture recognition unit  56  is a pose command or gesture command for indicating completion. When it is determined in step S 115  that it is not a pose command or gesture command indicating completion, the information selection process is completed. On the other hand, in step S 115 , for example, when a pose command indicating a pose of moving both arms down is supplied, the information selection control unit  32  determines in step S 116  that the pose command indicating completion is recognized and recognizes the completion of the process. 
         [0138]    The series of the processes described above are summarized below. 
         [0139]    That is, when a gesture of moving the palm in a substantially circular form as indicated by an arrow  351  as illustrated in the left arm LH 51  of a human body of a user in a state P 11  in  FIG. 14  is made, it is determined that starting is indicated and the process starts. At this time, as illustrated in the state P 11  in  FIG. 14 , the “A” column is selected as a consonant as default, and the vowel “A” is also selected. 
         [0140]    Then, a gesture of rotating the left arm LH 51  in the state P 11  counterclockwise by 90 degrees in the direction of an arrow  361  as indicated by the left arm LH 52  in a state P 12  is made, and a pose of pointing to the region  215  as indicated by the right arm RH 52  by moving from the right arm RH 51  is made. In this case, the consonant is moved from the “A” column to the “KA” column together with the gesture, and additionally, “KO” in the “KA” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “KO” is selected. 
         [0141]    Next, a gesture of rotating the left arm LH 52  in the state P 12  by 270 degrees clockwise in the direction of an arrow  371  as indicated by the left arm LH 53  in a state P 13  is made, and a pose of pointing to the region  305  as indicated by the right arm RH 53  without largely moving from the right arm RH 52  is made. In this case, the consonant is moved to the “WA” column through the “A” and “voiced sound mark” columns for each 90-degree rotation together with the gesture, and additionally, “N” in the “WA” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “N” is selected. 
         [0142]    And, a gesture of rotating the left arm LH 53  in the state P 13  by 450 degrees counter-clockwise in the direction of an arrow  381  as indicated by the left arm LH 54  in a state P 14  is made, and a pose of pointing to the region  212  as indicated by the right arm RH 54  by moving from the right arm RH 53  is made. In this case, the consonant is moved to the “NA” column through the “voiced sound mark,” “A,” “KA,” “SA,” and “TA” columns for each 90-degree rotation together with the gesture, and additionally, “NI” in the “NA” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “NI” is selected. 
         [0143]    Additionally, a gesture of rotating the left arm LH 54  in the state P 14  by 90 degrees clockwise in the direction of an arrow  391  as indicated by the left arm LH 55  in a state P 15  is made, and a pose of pointing to the region  212  as indicated by the right arm RH 55  in the same way as for the right arm RH 54  is made. In this case, the consonant is moved to the “TA” column together with the gesture, and additionally, “TI” in the “TA” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “TI” is selected. 
         [0144]    And, a gesture of rotating the left arm LH 55  in the state P 15  by 180 degrees clockwise in the direction of an arrow  401  as indicated by the left arm LH 56  in a state P 16  is made, and a pose of pointing to the region  211  as indicated by the right arm RH 56  by moving from the right arm RH 55  is made. In this case, the consonant is moved to the “HA” column through the “NA” column together with the gesture, and additionally, “HA” in the “HA” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “HA” is selected. 
         [0145]    Finally, as illustrated in a state P 16 , as indicated by the left arm LH 57  and the right arm RH 57 , a series of gestures of moving both arms down and a pose that indicate completion cause “KONNITIHA” (meaning “hello” in English) to be determined and entered. 
         [0146]    In this way, gestures and poses using right and left arms enable an entry of a character. At this time, a pose is recognized employing pose information, and a gesture is recognized employing chronological information of pose information. Therefore, false recognition, such as a failure to distinguish between right and left arms, that would occur if an option is selected and entered on the basis of the movement or the position of a single part of a human body can be reduced. 
         [0147]    In the foregoing, a technique of entering a character on the basis of pose information obtained from eight joints of the upper half of a body and movement of the parts is described as an example. However, three kinds of states of a state where the fingers are clenched in the palm (rock), a state where only index and middle fingers are extended (scissors), and a state of an open hand (paper), may be added to a feature. This can increase the range of variations in the method of identifying a vowel using a pose command, such as enabling switching among selection of a regular character in the state of rock, selection of a voiced sound mark in the state of scissors, and selection of a semi-voiced sound mark in the state of paper, as illustrated in the right part in  FIG. 11 , even when substantially the same way as in the method of identifying a vowel is used. 
         [0148]    And, in addition to kana characters, as illustrated in the left part in  FIG. 15 , “a,” “e,” “i,” “m,” “q,” “u,” and “y” may also be selected by a gesture of rotation in a way similar to the above-described method of selecting a consonant. Then, “a, b, c, d” for “a,” “e, f, g, h” for “e,” “i, j k, l” for “i,” “m, n, o, p” for “m,” “q, r, s, t” for “q,” “u, v, w, x” for “u,” and “y, z” for “y” may be selected in a way similar to the above-described selection of a vowel. 
         [0149]    Additionally, if identification employing the state of a palm is enabled, as illustrated in the right part in  FIG. 15 , “a,” “h,” “l,” “q,” and “w” may also be selected by a gesture of rotation in a way similar to the above-described method of selecting a consonant. Then, “a, b, c, d, e, f, g” for “a,” “h, i, j, k” for “h,” “l, m, n, o, p” for “l,” “q, r, s, t, u, v” for “q,” and “w, x, y, z” for “w” may be selected in a way similar to the above-described selection of a vowel. 
         [0150]    And, in the case illustrated in the right part in  FIG. 15 , even if identification employing the state of a palm is not used, the regions  211  to  215  imaginarily set in front of a person may be increased. In this case, for example, as illustrated in a state P 42  in  FIG. 16 , a configuration that has nine (=3*3) regions of regions  501  to  509  may be used. 
         [0151]    That is, for example, as indicated by the left arm LH 71  of a human body of a user in a state P 41  in  FIG. 16 , when a gesture of moving the palm in a substantially circular form as indicated by an arrow  411  is made, it is determined that starting is indicated, and the process starts. At this time, as illustrated in the state P 41  in  FIG. 16 , the “a” column is selected as a consonant by default, and “a” is also selected as a vowel. 
         [0152]    Then, when a gesture of rotating the left arm LH 71  in the state P 41  counterclockwise by 90 degrees in the direction of an arrow  412  as indicated by the left arm LH 72  in the state P 42  is made and a pose of pointing to a region  503  as indicated by the right arm RH 72  by moving from the right arm RH 71  is made, the consonant is moved from the “a” column to the “h” column together with the gesture, and additionally, “h” in the “h” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “h” is selected. 
         [0153]    Next, when a gesture of rotating the left arm LH 72  in the state P 42  by 90 degrees clockwise in the direction of an arrow  413  as indicated by the left arm LH 73  in a state P 43  is made and a pose of pointing to the region  505  as indicated by the right arm RH 73  from the right arm RH 72  is made, the consonant is moved to the “a” column for each 90-degree rotation together with the gesture, and additionally, “e” in the “a” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “e” is selected. 
         [0154]    And, when a gesture of rotating the left arm LH 73  in the state P 43  by 180 degrees counterclockwise in the direction of an arrow  414  as indicated by the left arm LH 74  in a state P 44  is made and a pose of pointing to the region  503  as indicated by the right arm RH 74  by moving from the right arm RH 73  is made, the consonant is moved to the “l” column through the “h” column for each 90-degree rotation together with the gesture, and additionally, “l” in the “l” column is identified as a vowel by the pose. In this state, when a gesture indicating determination is made, “l” is selected. 
         [0155]    Additionally, as indicated by the left arm LH 75  and the right arm RH 75  in a state P 45 , when a gesture indicating determination is made while the state P 44  is maintained, “l” is selected again. 
         [0156]    And, as indicated by the left arm LH 76  in a state P 46 , when a pose of pointing to the region  506  as indicated by the right arm RH 76  moved from the right arm RH 75  is made while the left arm LH 75  in the state P 45  is maintained, “o” in the “l” column is identified as a vowel. In this state, when a gesture indicating determination is made, “o” is selected. 
         [0157]    Finally, as illustrated by the left arm LH 77  and the right arm RH 77  in a state P 47 , a series of gestures of moving both arms down and a pose that indicate completion make an entry of “Hello.” 
         [0158]    Note that in the foregoing an example in which a consonant is moved by a single character for each 90-degree rotation angle is described. However, a rotation angle may not be used. For example, the number of characters of movement of a consonant may be changed in response to a rotation speed; for high speeds, the number of characters of movement may be increased, and for low speeds, the number of characters of movement may be reduced. 
         [0159]    And, an example in which coordinates of the position and an angle of each joint of a human body in a three-dimensional space are used as pose information is described. However, information, such as opening and closing of a palm or opening and closing of an eye and a mouse, may be added so as to be distinguishable. 
         [0160]    Additionally, in the foregoing, an example in which a kana character or a character of an alphabet is entered as an option is described. However, an option is not limited to only a character, and a file or folder may be selected using a file list or a folder list. In this case, a file or folder may be identified and selected by a creation date or a file size, like a vowel or consonant described above. One such example of the file may be a photograph file. In this case, the file may be classified and selected by information, such as the year, month, date, week, or time of obtaining an image, like a vowel or consonant described above. 
         [0161]    From the above, in recognition of a pose or gesture of a human body, even if there is a partial hidden part caused by, for example, crossing right and left arms, the right and left arms can be distinguished and recognized, and information can be entered while the best possible use of a limited space can be made. Therefore, desired information can be selected from among a large number of information options without increasing the amount of movement of an arm, suppressing a decrease in willingness to enter information caused by effort of entry operation reduces fatigue of a user, and an information selection process with ease of operation can be achieved. 
         [0162]    And, simultaneous recognition of different gestures made by right and left hands enables high-speed information selection and also enables selection by continuous operation, such as operation like drawing with a single stroke. Additionally, a large amount of information can be selected and entered using merely a small number of simple gestures, such as rotation or a change in the shape of a hand for determination operation, for example, sliding operation. Therefore, a user interface that enables a user to readily master operation and even a beginner to use it with ease can be achieved. 
         [0163]    Incidentally, although the above-described series of processes can be executed by hardware, it can be executed by software. If the series of processes is executed by software, a program forming the software can be installed on a computer incorporated in dedicated hardware or a computer capable of performing various functions using various programs being installed thereon, for example, a general-purpose personal computer from a recording medium. 
         [0164]      FIG. 17  illustrates a configuration example of a general-purpose personal computer. The personal computer incorporates a central processing unit (CPU)  1001 . The CPU  1001  is connected to an input/output interface  1005  through a bus  1004 . The bus  1004  is connected to a read-only memory (ROM)  1002  and a random-access memory (RAM)  1003 . 
         [0165]    The input/output interface  1005  is connected to an input unit  1006  including an input device from which a user inputs an operation command, such as a keyboard or a mouse, an output unit  1007  for outputting an image of a processing operation screen or a result of processing to a display device, a storage unit  1008  including a hard disk drive in which a program and various kinds of data are retained, and a communication unit  1009  including, for example, a local area network (LAN) adapter and performing communication processing through a network, typified by the Internet. It is also connected to a drive  1010  for writing data on and reading data from a removable medium  1011 , such as a magnetic disc (including a flexible disc), an optical disc (including a compact-disk read-only memory (CD-ROM) and a digital versatile disc (DVD)), a magneto-optical disc (mini disc (MD), or semiconductor memory. 
         [0166]    The CPU  1001  executes various kinds of processing in accordance with a program stored in the ROM  1002  or a program read from the removal medium  1011  (e.g., a magnetic disc, an optical disc, a magneto-optical disc, or semiconductor memory), installed in the storage unit  1008 , and loaded into the RAM  1003 . The RAM  1003  also stores data required for execution of various kinds of processing by the CPU  1001 , as needed. 
         [0167]    Note that in the present specification a step of describing a program recorded in a recording medium includes a process performed chronologically in the order being stated, of course, and also includes a process that is not necessarily performed chronologically and is performed in a parallel manner or on an individual basis. 
         [0168]    Out of the functional component elements of the information processing apparatus  11  described above in reference to  FIG. 1 , noncontact capture unit  31 , information selection control unit  32 , information device system control unit  34 , information display control unit  35 , display unit  35 , imaging unit  51 , human body pose estimation unit  52 , pose recognition unit  54 , and gesture recognition unit  56  may be implemented as hardware using a circuit configuration that includes one or more integrated circuits, or may be implemented as software by having a program stored in the storage unit  1008  executed by a CPU (Central Processing Unit). The storage unit  1008  is realized by combining storage apparatuses, such as a ROM (e.g., ROM  1002 ) or RAM ( 1003 ), or removable storage media (e.g., removal medium  1011 ), such as optical discs, magnetic disks, or semiconductor memory, or may be implemented as any additional or alternate combination thereof. 
       REFERENCE SIGNS LIST 
       [0169]      11  information input apparatus; 
         [0170]      31  noncontact capture unit; 
         [0171]      32  information selection control unit; 
         [0172]      33  information option database; 
         [0173]      34  information device system control unit; 
         [0174]      35  information display control unit; 
         [0175]      36  display unit; 
         [0176]      51  imaging unit; 
         [0177]      52  human body pose estimation unit; 
         [0178]      53  pose storage database; 
         [0179]      54  pose recognition unit; 
         [0180]      55  classified pose storage database; 
         [0181]      56  gesture recognition unit; 
         [0182]      57  pose history data buffer; and 
         [0183]      58  gesture storage database