PATENT DOCUMENT

Publication Number: US-9459758-B2
Application Number: US-201313904052-A
Country: US
Kind Code: B2

Title: Gesture-based interface with enhanced features

Abstract:
A method includes presenting, on a display coupled to a computer, an image of a keyboard comprising multiple keys, and receiving a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display. An initial portion of the sequence of 3D maps is processed to detect a transverse gesture performed by a hand of a user positioned in proximity to the display, and a cursor is presented on the display at a position indicated by the transverse gesture. While presenting the cursor in proximity to the one of the multiple keys, one of the multiple keys is selected upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps.

Claims:
We claim: 
     
       1. A method, comprising:
 presenting, on a display coupled to a computer, an image of a keyboard comprising multiple keys; 
 receiving a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display; 
 processing an initial portion of the sequence of 3D maps to detect a transverse gesture performed by the hand of the user positioned in proximity to the display, the transverse gesture including a movement of the hand in a direction that is parallel to a plane of the display; 
 presenting, on the display, a cursor at a position indicated by the transverse gesture; and 
 selecting, while presenting the cursor in proximity to one of the multiple keys, the one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
 
     
     
       2. The method according to  claim 1 , and comprising presenting the selected key in a text input area on the display. 
     
     
       3. The method according to  claim 2 , and comprising performing a search based on the selected key, and presenting, in a results area on the display, a result of the search. 
     
     
       4. The method according to  claim 1 , wherein a given one of the multiple keys comprises a space key and multiple alphanumeric keys, and comprising presenting the space key within a row of the multiple keys. 
     
     
       5. The method according to  claim 1 , and comprising selecting the one of the multiple keys upon detecting the grab gesture in the subsequent portion of the sequence of 3D maps. 
     
     
       6. An apparatus, comprising:
 a sensing device; 
 a display; and 
 a computer coupled to the sensing device and the display, and configured to present, on the display, an image of a keyboard comprising multiple keys, to receive a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display coupled to the computer, to process an initial portion of the sequence of 3D maps to detect a transverse gesture performed by the hand of the user positioned in proximity to the display, the transverse gesture including a movement of the hand in a direction that is parallel to a plane of the display, to present, on the display, a cursor at a position indicated by the transverse gesture, and to select, while presenting the cursor in proximity to one of the multiple keys, the one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
 
     
     
       7. The apparatus according to  claim 6 , wherein the computer is configured to present the selected key in a text input area on the display. 
     
     
       8. The apparatus according to  claim 7 , wherein the computer is configured to perform a search based on the selected key, and to present, in a results area on the display, a result of the search. 
     
     
       9. The apparatus according to  claim 6 , wherein a given one of the multiple keys comprises a space key and multiple alphanumeric keys, and wherein the computer is configured to present the space key within a row of the multiple keys. 
     
     
       10. The apparatus according to  claim 6 , wherein the computer is configured to select the one of the multiple keys upon detecting the grab gesture in the subsequent portion of the sequence of 3D maps. 
     
     
       11. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a user interface, cause the computer to present, on a display coupled to a computer, an image of a keyboard comprising multiple keys, to receive a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display, to process an initial portion of the sequence of 3D maps to detect a transverse gesture performed by the hand of the user positioned in proximity to the display, the transverse gesture including a movement of the hand in a direction that is parallel to a plane of the display, to present, on the display, a cursor at a position indicated by the transverse gesture, and to select, while presenting the cursor in proximity to one of the multiple keys, the one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
     
     
       12. A method, comprising:
 receiving, by a computer, a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to a display coupled to the computer; 
 detecting, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, the pointing gesture including a pointing of a finger of the hand; and 
 presenting, in response to the pointing gesture, one or more interactive objects on the display. 
 
     
     
       13. The method according to  claim 12 , wherein presenting the one or more interactive objects comprises presenting the one or more interactive objects along the edge of the display. 
     
     
       14. An apparatus, comprising:
 a sensing device; 
 a display; and 
 a computer coupled to the sensing device and the display, and configured to receive a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to the display, to detect, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, the pointing gesture including a pointing of a finger of the hand, and to present, in response to the pointing gesture, one or more interactive objects on the display. 
 
     
     
       15. The apparatus according to  claim 14 , wherein the computer is configured to present the one or more interactive objects by presenting the one or more interactive objects along the edge of the display. 
     
     
       16. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a user interface, cause the computer to receive a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to a display coupled to the computer, to detect, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, the pointing gesture including a pointing of a finger of the hand, and to present, in response to the pointing gesture, one or more interactive objects on the display. 
     
     
       17. A method, comprising:
 detecting, by a computer at least two hands of at least one user of the computer; 
 assigning, based on a respective position of each of the hands, a respective ranking value to each of the hands that indicates an intention to use the hand to interact with the computer; 
 selecting a hand from among the at least two hands responsively to the respective ranking values; 
 receiving a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer; and 
 analyzing the 3D maps to detect a gesture performed by the selected hand. 
 
     
     
       18. The method according to  claim 17 , wherein detecting the at least two hands comprises receiving a two-dimensional (2D) image, and identifying the at least two hands in the 2D image. 
     
     
       19. The method according to  claim 17 , wherein detecting the at least two hands comprises receiving, prior to receiving the sequence of 3D map, an initial set of 3D maps, and detecting the at least two hands in the initial sequence of 3D maps. 
     
     
       20. An apparatus, comprising:
 a sensing device; 
 a display; and 
 a computer coupled to the sensing device and the display, and configured to detect at least two hands of at least one user of the computer, to assign, based on a respective position of each of the hands, a respective ranking value to each of the hands that indicates an intention to use the hand to interact with the computer, to select a hand from among the at least two hands responsively to the respective ranking values, to receive a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer, and to analyze the 3D maps to detect a gesture performed by the selected hand. 
 
     
     
       21. The apparatus according to  claim 20 , wherein the computer is configured to detect the at least two hands by receiving a two-dimensional (2D) image, and to identify the at least two hands in the 2D image. 
     
     
       22. The apparatus according to  claim 20 , wherein the computer is configured to detect the at least two hands by receiving, prior to receiving the sequence of 3D map, an initial set of 3D maps, and to detect the at least two hands in the initial sequence of 3D maps. 
     
     
       23. A computer software product comprising a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer executing a user interface, cause the computer to detect at least two hands of at least one user of the computer, to assign, based on a respective position of each of the hands, a respective ranking value to each of the hands that indicates an intention to use the hand to interact with the computer, to select a hand from among the at least two hands responsively to the respective ranking values, to receive a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer, and to analyze the 3D maps to detect a gesture performed by the selected hand.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation-in-part of U.S. patent application Ser. No. 13/541,786, filed Jul. 5, 2012, which claims the benefit of U.S. Provisional Patent Application 61/504,339, filed Jul. 5, 2011, of U.S. Provisional Patent Application 61/521,448, filed Aug. 9, 2011, and of U.S. Provisional Patent Application 61/523,349, filed Aug. 14, 2011. This application also claims the benefit of U.S. Provisional Patent Application 61/652,899, filed May 30, 2012. All of the above related applications are incorporated herein by reference. 
    
    
     FIELD OF THE INVENTION 
     This invention relates generally to user interfaces for computerized systems, and specifically to user interfaces that are based on three-dimensional sensing. 
     BACKGROUND 
     Many different types of user interface devices and methods are currently available. Common tactile interface devices include the computer keyboard, mouse and joystick. Touch screens detect the presence and location of a touch by a finger or other object within the display area. Infrared remote controls are widely used, and “wearable” hardware devices have been developed, as well, for purposes of remote control. 
     Computer interfaces based on three-dimensional (3D) sensing of parts of the user&#39;s body have also been proposed. For example, PCT International Publication WO 03/071410, whose disclosure is incorporated herein by reference, describes a gesture recognition system using depth-perceptive sensors. A 3D sensor provides position information, which is used to identify gestures created by a body part of interest. The gestures are recognized based on a shape of a body part and its position and orientation over an interval. The gesture is classified for determining an input into a related electronic device. 
     As another example, U.S. Pat. No. 7,348,963, whose disclosure is incorporated herein by reference, describes an interactive video display system, in which a display screen displays a visual image, and a camera captures 3D information regarding an object in an interactive area located in front of the display screen. A computer system directs the display screen to change the visual image in response to changes in the object. 
     Documents incorporated by reference in the present patent application are to be considered an integral part of the application except that to the extent any terms are defined in these incorporated documents in a manner that conflicts with the definitions made explicitly or implicitly in the present specification, only the definitions in the present specification should be considered. 
     The description above is presented as a general overview of related art in this field and should not be construed as an admission that any of the information it contains constitutes prior art against the present patent application. 
     SUMMARY 
     There is provided, in accordance with an embodiment of the present invention a method, including presenting, on a display coupled to a computer, an image of a keyboard comprising multiple keys, receiving a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display, processing an initial portion of the sequence of 3D maps to detect a transverse gesture performed by a hand of a user positioned in proximity to the display, presenting, on the display, a cursor at a position indicated by the transverse gesture, and selecting, while presenting the cursor in proximity to the one of the multiple keys, one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
     There is also provided, in accordance with an embodiment of the present invention an apparatus, including a sensing device, a display, and a computer coupled to the sensing device and the display, and configured to present, on the display, an image of a keyboard comprising multiple keys, to receive a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display coupled to the computer, to process an initial portion of the sequence of 3D maps to detect a transverse gesture performed by a hand of a user positioned in proximity to the display, to present, on the display, a cursor at a position indicated by the transverse gesture, and to select, while presenting the cursor in proximity to the one of the multiple keys, one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
     There is further provided, in accordance with an embodiment of the present invention a computer software product, including a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to present, on a display coupled to a computer, an image of a keyboard comprising multiple keys, to receive a sequence of three-dimensional (3D) maps including a hand of a user positioned in proximity to the display, to process an initial portion of the sequence of 3D maps to detect a transverse gesture performed by a hand of a user positioned in proximity to the display, to present, on the display, a cursor at a position indicated by the transverse gesture, and to select, while presenting the cursor in proximity to the one of the multiple keys, one of the multiple keys upon detecting a grab gesture followed by a pull gesture followed by a release gesture in a subsequent portion of the sequence of 3D maps. 
     There is additionally provided, in accordance with an embodiment of the present invention a method, including receiving, by a computer, a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to a display coupled to the computer, detecting, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, and presenting, in response to the pointing gesture, one or more interactive objects on the display. 
     There is also provided, in accordance with an embodiment of the present invention an apparatus, including a sensing device, a display, and a computer coupled to the sensing device and the display, and configured to receive a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to the display, to detect, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, and to present, in response to the pointing gesture, one or more interactive objects on the display. 
     There is further provided, in accordance with an embodiment of the present invention a computer software product, including a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive a sequence of three-dimensional (3D) maps containing at least a hand of a user positioned in proximity to a display coupled to the computer, to detect, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, and to present, in response to the pointing gesture, one or more interactive objects on the display. 
     There is additionally provided, in accordance with an embodiment of the present invention a method, including detecting, by a computer at least two hands of at least one user of the computer, assigning, based on a position of each of the hands, a respective ranking value to each of the hands, selecting a hand from among the at least two hands responsively to the respective ranking values, receiving a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer; and analyzing the 3D maps to detect a gesture performed by the selected hand. 
     There is also provided, in accordance with an embodiment of the present invention an apparatus, including a sensing device, a display, and a computer coupled to the sensing device and the display, and configured to detect at least two hands of at least one user of the computer to assign, based on a position of each of the hands, a respective ranking value to each of the hands, to select a hand from among the at least two hands responsively to the respective ranking values, to receive a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer, and to analyze the 3D maps to detect a gesture performed by the selected hand. 
     There is further provided, in accordance with an embodiment of the present invention a computer software product, including a non-transitory computer-readable medium, in which program instructions are stored, which instructions, when read by a computer, cause the computer to detect at least two hands of at least one user of the computer, to assign, based on a position of each of the hands, a respective ranking value to each of the hands, to select a hand from among the at least two hands responsively to the respective ranking values, to receive a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to a display coupled to the computer, and to analyze the 3D maps to detect a gesture performed by the selected hand. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The disclosure is herein described, by way of example only, with reference to the accompanying drawings, wherein: 
         FIG. 1  is a schematic, pictorial illustration of a computer system implementing a non-tactile zoom-based user interface, in accordance with an embodiment of the present invention; 
         FIG. 2  is an schematic illustration of a tree data structure that the computer can present as a series of hierarchical ZoomGrid surfaces, in accordance with an embodiment of the present invention; 
         FIG. 3  is a flow diagram that schematically illustrates a method of interacting with the ZoomGrid surfaces, in accordance with an embodiment of the present invention; 
         FIGS. 4A-4D  are schematic pictorial illustrations showing a multi-level ZoomGrid control scheme, in accordance with an embodiment of the present invention; 
         FIG. 5  is a schematic illustration of a pile of interactive objects, in accordance with an embodiment of the present invention; 
         FIG. 6  is a schematic pictorial illustration of a first example of a ZoomGrid-based media player control, in accordance with an embodiment of the present invention; 
         FIG. 7  is a schematic pictorial illustration of a second example of a ZoomGrid-based media player control, in accordance with an embodiment of the present invention; 
         FIG. 8  is a pictorial illustration of an on-screen keyboard, in accordance with an embodiment of the present invention; 
         FIGS. 9A-9C  are schematic pictorial illustrations of a user of the computer system implementing a non-tactile zoom-based user interface performing an off-screen interaction, in accordance with an embodiment of the present invention; and 
         FIG. 10  is a schematic illustration of the user positioning both hands within a field of view of a sensing device coupled to the computer system implementing a non-tactile zoom-based user interface, in accordance with an embodiment of the present invention. 
     
    
    
     DETAILED DESCRIPTION OF EMBODIMENTS 
     When using physical tactile input devices such as buttons, rollers or touch screens, a user typically engages and disengages control of a user interface by touching and/or manipulating the physical device. Embodiments of the present invention provide methods and mechanisms for interacting with a display coupled to a computer executing a non-tactile zoom-based user interface that includes three-dimensional (3D) sensing, by a 3D sensor, of motion or change of position of one or more body parts, typically a hand or a finger, of the user. 
     In some embodiments the zoom-based user interface utilizes a ZoomGrid control scheme that enables the user to select a given interactive object from multiple interactive objects presented on a display. The ZoomGrid control scheme described hereinbelow utilizes a hierarchical data structure having multiple levels with multiple nodes, wherein non-leaf nodes may represent categories (e.g., movies and music), for example, while leaf nodes represent content (e.g., media files and software applications. 
     Using embodiments described herein, a user can perform 3D gestures to traverse the hierarchical data structure in order to find a specific node storing content and perform an operation on the content. In some embodiments, if the content comprises a movie, the user can perform 3D gestures to manipulate on-screen media controls for operations such as volume control, pause, seek, etc. In additional embodiments of the present invention, the user can perform 3D gestures to enter text via an on-screen keyboard, and point to areas just outside a display to select “hidden icons”. 
     When interacting with a computer executing a non-tactile zoom-based user interface, a user may be positioned so that both of the user&#39;s hands are positioned within a field of view of a 3D optical sensor coupled to the computer. Additionally, there may be more than one user positioned within the sensor&#39;s field of view. In embodiments of the present invention, the computer can analyze a position of each hand within the field of view, and identify which of the hands is most likely intentionally interacting with the non-tactile user interface by performing 3D gestures. 
     System Description 
       FIG. 1  is a schematic, pictorial illustration of a non-tactile zoom-based interface  20  (also referred to herein as the ZoomGrid interface) for operation by a user  22  of a computer  26 , in accordance with an embodiment of the present invention. The non-tactile zoom-based interface is based on a 3D sensing device  24  coupled to the computer, which captures 3D scene information of a scene that includes the body (or at least a body part, such as one or more of hands  30 ) of the user. Device  24  or a separate camera (not shown in the figures) may also capture video images of the scene. The information captured by device  24  is processed by computer  26 , which drives a display  28  so as to present and manipulate on-screen interactive objects  38 . Details of the operation of one appropriate type of 3D sensing device  24  are described in U.S. Patent Application Publication 2010/0007717, whose disclosure is incorporated herein by reference. 
     Computer  26 , executing zoom-based interface  20 , processes data generated by device  24  in order to reconstruct a 3D map of user  22 . The term “3D map” refers to a set of 3D coordinates measured, by way of example, with reference to a generally horizontal X-axis  32  in space, a generally vertical Y-axis  34  in space and a depth Z-axis  36  in space, based on device  24 . The 3D coordinates represent the surface of a given object, in this case the user&#39;s body. In embodiments described below, as user  22  moves hand  30  along Z-axis  36  and an X-Y plane  40 , computer  26  is configured to process the inputs received from the user in order to control location of a cursor  42  presented on display  28 . The Z-direction, i.e., the direction perpendicular to the plane of display  28 , is referred to in the present description and in the claims as the longitudinal direction, while directions within an X-Y plane, parallel to the plane of display, are referred to as transverse directions. 
     In one embodiment, device  24  projects a pattern of spots onto the object and captures an image of the projected pattern. Computer  26  then computes the 3D coordinates of points on the surface of the user&#39;s body by triangulation, based on transverse shifts of the spots in the pattern. Methods and devices for this sort of triangulation-based 3D mapping using a projected pattern are described, for example, in PCT International Publications WO 2007/043036, WO 2007/105205 and WO 2008/120217, whose disclosures are incorporated herein by reference. Alternatively, interface  20  may use other methods of 3D mapping, using single or multiple cameras or other types of sensors, as are known in the art. 
     Computer  26  typically comprises a general-purpose computer processor, which is programmed in software to carry out the functions described hereinbelow. The software may be downloaded to the processor in electronic form, over a network, for example, or it may alternatively be provided on non-transitory tangible media, such as optical, magnetic, or electronic memory media. Alternatively or additionally, some or all of the functions of the image processor may be implemented in dedicated hardware, such as a custom or semi-custom integrated circuit or a programmable digital signal processor (DSP). Although computer  26  is shown in  FIG. 1 , by way of example, as a separate unit from sensing device  24 , some or all of the processing functions of the computer may be performed by suitable dedicated circuitry within the housing of the sensing device or otherwise associated with the sensing device. 
     As another alternative, these processing functions may be carried out by a suitable processor that is integrated with display  28  (in a television set, for example) or with any other suitable sort of computerized device, such as a game console or media player. The sensing functions of device  24  may likewise be integrated into the computer or other computerized apparatus that is to be controlled by the sensor output. 
     Zoomgrid Surfaces 
       FIG. 2  is an schematic illustration of a hierarchical data structure  50  (also referred to herein as a tree) that computer  26  can present as a series of hierarchical ZoomGrid surfaces  52 , in accordance with an embodiment of the present invention. Each of the ZoomGrid surfaces comprises one or more interactive objects  38 . In the description herein, interactive objects  38  may be differentiated by appending a letter to the identifying numeral, so that interactive objects  38  comprise interactive objects  38 A- 38 N, and surfaces  52  may be differentiated by appending a letter to the identifying numeral so that surface  52  comprise surfaces  52 A- 52 E. 
     In operation, as user  22  traverses tree  50  and accesses a given interactive object  38 , computer  26  presents a given ZoomGrid surface  52  comprising sub-objects  38  (i.e., children nodes in tree  50 ) of the given interactive object. In the example shown in  FIG. 2 , the sub-objects can comprise menus, media objects or media player controls. 
     As described hereinbelow, starting from a given surface  52 , user  22  can traverse data structure  50  toward a given interactive object  38  (e.g., interactive object  38 L) by performing a Find gesture (also referred to herein as a Transverse gesture), followed by a Grab gesture, a Pull Gesture and a Release gesture. Likewise, starting from the given surface, user  22  can traverse data structure  50  toward root interactive object  38 A by performing a Find gesture, followed by a Grab gesture, a Push Gesture and a Release gesture. The terms grab, push and release are used in the present description and in the claims in their literal senses, to describe hand motions that would be used to graph, push and release a physical object, respectively, although in embodiments of the present invention these gestures are generally performed with respect to an interactive object, without there being any actual physical object in the hand. 
       FIG. 3  is a flow diagram that illustrates a method of interacting with ZoomGrid surfaces  52 , in accordance with an embodiment of the present invention. In an initialization step  60 , computer  26  arranges multiple interactive objects  38  as hierarchical data structure  50 , wherein each node of the hierarchical data structure is associated with a respective one of the multiple interactive objects. 
     In a presentation step  62 , computer  26  presents, on display  28 , a subset (i.e., a given surface  52 ) of interactive objects  38  that are associated with one or more child nodes of a first interactive object  38 . As described supra, user  22  traverses tree  50  by accessing a given interactive object  38 . Therefore, when initiating an interaction with ZoomGrid surfaces  52 , user initially accesses interactive object  38 A, and computer  26  presents interactive objects  38 B,  38 C and  38 D. Using embodiments described herein, user  22  can then traverse tree  50 . 
     In a receive step  64 , computer  26  receives, from sensing device  24 , a sequence of 3D maps that include at least a part of hand  30  positioned in proximity to display  28 , and in an identification step  66 , the computer identifies, in the sequence of 3D maps, a Find gesture followed by a Grab gesture followed by a longitudinal gesture followed by an Execute gesture. The term Execute gesture is used in the present description and in the claims in their literal senses, to describe a hand motion that user  22  performs subsequent to the longitudinal gesture in order to instruct computer  26  to perform an operation on a selected interactive object  38 . 
     Examples of the Execute gesture include:
         A time delay. The user can keep hand  30  steady for at least a specific time period. Keeping hand  30  steady for at least a specific period of time is also referred to as a TimeClick gesture, which is described in U.S. Patent Application Publication 2013/0044053, whose disclosure is incorporated herein by reference.   Perform a Release gesture. If the user pulls hand  30  back, the glass door stays “closed” until the user opens his hand.   Break on panning. If the user starts “swiping” hand  30  to the left or the right (i.e., a Find gesture), the glass door opens, and the computer scrolls data the presented interactive objects in a direction of the swiping motion.       

     The Find gesture is described in U.S. Patent Application Publication 2012/0223882, whose disclosure is incorporated herein by reference. To perform the Find gesture, user  22  moves hand  30  along X-Y plane  40 , and computer  26  can position cursor  42  on display  28  in response to the motion of the hand. 
     The Grab and the Release gestures are described in U.S. Patent Application Publication 2012/0204133, whose disclosure is incorporated herein by reference. To perform the Grab gesture, user  22  closed hand  30  by folding one or more fingers of hand  30  toward a palm of the hand. To perform the Release gesture, user  22  opens hand  30  from a closed or folded state. 
     Longitudinal gestures include a Push gesture and a Pull gesture, are also described in U.S. Patent Application Publication 2012/0204133, referenced above. User  22  can perform the Push gesture by moving hand  30  along Z-axis  36  toward display  28 . Likewise, user  22  can perform the Pull gesture by moving hand  30  along Z-axis  36  away from display  28 . 
     In a selection step  68 , in response to identifying the Find gesture, computer  26  selects a second interactive object  38  from the subset of interactive objects  38 . For example, to select the second interactive object, user can move hand  30  along X-Y plane  40 , and upon computer  26  responsively positioning cursor  42  over (or in proximity to) the second interactive object, the user can either transition to a longitudinal gesture or keeping the hand relatively steady for a specific period of time. 
     Finally, in a performance step  70 , in response to detecting the Release gesture, computer  26  performs an operation on the selected second interactive object. Examples of operations that computer  26  can perform on the second interactive object include, but are not limited to:
         Playing a media file. If the second interactive object comprises a media file (e.g., a movie), then computer  26  presents the media file on display  28  upon detecting the   Execute gesture comprising either a Release gesture or a TimeClick gesture.   Execute a software application. If the second interactive object comprises a software application, then computer  26  executes the software application upon detecting the Execute gesture comprising either a Release gesture or a TimeClick gesture.   Present a child surface  52 . If the longitudinal gesture comprises a Pull gesture, and the second interactive object is not associated with content (i.e., a media file or a software application), then the second interactive object comprises a child interactive object  38  of the first interactive object, and computer  26  presents one or more child interactive objects  38  of the second interactive object. For example, if the first interactive object comprises interactive object  38 C, then computer  26  initially presents interactive objects  38 H,  381  and  38 J. Upon user  22  selecting interactive object  381  via a Find gesture, computer  26  presents interactive objects  38 K,  38 L and  38 M upon detecting a Grab gesture followed by a Pull gesture and an Execute gesture comprising a Release gesture, a Find gesture or a TimeClick gesture.   Present a parent surface  52 . If the longitudinal gesture comprises a Push gesture, and the first interactive object is root interactive object  38 A, then the second interactive object comprises a parent interactive object  38  of the first interactive object, and computer  26  presents one or more child interactive objects  38  of the second interactive object. For example, if the first interactive object comprises interactive object  381 , then computer  26  initially presents interactive objects  38 K,  38 L and  38 M. Upon user  22  selecting interactive object  38 C via a Find gesture, computer  26  presents interactive objects  38 H,  381  and  38 J upon detecting a Grab gesture followed by a Push gesture and an Execute gesture comprising a Release gesture, a Find gesture or a TimeClick gesture.       

     After selecting the second interactive object in response to the Find gesture performed by the user, the computer can “zoom in” (i.e., increase the size) of the second interactive object in response to detecting user  22  performing a Grab gesture followed by a Pull gesture. While increasing the size of the second interactive object, computer  22  can present context information about the second interactive object. For example, if the second interactive object comprises a movie, computer  22  can present context information such as a plot and a list of actors, as the computer increases the size of the second interactive object. 
       FIGS. 4A-4D  are schematic pictorial illustrations showing a multi-level ZoomGrid control scheme based on tree  50 , in accordance with an embodiment of the present invention. In the example shown in  FIGS. 4A-4D , user  22  traverses ZoomGrid surfaces  52  to start playing a movie referenced by interactive object  38 L. Using embodiments described herein, a user gradually zooms in from viewing a menu of media types (ZoomGrid Surface  52 A in  FIG. 4A ), to types categories of movies (ZoomGrid Surface  52 C in  FIG. 4B ), to viewing the movies in a particular category (ZoomGrid Surface  52 D in  FIG. 4C ), and then zooming in to select interactive object  38 L for viewing ( FIG. 4D ). 
     Comfort Zones and Funnels 
     Two stabilization mechanisms described U.S. patent application Ser. No. 13/541,786 (referenced above) comprise comfort zones and funnels. While interacting with a multi-level ZoomGrid, computer  26  can define certain zoom levels as “comfort zones,” because they enable computer  26  to present interactive objects  38  in a aesthetic manner (for example, with an integer number of rows and columns of icons, with no icons cut off at the edges of the display). In other words computer  26  can “lock” user  22  into a comfort zone (i.e., a comfortable zoom level) while the user browses horizontally (e.g., using a Find gesture). 
     While in a comfort zone, if user  22  transversely moves hand (i.e., along X-axis  32  and/or Y-axis  34 ) while display  28  is in a comfort zone, the zoom may be locked, so that only significant motions along Z-axis  36  motions changes the zoom level. Specifically, computer  26  can be configured to assign less significance to hand motion detected along Z-axis  36  than hand motion detected along X-axis  32  and Y-axis  34 , while user  22  is performing a Find gesture. In other situations, the zoom levels can be biased in order to drive the display into a comfort zone in response to relatively small movement of hand  30  along Z-axis  36 . For example, if a given interactive object  38  comprises a folder of sub-objects  38 , then computer  26  can enlarge the given interactive object (and thereby display the sub-objects) upon detecting significant motion of hand  30  away from display  28 . 
     The “Funnel” mechanism enables computer  26  to accommodate any inadvertent transverse motion while user  22  is performing a longitudinal gesture. Specifically, computer  26  can be configured to assign less significance to hand motion detected along X-axis  32  and/or Y-axis  34  than hand motion detected on Z-axis  36 , while user  22  is performing a Pull or a Push gesture. Limiting the significance of any transverse motion as computer  26  enlarges (i.e., “zooms in” on) the active interactive object as the user performs a Pull gesture can create a “funnel” like sensation that can help guide the user towards a given interactive object  38  that the user intends to select. 
     The significance of the transverse motion can be inversely related to a location of hand  30  while performing a Pull gesture. In other words, computer  26  can assign less significance to any detected transverse motion of hand  30  as the distance between the hand and display  28  increases. In operation, if computer  26  “suspects” that user  22  has identified a given interactive object and detects the user starting to perform a Pull gesture, the computer can start to limit the significance of any detected transverse motion of hand  30 . As the Pull gesture progresses (and computer  26  further zooms in on the given interactive object) the computer can responsively decrease the significance of any detected transverse motion. 
     In some embodiments, the “funnel” paradigm can be extended to inhibit the association of hand  30  with a different interactive object  38  when the associated interactive object has been enlarged beyond a predetermined threshold size, responsively to user  22  moving hand  30  away from display  28 . In other words, upon computer  26  presenting the associated object at a size equal or greater to the predetermined size, the computer can substantially ignore any transverse movement of hand  30  along X-axis  32  and/or Y-axis  34 . 
     Glass Doors 
     As described supra, after selecting the second interactive object in response to a Find gesture performed by the user, the computer can “zoom in” (i.e., increase the size) of the second interactive object in response to detecting user  22  performing a Grab gesture followed by a Pull gesture. In embodiments of the present invention, as computer  26  increases the size of the second interactive object, the computer can also present a preview of the second interactive object. For example, in addition (or as an alternative) to increasing the size of the second interactive object, computer  26  can present a “preview” of the second interactive object&#39;s child nodes, similar to “peeking” through a glass door. 
     This glass door metaphor is based on a front glass wall of a store that includes sliding doors that open as a customer approaches the glass. In embodiments of the present invention, a “zoomable object” (i.e., a given interactive object  38  having child nodes) may use the glass door to control how a user interacts with data associated with the zoomable object. To view the data associated with the zoomable object, the user can pull hand  30  back away from the zoomable object, and either open the zoomable object (i.e., open the glass door) or “peek” inside the zoomable object. When the user opens the glass door, the user interface can transition to a comfort zone associated with the zoomable object&#39;s associated data. 
     For example, a given zoomable object may comprise an icon representing a collection of movies in a specific category (e.g., drama or comedy). As the user pulls his hand back away from the icon, computer  26  presents a listing of the movies in the category (i.e., data associated with the icon). Once the user “opens” the glass door, the user can perform an operation on the data associated with the icon (e.g., start playing one of the movies in the category) 
     Actions that can “open” the Glass Door include:
         Performing a TimeClick gesture. The user can keep hand  30  steady for at least a specific time period.   Performing a Release gesture. If the user pulls his hand back, the glass door stays “closed” until the user opens his hand.   Performing a Find gesture (i.e., “break on panning”). If the user starts “swiping” his hand to the left or the right (i.e., a Find gesture), the glass door opens, and the computer scrolls data the presented interactive objects in a direction of the swiping motion.       

     In embodiments of the present invention, the user can “peek” inside a given glass door without “opening” the door. When peeking inside, the user can view interactive objects  38  at that level, but may not be able to instruct computer  26  to perform an operation on the interactive objects at that level. For example, by pulling the hand back from the documentary movie icon (i.e., interactive object  381 ), the user can peek at the documentary movie selection, but the user can only select one of the movies after “opening” the glass door. User  22  can then pull hand  30  back to see the movies in a given category, and then push the hand forward to return to the different movie categories. By opening glass doors, user  22  can transition between different hierarchical levels of the zooming user interface. 
     For example, to start playing interactive object  38 L, in response to user  22  opening a first glass door to select Movies (i.e., interactive object  38 C), computer  26  responsively presents icons representing different categories of movies (i.e., interactive objects  38 H,  381  and  38 J). As user  22  peeks into the Documentary category (i.e., interactive object  381 ), computer  26  presents a grid of all the movies (represented by icons) in the Documentary category. 
     While surface  52 D in the example shown in  FIG. 2  comprises three interactive objects  38 , a given surface  52  may comprise any number of interactive objects  38 . For example, if a given surface  52  comprises 200 interactive objects  38 , then as user  22  peeks into the given interactive surface, computer  26  can present a “preview” comprising a grid of all the interactive objects of the given surface. However if the user “opens” the glass door to the given surface, then computer  26  can present a more legible subset the interactive objects of the given surface. Computer  26  can present the subset of interactive objects  38  as larger (and more viewable) icons, and the entire selection of icons can be scrolled left or right using embodiments describes in U.S. patent application Ser. No. 13/541,786, referenced above. 
     In other words, the user can see all the objects at the next hierarchical level when peeking at the level, even though the number of objects shown may limit the visibility of the objects. However of the user opens a glass door to a next hierarchical level, then a smaller (and scrollable) set of objects are shown that enable the user to easily interact with the presented objects. 
     Methods of peeking that computer  26  can implement to view content (i.e., interactive objects  38 ) at a given surface  52  include:
         Presenting interactive objects  38  in given surface  52 . Therefore the user can see if there are 20 or 200 interactive objects  38  in the given surface.   Presenting the interactive objects of the given surface in a “fanned out” format, in a manner similar to a fanned out pile of magazines.   When peeking into an interactive object that is associated with content (e.g., a media file or a software application), the computer can “zoom” an image and/or context information into “full-screen”, and not automatically allow zooming into higher zoom levels.       

     As described supra, when user  22  peeks into a given surface  52 , computer  26  can present the interactive objects of the given surface in a “fanned out” format, in a manner similar to a fanned out pile of magazines. When presenting objects at a given hierarchical level, computer  26  can use the following formula to present objects while looking at a first glass door and peeking inside a second glass door (i.e., at the next hierarchical level:
 
Ratio=(Amount of detail on glass door)/(Amount of detail in collapsed state)
 
where the numerator represents a number of interactive objects  38  at an adjacent lower hierarchical level (i.e., inside a given glass door) and the denominator represents a number of interactive objects  38  that computer  26  presents in detail at the current hierarchical level (i.e., when looking at the given glass door).
 
       FIG. 5  is a schematic illustration of a “pile”  80  of interactive objects  38  that is viewable from a given glass door, in accordance with an embodiment of the present invention. In the example shown in  FIG. 5 , the pile only shows one interactive object  38  (at the top of the pile) in detail. If the user peeks at the pile, then the user can see the six objects (comprising the pile) spread out. Therefore, the ratio in  FIG. 5  is 6/1. 
     Similar to the manner in which the glass door can be used to moderate a zoom-in gesture (i.e., a Pull gesture), a back door can be used to moderate a zoom-out gesture (i.e., a Push gesture). For example, if the user is at a given hierarchical level in  FIG. 5  that shows the six interactive objects spread out, the computer can present pile  80  upon the user performing a Push gesture. The user can “exit via the back door” by performing either a TimeClick gesture or a Release gesture. In other words, in a manner similar to peeking inside a glass door, user  22  can peek outside a back door and still remain at the same hierarchy level. 
     Examples of back door implementations include:
         If the user is looking at movie details, upon exiting the back door, computer  26  “collapses” the current interactive object and the zoom level is animated to a higher hierarchy level, e.g., a collection of movies.   If the user is watching a live television broadcast, computer  26  can present details on the next show to be broadcast.       

     As described supra, depending on a configuration of a given comfort zone, computer  26  can present a group of interactive objects  38  (e.g., icons) either in a pile, a grid or spread out in order to see details on each of the objects. In some embodiments, computer  26  can present the interactive objects in an interpolated layout. For example, a given interpolated layout may present a pile of 25 movies, with four of the movies fanned out. 
     In some embodiments, a given layout (e.g., interactive objects stacked or fanned out, as shown in  FIG. 5 ) can be dynamically changed. For example, visual effects such as stacking or unstacking a group of interactive objects  38  can be implemented and tied to a position of hand  30  along Z-axis  36 . In other words, computer  26  can change a layout of a given group of interactive objects  38  as user  22  performs a longitudinal gesture with hand  30 . 
     In additional embodiments, layout parameters can be changed to mix (i.e., interpolate) a given layout between presenting all the interactive objects as a “fanned out” pile and presenting the interactive objects as pile  80 . In some embodiments, computer  26  can dynamically present the interpolation as an animation. 
     In further embodiments, the interpolated layouts can be nested, thereby enabling computer  26  to mix three or more layouts. Examples of layout that computer  26  can mix include, but are not limited to:
         A stack.   A 10×1 grid.   A 5×5 grid.   A random layout.       

     Media Controls 
     While playing a media file, computer  26  can present media controls that user  22  can engage using embodiments described herein. Media controls may consist of controls for actions such as play/pause, seek, mute, volume up/down, next/previous track, fast-forward, rewind etc. In embodiments of the present invention, while computer  26  plays a media file (either in the foreground or in the background), the computer can place media player controls an “invisible” ZoomGrid surface  52 . During the playback, when the user performs a longitudinal gesture, the ZoomGrid surface containing the imaginary controls gradually becomes visible. User  22  can select the media controls in a manner similar to selecting any other interactive objects  38  presented on a ZoomGrid surface. 
     In operation, while playing a media file (i.e., subsequent to performing the operation on the selected interactive object  38  in step  70 ), computer  26  can receive an additional set of 3D maps and detect, in the additional set of 3D maps, an additional Grab gesture followed by an additional Find gesture followed by a further Grab gesture followed by an additional Release gesture. Computer  26  can present one or more media controls in response to the additional Grab gesture, and position cursor  42  in response to the additional Find gesture. Upon detecting the further Grab gesture, computer  26  can identify one of the one or more media controls presented in proximity to cursor  42 , and perform an operation associated with the one of the one or more media controls (e.g., increasing the volume) upon detecting the additional Release gesture. 
     A special behavior of the ZoomGrid based player controls mechanism is that upon selecting a given control, the controls surface can retract back and “disappear” (i.e., as if a spring connects it to the surface on which the media is playing). Toggle controls like mute/unmute or pause/play or buttons can be implemented again by the same embodiments as those used to select the interactive objects (i.e., the object selection triggers a given operation). Computer  26  can implement continuous controls such as volume and seek by identifying further gestures, as explained hereinbelow. 
       FIG. 6  is a schematic pictorial illustration of a first example of a ZoomGrid based media player control, in accordance with an embodiment of the present invention. While playing interactive object  38 L, computer  26  can present a one-dimensional ZoomGrid  90  in response to the user performing a Pull gesture. ZoomGrid  90  comprises a pause control  92 , a stop control  94 , play control  96 , and a seek control  98 . In the example shown in  FIG. 6 , user  22  has selected a volume control  100  using embodiments described herein, and can manipulate a volume slider icon  102  via transverse hand motions along X-axis  32 . 
     As described supra, computer  26  can implement continuous controls such as volume and seek by identifying further gestures performed by hand  30 . For example, if computer  26  identifies volume slider icon  102  in response to the additional Pull gesture, computer  26  can adjust an audio volume level (and reposition the volume slider icon) in response to detecting, in the additional set of 3D maps, a further Find gesture subsequent to the additional Pull gesture and prior to the additional Release gesture. Upon detecting the additional Release gesture subsequent to the further Find gesture, computer  26  can maintain the audio volume level indicated by a position of the volume slider icon on display  28 . 
       FIG. 7  is a schematic pictorial illustration of a second example of a ZoomGrid based media player control, in accordance with an embodiment of the present invention. In response to user  22  selecting seek control  78  using embodiments described herein, computer  26  presents a one-dimensional ZoomGrid  110  that comprises scrub points  112 A- 112 G. The scrub points comprise specific scenes in movie  38 L that user  22  can directly skip to by selecting one of the scrub points using the embodiments described herein. In the example shown in  FIG. 7 , user  22  has selected scrub point  112 A, and the computer responsively presents a given scene in movie  38 L that corresponds to the selected scrub point. 
     Zoomgrid Keyboard 
       FIG. 8  is a pictorial illustration of an on-screen keyboard  120  for use by non-tactile 3D user interface  20 , in accordance with an embodiment of the present invention. On-screen keyboard  120  comprises a keys area  122 , a text input area  124  and a results area  126 . In the example shown in  FIG. 8 , computer  26  presents, in keys area  122 , a space bar  128 A within a bottom row of alphanumeric keys  128 , thereby reducing space taken up by the on-screen keyboard on display  28 . 
     In embodiments of the present invention, computer  26  presents an image of keyboard  120  on display  28 , and receives a sequence of three-dimensional (3D) maps including a hand  30 . Upon processing an initial portion of the sequence of 3D maps and detecting hand  30  performing a Find gesture, computer  26  positions, on display  28 , cursor  42  at a position indicated by the Find gesture. While presenting cursor  42  in proximity to a given key  128 , and detecting in a subsequent portion of the sequence of 3D maps, a Grab gesture followed by a Pull gesture followed by a Release gesture, computer  26  selects the given key. 
     In operation, as user  22  performs a Find gesture by moving hand  30  along X-Y plane  40 , computer  26  responsively highlights a given key  128 . Upon performing a Grab gesture followed by a Pull gesture, computer  26  can convey visual feedback such as increasing the size of the given key (“C” as shown in the figure), and upon detecting the Release gesture, the computer can present a character associated with the given key in text input area  124 . In some embodiments, upon user  22  transitioning the hand&#39;s motion from the X-Y plane to the Z-axis (i.e., pulling the hand back), computer  26  can convey visual feedback such as increasing the size of the given key, as shown in  FIG. 8 . In additional embodiments, user  22  can delete the last character entered by performing a Push gesture (i.e., “undo”). 
     In some configurations, computer  26  can select a given key upon detecting, in the 3D maps, a Find gesture followed by a Grab gesture. In other words, while computer  26  is highlighting a given key  128 , the computer can select the given key upon detecting a Grab gesture while the given key is highlighted. 
     In the configuration shown in  FIG. 8 , as user  22  enters text using keyboard  120 , computer  26  can present search results in results area  126 . For example, as user starts entering a name of a movie, computer  26  can present one or more interactive objects  38  whose name matches text entered by the user. Using embodiments described herein, user  22  can select a given interactive object presented in results area  126 . 
     Off-Screen Interaction 
     While interacting with non-tactile 3D user interface  20 , user  22  typically manipulates interactive objects  38  presented on display  28 . However, when using a computer or a device controlled by a tactile user interface, there may be controls that are positioned outside the display. For example, Android™ smartphones have buttons positioned below the touch screen. 
     In embodiments of the present invention, upon computer  26  receiving a sequence of three-dimensional (3D) maps containing at least hand  30  positioned in proximity to display  28  and detecting, in the 3D maps, a pointing gesture directed toward a region external to the display and adjacent to an edge of the display, the computer can present one or more interactive objects  38  on display in response to the pointing gesture. In some embodiments, computer  26  can present the one or more interactive objects along the edge of the display that is adjacent to the regions. 
     A pointing gesture typically comprises user  22  pointing a finger of hand  30  toward display  28  to select a given interactive object  38  presented on the display, and are described in PCT International Publication WO 2012/107892, whose disclosure is incorporated herein by reference. In embodiments of the present invention, user  22  may perform a pointing gesture to a region outside display  28 . 
       FIGS. 9A-9C  are schematic pictorial illustrations of user  22  performing an off-screen interaction, in accordance with an embodiment of the present invention. In  FIG. 9A , user points hand  30  above the display toward a region  130 . In response to the user pointing hand at region  130 , computer  26  can zoom out any content that is currently being shown on the display, and present interactive objects  38  on display  28 , as shown in  FIG. 9B . Alternatively, computer  26  can dim the content while presenting interactive objects  28 , as shown in  FIG. 9C . 
     User Intention Rating System 
     While user  22  may have two hands  30  in proximity to sensing device  24 , the user interacts with the computer one of the hands at any given time. Additionally, there may be multiple individuals in a field of view of sensing device  24 . Embodiments of the present invention provide methods and systems for rating each hand  30  within the field of view of sensing device  24  in order to identify which of the hands is most likely intentionally interacting with non-tactile 3D user interface  20 . 
     In operation, computer  26  detects at least two hands  30  of at least one user  22 , and assigns a respective ranking value (also referred to herein as a rating) to each of the hands based on a position of each of the hands. Although the configuration of sensing device  24  shown in  FIG. 1  comprises a 3D sensor, the sensing device may also comprise a two-dimensional (2D) optical sensor. Therefore, in some embodiments, computer  26  can receive a two-dimensional (2D) image from sensing device  24 , and identify at least two hands  30  in the 2D image. In alternative embodiments computer  26  can receive an initial set of 3D maps from sensing device  24 , and detect at least two hands  30  in the initial sequence of 3D maps. 
     Computer  26  can then select a given hand  30  from among the at least two hands responsively to the respective ranking values, and upon receiving a sequence of three-dimensional (3D) maps containing at least the selected hand positioned in proximity to display  28 , computer  26  can analyze the 3D maps to detect a gesture performed by the selected hand. In embodiments where computer  26  identifies hands  30  from an initial sequence of 3D maps, the 3D maps that the computer analyzes to detect a gesture comprise a set of 3D maps that computer  26  receives subsequent to receiving the initial set of 3D maps that were used to detect the two or more hands. 
     In some embodiments, the computer can identify hands  30  within the field of view, detect poses of the user(s) positioned within the field of view, and assign a rating based on the position of each the hands and the pose of each of the users. If the rating for a given hand  30  exceeds a defined threshold, then the computer can accept gestures from the given hand (i.e., the hand is active). During times where there is no user  22  interacting with the system, the computer may require a more overt gesture in order to select a given hand  30  as being active. 
       FIG. 10  is a schematic illustration of user  22  with hands  30  positioned within a field of view  130  of sensing device  24 , in accordance with an embodiment of the present invention. In the description herein, hands  30  may be differentiated by appending a letter to the identifying numeral, so that hands  30  comprise hand  30 A and hand  30 B. 
     In the example shown in  FIG. 10 , user  22  raises hand  30 A (i.e., the left hand) and rests hand  30 B (i.e., the right hand) at the user&#39;s side. Computer  26  can apply a higher score to hand  30 A since the left hand is raised and positioned closer to the display. If there are additional individuals in the room, computer  26  can also take into account the fact that the user is positioned relatively close to the display and is facing the display, thereby increasing the rating of both the user&#39;s hands. 
     For example, computer  26  may be configured to identify gestures performed with both of the hands, and the rating can be used to identify which of the individuals is most likely interacting with user interface  20 . Additionally or alternatively, the hand ratings can be used for session management and for reducing false positives (i.e., reducing chances that the computer interprets a motion of a given hand  30  as a gesture, when the user did not intend to perform a gesture). 
     It will be appreciated that the embodiments described above are cited by way of example, and that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of the various features described hereinabove, as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing description and which are not disclosed in the prior art.

Metadata:
Filing Date: 20130529
Publication Date: 20161004
Grant Date: 20161004
Priority Date: 20110705
Inventors: BERENSON ADI
GALOR MICHA
POKRASS JONATHAN
SHANI RAN
SHEIN DANIEL
WEISSENSTERN ERAN
FREY MARTIN
HOFFNUNG AMIR
METUKI NILI
Assignee: APPLE INC
CPC Classifications: [{"code": "G06F3/04842", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F2203/04806", "inventive": false, "first": false, "tree": "[]"}, {"code": "G06F3/0482", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/017", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04842", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/017", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F3/04886", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/0482", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04886", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04815", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F2203/04806", "inventive": false, "first": false, "tree": "[]"}, {"code": "G06F3/04886", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/04815", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F3/04842", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/017", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F2203/04806", "inventive": false, "first": false, "tree": "[]"}, {"code": "G06F3/0482", "inventive": true, "first": false, "tree": "[]"}]
Family ID: 49236798