A method for using a gesture in connection with an interactive display system on which images are displayed and objects (76a, 76b) disposed on or proximate thereto are optically detectable, comprising the steps of: (a) forming an image with light reflected (80a, 80b, 80c) from an object (76a, 76b) that is disposed above a display surface of the interactive display system, the light (78a, 78b, 78c) that is thus reflected (80a, 80b, 80c) being detected after passing through the display surface; (b) processing the image to recognize a gesture that was made by the user using the object (76a, 76b), said gesture being recognized in response to detecting at least one connected component within the image, said connected component comprising adjacent pixels having an intensity above a predefined threshold; and (c) as a function of the gesture that was recognized, automatically producing a response that has previously been associated with the gesture, said response changing a state of the interactive display system; and wherein the object (76a, 76b) comprises at least a portion of an appendage of a user's body; wherein the step of detecting at least one connected component comprises the step of detecting a plurality of touch connected components (406; 414; 504; 604a; 604b; 604c;706; 806) and at least one hover connected component (408; 416; 502; 602a; 602b; 602c; 704; 804a; 804b) to generally determine an orientation of the appendage of the user, so as to enable the gesture to be correctly recognized, wherein a touch connected component corresponds to a contact of the display surface by the object, and a hover connected component corresponds to an object that is proximate the display surface, but not actually touching it, wherein different predefined threshold intensity levels are defined for hover objects and touch objects; the method further comprising the steps of: (a) analyzing temporal patterns of a plurality of sets of touch connected components as the touch connected components change overtime; (b) identifying appendages of each of a plurality of different users, where the appendages are hovering adjacent to the display surface; and (c) determining that groupings of multiple fingers in contact with the display surface belong to specific ones of the appendages, and thus, belong to specific users, based on an orientation of each appendage, and on the temporal patterns. (a) forming an image with light reflected (80a, 80b, 80c) from an object (76a, 76b) that is disposed above a display surface of the interactive display system, the light (78a, 78b, 78c) that is thus reflected (80a, 80b, 80c) being detected after passing through the display surface; (b) processing the image to recognize a gesture that was made by the user using the object (76a, 76b), said gesture being recognized in response to detecting at least one connected component within the image, said connected component comprising adjacent pixels having an intensity above a predefined threshold; and (c) as a function of the gesture that was recognized, automatically producing a response that has previously been associated with the gesture, said response changing a state of the interactive display system; and (a) analyzing temporal patterns of a plurality of sets of touch connected components as the touch connected components change overtime; (b) identifying appendages of each of a plurality of different users, where the appendages are hovering adjacent to the display surface; and (c) determining that groupings of multiple fingers in contact with the display surface belong to specific ones of the appendages, and thus, belong to specific users, based on an orientation of each appendage, and on the temporal patterns.