A method for providing a low-latency interaction in a virtual environment, the method comprising: receiving an initial image of a back of a user's hand and corresponding initial depth information that represent an initial actual position of the hand; using the initial depth information, projecting the initial image into the virtual environment to an initial virtual position, the initial virtual position corresponding to the initial actual position; providing a segmented version of the initial image to a display device for display in the virtual environment at the initial virtual position; receiving a subsequent image of the back of the user's hand and corresponding subsequent depth information that represent a subsequent actual position of the hand that is different from the initial actual position; using the subsequent depth information, projecting the subsequent image into the virtual environment to a subsequent virtual position, the subsequent virtual position corresponding to the subsequent actual position; providing a segmented version of the subsequent image to the display device for display in the virtual environment at the subsequent virtual position; and detecting a collision between a three-dimensional representation of the hand and a virtual object in the virtual environment or a physical object; wherein, regardless of the initial actual position and the subsequent actual position of the user's hand, the segmented version of the initial image and the segmented version of the subsequent image are provided to the display device for display at a virtual orientation in which the hand extends generally upwardly in the virtual environment. receiving an initial image of a back of a user's hand and corresponding initial depth information that represent an initial actual position of the hand; using the initial depth information, projecting the initial image into the virtual environment to an initial virtual position, the initial virtual position corresponding to the initial actual position; providing a segmented version of the initial image to a display device for display in the virtual environment at the initial virtual position; receiving a subsequent image of the back of the user's hand and corresponding subsequent depth information that represent a subsequent actual position of the hand that is different from the initial actual position; using the subsequent depth information, projecting the subsequent image into the virtual environment to a subsequent virtual position, the subsequent virtual position corresponding to the subsequent actual position; providing a segmented version of the subsequent image to the display device for display in the virtual environment at the subsequent virtual position; and detecting a collision between a three-dimensional representation of the hand and a virtual object in the virtual environment or a physical object; wherein, regardless of the initial actual position and the subsequent actual position of the user's hand, the segmented version of the initial image and the segmented version of the subsequent image are provided to the display device for display at a virtual orientation in which the hand extends generally upwardly in the virtual environment.