Fiducial rings in virtual reality

A virtual reality system enables a user to interact with virtual objects. The system includes a fiducial ring, an imaging device and a console. The fiducial ring includes a ring body that includes a plurality of fiducial markers that each correspond to a different location on the ring body. An imaging device is configured to capture one or more images of the fiducial ring. The console receives the images that include an image of one or more fiducial markers. Based on the received images of the fiducial markers, the console determines a location on the fiducial ring that corresponds to the imaged fiducial marker. The console determines a position of the fiducial ring based on the determined location of the fiducial marker on the fiducial ring. The console provides content to a head mounted display (HMD) based on the determined position of the fiducial ring.

BACKGROUND

The present disclosure generally relates to a system for haptic feedback to a user, and specifically to haptic devices that include fiducial markers to track the movement of a user's fingers in the virtual reality (VR) system.

Virtual reality (VR) is a simulated environment created by computer technology and presented to a user, such as through a VR system. Typically, a VR system includes a head-mounted display (HMD) that provides visual and audio information to the user. Conventional VR systems create virtual body parts (e.g., a virtual finger) in the simulated environment and use a tracking system to track a user movement in a physical space. The simulated environment presented to the user may be updated according to the user movement in the physical space.

SUMMARY

A locator assembly within a virtual reality environment enables a user to interact with virtual objects via individual fingers. The locator assembly includes one or more fiducial rings that can be worn on portion of the user's body. In some embodiments, a fiducial ring may be worn one, e.g., a finger, an arm, a leg, etc. Each fiducial ring includes a ring body that includes one or more fiducial markers. Each fiducial marker is positioned at a different location on the ring body. The fiducial markers may be configured as a unique combination on every fiducial ring.

An imaging device is configured to capture one or more images of each of the fiducial rings within the locator assembly. The imaging device may be an optical imaging device (e.g., visible light, infrared light, etc.), some other type of imaging device (e.g., radio frequency imaging, acoustic imaging (e.g., ultrasound), etc.), or some combination thereof. In one embodiment, the fiducial ring may include a sensor to sense a motion of the user's finger. Based on the sensed motion, the locator assembly may send a corresponding output to the console. The console may configure the imaging device to capture one or more images of the motion-sensed fiducial ring.

A console receives the images that include the fiducial markers on each fiducial ring. Based on the received images of the fiducial markers, the console determines a location of the fiducial ring that corresponds to the imaged fiducial marker. The console determines a position of the fiducial ring within the virtual reality environment, based on the determined location of the fiducial marker on the fiducial ring. Based on the determined position of the fiducial rings, the console provides content to a head-mounted display.

DETAILED DESCRIPTION

Configuration Overview

A haptic system includes one or more fiducial rings that each includes one or more fiducial markers to determine a position of a user's finger. A fiducial ring includes a set of fiducial markers that each correspond to a different location on the ring body. The fiducial ring is configured to fit on a portion of a user's finger, such as bottom of the finger, top of the finger or the bend of the finger. Alternatively, the ring can be configured to fit on any portion of a thumb of a user.

In one embodiment, a VR console may receive an image of the fiducial markers on each of the plurality of fiducial rings. Based on the received images, the VR console can determine a location on the fiducial ring that corresponds to an imaged fiducial marker. Based on the determined location of the fiducial marker, the VR console may determine a position of the fiducial ring and thus, the position of the user's finger in the virtual space.

In one embodiment, based on the determined position of the fiducial ring, a fast calibration information signal indicating a virtual touch event may be sent to the VR system. Responsive to the virtual touch event, the VR system provides haptic feedback to the user that includes perception of touch of a virtual object in the VR system.

System Overview

FIG. 1is a block diagram of a VR system environment100in which a VR console110operates. The system environment100shown byFIG. 1comprises a HMD105, an imaging device135, and a locator assembly140. WhileFIG. 1shows an example system100including one HMD105, one imaging device135, and one locator assembly140(e.g., a fiducial ring), in other embodiments any number of these components may be included in the system100. For example, there may be multiple VR headsets105each having an associated locator assembly140and being monitored by one or more imaging devices135, with each VR headset105, locator assembly140, and imaging devices135communicating with the VR console110. In alternative configurations, different and/or additional components may be included in the system environment100. Similarly, the functions can be distributed among the components in a different manner than is described here. For example, some or all of the functionality of the VR console110may be contained within the HMD105.

The HMD105is a head-mounted display that presents media to a user. Examples of media presented by the HMD105include one or more images, video, audio, or any combination thereof. In some embodiments, audio is presented via an external device (e.g., speakers and/or headphones) that receives audio information from the HMD105, the VR console110, or both, and presents audio data based on the audio information. In some embodiments, the HMD105may also act as an augmented reality (AR) HMD and/or mixed reality (MR) HMD. In these embodiments, the HMD105augments views of a physical, real-world environment with computer-generated elements (e.g., images, video, sound, etc.).

The HMD105includes an electronic display115, an optics block118, one or more locators120, one or more position sensors125, and an inertial measurement unit (IMU)130. The electronic display115displays images to the user in accordance with data received from the VR console110.

The optics block118magnifies received light from the electronic display115, corrects optical errors associated with the image light, and the corrected image light is presented to a user of the HMD105. An optical element may be an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, or any other suitable optical element that affects the image light emitted from the electronic display115. Moreover, the optics block118may include combinations of different optical elements. In some embodiments, one or more of the optical elements in the optics block118may have one or more coatings, such as anti-reflective coatings.

The locators120are objects located in specific positions on the HMD105relative to one another and relative to a specific reference point of the HMD105on the HMD105. A locator120may be a light emitting diode (LED), a corner cube reflector, a reflective marker, a type of light source that contrasts with an environment in which the HMD105operates, or some combination thereof. In embodiments where the locators120are active (i.e., an LED or other type of light emitting device), the locators120may emit light in the visible band (˜380 nm to 750 nm), in the infrared (IR) band (˜750 nm to 1 mm), in the ultraviolet band (10 nm to 380 nm), some other portion of the electromagnetic spectrum, or some combination thereof.

In some embodiments, the locators120are located beneath an outer surface of the HMD105, which is transparent to the wavelengths of light emitted or reflected by the locators120or is thin enough not to substantially attenuate the wavelengths of light emitted or reflected by the locators120. Additionally, in some embodiments, the outer surface or other portions of the HMD105are opaque in the visible band of wavelengths of light. Thus, the locators120may emit light in the IR band under an outer surface that is transparent in the IR band but opaque in the visible band.

The IMU130is an electronic device that generates fast calibration data (herein also referred to as “fast calibration information”) of the HMD105based on measurement signals received from one or more of the position sensors125. A position sensor125generates one or more measurement signals in response to motion of the HMD105. Examples of position sensors125include: one or more accelerometers, one or more gyroscopes, one or more magnetometers, another suitable type of sensor that detects motion, a type of sensor used for error correction of the IMU130, or some combination thereof. The position sensors125may be located external to the IMU130, internal to the IMU130, or some combination thereof.

Based on the one or more measurement signals from one or more position sensors125, the IMU130generates fast calibration data of the HMD105indicating an estimated position of the HMD105relative to an initial position of the HMD105. For example, the position sensors125include multiple accelerometers to measure translational motion (forward/back, up/down, left/right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, roll) of the HMD105. In some embodiments, the IMU130rapidly samples the measurement signals and calculates the estimated position of the HMD105from the sampled data. For example, the IMU130integrates the measurement signals received from the accelerometers over time to estimate a velocity vector and integrates the velocity vector over time to determine an estimated position of a reference point of the HMD105on the HMD105. Alternatively, the IMU130provides the sampled measurement signals to the VR console110, which determines the fast calibration data of the HMD105. The reference point of the HMD105is a point that may be used to describe the position of the HMD105. While the reference point of the HMD105may generally be defined as a point in space; however, in practice the reference point of the HMD105is defined as a point within the HMD105(e.g., a center of the IMU130).

The IMU130receives one or more calibration parameters of the HMD105from the VR console110. As further discussed below, the one or more calibration parameters of the HMD105are used to maintain tracking of the HMD105. Based on a received calibration parameter of the HMD105, the IMU130may adjust one or more IMU parameters (e.g., sample rate). In some embodiments, certain calibration parameters of the HMD105cause the IMU130to update an initial position of the reference point of the HMD105so it corresponds to a next calibrated position of the reference point of the HMD105. Updating the initial position of the reference point of the HMD105as the next calibrated position of the reference point of the HMD105helps reduce accumulated error associated with the determined estimated position. The accumulated error, also referred to as drift error, causes the estimated position of the reference point of the HMD105to “drift” away from the actual position of the reference point of the HMD105over time.

The locator assembly140is an apparatus for tracking a position and/or movement of a portion of user's body (e.g., arm, leg, finger, etc. In some embodiments, the locator assembly140may, e.g., be used to track a position and/or movement of a user's finger. The locator assembly140includes one or more fiducial rings170a. . .170n(generally termed as fiducial ring170). A fiducial ring170includes a ring body that includes one or more fiducial markers. In some embodiments, the fiducial rings170are employed to determine a physical position or movement of the locator assembly140. In another embodiment, the locator assembly140may receive, from the VR console110, a haptic feedback signal corresponding to haptic feedback to provide to the user. The fiducial ring170is configured to couple to a portion of a user's body. For example, a fiducial ring170may fit around an arm of a user, a finger of a user, etc.

The fiducial ring170includes fiducial markers located in specific positions on the locator assembly140relative to one another and relative to a specific reference point of the locator assembly140on the locator assembly140. A fiducial marker is substantially similar to a locator120except that a fiducial marker is part of the locator assembly140. In some embodiments, the fiducial marker may emit light similar to the locator120, however, in alternate embodiments, the fiducial marker may also or alternatively emit other signals. For example, the fiducial marker may emit acoustic signals, radio frequency signals, etc. Fiducial markers also may be passive as discussed below with regard toFIG. 2. Additionally, in some embodiments, the outer surface or other portions of the locator assembly140are opaque in the visible band of wavelengths of light. Thus, the fiducial marker may emit light in the IR band under an outer surface that is transparent in the IR band but opaque in the visible band.

In one embodiment, the haptic feedback signal indicates a position or a portion of the locator assembly140to be actuated, and an amount of actuation of the position or the portion of the locator assembly140for providing haptic feedback. In this embodiment, the amount of actuation is determined by, e.g., the VR console110, according to a virtual position of the locator assembly140corresponding to a physical position of the locator assembly140and a virtual position of a virtual object in a virtual space. In some embodiments, the VR console110may receive sensor information that includes a force value indicating an amount of pressure on the locator assembly140. In addition to the position of the locator assembly140, the VR console110may use the applied pressure/force information from the locator assembly140to determine the amount of actuation. The locator assembly140provides tactile perception of a user touching the virtual object, by actuating a haptic apparatus, according to the amount of actuation indicated by the haptic feedback signal.

The locator assembly140may optionally include a haptic apparatus that provides to the user the haptic feedback of touching a virtual object in a virtual space, according to the haptic feedback signal. For example, the locator assembly140determines a location and/or movement of a user's finger, corresponding to a virtual touch event in the virtual space. The haptic apparatus may restrict the movement of a user's finger as a perception of virtual touch in the virtual space. For example, if a user finger is in contact with a virtual object (e.g., a virtual wall) in a virtual space, the locator assembly140receives a haptic feedback signal that prevents a physical movement of the user finger to move in a direction through the virtual object in the virtual space. Accordingly, the user can receive a perception of contacting the virtual object.

In one embodiment, the locator assembly140includes a fiducial ring170. The fiducial ring170may be connected to a pressure sensor. The pressure sensor generates a signal as a function of the imposed pressure. For example, a movement of a user's finger or a bending of a user's finger may impose pressure on the pressure sensor connected to the fiducial ring170worn by the user. The pressure sensor may detect this imposed pressure and generate an electrical signal that indicates the amount of imposed pressure, generated as a result of the change in movement of the user's finger. Different embodiments of the locator assembly140and its operation are described in detail below with respect toFIGS. 2-4. In one embodiment, the locator assembly140is a fiducial ring through which the VR console110can detect and/or track a user finger/hand movement as described in detail with respect toFIGS. 2 through 4.

The imaging device135generates slow calibration data in accordance with calibration parameters received from the VR console110. Slow calibration data (herein also referred to as “slow calibration information”) of the HMD105includes one or more images showing observed positions of the locators120associated with the HMD105that are detectable by the imaging device135. Similarly, slow calibration data of the locator assembly140includes one or more images showing observed positions of the fiducial rings170associated with the locator assembly140that are detectable by the imaging device135. In one aspect, the slow calibration data includes one or more images of both the HMD105and locator assembly140. The imaging device135may include one or more cameras, one or more video cameras, and any other device capable of capturing images including one or more of the locators120and170, a radio frequency imaging device, an acoustic imaging (e.g., ultrasound, etc.), or some combination thereof. In embodiments, where the imaging device135captures optical images, the imaging device135may capture images in the visible band, the infrared band, the ultraviolet band, some other portion of the electromagnetic spectrum, or some combination thereof. Additionally, the imaging device135may include one or more filters (e.g., used to increase signal to noise ratio). The imaging device135is configured to detect light emitted or reflected from locators120and170in a field of view of the imaging device135. In embodiments where the locators120and170include passive elements (e.g., a retroreflector), the imaging device135may include a source that illuminates some or all of the locators120and170with an illumination signal (e.g., light, radio frequency signal, ultrasound, etc.), which reflect the illumination signal towards the source in the imaging device135. Slow calibration data is communicated from the imaging device135to the VR console110, and the imaging device135receives one or more calibration parameters from the VR console110to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.).

The VR console110provides media to the HMD105for presentation to the user in accordance with information received from one or more of: the imaging device135, the HMD105, and the locator assembly140. The VR console110may also instruct the locator assembly140to provide haptic feedback including a perception of a user contacting a virtual object. In the example shown inFIG. 1, the VR console110includes an application store145, a tracking module150, and a VR engine155. Some embodiments of the VR console110have different modules than those described in conjunction withFIG. 1. Similarly, the functions further described below may be distributed among components of the VR console110in a different manner than is described here.

The application store145stores one or more applications for execution by the VR console110. An application is a group of instructions, that when executed by a processor, generates content for presentation to the user. Content generated by an application may be in response to inputs received from the user via movement of the HMD105or the VR locator assembly140. Examples of applications include: gaming applications, conferencing applications, video playback application, or other suitable applications.

The tracking module150calibrates the VR system100using one or more calibration parameters and may adjust one or more calibration parameters to reduce error in determination of the position of the HMD105and/or the locator assembly140.

The tracking module150tracks movements of the HMD105and/or the locator assembly140using slow calibration information of the HMD105or the fiducial rings170of the locator assembly140from the imaging device135. The tracking module150determines positions of a reference point of the HMD105or the locator assembly140using observed locators from the slow calibration information and a model of the HMD105. The tracking module150also determines positions of a reference point of the HMD105using position information from the fast calibration information of the HMD105or the locator assembly140. Additionally, in some embodiments, the tracking module150may use portions of the fast calibration information, the slow calibration information, or some combination thereof of the HMD105or the locator assembly140, to predict a future location of the headset105or the locator assembly140. The tracking module150provides the estimated or predicted future position of the HMD105or the locator assembly140to the VR engine155.

The VR engine155executes applications within the system environment100and receives position information, acceleration information, velocity information, predicted future positions, or some combination thereof of the HMD105from the tracking module150. Based on the received information, the VR engine155determines content to provide to the HMD105for presentation to the user. For example, if the received information indicates that the user has looked to the left, the VR engine155generates content for the HMD105that mirrors the user's movement in a virtual environment. Additionally, the VR engine155performs an action within an application executing on the VR console110in response to detecting a motion of the locator assembly140and provides feedback to the user that the action was performed. In one example, the VR engine155instructs the HMD105to provide visual or audible feedback to the user. In another example, the VR engine155instructs the locator assembly140to provide haptic feedback including a perception of a user touching a virtual object.

In addition, the VR engine155receives position information, acceleration information, velocity information, predicted future positions or some combination thereof of the locator assembly140from the tracking module150and determines whether a virtual touch event occurred. A virtual touch event herein refers to an event of a user contacting a virtual object in a virtual space. For example, an image of a virtual object is presented to the user on the HMD105. Meanwhile, the VR engine155collectively analyzes positions of multiple sensors of the locator assembly140through the tracking module150, and generates a three dimensional mapping of the locator assembly140describing the position and the shape of the user's hand or fingers. The three dimensional mapping of the locator assembly140describes coordinates of various parts of the locator assembly140in a virtual space corresponding to physical positions of the user's hands along or fingers in reality. Responsive to the user performing an action to grab the virtual object or the user being contacted by the virtual object, the VR engine155determines that the virtual touch event occurred.

In one embodiment, the VR engine155compares coordinates of a virtual object and a coordinate of the locator assembly140in a virtual space to determine whether a virtual touch event occurred. The VR engine155obtains a coordinate of the virtual object in a virtual space, in accordance with an image presented via the HMD105. Additionally, the VR engine155obtains a coordinate of the locator assembly140(e.g., one or more fiducial rings) corresponding to a physical position of the user's fingers or hand or the three dimensional mapping of the user's fingers or hand, from the tracking module150. Then, the VR engine155compares the coordinate of the virtual object in the virtual space and the coordinate of the user's fingers or hand in the virtual space. For example, if two coordinates of the virtual object and the user's fingers or hand overlap or are approximate to each other within a predetermined distance for a predetermined amount of time (e.g., 1 second), the VR console110determines the virtual touch event occurred.

In one embodiment, the VR engine155performs rendering to adjust the position of a virtual object in response to the determined position or movement of the locator assembly140. Additionally, the VR engine155may generate a haptic feedback signal to simulate a user touching a virtual object (i.e., provide a perception to a user that the user is touching an object). Responsive to detecting the virtual touch event, the VR engine155determines permissible movements and impermissible movements of the user. For example, if a user's finger is in touch with a virtual object, the VR engine155determines that the user cannot bend the finger in a physical direction corresponding to a virtual direction through the virtual object, because the virtual mirror of the finger in the virtual space (i.e., mapping of the finger in the virtual space) is in contact with the virtual object in the virtual direction. For an additional example, the VR engine155determines that the user can lift the finger in another physical direction corresponding to another virtual direction away from the virtual object, because the virtual mirror of the finger is not in contact with any object in the virtual space in said another virtual direction. According to a list of permissible movements and impermissible movements, the VR engine155determines a portion (e.g., a coordinate or a position) of the locator assembly140to be actuated (e.g., activate a vibration force) and amount of actuation (e.g., a degree of vibration). The VR engine155provides the haptic feedback signal indicating the portion of the locator assembly140and the amount of actuation to the locator assembly140for executing the haptic feedback.

Example Locator Assembly

FIG. 2is a locator assembly200, in accordance with an embodiment. In some embodiments, the locator assembly200may be, e.g., the locator assembly140ofFIG. 1. The locator assembly200includes one or more fiducial rings220. In some embodiments, a fiducial ring220may be configured to fit on any portion of a phalange of a user. In one embodiment, a fiducial ring is worn on each finger of the hand, including the thumb. In another embodiment, multiple fiducial rings are worn on a single finger on different portions of the user's finger, for example, one at the bottom of the finger, one at the bend of the finger and one on the top portion of the finger.

The locator assembly200illustrated inFIG. 2is merely an example, and in different embodiments, a fiducial ring220includes fewer, more or different components than shown inFIG. 2. Moreover, in alternate embodiments, the fiducial ring220may be formed to fit other parts of a user's body (e.g., arm, leg, etc.).

Each fiducial ring220includes a ring body230and one or more fiducial markers240. The ring body230may be of an adjustable circular structure. The adjustable feature allows the ring body230across a range of finger sizes and may be adjusted to fit along a different portion of a finger such as a joint, a bottom portion of a figure or a top portion of a figure. The ring body230may be made of, e.g., elastomers (highly stretchable), springs in general, shape memory alloy springs (stretch and then return to their original shape when heated to body temperature).

A fiducial marker240is an object used as a point of reference or measure. Each fiducial marker240is a stationary (i.e., is located at a fixed position on the ring body220) locatable point on the ring body220and corresponds to a specific location on the ring body220, relative to a location of a second fiducial marker on the ring body220. The fiducial markers240are configured as a unique combination on the fiducial ring220for ease of recognizing a fiducial ring220via the imaging system. The configuration and operation of the fiducial markers240are similar to the fiducial marker described as a part of the locator assembly140ofFIG. 1. Therefore, the detailed description thereof is omitted herein for the sake of brevity.

A fiducial marker240may be an active device such as a light emitting diode (LED), a device emitting light in the visible band, an infrared band, an ultraviolet band or any other such active device, a device that emits acoustic signals, a device that emits radio frequency signals, or some combination thereof. Alternatively, a fiducial marker240may be a passive device such as a corner cube reflector, a reflective marker, a retro-reflective device, a passive reflector for acoustic signals (e.g., ultrasound), a passive reflector for radio frequency signals, or any other such passive device.

The user's finger position can be determined according to the slow calibration of the fiducial markers240from the imaging device135. In response to determination of movement of the user's finger, and the position of the user's finger, the locator assembly200receives haptic feedback including a perception of a user touching a virtual object. The haptic feedback can be provided to the user by the ring body230of the fiducial ring220, or a visual feedback can be provided by changing a light pattern on the fiducial markers225.

In one embodiment, the fiducial ring220is actuated to receive the haptic feedback in the visual or sensory form. For example, a visual haptic feedback may include displaying a specific light pattern on the fiducial markers240. A sensory haptic feedback may include vibrating the ring body or inflating/deflating a portion of the ring body220. The amount of inflation/deflation of the ring body may be adjusted as indicated by the received haptic feedback signal.

FIG. 3is a fiducial ring300, in accordance with an embodiment. The fiducial ring300is substantially the same as the fiducial ring220, except that it further includes a sensor320.

The sensor320detects events or changes in its surrounding environment such as a change in pressure, or a change in thermal energy and other such changes, and provides a corresponding output. In one embodiment, a sensor320is coupled to a portion of the ring body310(e.g., a portion corresponding to a fingertip, or a joint of the finger). The fiducial ring340is coupled to a corresponding finger portion330(e.g., a portion corresponding to a joint of the finger). In one embodiment, one or more of these components are placed beneath an outer surface of the ring body310, thus are not visible from the outside. Additionally or alternatively, some of these components are placed on an outer surface of the ring body310, and are visually detectable.

The sensor320senses a motion event on the ring body310or the portion of the user's finger and generates one or more measurement signals in response to motion of the fiducial ring340. In one embodiment, the configuration and operation of the sensor320is similar to the position sensors125of the HMD105ofFIG. 1. Therefore, the detailed description thereof is omitted herein for the sake of brevity.

In another embodiment, the sensor320is a pressure sensor. A pressure sensor generates one or more measurement signals in response to a change in pressure on the fiducial ring340, generated based on the expansion or compression of skin surrounding the fiducial ring340. The expansion or compression of skin may indicate fine motor movements of the user's finger, such as a bend in the finger.

A measurement signal may be generated in response to a change in pressure due to a bend in the finger coupled to the fiducial ring340. Examples of pressure sensors include: capacitive pressure sensors, electronic pressure sensors, strain gauges, electromagnetic pressure sensors, thermal sensor, an optical sensor, oscillator-based frequency sensors, resistive pressure sensors, or some combination thereof. The pressure sensor may be located external to the ring body310, or internal to the ring body310, or coupled externally to the ring body310using a coupling mechanism, or some combination thereof.

Based on the one or more measurement signals from one or more pressure sensors, the locator assembly300generates applied pressure/force information indicating an estimated pressure/force value on the fiducial ring340. For example, the pressure sensor may include an optical sensor such as optical fiber and measure a physical change of an optical fiber to detect the amount of force caused due to applied pressure on the fiducial ring340. In some embodiments, locator assembly300generates an applied pressure/force information signal including the estimated pressure/force value. Alternatively, the applied pressure/force information signal may include measurement signals indicating a change in pressure on the fiducial ring340. The locator assembly300sends the applied pressure/force information signal to the VR console110.

FIG. 4is a flow chart illustrating a process of providing haptic feedback responsive to a virtual touch event in a virtual space, in accordance with an embodiment. In one embodiment, the process ofFIG. 4is performed by a console (e.g., VR console110ofFIG. 1). Other entities may perform some or all of the steps of the process in other embodiments. Likewise, embodiments may include different and/or additional steps, or perform the steps in different orders.

The console determines410a virtual touch event occurred based on a user movement on the haptic device such as a fiducial ring170. In one embodiment, the console receives fast calibration data from the fiducial ring and/or slow calibration data from the imaging device, and then determines a finger/hand movement. In another embodiment, the console receives applied pressure/force information from the locator assembly140to determine a finger bend or a stretch.

In one approach, the console obtains 3-D map of the user's fingers describing coordinates of various parts of the fiducial ring in a virtual space corresponding to physical positions of the parts of the fiducial ring in reality based on the fast calibration data and/or the slow calibration data. The console compares the coordinate of the virtual object in the virtual space and the coordinate of the fiducial ring in the virtual space to determine whether a virtual touch event occurred.

In another approach, the console obtains pressure/force information including force values on the fiducial ring on various parts of the user's fingers in a virtual space, the user's fingers may be in contact with a virtual object. Based on the amount of force, and the position of the various parts of the user's fingers in the virtual space, the console determines a fine motor movement of the finger, such as a bend in the finger that is in contact with the virtual object, thus determining a virtual touch event occurred.

Responsive to determining the virtual touch event occurred, the console receives an image of the fiducial markers on the fiducial ring. Based on the received image and the position of the fiducial markers, the console determines a location of the fiducial ring in the virtual space.

Based on the determined location, the console determines420a position of the user's fingers in the virtual space. For example, responsive to determining that the user's finger moved in the virtual space towards a virtual wall, the console determines the location of the virtual wall and moves the user's finger to a position closer to the virtual wall. For example, responsive to the user pressing a virtual object (e.g., a ball) in a virtual space with a bottom surface of an index finger, the console determines such virtual touch event occurred, and identifies the bottom surface of the index finger is in contact with the virtual object. The console further determines the value of the force of the user's index finger when in contact with the virtual object.

The console generates430a haptic feedback signal describing details of the haptic feedback to be provided to the user, according to the determined position of each fiducial ring. In some embodiments, the amount of force on the virtual object is taken into consideration by the console to determine an amount of actuation to include within the haptic feedback signal. In one embodiment, the haptic feedback signal indicates which fiducial ring should be actuated (e.g., a coordinate), and an amount of actuation (e.g., amount of vibration).

The console transmits the haptic feedback signal440to the appropriate fiducial ring170within the locator assembly140for providing the haptic feedback. The fiducial ring receives the haptic feedback signal, and then provides haptic feedback to the user according to the haptic feedback signal.

Additional Configuration Information