Patent ID: 12229341

DETAILED DESCRIPTION

Electronic devices that are configured to be mounted on the body of a user may be used to gather user input and to provide a user with output. For example, electronic devices that are configured to be worn on one or more of a user's fingers, which are sometimes referred to as finger devices or finger-mounted devices, may be used to gather user input and to supply output. A finger device may, as an example, include an inertial measurement unit with an accelerometer for gathering information on finger motions such as finger taps or free-space finger gestures, may include force sensors for gathering information on normal and shear forces in the finger device and the user's finger, and may include other sensors for gathering information on the interactions between the finger device (and the user's finger on which the device is mounted) and the surrounding environment. The finger device may include a haptic output device to provide the user's finger with haptic output and may include other output components.

One or more finger devices may gather user input from a user. The user may use finger devices in operating a display and/or a virtual reality or mixed reality device (e.g., head-mounted equipment such as glasses, goggles, a helmet, or other device with a display). During operation, the finger devices may gather user input such as information on interactions between the finger device(s) and the surrounding environment (e.g., interactions between a user's fingers and the environment, including finger motions and other interactions associated with virtual content displayed for a user). The user input may be used in controlling visual output on the display. Corresponding haptic output may be provided to the user's fingers using the finger devices. Haptic output may be used, for example, to provide the fingers of a user with a desired texture sensation as a user is touching a real object or as a user is touching a virtual object. Haptic output can also be used to create detents and other haptic effects.

Finger devices can be worn on any or all of a user's fingers (e.g., the index finger, the index finger and thumb, three of a user's fingers on one of the user's hands, some or all fingers on both hands, etc.). To enhance the sensitivity of a user's touch as the user interacts with surrounding objects, finger devices may have inverted U shapes or other configurations that allow the finger devices to be worn over the top and sides of a user's finger tips while leaving the user's finger pads exposed. This allows a user to touch objects with the finger pad portions of the user's fingers during use. If desired, finger devices may be worn over knuckles on a user's finger, between knuckles, and/or on other portions of a user's finger. The use of finger devices on a user's finger tips is sometimes described herein as an example.

Users can use the finger devices to interact with any suitable electronic equipment. For example, a user may use one or more finger devices to supply input to a desktop computer, tablet computer, cellular telephone, watch, ear buds, or other accessory, to interact with a virtual reality or mixed reality system (e.g., a head-mounted device with a display), or to interact with other electronic equipment.

FIG.1is a schematic diagram of an illustrative system of the type that may include one or more finger devices. As shown inFIG.1, system8may include electronic device(s) such as finger device(s)10and other electronic device(s)24. Each finger device10may be worn on a finger of a user's hand. Additional electronic devices in system8such as devices24may include devices such as a laptop computer, a computer monitor containing an embedded computer, a tablet computer, a desktop computer (e.g., a display on a stand with an integrated computer processor and other computer circuitry), a cellular telephone, a media player, or other handheld or portable electronic device, a smaller device such as a wristwatch device, a pendant device, a headphone or earpiece device, a head-mounted device such as glasses, goggles, a helmet, or other equipment worn on a user's head, or other wearable or miniature device, a television, a computer display that does not contain an embedded computer, a gaming device, a remote control, a navigation device, an embedded system such as a system in which equipment is mounted in a kiosk, in an automobile, airplane, or other vehicle, a removable external case for electronic equipment, a strap, a wrist band or head band, a removable cover for a device, a case or bag that has straps or that has other structures to receive and carry electronic equipment and other items, a necklace or arm band, a wallet, sleeve, pocket, or other structure into which electronic equipment or other items may be inserted, part of a chair, sofa, or other seating (e.g., cushions or other seating structures), part of an item of clothing or other wearable item (e.g., a hat, belt, wrist band, headband, sock, glove, shirt, pants, etc.), or equipment that implements the functionality of two or more of these devices.

With one illustrative configuration, which may sometimes be described herein as an example, device10is a finger-mounted device having a finger-mounted housing with a U-shaped body that grasps a user's finger or a finger-mounted housing with other shapes configured to rest against a user's finger and device(s)24is a cellular telephone, tablet computer, laptop computer, wristwatch device, head-mounted device, a device with a speaker, or other electronic device (e.g., a device with a display, audio components, and/or other output components). A finger device with a U-shaped housing may have opposing left and right sides that are configured to receive a user's finger and a top housing portion that couples the left and right sides and that overlaps the user's fingernail.

Devices10and24may include control circuitry12and26. Control circuitry12and26may include storage and processing circuitry for supporting the operation of system8. The storage and processing circuitry may include storage such as nonvolatile memory (e.g., flash memory or other electrically-programmable-read-only memory configured to form a solid state drive), volatile memory (e.g., static or dynamic random-access-memory), etc. Processing circuitry in control circuitry12and26may be used to gather input from sensors and other input devices and may be used to control output devices. The processing circuitry may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors and other wireless communications circuits, power management units, audio chips, application specific integrated circuits, etc.

To support communications between devices10and24and/or to support communications between equipment in system8and external electronic equipment, control circuitry12may communicate using communications circuitry14and/or control circuitry26may communicate using communications circuitry28. Circuitry14and/or28may include antennas, radio-frequency transceiver circuitry, and other wireless communications circuitry and/or wired communications circuitry. Circuitry14and/or28, which may sometimes be referred to as control circuitry and/or control and communications circuitry, may, for example, support bidirectional wireless communications between devices10and24over wireless link38(e.g., a wireless local area network link, a near-field communications link, or other suitable wired or wireless communications link (e.g., a Bluetooth® link, a WiFi® link, a 60 GHz link or other millimeter wave link, etc.). Devices10and24may also include power circuits for transmitting and/or receiving wired and/or wireless power and may include batteries. In configurations in which wireless power transfer is supported between devices10and24, in-band wireless communications may be supported using inductive power transfer coils (as an example).

Devices10and24may include input-output devices such as devices16and30. Input-output devices16and/or30may be used in gathering user input, in gathering information on the environment surrounding the user, and/or in providing a user with output. Devices16may include sensors18and devices24may include sensors32. Sensors18and/or32may include force sensors (e.g., strain gauges, capacitive force sensors, resistive force sensors, etc.), audio sensors such as microphones, touch and/or proximity sensors such as capacitive sensors, optical sensors such as optical sensors that emit and detect light, ultrasonic sensors (e.g., ultrasonic sensors for tracking device orientation and location and/or for detecting user input such as finger input), and/or other touch sensors and/or proximity sensors, monochromatic and color ambient light sensors, image sensors, sensors for detecting position, orientation, and/or motion (e.g., accelerometers, magnetic sensors such as compass sensors, gyroscopes, and/or inertial measurement units that contain some or all of these sensors), muscle activity sensors (EMG) for detecting finger actions, radio-frequency sensors, depth sensors (e.g., structured light sensors and/or depth sensors based on stereo imaging devices), optical sensors such as self-mixing sensors and light detection and ranging (lidar) sensors that gather time-of-flight measurements, optical sensors such as visual odometry sensors that gather position and/or orientation information using images gathered with digital image sensors in cameras, gaze tracking sensors, visible light and/or infrared cameras having digital image sensors, humidity sensors, moisture sensors, and/or other sensors. In some arrangements, devices10and/or24may use sensors18and/or32and/or other input-output devices16and/or30to gather user input (e.g., buttons may be used to gather button press input, touch sensors overlapping displays can be used for gathering user touch screen input, touch pads may be used in gathering touch input, microphones may be used for gathering audio input, accelerometers may be used in monitoring when a finger contacts an input surface and may therefore be used to gather finger press input, etc.). If desired, device10and/or device24may include rotating buttons (e.g., a crown mechanism on a watch or finger device or other suitable rotary button that rotates and that optionally can be depressed to select items of interest). Alphanumeric keys and/or other buttons may be included in devices16and/or30. In some configurations, sensors18may include joysticks, roller balls, optical sensors (e.g., lasers that emit light and image sensors that track motion by monitoring and analyzing changings in the speckle patterns and other information associated with surfaces illuminated with the emitted light as device10is moved relative to those surfaces), fingerprint sensors, and/or other sensing circuitry. Radio-frequency tracking devices may be included in sensors18to detect location, orientation, and/or range. Beacons (e.g., radio-frequency beacons) may be used to emit radio-frequency signals at different locations in a user's environment (e.g., at one or more registered locations in a user's home or office). Radio-frequency beacon signals can be analyzed by devices10and/or24to help determine the location and position of devices10and/or24relative to the beacons. If desired, devices10and/or24may include beacons. Frequency strength (received signal strength information), beacon orientation, time-of-flight information, and/or other radio-frequency information may be used in determining orientation and position information. At some frequencies (e.g., lower frequencies such as frequencies below 10 GHz), signal strength information may be used, whereas at other frequencies (e.g., higher frequencies such as frequencies above 10 GHz), indoor radar schemes may be used). If desired, light-based beacons, ultrasonic beacons, and/or other beacon devices may be used in system8in addition to or instead of using radio-frequency beacons and/or radio-frequency radar technology.

Devices16and/or30may include haptic output devices20and/or34. Haptic output devices20and/or34can produce motion that is sensed by the user (e.g., through the user's fingertips). Haptic output devices20and/or34may include actuators such as electromagnetic actuators, motors, piezoelectric actuators, electroactive polymer actuators, vibrators, linear actuators (e.g., linear resonant actuators), rotational actuators, actuators that bend bendable members, actuator devices that create and/or control repulsive and/or attractive forces between devices10and/or24(e.g., components for creating electrostatic repulsion and/or attraction such as electrodes, components for producing ultrasonic output such as ultrasonic transducers, components for producing magnetic interactions such as electromagnets for producing direct-current and/or alternating-current magnetic fields, permanent magnets, magnetic materials such as iron or ferrite, and/or other circuitry for producing repulsive and/or attractive forces between devices10and/or24). In some situations, actuators for creating forces in device10may be used in squeezing a user's finger and/or otherwise directly interacting with a user's finger pulp. In other situations, these components may be used to interact with each other (e.g., by creating a dynamically adjustable electromagnetic repulsion and/or attraction force between a pair of devices10and/or between device(s)10and device(s)24using electromagnets).

If desired, input-output devices16and/or30may include other devices22and/or36such as displays (e.g., in device24to display images for a user), status indicator lights (e.g., a light-emitting diode in device10and/or24that serves as a power indicator, and other light-based output devices), speakers and other audio output devices, electromagnets, permanent magnets, structures formed from magnetic material (e.g., iron bars or other ferromagnetic members that are attracted to magnets such as electromagnets and/or permanent magnets), batteries, etc. Devices10and/or24may also include power transmitting and/or receiving circuits configured to transmit and/or receive wired and/or wireless power signals.

FIG.2is a top view of a user's finger (finger40) and an illustrative finger-mounted device10. As shown inFIG.2, device10may be formed from a finger-mounted unit that is mounted on or near the tip of finger40(e.g., partly or completely overlapping fingernail42). If desired, device10may be worn elsewhere on a user's fingers such as over a knuckle, between knuckles, etc. Configurations in which a device such as device10is worn between fingers40may also be used.

A user may wear one or more of devices10simultaneously. For example, a user may wear a single one of devices10on the user's ring finger or index finger. As another example, a user may wear a first device10on the user's thumb, a second device10on the user's index finger, and an optional third device10on the user's middle finger. Arrangements in which devices10are worn on other fingers and/or all fingers of one or both hands of a user may also be used.

Control circuitry12(and, if desired, communications circuitry14and/or input-output devices16) may be contained entirely within device10(e.g., in a housing for a fingertip-mounted unit) and/or may include circuitry that is coupled to a fingertip structure (e.g., by wires from an associated wrist band, glove, fingerless glove, etc.). Configurations in which devices10have bodies that are mounted on individual user fingertips are sometimes described herein as an example.

FIG.3is a cross-sectional side view of an illustrative finger device (finger-mounted device)10showing illustrative mounting locations46for electrical components (e.g., control circuitry12, communications circuitry14, and/or input-output devices16such as sensors18, haptic output devices20, and/or other devices22) within and/or on the surface(s) of finger device housing44. These components may, if desired, be incorporated into other portions of housing44.

As shown inFIG.3, housing44may have a U shape (e.g., housing44may be a U-shaped housing structure that faces downwardly and covers the upper surface of the tip of user finger40and fingernail42). During operation, a user may press against structures such as structure50. As the bottom of finger40(e.g., finger pulp40P) presses against surface48of structure50, the user's finger may compress and force portions of the finger outwardly against the sidewall portions of housing44(e.g., for sensing by sensors18mounted to the side portions of housing44). Lateral movement of finger40in the X-Y plane may also be sensed using sensors18on the sidewalls of housing44or other portions of housing44(e.g., because lateral movement will tend to press portions of finger40against some sensors18more than others and/or will create shear forces that are measured by force sensors that are configured to sense shear forces). Sensors18may, for example, include force sensors that measure how forcefully finger40is pressed against the sidewalls or other portions of housing44, and/or sensors18may include displacement sensors (e.g., optical distance sensors, ultrasonic distance sensors, other distance and/or displacement sensors) that measure changes in the distance D between finger40and sensors18.

Sensors18in device10may include ultrasonic sensors, optical sensors, inertial measurement units, strain gauges and other force sensors, capacitive sensors, radio-frequency sensors, and/or other sensors that may be used in gathering sensor measurements indicative of the activities of finger40and/or the activities of adjacent fingers. If desired, these sensors may also be used in mapping the contours of three-dimensional objects (e.g., by time-of-flight measurements and/or other measurements). For example, an ultrasonic sensor such as a two-dimensional image sensor or an ultrasonic sensor with a single ultrasonic transducer element may emit free-space ultrasonic sound signals that are received and processed after reflecting off of external objects. This allows a three-dimensional ultrasonic map to be generated indicating the shapes and locations of the external objects.

In some configurations, finger activity information (position, movement, orientation, etc.) may be gathered using sensors that are mounted in external electronic equipment (e.g., in a computer or other desktop device, in a head-mounted device or other wearable device, and/or in other electronic device24that is separate from device10). For example, optical sensors such as images sensors that are separate from devices10may be used in monitoring devices10to determine their position, movement, and/or orientation. If desired, devices10may include passive and/or active optical registration features to assist an image sensor in device24in tracking the position, orientation, and/or motion of device10. For example, devices10may include light-emitting devices such as light-emitting diodes, lasers (e.g., laser diodes, vertical cavity surface-emitting lasers, etc.), or other light sources and may operate at visible wavelengths, ultraviolet wavelengths, and/or infrared wavelengths. The light-emitting devices may be arranged in an asymmetric pattern on housing44and may emit light that is detected by an image sensor, depth sensor, and/or other light-based tracking sensor circuitry in device24(e.g., a head-mounted device, desktop computer, stand-alone camera-based monitoring systems, and/or other electrical equipment with an image sensor or other tracking sensor circuitry). By processing the received patterned of emitted light, device24can determine the position, orientation, and/or motion of device10. If desired, the light-emitting devices can be removable and/or customizable (e.g., a user can customize the location and type of light-emitting devices).

Tracking can also be performed that involves extrapolating from a known body part orientation (e.g., a finger orientation) to produce orientation information on other body parts (e.g., wrist and/or arm orientation estimated using inverse kinematics). Visual odometry sensors may, if desired, be included in devices10. These sensors may include image sensors that gather frames of image data of the surroundings of devices10and may be used in measuring position, orientation, and/or motion from the frame of image data. Lidar, ultrasonic sensors oriented in multiple directions, radio-frequency tracking sensors, and/or other finger device tracking arrangements may be used, if desired. In some arrangements, user input for controlling system8can include both user finger input and other user input (e.g., user eye gaze input, user voice input, etc.). For example, gaze tracking information such as a user's point-of-gaze measured with a gaze tracker can be fused with finger input when controlling device10and/or devices24in system8. A user may, for example, gaze at an object of interest while device10using one or more of sensors18(e.g., an accelerometer, force sensor, touch sensor, etc.) to gather information such as tap input (movement of device10resulting in measurable forces and/or accelerometer output when device10strikes a table top or other external surface), double-tap input, force input, multi-finger gestures (taps, swipes, and/or other gestures on external surfaces and/or the housing surfaces of multiple devices10), drag and drop operations associated with objects selected using a lingering gaze or other point-of-gaze input, etc. The finger input may include information on finger orientation, position, and/or motion and may include information on how forcefully a finger is pressing against surfaces (e.g., force information). Finger pointing input (e.g., the direction of finger pointing) may be gathered using radio-frequency sensors among sensors18and/or other sensors in device(s)10.

If desired, user input may include air gestures (sometimes referred to as three-dimensional gestures or non-contact gestures) gathered with sensors18(e.g., proximity sensors, image sensors, ultrasonic sensors, radio-frequency sensors, etc.). Air gestures (e.g., non-contact gestures in which a user's fingers hover and/or move relative to the sensors18of device10and/or in which device10hovers and/or moves relative to external surfaces) and/or touch and/or force-based input may include multifinger gestures (e.g., pinch to zoom, etc.). In some arrangements, a user may wear one or more finger devices10on both hands, allowing for two-hand tracking. For example, finger devices10on one hand may be used for detecting click or tap input and finger devices10on the other hand may be used for detecting more complex finger gestures. In some embodiments, a user may wear multiple devices10on one hand (e.g., on a thumb and index finger) and these devices may be used to gather finger pinch input such as pinch click gesture input, pinch-to-zoom input, and/or pinch force input. For example, a pinch click input may be detected when a tap (e.g., a peak in an accelerometer output signal) for a thumb device correlates with a tap for an index finger device and/or pinch force input may be gathered by measuring strain gauge output with strain gauges in devices10as the devices10press against each other. Pinch force can also be detected by measuring the size of the contact patch produced when a finger presses against a two-dimensional touch sensor (larger contact area being associated with larger applied force). In other arrangements, pinch click gesture input and pinch force input may be gathered using only a single finger device (e.g., by measuring motion or forces of the finger pad or finger pulp of the finger wearing the finger device as the user pinches, presses, or taps on the finger pad with a thumb finger or other finger).

By correlating user input from a first of devices10with user input from a second of devices10and/or by otherwise analyzing finger device sensor input, pinch gestures (e.g., pinch click or pinch tap gestures and/or pinch force input) and other finger device input may be detected and used in manipulating displayed content or taking other actions in system8. Consider, as an example, the use of a pinch gesture to select a displayed object associated with a user's current point-of-gaze. Once the displayed object has been selected based on the direction of the user's point-of-gaze (or finger point direction input) and based on the pinch gesture input or other user input, further user input gathered with one or more devices10may be used to rotate and/or otherwise manipulate the displayed object. For example, information on finger movement (e.g., rotational movement) may be gathered using an internal measurement unit or other sensor18in device(s)10and this rotational input used to rotate the selected object. In some scenarios, an object may be selected based on point-of-gaze (e.g., when a user's point-of-gaze is detected as being directed toward the object) and, following selection, object attributes (e.g., virtual object attributes such as virtual object appearance and/or real-world object attributes such as the operating settings of a real-world device) can be adjusted using strain gauge or touch sensor contact patch pinch input (e.g., detected pinch force between finger devices10that are being pinched together on opposing fingers) and/or can be adjusted using finger device orientation input (e.g., to rotate a virtual object, etc.).

If desired, gestures such as air gestures (three-dimensional gestures) may involve additional input. For example, a user may control system8using hybrid gestures that involve movement of device(s)10through the air (e.g., an air gesture component) and that also involve contact (and, if desired, movement) of a thumb or other finger against a two-dimensional touch sensor, force sensor, or other sensor18. As an example, an inertial measurement unit may detect user movement of finger40through the air (e.g., to trace out a path) while detecting force input, touch input, or other input (e.g., finger pinch input or other input to adjust a line or other virtual object that is being drawn along the path).

Sensors18in device10may, for example, measure how forcefully a user is moving device10(and finger40) against surface48(e.g., in a direction parallel to the surface normal n of surface48such as the −Z direction ofFIG.3) and/or how forcefully a user is moving device10(and finger40) within the X-Y plane, tangential to surface48. The direction of movement of device10in the X-Y plane and/or in the Z direction can also be measured by the force sensors and/or other sensors18at locations46.

Structure50may be a portion of a housing of device24, may be a portion of another device10(e.g., another housing44), may be a portion of a user's finger40or other body part, may be a surface of a real-world object such as a table, a movable real-world object such as a bottle or pen, or other inanimate object external to device10, and/or may be any other structure that the user can contact with finger40while moving finger40in a desired direction with a desired force. Because motions such as these can be sensed by device10, device(s)10can be used to gather pointing input (e.g., input moving a cursor or other virtual object on a display such as a display in devices24), can be used to gather tap input, swipe input, pinch-to-zoom input (e.g., when a pair of devices10is used), or other gesture input (e.g., finger gestures, hand gestures, arm motions, etc.), and/or can be used to gather other user input.

Sensors18may include a touch sensor formed from an array of capacitive touch sensor electrodes such as electrodes52overlapping the side and/or top surfaces of housing44. Touch sensor electrodes52of sensor18may be used to gather touch input such as input from direct contact and/or close proximity with a different finger of the user or other external object. In the example ofFIG.3, touch sensor electrodes52may form a touch input area on the outer surface of the side(s) of device10.

System8may have display devices or other devices that present image content to a user. One or more finger devices10may be used to interact with the display devices. For example, finger devices10may be used to scroll through menu options on a display, to scroll through web browser pages on a display, to provide input to computer-aided design software that is running on a display device, and/or to provide other input to a display (e.g., a desktop display, a laptop display, a cellular telephone display, a tablet display, or other display that displays two-dimensional image content, and/or displays that display three-dimensional image content such as head-mounted displays, etc.).

FIGS.4,5,6, and7show illustrative types of user input that may be detected with one or more finger devices and that may be used to provide input to a device with a display.

In the example ofFIG.4, finger device10is being used to detect finger input to the finger pulp. In particular, sensors18in device10may measure the activity of finger40-1(e.g., a pointer finger or other suitable finger wearing device10) as finger40-2(e.g., a thumb or other suitable finger) makes contact with finger pulp40P of finger40-1. This may include, for example, detecting how forcefully finger40-2presses against finger pulp40P (e.g., how forcefully finger pulp40P is pressed inwardly towards the fingernail), detecting small changes in distance between finger40-1and sensors18(and/or detecting how forcefully finger40-1is pressed against the sidewalls of device10) as finger40-2moves finger pulp40P around, from side to side, from front to back, etc., and/or measuring other properties of finger40-1as finger40-2makes contact with finger pulp40P. Sensors18may detect lateral movement of finger pulp40P (e.g., movement in the X-Y plane) in which pulp40P moves relative to finger40-2and may also detect movement of finger pulp40P that results in shear force in the X-Y plane with little or no actual movement of pulp40P relative to finger40-2. Sensors18may detect and distinguish between a tap (e.g., where finger40-2lightly and briefly contacts finger pulp40P), a press (e.g., where finger40-2forcefully and briefly contacts finger pulp40P), a pinch (e.g., where finger40-2forcefully contacts finger pulp40P for a longer time period), and/or other finger input to finger pulp40P.

FIG.5shows an example in which finger device10is being used to detect finger input on a surface of structure50. In particular, sensors18in device10may measure the activity of finger40(e.g., a pointer finger or other suitable finger wearing device10) as finger40makes contact with the surface of structure50. This may include, for example, detecting how forcefully finger40presses against structure50(e.g., how forcefully finger pulp40P is pressed inwardly towards the fingernail), detecting small changes in distance between finger40and sensors18(and/or detecting how forcefully finger40is pressed against the sidewalls of device10) as finger40moves finger pulp40P on structure50around, from side to side, from front to back, etc., and/or measuring other properties of finger40as finger40makes contact with structure50. Sensors18may detect lateral movement of finger40(e.g., movement in the X-Y plane) in which pulp40P moves relative to the surface of structure50and may also detect movement of finger40that results in shear force in the X-Y plane with little or no actual movement of pulp40P relative to the surface of structure50. Sensors18may detect and distinguish between a tap (e.g., where finger40lightly and briefly contacts structure50), a press (e.g., where finger40forcefully but briefly contacts structure50), a press-and-hold (e.g., where finger40forcefully contacts structure50for a longer time period), and/or other finger input on structure50.

FIG.6shows an example in which finger device10is being used to detect finger input on device10. In particular, device10may be worn on finger40-1(e.g., a pointer finger or other suitable finger) and may detect activities of finger40-2(e.g., a thumb or other suitable finger) as it makes contact with and/or as it comes in proximity to device10. This may include, for example, detecting how forcefully finger40-2presses against device10and/or detecting swipes, pinches, taps, presses, press-and-holds, or other gestures on device10. Sensors18may, for example, include a touch sensor formed from a one-dimensional or two-dimensional array of capacitive touch sensor electrodes (e.g., electrodes52ofFIG.3) and/or may include other sensors for detecting touch input, force input, and/or other input involving contact between the user's fingers and device10.

FIG.7is an example in which device10is being used to detect the proximity of one or more adjacent fingers. In particular, device10may be worn on finger40-1(e.g., a pointer finger or other suitable finger) and may detect activities of finger40-2(e.g., a middle finger or other suitable finger) as it makes contact with and/or as it comes in proximity to device10. For example, sensors18may detect when the middle finger is in contact with and/or in proximity to the pointer finger wearing device10(e.g., when finger40-2and finger40-1are side-to-side, when finger40-2is stacked on top of or below finger40-1, etc.). Detecting when finger40-1is in contact with or close proximity to finger40-1may be used to provide a different type of input than that associated with a single finger. For example, finger gestures made with two side-by-side fingers as shown inFIG.7may be used to scroll through content on a display whereas finger gestures made with a single finger may be used to move a cursor on a display.

If desired, the finger gestures ofFIGS.4,5,6, and7may be combined with one another and/or combined with other finger gestures to provide different types of user input to a display. As an example, a user may select an item on a display by tapping or pinching finger pulp40P (e.g., as shown inFIG.4) and, once the item has been selected, the user may manipulate the selected item by providing touch input to a touch sensor on device10(e.g., as shown inFIG.6). Multi-finger gestures may be detected by detecting an adjacent finger as the user pinches against the finger pulp of a finger wearing device10(e.g., as discussed in connection withFIG.4), by detecting an adjacent finger as the user presses a finger wearing device10against a surface (e.g., as discussed in connection withFIG.5), by detecting an adjacent finger as the user touches the outside of device10(e.g., as discussed in connection withFIG.6), etc.

If desired, one or more finger devices10may be used to provide track pad input to an electronic device. In a track pad arrangement, a user may make track pad gestures (e.g., taps, presses, pinch-to-zoom, scrolling, swiping, etc.) on any surface (e.g., a non-touch-sensitive surface) and/or in the air to provide input to an electronic device. Consider, as an example, a scenario of the type shown inFIG.8. In this example, device24has a display such as display54that is configured to display images for user58. The image may include one or more objects (e.g., visual items) such as object56. Object56may be a cursor or any other displayed content.

User58may provide input to electronic device24using finger device10. In some arrangements, user58may wish to provide track pad type input to control objects on display54such as cursor56or other items on display54. When the user wishes to provide track pad type input, the user may define the boundaries of the region that is intended to be used as a track pad region. For example, a user may define the boundaries of an input region on a surface such as surface64(e.g., a table, desk, or other surface) by tapping in locations60on surface64with the finger wearing device10. Device10may have sensors18that detect the first, second, third, and fourth taps in locations60on surface64(e.g., taps of the type described in connection withFIG.5) and may register region62within locations60as an input region for receiving touch pad type input.

Once a user has defined the boundaries of an input region62on surface64, the user may provide input to region62as if region62were a track pad (e.g., a touch-sensitive surface). In particular, device10may detect finger gestures (e.g., taps, swipes, pinch-to-zoom gestures, two-finger scrolling gestures, and other finger gestures) on surface64in region62which may be used to control electronic device24and/or objects being displayed on display54such as cursor56. Finger gestures in region62may be interpreted differently than finger gestures outside of region62, if desired. In this way, device10may be used to turn any surface into a touch pad surface. The use of four points to define the boundaries of input region62is merely illustrative. If desired, a user may define only one point, only two points, three or more points, and/or may draw a rectangle, circle, or other shape with his or her finger on the desired surface to turn that region of the surface into an input region. Arrangements in which the boundaries of an input region are predefined may also be used.

In addition to or instead of turning surfaces such as surface64into an input region, user58turn a region of free space into an input region. This may be achieved by defining an area in space that is to be used as the input region. For example, user58may pinch his or her fingers in locations66to define the boundaries of input region68in space. Device10may detect first, second, third, and fourth pinches in locations66(e.g., pinches, taps, or other finger-to-finger input of the type described in connection withFIG.4) and may register region68within locations66as an input region for receiving touch pad type input or other suitable user input. Once a user has defined the boundaries of input region68, the user may provide input to region68as if region68were a track pad. In particular, device10may detect finger gestures (e.g., taps, swipes, pinch-to-zoom gestures, two-finger scrolling gestures, and other finger gestures) in region68which may be used to control electronic device24and/or objects being displayed on display54such as cursor56. Finger gestures in region68may be interpreted differently than finger gestures outside of region68, if desired. In this way, device10may be used to turn any region of space into an input region. The use of four points to define the boundaries of input region68is merely illustrative. If desired, a user may define only one point, only two points, three or more points, and/or may draw a rectangle, circle, or other shape with his or her finger in the desired region to turn that region of space into an input region. Arrangements in which the boundaries of an input region are predefined may also be used.

FIG.9illustrates how a user may use his or her hand as a track pad surface. When the user wishes to provide track pad type input, the user may define the boundaries of an input region on his or her hand by tapping in locations70on the palm of his or her hand (or on any other portion of the user's body) with the finger wearing device10. Device10may detect the first, second, third, and fourth taps in locations70(e.g., taps of the type described in connection withFIG.5) and may register region72within locations70as an input region for receiving touch pad type input. Once a user has defined the boundaries of an input region72on his or her hand, the user may provide input to region72as if region72were a track pad (e.g., a touch-sensitive surface). In particular, device10may detect finger gestures (e.g., taps, swipes, pinch-to-zoom gestures, two-finger scrolling gestures, and other finger gestures) in region72which may be used to control an electronic device and/or objects on a display (e.g., display54ofFIG.8). Finger gestures in region72may be interpreted differently than finger gestures outside of region72, if desired. In this way, device10may be used to turn any surface of the user's body into a touch pad surface. The use of four points to define the boundaries of input region72is merely illustrative. If desired, a user may define only one point, only two points, three or more points, and/or may draw a rectangle, circle, or other shape with his or her finger on a body part to turn that region of the body into an input region. Arrangements in which the boundaries of an input region are predefined may also be used.

FIG.10illustrates how a user may use a movable object as a track pad surface. When the user wishes to provide track pad type input, the user may define the boundaries of an input region on an object such as object78by tapping in locations74on object78with the finger wearing device10. Device10may detect the first, second, third, and fourth taps in locations74(e.g., taps of the type described in connection withFIG.5) and may register region76within locations74as an input region for receiving touch pad type input. Once a user has defined the boundaries of input region76on object78, the user may provide input to region76as if region76were a track pad (e.g., a touch-sensitive surface). In particular, device10may detect finger gestures (e.g., taps, swipes, pinch-to-zoom gestures, two-finger scrolling gestures, and other finger gestures) in region76which may be used to control an electronic device and/or objects on a display (e.g., display54ofFIG.8). Finger gestures in region76may be interpreted differently than finger gestures outside of region76, if desired. In this way, device10may be used to turn any surface of any object (e.g., a non-touch-sensitive surface of a cellular telephone, a non-touch-sensitive surface of an object without electronics, etc.) into a touch pad surface. The use of four points to define the boundaries of input region76is merely illustrative. If desired, a user may define only one point, only two points, three or more points, and/or may draw a rectangle, circle, or other shape with his or her finger on the desired surface to turn that region of the surface into an input region. Arrangements in which the boundaries of an input region are predefined may also be used.

If desired, haptic output (e.g., a click) can be provided as the user provides track pad input to a surface or to the air. For example, haptic output detents (vibrations that are supplied when the user's finger position coincides with predetermined locations) may be supplied during user finger interactions to indicate the selection of an item, to simulate clicking, and/or to provide other output.

FIG.11is another example showing how device10may be used to turn an object into an input device. In the example ofFIG.11, object80may be a pen or pencil that does not contain any circuitry. A user wearing one or more finger devices10may rotate object80about its longitudinal axis, may move the tip of object80across a surface (e.g., the surface of structure50), and/or may tap or press the tip of object80on the surface of structure50, and/or may make other movements of object80. During movement of object80, one or more finger devices10can gather information on the interactions between finger device10and object80and can thereby be used in determining the location, orientation, and movement of object80. If desired, device10may be used to detect movement of object80relative to the surface of structure50(e.g., so that object80becomes a stylus and device10detects stylus input on structure50). This in turn may be used to control an electronic device and/or objects on a display (e.g., display54ofFIG.8).

FIG.12is a perspective view of an illustrative display and finger device being used to provide input to the display. As shown inFIG.12, electronic device24may include display54and an optical sensor such as a gaze detection sensor82(sometimes referred to as a gaze detector, gaze tracker, gaze tracking system, or eye monitoring system). Gaze detection sensor82may, for example, include image sensors, light sources, and/or other equipment that is used in monitoring the eyes of a user. This system may include one or more visible and/or infrared cameras that face a viewer's eyes and capture images of the viewer's (user's) eyes. The orientation of the user's gaze may be used to determine which location on display54the user is looking at (sometimes referred to as the user's point-of-gaze). If desired, device24may use gaze tracking information such as information on the user's point-of-gaze in determining which actions to take in system8. For example, a gaze tracking system may determine that a user's point-of-gaze is directed towards a first object and not a second object and may respond by assuming that the user is visually selecting the first object and not the second object. Finger input and/or other user input may be used in combination with input such as point-of-gaze information in determining which actions are to be taken.

In the example ofFIG.12, display54is being used to display image content such as items80. Items80may be a list of menu options, a list of movie options, a channel guide, or any other suitable display content. A user may use finger device10to navigate the content being displayed on display54. For example, display54may display a selection indicator such as selection indicator90which lets the user know which item80is currently selected. The user may move selection indicator90to different items80by providing input via finger device10. In the example ofFIG.12, device10is worn on finger40-1and the user is providing input to device10using finger40-2(e.g., as described in connection withFIGS.4and6). For example, the user may swipe to the right (e.g., in direction92) on sensor18(e.g., a two-dimensional array of touch sensor electrodes such as electrodes52ofFIG.3) to move selection indicator90on display54to the right (e.g., in direction84). The user may swipe downward on sensor18to move selection indicator90on display54to downward (e.g., in direction86). If desired, display54may display a visual aid such as scroll bar88or other suitable visual aid for indicating where among items80the selection indicator90is currently located.

If desired, haptic output (e.g., a click) can be provided as the user navigates items80on display54. For example, haptic output detents (vibrations that are supplied when the user's finger position coincides with predetermined locations) may be supplied during user finger interactions to indicate movement of selection indicator90, to indicate when an item80has been selected, and/or to provide other output to the user.

The use of finger40-2to provide touch input to the outer surfaces of device10(as discussed in connection withFIG.6) to navigate items on display54is merely illustrative. If desired, the user may navigate items on display54by using finger40-2to provide touch input to the finger pulp of finger40-1(as discussed in connection withFIG.4). In both types of input arrangements, device10may be used to detect swipes (e.g., for moving selection indicator90) and clicks (e.g., hard presses on sensor18and/or on pulp40P) for selecting item80on which selection indicator90is located. Track pad type input (e.g., of the type described in connection withFIGS.5,8,9,10, and11) may also be used to navigate items80on display54. These examples are merely illustrative. Finger device10may be used to provide other types of input to display54.

In the example ofFIG.13, display54is being used to display image content such as items96and94. Items96and94may be different documents, tabs, programs, websites, or any other suitable display content. A user may use finger device10to navigate the content being displayed on display54. For example, the user may scroll down on item94by pinching his or her finger40-2against finger pulp40P of finger40-1wearing device10and, while pinching, may pull his or her fingers upward in direction100. This combination of pinching and pulling upward by the user may in turn cause the bottom of item94to move upward in direction98by an amount proportional to the upward motion of the user's pinched fingers.

If desired, device24may determine which item is being scrolled using the pinch and pull hand gesture using gaze tracker82. For example, gaze tracker82may detect that the user is looking towards the bottom of item94during the initial pinching, and control circuitry in device24may interpret this gaze information to mean that the pinch and pull upward motion is intended to manipulate item94and may move the bottom of item94upward in direction98accordingly. When the user wishes to manipulate item96, the user may direct his or her gaze at item96(which may be detected by gaze tracker82) and may make hand or finger gestures with device10.

If desired, haptic output (e.g., a click) can be provided as items96and94are manipulated on display54. For example, haptic output detents (vibrations that are supplied when the user's finger position coincides with predetermined locations) may be supplied during user finger interactions to indicate the selection of an item, to indicate that no further downward scrolling is possible because the end of the item has been reached, and/or to provide other output.

In the example ofFIG.14, display54is being used to display image content such as image content associated with a computer-aided design software program. The software program may be used to create two-dimensional and/or three-dimensional illustrations of an engineering project, architectural project, manufacturing project, or other project. In the example ofFIG.14, display54is used to display computer-aided design items102and104. A user may use finger device10to create and manipulate computer-aided design items102and104. Because finger device10detects movement of finger40and adjacent fingers in space, finger device10may allow for a wider range of user input gestures for manipulating computer-aided designs with six degrees of freedom.

For example, the user may make a pinch gesture to select an item that the user wishes to manipulate. If desired, device24may use gaze tracker82to determine which item the user is looking at while making the pinch gesture and thus which item the user is selecting. Point-of-gaze information and user input to finger device10may also be used to select which axis the user wishes to manipulate in the computer-aided drawings. For example, the user may manipulate axis104-1or104-2of item104on display54by pinching fingers40-1and40-2together (or providing other suitable input with finger device10) while directing his or her gaze to the desired axis.

Once the displayed item (e.g., item102,104, and/or an individual axis of item102or item104) has been selected based on the direction of the user's point-of-gaze (or finger point direction input) and based on the pinch gesture input or other user input with device10, further user input gathered with one or more devices10may be used to rotate and/or otherwise manipulate the selected item. Information on finger movement (e.g., rotational movement) may be gathered using an internal measurement unit or other sensor18in device(s)10and this rotational input used to rotate the selected object. As shown inFIG.14, for example, a user may pinch finger pulp40P of finger40-1with finger40-2while rotating fingers40-1and40-2in direction106, which in turn may be used to rotate item102in direction108.

Display24may provide visual aids to let the user know which item or axis is being manipulated. Different types of input to device10may be used to make fine control adjustments and course control adjustments, if desired. For example, a finger pinch of the type shown inFIG.4may be used to make fine control adjustments and touch input to the outer surface of device10as shown inFIG.6may be used to make course control adjustments, if desired.

If desired, haptic output (e.g., a click) can be provided as items102and104are manipulated on display54. For example, haptic output detents (vibrations that are supplied when the user's finger position coincides with predetermined locations) may be supplied during user finger interactions to indicate the selection of an item, to indicate the locking of a rotational axis, to indicate object-to-object collision, etc.

As described above, one aspect of the present technology is the gathering and use of information such as sensor information. The present disclosure contemplates that in some instances, this gathered data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data can include demographic data, location-based data, telephone numbers, email addresses, twitter ID's, home addresses, data or records relating to a user's health or level of fitness (e.g., vital signs measurements, medication information, exercise information), date of birth, eyeglasses prescription, username, password, biometric information, or any other identifying or personal information.

The present disclosure recognizes that the use of such personal information, in the present technology, can be used to the benefit of users. For example, the personal information data can be used to deliver targeted content that is of greater interest to the user. Accordingly, use of such personal information data enables users to calculated control of the delivered content. Further, other uses for personal information data that benefit the user are also contemplated by the present disclosure. For instance, health and fitness data may be used to provide insights into a user's general wellness, or may be used as positive feedback to individuals using technology to pursue wellness goals.

The present disclosure contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. Such policies should be easily accessible by users, and should be updated as the collection and/or use of data changes. Personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection/sharing should occur after receiving the informed consent of the users. Additionally, such entities should consider taking any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices. In addition, policies and practices should be adapted for the particular types of personal information data being collected and/or accessed and adapted to applicable laws and standards, including jurisdiction-specific considerations. For instance, in the United States, collection of or access to certain health data may be governed by federal and/or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), whereas health data in other countries may be subject to other regulations and policies and should be handled accordingly. Hence different privacy practices should be maintained for different personal data types in each country.

Despite the foregoing, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the collection of personal information data during registration for services or anytime thereafter. In another example, users can select not to provide certain types of user data. In yet another example, users can select to limit the length of time user-specific data is maintained. In addition to providing “opt in” and “opt out” options, the present disclosure contemplates providing notifications relating to the access or use of personal information. For instance, a user may be notified upon downloading an application (“app”) that their personal information data will be accessed and then reminded again just before personal information data is accessed by the app.

Moreover, it is the intent of the present disclosure that personal information data should be managed and handled in a way to minimize risks of unintentional or unauthorized access or use. Risk can be minimized by limiting the collection of data and deleting data once it is no longer needed. In addition, and when applicable, including in certain health related applications, data de-identification can be used to protect a user's privacy. De-identification may be facilitated, when appropriate, by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of data stored (e.g., collecting location data at a city level rather than at an address level), controlling how data is stored (e.g., aggregating data across users), and/or other methods.

Therefore, although the present disclosure broadly covers use of personal information data to implement one or more various disclosed embodiments, the present disclosure also contemplates that the various embodiments can also be implemented without the need for accessing such personal information data. That is, the various embodiments of the present technology are not rendered inoperable due to the lack of all or a portion of such personal information data.

The foregoing is merely illustrative and various modifications can be made to the described embodiments. The foregoing embodiments may be implemented individually or in any combination.