System and method for remote touch detection

Embodiments of the present invention disclose a system and method for remote touch detection. According to one embodiment, an infrared light source is projected across a front surface of a display via at least one light emitting device. Furthermore, an infrared signal is transmitted in a direction of the front surface of the display via a remote pointing device. A surface target position is then determined based on measurement data caused by disruption of a portion of the infrared light source of the light emitting device by either the infrared signal or a physical touch from a user.

BACKGROUND

Providing efficient and intuitive interaction between a computer system and users thereof is essential for delivering an engaging and enjoyable user-experience. Today, most computer systems include a keyboard for allowing a user to manually input information into the computer system, and a mouse for selecting or highlighting items shown on an associated display unit. As computer systems have grown in popularity, however, alternate input and interaction systems have been developed. For example, touch-based, or touchscreen, computer systems allow a user to physically touch the display unit and have that touch registered as an input at the particular touch location, thereby enabling a user to interact physically with objects shown on the display of the computer system.

DETAILED DESCRIPTION OF THE INVENTION

While touch technology is an exciting and natural means of user interface, it can still be improved. One fundamental drawback of conventional touchscreen interface systems is that the user must be within a few feet of the touchscreen computer for operation. In addition to limiting user movement and system placement, physical touching of the display screen may also cause arm and shoulder fatigue with extended use of the system.

Embodiments of the present invention provide a system and method for remote touch detection capable utilizing existing touchscreen computer system without hardware modification. That is, the touchscreen computer vision system is capable of supporting both physical touch, or “black or shadow” detection, and remote touch, or “white” detection. As such, embodiments in accordance with the present invention allow system interaction with an laser beam or infrared signal such as one from a laser pointer, thereby enabling users to remotely interface with a touchscreen system while maintaining line-of-sight contact of the laser beam with the front surface of the display.

Several advantages are afforded by the remote touch input method of the present embodiments. For example, the operating user does not need to be in close proximity to the touch display in order to interface with it. Accordingly, such a configuration allows for greater user mobility and more flexible placement of the touchscreen display. Furthermore, the remote touch system of the present embodiments helps to alleviate arm and shoulder fatigue caused by extended sessions of physically touching a vertical screen since user input can now be done from a comfortable location with only the press of a button or a flick of the wrist from the remote pointing device.

Referring now in more detail to the drawings in which like numerals identify corresponding parts throughout the views,FIG. 1Ais a three-dimensional perspective view of an all-in-one computer having multiple optical sensors, whileFIG. 1Bis a top down view of a display device and optical sensors including the field of views thereof according to an embodiment of the present invention. As shown inFIG. 1A, the system100includes a housing105for enclosing a display panel109and light emitting devices113aand113band optical sensors110aand110b. The system also includes input devices such as a keyboard120and a mouse125for text entry, navigating the user interface, and manipulating data by a user for example.

The display system100may includes a display panel109and a transparent layer107in front of the display panel109, though the transparent layer107may be omitted in certain embodiments. The front side of the display panel109is the surface that displays an image and the back of the panel109is opposite the front. Light emitting devices113aand113band optical sensors110aand110bcan be on the same side of the transparent layer107as the display panel109to protect the optical sensors from contaminates. In an alternative embodiment, the light emitting device and optical sensors110aand110bmay be in front of the transparent layer107. The transparent layer107can be glass, plastic, or another transparent material. The display panel109may be a liquid crystal display (LCD) panel, a plasma display, a cathode ray tube (CRT), an OLED or a projection display such as digital light processing (DLP), for example. In one embodiment, mounting the light emitting devices113aand113band optical sensors110aand110bin an area of the display system100that is outside of the perimeter of the of the display panel109provides that the clarity of the transparent layer is not reduced by the light emitting devices or optical sensors.

In one embodiment, optical sensors110aand110brepresent may two-dimensional cameras including a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors for example, and are configured to receive external light or shadows and convert the light or shadow to data. In another embodiment, optical sensors110aand110brepresent three-dimensional optical sensors configured to report a three-dimensional depth snap to a processor. The three-dimensional optical sensors110aand110bcan determine the depth of an object located within its respective field of view115aand115b. The depth map changes over time as an object and signal moves in the respective field of view115aof optical sensor110a, or within the field of view115bof optical sensor115b. According to one embodiment, the depth of the object can be used to determine if the object is within a programmed distance of the display panel but not actually contacting the front side of the display panel. For example, the object may be a user's hand or finger approaching the front side of the display panel109, or an infrared signal emitted from a laser pointing device operated by a user. Still further, and according to one embodiment, optical sensors110aand110bare positioned at top most corners around the perimeter of the display panel109such that each field of view115aand115bincludes the areas above and surrounding the display panel109. As such, a touch input such as a user's hand or infrared signal for example, may both be detected, and any associated motions around the perimeter and in front of the computer system100can be accurately interpreted by the computer processor.

FIG. 2is a simplified block diagram of the computer vision system according to an embodiment of the present invention. As shown in this exemplary embodiment, the system200includes a processor220coupled to a display unit230, a system control unit240, a computer-readable storage medium225, light emitter210, and optical sensor213configured to capture touch input204, or measurement data related to an object or input disruption near the front of the display unit230. In one embodiment, processor220represents a central processing unit configured to execute program instructions. Display unit230represents an electronic visual display or touch-sensitive display such as a desktop flat panel monitor configured to display images and a graphical user interface for enabling interaction between the user and the computer system. Storage medium225represents volatile storage (e.g. random access memory), non-volatile store (e.g. hard disk drive, read-only memory, compact disc read only memory, flash storage, etc), or combinations thereof. In one embodiment, system control unit240may represent an application program or user interface control module configured to receive and process measurement data of a detected object. Furthermore, storage medium225includes software228that is executable by processor220and, that when executed, causes the processor220to perform some or all of the functionality described herein.

FIG. 3is an illustrative example of operation of the light emitting device and optical sensor according to an embodiment of the invention. The light emitting device325is configured to project a light source327such as an infrared light or a laser light source for example, that emits light and may be invisible to the user. In one embodiment, the light emitting device325emits a structured light that is projected as a light pattern such as a plane, grid, or more complex shape at a known angle. The way that the light pattern deforms when disrupted by an object or light source, allows the computer vision system to calculate the depth and surface information of the object or light source. For example, the optical sensor315can receive disrupted light329as measurement data, which is reflected from a light emitting device325from an object320for example. Furthermore, the light emitting device325can be in any position relative to the optical sensor315that allows the light source327to be disrupted or reflected off the object320and be captured by the optical sensor315. The infrared light325can be disrupted and reflected from an object320such as user's hand or other light source, so as to be captured by the optical sensor315.

FIGS. 4A and 4Bare top down perspective views of the computer vision system and operating environments thereof according to an embodiment of the present invention. As shown in the embodiment ofFIG. 4A, the touchscreen system includes a display405for enclosing light emitting devices412aand412band optical sensors410aand410b. According to the present embodiment, light emitting devices412aand412bproject light source417aand417brespectively. More specifically, the light emitters412aand412bprojects a plane of infrared light417aand417bthat covers the front surface407of the display panel405. Optical sensors410aand410binclude respective fields of views415aand415bconfigured to detect and capture disruptions in either light source417aor417bcaused by an input object or device approaching the front display surface407. The object may be a physical object such as a finger or stylus is in contact with the display surface407as shown inFIG. 4A, or an invisible input or infrared signal emitted onto the front surface407of the display as shown inFIG. 4B.

More particularly, an object such as a user's hand428may approach the display405and cause a disruption in either light source417aor417bat position425inFIG. 4A. Similarly, a remote pointing device430operated by a user may output an infrared signal432in the direction of the display405so as to cause a disruption in the plane of either light source417aor417bat position425inFIG. 4B. When the infrared signal432disrupts the plane of either light source417aor417b, the disruption is accordingly detected by one of the optical sensors410aor410b. Thereafter, a processing controller receives measurement or disruption information from the optical sensors in the form of a shadow data where the disrupted area may appear black or white for example.

FIG. 5Aillustrates an exemplary environment of a user interacting with the computer vision system, whileFIG. 5Billustrates a top down view perspective view of the computer vision system and an operating user shown inFIG. 5Aaccording to one embodiment of the present invention. As shown inFIGS. 5A and 5B, the operating environment includes a user535aiming a remote pointing device530at a display system502. The user may be positioned further away (e.g. greater than one meter) from the display system than traditional touchscreen operating environments. Like the previous embodiments, the display system includes a light emitting device510configured to project a light source plane525across the front surface507of the display panel505, and an optical sensor510configured with a field of view515. Furthermore, as the infrared signal532from the remote pointing device530approaches the front surface of the display panel, the signal532eventually breaks and disrupts a portion of the light source525or plane thereof.

As shown in the embodiment ofFIG. 5A, the plane of the light source525is disrupted by the infrared signal532near position527, which is proximate to the front surface of the display panel. Thereafter, the optical sensor510captures the light caused by the disruption and associates measurement data therewith. For example, in the case of a physical disruption from a stylus or hand for example, the optical sensors are configured to capture measurement data having an approximate “black” color value at or near (0, 0, 0) within a RGB color model caused by the change in intensity of the light source emitted by the light emitters at the area of disruption. Similarly, in the case of a disruption from a light source such as a laser beam or infrared signal from a remote pointing device for example, the optical sensors are also configured to simultaneously capture measurement data having an approximate “white” color value at or near (255, 255, 255) within a RGB color model caused by the change in intensity of the light source emitted by the light emitters at the area of disruption. Based upon this disruption and measurement data, the processor of the computer vision system can determine the approximate location of the disruption and the surface target position, which is the desired touch input location on the front surface of the display panel. Furthermore, and as shown in the embodiment ofFIGS. 5A-5B, the user535and the user's hand is in a natural and relaxed positioned while interacting with the computer vision system502.

FIG. 6illustrates the processing steps for remote touch detection according to an embodiment of the present invention. In step602, the computer vision system projects an infrared light source as a plane to cover the front surface of the display device. Next, in step604an infrared signal from a user operating a remote pointing device directed towards the front surface of the display, is received at the light source projected by the light emitting device. A disruption in the light source is then captured by the optical sensor as measurement or disruption data in step606. That is, the optical sensor detects the relative location of the disruption on the front surface of the display caused by the infrared signal or physical object breaking and disrupting the plane of the light source from the light emitting device.

In step608, the processor calculates the surface target position based on measurement data captured by the optical sensor. Next, in step610, the system determines if the surface target position has moved, or if the user is moving the remote pointing device or physical object across the front display surface, and if so, updates the surface target position accordingly. According to one embodiment, once the surface target position is stationary for a predetermined time, the processor registers the surface target position as a touch input location in step612for determining an appropriate operation of the computer system.

Embodiments of the present invention provide a method for implementing a remote pointing device for use with touchscreen computer vision systems. In particular, the computer vision system of the present embodiments is configured to detect touch inputs caused by physical objects contacting a front surface of the display panel, in addition to touch inputs caused by an infrared light source contacting the front surface of the display panel. As such, a laser pointing device operated by a user and emitting an infrared signal may serve as a remote operating device for controlling and operating a touchscreen computing system.

Many advantages are afforded by the remote touch detection system and method according to embodiments of the present invention. For instance, the display panel of the touchscreen computing system may be placed in a location in which physical touch of the display is difficult or impossible (e.g. ceiling mounted). In such a case, the user may operate the computing system remotely from a more comfortable position than normal (e.g. lying down). Still further, the computer vision system of the present embodiments helps to reduce upper body fatigue caused by prolonged extension of a user's arms when physically contacting the display panel with their hands or a stylus for example. Moreover, embodiments of the present invention can be beneficial for users with physical disabilities by attaching the laser emitter to a headband or body part other than the user's hand for example.

Furthermore, while the invention has been described with respect to particular embodiments, one skilled in the art will recognize that numerous modifications are possible. For example, although exemplary embodiments depict an all-in-one computer as the representative display panel of the computer vision system, the invention is not limited thereto. For example, the computer vision system of the present embodiments may be implemented in a netbook, a tablet personal computer, a cell phone, or any other electronic device having a display panel, light emitting device, and optical sensor.

Still further, a single light emitting device and single optical sensor may be utilized in the computer visions system in lieu of the two emitting devices and two optical sensors depicted in the figures. Thus, although the invention has been described with respect to exemplary embodiments, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.