Abstract:
A display system (AP 1 ) has plural cameras ( 11 L an d 11 R) having fields of view ( 31 L and  31 R) that extend more orthogonal to than parallel to a display screen ( 21 ) for viewing distal objects. Reflectors ( 33 L and  33 R) redirect touchscreen light ( 36 ) from a direction along the display screen to a direction toward the cameras. A video processor ( 19 ) locates a 2D position relative to the display screen of an object proximate to the display screen as a function of images generated by the cameras in response to the touchscreen light.

Description:
BACKGROUND 
     A “touchscreen” is a display that can detect the presence and location of a touch, e.g., by a finger, within the display area. Touchscreen capabilities can be enabled by a range of technologies including resistive, surface acoustic wave, capacitive, infrared, strain gage, diffused laser imaging, optical imaging, dispersive signal technology, and acoustic pulse recognition. A touchscreen allows user input without requiring a separate device such as a mouse or trackpad. Unlike those devices, a touchscreen enables a user to interact with what is displayed directly on the screen, where it is displayed, rather than indirectly. 
     Touchscreens are incorporated increasingly in small devices such as cell phones, PDAs, digital audio players, and navigation units. Large touchscreen displays are also increasingly common in kiosks and other commercial settings. However, displays for desktop computers usually do not provide touchscreen capabilities. 
     TouchSmart™ computers, available from Hewlett-Packard Company, are notable exceptions. The computers include and are visually dominated by a touchscreen display. Infrared emitters at the upper corners of the display radiate light that is normally reflected by opposing display bezels. Linear detector arrays detect the reflections and any shadows to locate a touch trigonometrically. The touchscreen capabilities add significantly to the cost of the computers. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The features and advantages of the invention as well as additional features and advantages thereof will be more clearly understood hereinafter as a result of a detailed description of a preferred embodiment of the invention when taken in conjunction with the following drawings in which: 
         FIG. 1  is a schematic diagram of a computer system incorporating a touchscreen display in accordance with an embodiment of the invention. 
         FIG. 2  is a schematic top plan view of a display of  FIG. 1  with reflectors in “webcam-mode” positions permitting sensors to be used as a stereo webcam. 
         FIG. 3  is a schematic top plan view of the display of  FIG. 1  with reflectors in “touchscreen-mode” positions permitting sensors to be used to effect touchscreen capabilities. 
         FIG. 4  is a flow chart of a method in accordance with an embodiment of the invention. 
     
    
    
     These figures depict implementations/embodiments of the invention and not of the invention itself. 
     DETAILED DESCRIPTION 
     The present invention provides for using a pair of webcams (e.g., instead of a pair of linear array sensors) to provide touchscreen functionality. Cost-savings are achieved in part due to economies of scale: webcam sensors are less expensive than, for example, the linear sensors used in some touchscreen implementations. Cost-effectiveness is further enhanced since the cameras can also be used as webcams, providing for stereo imaging and 3D gesture control. 
     An all-in-one computer AP 1  in accordance with an embodiment of the invention is shown in  FIG. 1 . Computer AP 1  has stereo cameras  11 L and  11 R, which are used: 1) for touchscreen functionality and 2) webcam functionality, in other words for imaging and for tracking finger or stylus position on a display. While the illustrated embodiment relates to an all-in-one computer (computer and display as monolithic unit), other embodiments relate to a display that is separate from the computer itself. 
     All-in-one computer AP 1  includes a housing  13 , processors  15 , computer-readable storage media  17  (including main memory, disk, and flash based storage), a multi-image video processor  19 , and communications (including I/O) devices, including a display screen  21 . Housing  13  provides a frame  23  for display screen  21 , which includes a glass plate  25 , as shown in  FIGS. 2 and 3 . Frame  23  and display screen  21  are bridged by a retroreflective bezel  27 . 
     Cameras  11 L and  11 R are located beneath glass plate  25  near respective upper corners  29 L and  29 R of display screen  21 , shown in  FIG. 1 . Cameras  11 L and  11 R ( FIG. 1 ) have respective fields of view  31 L and  31 R ( FIG. 2 ). Fields of view  31 L and  31 R are directed generally outward, with some inclination toward their common center so that they converge toward an expected user position as desired for their uses as stereo webcams and for 3D gesture control. Even with the inclination, fields of view  31 L and  31 R are more orthogonal to than parallel to (the 2D surface of) display screen  21 . This allows cameras  11 L and  11 R to detect “object light” from objects centered relative to and at least a foot from display screen  21  so that the objects can be imaged. 
     So that cameras  11 L and  11 R can be used for touchscreen functionality, eyelid reflectors  33 L and  33 R can be moved to positions respectively occluding fields of view  31 L and  31 R ( FIG. 2 ). In these positions, reflectors  33 L and  33 R cause “touchscreen” light arriving from bezel  27  along display screen  21  (and thus not within fields of view  31 L and  31 R) to be redirected toward respective cameras  11 L and  11 R. In one embodiment, this light is provided by an emissive bezel surface. In the illustrated embodiment, this light is provided by point-source infrared emitters  35 L and  35 R, roughly co-located with the cameras  11 L and  11 R, respectively. In this case, bezel  27  is retroreflective in that incident light  34  from an emitter  35 L,  35 R is reflected as touchscreen light  36  back toward the emitter by the bezel with a minimum of scattering, as shown in  FIG. 3 . Eyelid reflectors  33 L and  33 R have respective apertures  37 L and  37 R ( FIG. 3 ), through which light from emitters  35 L and  35 R is directed toward opposing sides of retroreflective bezel  27 . 
     Each emitter  35 L,  35 R emits infrared light (IR)  35  and directs it to the two opposite sides of bezel  27 . More specifically, emitter  35 L directs IR light to the bottom side  39 B ( FIG. 1 ) and right side  39 R of retroreflective bezel  27 , while emitter  35 R directs IR light to the bottom side  39 B and left side  39 L of bezel  27 . Emitters  35 L and  35 R and bezel  27  are not in the fields of view  31 L and  31 R ( FIG. 2 ). Thus, with reflectors  33 L and  33 R in their webcam mode positions, as shown in  FIG. 2 , neither the IR light  34  emitted by emitters  35 L and  35 R, nor the IR light  35  reflected by bezel  27  impinges on cameras  11 L and  11 R. IR cut filters  41 L and  41 R ( FIG. 2 ) limit the light reaching cameras  11 L and  11 R to visible light so that the cameras  11 L and  11 R provide images that more closely match what a person sees and are not overwhelmed by IR light. 
     In  FIG. 1 , cameras  11 L and  11 R are connected to video processor  19 , which performs processing of the digital signals from the cameras. Video processor  19  detects the positions of reflectors  33 L and  33 R to distinguish touchscreen and webcam modes. Video processor  19  communicates with other computer components using an internal USB connection. In alternative embodiments, IEEE 1394 (firewire) or other protocol connections are used. In a 3D gesture submode, the processing reduces the stereo images into a displacement map (distance information) that computer AP 1  can then interpret and respond to as commands. 
     Light transmitting along display screen  21  can be blocked by an object touching or otherwise sufficiently close to display screen  21 . When computer AP 1  is in touchscreen mode (with reflectors  33 L and  33 R occluding fields of view  31 L and  31 R, shown in  FIG. 2 ), video processor  19  can detect and locate the resulting shadows. Video processor  19  trigonometrically determines the 2D display screen location of the object by comparing the location of breaks (shadows) as seen by cameras  11 L and  11 R. 
     In  FIG. 1 , media  17  has computer-readable data and programs of computer-executable instructions encoded thereon. One such program is a touchscreen/webcam mode controller  43 , which provides an interface for a user to switch between webcam and touchscreen modes for computer AP 1 . When a user switches to touchscreen mode, reflectors  33 L and  33 R are moved into and thus occlude fields of view  31 L and  31 R. A user can select webcam mode, by moving reflectors  33 L and  33 R to their webcam positions ( FIG. 2 ), out of fields of view  31 L and  31 R. In the illustrated embodiment, reflectors  33 L and  33 R are moved manually by a user and the movement is detected by mode controller  43 . In an alternative embodiment, the reflectors are motorized and controlled by mode controller software. 
     A user can select among several modes provided by video processor  19 : 1) in touchscreen mode, video processor  19  determines touch locations; 2) in raw mode, video processor  19  provides a pair of raw video signals; in 2D webcam mode, a pair of raw video images (mostly for webcam mode); 3) in 2D webcam mode, video processor  19  combines raw images to provide merged 2D video images; 4) in 3D webcam mode, video processor  19  combines raw video images to provide 3D images; 5) in 3D command mode, video processor  19  combines raw video images to gestures which can be converted to commands. The latter mode provides for gesture-based control of computer AP 1  as an alternative to touchscreen control. This gesture-based control can be modeless (no need to move reflectors  33 L and  33 R into position) and more comfortable than touchscreen control (some people begin to feel uncomfortable when holding out their arms for touchscreen operation for long periods of time). 
     A method ME 1  in accordance with the invention is flow charted in  FIG. 4 . At step M 1 , a user switches modes (e.g., by manually moving reflectors  33 L and  33 R) between a touchscreen mode M 11  and a webcam mode M 21 . In the case the switch is to touchscreen mode M 11 , reflectors  33 L and  33 R are moved so that they occlude respective fields of view  31 L and  31 R of cameras  11 L and  1 R at step M 12 . In addition, IR emitters  35 L and  35 R can be turned on. Reflectors  33 L and  33 R redirect light from paths along and therefore more parallel to than orthogonal to display screen  21  to a path more orthogonal to than parallel to display screen  21  at step M 13 . If a finger or stylus or a similar object contacts or at least approaches the display screen, it will block light transmitting more parallel to than orthogonal to the display screen. In such a case, shadows can be detected in video images at step M 14 . The positions of shadows in the respective camera images can be used to locate a 2D position of the object relative to the display screen at step M 15 . This position information can be used to interpret two-dimensional gestures (touch, slide, etc.) so that the gestures can be interpreted (converted to) commands at step M 16 . 
     In the case that, at step M 1 , a user switches to a webcam mode M 21 , reflectors  33 L and  33 R are moved out of respective camera fields of view  31 L and  31 R at step M 22 ; in addition, emitters  35 L and  35 R can be turned off. Removing reflectors  33 L and  33 R allows light transmitted more orthogonal to than parallel to the display screen to reach cameras  11 L and  11 R at step M 23  without being redirected. This allows plural video images of a distal object, e.g., a user head, to be generated at step M 24 . The plural images can be combined to generate a unified 2D or 3D video image at step M 25 . In addition, 3D gestures detected in the video images can be converted to commands (“select”, “move”, etc.) at step M 26 . 
     The technology described herein provides for reducing the marginal cost of touchscreen technology by 1) using widely available and economical (webcam) cameras for touchscreen technology; and 2) arranging for the touchscreen components (e.g., cameras) to be used for other purposes (e.g., webcam functionality plus 3D gesture input). By providing for both touchscreen and gesture functionality, a bridge is provided between familiar touchscreen control and emerging gesture control technologies. 
     The foregoing description, for purposes of explanation, has been described with reference to specific embodiments. However, the illustrative discussions are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in view of the disclosed teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, to thereby enable others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. 
     As used herein, “touchscreen” refers to any technology that is used to identify a position on a display screen that is contacted by a finger, stylus, or similar object. While some touchscreen technologies actually detect touch, the image-based technology described herein is not touch-based in that sense. Instead, the present invention detects position by imaging and it does not matter whether the object being imaged is actually touching the display screen or is only close enough to the display screen to interrupt light transmitted along the display screen. 
     Herein, “distal” means far enough from a display screen that it cannot be detected by light used to locate 2D position in touchscreen mode; in practice this can be one centimeter or more from the display screen. Herein, “proximal” means in contact with or near enough to be detected by light used to locate 2D position in touchscreen mode. Typically, proximal is less than one centimeter. 
     The invention provides for many alternatives to the illustrated embodiment. The invention provides for all-in-one and separate computer and display embodiments. The invention can be applied to appliances not typically considered to be computers (although they may include computer components), e.g., televisions and display panels on other devices such as printers. The cameras may be located in other places, e.g., bottom corners or along the sides of displays. More than two cameras can be used; e.g., a third camera at the center top or bottom can help resolve the location of an object touching the display. The standard center webcam might be combined with the stereo cameras to disambiguate two touches in a touch screen mode or to refine a displacement map as well as provide a savings on interface logic and circuit board cost. 
     The shape of the reflectors can be determined by those skilled in the art given that the opposing sides of the bezel must be within the cameras&#39; field of view as modified by the reflectors. Depending on the embodiment, reflectors can translate or rotate between positions. Alternatively, a non-movable reflector can be used. For example, the reflector can be of material that can be electrically controlled so that it is more reflective or more transmissive. In another embodiment, the reflector works as a half-silvered mirror or beam splitter, allowing some light to pass (for touchscreen use) and some to be transmitted (for webcam use). 
     In the illustrated embodiment, infrared radiation is used in touchscreen mode and visible light is used in webcam mode. Other embodiments make use of this split spectrum to permit modeless operation or to provide for mode switching by switching spectra. In the illustrated embodiment, reflectors can be moved in position between an IR cut filter and a camera. In an alternative embodiment, the emitters and touchscreen functionality make use of visible rather than IR spectrum. This allows an IR cut filter to be coupled to the camera; rather than to be separated from the camera by the reflector. Instead of emitters being co-located with the cameras, the bezel can be emissive (either IR or visible spectrum). In another embodiment, cameras are rotated according to mode; in this embodiment, no reflectors are required. These and other variations upon and modifications to the illustrated embodiment are provided by the present invention, the scope of which is defined by the following claims.