Source: http://www.google.com/patents/US8035612?dq=5527183
Timestamp: 2014-04-18 17:12:32
Document Index: 593405410

Matched Legal Cases: ['Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 06010825', 'Application No. 08830473', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 2002312346', 'Application No. 2', 'Application No. 02815206', 'Application No. 02815206', 'Application No. 02815206', 'Application No. 200480034551', 'Application No. 04', 'Application No. 1582', 'Application No. 10', 'Application No. 2006', 'Application No. 2008299883', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 2449300', 'Application No. 02739710', 'Application No. 02739710', 'Application No. 02739710']

Patent US8035612 - Self-contained interactive video display system - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA self-contained interactive video display system. A projector projects a visual image onto a screen for displaying the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen. An illuminator illuminates an...http://www.google.com/patents/US8035612?utm_source=gb-gplus-sharePatent US8035612 - Self-contained interactive video display systemAdvanced Patent SearchPublication numberUS8035612 B2Publication typeGrantApplication numberUS 10/946,263Publication dateOct 11, 2011Filing dateSep 20, 2004Priority dateMay 28, 2002Also published asEP1695196A2, US8035624, US20050162381, US20080150913, WO2005057921A2, WO2005057921A3Publication number10946263, 946263, US 8035612 B2, US 8035612B2, US-B2-8035612, US8035612 B2, US8035612B2InventorsMatthew Bell, Philip Gleckman, Joshua Zide, Helen ShaughnessyOriginal AssigneeIntellectual Ventures Holding 67 LlcExport CitationBiBTeX, EndNote, RefManPatent Citations (106), Non-Patent Citations (228), Referenced by (6), Classifications (16), Legal Events (5) External Links: USPTO, USPTO Assignment, EspacenetSelf-contained interactive video display systemUS 8035612 B2Abstract A self-contained interactive video display system. A projector projects a visual image onto a screen for displaying the visual image, wherein the projector projects the visual image onto a back side of the screen for presentation to a user on a front side of the screen. An illuminator illuminates an object near the front side of the screen. A camera detects interaction of an illuminated object with the visual image, wherein the screen is at least partially transparent to light detectable to the camera, allowing the camera to detect the illuminated object through the screen. A computer system directs the projector to change the visual image in response to the interaction.
projecting means for projecting visual images;
depicting means comprising a scattering polarizer for depicting visual images projected by the projecting means, wherein the projecting means is configured to emit polarized light in a polarization substantially scattered by the scattering polarizer;
sensor means sensitive to light of a polarization not substantially scattered by the scattering polarizer positioned on a first side of the depicting means for acquiring images representative of at least a portion of one or more objects positioned on a second side of the depicting means, wherein the first and second sides of the depicting means are substantially opposite sides of the depicting means; and
computing means for detecting an interaction of at least one of the one or more objects with at least a portion of at least one of the visual images projected by the projecting means.
2. The display system of claim 1, wherein the computing means is configured to update the visual images projected by the projecting means in response to the detected interaction.
3. The display system of claim 1, wherein the depicting means comprises a time-based material configured to change from substantially translucent to substantially transparent in response to a state of the projecting means.
4. The display system of claim 1, wherein the depicting means comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein the projecting means projects light onto the depicting means from at least the particular angle.
5. The display system of claim 1, wherein the depicting means comprises a screen that is at least partially transparent.
6. An interactive video display system comprising:
a projector configured to project a visual image;
a screen positioned so that the visual image is projected onto a back side of the screen, wherein the screen is at least partially transparent and comprises a material that is substantially translucent at a particular angle and substantially transparent at other angles, wherein the projector projects light onto the screen from the particular angle;
a camera configured to acquire images representative of at least a portion of an object near a front side of the screen that is opposite the back side of the screen; and
a computer system configured to receive the images from the camera and, based at least on the images, to detect an interaction of the object with at least a portion of the visual image and to direct the projector to change the visual image in response to the interaction.
7. A method of projecting visual images onto a screen, the method comprising:
projecting a visual image onto a first side of a screen, wherein the screen is at least partially transparent and configured for viewing from a second side of the screen opposite the first side of the screen;
acquiring images from an imaging device positioned on the first side of the screen, wherein the images are representative of at least a portion of an object positioned near the screen;
illuminating the object with strobed illumination that is synced with the imaging device; and
detecting an interaction of the object with at least a portion of the visual image based at least partly on the acquired images.
updating the visual image in response to the interaction.
9. An interactive video display system comprising:
a screen positioned so that the visual image is projected onto a back side of the screen, wherein the screen is at least partially transparent;
a camera configured to acquire images representative of at least a portion of an object near a front side of the screen that is opposite the back side of the screen;
an illuminator configured to illuminate the object, wherein the illumination is strobed in time with exposures of the camera; and
10. The interactive video display system as recited in claim 9, further comprising an illuminator configured to illuminate the object with infrared illumination, and wherein the camera is an infrared camera for detecting infrared illumination.
11. The interactive video display system as recited in claim 9 wherein linear polarizing films of perpendicular orientations are placed adjacent to the front side and the back side of the screen.
12. The interactive video display system as recited in claim 9 wherein the projector emits polarized light, and a linear polarizing film of perpendicular orientation is placed adjacent to the front side of the screen.
13. The interactive video display system as recited in claim 1 wherein the projector, the camera, the illuminator, and the computer system are comprised within an enclosure, and wherein one side of the enclosure comprises the screen.
14. The interactive video display system as recited in claim 9 further comprising a mirror for projecting the visual image onto the screen, wherein the projector is pointed indirectly at the screen and the mirror redirects the visual image to the screen.
15. The interactive video display system as recited in claim 1, wherein the illuminator projects illumination through the screen onto the object.
16. The interactive video display system as recited in claim 15, wherein the illuminator is positioned so as to reduce any potential for glare effects on the camera.
17. The interactive video display system as recited in claim 9 wherein the illuminator is located next to the screen such that the illuminator does not project illumination through the screen.
18. The interactive video display system as recited in claim 9 comprising a plurality of illuminators, wherein the illuminators are located next to the screen and behind the screen.
19. The interactive video display system as recited in claim 9 wherein the projector is an off-axis projector.
20. The interactive video display system as recited in claim 9 further comprising a Fresnel lens positioned adjacent to the screen to correct distortion of a view of the camera.
21. The interactive video display system as recited in claim 9, further comprising a series of mirror strips positioned at a distance from the screen to correct distortion of a view of the camera.
22. The interactive video display system as recited in claim 9 wherein the screen comprises a scattering polarizer, wherein the projector emits polarized light in a polarization substantially scattered by the scattering polarizer, and the camera is sensitive to light of a polarization not substantially scattered by the scattering polarizer.
23. The interactive video display system as recited in claim 9 wherein the screen comprises a material that substantially scatters light of wavelengths emitted by the projector, but does not substantially scatter wavelengths detectable to the camera.
24. The interactive video display system as recited in claim 9 wherein the screen comprises a time-based material that can change from substantially translucent to substantially transparent, wherein the screen is substantially translucent when the projector is projecting the visual image, and is substantially transparent when the camera is detecting objects near the front side of the screen.
25. The interactive video display system as recited in claim 9 wherein an image of the camera is calibrated to the visual image such that the interaction caused by the object is matched to a physical position of the object near to the screen.
26. The interactive video display system as recited in claim 9 wherein the interactive video display system is operable to determine information about a distance of the object from the screen.
27. The interactive video display system as recited in claim 26 wherein the camera is a stereo camera.
28. The interactive video display system as recited in claim 26 wherein the camera is a time-of-flight camera.
29. The interactive video display system as recited in claim 28 wherein the time-of-flight camera is positioned such that the time-of-flight camera does not reflect back onto itself.
30. The interactive video display system as recited in claim 1 wherein the interactive video display system provides touchscreen functionality when the object is proximate the screen.
31. The interactive video display system as recited in claim 30 further comprising a transparent touchscreen adjacent the front side of the screen.
32. The interactive video display system as recited in claim 30 further comprising an edge-lit transparent sheet adjacent the front side of the screen, and wherein the camera is operable to distinguish light created when the object comes in contact with the edge-lit transparent sheet.
33. The interactive video display system as recited in claim 9 wherein the object is a body part of a human user.
34. The interactive video display system as recited in claim 9 wherein the interactive video display system may be tiled with a second interactive video display system.
35. The interactive video display system as recited in claim 9 further comprising a second camera, a first Fresnel lens for the camera, and a second Fresnel lens for the second camera, allowing spaces viewed by the camera and the second camera to be stitched together.
CROSS-REFERENCE TO RELATED APPLICATION This application is a Continuation-in-Part Patent Application claiming priority from U.S. patent application Ser. No. 10/160,217, filed on May 28, 2002 now U.S. Pat. No. 7,259,747, entitled �INTERACTIVE VIDEO DISPLAY SYSTEM,� by Bell, and assigned to the assignee of the present application, which is herein incorporated by reference. This application also claims priority from U.S. Provisional Patent Application No. 60/504,375, filed on Sep. 18, 2003, entitled �SELF-CONTAINED INTERACTIVE VIDEO DISPLAY SYSTEM,� by Bell, and assigned to the assignee of the present application, from U.S. Provisional Patent Application No. 60/514,024, filed on Oct. 24, 2003, entitled �METHOD AND SYSTEM FOR PROCESSING CAPTURED IMAGE INFORMATION IN AN INTERACTIVE VIDEO SYSTEM,� by Bell, and assigned to the assignee of the present application, from U.S. Provisional Patent Application No. 60/528,439, filed on Dec. 9, 2003, entitled �SELF-CONTAINED INTERACTIVE VIDEO DISPLAY SYSTEM AND FEATURES RELATING THERETO,� by Bell, and assigned to the assignee of the present application, and from U.S. Provisional Patent Application No. 60/554,520, filed on Mar. 18, 2004, entitled �METHOD AND SYSTEM FOR ALLOWING A CAMERA TO VIEW AN AREA IN FRONT OF A DISPLAY BY IMAGING IT THROUGH THE DISPLAY,� by Bell et al., and assigned to the assignee of the present application, all of which are herein incorporated by reference.
Computer 110 processes the camera 115 input to discern on a pixel-by-pixel basis what portions of the volume in front of screen 130 are occupied by people (or moving objects) and what portions of screen 130 are background. Computer 110 accomplishes this by developing several evolving models of what the background is supposed to look like, and then comparing its concepts of the background to what camera 115 is currently seeing. Alternatively, components of computer 110 that process camera 115 input are collectively known as the vision system. Various embodiments of this vision system are described in co-pending U.S. patent application Ser. No. 10/160,217, filed on May 28, 2002, entitled �INTERACTIVE VIDEO DISPLAY SYSTEM,� by Bell, and assigned to the assignee of the present application, in U.S. Provisional Patent Application No. 60/504,375, filed on Sep. 18, 2003, entitled �SELF-CONTAINED INTERACTIVE DISPLAY SYSTEM,� by Bell, and assigned to the assignee of the present application, and in U.S. Provisional Patent Application No. 60/514,024, filed on Oct. 24, 2003, entitled �METHOD AND SYSTEM FOR PROCESSING CAPTURED IMAGE INFORMATION IN AN INTERACTIVE VIDEO SYSTEM,� by Bell, and assigned to the assignee of the present application, all of which are herein incorporated by reference.
In another embodiment, the illuminators can be placed in front of the screen in a way that allows them to easily illuminate all locations in front of the screen. One configuration, in which the illuminator protrudes from the front of the screen, is shown in FIG. 5A; another configuration, in which the display surface is recessed, is shown in FIG. 5B. In the embodiments described in FIGS. 4A, 4B, 5A and 5B, the illuminators may be placed at regular intervals around the screen, in a continuous line, or at strategic locations. These illumination strategies shown in FIGS. 4A, 4B, 5A, and 5B can also be combined with illuminators that are behind the screen and shine through the screen.
In certain instances, while small particle scattering has a λ−4 wavelength dependence for single-particle scattering, most diffusers employ multiple scattering to effect adequate diffusion. In multiple scattering the wavelength dependence of the scattered light is far more neutral, as is demonstrated by the color of milk. The intensity of light coherently scattered by small particles is also known to be proportional to (n−1)2, where n is the relative index of refraction between the particle and the host matrix. In the present invention it is this property of scattering that is exploited to allow transparency in the capture channel and haze in the display channel. The method used here is compatible with but is not limited to the use of IR in the capture channel. The dispersion of normal polymer materials is inadequate to create contrast between the visible and near IR. Instead, we label the capture channel with one state of polarization state and the display channel with the orthogonal state. We also employ a screen which has the property of index matching (n=1) between the matrix and the dispersed particles for one state of polarization and index mismatching (n≠1) for the orthogonal state. In this way, the screen will have substantial haze for the display channel and substantial transparency in the capture channel. Materials may be tuned to effect a near-perfect index match at the capture channel which may have a very narrow spectrum (20 nm or so). Two primary metrics can be used to define the performance of this type of screen: the single piece transmission (Tsp) and the polarizer efficiency (PE). These quantities are defined as follows in Equations 1 and 2:
T sp=(T ∥ +T ⊥)/2 (1) PE=|(T∥ −T ⊥)/(T ∥ +T ⊥)| (2)
Multi-User Touchscreens and 3D Data; Brightness Ratios
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS2917980Dec 30, 1955Dec 22, 1959Mergenthaler Linotype GmbhLenslet assembly for photocomposing machinesUS3068754Jul 30, 1958Dec 18, 1962Corning Giass WorksPrismatic light transmitting panelUS3763468Oct 1, 1971Oct 2, 1973Energy Conversion Devices IncLight emitting display array with non-volatile memoryUS4053208Feb 3, 1976Oct 11, 1977Fuji Photo Film Co., Ltd.Rear projection screensUS4275395Sep 28, 1979Jun 23, 1981International Business Machines CorporationInteractive projection display systemUS4573191Mar 29, 1984Feb 25, 1986Tokyo Shibaura Denki Kabushiki KaishaStereoscopic vision systemUS4725863Feb 20, 1987Feb 16, 1988United Kingdom Atomic Energy AuthorityStereo cameraUS4843568Apr 11, 1986Jun 27, 1989Krueger Myron WReal time perception of and response to the actions of an unencumbered participant/userUS4887898Apr 13, 1988Dec 19, 1989Rowe Furniture CorporationFabric projection systemUS4948371 *Apr 25, 1989Aug 14, 1990The United States Of America As Represented By The United States Department Of EnergySystem for training and evaluation of security personnel in use of firearmsUS5001558Jun 11, 1985Mar 19, 1991General Motors CorporationNight vision system with color video cameraUS5138304Aug 2, 1990Aug 11, 1992Hewlett-Packard CompanyProjected image light penUS5151718Dec 31, 1990Sep 29, 1992Texas Instruments IncorporatedSystem and method for solid state illumination for dmd devicesUS5239373Dec 26, 1990Aug 24, 1993Xerox CorporationVideo computational shared drawing spaceUS5276609 *May 28, 1991Jan 4, 1994Durlach David M3-D amusement and display deviceUS5319496Nov 18, 1992Jun 7, 1994Photonics Research IncorporatedOptical beam delivery systemUS5325472Apr 12, 1991Jun 28, 1994Matsushita Electric Industrial Co., Ltd.Image displaying system for interactively changing the positions of a view vector and a viewpoint in a 3-dimensional spaceUS5325473Oct 11, 1991Jun 28, 1994The Walt Disney CompanyApparatus and method for projection upon a three-dimensional objectUS5426474Mar 22, 1994Jun 20, 1995Innersense, Inc.Light projection system using randomized fiber optic bundleUS5436639Mar 15, 1994Jul 25, 1995Hitachi, Ltd.Information processing systemUS5442252Nov 16, 1992Aug 15, 1995General Electric CompanyLenticulated lens with improved light distributionUS5454043Jul 30, 1993Sep 26, 1995Mitsubishi Electric Research Laboratories, Inc.Dynamic and static hand gesture recognition through low-level image analysisUS5497269Jun 25, 1992Mar 5, 1996Lockheed Missiles And Space Company, Inc.Dispersive microlensUS5510828Mar 1, 1994Apr 23, 1996Lutterbach; R. StevenInteractive video display systemUS5526182Feb 17, 1993Jun 11, 1996Vixel CorporationMultiple beam optical memory systemUS5528263Jun 15, 1994Jun 18, 1996Daniel M. PlatzkerInteractive projected video image display systemUS5528297Aug 25, 1994Jun 18, 1996Deutsche Thomson-Brant GmbhConvertible video camera/projectorUS5534917May 9, 1991Jul 9, 1996Very Vivid, Inc.Video image based control systemUS5548694Jan 31, 1995Aug 20, 1996Mitsubishi Electric Information Technology Center America, Inc.Collision avoidance system for voxel-based object representationUS5591972Aug 3, 1995Jan 7, 1997Illumination Technologies, Inc.Apparatus for reading optical informationUS5594469Feb 21, 1995Jan 14, 1997Mitsubishi Electric Information Technology Center America Inc.Hand gesture machine control systemUS5633691Jun 7, 1995May 27, 1997Nview CorporationStylus position sensing and digital camera with a digital micromirror deviceUS5703637Oct 26, 1994Dec 30, 1997Kinseki LimitedRetina direct display device and television receiver using the sameUS5808784Sep 5, 1995Sep 15, 1998Dai Nippon Printing Co., Ltd.Lens array sheet surface light source, and transmission type display deviceUS5861881Feb 8, 1996Jan 19, 1999Actv, Inc.Interactive computer system for providing an interactive presentation with personalized video, audio and graphics responses for multiple viewersUS5882204Jul 13, 1995Mar 16, 1999James W. LacyFootball interactive simulation trainerUS5923380Jan 25, 1996Jul 13, 1999Polaroid CorporationMethod for replacing the background of an imageUS5923475Nov 27, 1996Jul 13, 1999Eastman Kodak CompanyLaser printer using a fly's eye integratorUS5953152Sep 23, 1997Sep 14, 1999Light & Sound Design, Ltd.Programmable light beam shape altering device using programmable micromirrorsUS5969754Dec 9, 1996Oct 19, 1999Zeman; Herbert D.Contrast enhancing illuminatorUS5978136Dec 17, 1997Nov 2, 1999Seiko Epson CorporationOptical element, polarization illumination device, and projection display apparatusUS5982352Jun 29, 1995Nov 9, 1999Pryor; Timothy R.Method for providing human input to a computerUS6008800Aug 15, 1994Dec 28, 1999Pryor; Timothy R.Man machine interfaces for entering data into a computerUS6058397Apr 8, 1997May 2, 2000Mitsubishi Electric Information Technology Center America, Inc.3D virtual environment creation management and delivery systemUS6075895Jun 20, 1997Jun 13, 2000HoloplexMethods and apparatus for gesture recognition based on templatesUS6084979Jun 20, 1996Jul 4, 2000Carnegie Mellon UniversityMethod for creating virtual realityUS6088612Apr 4, 1997Jul 11, 2000Medtech Research CorporationMethod and apparatus for reflective glare removal in digital photography useful in cervical cancer detectionUS6097369Feb 2, 1995Aug 1, 2000Wambach; Mark L.Computer mouse gloveUS6106119Nov 9, 1999Aug 22, 2000The Board Of Trustees Of The Leland Stanford Junior UniversityMethod for presenting high level interpretations of eye tracking data correlated to saved display imagesUS6118888Feb 25, 1998Sep 12, 2000Kabushiki Kaisha ToshibaMulti-modal interface apparatus and methodUS6125198Nov 19, 1998Sep 26, 2000Matsushita Electric Industrial Co., Ltd.Method of matching stereo images and method of measuring disparity between these itemsUS6166744Sep 15, 1998Dec 26, 2000Pathfinder Systems, Inc.System for combining virtual images with real-world scenesUS6176782Dec 22, 1997Jan 23, 2001Philips Electronics North America Corp.Motion-based command generation technologyUS6191773Apr 25, 1996Feb 20, 2001Matsushita Electric Industrial Co., Ltd.Interface apparatusUS6198487Feb 26, 1999Mar 6, 2001Intergraph CorporationOle for design and modelingUS6198844Jan 27, 1999Mar 6, 2001Konica CorporationImage processing apparatusUS6263339Aug 25, 1998Jul 17, 2001Informix Software, Inc.Dynamic object visualization and code generationUS6292171 *Mar 31, 1999Sep 18, 2001Seiko Epson CorporationMethod and apparatus for calibrating a computer-generated projected imageUS6308565Oct 15, 1998Oct 30, 2001Impulse Technology Ltd.System and method for tracking and assessing movement skills in multidimensional spaceUS6323895Jun 10, 1998Nov 27, 2001Namco Ltd.Image generating system and information storage medium capable of changing viewpoint or line-of sight direction of virtual camera for enabling player to see two objects without interpositionUS6333735May 24, 1999Dec 25, 2001International Business Machines CorporationMethod and apparatus for mouse positioning device based on infrared light sources and detectorsUS6335977May 28, 1998Jan 1, 2002Mitsubishi Denki Kabushiki KaishaAction recognizing apparatus and recording medium in that action recognizing program is recordedUS6339748 *Nov 5, 1998Jan 15, 2002Seiko Epson CorporationCoordinate input system and display apparatusUS6349301Feb 24, 1998Feb 19, 2002Microsoft CorporationVirtual environment bystander updating in client server architectureUS6353428Feb 10, 1998Mar 5, 2002Siemens AktiengesellschaftMethod and device for detecting an object in an area radiated by waves in the invisible spectral rangeUS6359612Sep 29, 1999Mar 19, 2002Siemens AktiengesellschaftImaging system for displaying image information that has been acquired by means of a medical diagnostic imaging deviceUS6388657Dec 31, 1998May 14, 2002Anthony James Francis NatoliVirtual reality keyboard system and methodUS6400374Sep 18, 1996Jun 4, 2002Eyematic Interfaces, Inc.Video superposition system and methodUS6407870Oct 28, 1999Jun 18, 2002Ihar HurevichOptical beam shaper and method for spatial redistribution of inhomogeneous beamUS6414672Jul 6, 1998Jul 2, 2002Sony CorporationInformation input apparatusUS6445815Mar 12, 1999Sep 3, 2002Canon Kabushiki KaishaMeasurement of depth image considering time delayUS6454419Mar 19, 2001Sep 24, 2002Seiko Epson CorporationIndicated position detection by multiple resolution image analysisUS6480267Dec 26, 2000Nov 12, 2002Kabushiki Kaisha TopconWavefront sensor, and lens meter and active optical reflecting telescope using the sameUS6491396Feb 14, 2001Dec 10, 2002Seiko Epson CorporationProjector modulating a plurality of partial luminous fluxes according to imaging information by means of an electro-optical deviceUS6501515Oct 13, 1998Dec 31, 2002Sony CorporationRemote control systemUS6522312Mar 23, 1998Feb 18, 2003Canon Kabushiki KaishaApparatus for presenting mixed reality shared among operatorsUS6545706Jul 30, 1999Apr 8, 2003Electric Planet, Inc.System, method and article of manufacture for tracking a head of a camera-generated image of a personUS6552760Feb 17, 2000Apr 22, 2003Fujitsu LimitedLuminaire with improved light utilization efficiencyUS6598978Jul 16, 2001Jul 29, 2003Canon Kabushiki KaishaImage display system, image display method, storage medium, and computer programUS6607275Mar 20, 2002Aug 19, 2003The Neiman Marcus Group, Inc.Merchandise display case and systemUS6611241Nov 25, 1998Aug 26, 2003Sarnoff CorporationModular display systemUS6654734Aug 30, 2000Nov 25, 2003International Business Machines CorporationSystem and method for query processing and optimization for XML repositoriesUS6658150Dec 1, 2000Dec 2, 2003Honda Giken Kogyo Kabushiki KaishaImage recognition systemUS6661918Dec 3, 1999Dec 9, 2003Interval Research CorporationBackground estimation and segmentation based on range and colorUS6677969Sep 24, 1999Jan 13, 2004Sanyo Electric Co., Ltd.Instruction recognition system having gesture recognition functionUS6707054Mar 21, 2002Mar 16, 2004Eastman Kodak CompanyScannerless range imaging system having high dynamic rangeUS6707444Aug 18, 2000Mar 16, 2004International Business Machines CorporationProjector and camera arrangement with shared optics and optical marker for use with whiteboard systemsUS6712476Nov 17, 2000Mar 30, 2004Seiko Epson CorporationProjection display apparatus and method of displayUS6720949 *Aug 21, 1998Apr 13, 2004Timothy R. PryorMan machine interfaces and applicationsUS6732929Feb 1, 2002May 11, 2004Metrologic Instruments, Inc.Led-based planar light illumination beam generation module employing a focal lens for reducing the image size of the light emmiting surface of the led prior to beam collimation and planarizationUS6747666Jan 26, 2001Jun 8, 2004New York UniversityMethod and system for facilitating wireless, full-body, real-time user interaction with digitally generated text dataUS6752720Jun 15, 2000Jun 22, 2004Intel CorporationMobile remote control video gaming systemUS6754370Aug 14, 2000Jun 22, 2004The Board Of Trustees Of The Leland Stanford Junior UniversityReal-time structured light range scanning of moving scenesUS6791700May 30, 2003Sep 14, 2004Ricoh Company, Ltd.Coordinate inputting/detecting apparatus, method and computer program product designed to precisely recognize a designating state of a designating device designating a positionUS6826727Nov 24, 1999Nov 30, 2004Bitstream Inc.Apparatus, methods, programming for automatically laying out documentsUS6831664Mar 22, 2002Dec 14, 2004Koninklijke Philips Electronics N.V.Low cost interactive program control system and methodUS6871982Jan 22, 2004Mar 29, 2005Digital Optics International CorporationHigh-density illumination systemUS6877882Mar 12, 2003Apr 12, 2005Delta Electronics, Inc.Illumination system for a projection systemUS6912313May 31, 2001Jun 28, 2005Sharp Laboratories Of America, Inc.Image background replacement methodUS6965693Aug 21, 2000Nov 15, 2005Sony CorporationImage processor, image processing method, and recorded mediumUS6971700Jul 2, 2004Dec 6, 2005Grupo Antolin-Ingenieria, S.A.Motor vehicle seatUS6975360Dec 1, 2000Dec 13, 2005Hewlett-Packard Development Company, L.P.Image detector method and apparatus including plural detector regions and image illuminatorsUS6999600Jan 30, 2003Feb 14, 2006Objectvideo, Inc.Video scene background maintenance using change detection and classificationUS7015894Sep 27, 2002Mar 21, 2006Ricoh Company, Ltd.Information input and output system, method, storage medium, and carrier waveUS20010012001 *Jul 6, 1998Aug 9, 2001Junichi RekimotoInformation input apparatusUS20020041327 *Jul 23, 2001Apr 11, 2002Evan HildrethVideo-based image control system* Cited by examinerNon-Patent CitationsReference1"EffecTV" Version 0.2.0 released Mar. 27, 2001, available online at .2"Index of EffecTV", as downloaded on Apr. 30, 2007 at .3"EffecTV" Version 0.2.0 released Mar. 27, 2001, available online at <http://web.archive.org/web/20010101-20010625re�http://effectv.sourceforge.net>.4"Index of EffecTV", as downloaded on Apr. 30, 2007 at <http://effectv.cvs.sourceforge.net/effectv/EffecTV/?pathrev=rel�0�2�0>.52001 Symposium on Interactive 3D Graphics program description, ACM SIGGRAPH, held Mar. 19-21, 2001, Research Triangle Park, NC, downloaded from ; cited during opposition of European Application No. 02739710.8 filed Jun. 4, 2002.62001 Symposium on Interactive 3D Graphics program description, ACM SIGGRAPH, held Mar. 19-21, 2001, Research Triangle Park, NC, downloaded from <http://www.allconferences.com/conferences/2000830092631/>; cited during opposition of European Application No. 02739710.8 filed Jun. 4, 2002.7Affidavit of Daniel Barthels regarding EffecTV, dated May 15, 2007 (partial machine translation), cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.8Announcement: Workshop on Perceptual User Interfaces, The Banff Rocky Mountain Resort, Banff, Alberta, Canada, Oct. 20-21, 1997, can be found at , cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.9Announcement: Workshop on Perceptual User Interfaces, The Banff Rocky Mountain Resort, Banff, Alberta, Canada, Oct. 20-21, 1997, can be found at <http://www.research.microsoft.com/PUIWorkshop/>, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.10ART+COM Bodymover 2000, as downloaded on Aug. 21, 2009 from <http://www.artcom.de/index.php?option=com-acprojects&page=6&id=28&Itemid=144&details=0&lang=en>, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.11ART+COM Bodymover 2000, as downloaded on Aug. 21, 2009 from <http://www.artcom.de/index.php?option=com�acprojects&page=6&id=28&Itemid=144&details=0&lang=en>, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.12Article 96(2) Communication dated Feb. 25, 2005 in European Application No. 02739710.8.13Article 96(2) Communication dated Mar. 31, 2004 in European Application No. 02739710.8.14Bodymover Body Movement as a Means to Obtain an Audiovisual Spatial Experience, 2000 ART+COM AG Berlin; <http://www.artcom.de/index.php?option=com-acprojects&page=6&id=28&Itemid=144& details=0&lang=en>, as downloaded on Aug. 8, 2005.15Bodymover Body Movement as a Means to Obtain an Audiovisual Spatial Experience, 2000 ART+COM AG Berlin; <http://www.artcom.de/index.php?option=com�acprojects&page=6&id=28&Itemid=144& details=0&lang=en>, as downloaded on Aug. 8, 2005.16Brown, Matthew, et al. "Multi-Image Matching using Multi-Scale Oriented Patches," Technical Report, Dec. 2004, pp. 1-48, available online at .17Brown, Matthew, et al. "Multi-Image Matching using Multi-Scale Oriented Patches," Technical Report, Dec. 2004, pp. 1-48, available online at <ftp://ftp.research.microsoft.com/pub/tr/TR-2004-133.pdf>.18Brown, Matthew, et al., "Multi-Image Matching using Multi-Scale Oriented Patches," Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Conference Publication Date: Jun. 20-25, 2005, 8 pgs.19Buxton, Bill, "Multi-Touch Systems That I Have Known and Loved," accessed Mar. 21, 2007, http://billbuxton.com/multitouchOverview.html.20Communication dated Dec. 10, 2008 from Patentanwalt attaching article by Katy Bachman, entitled "Reactrix Up for Sale," cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.21Crouser, P.D., et al., "Unattenuated tracer particle extraction through time-averaged, background image subtraction with outlier rejection," Experiments in Fluids, 22, 1997, 220-228, Springer-Verlag.22Dachselt, Raimund, et al., "CONTIGRA: An XML-Based Architecture for Component-Oriented 3D Applications, 3D Technologies for the World Wide Web, Proceedings of the Seventh International Conference on 3D Technology," ACM, Feb. 24-28, 2002, pp. 155-163.23Davis, J.W., et al., "SIDEshow: A Silhouette-based Interactive Dual-screen Environment," Aug. 1998, MIT Media Lab Tech Report No. 457.24Decision revoking the European Patent in European Application No. 02739710.8 dated Dec. 28, 2009.25Demarest, Ken, "Sand," 2000, Mine Control, art installation, available online at , downloaded on Mar. 15, 2007.26Demarest, Ken, "Sand," 2000, Mine Control, art installation, available online at <http://www.mine-control.com>, downloaded on Mar. 15, 2007.27EffecTV Software Source: effect module, dated May 20, 2001 (German); cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.28Elgammal, Ahmed, et al., "Non-parametric Model for Background Subtraction," Jun. 2000, European Conference on Computer Vision, Lecture Notes on Computer Science, vol. 1843, pp. 751-767.29Examiner Interview Summary in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated Jul. 28, 2009.30Examiner Interview Summary Record in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Aug. 15, 2006.31Examiner Interview Summary Record in U.S. Appl. No. 10/866,495, dated Nov. 18, 2009.32Examiner Interview Summary Record in U.S. Appl. No. 10/946,084, dated Aug. 27, 2010.33Examiner Interview Summary Record in U.S. Appl. No. 11/929,778, dated Sep. 22, 2010.34Examiner Interview Summary Record in U.S. Appl. No. 11/929,947, dated Dec. 14, 2010.35Examiner Interview Summary Record in U.S. Appl. No. 11/982,290, dated Aug. 5, 2011.36Examiner Interview Summary Record in U.S. Appl. No. 11/982,290, dated Mar. 11, 2011.37Experience Microsoft Surface, downloaded from http://www.microsoft.com/surface/Pages/Product/Specifications.aspx on Sep. 24, 2009.38Extended Search Report for European Application No. 06010825.5, filed Jun. 4, 2002, dated Jul. 10, 2006.39Extended Search Report for European Application No. 08830473.8, dated Apr. 19, 2011.40Final Rejection in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated May 26, 2009.41Final Rejection in U.S. Appl. No. 11/197,941, dated Mar. 5, 2007.42Final Rejection in U.S. Appl. No. 11/507,976, dated May 13, 2008.43Final Rejection in U.S. Appl. No. 11/982,290, dated Jul. 19, 2011.44Foerterer, Holger, "Fluidum," 1999, art installation, description available online at , downloaded on Mar. 16, 2007.45Foerterer, Holger, "Helikopter," 2001, art installation, description available online at http://www.foerterer.com/helikopter/index.htm>, downloaded on Mar. 16, 2007.46Foerterer, Holger, "Fluidum," 1999, art installation, description available online at <http://www.foerterer.com/fluidum>, downloaded on Mar. 16, 2007.47Freeman, William, et al., "Computer vision for interactive computer graphics," May-Jun. 1998, IEEE Computer Graphics and Applications, vol. 18, No. 3, pp. 42-53.48Frisken, Sarah F. et al., "Adaptively Sampled Distance Fields: A General Representation of Shape for Computer Graphics," Jul. 23-28, 2000, Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 249-254.49Fujihata, Masaki, "Beyond Pages," 1995, art installation, description available online at , downloaded on Mar. 16, 2007.50Fujihata, Masaki, "Beyond Pages," 1995, art installation, description available online at <http://on1.zkm.de/zkm/werke/BeyondPages>, downloaded on Mar. 16, 2007.51Goetz, Frank, et al., "An XML-based Visual Shading Language for Vertex and Fragment Shaders," 3D Technologies for the World Wide Web, Proceedings of Ninth International Conference on 3D Technology; ACM, Apr. 5-8, 2004; pp. 87-97.52GroundFX Document, GestureTek (Very Vivid, Inc.), description available online at , downloaded on Aug. 11, 2006.53GroundFX Document, GestureTek (Very Vivid, Inc.), description available online at <http://www.gesturetek.com/groundfx>, downloaded on Aug. 11, 2006.54Haller, Michael et al., "Coeno-Storyboard: An Augmented Surface for Storyboard Presentations," Mensch & Computer 2005, Sep. 4-7, 2005, Linz, Austria.55Han, Jefferson Y., "Low-Cost Multi-Touch Sensing Through Frustrated Total Internal Reflection," Oct. 23-26, 2005, ACM Symposium on User Interface Software and Technology (UIST).56Harville, Michael et al., "Foreground Segmentation Using Adaptive Mixture Models in Color and Depth," Jul. 8, 2001, Proceedings of IEEE Workshop on Detection and Recognition of Events in Video, pp. 3-11.57Hemmer, Raphael Lozano, "Body Movies," 2002, art project/installation, description available online at <http://www.lozano-hemmer.com/eproyecto.html>, downloaded on Mar. 16, 2007.58Hoff, Kenneth E. III et al, "Fast and Simple 2D Geometric Proximity Queries Using Graphics Hardware," Mar. 19-21, 2001, Proc. of the 2001 Symposium on Interactive 3D Graphics, pp. 145-148.59International Preliminary Examination Report for PCT/US2002/017843, filed Jun. 4, 2002.60International Preliminary Examination Report on Patentability for PCT/US2004/035477, filed Oct. 25, 2004.61International Preliminary Report on Patentability for PCT/US2004/035478, filed Oct. 25, 2004.62International Preliminary Report on Patentability for PCT/US2004/041318, filed Dec. 9, 2004.63International Preliminary Report on Patentability for PCT/US2004/041319, filed Dec. 9, 2004.64International Preliminary Report on Patentability for PCT/US2004/041320, filed Dec. 9, 2004.65International Preliminary Report on Patentability for PCT/US2005/008984, filed Mar. 18, 2005.66International Preliminary Report on Patentability for PCT/US2006/030720, filed on Aug. 4, 2006.67International Preliminary Report on Patentability for PCT/US2008/059900, filed on Apr. 10, 2008.68International Preliminary Report on Patentability for PCT/US2008/10750, filed Sep. 15, 2008.69International Search Report for PCT/US03/40321, filed Dec. 15, 2003.70International Search Report for PCT/US2002/017843, filed Jun. 4, 2002.71International Search Report for PCT/US2004/035477, filed Oct. 25, 2004.72International Search Report for PCT/US2004/035478, filed Oct. 25, 2004.73International Search Report for PCT/US2004/041318, filed Dec. 9, 2004.74International Search Report for PCT/US2004/041319, filed Dec. 9, 2004.75International Search Report for PCT/US2004/041320, filed Dec. 9, 2004.76International Search Report for PCT/US2005/008984, filed Mar. 18, 2005.77International Search Report for PCT/US2006/030720, filed Aug. 4, 2006.78International Search Report for PCT/US2008/059900, filed Apr. 10, 2008.79International Search Report for PCT/US2008/10750, filed Sep. 15, 2008.80Invitation to Pay Additional Fees and Partial ternational Search Report on Patentability for PCT/US2004/035478, filed Oct. 25, 2004.81Ivars Peterson, "Artificial reality; combining a person's live video image with computer graphics suggests novel ways of working and playing with computers" Science News, Jun. 22, 1985.82Jabri, Sumer et al., "Detection and Location of People in Video Images Using Adaptive Fusion of Color and Edge Information;" presented at the Int. Conf. Pattern Recognition, Barcelona, Spain, 2000.83Joyce, Arthur W. III, et al., "Implementation and capabilities of a virtual interaction system," Sep. 10-11, 1998, Proceedings 2nd European Conference on Disability, Virtual Reality and Associated Technologies, Skovde, Sweden, pp. 237-245.84Katz Itai et al., "A Multi-Touch Surface Using Multiple Cameras," Oct. 3, 2007, Advanced Concepts for Intelligent Vision Systems, vol. 4678/2007.85Keays, Bill, "Using High-Bandwidth Input/Output in Interactive Art," Jun. 1999, Master's Thesis, Massachusetts Institute of Technology, School of Architecture and Planning.86Keays, Bill, "metaField Maze," 1998, exhibited at Siggraph'99 Emerging Technologies and Ars Electronica Aug. 8-13, 1999, description available online at <http://www.billkeays.com/metaFieldInfosheet1A.pdf>.87Khan, Jeff; "Intelligent Room with a View"; Apr.-May 2004, RealTime Arts Magazine, Issue 60, available online at www.realtimearts.net/article/60/7432.88Kjeldesn, Rick et al., "Dynamically Reconfigurable Vision-Based User Interfaces," Apr. 2003, 3rd International Conference on Vision Systems (ICVVS '03), Graz, Austria, pp. 6-12.89Kjeldsen, R. et al., "Interacting with Steerable Projected Displays," May 20-21, 2002, Proceedings of the 5th International Conference on Automatic Face and Gesture Recognition, Washington, D.C.90Kreuger, Myron, "Videoplace-An Artificial Reality," Apr. 1985, Conference on Human Factors in Computing Systems, San Francisco, California, pp. 35-40.91Kreuger, Myron, "Videoplace," 1969 and subsequent, summary available online at <http://www.jtnimoy.com/itp/newmediahistory/videoplace>, downloaded Mar. 21, 2007.92Kreuger, Myron, "Videoplace�An Artificial Reality," Apr. 1985, Conference on Human Factors in Computing Systems, San Francisco, California, pp. 35-40.93Kurapati, Kaushal, et al., "A Multi-Agent TV Recommender," Jul. 13-14, 2001, Workshop on Personalization in Future TV, pp. 1-8, XP02228335.94Lamarre, Mathieu, et al., "Background subtraction using competing models in the block-DCT domain," Pattern Recognition, 2002 Proceedings, 16 International Conference in Quebec City, Que., Canada, Aug. 11-15, 2002, Los Alamitos, CA, USA, IEEE Comput Soc US.95Lantagne, Michel, et al., "VIP: Vision tool for comparing Images of People," Vision Interface, Jun. 11-13, 2003, pp. 1-8.96Leibe B, et al: Toward Spontaneous Interaction With the Perceptive Workbench IEEE Computer Graphics and Applications, IEEE Inc. New Your, US, vol. 20, No. 6 Nov. 2000, pp. 54-64, XP000969594, ISSN: 0272-1716 the whole document.97Lengyel, Jed et al., "Real-Time Robot Motion Planning Using Rasterizing Computer Graphics Hardware," Aug. 1990, ACM SIGGRAPH Computer Graphics, vol. 24, Issue 4, pp. 327-335.98Letter dated May 16, 2007 from Christian Zuckschwerdt regarding EffecTV, (partial machine translation), cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.99Letter of the opponent O2 dated May 28, 2010 in European Application No. 02739710.8, filed Jun. 4, 2002.100Levin, Golan "Computer Vision for Artists and Designers: Pedagogic Tools and Techniques for Novice Programmers," Aug. 2006, AI & Society, vol. 20, Issue 4, pp. 462-482.101Lin, Mingxiu et al., "A New Approach for Vision-based Rear Vehicle Tracking," Key Laboratory of Integrated Automation of Process Industry, Ministry of Education, Northeastern University, Shenyang, Liaoning Province, China, held May 23-25, 2007, pp. 107-111.102Livingston, Mark Alan, "Vision-based Tracking with Dynamic Structured Light for Video See-through Augmented Reality," 1998, Ph.D. Dissertation, University of North Carolina at Chapel Hill, cited on Jun. 18, 2007 during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.103MacIver, Malcolm, et al., "Body Electric," Apr. 15-Jun. 29, 2003, art installation, description available online at <http://www.neuromech.northwestern.edu/uropatagium/#ArtSci>.104Malik, Shahzad et al., "Visual Touchpad: A Two-Handed Gestural Input Device," Oct. 13-15, 2004, International Conference on Multimodal Interfaces (ICMI '04).105Mandala Systems, "Video Gesture Control System Concept," 1986, description available online at <http://www.vividgroup.com/tech.html>, downloaded on Mar. 16, 2007.106Maria Langer, "Mac OS X 10.2: Visual QuickStart Guide," Sep. 17, 2002, Peachpit Press, P. 111.107Microsoft Surface multi-touch interface table unveiled, May 30, 2007, downloaded from http://www.dancewithshadows.com/tech/microsoft-surface.asp.108Microsoft Surface Web Page, downloaded from http://www.microsoft.com/surface/Pages/Product/WhatIs.aspx on Sep. 24, 2009.109Microsoft Surface, downloaded from http://en.wikipedia.org/wiki/Microsoft-surface on Sep. 24, 2009.110Microsoft Surface, downloaded from http://en.wikipedia.org/wiki/Microsoft�surface on Sep. 24, 2009.111Mitsubishi DiamondTouch, http://www.merl.com/projects/DiamondTouch/, visited Mar. 21, 2007.112Mo, Zhenyao "SmartCanvas: A Gesture-Driven Intelligent Drawing Desk System," Jan. 9-12, 2005, Proceedings of Intelligent User Interfaces (IUI '05).113Morano, Raymond A. et al., "Structured Light Using Pseudorandom Codes," Mar. 1998, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, No. 3.114Morris, T. et al., "Real-Time Fingertip Detection for Hand Gesture Recognition," Sep. 9-11, 2002, Advanced Concepts for Intelligent Vision Systems (ACIVS '04), Ghent University, Belgium.115Muench, Wolfgang, "Bubbles", Prix Ars Electonica Catalog 1999, Springer-Verlag, Berlin, Germany; available online at <http://hosting.zkm.de/wmuench/bub/text>, downloaded on Mar. 16, 2007.116Non-Final Rejection in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated Aug. 6, 2008.117Non-Final Rejection in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated Jan. 29, 2008.118Non-Final Rejection in U.S. Appl. No. 11/083,851, dated Mar. 8, 2006.119Non-Final Rejection in U.S. Appl. No. 11/101,900, dated Aug. 6, 2007.120Non-Final Rejection in U.S. Appl. No. 11/197,941, dated Mar. 8, 2006.121Non-Final Rejection in U.S. Appl. No. 11/197,941, dated Sep. 6, 2006.122Non-Final Rejection in U.S. Appl. No. 11/507,976, dated Oct. 29, 2008.123Non-Final Rejection in U.S. Appl. No. 11/507,976, dated Sep. 17, 2007.124Non-Final Rejection in U.S. Appl. No. 11/929,778, dated Jul. 27, 2010.125Non-Final Rejection in U.S. Appl. No. 11/929,947, dated Oct. 21, 2010.126Non-Final Rejection in U.S. Appl. No. 11/982,290, dated Feb. 15, 2011.127Notice of Allowance in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Jun. 26, 2007.128Notice of Allowance in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Oct. 18, 2006.129Notice of Allowance in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated Feb. 22, 2010.130Notice of Allowance in U.S. Appl. No. 10/946,414 (USPN 7,710,391), dated Nov. 17, 2009.131Notice of Allowance in U.S. Appl. No. 11/083,851, dated Sep. 12, 2006.132Notice of Allowance in U.S. Appl. No. 11/197,941, dated Oct. 26, 2007.133Notice of Allowance in U.S. Appl. No. 11/507,976, dated Jan. 20, 2010.134Notice of Allowance in U.S. Appl. No. 11/507,976, dated Jul. 13, 2010.135Notice of Allowance in U.S. Appl. No. 11/507,976, dated Sep. 9, 2009.136Notice of Allowance in U.S. Appl. No. 11/929,778, dated Jan. 3, 2011.137Notice of Allowance in U.S. Appl. No. 11/929,778, dated Jul. 29, 2011.138Notice of Allowance in U.S. Appl. No. 11/929,778, dated May 24, 2011.139Notice of Allowance in U.S. Appl. No. 11/929,947, dated Apr. 1, 2011.140Notice of Allowance in U.S. Appl. No. 11/929,947, dated Jun. 30, 2011.141Notice of Opposition in European Application No. 02739710.8 dated Aug. 23, 2010.142Notice of Opposition in European Application No. 02739710.8 dated May 14, 2007.143Observation by third party Michael Saup dated Jan. 17, 2005, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.144Observation by third party Petra Trefzger dated Jan. 17, 2005, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.145Observation by third party Simon Penny dated Jan. 17, 2005, cited during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.146Office Action in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Apr. 22, 2004.147Office Action in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Jun. 27, 2006.148Office Action in U.S. Appl. No. 10/866,495, dated Apr. 6, 2007.149Office Action in U.S. Appl. No. 10/866,495, dated Aug. 21, 2009.150Office Action in U.S. Appl. No. 10/866,495, dated Jan. 23, 2008.151Office Action in U.S. Appl. No. 10/866,495, dated Oct. 17, 2007.152Office Action in U.S. Appl. No. 10/866,495, dated Sep. 4, 2008.153Office Action in U.S. Appl. No. 10/946,08, dated Dec. 7, 2009.154Office Action in U.S. Appl. No. 10/946,084, dated Apr. 28, 2011.155Office Action in U.S. Appl. No. 10/946,084, dated Apr. 9, 2009.156Office Action in U.S. Appl. No. 10/946,084, dated Dec. 8, 2010.157Office Action in U.S. Appl. No. 10/946,084, dated Jul. 10, 2008.158Office Action in U.S. Appl. No. 10/946,084, dated Jun. 28, 2010.159Official Communication in Australian Application No. 2002312346, dated Nov. 14, 2006.160Official Communication in Canadian Patent Application No. 2,449,300, dated Nov. 9, 2009.161Official Communication in Chinese Application No. 02815206.9 dated Dec. 12, 2008.162Official Communication in Chinese Application No. 02815206.9 dated Mar. 1, 2010.163Official Communication in Chinese Application No. 02815206.9 dated Sep. 25, 2009.164Official Communication in Chinese Application No. 200480034551.4, dated Feb. 19, 2008.165Official Communication in European Patent Application No. 04 813 622.0, dated Mar. 4, 2008.166Official Communication in Indian Patent Application No. 1582/KOLNP/2003, dated May 22, 2006.167Official Communication in Korean Patent Application No. 10-2003-7015990, dated Nov. 8, 2006.168Official Communication in Korean Patent Application No. 2006-7011270, dated Mar. 8, 2011.169Official Report in Australian Application No. 2008299883, dated Dec. 8, 2010.170Paradiso, Joseph et al., "New Sensor and Music Systems for Large Interactive Surfaces," Aug. 2000, Proceedings of the Interactive Computer Music Conference, Berlin, Germany, pp. 277-280.171Patent Withdrawal Notice in U.S. Appl. No. 10/160,217 (USPN 7,259,747), dated Mar. 22, 2007.172Penny, Simon, "Fugitive"; Oct. 1997; <http://www.ace.uci.edu/penny/works/fugitive/fugitive.html>.173Penny, Simon, et al., "Fugitive II," Jan. 8-Mar. 14, 2004, Australian Center for the Moving Image, art installation, description available online at <http://www.acmi.net.au/fugitive.jsp?>.174Penny, Simon, et al.; Traces: Wireless full body tracking in the Cave, Dec. 16-18, 1999; Japan; ICAT Virtual Reality Conference; <http://turing.ace.uci.edu/pennytexts/traces/>.175Pinhanez, C. et al., "Ubiquitous Interactive Graphics," Jul. 29-31, 2003, IBM Research Report RC22495, available at <http://www.research.ibm.com/ed/publications/rc22495.pdf>.176Pinhanez, C., "The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces," Ubicomp 2001: Ubiquitous Computing: International Conference Atlanta, Georgia, USA, Sep. 30-Oct. 2, 2001 Proceedings (Lecture Notes in Computer Science), pp. 315-331.177Plasma; 3 pages; , cited in U.S. Appl. No. 10/160,217, filed Aug. 8, 2005.178Plasma; 3 pages; <http://www.particles.de/paradocs/plasma/index.html>, cited in U.S. Appl. No. 10/160,217, filed Aug. 8, 2005.179Provision of the minutes in European Application No. 02739710.8 dated Dec. 28, 2009.180R111, The Transformation From Digital Information to Analog Matter, available online at , cited on Jan. 17, 2005 during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.181R111, The Transformation From Digital Information to Analog Matter, available online at <http://www.particles.de/paradocs/r111/10mkp2004/hmtl/r111�text111hock04.html>, cited on Jan. 17, 2005 during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.182Reactrix, Inc. website, Mar. 28, 2003, <http://web.archive.org/web/20030328234205/http://www.reactrix.com> and <http://web.archive.org/web/20030328234205/http://www.reactrix.com/webdemo.php>.183Reactrix, Inc. website, Mar. 28, 2003, and .184Rekimoto, J., et al., "Perceptual Surfaces: Towards a Human and Object Sensitive Interactive Display," Oct. 19-21, 1997, Proceedings of the Workshop on Perceptual User Interfaces, Banff, Canada, pp. 30-32.185Rekimoto, Jun, "SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces." vol. No. 4, Issue No. 1, pp. 113-120, Apr. 2002.186Ringel, M. et al., "Barehands: Implement-Free Interaction with a Wall-Mounted Display," Mar. 31-Apr. 5, 2001, Proceedings of the 2001 ACM CHI Conference on Human Factors in Computing Systems (Extended Abstracts), p. 367-368.187Rogala, Miroslaw, "Lovers Leap," Nov. 21-26, 1995, art installation, Dutch Electronic Arts Festival, description available online at <http://wayback.v2.nl/DEAF/persona/rogala.html>.188Rokeby, David, "softVNS 2 real time video processing and tracking software for Max;" SoftVNS 2 downloads, as downloaded from on Mar. 16, 2007.189Rokeby, David, "Very Nervous System (VNS)," Mar. 1995, Wired Magazine, available online at ; sold as software at .190Rokeby, David, "softVNS 2 real time video processing and tracking software for Max;" SoftVNS 2 downloads, as downloaded from <http://homepage.mac.com/davidrokeby/softVNS.html> on Mar. 16, 2007.191Rokeby, David, "Very Nervous System (VNS)," Mar. 1995, Wired Magazine, available online at <http://www.wired.com/wired/archive/3.03/rokeby.html>; sold as software at <http://homepage.mac.com/davidrokeby/softVNS.html>.192Sato, Yoichi, et al., "Fast Tracking of Hands and Fingertips in Infrared Images for Augmented Desk Interface," Mar. 2000, 4th International Conference on Automatic Face-and Gesture-Recognition, Grenoble, France.193Sato, Yoichi, et al., "Fast Tracking of Hands and Fingertips in Infrared Images for Augmented Desk Interface," Mar. 2000, 4th International Conference on Automatic Face�and Gesture-Recognition, Grenoble, France.194Schneider, John K., "Improved Fingerprint System Using Rolled and Multi-segmented Techniques," U.S. Appl. No. 60/575,952, filed Jun. 1, 2004, pp. 1-6.195Screenshots of Reactrix Product Demo Video, Mar. 28, 2003, .196Screenshots of Reactrix Product Demo Video, Mar. 28, 2003, <http://web.archive.org/web/20030407174258/http://www.reactrix.com/demo/reactrix�demo.wmv>.197Sester, Marie, "Access," Dec. 2001, Interaction 99 Biennial Catalog, Gifu, Japan, available online at <http://www.accessproject.net/concept.html>.198Snibbe, Scott, "Boundary Functions," Sep. 7-12, 1998, art installation, description available online at <http://snibbe.com/scott/bf/index.html>.199Snibbe, Scott, "Screen Series," 2002-2003 art installation, description available online at <http://snibbe.com/scott/screen/index.html>, downloaded on Mar. 16, 2007.200Sonneck, Georg, et al., "Optimized One-to-One Personalization of Web Applications using a Graph Based Model," IEEE-22, Apr. 26, 2003, 9 pgs.201Sparacino, Flavia, "(Some) computer visions based interfaces for interactive art and entertainment installations," 2001, Inter�Face Body Boundaries, Anomalie digital�arts, No. 2, Paris, France, <http://alumni.media.mit.edu/�flavia/publications.html>, cited on Jul. 15, 2005 during opposition of Canadian Application No. 2449300, filed Jun. 4, 2002.202Sparacino, Flavia, et al., "Media in performance: interactive spaces for dance, theater, circus and museum exhibits," Nov. 2000, IBM Systems Journal, vol. 39, No. 3-4, pp. 479-510.203Stauffer, Chris, et al., "Learning Patterns of Activity Using Real-Time Tracking," Aug. 2000, IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 22, No. 8, pp. 747-757.204Summons to Attend Oral Proceedings in European Application No. 02739710.8, dated Aug. 12, 2005.205Summons to Attend Oral Proceedings in European Application No. 02739710.8, dated Jun. 12, 2009.206Supreme Particles, "Plasma/Architexture," 1994, available online at <http://www.particles.de/paradocs/plasma/plasma�e.html>, downloaded on May 21, 2007.207Supreme Particles; R111, 1999, available online at <http://www.r111.org>, obtained Jul. 12, 2005, XP-002989704.208Tan, P, et al., "Highlight Removal by Illumination-Constrained Inpainting," Ninth IEEE International Conference on Computer Vision, Oct. 13-16, 2003.209The History of Microsoft Surface, downloaded from http://www.microsoft.com/presspass/presskits/surfacecomputing/docs/SurfaceHistoryBG.doc on Sep. 24, 2009.210Torr, P.H.S. et al., "The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix," Sep./Oct. 1997, International Journal of Computer Vision, vol. 24, No. 3, pp. 271-300.211Toth, Daniel et al., "Illumination-Invariant Change Detection," Apr. 2-4, 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation, p. 3.212Trefzger, Petra, "Vorwerk," 2000, art installation, description available online at <http://www.petracolor.de/> cited on Jan. 17, 2005 during opposition of European Application No. 02739710.8, filed Jun. 4, 2002.213Utterback, Camille, et al., "Text Rain," 1999, art installation, available online at <www.camilleutterback.com/textrain.html>, Mar. 16, 2007.214Vogt, Florian et al., "Highlight Substitution in Light Fields," IEEE International Conference on Image Processing, Sep. 22-25, 2002.215Wang, Junxian, et al., "Specular reflection removal for human detection under aquatic environment," Jun. 27-Jul. 2, 2004 IEEE Conference on Computer and Pattern Recognition Workshop (CVPRW04) vol. 8, p. 130.216Wellner, Pierre, "Digital Desk Calculator:Tangible Manipulation on a Desktop Display" Proceedings of the Symposium on User Interface Software and Technol (UIST), Hilton Head, S. Carolina, Nov. 11-13, 1991.217Wellner, Pierre, "Interacting with paper on the DigitalDesk," Jul. 1993, Communications of the ACM, Special issue on computer augmented environments: back to the real world, vol. 36, Issue 7, pp. 87-96.218Wilson, Andrew, "PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System," ACM Symposium on User Interface Software and Technology (UIST), Oct. 23-27, 2005, Seattle, Washington, U.S.A.219Written Opinion for PCT/US2002/017843, filed Jun. 4, 2002, dated Feb. 5, 2003.220Written Opinion of the International Searching Authority for PCT/US2004/035477, filed Oct. 25, 2004.221Written Opinion of the International Searching Authority for PCT/US2004/035478, filed Oct. 25, 2004.222Written Opinion of the International Searching Authority for PCT/US2004/041318, filed Dec. 9, 2004.223Written Opinion of the International Searching Authority for PCT/US2004/041319, filed Dec. 9, 2004.224Written Opinion of the International Searching Authority for PCT/US2004/041320, filed Dec. 9, 2004.225Written Opinion of the International Searching Authority for PCT/US2005/008984, filed Mar. 18, 2005.226Written Opinion of the International Searching Authority for PCT/US2006/030720, filed Aug. 4, 2006.227Written Opinion of the International Searching Authority for PCT/US2008/059900, filed Aug. 10, 2008.228Xiao, Yang; "Throughput and Delay Limits of IEEE 802.11," IEEE Communications Letters, vol. 6, No. 8, pp. 355-357, Aug. 2002.Referenced byCiting PatentFiling datePublication dateApplicantTitleUS8625845Feb 16, 2010Jan 7, 2014Quantum Signal, LlcOverlaying virtual content onto video stream of people within venue based on analysis of the people within the video streamUS20090219253 *Feb 29, 2008Sep 3, 2009Microsoft CorporationInteractive Surface Computer with Switchable DiffuserUS20090273576 *Jul 10, 2009Nov 5, 2009Blythe Michael MDisplay deviceUS20100134409 *Nov 30, 2008Jun 3, 2010Lenovo (Singapore) Pte. Ltd.Three-dimensional user interfaceUS20120086636 *Nov 16, 2010Apr 12, 2012Hon Hai Precision Industry Co., Ltd.Sensor ring and interactive system having sensor ringUS20120188238 *Mar 2, 2012Jul 26, 2012Sharp Kabushiki KaishaMultiple-view directional display* Cited by examinerClassifications U.S. Classification345/156, 345/158International ClassificationG06F3/00, G06F3/042, G06F3/033, G09G5/00, G03B21/14, G06F3/01Cooperative ClassificationA63F2300/1068, G06F3/0425, G06F3/0428, A63F2300/1087, G06F3/011European ClassificationG06F3/01B, G06F3/042P, G06F3/042CLegal EventsDateCodeEventDescriptionApr 9, 2012ASAssignmentFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTELLECTUAL VENTURES HOLDING 67 LLC;REEL/FRAME:028012/0662Owner name: MICROSOFT CORPORATION, WASHINGTONEffective date: 20120216Jun 2, 2009ASAssignmentOwner name: INTELLECTUAL VENTURES HOLDING 67 LLC, NEVADAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DHANDO INVESTMENTS, INC.;REEL/FRAME:022766/0584Effective date: 20090409May 28, 2009ASAssignmentOwner name: DHANDO INVESTMENTS, INC., DELAWAREFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:REACTRIX (ASSIGNMENT FOR THE BENEFIT OF CREDITORS), LLC;REEL/FRAME:022745/0484Effective date: 20090409May 19, 2009ASAssignmentOwner name: REACTRIX (ASSIGNMENT FOR THE BENEFIT OF CREDITORS)Free format text: CONFIRMATORY ASSIGNMENT;ASSIGNOR:REACTIX SYSTEMS, INC.;REEL/FRAME:022710/0044Effective date: 20090406Jan 24, 2005ASAssignmentOwner name: REACTRIX SYSTEMS, INC., CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BELL, MATTHEW;GLECKMAN, PHILIP;ZIDE, JOSHUA;AND OTHERS;REEL/FRAME:016181/0188;SIGNING DATES FROM 20041223 TO 20050113Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BELL, MATTHEW;GLECKMAN, PHILIP;ZIDE, JOSHUA;AND OTHERS;SIGNING DATES FROM 20041223 TO 20050113;REEL/FRAME:016181/0188RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google