Source: http://www.google.co.uk/patents/US8094928
Timestamp: 2014-09-17 09:36:52
Document Index: 133860628

Matched Legal Cases: ['Application No. 200680042234', 'Application No. 06816854', 'Application No. 06816854', 'Application No. 200680042234', 'Application No. 06816854', 'Application No. 06816854', 'Application No. 06816854']

Patent US8094928 - Stereo video for gaming - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign in<nobr>Advanced Patent Search</nobr>PatentsA real-time stereo video signal of a captured scene with a physical foreground object and a physical background is received. In real-time, a foreground/background separation algorithm is used on the real-time stereo video signal to identify pixels from the stereo video signal that represent the physical...http://www.google.co.uk/patents/US8094928?utm_source=gb-gplus-sharePatent US8094928 - Stereo video for gamingAdvanced Patent SearchPublication numberUS8094928 B2Publication typeGrantApplication numberUS 11/272,950Publication date10 Jan 2012Filing date14 Nov 2005Priority date14 Nov 2005Also published asCN101305401A, CN101305401B, EP1960970A1, EP1960970A4, EP1960970B1, US20070110298, US20120071239, WO2007055865A1Publication number11272950, 272950, US 8094928 B2, US 8094928B2, US-B2-8094928, US8094928 B2, US8094928B2InventorsThore K H Graepel, Andrew Blake, Ralf HerbrichOriginal AssigneeMicrosoft CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (11), Non-Patent Citations (60), Referenced by (12), Classifications (18), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetStereo video for gamingUS 8094928 B2Abstract A real-time stereo video signal of a captured scene with a physical foreground object and a physical background is received. In real-time, a foreground/background separation algorithm is used on the real-time stereo video signal to identify pixels from the stereo video signal that represent the physical foreground object. A video sequence is produced by rendering a 3d virtual reality based on the identified pixels of the physical foreground object.
BACKGROUND Three dimensional (3d) graphics, in particular, simulated 3d realms or worlds, sometimes called 3d virtual reality, is a well known area of computer graphics, which typically involves rendering two dimensional images of 3d models and scenery in a 3d coordinate space. Most modern game consoles are designed specifically to be able to process 3d graphics in real-time, and many games for game consoles are based on a simulated 3d or virtual reality.
DETAILED DESCRIPTION Stereo cameras and algorithms for processing stereo video data have progressed to the point where it possible to reliably acquire certain image information about captured objects in real time. A number of publications may be consulted. For example, �Bi-layer segmentation of binocular stereo video� (Vladimir Kolmogorov, Antonio Criminisi, Andrew Blake, Geoffrey Cross, Carsten Rother. 2005 San Diego, Calif., US Proc. IEEE Computer Vision and Pattern Recognition) discusses techniques for separating foreground objects from their background by fusing color/contrast analysis with stereo pattern matching. Regarding basic stereo matching, see also Y. Ohta and T. Kanade, Stereo by intra- and inter-scan line search using dynamic programming, IEEE Trans. on PAMI, 7(2), 1985; I. J. Cox, S. L. Hingorani, and S. B. Rao, A maximum likelihood stereo algorithm, CVIU, 63(3):542-567, 1996; D. Scharstein and R. Szeliski, A taxonomy and evaluation of dense two-frame stereo correspondence algorithms, IJCV, 47(1-3), 2002. Regarding dealing with occlusion on object boundaries, see P. N. Belhumeur, A Bayesian-approach to binocular stereopsis, Int. J. Computer Vision, 19(3):237-260, August 1996; D. Geiger, B. Ladendorf, and A. Yuille, Occlusions and binocular stereo, Int. J. Computer Vision, 14:211-226, 1995; and A. Criminisi, J. Shotton, A. Blake, and P. H. S. Torr, Gaze manipulation for one to one teleconferencing, In Proc. ICCV, 2003.
The disparity map 124 is compared to at least a portion of the kernel image 126 to determine matching disparity values. A disparity-based kernel image is a model or template disparity map that is compared against the disparity map 124. The use of a kernel image is optional. A kernel image can be used to rapidly recover the approximate silhouette of an object. A more precise but costly alternative, discussed in the next paragraph, is to use optimization methods to define a binary mask of foreground vs. background points. The kernel image 126 can be an array of pixel values which represent the stereo disparity of an object to be located or searched for. More particularly, the kernel image 126 is an encoding of the silhouette of the object to be located as well as surface shape of the object to be located, e.g., the �bumpiness� or depth of the object. In this manner, the kernel image 126 indicates the 3d surface shape of the object to be located from a point of view. The kernel image 126 can be, for example, a predetermined disparity map of a generic torso shape or any other shape or object. The kernel image 126 can be calculated in advance, or derived from a previous disparity map, or otherwise obtained. The kernel image 126 can be an approximation of the object that it represents, in other words, a rough model of the object. The disparity map 124 can also be used to determine the depth or distance of pixels relative to the stereo camera 50. An average of these distances (a distance to the object) can be used to scale the kernel image 126 before disparity map 124 is searched against the kernel image 126. As discussed below, color/contrast information 128, possibly from a preceding disparity map or previous stereo video frames, can be used in separating the background.
Foreground/background separation can be performed by fusing a stereo-based segmentation algorithm with a color/contrast based segmentation algorithm. Algorithms for automatically separating layers using color/contrast or stereo alone are often prone to errors. By fusing color/contrast analysis with stereo matching information, layers can be inferred accurately and efficiently. A Layered Dynamic Programming (LDP) algorithm can be used to solve stereo in an extended 6-state space that represents both foreground/background layers and occluded regions. The resulting stereo-match likelihood is then fused with a contrast-sensitive color model that is learned on the fly, and stereo disparities are obtained by dynamic programming. A second algorithm, Layered Graph Cut (LGC), can be used to marginalize the stereo match likelihood over foreground and background hypotheses for fusion with a contrast-sensitive color model like the one used in LDP. Segmentation is then solved efficiently by a ternary graph cut. In sum, the device driver 78 uses one or more algorithms for fast and reliable foreground/background segregation using stereo and/or color/contrast information, which produces a separated foreground object 130. For additional details, see �Bi-layer segmentation of binocular stereo video�, by Vladimir Kolmogorov, Antonio Criminisi, Andrew Blake, Geoffrey Cross, Carsten Rother (US Proc. IEEE Computer Vision and Pattern Recognition, 2005).
Having separated stereo images of one or more objects in a scene, different types of information about objects in a scene can then be determined 132. For example, different types of objects can be identified by using different kernel images 126. If an object has been separated from the background, that object can be identified by comparing it to different kernel images. Stereo-based depth information can also be obtained. A virtual or cyclopean image of the object can be computed from the left and right image using ordinary geometry-based techniques. The location of the separated object in the stereo-based image and/or an input image may be indicated in any suitable manner. For example, the disparity data, pixel locations, or any other suitable indicator of the located object may be associated with the image as meta-data. The image with the located object may be used by the display manipulator module to perform some action or it may be sent to another application. Artifacts in the generated image can be corrected using a split-patch search algorithm, which may involve: restricting candidate patches to those lying on corresponding (left or right) epipolar lines; constraining a search region using tight, geometric depth bounds; and applying exemplar-based synthesis sparsely, where flagged by an inconsistency test. For further details, see �The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search�, by Antonio Criminisi, Andrew Blake, (US Proc. IEEE Computer Vision and Pattern Recognition, 2004). Border matting is an alternative method for correcting artifacts and obtaining pixel or subpixel precision. For details, see V. Kolmogorov, A. Criminisi, A. Blake, G. Cross, C. Rother, Probabilistic fusion of stereo with color and contrast for bi-layer segmentation, June 2005, MSR-TR-2005-35.
It should be appreciated that stereo image analysis as discussed above can be repeatedly performed in real time on paired frames of the stereo video signal. This allows real time operations such as tracking the changing position of an object, providing accurate real time �cut out� video of an object as it moves and changes (i.e., video of an object with the background cleanly removed regardless of the nature of the background), and providing a dynamic depth map of an object as it moves or changes in real time.
FIG. 6 shows how stereo-derived object information can be used in conjunction with a game and VR engine. A stereo image pair is received 150 from a stereo video signal. Using one or more techniques discussed above, for example stereo matching segmentation fused with color/contrast segmentation, foreground/background separation is performed to separate 152 one or more objects from the background. Information about the one or more objects is obtained 154, for example, depth information, a well-defined image of the object, the identity of the one or more objects, etc. This information is provided 156 to the game program. The game program receives 158 the object information and uses it (some examples follow, see FIGS. 7-10) to affect or modify 160 the behavior or �play� of the game, and/or the appearance of the game, or other aspects of the game. As instructed by the game program, the render engine renders 162 the game as modified 160 in accordance with the stereo-derived object information.
It should be noted that boundary recovery to pixel precision (or better) can have allow not just determining the location of an object (e.g., �a limb�) but its precise outline, shape, and interior texture. Thus the entire shape and texture of the object can be reproduced elsewhere, and can be subjected to transformations of color or shape or texture along the way.
FIG. 9 shows an example of mapping a separated image of a foreground physical object to a model that is then rendered and displayed. The boxes 230 in FIG. 9 represent a real world scene as seen by a stereo camera, in this case, a person's torso in a room. An image of an object is extracted 232 from stereo images of a scene using techniques discussed above. For example, a kernel disparity image of the rough form of a human torso can be used for foreground/background separation, possibly in conjunction with other techniques. In one embodiment, the extracted image can include depth values of the pixels of the image. In other words, a 3d image of the detected object. In the example of FIG. 9, by keying on facial features, the extracted image is processed further to obtain a particular portion of the object�the face. The original or the refined images 234 can be normalized so that the edge pixels have a depth of zero. In the example, an image of the face could also be obtained from a suitable face-like kernel image.
In another embodiment, the extracted image of the object is not mapped 238 to a model. Techniques for stereo-based foreground/background separation have advanced to the point where foreground images can be separated cleanly and efficiently, even if the background has a same color as the object in the foreground. Furthermore, the images can be separated and synthesized in such a manner that the images are significantly free of artifacts. In other words, an accurate profile of the object can be obtained; the background is accurately removed independent of the nature of the background. Extracted images usually have a quality comparable to images obtained using blue or green screen separation; the images are sharp and accurate representations of the object. Therefore, an image of an object can be displayed directly in a game or 3d virtual reality, either as a planar surface, or as a 3d surface, possibly with some modeled �backing� to allow 3d non-frontal viewing.
In another embodiment, the extracted image is co-displayed with the 3d virtual reality, but is not incorporated into the 3d virtual reality. For example, if a number of players are participating in a same 3d virtual reality (each with a stereo camera), each player's �heads up display� (user interface) may include images or real time video of the head/torso of each participant. The general idea of using stereo techniques to extract foreground images cleanly separated from the background and immersing the images in a game can take other forms. For example, extracted images or video can be displayed as two-dimensional images, whether in a two-dimensional game or a three-dimensional game. As another example, extracted images could be displayed in a virtual monitor (within the game) or an instant-messenger type of application (within the game or as part of the game interface). A remote partner or combatant can be seen, in some form, within scenes in a game.
Stereo-based foreground/background separation is also useful for object recognition. FIG. 10 shows how stereo-based object recognition can be used for 3d gaming. As discussed above, the stereo video signal can be searched for kernel images of different types of objects, thus performing a form of object detection or recognition; if a kernel image is matched to a portion of a captured scene, the object associated with that kernel image is deemed to be present in the scene. Consider an example where there are three kernel images (not shown) to be searched for: a kernel image of a briefcase; a kernel image of a flashlight or cylindrical object; and a kernel image of an arm/hand holding nothing. In this example, the game is a type where the player controls a 3d avatar, character, vehicle, etc. that is rendered and displayed. In a first physical scene 260 A, a real person is holding a briefcase. One or more pairs of stereo frames are processed to recognize 262 the object (e.g., �a briefcase�).
In response to the recognition 262, the game causes the 3d character to �hold� a corresponding virtual object such as a virtual briefcase. The holding of the object can simply be implemented as a change to the state of the character (e.g., a flag is set indicating that the virtual object is currently being held) without any corresponding change in what is displayed or rendered. Additionally or alternatively, the virtual holding can be implemented by causing the 3d character to be rendered to appear to hold a virtual object associated with the matched kernel image, as seen in rendered characters 264. Similarly, in scene 260 B, a flashlight is recognized 262 and the game character is modified and/or rendered accordingly. If the recognizing 262 is handedness sensitive, then if the flashlight is in the same hand as the briefcase was, the character is made to stop holding the virtual briefcase, or if the flashlight is in the real person's other hand, then the character might be made to virtually hold both virtual objects. In scene 260 C, an empty arm/hand is recognized and the game character is rendered accordingly. In this manner, a person with a set of real objects can control the objects virtually held by a game character by picking up any of the corresponding real objects. Real objects held in a hand can be recognized by using both the kernel image of the arm/hand and the kernel images of the other objects to detect which object is currently in a hand of the person. As mentioned earlier, kernel disparity images can be obtained in advance (e.g., part of the content embedded in a particular game), or during a training process where an object is held in front of the stereo camera, or from disparity maps extracted from earlier processed scenes.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS680689820 Mar 200019 Oct 2004Microsoft Corp.System and method for automatically adjusting gaze and head orientation for video conferencingUS687372330 Jun 199929 Mar 2005Intel CorporationSegmenting three-dimensional video images using stereoUS7164784 *30 Jul 200216 Jan 2007Mitsubishi Electric Research Laboratories, Inc.Edge chaining using smoothly-varying stereo disparityUS20040151366 *4 Feb 20035 Aug 2004Nefian Ara V.Gesture detection from digital video imagesUS20040239670 *29 May 20032 Dec 2004Sony Computer Entertainment Inc.System and method for providing a real-time three-dimensional interactive environmentUS20040252862 *13 Jun 200316 Dec 2004Sarnoff CorporationVehicular vision systemUS200402582797 Apr 200423 Dec 2004Sarnoff CorporationMethod and apparatus for pedestrian detectionUS20090073170 *21 Oct 200519 Mar 2009Koninklijke Philips Electronics, N.V.Disparity mapEP1072018A112 Apr 199931 Jan 2001Eyematic Interfaces, Inc.Wavelet-based facial motion capture for avatar animationWO2001013645A217 Aug 200022 Feb 2001Orad Hi Tec Systems LtdNarrow bandwidth broadcasting systemWO2002007839A224 Jul 200131 Jan 2002Jestertek IncVideo-based image control system* Cited by examinerNon-Patent CitationsReference1"Integrated person tracking using stereo, color, and pattern detection", T. Darrell, G. Gordon, M. Harville and J. Woodfill, Proceedings of the 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR '98, Santa Barbara, CA, Jun. 23-25, 1998. [IEEE Computer Society Conference on Computer fision and Pattern Recognition], Los Alamitos, CA.: IE, vol. CONF. 17, Jun. 23, 1998, pp. 601-608.2"Projects Regarding Virtual Reality : Motion Capture for Avatar Construction in Interactive Virtual Environments", http://www.comp.mq.edu.au/undergrad/honours/projects/proposals/manolya.htm, (Nov. 2005).3"Projects Regarding Virtual Reality : Motion Capture for Avatar Construction in Interactive Virtual Environments", http://www.comp.mq.edu.au/undergrad/honours/projects/proposals/manolya.htm.4"Xbox", Anonymous, Wikipedia, Oct. 31, 2005, pp. 1-10, Retrieved from the Internet: URL: http://en.wikipedia.org/w/index.php?title=Xbox&oldid=27003388 on Apr. 29, 2010.5Bahadori, S., et al., "A Stereo Vision System for 3D Reconstruction and Semi-Automatic Surveillance of Museum Areas," http://www.dis.uniromal.it/~iocchi/publications/iocchi-plt-aiia03.pdf, (2003).6Bahadori, S., et al., "A Stereo Vision System for 3D Reconstruction and Semi-Automatic Surveillance of Museum Areas," http://www.dis.uniromal.it/~iocchi/publications/iocchi-plt-aiia03.pdf.7Bahadori, S., et al., "A Stereo Vision System for 3D Reconstruction and Semi-Automatic Surveillance of Museum Areas," http://www.dis.uniromal.it/�iocchi/publications/iocchi-plt-aiia03.pdf, (2003).8Bahadori, S., et al., "A Stereo Vision System for 3D Reconstruction and Semi-Automatic Surveillance of Museum Areas," http://www.dis.uniromal.it/�iocchi/publications/iocchi-plt-aiia03.pdf.9Chinese Office Action cited in Chinese Application No. 200680042234.6 dated Jun. 22, 2011.10Chong, L., et al., "Video Stereoscopic Avatars for the CAVE(TM) Virtual Environments," http://zx81.isl.uiuc.edu/camilleg/ismar04.pdf, (2004).11Chong, L., et al., "Video Stereoscopic Avatars for the CAVE(TM) Virtual Environments," http://zx81.isl.uiuc.edu/camilleg/ismar04.pdf.12Chong, L., et al., "Video Stereoscopic Avatars for the CAVE� Virtual Environments," http://zx81.isl.uiuc.edu/camilleg/ismar04.pdf, (2004).13Chong, L., et al., "Video Stereoscopic Avatars for the CAVE� Virtual Environments," http://zx81.isl.uiuc.edu/camilleg/ismar04.pdf.14Cordea, M.D., et al., "3D Head Pose Recovery for Interactive Virtual Reality Avatars," IEEE Instrumentation and Measurement Technology Conference, Budapest, Hungary, May 21-23, 2001.15 *Criminisi, A.-"The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search"-IEEE-Jul. 2004.16 *Criminisi, A.�"The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search"�IEEE�Jul. 2004.17Eisaku Hori, et al., "3D Input Interface via Hand Motion Using Stereo Camera System for Wearable PC Environment," Presented in Proceedings of MVA2002 (IAPR Workshop on Machine Vision Applications, Dec. 11-12, 2002, Nara, Japan), pp. 285-288.18European Office Action in Related Case Application No. 06816854.1 Dated Jan. 5, 2011.19European Search Report Ref EP58192TE900kap, Application No. 06816854.1-2218 / 1960970 PCT/US2006040043. Mailed Feb. 3, 2009. 17 Pages.20European Search Report, EP 06 81 6854, by Scholz, Volker, Jan. 22, 2009.21Garcia, L. V., "MeetMe@CyberStage: Immersive Stereoscopic Tlepresence," 2001.22Gordon G et al: "Background estimation and removal based on range and color" 19990623-19990625, vol. 2, Jun. 23, 1999, pp. 459-464.23Gordon, et al., "Background Estimation and Removal Based on Range and Color", cvpr, vol. 2, 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99)-vol. 2. 6 Pages.24Gordon, et al., "Background Estimation and Removal Based on Range and Color", cvpr, vol. 2, 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99)�vol. 2. 6 Pages.25Hilton A et al: "3-D studio production of animated actor models Visual media production" IEE Proceedings: Vision, Image and Signal Processing, Institution of Electrical Engineers, GB, vol. 152, No. 4, Aug. 5, 2005, pp. 481-490.26Hilton, et al., "3-D Studio Production of Animated Actor Models", IEE Proceedings-Vision, Image and Signal Processing, vol. 152, Issue 4, Aug. 5, 2005. pp. 481-490.27Hilton, et al., "3-D Studio Production of Animated Actor Models", IEE Proceedings�Vision, Image and Signal Processing, vol. 152, Issue 4, Aug. 5, 2005. pp. 481-490.28Hilton, et al; "3D Studio Production of Animated Actor Models"; Vision, Image and Signal Processing, IEE Proceedings; vol. 152, Aug. 4-5, 2005. pp. 481-490.29Int. Preliminary Report on Patentability cited in PCT Application No. PCT/US2006/040043 dated May 22, 2008.30International Search Report; Korean Intellectual Property Office; Feb. 14, 2007.31Kleiner, M., et al., "Multi-viewpoint video capture for facial perception research," http://www.kyb.mpg.de/publications/pdfs/pdf3058.pdf, (2004).32Kleiner, M., et al., "Multi-viewpoint video capture for facial perception research," http://www.kyb.mpg.de/publications/pdfs/pdf3058.pdf.33Kolmogorov, et al., "Bi-Layer Segmentation of Binocular Stereo Video", IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005. Jun. 20-25, 2005. 8 Pages.34Kolmogorov, V. et al; "Bi-Layer Segmentation of Binocular Stereo Video"; Computer Vision and Pattern Recognition, 2005; In: IEEE Computer Society Conference; vol. 2, Jun. 20-25, 2005; pp. 407-414.35 *Konrad, T.-"Navigating in Virtual Environments using a Vision-based Interface"-ACM-2004, pp. 113-120.36 *Konrad, T.�"Navigating in Virtual Environments using a Vision-based Interface"�ACM�2004, pp. 113-120.37 *Mittal, A.-"M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene"-Computer Vision-2003.38 *Mittal, A.�"M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene"�Computer Vision�2003.39Morency, L.-P., et al., "From Conversational Tooltips to Grounded Discourse: Head Pose Tracking in Interactive Dialog Systems," ICMI'04, Oct. 13-15, 2004, State College, Pennsylvania, USA.40Office Action cited in related Chinese Application No. 200680042234.6 dated Mar. 3, 2010.41Office Action cited in related European Application No. 06816854.1-2218 dated Apr. 8, 2009.42Office Action cited in related European Application No. 06816854.1-2218 dated May 11, 2010.43Office Action for European Application No. 06816854.1-2218 dated Sep. 29, 2009.44Ogi, T., et al., "Usage of Video Avatar Technology for Immersive Communication," First International Workshop on Language Understanding and Agents for Real Work Interaction, 2003, http://www.cl.cs.titech.ac.ip/sinpro/WS/2003.7.13/Ogi.pdf.45Rajan, V., et al., "A Realistic Video Avatar System for Networked Virtual Environments," http://www.evl.uic.edu/aej/papers/IPT2002.pdf, (2002).46Rajan, V., et al., "A Realistic Video Avatar System for Networked Virtual Environments," http://www.evl.uic.edu/aej/papers/IPT2002.pdf.47Search Report in Related Case Application No. PCT/US2006040043 Dated Feb. 14, 2007.48Sheriff, L., "MS boffins build real-time stereocam," The Register, 10th Jun. 2004, http://www.theregister.com/2004/06/10/3d-avatar/.49Sheriff, L., "MS boffins build real-time stereocam," The Register, 10th Jun. 2004, http://www.theregister.com/2004/06/10/3d�avatar/.50Tollmar, K., et al., "Gesture + Play: Full-Body Interaction for Virtual Environments," CHI 2003: New Horizons, CHI 2003, Apr. 5-10m, 2003, Ft. Lauderdale, Florida, USA.51Triclops(TM) Software Development Kit (SDK) Technical Specification, Point Grey Research, http://www.ptgrey.com/products/triclopsSDK/triclops.pdf, (Feb. 2004).52Triclops(TM) Software Development Kit (SDK) Technical Specification, Point Grey Research, http://www.ptgrey.com/products/triclopsSDK/triclops.pdf.53Triclops� Software Development Kit (SDK) Technical Specification, Point Grey Research, http://www.ptgrey.com/products/triclopsSDK/triclops.pdf, (Feb. 2004).54Triclops� Software Development Kit (SDK) Technical Specification, Point Grey Research, http://www.ptgrey.com/products/triclopsSDK/triclops.pdf.55Wren, C.R., et al., "Combining Audio and Video in Perceptive Spaces," M.I.T. Media Laboratory Perceptional Computing Section Technical Report No. 511, 1st International Workshop on Managing Interactions.56Wren, C.R., et al., "Combining Audio and Video in Perceptive Spaces," M.I.T. Media Laboratory Perceptional Computing Section Technical Report No. 511, lst International Workshop on Managing Interactions (1999).57Yonemoto S et al: "Vision-based 3D direct manipulation interface for smart interaction" Pattern Recognition, 2002, Proceedings. 16th International Conference on Quebec City, Que., Canada Aug. 11-15, 2002, Los Alamitos, CA, USA, IEEE Comput. Soc., US, vol. 2, Aug. 11, 2002, pp. 655-658.58Yonemoto, et al., "Vision-based 3D Direct Manipulation Interface for Smart Interaction", Proceedings 16th International Conference on Pattern Recognition, 2002. vol. 2, pp. 655-658.59Youngjung Suh, et al., "2.5D Video Avatar Augmentation for VRPhoto," ICAT 2002, Dec. 4-6, Tokyo, Japan.60Zabih et al., "Non-parametric Local Transforms for Computing Visual Correspondence" In Proceedings of European Conference on Computer Vision, Stockholm, Sweden, May 1994, pp. 151-158 http://www.tyzx.com/PDFs/census.pdf.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS8447098 *23 Nov 201021 May 2013Adobe Systems IncorporatedModel-based stereo matchingUS8602887 *3 Jun 201010 Dec 2013Microsoft CorporationSynthesis of information from multiple audiovisual sourcesUS8630455 *20 Jul 201114 Jan 2014SET CorporationMethod and system for audience digital monitoringUS863902016 Jun 201028 Jan 2014Intel CorporationMethod and system for modeling subjects from a depth mapUS867061530 Sep 201011 Mar 2014Flashfoto, Inc.Refinement of segmentation markupUS8682029 *12 Dec 200825 Mar 2014Flashfoto, Inc.Rule-based segmentation for objects with frontal view in color imagesUS20100278426 *12 Dec 20084 Nov 2010Robinson PiramuthuSystems and methods for rule-based segmentation for objects with full or partial frontal view in color imagesUS20110254837 *19 Apr 201120 Oct 2011Lg Electronics Inc.Image display apparatus and method for controlling the sameUS20110300929 *3 Jun 20108 Dec 2011Microsoft CorporationSynthesis of information from multiple audiovisual sourcesUS20120027299 *20 Jul 20112 Feb 2012SET CorporationMethod and system for audience digital monitoringUS20120071239 *28 Nov 201122 Mar 2012Microsoft CorporationStereo video for gamingUS20130129190 *23 Nov 201023 May 2013Scott D. CohenModel-Based Stereo Matching* Cited by examinerClassifications U.S. Classification382/154, 382/173, 345/419, 345/619International ClassificationG06K9/34, G06K9/00, G09G5/00, G06T15/00Cooperative ClassificationG06T15/00, G06T2207/10012, G06K9/00355, H04N13/0239, A63F2300/1093, G06T7/0075European ClassificationH04N13/02A2, G06T15/00, G06T7/00R7S, G06K9/00G2Legal EventsDateCodeEventDescription20 Mar 2012CCCertificate of correction13 Feb 2006ASAssignmentOwner name: MICROSOFT CORPORATION,WASHINGTONFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GRAEPEL, THORE KH;BLAKE, ANDREW;HERBRICH, RALF;US-ASSIGNMENT DATABASE UPDATED:20100225;REEL/FRAME:17159/735Effective date: 20051110Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GRAEPEL, THORE KH;BLAKE, ANDREW;HERBRICH, RALF;REEL/FRAME:017159/0735Owner name: MICROSOFT CORPORATION, WASHINGTONRotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google