Patent ID: 11936840
Assignee: TANZLE, INC.
Field: Audio-visual technology (Electrical engineering)
Classification: CPC H  G | IPC G  H

Claim 14:
15. A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a stereoscopic video image of a physical scene captured by a location and/or orientation tracked camera, the physical scene comprising an object;
storing data providing a model defining (i) dimensions of a three-dimensional green-screen region and (ii) a location and/or orientation of the three-dimensional green-screen region relative to the object;
receiving data identifying a location of the object;
identifying a three-dimensional region of space in the physical scene for the three-dimensional green-screen region based on (i) the location and/or orientation of the camera, and (ii) the data defining dimensions of the three-dimensional green-screen region and the location and/or orientation of the three-dimensional green-screen region relative to the object;
identifying an eyepoint pair in the physical scene;
identifying, for each eyepoint of the eyepoint pair, a position and/or orientation of the three-dimensional green-screen region relative to the eyepoint; and
rendering a modified stereoscopic video image onto a display surface in which the three-dimensional green-screen region is replaced with a depiction of a virtual three-dimensional object, the rendering comprising:
obtaining a three-dimensional model of the virtual three-dimensional object,
determining a three-dimensional boundary of the three-dimensional green-screen region,
determining, for each eyepoint of the eyepoint pair, a respective different perspective of the virtual three-dimensional object based on (i) the three-dimensional model of the virtual three-dimensional object and (ii) the position and/or orientation of the three-dimensional green-screen region relative to the eyepoint, and
for each eyepoint of the eyepoint pair, performing a projection rendering of the three-dimensional model of the virtual three-dimensional object based on the determined perspective of the eyepoint such that the depiction of the virtual three-dimensional object is constrained to be within the determined three-dimensional boundary of the three-dimensional green-screen region.