Patent Document ID: 10062210
Application ID: 14189838
Patent Status: 1

Claim One:
1. A method, comprising: receiving at least one video frame of an environment; generating a surface reconstruction of the environment; projecting a plurality of rays within the surface reconstruction of the environment; generating illumination data of the environment from the at least one video frame; determining a subset of rays from the plurality of rays in the environment, wherein the determining a subset of rays from the plurality of rays comprises (1) applying adaptive sampling techniques based on a 4-D subdivision of a ray space comprising the plurality of rays into a plurality of 4-D regions and further based on an approximated variance analysis, wherein the adaptive sampling techniques comprise applying a weighting mask to the surface reconstruction of the environment and inspecting the weighting mask to determine a location where more samples should be computed, wherein the approximated variance analysis comprises computing, for each 4-D region, a variance value based on visibility information for rays in the 4-D region, and using the variance value as a contributing factor to the weighting mask, and (2) applying differential rendering techniques to the environment and analyzing shadow information as a result thereof; generating estimated real world lighting data of the environment associated with the at least one video frame, based on the surface reconstruction, the illumination data, and the plurality of rays excluding the subset of rays; and rendering a virtual object over the at least one video frame using the estimated real world lighting data.