Patent ID: 11972529
Assignee: SNAP INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A computer-implemented method for displaying virtual content in a device comprising, at a processor of the device:
obtaining a texture map and a 3D mesh corresponding to a real-world object or environment;
determining localization of the device relative to the real-world object or environment;
identifying a portion of the real-world object or environment that is visible in a current camera feed captured using an image sensor of the device;
accessing modified texture data for the portion of the real-world object or environment directly from the current camera feed captured using the image sensor;
replacing texture data associated with the portion of the real-world object or environment in the texture map with the modified texture data, wherein the modified texture data is created by:
assigning a tracking reliability value to each frame captured from the image sensor, each tracking reliability value assigned based on data received by an inertial measurement unit sensor during capture of each frame; and
creating the modified texture data using a weighted blending process, wherein the weights of the weighted blending process are based on the tracking reliability values;
storing the modified texture data in association with the texture map;
receiving a transformation input from a source device;
generating virtual content by applying a transformation corresponding to the transformation input to the 3D mesh, the transformation mapping a plurality of vertices of the 3D mesh to a plurality of transformed vertices in a transformed geometry and populating the transformed vertices with texture data according the texture map, wherein the transformation comprises at least one of a reflection, a deformation, and a duplication; and
processing at least one current image of the real-world object or environment, captured using the image sensor, to superpose a view of the virtual content over the at least one current image.