Source: http://patents.com/us-10229544.html
Timestamp: 2019-04-22 16:03:44
Document Index: 153648014

Matched Legal Cases: ['Application No. 13198216', 'Application No. 13198216', 'Application No. 13198216', 'Application No. 201310757195', 'Application No. 201310757195', 'Application No. 2013']

US Patent # 1,022,9544. Constructing augmented reality environment with pre-computed lighting - Patents.com
United States Patent 10,229,544
Steed , et al. March 12, 2019
Family ID: 1000003875256
15/815,634
US 20180075663 A1 Mar 15, 2018
15298528 Oct 20, 2016 9892562
13668953 Dec 20, 2016 9524585
2003/0179218 September 2003 Martins et al.
2011/0175913 July 2011 Sloan et al.
Ritschel, Tobias, et al. "The state of the art in interactive global illumination." Computer Graphics Forum. vol. 31. No. 1. Blackwell Publishing Ltd, 2012. (Year: 2012). cited by examiner .
"Precomputed Radiance Transfer", Retrieved From: https://en.wikipedia.org/wiki/recomputed_Radiance_Transfer , Mar. 14, 2013, 2 Pages. cited by applicant .
"Final Office Action Issued in U.S. Appl. No. 13/668,953", dated Apr. 14, 2015, 12 Pages. cited by applicant .
"Final Office Action Issued in U.S. Appl. No. 13/668,953", dated Dec. 23, 2015, 16 Pages. cited by applicant .
"Non-Final Office Action Issued in U.S. Appl. No. 13/668,953", dated Aug. 6, 2015, 14 Pages. cited by applicant .
"Non-Final Office Action Issued in U.S. Appl. No. 13/668,953", dated Oct. 24, 2014, 12 Pages. cited by applicant .
"Notice of Allowance Issued in U.S. Appl. No. 13/668,953", dated Jul. 15, 2016, 5 Pages. cited by applicant .
"Office Action Issued in European Patent Application No. 13198216.7", dated May 18, 2017, 4 Pages. cited by applicant .
"Office Action Issued in European Patent Application No. 13198216.7", dated Oct. 19, 2017, 10 Pages. cited by applicant .
"Supplemental Search Report Issued in European Patent Application No. 13198216.7", dated Mar. 21, 2014, 10 Pages. cited by applicant .
"Non-Final Office Action Issued in U.S. Appl. No. 15/298,528", dated Dec. 2, 2016,14 Pages. cited by applicant .
"First Office Action Issued in Chinese Patent Application No. 201310757195.X", dated Feb. 26, 2016, 11 Pages. cited by applicant .
"Notice of Allowance Issued in Chinese Patent Application No. 201310757195.X", dated Aug. 31, 2016, 4 Pages. cited by applicant .
"Notice of Allowance Issued in Japanese Patent Application No. 2013-261856", dated Nov. 30, 2017, 3 Pages. cited by applicant .
Diverdi, Stephen, "Towards Anywhere Augmentation", Retrieved from https://techtransfer.universityofcalifomia.edu/NCD/media/DiVerdi.pdf , Sep. 2007, 95 Pages. cited by applicant .
Fuchs, Carsten, "Methods for Real-Time Lighting", Retrieved From: https://www.cafu.de/files/Uni-DA-RealTimeLighting.pdf , Feb. 18, 2005, 122 Pages. cited by applicant .
Green, Chris, "Efficient Self-Shadowed Radiosity Normal Mapping", In Proceedings of the Advanced Real-Time Rendering in 3D Graphics and Games Course--SIGGRAPH 2007, Aug. 5, 2007, 4 Pages. cited by applicant .
Grosch, et al., "Consistent interactive augmentation of live camera images with correct near-field illumination", In Proceedings of the 2007 ACM symposium on Virtual reality software and technology, Nov. 5, 2007, pp. 125-132. cited by applicant .
Havemann, Sven, "Generative Mesh Modeling", Retrieved From: https://publikationsserver.tu-braunschweig.de/receive/dbbs_mods_00000008 , Nov. 16, 2005, 303 Pages. (Submitted in 2 parts). cited by applicant .
Hilton, Adrian, "Scene Modelling from Sparse 3D Data", In Journal of Image and Vision Computing, vol. 23, Issue 10, Sep. 20, 2005, pp. 1-21. cited by applicant .
Hyvarinen, Juha, "Surface Reconstruction of Point Clouds Captured with Microsoft Kinect", Retrieved From: https://www.theseus.fi/bitstream/handle/10024/42161/Hyvarinen_Juha.pdf , Retrieved Date: Mar. 10, 2014, 50 Pages. cited by applicant .
Jung, et al., "Enhancing X3D for advanced MR appliances", In Proceedings of the 12th International Conference on 3D Web Technology, Apr. 15, 2007, pp. 27-37. cited by applicant .
Kanbara, et al., "A Stereoscopic Video See-Through Augmented Reality System Based on Real-Time Vision-Based Registration", In Proceedings of the IEEE Virtual Reality, Mar. 18, 2000, 8 Pages. cited by applicant .
Kristensen, et al., "Precomputed local radiance transfer for real-time lighting design", In Proceedings of the ACM Transactions on Graphics, vol. 24, Issue 3., Jul. 2005, pp. 1208-1215. cited by applicant .
Loos, et al., "Delta Radiance Transfer", In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Mar. 9, 2012, pp. 1-6. cited by applicant .
Loscos, et al., "Interactive Virtual Relighting and Remodeling of Real Scenes", In Proceedings of the Workshop on Eurographics Rendering, Apr. 1999, pp. 1-19. cited by applicant .
"International Search Report and Written Opinion Issued in PCT Application No. PCT/US2013/068363", dated May 4, 2015, 19 Pages. cited by applicant .
"ISA European Patent Office, Invitation to Pay Additional Fees and, where Applicable, Protest Fee Issued in Application No. PCT/US2013/068363", dated Feb. 23, 2015, 11 Pages. cited by applicant .
Riera, et al., "Lighting simulation in augmented reality scenes: Teaching experience in interior design", In Proceedings of the 7th Iberian Conference on Information Systems and Technologies (CISTI), Jun. 20, 2012, pp. 1-6. cited by applicant .
Sloan, et al., "Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments", In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, vol. 21, Issue 3, Jul. 23, 2002, pp. 527-536. cited by applicant .
Wang, et al., "Relighting in spatial augmented reality", In Proceedings of the International Conference on Audio Language and Image Processing (ICALIP), Nov. 23, 2010, 6 Pages. cited by applicant .
Tsingos, Nicolas, "Pre-Computing Geometry-Based Reverberation Effects for Games", In Proceedings of the AES 35th International Conference: Audio for Games, Feb. 11, 2009, pp. 1-10. cited by applicant .
Van Krevelen, et al., "A Survey of Augmented Reality Technologies, Applications and Limitations", In Proceedings of the International Virtual Reality, vol. 9, Issue 2, Jun. 2010, 19 Pages. cited by applicant .
Ventura, et al., "Online environment model estimation for augmented reality", In Proceedings of the 8th IEEE International Symposium on Mixed and Augmented Reality, Oct. 19, 2009, pp. 103-106. cited by applicant.
1. A physical computer-readable storage device comprising stored instructions executable by a computing system to display an augmented reality image by: receiving image data acquired by an image sensor, the image data capturing an image of a local environment of the image sensor; identifying a physical surface of the local environment via the image data; constructing an image of a virtual structure for display over the surface in spatial registration with the surface by fitting a first modular image segment of the virtual structure to the surface based on a geometry of the surface by one or more of scaling, rotating, clipping, and morphing the first modular image segment, and then positioning a second modular image segment of the virtual structure adjacent to the first modular image segment to form an appearance of the virtual structure, each modular image segment having an appearance of a portion of the virtual structure; and outputting the image for display by a display device.
2. The physical computer-readable storage device of claim 1, wherein the first and second modular image segments comprise a pre-computed global illumination effect.
3. The physical computer-readable storage device of claim 2, wherein the pre-computed global illumination effect comprises a pre-computed directional lighting effect.
4. The physical computer-readable storage device of claim 2, wherein the pre-computed global illumination effect comprises a pre-computed radiance transfer function.
5. The physical computer-readable storage device of claim 1, wherein the instructions executable to identify the physical surface of the local environment are executable to perform a mesh analysis of the local environment.
6. The physical computer-readable storage device of claim 1, wherein the physical surface comprises one or more of a wall and a ceiling.
7. The physical computer-readable storage device of claim 1, wherein the physical surface comprises a non-structural object within the local environment.
8. The physical computer-readable storage device of claim 1, wherein the instructions are further executable to identify a lighting characteristic of the local environment via the image data, and modulate an appearance of the first and second modular image segments based upon the lighting characteristic of the local environment.
9. The physical computer-readable storage device of claim 8, wherein the lighting characteristic comprises a color characteristic of the local environment, and wherein the instructions executable to modulate the appearance of the first and second modular image segments are executable to impart the color characteristic to the modular image segments.
10. The physical computer-readable storage device of claim 8, wherein the lighting characteristic comprises a location of a physical light in the local environment, and wherein the instructions executable to modulate the appearance of the first and second modular image segments are executable to compute a light effect arising from a virtual point light at the location of the physical light.
11. The physical computer-readable storage device of claim 1, wherein the instructions executable to construct the image of the virtual structure are executable to identify a local lighting characteristic, and select a set of modular image segments determined to have a similar pre-computed lighting effect.
12. The physical computer-readable storage device of claim 1, wherein the instructions executable to construct the image of the virtual structure are executable to apply a connectivity constraint to select adjacent modular image segments.
13. A physical computer-readable storage device comprising stored instructions executable by a computing system to display an augmented reality image by: receiving image data acquired by an image sensor, the image data capturing an image of a local environment of the image sensor; identifying a physical feature of the local environment via the image data, the physical feature comprising a furniture structure; constructing an image of a virtual structure for display over the physical feature in spatial registration with the physical feature by fitting a first modular image segment of the virtual structure to the feature based on a geometry of the feature by one or more of scaling, rotating, clipping, and morphing the first modular image segment, and then positioning a second modular image segment of the virtual structure adjacent to the first modular image segment to form an appearance of the virtual structure, each modular image segment having an appearance of a portion of the virtual structure; and outputting the image for display by a display device.
14. The physical computer-readable storage device of claim 13, wherein the instructions executable to identify a physical feature of the local environment are executable to perform a mesh analysis of the local environment.
15. The physical computer-readable storage device of claim 13, wherein the first and second modular image segments comprise a pre-computed global illumination effect.
16. The physical computer-readable storage device of claim 15, wherein the pre-computed global illumination effect comprises one or more of a pre-computed directional lighting effect and radiance transfer function information.
17. The physical computer-readable storage device of claim 13, wherein the instructions are further executable to detect a lighting characteristic of the local environment via the image data, and modulate an appearance of the first and second modular image segments based upon the lighting characteristic of the local environment.
18. The physical computer-readable storage device of claim 13, wherein the instructions are further executable to detect a local lighting characteristic, and select a set of modular image segments determined to have a similar pre-computed lighting effect as the local lighting characteristic.
19. A system for displaying an augmented reality image, the system comprising: an image sensor; a logic machine; and a storage machine holding instructions executable by the logic machine to receive image data acquired by the image sensor, the image data capturing an image of a local environment of the image sensor; identify a physical surface of the local environment via the image data; construct an image of a virtual structure for display over the surface in spatial registration with the surface by fitting a first modular image segment of the virtual structure to the surface based on a geometry of the surface by one or more of scaling, rotating, clipping, and morphing the first modular image segment, and then positioning a second modular image segment of the virtual structure adjacent to the first modular image segment to form an appearance of the virtual structure, each modular image segment having an appearance of a portion of the virtual structure; and output the image for display by a display device.
Previous Patent US 10,229,543 | Next Patent US 10,229,545