Source: http://www.google.com/patents/USRE41175?ie=ISO-8859-1&dq=2040248
Timestamp: 2015-03-27 03:39:16
Document Index: 555317338

Matched Legal Cases: ['application No. 11', 'application No. 12', 'application No. 10', 'Application No. 60', 'art 3', 'art 1', 'art 5']

Patent USRE41175 - GPS-enhanced system and method for automatically capturing and co ... - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA system for capturing a virtual model of a site includes a range scanner for scanning the site to generate range data indicating distances from the range scanner to real-world objects. The system also includes a global positioning system (GPS) receiver coupled to the range scanner for acquiring GPS...http://www.google.com/patents/USRE41175?utm_source=gb-gplus-sharePatent USRE41175 - GPS-enhanced system and method for automatically capturing and co-registering virtual models of a siteAdvanced Patent SearchPublication numberUSRE41175 E1Publication typeGrantApplication numberUS 11/480,248Publication dateMar 30, 2010Filing dateJun 30, 2006Priority dateJan 22, 2002Fee statusPaidAlso published asUS6759979, US20030137449, WO2003062849A2, WO2003062849A3Publication number11480248, 480248, US RE41175 E1, US RE41175E1, US-E1-RE41175, USRE41175 E1, USRE41175E1InventorsRobert M. Vashisth, James U. Jensen, James W. BungerOriginal AssigneeIntelisum, Inc.Export CitationBiBTeX, EndNote, RefManPatent Citations (25), Non-Patent Citations (137), Referenced by (12), Classifications (22), Legal Events (2) External Links: USPTO, USPTO Assignment, EspacenetGPS-enhanced system and method for automatically capturing and co-registering virtual models of a site
The applicationNotice: More than one reissue application has been filed for the reissue of Pat. No. 6,759,979. The reissue applications are application No. 11/480,248 (the present application filed on Jun. 30, 2006 ) and application No. 12/362,954 (filed on Jan. 30, 2009 ), all of which are reissue or divisional reissue applications of Pat. No. 6,759,979. The present application is a reissue application of U.S. Pat. No. 6,759,979 (application No. 10/348,275 ), which claims the benefit of U.S. Provisional Application No. 60/350,860, filed on Jan. 22, 2002, for �System and Method for Generating 3-D Topographical Visualizations,� with inventors Munish Vashisth and James U. Jensen, whicheach application isidentified above being incorporated herein by this reference in its entirety.
Physical limitations of the range scanner constrain the maximum resolution of the range data, which decreases with distance from the range scanner. At large distances, the range scanner may not be able to discern surface details of an object. A lack of continuous spatial data (gaps between points) and a lack of color attributes are significant limitations of conventional range scanners. Furthermore, a range scanner only scans objects within the lidar's line-of-sight. As a result, no data is collected for the side of an object opposite to the lidar or for objects obscured by other objects (�occlusions�).
Unfortunately, the merging of sets of range data is not automatic. Human decision-making is generally required at several steps in the merging process. For instance, a human surveyor is typically needed to determine the relative distances between the range scanning locations and the scanned area. Furthermore, a human operator must manually identify points in common (�fiducials�) between multiple sets of range data in order to align and merge the sets into a single model. Such identification is by no means easy, particularly in the case of curved surfaces. The need for human decision-making increases the cost of modeling and the likelihood of error in the process.
In one embodiment, the range scanner 102 sends the range data, digital images, and GPS data to a computer 108, where they are used to create the visualization 112. The visualization 112 may be interactive, e.g., a user may �walk through� the site 104 depicted in the visualization 112. In addition, the user may delete or move objects depicted in the visualization 112 or modify the visualization 112 in other ways. Such visualizations 112 are highly beneficial in the fields of architecture, landscape design, land use, erosion control, etc.
The digital camera 110 may include a PowerShot G2� camera available from Canon, Inc. In one configuration, the digital camera 110 is capable of capturing images with a resolution of 2272�1704 pixels at a rate of approximately 2.5 images per second. The digital camera 110 may be included within, attached to, or otherwise integrated with the range scanner 102. In alternative embodiments, the range scanner 102 includes multiple digital cameras 110.
The GPS receiver 116 may be embodied as a standard mapping-grade receiver, which may support L-band differential GPS (DGPS). Where higher accuracy is needed, survey-grade receivers may be used, such as a carrier phase (CPH) or real-time kinematic (RTK) GPS. In such embodiments, a base station (not shown) having a known Earth location broadcasts an error correction signal that is used by the GPS receiver 116 to achieve accuracy to within a few centimeters. An example of a suitable GPS receiver 116 is the ProMark2� survey system available from Ashtech, Inc. of Santa Clara, Calif. Like the digital camera 110, the GPS receiver 116 may be included within, attached to, or otherwise integrated with the range scanner 102.
Some GPS receivers 116 may include compasses, gyroscopes, inertial navigation systems, etc., for providing highly accurate bearing and/or other orientation information. For example, the ProMark2� survey system described above provides an azimuth reading. Similarly, a bearing may be obtained indirectly from GPS readings, since two precise GPS coordinates define a bearing. Thus, the orientation indicator 202 need not be separate component.
The communication interface 210 of the range scanner 102 is coupled to, or in communication with, a similar communication interface 212 within the computer 108. The computer 108 may be embodied as a standard IBM-PC� compatible computer running a widely-available operating system (OS) such as Windows XP� or Linux�.
The computer 108 also includes a central processing unit (CPU) 214, such as an Intel� x86 processor. The CPU 214 is coupled to a standard display interface 216 for displaying text and graphics, including the visualization 112, on the monitor 114. The CPU 214 is further coupled to an input interface 218 for receiving data from a standard input device, such as a keyboard 220 or mouse 222.
In one configuration, the computer 108 includes a mass storage device 236, such as a hard disk drive, optical storage device (e.g., DVD-RW), or the like, which may be used to store any of the above-described modules or data structures. Hence, references herein to �memory� or �storage media� should be construed to include any combination of volatile, non-volatile, magnetic, or optical storage media.
Referring to FIG. 3, one or more virtual models 234 of a site 104 may be captured at each of a number of positions or vantage points. At each position, the range scanner 102 generates range data 302 indicating distances to objects (e.g., a tree) within the site 104. A set of range data 302 is sometimes referred to as a �range image,� although the range data 302 need not be stored or presented in a conventional image format. The terms �range data� and �range image� are used herein interchangeably.
In certain instances, a data structure lacking one or more of the above-described elements may still be referred to as a �virtual model.� For example, a virtual model 234 may not include the digital images 306 or certain orientation information 305 (such as tilt data where the range scanner 102 is level during scanning).
FIG. 4 depicts the three sets of range data 302a-c from top-down views rather than the side views of FIG. 3. As shown, each set represents only a portion of the site 104, since the range scanner 102 is not able to �see� behind objects.
Referring to FIG. 6, in order to combine or �co-register� the virtual models 234 from the various scanning positions, the transformation module 229 transforms the range data 302a-c from their respective scanning coordinate systems 402a-c to a single modeling coordinate system 602 that is independent of the scanning positions and the orientation of the range scanner 102.
GPSE is the UTM �easting,� GPSH is the altitude (typically the height above the reference ellipsoid), and GPSN is the UTM �northing.�
The UTM easting and northing for a number of points in the modeling coordinate system 602 are shown in FIG. 7, and are typically represented on maps using the �mE� (meters East) and �mN� (meters North) labels.
As illustrated in FIG. 7, a co-registered model 702 is not perfect. Noise and other sources of error may result in various gaps, incongruities, regions of overlap, etc. Thus, while the co-registration module 228 automatically places the range data 302a-c within close proximity to their expected locations, eliminating the need for human decision-making, the range data 302a�c are not truly merged. For example, two separate points may exist within the co-registered model 702 that should actually refer to the same physical location in the site 104.
Referring to FIG. 8, a merging module 230 addresses this problem by merging the range data 302a-c from the co-registered model 702 into a single merged model 802. The merging module 230 makes fine adjustments to the transformed range data 302a�c, eliminating the gaps, incongruities, and regions of overlap. In addition, the merging module 230 may eliminate redundancy by merging points from the transformed range data 302a that represent the same physical location. This is accomplished, in one embodiment, using an iterative closest point (ICP) algorithm, as known to those of skill in the art.
In one embodiment, the merging module 230 incorporates the Scanalyze� product available from Stanford University. Scanalyze� is an interactive computer graphics application for viewing, editing, aligning, and merging range images to produce dense polygon meshes.
Scanalyze� processes three kinds of files: triangle-mesh PLY files (extension .ply), range-grid PLY files (also with extension .ply), and SD files (extension .sd). Triangle-mesh PLY files encode general triangle meshes as lists of arbitrarily connected 3D vertices, whereas range-grid PLY files and SD files encode range images as rectangular arrays of points. SD files also contain metadata that describe the geometry of the range scanner 102 used to acquire the data. This geometry is used by Scanalyze� to derive line-of-sight information for various algorithms. PLY files may also encode range images (in polygon mesh form), but they do not include metadata about the range scanner and thus do not provide line-of-sight information.
Once the PLY or SD files have been loaded, they can be pairwise aligned using a variety of techniques�some manual (i.e. pointing and clicking) and some automatic (using a variant of the ICP algorithm).
The visualization module 232 also decomposes the digital images 306 into textures 904, which are then applied to the polygon mesh 902. In essence, the digital images 306 are �draped� upon the polygon mesh 902. Due to the relatively higher resolution of the digital images 306, the textures 904 add a high degree of realism to the visualization 112. Techniques and code for applying textures 904 to polygon meshes 902 are known to those of skill in the art.
In one embodiment, the mesh 902 and textures 904 are used to create the visualization 112 of the site 104 using a standard modeling representation, such as the virtual reality modeling language (VRML). Thereafter, the visualization 112 can be viewed using a standard VRML browser, or a browser equipped with a VRML plugin, such as the Microsoft� VRML Viewer. Of course, the visualization 112 could also be created using a proprietary representation and viewed using a proprietary viewer.
As depicted in FIG. 9, the browser may provide navigation controls 906 to allow the user to �walk through� the visualization 112. In addition, the user may delete or move objects shown in the visualization 112 or modify the visualization 112 in other ways. As noted, such visualization 112 are highly beneficial in the fields of architecture, landscape design, land use, erosion control, and the like.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS5337149Nov 12, 1992Aug 9, 1994Kozah Ghassan FComputerized three dimensional data acquisition apparatus and methodUS5988862Apr 24, 1996Nov 23, 1999Cyra Technologies, Inc.Integrated system for quickly and accurately imaging and modeling three dimensional objectsUS6166744Sep 15, 1998Dec 26, 2000Pathfinder Systems, Inc.System for combining virtual images with real-world scenesUS6246468Oct 23, 1998Jun 12, 2001Cyra TechnologiesIntegrated system for quickly and accurately imaging and modeling three-dimensional objectsUS6249600Nov 26, 1997Jun 19, 2001The Trustees Of Columbia University In The City Of New YorkSystem and method for generation of a three-dimensional solid modelUS6292215Jul 24, 1997Sep 18, 2001Transcenic L.L.C.Apparatus for referencing and sorting images in a three-dimensional systemUS6307556Mar 27, 1995Oct 23, 2001Geovector Corp.Augmented reality vision systems which derive image information from other vision systemUS6330523Oct 23, 1998Dec 11, 2001Cyra Technologies, Inc.Integrated system for quickly and accurately imaging and modeling three-dimensional objectsUS6420698Oct 23, 1998Jul 16, 2002Cyra Technologies, Inc.Integrated system for quickly and accurately imaging and modeling three-dimensional objectsUS6473079Oct 23, 1998Oct 29, 2002Cyra Technologies, Inc.Integrated system for quickly and accurately imaging and modeling three-dimensional objectsUS6526352Jul 19, 2001Feb 25, 2003Intelligent Technologies International, Inc.Method and arrangement for mapping a roadUS6664529 *Jan 15, 2002Dec 16, 2003Utah State University3D multispectral lidarUS6759979Jan 21, 2003Jul 6, 2004E-Businesscontrols Corp.GPS-enhanced system and method for automatically capturing and co-registering virtual models of a siteUS20010010546Sep 26, 1997Aug 2, 2001Shenchang Eric ChenVirtual reality cameraUS20020060784Jan 15, 2002May 23, 2002Utah State University3D multispectral lidarUS20030090415Oct 24, 2002May 15, 2003Mitsui & Co., Ltd.GPS positioning systemUS20040105573Sep 30, 2003Jun 3, 2004Ulrich NeumannAugmented virtual environmentsUS20050057745Sep 14, 2004Mar 17, 2005Bontje Douglas A.Measurement methods and apparatusWO1997040342A2Apr 24, 1997Oct 30, 1997Mark BrunkhartIntegrated system for imaging and modeling three-dimensional objectsWO2001004576A1Jul 14, 2000Jan 18, 2001Cyra Technologies IncMethod for operating a laser scannerWO2001088565A2Apr 27, 2001Nov 22, 2001Cyra Technologies IncApparatus and method for identifying the points that lie on a surface of interestWO2001088566A2Apr 27, 2001Nov 22, 2001Cyra Technologies IncSystem and method for acquiring tie-point location information on a structureWO2001088741A2Apr 27, 2001Nov 22, 2001Cyra Technologies IncSystem and method for concurrently modeling any element of a modelWO2001088849A2Apr 27, 2001Nov 22, 2001Cyra Technologies IncApparatus and method for forming 2d views of a structure from 3d point dataWO2002016865A2Aug 24, 2001Feb 28, 20023Shape ApsObject and method for calibration of a three-dimensional light scanner* Cited by examinerNon-Patent CitationsReference1""Automatic" Multimodal Medical Image Fusion," http://csdl2.computer.org/persagen/DLAbsToc.jsp, Jul. 7, 2006, pp. 1.2"2-D and 3-D Image Registration: A Tutorial," http://www.cs.wright.edu/~agoshtas/CVPR04_Registration_Tutorial.html, Jul. 7, 2006, pp. 1-3.3"3D Reality Modelling: Photo-Realistic 3D Models of Real World Scenes," Vitor Sequeira, Jo�o G.M. Gon�alves, Proceedings of the First International Symposium on 3D Data Processing Visualization and Transmission, 2002, pp. 1-8.4"A Contour-Based Approach to Multisensor Image Registration," Hui Li, B. S. Manjunath, Sanjit K. Mitra, IEEE Transactions On Image Processing, vol. 4, No. 3, Mar. 1995, pp. 320-334.5"A flexible mathematical model for matching of 3D surfaces and attributes," Devrim Akca, Armin Gruen, Electronic Imaging, SPIE vol. 5665, 2005, pp. 184-195.6"A Multi-Resolution ICP with Heuristic Closest Point Search for Fast and Robust 3D Registration of Range Images," Timoth�e Jost and Heinz H�gli, Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-7.7"A Point-and-Shoot Color 3D Camera," Askold V. Strat, Manuel M. Oliveira, Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.8"A simple MATLAB interface to FireWire cameras," F. W�rnle, May 2006, pp. 1-25.9"A Survey of Medical Image Registration," J. B. Antoine Maintz and Max A. Viergever, Image Sciences Institute, Utrecht University Hospital, Utrecht, the Netherlands, Oct. 16, 1997, pp. 1-37.10"Adaptive Enhancement of 3D Scenes using Hierarchical Registration of Texture-Mapped 3D models," Srikumar Ramalingam and Suresh K. Lodha, Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.11"Advanced Nonrigid Registration Algorithms for Image Fusion," Simon K. Warfield et al., Brain Mapping: The Methods, Second Edition, 2002, pp. 661-690.12"Alignment by Maximization of Mutual Information," Paul A. Viola, Massachusetts Institute of Technology, 1995, pp. 1-156.13"An Integrated Multi-Sensory System for Photo-Realistic 3D Scene Reconstruction," Kia Ng et al., School of Computer Studies, University of Leeds, European Commission-Joint Research Centre, pp. 1-9.14"Automated reconstruction of 3D models from real environments," V. Sequeira, K. Ng, E. Wolfart, J.G.M. Gon�alves, D. Hogg, ISPRS Journal of Photogrammetry & Remote Sensing 54, 1999, pp. 1-22.15"Automated Registration and Evaluation of Laser Scanner Point Clouds and Images, Automatic Point Cloud Registration Using Template Shaped Targets," http://www.photogrammetry.ethz.ch/research/pointcloud/withtargets.html, Aug. 25, 2004.16"Automated Registration, 3DD Optix 400 Series,"3D Digital Corp., Apr. 2004.17"Automatic Registration of Range Images Based on Correspondence of Complete Plane Patches," Wenfeng He, Wei Ma, Hongbin Zha, Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-6.18"Combining texture and shape for automatic crude patch registration," Joris Vanden Wyngaerd et al., Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.19"Consistent Linear-Elastic Transformations for Image Matching," Gary E. Christensen, A. Kuba et al. (Eds.): IPIM'99, LNCS 1613, 1999, pp. 224-237.20"Edge and Line Detection Image Fusion Systems Research," http://www.imgfsr.com/ifsr_is_ed.html, Jul. 7, 2006, pp. 1-8.21"Effective Nearest Neighbor Search for Aligning and Merging Range Images," Ryusuke Sagawa et al., Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling. 2003, pp. 1-8.22"Enhanced, Robust Genetic Algorithms for Multiview Range Image Registration," Luciano Silva et al., Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.23"Fast Normalized Cross-Correlation," http://www.idiom.com/~zilla/Work/nvisionInterface/nip.htm, Jul. 7, 2006, pp. 1-11.24"Feature point detection in multiframe images," Barbara Zitova et al., Czech Pattern Recognition Workshop 2000, Feb. 2, 2000-Feb. 4, 2000, pp. 1-6.25"Fully automatic registration of multiple 3D data sets," Daniel F. Huber, Martial Hebert, Image and Vision Computing 21, 2003, pp. 637-650.26"Fusion and interpretation of medical images," http://www.research.ibm.com/hc/VISUALIZE/visualize.html, Jul. 7, 2006, pp. 1-3.27"Future standard," http://www.medicalimagingmag.com/issues/articles/2000-08_01.asp, Jul. 7, 2006, pp. 1.28"Gaussian Random Fields on Sub-Manifolds for Characterizing Brain Surfaces," http://portal.acm.org/citation.cfm?id=645595.660540&coll=GUIDE&d..., Jul. 6, 2006, pp. 1-3.29"HLODs: Hierarchical Levels of Detail, hierarchical Simplification for Faster Display of Massive Geometric environments," Department of Computer Science, University of North Carolina at Chapel Hill, Feb. 2004.30"Image Fusion Systems Research," http://www.imgfsr.com/, Jul. 7, 2006, pp. 1-2.31"Image Fusion-a new era in diagnosis," Dr Michael Kitchener, Imaging Update, Issue 8, Apr. 2002, pp. 1-4.32"Image Guidance Laboratories," Complete Laboratory Publications, http://www-igl.stanford.edu/papers.php?pg=main..., Jul. 7, 2006.33"Image Image-Guided Interventions Workshop Guided Interventions Workshop," John Haller, National Institute of Biomedical Imaging and Bioengineering, May 13, 2004-May 14, 2004, pp. 2-11.34"Image Registration and Mosaicking," http://prettyview.com/mtch/mtch.shtml, Jul. 7, 2006, pp. 1.35"Image registration methods: a survey," Barbara Zitova et al., Department of Image Processing, Institute of Information Theory and Automation, Academy of Sciences of the Czech Republic, Jun. 26, 2003, pp. 977-1000.36"Image-Based Registration of 3D-Range Data Using Feature Surface Elements," Gerhard Heinrich Bendels et al., Institute for Computer Science II-Computer Graphics, University of Bonn, Germany, 2004, pp. 1-10.37"Infrastructure for Image Guided Surgery," John W. Haller, Timothy C. Ryken, Thomas A. Gallagher and Michael W. Vannier, National Institute of Neurological Disease and Stroke, Jun. 28, 2001, pp. 1-6.38"Inverse Problems: Image Restoration and Parameter Identification," http://www.mit.jyu.fi/majkir/tutkimus/index.html, Tommi K�rkk�inen and Kirsi Majava, University of Jyv�skyl� Dept. of Mathematical Information Technology, Aug. 2, 2004, pp. 1-3.39"Investigations of Image Fusion," http://www.ece.lehigh.edu/SPCRL/IF/image_fusion.htm, Jul. 7, 2006, pp. 1-13.40"iPhotoMeasure-The Contractors Photo Measuring Tool," http://www.iphotomeasure.com/faq.asp, Jan. 25, 2007.41"Keith Price Bibliography Fusion of Medical Data," http://iris.usc.edu/Vision-Notes/bibliography/match-p1504.html, Jul. 7, 2006.42"Least Squares 3D Surfaces Matching," Armin Gruen, Devrim Akca, Geospatial Goes Global: From Your Neighborhood to the Whole Planet, ASPRS 2005 Annual Conference, Mar. 7, 2005-Mar. 11, 2005, pp. 1-13.43"Lecture Notes In Computer Science," http://portal.acm.org/toc.cfm?id=645595&type=proceeding&coll=GUI..., Jul. 6, 2006, pp. 1-6.44"Leica Cyclone 5.6 Register," Leica Geosystems, Heerbrugg, Switzerland, 2006.45"Medical image fusion by wavelet transform modulus maxima," Guihong Qu, Dali Zhang and Pingfan Yan, Department of Automation, Tsinghua University, Beijing 100084,China, May 10, 2001, pp. 184-190.46"Medical Image Fusion," http://www.uihealthcare.com/news/currents/vol1issue1/figimfus.html, Jul. 6, 2006, pp. 1-4.47"Medical Image Processing," http://www.sce.carleton.ca/faculty/adler/elg7173/elg7173.html, Jul. 7, 2006, pp. 1-5.48"Multi-modality image registration using mutual information based on gradient vector flow," Yujun Guo, May 1, 2006, pp. 1-31.49"Multi-Sensor Image Fusion Using the Wavelet Transform," Hui Li, B.S. Manjunath, Sanjit K. Mitra, IEEE, 1994, pp. 51-55.50"Non-parametric 3D Surface Completion," Toby P. Breckon, Robert B. Fisher, Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.51"Open Scene Graph Home Page," http://www.openscenegraph.org/osgwiki/pmwiki.php/Home/HomePage, Jan. 25, 2007.52"Project: Fully Automated Registration and Composite Generation of Multisensor and Multidate Satellite Image Data," http://vision.ece.ucsb.edu/registration/satellite/, Jul. 7, 2006, pp. 1-2.53"Projective Surface Matching of Colored 3D Scans," Kari Pulli et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.54"Registration and Integration of Textured 3-D Data," Andrew Johnson and Sing Bing Kang, Digital Equipment Corporation Cambridge Research Lab, Sep. 1996, pp. 1-48.55"Reliability of Functional MRI for Motor and Language Cortex Activation," http://dolphin.radiology.uiowa,edu/ge/BME/public_html/projects/project, Jul. 6, 2006, pp. 1.56"Robust Detection of Significant Points in Multiframe Images," http://staff.utia.cas.cz/zitova/corners.htm, Jul. 7, 2006, pp. 1-5.57"Selected Papers on Image Registration, Image Fusion Systems Research," http://www.imgfsr.com/ifsr_irb.html, Jul. 7, 2006.58"The Registration of Three Dimensional Images from one or more Imaging Modalities," http://www.biomed.abdn.ac.uk/Abstracts/A00331/, Jul. 7, 2006, pp. 1-11.59"Workshop on symmetries, inverse problems and image processing," http://www.indmath.uni-linz.ac.at/people/bila/Workshop.html, Jan. 12, 2005, pp. 1-4.60"2-D and 3-D Image Registration: A Tutorial," http://www.cs.wright.edu/�agoshtas/CVPR04_Registration_Tutorial.html, Jul. 7, 2006, pp. 1-3.61"3D Modeling of Outdoor Environments by Integrating Omnidirectional Range and Color Images," Toshihiro ASAI et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.62"An Integrated Multi-Sensory System for Photo-Realistic 3D Scene Reconstruction," Kia Ng et al., School of Computer Studies, University of Leeds, European Commission�Joint Research Centre, pp. 1-9.63"Construction of Large-Scale Virtual Environment by Fusing Range Data, Texture Images, and Airborne Altimetry Data," Conny Riani Gunadi et al., Proceedings of the First International Symposium on 3-D Data Processing Visualization and Transmission, 2002 pp. 1.64"Construction of Large-Scale Virtual Environment by Fusing Range Data, Texture Images, and Airborne Altimetry Data," Conny Riani Gunadi et al., Proceedings of the First International Symposium on 3-D Data Processing Visualization and Transmission, 2002, pp. 1-4.65"Correction of color information of a 3D model using a range intensity image," Kazunori Umeda et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.66"Evaluating Collinearity Constraint for Automatic Range Image Registration," Yonghuai Liu et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.67"Fast Alignment of 3D Geometrical Models and 2D Color Images using 2D Distance Maps," Yumi Iwashita et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.68"Fast Normalized Cross-Correlation," http://www.idiom.com/�zilla/Work/nvisionInterface/nip.htm, Jul. 7, 2006, pp. 1-11.69"Image Fusion�a new era in diagnosis," Dr Michael Kitchener, Imaging Update, Issue 8, Apr. 2002, pp. 1-4.70"Image-Based Object Editing," Holly Rushmeier et al., Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.71"Image-Based Registration of 3D-Range Data Using Feature Surface Elements," Gerhard Heinrich Bendels et al., Institute for Computer Science II�Computer Graphics, University of Bonn, Germany, 2004, pp. 1-10.72"iPhotoMeasure�The Contractors Photo Measuring Tool," http://www.iphotomeasure.com/faq.asp, Jan. 25, 2007.73"Registration and Fusion of Intensity and Range Data for 3D Modelling of Real World Scenes," Paulo Dias et al., Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, 2003, pp. 1-8.74"Semi-automatic range to range registration: a feature-based method," Chen Chao et al., Proceedings of the Fifth International Conference on 3-D Digital Imaging and Modeling, 2005, pp. 1-8.75 *A Volumetric Method for Building Complex Models from Range Images by Brian Curless et al.; Proc. SIGGRAPH '96, Aug. 1995; pp. 1-10.76 *Allen, P. et al, Avenue: automated site modeling in urban environments-3-D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on, 2001, pp. 357-364.77 *Allen, P. et al, Avenue: automated site modeling in urban environments�3-D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on, 2001, pp. 357-364.78Ameesh Makadia, et al., "Fully Automatic Registration of 3D Point Clouds," Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006.79Andrew Johnson, et al. "Registration and Integration of Textured 3-D Data," Bibliography, 1996.80Armin Gruen, et al., "Least Squares 3D Surface and Curve Matching," Elsevier, ISPRS Journal of Photogrammetry & Remote Sensing 59, Feb. 16, 2005.81Ayman F. Habib, et al., "Automatic Surface Matching for the Registration of LIDAR Data and MR Imagery," ETRI Journal, vol. 28, No. 2, Apr. 2006.82B. Vangelder, et al., "Modern Technologies for Design Data Collection. " Civil Engineering, Joint Transportation Research Program, Purdue Libraries, 2005.83 *Beraldin, J-A. et al, "Portable Digital 3-d Imaging System For Remote Sites"-Circuits and Systems, 1998. ISCAS '98. Proceedings of the 1998 IEEE International Symposium on, pp. V488-V493.84 *Beraldin, J-A. et al, "Portable Digital 3-d Imaging System For Remote Sites"�Circuits and Systems, 1998. ISCAS '98. Proceedings of the 1998 IEEE International Symposium on, pp. V488-V493.85 *Berladin, J-A. et al, "Object Model Creation From Multiple Range Images: Acquisition, Calibration, Model Building And Verification," 3-D Digital Imaging and Modeling, 1997. Proceedings., International Conference on Recent Advances,pp. 326-333.86 *Besl, Paul, "Geometric modeling and computer vision"-Proceedings of the IEEE, vol. 76, No. 8, Aug. 1988, pp. 936-958.87 *Besl, Paul, "Geometric modeling and computer vision"�Proceedings of the IEEE, vol. 76, No. 8, Aug. 1988, pp. 936-958.88Christoph Dold, "Extended Gaussian Images for the Registration of Terrestrial Scan Data," Institute of Cartography and Geoinformatics, Workshop "Laser Scanning 2005," Sep. 1214, 2005.89Christoph Dold, et al., "Automatic Matching of Terrestrial Scan Data as a Basis for the Generation of Detailed 3D City Models," International Archives of Phtotogrammetry Remote Sensing and Spatial Information Sciences, vol. 35, Part 3, p. 1091-1096, 2004.90D. Akca, "Registration of Point Clouds Using Range and Intensity Information," International Workshop on Recording, Modeling and Visualization of Cultural Heritage, 2005.91David T. Gering, et al., An Integrated Visualization System for Surgical Planning and Guidance using Image Fusion and Interventional Imagining, MIT AI Laboratory, 1999.92Dinesh Manadhar, "Extraction of linear features from vehicle-borne laser data," Remote Sensing, Singapore, vol. 2, p. 113-1118, Nov. 5-9, 2001.93Faysal Boughorbal, et al., "Registration and Integration of Multi-Sensor Data for Photo-realistic Scene Reconstruction," Applied Imagery Pattern Recognition, Sponsored by SPIE, Washington, DC, Oct. 13-15, 1999.94Fiorella Sgallari, "Numerical Solution of Inverse Problems in Image Processing," University of Bologna, Presented at Association for Computing Machinery Conference, Jan. 13, 2006.95Geoff Jacobs, "Field Productivity Factors in Laser Scanning, Part 1," Professional Surveyor Magazine, Jan. 2007.96Google Search, "Medical Image Fusion," http://www.google.com/search?hl=cn&q=medical+image+fusion&btnG..., Jul. 7, 2006.97 *Gueorguiev, A. et al, "Design, architecture and control of a mobile site-modeling robot"-Robotics and Automation, 2000. Proceedings. ICRA '00. IEEE International Conference on, pp. 3266-3271.98 *Gueorguiev, A. et al, "Design, architecture and control of a mobile site-modeling robot"�Robotics and Automation, 2000. Proceedings. ICRA '00. IEEE International Conference on, pp. 3266-3271.99Hans J. Johnson, et al., "Consistent Landmark and Intensity-based Image Registration," IEEE Transactions on Medical Imaging, p. 126, 2002.100Helmut Pottmann, et al., "Registration without ICP," Geometric Modeling and Industrial Geometry Group, Vienna University of Technology, Mar. 5, 2004.101Hendrik P. A. Lensch, et al., "Automated Texture Registration and Stitching for Real World Models," Max-Planck-Institute for Computer Science, Baarbrucken, Germany, 2000.102Isi-Gi, Dec. 21, 2006.103 *Kamgar-Parsi et al, "Registration Algorithms For Geophysical Maps"-Oceans '97, MTS/IEEE Conference Proceedings, pp. 974-980.104 *Kamgar-Parsi et al, "Registration Algorithms For Geophysical Maps"�Oceans '97, MTS/IEEE Conference Proceedings, pp. 974-980.105Kia Ng, et al., "An Integrated Multi-Sensory System for photo-Realistic 3D Scene Reconstruction," ISPRS Comm, p. 356-363, 1998.106 *Klein, Konrad et al, "View Planning for the 3D Modelling of Real World Scenes," Proc. of the 2000 IEEE/RSJ International Co on Intelligent Robots and Systems, pp. 943-948.107 *Kropp,A. et al, OMNIVIS'00: "Acquiring and Rendering High-Resolution Spherical Mosaics", pp. 1-7.108Kwang-Ho Bae, et al., "Automated registration of Unorganised Point Clouds from Terrestrial Laser Scanners," Department of Spatial Sciences, Curtin University of Technology, Perth, Australia, 2004.109 *Li, Rongxing, "Mobile Mapping-An Emerging Technology For Spatial Data Acquisition," Dept. of Civil and Environmental Engineering and Geodetic Science, The Ohio State University, 2000, pp. 1-23.110 *Li, Rongxing, "Mobile Mapping�An Emerging Technology For Spatial Data Acquisition," Dept. of Civil and Environmental Engineering and Geodetic Science, The Ohio State University, 2000, pp. 1-23.111Matthew P. Tait, "Point Cloud registration: Current State of the Science," Schulich School of Engineering, University of Calgary, Mar. 27, 2006.112Mehdi Bouroumand, et al., "The Fusion of Laser Scanning and Close Range Photogrammetry in Bam Laser-Photogrammetric Mapping of Bam Citadel (Arg-E-Bam)/Iran," KNT University in Tehran, 2004.113 *Modeling and Rendering of Real Environments by Wagner T. Correa et al.; RITA, vol. IX, Numero 1, Aug. 2002; pp. 1-32.114Naser El-Sheimy, et al., "Digital Terrain Modeling, Acquisition, Manipulation, and Applications," Artech House, Inc., p. 117-119, 184-186, 2005.115Natasha Gelfand, et al., "Robust Global Registration," Eurographics Symposium on Geometry Processing, 2005.116Nathaniel Williams, et al., "Automatic Image Alignment for 3D Environment Modeling," Bibliography, Presented at the Siggraph conference, Aug. 11, 2004.117Nathaniel Williams, et al., "Automatic Image Alignment for 3D Environment Modeling," Department of Computer Science, University of North Carolina at Chapel Hill, 2004.118Nhat Xuan Nguyen, "Numerical Algorithms for Image Supreresolution," http://citeseer.ist.psu.edu/nguyen00numerical.html, 2000.119Paul Rademacher, "Ray Tracing: Graphics for the Masses," ACM, New York, vol. 3, Issue 4, p. 3-7, 1997.120Pedro F. Felzenswalb, et al., "Pictorial Structures for Object Recognition," Artificial Intelligence Lab, Massachusetts Institute of Technology, Computer Science Department, Cornell University, 2003.121Peter K. Allen, et al., "3D Modeling of Historic Sites Using Range and Image Data," Dept. of Computer Science, Columbia University, Dept. of Computer Science, Hunger College, May 16, 2008.122Richard A. Robb, et al., "Adaptive Piece-wise. Registration for Automated Fusion of Point Cloud Coordinates and Anatomic Volume Images," http://www.mayoclinictechnology.com/tc/software/MMV-04-247-565.html, 2004.123S.T. Dijkman, et al., "Semi Automatic Registration of Laser Scanner Data," International Archives of Photogrammetry Remote Sensing and Spatial Information Sciences, Natural Resources, Canada, vol. 34, Part 5, . 12-17, 2002.124 *Sato, Yukio et al, "Three-Dimensional Shape Reconstruction by Active Rangefinder," Proc. CVPR., IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Jun. 1993, pp. 142-147.125 *Scanalyze: a system for aligning and merging range data; http://graphics.standford.edu/software/scanalyze; dated Dec. 9, 2002; pp. 1-7.126 *Soucy, M. et al, "A general surface approach to the integration of a set of range views"-Pattern Analysis and Machine Intelligence, vol. 17, No. 4,, IEEE Transactions on, Apr. 1995, pp. 34-358.127 *Soucy, M. et al, "A general surface approach to the integration of a set of range views"�Pattern Analysis and Machine Intelligence, vol. 17, No. 4,, IEEE Transactions on, Apr. 1995, pp. 34-358.128Steven Alexander Sablerolle, "Automatic Registration of Laser Scanning Data and Colour Images," TU Delft, Faculty of Civil Engineering, Final Presentation Msc. Geomatics, Oct. 31, 2006.129Steven Alexander Sablerolle, "Automatic Registration of Laser Scanning Data and Colour Images," TUDelft, Oct. 2006.130Steven Sablerolle, "Graduation Research Report," TU Delft, Sep. 2006.131T. Rabbani, et al., "Segmentation of Point clouds Using Smoothness Constraint," ISPRS Commission V Symposium 'Image Engineering and Vision Metrology,' 2006.132T. Rabbani, et al., "Segmentation of Point clouds Using Smoothness Constraint," ISPRS Commission V Symposium �Image Engineering and Vision Metrology,� 2006.133Tahir Rabbani Shah, "Automatic Reconstruction of Industrial Installations Using Point Clouds and Images," Geodesy 6.2, Nederlandse Commissie voor Geodesie Netherlands Geodetic Commission, Delft, May 2006.134Tahir Rabbani, et al., "Efficient Hough Transform for Automatic detection of Cylinders in Point Clouds," Workshop "Laser Scanning 2005," Enschede, the Netherlands, Sep. 12-14, 2005.135Wikipedia, the free encyclopedia, "Level of Detail," http://en.wikipedia.org/wiki/Level_of_detail_(programming), May 23, 2008.136Yu Lifeng, et al., "Multi-Modality Medical Image Fusion Based on Wavelet Pyramid and Evaluation," Peking University, Beijing China, 1994.137 *Zippered Polygon Meshes from Range Images by Greg Turk et al.; Proc. SIGGRAPH '94, Jul. 1994; pp. 1-8.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS8207964 *Feb 22, 2008Jun 26, 2012Meadow William DMethods and apparatus for generating three-dimensional image data modelsUS8379191 *Jun 21, 2005Feb 19, 2013Leica Geosystems AgScanner system and method for registering surfacesUS8436901 *Feb 20, 2009May 7, 2013Id. Fone Co., Ltd.Field monitoring system using a mobile terminal and method thereofUS8558848Mar 9, 2013Oct 15, 2013William D. MeadowWireless internet-accessible drive-by street view system and methodUS8600713 *Sep 15, 2011Dec 3, 2013National Central UniversityMethod of online building-model reconstruction using photogrammetric mapping systemUS8884950Jul 29, 2011Nov 11, 2014Google Inc.Pose data via user interactionUS8890866Mar 9, 2013Nov 18, 2014Visual Real Estate, Inc.Method and apparatus of providing street view data of a comparable real estate propertyUS8902226Mar 12, 2013Dec 2, 2014Visual Real Estate, Inc.Method for using drive-by image data to generate a valuation report of a selected real estate propertyUS20100118116 *Jun 28, 2007May 13, 2010Wojciech Nowak TomaszMethod of and apparatus for producing a multi-viewpoint panoramaUS20110001795 *Feb 20, 2009Jan 6, 2011Hyun Duk UhmField monitoring system using a mobil terminalUS20120265494 *Sep 15, 2011Oct 18, 2012National Central UniversityMethod of Online Building-Model Reconstruction Using Photogrammetric Mapping SystemUSRE45264 *Apr 11, 2013Dec 2, 2014Visual Real Estate, Inc.Methods and apparatus for generating three-dimensional image data models* Cited by examinerClassifications U.S. Classification342/357.31, 345/419, 348/218.1, 345/629International ClassificationG01S19/42, G01C15/00, G01S17/08, G01S19/48, G01S17/02, G06T15/00, H04N5/225, G01S5/14, G09G5/00Cooperative ClassificationG01S19/42, G01C15/00, G01S17/08, G01C11/02, G01S17/023European ClassificationG01S19/42, G01C11/02, G01S17/02C, G01C15/00Legal EventsDateCodeEventDescriptionDec 20, 2011FPAYFee paymentYear of fee payment: 8May 8, 2008ASAssignmentOwner name: SQUARE 1 BANK,NORTH CAROLINAFree format text: SECURITY INTEREST;ASSIGNOR:INTELISUM, INC.;REEL/FRAME:020930/0037Effective date: 20070518Owner name: SQUARE 1 BANK, NORTH CAROLINAFree format text: SECURITY INTEREST;ASSIGNOR:INTELISUM, INC.;REEL/FRAME:020930/0037Effective date: 20070518RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services