INTEGRATION OF PHOTOGRAMMETRIC AND LIDAR DATA IN A MULTI PRIMITIVE TRIANGULATION PROCEDURE
Year: 2007

Photogrammetry is the art and science of deriving accurate metric and descriptive information from analog and digital images. Photogrammetric reconstruction produces surfaces that are rich in semantic information, which can be clearly recognized in the captured imagery. The inherent redundancy associated with photogrammetric restitution results in highly accurate surfaces. Nevertheless, the amount of effort and time required by the photogrammetric reconstruction procedure is a major disadvantage. In LIDAR mapping, however, spatial coordinates of object space points are directly acquired, enabling quick turnaround of mapping products. Still, the positional nature of LIDAR points makes it difficult to derive semantic surface information such as discontinuities and types of observed structures. Additionally, no inherent redundancy is available in the reconstructed surfaces that may be utilized to enhance the accuracy of the final product. The complementary characteristics of photogrammetry and LIDAR, if exploited, can lead to a more complete surface description. The synergic advantages of both systems can be fully realized only after precise calibration of both systems and successful registration of the photogrammetric and LIDAR data relative to a common reference frame. Generally, a registration methodology has to deal with three issues: registration primitives, transformation function, and similarity measure. In this paper, two methodologies are introduced for the co registration of LIDAR and photogrammetric datasets. One track of methodologies uses straight lines while the other uses planar patches as the registration primitives. The mathematical model and similarity measures corresponding to both types of primitives are introduced. The presented approaches can be divided into two categories. The first category includes new constraints that can be added to current bundle adjustment procedures. On the other hand, the second category utilizes existing point based bundle adjustment procedures after manipulating the variance covariance matrices of the linear and areal primitives. The proposed methodologies are tested and proved efficient in a multi sensor triangulation environment. For the feasibility test, IKONOS stereo pair and medium format aerial image block are integrated with LIDAR data in a single bundle adjustment procedure while implementing linear and areal features.  A diverse range of spatial data acquisition systems is now available onboard satellite, aerial, and terrestrial mapping platforms. The diversity starts with analog and digital frame cameras and continues to include linear array scanners. In the past few years, imaging sensors witnessed vast development as a result of an enormous advancement in the digital technology. For example, the increasing sensor size and storage capacity of digital frame cameras lead to their application in traditional and new mapping functions. However, due to technical limitations, current digital frame cameras are not capable of providing geometric resolution and ground coverage similar to those associated with analog frame cameras. To alleviate such a limitation, push broom scanners (line cameras) have been developed and used onboard satellite and aerial platforms. However, rigorous modeling of the imaging process of push broom scanners is far more complicated than the modeling of frame sensors (Poli, 2004; Toutin, 2004 a; Toutin, 2004 b) For example, in the absence of the georeferencing parameters, the narrow Angular Field of View (AFOV) associated with current push broom scanners are causing instability in the triangulation procedure due to strong correlation between the components of the Exterior Orientation Parameters (EOP) Therefore, there has been a tremendous body of research that dealt with the derivation of alternative models for line cameras to circumvent such instability (Tao et al, 2004; Habib et al, 2004; Fraser and Hanley, 2003) In addition to line cameras, the low cost of medium format digital frame cameras is leading to their frequent adoption by the mapping community. Apart from imaging systems, LIDAR scanning is rapidly taking its place in the ASPRS 2007 Annual Conference Tampa, Florida May 7 11, 2007 mapping industry as a fast and cost effective 3D data acquisition technology. The increased accuracy and affordability of GPS/INS systems are the main reasons behind their expanding role in orienting LIDAR sensors. Considering the characteristics of acquired spatial data from imaging and LIDAR systems, one can argue that their integration will be beneficial for accurate and complete description of the object space. It is evident that the disadvantages of one system can be compensated for by the advantages of the other system (Baltsavias, 1999; Satale and Kulkarni, 2003) However, the synergic characteristics of both systems can be fully utilized only after ensuring that both datasets are georeferenced relative to the same reference frame (Habib and Schenk, 1999) Traditionally, photogrammetric geo referencing is either indirectly established with the help of Ground Control Points (GCP) or directly defined using GPS/INS units on board the imaging platform (Cramer et al, 2000) On the other hand, LIDAR geo referencing is directly established through the GPS/INS components of a LIDAR system. In this regard, this paper presents alternative methodologies for utilizing LIDAR features as a source of control for photogrammetric geo referencing. These methodologies have two main advantages. First, they will ensure the co alignment of the LIDAR and photogrammetric data to a common reference frame as defined by the GPS/INS unit of the LIDAR system. Moreover, LIDAR features will eliminate the need for ground control points to establish the geo referencing parameters of the photogrammetric data. The possibility of utilizing LIDAR data as a source of control for photogrammetric geo referencing hinges on the ability to identify common features in both datasets. Therefore, the first objective of the developed methodologies is concerned with the selection of appropriate primitives. Afterwards, the mathematical models, which can be utilized in the triangulation procedure, for relating LIDAR and photogrammetric primitives will be introduced. The paper will start by offering a brief discussion of the appropriate primitive for relating the LIDAR and photogrammetric data as well as the respective mathematical models for their incorporation in the triangulation procedure. The feasibility and the performance of the developed multi primitive and multi sensor triangulation will be outlined through the experimental results section using real data. Finally, the paper will conclude with final remarks and recommendations for future research. APPROPRIATE PRIMITIVES A triangulation process relies on the identification of common primitives for relating the involved datasets to the reference frame defined by the control information. Traditionally, photogrammetric triangulation has been based on point primitives. Considering photogrammetric and LIDAR data, relating the laser footprint to the corresponding point in the imagery is almost impossible. Therefore, point primitives are not appropriate for the task at hand. Linear and areal features are other potential primitives that can be more suitable for relating LIDAR and photogrammetric data. Linear features can be directly identified in the imagery while conjugate LIDAR lines can be extracted through planar patch segmentation and intersection. Alternatively, LIDAR lines can be directly identified in the intensity images produced by most of today's LIDAR systems. It should be noted that extracted linear features from planar patch segmentation and intersection are more accurate than the extracted features from intensity images. The lower quality of extracted features from the intensity images is caused by the utilized interpolation process for converting the irregular LIDAR footprints to a raster grid. Other than linear features, areal primitives in photogrammetric datasets can be defined using their boundaries, which can be identified in the imagery. Such primitives include, for example, roof tops, lakes, and other homogeneous regions. In LIDAR datasets, areal regions can be derived through planar patch segmentation techniques. Another issue that is related to the primitives selection is their representation in both photogrammetric and LIDAR data. In this regard, image space lines can be represented by a sequence of image points along the feature. This is an appealing representation since it can handle image space linear features in the presence of distortions as they will cause deviations from straightness in the image space. Furthermore, such a representation will allow for the inclusion of linear features in scenes captured by line cameras since perturbations in the flight trajectory would lead to deviations from straightness in image space linear features corresponding to object space straight lines (Habib et al. 2002) It should be noted that the selected intermediate points along corresponding line segments in overlapping scenes need not be conjugate. As for the LIDAR data, object lines can be represented by their end points. The points defining the LIDAR line need not be visible in the imagery. For areal primitives, photogrammetric planar patches can be represented by three points (e.g. corner points) along their boundaries. These points should be identified in all overlapping imagery. Similar to the linear features, this representation is valid for scenes captured by frame and line cameras. On the other hand, LIDAR patches can be represented by the footprints defining that patch. These points can be directly derived from planar patch segmentation techniques. Having settled the primitive selection and representation issues, the next section will focus on explaining the proposed mathematical models for relating