## 4 Results
### Data Processing
#### 4.2.2 Spatialization of Contemporary and Historical Photographs  
One pipeline task is the spatialization of historical photographs. Part of this task is to collect and identify spatiotemporally corresponding images via textual image metadata (e.g., placenames). The next step is to detect similar views via overlapping segments in historical and contemporary photographs and to calculate a relative position via a feature-based orientation/positioning pipeline, and combine them with contemporary, oriented data to identify absolute positions. While digital photographs already contain data which enable, at least for Streetview data, an out-of-the-box use, historical photographs have to be processed to retrieve orientation information. Exact determination of the position and orientation of contemporary and historical photographs remains a crucial step for automatic photogrammetric 3D modelling and texturization of 3D models. Several strategies have been developed within the Jena4D research group. Image Processing for Contemporary Photographs  
Regarding the contemporary image material retrieved from different sources, initial experiments deal with the retrieved Google Street View images. The Google Street View API only provides sparse information such as the latitude and longitude of the photograph in UTM coordinates and one angle for the orientation. Thus, several assumptions need to be made:  
1. As there are always three images taken at one position, we assume a field of view (FOV) of 120\({}^{\circ}\). 2. As only one angle is given, we assume that this is the rotation around the yaw axis. We assume that the other two angles are close to 0\({}^{\circ}\). 3. As the height coordinate of the image is not given, we estimate the respective elevation using the API ofopentopodata.org and the EU digital elevation model EU-DEM with 25 m resolution. We add 2 m to the retrieved height because Google's camera is usually mounted on a car. Considering these multiple assumptions for the contemporary images, the accuracy is quite remarkable (Figure 7). It is planned to use these images in areas where no other (historical) image material is available. Considering the quality of the overlay, it is possible that matching the 3D model and image content may improve the image orientation. A similar workflow has not yet be established for Mapillary images. The Mapillary API provides even less image orientation information, and the global coordinates seem to be more inaccurate than those for Google Street View, placing the images in the wrong spot in the 4D application. Further, orientation angle information is not given at all, so the viewing direction would have to be estimated by an iterative approach to test matching between images and given OpenStreetMap (OSM) panoramas. The last retrieval image data source is Wikimedia Commons photographs. While these images are usually not available for all possible urban scenes, they can be especially relevant for specific landmarks. For famous sights, several hundreds of images often exist. Figure 7: Automatically oriented images obtained from Google Street View. The image is shown with a semi-transparent overlay over the environment and OpenStreetMap 3D models behind it. From (**left**) to (**right**): three scenes in Italy in Pisa, Bologna, and Massa. However, no orientation information is given for this material. Thus, for the Wikimedia Commons images, we followed the SfM method that is used on historical urban images and explained in the following. Image Orientation for Historical Photographs  
The strategy for orientation of historical images is explained in detail in [130,290]. Here, the process is summarized, and recent improvements are highlighted.