# Investigating Lunar Boulders at the Apollo 17 Landing Site Using Photogrammetry and Virtual Reality

Stéphane Le Mouélic, Pauline Enguehard, Harrison H. Schmitt, Gwénaël

Caravaca, Benoît Seignovert, Nicolas Mangold, Jean-Philippe Combe,

François Civet

► To cite this version:

Stéphane Le Mouélic, Pauline Enguehard, Harrison H. Schmitt, Gwénaël Caravaca, Benoît Seignovert, et al.. Investigating Lunar Boulders at the Apollo 17 Landing Site Using Photogrammetry and Virtual Reality. Remote Sensing, 2020, Special Issue Celebrating the 50th Anniversary of the Moon Landing with Views of Earth from Satellite and Apollo Missions, 12 (11), pp.1900. 10.3390/rs12111900. hal-02879333

HAL Id: hal-02879333

https://hal.science/hal-02879333

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

Article

# Investigating Lunar Boulders at the Apollo 17 Landing Site Using Photogrammetry and Virtual Reality

Stéphane Le Mouélic<sup>1,*</sup>,<sup>5</sup> Pauline Enguehard<sup>1</sup>, Harrison H. Schmitt<sup>2</sup>, Gwénaël Caravaca<sup>1,5</sup>, Benoît Seignovert<sup>3,5</sup>, Nicolas Mangold<sup>1</sup>, Jean-Philippe Combe<sup>4</sup> and François Civet<sup>5</sup>

Received: 11 May 2020; Accepted: 9 June 2020; Published: 11 June 2020

<figure><img src="image_3.png" /></figure>

check for
updates

**Abstract:** The Taurus-Littrow valley on the Moon was the location of intensive geologic fieldwork during three days in December 1972. In situ activities at sampling stations were systematically documented by the astronauts using a series of overlapping images taken with their Hasselblad cameras. We investigated how this Apollo image archive can be used to perform 3-D reconstructions of several boulders of interest using close-range photogrammetry. We specifically focused on seven different boulders located at Stations 2, 6, and 7, at the foot of South and North Massifs, respectively. These boulders represent samples from highland materials, which rolled down the slopes of the surrounding hills. We used the Agisoft Metashape software to compute 3-D reconstructions of these boulders, using 173 scanned images as input. We then used either a web-based platform or a game engine to render the models in virtual reality. This allowed the users to walk around the boulders and to investigate in detail their morphology, fractures, vesicles, color variations, and sampling spots, as if standing directly in front of them with the astronauts. This work suggests that many features can be reconstructed in other sites of the Apollo missions, so as other robotic landing sites. Virtual reality techniques coupled to photogrammetry is thus opening a new era of exploration, both for past and future landing sites.

**Keywords:** sample documentation; remote sensing; photogrammetry; moon; 3-D; Apollo; geology; virtual reality

## 1. Introduction

The constant development of computer hardware (notably graphic computing units used in game engines) and the availability since 2016 of technologically mature virtual reality (VR) headsets to the general public opens a new era of geological exploration. In planetary exploration, the use of simulated visual experience, commonly known as virtual reality, is quite an old concept [1], but the technique is only starting to become mature and progressively widespread in our community. It can be coupled with the possibility of reconstructing high-resolution 3-D data sets, such as digital outcrop models (DOMs), from a

<footnote n="1">¹ Laboratoire de Planétologie et Géodynamique, CNRS UMR6112, Université de Nantes, 44322 Nantes, France; pauline.enguehard@etu.univ-nantes.fr (P.E.); gwenael.caravaca@univ-nantes.fr (G.C.); nicolas.mangold@univ-nantes.fr (N.M.)</footnote>

<footnote n="2">² Department of Engineering Physics, University of Wisconsin-Madison, P.O. Box 90730, Albuquerque, NM 87199, USA; hhschmitt@earthlink.net</footnote>

<footnote n="3">³ Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA 91109, USA; benoit.a.seignovert@jpl.nasa.gov</footnote>

<footnote n="4">⁴ Bear Fight Institute, 22 Fiddler's Road, Winthrop, WA 98862, USA; jean-philippe_combe@bearfightinstitute.com</footnote>

<footnote n="5">⁵ VR2Planets, 44322 Nantes, France; fcivet@vr2planets.com</footnote>

<footnote>* Correspondence: stephane.lemouelic@univ-nantes.fr; Tel.: +33-2-5-12-55-69</footnote>
set of overlapping photographs using structure-from-motion photogrammetry (e.g., [2–8]). By doing so, VR enables geoscientists to complement, quantitatively evaluate, and analytically enhance field observations, and also to eventually perform analysis at multiple scales and optical manipulation, which is not directly possible in the field with conventional methods [9–12]. This is particularly important for remote locations that are not easily accessible, such as planets, moons, and small bodies of the solar system. VR also allows the possibility of easily extracting and visualizing the scientific content of a set of images, and sharing the results with the community through web-based platforms or other digital systems, such as game engines, thanks to their optimized 3-D rendering capabilities [13,14]. This new immersive technique potentially allows the users to explore 3-D reconstructions of planetary landscapes as if they were physically present “on the field”. We investigated here how the in situ lunar imagery can be coupled to the orbital imagery to provide the user the possibility of exploring the lunar surface in VR. We focused on the reconstruction of the Apollo 17 lunar landing site, with a particular emphasis on the reproduction of the geological settings related to the melt-breccia contacts in boulders documented in photographs at Stations 6 and 7 at the foot of the North Massif in the Taurus-Littrow valley, and on boulders at Station 2 at the foot of the South Massif [8]. The location of these three sites is shown in Figure 1a on the traverse map produced from [15], building on the initial work of [16], together with panoramas taken in situ near the three areas of boulders. The topic of this paper is to describe the reconstruction process of the main boulders, from the input data up to the visualization of the products. The results of the 3-D reconstructions of the boulders are shared through a public web-based platform.

The Apollo 17 crew spent three days at the Taurus-Littrow site in December 1972. The southernmost part of the traverse was reached during the second extra vehicular activity (EVA), at Station 2 located at the foot of South Massif (Figure 1a,b). During the third EVA, the sampling sites Station 6 and Station 7 were selected as a suitable location to collect possible ancient highland material from the North Massif (Figure 1c,d). North Massif rises about 1400 m above Station 6, and probably formed as the result of the giant impact that created the Serenitatis basin [8]. Sampling impact melts are particularly useful for the dating of major events, the radio-isotopic age of the sampled rock being reset by the melting during the impact. Radio-isotopic dating, however, continues to evolve, depending on the ability to precisely identify what is being dated. Whereas other locations can be envisaged in further works, we focused our efforts here on the Station 6 and Station 7 boulders at the base of North Massif, located on the northernmost portion of the EVA-3 traverse, and the Station 2 boulders at the base of the South Massif. In the first section, we present the orbital and in situ data used as input, as well as the software solution used for the photogrammetric process. In the second part, we detail the 3-D reconstructions of the boulders at these key sampling sites of the Apollo 17 field campaign: Station 2, 6, and 7. We then present the integration of the products into VR in the third section, before discussing the results and concluding the study.
<figure><img src="image_4.png" /><figcaption>**Figure 1.** (a) Apollo 17 traverse map (adapted from [15]) showing the localization of Stations 2, 6, and 7 (red circles). (b) Panorama taken by E. Cernan at Station 2 (frames AS17-137-20926 to 20956). (c) Panorama taken by at H. H. Schmitt at Station 6 (frames AS17-141-21575 to 21603), showing several boulder fragments to the left of the lunar roving vehicle (LRV). (d) Panorama taken by E. Cernan at station 7 (frames AS17-146-22339 to 22363). The main boulder is on the right of the LRV.</figcaption></figure>

## 2. Materials and Methods

### 2.1. Context Orbital Imagery

We first used the available orbital imagery to map the Taurus-Littrow valley using the ArcGIS Geographic Information System. The purpose was to give a broader regional context of the sampling sites once integrated into virtual reality, and to precisely locate the sites of interest. The general context was computed using a subset of the SELenological and Engineering Explorer (SELENE) "Kaguya" Terrain Camera (TC) global orthomosaic at 4096 pixels per degrees [17], available on the United States Geological Survey (USGS) astrogeology science center website. With a resolution of ~10 m per pixel, this data set bridges the gap between Clementine and the LROC WAC (~100 m/pixel) and the LRO NAC (~1 m/pixel) and therefore provides relevant contextual information. The Kaguya-TC mosaic was draped over a digital elevation model produced from the merging of LRO Lunar Orbiter Laser Altimeter (LOLA) data and SELENE Kaguya TC data, available at a horizontal resolution of 512 pixels per degree (corresponding to ~59 m per pixel at the equator) and a typical vertical accuracy
~3 to 4 m [17,18]. A global color ratio composite from Clementine UVVIS is used to provide spectral information at the regional scale [19,20].

For the highest spatial resolution, we over imposed an Apollo 17 LROC NAC orthomosaic at 50 cm per pixel and the corresponding associated 1.5-m-scale LROC DEM, both also being available on the USGS astrogeology Center website [15]. This mosaic was produced from images M137353046-L/-R, M152669024-L/-R, M134985003-R, M168000580-L/-R, M119652859-L/-R, M109032389-R, and orthorectified using the 1.5 m DEM. Figure 2 shows a selection of global views of these cartographic products once integrated into a 3-D rendering system.

<figure><img src="image_5.png" /><figcaption>**Figure 2.** (a) Perspective view of a Kaguya-TC mosaic draped over a LOLA-Kaguya TC merged digital elevation model, providing the context of the Taurus-Littrow valley (north is pointing on the upper right corner, and the tile dimension is 55 × 55 km). The Lunar Reconnaissance Orbiter NAC 50 cm mosaic appears in a lighter shade in the center of the frame. (b) Clementine UVVIS Color ratio composite draped over the same DEM. (c)–(f) display progressive close-up views showing the location of the reconstructed boulders at Station 6 and 7 at the foot of North Massif that is discussed in Section 3. The distance between Station 6 and 7 is 490 m.</figcaption></figure>

## 2.2. *The Hasselblad Surface Image Data Set*

During their EVAs, the astronauts used two Hasselblad 500 EL data cameras attached to the chest of their spacesuits. Cameras were calibrated and equipped with a reseau plate in front of the image plane to provide a photogrammetric reference [21]. The grid consists in 5 × 5 mm crosses at 10-mm intervals, imaged in each frame of the 70-mm film. The cameras were fitted with a Zeiss Biogon lens with a 60-mm focal length, thus providing a moderately wide field of view. The astronauts systematically acquired single frames, full panoramas, overlapping images, and stereo pairs along their traverse path and at sampling stations to document the samples' collection and allow post-flight geological analysis [22].

We mainly used two different online resources to compile the relevant imagery database for our research project. Both are based on original film scans performed at the Johnson Space Center [23].
We first used the "Apollo in real time" (https://apollo17.org) website, which provides a very user-friendly possibility to evaluate how many images are available at each location, and when (and where) they were sequentially taken during the mission. Images directly accessible from this website have a resolution of 2100 × 2100 pixels. While being generally well contrasted and color balanced, we rather obtained a better accuracy in our photogrammetric process when using the same images downloaded from the "Project Apollo Archive" page on Flickr (https://www.flickr.com/projectapolloarchive), which complements the comprehensive Apollo Lunar Surface Journal source of information. The highest pixel resolution provided on the Flickr website corresponds to a size of 4175 × 4175 pixels. An example of the images used as input is shown in Figure 3. These black and white and color images correspond to 1800 dpi Hasselblad film scans produced in 2005 at the Johnson Space Center [23]. Considering that the Hasselblad film is 70 mm on the diagonal, this corresponds to an equivalent pixel size of ~12 µm. It should be noted that the original oversampled scans with a resolution up to 14,000 × 16,000 pixels by photo are also available at the Apollo Image Archive (https://tothemoon.ser.asu.edu/gallery/Apollo) from the Arizona State University. In this study, we decided to work with the 4175 × 4175 pixel resolution images for optimum handling with our current computing resources.

<figure><img src="image_6.png" /><figcaption>**Figure 3.** Example of the images of H. H. Schmitt taking rock samples during the third extravehicular activity at Station 7 (images labeled AS17-146-22335 to AS17-146-22338). There is sufficient overlap between these kind of sample images to allow a photogrammetric 3-D reconstruction of the boulder.</figcaption></figure>

The cameras were calibrated before flight to evaluate their internal orientation and lens distortion [24]. In the photogrammetric process that we describe in further sections, this information was directly derived from the set of pictures themselves by the structure-from-motion optimization algorithm.

## 2.3. *Structure from Motion Principle and Software Choice*

The structure-from-motion (SfM) photogrammetry principle relies on the identification of tie-points on a set of overlapping images taken from different points of views to reconstruct a shape in 3-D [2]. When a sufficient number of overlapping images of a given scene is available as input, it is possible to compute the position of tie points in a 3-D space, together with optical parameters linked to the camera used to take the pictures. Using a dense point cloud, a 3-D mesh can be reconstructed to provide
the shape of the elements constituting the landscape. Individual images are finally mosaicked and
wrapped over the mesh as a texture to provide a photorealistic final 3-D model that can be rendered by
various common 3-D processing software. Several commercial or freeware software can be used to
perform all the processing steps.

An example of photogrammetric reconstruction made with the iWitness ProTM 3.6 software [25] on a series of six overlapping images of the 70019 sample area is shown in [8]. Another example of photogrammetric reconstruction is presented in [26], using Agisoft PhotoScan Professional (recently renamed "Metashape") on descent images of the Apollo 15 landing site. In this work, we also relied on Agisoft Metashape (v. 1.6.2, [27]), which is one of the most widely used SfM solutions in the field of planetary science for geological applications (e.g., [3,5–7,11,12]).

**3. Results**

3.1. *Three-Dimensional Reconstruction of the Station 7 Boulder*

We started our analysis with the Station 7 boulder, which consists in a single ~2-m-high block that
had rolled down from several hundreds of meters higher on the Massif (Figures 1a,d and 2, and [8]).
The objectives of the Apollo 17 mission at this location were to characterize and sample both Massif
and dark mantle materials [16], although eventually no dark mantle materials were identified at Station
7. A first panorama was acquired south of the rover (images AS17-141-21646 to AS17-141-21664)
and a second panorama was acquired north of the rover (images AS17-146-22339 to AS17-146-22363).
This boulder stands on a 13.6° slope, just above the break in the slope between the Massif and the valley
floor. The block was originally described during the in situ exploration as a vesicular anorthositic
gabbro, but it was later found to be a fine-grained impact melt-breccia [8,28].

We used the set of 41 images labeled AS17-146-22298 to AS17-146-22338 as input to reconstruct
the main boulder using the photogrammetric process. First, a mask was applied to remove 75 lines
and columns of pixels at the edge of each image, corresponding to the diffuse limit of the scanned film.
All 41 images were then successfully automatically aligned with Agisoft Metashape using ~450,000 tie
points, allowing for the generation of a sparse point cloud. In the next step, a dense cloud of 30 million
points was generated, allowing the computation of a detailed 3-D mesh. Considering the relatively
simple shape of the boulder, we produced a mesh of 300,000 polygons, draped with two textures
of 8192 × 8192 pixels. This represents a good tradeoff between the level of detail and the computer
resources' optimization of the 3-D model, for further handling into VR systems. During the texture
computation, the mosaicking of the 41 original images was performed using an average of the values
on each polygon of the 3-D model.

Figure 4 shows different views of the textured 3-D model reconstructed by the photogrammetric
software. The astronauts documented approximatively one-half of the boulder. The 41 positions and
orientations of the camera computed by Metashape are indicated at their exact location (annotated
squares) around the perspective view in the lower left of Figure 4. The inset (white rectangle) shows
the level of detail achieved during the reconstruction process. Arrows point, for example, to a veinlet
of dark very fine-grained melt-breccia that stands out on the model and that can be traced throughout
various parts of the boulder. It presents potential similarities with impact pseudotachylites [29].
The light patch (arrow) in the middle of the left view corresponds to the markings left when collecting
sample 77115. Sample 77135 was taken above this area on the left.

In the final processing step, the scaling can be determined using a known reference such as the
gnomon (ideally) or any piece of hardware, when visible in the original images. Any footprint can also
be useful for this purpose. However, this step can prove to be problematic when no known reference is
available on the scene. Another important scaling indication is given by the position of the cameras
with respect to the surrounding ground, provided that enough ground is present in the 3-D model,
and considering that the cameras were systematically attached to the chest of the two 1.83 m- (Cernan)
and 1.75 m-tall (Schmitt) astronauts. In this specific case at Station 7, we relied on the astronaut's suited height on the original pictures to evaluate the scale.

<figure><img src="image_7.png" /><figcaption>**Figure 4.** Photogrammetric reconstruction (textured 3-D model) of the 2-m-high boulder at Station 7. The position and orientation of the 41 entry Apollo images (AS17-146-22298 to AS17-146-22338) as computed by Metashape are indicated on the left perspective view (annotated squares), together with the location of samples 77135 and 77115. The inset shows a close-up of the area located in the white rectangle on the front view, to illustrate the level of details allowed by the 3-D reconstruction. Arrows point to a 3–5 cm vein.</figcaption></figure>

## 3.2. *Three-Dimensional Reconstruction of the Station 6 Boulders*

Station 6 is located at 490 m west of Station 7, north of the Henry crater (Figure 1a). Station 6 offered the opportunity to explore a collection of five large boulder fragments and regolith [8], lying at the base of a long boulder trail descending from the North Massif (Figures 1c and 2). The fragments originated from a single boulder, which rolled down the hill, leaving a 10-m-wide and 1-km-long trail, and broke apart about 30 m above the valley floor, at the transition between the ~26° slope of the massif to a 15.4° slope [15] near the massif's contact with the valley floor. Activities at this site included the collection of samples, a tube core and a traverse gravimeter measurement, all these activities being documented by many photographs, of which we found 82 that were suitable for a 3-D reconstruction. Two additional sets of panoramas were acquired to provide a comprehensive view of the site. H. H. Schmitt recorded the South Pan panorama with frames AS17-141-21575 to AS17-141-21603. E. Cernan recorded the second panorama while being north of the boulder, with frames 140-21483 to AS17-140-21509.

To reconstruct this complex area from the available imagery, and considering the incomplete spatial overlap between images of the main boulder fragments, we had to split the photogrammetric project for this scene into three separate chunks, allowing an independent treatment of each fragment. The first chunk, corresponding to the northern most fragment referred to as Boulder 1 in [8,15,16], was reconstructed from a subset of 42 images labeled AS17-140-21441 to AS-140-21482 (Figure 5). The entry images were aligned, generating a sparse point cloud of about 300,000 tie-points. A dense cloud of 46 million points was computed during the SfM process, allowing the generation of a
mesh of 9 million polygons, that we downsampled down to 1 million polygons for better computer performances during the visualization process. The computed location and orientation of the 42 images is shown on the perspective view on the right panel of Figure 5. The inset shows an enlargement on the locations where samples 76295 and 76275 were extracted.

<figure><img src="image_8.png" /><figcaption>**Figure 5.** Photogrammetric reconstruction (textured 3-D model) of the northern most boulder fragment at Station 6, informally named Boulder 1 by previous authors. The position and orientation of the 42 entry Apollo images AS17-140-21441 to As-140-21482 computed from Metashape, are indicated in the perspective view on the right.</figcaption></figure>

The second chunk covers the fragments 2 and 3 of the boulders. It was produced using a subset of seven images labeled AS17-140-21434 to AS17-140-21440 (Figure 6). The alignment step generated a sparse point cloud of about 40,000 tie-points. A dense point cloud of 7 million points then allowed the generation of a mesh of 500,000 polygons. A significant part of the soil was reconstructed in the 3-D model, thanks to the overlap between images. This proved to be useful to give a reference for the orientation and the scaling of the model.

Finally, a third chunk using a subset of 33 images labeled AS17-140-21401 to AS17-140-21433 was used to reconstruct the fragments 4 and 5 (Figure 7). The alignment step generated a sparse point cloud of about 250,000 tie-points. A dense point cloud of 30 million points then allowed the generation of a mesh that we downsampled to 2 million polygons. This third chunk is of particular interest as the gnomon used by Apollo astronauts was fully reconstructed in the 3-D model, thanks to the fact that it appears in multiple overlapping images. The gnomon is 53 cm when stowed, and 62 cm high when deployed. A partial footprint is also visible in the soil just left of the gnomon, allowing for a secondary scaling of the model, in addition to the camera's height. This part of the boulder is a vesicular tan-gray breccia, with scattered distinctive light-colored clasts [30] that were accurately represented in the 3-D model, so as the vesicles.
<figure><img src="image_9.png" /><figcaption>**Figure 6.** Photogrammetric reconstruction (textured 3-D model) of fragments 2 and 3 at Station 6. The location and orientation of the seven entry Apollo images AS17-140-21434 to AS17-140-21440 as computed by Metashape are indicated by annotated squares. A significant part of the ground underneath the boulders was reconstructed thanks to significant overlap in the entry images.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Figure 7.** Photogrammetric reconstruction (textured 3-D model) of the fragments 4–5 at Station 6. The position and orientation of the 33 Apollo images labeled AS17-140-21401 to AS17-140-21433 computed by Metashape is indicated by annotated squares on the bottom perspective view. The gnomon is fully reproduced in 3-D, as is a footprint, allowing for a precise scaling of the model. The inset shows the markings (white arrow) left by astronauts when hammering the boulder for sampling.</figcaption></figure>
### 3.3. *Three-Dimensional Reconstruction of the Station 2 Boulders*

The rocks of the South Massif were one of the main geological objectives of the Apollo 17 mission. Station 2 is located 7.4 km southwest of the lunar module, at the foot of the South Massif, at the massif slope's intersection with the southeast rim of the Nansen moat (Figure 1), the latter probably formed by a pull-apart due to thrust faulting [8]. The boulders at the feet of the hill slopes may contain clues about the origination of the Serenitatis basin-forming event. Several sets of images taken at Station 2 at the edge between the plain and the South Massif hill appear suitable for a photogrammetric processing, in particular on three main boulders.

Boulder 1 is an interesting ~2 m-diameter foliated and layered fragment-rich breccia, lying right at the break of the slope. Four samples were collected across the layering (samples 72255, 72215, 72235, and 72275). We used a set of 20 images labeled AS17-137-20900 to AS17-137-20909 and AS17-138-21029 to AS17-138-21042 to generate the textured 3-D model shown in Figure 8. The second part of the data set, covering the right part of the boulder, is in black and white with slightly overexposed images that we had to homogenize. This does not seem to affect the photogrammetric treatment. The subtle difference in color between the left and right parts of the boulder should, however, not be considered. The alignment generated a sparse point cloud of 88,000 tie-points. A dense point cloud of about 13 million points allowed the generation of a mesh downsampled to 400,000 polygons. This boulder appears much less coherent than the tan-gray or blue-gray breccia types [30]. It contains large clasts of both dark- and light-colored breccias in a generally light-colored matrix.

<figure><img src="image_11.png" /><figcaption>**Figure 8.** Photogrammetric reconstruction (textured 3-D model) of boulder 1 at Station 2. The location and orientation of the entry Apollo images labelled AS17-137-20900 to AS17-137-20909 and AS17-138-21029 to AS17-138-21042 computed by Metashape is indicated by annotated rectangles in the bottom perspective view. The inset (white rectangle) shows the quality of the textured 3-D model, with arrows pointing to the blue-gray markings left at the location of samples 72255, 72215, and 72235.</figcaption></figure>
The second place of interest at Station 2 is a ~3-m-large boulder labeled “Boulder 2”, which rests on the Massif’s 14.9° slope (Figure 1). Figure 9 presents the 3-D reconstruction that we obtained using 19 images labeled AS17-137-20912 to AS17-137-20956. The alignment generated a sparse point cloud of about 136,000 tie-points. The dense point cloud of 8 million points allowed the generation of a mesh of 500,000 polygons. The gnomon is present in several images, allowing its 3-D reconstruction together with the boulder. The boulder is similar in texture to the ones observed at intrusive melt-breccias at Stations 6 and 7.

<figure><img src="image_12.png" /><figcaption>**Figure 9.** Photogrammetric reconstruction (textured 3-D model) of boulder 2 at Station 2. The location and orientation of the entry Apollo images labeled AS17-137-20912 to AS17-137-20925 as computed by Metashape are indicated in the bottom perspective view. The inset shows the location where samples 72335 and 72315 were extracted.</figcaption></figure>

Finally, Boulder 3 at Station 2 was particularly interesting, as it contained crushed crystalline dunite, which turned out to be possibly ~4.55 billion years old [31,32] and may have originally crystallized near the base of the lunar magma ocean [33]. We used 11 images labeled AS17-137-20963 to AS17-137-20973 to reconstruct this ~70-cm breccia boulder (Figure 10). The alignment generated a sparse point cloud of about 67,000 tie-points. The dense point cloud of about 13 million points was used to generate a mesh of 200,000 polygons, taking into account the simple shape of the area. Only a part of the gnomon was reproduced in 3-D. Several 2- to 4-cm clasts and one 10-cm light-gray clast
clearly appeared in the 3-D model, as two planar perpendicular fractures [30]. Samples 72415-72418 were chipped from a large dunite clast (bright white spot in Figure 10). A sample (72435) of the boulder melt-breccia matrix was chipped from the blue-gray patch. It contains lithic and mineral clasts in a finely crystalline deep bluish-gray matrix.

<figure><img src="image_13.png" /><figcaption>**Figure 10.** Photogrammetric reconstruction (textured 3-D model) of the 40-cm boulder 3 at Station 2. The location and orientation of the 11 entry Apollo images labeled AS17-137-20963 to AS17-137-20973 as computed by Metashape are indicated by annotated squares in the bottom view. Sample 72435 (matrix) was extracted from the blue-gray patch. Samples 72415-72418 were extracted from a large dunite clast (white patch). These sampling areas are well visible on the texture of the reconstructed boulder.</figcaption></figure>

## 3.4. Integration into Virtual Reality of the 3-D Models

Once accurately reconstructed using SfM photogrammetry, the boulders' 3-D models could be integrated into an immersive visualization system in order to be manipulated and analyzed, as if the user were standing by himself in front of the boulders. Beyond the scientific analyses itself, the objective was also to easily share interpretations, enhance scientific communication, or allow student education. For this step, we used two approaches. The first relied on a web-based public platform. The second made use of a game engine. A comparison of the respective pros and cons of each approach for a terrestrial test case is discussed in [14].

### 3.4.1. Publication of the Models on a Web-Based Platform

The Sketchfab 3-D meshes repository platform (www sketchfab.com) provides a digital access and intuitive basic controls to interact with online 3-D objects of various scales, from centimeter-scale samples to large digital outcrop models, without the need for specific software and hardware. The visualization can be done on either a traditional computer screen, on a smartphone, or using a virtual reality headset,
directly online. When using a smartphone or a tablet, the 3-D models can also be viewed in augmented
reality. Contributors are allowed to upload models using a web-based interface, and to define custom
rendering options, such as lighting, material properties, and the size and orientation of the model with
respect to the VR rendering. Annotations can be added to the model, in order to highlight areas of
interest (such as sampling areas) and give relevant information. This makes the exchange of data very
user friendly. Snapshots of the full 3-D models of the Station 2 and Station 6 boulders integrated in
Sketchfab are displayed in Figure 11, together with a character showing the scaling of the objects in VR.
The models are available online (e.g., https://sketchfab.com/LPG-3D).

<figure><img src="image_14.png" /><figcaption>**Figure 11.** Snapshots of the reconstructed boulder models that can be viewed online on the web-based Sketchfab platform (e.g., https://sketchfab.com/LPG-3D). Any user equipped with a virtual reality headset can stand in front of these boulders and investigate them in detail at a true scale in virtual reality. The character is 1.80 m tall.</figcaption></figure>

The rendering in VR allows in particular the user to investigate the morphology of the boulders at
real scales. It provides a very user-friendly way to visualize and analyze the scientific content of the
whole set of integrated images. For example, the distribution and geometry of the small dark veins,
which are intruding noritic breccia clasts in the Station 7 boulder comprising angular minerals and lithic
clasts, are clearly visible in VR. The latest generations of VR headsets have a screen pixel resolution,
which is now good enough to distinguish the fine details on the textures. Similarly, the renderings
could potentially allow quantitative evaluation of the vesicle patterns in the boulders at Station 6.

The size limitation on the uploadable meshes and textures in Sketchfab (50 MB in total for free accounts and 200 MB for pro accounts), and the availability of basic tools only with limited online computing resources, prevent the user from navigating in a complete environment. In particular, this web-based solution is not well suited to the integration of orbital imagery with multiple digital outcrop models. For this more evolved purpose, we rather used a game engine, which optimizes the use of computing resources thanks to the graphical processing unit.
### 3.4.2. Integration into a Game Engine

Compared to web-based platforms, game engines such as Unity 3D or Unreal, developed for and by the game industry, provide a much more versatile way to integrate, manipulate, and merge a large quantity of planetary orbital and in situ data (e.g., [9,10,14]). For our project, we selected the Unity 3D platform (version 2019.2.15 f1), which is thoroughly documented through user manuals, extensive tutorials, and forums. We designed a simple scene using the Unity terrains asset, which allows the generation of real terrains based on 16-bit DEM files for height values, draped by corresponding texture images. The Kaguya TC, Clementine UVVIS, and LRO NAC mosaics discussed in Section 2.1 were used as input to generate these various terrains’ renderings (Figures 2 and 12). The advantages of using the built-in Unity terrains generator is that optimization functions are already implemented to reduce the load on computer resources when displaying data at various scales. The level of details depends mostly on the distance of the player (the rendering camera) with respect to the ground.

<figure><img src="image_15.png" /><figcaption>**Figure 12.** Integration in a game engine, showing a scene composed using 3-D products derived from both orbital imagery and photogrammetric reconstructions based on in situ photographs of the boulders at Station 6 (up) and Station 7 (bottom). This solution provides insights toward a "VR field geology" characterization of the impact melt boulders in Taurus Littrow.</figcaption></figure>

The 3-D models of the outcrops can then be directly added to the scene by using a simple drag and drop function on the scene hierarchy and placed at the relevant location with the right orientation and scaling. We used the Autodesk FilmBoX (.fbx) file export format from Metashape, associated with corresponding jpeg textures, for this purpose. The advantage of the .fbx format is to export the camera positions together with the 3-D model. Navigation within the scene and interactions with elements were programmed in C#. In order to provide the user with the possibility of freely exploring the entire scene, we used a first-person movement script, allowing either to walk at the
surface or to fly over the landscape. Various components can be added to the scene, such as a skybox with stars, a 3-D model of the lunar module, in situ 360° panoramas, or sound recordings of the actual astronauts' conversations, which are triggered when the user is approaching relevant elements, such as the boulders. A user interface control panel is available to switch between different textures (i.e., Kaguya TC versus Clementine) or to directly go to points of interest.

# 4. Discussion and Future Work

One of the main interests of the VR reconstruction is to give a real sense of scales and shapes, without the deformations induced by reprojections on a conventional computer screen. The surface of the Moon lacks atmospheric scattering (inducing distance effects on Earth) and known features, such as trees, roads, or any known reference, that could help the brain to figure out the distances, and therefore the size (and real shape) of landscapes elements. This problem is easily overcome by VR. One main advantage of the 3-D reconstruction compared to single images is it also provides a synoptic view of the rocks and/or outcrops, including their complex real shape, without the loss of any scientific information from the original set of data. In the case of the boulders at Station 6, the detailed vesicular nature of the rock clearly appears when viewed in VR, while it is not so obvious when seen on a conventional flat screen. The exact emplacement of several lunar rock samples also appears readily when viewed in VR (ex. inset of Figure 8). Further work using the 3-D scans of lunar samples themselves might allow in some cases to replace the samples directly in their exact position in VR, which might, for example, be useful for paleo magnetism studies that require the exact position of the sample to be of any relevance [8]. The "astromaterials 3D" initiative at the Astromaterials Research and Exploration Science division at the Johnson Space Center is currently aiming at providing research-grade photogrammetric exterior 3-D models of several tens of lunar samples. The detailed comparison of these 3-D meshes of lunar samples with 3-D meshes of originating rocks, when available and sufficiently detailed, could, for example, be performed using the open source "CloudCompare" (www.cloudcompare.org) software, to look for matching correspondences on the common faces of the biggest samples. This search for correspondences can be envisaged in further studies, when a sufficiently high number of samples will be digitized and available.

Sampling activities during extra-vehicular activities were extremely limited in time, in order to maximize the scientific return of the mission. Using the VR reconstruction of the main places of interest allows users to virtually "come back" to the sites, to analyze them without any time (and space) constraints. Whereas it is obviously not the same as being there, it can help to bridge the gap between a simple computer screen experience and reality.

The photogrammetry technique can also eventually be applied to descent images when available to compute a local high-resolution digital elevation model [26] and to retrieve the descent spacecraft trajectory [34]. Future surface photogrammetric work could also be carried out with the full-resolution scans of the Apollo Image Archive. However, with ~300 MB per image, this would require a very powerful computing system, for an improvement which is still to be evaluated, depending often on the focus quality of the views. These original high-resolution scans also need to be centered and cropped in a consistent manner before being usable in an automated photogrammetric pipeline. It could also be interesting to further work on the lighting conditions in the reconstructed virtual world, in order to merge orbital data with ground-based data taken with a solar azimuth and elevation as close as possible. Figure 12 uses the LRO image M129086118LR for this purpose. Another optimization of the final products could be the generation of normal maps to realistically render the micro relief on the boulders without overloading the graphic processor. Whereas this falls beyond the scope of this work, game engines provide the possibility to fine-tune all the sources of light and their mutual interactions, including multiple scattering effects, which should help in providing an even more realistic experience. This will be particularly interesting with the next generation of graphic cards, which take into account the ray tracing.
**5. Conclusions**

We reconstructed in 3-D and integrated in virtual reality the boulders explored by Apollo 17 astronauts during their EVA3 at Station 6 and Station 7 and the three main boulders observed at Station 2. Whereas recent photogrammetric projects are now mainly carried out with images taken from digital cameras, we observed that the reconstruction process can still work easily with Apollo images, thanks to the quality and homogeneity of the scans of the original films performed at the Johnson Space Center performed in the last decade. We provide reconstructed 3-D models that can be viewed online either on a traditional computer screen, or directly in virtual and augmented reality, using the web-based Sketchfab platform. The rendering in the virtual world can eventually be merged with orbital imagery and in situ 360° panoramas to provide the general context. This approach could ultimately allow the placement of the collected lunar samples in their original context, and help to precisely characterize their sampling area. Particular care has to be taken to handle the lighting conditions in the virtual world to provide a realistic experience.

It is very likely that geological fieldtrips will be available in virtual reality in the near future in many places of interest, either for scientific investigations or for student education, as well as public outreach and historic conservation. This is particularly useful in the case of remote (and almost inaccessible) places, such as moons and planets, where only 12 human beings have experienced a physical field trip so far. The work that we carried out here suggest that the versatility of this approach allows similar reproductions of the geological settings documented in many photographs of the Apollo collection. In the future, similar reconstructions could also be applied to the imagery acquired by the ongoing and forthcoming lunar robotic missions, preparing the next generation of exploration. Realistic 3-D surface models can, for example, be used in simulations dedicated to the training of future astronauts taking an active role in the forthcoming Artemis lunar exploration program.

**Author Contributions:** Conceptualization, methodology, S.L.M., P.E.; software, S.L.M., G.C., P.E., F.C.; validation, H.H.S.; image resources, H.H.S., B.S., J.-P.C.; writing—original draft preparation, review and editing, S.L.M.; supervision, N.M.; project administration, funding acquisition, N.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** Part of this research (G.C.) benefited from the support of the European Commission Research Executive Agency under the Horizon 2020 "PlanMap" project (grant n° 776276). J.-Ph. Combe was funded by the NASA Lunar Data Analysis Program 17-LDAP17_2-0067.

**Acknowledgments:** The authors thank the NASA/JSC/Arizona State University for providing the scanned imagery of the Apollo missions. We are grateful to three anonymous reviewers for their constructive remarks, and we also would like to thank Carolyn Van Der Bogert for useful discussions.

**Conflicts of Interest:** The authors declare no conflict of interest.

**References**

1. McGreevy, M.W. Virtual reality and planetary exploration. In *Virtual Reality*; Elsevier: Amsterdam, The Netherlands, 1993; pp. 163–197, ISBN 0-12-745045-9.

2. Ullman, S. The interpretation of structure from motion. *Proc. R. Soc. Lond. Ser. B Biol. Sci.* **1979**, *203*, 405–426.
[CrossRef]

3. Verhoeven, G. Taking computer vision aloft-archaeological three-dimensional reconstructions from aerial photographs with photoscan. *Archaeol. Prospect.* **2011**, *18*, 67–73. [CrossRef]

4. Favalli, M.; Fornaciai, A.; Isola, I.; Tarquini, S.; Nannipieri, L. Multiview 3D reconstruction in geosciences. *Comput. Geosci.* **2012**, *44*, 168–176. [CrossRef]

5. Arbuées, P.; García-Sellés, D.; Granado, P.; Lopez-Blanco, M.; Muñoz, J. A method for producing photorealistic digital outcrop models. In *Proceedings of the 74th EAGE Conference and Exhibition Incorporating EUROPEC, Copenhagen, Denmark, 4–7 June 2012*; p. cp-293-00121, Abstract #D029.

6. Tavani, S.; Granado, P.; Corradetti, A.; Girundo, M.; Iannace, A.; Arbués, P.; Muñoz, J.A.; Mazzoli, S. Building a virtual outcrop, extracting geological information from it, and sharing the results in Google Earth via OpenPlot and Photoscan: An example from the Khaviz Anticline (Iran). *Comput. Geosci.* **2014**, *63*, 44–53.
[CrossRef]
7. Ostwald, A.; Hurtado, J. 3D models from structure-from-motion photogrammetry using Mars science laboratory images: Methods and implications. In Proceedings of the 48th Lunar and Planetary Science Conference, The Woodlands, TX, USA, 20–24 March 2017. LPI Contribution No. 1964, id.1787.

8. Schmitt, H.H.; Petro, N.E.; Wells, R.A.; Robinson, M.S.; Weiss, B.P.; Mercer, C.M. Revisiting the field geology of Taurus-Littrow. *Icarus* **2017**, *298*, 2–33. [CrossRef]

9. Civet, F.; Le Mouélic, S.; Le Menn, E.; Beaunay, S. Using Virtual Reality for Outreach Purposes in Planetology. In Proceedings of the American Astronomical Society, DPS meeting #48, Washington, DC, USA, 16–21 October 2016. id.419.10.

10. Le Mouélic, S.; L'Haridon, J.; Civet, F.; Mangold, N.; Triantafyllou, A.; Massé, M.; Le Menn, E.; Beaunay, S. Using virtual reality to investigate geological outcrops on planetary surfaces. In Proceedings of the 20th EGU General Assembly, EGU2018, Conference held, Vienna, Austria, 4–13 April 2018; p. 13366.

11. Triantafyllou, A.; Watlet, A.; Le Mouélic, S.; Camelbeeck, T.; Civet, F.; Kaufmann, O.; Quinif, Y.; Vandycke, S. 3-D digital outcrop model for analysis of brittle deformation and lithological mapping (Lorette cave, Belgium). *J. Struct. Geol.* **2019**, *120*, 55–66. [CrossRef]

12. Caravaca, G.; Le Mouélic, S.; Mangold, N.; L'Haridon, J.; Le Deit, L.; Massé, M. 3D digital outcrop model reconstruction of the Kimberley outcrop (Gale crater, Mars) and its integration into Virtual Reality for simulated geological analysis. *Planet. Space Sci.* **2020**, *182*, 104808. [CrossRef]

13. Mat, R.C.; Shariff, A.R.M.; Zulkifli, A.N.; Rahim, M.S.M.; Mahayudin, M.H. Using game engine for 3D terrain visualisation of GIS data: A review. In *IOP Conference Series: Earth and Environmental Science*; IOP Publishing: Bristol, UK, 2014; Volume 20, p. 012037.

14. Nesbit, P.R.; Boulding, A.D.; Hugenholtz, C.H.; Durkin, P.R.; Hubbard, S.M. Visualization and Sharing of 3D Digital Outcrop Models to Promote Open Science. *GSA Today* **2020**, *30*, 4–10. [CrossRef]

15. Haase, I.; Wählisch, M.; Gläser, P.; Oberst, J.; Robinson, M. Coordinates and Maps of the Apollo 17 Landing Site. *Earth Space Sci.* **2019**, *6*, 59–95. [CrossRef]

16. Wolfe, E.W.; Bailey, N.G.; Lucchitta, B.K.; Muehlberger, W.R.; Scott, D.H.; Sutton, R.L.; Wilshire, H.G. Geologic investigation of the Taurus-Littrow Valley: Apollo 17 landing site. *U.S. Geol. Surv. Prof. Pap.* **1981**, *1080*, 225–280.

17. Haruyama, J.; Matsunaga, T.; Ohtake, M.; Morota, T.; Honda, C.; Yokota, Y.; Torii, M.; Ogawa, Y. LISM Working Group. Global lunar-surface mapping experiment using the Lunar Imager/Spectrometer on SELENE. *Earth Planets Space* **2008**, *60*, 243–255. [CrossRef]

18. Barker, M.K.; Mazarico, E.; Neumann, G.A.; Zuber, M.T.; Haruyama, J.; Smith, D.E. A new lunar digital elevation model from the Lunar Orbiter Laser Altimeter and SELENE Terrain Camera. *Icarus* **2016**, *273*, 346–355. [CrossRef]

19. McEwen, A.S.; Robinson, M.S. Mapping of the Moon by Clementine. *Adv. Space Res.* **1997**, *19*, 1523–1533. [CrossRef]

20. Lucey, P.G.; Blewett, D.T.; Taylor, G.J.; Hawke, B.R. Imaging of lunar surface maturity. *J. Geophys. Res. Planets* **2000**, *105*, 20377–20386. [CrossRef]

21. Kammerer, J.; Zeiss, C. The moon camera and its lenses. *Opt. Eng.* **1972**, *11*, 73–78. [CrossRef]

22. Batson, R.M.; Larson, K.B.; Tyner, R.L. Apollo 17 lunar surface photography, in Geologic Investigation of the Taurus-Littrow Valley: Apollo 17 Landing Site. *U.S. Geol. Surv. Prof. Pap.* **1981**, *1080*, 225–280.

23. Lawrence, S.J.; Robinson, M.S.; Broxton, M.; Stopar, J.D.; Close, W.; Grunsfeld, J.; Ingram, R.; Jefferson, L.; Locke, S.; Mitchell, R.; et al. The Apollo digital image archive: New research and data products. In Proceedings of the NLSI Lunar Science Conference, California, CA, USA, 22–23 July 2008; p. 2066. Available online: https://www.lpi.usra.edu/meetings/nlsc2008/pdf/2066.pdf (accessed on 11 May 2020).

24. Borgeson, W.T.; Batson, R.M. *Photogrammetric Calibration of Apollo Film Cameras*; Usgs Open-File Report N69-N27911; U.S. Geological Survey: Reston, VA, USA, 1969.

25. iWitnessPro. 2015. Available online: https://www.photometrix.com.au/i witness/ (accessed on 11 May 2020).

26. Manheim, M.; Wagner, R.; Klem, S.; Robinson, M. Photoscan DEMs from Apollo 15 Hasselblad photographs. In Proceedings of the European Planetary Science Congress, Berlin, Germany, 16–21 September 2018; Volume 12. EPSC2018-996.

27. Agisoft LLC. Metashape Professional. 2020. Available online: https://www.agisoft.com (accessed on 30 January 2020).
28. Meyer, C. Lunar Sample Compendium. In Proceedings of the 41st Lunar and Planetary Science Conference, The Woodlands, TX, USA, 1–5 March 2010; p. 1016, LPI Contribution No. 1533.

29. Reimold, W.U. Pseudotachylite in impact structures—generation by friction melting and shock brecciation?: A review and discussion. *Earth Sci. Rev.* **1995**, *39*, 247–265. [CrossRef]

30. Schmitt, H.H.; Cernan, E.; Lyndon, B.; Johnson Space Center. *Apollo 17: Preliminary Science Report*; Scientific and Technical Information Office: Washington, DC, USA, 1973.

31. Papanastassiou, D.A.; Wasserburg, G.J. Rb-Sr study of a lunar dunite and evidence for early lunar differentiates. In Proceedings of the Lunar and Planetary Science Conference, New York, NY, USA, 17–21 March 1975; pp. 1467–1489.

32. Ryder, G. Chemical variation and zoning of olivine in lunar dunite 72415: Near-surface accumulation. In Proceedings of the 22nd Lunar and Planetary Science Conference, Houston, TX, USA, 18–22 March 1991; Lunar Planetary Institute: Houston, TX, USA, 1992; pp. 373–380.

33. Schmitt, H.H. Symplectites in dunite 71415 and troctolite 76535 indicate mantle overturn beneath lunar near-side. In Proceedings of the 47th Lunar and Planetary Science Conference, The Woodlands, TX, USA, 21–25 March 2016; p. 2339, LPI Contribution No. 1903.

34. Binet, R.; Grizonnet, M.; Torres, A.; Malapert, J.-C.; Jocteur-Bronzier, F. Lunar Landing Site Localization, Trajectory Inversion, and DTM Update from CHANG'E-3 Descent Images. In Proceedings of the Lunar and Planetary Science Conference, The Woodlands, TX, USA, 18–22 March 2019. LPI Contribution No. 2132, id.2433.

<figure><img src="image_16.png" /></figure>

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).