# FAMU Digital Documentation, a VR narrative for architectural education

Laurent Lescop, Andrew Chin, Sang Bum Park, Anahid Ehtemami, Shonda Bernardin

► To cite this version:

Laurent Lescop, Andrew Chin, Sang Bum Park, Anahid Ehtemami, Shonda Bernardin. FAMU Digital Documentation, a VR narrative for architectural education. 15th Biennial International Conference of the European Architectural Envisioning Association EAEA15, Danilo Di Mascio, Sep 2021, Huddersfield, United Kingdom. pp.483-494. hal-03523016

HAL Id: hal-03523016

https://hal.science/hal-03523016

Submitted on 12 Jan 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Laurent Lescop¹, Andrew Chin², Sang Bum Park², Anahid Ehtemami³, Shonda Bernardin³

¹ENSA Nantes CRENAU/ AAU UMR-CNRS1563, Nantes, France ; ²School of Architecture and Engineering Technology FAMU Tallahassee, Florida, USA ; ³Electrical & Computer Engineering dpt FAMU-FSU College of Engineering Tallahassee, USA

# FAMU Digital Documentation, a VR narrative for architectural education

## Introduction

Florida A&M University is typically recognized as an institution that embraces the potential of under-represented populations and balances a commitment to humanities as well as STEM (*Science, Technology, Engineering, and Mathematics*) centered research. While both characterizations are correct, FAMU's academic breadth provides an opportunity to demonstrate the potential for emerging technology as a tool for cultural memory and historic preservation. A multidisciplinary team has been gathered for the creation of a virtual and augmented reality experience of Florida's historic landscapes.

The "FAMU Digital Documentation Project" recognizes that 3D documentation has emerged as the preeminent technology to support heritage preservation around the world. Organizations like UNESCO, the World Monuments Fund and Smithsonian actively participate in programs for 3D documentation of historic objects and sites. Recently, the University of Florida Historic Preservation Program launched the Envision Heritage initiative with the mission of exploring new and emerging technologies in preservation. In contrast to the UF project's focus on Iraq, Myanmar, Barbados and Thailand, the "FAMU Digital Documentation Project" focuses on Florida's forgotten historic landscapes.

The FAMU project is organized in three parts;

1. Scanner, drone and camera data collection

2. digital campus model creation

3. augmented reality application

The project focuses on Lee Hall, Coleman Library and the Meek-Eaton, Black Archives Research Center and Museum. All three structures contribute to FAMU's designation as a National Historic District. All data, 3D scans, photogrammetry, photos, 3D models are gathered, simplified and reorganized for Virtual Reality.

The Virtual Reality (VR) integrates the real-world imagery with the computer-generated 3D objects. The new product will allow guests to appreciate Florida's historic landscape by juxtaposing the present and its historic condition. This project is also dedicated to the long path to equality for the Afro-American community.
## Historical significance and context

<figure><img src="image_2.png" /><figcaption>Figure 1 - Stokely Carmichael (Kwame Ture) addresses Florida A&M students on Black Power. 1967 (www crmvet org)</figcaption></figure>

Florida A&M University (FAMU) is Florida's only historically black,

state-supported institution of higher education. The University and its graduates have had a profound impact on Florida's economic, educational, social, political and religious development. This has helped advance African Americans in neighboring states and across the country.

On May 6, 1996, the FAMU campus and its surrounding community were designated by the United States Department of the Interior as a National Historic District based on the institution's historic significance and the architectural style of its oldest buildings. The designation also recognized the national achievements and contributions of FAMU students, alumni, faculty and staff, both past and present. The historic district designation resulted from cooperative efforts between FAMU, the Carrie Meek-James N. Eaton, Sr. Black Archives Research Center and Museum, the Florida Bureau of Historical Preservation, Division of Historical Resources, the U.S. Department of the Interior, and FAMU's School of Architecture + Engineering Technology.

Significant African American history landscapes are often formally recognized - the Zora Neale Hurston House, Kingsley Plantation State Historic Site or the Ritz Theater in Jacksonville, Florida. But few places in Florida enjoy this level of recognition. There are even more secondary sites of both ordinary and everyday experiences. In Tallahassee, such stories might otherwise be lost due to suburban growth and the reduction in state funds. This is evident in the recent tree removal in downtown Tallahassee's Cascades Park and the demolition of the Art Deco the Firestone and Bloxham Annex buildings on East Gaines Street.

On Florida's college campuses, funds are occasionally provided for new construction, but the maintenance, repair and renovation of historic sites is easily forgotten. By highlighting the African-American landmarks on FAMU's campus we can stimulate revitalization and foster interest in places that today seem to exist without history or meaning. Indeed, these places can serve as anchors reviving our sense of community.
# Methodology

Here, the workflow from laser or photogrammetric survey to modeling of missing parts and then optimization for real-time visualization within the Quest virtual headset necessitates the control of a large number of parameters, which will be discussed in greater detail below. The expertise stems from the fact that an error on one of the links will result in the entire process being penalized.

FAMU project consisted in recording 6 important buildings: the Black Archives (interior and exterior), Coleman Building, Jackson Davis Building, Jones Hall Building, Lee Hall Building and Tucker Hall Building. In addition to the buildings, the large central plaza of the campus is also modeled, representing a rectangle of 120m by 65m or 7,800m² of gardens, walkways, street furniture and remarkable elements such as the obelisk bearing the Eternal Flame. For everything visible from the ground, a FARO Focus Laser Scanner was used to conduct the laser survey. A DJI Mavic Air 2 drone completed the roof survey.

The technical challenge is as follows: very large surfaces with greenery producing millions of points that are extremely difficult to process, large buildings with roofs that are not visible from the ground and thus inaccessible to the scanner, dozens of laser survey points producing very dense point clouds that must then be associated with a photogrammetric survey performed by a drone. Photogrammetry produces a cloud of points, a mesh, and a textured model, the precision of which is dependent on the shooting distance. The density of the model from the scanner will be affected by the distance, while the accuracy will be less affected.

<figure><img src="image_3.png" /><figcaption>Figure 2 - workflow from survey to Real Time</figcaption></figure>

Thus, the workflow consists of five major steps: the geometric data survey, the generation of raw 2D and 3D data, and the cleaning of the data to retain only the useful information. Indeed, there is no distinction between the various elements of the site in a laser or photogrammetric survey; the ground, the trees, and the buildings form a single entity that must be segregated. The data is then assembled and organized to create models that must be completed when an obstacle obscures the laser beam's path or when an element is not visible. The
entire model is then simplified and prepared to be exported in the application,
allowing for a virtual visit.

<figure><img src="image_4.png" /><figcaption>Figure 3 - Workflow part 1</figcaption></figure>

The most difficult challenge, and the one that requires the most technical expertise, is the transition from raw data to transformed data. Scanning, like photogrammetry, records information that extends far beyond the subject's boundaries. A Faro captures information well beyond the limits of the target area, with a range of 70 to 350m depending on the equipment. With photogrammetry, it is possible to more "define" the subject in post-processing, but without having a strict limitation.

Lasergrammetry / Photogrammetry : technologies and differences

To refresh your memory, lasergrammetry is a technology that involves
shooting a laser beam at varying angles in azimuths and elevations to measure
distances. There are two techniques: time of flight and phase shift. The path
between the emission of a laser pulse and the reception of the wave reflected by
the object to be acquired is measured in the first case. In the second case, we
compare the phases of an emitted signal with those of the returned signal. Each
point found in this manner is expressed in polar coordinates. The most recent
sensors contain a photographic sensor and generate a cloud of colored points.

Photogrammetry is a technique that dates back to the mid-nineteenth century (Albota, 1976), shortly after the invention of photography. The principle entails determining the position in space of a point on a photograph. To do so, we compute the segment that connects this point to the camera's optical center. The calculation is repeated for the next photo shifted by a few degrees and by triangulation we can determine where this point is in space. The power of
computers now makes it possible to correlate and position thousands of points and thus create clouds of millions of points with a set of several hundred photos. The resulting cloud is then meshed and textured to create a 3D model that closely resembles the subject in appearance and dimension. The accuracy of the two techniques is nearly identical (Remondino, 2011), but each has its own advantages; lasergrammetry produces a scaled and geo-referenced output cloud, whereas photogrammetry allows for work at multiple resolutions. Both techniques, however, are hampered by reflective or transparent surfaces, uniform blacks or whites, or too thin volumes. In all of these cases, complementary elements must be manually modelled. The challenge of our project is thus to assemble and make heterogeneous sources coherent using various methods and, in some cases, different operators. The post-processing, which consists in filtering the data, assembling it, and making it consistent, must be done with a complete understanding of the entire chain and the final goal, which is real time.

## Data treatment

Both laser and photo techniques generate millions of points and files that can be several gigabytes in size for each building. The machines' power is put to the test. The paradox is that millions of points are collected, but the information is still incomplete in many areas. One of the resulting clouds is depicted in the illustration below. At first glance, it appears to be very complete, but one can clearly see the large missing surfaces on the ground, the partially raised facades, and the uncovered roofs.

<figure><img src="image_5.png" /><figcaption>Figure 4 - Coleman Building, raw data from laser scanner and after adding complementary data.</figcaption></figure>

To balance the density of the mesh with the quality and size of the textures, a careful choice must be made. A mesh that is too dense will not be smooth in real time, but too much simplification will remove all the details, making the immersive experience unconvincing. The solution is to compensate for the geometric simplification with texture detail and to obtain the detail reliefs with Normal Maps (Blinn, 1978), which can be fabricated from the geometry in high definition. Depending on the light source, the Normal Map simulates a relief effect. It's very useful because it allows you to replace thousands of polygons used to draw bricks with a few polygons and a texture. The advantages for real-time are spectacular.
<figure><img src="image_6.png" /><figcaption>Figure 5 - sampling of mesh resolution with number of triangles</figcaption></figure>

For the development of this project, several software tools were tested and used. We argued that there was no reason to favor one tool over another. Knowing the project's objectives, each partner would test and compare their tools to those of the others. It was necessary to exercise caution in determining what stemmed from the software's functionalities and what stemmed from the user's dexterity. Indeed, a poorly used tool may be less adapted if its user is unfamiliar with it, and conversely, a seemingly rudimentary tool may produce unexpected results in the hands of an expert.

<figure><img src="image_7.png" /><figcaption>Figure 6 - Workflow part 2</figcaption></figure>

What matters in this type of project is the fluidity of the different steps, as well as the avoidance of information loss. The choice of formats is thus critical because
it is the formats that serve as a link between the various partners and from one
stage to the next. For this project, we chose the 3D formats XYZ, PLY, OBJ, FBX,
PLY based on the needs and complexity of the models, with XYZ and PLY
dedicated to point clouds and OBJ and FBX capable of storing mesh, texture, and
even animation for the latter. The JPG and PNG image formats have been retained.
They are both compressed formats, with the JPG being destructive and the PNG
being non-destructive.

<figure><img src="image_8.png" /><figcaption>Figure 7 - Workflow part 3</figcaption></figure>

Virtual Reality application

The final step is to create a virtual application that is compatible with the Oculus Quest headset, which was chosen for its power and freedom of movement. The Oculus Quest was released in May 2019, just as our project was getting underway. Unlike previous headsets, including the well-known HTC Vive, the Quest operates autonomously and does not require external sensors to locate itself in space. The user defines the wandering area, which greatly enhances the immersive experience. The weakness is that the data is embedded in the headset, and with a Qualcomm Snapdragon 835, 4 GB of RAM, and a 2.3 GHz CPU, users should therefore understand how to control the flow of data that you want to process.

Technical aspects

Unity software was used to create the immersive content for the headset. Other
options, such as Unreal, were available to us (Anahid and Al. 2021). Both are open
to academic developments like ours. Unity is especially well suited for lightweight
platforms like Quest. However, due to the large size of the site and the complexity of the models, visual strategies were required to maintain the quality of the experience in a cost-effective manner. The site model is low-resolution, but since it is seen from a distance, this is not a problem. The users' default position is in front of the main facade, which allowed us to simplify the other facades while improving display quality. By concentrating the user's attention on a few controlled points, the user will be less inclined to wander around the building. However, we used the vegetation to create invisible barriers that limit movement.

<figure><img src="image_9.png" /><figcaption>Figure 8 - Coleman and the Carnegie Library Bell. Around, greenery rendered with Point Cloud</figcaption></figure>

The vegetation itself was a major source of contention. It is abundant throughout the campus and is essential to the visual quality and spatial location, but there is a significant constraint in the number of polygons. Finally, a point cloud display enabled a visual and performance compromise by giving the plant a pointillist aspect. Furthermore, as you move away, the point clouds gain visual density, which corresponds to the desired effect.

## Narrative aspects

Our application is divided into two phases: the initial introduction to the virtual world and the visits to each site. The welcoming was thoroughly discussed. Indeed, the invitation to enter the virtual world is critical to the success of an immersive narrative experience (Lescop, 2017). For this application, we chose a neutral space in the center of which is a tray carrying the site model. This provides a global view of the theme as well as a global view of the site. The names of the buildings encroach on the map, and clicking on them transports you to the location as if as you were there.

<figure><img src="image_10.png" /><figcaption>Figure 9 - Launch room with the scaled model as prepared in Unity</figcaption></figure>
The idea behind the application is to be positioned in front of the building to investigate. The user explores the responsive elements with the controller; a kind of Pad in the left hand indicates points of interest and provides access to information. The visit is accompanied by an audio commentary that provides context information. As a result, each building is informed, and the content can be turned into a history or architecture course.

<figure><img src="image_11.png" /><figcaption>Figure 10 – Explanation boards for Quad, Jackson Davis and Lee Hall</figcaption></figure>

## From Quest 1 to Quest 2

Quest 2 was released in October 2020 and provides superior performance. The resolution jumps from 1440 x 1600 to 1832 x 1920, while the refresh rate increases from 72Hz to 90Hz. As a result, there is a better immersion with more precise details, greater reading comfort, and a much more precise and enveloping restitution of the lights. The Qualcomm Snapdragon XR2 chip and 6GB of RAM improve performance, but is still dependent on very good scene organization and a strategy of gradual launch with the use of the LOD (level of detail).

<figure><img src="image_12.png" /><figcaption>Figure 11 - Quest 1 (above) Quest 2 (below) visual comparison same APK</figcaption></figure>
Discussion

The FAMU Digital Documentation application was the result of a lengthy development process in which the entire team was involved at all stages. Being involved throughout the process allows you to understand the consequences of each narrative or technical choice, at least in the academic context. In a broader sense, the project serves as a vital repository for the memory of African American struggles for equality. We give this memory permanence and temporal continuity by inscribing it in space. The virtual may be a more direct and playful way of accessing culture. It allows you to see and experiment. The redundancy of information, text, commentary, and image, inscribes the information more permanently in the memory (Endres, 2015). The virtual application is not the end, but it is an important step toward gaining a broader and more nuanced understanding of our current issues.

<figure><img src="image_13.png" /><figcaption>Figure 12 - Inside the Black Archives</figcaption></figure>

Latest developments

The pandemic that began in Europe and the United States in March 2020 has greatly hampered the application's spread. Demonstrations were to be held in schools or at the Black Archives, with the headset, which had been prepared for the occasion, being presented to the public directly. We acknowledge that learning to use a virtual application with a headset is not always easy, and that assistance is required to reassure users, explain how the controllers work, and prevent potential disruptions that could pollute the experience.

As a result, we attempted to place it on a distribution platform. The Facebook-controlled Oculus Store is relatively closed, and publishing an application is a complicated process. We were not prepared to prepare such a publication as part of an academic project. Another option is to use the SideQuest platform. SideQuest is part of the Khronos Group and promotes itself as a platform for launching applications that will later be sold in the official Oculus Store. But, once again, the general approach appeared relatively complex, not so much for publication as it did to show our future users how to find, install, and then use our application.
The idea was therefore to look for a platform that was free to access and had few constraints on the user side. After evaluating a large number of solutions, we identified social platforms that have seen their development accelerate during the pandemic. Social platforms were originally designed to provide meeting places, but in fact have evolved to replace offices, universities or classrooms. They have abandoned their purely playful nature in order to incorporate presentation, drawing, animation, and, of course, conversation functions. We began by considering three platforms: Engage (engagevr.io), Hub Mozilla (hubs.mozilla.com), and Spatial (spatial.io). By conducting a thematic search, Engage and Spatial were discovered in the Oculus store and Hub Mozilla. Engage and Spatial have a free and a paid version, Hub Mozilla is totally free.

<table><thead><tr><th></th><th>Engage</th><th>Hub Mozilla</th><th>Spatial</th></tr></thead><tbody><tr><th>Cost</th><td>Free/pay</td><td>Free</td><td>Free/pay</td></tr><tr><th>Sim. Visitors</th><td>3 free / 70 pay</td><td>25</td><td>50</td></tr><tr><th>3D upload</th><td>Library</td><td>Custom</td><td>Custom</td></tr><tr><th>Cust. Environment</th><td>No</td><td>yes</td><td>Yes</td></tr><tr><th>Graphic upload</th><td>Yes</td><td>Yes</td><td>Yes</td></tr><tr><th>Avatars</th><td>Toon</td><td>Toon</td><td>Real</td></tr><tr><th>interactions</th><td>Yes</td><td>Yes</td><td>Yes</td></tr></tbody></table>

<figure><img src="image_14.png" /><figcaption>Figure 13 - Into Spatial, presentation room</figcaption></figure>

The decision was made to go with Spatial because it provides the most flexibility. Spatial allows for very easy customization of the environment, allowing for a virtual visit on a 1/1 scale. Spatial also allows for a greater number of concurrent visitors, and the fact that the avatars are realistic adds to the realism of the experience. As we can see, the pandemic context has prompted us to reconsider our strategy and adapt to new constraints. At the same time, these new
constraints have clearly revealed new opportunities, which seem to us to herald
new configurations and practices in the fields of pedagogy, mediation, and
museography. The multiplatform experience enables a large audience to benefit
from an experience that is tailored to their material and temporality. This solution
also restores a level of sociability to VR experiences that has been sorely lacking.

<figure><img src="image_15.png" /><figcaption>Figure 14 - In augmented Reality, the same room in a kitchen !</figcaption></figure>

Bibliography

Albota, M.G., 1976, *Short Chronological History of Photogrammetry*,
Proceedings of XIII Congress of the International Society for
Photogrammetry, Commission VI, Helsinki, Finland

Blinn J.F., 1978, *Simulation of Wrinkled Surfaces*, Siggraph Atlanta, GA, USA

Ehtemami, A. & al. 2021, *Visualizing historical architectural knowledge through Virtual Reality*, IEEE VR 2021; Lisbonne, Portugal

Endres, M., & al., 2015, *Working Memory Capacity and Redundant Information Processing Efficiency*. Frontiers in Psychology. 6. 594. 10.3389/fpsyg.2015.00594.

Fribourg R., Argelaguet F., Lécuyer A., Hoyet L., *Avatar and Sense of Embodiment: Studying the Relative Preference between Appearance, Control and Point of View*, IEEE Transactions in Visualization and Computer Graphics (TVCG) IEEE VR 2020 special issue

Lescop.L. 2017 360 vision, from panoramas to VR. In Tom Maver, editor, Envisioning Architecture: Space / Time / Meaning, Volume 1, pages 226–232, Glasgow, United Kingdom, September 2017d. Tom Maver, Paul Chapman, Christopher Platt, Victor Portela and David Eaton, Published by the Mackintosh School of Architecture and the School of Simulation and Visualization at the Glasgow School of Art. Scotland

Remondino, F., 2011. *Heritage Recording and 3D Modeling with Photogrammetry and 3D Scanning*. In: Remote Sensing. Vol. 3, n° 12, p. 1104-1138. DOI 10.3390/rs3061104.