# Autonomous Service Drones for Multimodal Detection and Monitoring of Archaeological Sites
## 3 Materials and Methods
### Proposed Built Cultural Heritage Detection Algorithm  
We develop a neural network-based detection framework to handle the orthomosaics produced through the video mosaicking algorithm. The proposed framework uses a pre-trained YOLO model that detects built cultural heritage within an orthomosaic. The detected bounding boxes of the cultural heritage structures are then handed a segmentation tool for post-processing. We used a YOLOv2 network trained for a \(512\times 736\) input sample images. Each sample is associated with a file containing the class and the bounding box information for each area in the sample. Input samples are automatically resized to the specified size in the network parameters.  
The YOLO architecture involves a convolutional neural network used to extract visual features from the input samples. Assuming multi-level grid shaped cultural heritage area, each sample is divided into multi-level grid with \(S\times S,(S\in[8,16,32])\) cells, and the network predict whether a heritage structure exists in each cell in addition to four float parameters defining the bounding box of the structure. The prediction target for each pit is stored in a tensor with the following format:  
\(S\times S\times[C_{x},C_{y},w,h],S=32\) for the output probability map as in Figure 7.  
where:  
\(C_{x}=x\)-axis centroid of the bounding box.  
\(C_{y}=y\)-axis centroid of the bounding box.  
\(w=\)Width of the bounding box.  
\(h=\)Height of the bounding box.  
Data Generation  
To create a reliable network for architectural cultural heritage detection, an unbiased data set of a large size is essential. The input to the network requires an orthomosaic for various heritage structures captured using drones. To facilitate the process of data acquisition, we built a physical model in the lab environment simulating a grid of cultural heritage structures. We used tissue boxes to simulate the depth and placed heated metal bars on top of each box to cater for the thermal difference expected between the floor and the structure of interest. We simulated deformation over time using newspapers of  
Figure 6: Full pipeline for the proposed mosaicking algorithm using RGB flight video.  
Figure 7: Prediction class probability map.  
different volumes. We used this model to generate most of the data by artificially creating samples of different orthomosaics using images of individual structures for the purpose of training and testing our proposed algorithms. For that, we programmed a data generation tool that arranges the raw site images from a given data source with randomized order and orientation to generate different orthomosaic samples. The generated samples are also submitted for randomizing grid size and visual shift. Figure 8 shows samples of the raw heritage structure images we used.  
The generated data set is processed through a data augmentation tool to increase the data set size by 600%. We designed the augmenter to generate copies of each original sample while randomly applying horizontal and vertical shifting, zooming, individual manipulation of RGB channels, brightness adjustment, and perspective transformation.