# Three-Dimensional Human Head Reconstruction Using Smartphone-Based Close-Range Video Photogrammetry
## 2 Materials and Methods
### Frames Selection Methods  
Two goals may be achieved simultaneously by performing frame selection--removing redundant data from the dataset to reduce the dataset and, as a result, reduce the computational load, and removing images that are low quality due to motion and defocus blur. We implemented and tested two different methods for frame selection. The first method is a straightforward extension of the simplest frame reduction strategy where every \(N\)-th frame is selected. The modification is made by integrating image quality estimates into the frame selection process. Image quality is estimated for every frame, and instead of selecting every \(N\)-th frame, the frame with the highest quality from \(N\) consecutive frames is selected. The image quality estimation method is presented below. The second branch of the baseline algorithm is created by adding the simplest frame reduction strategy together with the previously added image background elimination and is labeled as _Pipeline_ 3 with sub-branches [a | b] (Table 1). This frame reduction strategy is implemented as _4. Image quality estimation_ and _5. Frame selection_ steps of the generalized 3D reconstruction pipeline (Figure 1). Image Quality Estimation  
The image sharpness metric was used as an estimate of image quality for the frame selection. This algorithm implements the _4. Image quality estimation_ step of the generalized 3D reconstruction pipeline (Figure 1) when _Pipeline_ 3[a | b] is selected (Table 1). Key algorithm steps for image sharpness estimation:  
Figure 2: Diagram of the convolutional neural network (CNN) model that was used for human head detection showing the main convolutional layers. The model is composed of a Single-Shot Detector (SSD) head and a VGG-16 backbone. Features from the last 15 convolutional layers were investigated for suitability to sort images by similarity in order to predict the relative location of the frames in the 3D space. Layer names and filter shapes are presented above the boxes (notations of convolutional layers). The numbers on the boxes specify the shape of the feature layers. The model adopted in this research is developed by the authors of LAEO-Net [82]. 1. Detect the head region defined by BBox (it is already detected in the background removal step);
2. Calculate Region of Interest (RoI) parameters: define the size of square as the largest edge of head BBox;
3. Crop RoI part and resize to 256 \(\times\) 256 px image patch;
4. Filter patch using Laplacian of Gaussian (LoG) filter (3 \(\times\) 3 filter size, \(\sigma=0.5\));
5. Calculate the variance of filtered patch;
6. A larger variance represents a higher image sharpness. Frame pose prediction by ordering images according to similarity is a crucial step to create a subset of images that covers a wide area around the object and contains evenly spaced images. Here, we define image similarity in terms of camera pose in 3D space. Pictures or frames having similar poses will likely be similar if the scene is static. Image ordering by similarity is a proxy task to predict the relative poses of the frames. Having relative poses, we could select the best quality image from the image group corresponding to a predefined region of the surrounding space. To imitate image ordering in 3D space or in 2D space, if we assume that the camera keeps an approximately constant distance from the head, we would like to get 3D or 2D embeddings of the images. To get image embeddings in 2D or 3D, a possible solution would be to collect multidimensional feature vectors that describe images containing a head from the CNN that were used to detect heads in the images and later to reduce dimensionality. The CNN model is trained to detect heads, so the features extracted by the network should serve as good descriptors of the head image patch. Additionally, it would be the third task where the same CNN model serves, i.e., head detection for initial feature point selection, for image quality evaluation as the RoI provider, and here, as a feature extractor for image description.