# Finding the Least Motion-Blurred Image by Reusing Early Features of Object Detection Network
## 2 Materials and Methods
### Motivation and Method for Finding the Least Motion-Blurred Image
#### 2.1.1 Proposed Method  
A suitable method would be capable of finding the least motion-blurred image by evaluating the relevant image region. If the background of the image is not static, the object (human head, in our case) should be reconstructed using only the part of the image without the background. Having non-static background is the typical situation when the head is imaged by a handheld device and the head is not immobilized. So the background can be eliminated by detecting the region of the head. A head detector is necessary for such a situation. If the head detector is a convolutional neural network (CNN), it is possible to reuse activations of the convolutional layers as features for other tasks other than head detection (Figure 2). Having information about the head's location in the image, we can gather only relevant parts of feature maps, i.e., parts that are spatially corresponding to the area in the input image where the head is located (Figure 3). Image features that define sharpness or motion blur should be found in lower-level image details. These image details should be encoded by the early features of the CNN. Later layers of the CNN encode progressively higher-level structural information about the object it learned to detect. So we will test the first ten feature maps of the CNN for their suitability to provide useful features for the motion blur evaluation of the input image. The CNN head detector [81] used in the experiment is a single-shot detector (SSD) [82] with the VGG-16 [83] backbone. The model adopted in this research was developed by the authors of LAEO-Net [84]. Shapes of the feature maps and names of the convolutional layers that generated these maps are provided in Figure 2 and Table 1.  
Figure 2: Partial structure of the convolutional neural network (CNN) model used for human head detection. Features from the first 10 feature maps (outputs of convolutional layers) of the VGG-16 backbone were investigated for suitability to predict motion blur level in the presented image. Names and filter shapes of the convolutional layers are given above the boxes that denote feature maps. The numbers in the boxes specify the shape of the feature maps. After the head is detected in the image, the corresponding spatial region can be located in the feature map. Features from this region can be used to score an image for motion blur level in the head's region of an image. All features from the region or just a subset of features could be used. As the spatial size of feature maps reduces twice after each max-pooling, the region that corresponds to the head region also gets smaller. In order to equalize the number of features from different feature maps and to decrease the amount of training data, we chose to sample 50 features from the region for the train set and 100 features for the test set. The sampling pattern was different as well. For the training, features were sampled from the region randomly (see Figure 3c). This strategy additionally should work as overfitting prevention by impeding the model from catching some patterns in the data related to facial features. Feature sampling for testing is performed using a uniform pattern (see Figure 3c). Differences in feature vectors from the same or near similar face locations will ultimately describe mostly quality-related image patterns because quality-unrelated image patterns are near constant. The method will be picking the least motion-blurred image from the set of images; therefore, rank-1 performance (right image is returned as top-1 prediction) is important, but not the absolute predicted score of image quality. A ranking algorithm in this situation solves the problem. We chose the LightGBM ranker model for learning to score images according to their motion blur distortions. Traditional gradient boosting decision tree was used, which is implemented in Python's lightgbm package as LGBMRanker class. LambdaRank was used as an objective function.