# Multi-Frame Labeled Faces Database: Towards Face Super-Resolution from Realistic Video Sequences
## 3 Materials and Methods
### Evaluation Metrics
#### 3.4.1 Objective Measures  
The advantage of objective measures is that they provide a consistent metric for evaluation of the quality. All the metrics are described by mathematical equations or are based on open-source software, so the results can be reproduced. The following objective metrics were used. Mean Square Error (MSE) measures the average of the squares of the difference between the actual value and estimated value, i.e., the difference between the images [41]. _MSE_ is computed as  
\[MSE=\frac{1}{m\cdot n}\sum_{i=1}^{m}\sum_{j=1}^{n}(I(i,j)-\hat{I}(i,j))^{2}, \tag{3}\]  
where \(I\) is the original image and \(\hat{I}\) is the estimated image, both of width \(m\) and height \(n\). The best possible result of MSE can be 0--no difference. In general, the lower MSE value is considered better. Peak signal-to-noise ratio (PSNR) is the ratio between the maximum possible power of a signal and the power of corrupting noise [42]:  
\[PSNR=10\cdot\log_{10}(\frac{R^{2}}{MSE}), \tag{4}\]  
where \(R\) is the maximal variation in the input image data. The images are more similar to higher PSNR results (the unit of PSNR is a decibel). Structural similarity (SSIM) [43] is a method for predicting the quality of digital television and images:  
\[SSIM(I,\hat{I})=\frac{(2\mu_{I}\mu_{\hat{I}}+C_{1})+(2\sigma_{\hat{I}\hat{I}}+ C_{2})}{(\mu_{\hat{I}}^{2}+\mu_{\hat{I}}^{2}+C_{1})(\sigma_{\hat{I}}^{2}+ \sigma_{\hat{I}}^{2}+C_{2})}, \tag{5}\]  
where \(\mu\) is the mean, \(\sigma^{2}\) is the variance, \(\sigma\) is the covariance of \(I\) and \(\hat{I}\) and \(C_{1}\) and \(C_{2}\) are constants. The best possible result of SSIM is 1, a higher value represents better image similarity. Cumulative probability of blur detection (_CPBD_) [44] is an index to evaluate the sharpness of an image:  
\[CPBD=P(P_{BLUR}\leq P_{JNB})=\sum_{P_{BLUR}=0}^{P_{BLUR}=P_{BLUR}}P(P_{BLUR}), \tag{6}\]  
where  
\[P_{BLUR}=P(e_{i})=1-exp(-\left|\frac{w(e_{i})}{w_{JNB}(e_{i})}\right|^{\beta}), \tag{7}\]where \(w(e_{i})\) is the width of the edge \(e_{i}\), \(w_{INB}\) is the JNB (Just Noticeable Blur) [45] edge width which depends on the local contrast around the edge and \(\beta\) is a parameter with a median value of 3.6 according to [46]. In the scope of this work, the sharpness is computed as the difference between the ground truth image CPBD and the predicted image CPBD. Facial recognition similarity rate index (next FR rate), which is based on the Face-Recognition framework [31], is used as another objective metric. The project is not state-of-the-art in the area of face recognition, however, it is reproducible and accessible for free. The algorithm extracts 128 measures from the face based on a neural network called Facenet [32] and creates a feature vector. Similarity is computed from the feature vector of the ground truth image and the feature vector of the predicted image using the Euclidean distance. A lower value determines a better \"face match\", according to [32], distance under 0.6 is considered as the same person. Unfortunately, in some cases the Face-recognition algorithm failed to extract face features, therefore, it was not possible to measure the FR rate for the given image. The images which failed in feature extraction differ for each tested method. Because of that the FR rate (all) metric is computed from all the images in which it was possible to extract face features for the given tested model (no dependency on the other models). To make this metric more valuable, the FR rate (\(\cap\)) is defined as the metric computed only from the images in which all tested models were able to extract face features (intersection of the images over all the models). The last metric--the face features extraction fail rate is simply defined as how many face images the Face-recognition failed at all (FR failed). This metric indicates the number of images in which it was not possible to extract face features, therefore, these images should not be used for face recognition.