# Multi-Frame Labeled Faces Database: Towards Face Super-Resolution from Realistic Video Sequences
## 3 Materials and Methods
### Training and Testing Data  
For training and evaluation, we introduce a new dataset called Multi-frame Labeled Faces Database--MLFDB. The main difference to the existing datasets is that this dataset does not contain just a single facial image, but it contains a sequence of consecutive frames in a video record. This sequence of images is in a low-resolution (32 \(\times\) 32 pixels) and serves as an input of the super-resolution methods. Additionally for each of those sequences, there was added one frame in a higher resolution (64 \(\times\) 64 pixels) which serves as a ground truth, i.e., the optimal output of the super-resolution method (see Figure 4).  
What should be emphasized here is that the faces in the dataset were selected on the borderline where identification is difficult (i.e., each low-resolution frame of 32 \(\times\) 32 pixels) and when it is easier to use it for identification (i.e., higher resolution image with 64 \(\times\) 64 pixels). The main objective is not to have the resulting facial images good looking, but rather that they can be used for identification.  
The dataset is of a comparable size, as are the major face datasets (it contains 17,426 faces and approximately 6000-7000 unique persons). It tries to avoid the problem of bias, i.e., the recordings are from various real-world scenarios where faces are nearby to each other, a face is partially covered including glasses, beard, scarr or other coverings (e.g., face paintings in some cases). The dataset also contains various facial images including various lighting conditions (i.e., high, low, and poor lighting quality conditions), various recording hardware was used, videos were encoded using various algorithms, faces contain a wide range of ages, faces were recorded from various poses, and they contain various emotional expressions (including smile, fear, anger or surprise). It also  
Figure 4: An example of sample data. Seven low-resolution images (32 \(\times\) 32 px) and label (64 \(\times\) 64 px).  
Figure 3: Scheme of the experiment. The dataset consisting of several low-resolution images and one high-resolution target image was used for training and evaluation of a couple of different super-resolution methods. These methods were evaluated using several objective and subjective metrics.  
covers all main ethnic groups including Europeans (i.e., Middle Easterners and Mediterraneans), East Indians, Asians, American Indians, Africans, Melanesians, Micronesians, Polynesians, Australians, and Aborignines.  
The created dataset was released online and can be used to reproduce the experiment described in this paper. Although it was primarily designed for multi-frame processing, it is expected its utilization will be wider.  
Youtube Faces DB [9] can be considered as a similar dataset to introduced MLFDB. However, it differs in several parameters: Youtube Faces DB contains faces that are not on its borderline when identification is possible and simple downsampling the facial images can bring bias and over-fitting. Moreover, it contains only 1595 different people, but MLFDB contains approximately 6000-7000 different people (see more details in Section 5.1), and also it is not designed for super-resolution purposes.