## 2 Methodology
### Data Collection and Pre-Processing
#### 2.2.2 Image Resolution Degradation and Enhancement  
The objective of Super Resolution (SR) methods is to restore an output as similar as possible to the original high resolution (HR) input by feeding to the network the low resolution (LR) image. Usually, we don't have access to two perfectly synchronized cameras with same parameters but different spatial resolutions: higher and lower. Thus, the common approach is to create a synthetic LR image using HRdata. It is often achieved by down-scaling and then up-scaling of the HR image with inverse scales, i.e., 1/2 and then 2. In this way, the image has the same size but one can observe the effect of resolution degradation, as presented in Figure 2a,b. In the case of Single Image Super Resolution (SISR) models that use convolutional filters, generated LR images are fed to the Neural Network (NN) that sequentially applies the stack of filters with weights adjusted during model training to restore high resolution features. If a stride parameter of convolutions is set to 1 and no pooling is used, the restored output has the same size as the LR input, but important components e.g., facial features are restored as shown in Figure 2c. The majority of DL solutions aimed at generating hallucinated face representations have been focused on visible light images [30; 31; 48]. Yet, it is very important to notice that thermal data have different characteristic that images acquired in the visible light. Due to the heat flow in objects, features present in thermal images are more blurred and a distance between specific facial areas is bigger, leading to a lower contrast and smoother color changes between them (see Figure 3). Hence, motivated by this finding, we designed our custom convolutional-based SR NN, which allows for widening of the receptive field without introducing new parameters. In this way, we show improvement of features restoration as the model gets more details about distant dependencies between interesting components. The architecture of the introduced SR NN is presented in Figure 4.  
Figure 2: Examples of corresponding images from the Lepton 8-bit dataset. The main difference of our network comparing to previous solutions is the use of residual blocks for building representation of the input image in a form of embeddings. Residual blocks have been already proved to ease the model optimization process, allowing for a significant depth increase without the risk of over-fitting [49]. After building the representation of the image, it is mapped to another vector representation once a non-linear activation function is applied. Intuitively, we may treat this mapping as the process of generating high-resolution patches from input representations, as suggested by the pioneer work in the area of applying DL to the SR task [29]. At this step, similarly to [48], we supervise the recursion to minimize the risk of vanishing gradient problem (in Figure 4 it is marked with all 9 gray outputs from recursions being passed to the final convolution operation). In addition, it is usually advisable to correlate the LR image with the reconstructed output, as in general they are very similar except the lack of detailed features in the LR input. This correlation is realized in our model by the skip connection that concatenates the input and the restored outputs in the concat layer. Figure 4: Architecture of the proposed Convolutional-based Super Resolution model used in the study for enhancing thermal sequences from which respiratory rates were extracted. Figure 3: Comparison of pixel values differences (blue plot) at the eye level. Thermal and visible light image taken simultaneously using the FLIR® One Gen 2 camera. It can be observed that the thermal image is characterized by smoother changes and lower differences between adjacent pixels values than the visible light image. The figure presents one of the authors who has agreed to publish his photography.