# Towards Facial Biometrics for ID Document Validation in Mobile Devices
## 5 Datasets  
It is important to note that the deployment of a mobile face recognition system for document security purposes implies dealing with images that are captured by the portable digital camera from the physically printed documents.  
As an example, some works directed on document scanning scenario (which is constrained with the absence of perspective transformations) introduce collected print-scan datasets to deal with that problem [28] or methods for generating such print/scan datasets automatically [29]. However, the last option usually can barely cover all aspects of illumination and acquisition distortions with various capture devices.  
The document acquisition with a portable digital camera introduces even more variable noise to the captured images, which applies to the perspective inaccuracies and more severe lighting distortions.  
Following the original purpose of protecting the ID document at the moment of its personalization, for all images from the chosen original dataset, we collected their print/capture digital copies trying to cover possible noise and distortion variations during the process. Acquired images were then automatically frontalized (see examples in Figures 4 and 5).  
The differential verification with such an approach implies estimating the similarity between the original and the captured images in a single verification iteration. In order to retain such differential manner of the processing scenario, we labeled the sets of collected images similarly for original and captured ones.  
A similar print/capture dataset (Print/Capture Aberdeen [30]) was obtained by [PERSON] et al. [5]. We extend that approach to several other datasets that contain frontal face images complied with travel document standards [4] (Utrecht [30] and AR [31]), including various level of acquisition parameter variations. As a result, we obtain several print/capture datasets ([[https://github.com/vistteam-isr-uc/trustfaces-template-verification](https://github.com/vistteam-isr-uc/trustfaces-template-verification)]([https://github.com/vistteam-isr-uc/trustfaces-template-verification](https://github.com/vistteam-isr-uc/trustfaces-template-verification)) (accessed on 30 June 2021)) which we call as follows :  
* Print/Capture Aberdeen v2 (89 identities, 15 k captures);
* Print/Capture Utrecht (67 identities, 16 k captures); and
* Print/Capture AR (135 identities, 29 k captures).  
In the process of training on the templates extracted from such datasets, the network learns the proper weighting for the particular template components. At the same time, such an approach optimizes the further face verification process by learning existing irregularities related to printing, digital capture, misalignment and rectification process.  
At the same time, practical deployment in the mobile application assumes handling the stream of frames from which only a few are selected for processing when the bad quality ones (over illuminated or occluded) may be skipped. Such practical details are usually important to be accounted in the early research stage. That is why one has to be careful choosing the strategy for selecting images to be included in the dataset, reducing ones that will likely be skipped during the deployment.  
We achieve that by carefully designing a rectification process that is primarily directed on the eliminating perspective deformation of the document on the captured image.