dataesr/scientific-paragraphs-categorization huggingface dataset:
We additionally produced a sample of 80 000 more realistic CFIS-like images derived from real COSMOS data. Results demonstrate that the BlendHunter weights, learned on the parametric models, can successfully be transferred to this more complex data set and achieve >90% classification accuracy, outperforming SEP by a few percent on the full sample when the appropriate weights are used. The results also indicate that BlendHunter is capable of achieving results that are roughly consistent with those of SEP even when weights are used that have been trained with a significantly different noise standard deviation. BlendHunter notably outperforms SEP by 5-15% for blends in which the galaxies are separated by less than ten pixels. This is an interesting result as these are precisely the cases that generate biases in galaxy shape measurements  (MacCrann et al.
Acknowledgements. The authors wish to acknowledge the  COSMIC  project funded by the  CEA DRF-Impulsion  call in 2016, the  CrossDisciplinary Program on Numerical Simulation  ( SILICOSMIC  project in 2018) of  CEA , the  French Alternative Energies and Atomic Energy Commission. The Euclid Collaboration, the  European Space Agency  and the support of the  Centre National d'Etudes Spatiales. This work was also supported by the  ANR   AstroDeep  project -grant  19-CE23-0024-01. This work has made use of the  CANDIDE Cluster at the Institut d'Astrophysique de Paris  and made possible by grants from the  PNCG  and the  DIM-ACAV. This work is based on data obtained as part of the  Canada-France Imaging Survey , a  CFHT large program  of the  National Research Council of Canada  and the  French Centre National de la Recherche Scientifique.
Identity documents recognition is an important sub-field of document analysis, which deals with tasks of robust document detection, type identification, text fields recognition, as well as identity fraud prevention and document authenticity validation given photos, scans, or video frames of an identity document capture. Significant amount of research has been published on this topic in recent years, however a chief difficulty for such research is scarcity of datasets, due to the subject matter being protected by security requirements. A few datasets of identity documents which are available lack diversity of document types, capturing conditions, or variability of document field values. In addition, the published datasets were typically designed only for a subset of document recognition problems, not for a complex identity document analysis.
In this paper, we present a dataset MIDV-2020 which consists of 1000 video clips, 2000 scanned images, and 1000 photos of 1000 unique mock identity documents, each with unique text field values and unique artificially generated faces, with rich annotation. For the presented benchmark dataset baselines are provided for such tasks as document location and identification, text fields recognition, and face detection. With 72409 annotated images in total, to the date of publication the proposed dataset is the largest publicly available identity documents dataset with variable artificially generated data, and we believe that it will prove invaluable for advancement of the field of document analysis and recognition.
The global demand for reliable automated identity document processing systems lead to the need of developing new methods for document detection, identification, and location; segmenting the document into relevant fields; extraction of text fields and graphical elements, such as signatures and faces; as well as anti-spoofing, validating the document authenticity, and detection of markers of potential fraud. The variation of input sources has also been growing: while traditional methods of scanning identity documents using flatbed or specialized scanners are still relevant, remote identification procedures now require the document analysis systems to process camera-captured photos, as well as perform ID recognition in a video stream. One of the most important aspects of effective research of new document analysis methods is the availability of openly accessible datasets for training and benchmarking.
However, when it comes to identity documents, this issue becomes very hard to resolve, since identity documents by their nature contain sensitive personal data, which is protected by worldwide regulations. As was mentioned in the previous sections, since identity documents by their nature contain sensitive information, there are very few publicly available datasets of identity document images, and those which exist contain either partial information, or contain synthetic examples of ungenuine documents. Existing datasets dedicated specifically to identity document images include LRDE Identity Document Image Database (LRDE IDID)  [5] , the recently published Brazilian Identity Document Dataset (BID Dataset)  [4] , and the Mobile Identity Document Video dataset family (MIDV)  [6, 7] , to which the dataset presented in this paper also belongs.
Some larger datasets, dedicated to address the issues of a broader document analysis problem, such as the ones from SmartDoc family  [8] , also contain identity document images. Existing datasets of identity document images have disadvantages which present themselves when the datasets are used by researchers as benchmarks for specific identity document analysis tasks. The LRDE IDID  [5]  and the identity documents subset of SmartDoc  [8]  comprise a small amount of document samples, which allows them only to be used as reference benchmarks, without deeper analysis of identity processing methods. BID Dataset  [4]  addresses that issue, featuring 28800 synthetically generated document images with 8 different document types. At the same time, the images of BID Dataset were generated with artificial inscription of text field values over the automatically masked document regions, which might lead to the field presentation widely different from the one on actual documents, and the document owner faces were blurred in each document image, which make the dataset impossible to use for evaluation of face detection and location methods.
In addition, BID Dataset only features ideally cropped documents, however some further datasets from the same team include synthetic scans and photographs as well. The first dataset of the MIDV family was MIDV-500  [6] , which contained 500 video clips of 50 identity documents, 10 clips per document type. The identity documents had different types, and mostly were «sample» or «specimen» documents which could be found in WikiMedia and which were distributed under public copyright licenses. The dataset focused on mobile video capture, featured clips shot using two smartphones (Apple iPhone 5 and Samsung Galaxy S3) under five distinct conditions: «Table», «Keyboard», «Hand», «Partial», and «Clutter». The conditions represented in MIDV-500 thus had some diversity regarding the background and the positioning of the document in relation to the capturing process, however they did not include variation in lighting conditions, or significant projective distortions.
To address the latter issues a dataset MIDV-2019  [7]  was later published as an extension of MIDV-500, which featured video clips captured with very low lighting conditions and with higher projective distortions. Video clips of MIDV-2019 were captured using Apple iPhone XS Max and Samsung Galaxy S10. The dataset was also supplemented with photos and scanned images of the same document types  [9]  to represent typical input for server-side identity document analysis systems. Since its publication, MIDV-500 dataset and its extension MIDV-2019 were used to evaluate the methods of identity document images classification  [10] [11] [12] ; identity document location  [9, 13] , including the methods based on semantic segmentation  [14] ; detecting of faces on images of identity documents  [15] ; and methods related to text fields recognition, including single text line recognition  [16] , per-frame recognition results combination  [17, 18]  and making a stopping decision in a video stream  [19, 20].
The dataset was also used to evaluate the methods of choosing a single best frame in the identity document video capture  [21]  and assessing the quality of the frame for its processing by an identity analysis system  [22] , detection and masking of sensitive and private information  [23]  and general ID verification  [24]. The main disadvantage of the datasets of the MIDV family is the scarcity of different document samplesall images were made using the same 50 physical document samples. The fact that the text field data and the graphical data (signatures, faces) are not variable, makes it harder to produce reliable evaluations of identity document analysis algorithms, or even to calculate meaningful benchmarks. In this paper, we present a new dataset MIDV-2020, based on the 10 document types featured previously in MIDV-500 and MIDV-2019.
One of the main goals of the MIDV-2020 dataset, presented in this paper, is to provide variability of the text fields, faces, and signatures, while retaining the realism of the dataset. The dataset consists of 1000 different physical documents (100 documents per type), all with unique artificially generated faces, signatures, and text fields data. Each physical document was photographed, scanned, and for each a video clip was captured using a smartphone. The ground truth includes ideal text field values, geometrical position of documents and faces in each photo, scan, and video clip frame (with 10 frames-per-second annotation). We believe that MIDV-2020 will prove invaluable for the advancement of the field of document analysis and recognition, as well as serve as a benchmark for modern identity document processing systems.
Composition of the dataset The set of base document types for MIDV-2020 comprises 10 document types, each present in previously published MIDV-500 and MIDV-2019 datasets. The identity document types of MIDV-2020 are listed in Table  1 , with the codes of the PRADO database  [25]  for each document type except the Internal passport of Russia. 100 sample documents were created for each of the 10 document types present in the dataset. To create unique sample documents the original sample images obtained from Wikimedia Commons (we used the same source images as in MIDV-500  [6] ) were edited to remove non-persistent data (such as signature, photo, and text field values). Then, the generated text field values for each field were added to the images using a font which closely resembles the original font of the sample.
In addition, for each document a mock signature was drawn, vaguely resembling the spelling of the last name, and a unique face picture was added. The values of gender, birth date, issue date, and expiry date were generated in accordance with the specifics of issuing countries and a pre-set distribution of age and gender parameters: 1. 80% of generated documents correspond to adult holders (ages 18 through 60), 10% of generated documents correspond to seniors (ages 60 through 80) and 10% to children and adolescents (17 or less) depending on the minimum age for the document issue. 2. 50% of the generated documents correspond to female holders, and 50% to male holders. Names and addresses were generated using the databases of existing names and examples of addresses available online, using Wikipedia listings of names  [26] , and online name generators  [27].
Finally, for document types where some text fields are oriented vertically, an additional orientation attribute is provided for such text fields which indicates the angle of the field's counterclockwise rotation expressed in degrees. The scanned images were captured in TIFF format, then converted to JPEG using ImageMagick 7. 0. 11 with default parameters. The JPEG images with their annotations are placed in the dataset within the archives «scan_upright. tar» and «scan_rotated. tar». The original TIFF images are placed within the archives «scan_upright_tif. tar» and «scan_rotated_tif. tar». There are 1000 upright scans and 1000 rotated scans in total. The names of the scanned images correspond to the names of the template image from which the physical document was created. The annotations for scanned images are provided in JSON format readable with VGG Image Annotator v2.
Photos For each physical document sample a photo was taken, given various conditions and two smartphones. Half of the photos were captured using Apple iPhone XR, and the other half using Samsung S10. The following capturing conditions were used: 1. Low lighting conditions (20 documents of each type); 2. Keyboard as a background (10 documents of each type); 3. Natural lighting, captured outdoors (10 documents of each type); 4. Table  as a   The ranges of codes of the document samples which correspond to each of the listed conditions are specified in Table  2. All photos are stored in JPEG format and have the resolution of 2268x4032 pixels. The images with annotations are placed in the dataset within the «photo. tar» archive. The names of the images correspond to the names of the template from which the physical document was created.
Benchmarks While the main goal of the paper is to present an identity document dataset MIDV-2020, in order to provide a baseline for future research involving the dataset, in the following sections several benchmarks using MIDV-2020 will be presented. Based on the topics for which the datasets of the MIDV family were used (see the Introduction section), the benchmark will include the task of content-independent document boundaries location, semantic segmentation of the document body from the background, feature-based document location and type identification, text fields recognition, and face detection.
As shown in  [35] , the representatives of all three classes successfully solve the problem given a subset of an open dataset SmartDoc  [39] , which complies to the problem statement. There is an openly available model from  [32] , which uses a convolutional neural network for document vertex detection. For the purposes of a baseline evaluation, we used a model trained using a dataset of A4-size documents (SmartDoc subset and own-collected). We also asked the authors of  [35] [36] [37]  to evaluate their algorithms using MIDV-2020. The method described in  [35]  uses straight lines (for generating quadrangle hypotheses) and salient regions (for comparing and ranging the hypotheses). This algorithm shows state-of-the-art results on the open dataset MIDV-500  [6]  and can be used on low-end computational devices.
The algorithms  [35]  and  [37]  both require the knowledge of the document's aspect ratio and the knowledge of the camera internal parameters (focal length and position of a principal point). Although we do not know the latter from our problem statement, in experiments with  [35]  and  [37]  we used the focal length equal to 0. 705 from the diagonal of the image and the position of principal point in the center of the image. The algorithm  [36]  as well as  [35]  uses straight lines and salient regions however does not require knowledge of the aspect ratio and internal parameters of the camera. In order to estimate the accuracy given a dataset of images we will use the following statistics: the rate of correct answers for the two binary quality criteria: IoU 0. 9  (see Table  4 ) and 4  D 0.
02  (see Table  5 ), as well as a mean Jaccard score gt IoU (see Table  6 ). The source code of the statistics implementation published at https://github. com/SmartEngines/hough_document_locali zation, as referenced in  [35]. Example outputs of the evaluated systems are presented in Fig. 5. Note that on the Fig. 5a  the output of the  [35]  and  [37]  totally overlap, the same on the Fig. 5c , but for systems  [35] [36] [37]. [32]magenta,  [35]  red,  [36]  green,  [37]  blue The results of the system presented in  [32] , while being quite low, correspond to the data presented in the original paper: the mean value of gt IoU on the 5-th subset of the SmartDoc dataset (excluding the training subset) was reported to be 0. 67. Lower results on the 5-th subset of SmartDoc was explained by the presence of additional objects near the target document.
The results of the evaluation are presented in Table  7. The results of  [13]  were obtained using a published source code at https://github. com/ricardobnjunior/HU-PageScan, referenced in the original paper. Examples of the semantic segmentation results on images from various MIDV-2020 subsets are presented in Fig. 6. a) b) c) d) e) f) g) h) i) j) k) l) Fig. 6. Examples of semantic results of images from MIDV-2020 dataset (the source images correspond to the ones presented in Fig. 5. a) source image; b) result of  [13] ; c) result of  [14] ; d) source image; e) result of  [13] ; f) result of  [14] ; g) source image; h) result of  [13] ; i) result of  [14] ; j) source image; k) result of  [13] ; l) result of  [14]  Low results of the HoughEncoder method  [14]  on the «Upright scans» and «Rotated scans» subsets of MIDV-2020 can be explained by its training on MIDV-500 and MIDV-2019 datasets, which do not feature scanned images.
Feature-based document location and identification The previous two sections considered document location methods which did not directly rely on the internal document content (like the boundary analysis methods), or which learned the differences between document content and background (like in the case of semantic segmentation) given a train dataset. Identity documents in general can be classified as semi-structured objectstheir content consists of static elements of the document template and variable personal data. Within a single document template type the exact or approximate position of the variable personal data is known. Let us consider the task of document type identification given a scanned or photographed image. Given a predefined set of identity document templates (which, in a general case, may contain hundreds or thousands of different templates) the task is to classify the image to one of the classes representing a template, or to a null class, which correspond to the absence of a known document type.
The state-of-theart approach  [40]  for solving such a problem is based on local features matching. The typical base features are keypoints and their descriptors, however the feature space can be expanded with line segments, quadrangles, vanishing points, etc. [9, 41]. To provide a baseline for evaluation of such methods using MIDV-2020 dataset we used the algorithm described in  [42] : 1. For each ideal document template an ideal image is used to mark up the zones with variable personal data. 2. Keypoints and their descriptors are extracted in an ideal document template image and in the analyzed image. The keypoints inside the variable data zones are discarded. 3. A set of direct matches is constructed. For each local descriptor of the analyzed image the closest descriptor from the set of all stored ideal image descriptors is found.
Within the scope of the benchmark evaluation, in order to maximize reproducibility, the search for direct and inverse matching was performed without any randomized approximation, and no additional geometric restrictions were imposed, following the procedure described in  [11]. For keypoints detection the SURF method was employed  [43] , and for matching the keypoints we evaluated the SURF descriptor  [43] , and the binary descriptor BEBLID  [44]  in 256-and 512-dimension variants. The accuracy of document type identification as measured on MIDV-2020 dataset is presented in Table  8. For an end-to-end system it is important not only to correctly identify the document type, but to correctly find the coordinates of the document's vertices as well, especially if the procedures for text fields extraction and recognition depend on a preliminary rectification of the document page.
), we used the coordinates provided in the dataset ground truth. To obtain rectified field images we extended the annotated bounding boxes by 20% in each direction. If the field contained letters that cross the baseline or the cap line (this information is provided in the ground truth as additional attributes of each annotated field), we applied an additional 10% padding. We divided all fields in the documents into four groupsnumerical fields, fields written using numerals and letters of the English alphabet (ISO basic Latin alphabet), fields written using numerals and local alphabets, and machine-readable zones (MRZ). We excluded four fields («authority», «code», «nationality», and «birth_place_eng») of the passport of Greece from our experiment as they contain bilingual (Greek and English) text.
Instead of detecting live faces under spontaneous scenarios as in  [48, 49] , detecting face on identity documents is a more challenging problem due to the conditions under which the images of identity documents have been captured being more complex and with much less constraints. The face detection on identity documents encounters more challenges such as partial face, defocus, reflection, the varying shift in space, unconstrained shooting angle and the varying illumination conditions as shown in Fig. 9. Thanks to the great advances in deep neural network architectures, face detection in general face images has made great progress in the past decade  [50] [51] [52] [53] [54] [55] [56] [57] [58] [59]. FDDB  [48]  and WiderFace  [49]  are the most widely used datasets for training and evaluating face detection algorithms.
However, FDDB and WiderFace only include the general/liveness face images in the spontaneous scenarios having no face images from the identity documents. Although FDDB and WiderFace contain the faces captured under the «unconstrained» environment, the face images on the identity documents included in the proposed MIDV-2020 have more variations as shown in Figure  9. Besides, the resolution of images in FDDB and WiderFace are relatively low (from 40x40 to 400x400 maximum) compared to the images in MIDV-2020 (2160x3840). Thus, not only for the research of face detection on identity documents but also for the research of the generalization of face detection algorithms, the MIDV-2020 is an essential supplement for the current face detection datasets. Benefiting from the blooming of the deep learningbased generic object detection, face detection makes great progress on the extreme and real variation problem including scale, pose, occlusion, expression, makeup, illumination, blur etc.
TinaFace  [53]  achieved impressive results on unconstrained face detection by revising a generic object detector. However, nowadays the face detection methods seem to make the overall algorithms and systems become more and more complex. In order to provide a strong but simple baseline method for face detection in the images of identity documents, MTCNN is selected in this work as the face detection method. MTCNN is almost the most widely used face detection method of today given its high stability, low latency and simplicity. The evaluation of face detection on MIDV-2020 has been conducted on the five different types of images of identity documents respectively. The ground truth of bounding box of faces in MIDV-2020 has been semiautomatically annotated, i. e. , the faces in the dataset have been firstly detected automatically by pretrained MTCNN and then the detected bounding boxes have been calibrated manually to serve as the ground truth of the bounding boxes of faces in MIDV-2020.