# Recovering Microscopic Images in Material Science Documents by Image Inpainting
## 1 Introduction  
Microscopic images in material science documents (e.g., books, articles, and reports) are generally used to display key structures in a target material [1]. The images are produced by various types of microscopic imaging tools, such as optical microscopy, transmission electron microscopy (TEM), scanning TEM (STEM), scanning electron microscopy (SEM), etc. [2; 3; 4; 5; 6]. Due to the advancement and widespread use of such microscopy instruments, micro-/ nano-scale material images have become more sophisticated and prevalent in material science documents. In the material science field, there is a massive volume of online documents that have been published. In addition, digitalized material datasets, e.g., texts, graphs, and images, are included within them. Due to open access movements with respect to scientific publishing, a huge number of digitized material images are freely available to be downloaded via the processes of web crawling or scraping, which are data mining techniques [7]. If one learns even a small degree of python programming regarding the processes of data scraping, then the material images through microscopy are easily obtained and utilized. Since the advent of the data-driven approach, these material datasets are useful in terms of feeding artificial intelligence (AI) models that enable them to efficiently find explicit material properties [8]. In particular, AI-driven analysis on such material images can accelerate the process of diagnosing real space geometrical information with a higher accuracy instead of the analysis conducted by humans [9; 10]. To guarantee a high-quality AI model, it is necessary that the training dataset consists of abundant and actual microscopic images [11]. Following this, the diverse experimental images obtained via microscopy and which are stored in the material science documents are applicable to the training dataset. However, it gives rise to two main issues. Firstly, when we inspect the microscopic images acquired in the documents, several marks, such as letters, scale bars, and lines/arrows, are added into the raw microscopic photos for the purposes of either expressing image information or for highlighting sections [12; 13; 14]. For example, letters explain ordering in the figure set, the scale bar shows imaging resolution, and lines/arrows indicate remarkable objects. Therefore, one needs to remake the published images to remove the marks. Indeed, image inpainting techniques have been developed in the computer vision field for the purposes of filling in the adequate information required in the missing parts of target images [15]. Moreover, in recent years, they were designed by not only rule-based models but also deep learning-related models in order to achieve a good performance in terms of accuracy and speed, albeit these models were focused only on retrieving original figures regarding people and landscapes [16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26]. Thus, we applied various image inpainting models with the aim of recovering microscopic images in material science documents, specifically in regard to solving the problem detailed above. In this study, we contribute the main things, as follows: (1) we proposed new methods regarding the preprocessing of inputs with statistical and threshold-based masks, specifically with respect to the latest image inpainting methods for recovering microscopic images in material science documents; and (2) we compared the performances of the various image inpainting models in terms of their restorative performance factors, which were related to the similarity with the ground truth of the images. The remainder of this paper is organized as follows: Section 2 introduces the related work and backgrounds of the microscopic images in material science documents and the image inpainting techniques that were utilized in the computer vision field.