# Recovering Microscopic Images in Material Science Documents by Image Inpainting
## 3 Results
### Generating Inputs Using Masks  
When we applied the image inpainting methods, the inputs should have been pre-processed as the microscopic images in the material science documents were convolved with a mask. Then, the mask was filled in either black or white colored pixel values for specifying normal or undesired areas. Generally, this would be a binary image and should have the same size as the input image [24; 26]. In the mask image, the pixels that should be preserved are those which are black, and the pixels in the undesired areas are in white [15]. For training the model, we used a random mask [23]. Next, we used the mask that covers the two additives, i.e., the alphabet characters and scale bars, on the images for testing the model. There was, initially, a lack of information (e.g., type, location, and length) for the additives. However, the masks were then generated by two ways: statistical masking and threshold-based masking. Firstly, we investigated the statistical positions and sizes of the two additives (the alphabet characters and scale bars) on the four corners in the 1100 given images. In Figure 3a, we observed that the averaged positions of the alphabet characters in the top-left and top-right were (23, 233) and (225, 228), respectively, whereas the averaged positions of the scale bars in the bottom-left and bottom-right were (41, 20) and (212, 26), respectively. In order to measure the average size of the alphabet characters and the scale bars, we estimated a cumulative distribution function (CDF) with respect to the ratio of their sizes and their probabilities, as shown in Figure 3b. Although all the additives in the four regions were only the characters and the scale bars, the different average sizes of the additives in the four regions were observed. The characters appeared at the top, but the scale bars appeared at the bottom. Since the sizes of characters and scale bars were different, the average sizes of the additives between the top and bottom were different. Furthermore, the average sizes of the additives among the four regions can be different because of the personal preference of the authors. For instance, one used the additive with parenthesis as like \"(b)\" (see Figure 1b), and the other one used the additive without parenthesis as like \"b\" (see Figure 1c). Then, we observed that the maximum sizes of the 80% majority additives in the top-left, top-right, bottom-left, and bottom-right were 36 (pixels) \(\times\) 36 (pixels), 41 (pixels) \(\times\) 41 (pixels), 49 (pixels) \(\times\) 49 (pixels), and 51 (pixels) \(\times\) 51 (pixels), respectively. Therefore, we decided on the statistical mask that covers the four squares at (23, 233), (225, 228), (41, 20), and (212, 26), respectively. Next, we designed an algorithm that generates a threshold-based mask. When we investigated 1100 images, we checked that the microscopic images mostly took on extremely white-colored additives against a dark background. Therefore, we defined the threshold-based mask to include white pixels of a higher threshold as the additives, as well as black pixels of a lower threshold for the others. For distinguishing the pixels of the additives, as well as the others, we used a threshold value of 200 in gray scale. In Figure 4a, this can be black on the additive, but white around the additive. When we used only a threshold value in order to distinguish the pixels of these, the additive filled the black pixels remains as a non-removable object, as can be seen in Figure 4b. Therefore, we added the function that the pixels close to the white pixels possess a white color in the algorithm. We observed that the regions of the additives can possess white, and the regions of the others can possess black with respect to the result of the algorithm (see Figure 4c).