# Focused and TSOM Images Two-Input Deep-Learning Method for Through-Focus Scanning Measuring
## 1 Introduction  
Metrology and defect inspection are significant steps in the fabrication of semiconductor integrated circuits and nanoscale devices industry. Several methods are currently used for nanoscale measurement, such as scanning electron microscope (SEM) [1, 2] based on electronic excitation scattering measurement and atomic force microscope (AFM) [3] based on probe measurement. Although these above methods show high measuring accuracy in measurement, they are difficult to be applied to online real-time detection due to their low measuring efficiency, high cost and destructions. Additionally, methods based on optical scattering are unable to achieve accurate nanoscale measurement [4] due to the limitation of diffraction limit of optical imaging system though they are fast and efficient. Through-focus scanning optical microscopy (TSOM) is a novel, fast and non-destructive micro/nano-scale measurement method based on computational imaging [5, 6, 7, 8, 9]. Different from traditional optical measurement, TSOM is based on the view of \"what you get is not what you see\". TSOM captures a series of images consisting of a focused image and a lot of defocused images of the imaging target by scanning along the optical axis. Captured images are stacked according to spatial position to form an image cube (TSOM cube), and the middle sectional view of the TSOM cube is taken to generate the TSOM image. The features of the TSOM image are sensitive to the nanoscale size change and the nanoscale detect of the measuring target, for which the parameters of the measuring target can be retrieved from the TSOM image. By using TSOM, the diffraction limit of optical imagingsystem can be avoided so that the ordinary wide field optical microscope can achieve the dimensional measurement in nanoscale. TSOM has been applied in mask-defect detection [10], nanoparticle shape evaluation [11], nanoscale dimensional measurements of three-dimensional structures [12] and overlay analysis [13; 14]. Three kinds of TSOM methods are used to measure the nanoscale parameters of the target: the library-matching TSOM method [15; 16], the machine-learning TSOM method [17] and the deep-learning TSOM method [18; 19]. Library-matching TSOM method is a classical measurement method of TSOM [15; 16]. In this method, the experimental TSOM image of the target is matched with the simulated TSOM image library established by simulation algorithm. Then, the parameters of the target can be recorded from the best match. Library-matching TSOM method need much time for establishing the simulated TSOM image library, and its measuring accuracy is limited by the fineness of the established library. [PERSON] et al. proposed the machine-learning TSOM method according to TSOM image features [17]. In their method, three kinds of image feature extraction algorithms are used to extract TSOM image features: the gray level co-occurrence matrix (GLCM), the local binary pattern (LBP) and the gradient direction histogram (HOG). Then, the extracted TSOM image features are input into different machine-learning models in different combinations for training and measuring. Finally, they get more accurate measurement results than the library-matching method. However, due to the limitation of feature extraction algorithms, the machine-learning TSOM method is still not accurate enough. Deep learning has developed rapidly in recent years and has been widely used in various fields, such as image recognition [20; 21], biomedical sciences [22; 23] and computational imaging [24]. The earliest deep-learning TSOM method proposed by [PERSON] et al. is based on the idea of image classification [18]. With setting different parameter sizes as the classifying categories, this method uses a single-column Convolutional Neural Network (CNN) to calculate the probability of the TSOM image being assigned to each category. Finally, they obtain the measurement results from weighted average of all categories. The method is simple and convenient, but the measurement range is small and the accuracy is limited.