Detecting outliers and anomalies for OCD metrology machine learning

A system and methods for OCD metrology are provided including receiving training data for training an OCD machine learning (ML) model, including multiple pairs of corresponding sets of scatterometric data and reference parameters. For each of the pairs, one or more corresponding outlier metrics are by calculated and corresponding outlier thresholds are applied whether a given pair is an outlier pair. The OCD MIL model is then trained with the training data less the outlier pairs.

FIELD OF THE INVENTION

The present invention relates generally to the field of optical inspection of integrated circuit wafer patterns, and in particular to algorithms for measurement of wafer pattern parameters.

BACKGROUND

Integrated circuits (ICs) are produced on semiconductor wafers through multiple steps of depositing, altering, and removing thin layers that build up into stacked structures on the wafers. These stacked structures, or “stacks,” are typically formed in repetitive patterns that, like diffraction gratings, have optical properties. Modern metrology methods for measuring critical dimensions (CDs) and material properties of these patterns exploit these optical properties. Hereinbelow, CDs and material properties are also referred to as “pattern parameters,” or simply as “parameters.” These parameters may include the height, width, and pitch of stacks. As described by Dixit, et al., in “Sensitivity analysis and line edge roughness determination of 28-nm pitch silicon fins using Mueller matrix spectroscopic ellipsometry-based optical critical dimension metrology,” J. Micro/Nanolith. MEMS MOEMS. 14(3), 031208 (2015), incorporated herein by reference, pattern parameters may also include: side wall angle (SWA), spacer widths, spacer pull-down, epitaxial proximity, footing/undercut, over-fill/under-fill of 2-dimentional (HKMG), 3-dimentional profile (FinFETs) and line edge roughness (LER).

Optical critical dimension (OCD) metrology employs methods of scatterometry to measure scatterometric data, that is, reflected light radiation that is indicative of optical properties of patterns. A measurement set of scatterometric data (which may also be referred to as a scatterometric signature) may include data points of reflected irradiance versus an incident angle of radiation (which may be zeroth-order measurements). Alternatively, or additionally, scatterometric data may include spectrograms that are measures of reflected radiation intensity over a range of wavelengths or frequencies. Additional types of scatterometric data known in the art may also be applied in OCD metrology.

U.S. Pat. No. 6,476,920 to Scheiner and Machavariani, “Method and apparatus for measurements of patterned structures,” incorporated herein by reference, describes development of an “optical model” (also referred to as “physical model”), which is a function (i.e., a set of algorithms) defining a relation between reflected radiation and the physical structure of a wafer. That is, optical models are theoretical models, based on physical laws of optics, determining how light is reflected from patterns with known parameters. Such optical models can be applied to generate, from a set of known pattern parameters, an estimate of scatterometry data that would be measured during spectrographic testing. Optical models can also be designed to perform the converse (or “inverse”) function, of predicting (i.e., “estimating”) pattern parameters based on measured scatterometry data. Optical models used in practice are typically tuned to conform with measured data.

Optical models are commonly applied for OCD metrology during IC production to measure, based on scatterometric measurements, whether wafer patterns are being fabricated with correct parameters. Each pattern of a given wafer may be measured to determine how much the parameters of each patterns varies from a design specification or from a mean value.

As an alternative to optical modeling, machine learning (ML) techniques may be applied to predict pattern parameters based on scatterometry data. For example, as described in PCT patent application WO 2019/239380 to Rothstein, et al., incorporated herein by reference, a machine learning model may be trained to identify correspondences between measured scatterometry data and reference parameters measured by methods described below. After an ML model is trained to predict parameters from scatterometry data, it may then be applied to make such parameter predictions during IC production.

Exemplary scatterometric tools for measuring (acquiring) scatterometry data (e.g., spectrograms) may include spectral ellipsometers (SE), spectral reflectometers (SR), polarized spectral reflectometers, as well as other optical critical dimension (OCD) metrology tools. Such tools are incorporated into OCD metrology systems currently available. One such OCD metrology system is the NOVA T600® Advanced OCD Metrology tool, commercially available from Nova Measuring Instruments Ltd. of Rehovot, Israel, which takes measurements of pattern parameters that may be at designated test sites or “in-die.” Additional methods for measuring critical dimensions (CDs) include interferometry, X-ray Raman spectrometry (XRS), X-ray diffraction (XRD), and pump-probe tools, among others. Some examples of such tools are disclosed in U.S. Pat. Nos. 10,161,885, 10,054,423, 9,184,102, and 10,119,925, and in international pending patent application publication WO2018/211505, all assigned to the Applicant and incorporated herein by reference in their entirety.

High accuracy methods of measuring pattern parameters that do not rely on the optical models described above include wafer measurements with equipment such as CD scanning electron microscopes (CD-SEMs), atomic force microscopes (AFMs), cross-section tunneling electron microscopes (TEMs), or X-ray metrology tools.

A shortcoming of ML modeling is the reliance on measured training data that is assumed to accurately reflect the characteristics of data that will be encountered in subsequent production. Problems with OCD equipment or system calibration may cause anomalies and measurement errors that are non-repeatable and not readily identified. In such cases, the training data will lead to training of ML models with features that are not indicative of real data characteristics.

Embodiments of the present invention as disclosed hereinbelow help to overcome these shortcomings.

SUMMARY

Embodiments of the present invention provide a system and methods for OCD metrology, providing steps that include receiving training data, for training an OCD machine learning (ML) model, including multiple pairs of corresponding input and output training data. The input data of each pair includes a set of scatterometric data measured from a wafer pattern and the output data of each pair includes a reference parameter measured from the same wafer pattern. For each of the pairs, one or more corresponding outlier metrics may be calculated, by calculating one or more of: 1) a difference between the reference parameter and a corresponding predicted parameter that is predicted from the corresponding set of scatterometric data; 2) a difference between a merit function of the set of scatterometric data and a deviation from a distribution of merit functions; and 3) a difference between the reference parameter and a mean or a median of reference parameters. For each of the one or more outlier metrics of each of the pairs, a corresponding outlier threshold may be applied to the outlier metric to determine whether the pair is an outlier pair, thereby determining, from among the training data, one or more outlier pairs. The OCD ML model may then be trained from the training data less the one or more outlier pairs. Training teaches the OCD ML model to predict, from new scatterometric data, respective new predicted parameters.

In some embodiments, calculating the one or more outlier metrics includes calculating a Cook's distance, which includes calculating a squared difference between the reference parameter and the corresponding predicted parameter, and multiplying the squared difference by a leverage factor. The leverage factor indicates an influence of the squared difference on the Cook's distance. After determining from the Cook's distance that a given pair is an outlier pair, the given outlier pair may be removed from the training data, after which a new Cook's distance may be calculated.

The distribution of merit functions may be a distribution of merit functions of all sets of scatterometric data in the training data, and the merit function for a given set of scatterometric data Simay be calculated by one or more of a Euclidian distance, a Minikowski distance, a Chebyshev distance, and a Mahalonbis distance between Siand all sets of scatterometric data in the training data. The outlier threshold may be a skewed box plot threshold, and an upper threshold W3 of the skewed box plot threshold may be a function of the form W3=a (IQR)eb MC, where IQR is the inter-quartile range of the distribution, a and b are constants in the range of 1 to 5, and MC is a medcouple function of the distribution.

In further embodiments, the one or more outlier metrics may be calculated as a difference between the reference parameter and a mean of all reference parameters in the training data set, and the outlier threshold may be a box plot threshold.

The reference parameter may be one of multiple reference parameters received in the training data and corresponding to a given set of scatterometric data. Each of the multiple reference parameters and the corresponding set of scatterometric data may be processed as a pair of corresponding input and output training data. In some embodiments, determining that a pair of training data is an outlier pair may include determining that other pairs of training data including the set of scatterometric data are also outlier pairs.

In some embodiments, the predicted parameters are predicted by applying an optical model or a previously generated ML model to the corresponding sets of scatterometric data. The reference parameters may be measured with high accuracy metrology by one or more of a CD scanning electron microscope (CD-SEM), an atomic force microscope (AFM), a cross-section tunneling electron microscope (TEM), or an X-ray metrology tool. The training data may be measured from multiple wafer patterns located on one or more wafers.

In further embodiments, there are also provided one or more non-transitory, machine-accessible storage media having instructions stored thereon, the instructions, when executed by a machine, causing the machine to implement the processes described above.

DETAILED DESCRIPTION

Embodiments of the present invention provide systems and methods for generating machine learning (ML) models for optical critical dimension (OCD) monitoring, with outlier pre-processing of training data.

FIG.1is a schematic diagram of a system10for generating a machine learning model for OCD metrology, with outlier pre-processing, in accordance with an embodiment of the present invention.

The system10may operate within a production line (not shown) for production and monitoring of wafers12. As indicated, wafers12include patterns14. These patterns have parameters, such as height (“h”), width (“w”), and pitch (“p”), as indicated in the pattern enlargement14a, as well as other parameters described in the Background above. Typically, wafers have multiple regions, or segments, or “dies” that are designed to have the same patterns (i.e., the same pattern design is used to manufacture all of the patterns), though fabrication variability may cause slight changes in these patterns. Machine learning models described above are typically designed to predict pattern parameters from a wide range of pattern geometries. For each pattern, a set of multiple parameters may be measured, referred to hereinbelow as “reference parameters.” It is to be understood that where methods below are described with respect to a single reference parameter being measured from a given pattern, the method may be extended for the case of a set of multiple reference parameters being measured for the given pattern.

The system10includes a light source20, which generates a beam of light22of a predetermined wavelength range. The beam of light22is reflected from the wafer patterns14(indicated as reflected, or “scattered,” light24) towards a spectrophotometric detector26. In some configurations, the light source and spectrophotometric detector are included in an OCD metrology system30(e.g., ellipsometer or a spectrophotometer). The construction and operation of the metrology system30may be of any known kind, for example, such as disclosed in U.S. Pat. Nos. 5,517,312, 6,657,736, and 7,169,015, and in international pending patent application publication WO2018/211505, all assigned to the Applicant and incorporated herein by reference in their entirety. Typically the metrology system30includes additional components, not shown, such as light directing optics, which may include a beam deflector having an objective lens, a beam splitter and a mirror. Additional components of such systems may include imaging lenses, polarizing lenses, variable aperture stops, and motors. Operation of such elements is typically automated by computer controllers, which may include I/O devices and which may also be configured to perform data processing tasks, such as generating scatterometry data32.

The scatterometry data32generated by the metrology system30typically includes various types of graphical data34, which may be represented in vector form (e.g., a spectrogram, whose data points are measures of reflected light intensity at different light wavelengths, or a mapping of reflected irradiance vs. incident angle). As described above, variations between sets of scatterometric data are indicative of differing pattern parameters. In typical OCD metrology, the range of light that is measured may cover the visible light spectrum and may also include wavelengths in ultraviolet and infrared regions. A typical spectrogram output for OCD metrology may have 245 data points covering a wavelength range of 200 to 970 nm.

In embodiments of the present invention, sets of scatterometric data32from respective wafer patterns14, and sets of corresponding reference parameters34from the same respective wafer patterns, are acquired as pairs of corresponding input and output training data for machine learning (ML). The sets of scatterometric data32are typically the input data, each set being paired with a set of corresponding reference parameters34that form the output data.

Before ML modeling is performed, a pre-processing outlier filter36may be employed to remove pairs of training data that have anomalous characteristics. Multiple methods of pre-processing outlier filtering are described hereinbelow. In some embodiments, multiple parallel methods are used to determine if pairs of training data are outlier pairs, such that a pair may be determined to be an outlier if any one or more of the methods indicates that it is an outlier.

After pre-processing, a computer system including ML tools known in the art, referred to herein as an ML modeling system40, may be configured for training an ML model for OCD metrology. Input data typically includes the sets of scatterometric data34, as described above, corresponding to the reference parameters44, which may be used as target output for ML training. The reference parameters may be acquired from patterns of one or more wafers by high accuracy means known in the art, such as described above (e.g., CD-SEM, AFM, TEM, X-ray metrology, or high accuracy OCD spectroscopy relying on optical modeling). After training, the ML model is used to predict pattern parameters based on sets of scatterometric data, which may be applied, for example, in the monitoring of wafer production.

The ML modeling system40and the pre-processing outlier filter may operate independently of the metrology system30or the systems may be integrated into a single computing platform.

FIG.2is a flow diagram depicting a process200for generating a machine learning model for OCD metrology, with outlier pre-processing, in accordance with an embodiment of the present invention; Process200may be implemented by the system10for OCD metrology, described above, and in particular by the pre-processing outlier filter36and by the ML modeling system40.

A first step214includes receiving training data that will be used for subsequent training of a machine learning (ML) model for OCD metrology. For each wafer pattern a pair of input and output training data is received, as described above, the input data being a set of scatterometric data, and the output data being one or more reference parameters. Outlier methods described below refer to a single reference parameter being acquired for each pair of training data. If additional reference parameters are acquired for a given set of scatterometric data (i.e., with respect to a given wafer patter), then the methods described below may be implemented by considering each of the multiple reference parameters, together with its corresponding set of scatterometric data, to be a distinct pair of training data. (However, in some embodiments, if one pair including a given set of scatterometric data is determined to be an outlier pair, all pairs associated with that set of scatterometric data may be determined to be outlier pairs. Alternatively, such pairs may be treated separately with respect to outlier treatment.)

After data acquisition at step214, three types of outlier filters may be applied to the training data pairs. Each type of filter determines a respective outlier metric with respect to the training data pair. Outlier metrics for the respective methods are then compared with respective outlier thresholds. In some embodiments, one or more of the outlier filters are operated in parallel, and a training pair is determined to be an outlier pair if any of the outlier filters determines it is an outlier pair (i.e., even if the pair is not an outlier according to some of the outlier filters.)

A first outlier filter220determines outliers based on comparing reference parameters with predicted parameters. At a first step222of the filter, a prediction model, such as an optical model or ML model, is applied to each set of acquired scatterometric data to generate a predicted parameter. (If each set of acquired scatterometric data is associated with multiple reference parameters, then multiple predicted parameters may be generated at step222.)

At a second step224of the outlier filter, an outlier metric and subsequent outlier threshold are applied to determine if a difference between the corresponding reference and predicted parameters indicates that the corresponding pair of training data (i.e., the reference parameter and the corresponding set of scatterometric data) is an outlier pair.

One method of determining an outlier metric with respect to the difference between the reference and predicted parameters is to calculate the outlier metric as a Cook's distance. Cook's distance is typically used to measure outliers in a regression model. The formula for the Cook's distance, Di(i.e., the outlier metric), for a given data point i in a regression model is calculated as:

To apply the Cook's distance in the case of pairs of reference and predicted parameters, the terms of the Cook's distance may be defined as follows.

The squared error term ei2may be set to:

ei2=(pireference−pipredicted)2, that is, ei2is set as the squared difference between the reference and predicted parameters.

The term “A” is a normalization constant, which may be set, for example, to 1.

The term hiiis a leverage term, indicating the weight of the given data point (i.e., the weight of the parameter pair) in the calculation of the Cook's distance. The terms hiiare the terms of the diagonal matrix H, which may be set according to the equation:
H=S(STS)−1ST,

where S is a matrix of all the sets of scatterometric data (i.e., for n sets of data, each of k data points, S is an n×k matrix). A typical threshold for determining whether a data point is an outlier is Di>1. A range of thresholds between 1 and 3 may be used depending on how restrictive the outlier threshold should be for a given environment. Alternatively, Dimay be determined to be an outlier according to a percentile value using the F-distribution. A percentile of over 50 may be used as a threshold.

Alternatively or in addition to the Cook's distance, an outlier metric may be set with respect to the squared difference between the reference and predicted parameters, without the leverage function. For example, a distribution of squared differences of all pairs may be generated, and the difference of any pair's squared difference from a measure of the distribution may be used as an outlier metric. For example, a measure of the distribution may be in units of quartiles of the distribution, with a threshold set by a Tukey box plot threshold, as indicated in graph300ofFIG.3. Outliers may be determined as being outside an inter-quartile range (IQR), i.e., less than Q1 or greater than Q3. The box plot may include “whiskers,” W1 and W3, extending beyond the respective Q1 and Q3 quartiles. For a skewed distribution, as indicated in the figure, the primary area of interest for determining outliers would be when the squared difference is greater than W3. Examples of setting such a threshold are described below with respect to outlier filter230.

In parallel with filter220, filters230and240may be applied in outlier preprocessing, the latter filters being implemented without generation of the additional corresponding predicted parameters.

Filter230calculates outlier metrics with respect to the sets of scatterometric data. Each set of scatterometric data may be indicated as vector Si(e.g., a vector of irradiance values for a given range of wavelengths). Each element k of a set (i.e., a data point corresponding to wavelength) may be indicated as Ski. A mean value of all values of Skimay be indicated asSk. For each Si, the outlier metric may be calculated as a distance to a representative distribution or vector of the multiple sets of scatterometric data. Such a distance is also referred to herein as a “merit function.”

Various methods of calculating such a merit function are as follows.

1. Euclidian distance (measured as a difference between each element of a set Siand the average for all sets):
xi2=Σk|Ski−Sk|2.

where S is the matrix of all sets of scatterometric data, as described above.

The various examples of outlier metrics described above are “skewed” distributions, such as presented inFIG.3. A one-side box plot may be used to set a threshold, that is, a box plot where W1 is set to the minimum distance, i.e., zero, such that outliers are only detected above the threshold W3. The threshold W3 may be set as a function of a “medcouple” (MC), described by Brys, et al., in “A robust measure of skewness,”Journal of Computational and Graphical Statistics,13 (4): 996-1017 (2004). An exemplary form of such a setting may be of the form:

W3=a (IQR)eb MC, where IQR is the inter-quartile range of the distribution, a and b are constants in the range of 1 to 5, and MC is the medcouple function.

The medcouple function MC may be calculated as:

MC=modxi⩽Q2⩽xj⁢h⁡(xi,xj)
with Q2the sample median and where all xi≠xjthe kernel function h is given by

Filter240calculates outlier metrics with respect to the reference parameters of the training data. A straightforward outlier metric may be determine, at a step242, a normalized distance between a given reference parameter and a mean or median of all the reference parameters. As opposed to the skewed distribution of graph300ofFIG.3, such as distribution could be expected to be more symmetrical. A box plot threshold may then be set at a step244to determine pairs of training data that are outliers. The box plot may have “whiskers” extending beyond the Q1 and Q3 quartiles, might be set, for example, to 3×(IQR), in either direction. For reference parameters outside this range, the corresponding training data pair would be considered an outlier pair.

Following the parallel one or more steps of outlier filters described above, at a step250the outlier pairs would be removed from the training data before training of a ML model. Training would then be performed at a step260. An ML model, such as a neural network described below with respect toFIG.4, would be trained from the remaining training data. The resulting ML model, in production, would be used to predict parameters of new patterns from new scatterometric data.

FIG.4is a schematic diagram of an exemplary ML model, such as a neural network400that is trained following outlier filtering, in accordance with an embodiment of the present invention. During training, the pairs of training data not removed as outliers are provided in a supervised learning manner. Input data, i.e., the sets of scatterometric data Si, are fed to input layers420, followed by hidden layers430. The number of nodes of the output layer440is equal to the number corresponding reference parameters, i.e., if there are more than one set of “pairs” measured for each wafer pattern. These reference parameters, may be as shown inFIG.1, e.g., height, width, and pitch of a given wafer stack. In effect, training the ML model creates a mapping that may subsequently be applied to new sets of scatterometric data to corresponding new predicted parameters. Training is typically performed according to standard ML training methods, which may include, for example, L2 regularization. Typically, the loss function that a NN is trained to minimize is a mean squared error (MSE) loss function. Preferably, validation would follow training, with a validation set of data using data sets acquired from different wafers than those used to acquire the data of the training data set. Methods of outlier filtering described above may also be applied in pre-processing of the validation data.

It is to be understood that processing elements shown or described herein are preferably implemented by one or more computers in computer hardware and/or in computer software embodied in a non-transitory, computer-readable medium in accordance with conventional techniques, such as employing a computer processor, a memory, I/O devices, and a network interface, coupled via a computer bus or alternate connection arrangement.

Unless otherwise described, the terms “processor” and “device” are intended to include any processing device, such as, for example, one that includes a CPU (central processing unit) and/or other processing circuitry (e.g., GPUs), and may refer to more than one processing device. Various elements associated with a processing device may be shared by other processing devices.

The term “memory” as used herein is intended to include memory associated with a processor or CPU, such as, for example, RAM, ROM, a fixed memory device (e.g., hard drive), a removable memory device (e.g., diskette, tapes), flash memory, etc. Such memory may be considered a computer readable storage medium.

In addition, phrases “input/output devices” or “I/O devices” may include one or more input devices (e.g., keyboard, mouse, scanner, HUD, etc.) for entering data to the processing unit, and/or one or more output devices (e.g., speaker, display, printer, HUD, AR, VR, etc.) for presenting results associated with the processing unit.