Patent ID: 12229936

DETAILED DESCRIPTION OF VARIOUS EXAMPLE EMBODIMENTS

FIG.1is an illustrative block diagram illustrating a super resolution scanning electron microscope (SEM) image implementing system according to some example embodiments.

Referring toFIG.1, a super resolution SEM image implementing system10according to some example embodiments includes a central processing unit (CPU)200and a graphics processing unit (GPU)300, a neural processing unit (NPU)400, and a super resolution SEM image implementing device100.

The central processing unit200may control an overall operation of the super resolution SEM image implementing system10; for example, operations of other components constituting the super resolution SEM image implementing system10.

The super resolution SEM image implementing system10may include an accelerator, which is a dedicated circuit for a high-speed data operation such as an artificial intelligence (AI) data operation. The accelerator may include, for example, the graphics processing unit300, the neural network processing unit400, and/or a data processing unit (DPU) (not illustrated).

The super resolution SEM image implementing system10includes the super resolution SEM image implementing device100that implements a low resolution SEM image as a super resolution SEM image.

When an SEM image obtained through an SEM apparatus is obtained as a super resolution image and evaluation of and/or an adjustment of a process of a semiconductor device is performed based on the super resolution image, a long turn-around time (TAT) may occur. Alternatively, when the evaluation of and/or an adjustment of the process of the semiconductor device is performed using a low resolution SEM image in order to decrease a TAT, accuracy of the evaluation may be decreased. The SEM image may be obtained during the process of semiconductor fabrication, e.g. as part of a critical-dimension (CD) measurement and/or adjustment process.

Accordingly, by implementing a super resolution SEM image using deep learning based on a low resolution SEM image obtained through the SEM apparatus by the super resolution SEM image implementing device100, the TAT may be decreased, and additionally the accuracy of the evaluation of the process of the semiconductor device may be increased.

Hereinafter, an operation and a method of implementing a super resolution SEM image using deep learning based on a low resolution SEM image obtained through the SEM apparatus by the super resolution SEM image implementing device100will be described.

FIG.2is an illustrative block diagram illustrating a super resolution SEM image implementing device according to some example embodiments.

The super resolution SEM image implementing device100includes a cropping unit110, a buffer112, an upscaling unit120, a noise canceling unit130, and a merging unit140. Each of the components described inFIG.2are illustrated as different elements; however, example embodiments are not limited thereto. Some of the functions described with reference to one or more elements inFIG.2may be performed by one or more other elements described inFIG.2. The functions of the elements illustrated inFIG.2may be performed by a processor, e.g. a processor configured to execute machine-readable instructions that, when executed, cause the processor to perform various functions.

The cropping unit110may perform cropping on the low resolution SEM image. An operation of the cropping unit110will be described with reference toFIG.3.

FIG.3is an illustrative view for describing a cropping operation of a cropping unit.

Referring toFIGS.2and3, the cropping unit110receives a first low resolution SEM image LR_S_I1to an N-th low resolution SEM image LR_S_I N obtained through an SEM apparatus, where N is a natural number greater than 1. Hereinafter, an operation and an implementing method of the super resolution SEM image implementing device according to some example embodiments will be described with respect to the first low resolution SEM image LR_S_I1, and this description may also be applied to other low resolution SEM images (e.g., the N-th low resolution SEM image LR_S_IN N).

As described herein, a low resolution SEM image may refer to an SEM image having resolution lower than/less than that of a super resolution SEM image.

The first low resolution SEM image LR_S_I1received by the cropping unit110may be, for example, an image after an etching process for forming fins is performed with respect to a FinFET semiconductor element. Alternatively or additionally, the first low resolution SEM image LR_S_I1received by the cropping unit110may be an image after a trench etching process for forming shallow trench insulators (STIs) is performed with respect to a dynamic random access memory (DRAM) semiconductor element. Alternatively or additionally, the first low resolution SEM image LR_S_I1received by the cropping unit110may be an image after an etching process for forming gate lines is performed with respect to a NAND semiconductor element. Alternatively or additionally, the first low resolution SEM image LR_S_I1received by the cropping unit110may be an image after an etching process for forming channel holes is performed with respect to a VNAND semiconductor element. The first low resolution SEM image LR_S_I1received by the cropping unit110is not limited to the above-described examples, and may be an image in a processing process for various semiconductor devices.

The cropping unit110may crop the first low resolution SEM image LR_S_I1. For example, the cropping unit110may crop the first low resolution SEM image LR_S_I1to generate a plurality of cropped images (first cropped image C_I1to sixteenth cropped image C_I16). For reference, the number of cropped images generated by cropping the first low resolution SEM image LR_S_I1by the cropping unit110is not limited to 16, and may be greater than 16 or less than 16.

The first cropped image C_I1generated by cropping the first low resolution SEM image LR_S_I1by the cropping unit110may be generated, for example, as illustrated inFIG.4.

FIG.4is an illustrative view for describing a cropped image.

Referring toFIGS.2to4, the first cropped image C_I1generated by cropping the first low resolution SEM image LR_S_I1by the cropping unit110is illustrated. Hereinafter, a description for the first cropped image C_I1may also be applied to other cropped images (second cropped image C_I2to sixteenth cropped image C_I16).

The cropping unit110may store a first position, which is position information on the first cropped image C_I1, in the buffer112. The buffer112may be implemented as a static random access memory (SRAM) buffer and/or a dynamic random access memory (DRAM) buffer; however, example embodiments are not limited thereto.

For example, the cropping unit110may store a first position including a horizontal starting point HSP, a vertical starting point VSP, a horizontal range HR, and a vertical range VR for cropping the first cropped image C_I1, in the buffer112. Although the horizontal starting point HSP and the vertical starting point VSP are illustrated as being in the bottom-left of the first cropped image C_I1inFIG.4, example embodiments are not limited thereto, and the starting points may be at other positions in the first cropped image C_I1.

The cropping unit110may crop the first low resolution SEM image LR_S_I1so that there are no portions overlapping each other between the plurality of cropped images (first cropped image C_I1to sixteenth cropped image C_I16) generated by cropping the first low resolution SEM image LR_S_I1.

For example, the plurality of cropped images (first cropped image C_I1to sixteenth cropped image C_I16) may be cropped without areas overlapping each other within the first low resolution SEM image LR_S_I1.

Alternatively, the cropping unit110may crop the first low resolution SEM image LR_S_I1so that there are portions overlapping each other between the plurality of cropped images (first cropped image C_I1to sixteenth cropped image C_I16) generated by cropping the first low resolution SEM image LR_S_I1.

A detailed description therefor will be described below with reference toFIGS.5and6.

FIG.5is an illustrative view for describing a plurality of cropped images adjacent to each other.FIG.6is an illustrative view for describing cropped images generated through another cropping operation of the cropping unit.

Referring toFIGS.2,3,5, and6, the first cropped image C_I1and the second cropped image C_I2may be in contact with each other on the basis of/along a boundary line B_L.

In this case, the cropping unit110may designate a first horizontal range HR1from a first horizontal starting point HSP1where cropping starts, as a cropping area, and perform the cropping. In addition, the cropping unit110may designate a second horizontal range HR2from a second horizontal starting point HSP2where cropping starts, as another cropping area, and perform the cropping.

For example, the cropping unit110may generate a first-first cropped image C_I1′ cropped so as to include an area permeating from the boundary line B_L into the second cropped image C_I2by a second length L2(area patterned as diagonal lines from the upper left side to the lower right side) with respect to the first cropped image C_I1.

In addition, the cropping unit110may generate a second-first cropped image C_I2′ cropped so as to include an area permeating from the boundary line B_L into the first cropped image C_I1by a first length L1(area patterned as diagonal lines from the upper right side to the lower left side) with respect to the second cropped image C_I2.

For reference, the first length L1and the second length L2may be the same as, or different from each other.

Referring toFIG.2again, the upscaling unit120may upscale the plurality of cropped images (first cropped image C_I1to sixteenth cropped image C_I16) generated through the cropping unit110to generate a plurality of upscaled images.

An operation of the upscaling unit120will be described in detail with reference toFIGS.7and8.

Hereinafter, it will be described by way of example to upscale the first cropped image C_I1to generate a first upscaled image.

FIG.7is an illustrative view for describing an operation of an upscaling unit.

Referring toFIGS.2and7, the upscaling unit120may first perform residual learning on the first cropped image C_I1. In more detail, the upscaling unit120may generate a plurality of residual blocks for stability of learning with respect to the first cropped image C_I1.

For example, the upscaling unit120may generate16residual blocks.

Thereafter, the upscaling unit120connects the front and the rear of the16residual blocks to each other using a skip connection for the16residual blocks to optimize filter parameters.

Thereafter, the upscaling unit120may generate a first upscaled image U_I1through a deconvolution and/or upsampling operation. For example, an upsampling operation for an upscaling multiple of 2 may be performed. Alternatively, for example, an upsampling operation for an upscaling multiple of 4 may be performed.

In this case, the upscaling unit120may generate the first upscaled image U_I1using a mean square error (MSE) loss function and/or a mean absolute error (MAE) loss function between the first upscaled image U_I1and a first ground truth image GT_I1, generated through a deep learning-based network. For example, the upscaling unit120may generate the first upscaled image U_I1using an image loss. The first ground truth image GT_I1may be a high resolution image obtained from an SEM, corresponding to the first cropped image C_I1.

Alternatively or additionally, the upscaling unit120may generate the first upscaled image U_I1using perceptual loss between the first upscaled image U_I1and a first ground truth image GT_I1, generated using a convolutional neural network (CNN) such as VGGNet and/or Resnet, which are various deep learning-based image discriminator networks. The first ground truth image GT_I1may be a high resolution image obtained from an SEM, corresponding to the first cropped image C_I1.

A discriminator network may continuously compare the first upscaled image U_I1and the first ground truth image GT_I1with each other to increase resolution of the first upscaled image U_I1. The first ground truth image GT_I1may be a high resolution image obtained from an SEM, corresponding to the first cropped image C_I1.

FIG.8is an illustrative view for describing an upscaled image generated through an upscaling operation of the upscaling unit.

Referring toFIGS.2,4, and8, a first upscaled image U_I1may be generated from the first cropped image C_I1through the operation of the upscaling unit120described above.

Referring toFIG.2again, the noise canceling unit130may cancel noise from the first upscaled image U_I1to generate a first noise canceled image.

An operation of the noise canceling unit130will be described with reference toFIG.9.

FIG.9is an illustrative view for describing an operation of a noise canceling unit.

Referring toFIG.9, the noise canceling unit130includes an encoder132performing convolution on the first upscaled image U_I1and a decoder134performing deconvolution on the first upscaled image U_I1on which the convolution is performed.

When a neural network model is learned, the deeper the layer of the neural network model, the better the learning result, but if the layer becomes too deep and/or the number of nodes is excessively increased, information loss may occur and/or a problem that weights are updated in an erroneous direction may occur.

Accordingly, in order to use information of the previous layer, a selective skip connection that connects the information of the previous layer may be applied.

For example, the noise canceling unit130may suppress loss of structural information for the first upscaled image U_I1received as an input of the noise canceling unit130, and at the same time, cancel the noise from the first upscaled image U_I1, through the selective skip connection between the encoder132and the decoder134.

FIG.10is an illustrative view for describing a noise canceled image generated through a noise canceling operation of the noise canceling unit.

Referring toFIGS.2,9, and10, a first noise canceled image N_C_I1in which the noise is canceled from the first upscaled image U_I1may be generated through the noise canceling unit130.

Referring toFIG.2again, the merging unit140may merge a plurality of noise canceled images (e.g., a plurality of noise canceled image including the first noise canceled image N_C_I1) generated through the first low resolution SEM image LR_S_I1with each other.

In this case, the merging unit140may merge the plurality of noise canceled images (e.g., the plurality of noise canceled image including the first noise canceled image N_C_I1) with each other based on position information of each of the cropped images stored in the buffer112.

As a result, one first super resolution SEM image may be generated.

FIG.11is an illustrative view for describing a super resolution SEM image generated through a merging unit.

Referring toFIGS.2and11again, the merging unit140may merge the plurality of noise canceled images (e.g., the plurality of noise canceled image including the first noise canceled image N_C_I1) generated through the first low resolution SEM image LR_S_I1with each other to generate a first super resolution SEM image SR_S_I.

In this case, when the cropping unit110of the super resolution SEM image implementing device100according to some example embodiments performs the cropping in such a way that the overlapping area is included as in the first-first cropped image C_I1′ and the second-first cropped image C_I2′, the merging unit140may perform merging by removing the overlapping area with respect to at least one cropped image.

FIG.12is an illustrative flowchart for describing a super resolution SEM image implementing method according to some example embodiments.

Referring toFIGS.2and12, a plurality of cropped images are generated by cropping a low resolution scanning electron microscope (SEM) image through the cropping unit110(S100).

Then, a plurality of upscaled images are generated by performing upscaling on each of the plurality of cropped images through the upscaling unit120(S110).

Then, a plurality of noise canceled images are generated by canceling noise from each of the plurality of upscaled images through the noise canceling unit130(S120).

Then, a plurality of noise canceled images are merged with each other (S130), and one super resolution SEM image is generated (S140).

Optionally, a semiconductor device may be fabricated based on the super resolution SEM image (S150).

FIG.13is an illustrative graph illustrating a critical dimension (CD) distribution of the super resolution SEM image generated through the super resolution SEM image implementing method according to some example embodiments.

Referring toFIG.13, a critical dimension (CD) extracted from the super resolution (SR) SEM images generated from the low resolution SEM images through the super resolution SEM image implementing device100according to some example embodiments is shown on a vertical axis.

In addition, a CD extracted from high resolution SEM images obtained from an SEM device is shown on a horizontal axis.

It can be seen throughFIG.13that the super resolution (SR) SEM images generated from the low resolution SEM images through the super resolution SEM image implementing device100according to some example embodiments are very close to the high resolution SEM images obtained from the SEM device.

FIG.14is an illustrative block diagram illustrating another super resolution SEM image implementing system according to some example embodiments.

Referring toFIG.14, another super resolution SEM image implementing system12according to some example embodiments is different from the super resolution SEM image implementing system10according to some example embodiments illustrated inFIG.1in that it further includes an anomaly detection unit600.

FIG.15is an illustrative block diagram for describing an anomaly detection unit according to some example embodiments.

Referring toFIG.15, the anomaly detection unit600includes a training unit610, a deep learning application unit620, and an anomaly detector630.

The training unit610will be described with reference toFIG.16.

FIG.16is an illustrative view for describing an operation of a training unit of the anomaly detection unit.

Referring toFIGS.15and16, the training unit610performs image translation learning training through comparison with a layout image based on the super resolution SEM image SR_S_I received through the super resolution SEM image implementing device100.

Referring toFIG.15again, SEM image translation is performed by the deep learning application unit620through a deep learning model on which learning training is performed through the layout image from the training unit610.

An operation of the deep learning application unit620will be described with reference toFIG.17.

FIG.17is an illustrative view for describing an operation of the deep learning application unit of the anomaly detection unit.

Referring toFIGS.15and17, the deep learning application unit620performs SEM image translation based on an image translation deep learning model on which learning training is performed from the training unit610using machine learning such as at least one of an artificial neural network (ANN) or using deep learning such as a deep neural network (DNN), a convolutional neural network (CNN), or a generative adversarial network.

Referring toFIG.15again, the anomaly detector630detects abnormal images through an SEM image generated by the deep learning application unit620. An operation of the anomaly detector630will be described with reference toFIG.18.

FIG.18is an illustrative block diagram for describing an operation of the anomaly detector of the anomaly detection unit.

Referring toFIGS.15and18, the anomaly detector630detects abnormal images through an SEM image generated by the deep learning application unit620.

For example, the anomaly detector630may detect abnormal images in which defects are generated.

FIG.19is an illustrative flowchart for describing an operation of the anomaly detection unit according to some example embodiments.

Referring toFIGS.15and19, in the anomaly detection unit600according to some example embodiments, the training unit610performs training through a layout image and translation learning based on the super resolution SEM image SR_S_I received through the super resolution SEM image implementing device100(S200).

Then, the deep learning application unit620performs SEM image translation through a layout image on which learning training is performed from the training unit610and a model on which learning training is performed based on an SEM image (S210).

Then, the anomaly detector630detects abnormal images through an SEM image generated by the deep learning application unit620(S220).

According to some example embodiments, by generating images with a low-resolution CD-SEM where the images are cropped and upscaled, a turn-around time (TAT) of imaging may be low. Furthermore by creating a super-resolution based on the cropped and upscaled images, a quality of the images may be improved, and a quality of semiconductor devices that are fabricated may be improved.

Any of the elements and/or functional blocks disclosed above may include or be implemented in processing circuitry such as hardware including logic circuits; a hardware/software combination such as a processor executing software; or a combination thereof. For example, the processing circuitry more specifically may include, but is not limited to, a central processing unit (CPU), an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a System-on-Chip (SoC), a programmable logic unit, a microprocessor, application-specific integrated circuit (ASIC), etc. The processing circuitry may include electrical components such as at least one of transistors, resistors, capacitors, etc. The processing circuitry may include electrical components such as logic gates including at least one of AND gates, OR gates, NAND gates, NOT gates, etc.

Although various example embodiments have been described above with reference to the accompanying drawings, it will be understood by those of ordinary skill in the art that various embodiments are not limited thereto and may be implemented in many different forms without departing from the technical idea or essential features thereof. Therefore, it should be understood that example embodiments set forth herein are merely examples in all respects and not restrictive.