Query: machine learning semiconductor CNN

Candidate papers:

abstract: Defect inspection (to detect, classify, measure and analyze) has long been a challenging task in semiconductor manufacturing (MFG) domain. This paper discusses a new Machine Learning (ML) approach which can be used to assist defect inspection in different MFG scenarios. Associated solutions have been developed and applied to the surface defect inspection for substrate components, used in our IC packaging MFG.During the past decade, the help from image based Automated Optical Inspection (AOI) equipment has significantly reduced manual efforts in substrate/PCB defect examination, but is still insufficient in defect classification automation. Recently, the adoption of ML and Convolution Neural Network (CNN) based Deep Learning technologies raised much hoping to advance the defect classification automation to a new level that acceptable for rigid MFG practices. Most of the published CNN models, however, tend to use large number of learning parameters (floating point variables) during computing in order to gain high image recognition accuracy. The parameters' massive blow up often causes heavy power gobbling computation. While applying CNN to our substrate defect datasets, each of which contains hundreds of thousands of high resolution images, collected across different MFG processes and produced products, a training run could take days or even weeks to finish. Also, more the learning variables for training are used, longer the inference time will be. The situations make these complex CNN models hard to meet our overnight retraining and in line inference time constraint. Therefore, to develop a more efficient ML method is strongly preferred in our MFG environment.In this paper, we develop a feature spanning ML approach which takes the feature of an image as a base. Through mathematical transformations the base is spanned to fit into an accumulated and gradually divided feature of classes space. During the feature spanning process, the initialization of parameters is tightly controlled, which means the redundancy blow up is carefully calculated to optimize the usage of computation resource.To demonstrate the advantage of our ML method, the public dataset CIFAR 10 is used for benchmarking. CIFAR 10 dataset contains sufficient diversity and is small enough to have quick observations on computing performance. Our results indicate that the parameters used by our method are only 5.6% of ResNet 101's. We also apply this new ML technology to the inspection of several substrate defect types. In particular, defects on solder mask are hard to be recognized because their pattern's colors are quite similar to the substrate background. The benchmarking shows very competitive results as compared with other CNNs' Further benchmarking shows our ML method holds high degree of shift invariance property, implying that our method can help to resist MFG condition changes during the defect inspection operation.Our ML approach requires much less learning variables, and thus can achieve very fast training and inference speed. While adopted in MFG production lines, it helps to reduce computing cost/energy and comply with Green Factory policy. Besides, our ML technology has high scalability, capable of performing heterogeneous learning on data combined from different aspects, such as image plus CAM design reference, Z height or time series signal waveform. For continuous development efforts, we are making the ML behavior in our AI for MFG applications more explainable, controllable, and be self adaptive to changes in MFG environment.
author_names: Yi-Chung Hsu

Ping-Yen Kuo

Wong-Shian Huang
title: A Novel Feature spanning Machine Learning Technology for Defect Inspection
venue: 2019 14th International Microsystems, Packaging, Assembly and Circuits Technology Conference (IMPACT)
year: 2019

abstract: Automated visual inspection in the semiconductor industry aims to detect and classify manufacturing defects utilizing modern image processing techniques. While an earliest possible detection of defect patterns allows quality control and automation of manufacturing chains, manufacturers benefit from an increased yield and reduced manufacturing costs. Since classical image processing systems are limited in their ability to detect novel defect patterns, and machine learning approaches often involve a tremendous amount of computational effort, this contribution introduces a novel deep neural network based hybrid approach. Unlike classical deep neural networks, a multi stage system allows the detection and classification of the finest structures in pixel size within high resolution imagery. Consisting of stacked hybrid convolutional neural networks (SH CNN) and inspired by current approaches of visual attention, the realized system draws the focus over the level of detail from its structures to more task relevant areas of interest. The results of our test environment show that the SH CNN outperforms current approaches of learning based automated visual inspection, whereas a distinction depending on the level of detail enables the elimination of defect patterns in earlier stages of the manufacturing process.
author_names: Tobias Schlosser

Frederik Beuth

Michael Friedrich

Danny Kowerko
title: A Novel Visual Fault Detection and Classification System for Semiconductor Manufacturing Using Stacked Hybrid Convolutional Neural Networks
venue: 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA)
year: 2019

abstract: An automatic defect classification (ADC) system identifies and classifies wafer surface defects using scanning electron microscope images. By classifying defects, manufacturers can determine whether the wafer can be repaired and proceed to the next fabrication step. Current ADC systems have high defect detection performance. However, the classification power is poor. In most work sites, defect classification is performed manually using the naked eye, which is unreliable. This paper proposes an ADC method based on deep learning that automatically classifies various types of wafer surface damage. In contrast to conventional ADC methods, which apply a series of image recognition and machine learning techniques to find features for defect classification, the proposed model adopts a single convolutional neural network (CNN) model that can extract effective features for defect classification without using additional feature extraction algorithms. Moreover, the proposed method can identify defect classes not seen during training by comparing the CNN features of the unseen classes with those of the trained classes. Experiments with real datasets verified that the proposed ADC method achieves high defect classification performance.
author_names: Sejune Cheon

Hankang Lee

Chang Ouk Kim

Seok Hyung Lee
title: Convolutional Neural Network for Wafer Surface Defect Classification and the Detection of Unknown Defect Class
venue: IEEE Transactions on Semiconductor Manufacturing
year: 2019

abstract: Abstract The similarity of gray characteristics and shapes of the defects, stems, and calyxes in apple images is a difficult challenge for automatic identification and detection of apple defects in the machine vision field. This paper attempts towards finding an automatic and efficient way to identify apple defects. An automatic detection method is proposed for apple defects based on laser induced light backscattering imaging and convolutional neural network (CNN) algorithm. Laser backscattering spectroscopic images of apples are obtained using semiconductor laser. We take preprocessing steps to get the finest image dataset for CNN. An AlexNet model with an 11 layer structure is established and trained to identify apple defects. We analyze how well the model does with the recognition effects of apple defects. The proposed CNN model for the detection of apple defects achieves a higher recognition rate of 92.5% and the accuracy is better than conventional machine learning algorithms. The method based on laser backscattering imaging analysis and CNN theory provides an idea and theoretical basis for efficient, non destructive, and online detection of fruits quality.
author_names: Ang Wu

Juan-hua Zhu

Taiyong Ren
title: Detection of apple defect using laser induced light backscattering imaging and convolutional neural network
venue: Comput. Electr. Eng.
year: 2020

abstract: With semiconductor technology progressing beyond 5nm node, there is tremendous pressure on computational lithography to achieve both accuracy and speed. One very promising technique to accomplish this mission is to take full advantage of the maturing machine learning techniques based on neural network architecture. Some success has been achieved using convolution neural network (CNN) to obtain inverse lithography technology (ILT) solution with significantly less computational time. In general, CNN architecture consists of feature extraction layers and nonlinear mapping function construction layers. To train a CNN model requires a large amount of data and computational resource. To maintain certain intrinsic symmetries of imaging behavior, the feature extraction layers must be carefully engineered using weight sharing techniques or using well balanced training samples of different orientations, otherwise, feature extraction part will be skewed. It is therefore very desired to have a scheme that can obtain optimal feature vector for machine learning based computational lithography automatically without the need of feature extraction layers in CNN. In this paper, we will make an attempt to describe such a scheme and present our test results on machine learning based OPC and ILT solution. It should be understood that machine learning based computational lithography solutions do not possess the capability to replace conventional OPC or ILT completely due to its lack of required accuracy. However, it can provide an initial solution that is close enough to final OPC solution or ILT solution, therefore fast OPC and fast ILT can be realized.
author_names: Xuelong Santa Clara Shi

Yuhang Zhao

Shoumian Cheng

Ming Li

Wei Yuan

Leon Yao

Wenhao Zhao

Yanjun Xiao

Xiao Hui Kang

Angmar Li
title: Optimal feature vector design for computational lithography
venue: Advanced Lithography
year: 2019

abstract: Computer vision and classification methods have become increasingly wide spread in recent years due to ever increasing access to computation power. Advances in semiconductor devices are the basis for this growth, but few publications have probed the benefits of data driven methods for improving a critical component of semiconductor manufacturing, the detection and inspection of defects for such devices. As defects become smaller, intensity threshold based approaches eventually fail to adequately discern differences between faulty and non faulty structures. To overcome these challenges we present machine learning methods including convolutional neural networks (CNN) applied to image based defect detection. These images are formed from the simulated scattering of realistic geometries with and without key defects while also taking into account line edge roughness (LER) LER is a known and challenging problem in fabrication as it yields additional scattering that further complicates defect inspection. Simulating images of an intentional defect array, a CNN approach is applied to extend detectability and enhance classification to these defects, even those that are more than 20 times smaller than the inspection wavelength.
author_names: Mark-Alexander Henn

Hui Zhou

Bryan M Barnes
title: Data driven approaches to optical patterned defect detection.
venue: OSA continuum
year: 2019

abstract: Computer vision and classification methods have become increasingly wide spread in recent years due to ever increasing access to computation power. Advances in semiconductor devices are the basis for this growth, but few publications have probed the benefits of data driven methods for improving a critical component of semiconductor manufacturing, the detection and inspection of defects for such devices. As defects become smaller, intensity threshold based approaches eventually fail to adequately discern differences between faulty and non faulty structures. To overcome these challenges we present machine learning methods including convolutional neural networks (CNN) applied to image based defect detection. These images are formed from the simulated scattering of realistic geometries with and without key defects while also taking into account line edge roughness (LER) LER is a known and challenging problem in fabrication as it yields additional scattering that further complicates defect inspection. Simulating images of an intentional defect array, a CNN approach is applied to extend detectability and enhance classification to these defects, even those that are more than 20 times smaller than the inspection wavelength. (c) 2019 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
title: Data driven approaches to optical patterned defect detection
year: 2019

abstract: With the development of machine learning technology, the exploration of energy efficient and flexible architectures for object inference algorithms is of growing interest in recent years. However, not many publications concentrate on a coarse grained reconfigurable architecture (CGRA) for object inference algorithms. This paper provides a stream processing, dual track programming CGRA based approach to address the inherent computing characteristics of algorithms in object inference. Based on the proposed approach, an architecture called stream dual track CGRA (SDT CGRA) is presented as an implementation prototype. To evaluate the performance, the SDT CGRA is realized in Verilog HDL and implemented in Semiconductor Manufacturing International Corporation 55 nm process, with the footprint of 5.19 mm2 at 450 MHz. Seven object inference algorithms, including convolutional neural network (CNN) {k} means, principal component analysis (PCA) spatial pyramid matching (SPM) linear support vector machine (SVM) Softmax, and Joint Bayesian, are selected as benchmarks. The experimental results show that the SDT CGRA can gain on average 343.8 times and 17.7 times higher energy efficiency for Softmax, PCA, and CNN, 621.0 times and 1261.8 times higher energy efficiency for {k} means, SPM, linear SVM, and Joint Bayesian algorithms when compared with the Intel Xeon E5 2637 CPU and the Nvidia TitanX graphics processing unit. When compared with the state of the art solutions of AlexNet on field programmable gate array and CGRA, the proposed SDT CGRA can achieve a 1.78 times increase in energy efficiency and a 13 times speedup, respectively.
author_names: Xitian Fan

Di Wu

Wei Cao

Wayne Luk

Lingli Wang
title: Stream Processing Dual Track CGRA for Object Inference
venue: IEEE Transactions on Very Large Scale Integration (VLSI) Systems
year: 2018

abstract: Computation in memory (CIM) is a promising avenue to improve the energy efficiency of multiply and accumulate (MAC) operations in AI chips. Multi bit CNNs are required for high inference accuracy in many applications [1 5] There are challenges and tradeoffs for SRAM based CIM: (1) tradeoffs between signal margin, cell stability and area overhead; (2) the high weighted bit process variation dominates the end result error rate; (3) trade off between input bandwidth, speed and area. Previous SRAM CIM macros were limited to binary MAC operations for fully connected networks [1] or they used CIM for multiplication [2] or weight combination operations [3] with additional large area near memory computing (NMC) logic for summation or MAC operations.
author_names: Xin Si

Ssu-Yen Wu
title: 24.5 A Twin 8T SRAM Computation In Memory Macro for Multiple Bit CNN Based Machine Learning
venue: 2019 IEEE International Solid State Circuits Conference (ISSCC)
year: 2019

abstract: A computer aided diagnosis (CAD) system based on mammograms enables early breast cancer detection, diagnosis, and treatment. However, the accuracy of the existing CAD systems remains unsatisfactory. This paper explores a breast CAD method based on feature fusion with convolutional neural network (CNN) deep features. First, we propose a mass detection method based on CNN deep features and unsupervised extreme learning machine (ELM) clustering. Second, we build a feature set fusing deep features, morphological features, texture features, and density features. Third, an ELM classifier is developed using the fused feature set to classify benign and malignant breast masses. Extensive experiments demonstrate the accuracy and efficiency of our proposed mass detection and breast cancer classification method.
author_names: Zhiqiong Wang

Mo Li

Huaxia Wang

Hanyu Jiang

Yu-dong Yao

Hao Zhang

Junchang Xin
title: Breast Cancer Detection Using Extreme Learning Machine Based on Feature Fusion With CNN Deep Features
venue: IEEE Access
year: 2019