# Heat Treatment Management of Radio Frequency Integrated Circuit IC Based on Deep Learning

Authors: Weijia Zhang; Qiancheng Rao; Zhenjie Gu
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Integrated circuit (IC) is a microchip that consists of several electronic components which works as a single unit. Now, aiming at the problem of how to effectively dissipate the heat of the radio frequency integrated circuit in the work, in order to effectively manage the heat dissipation of the chip and to improve the working life of the IC chip, this paper uses the algorithm of convolutional neural network to analyze the IC chip based on deep learning. Identify and manage heat dissipation. The experiments show that the algorithm can effectively identify heat treatment, further improve the working time limit of IC circuits, and provide a reference for the research of IC heat treatment management.


---

## Body

## 1. Introduction

With the development of electronic manufacturing technology, the quality standards of electronic products are getting higher and higher. As the main component of electronic products, the production process standard of PCBA has also been improved. The quality of PCBA determines the quality of electronic products. Therefore, the quality inspection of PCBA is strictly controlled in the industry. The quality inspection of PCBA is mainly to detect the soldering defects of its components. The heat generated during soldering defects is the PCBA quality inspection. The key points and difficulties are as follows: Because the volume of SMD components is getting smaller and smaller, the detection of soldering defects becomes more and more difficult, especially the detection of soldering defects of IC devices. IC devices have the characteristics of many pins and small spacing. These characteristics make the detection of welding defects of IC devices more difficult than other components. Common IC pin solder joint defects mainly include empty soldering, virtual soldering, bridging, little tin, and missing parts.At present, most domestic companies generally use optical equipment combined with image processing software to complete PCBA detection, that is, use AOI technology to detect soldering defects of IC devices. The algorithms currently used by AOI are mainly methods such as statistical modeling and template matching. These algorithms have two shortcomings in the detection of welding defects for IC-type devices. First, due to its large number of pins and small spacing, IC devices require far higher welding accuracy and recognition accuracy than other components. IC pin solder joints are too small, and it is very difficult to manually extract features using traditional algorithms. Insufficient feature extraction will affect the final defect detection accuracy. Second, when AOI inspects PCBA production lines with multiple models and small batches, because there are many types of IC devices, repeated modeling is required, which is time-consuming and cumbersome. Therefore, the detection of soldering defects of IC devices is the key and difficult point in PCBA detection [1].The research content of this paper is aimed at the problem of repeated modeling when AOI detects the soldering defects of multi-type IC pins. In the research of this article, the deep learning method is used for the thermal management of IC welding inspection. The advantage of deep learning is that it can extract feature information through learning by itself, which solves the problem of manual extraction of IC thermal management feature information. In addition, because the data set of the training sample uses the pin solder joint diagrams of different types of IC devices, the successfully trained network can directly detect the soldering defects of multiple types of IC pins, which solves the problem of facing multiple types of IC devices. The problem of repeated modeling has greatly improved the detection efficiency of AOI [2, 3].The rest of the paper is arranged as follows: Section2 focuses on overview of deep learning, while Section 3 is concerned about establishment of sample data set and training environment and Section 4 is related to experimental results and analysis of network model selection. The last section is the conclusion followed by the future work.


## 2. Overview of Deep Learning

### 2.1. Neural Network

Deep learning is a new field of machine learning. It analyzes data by simulating the human brain for analysis and learning and the mechanism of the human brain. Its essence is to let the computer build a learning model by learning many data samples so that the computer can learn and judge autonomously like a human [4].


#### 2.1.1. Neurons

The neuron includes input, output, and calculation modules, and its model is shown in Figure1. First, the neuron performs a weighted summation of the input signals and then outputs the output after performing nonlinear mapping through a nonlinear function. The nonlinear function is also called the activation function [5]. The output formula is
(1)z=gx1⋅w1+x2⋅w2+x3⋅w3.Figure 1
Neuron model.


#### 2.1.2. Shallow Neural Network

This article uses a two-layer neural network as an example to introduce the shallow neural network. The two-layer shallow neural network is mainly composed of three parts, including the input layer, the hidden layer, and the output layer [6–8]. Two adjacent layers can be connected to each other, but they cannot cross layer connection. The structure of the two-layer neural network is shown in Figure 2.Figure 2
Two-layer neural network.The calculation formula for the hidden layer is(2)α12=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31,α22=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31.The way to calculate the final output z is to use the middle layerα12 and α22 and the second weight matrix calculation:
(3)z=gα12⋅w1,12+α22⋅w1,22.If the prediction target is a vector, you need to add more nodes in the output layer. Take two outputs as an example, as shown in Figure3.Figure 3
Multioutput two-layer neural network.


### 2.2. Convolutional Neural Network

Convolutional neural network (CNN) is a deep learning network composed of neurons with learnable weights and deviations. It is mainly used to classify images, cluster based on similarity, and target in the scene recognition. Convolutional neural networks use end-to-end learning capabilities to directly input original images without excessive manual processing, which greatly reduces work complexity and labor costs. At the same time, the weight sharing mechanism is adopted between the neurons of CNN, which reduces the network training parameters and greatly reduces the amount of calculation and model complexity. Its main components are as follows: input layer (input image), convolutional layer, pooling layer, and fully connected layer [9, 10]. The introduction of local receptive fields and weight sharing makes convolutional neural networks have huge advantages.


#### 2.2.1. Partial Receptive Field

The correlation between local pixels in the image is stronger, and the correlation between pixels farther apart is weaker. Therefore, the neurons in the neural network do not need to perceive the entire image, but only need to make partial connections. Neurons use local perception instead of global image perception and then synthesize all local information to achieve the purpose of enhancing image information, which greatly reduces network parameters and calculations. The schematic diagram of local connections is shown in Figure4.Figure 4
Receptive field


#### 2.2.2. Weight Sharing

Although the local receptive field can reduce the network parameters, in the deep convolutional neural network, only the local connection method cannot minimize the network parameters, and the weight sharing method can further reduce the number of parameters. Weight sharing means that the connection weights of some neurons in the previous layer and the neurons in the current network layer are the same. The neurons of the same feature map in the convolutional neural network use the same convolution kernel so that the weight sharing method reduces the training parameters. At the same time, the trained network has a stronger generalization ability and realizes the independence of image features and position.


#### 2.2.3. Multiple Convolution Kernels

According to the principle of weight sharing, it can be known that when a convolution kernel performs a convolution operation, only one feature is obtained, which is not sufficient. The more features learned, the higher the detection accuracy of the neural network. Therefore, multiple convolution kernels must be introduced for feature extraction. For example, when 15 convolution kernels perform convolution operations, 15 different features can be learned. As shown in Figure5, it can be seen from the figure, the types of convolution kernels are represented by different colors, and the result of convolution by each convolution kernel will generate an image, which can be regarded as different channels of an image.Figure 5
Multiple convolution kernels.


#### 2.2.4. Activation Function

CNN has a strong feature extraction and feature expression capabilities, mainly because of its nonlinear processing unit, namely, the activation function. The activation function can improve the nonlinear expression ability of CNN and can be used to solve the linear inseparable situation. It is a very important part of CNN. The following are three typical nonlinear activation functions commonly used:(1)
The Sigmoid function is the most widely used activation function. Its value range is (0,1). The Sigmoid function is expressed as(4)Sigmiodx=11+e−x.As shown in Figure6, the Sigmoid function is monotonically increasing and is suitable for input normalization. But the derivative on both sides of the Sigmoid function tends to 0, there is soft saturation, and the gradient disappears easily, which makes training problems, and the function contains exponentiation, which is very time-consuming to solve. When training deep networks, it will greatly increase training time.
(2)
The range of Tanh function is (-1,1), and the mathematical formula is(5)Tanhx=1−e−2x1+e−2x.Figure 6
Sigmoid function diagram.As shown in Figure7, compared with the Sigmoid function, the Tanh function converges faster. But similar to the Sigmoid function, the Tanh function also has soft saturation, and it is prone to the disappearance of the gradient.
(3)
The ReLU function is a piecewise linear function with a value range of, and the mathematical formula is as follows and shown in Figure8:(6)ReLUx=max0,x.Figure 7
Tanh’s function graph.Figure 8
ReLU function diagram.


## 2.1. Neural Network

Deep learning is a new field of machine learning. It analyzes data by simulating the human brain for analysis and learning and the mechanism of the human brain. Its essence is to let the computer build a learning model by learning many data samples so that the computer can learn and judge autonomously like a human [4].


### 2.1.1. Neurons

The neuron includes input, output, and calculation modules, and its model is shown in Figure1. First, the neuron performs a weighted summation of the input signals and then outputs the output after performing nonlinear mapping through a nonlinear function. The nonlinear function is also called the activation function [5]. The output formula is
(1)z=gx1⋅w1+x2⋅w2+x3⋅w3.Figure 1
Neuron model.


### 2.1.2. Shallow Neural Network

This article uses a two-layer neural network as an example to introduce the shallow neural network. The two-layer shallow neural network is mainly composed of three parts, including the input layer, the hidden layer, and the output layer [6–8]. Two adjacent layers can be connected to each other, but they cannot cross layer connection. The structure of the two-layer neural network is shown in Figure 2.Figure 2
Two-layer neural network.The calculation formula for the hidden layer is(2)α12=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31,α22=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31.The way to calculate the final output z is to use the middle layerα12 and α22 and the second weight matrix calculation:
(3)z=gα12⋅w1,12+α22⋅w1,22.If the prediction target is a vector, you need to add more nodes in the output layer. Take two outputs as an example, as shown in Figure3.Figure 3
Multioutput two-layer neural network.


## 2.1.1. Neurons

The neuron includes input, output, and calculation modules, and its model is shown in Figure1. First, the neuron performs a weighted summation of the input signals and then outputs the output after performing nonlinear mapping through a nonlinear function. The nonlinear function is also called the activation function [5]. The output formula is
(1)z=gx1⋅w1+x2⋅w2+x3⋅w3.Figure 1
Neuron model.


## 2.1.2. Shallow Neural Network

This article uses a two-layer neural network as an example to introduce the shallow neural network. The two-layer shallow neural network is mainly composed of three parts, including the input layer, the hidden layer, and the output layer [6–8]. Two adjacent layers can be connected to each other, but they cannot cross layer connection. The structure of the two-layer neural network is shown in Figure 2.Figure 2
Two-layer neural network.The calculation formula for the hidden layer is(2)α12=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31,α22=gα11⋅w1,11+α21⋅w1,21+α31⋅w1,31.The way to calculate the final output z is to use the middle layerα12 and α22 and the second weight matrix calculation:
(3)z=gα12⋅w1,12+α22⋅w1,22.If the prediction target is a vector, you need to add more nodes in the output layer. Take two outputs as an example, as shown in Figure3.Figure 3
Multioutput two-layer neural network.


## 2.2. Convolutional Neural Network

Convolutional neural network (CNN) is a deep learning network composed of neurons with learnable weights and deviations. It is mainly used to classify images, cluster based on similarity, and target in the scene recognition. Convolutional neural networks use end-to-end learning capabilities to directly input original images without excessive manual processing, which greatly reduces work complexity and labor costs. At the same time, the weight sharing mechanism is adopted between the neurons of CNN, which reduces the network training parameters and greatly reduces the amount of calculation and model complexity. Its main components are as follows: input layer (input image), convolutional layer, pooling layer, and fully connected layer [9, 10]. The introduction of local receptive fields and weight sharing makes convolutional neural networks have huge advantages.


### 2.2.1. Partial Receptive Field

The correlation between local pixels in the image is stronger, and the correlation between pixels farther apart is weaker. Therefore, the neurons in the neural network do not need to perceive the entire image, but only need to make partial connections. Neurons use local perception instead of global image perception and then synthesize all local information to achieve the purpose of enhancing image information, which greatly reduces network parameters and calculations. The schematic diagram of local connections is shown in Figure4.Figure 4
Receptive field


### 2.2.2. Weight Sharing

Although the local receptive field can reduce the network parameters, in the deep convolutional neural network, only the local connection method cannot minimize the network parameters, and the weight sharing method can further reduce the number of parameters. Weight sharing means that the connection weights of some neurons in the previous layer and the neurons in the current network layer are the same. The neurons of the same feature map in the convolutional neural network use the same convolution kernel so that the weight sharing method reduces the training parameters. At the same time, the trained network has a stronger generalization ability and realizes the independence of image features and position.


### 2.2.3. Multiple Convolution Kernels

According to the principle of weight sharing, it can be known that when a convolution kernel performs a convolution operation, only one feature is obtained, which is not sufficient. The more features learned, the higher the detection accuracy of the neural network. Therefore, multiple convolution kernels must be introduced for feature extraction. For example, when 15 convolution kernels perform convolution operations, 15 different features can be learned. As shown in Figure5, it can be seen from the figure, the types of convolution kernels are represented by different colors, and the result of convolution by each convolution kernel will generate an image, which can be regarded as different channels of an image.Figure 5
Multiple convolution kernels.


### 2.2.4. Activation Function

CNN has a strong feature extraction and feature expression capabilities, mainly because of its nonlinear processing unit, namely, the activation function. The activation function can improve the nonlinear expression ability of CNN and can be used to solve the linear inseparable situation. It is a very important part of CNN. The following are three typical nonlinear activation functions commonly used:(1)
The Sigmoid function is the most widely used activation function. Its value range is (0,1). The Sigmoid function is expressed as(4)Sigmiodx=11+e−x.As shown in Figure6, the Sigmoid function is monotonically increasing and is suitable for input normalization. But the derivative on both sides of the Sigmoid function tends to 0, there is soft saturation, and the gradient disappears easily, which makes training problems, and the function contains exponentiation, which is very time-consuming to solve. When training deep networks, it will greatly increase training time.
(2)
The range of Tanh function is (-1,1), and the mathematical formula is(5)Tanhx=1−e−2x1+e−2x.Figure 6
Sigmoid function diagram.As shown in Figure7, compared with the Sigmoid function, the Tanh function converges faster. But similar to the Sigmoid function, the Tanh function also has soft saturation, and it is prone to the disappearance of the gradient.
(3)
The ReLU function is a piecewise linear function with a value range of, and the mathematical formula is as follows and shown in Figure8:(6)ReLUx=max0,x.Figure 7
Tanh’s function graph.Figure 8
ReLU function diagram.


## 2.2.1. Partial Receptive Field

The correlation between local pixels in the image is stronger, and the correlation between pixels farther apart is weaker. Therefore, the neurons in the neural network do not need to perceive the entire image, but only need to make partial connections. Neurons use local perception instead of global image perception and then synthesize all local information to achieve the purpose of enhancing image information, which greatly reduces network parameters and calculations. The schematic diagram of local connections is shown in Figure4.Figure 4
Receptive field


## 2.2.2. Weight Sharing

Although the local receptive field can reduce the network parameters, in the deep convolutional neural network, only the local connection method cannot minimize the network parameters, and the weight sharing method can further reduce the number of parameters. Weight sharing means that the connection weights of some neurons in the previous layer and the neurons in the current network layer are the same. The neurons of the same feature map in the convolutional neural network use the same convolution kernel so that the weight sharing method reduces the training parameters. At the same time, the trained network has a stronger generalization ability and realizes the independence of image features and position.


## 2.2.3. Multiple Convolution Kernels

According to the principle of weight sharing, it can be known that when a convolution kernel performs a convolution operation, only one feature is obtained, which is not sufficient. The more features learned, the higher the detection accuracy of the neural network. Therefore, multiple convolution kernels must be introduced for feature extraction. For example, when 15 convolution kernels perform convolution operations, 15 different features can be learned. As shown in Figure5, it can be seen from the figure, the types of convolution kernels are represented by different colors, and the result of convolution by each convolution kernel will generate an image, which can be regarded as different channels of an image.Figure 5
Multiple convolution kernels.


## 2.2.4. Activation Function

CNN has a strong feature extraction and feature expression capabilities, mainly because of its nonlinear processing unit, namely, the activation function. The activation function can improve the nonlinear expression ability of CNN and can be used to solve the linear inseparable situation. It is a very important part of CNN. The following are three typical nonlinear activation functions commonly used:(1)
The Sigmoid function is the most widely used activation function. Its value range is (0,1). The Sigmoid function is expressed as(4)Sigmiodx=11+e−x.As shown in Figure6, the Sigmoid function is monotonically increasing and is suitable for input normalization. But the derivative on both sides of the Sigmoid function tends to 0, there is soft saturation, and the gradient disappears easily, which makes training problems, and the function contains exponentiation, which is very time-consuming to solve. When training deep networks, it will greatly increase training time.
(2)
The range of Tanh function is (-1,1), and the mathematical formula is(5)Tanhx=1−e−2x1+e−2x.Figure 6
Sigmoid function diagram.As shown in Figure7, compared with the Sigmoid function, the Tanh function converges faster. But similar to the Sigmoid function, the Tanh function also has soft saturation, and it is prone to the disappearance of the gradient.
(3)
The ReLU function is a piecewise linear function with a value range of, and the mathematical formula is as follows and shown in Figure8:(6)ReLUx=max0,x.Figure 7
Tanh’s function graph.Figure 8
ReLU function diagram.


## 3. Establishment of Sample Data Set and Training Environment

### 3.1. Establishment of Sample Data Set

The sample database of this deep learning-based IC pin welding defect detection experiment comes from the customer company of Dongguan Shenzhou Vision Technology Co., Ltd., and is collected and sorted by me. The research purpose of this topic is to use the deep learning method to find a more versatile method for welding defect detection in AOI. Different types of IC devices can be directly tested for welding defects, aiming to solve the defects that require remodeling in traditional methods. Therefore, the IC pin solder joint images collected this time are obtained from a variety of different types of IC devices, which ensures the diversity of training samples and test samples. After reasonable data amplification, a sample library that meets the needs of this experiment was finally obtained. The IC pin solder joint diagrams collected this time are mainly divided into six categories, including virtual soldering, empty soldering, short circuit, missing parts, little tin, and normal. Figure9 is a partially collected IC overall image, and Figure 10 is a partial IC pin solder joint image in the training sample.Figure 9
IC overall diagram.Figure 10
Part of the IC pin solder joint diagram.


### 3.2. Selection of Convolutional Neural Network Model

Different neural network models have different classification effects on images. For different databases, different network models have different classification effects. Therefore, choosing a suitable network model is particularly important in the process of image classification using deep learning. The networks to be selected in this research are Le Net, Alex Net, and Google Net. The following will compare the classification effects of the three network models through specific experiments.This experiment is aimed at six types of solder joints, including virtual soldering, empty soldering, short circuit, missing parts, low tin, and normal. The preselected network models are Le Net, Alex Net, and Google Net. These three network models are basically trained using gradient-based algorithms, so the general process is similar, and the training process is as follows:(1)
Preprocess and classify the collected samples, and do a good job of labeling(2)
Initialization of weights(3)
Select the training sample as input, and obtain the output value after convolution operation, downsampling, and fully connected layer(4)
Calculate the error value between the actual output value and the target value(5)
Compare the magnitude of the error, if it is higher than the expected value, pass the error value back to the network through backpropagation, and adjustIf the weights and thresholds in the network are lower than the expected value, stop training, and save the network parameters.


## 3.1. Establishment of Sample Data Set

The sample database of this deep learning-based IC pin welding defect detection experiment comes from the customer company of Dongguan Shenzhou Vision Technology Co., Ltd., and is collected and sorted by me. The research purpose of this topic is to use the deep learning method to find a more versatile method for welding defect detection in AOI. Different types of IC devices can be directly tested for welding defects, aiming to solve the defects that require remodeling in traditional methods. Therefore, the IC pin solder joint images collected this time are obtained from a variety of different types of IC devices, which ensures the diversity of training samples and test samples. After reasonable data amplification, a sample library that meets the needs of this experiment was finally obtained. The IC pin solder joint diagrams collected this time are mainly divided into six categories, including virtual soldering, empty soldering, short circuit, missing parts, little tin, and normal. Figure9 is a partially collected IC overall image, and Figure 10 is a partial IC pin solder joint image in the training sample.Figure 9
IC overall diagram.Figure 10
Part of the IC pin solder joint diagram.


## 3.2. Selection of Convolutional Neural Network Model

Different neural network models have different classification effects on images. For different databases, different network models have different classification effects. Therefore, choosing a suitable network model is particularly important in the process of image classification using deep learning. The networks to be selected in this research are Le Net, Alex Net, and Google Net. The following will compare the classification effects of the three network models through specific experiments.This experiment is aimed at six types of solder joints, including virtual soldering, empty soldering, short circuit, missing parts, low tin, and normal. The preselected network models are Le Net, Alex Net, and Google Net. These three network models are basically trained using gradient-based algorithms, so the general process is similar, and the training process is as follows:(1)
Preprocess and classify the collected samples, and do a good job of labeling(2)
Initialization of weights(3)
Select the training sample as input, and obtain the output value after convolution operation, downsampling, and fully connected layer(4)
Calculate the error value between the actual output value and the target value(5)
Compare the magnitude of the error, if it is higher than the expected value, pass the error value back to the network through backpropagation, and adjustIf the weights and thresholds in the network are lower than the expected value, stop training, and save the network parameters.


## 4. Experimental Results and Analysis of Network Model Selection

In this experiment, the training sample data set includes a total of 3180 IC pin solder joint images, of which 800 are qualified and 2380 are defective, which are divided into 5 types of defects, and the test set totals 1,080. To facilitate training, all pictures are normalized to256×256, and the learning rate is set to 0.001. After the training of the three network models was completed, the IC pin solder joint images in the test set were detected and classified, and the entire experimental training and test process data were visualized, and the corresponding training and test loss functions were obtained, as well as the test. The changing process of accuracy. The specific experimental results are analyzed as follows: Figures 11 and 12 are schematic diagrams of the changes of related indexes in the process of classification experiment of IC pin solder joint defects based on Le Net network model. As shown in Figure 11, as the network training time increases, the training set loss function first slowly decreases and finally stabilizes after about 400 iterations, indicating that the network training is basically completed at this time. From Figure 12, it can be seen that after about 400 iterations, the loss function of the test set tends to be stable, and the recognition rate of the test set has reached 70%.Figure 11
LeNet training loss change graph.Figure 12
LeNet test accuracy and loss change graph.Figures13 and 14 are schematic diagrams of changes in related indexes during the IC pin solder joint defect classification experiment based on the Alex Net network model. As shown in Figure 13, as the network training time increases, the training set loss function first slowly decreases and finally stabilizes after about 300 iterations, indicating that the network training is basically completed at this time. It can be seen from Figure 14 that after about 300 iterations, the loss function of the test set tends to be stable, and the recognition rate of the test set has reached 74%.Figure 13
Alex Net training loss change graph.Figure 14
Alex Net test accuracy and loss change graph.


## 5. Conclusion

Aiming at the management of IC pin heat treatment, a method based on deep learning is proposed to detect IC pin soldering defects. Collect a large number of different types of IC device pin solder joint maps to establish a deep learning sample data set. The Alex Net network is selected as the experimental basic network through comparative experiments, and then the Alex Net network model is improved, the network is trained and tested, and several common defects in IC devices are successfully detected and classified, and the recognition rate can reach 94.23%. Within the allowable error range of industrial testing, the problem of feature extraction difficulties and the need to re-model different types of IC devices are solved. Compared with traditional detection methods, this method is more versatile.



---

*Source: 6886072-2022-05-13.xml*