# Analog Circuit Soft Fault Diagnosis Based on Sparse Random Projections and K-Nearest Neighbor

Authors: Jian Sun; Guobin Hu; Chenghua Wang
Journal: Scientific Programming
 (2021)

---

## Abstract

Analog circuit fault diagnosis is a key problem in theory of circuit networks and has been investigated by many researchers in recent years. An approach based on sparse random projections (SRPs) and K-nearest neighbor (KNN) to the realization of analog circuit soft fault diagnosis has been presented in this paper. The proposed method uses the wavelet packet energy spectrum and sparse random projections to preprocess the time response for feature extraction. Then, the variables of the fault features are constructed, which are used to form the observation sequences of K-nearest neighbor classifier. K-nearest neighbor classifier is used to accomplish the fault diagnosis of analog circuit. In this paper, four-opamp biquad high-pass filter has been used as simulation example to verify the effectiveness of the proposed method. The simulations show that the proposed method offers higher correct fault location rate in analog circuit soft fault diagnosis application as compared with the other methods.


---

## Body

## 1. Introduction

With the rapid development of the microelectronics industry, fault diagnosis of mixed-signal integrated circuits is an essential area of scientific research [1–5]. At present, there are well-consolidated fault diagnosis techniques for digital circuits. However, fault diagnosis of analog circuits is still a challenging and complex task due to the lack of the simple fault model, the presence of component tolerance, the disturbance of noise, and the influence of nonlinear property [6, 7]. In recent years, a number of scientists are getting more attention in the fields of analog circuit fault diagnosis, and many methods have been appeared [8–12].The techniques used in the analog circuit fault diagnosis can be divided into two broad categories: the parameter estimation methods and pattern recognition methods. The parameter estimation methods require complex mathematical models. Thus, the application of parameter estimation methods is very limited in practice [13]. Recently, pattern recognition methods have been widely used in analog circuit fault diagnosis. Fault feature extraction and fault classification are two critical processes for analog circuit fault diagnosis in pattern recognition methods. Through fault feature extraction, dimensions of fault feature can be reduced and redundancies of fault feature can be removed. It would be convenient to recognize fault category of analog circuit [14–17]. It is common to see that signal processing methods, information entropy methods, frequency analysis methods, and time-domain analysis methods are used to extract fault feature in analog circuit fault diagnosis [18]. In addition, principle component analysis (PCA) and linear discriminant analysis (LDA) are used to reduce the dimensionality of fault feature in analog circuit fault diagnosis [19, 20]. On the other hand, to design a robust classifier is also of the key importance for analog circuit fault diagnosis. Some machine learning algorithms are widely applied to diagnosis analog circuit fault, such as artificial neural network (ANN), support vector machine (SVM), support vector data description (SVDD), and Bayes network and fuzzy theory [21–23]. However, the network structure, weight, and threshold of artificial neural network are difficult to determine, and artificial neural network suffers from low convergence speed in fault diagnosis, which lead to be a poor efficiency in analog circuit fault diagnosis based on artificial neural network. Penalty parameter and kernel parameter influence SVM performance. It is difficult to determine kernel functions of SVM [24, 25]. These affect the performance in analog circuit fault diagnosis.K-nearest neighbor is a machine learning algorithm based on statistical characteristic. It classifies the test sample throughout estimating the distances of different eigenvalues and has the advantage of simple structure, no need for training and estimation parameter. The KNN algorithm has a good classification effect on overlap data. Thus, it is appropriate to use KNN classifier for analog circuit fault diagnosis. It has been recognized that fault diagnosis based on KNN can effectively deal with some characteristics of analog circuit, such as component tolerance and nonlinearity [26]. However, the computation complexity and storage space involved in neighbors searching of KNN prevent it from online monitoring, especially for multiple faults and high-dimensional cases [27]. One possible solution to this problem is to reduce the dimensionality of the data, ideally without losing predictive performance. The reason why KNN can deal with the problems in analog circuit fault is that the distances between local samples are utilized to perform fault detection. Therefore, the dimension reduction cannot cause the distance distortion and must guarantee the performance of KNN in the subspace. Random projections are a powerful method of dimensionality reduction that preserves the structure of the data without introducing significant distortion. The similarity of data vectors is preserved well under random projections. Random projections do not need to decompose matrix, so it have lower computational requirement. Sparse random projections can further improve computational efficiency, and it is extremely suitable for applications of high dimension and limited samples. The advantages of SRP make it be very suitable for analog circuit fault diagnosis [28, 29]. A fault diagnosis method combining random projection and K-nearest neighbor rule can realize reducing computational complexity while guaranteeing the performance of fault detection.To improve efficiency of analog circuit fault diagnosis, a new approach is introduced to diagnose fault via sparse random projection and K-nearest neighbor classifier for analog circuit in this paper. The proposed method not only can reduce the computational complexity and storage space but also approximately guarantee the advantages of KNN classifier in dealing with the problems of component tolerance and nonlinearity that often exist in analog circuit.


## 2. Materials and Methods

### 2.1. Sparse Random Projections

Random projections are an approximate algorithm for estimating distances between pairs of points in a high-dimensional vector space. It uses a suitable scale random matrix whose columns have unit lengths to project the high-dimensional feature vector down to a much lower-dimensional subspace, thus achieving the dimensional reduction of the original data set. Random projections are extremely suitable for applications of high dimension and limited samples [30]. Recently, it has been extensively applied in several areas, such as machine learning, image processing, and compressed sensing. The random projections algorithm is implemented as follows.LetXd×n be an original high-dimensional matrix, where d is the dimensionality. Let Rk×d be random projection matrix, where k≤d. The transformation of dimensionality reduction is given by the following equation:(1)Xk×NRP=Rk×dXd×N,where Xk×NRP is a low-dimensional matrix obtained after random projections. Random projections reduce data dimension, while approximately preserving the distance between pairs of mapping points in the Euclidean space. The data vectors in the low-dimensional matrix have important characteristic information of original data vectors. Therefore, random projections preserve the similarities of data vectors to a high degree. This theory can be derived from the famous JL (Johnson–Lindenstrauss) theorem. JL theorem is defined as follows:(2)k≥4ε22−ε33−klnn,where W is an arbitrary dataset in d-dimensional Euclidean space. There is a map f:Rd⟶Rk, and any two data u and v in set W satisfy the following formula:(3)1−εu−v2≤fu−fv2≤1+εu−v2.JL theorem indicates thatn data in high-dimensional Euclidean space can map into Ologn/ε2 dimensional subspace, and the distance between these data after projection is still approximate. The change distance is within 1±ε disturbance bound.The above analysis shows that the key to achieving data dimensionality reduction using the random projections method is to determine the appropriate random projections matrix. At present, the Gaussian random matrix is commonly used. Each of its elements is independent and subject to standard normal distribution. However, the Gaussian random matrix is more complicated for constructing large-scale random matrices and has a higher computational complexity for the projection operation of data. Bingham et al. [31] proposed that two simple distributions produce a random projections matrix. The elements are independent of each other in the Gaussian random matrix. Their values will be present in the form as follows:(4)ri,j=+1,P=12−1,P=12,(5)ri,j=+1,P=160P=23−1,P=16.The above two distributions greatly reduce the computational complexity of random projections matrix mapping conversion. Especially in equation (5), it can be seen from the construction that the matrix is very sparse. The elements in matrix have values of 0 with probability 2/3, and 2/3 projection calculated amount can be saved. So, the operation is faster. The sparse random projections matrix is proposed in this paper.


### 2.2. K-Nearest Neighbor

KNN is a supervised classifier. This algorithm calculates the similarity between the data samples and all neighbors and finds thek most similar samples. This algorithm classifies a new entity based on decision [32]. The time required may be significantly reduced by using the dimensionality reduction while still maintaining a high accuracy in KNN. Assume that the fault sample to be diagnosed is Xs. Given training data set U=xi,ci|i=1,2,⋯,N, which contains m categories. The category set is C=cl|l=1,2,⋯,m, where training sample xi is a p-dimensional column vector and ci is the category label corresponding to xi. Steps in KNN classifier algorithms are as follows:(1)
Calculate the distance between the samplesXs to be classified and all training samples. Euclidean distance, Manhattan distance, and Minkowski distance are usually used as the distance measured between the test object and every object in the set of training objects. Euclidean distance function is used in this paper.(6)dxs,xi=xs−xiTxs−xi.(2)
Select nearest neighbor parameterk. Find K-nearest neighbor samples Xs according to the calculated Euclidean distance, and it is written as follows: Qs=xsj,csj|j=1,2,⋯,k, where xsj is the jth 1≤j≤k nearest neighbor sample of the sample Xs to be classified and csj is the category corresponding to xsj.(3)
Vote based on the category information of the K-nearest neighbor samples inQs. The result of the vote is written as follows: V=v1,v2,⋯,vl,⋯vM. To make decisions according to the voting results, the decision rule is as follows:(7)cxs=argmaxvl.


## 2.1. Sparse Random Projections

Random projections are an approximate algorithm for estimating distances between pairs of points in a high-dimensional vector space. It uses a suitable scale random matrix whose columns have unit lengths to project the high-dimensional feature vector down to a much lower-dimensional subspace, thus achieving the dimensional reduction of the original data set. Random projections are extremely suitable for applications of high dimension and limited samples [30]. Recently, it has been extensively applied in several areas, such as machine learning, image processing, and compressed sensing. The random projections algorithm is implemented as follows.LetXd×n be an original high-dimensional matrix, where d is the dimensionality. Let Rk×d be random projection matrix, where k≤d. The transformation of dimensionality reduction is given by the following equation:(1)Xk×NRP=Rk×dXd×N,where Xk×NRP is a low-dimensional matrix obtained after random projections. Random projections reduce data dimension, while approximately preserving the distance between pairs of mapping points in the Euclidean space. The data vectors in the low-dimensional matrix have important characteristic information of original data vectors. Therefore, random projections preserve the similarities of data vectors to a high degree. This theory can be derived from the famous JL (Johnson–Lindenstrauss) theorem. JL theorem is defined as follows:(2)k≥4ε22−ε33−klnn,where W is an arbitrary dataset in d-dimensional Euclidean space. There is a map f:Rd⟶Rk, and any two data u and v in set W satisfy the following formula:(3)1−εu−v2≤fu−fv2≤1+εu−v2.JL theorem indicates thatn data in high-dimensional Euclidean space can map into Ologn/ε2 dimensional subspace, and the distance between these data after projection is still approximate. The change distance is within 1±ε disturbance bound.The above analysis shows that the key to achieving data dimensionality reduction using the random projections method is to determine the appropriate random projections matrix. At present, the Gaussian random matrix is commonly used. Each of its elements is independent and subject to standard normal distribution. However, the Gaussian random matrix is more complicated for constructing large-scale random matrices and has a higher computational complexity for the projection operation of data. Bingham et al. [31] proposed that two simple distributions produce a random projections matrix. The elements are independent of each other in the Gaussian random matrix. Their values will be present in the form as follows:(4)ri,j=+1,P=12−1,P=12,(5)ri,j=+1,P=160P=23−1,P=16.The above two distributions greatly reduce the computational complexity of random projections matrix mapping conversion. Especially in equation (5), it can be seen from the construction that the matrix is very sparse. The elements in matrix have values of 0 with probability 2/3, and 2/3 projection calculated amount can be saved. So, the operation is faster. The sparse random projections matrix is proposed in this paper.


## 2.2. K-Nearest Neighbor

KNN is a supervised classifier. This algorithm calculates the similarity between the data samples and all neighbors and finds thek most similar samples. This algorithm classifies a new entity based on decision [32]. The time required may be significantly reduced by using the dimensionality reduction while still maintaining a high accuracy in KNN. Assume that the fault sample to be diagnosed is Xs. Given training data set U=xi,ci|i=1,2,⋯,N, which contains m categories. The category set is C=cl|l=1,2,⋯,m, where training sample xi is a p-dimensional column vector and ci is the category label corresponding to xi. Steps in KNN classifier algorithms are as follows:(1)
Calculate the distance between the samplesXs to be classified and all training samples. Euclidean distance, Manhattan distance, and Minkowski distance are usually used as the distance measured between the test object and every object in the set of training objects. Euclidean distance function is used in this paper.(6)dxs,xi=xs−xiTxs−xi.(2)
Select nearest neighbor parameterk. Find K-nearest neighbor samples Xs according to the calculated Euclidean distance, and it is written as follows: Qs=xsj,csj|j=1,2,⋯,k, where xsj is the jth 1≤j≤k nearest neighbor sample of the sample Xs to be classified and csj is the category corresponding to xsj.(3)
Vote based on the category information of the K-nearest neighbor samples inQs. The result of the vote is written as follows: V=v1,v2,⋯,vl,⋯vM. To make decisions according to the voting results, the decision rule is as follows:(7)cxs=argmaxvl.


## 3. Analog Circuit Fault Diagnosis Model Based on SRP-KNN

Use four-opamp biquad high-pass filter circuit shown in Figure1 as example to introduce the analog circuit soft fault diagnosis method based on SRP-KNN. There are four opamps, two capacitors, and ten resistors in four-opamp biquad high-pass filter circuit. Capacitors and resistors are used as fault components to diagnose.Figure 1
Four-opamp biquad high-pass filter circuit.Figure2 shows four-opamp biquad high-pass filter amplitude-frequency response of different fault modes stimulated by a 1V AC signal. It can be seen from Figure 2 that when a component of the circuit fails, its amplitude-frequency response curve will change. Therefore, the fault characteristics of the analog circuit can be acquired through the response curve.Figure 2
Amplitude-frequency response curve of four-opamp biquad high-pass filter.The flow chart of analog circuit fault diagnosis based on SRP-KNN is shown in Figure3. In the first place, input a drive signal to the analog circuit under test. Simultaneously, collect wavelet packet energy spectrum fault features at the output of the circuit. Using the sparse random projections algorithm to reduce the dimensions of the collected fault feature, normalization processing is applied to these data. The samples are divided into train samples and test samples. The results of analog circuit fault diagnosis are obtained by KNN classifier.Figure 3
Flow chart of analog circuit fault diagnosis based on SRP-KNN.


## 4. Experiment Results and Analysis of Simulation

The example for analog circuit is a four-opamp biquad high-pass filter, which is shown as Figure1. It consists of 4 operational amplifiers, 2 capacitors, and 10 resistors. The nominal value of each component is indicated in Figure 1 wherein the resistor and capacitor each has a tolerance of 1%. If the values of resistor and capacitor are outside the tolerance range and are within 50%, then the circuit is considered to have a soft fault.Select C1, C2, R1, R2, R3, and R4 as diagnostic components by sensitivity analysis. The soft fault of each component has two modes: a large soft fault and a small soft fault. Use ↑ to indicate that the component has a large fault and ↓ to indicate that the component has a small fault. There are 13 soft fault modes including fault-free state (NF), which are listed in Table 1.Table 1
Soft fault modes of four-opamp biquad high-pass filter.
Fault codeFault modelNominal valueFault valueF1C1↑5nF6nFF2C1↓5nF4nFF3C2↑5nF6nFF4C2↓5nF4nFF5R1↑6.2 kΩ7.5 kΩF6R1↓6.2 kΩ5 kΩF7R2↑6.2 kΩ7.5 kΩF8R2↓6.2 kΩ5 kΩF9R3↑6.2 kΩ7.5 kΩF10R3↓6.2 kΩ5 kΩF11R3↑1.6 kΩ2 kΩF12R4↓1.6 kΩ1.2 kΩF13NF————OrCAD10.5 software is used to produce the simulation fault data according to the fault modes in Table1. The circuit is inputted a pulse stimulus with the amplitude of 5 V and pulse width of 10 us and implemented time-domain transient analysis and 50 Monte Carlo analyses. In the time-domain transient analysis, the run-to-time and maximum step size are set as 0.5 ms and 4 us. In the Monte Carlo analysis, the number of runs is set to 50 and distribution is set to uniform distribution. There are 50 time-domain response curves for each fault mode (including fault-free). In each time-domain transient response curve, there are 125 points. The time-domain transient fault response curves of four-opamp biquad high-pass filter (C1 big fault, C2 small fault, and normal state) using the time-domain transient analysis and the Monte Carlo analysis are shown in Figure 4.Figure 4
Fault response curves of four-opamp biquad high-pass filter.Then, three-layer Haar wavelet packet decomposition is performed on the collected fault signal. Eight bands wavelet packet reconstruction signals of the third layer are obtained, respectively. The energy spectrums in its eight bands are calculated, and the energy spectrum values are taken as fault features. Fault curves of 3-layer Haar wavelet packet decomposition (C1 big fault, C2 small fault, and normal state) are drawn in Figure 5.Figure 5
Fault curves of 3-layer Haar wavelet packet decomposition.Let the quadratic sum of each frequency band signal decomposed by wavelet packet be the wavelet packet energy spectrum. Assumedi,jk denotes the signal decomposed by wavelet packet. The energy of the signal in each frequency band is given by(8)Ei,j=∑K=1Ndi,jk2,j=0,1,⋯2i−1,where N denotes the length of original signal. Let the wavelet packet energy spectrum be E, which is constituted of all Ei,j as follows:(9)E=Ei,0,Ei,1,⋯Ei,j,⋯,Ei,2i−1.When the energy is large, it can be normalized as follows:(10)∑E=Ei,0+Ei,1+⋯+Ei,j+⋯+Ei,2i−1.The feature vector is defined as follows:(11)E′=E∑E.50 samples are collected for each fault model, and each sample has 8 attributes. Partial fault features of the wavelet packet energy spectrum are shown as Figure6, where x-axis represents eight bands by three-layer wavelet packet decomposition, y-axis represents analog circuit fault category, and z-axis represents energy value by wavelet packet decomposition.Figure 6
Partial wavelet packet energy spectrum fault features.In order to reduce redundancy and improve the recognition rate of analog circuit fault diagnosis, the sparse random projections algorithm mentioned above is used to further extract the fault features of analog circuit. Figure7 is a three-dimensional fault information distribution map extracted with sparse random projections.Figure 7
Scatter plot of the 3D fault feature extracted with SRP.Each fault feature is clearly divided in Figure7. It shows that the sparse random projections feature extraction algorithm can reflect the essence of analog circuit faults and has better clustering and distinguishing ability for analog circuit faults, which creates a good condition for subsequent fault diagnosis.In order to diagnosis circuit fault through KNN classifier, the extracted fault features are divided into two data sets. 30 samples are used as training data, and the rest 20 sample are test data. The total training data and test data are 390 and 260, respectively.Table2 lists diagnosis results of four-opamp biquad high-pass filter circuit using different classifiers. From Table 2, the proposed method diagnoses all kinds of faults correctly. It achieves 100% classification. The accuracy rate of RBF classifier is 65.77%, and the fuzzy fault number is 8. The result of SVM is 81.92%. Its fuzzy fault number is 9. The accuracy rate of BayesNet and decision tree is 97.31% and 96.54%, respectively, and the fuzzy fault number is 4 and 5, respectively. Obviously, the proposed method is better than other methods.Table 2
Diagnosis results of four-opamp biquad high-pass filter circuit.
Fault codeThe proposed method (%)RBFclassifier (%)BayesNet (%)Decision tree (%)SVM (%)F11003510090100F210010010010090F31001008510055F410010909070F5100010010090F61009510010090F71007010010090F810055100100100F910010010095100F10100100100100100F1110001009050F121001009510065F1310090959065Average result10065.7797.3196.5481.92Fuzzy fault sets are listed in Table3. The SRP-KNN method obtains the least fuzzy faults. RBF classifier cannot locate fuzzy fault set of {F2, F4, F5, F6, F7, F8, F11, F13}. SVM cannot locate fuzzy fault set of {F2, F3, F4, F5, F6, F7, F11, F12, F13}. Fuzzy fault set of {F3, F4, F12, F13} and {F1, F4, F9, F11, F13} cannot be correctly diagnosed for the methods in BayesNet and decision tree. These show that the proposed method obtains a decent performance for four-opamp biquad high-pass filter.Table 3
Fuzzy fault number for various diagnosis methods.
MethodFuzzy fault numberFuzzy faultRBF classifier8F1, F4–F8, F11, F13BayesNet4F3, F4, F12, F13Decision tree5F1, F4, F9, F11, F13SVM9F2–F7, F11–F13The proposed method0—In order to further illustrate the superiority of the proposed method, white noise is added to the fault information. In order to reflect the randomness of noise, the experiment is repeated ten times, and the average value of the ten experiments is recorded as the final diagnosis result. Table4 lists diagnosis results in the noisy environment.Table 4
Diagnosis results in the noisy environment.
Fault codeThe proposed method (%)BayesNet (%)Decision tree (%)SVM (%)F110099.599.593F2989492.580F3999096.554.5F410096.59560.5F510093.59676F6100959578F710010099.577F810097.59994F910010094.597.5F1010010099.592.5F11989693.557F1299.592.59665.5F1399969557.5Average result99.596.1996.2775.62From Table4, the accuracy of fault diagnosis has decreased slightly in the noisy environment. This shows that noise can affect the results of analog circuit fault diagnosis. The accuracy rate of the proposed method is 99.5%. The accuracy rate of BayesNet and decision tree is 96.19% and 96.27%, respectively. The accuracy rate of SVM is only 75.62%, which is much lower. The proposed method can diagnose eight fault modes, and BayesNet can only diagnose three fault modes. None of the fault modes can be completely diagnosed correctly by decision tree and SVM. It can be seen that SVM performs the worst in the noisy environment.Fault diagnosis accuracy rate curves of ten experiments in noisy environment are drawn in Figure8. From Figure 8, the accuracy rate of the proposed method is always the highest, and it is 100% in seven experiments. The accuracy rate of BayesNet, decision tree, and SVM is significantly lower than that of the proposed method. In addition, the accuracy rate of BayesNet, decision tree, and SVM fluctuates greatly. It indicates that these methods are more sensitive to noise. Therefore, the proposed method is significantly better than BayesNet, decision tree, and SVM in noisy environment. In summary, the fault diagnosis results prove that the introduced method can more effectively diagnose soft faults for analog circuit.Figure 8
Fault diagnosis accuracy curves in noisy environment.


## 5. Conclusion

In this paper, sparse random projections and K-nearest neighbor classifier techniques are applied for fault diagnosis in analog circuit. The study indicates that the preprocessing technique has a significant impact on fault diagnosis of analog circuit. The proposed method uses the wavelet packet energy spectrum and sparse random projections as preprocessor to extract analog circuit fault feature. It is contributed to reduce noise interference and test cost in fault diagnosis, and dimensionality reduction using SPR can lead to more effective KNN classification. Experiment results show that the introduced SRP-KNN-based method performs better than existing methods for analog circuit fault diagnosis even when noise is added. According analog circuit theory, the response of the nonlinear system is different from that of the linear system. Hence, the proposed method will be studied to diagnose nonlinear circuit fault in our further research.



---

*Source: 8040140-2021-11-02.xml*