# Enhanced Anomaly-Based Fault Detection System in Electrical Power Grids

Authors: Wisam Elmasry; Mohammed Wadi
Journal: International Transactions on Electrical Energy Systems
 (2022)

---

## Abstract

Early and accurate fault detection in electrical power grids is a very essential research area because of its positive influence on network stability and customer satisfaction. Although many electrical fault detection techniques have been introduced during the past decade, the existence of an effective and robust fault detection system is still rare in real-world applications. Moreover, one of the main challenges that delays the progress in this direction is the severe lack of reliable data for system validation. Therefore, this paper proposes a novel anomaly-based electrical fault detection system which is consistent with the concept of faults in the electrical power grids. It benefits from two phases prior to training phase, namely, data preprocessing and pretraining. While the data preprocessing phase executes all elementary operations on the raw data, the pretraining phase selects the optimal hyperparameters of the model using a particle swarm optimization (PSO)-based algorithm. Furthermore, the one-class support vector machines (OC-SVMs) and the principal component analysis (PCA) anomaly-based detection models are exploited to validate the proposed system on the VSB dataset which is a modern and realistic electrical fault detection dataset. Finally, the results are thoroughly discussed using several quantitative and statistical analyses. The experimental results confirm the effectiveness of the proposed system in improving the detection of electrical faults.


---

## Body

## 1. Introduction

Nowadays, there is a rapid growth in the electrical power grids in terms of size and complexity [1]. This growth includes all sectors of electrical power industry starting from generation to transmission and distribution [2]. One of the conventional problems encountered in the electrical power system is the sudden occurrence of electrical faults across transmission or distribution lines [3]. An electrical fault is deemed to be an abnormal change in current and voltage values, that is, higher values of current and voltage than those commonly expected to be under normal operating conditions. This deviation of voltage and current from nominal states is caused by human errors, environmental conditions, and equipment failures [4]. Furthermore, when an electrical fault occurs, it imposes excessively high current to flow across the network that may cause damage to devices and equipment [5]. Therefore, an early and accurate fault detection is pivotal to prevent equipment damage, service interruption, and loss of human and animal lives [6].Although electrical fault detection systems based on binary classification have been extensively researched during the last decade [7], it was reported that there is a research gap in this domain including the automation and validation of the system [8]. Hence, there is a dire need for an intelligent system that acts efficiently in the real-world power systems.The anomaly detection in machine learning refers to a special class of detection methods that seeks to identify anomalous samples or events in a dataset [9]. Basically, anomalies (also named outliers) are extremely different from the expected pattern of a dataset, and they are quantitatively scarce compared to the majority (normal) of samples [10]. Likewise, electrical faults rarely occur in real-world power systems (less than 5%) while the rest of the signals are normal [11]. Therefore, employing anomaly detection will fit the electrical fault detection problem instead of using the conventional binary classification where there is a need of enough amount of faulty signals in the dataset to prevent the binary classifier from biasing to the “normal” class [12]. The anomaly-based detection models are being trained on the normal samples exclusively in such a way that these models can discover the normal behaviour of data. Then, they can detect any unseen data which deviate from the preserved behaviour [13, 14].The aims and contributions of this research are threefold, as follows.(i)
An anomaly-based detection system is suggested to reveal electrical faults as they occur in the electrical power grids.(ii)
To enhance detection of electrical faults, two preliminary phases are introduced just before the training phase.(iii)
The VSB dataset is utilized to validate two anomaly-based detection models leveraged from the proposed technique.The rest of this paper is organized as follows. A list of related works in the domain of electrical fault detection is introduced in Section2. Section 3 describes the main characteristics of the VSB dataset. In Section 4, the proposed anomaly-based detection system is explained in detail with a brief description of each of the used models. Then, Section 5 presents which evaluation metrics are used along with their formulas, and the experimental results are also given. The obtained results are discussed and the proposed system is validated using various analytical and statistical aspects in Section 6. In Section 7, the conclusion of the study is drawn.


## 2. Literature Review

In the open literature, dozens of studies had been published in the area of electrical fault detection and classification. The artificial neural network (ANN) classifier has been utilized in many previous studies for fault detection. For instance, Jamil et al. tested an ANN model for three-phase power line fault detection and classification [15]. They used simulated data in MATLAB and obtained good results. Similarly, several ANN models with different structures were introduced to detect electrical faults on simulated dataset [16]. In [17], an ANN model was exploited to classify and detect electrical faults in simulated six-phase transmission line. Atul and Navita simulated a double-circuit transmission line using MATLAB and applied an ANN classifier on their simulated data for the purpose of fault detection and classification [18]. The fault detection and fault location in extra high voltage (EHV) environments were investigated using an ANN model and simulated dataset [19]. In [20], electrical faults in simulated transmission line were detected and classified using an ANN model. A simulation of the Nigerian power system using MATLAB was introduced in [21, 22]. Then, they used the simulated data for fault detection, fault classification, and fault location using an ANN model. Different ANN architectures with the backpropagation (BP) technique were proposed for both fault detection and classification [23, 24].Three variations of the ANN model, which are the adaptive network fuzzy inference system (ANFIS), probabilistic neural network (PNN), and generalized regression neural network (GRNN), were utilized for three different tasks, namely, fault detection, fault classification, and fault location [25]. The proposed models were trained and tested using simulated dataset in Simulink. Ekici et al. extended the former study by using the PNN model to classify faults while they used the resilient propagation (RPROP) technique to identify the location of faults [26].In [27], the prediction of fault occurrence and fault location using the concurrent fuzzy logic (CFL) was employed on data of different cases of simulated transmission lines. A novel technique for detecting fault locations in simulated transmission line was introduced in [28]. The frequency transformations, such as the wavelet transform, were also used for fault detection. Koley et al. suggested a hybrid technique for fault classification, fault location, and fault detection tasks [29]. The suggested technique in the former study exploited the wavelet transform along with the modular ANN model to accomplish these tasks in simulated six-phase transmission lines. In [30], Wani et al. investigated the effectiveness of using the wavelet transform and different ANN architectures on simulated data to detect and classify faults as well as to identify their locations.Another widely used machine learning method for fault detection is the support vector machine (SVM). For example, three SVM models with different kernel functions were used to detect faults using simulated data [31]. It was reported in the former study that the SVM model based on the Gaussian radial basis kernel function (RBF) was superior to other SVM models. Singh et al. performed fault detection and fault classification using a SVM classifier [32]. They also simulated a 3-phase transmission line in MATLAB framework and exploited the simulated data to validate their SVM model. In [33], a novel method using the SVM model was developed in order to detect faults and their type and location in simulated transmission lines.In a similar study [34], a new seasonal and trend decomposition using loess (STL) method was proposed, and a SVM model with the RBF kernel was utilized to recognize partial discharge (PD) activities. They trained and tested their model on the VSB dataset [35]. They obtained a recall value of 88% of actual PD signals. A unique anomaly-based fault detection technique is proposed and investigated by using the OC-SVM and PCA anomaly-based models [36]. The two models are validated on the VSB dataset and gained a good performance with accuracy of 80%.To put all together, most of aforementioned studies in the area of electrical fault detection suffer from the following two shortcomings. (i) They mainly depended on the binary classification-based methods to detect faults in transmission lines, which is inappropriate in the case of electrical faults since the electrical faults are rare in reality [36]. (ii) They exploited simulated datasets to validate their proposed techniques, which cannot accurately represent the actual pattern of electrical faults in real-world power systems [3, 6–8]. Therefore, proposing an enhanced anomaly-based electrical fault detection system that is based on real-time data is still very desirable.


## 3. VSB Dataset

The VSB (Technical University of Ostrava) dataset is a modern dataset which was published online in Kaggle Competition website in 2018 [35]. In addition to that, it is a realistic fault detection dataset because it was created by the ENET Center at Technical University of Ostrava using a new device for capturing electrical signals passed through real power lines [37]. Regarding structure of the VSB dataset, it has 8712 samples, and each sample is merely an electrical signal that has 800,000 voltage measurements stored as integer values. These signals are captured from a real 3-phase electrical power grid that operates at 50 Hz, and all signals are recorded over a single complete grid cycle (20 milliseconds).Furthermore, there is a feature, named “Class,” in the VSB dataset that determines the type of each signal, i.e., “normal” and “faulty” classes are labeled as “0” and “1,” respectively. On the other hand, the majority of samples in the VSB dataset belong to normal signals (8187 samples), while the rest (525 samples) are faulty signals. This severe defect between the number of normal and faulty samples in the VSB dataset may lead to poor classification results because the classifiers will bias to the majority class (“normal”). Hence, employing anomaly-based detection models is inevitable with such an imbalanced dataset.


## 4. Methodology and Models

The proposed system seeks to recognize anomalous patterns in the electrical power line’s voltage signals. Since anomaly-based detection models cannot deal with an electrical signal in its raw form, the input signal has to undergo a preprocessing phase where voltage measurements of the signal are filtered from noise and decomposed into chunks. Afterwards, a feature extraction process is executed to characterize the pattern of voltage measurements, and then these features are put into a data record and normalized. These data records of the input signals are used along with a PSO-based algorithm in the pretraining phase to determine the optimal hyperparameter vector of the selected anomaly-based detection model which fits the used model with underlying fault detection task. Thereafter, the optimized anomaly-based detection model is trained on data records of the normal signals which help the used model to build precisely a profile for the normal signals. Finally, the trained anomaly-based fault detection model will be ready to detect any faulty signals from the normal signals. Figure1 shows the mechanism of the proposed anomaly-based electrical fault detection system. In the next sections, the methodology of executing our empirical experiments which accomplishes the aforementioned proposed system and a brief description of anomaly-based detection models are explained in detail.Figure 1
Mechanism of the proposed anomaly-based fault detection system.


### 4.1. Methodology

Our methodology is developed to be evident and uncomplicated. It comprises four successive phases, namely, data preprocessing, pretraining, training, and testing. These phases are elaborated in the following sections. Figure2 depicts the diagram of our methodology.Figure 2
Diagram of experiments’ methodology.


#### 4.1.1. Data Preprocessing Phase

As the name of this phase may imply, it performs all elementary operations on the samples of the VSB dataset. It is very vital because it prepares data for modeling and analyzing by the used anomaly-based fault detection models. Five successive operations are applied on the VSB dataset in this phase: signal denoising, signal decomposition, feature extraction, data normalization, and dataset splitting, as follows.(1) Signal Denoising. Generally, machine learning specialists or scientists concern with applying machine learning methods or algorithms on the dataset instead of collecting samples or observations. Thus, they consider the collected data as a ready-for-use dataset and no further work is required. Unfortunately, this is not correct most of the time because usually the collected data are dirty, that is, containing a noise. There are several causes of noise existence in the collected data such as failure of the measurement devices, unexpected event, or casual environmental conditions. Indeed, there is no way to prevent capturing noise during data collecting process. In front of this fact, there is no option rather than accepting the existence of noise in the collected data. On the contrary, using dirty data to train models has a serious downside regarding the quality of data modeling and analyzing. This can be explained by the fact that all the descriptive statistics of the collected data such as the mean and standard deviation are sensitive to noise which can cause tests to either miss significant findings or distort real results. Therefore, the most effective solution in this case is filtering data from the noise [38].To have better results in data cleaning, the concept of noise inside data should be realized firstly. As mentioned in Section1, noise is another face of outliers in data. The noise in data is a set of rare samples that fall far away from the majority of data. There is no precise way to identify noise in general. But with the help of the concept of noise explained above, some statistical methods can be utilized to find out noise candidates. One of the well-known noise filtering methods is the interquartile range (IQR). The advantages of using the IQR method are not only because it does not depend on a specific distribution of data but also because it is relatively robust to the presence of noise compared to the other quantitative methods [39] In this paper, each signal in the VSB dataset is filtered from noise separately, as follows. Firstly, a copy of all voltage measurements of particular signal is saved and ordered in an ascending order. Afterwards, the first quartile Q1 (25% percentile) and the third quartile Q3 (75% percentile) of the signal measurements are calculated. Then, the value of IQR (the middle 50% of the signal) is computed according to the following equation:(1)IQR=Q3−Q1.After that, the value of IQR is multiplied byk value which is an adjustment factor. The aim of using such a factor is to determine the strength of outliers. In statistics, there are two widely used values of k, which are 1.5 and 3 [39]. While the value of 1.5 is used to identify weak (minor) outliers, the value of 3 is used to determine strong (major) outliers in data. In the electrical fault detection problem, two types of outliers are familiar: fault measurements which slightly deviate from the normal values and noise measurements which extremely differ from the normal values. As a result of that, the fault measurements have to be kept in order to accomplish the fault detection task, whereas the noise measurements have to be removed. Accordingly, in this paper, k value is set to be 3.(2)Threshold=3∗IQR.Thereafter, thethreshold value is exploited to determine the lower and upper fences of the noise measurements using (3 and 4), respectively.(3)Lower_Fence=Q1−threshold,(4)Upper_Fence=Q3+threshold.Finally, any voltage measurements of particular signal less than the lower fence value or greater than the upper fence value will be removed from the original signal data in the VSB dataset. The filtering process will step into the next signal in the VSB dataset and repeat the same procedure until all signals in the VSB dataset are filtered successfully.(2) Signal Decomposition. After signal denoising process is finished, the remaining voltage measurements of the ith signal in the VSB dataset are equal to 800,000−l, where l is the number of voltage measurements in the ith signal that are identified as a noise and removed. However, detecting faults in the remaining voltage measurements of each signal is still a difficult task because the few faulty measurements are located within a wide range of normal measurements of the signal. To overcome this problem, the signal decomposition process is indispensable [40].Signal decomposition is the process of partitioning the remaining voltage measurements of each signal into smaller chunks that are easier to detect faults within them. This will yield to shorten the range of voltage measurements in each chunk significantly and foster better understanding of faults within the range of their related normal measurements [41]. Obviously, if there are more chunks, the performance of the model will be raised [41]. However, in order to explore the performance differences in this paper, each signal in the VSB dataset is decomposed into 1, 2, 4, and 8 chunks in separate experiments. Let M denote the number of chunks; then, the signal decomposition process breaks up the remaining voltage measurements of each signal in the VSB dataset into M chunks, as follows:(5)chunk_size=ROUNDremaining_measurements_of_signaliM,where chunk_size is the size of the chunk and ROUND is a function that rounds a number to an integer value.(6)Chunkji=Xj−1∗chunk_size+1i,Xj−1∗chunk_size+2i,…,Xj−1∗chunk_size+chunk_sizei,j=1,2,…,M,where Xdi is the dth voltage measurement of the ith signal and Chunkji is the jth chunk of the ith signal.(3) Feature Extraction. Although the number of voltage measurements for each signal is reduced after performing signal denoising and signal decomposition, the number of voltage measurements in each chunk of the signal is estimated at tens or hundreds of thousands. This due to the fact that originally each signal in the VSB dataset consists of a numerous number of voltage measurements (800,000). Basically, each of the remaining voltage measurements of the signal will act as an input to the used models with a high-dimensional input space. Such a high-dimensional space of inputs is one of the emerging challenges in machine learning domain because high dimensionality is impracticable for the most of machine learning models and definitely it will cause a model’s failure with poor performance [12].To overcome this problem, a feature extraction process is performed to reduce dimensions of the feature space. Thus, 19 features from the existing voltage measurements are extracted for each chunk of the signal separately. After extracting features from all chunks of the signal, all extracted features are combined together along with the “Class” label of the signal in order to form a data record for that signal. The feature extraction process stops when all signals are processed and all resulting data records are put into a new dataset (the reduced dataset). Obviously, the number of features in the reduced dataset differs according to the specified number of chunks, that is, it is equal to19∗M+1, where M is the number of chunks.Indeed, the extracted features from each chunk of the signal are widespread statistics which can give us an informative picture about distribution and behaviour of the voltage measurements. All the extracted features are numeric values, described as follows.(i)
Mean is the average of a set of numbers and can be obtained using the following equation:(7)Meanj=∑i=1chunk_sizeXijchunk_size,where Meanj is the mean of the jth chunk of the signal and Xij is the ith voltage measurement of the jth chunk of the signal.(ii)
Standard deviation is a statistical measure that gives information about dispersion of a discrete set of numbers. As the value of standard deviation increases, the variation among these numbers increases too, and vice versa.(8)Standard_deviation=∑i=1chunk_sizeXij−X¯j2chunk_size,where Standard_deviationj is the standard deviation of the jth chunk of the signal and Xj¯ is the mean of the jth chunk of the signal.(iii)
Maximum value of the voltage measurements existed in particular chunk of the signal.(iv)
Minimum value of the voltage measurements existed in particular chunk of the signal.(v)
Percentile is a statistical measure that allows the chunk to be analyzed in terms of percentage [42]. For instance, the nth percentile is a number where n% of the voltage measurements fall below that number. In this paper, the 1%, 25%, 50%, 75%, and 99% percentiles of each chunk of the signal are computed. Mathematically, the percentile value can be obtained by selecting the element of rank z after sorting the voltage measurements of particular chunk in an ascending order.(9)z=P100×chunk_size,where P is the value of percentage.(vi)
Relative percentile is the amount of deviation of a specific data from the mean. In this study, the 0%, 1%, 25%, 50%, 75%, 99%, and 100% relative percentiles of each chunk of the signal are calculated using the following equation:(10)P%_Relative_Percentilej=P%_Percentilej−Meanj,where P%_Relative_Percentilej is the P% relative percentile of the jth chunk of the signal and P%_Percentilej is the P% percentile of the jth chunk of the signal.(vii)
Lower and upper bounds are the lowest and highest bands of the voltage measurements of particular chunk of the signal, as calculated using the following equations.(11)Lower_Boundj=Meanj−Standard_deviationj,Upper_Boundj=Meanj+Standard_deviationj,where Lower_Boundj and Upper_Boundj are the lower and upper bands of the jth chunk of the signal, respectively.(viii)
Height is the distance measured from the minimum to the maximum of the voltage measurements of particular chunk of the signal.(12)Heightj=Maximumj−Minimumj,where Heightj, Maximumj, and Minimumj are the height, maximum, and minimum of the jth chunk of the signal, respectively.(4) Data Normalization. For each data record in the reduced dataset, features (except the “Class” feature) are normalized into [0,1] using the min-max transformation, as follows.(13)xi=xi−MinMax−Min,where xi is the numeric feature of the ith data record in the reduced dataset and Min and Max are the minimum and maximum values for each numeric feature, respectively.(5) Dataset Splitting. The reduced dataset obtained from the previous step is split according to the concept of anomaly detection, that is, the training set contains only normal samples, whereas the test set contains both normal and faulty samples. Hence, 7100 (81.5%) of normal samples are randomly selected without replacement and inserted in the training set. The rest of normal samples (1087) and all faulty samples (525) are put into the test set with a total of 1612 samples (18.5%). Table 1 summarizes the main characteristics of the reduced dataset after the data preprocessing phase is finished. Figure 3 shows the flowchart of data preprocessing and its operations.Table 1
Main characteristics of the reduced dataset.
CharacteristicsValueYear2018Samples8712Classes2Data typeNumericNumber of features20 for 1 chunk39 for 2 chunks77 for 4 chunks153 for 8 chunksTraining set distributionNormal = 7100Faulty = 0Total = 7100Test set distributionNormal = 1087Faulty = 525Total = 1612Figure 3
Flowchart of the data preprocessing phase (M: number of chunks, i: current iteration, Si: the ith signal in the VSB dataset, Si′: the filtered signal of Si, recordi: the ith record in the reduced dataset, and N: the number of signals in the VSB dataset).


#### 4.1.2. Pretraining Phase

The pretraining phase is designed with the aim of improving the performance of the anomaly-based detection model by selecting its optimal hyperparameters which fit the electrical fault detection task. There are many swarm intelligence-based metaheuristics that can be utilized for hyperparameter selection, but the PSO-based algorithm which is proposed by Elmasry et al. [13] has attracted much attention due to its simplicity, stability, and generality [14]. Figure 4 depicts the diagram of the PSO-based algorithm and its functionality.Figure 4
Diagram of the PSO-based algorithm for hyperparameter selection.The basic idea behind the PSO-based algorithm is that it selects the hyperparameter vector of particular model that maximizes the accuracy of that model on the given dataset. Accordingly, the first step in the PSO-based algorithm is to adjust it with the optimal operating parameters. Table2 shows the values of main operating parameters of the PSO-based algorithm. The selected values in Table 2 are obtained after executing a grid search for each PSO parameter in its recommended domain. In addition to that, the domains in Table 2 are recommended in many theoretical and empirical previous studies [12, 13]. Afterwards, the user determines a list of the model’s hyperparameters and their recommended domains.Table 2
PSO parameters and their domains and selected values [12, 13].
PSO parameterDomainSelected valueSwarm size[5,40]40Velocity (min)[0,1]0Velocity (max)[0,1]1Coefficients of acceleration[1,6]1.43Constant of inertia weight[0.39,0.99]0.69Number of iterations (max)[20,100]50Stopping factor[0.01,0.001]0.001Then, in the second step, a copy of the training set is split into two independent sets: training only and validation. The hold-out sampling without replacement method is utilized to select randomly 6850 normal samples in the training-only data. The same sampling method is used to select randomly 250 normal samples as well as 125 faulty samples in the validation set (375 samples). Thereafter, for each iteration, the PSO-based algorithm tries many possible combinations of the model’s hyperparameters within their specified ranges. Once the model is tuned by a set of hyperparameters, the training-only data will be used for training and the validation sets for testing. Then, the accuracy value of the model will be computed and stored.Finally, when the stopping criteria are satisfied, the third step of the PSO-based algorithm outputs the optimal hyperparameter vector which maximized the accuracy value of the given anomaly-based model over all iterations. Table3 presents the hyperparameters of the used models, their ranges, and the optimal values after finishing the pretraining phase.Table 3
The resulting optimal hyperparameters of each anomaly-based detection model.
ModelParameterRangeOptimal valueOC-SVMNu[0.001,0.1]0.1Step = 0.01Epsilon[0.001,0.1]0.001Step = 0.01PCARank[2,10]2Step = 2Oversampling[2,10]4Step = 2Center{True, False}False


#### 4.1.3. Training and Testing Phases

The training phase is started when the optimized anomaly-based model is constructed using the optimal hyperparameters and trained on the full training set. Subsequently, the testing phase is put forward by testing the trained model on the test set. Finally, the obtained outcomes are stored for further processing later.


### 4.2. Anomaly-Based Detection Models

The experiments are designed and executed in the Azure Machine Learning (AML) studio [43] using the OC-SVM and PCA anomaly-based detection models. The AML is a free cloud-based platform that can provide users with many useful capabilities. For instance, it is a collaborative tool for designing and analyzing various machine learning experiments with massive computing resources [44, 45]. Sections 4.2.1 and 4.2.2 give a brief description of the used models and their operating parameters.


#### 4.2.1. One-Class Support Vector Machine

The OC-SVM is a special case of the traditional support vector machine where it learns from the training data to identify the majority class among other classes. To accomplish this goal, the OC-SVM model only trains on data belonging to a class that has a vast majority of the dataset samples (“normal” class of the signals). This helps the OC-SVM model to infer properties of the normal samples, and from these properties, it can determine the boundaries of these samples [46].Mathematically, the OC-SVM model tries to identify the smallest hypersphere which contains all the normal samples inside. Furthermore, samples located on the boundary of the hypersphere are known as support vectors, and those located outside the hypersphere are considered as anomalies. Accordingly, the problem can be defined as the following constrained optimization form [47]:(14)minr,c,ζr2+1νN∑i=1Nζisubject to:Φxi−c2≤r2+ζi∀i=1,2,3,…,N,where r, c, ζi, ν, N, xi, and Φxi−c2 are the hypersphere radius, the hypersphere center, the ith slack value, the nu hyperparameter value, the number of training set samples, the ith sample in the training set, and the distance between the ith sample and the hypersphere center, respectively. The slack value of a sample means the distance between this sample and the support vectors, and if this value is less than or equal to zero, then the sample will be inside the hypersphere. Otherwise, it will be considered as an outlier. From Karush–Kuhn–Tucker optimal conditions, the center of the hypersphere can be found [48], as follows.(15)c=∑i=1NαiΦxi,where αi’s are the solutions of the following constrained optimization problem:(16)maxα∑i=1Nαikxi,xi−∑i,j=1Nαiαjkxi,xjsubject to:∑i=1Nαi=1and 0≤αi≤1νN∀i=1,2,…,N,where k is the kernel function of the OC-SVM model. In this paper, the RBF kernel function is used since it is very common.There are two hyperparameters which control the performance of the OC-SVM model, namely, nu (ν) and epsilon (ϵ). The nu hyperparameter is a value that determines the upper bound on the fraction of outliers [49]. This upper bound lets the user to trade off between outliers and normal cases. Moreover, the epsilon hyperparameter is deemed to be a stopping factor which affects the number of iterations reached when optimizing the OC-SVM model. Once the value is exceeded, the OC-SVM model stops iterating on a solution [46].


#### 4.2.2. Principle Component Analysis

The PCA was firstly proposed by Karl Pearson in 1901 [50]. It is frequently exploited in machine learning to explore data because it not only describes the inner structure of data but also determines the variance in data. It is deemed to be an orthogonal linear transformation method that converts the data space into a more compact space, i.e., the principal components. This can be handled by analyzing data and looking for correlations among the features to determine the combination of values that best describes differences in outcomes.The PCA-based anomaly detection model only trains on the normal samples and learns from them which feature set constitutes the “normal” class. In the testing phase, each unseen sample is projected on the eigenvectors as well as a normalized error value is computed in order to identify whether this sample is normal or not. The PCA-based anomaly detection model has three hyperparameters which can adjust its performance, namely, oversampling, rank, and center [51].LetX denote the training set matrix of size NxP where the sample mean of each column is a zero empirical mean. Then, the PCA algorithm computes a set of size l of P-dimensional vectors of coefficients wk that transform each sample xi to a new principle component scores vector ti using the following equation:(17)tik=xi⋅wk,∀i=1,2,…,Nandk=1,2,…l,where the first coefficient w1 can be computed as follows.(18)w1=argmaxwTXTXwwTw.Finally, thekth coefficient can be calculated by abstracting the first k−1 principle components from the matrix X using (19), and then wk coefficient can be found using the following equation:(19)X^k=X−∑s=1k−1XwswTs,(20)wk=argmaxwTX^kTX^kwwTw.


## 4.1. Methodology

Our methodology is developed to be evident and uncomplicated. It comprises four successive phases, namely, data preprocessing, pretraining, training, and testing. These phases are elaborated in the following sections. Figure2 depicts the diagram of our methodology.Figure 2
Diagram of experiments’ methodology.


### 4.1.1. Data Preprocessing Phase

As the name of this phase may imply, it performs all elementary operations on the samples of the VSB dataset. It is very vital because it prepares data for modeling and analyzing by the used anomaly-based fault detection models. Five successive operations are applied on the VSB dataset in this phase: signal denoising, signal decomposition, feature extraction, data normalization, and dataset splitting, as follows.(1) Signal Denoising. Generally, machine learning specialists or scientists concern with applying machine learning methods or algorithms on the dataset instead of collecting samples or observations. Thus, they consider the collected data as a ready-for-use dataset and no further work is required. Unfortunately, this is not correct most of the time because usually the collected data are dirty, that is, containing a noise. There are several causes of noise existence in the collected data such as failure of the measurement devices, unexpected event, or casual environmental conditions. Indeed, there is no way to prevent capturing noise during data collecting process. In front of this fact, there is no option rather than accepting the existence of noise in the collected data. On the contrary, using dirty data to train models has a serious downside regarding the quality of data modeling and analyzing. This can be explained by the fact that all the descriptive statistics of the collected data such as the mean and standard deviation are sensitive to noise which can cause tests to either miss significant findings or distort real results. Therefore, the most effective solution in this case is filtering data from the noise [38].To have better results in data cleaning, the concept of noise inside data should be realized firstly. As mentioned in Section1, noise is another face of outliers in data. The noise in data is a set of rare samples that fall far away from the majority of data. There is no precise way to identify noise in general. But with the help of the concept of noise explained above, some statistical methods can be utilized to find out noise candidates. One of the well-known noise filtering methods is the interquartile range (IQR). The advantages of using the IQR method are not only because it does not depend on a specific distribution of data but also because it is relatively robust to the presence of noise compared to the other quantitative methods [39] In this paper, each signal in the VSB dataset is filtered from noise separately, as follows. Firstly, a copy of all voltage measurements of particular signal is saved and ordered in an ascending order. Afterwards, the first quartile Q1 (25% percentile) and the third quartile Q3 (75% percentile) of the signal measurements are calculated. Then, the value of IQR (the middle 50% of the signal) is computed according to the following equation:(1)IQR=Q3−Q1.After that, the value of IQR is multiplied byk value which is an adjustment factor. The aim of using such a factor is to determine the strength of outliers. In statistics, there are two widely used values of k, which are 1.5 and 3 [39]. While the value of 1.5 is used to identify weak (minor) outliers, the value of 3 is used to determine strong (major) outliers in data. In the electrical fault detection problem, two types of outliers are familiar: fault measurements which slightly deviate from the normal values and noise measurements which extremely differ from the normal values. As a result of that, the fault measurements have to be kept in order to accomplish the fault detection task, whereas the noise measurements have to be removed. Accordingly, in this paper, k value is set to be 3.(2)Threshold=3∗IQR.Thereafter, thethreshold value is exploited to determine the lower and upper fences of the noise measurements using (3 and 4), respectively.(3)Lower_Fence=Q1−threshold,(4)Upper_Fence=Q3+threshold.Finally, any voltage measurements of particular signal less than the lower fence value or greater than the upper fence value will be removed from the original signal data in the VSB dataset. The filtering process will step into the next signal in the VSB dataset and repeat the same procedure until all signals in the VSB dataset are filtered successfully.(2) Signal Decomposition. After signal denoising process is finished, the remaining voltage measurements of the ith signal in the VSB dataset are equal to 800,000−l, where l is the number of voltage measurements in the ith signal that are identified as a noise and removed. However, detecting faults in the remaining voltage measurements of each signal is still a difficult task because the few faulty measurements are located within a wide range of normal measurements of the signal. To overcome this problem, the signal decomposition process is indispensable [40].Signal decomposition is the process of partitioning the remaining voltage measurements of each signal into smaller chunks that are easier to detect faults within them. This will yield to shorten the range of voltage measurements in each chunk significantly and foster better understanding of faults within the range of their related normal measurements [41]. Obviously, if there are more chunks, the performance of the model will be raised [41]. However, in order to explore the performance differences in this paper, each signal in the VSB dataset is decomposed into 1, 2, 4, and 8 chunks in separate experiments. Let M denote the number of chunks; then, the signal decomposition process breaks up the remaining voltage measurements of each signal in the VSB dataset into M chunks, as follows:(5)chunk_size=ROUNDremaining_measurements_of_signaliM,where chunk_size is the size of the chunk and ROUND is a function that rounds a number to an integer value.(6)Chunkji=Xj−1∗chunk_size+1i,Xj−1∗chunk_size+2i,…,Xj−1∗chunk_size+chunk_sizei,j=1,2,…,M,where Xdi is the dth voltage measurement of the ith signal and Chunkji is the jth chunk of the ith signal.(3) Feature Extraction. Although the number of voltage measurements for each signal is reduced after performing signal denoising and signal decomposition, the number of voltage measurements in each chunk of the signal is estimated at tens or hundreds of thousands. This due to the fact that originally each signal in the VSB dataset consists of a numerous number of voltage measurements (800,000). Basically, each of the remaining voltage measurements of the signal will act as an input to the used models with a high-dimensional input space. Such a high-dimensional space of inputs is one of the emerging challenges in machine learning domain because high dimensionality is impracticable for the most of machine learning models and definitely it will cause a model’s failure with poor performance [12].To overcome this problem, a feature extraction process is performed to reduce dimensions of the feature space. Thus, 19 features from the existing voltage measurements are extracted for each chunk of the signal separately. After extracting features from all chunks of the signal, all extracted features are combined together along with the “Class” label of the signal in order to form a data record for that signal. The feature extraction process stops when all signals are processed and all resulting data records are put into a new dataset (the reduced dataset). Obviously, the number of features in the reduced dataset differs according to the specified number of chunks, that is, it is equal to19∗M+1, where M is the number of chunks.Indeed, the extracted features from each chunk of the signal are widespread statistics which can give us an informative picture about distribution and behaviour of the voltage measurements. All the extracted features are numeric values, described as follows.(i)
Mean is the average of a set of numbers and can be obtained using the following equation:(7)Meanj=∑i=1chunk_sizeXijchunk_size,where Meanj is the mean of the jth chunk of the signal and Xij is the ith voltage measurement of the jth chunk of the signal.(ii)
Standard deviation is a statistical measure that gives information about dispersion of a discrete set of numbers. As the value of standard deviation increases, the variation among these numbers increases too, and vice versa.(8)Standard_deviation=∑i=1chunk_sizeXij−X¯j2chunk_size,where Standard_deviationj is the standard deviation of the jth chunk of the signal and Xj¯ is the mean of the jth chunk of the signal.(iii)
Maximum value of the voltage measurements existed in particular chunk of the signal.(iv)
Minimum value of the voltage measurements existed in particular chunk of the signal.(v)
Percentile is a statistical measure that allows the chunk to be analyzed in terms of percentage [42]. For instance, the nth percentile is a number where n% of the voltage measurements fall below that number. In this paper, the 1%, 25%, 50%, 75%, and 99% percentiles of each chunk of the signal are computed. Mathematically, the percentile value can be obtained by selecting the element of rank z after sorting the voltage measurements of particular chunk in an ascending order.(9)z=P100×chunk_size,where P is the value of percentage.(vi)
Relative percentile is the amount of deviation of a specific data from the mean. In this study, the 0%, 1%, 25%, 50%, 75%, 99%, and 100% relative percentiles of each chunk of the signal are calculated using the following equation:(10)P%_Relative_Percentilej=P%_Percentilej−Meanj,where P%_Relative_Percentilej is the P% relative percentile of the jth chunk of the signal and P%_Percentilej is the P% percentile of the jth chunk of the signal.(vii)
Lower and upper bounds are the lowest and highest bands of the voltage measurements of particular chunk of the signal, as calculated using the following equations.(11)Lower_Boundj=Meanj−Standard_deviationj,Upper_Boundj=Meanj+Standard_deviationj,where Lower_Boundj and Upper_Boundj are the lower and upper bands of the jth chunk of the signal, respectively.(viii)
Height is the distance measured from the minimum to the maximum of the voltage measurements of particular chunk of the signal.(12)Heightj=Maximumj−Minimumj,where Heightj, Maximumj, and Minimumj are the height, maximum, and minimum of the jth chunk of the signal, respectively.(4) Data Normalization. For each data record in the reduced dataset, features (except the “Class” feature) are normalized into [0,1] using the min-max transformation, as follows.(13)xi=xi−MinMax−Min,where xi is the numeric feature of the ith data record in the reduced dataset and Min and Max are the minimum and maximum values for each numeric feature, respectively.(5) Dataset Splitting. The reduced dataset obtained from the previous step is split according to the concept of anomaly detection, that is, the training set contains only normal samples, whereas the test set contains both normal and faulty samples. Hence, 7100 (81.5%) of normal samples are randomly selected without replacement and inserted in the training set. The rest of normal samples (1087) and all faulty samples (525) are put into the test set with a total of 1612 samples (18.5%). Table 1 summarizes the main characteristics of the reduced dataset after the data preprocessing phase is finished. Figure 3 shows the flowchart of data preprocessing and its operations.Table 1
Main characteristics of the reduced dataset.
CharacteristicsValueYear2018Samples8712Classes2Data typeNumericNumber of features20 for 1 chunk39 for 2 chunks77 for 4 chunks153 for 8 chunksTraining set distributionNormal = 7100Faulty = 0Total = 7100Test set distributionNormal = 1087Faulty = 525Total = 1612Figure 3
Flowchart of the data preprocessing phase (M: number of chunks, i: current iteration, Si: the ith signal in the VSB dataset, Si′: the filtered signal of Si, recordi: the ith record in the reduced dataset, and N: the number of signals in the VSB dataset).


### 4.1.2. Pretraining Phase

The pretraining phase is designed with the aim of improving the performance of the anomaly-based detection model by selecting its optimal hyperparameters which fit the electrical fault detection task. There are many swarm intelligence-based metaheuristics that can be utilized for hyperparameter selection, but the PSO-based algorithm which is proposed by Elmasry et al. [13] has attracted much attention due to its simplicity, stability, and generality [14]. Figure 4 depicts the diagram of the PSO-based algorithm and its functionality.Figure 4
Diagram of the PSO-based algorithm for hyperparameter selection.The basic idea behind the PSO-based algorithm is that it selects the hyperparameter vector of particular model that maximizes the accuracy of that model on the given dataset. Accordingly, the first step in the PSO-based algorithm is to adjust it with the optimal operating parameters. Table2 shows the values of main operating parameters of the PSO-based algorithm. The selected values in Table 2 are obtained after executing a grid search for each PSO parameter in its recommended domain. In addition to that, the domains in Table 2 are recommended in many theoretical and empirical previous studies [12, 13]. Afterwards, the user determines a list of the model’s hyperparameters and their recommended domains.Table 2
PSO parameters and their domains and selected values [12, 13].
PSO parameterDomainSelected valueSwarm size[5,40]40Velocity (min)[0,1]0Velocity (max)[0,1]1Coefficients of acceleration[1,6]1.43Constant of inertia weight[0.39,0.99]0.69Number of iterations (max)[20,100]50Stopping factor[0.01,0.001]0.001Then, in the second step, a copy of the training set is split into two independent sets: training only and validation. The hold-out sampling without replacement method is utilized to select randomly 6850 normal samples in the training-only data. The same sampling method is used to select randomly 250 normal samples as well as 125 faulty samples in the validation set (375 samples). Thereafter, for each iteration, the PSO-based algorithm tries many possible combinations of the model’s hyperparameters within their specified ranges. Once the model is tuned by a set of hyperparameters, the training-only data will be used for training and the validation sets for testing. Then, the accuracy value of the model will be computed and stored.Finally, when the stopping criteria are satisfied, the third step of the PSO-based algorithm outputs the optimal hyperparameter vector which maximized the accuracy value of the given anomaly-based model over all iterations. Table3 presents the hyperparameters of the used models, their ranges, and the optimal values after finishing the pretraining phase.Table 3
The resulting optimal hyperparameters of each anomaly-based detection model.
ModelParameterRangeOptimal valueOC-SVMNu[0.001,0.1]0.1Step = 0.01Epsilon[0.001,0.1]0.001Step = 0.01PCARank[2,10]2Step = 2Oversampling[2,10]4Step = 2Center{True, False}False


### 4.1.3. Training and Testing Phases

The training phase is started when the optimized anomaly-based model is constructed using the optimal hyperparameters and trained on the full training set. Subsequently, the testing phase is put forward by testing the trained model on the test set. Finally, the obtained outcomes are stored for further processing later.


## 4.1.1. Data Preprocessing Phase

As the name of this phase may imply, it performs all elementary operations on the samples of the VSB dataset. It is very vital because it prepares data for modeling and analyzing by the used anomaly-based fault detection models. Five successive operations are applied on the VSB dataset in this phase: signal denoising, signal decomposition, feature extraction, data normalization, and dataset splitting, as follows.(1) Signal Denoising. Generally, machine learning specialists or scientists concern with applying machine learning methods or algorithms on the dataset instead of collecting samples or observations. Thus, they consider the collected data as a ready-for-use dataset and no further work is required. Unfortunately, this is not correct most of the time because usually the collected data are dirty, that is, containing a noise. There are several causes of noise existence in the collected data such as failure of the measurement devices, unexpected event, or casual environmental conditions. Indeed, there is no way to prevent capturing noise during data collecting process. In front of this fact, there is no option rather than accepting the existence of noise in the collected data. On the contrary, using dirty data to train models has a serious downside regarding the quality of data modeling and analyzing. This can be explained by the fact that all the descriptive statistics of the collected data such as the mean and standard deviation are sensitive to noise which can cause tests to either miss significant findings or distort real results. Therefore, the most effective solution in this case is filtering data from the noise [38].To have better results in data cleaning, the concept of noise inside data should be realized firstly. As mentioned in Section1, noise is another face of outliers in data. The noise in data is a set of rare samples that fall far away from the majority of data. There is no precise way to identify noise in general. But with the help of the concept of noise explained above, some statistical methods can be utilized to find out noise candidates. One of the well-known noise filtering methods is the interquartile range (IQR). The advantages of using the IQR method are not only because it does not depend on a specific distribution of data but also because it is relatively robust to the presence of noise compared to the other quantitative methods [39] In this paper, each signal in the VSB dataset is filtered from noise separately, as follows. Firstly, a copy of all voltage measurements of particular signal is saved and ordered in an ascending order. Afterwards, the first quartile Q1 (25% percentile) and the third quartile Q3 (75% percentile) of the signal measurements are calculated. Then, the value of IQR (the middle 50% of the signal) is computed according to the following equation:(1)IQR=Q3−Q1.After that, the value of IQR is multiplied byk value which is an adjustment factor. The aim of using such a factor is to determine the strength of outliers. In statistics, there are two widely used values of k, which are 1.5 and 3 [39]. While the value of 1.5 is used to identify weak (minor) outliers, the value of 3 is used to determine strong (major) outliers in data. In the electrical fault detection problem, two types of outliers are familiar: fault measurements which slightly deviate from the normal values and noise measurements which extremely differ from the normal values. As a result of that, the fault measurements have to be kept in order to accomplish the fault detection task, whereas the noise measurements have to be removed. Accordingly, in this paper, k value is set to be 3.(2)Threshold=3∗IQR.Thereafter, thethreshold value is exploited to determine the lower and upper fences of the noise measurements using (3 and 4), respectively.(3)Lower_Fence=Q1−threshold,(4)Upper_Fence=Q3+threshold.Finally, any voltage measurements of particular signal less than the lower fence value or greater than the upper fence value will be removed from the original signal data in the VSB dataset. The filtering process will step into the next signal in the VSB dataset and repeat the same procedure until all signals in the VSB dataset are filtered successfully.(2) Signal Decomposition. After signal denoising process is finished, the remaining voltage measurements of the ith signal in the VSB dataset are equal to 800,000−l, where l is the number of voltage measurements in the ith signal that are identified as a noise and removed. However, detecting faults in the remaining voltage measurements of each signal is still a difficult task because the few faulty measurements are located within a wide range of normal measurements of the signal. To overcome this problem, the signal decomposition process is indispensable [40].Signal decomposition is the process of partitioning the remaining voltage measurements of each signal into smaller chunks that are easier to detect faults within them. This will yield to shorten the range of voltage measurements in each chunk significantly and foster better understanding of faults within the range of their related normal measurements [41]. Obviously, if there are more chunks, the performance of the model will be raised [41]. However, in order to explore the performance differences in this paper, each signal in the VSB dataset is decomposed into 1, 2, 4, and 8 chunks in separate experiments. Let M denote the number of chunks; then, the signal decomposition process breaks up the remaining voltage measurements of each signal in the VSB dataset into M chunks, as follows:(5)chunk_size=ROUNDremaining_measurements_of_signaliM,where chunk_size is the size of the chunk and ROUND is a function that rounds a number to an integer value.(6)Chunkji=Xj−1∗chunk_size+1i,Xj−1∗chunk_size+2i,…,Xj−1∗chunk_size+chunk_sizei,j=1,2,…,M,where Xdi is the dth voltage measurement of the ith signal and Chunkji is the jth chunk of the ith signal.(3) Feature Extraction. Although the number of voltage measurements for each signal is reduced after performing signal denoising and signal decomposition, the number of voltage measurements in each chunk of the signal is estimated at tens or hundreds of thousands. This due to the fact that originally each signal in the VSB dataset consists of a numerous number of voltage measurements (800,000). Basically, each of the remaining voltage measurements of the signal will act as an input to the used models with a high-dimensional input space. Such a high-dimensional space of inputs is one of the emerging challenges in machine learning domain because high dimensionality is impracticable for the most of machine learning models and definitely it will cause a model’s failure with poor performance [12].To overcome this problem, a feature extraction process is performed to reduce dimensions of the feature space. Thus, 19 features from the existing voltage measurements are extracted for each chunk of the signal separately. After extracting features from all chunks of the signal, all extracted features are combined together along with the “Class” label of the signal in order to form a data record for that signal. The feature extraction process stops when all signals are processed and all resulting data records are put into a new dataset (the reduced dataset). Obviously, the number of features in the reduced dataset differs according to the specified number of chunks, that is, it is equal to19∗M+1, where M is the number of chunks.Indeed, the extracted features from each chunk of the signal are widespread statistics which can give us an informative picture about distribution and behaviour of the voltage measurements. All the extracted features are numeric values, described as follows.(i)
Mean is the average of a set of numbers and can be obtained using the following equation:(7)Meanj=∑i=1chunk_sizeXijchunk_size,where Meanj is the mean of the jth chunk of the signal and Xij is the ith voltage measurement of the jth chunk of the signal.(ii)
Standard deviation is a statistical measure that gives information about dispersion of a discrete set of numbers. As the value of standard deviation increases, the variation among these numbers increases too, and vice versa.(8)Standard_deviation=∑i=1chunk_sizeXij−X¯j2chunk_size,where Standard_deviationj is the standard deviation of the jth chunk of the signal and Xj¯ is the mean of the jth chunk of the signal.(iii)
Maximum value of the voltage measurements existed in particular chunk of the signal.(iv)
Minimum value of the voltage measurements existed in particular chunk of the signal.(v)
Percentile is a statistical measure that allows the chunk to be analyzed in terms of percentage [42]. For instance, the nth percentile is a number where n% of the voltage measurements fall below that number. In this paper, the 1%, 25%, 50%, 75%, and 99% percentiles of each chunk of the signal are computed. Mathematically, the percentile value can be obtained by selecting the element of rank z after sorting the voltage measurements of particular chunk in an ascending order.(9)z=P100×chunk_size,where P is the value of percentage.(vi)
Relative percentile is the amount of deviation of a specific data from the mean. In this study, the 0%, 1%, 25%, 50%, 75%, 99%, and 100% relative percentiles of each chunk of the signal are calculated using the following equation:(10)P%_Relative_Percentilej=P%_Percentilej−Meanj,where P%_Relative_Percentilej is the P% relative percentile of the jth chunk of the signal and P%_Percentilej is the P% percentile of the jth chunk of the signal.(vii)
Lower and upper bounds are the lowest and highest bands of the voltage measurements of particular chunk of the signal, as calculated using the following equations.(11)Lower_Boundj=Meanj−Standard_deviationj,Upper_Boundj=Meanj+Standard_deviationj,where Lower_Boundj and Upper_Boundj are the lower and upper bands of the jth chunk of the signal, respectively.(viii)
Height is the distance measured from the minimum to the maximum of the voltage measurements of particular chunk of the signal.(12)Heightj=Maximumj−Minimumj,where Heightj, Maximumj, and Minimumj are the height, maximum, and minimum of the jth chunk of the signal, respectively.(4) Data Normalization. For each data record in the reduced dataset, features (except the “Class” feature) are normalized into [0,1] using the min-max transformation, as follows.(13)xi=xi−MinMax−Min,where xi is the numeric feature of the ith data record in the reduced dataset and Min and Max are the minimum and maximum values for each numeric feature, respectively.(5) Dataset Splitting. The reduced dataset obtained from the previous step is split according to the concept of anomaly detection, that is, the training set contains only normal samples, whereas the test set contains both normal and faulty samples. Hence, 7100 (81.5%) of normal samples are randomly selected without replacement and inserted in the training set. The rest of normal samples (1087) and all faulty samples (525) are put into the test set with a total of 1612 samples (18.5%). Table 1 summarizes the main characteristics of the reduced dataset after the data preprocessing phase is finished. Figure 3 shows the flowchart of data preprocessing and its operations.Table 1
Main characteristics of the reduced dataset.
CharacteristicsValueYear2018Samples8712Classes2Data typeNumericNumber of features20 for 1 chunk39 for 2 chunks77 for 4 chunks153 for 8 chunksTraining set distributionNormal = 7100Faulty = 0Total = 7100Test set distributionNormal = 1087Faulty = 525Total = 1612Figure 3
Flowchart of the data preprocessing phase (M: number of chunks, i: current iteration, Si: the ith signal in the VSB dataset, Si′: the filtered signal of Si, recordi: the ith record in the reduced dataset, and N: the number of signals in the VSB dataset).


## 4.1.2. Pretraining Phase

The pretraining phase is designed with the aim of improving the performance of the anomaly-based detection model by selecting its optimal hyperparameters which fit the electrical fault detection task. There are many swarm intelligence-based metaheuristics that can be utilized for hyperparameter selection, but the PSO-based algorithm which is proposed by Elmasry et al. [13] has attracted much attention due to its simplicity, stability, and generality [14]. Figure 4 depicts the diagram of the PSO-based algorithm and its functionality.Figure 4
Diagram of the PSO-based algorithm for hyperparameter selection.The basic idea behind the PSO-based algorithm is that it selects the hyperparameter vector of particular model that maximizes the accuracy of that model on the given dataset. Accordingly, the first step in the PSO-based algorithm is to adjust it with the optimal operating parameters. Table2 shows the values of main operating parameters of the PSO-based algorithm. The selected values in Table 2 are obtained after executing a grid search for each PSO parameter in its recommended domain. In addition to that, the domains in Table 2 are recommended in many theoretical and empirical previous studies [12, 13]. Afterwards, the user determines a list of the model’s hyperparameters and their recommended domains.Table 2
PSO parameters and their domains and selected values [12, 13].
PSO parameterDomainSelected valueSwarm size[5,40]40Velocity (min)[0,1]0Velocity (max)[0,1]1Coefficients of acceleration[1,6]1.43Constant of inertia weight[0.39,0.99]0.69Number of iterations (max)[20,100]50Stopping factor[0.01,0.001]0.001Then, in the second step, a copy of the training set is split into two independent sets: training only and validation. The hold-out sampling without replacement method is utilized to select randomly 6850 normal samples in the training-only data. The same sampling method is used to select randomly 250 normal samples as well as 125 faulty samples in the validation set (375 samples). Thereafter, for each iteration, the PSO-based algorithm tries many possible combinations of the model’s hyperparameters within their specified ranges. Once the model is tuned by a set of hyperparameters, the training-only data will be used for training and the validation sets for testing. Then, the accuracy value of the model will be computed and stored.Finally, when the stopping criteria are satisfied, the third step of the PSO-based algorithm outputs the optimal hyperparameter vector which maximized the accuracy value of the given anomaly-based model over all iterations. Table3 presents the hyperparameters of the used models, their ranges, and the optimal values after finishing the pretraining phase.Table 3
The resulting optimal hyperparameters of each anomaly-based detection model.
ModelParameterRangeOptimal valueOC-SVMNu[0.001,0.1]0.1Step = 0.01Epsilon[0.001,0.1]0.001Step = 0.01PCARank[2,10]2Step = 2Oversampling[2,10]4Step = 2Center{True, False}False


## 4.1.3. Training and Testing Phases

The training phase is started when the optimized anomaly-based model is constructed using the optimal hyperparameters and trained on the full training set. Subsequently, the testing phase is put forward by testing the trained model on the test set. Finally, the obtained outcomes are stored for further processing later.


## 4.2. Anomaly-Based Detection Models

The experiments are designed and executed in the Azure Machine Learning (AML) studio [43] using the OC-SVM and PCA anomaly-based detection models. The AML is a free cloud-based platform that can provide users with many useful capabilities. For instance, it is a collaborative tool for designing and analyzing various machine learning experiments with massive computing resources [44, 45]. Sections 4.2.1 and 4.2.2 give a brief description of the used models and their operating parameters.


### 4.2.1. One-Class Support Vector Machine

The OC-SVM is a special case of the traditional support vector machine where it learns from the training data to identify the majority class among other classes. To accomplish this goal, the OC-SVM model only trains on data belonging to a class that has a vast majority of the dataset samples (“normal” class of the signals). This helps the OC-SVM model to infer properties of the normal samples, and from these properties, it can determine the boundaries of these samples [46].Mathematically, the OC-SVM model tries to identify the smallest hypersphere which contains all the normal samples inside. Furthermore, samples located on the boundary of the hypersphere are known as support vectors, and those located outside the hypersphere are considered as anomalies. Accordingly, the problem can be defined as the following constrained optimization form [47]:(14)minr,c,ζr2+1νN∑i=1Nζisubject to:Φxi−c2≤r2+ζi∀i=1,2,3,…,N,where r, c, ζi, ν, N, xi, and Φxi−c2 are the hypersphere radius, the hypersphere center, the ith slack value, the nu hyperparameter value, the number of training set samples, the ith sample in the training set, and the distance between the ith sample and the hypersphere center, respectively. The slack value of a sample means the distance between this sample and the support vectors, and if this value is less than or equal to zero, then the sample will be inside the hypersphere. Otherwise, it will be considered as an outlier. From Karush–Kuhn–Tucker optimal conditions, the center of the hypersphere can be found [48], as follows.(15)c=∑i=1NαiΦxi,where αi’s are the solutions of the following constrained optimization problem:(16)maxα∑i=1Nαikxi,xi−∑i,j=1Nαiαjkxi,xjsubject to:∑i=1Nαi=1and 0≤αi≤1νN∀i=1,2,…,N,where k is the kernel function of the OC-SVM model. In this paper, the RBF kernel function is used since it is very common.There are two hyperparameters which control the performance of the OC-SVM model, namely, nu (ν) and epsilon (ϵ). The nu hyperparameter is a value that determines the upper bound on the fraction of outliers [49]. This upper bound lets the user to trade off between outliers and normal cases. Moreover, the epsilon hyperparameter is deemed to be a stopping factor which affects the number of iterations reached when optimizing the OC-SVM model. Once the value is exceeded, the OC-SVM model stops iterating on a solution [46].


### 4.2.2. Principle Component Analysis

The PCA was firstly proposed by Karl Pearson in 1901 [50]. It is frequently exploited in machine learning to explore data because it not only describes the inner structure of data but also determines the variance in data. It is deemed to be an orthogonal linear transformation method that converts the data space into a more compact space, i.e., the principal components. This can be handled by analyzing data and looking for correlations among the features to determine the combination of values that best describes differences in outcomes.The PCA-based anomaly detection model only trains on the normal samples and learns from them which feature set constitutes the “normal” class. In the testing phase, each unseen sample is projected on the eigenvectors as well as a normalized error value is computed in order to identify whether this sample is normal or not. The PCA-based anomaly detection model has three hyperparameters which can adjust its performance, namely, oversampling, rank, and center [51].LetX denote the training set matrix of size NxP where the sample mean of each column is a zero empirical mean. Then, the PCA algorithm computes a set of size l of P-dimensional vectors of coefficients wk that transform each sample xi to a new principle component scores vector ti using the following equation:(17)tik=xi⋅wk,∀i=1,2,…,Nandk=1,2,…l,where the first coefficient w1 can be computed as follows.(18)w1=argmaxwTXTXwwTw.Finally, thekth coefficient can be calculated by abstracting the first k−1 principle components from the matrix X using (19), and then wk coefficient can be found using the following equation:(19)X^k=X−∑s=1k−1XwswTs,(20)wk=argmaxwTX^kTX^kwwTw.


## 4.2.1. One-Class Support Vector Machine

The OC-SVM is a special case of the traditional support vector machine where it learns from the training data to identify the majority class among other classes. To accomplish this goal, the OC-SVM model only trains on data belonging to a class that has a vast majority of the dataset samples (“normal” class of the signals). This helps the OC-SVM model to infer properties of the normal samples, and from these properties, it can determine the boundaries of these samples [46].Mathematically, the OC-SVM model tries to identify the smallest hypersphere which contains all the normal samples inside. Furthermore, samples located on the boundary of the hypersphere are known as support vectors, and those located outside the hypersphere are considered as anomalies. Accordingly, the problem can be defined as the following constrained optimization form [47]:(14)minr,c,ζr2+1νN∑i=1Nζisubject to:Φxi−c2≤r2+ζi∀i=1,2,3,…,N,where r, c, ζi, ν, N, xi, and Φxi−c2 are the hypersphere radius, the hypersphere center, the ith slack value, the nu hyperparameter value, the number of training set samples, the ith sample in the training set, and the distance between the ith sample and the hypersphere center, respectively. The slack value of a sample means the distance between this sample and the support vectors, and if this value is less than or equal to zero, then the sample will be inside the hypersphere. Otherwise, it will be considered as an outlier. From Karush–Kuhn–Tucker optimal conditions, the center of the hypersphere can be found [48], as follows.(15)c=∑i=1NαiΦxi,where αi’s are the solutions of the following constrained optimization problem:(16)maxα∑i=1Nαikxi,xi−∑i,j=1Nαiαjkxi,xjsubject to:∑i=1Nαi=1and 0≤αi≤1νN∀i=1,2,…,N,where k is the kernel function of the OC-SVM model. In this paper, the RBF kernel function is used since it is very common.There are two hyperparameters which control the performance of the OC-SVM model, namely, nu (ν) and epsilon (ϵ). The nu hyperparameter is a value that determines the upper bound on the fraction of outliers [49]. This upper bound lets the user to trade off between outliers and normal cases. Moreover, the epsilon hyperparameter is deemed to be a stopping factor which affects the number of iterations reached when optimizing the OC-SVM model. Once the value is exceeded, the OC-SVM model stops iterating on a solution [46].


## 4.2.2. Principle Component Analysis

The PCA was firstly proposed by Karl Pearson in 1901 [50]. It is frequently exploited in machine learning to explore data because it not only describes the inner structure of data but also determines the variance in data. It is deemed to be an orthogonal linear transformation method that converts the data space into a more compact space, i.e., the principal components. This can be handled by analyzing data and looking for correlations among the features to determine the combination of values that best describes differences in outcomes.The PCA-based anomaly detection model only trains on the normal samples and learns from them which feature set constitutes the “normal” class. In the testing phase, each unseen sample is projected on the eigenvectors as well as a normalized error value is computed in order to identify whether this sample is normal or not. The PCA-based anomaly detection model has three hyperparameters which can adjust its performance, namely, oversampling, rank, and center [51].LetX denote the training set matrix of size NxP where the sample mean of each column is a zero empirical mean. Then, the PCA algorithm computes a set of size l of P-dimensional vectors of coefficients wk that transform each sample xi to a new principle component scores vector ti using the following equation:(17)tik=xi⋅wk,∀i=1,2,…,Nandk=1,2,…l,where the first coefficient w1 can be computed as follows.(18)w1=argmaxwTXTXwwTw.Finally, thekth coefficient can be calculated by abstracting the first k−1 principle components from the matrix X using (19), and then wk coefficient can be found using the following equation:(19)X^k=X−∑s=1k−1XwswTs,(20)wk=argmaxwTX^kTX^kwwTw.


## 5. Experimental Results

The data preprocessing and pretraining phases are carried out using the Python programming language version 3.9.6 [52] associated with the NumPy library [53]. On the other hand, the training and testing phases are hosted in the AML environment. The next two sections present the evaluation metrics and experimental results.


### 5.1. Evaluation Metrics

The outcome of the testing phase is merely a binary classification process indicating whether a sample is “normal” or “faulty.” Hence, a confusion matrix will be constructed after the testing phase is finished. This confusion matrix has four cells which contain the following measures: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). These measures are required to compute eight commonly used evaluation metrics, as follows.(i)
Accuracy is the ratio of the number of true classifications to the size of test set.(21)Accuracy=TP+TNTP+TN+FP+FN.(ii)
Precision is the ratio of the number of correctly classified faulty samples to all samples labeled as “faulty.”(22)Precision=TPTP+FP.(iii)
Recall is the ratio of the number of correctly classified faulty samples to all faulty samples. It is also named hit, true positive rate (TPR), detection rate (DR), or sensitivity.(23)Recall=TPTP+FN.(iv)
F1 score is a balanced metric that includes both the recall (R) and precision (P) values. It is also called F1 metric.(24)F1score=2×P×RP+R.(v)
False alarm rate (FAR) is the ratio of the number of normal samples that is wrongly classified as “faulty” to all normal samples. It is also named false positive rate (FPR).(25)FAR=FPFP+TN.(vi)
Specificity is the ratio of the number of correctly classified normal samples to all normal samples. It is also known as true negative rate (TNR).(26)Specificity=TNTN+FP.(vii)
False negative rate (FNR) is the ratio of the number of faulty samples that is wrongly classified as “normal” to all faulty samples.(27)FNR=FNFN+TP.(viii)
Matthews correlation coefficient (MCC) is a balanced measure that, when calculated, considers all major outcomes of classification. Usually, the MCC metric is useful in the case of using imbalanced datasets [54]. In addition to that, the MCC value can be in [−1,1], where -1 and 1 values indicate weak and perfect classifiers, respectively [55].(28)MCC=TP×TN−FP×FNTP+FN×TP+FP×TN+FP×TN+FN.


### 5.2. Performance Analysis

The evaluation metrics, mentioned in Section5.1, will be the key factors for assessing the performance of the used anomaly-based detection models in the electrical fault detection. Obviously, the higher the values of true classification metrics and the lower the values of misclassification metrics, the more effective the model. Table 4 shows the percentage of the evaluation metrics for each anomaly-based detection model in all experiments. Furthermore, the bold and italicized values in Table 4 represent the best results of the used models in the same experiment and among all experiments, respectively, whereas the “Normal” column in Table 4 refers to the results of the used models without using our proposed system, and the “Enhanced” columns refer to the results when using the proposed system.Table 4
Results of our empirical experiments.
ModelEvaluation metricNormalEnhancedNumber of chunks1248OC-SVMAccuracy75.6883.8187.7290.5194.42Precision64.8878.4582.7785.7794.30Recall55.2469.3378.6784.9588.19F1 score59.6773.6180.6685.3691.14FAR14.449.207.916.812.58Specificity85.5690.8092.0993.1997.42FNR44.7630.6721.3315.0511.81MCC42.7162.2471.7278.3487.18PCAAccuracy73.3982.1385.3092.6295.78Precision60.2176.2779.8890.2896.73Recall53.9065.5273.3386.6790.10F1 score56.8870.4976.4688.4493.29FAR17.209.848.924.511.47Specificity82.8090.1691.0895.4998.53FNR46.1034.4826.6713.339.90MCC37.8458.1365.9383.0590.34Clearly, the proposed anomaly-based fault detection system enhanced the performance of all models compared to same models without using the proposed system. This can be noticed when comparing the values of evaluation metrics of the used models in “Normal” and “Enhanced for 1 chunk” columns in Table4. For instance, the recall values are increased by 14%, and the FAR values are decreased by 8%. This can be explained by the impact of data preprocessing and pretraining phases that helps anomaly-based detection models to detect faults aggressively. Moreover, when the number of chunks is increased, the values of evaluation metrics for all models are improved significantly. For example, the recall and FAR values of the OC-SVM and PCA models for one chunk are (69.33%, 9.20%) and (65.52%, 9.84%), respectively. But when the number of chunks is increased to 8, they become (88.19%, 2.58%) and (90.10%, 1.47%).Unfortunately, improving the model’s performance by partitioning the voltage measurements of the signal into more chunks does not come without a penalty, that is, as the number of chunks increases, the complexity of the model’s structure will be increased too. This is because increasing the number of chunks will result in increasing the number of extracted features from these chunks, and accordingly the number of model’s inputs will be increased linearly. Such a feature space with high dimensionality will become less reliable for shallow machine learning models. Hence, using deep learning models in such cases will be more efficient [11, 13].Regarding performance of the OC-SVM and PCA models, the OC-SVM model is superior to the PCA model when it is applied to a low number of chunks, whereas the PCA model outperformed the OC-SVM model only when the number of chunks is equal or greater than 4. This is due to the ability of the PCA model to find out the smallest feature subset in high dimensional space and to explore the characteristics of “normal” samples with this subset of features. On the other hand, this advantage of the PCA model does not exist in the support vector machines at all. However, all the used models are still fit to the electrical fault detection.To put all together, the proposed anomaly-based system improved the electrical fault detection, but it encounters an immediate challenge, namely, the increase of feature space which can be resolved by using deep learning methods instead of using shallow learners and harnessing feature selection algorithms to eliminate any irrelevant or redundant features [12]. Due to space limitation, only the accuracy, recall, FAR, and MCC evaluation metrics are presented in Figure 5. Figure 5 presents a visual comparison between the used models when the number of chunks varies. The performance of the used models is enhanced drastically as the number of chunks increases, in such a way that the accuracy, recall, and MCC values for all used models are increased as well as the FAR value is decreased in all cases. Another way to visually interpret the performance of the used models is to draw the critical difference diagram (CDD) [56]. Figure 6 depicts the CDD of the used models for all number of chunks. It is worthy to mention that the notation (Modelm) in Figure 6 indicates experiment of the Model with m chunks. The critical difference (CD) value, which is drawn above the figure as a bar, equals 7.4244.Figure 5
Comparison between some evaluation metrics of the used models. (a) Accuracy. (b) Recall. (c) FAR. (d) MCC.
(a)(b)(c)(d)Figure 6
Critical difference diagram of the used models for different number of chunks.Another critical issue in analyzing the performance of an electrical fault detection system is related to the ability of that system to detect faults regardless to type of faults. Indeed, there are mainly two types of faults in the electrical power system. Those are symmetrical and unsymmetrical faults [3]. Firstly, the symmetrical faults (also named as balanced faults) are considered as very serious but infrequent faults in the electrical power grids [6]. Specifically, the symmetrical faults come with two forms in the 3-phase grid, namely, the three lines to ground (L-L-L-G) and three lines (L-L-L) faults [8]. According to many practical studies, these faults are likely to occur with 2 to 5 percent in the entire electrical power system [7]. Although such faults rarely occur, their occurrence usually causes severe damage to both the electrical power system and equipment [7]. However, the entire electrical power system remains balanced even with that damage [7].On the other hand, the unsymmetrical faults, also called unbalanced faults, are predominant and less dangerous than the symmetrical faults [8]. Three forms of unsymmetrical faults could occur in the 3-phase electrical grid, namely, the line to ground (L-G), line to line (L-L), and double line to ground (L-L-G) faults [7]. It was reported that they are very common in the electrical power systems with a percentage of 65% to 70% for the line to ground faults, 15% to 20% for the double line to ground faults, and 5% to 10% for the line to line faults [3]. Even though they are safer to the electrical power system and equipment than symmetrical faults, they cause unbalancing in the entire system in such a way that they generate unbalanced current to flow in the 3 phases [6].Regarding the VSB dataset structure based on the fault types, there are 525 faulty samples: 4.5 percent of them are symmetrical faults, whereas 95.5 percent are unsymmetrical faults (line to ground = 69.15%, double line to ground = 18.74%, and line to line = 7.61%). This confirms the reliability of the VSB dataset, and it reflects the occurrence pattern of electrical faults in real-world electrical power grids. Figure7(a) shows the percentage of detection rate for each of symmetrical and unsymmetrical faults when using the proposed system or not. Obviously, using the proposed system enhanced detection of the symmetrical faults by 43.98% and the unsymmetrical faults by 32.01% compared to those when not using the proposed system. Furthermore, Figure 7(b) elaborates the impact of number of chunks in detecting the symmetrical and unsymmetrical faults. It can be perceived from Figure 7(b) that increasing the number of chunks helps both the OC-SVM and PCA models to detect more faults whether they are symmetrical or unsymmetrical. Therefore, the proposed system is effective considering detection of all types of electrical fault.Figure 7
Percentage of detection rate for the symmetrical and unsymmetrical faults when (a) using the proposed system or not and (b) using different number of chunks for each model.
(a)(b)


### 5.3. ROC Analysis

Alternatively, the ranking methods can be utilized to trade off between several models which are applied on the same dataset in order to choose the optimal classifier. One of widespread ranking methods is the receiver operating characteristic (ROC) curve. The ROC curve is a plot of the recall metric as a function of the FAR metric of a classifier [57]. The reference line with a model’s performance equal to 50% is the diagonal line of the ROC curve. Moreover, the classifier reaches 100% of the performance only if its ROC curve located on the top-left corner. Figure 8 shows the ROC curves of the used models per number of chunks.Figure 8
ROC curves of the used models for (a) 1 chunk, (b) 2 chunks, (c) 4 chunks, and (d) 8 chunks.
(a)(b)(c)(d)The area under the ROC curve (AUC) is a numerical measure that describes the corresponding ROC curve quantitatively [58]. The AUC value of a classifier is in [0,1], where the higher the AUC value, the better the performance of the classifier. Furthermore, the AUC value of a classifier can be computed approximately using equation (29) [59]. Table 5 presents the AUC values of the ROC curves which are plotted in Figure 8. Based on findings, all used models have convenient performance particularly when the number of chunks is higher than 2. Moreover, the impact of the signal decomposition process on enhancing the performance of the used anomaly-based detection models is evident, especially when the number of chunks equals 8 in such a way that the AUC values for all models are higher than 0.9.(29)AUC=12×recall+specificity.Table 5
AUC values of the ROC curves.
Number of chunksModelsAUC1OC-SVM0.800 7PCA0.778 42OC-SVM0.853 8PCA0.822 04OC-SVM0.890 7PCA0.910 88OC-SVM0.928 1PCA0.943 1The bold values provide the best results between the used models for each number of chunks.


## 5.1. Evaluation Metrics

The outcome of the testing phase is merely a binary classification process indicating whether a sample is “normal” or “faulty.” Hence, a confusion matrix will be constructed after the testing phase is finished. This confusion matrix has four cells which contain the following measures: true positive (TP), true negative (TN), false positive (FP), and false negative (FN). These measures are required to compute eight commonly used evaluation metrics, as follows.(i)
Accuracy is the ratio of the number of true classifications to the size of test set.(21)Accuracy=TP+TNTP+TN+FP+FN.(ii)
Precision is the ratio of the number of correctly classified faulty samples to all samples labeled as “faulty.”(22)Precision=TPTP+FP.(iii)
Recall is the ratio of the number of correctly classified faulty samples to all faulty samples. It is also named hit, true positive rate (TPR), detection rate (DR), or sensitivity.(23)Recall=TPTP+FN.(iv)
F1 score is a balanced metric that includes both the recall (R) and precision (P) values. It is also called F1 metric.(24)F1score=2×P×RP+R.(v)
False alarm rate (FAR) is the ratio of the number of normal samples that is wrongly classified as “faulty” to all normal samples. It is also named false positive rate (FPR).(25)FAR=FPFP+TN.(vi)
Specificity is the ratio of the number of correctly classified normal samples to all normal samples. It is also known as true negative rate (TNR).(26)Specificity=TNTN+FP.(vii)
False negative rate (FNR) is the ratio of the number of faulty samples that is wrongly classified as “normal” to all faulty samples.(27)FNR=FNFN+TP.(viii)
Matthews correlation coefficient (MCC) is a balanced measure that, when calculated, considers all major outcomes of classification. Usually, the MCC metric is useful in the case of using imbalanced datasets [54]. In addition to that, the MCC value can be in [−1,1], where -1 and 1 values indicate weak and perfect classifiers, respectively [55].(28)MCC=TP×TN−FP×FNTP+FN×TP+FP×TN+FP×TN+FN.


## 5.2. Performance Analysis

The evaluation metrics, mentioned in Section5.1, will be the key factors for assessing the performance of the used anomaly-based detection models in the electrical fault detection. Obviously, the higher the values of true classification metrics and the lower the values of misclassification metrics, the more effective the model. Table 4 shows the percentage of the evaluation metrics for each anomaly-based detection model in all experiments. Furthermore, the bold and italicized values in Table 4 represent the best results of the used models in the same experiment and among all experiments, respectively, whereas the “Normal” column in Table 4 refers to the results of the used models without using our proposed system, and the “Enhanced” columns refer to the results when using the proposed system.Table 4
Results of our empirical experiments.
ModelEvaluation metricNormalEnhancedNumber of chunks1248OC-SVMAccuracy75.6883.8187.7290.5194.42Precision64.8878.4582.7785.7794.30Recall55.2469.3378.6784.9588.19F1 score59.6773.6180.6685.3691.14FAR14.449.207.916.812.58Specificity85.5690.8092.0993.1997.42FNR44.7630.6721.3315.0511.81MCC42.7162.2471.7278.3487.18PCAAccuracy73.3982.1385.3092.6295.78Precision60.2176.2779.8890.2896.73Recall53.9065.5273.3386.6790.10F1 score56.8870.4976.4688.4493.29FAR17.209.848.924.511.47Specificity82.8090.1691.0895.4998.53FNR46.1034.4826.6713.339.90MCC37.8458.1365.9383.0590.34Clearly, the proposed anomaly-based fault detection system enhanced the performance of all models compared to same models without using the proposed system. This can be noticed when comparing the values of evaluation metrics of the used models in “Normal” and “Enhanced for 1 chunk” columns in Table4. For instance, the recall values are increased by 14%, and the FAR values are decreased by 8%. This can be explained by the impact of data preprocessing and pretraining phases that helps anomaly-based detection models to detect faults aggressively. Moreover, when the number of chunks is increased, the values of evaluation metrics for all models are improved significantly. For example, the recall and FAR values of the OC-SVM and PCA models for one chunk are (69.33%, 9.20%) and (65.52%, 9.84%), respectively. But when the number of chunks is increased to 8, they become (88.19%, 2.58%) and (90.10%, 1.47%).Unfortunately, improving the model’s performance by partitioning the voltage measurements of the signal into more chunks does not come without a penalty, that is, as the number of chunks increases, the complexity of the model’s structure will be increased too. This is because increasing the number of chunks will result in increasing the number of extracted features from these chunks, and accordingly the number of model’s inputs will be increased linearly. Such a feature space with high dimensionality will become less reliable for shallow machine learning models. Hence, using deep learning models in such cases will be more efficient [11, 13].Regarding performance of the OC-SVM and PCA models, the OC-SVM model is superior to the PCA model when it is applied to a low number of chunks, whereas the PCA model outperformed the OC-SVM model only when the number of chunks is equal or greater than 4. This is due to the ability of the PCA model to find out the smallest feature subset in high dimensional space and to explore the characteristics of “normal” samples with this subset of features. On the other hand, this advantage of the PCA model does not exist in the support vector machines at all. However, all the used models are still fit to the electrical fault detection.To put all together, the proposed anomaly-based system improved the electrical fault detection, but it encounters an immediate challenge, namely, the increase of feature space which can be resolved by using deep learning methods instead of using shallow learners and harnessing feature selection algorithms to eliminate any irrelevant or redundant features [12]. Due to space limitation, only the accuracy, recall, FAR, and MCC evaluation metrics are presented in Figure 5. Figure 5 presents a visual comparison between the used models when the number of chunks varies. The performance of the used models is enhanced drastically as the number of chunks increases, in such a way that the accuracy, recall, and MCC values for all used models are increased as well as the FAR value is decreased in all cases. Another way to visually interpret the performance of the used models is to draw the critical difference diagram (CDD) [56]. Figure 6 depicts the CDD of the used models for all number of chunks. It is worthy to mention that the notation (Modelm) in Figure 6 indicates experiment of the Model with m chunks. The critical difference (CD) value, which is drawn above the figure as a bar, equals 7.4244.Figure 5
Comparison between some evaluation metrics of the used models. (a) Accuracy. (b) Recall. (c) FAR. (d) MCC.
(a)(b)(c)(d)Figure 6
Critical difference diagram of the used models for different number of chunks.Another critical issue in analyzing the performance of an electrical fault detection system is related to the ability of that system to detect faults regardless to type of faults. Indeed, there are mainly two types of faults in the electrical power system. Those are symmetrical and unsymmetrical faults [3]. Firstly, the symmetrical faults (also named as balanced faults) are considered as very serious but infrequent faults in the electrical power grids [6]. Specifically, the symmetrical faults come with two forms in the 3-phase grid, namely, the three lines to ground (L-L-L-G) and three lines (L-L-L) faults [8]. According to many practical studies, these faults are likely to occur with 2 to 5 percent in the entire electrical power system [7]. Although such faults rarely occur, their occurrence usually causes severe damage to both the electrical power system and equipment [7]. However, the entire electrical power system remains balanced even with that damage [7].On the other hand, the unsymmetrical faults, also called unbalanced faults, are predominant and less dangerous than the symmetrical faults [8]. Three forms of unsymmetrical faults could occur in the 3-phase electrical grid, namely, the line to ground (L-G), line to line (L-L), and double line to ground (L-L-G) faults [7]. It was reported that they are very common in the electrical power systems with a percentage of 65% to 70% for the line to ground faults, 15% to 20% for the double line to ground faults, and 5% to 10% for the line to line faults [3]. Even though they are safer to the electrical power system and equipment than symmetrical faults, they cause unbalancing in the entire system in such a way that they generate unbalanced current to flow in the 3 phases [6].Regarding the VSB dataset structure based on the fault types, there are 525 faulty samples: 4.5 percent of them are symmetrical faults, whereas 95.5 percent are unsymmetrical faults (line to ground = 69.15%, double line to ground = 18.74%, and line to line = 7.61%). This confirms the reliability of the VSB dataset, and it reflects the occurrence pattern of electrical faults in real-world electrical power grids. Figure7(a) shows the percentage of detection rate for each of symmetrical and unsymmetrical faults when using the proposed system or not. Obviously, using the proposed system enhanced detection of the symmetrical faults by 43.98% and the unsymmetrical faults by 32.01% compared to those when not using the proposed system. Furthermore, Figure 7(b) elaborates the impact of number of chunks in detecting the symmetrical and unsymmetrical faults. It can be perceived from Figure 7(b) that increasing the number of chunks helps both the OC-SVM and PCA models to detect more faults whether they are symmetrical or unsymmetrical. Therefore, the proposed system is effective considering detection of all types of electrical fault.Figure 7
Percentage of detection rate for the symmetrical and unsymmetrical faults when (a) using the proposed system or not and (b) using different number of chunks for each model.
(a)(b)


## 5.3. ROC Analysis

Alternatively, the ranking methods can be utilized to trade off between several models which are applied on the same dataset in order to choose the optimal classifier. One of widespread ranking methods is the receiver operating characteristic (ROC) curve. The ROC curve is a plot of the recall metric as a function of the FAR metric of a classifier [57]. The reference line with a model’s performance equal to 50% is the diagonal line of the ROC curve. Moreover, the classifier reaches 100% of the performance only if its ROC curve located on the top-left corner. Figure 8 shows the ROC curves of the used models per number of chunks.Figure 8
ROC curves of the used models for (a) 1 chunk, (b) 2 chunks, (c) 4 chunks, and (d) 8 chunks.
(a)(b)(c)(d)The area under the ROC curve (AUC) is a numerical measure that describes the corresponding ROC curve quantitatively [58]. The AUC value of a classifier is in [0,1], where the higher the AUC value, the better the performance of the classifier. Furthermore, the AUC value of a classifier can be computed approximately using equation (29) [59]. Table 5 presents the AUC values of the ROC curves which are plotted in Figure 8. Based on findings, all used models have convenient performance particularly when the number of chunks is higher than 2. Moreover, the impact of the signal decomposition process on enhancing the performance of the used anomaly-based detection models is evident, especially when the number of chunks equals 8 in such a way that the AUC values for all models are higher than 0.9.(29)AUC=12×recall+specificity.Table 5
AUC values of the ROC curves.
Number of chunksModelsAUC1OC-SVM0.800 7PCA0.778 42OC-SVM0.853 8PCA0.822 04OC-SVM0.890 7PCA0.910 88OC-SVM0.928 1PCA0.943 1The bold values provide the best results between the used models for each number of chunks.


## 6. Discussion

In this section, the obtained results in Section5 using several statistical and comparative analyses are deeply discussed.


### 6.1. Stability and Sensitivity Analyses

To prove the stability of the proposed system and consistency of the results, some statistical tests can be performed. Due to space limitation, only the Friedman test is applied on the results in Table4. Friedman test is a renowned statistical test which seeks to discover the distinctions between several repeated treatments [60]. The Friedman test has many advantages such as its simplicity and generality, and it is a nonparametric test that assumes that your data do not come from a specific distribution. Practically, in Table 4, there are four experiments related to different number of chunks. In addition to that, in each experiment, there are two subjects related to the used models. The Friedman test assumes in its null hypothesis that all these experiments have identical effects. To reject the null hypothesis, two main conditions must be satisfied. The first is that the critical value FC is less than the calculated statistic FS. The second condition is that the significance level value (α) is larger than the calculated probability value P_value. In this study, the α value is selected to be 0.05 because it is very common. Table 6 shows the results of the Friedman test when it is applied on TP, TN, FN, and FP outcomes. From Table 6, the null hypothesis of the Friedman test is rejected because the two conditions are satisfied in all cases (FC <FS and α >P_value). Therefore, the outcomes of the used models are significant and different from each other.Table 6
Results of the Friedman test.
OutcomeFCFSαP_valueTN680.050.001 56FP680.050.001 56TP680.050.001 56FN680.050.001 56Sensitivity analysis is often applied in classification tasks to specify the relationship between input (independent) and target (dependent) variables under a given set of assumptions [61]. Due to space limitations, only the number of chunks as an input variable is analyzed to show its influence on the recall value of the used models as an output. Indeed, the sensitivity analysis can be done using various approaches, but the one-at-a-time (OAT) analysis is the most common approach. In the first step in the OAT analysis, the base case of the models is defined which in this study is the recall values of the used models with one chunk. Afterwards, the recall values of the used models will be calculated for different number of chunks, leaving all other assumptions unchanged. Finally, the sensitivity statistics will be calculated using the following formula [62].(30)Sensitivity statistic=%changein the output variable%changein the input variable.The higher the sensitivity statistic is, the more sensitive the recall is to changes in the number of chunks. Table7 presents the results of the OAT sensitivity analysis when the base case is one chunk. From Table 7, the recall value of the used anomaly detection models is sensitive to the number of chunks and it significantly increases as the number of chunks increases.Table 7
Results of the OAT sensitivity analysis.
Number of chunksSensitivity statistic (%)OC-SVMPCA1−−213.4711.92422.5332.28827.2037.52


### 6.2. Feature Selection Methods

In this section, the impact of performing feature selection process in the data preprocessing phase is examined. Some recent feature selection methods such as the fitness proportionate selection binary particle swarm optimization and entropy (FPSBPSO-E) [12, 63, 64], stochastic fractal search-based guided whale optimization algorithm (SFS-guided WOA) [65], hybrid of grey wolf optimization and particle swarm optimization (GWO-PSO) [66], hybrid of grey wolf optimization and genetic algorithm (GWO-GA) [66], biogeography-based optimizer (BBO) [67], firefly algorithm (FA) [68], and satin bowerbird optimizer (SBO) [69] are compared in terms of the mean of AUC and feature reduction rate (FRR) [12] metrics. The FRR metric is the complement of the ratio of selected features to all feature set and can be calculated as follows [12].(31)FRR=1−number of selected featuresnumber of all features.The experiment is conducted on number of chunks equal to 1, that is, the size of full feature set is 20. Furthermore, the anomaly-based detection models are trained and tested on the feature subsets which are generated by the feature selection methods. Table8 presents the results of feature selection process when using different methods. It can be perceived that the FPSBPSO-E method not only is better than other feature selection methods in terms of performance but also selected the smallest feature subset.Table 8
Results of using some feature selection methods in the pretraining phase.
Feature selection methodNumber of selected featuresFRR (%)AUC (%)SFS-guided WOA143085.56GWO-PSO162083.73GWO-GA152584.82BBO171582.65FA181081.97SBO19581.22FPSBPSO-E133586.11The bold values provide the best results among all the used feature selection methods.


### 6.3. Hyperparameter Optimization Methods

The performance of the used PSO-based hyperparameter optimization method is evaluated by comparing it with other popular optimization algorithms such as the original genetic algorithm (GA) [70], grasshopper optimization algorithm (GOA) [71], whale optimization algorithm (WOA) [72], grey wolf optimization (GWO) [73], bat algorithm (BA) [74], and multiverse optimization (MVO) [75] in terms of the mean AUC values of the anomaly-based detection models. Table 9 shows the results of hyperparameter optimization process when using different algorithms. Clearly, the used PSO-based algorithm for hyperparameter selection outperformed other optimization algorithms.Table 9
Results of using some hyperparameter selection algorithms in the pretraining phase.
Optimization algorithmAUC (%)GA78.39GOA76.42WOA78.15GWO79.05BA75.54MVO77.70PSO80.76The bold values provide the best results among all the used hyperparameter selection algorithms.


### 6.4. Anomaly-Based Models vs. Binary Models

This section is dedicated to investigate performance differences between some binary classification models and the used anomaly-based detection models in the electrical fault detection problem. The binary classification models such as the ANN [45], support vector machine (SVM) [45], Naive Bayes (NB) [76], boosted decision tree (BDT) [77], decision forest (DF) [78], decision jungle (DJ) [79], and quantum support vector machine (QSVM) [80] are utilized without using the proposed fault detection system. Then, their performance is compared to performance of the OC-SVM and PCA anomaly-based detection models using the proposed system. Table 10 presents the results of binary classification and anomaly-based detection models in terms of the AUC metric. It can be noticed that the anomaly-based detection models are superior to the binary classification models due to the cooperated mechanisms in the proposed fault detection system.Table 10
Comparison between some binary classification and anomaly-based detection models (one chunk).
Detection methodModelAUC (%)Binary classificationANN52.16SVM60.57NB57.99BDT61.23DF65.55DJ68.06QSVM70.31Anomaly-based modelsOC-SVM80.07PCA77.84The bold value provides the best result among all used models.


### 6.5. Comparison with Related Works

As explained in Section2, most of the previous studies in electrical fault detection used simulated datasets which cannot reflect the pattern of electrical faults in the real-world electrical power grids. Our proposed anomaly-based fault detection system is compared only with two studies [34, 36], since they used the same real-time dataset (VSB dataset) in their works. Thus, the readers can compare the results easily. Table 11 presents a comparison between our findings and those of related works. Obviously, the proposed system enhanced the performance of all anomaly-based detection models compared to the models in the studies [34, 36]. This confirms the effectiveness of the proposed fault detection system in the electrical fault detection problem.Table 11
Comparison with some related works.
StudyTechniqueAccuracyPrecisionRecallF1 score[34]STL + SVM (linear kernel)−0.720.70.7STL + SVM (polynomial kernel with degree 6)−0.670.550.45STL + SVM (RBF kernel)−0.750.730.72STL + SVM (sigmoid kernel)−0.530.520.5[36]Anomaly detection + SVM (RBF kernel)0.798 40.698 40.670 50.684 2Anomaly detection + PCA0.792 80.667 30.725 70.695 3Our studyAnomaly detection + PSO + OC-SVM0.944 20.943 00.881 90.911 4Anomaly detection + PSO + PCA0.957 80.967 30.901 00.932 9The bold values provide the best results among all models.


## 6.1. Stability and Sensitivity Analyses

To prove the stability of the proposed system and consistency of the results, some statistical tests can be performed. Due to space limitation, only the Friedman test is applied on the results in Table4. Friedman test is a renowned statistical test which seeks to discover the distinctions between several repeated treatments [60]. The Friedman test has many advantages such as its simplicity and generality, and it is a nonparametric test that assumes that your data do not come from a specific distribution. Practically, in Table 4, there are four experiments related to different number of chunks. In addition to that, in each experiment, there are two subjects related to the used models. The Friedman test assumes in its null hypothesis that all these experiments have identical effects. To reject the null hypothesis, two main conditions must be satisfied. The first is that the critical value FC is less than the calculated statistic FS. The second condition is that the significance level value (α) is larger than the calculated probability value P_value. In this study, the α value is selected to be 0.05 because it is very common. Table 6 shows the results of the Friedman test when it is applied on TP, TN, FN, and FP outcomes. From Table 6, the null hypothesis of the Friedman test is rejected because the two conditions are satisfied in all cases (FC <FS and α >P_value). Therefore, the outcomes of the used models are significant and different from each other.Table 6
Results of the Friedman test.
OutcomeFCFSαP_valueTN680.050.001 56FP680.050.001 56TP680.050.001 56FN680.050.001 56Sensitivity analysis is often applied in classification tasks to specify the relationship between input (independent) and target (dependent) variables under a given set of assumptions [61]. Due to space limitations, only the number of chunks as an input variable is analyzed to show its influence on the recall value of the used models as an output. Indeed, the sensitivity analysis can be done using various approaches, but the one-at-a-time (OAT) analysis is the most common approach. In the first step in the OAT analysis, the base case of the models is defined which in this study is the recall values of the used models with one chunk. Afterwards, the recall values of the used models will be calculated for different number of chunks, leaving all other assumptions unchanged. Finally, the sensitivity statistics will be calculated using the following formula [62].(30)Sensitivity statistic=%changein the output variable%changein the input variable.The higher the sensitivity statistic is, the more sensitive the recall is to changes in the number of chunks. Table7 presents the results of the OAT sensitivity analysis when the base case is one chunk. From Table 7, the recall value of the used anomaly detection models is sensitive to the number of chunks and it significantly increases as the number of chunks increases.Table 7
Results of the OAT sensitivity analysis.
Number of chunksSensitivity statistic (%)OC-SVMPCA1−−213.4711.92422.5332.28827.2037.52


## 6.2. Feature Selection Methods

In this section, the impact of performing feature selection process in the data preprocessing phase is examined. Some recent feature selection methods such as the fitness proportionate selection binary particle swarm optimization and entropy (FPSBPSO-E) [12, 63, 64], stochastic fractal search-based guided whale optimization algorithm (SFS-guided WOA) [65], hybrid of grey wolf optimization and particle swarm optimization (GWO-PSO) [66], hybrid of grey wolf optimization and genetic algorithm (GWO-GA) [66], biogeography-based optimizer (BBO) [67], firefly algorithm (FA) [68], and satin bowerbird optimizer (SBO) [69] are compared in terms of the mean of AUC and feature reduction rate (FRR) [12] metrics. The FRR metric is the complement of the ratio of selected features to all feature set and can be calculated as follows [12].(31)FRR=1−number of selected featuresnumber of all features.The experiment is conducted on number of chunks equal to 1, that is, the size of full feature set is 20. Furthermore, the anomaly-based detection models are trained and tested on the feature subsets which are generated by the feature selection methods. Table8 presents the results of feature selection process when using different methods. It can be perceived that the FPSBPSO-E method not only is better than other feature selection methods in terms of performance but also selected the smallest feature subset.Table 8
Results of using some feature selection methods in the pretraining phase.
Feature selection methodNumber of selected featuresFRR (%)AUC (%)SFS-guided WOA143085.56GWO-PSO162083.73GWO-GA152584.82BBO171582.65FA181081.97SBO19581.22FPSBPSO-E133586.11The bold values provide the best results among all the used feature selection methods.


## 6.3. Hyperparameter Optimization Methods

The performance of the used PSO-based hyperparameter optimization method is evaluated by comparing it with other popular optimization algorithms such as the original genetic algorithm (GA) [70], grasshopper optimization algorithm (GOA) [71], whale optimization algorithm (WOA) [72], grey wolf optimization (GWO) [73], bat algorithm (BA) [74], and multiverse optimization (MVO) [75] in terms of the mean AUC values of the anomaly-based detection models. Table 9 shows the results of hyperparameter optimization process when using different algorithms. Clearly, the used PSO-based algorithm for hyperparameter selection outperformed other optimization algorithms.Table 9
Results of using some hyperparameter selection algorithms in the pretraining phase.
Optimization algorithmAUC (%)GA78.39GOA76.42WOA78.15GWO79.05BA75.54MVO77.70PSO80.76The bold values provide the best results among all the used hyperparameter selection algorithms.


## 6.4. Anomaly-Based Models vs. Binary Models

This section is dedicated to investigate performance differences between some binary classification models and the used anomaly-based detection models in the electrical fault detection problem. The binary classification models such as the ANN [45], support vector machine (SVM) [45], Naive Bayes (NB) [76], boosted decision tree (BDT) [77], decision forest (DF) [78], decision jungle (DJ) [79], and quantum support vector machine (QSVM) [80] are utilized without using the proposed fault detection system. Then, their performance is compared to performance of the OC-SVM and PCA anomaly-based detection models using the proposed system. Table 10 presents the results of binary classification and anomaly-based detection models in terms of the AUC metric. It can be noticed that the anomaly-based detection models are superior to the binary classification models due to the cooperated mechanisms in the proposed fault detection system.Table 10
Comparison between some binary classification and anomaly-based detection models (one chunk).
Detection methodModelAUC (%)Binary classificationANN52.16SVM60.57NB57.99BDT61.23DF65.55DJ68.06QSVM70.31Anomaly-based modelsOC-SVM80.07PCA77.84The bold value provides the best result among all used models.


## 6.5. Comparison with Related Works

As explained in Section2, most of the previous studies in electrical fault detection used simulated datasets which cannot reflect the pattern of electrical faults in the real-world electrical power grids. Our proposed anomaly-based fault detection system is compared only with two studies [34, 36], since they used the same real-time dataset (VSB dataset) in their works. Thus, the readers can compare the results easily. Table 11 presents a comparison between our findings and those of related works. Obviously, the proposed system enhanced the performance of all anomaly-based detection models compared to the models in the studies [34, 36]. This confirms the effectiveness of the proposed fault detection system in the electrical fault detection problem.Table 11
Comparison with some related works.
StudyTechniqueAccuracyPrecisionRecallF1 score[34]STL + SVM (linear kernel)−0.720.70.7STL + SVM (polynomial kernel with degree 6)−0.670.550.45STL + SVM (RBF kernel)−0.750.730.72STL + SVM (sigmoid kernel)−0.530.520.5[36]Anomaly detection + SVM (RBF kernel)0.798 40.698 40.670 50.684 2Anomaly detection + PCA0.792 80.667 30.725 70.695 3Our studyAnomaly detection + PSO + OC-SVM0.944 20.943 00.881 90.911 4Anomaly detection + PSO + PCA0.957 80.967 30.901 00.932 9The bold values provide the best results among all models.


## 7. Conclusion

Effective fault detection in electrical power industry will lead to maximizing service quality as well as minimizing economic losses. Therefore, an anomaly-based fault detection system is proposed to cope with drawbacks and limitations of the existing electrical fault detection systems. The proposed system consists of four phases, namely, data preprocessing, pretraining, training, and testing. Most of the work is done in the first two phases. The data preprocessing phase prepares raw data of the signals by performing signal denoising, signal decomposition, feature extraction, data normalization, and dataset splitting. On the other hand, the pretraining phase selects the optimal hyperparameters needed by the anomaly-based fault detection model using a PSO-based algorithm. Moreover, two anomaly-based fault detection models leveraged from the proposed system are validated on the VSB dataset. The experimental results show a significant improvement in electrical fault detection which increased the true classification metrics such as the accuracy and recall metrics by 14% as well as decreased the misclassification metrics such as the FAR metric by 8% compared to same models without using the proposed system. Based on our results, the OC-SVM model is better than the PCA model only in the case of a low number of chunks, and vice versa. However, increasing the number of chunks led to increase in the feature space that weakened the used models. Thus, using the proposed system with deep learning models for further enhancement in electrical fault detection is recommended as a possible future work.



---

*Source: 1870136-2022-02-14.xml*