# Transmission Line Fault Classification Using Conformer Convolution-Augmented Transformer Model
## 1 Introduction
This approach not only streamlines the workflow but also elevates the model's adaptability across diverse fault scenarios, ensuring its efficacy in real-world applications without the need of expert knowledge for feature extraction. Applying the model to real-world data introduces numerous challenges, particularly concerning the diverse sampling rates of input data, where different types of relays each operate at specific sampling rates. In Taiwan, these rates range from 4 to 188.5 samples per cycle. In contrast, simulation-based studies typically utilize consistent sampling rates, such as 20 kHz (333.33 samples per cycle for 60 Hz or 400 samples per cycle for 50 Hz) [3; 5; 7], 10 kHz (200 samples per cycle for 50 Hz) [8  
Notably, Model III, which simplifies input data to lower sampling rates, demonstrates that detailed high-frequency information may have minimal impact on fault classification. This finding suggests that for our specific classification tasks, focusing on lower frequency data could be sufficient, challenging the conventional emphasis on high-frequency analysis. The existing literature has proposed a repertoire of machine learning models on transmission line fault classification, encompassing the Artificial Neural Network (ANN) [12], Extreme Learning Model (ELM) [3], Radial Basis Function (RBF) neural network [13], Convolutional Neural Network (CNN) [7; 8], Long Short-Term Memory (LSTM) [11], self-attention CNN (SAT-CNN) [10], Support Vector Machine (SVM) [14; 16], and Decision Tree (DT) [9]. Furthermore, ensemble techniques have been extensively employed across various domains utilizing ML-based methodologies [17; 18], and similarly, transmission line fault classification also utilizes these techniques [19]. In this study, our aim is to develop a model distinguished by its exceptional generalizability, thus significantly reducing the reliance on expert feature extraction. Therefore, our study utilizes the Conformer Convolution-Augmented Transformer model, known for integrating local and global information efficiently. The Conformer has the ability to outperform LSTM and other Recurrent Neural Networks (RNNs), which are two common models used for handling time-relevant input data, in terms of classification accuracy with the same set of parameters [20], which is analogous to the three-phase voltage and current signals used in our study. In addition, the Conformer achieves higher accuracy in a similar training time compared to LSTM and other RNNs [20]. Notably, in Automatic Speech Recognition (ASR), the Conformer model has demonstrated its effectiveness by surpassing the performance of previous Transformer- and CNN-based models, achieving state-of-the-art accuracies [21]. The main contributions of this study are listed below:  
* Our ML-based model can classify time series inputs without the need for expert feature extraction. * The window selection for fault classification must encompass pre-fault and fault states. Nonetheless, we have observed that some historical event data also include post-fault states. Consequently, our model is designed to successfully classify faults using data that either include or exclude post-fault states. * Resampling the data using frequency domain information provides several advantages, including handling non-integer sampling rates, benefiting from batch normalization, and improving efficiency. * Resampling the data to four samples per cycle is sufficient for fault classification, even if the resulting signal may not perfectly resemble a sine wave. This finding suggests that for our specific classification tasks, focusing on lower frequency data could be sufficient, challenging the conventional emphasis on high-frequency analysis. * We conduct a comparative analysis between our method and the ELM described in [3], which has been compared with 11 state-of-the-art approaches spanning from 2015 to 2022.