# High-impedance Fault Detection in DC Microgrid Lines Using Open-Set Recognition
## 3 Fault Detection
### Open-Set Recognition
#### 3.2.2 Open-Set Recognition Using NNs  
The same problem persists in classification with neural networks, as samples from unknown classes are often classified with high confidence [37]. The reason for this behavior is due to the closed-set nature of NNs, i.e., the input sample is always assigned to one of the known classes. Since the final layer of the NN is softmax, the probability of the input belonging to each class is determined and the one with the highest probability is selected. However, it has been shown that there are \"fooling\" inputs that can lead to a high score in the classification even though they are significantly different from the rest of the samples belonging to that class [49]. Much worse, adversarial samples can also be created. These samples visually resemble those in the training dataset but result in high confidence output that is incorrect. To address this issue, several approaches have been proposed [37]. However, one of the recent approaches that became very popular shows that if a stronger classifier is used, its open-set performance is on par with other OSR approaches. The authors of [50] concluded that a classifier's open-set performance correlates positively with its closed-set performance across different datasets, model architectures, and objectives. However, there are certain additions to the classifier's training procedure that allow for such performance improvement. The first is to augment the dataset with more data, as this increases the accuracy. The same effect is achieved by warming up the learning rate and smoothing the labels. The former helps with numerical instabilities at the beginning of the training, since the parameters (weights and biases) are random values that can be far from their optimal values. Hence, the learning process is started with a learning rate lower than the initial one (\(\eta\)) and gradually increased until it equals the initial rate [51]. In this work, a linear increase is used. Since the one-hot encoded representation of the label vector \(\mathbf{y}\) is equal to 1 at the location \(i=y\) and 0 otherwise, label smoothing affects the labels by decreasing the value of the true label for a small constant \(\epsilon\) and increasing the other element values to \(\epsilon/(K-1)\), where \(K\) is the number of classes:  
\[\hat{\mathbf{y}}_{i}= \begin{cases}1-\epsilon,&\text{if }i=y,\\ \epsilon/(K-1),&\text{otherwise}.\end{cases}  \tag{8}\]  
This leads to a change in the optimal solution of the loss function:  
\[\mathbf{z}_{i}^{*}= \begin{cases}\log((K-1)(1-\epsilon)/\epsilon)+\alpha,& \text{if }i=y,\\ \alpha,&\text{otherwise},\end{cases}  \tag{9}\]  
where \(\alpha\) is any real number. Constant \(\epsilon\) affects the solution by making it finite, which is shown to increase the generalization capability [52]. It is also beneficial to train the model longer as compared to the normal setting by using the cosine learning rate decay. This decay reduces the learning rate from the initial value to zero according to the cosine law, which benefits the training progress [52]. The expression for the decay is as follows:  
\[\eta_{t}=\frac{1}{2}\bigg{(}1+\cos\bigg{(}\frac{t\pi}{T}\bigg{)}\bigg{)}\eta, \tag{10}\]  
where \(\eta_{t}\) is the learning rate at epoch \(t\), \(T\) is the total number of epochs, and \(\eta\) is the initial learning rate. Finally, the maximum logit score (MLS) is used as an open-set indicator instead of the normalized softmax score (note that MLS is used as an indicator, but softmax is used during the training and testing phase of the classifier). Logits are outputs of the penultimate layer of the NN that are given as inputs to the softmax layer. When they are normalized, the information about their magnitude, which proves to be a good indicator, is lost. Therefore, open-set detection is performed directly on logits, and in this case specifically on the logit with the highest value.