# Unsupervised Machine Learning Techniques for Detecting PLC Process Control Anomalies
## 4 Proposed Method
### OCSVM-Based Detection Approach  
[PERSON] et al. [43] proposed OCSVM, a maximum-based classifier established on support vector machines. The OCSVM is an unsupervised anomaly-detection algorithm that learns a decision function for separating the normal class from the anomalies [44]. Given a training dataset \(\{X_{i}|i=1,2,3\cdots,n\}\) where \(X_{i}\in R^{d}\), the OCSVM separates the data points from the origin in the feature space by a hyperplane and maximizes the distance from the hyperplane to the origin. OCSVM finds a decision function \(f_{F}\) that separates the data points into positive and negative scores. The positive scores represent the region in the feature space where \(X_{i}\in F\), and \(F\) is the set that carries a high concentration of the data points, also known as the minimum-volume set. The negative scores represent all other data points or anomalies. High dimensional Hilbert space \(H\), can be used to transform each data point \(X_{i}\) via a feature map \(\Phi:R_{d}\gets H\) generated by a positive-definite kernel, \(k(X,X^{\prime})\). The optimization problem for separating the data from the origin in the OCSVM is therefore given by  
\[\begin{split}\min_{w,b}&\frac{1}{2}\|w\|^{2}-\frac {C}{N}\sum_{n=1}^{N}\xi-b\\ s.t:&\langle w,\Phi(X_{i})\rangle\geq b-\xi_{i}, \forall i,\\ \xi_{i}&\geq 0,\forall i\end{split} \tag{1}\]  
where \(b\) is the variable that controls the algorithm's bias. The optimization problem is formulated such that \(w\cdot\phi(X)-b\) is positive for as many \(N\) training examples as possible. The \(C\) value is a hyperparameter that serves as the differential weight of the normal data points compared to the anomalous data points. The value, \(\
u=1/C\) is regarded as the prior probability that a data point in the training set is an anomaly, thereby regulating the trade-off between false positives and false negatives in the model. The slack variable \(\xi\) allows some data points in a nonseparable dataset to be within the margin. As a result, for the given data \(X\), the decision function \(f_{F}(X_{n:})\) is  
\[f_{F}(X_{n:})=w^{T}\Phi(X_{n:})-b \tag{2}\]  
The function definition in (2) is responsible for separating the data points from the origin by determining whether a point is in the positive or negative set. The width of the margin is controlled by \(b\in[0,1]\) and \(w\) is the normal vector of the hyperplane. The input data is projected into a nonlinear high-dimensional space by \(\Phi(X_{n:})\), and the slack variable \(\xi\) models the separation errors in the same way as the feature space of (1). Therefore, the overall OCSVM objective function is  
\[\min_{w,b}\frac{1}{2}\|w\|^{2}+\frac{1}{\
u N}\sum_{n=1}^{N}\max(0,b-\langle w,\Phi(X_{n:})\rangle)-b \tag{3}\]  
While the literature reports different variations of OCSVM, this work presents an OCSVM model that is developed by using the same model parameters in [13] to serve as a baseline upon which our proposed methods could be compared. OCSVM model learning process is controlled by using hyperparameters. Table 2 shows the hyperparameters for the OCSVM. According to [13], the modeling parameters in Table 2 were  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Dataset** & **No. of Records** & **\% Anomalies** \\ \hline Training set & 41,580 & n/a \\ Test Set 1 & 5000 & 10 \\ Test Set 2 & 7000 & 10 \\ Test Set 3 & 13,130 & 20 \\ Test Set 4 & 15,000 & 30 \\ Test Set 5 & 18,270 & 50 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Number of records and proportion of anomalies in training and test data sets.  
selected as optimal hyper-parameters for the OCSVM algorithm after investigating various hyperparameter ranges.