# Unsupervised Machine Learning Techniques for Detecting PLC Process Control Anomalies
## 4 Proposed Method
### Isolation Forest-Based Detection Approach  
Isolation forest (IF) is an unsupervised learning technique that builds binary trees ensemble for a given dataset for anomaly detection [48, 49, 50] IF assumes that anomalies make up the minority of a given dataset. As a result, anomalies have attribute values that are different from the normal instances. IF uses several isolation trees and trains each tree on a subset of the training dataset. IF uses the following parameters for constructing the binary trees:  
1. Total number of isolation trees \((n_{t})\);
2. Sample size of training data subset used to train each isolation tree \((n_{max})\);
3. Maximum number of features representing a subset of the data features used to train each tree \((f_{max})\). Algorithm 4 summarizes the IF algorithm training process. During training, IF recursively partitions the training data with an axis-parallel cut at randomly chosen partition points in randomly selected attributes. Next, IF isolates the partitioned instances into nodes with fewer and fewer instances until the points are isolated into singleton nodes containing one instance [48]. IF randomly selects attributes splits \(q\) and a split subset \(p\) within a specified range, resulting in a left \((X_{i})\) and right \((X_{r})\) subsets of the data each time \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Activation Function** & \(\mathbf{\
u}\) & **Learning Rate** & **No. of Hidden Layers** \\ \hline ReLU & 0.04 & 0.0001 & 32 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Model hyperparameters for OCNN. until all training samples are isolated into singleton nodes. Algorithm 5 summarizes the recursive binary splitting concept for separating anomalies by IF. ```
Input:\(X\)--input data, \(n_{t}\)--number of trees, \(n_{max}\)--sub-sampling size, \(f_{max}\)--attributes of data subset Output: a set of \(n_{t}\) iTrees  Initialize Forest for(\(i=1\) to \(n_{t}\))do \(X^{\prime}\)\(\leftarrow\)\(sample(X,f_{max},n_{max})\) \(Forest\)\(\leftarrow\)\(Forest\cup iTree(X^{\prime})\) end return Forest
```  
**Algorithm 4**Train \(IF(X,n_{t},n_{max},f_{max})\). Algorithm 5 shows that after each split, isolation tree \((iTree)\) produces a node which is either an internal node \((inNode)\) or external node \((exNode)\) depending on whether there is a further possibility of splitting the former into subsequent split regions. Consequently, the two internal node subsets (\(X_{l}\) and \(X_{r}\)) are split further until they reach an external node. External nodes are considered as leaves of branches when the maximum tree depth is reached or the last nodes in branches when the data subset size of the region is one. ```
Input:\(X\)--input data, \(n_{t}\)--number of trees, \(n_{max}\)--sub-sampling size, \(f_{max}\)--attributes of data subset Output: an iTrees if\(X^{\prime}\) is a singleton node then  return \(exNode\{Size\)\(\leftarrow\)\(|X|\}\); else  let \(Q\) be a list of attributes in \(X\)  randomly select an attribute \(q\in Q\) randomly select a split point \(p\) from \(max\)  and \(min\) values of attributes \(q\) in \(X^{\prime}\) \(X_{l}\)\(\leftarrow\)\(filter(X^{\prime},q<p)\) \(X_{r}\)\(\leftarrow\)\(filter(X,q\geq p)\)  return \(inNode\{Left\)\(+\)\(iTree(X_{l})\), \(Right\)\(\leftarrow\)\(iTree(X_{r}),SplitAttribute\)\(\leftarrow\)\(q,SplitValue\)\(\leftarrow\)\(p\)  end if
```  
**Algorithm 5**Train \(iTree(X^{\prime})\). Anomalous events are considerably different from the normal data points, and so the smaller paths in the isolation tree construction correspond to the lower dimensionality of the subspaces in which the anomalies have been isolated. IF works under the implicit assumption that it is more likely to isolate subspaces of lower dimensionality created by random splits [48].