## 6 Conclusions  
In this article, we presented a detailed entropy analysis using network traffic features followed by a classifier-based analysis to detect anomalies in industrial control systems. We based our work on Modbus over TCP/IP data collected from an HVAC system at the University of Alabama Huntsville research lab. The data had both malicious and non-malicious traffic representing various stages of the HVAC operations (auto cool, auto heat, normal, and random). Of the various attacks that were simulated, we chose the top three that have been discussed in the literature, namely Denial of Service (DOS), Man-in \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & \multicolumn{6}{c}{**DOS: No Resampling**} \\ \hline  & **Accuracy** & **Recall** & **FPR** & **Precision** & **F-Measure** & **ROC** \\ \hline BayesNet & 97.54\% & 97.50\% & 17.00\% & 99.30\% & 98.39\% & 95.30\% \\ Naive Bayes & 96.72\% & 96.70\% & 17.50\% & 99.30\% & 97.98\% & 90.40\% \\ J48 & 99.23\% & 99.20\% & 18.40\% & 99.30\% & 99.25\% & 92.70\% \\ Simple Logistic & 97.03\% & 97.00\% & 57.20\% & 97.90\% & 97.45\% & 75.20\% \\ SVM & 96.83\% & 96.80\% & 59.00\% & 97.80\% & 97.30\% & 68.90\% \\ Multilayer Percep & 97.24\% & 97.20\% & 36.10\% & 98.30\% & 97.75\% & 91.00\% \\ Random Forest & 99.24\% & 99.20\% & 17.20\% & 99.30\% & 99.25\% & 95.60\% \\ Decision Table & 99.14\% & 99.10\% & 20.60\% & 99.20\% & 99.15\% & 97.00\% \\ Average for all classifiers & 97.87\% & 97.90\% & 30.40\% & 98.80\% & 98.35\% & 88.30\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Classifier results for DOS No Resampling: 60–40% training vs. testing. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline  & **MITM: 80\% Non-Attack and 20\% Attack** & & & \\ \hline  & **Accuracy** & **Recall** & **FPR** & **Precision** & **F-Measure** & **ROC** \\ \hline BayesNet & 81.80\% & 81.80\% & 54.70\% & 84.90\% & 83.32\% & 78.40\% \\ Naive Bayes & 79.12\% & 79.10\% & 77.20\% & 80.30\% & 79.70\% & 55.30\% \\ J48 & 86.91\% & 86.90\% & 43.90\% & 87.80\% & 87.35\% & 84.30\% \\ Simple Logistic & 80.34\% & 80.30\% & 78.40\% & 80.30\% & 80.30\% & 47.10\% \\ SVM & 80.34\% & 80.30\% & 78.40\% & 80.30\% & 80.30\% & 51.00\% \\ Multilayer Percep & 82.19\% & 82.20\% & 70.00\% & 82.00\% & 82.10\% & 65.30\% \\ Random Forest & 86.41\% & 86.40\% & 38.30\% & 89.00\% & 87.68\% & 86.10\% \\ Decision Table & 85.35\% & 85.30\% & 51.20\% & 86.00\% & 85.65\% & 84.20\% \\ Average for all classifiers & 82.81\% & 82.80\% & 61.50\% & 83.80\% & 83.30\% & 69.00\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: MITM Classifier results for 80–20 (non-attack vs. attack): 75–25% training vs. testing split. the-middle (MITM), and Reconnaissance. Our detailed entropy analysis corroborates our earlier findings that, although some degree of anomaly may be detected using univariate and bivariate entropy analysis for DOS and MITM attacks, not much information may be obtained for the initial reconnaissance, thus preventing early stages of attack detection in the Cyber Kill Chain. We extended our anomaly detection technique using entropy analysis presented in this paper by using a classification-based approach using two Modbus features in addition to the three TCP/IP features selected earlier. Eight different classifiers, including tree-based classifiers, were compared: BayesNet, Naive Bayes, J48, Simple Logistic, SVM, Multilayer Perceptron, random forest, and Decision Table. Binary classification, performed using Weka's default values for each classifier, used the features packet size, inter-packet, packet process time, and two additional Modbus application protocol header features: Modbus frame length and function code value. Our results showed that the classification results of the DOS attacks were much higher than the MITM attacks. In terms of classifiers, J48 and random forest had the best classification results and can be considered comparable.