# A Survey on Machine-Learning Based Security Design for Cyber-Physical Systems
## 5 Potential Research Directions
### Real-Time Attack Detection in ML  
General ML applications, such as obstacle detection, require high accuracy and precision. Therefore, general ML applications are evaluated with four performance metrics [102]: accuracy (9), precision (10), recall (11), and the F1-score (12). However, conventional evaluation metrics do not consider time-related metrics. Due to the real-time characteristics of physical dynamics in a CPS, security-critical ML applications must be evaluated with not only the four conventional metrics but also the time to detection metric. Theoretically, physical dynamics can diverge to infinity, but the state of physical systems in the real world has a finite boundary. When the state of the physical system violates the boundary, the physical system becomes irreparable, which means disruption. Therefore, a cyber-physical attack must be detected before the state of the physical system exceeds the repairable boundary. Detection deadlines are determined by system dynamics, the state of the physical system when the attack starts, and by the type of attack. Figure 7 shows one example of a real-time detection constraint under a DoS attack with a ball-beam control system, which is a well-known physical control system [133]. The ball-beam system tilts a beam to regulate the position of a ball rolling off the beam due to gravity; therefore, the ball-beam system is intrinsically unstable. When a DoS attack (7) is launched against a ball-beam control system, the ball rolls off the beam due to the inherently unstable characteristics of the system. We consider a control scenario in which the range of the beam is 0 m to 1 m and the reference ball position is 0.5 m. If the position of the ball exceeds the range of the beam, we then consider the physical system to be irreparable. In this scenario, we launch a DoS attack at 15 s; then, the position of the ball drastically decreases and exceeds the range of the beam at 25.75 s. Therefore, the ball-beam control system becomes irreparable. To avoid destroying the physical system, the cyber-physical attack detection strategy should succeed before the physical system reaches the irreparable state boundary. The attack detection deadline is determined to be after 10.75 s, which in this scenario is from the start of the DoS attack to the point where the physical system becomes irreparable. Many studies into cyber-physical attack detection have been conducted; however, the research on cyber-physical security that takes real-time constraints into consideration is insufficient [27]. In the physical system layer, a system knowledge-based attack detection strategy with KF is proposed in [14], which supports real-time attack detection from sensor \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Reference** & **Defense Against** & **ML Model** & **Validation** \\ \hline
[125; 128; 129] & Application software & Various & Experiment \\
[126; 127] & Application software & CNN & Experiment \\
[130] & Application software & Softmax & Experiment \\
[131] & Computing hardware & Various & Experiment \\
[132] & Computing hardware & CNN & Experiment \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Summary of ML-based anomaly detection methods in the application layer. attacks and DoS attacks on a smart grid. A real-time image sensor attack detection method with non-linear physical-dynamics knowledge is proposed in [134], where the proposed method is validated for a vehicle and detects the sensor attack within 0.1 s. In the network layer, a real-time packet manipulation-attack detection method is proposed targeting CBTC systems, where distributed network devices continuously monitor the exploitation of the ARP protocol [16]. In [58], a CPS framework provides detection against sophisticated sensor attacks through the network in real-time, where the real-time constraint is determined by the irreparable state condition of the physical system.