## 5 Position and Opportunities for Further Research
### Artificial Intelligence for Safety-Critical ICPS  
CPS are challenging to get right, therefore formal verification provides rigorous ways of establishing safety of controllers with respect to a physical model of the system under control. However, formal verification can guarantee safe system operation, but only if there are no discrepancies between the real implementation and the verified model. Such discrepancies between reality and model are imminent in physical systems operating in open environments. Moreover, to overcome complexity challenges and to make it possible to be analysed, often the verifiable models of a system use simplifying abstractions.  
CPSs can leverage artificial intelligence and machine learning (ML) algorithms to act well in open environments, and therefore, in recent years, there has been a big interest in applying ML components into safety-critical CPS. In difference to formal methods, ML does not need a knowledge of the behaviour of the whole system and, instead, uses learning algorithms that generalise responses from static data (e.g., a set of labelled images to classify) or from dynamic experience (e.g., responses to trial and error). However, it is challenging to achieve high-levels of safety assurance due to the complexity and unpredictability of ML techniques. Moreover, ML algorithms also raise additional safety issues since: (1) most expressive and powerful ML models are not transparent and behave as a black box; and (2) the ML training data are usually incomplete. In reinforcement learning (RL) [63], for example, a controller or an agent perceives the state of the environment, and it acts in order to maximise the long-term return that is based on a real valued reward signal, without a need for a perfect model of the environment. The RL agent needs to balance exploitation with exploration, where exploitation is the strategy to select best action based on previously learned policy (i.e., behaviour), while exploration is a strategy to search for better policies using actions outside the current, learned, policy. Exploration creates opportunities, but also induces risk that actions selected during this phase will not generate increased reward. Therefore, most approaches towards reinforcement learning provide no guarantee about the safety of the learned controller or about the safety of actions taken during learning, which is against the best practices demands of safety critical CPS, such as planes [64] or vehicles [65]. Safe reinforcement learning [66] is am emerging field in artificial intelligence that addresses control problems in which it is important to respect safety constraints.  
Another aspect related to artificial intelligence and CPS is the introduction of cloud platforms. While the fog and cloud bring immense computational power to CPS, which can be further used for artificial intelligence training and inference, they also introduce system level resilience and communication latency assurance challenges which can be fundamentally important to resolve in the CPSs that have real-time and safety-criticality requirements. In practise, the more safety critical a system is, the closer to the system the inference and decisions should be made. Finally, in relation to the research questions posed in Section 1, artificial intelligence and formal verification combined can be of great help in the development of time-critical and secure CPSs. Nevertheless, there is a need for future work to resolve the above mentioned challenges in utilising the combination of artificial intelligence and formal verification techniques in time- and safety-critical CPSs.