# Artificial Intelligence to Reshape the Healthcare Ecosystem
## 2 AI Techniques for Healthcare
### Reinforcement Learning  
When dealing with particularly complex systems, characterized by non-deterministic evolutionary dynamics, reinforcement learning (RL) techniques can be very useful. This can happen in various healthcare sectors. For example, in the definition of personalized medical and care techniques, evolutionary patterns may occur that cannot be framed in general models that can be trained through a dataset collected from different patients. Another difficult situation is clinical and care robotics, in which robots must behave in accordance with the needs of the environment in which they operate [106]. In situations of this type, RL techniques may be crucial. The general objective of RL techniques is to allow an agent to learn through continuous interaction with the environment where it resides and has to interact [107; 108]. Learning consists of discovering suitable strategies by making decisions and receiving related rewards, either positive or negative, according to the results due to the selected actions. Thus, the objective can be synthesized as learning the strategy that maximizes the final reward. Learning _iterations_ consist of consecutive elementary interactions between  
Figure 9: Schematic representation of the forward and reverse diffusion processes. the agent and the environment, called _steps_. Interactions, along with their rewards, are associated with a _state_ of the system. In such systems, a state consists of a collection of parameter values that are sufficient to model how the system evolves over time, that is, the system state sequence. Thus, the association between states, actions, and rewards is the basis of training. In summary, at each step, the agent makes decisions, interacts with the environment, characterized by a state, receives the reward, and adapts its behavior based on the collected reward. Steps are repeated over time until the agent's performance is acceptable. RL algorithms include _policy_ functions. These functions determine the strategies that map states with the actions taken by the agent. Therefore, the policy must determine the agent's behavior in all states of the environment. Policies can be of different natures. Essentially they can be deterministic or statistical. In the context of healthcare AI systems, statistical policies are typically used. They are indicated as \(\pi_{\theta}(a|s)\), and represent the probability of selecting the action \(a\) in the state \(s\). The parameters in the vector \(\theta\) represent the learned parameters determining the optimal policy. In recent years, RL research has produced numerous algorithms characterized by different properties in terms of state representation, stability, and convergence speed. The interested reader can find details in [109; 110; 111; 102]. Research on the application of RL techniques in different fields in healthcare is still in its early phase, but some contributions are promising. For example, Ref. [113] shows an application of RL techniques for discovering new treatments and personalizing existing ones. Application to dynamic treatment regimes in chronic diseases for automated medical diagnosis is investigated in [114]. Robotics is one of the main fields of application of RL. Therefore, robotic systems used in healthcare are largely influenced by the applications of these techniques. Ref. [106] presents an application of RL to precision robotics and the determination of an optimal strategy driving the growth of cell cultures and the development of therapeutic solutions. However, recent results also show how research is still in progress. For example, in [115] the authors focus on oncology and identify current challenges and pitfalls to be considered in order to successfully implement RL-based decision support systems for precision oncology.