# Considerations for Developing Robot-Assisted Crisis De-Escalation Practices
## 5 Training the Robot
There is also the issue of sample inefficiency, given that obtaining the large set of data required to train an agent is problematic. First and foremost, there are significant ethical issues associated with sending an untrained or partially trained agent into situations with agitated individuals to engage in the trial-and-error process of reinforcement learning. Ethical issues aside, it is also hard to imagine where to acquire a sample of agitated individuals large enough to train the agent. For example, training an agent to play Atari 2600 games required millions of attempts to play the game [61]. De-escalation involves a significantly larger action space and more complicated reward function than Atari 2600 games; as such, the number of training attempts required would likely be much larger. Offline reinforcement learning offers some promise in overcoming these challenges. In offline reinforcement learning, the agent trains on a fixed dataset of previously collected experiences with known trajectories. The agent interacts with the dataset to collect a set of experiences to learn a policy without engaging with a real-world environment. Offline reinforcement learning is particularly valuable for tasks for which real-world interaction is prohibitively dangerous or expensive, as in the case for training a robot in de-escalation. This learning paradigm is limited, however, by the datasets that are available for training. With police bodycams becoming a norm within the field, it is possible that footage collected from police interactions could be one dataset on which robots could be trained. [PERSON] and [PERSON] [62] offer a systematic methodology for coding and analyzing video data collected from bodycams for patterns of escalation and de-escalation, and observations from their study regarding the strengths and limitations of bodycam footage may help to guide machine learning with this type of data. Police interactions are a highly specific context for de-escalation, however, and would likely not be adequate to train a robot for de-escalation in other settings. As such, it would be beneficial to acquire datasets representative of other fields in which de-escalation practices are often implemented (e.g., emergency medicine, mental health nursing, special education). With reinforcement learning, the solutions that machines devise to maximize reward functions are often not the outcomes intended. What is intended is not always consistent with what is incentivized, and it can be difficult at times to capture exactly what an agent should be rewarded to do. Consequently, reward functions use imperfect but easily quantified proxies for desired outcomes. In one example, researchers training an agent on a motorboat race game encountered a problem in which the agent learned to maximize its score by circling around and hitting targets repeatedly, but without finishing the race [63]. With de-escalation, it is not hard to imagine reward functions that could be maximized through strategies causing more harm than good, or strategies that are inherently dangerous. For example, a quick way to reduce agitation would be to cause a person to lose consciousness, but this is not an acceptable strategy for de-escalation. Lastly, there remains the challenge of equipping robots with the requisite hardware and software to sense the information needed for decision making and to adequately convey the appropriate response to their human interactional partner.