# Designing Procedure Execution Tools with Emerging Technologies for Future Astronauts
## 1 Introduction
### Usability of Emerging Technologies  
The process of integrating these emerging technologies into operations is not straightforward, particularly in safety-critical domains like human spaceflight. AR interactions and IoT sensors have increased the possibilities for user centered interactions of products, potentially enhancing user experience. Yet how exactly can these emerging technologies enhance, not detract, from procedure training and execution? The HCI Group has been building software for space mission control and operations for more than ten years (e.g., [21; 22; 23]). Fundamental to our software development approach is pairing it with HCI research and methods, emphasizing usability and user experience through thoughtful and purposeful designs. As such, NASA Ames Research Center's HCI Group, in collaboration with San Jose State University Research Foundation and HCI graduate students at Carnegie Mellon University, have studied the use of emerging technology to enhance user experience during procedure training and execution for future LDEMs.  
The emerging technologies we have focused our prototypes and investigations on are AR and IoT. We believe these technologies can revolutionize LDEM human spaceflight operations, as they have the potential to significantly improve astronauts' task performance without the constant assistance traditionally provided by MCC. Currently, astronauts onboard ISS depend on procedures to complete assigned tasks and resolve unexpected issues. Procedures are digital written documents with images and, occasionally, embedded videos delineating step-by-step instructions. Procedures are developed and certified by a large team of ground controllers and are made available to crew when completing tasks. While astronauts are trained on the tasks they must complete in space, it is often years between training and execution. Due to the long time between training and execution, concepts such as just-in-time training and refresher training may be needed for future LDEMs. Astronauts rely on these procedures, in addition to the support provided by MCC, to clarify any steps or fix missteps. We believe that AR and IoT can improve procedure execution by providing more contextual information, preventing missteps, and guiding astronauts, all without relying on real-time, continuous communication with Earth.  
The HCI Group at NASA Ames Research Center, collaborating with the San Jose State University Research Foundation and Carnegie Mellon University, have conducted several research investigations with the purpose of testing and validating whether these emerging technologies can provide, at an early stage, promising feedback on successful usability. In this paper, we outline the different approaches that we have taken to test the interaction benefits that such tools provide. The goal of these approaches is to increase crew autonomy by providing crew members with additional sources of information, execution tools, or both, by leveraging emerging technologies. This helps to enable crew members to act without the need for constant real-time communication with MCC by better integrating them with data generated by the habitat, assisting with procedure execution, and by providing just-in-time training. While we do not explicitly aim to replace MCC, these tools and techniques are designed to help crew members complete their tasks in missions with increased communication latencies. Four studies are outlined herein, describing the research conducted and key findings for each specific usability topic. The paper discusses the different approaches in which emerging technology in procedure execution tools have been addressed by the HCI team. These studies focus on habitat state feedback, integration of a digital procedure in an analogous domain, designing a prototype AR system, and crew training improvements.