# Designing Procedure Execution Tools with Emerging Technologies for Future Astronauts
## 1 Introduction
### Training and Task Execution Research  
The future of training and procedure execution for NASA missions to the Moon (with Artemis targets in 2024 [3]) and Mars (targets in the 2030s and beyond [4]) is likely to include a combination of virtual and augmented reality to display information collected by a ubiquitous collection of Internet of Things (IoT) sensors within the spacecraft, habitat, and nearby robotic agents. NASA has been integrating advanced display technologies into training and procedure execution since the early 1990s. NASA Johnson Space Center's Virtual Reality Lab was created in 1991, and astronauts have been using virtual reality (VR) there since 1993 to train and plan for spacewalks and operate robotic arms. More recently, astronauts onboard the ISS have used commercially available augmented reality (AR) headsets such as Google Glass and the Microsoft HoloLens to assist with scientific experiments and demonstrate potential uses for enabling astronauts to better complete procedures with the help of remote support from MCC. These opportunities demonstrate NASA's desire to integrate these advanced displays into future missions, but research is required to identify opportunities where VR/AR can provide the most benefit over traditional training. Care must also be taken to avoid instances where training and simulated task execution in VR/AR does not accurately align with reality in order to avoid poor transfer of training to actual operation while in space. Astronauts onboard the Space Shuttle flew with large volumes containing detailed procedures of the tasks they were expected to conduct during their missions (see Figure 1). Though these procedures have largely been digitized to Portable Document Format (PDF) files, emergency procedures for the ISS continue to be available on paper in case of power failure. In order to advance the state of the art for spaceflight procedures, we have looked at analogous work processes embedded with technicians on Earth. Technicians often carry large manuals or tablets with reference material into the field with them, where a hands-free digital interface would greatly improve ground procedure execution. Like astronauts, technicians typically need their hands free to execute work. Heads-up displays are a demonstrated technology solution with successful adoption in the domain of technician field execution. Field service technician work execution technologies within industrial manufacturing domains are already heavily invested in hands-free mixed reality experiences. Additionally, aviation, aeronautics, and product manufacturing contexts have demonstrated the utility of checklists. However, much of the research around the use of procedures and checklists has focused on training. Research investigating the effects of virtual and augmented reality training has previously shown improved performance compared to conventional types of training. This research in virtual and augmented reality training has primarily focused on surgery, assembly, and maintenance skills. All three of these tasks are analogous to the types of tasks astronauts are required to accomplish during the completion of procedures in their missions. Surgical training has seen particular focus by researchers due to the inherent high-risk nature of the task, as well as the expense and limited time of expert instructors [5; 6; 7; 8]. The use of VR to simulate surgical tasks was first proposed by [PERSON] [5] in the early 1990s [5]. They used an off-the-shelf VR head mounted display (HMD) and a \"DataGlove\", which essentially acted as a joystick, for the user to interact with the scene.