Procedure Automation: Sharing Work with Users

An area of interest for NASA is the use of procedures as the basis of task automation. The PRIDE software was developed to author and execute electronic procedures for NASA spacecraft and habitat operations. We describe our approach for modeling human-automation work based on a procedure language, and allocating and execution tasks among a human-automation team. We illustrate our approach with examples of collaborative work using procedure automation. Procedure Automation for NASA The PRIDE software was developed to author and execute electronic procedures for NASA spacecraft and habitat operations. The nature of work in NASA operations requires specialized knowledge about complex systems that may be used infrequently. Additionally, error consequences when performing the job can be significant. NASA uses procedures as a means of “planning ahead” how operators will perform both nominal and off-nominal work, to mitigate the risks of operating in such a high criticality domain. Procedures are used to manage spacecraft and habitat systems, perform Extra Vehicular Activities (EVAs), and conduct space science and exploration. Astronauts and flight controllers are trained using procedures. Qualifying for flight control positions includes performance using procedures. Thus, NASA users are familiar with procedures and procedures are well-maintained. NASA is interested is the use of procedures as the basis of task automation. As astronauts move deeper into space, their workload is expected to increase because Earth-based flight controllers will not be in continuous real-time communication. Task automation has potential to reduce astronaut workload for such missions. It also can improve response time as communication latency with Earth increases. And automation can prove beneficial in performing tasks prone to human error, such as vigilance monitoring. One challenge in automating procedures is capturing procedure knowledge that can be used both for manual and Copyright © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. automated execution. These task models often are built when the manual procedure is first documented, and well before the automation is available. Thus, our approach must produce electronic procedures for either manual or automated execution. NASA procedure authors are subject matter experts, so we also need an approach to task modeling that does not require computer programming skills. Another challenge in automating procedures is communicating automation behavior and its effects on spacecraft and habitat systems. The introduction of task automation into NASA operations requires establishing operator trust that automation is reliable and predictable. Even when operating at a high level of automation, it is expected that operators must maintain awareness of automation actions, because they are responsible to direct and manage automation. It also is expected that operators will intervene when automation or system behavior is different than expected. The PRIDE electronic procedure software was developed to address these challenges. It consists of a procedure editor (Pride Author), web-based display server (Pride View), and an automation engine (PAX). We describe our approach for modeling human-automation work based on a procedure language, and allocating and executing tasks among a human-automation team. We illustrate our approach with examples of collaborative work using procedure automation. We summarize our studies of performance with procedure automation. We propose to present our position with demonstration at the workshop. Modeling Human Work for Automation Inspection of the procedures used by NASA human space flight reveals an underlying action vocabulary and grammar for using this vocabulary that has a clearly defined semantics. When managing spacecraft or habitats, operators need to perform actions such as 1) send commands to a system 2) verify sensed values are as expected 3) record sensed values at a specific point in the procedure, and 4) wait for a sensed value to reach a target value. These atomic actions are composed into checklists with conditional action sequencing, such as 1) performing a subset of actions conditional upon situated information, and 2) looping through a subset of actions until a condition is true. PRIDE task models are represented using a procedure representation language (PRL; Kortenkamp, et al., 2008) that abstracts this vocabulary in a set of instruction types for building the action sequences seen in procedure checklists. One user of PRIDE procedures is the procedure author who creates and modifies the PRL. For NASA, procedure authors are subject matter experts. They usually are engineers, scientists, or mathematicians. While they understand how to use a computer, they often have no background in computer programming. They typically use Microsoft Word to author procedure documents that are translated into XML files by a programmer. The Pride Author software provides a way for authors to produce XML directly while manipulating instruction objects (Izygon, et al 2008). To add an instruction, the author drags the desired instruction type (corresponding to an action) into a central canvas area. This produces an instance of that type. Manipulation of these instruction objects in the canvas automatically produces PRL in the background. What the author sees is an action-object pair similar to what they typed into the Word document e.g., a valve enable command is displayed “Cmd CO2 Vent Valve Enable”. The author also drags items from a model of the system commands and data (called the System Representation; Bell, et al., 2015) to insert references to system commands and telemetry verifies. Figure 1 shows an example of the procedure editing user interface for building PRL procedures. When executing the procedure, both the operator and automation use the same PRL task model to perform tasks. This model combines information to instruct a person what actions to take with information needed to execute those actions. Thus, a task to compare a sensed instrument reading to a target value will include both operator directions for what values to compare and data references for accessing current sensed readings. This model is used to generate a web user interface of the procedure document that is directly manipulated by a person to perform the task. The same model is used by the software to automate tasks. As the procedure instructions are executed, the procedure display is annotated with information about the state of execution (what has been done, what remains to be done); see Figure 2. The same annotations are used whether a person or automation performs the task. Thus, the same task-based user interface is used to monitor the actions of automation as is used to perform actions manually. This shared task model is the basis of humanautomation communication about the task. Structuring the work of automation according to human work improves the transparency of automation actions. This approach provides a means for establishing common ground about the ongoing task that should improve operator understanding of automation behavior (Clark and Brennan, 1991). Sharing Task Responsibility with Automation Shared human-automation work for complex, high risk domains benefits from the ability to tailor the task allocations to the situation. For example, workload balancing may require a redistribution of tasks among the humanautomation team. For electronic procedures, this means shifting or sharing the responsibility to perform instructions or make decisions between operators and automation. Each instruction is designated as manual only or automatable. Manual only instructions can only be performed by a person. Automatable instructions can be performed either by automation or a person. For the domains in which PRIDE procedures have been used, the ability to designate an instruction as Automated Only has not been needed. These designations are made when the procedure is auFigure 1. Pride Author User Interface Figure 2. Pride View User Interface thored, and can be adjusted as needed when a procedure is performed (Schreckenghost, et al., 2008). Responsibility to complete an instruction can be shared by the operator and automation. Instructions have an optional Witness property indicating when a person should approve the action taken by automation before proceeding to the next instruction. Failure of a human witness to approve the instruction is considered anomalous execution. Procedure instructions are designed to be executed in the order shown in the procedure. When performing instructions manually, however, the user is able to alter the order of execution. PRIDE provides functionality (oversight mode) to alert the user when doing an instruction out of order, but such re-ordering is not prevented. When performing instructions automatically, the order of execution is enforced by the automation (guided mode). The currently “active” instruction is indicated by a colored, labeled focus bar placed behind the instruction. The operator can only manipulate command buttons or other interaction forms in the active instruction; all other instructions are disabled for manipulation until the focus bar reaches them. Procedures can be composed of a mix of Manual Only and Automatable instructions. When operating in guided mode, the automation will pause when it reaches a Manual Only instruction. The interaction forms for that instruction are enabled for manipulation. If the user completes the manual action, the focus bar moves to the next instruction and automation resumes, if the instruction is designated Automatable. The user also has the option to skip the instruction, fail the instruction, or stop automation. Examples of Collaboration with Automation Multiple procedures can execute concurrently, operating at different levels of automation and with different type