## 1 Introduction
### Research Questions and Objectives  
Over the past few years, continuous research efforts on human-machine partnering and related methodologies have brought the HMT concept from functional design to reality. The potential benefits of HMT could be limitless and could prompt a new industries in the 21 st century. Recently, it has started gaining the recognition it deserves with its multidisciplinary applications. However, HMT is a complex subject and requires rigorous case studies in a controlled environment before its use. Human factor studies are a critical variable of interest in HMT. There is a significant technical advance achieved in autonomous systems (i.e., the machines) through AI, but that is not enough to fully realize an HMT. It requires a different approach to synergize both human and machine performance. In such cases, human performance is a subject of a mental model, task complexity, environment, and interface efficacy, thereby requiring careful assessment. Effective feedback methods are needed that are embedded in every stage of HMT building and operation. This can be achieved by metrics that evaluate an HMT's effectiveness and its agents (e.g., human, machine, and team) on different levels. To this end, we propose several research questions and associated objectives of this study. Figure 1 presents proposed human-in-loop (HIL) simulation forhuman-machine teaming (HMT) standardization. Figure 1: A graphical representation of the proposed novel human-in-loop (HIL) simulation for human–machine teaming (HMT) standardization (reproduced from [5]). Research questions: While there are ongoing research studies taking place in HMT-based technologies, there is little know-how available in the literature regarding standardization or benchmarking of teaming applications. This is despite the fact that it is critically important to compose rules and roles, validation schema, and approval odds for an HMT application. To achieve this, we pose the following research questions:  
1. Could we identify metrics that can be used to measure HMT and its components, and develop a comprehensive measurement method? 2. Could we develop and customize an intelligent agent (IA) that can be used as a machine teammate in HMT? 3. How do we study the developed IA using machine metrics identified in the first research question? Are human-in-the-loop (HIL) simulations (that utilizes the HMT) conducted in a controlled environment would be sufficient? 4. Could performing inferential and exploratory studies on the data collected from HIL simulation studies reveal additional insights that may help standardize or benchmark HMTs for a desired performance level? Objectives: Testing a hypothesized method and achieving a reliable HMT require a thorough simulation study on a task-by-task basis. However, there are no common methodologies to study how these systems perform as a team. There are few research publications dedicated to study the performance of a HMT in a single task such as navigation in crowded areas or navigation in stressful situations. Hence, we established the following objectives for this research study to test the above hypothesized research process:  
1. Study and perform a meta-analysis of various HMT metrics available in the literature and establish their correlations. (This was accomplished in a previous work [4]). 2. Using the data collected from the case studies, classify metrics into application and characteristic groups. Using identified groups, establish common sets of metrics that can be used to measure synthetic-assistant-based HMT. 3. Design and develop a SA architecture, integrated with an identified interface (e.g., voice results in a voice-based synthetic assistant, VBSA). This serves as a prerequisite to the HMT design in the HMT architecture development. 4. Propose a generalized architecture followed by components and fundamental blocks involved in teaming. 5. Develop a VBSA that can help in ECP training in real-world, controlled environments.