. The rapid increase in integration of intelligent systems in every corner of technology created an emerging ﬁeld called Human-Machine Teaming (HMT). In HMT, human and machine collaborate with one another to accom- plish a common goal or task. To achieve the best performance of a team, it is necessary to build trust and cohesion among all teammates (machines andhumans). Furthermore, in a team, it is an established fact that a team member ability to predict fellow member ’s future course of action and have an accurate picture is a valuable asset and will result in better team dynamics and teamperformance. To realize such an ability we are proposing a human error pre-dicting methodology that could give an intelligent system a better understanding of human actions in advance. In the proposed method, we used eye-tracking metrics such as gaze density and cognitive state to predict human errors. Theresults obtained with the proposed and developed methods are found to be efﬁcient in predicting human error probability. Keywords: Cohesion /C1Decision trees /C1Eye-Tracking Human-Machine Teaming /C1Support Vector Machine 1 Introduction The upsurge in the incorporation of intelligent frameworks in almost every aspect of human-life has laid the foundation for better possibilities pertaining to increasedinteractions with machines and devices. On that note, the complexity in collaboration with machines has led to the development of a new concept called human-machine teaming (HMT), an emerging ﬁeld. Indeed, the evolving relationship between human and machines has become a primary intent with most of the developments aimed ataddressing the complexity factors. Today, the intelligence emanating from machineshas been instrumental, penetrating various elements of human life as indicated by IBMsurvey results [ 1]. The international organization claims that 75% of respondents in their survey anticipate that intelligent machines ought to have a meaningful impact onbusiness in the next three years. Also adding to that, 70% of organizations who ©Springer International Publishing AG, part of Springer Nature 2019 R. L. Boring (Ed.): AHFE 2018, AISC 778, pp. 47 –57, 2019. https://doi.org/10.1007/978-3-319-94391-6_5 participated in the survey believed that intelligent machines could result in a higher- value work for employees if adjustments were made to achieve most effective HMTs[1]. Essentially, the machines have made it possible to push the performance horizon of humans beyond the conventional threshold (physical, emotional, and cognitive domains). Moreover, this is apparent from the results of technologies such as bioa- coustics sensing, quanti ﬁed self and human augmentation focusing on augmented reality, as well as voice and gesture control. Despite these, there is still a dearth ofcomprehensive understanding of dynamics of trust and cohesion between people andmachines [ 2]. 1.1 Frameworks of Human-Machine Teams Associations and processes that take place in realistic contexts characterize the cus- tomary HMT framework (Fig. 1). Additionally, the machines interact with the humans through various interfaces and controls depending on the type of system application.These interfaces mostly entail panels and indicators for display in addition to decisionsupport tools. Conversely, the controls help humans in automation and system oper-ation and provide a room to standardize ways for implementing human intentions andstrategies [ 3]. Despite the practicality of this explanation, it only refers to the most typical HMTs such as autonomous assembly chains used in factories. The HMT does not allude to thecontemporary human-machine interaction which is a multi-disciplinary ﬁeld focusing on facets such as arti ﬁcial intelligence, robotics, voice control interfaces, and social sciences. Today, HMTs have a common share of collaborative efforts from both entitieswith a mutual goal of accomplishing a speci ﬁc goal and task. Nonetheless, despite Fig. 1. Typical structure of a human-machine teaming system48 P. Damacharla et al. having an intelligent framework, the machines do not have comprehensive control over a task [ 4]. Therefore, the task accomplishment requires leveraging best skills of human and machine. For instance, humans have optimal skills such as Correlation identi ﬁ- cation, Perspective & Emotional intelligence while the machines are more dominant in Scouring of Data, All-pairs testing, and Classi ﬁcation. Some of the common machines used in HMT include hardware robots with arti ﬁcial intelligence based operating systems or complete software agents [ 4]. Moreover, the completion of the multi-agent setup includes linked data resources that positively contribute to the interactions. Machines are the simulated agents with capabilities of perception and action in the natural world. In the modern setting, their utilization has become a paramount factor intechnologically advanced societies that play equally signi ﬁcant roles in crucial domains [2]. In the context of HMTs, a range of elements comes into play in determining the degree at which human trusts the machine and their level of productivity in dealing with speci ﬁc tasks. Some of these elements include the reliability of the machine, the self-con ﬁdence of the humans, and the synchronized workload [ 3]. Certain tasks require the machines to classify objects, identify, and detect humans and their emotions. Typically, the urge for energetic dimensions prompts the expansionof all the sub- ﬁelds of human-machine operations. Moreover, there are increased possibilities and potential for HMT [ 5]. Consequently, with the expansion of human- machine cohesions, the current and advanced technologies could facilitate scenarios where the team members collaboratively assist each other in control, coordination, and achievement of tasks without any errors [ 5]. Both humans and machines can be erroneous, particularly in circumstances with high levels of fuzziness. Therefore, a better standardization of cohesion betweenmachines and human is a signi ﬁcant necessity for a good team performance. Com- mendably, industrial machines have been a major integration into industrial assemblylines and have indicated collaborative effectiveness with humans. However, as part ofthe team ’s functionality, it is signi ﬁcant that team members have the forecast for courses of action of fellow members for improved accuracy through minimal errors [5,6]. 1.2 Overview of the Paper Despite the prevalence of multiple protuberant approaches that utilize facial lexes to identify prompt errors or task-related performance errors, HMTs lack the capabilities offorecasting errors during task execution. Resolutely, the paper entails the proposition ofan approach to exploit eye tracking with eye point data as a means for measuring theconcentrated area followed by utilization of data to forecast both the type and per-centage of errors. Notably, the proposed plan utilizes a chronological method tomeasure cognitive state to predict human errors. In Sect. 2, we brie ﬂy discuss different human error identi ﬁcation technologies studied over the course of time, and in Sect. 3 we propose an error prediction method and discuss its implementation. Section 4 discusses results and effects of results, and Sect. 5concludes the research work.Human Error Prediction Using Eye Tracking to Improvise Team Cohesion 49 2 Background and Related Work Over the years, there have been multiple publications of work focusing on the pre- diction and identi ﬁcation of human errors in completing a task or interacting with a complex machine. These literature could be ef ﬁcient in informing humans and inﬂuencing ef ﬁcient interactions with the machines. In 2015, Mu et al. highlighted that the increasing dependency on machines was marked with augmented accidents ema-nating from human error [ 7]. This applied as the basis for indicating the need for a human reliability analysis. In their paper, they constructed a prediction method ofhuman error probability through the integration of performance shaping factors andBayesian networks theory. The results of the study showed that the method couldeffectively address challenges on correlation and the dissimilar weight. The robotics department of MIT has also been one among a few research institu- tions at the forefront making developments on the subject matter. Their study aimed at discovering how interface design could in ﬂuence the calibration of trust and reliability between humans and machines. The primary aspects of interest were the probableadvantages of predictive alarm systems that could identify human errors before theyoccurred [ 8]. Contrary to the traditional binary alarms that indicate solely ‘threats ’or ‘no threats ’, the predictive alarms would give additional details on the con ﬁdence levels and the insistence of the forewarned error. Paper concluded that the predictive alarmwould be in ﬂuential in helping alleviate the ‘cry wolf ’effect as a sensation popular in high-risk industries operating with both humans and machines [ 8]. In a different study, Yang et al. formulated a method entailing human respondents performing a memory and recognition task with the help of an automated decision aid.In the study, the respondents were asked to memorize images and were later asked torecognize them in a gallery of relatively similar images [ 9,10]. The automated decision aid helped the respondents by recommending image to select, which was very reliablefor some of the participants but deemed undependable for others. The results of thestudy showed that the respondents under-trusted highly reliable aids and over-trusted the unreliable ones. In conclusion, the study showed that cohesion between humans and machines evolves and stabilizes over time, which positively in ﬂuences the prevalence of errors [ 9,10]. Resolutely, based on recent studies, the identi ﬁed works which made positive contributions to the subject are presented in Table 1. However, the study focuses more on the prediction and identi ﬁcation of human errors in a task. The studies are aimed at developing cohesion process between humans and machines but fail to indicate how itcan be improved through a form of contemporary interaction where machines also get to observe humans back. This leaves an unexplored area in HMTs warrants a two-way communication between human and machine through mutual feedback.50 P. Damacharla et al. 3 Proposed Methods The primary reason for selecting Eye-tracking mechanisms for human errors detection is its possibility to develop methods task independently and can be adapted to most ofthe environments [ 13]. Studies show that eye tracking methods can track eye motions to the highest degree of accuracy possible. The methodology section looks at gazing eye mechanisms and measurement of cognitive eye movement to estimate error-makingpossibilities. The method section will base the ﬁndings of the error prediction mech- anism on the outcomes of experimental processes derived by using gazing mechanismsand webcam applications to identify incidences of errors. 3.1 Video Monitoring Based Gaze Tracking Video-based method of eye tracking provides a suitable method of gaze tracking. The experiment relied on non-invasive and video-based tracking method to attain accurateresults of error prediction prior to errors made. The non-invasive method relies on amulti-camera or mono-camera eye tracker that most common systems are equipped with (Fig. 2). A major bene ﬁt of this methodology is that it entails minimal mathe- matical details, which makes the calculation processes and methods of generatingresults less complex. Theﬁrst step was to illuminate the eye with an infrared light source as it was essential in the production of glints on the cornea. Here, the re ﬂection generated (corneal re ﬂection), was a useful output source to help identify the reference point for the estimation of the gaze. The glint vector produced by the pupil remains constantTable 1. Comparison of human error prediction studies to improve HMTs Research study Area Results and conclusions An increasing dependency onmachines Vs human error [ 10]Construction of a prediction method of human errorprobability through the integration of performance shaping factors and Bayesiannetworks theoryThe method could effectively address challenges oncorrelation and the dissimilar weight using Bayesian networks. Can be concluded,the method predicts humanerrors though actions sequence data The probable advantages ofpredictive human error alarm systems [ 11]Discovering how interface design could in ﬂuence the calibration of trust and reliability between humans and machinesThe predictive alarm would be inﬂuential in helping alleviate the ‘cry wolf ’effect as a sensation popular in high-risk industries operating with bothhumans and machines Humans calibrate their trust in machines based on past experiences[12]Memory and recognition task with the help of an automated decision aidThe cohesion between humans and machines evolves and stabilizes over time, whichpositively in ﬂuences the prevalence of errorsHuman Error Prediction Using Eye Tracking to Improvise Team Cohesion 51 irrespective of the eye and head movement. But, employing a single camera may not produce the desired illumination effect. The above gazing method bases its workingmechanism on the ability to trace the directions of eye movements [ 14]. Accordingly, the process provides evidence on the ability to generate different sets of images for different eye movements. When the eye is captured on various actions, it is simpler todevelop feasible mechanisms upon which the simulation of errors can also be identi-ﬁed. The method is implemented using stranded software library called Open Gaze and Mouse Analyzer (OGMA), with attention maps calculated as the Gaussian distributionof each ﬁxation in a stimulus slide using Eq. 1[14]. fx ;yðÞ ¼ e/C0x2þy2 2r2 2pr2x;y2/C0 s;s½/C138 ð 1Þ Suitable outcomes on the assessments were dependent on the results garnered in the process. For instance, irrespective of whether a post-completion error occurred or not,the data collected in the process would be useful in generating a ﬁxation measure to conduct better prediction capacities of alerting human beings on the nature of errors that they commit. 3.2 Predicting Human Error Through Cognitive State Measurement Eye movement measures are dependent on the cognitive state of the individual. The fact that the eye movements are an essential aspect in evaluating human abilities toavoid making errors implies that a better understanding of the cognitive state of aperson would yield superior outcomes. We analyze the commonly used mean pupildiameter change (MPDC) [ 15], to evaluate cognitive state, which is calculated as the average for the six curvy road segments of the difference between the mean pupildiameter in a given curvy segment and the overall mean pupil diameter for a givensubject. Polar research results proved that MPDC is a function of cognitive state [13,15]. Fig. 2. OGMA eye-tracking working and setup52 P. Damacharla et al. By using MPDC values and ﬁxation values obtained through Eq. 1we would be able to predict potential human error, because, a human error is going to be a functionof human cognitive state and human gaze point on a screen. Other parameters that mayplay a role for a human to make an error are natural skills, knowledge, experience, environment, and auxiliary systems. For the experiment, simulation condition the paper considers as other parameters effect are standardized and represented with singlevariable value. 3.3 Data Generated and Processing Using two major factors that from eye tracking data gaze and MPDC we will be able to predict the probability of distraction or probability of human going to commit errorassuming rest of the conditions are ideal. Eye tracking data generated from Eq. 1is a density value that ranges between to 0 to positive making higher the value better thehuman viewing targeted area. Whereas, MPDC ranges between −0.8 mm to 1.2 mm [13,15] signi ﬁcantly in general population. According to studies, the larger the pupil diameter the more the cognitive load on human, and the more it directly relates to a higher probability for a human to make a mistake. We normalized gaze values from 0 to 1 and MPDC values inverted to get direct relations from chances for humans to make errors that can be represented by Eq. 2. Using Eq. 2we have generated different sets of synthetic data that is used to train machine learning prediction models to predict human error chances. human error chances ¼gaze /C3w1 ðÞ þ /C0 MPDC /C3w2 ðÞ weights w1 þw2¼1ð2Þ 3.4 Prediction Methods Support Vector Machines is a classi ﬁcation method that maps data points onto a different dimensional space that includes high dimensional space. In this technique, asupport vector network maps a known two input factors into an unknown data spacethrough nonlinear mapping techniques [ 16]. In general, SVMs offer fast training in a distributed manner with the ef ﬁcient use of processing resources. Due to the relatedly binary nature of SVM outputs, this technique will be one of the optimal ﬁt for our data. SVM is implemented with regression learning in this paper for our data. In addition tothe training and validation accuracy metrics an additional metric, known as the ﬁt score, can be applied to such binary classi ﬁers as SVMs. Decision Tree Learning. A method used classify data by sorting down the tree from the root to child nodes. Each node in tree speci ﬁes a test case and each child node represents one of the possible value for that test case. An instance is classi ﬁed from root to tree, then move down to tree branches [ 17]. Decision tree learning is implemented with regression learning in this paper for our data. Regression tree best suits for a continuous data type which makes this model a good ﬁt for the output data type of this research.Human Error Prediction Using Eye Tracking to Improvise Team Cohesion 53 4 Results and Discussion 4.1 Results This section provides the results for the MATLAB simulation data and python implementation with respect to both accuracy and computational runtime. In addition,discussion prediction methods are given. Also discoursed are the key distinctions andobservations between human error detection capabilities through machine learning.Figure 3presents data generated by MATLAB statistical methods with weights w1 from 0.9 to 0.6, it was assumed that focusing on the different surface than desired is agreater reason for a human to make more mistakes in a critical task execution than the cognitive load at that time. However, for the sake of experiments we have provided various weights, however, the combination of two weights is always equal to one. Oneof the critical points that needs mention regarding Eq. 2is it derived in a way such that greater the result of the equation is, greater the chances for human to make more errors. Figure 4(a) and (b) provide a comprehensive analysis of the SVM model. 4(b) provides the respective training and validation accuracies by cvalue and by penalty term. Figure 4(a) provides the computational runtime with respect to cand penalty. The results show in SVM the model trained to predict human error results in near 98% accuracy. Fig. 3. The graph presents eye tracking with corresponds to human error possibility with various weights for gaze and MPDC54 P. Damacharla et al. Figure 5(a) and (b) provide a comprehensive analysis of the decision tree model. 5(b) provides the respective training and validation percentile accuracies by minimum sample by leaf and sample for the split. Figure 4(a) provides the computational runtime with respect to minimum sample for leaf and split. Theﬁrst method of understanding the cognitive mechanisms of human intuition indicated that better results could be yielded when implemented to facilitate real-timetracking. The ef ﬁciencies of machine learning models to predict human error proba- bility through eye tracking prove to be effective from Figs. 4(b) and 5(b). However, there is practical validation still remaining to make any conclusion on ef ﬁciency in predicting human errors in real tasks. With that, there is still need for a study in human-machine teaming affected by this through eye tracking before we conclude this method Fig. 4. (a). SVM model computational characteristic and (b) accuracy characteristics. Fig. 5. (a). Decision trees model computational characteristics and (b) decision tree accuracy characteristics.Human Error Prediction Using Eye Tracking to Improvise Team Cohesion 55 to be effective. Those studies still remaining to be done. For now, according to our results, it can be concluded that machine learning based methods such as SVM andDecision tree methods are effective in predicting human errors only through eyetracking data. 4.2 Discussion To recap the proposed method in simple terms, the paper presents a study about predicting human errors in HMT environments through eye tracking data. Gaze density and MPDC are two important features extracted from eye tracking data and utilized inerror predicting model development. Gaze density represents human eye physicallocation on a two-dimensional space which is directly proportional to human errorprobability whereas, MPDC gives an inverse relation of congestive load and anotherfactor in human error probability. Both are statistically used to predict chance where ahuman is going to make an error to alert the person. However, failure to look at theﬁxated points implies that the user is highly likely to make an initial error at the ﬁrst instance. Nonetheless, the results and studies showed that there are incidences when post-completion errors diminish accuracies, thereby leading to inevitable events. Theefﬁciency of the process is also a by-product of different eye measures, which are developed to feasibly provide relevant results of the movements parallel with theperformance of a task. The predictors from the regression model may vary dependingon the degree of the error change and the actual probabilities notably predicted. 5 Conclusions and Future Work Real-time error prevention mechanisms can be attained after effective learning of theway they occur. The human eye is often exposed to various intuitions all of which havea signi ﬁcant effect on the possibility of making an error. First, the paper illustrated that the ability to predict real-time eye tracking and data collection. This paper presentsSVM and decision tree based prediction models that can use eye-tracking data topredict human error probability. Besides, the ability to gauge human intuitions provides a feasible support user interaction in various ways. Prediction models can be used in alerting human before committing a mistake. Human error models will be helpful tomachines to understand the human state that bonds the cohesion between human andmachine in HMTs. Future work that needs to be conducted in implementing the modelin real time to test human subject data for validation. Human studies need to beconducted to understand the practical feasibility of human alert system in HMTs, andperformance studies have to be conducted in HMTs with this feature included. Acknowledgments. The University of Toledo and Round 1 Award from the Ohio Federal Research Jobs Commission (OFMJC) through Ohio Federal Research Network (OFRN) fund this research project; authors also appreciate support of the Paul A. Hotmer Family CSTAR(Cybersecurity and Teaming Research) Lab and EECS (Electrical Engineering and ComputerScience) Department at the University of Toledo.56 P. Damacharla et al.