# Augmented Reality in Surgical Navigation: A Review of Evaluation and Validation Metrics
## 3 Evaluation and Validation of AR Surgical Technology
The main challenge that AR technology is currently facing is the accuracy. Aside from a qualitative visual evaluation, there is currently no way to measure the correctness of an augmentation or predict the error. This is owing to the utilization of several complicated systems, each of which contributes to the uncertainty. For instance, optical tracking is widely utilized as a source of information in augmented reality for camera movement and fiducial pointing in AR [76]. However, as mentioned in [90], camera calibration, tracking, and hand-eye calibration errors are all accumulated by such systems. A clear requirement for validation methods in surgical AR is created by this lack of precision [1]. Validation requires demonstrating that perhaps the system performs as expected. Validation will mostly consist of establishing that implemented AR improves the knowledge and understanding of the operational domain if AR is designed to deliver an ideal understanding of the work domain. Validation has primarily been studied in terms of its impact on perception so far. As a result, validation is carried out as a research including a representative group of users, covering a wide range of abilities, age, technological skills, gender, and other applicable characteristics, as well as under unique situations such as fatigue and stress. AR designers want to know how their system affects decision-making and action, and there are numerous metrics that may be used to evaluate and validate this system, which are mentioned in [91]. Figure 8 presents one of the evaluation metrics which can be classified into four groups. The first is about the user who is testing the AR system (e.g., head-mounted system). The system's human elements, such as the system's ergonomics, have a direct impact on the user (e.g., comfort of use). The system may also have an effect on the user's technical (dexterity, tremor) and non-technical skills (such as situation awareness, stress, or risk anticipation). The technology may have an impact on the entire medical team by raising situation awareness and influencing human factors. Although it is well understood that the success of a process is primarily determined by its goal, the procedure used to achieve the goal, such as minimal duration, minimal energy, minimal workspace, cost, repeatability, or standardization, may be of interest and serve as a metric for evaluation of the AR system. Finally, the system may have an impact on the patient by affecting clinical scores, operating room time, or recovery time. When conducting AR validation studies, it is critical to follow the standard procedures to ensure accurate reporting and interpretation of the findings. It is relevant to describe the clear design and reporting in the context of assessing image-guided navigation systems [92]. It is also critical to be able to clearly specify the conditions under which validation can be carried out in order to acquire a thorough grasp of the results and any potential bias. The suggested validation workflow is shown in Table 2 based on the operator level, which ranges from low to high clinical realism (i.e., from engineer to medical student to the surgeon). It describes the parameters of the place/setup where the AR system is being used by the various operators, which is laboratory for an engineer, simulated operation room for a medical student and operation room for the surgeon. An engineer's work is mainly focused on technical scenarios and testing with simulations, whereas a medical student performs a simulated procedure using phantoms, and the real procedure is being performed by the surgeon using a clinical dataset. Following the above described evaluation and validation method step by step can aid in the development of a better AR surgical technique. We advise assessing the degree of expertise in the work domain for each operator. This can be calculated using surveys such as the one given in [74], which has questions that span the entire abstraction hierarchy of the relevant work domain.