# Interactions in Augmented and Mixed Reality: An Overview
## 3 Research Results
### Visual-Based Modality
#### 3.1.1 Gesture-Based  
When interacting through the gesture-based context, computers get visual input and recognize body language without any other sensory information. _Eye gaze detection_ is included in this context of interaction. It is characterized by two major issues: (a) avoidance of unintended actions and (b) limitations related to eye movement accuracy. The gaze and dwell interaction model [69] is used for this method, as described by Microsoft [70], where basically the user needs to look at an object and retain this action by staring to select it. It has a high accessibility rate [71] as even severely constrained users can perform this interaction. In 2017, [PERSON] et al. explored the advantages of this interaction by exploiting some of the basic functionalities of the human eye [72]. This method takes advantage of the eye inertial and the natural vestibulo-ocular reflex (the ability to lock a target regardless of head movements) and is used with head-mounted displays. The authors conclude that more research should be done analyzing the collected large-scale _eye movement tracking_ data and improving user experience. [PERSON] [73] examined techniques and challenges related to eye movement interactions. He proposed an approach based on separating the actual data (eye movements) from noise and then estimating the user's intention of interaction. An interesting interaction method related to the eyes is _pupil dilation detection_. In [74], the authors used this method as a reliable indicator of cognitive workload. In 2017 [PERSON] et al. [75] performed task classification in HCI via the visual channel. The authors combined _facial expression analysis_ and eye-gaze metrics for computer-based task classification. The outcome was that these two interaction methods combined resulted in higher classification accuracy. Facial-expression interactions exploit the _face detection_[76] method and can be used in controlling a user's virtual avatar facial expressions [77] in _facial emotion analysis_[78] and emotion recognition [79]. _Face recognition_[80] also exploits the _face detection_ method but is usually used to identify a person by facial characteristics. In 2017 [PERSON] et al. [81] conducted a review on human emotion recognition which could assist in teaching social intelligence in machines. Techniques commonly used to exploit _emotion recognition_ and _emotion analyses_ methods include, without being limited to, the geometric feature-based processes [82] and machine learning [83]. In the context of emotion analyses and human psychology, another method interprets body language. The body movements in immersive realities and how they enhance the sense of presence is examined in-depth by [PERSON]'s and [PERSON]'s work [84]. In 2020 [PERSON] [85]  
Figure 2: Displaying all categories of the visual-based modality. applied the Kinect Skeletal Tracking (KST) System in an augmented reality application to improve the social interaction of children with autism spectrum disorder (ASD). They used a Kinect camera to scan the therapist's body gestures and visualized them on a 3D virtual character. In another work [86], the authors used a webcam exploiting the user's _body movement tracking_ method to interact with the AR system. The user had to do simple tasks like jumping, stretching, or boxing to \"hit\" the correct answers presented in the virtual world. Such methods utilizing physical exercise make the learning process more appealing to students of younger ages. [PERSON] et al. [87] exploited the _body posture analysis_ method to locate a person's two hands perceived by a Firewire camera and superimpose artificial fire on them in real time. Algorithms of hand tracking and gesture recognition are used to detect the gestures through a camera sensor and perform functions accordingly. In 2016 [PERSON] et al. [88] introduced a solution for real-time 3D _hand gesture recognition_. They used the embedded 2D camera of a mobile device, supporting 2D and 3D tracking, joint analysis and 10+ degrees of freedom.