## 2 Background
### Related Research  
MI systems have been reviewed by several researchers from various perspectives. [PERSON] and [PERSON] [5] conducted a review of MI systems that covered input modalities, system architectures, and use cases in a number of different themes (input modalities, system architectures, and applications), which were used to categorize the findings. For input modalities, they organized their findings that are based on the type of modality, such as body and hand gestures, facial expressions, and voice. Another review of MI systems was conducted by [PERSON] [6]. [PERSON] [6] building upon the findings of [PERSON] and [PERSON] [28], provided insights into a variety of input modalities with examples. In the list of presented by [PERSON] [6], there are four modalities: (1) \"visual\", which includes facial expressions, gaze, and body gestures; (2) \"auditory\", which includes speech and non-speech sound; (3) \"touch\", which indicates pressure and actions performed by fingers; and (4) \"motion capture\", which records movement of the target (sender) by sensor devices. [PERSON] et al. [20] presented a similar list of input modalities by including biosignal sensors, such as the electroencephalogram, electromyogram, and electrooculogram. [PERSON] and [PERSON] [19] conducted a systematic review that focused on output modalities used to engage users with materials in art exhibitions. They identified visual, haptic, auditory, gustation, and thermoception modalities that were used in 154 cases. However, we noted different uses of terminology for each modality, as the classification of modalities was dependent on the perspectives of the researchers. For example, [PERSON] et al. [20] described touch-based interaction as a \"haptic-based\" modality, whereas [PERSON] and [PERSON] [5] described the same interaction as a \"touch\" modality. Therefore, a taxonomy of modalities is necessary, which was developed by [PERSON] and [PERSON] [29], who presented a taxonomy of input and output modalities used in HCI. Their taxonomy, which is depicted in Figure 2, was evaluated by six HCI experts, who verified its usefulness as a tool for categorizing interaction modalities. As IoT and AR are technologies that can be used in MI systems, a number of studies using both technologies within one MI system have been published. [PERSON] et al. [21] explored an existing mobile application that utilized IoT and AR in the context of a smart city, while [PERSON] et al. [22] analyzed 79 studies that were published from 2000 to 2018 that developed systems using IoT and AR across several platforms and different fields. The survey conducted by [PERSON] et al. [22] revealed that over 60 systems utilized  
Figure 2: Taxonomy of interaction modalities proposed by [PERSON] and [PERSON] [29]. vision (light) as input and output modalities, while other modalities, such as sound, touch, motion, and temperature, were used in fewer than 20 systems. Furthermore, they only identified 26 studies that constructed an MI system. [PERSON] et al. [30] analyzed studies that were published between 2014 and 2018 that proposed MI systems using AR. From 14 studies, the researchers identified eight input modalities (gesture, speech, face, touch, adaptive, motion, tangible, and click) and reviewed the proposed MI system architectures. Furthermore, when a virtual world in AR is mapped onto a real-world geographical space, the combination, called mixed reality (MR), becomes an important topic that can attract researchers' interest as a novel technology. [PERSON] and [PERSON] [31] presented a review of input modalities used in 10 MI systems operating in MR environments. They found that gesture and speech were used in six MI systems, while, other modalities, such as facial recognition and eye/head tracking, were used in three MI systems. In addition, they listed tangible interfaces, which are based on physical objects that are used to manipulate virtual objects, as one of the input modalities. Our analysis revealed that MI systems have been studied from various perspectives.