## 5 Discussion
### RQ 1. Which Modalities Are Used in MI Systems with IoT and AR? We identified five unique modalities that were used as input and two unique modalities that were used as output in the reviewed studies, according to the results presented in Section 4.4. Each modality had several input sources from an agent, except for the audition. Because AR requires a view of the real world when overlaying virtual content, the vision modality was employed in every reviewed study. Moreover, vision was also used as an output modality in every reviewed study, as AR content is typically presented in either 2D or 3D graphics. Touch was the second most used input modality, possibly because the most frequently used interaction device was a mobile device (e.g., smartphone, tablet PC), which an agent primarily touches to perform interactions. While a mobile device was the most commonly used type of interaction device, an HMD was the second most used device. The third most implemented input modality was proprioception, as many interactions are performed by hands (e.g., hand gestures, position of hands). Kinematics and audition were the modalities least used as input. As an input modality, kinematics was only used in specially designed environments to detect the movement of physical objects sensed by IoT devices, whereas audition was used in two studies that utilized HMDs. Furthermore, audition was the least used output modality in the reviewed studies. Thus, our findings suggest that audition is rarely used as either an input or output modality. Voice commands are preferred by agents over hand gestures as the input modality when using an HMD, as [PERSON] et al. [45] determined through user evaluation. Therefore, the infrequency of audition as an input modality is remarkable, since both of the most commonly used interaction devices (i.e., mobile devices and HMD) support voice input from the agent. This infrequency may be caused by the type of interaction device and how familiar the agent is with the input modality. For example, mobile devices are frequently used in MI systems and they are a popular type of interaction device used by agents. They are designed to have primarily touch-based interaction on the screen; thus,  
Figure 7: Yearly distribution of interaction devices. agents are familiar with touch as an input modality. Moreover, environmental and social conditions may also limit the use of audition as an input modality. Thus, these findings can influence MI system developers to prioritize mobile devices as the interaction device and employ touch as the primary input modality. The principle of familiarity can also be applied to an HMD. We only identified two studies that utilized voice commands as input [45; 49]. Both of the MI systems were implemented on the HoloLens, and hand gestures could be used as the input modality along with voice commands. However, [PERSON] et al. [45] found, through user evaluation of their HoloLens application, that the participants tended to use hand gestures with voice commands, although they reported preferring using only voice commands. This observation suggests that the similarity between hand gestures (e.g., finger tapping) and touch-based interaction (e.g., touching a touchscreen) provided the agents with sufficient familiarity. Therefore, even when the participants preferred using only voice commands, they continued to use both hand gestures and voice commands to interact with the MI system. This phenomenon can also be supported with one of the myths of multimodal interaction discussed by [PERSON] [53], which claims that higher efficiency is the main advantage of multimodal systems. [PERSON] [53] argued that the main advantages of MI systems are not about efficiency enhancement, but rather about other factors, such as the flexibility of selecting from multiple input modalities, which was demonstrated by [PERSON] et al. [45]. This lack of consensus regarding efficiency improvement may be caused by missing knowledge regarding the relationship between agents' cognitive processes and each modality [5; 6].