# Augmenting Industrial Control Rooms with Multimodal Collaborative Interaction Techniques
###### Abstract  
The German manufacturing industry has been carrying out new developments towards the next industrial revolution, focusing on smart manufacturing environments. Our work emphasizes human-centered control rooms in the context of production plants. Increased automation does not have to come with less human control. Therefore, we report on multimodal collaborative interaction techniques to augment industrial control rooms. In particular, we include mobile workers who use the control room while being in the production hall using tablets or specifically mixed reality glasses. Collaborative annotation dashboards support discussions and a shared understanding among analysts. Manufacturing-related data can be integrated into business analytics environments so that holistic analyses can be performed. Multimodal interaction techniques can support effective interaction with the control room based on the users' preferences. Immersive experience through mixed reality-based three-dimensional visualizations and interaction possibilities support users in obtaining a clear understanding of the underlying data. control room; multimodel interaction; augmented reality; mixed reality +
Footnote †: journal:  
0000-0001-9999-4000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-0000-  
Our augmented digital control room includes technologies for industrial analytics, e.g., data integration or intelligent algorithms to identify and classify anomalies or predict maintenance tasks. However, in particular it focuses on integrating innovative interaction techniques, such as collaborative and contextual dashboards, speech and gesture control, as well as mixed reality (MR) smart glasses. Multimodal interfaces provide users with a number of different ways of interacting with a system. This might relate to the usage of diverse devices or to natural modes of communication, such as speech, body gestures, or handwriting [8]. The augmented digital control room, presented in this paper, is built upon the digital control room Section 3.2, which in turn is built upon the digital boardroom Section 3.1. Users of the digital boardroom have different situation-dependent preferences with respect to the mode of interaction [9]. Thus, it is important to provide users with different interaction possibilities. Multimodal interaction techniques support the access of the same functionality in different ways depending on the current situation [10]. MR devices, such as the Microsoft(r) (MS) HoloLens 2, support an immersive experience combining the real world with the virtual one. For example, three-dimensional holographic visualizations can be used to improve the human perception and understanding of the underlying data. We propose contextual dashboards, such as ones presenting the workload of a nearby machine, three-dimensional holographic visualizations, such as scatterplots, and eye-supported details on demand. In addition, collaboration support is crucial. Industry 4.0 usage scenarios also point out \"networked manufacturing\" [1], because of which dynamic networked organizations are important. They are based on distributed cooperating teams who need support for the development and execution of joint business processes, which are highly variable [11]. Furthermore, decision making is inherently a collaborative task [12]. Well-founded decisions require the participation of several analysts, such as domain experts, line-of-business managers, customer representatives, or key suppliers. Thus, our augmented digital control room includes collaboration facilities for both--remote as well as face-to-face situations. In particular, we include mobile workers who use the control room while being in the production hall using tablets or MR glasses. With the help of Microsoft Shared Experience, for example, holograms and dashboards can be shared and interacted with collaboratively. The remainder of this paper is organized as follows. In Section 2 related work is presented and discussed.