# Control Rooms in Safety Critical Contexts: Design, Engineering and Evaluation Issues

Tilo Mentler, Philippe Palanque, Susanne Boll, Chris Johnson, Kristof van Laerhoven

► To cite this version:

Tilo Mentler, Philippe Palanque, Susanne Boll, Chris Johnson, Kristof van Laerhoven. Control Rooms in Safety Critical Contexts: Design, Engineering and Evaluation Issues. IFIP WG 13.5 Workshop @18th IFIP TC 13 International Conference on Human-Computer-Interaction (INTERACT 2021), IFIP: International Federation for Information Processing, Aug 2021, Bari, Italy. pp.542-547, 10.1007/978-3-030-85607-6_72. hal-03376237

HAL Id: hal-03376237

https://hal.science/hal-03376237

Submitted on 21 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Control Rooms in Safety Critical Contexts:
Design, Engineering and Evaluation Issues

## IFIP WG 13.5 Workshop at INTERACT 2021

Tilo Mentler¹, Philippe Palanque², Susanne Boll³, Chris Johnson⁴,
Kristof Van Laerhoven⁵

¹ Trier University of Applied Sciences, Germany
T.Mentler@inf.hochschule-trier.de

² Université Toulouse III – Paul Sabatier, France
palanque@irit.fr

³ University of Oldenburg, Oldenburg, Germany
susanne.boll@uni-oldenburg.de

⁴ Queen's University Belfast, United Kingdom
c.w.johnson@qub.ac.uk

⁵ University of Siegen, Germany
kvl@eti.uni-siegen.de

**Abstract.** Human-Computer Interaction (HCI) research has been focussing on the design of new interaction techniques and the understanding of people and the way they interact with computing devices and new technologies. The ways in which the work is performed with these interactive technologies has arguably been less of a focus. This workshop aims at addressing this specific aspect of Human-Computer Interaction in the control rooms domain. Control rooms are crucial elements of safety-critical infrastructures (e.g., crisis management, emergency medical services, fire services, power supply, or traffic management). They have been studied in terms of Human-Computer Interaction with respect to routine and emergency operations, human-machine task allocation, interaction design and evaluation approaches for more than 30 years. However, they are dynamic and evolving environments with, for instance, the gradual introduction of higher levels of automation/autonomy. While state of the art control rooms are still characterized by stationary workstations with several smaller screens and large wall-mounted displays, introducing mobile and wearable devices as well as IoT solutions could enable more flexible and cooperative ways of working. The workshop aims at understanding how recent technologies in HCI could change the way control rooms are designed, engineered and operated. This workshop is organized by the IFIP WG 13.5 on Human Error, Resilience, Reliability and Safety in System Development.

**Keywords:** Control room, Pervasive computing, Safety, Usability, UX, Security, Resilience, Dependability.
# 1 Overview and goals

While Human Factors approaches [1] and User-Centered Design [19] have been used to design and evaluate control rooms for a long time, the interaction technologies they offer are usually several steps behind what can be found in other areas such as home entertainment or gaming. Control rooms are deployed in very different domains such as crisis management, emergency medical services, intensive care units, fire services, power supply, maritime navigation, or traffic management.

More recent work has been focussing on the use of new technologies in the context of control rooms exploiting new interaction techniques and new interaction technologies introduced in research contributions from the Human-Computer Interaction area. Speech was considered early as an input modality [6] while the issues raised by its deployment was questioned in [8]. Auditory information was used as an alerting system in addition to the traditional display of information in [11] even though studies have shown [9] that human brain filters out that information in case of high workload or stress. Tactile feedback was introduced in cockpits more recently [12] (but this feedback directly used in combination with information displays) while tangible interactions were introduced as a mean to bring interactions closer to the ones on physical knobs [7]. Multi-touch interactions have also been recently studied both in control rooms [15] and interactive cockpits [16] raising multiple issues. Ambient displays [20] [ref] and head mounted displays [21] [ref] have been used to add a digital layer of information on top of control room elements to support situation awareness and attention in safety critical environments. This introduction of nonstandard (at the time) modalities, led to multimodal interactions (both in terms of input and output) but they are still considered a challenge in 2016 [2] while the early work from Bolt in that domain was introduced more than 25 years earlier [3]. Multimodality as input was however introduced successfully in industrial control rooms [5] or military aircraft cockpits [14] while multimodal output was largely used for alerting flying crews [13].

Beyond interaction technologies, the centralised nature of control rooms was questioned [10] and mobile solutions were proposed to support control and monitoring activities on the move. However, the introduction of these technologies in critical systems raised immediately the issue of their dependability and their security as soon as the environment loses its closeness nature.

While state of the art in deployed control rooms is still characterized by stationary workstations with several smaller screens and large wall-mounted displays, introducing mobile and wearable devices as well as IoT solutions could enable more flexible and cooperative ways of working. However, turning control rooms into pervasive computing environments raises user-related challenges such as usability and user experience, system-related ones such as reliability and dependability and more global ones such as safety and security [17]. However, it is clear that control rooms must evolve and integrate advanced interaction technologies [18] so that operations can be improved.

This workshop promotes sharing of experiences in designing, implementing, and evaluating interactive systems in control rooms. We are especially interested by contributions presenting theories, methods, and approaches for considering them as pervasive computing environments and interdependencies between all the properties listed above. In this workshop we consider control rooms in their broader sense including (ship)
bridges, cockpits, and operating rooms. This workshop is organized by the IFIP Working Group 13.5 on Human Error, Resilience, Reliability, Safety and System Development.

## 2 The specificities of control rooms

These control rooms are characterized by their operations that must be resilient to adverse events and unforeseen situations including human errors, system failures, environmental adversary conditions and malicious attacks. One of the specific challenges in their development is the demonstration that they meet requirements imposed by regulatory authorities.

As discussed in [4] control rooms present very specific aspects that make them very different from other interactive systems:

* They allow monitoring and control of complex cyber-physical systems;

* Their operations are usually defined and regularly assessed by regulatory authorities;

* They are operated by trained professional users that are usually qualified by dedicated organizations;

* They usually involve multiple operators who have to coordinate their actions to reach their goals;

* Operators have different levels of authority which grant different access to functions and data;

* Operator errors might have catastrophic consequences on people and goods;

* Failures in the display and control system might have catastrophic consequences on people and goods;

* Their deployment requires the acquisition of a certification from external organizations;

* They are operated in close worlds reducing security risks.

This list is far from being exhaustive but it gives a set of characteristics to be taken into account when considering the design of new types of control rooms and the constraints that have to be addressed.

## 3 Target audience and expected outcomes

This workshop is open to everyone who is interested in the aspects related to the design, the engineering, the evaluation, the deployment, the maintenance and the certification of control rooms. We expect a high participation of IFIP working group 13.5 members. We invite participants to present position papers describing real-life case studies that illustrate, if possible, how a new technology would enhance control room operations. They could also highlight the trade-offs between two or more properties of interactive systems such as user experience and dependability or usability and security. The way the new technology will be addressing some known or envisioned problem in a control should be presented in the contribution. We are also interested in methods, theories and tools for control rooms development as long as they address some user interface properties. Accepted position papers are published in INTERACT 2021 adjunct conference
proceedings. We also expect to discuss at the workshop how to disseminate individual
contributions to the community in a special issue in a journal or in an edited volume.

4 Structure of the workshop

This proposal encompasses a full-day workshop organized around presentation of po-
sition papers and working activities in small groups. From the set of contributions, a
subset of selected case studies is invited to be presented at the beginning of the work-
shop and is used to support the discussion that follows. The morning sessions are ded-
icated to welcoming participants and presenting case studies. Participants are invited to
comment the case studies and to report similar experiences. The afternoon sessions are
devoted to interactive sessions, where participants are engaged to work in small groups
to propose solutions to the issues raised by the case studies presented in the morning.
Proposed solutions will be compiled and compared. Based on the lessons learned, par-
ticipants will draft an agenda of future work that can be accomplished.

5 Workshop organizers

**Tilo Mentler** is a professor of Human Computer Interaction and User Experience at Trier University of Applied Sciences. His research is focused on human-centered design in safety-critical contexts (e.g. mobile devices and mixed reality in healthcare, novel approaches to critical infrastructure). Currently, he works on control rooms as pervasive computing environments and examines the role of user experience in safety-critical settings. Prof. Mentler is chair of the special interest group "Usable Safety & Security" within the German Informatics Society (GI), member of the IFIP Advisory Board of GI and has been the GI representative in the IFIP Domain Committee on IT in Disaster Risk Reduction.

**Philippe Palanque** is Professor in Computer Science at University of Toulouse 3, where he leads the Interactive Critical Systems research group. Since the late 1980s he has been working on the development and application of formal description techniques for interactive systems. He has worked on research projects at the Centre National d'Études Spatiales (CNES) for more than 10 years and on software architectures and user interface modelling for interactive cockpits in large civil aircraft (funded by Air-bus). The main driver of Philippe's research over the last 20 years has been to address in an even way usability, safety and dependability in order to build trustable safety-critical interactive systems. As for conferences he is TPC co-chair of EICS 2021 and is a member of the ACM CHI steering committee. He is a member of CHI academy, the chair of the IFIP TC 13 committee on Human-Computer Interaction, and secretary of IFIP WG 13.5.

**Susanne Boll** is a full professor for Media Informatics and Multimedia Systems at the University of Oldenburg and a member of the board of the OFFIS-Institute for Information Technology. Her research focus lies in the field of human-computer interaction (HCI). In her ongoing research, she is designing novel interaction technology for a respectful and beneficial cooperation of human and technology in a future automated world. She works on novel interaction methods for a safe cooperation and humans in
safety-critical automated environments such as automated driving, health care and pro-
duction. Her scientific results have been published in top venues in her field such as
ACM CHI, MobileHCI, AutomotiveUI, as well as internationally recognized journals.
Prof. Boll was named a Fellow of the German Informatics Society in October 2020.
She was named a Distinguished Member of the ACM in 2019 and is an elected member
of acatech, The German National Academy of Science and Engineering.

**Chris Johnson** is Professor and Pro Vice Chancellor of Engineering and Physical Sciences at Queen's University Belfast. Over the last 20+ years, he has authored more than 200 peer reviewed publications, including one of the first textbooks on Accident and Incident Reporting. He has held fellowships from NASA, the US Air Force and Navy. He is a Scientific Advisor to the EC SESAR JU on the future of air traffic management and helped author guidelines on incident reporting for EUROCONTROL and for the European Railway Agency. He also helped the UK Department for Transport to develop the national cyber security strategy for aviation. In 2021, he is one of some fifteen expert witnesses retained to support the public inquiry into the Grenfell Tower fire.

**Kristof Van Laerhoven** is Professor in Ubiquitous Computing at the University of Siegen, Germany. His research interests span the areas of wearable and distributed sensing systems that focus on machine learning challenges, such as recognising what human users are doing, what they are focusing on, and how stressed they are. He is wearable department co-editor at IEEE Pervasive, is co-editor of Springer Adaptive Environments, associate editor for ACM IMWUT, and was general co-chair for ACM UbiComp/ISWC in 2020. More information can be found on http://ubicomp.eti.uni-siegen.de.

References

[1] Paulo VR Carvalho, Isaac L. dos Santos, Jose Orlando Gomes, Marcos RS Borges, and Stephanie Guerlain. 2008. Human factors approach for evaluation and redesign of human--system interfaces of a nuclear power plant simulator. Displays 29, 3, 273–284

[2] Tomi Heimonen, Jaakko Hakulinen, Sumita Sharma, Markku Turunen, Lauri Lehtikunnas, and Hannu Paunonen. 2016. Multimodal interaction in process control rooms: are we there yet? In Proceedings of the 5th ACM International Symposium on Pervasive Displays (PerDis '16). ACM DL, 20–32.

[3] Richard A. Bolt. 1980. "Put-that-there": Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques (SIGGRAPH '80). Association for Computing Machinery, New York, NY, USA, 262–270.

[4] Ronald L. Boring, Jacques Hugo, Christian M. Richard, and Donald D. Dudenhoeffer. SIG: the role of human-computer interaction in next-generation control rooms. CHI '05 Extended Abst. on Human Factors in Computing Systems (CHI EA '05). ACM DL, 2033–2034.

[5] Johan Fagerlönn, Kristin Hammarberg, Stefan Lindberg, Anna Sirkka, and Sofia Larsson. 2017. Designing a Multimodal Warning Display for an Industrial Control Room. 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences (AM '17). ACM DL, Article 46, 1–5. DOI:https://doi.org/10.1145/3123514.3123516

[6] C. Baber, D.M. Usher, R.B. Stammers, R.G. Taylor, Feedback requirements for automatic speech recognition in the process control room, International Journal of Man-Machine Studies, Volume 37, Issue 6, 1992, Pages 703-719, ISSN 0020-7373, https://doi.org/10.1016/0020-7373(92)90064-R.
[7] Jens Müller, Tobias Schwarz, Simon Butscher, and Harald Reiterer. 2014. Back to tangibility: a post-WIMP perspective on control room design. International Working Conference on Advanced Visual Interfaces (AVI '14). ACM, DL, 57--64.

[8] Kenneth Huber. 2005. Does speech technology have a place in the control room? In IEEE Power Engineering Society General Meeting, Vol. 3: 2702--2703

[9] Frédéric Dehais, Michael Causse, Vachon F, Régis N, Menant E, Tremblay S. Failure to detect critical auditory alerts in the cockpit: evidence for inattentional deafness. Hum Factors. 2014 Jun;56(4):631-44. doi: 10.1177/0018720813510735. PMID: 25029890.

[10] Oskar Juhlin and Alexandra Weilenmann. 2001. Decentralizing the control room: mobile work and institutional order. In Proceedings of the European Conference on Computer Supported Cooperative Work (ECSCW'01). Kluwer Academic Publishers, Norwell, MA, USA, 379--397 http://dx.doi.org/10.1007/0-306-48019-0_20

[11] Patterson, R. D., and T. F. Mayfield. "Auditory Warning Sounds in the Work Environment [and Discussion]." Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, vol. 327, no. 1241, 1990, pp. 485-492. JSTOR, www.jstor.org/stable/55320. Accessed 10 Mar. 2021.

[12] T. Nojima and K. Funabiki, "Cockpit display using tactile sensation," First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference, Pisa, Italy, 2005, pp. 501-502, doi: 10.1109/WHC.2005.27.

[13] Selcon SJ, Taylor RM, Shadrake RA. Multi-Modal Cockpit Warnings: Pictures. Words. or Both? Proceedings of the Human Factors Society Annual Meeting. 1992;36(1):57-61.

[14] Rémi Bastide, David Navarre, Philippe Palanque, Amélie Schyn, and Pierre Dragicevic. 2004. A model-based approach for real-time embedded multimodal systems in military aircrafts. 6th int. conference on Multimodal interfaces (ICMI '04). ACM DL, 243-250.

[15] Elaf Selim and Frank Maurer. 2010. EGrid: supporting the control room operation of a utility company with multi-touch tables. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM, 289--290.

[16] Andy Cockburn, Carl Gutwin, Philippe Palanque, Yannick Deleris, Catherine Trask, Ashley Coveney, Marcus Yung, and Karon MacLean. 2017. Turbulent Touch: Touchscreen Input for Cockpit Flight Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM DL, 6742-6753.

[17] Philippe Palanque, Sandra Basnyat, Regina Bernhaupt, Ronald Boring, Chris Johnson, and Peter Johnson. 2007. Beyond usability for safety critical systems: how to be sure (safe, usable, reliable, and evolvable)? In CHI '07 Extended Abstracts on Human Factors in Computing Systems (CHI EA '07). ACM DL, 2133-2136.

[18] Roth, E., & O'Hara, J. (2002). Integrating digital and conventional human system interface technology: Lessons learned from a control room modernization program. NUREG/CR-6749). Washington, DC: US Nuclear Regulatory Commission

[19] Ulrich, T.A., Boring, R.L.: Example user centered design process for a digital control system in a nuclear power plant. In: Proceedings of the Human Factors and Ergonomics Society 57th Annual Meeting, pp. 1727-1731 (2013)

[20] Vanessa Cobus, Wilko Heuten, and Susanne Boll. 2017. Multimodal head-mounted display for multimodal alarms in intensive care units. In Proceedings of the 6th ACM International Symposium on Pervasive Displays (PerDis '17). Association for Computing Machinery, New York, NY, USA, Article 26, 1-2. DOI:https://doi.org/10.1145/3078810.3084349

[21] Tim Claudius Stratmann, Felix Kempa, and Susanne Boll. 2019. LAME: light-controlled attention guidance for multi-monitor environments. In Proceedings of the 8th ACM International Symposium on Pervasive Displays (PerDis '19). Association for Computing Machinery, New York, NY, USA, Article 7, 1-5. DOI:https://doi.org/10.1145/3321335.3324935