# Interaction and Humans in Internet of Things

Markku Turunen, Daniel Sonntag, Klaus-Peter Engelbrecht, Thomas Olsson, Dirk Schnelle-Walka, Andrés Lucero

► To cite this version:

Markku Turunen, Daniel Sonntag, Klaus-Peter Engelbrecht, Thomas Olsson, Dirk Schnelle-Walka, et al. Interaction and Humans in Internet of Things. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.633-636, 10.1007/978-3-319-22723-8_80. hal-01610768

HAL Id: hal-01610768

https://inria.hal.science/hal-01610768

Submitted on 5 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
Interaction and Humans in Internet of Things

Markku Turunen¹ (panel coordinator), Daniel Sonntag²,
Klaus-Peter Engelbrecht³, Thomas Olsson⁴, Dirk Schnelle-Walka⁵, Andrés Lucero⁶

¹University of Tampere, Finland

markku.turunen@sis.uta.fi

²DFKI, Germany

Daniel.Sonntag@dfki.de

³Technische Universität Berlin, Germany

Klaus-Peter.Engelbrecht@telekom.de

⁴Tampere University of Technology, Finland

thomas.olsson@tut.fi

⁵S1nn GmbH & Co. KG, Germany

Dirk. schnelle@jvoicexml.org

⁶University of Southern Denmark, Denmark

lucero@mci.sdu.dk

**Abstract.** Internet of Things is mainly about connected devices embedded in our everyday environment. Typically, 'interaction' in the context of IoT means interfaces which allow people to either monitor or configure IoT devices. Some examples include mobile applications and embedded touchscreens for control of various functions (e.g., heating, lights, and energy efficiency) in environments such as homes and offices. In some cases, humans are an explicit part of the scenario, such as in those cases where people are monitored (e.g., children and elderly) by IoT devices. Interaction in such applications is still quite straightforward, mainly consisting of traditional graphical interfaces, which often leads to clumsy co-existence of human and IoT devices. Thus, there is a need to investigate what kinds of interaction techniques could provide IoT to be more human oriented, what is the role of automation and interaction, and how human originated data can be used in IoT.

**Keywords:** IoT, novel interaction means, automation

# 1 Introduction

Most successful Internet of Things applications nowadays provide quite straightforward user interfaces in comparison to the vast capabilities a smart device with smart materials, sensors and web access could actually provide. When thinking of the most successful practical IoT applications, human-technology interaction is mostly focused on graphical mobile or web interfaces – and in some cases embedded interaction devices such as touch screen panels. Typical examples include “smart home” devices,
such as thermostats, ovens, and light fixtures. Since it is assumed that the IoT devices
operate mostly automatically, the control interfaces are mainly targeted for initial
setups or other more individual configuration operations. Another frequent task is
monitoring of the device state. Situated control of IoT devices is less frequent in cur-
rent applications, but may gain importance once more of our everyday objects are
connected to the IoT. For situated control, it is particularly desirable that we can in-
teract with the device directly, rather than indirectly using graphical user interfaces.
Moreover, the smart capabilities of IoT devices should be exploited for a more hu-
man-oriented, i.e. natural and tangible, interaction. To make IoT devices more human
oriented we need to explore novel ways to interact with them in our daily life, re-
search how user-friendly automation can be achieved, and analyze how human origi-
nated data can be utilized. In this panel, the key topics related to interaction in IoT are
covered from these perspectives.

## 2 Interaction techniques for IoT

As stated, most human-technology interfaces in IoT are still rather traditional, and usually based on GUIs. However, more natural multimodal interaction means, such as spoken and gestural interaction, for example, could provide more efficient and pleasant interaction with IoT devices. Still, such interfaces are not studied widely in IoT context, even there are some cases, which are mostly related to “non-serious” applications such as toys and other gadgets. What we need are success stories – or at least some stories – from experiences of different interaction techniques which have been studied in everyday IoT scenarios. The participants of the panel will demonstrate concrete examples and findings from novel IoT interfaces.

## 3 Automation and Interaction

One of the most interesting aspects of many IoT applications is the relation of auto-
mation and human control. For example, when IoT devices are used to automate cer-
tain functions in home environments, this creates huge challenges for acceptance of
the technology, including critical factors such as safety, feeling of control, and priva-
cy, among others. The key question is in which cases automation is more suitable and
when human control should be preferred. The major factors for successful design for
co-existence of automation and explicit interaction of IoT devices are discussed.

## 4 Human-related information in IoT

In addition to the explicit interaction between humans and devices, human originated
information is often highly valuable or even crucial in many IoT applications. Most
obvious examples include IoT applications monitoring humans. Practical examples
include self-monitoring applications, which are typically related to health and wellbe-
ing. Other examples include applications for monitoring of other people, such as chil-
dren and elderly. Topics for discussion include novel means to gather, process and distribute human-oriented data, as well as key questions related to the human-factors, such as safety and privacy. Finally, it will be discussed how all of this affects and changes human behavior in the long run.

## 5 Panel members

**Markku Turunen** is a professor of Interactive Technology at the School of Information Sciences in University of Tampere, Finland. He has been worked in the Tampere Unit for Computer-Human Interaction (TAUCHI) since 1998, leading a group on pervasive interaction. His fields of expertise include novel interaction techniques, software architectures for interactive systems, pervasive applications, interactive solutions for industrial settings, user experience of multimodal interaction, ecological valid evaluation methods (including showrooms, living labs, and long-term pilot studies) with representative user groups, and commercialization or research results. His teaching portfolio includes multiple courses on pervasive and multimodal interaction. He is also a CEO of Multisense Oy, a startup company specializing on advanced human-technology interaction.

Dr. **Daniel Sonntag** is a principal researcher and principal investigator/project leader at DFKI (Intelligent User Interfaces), reader at Saarland University and University of Kaiserslautern, and permanent member of the editorial board of the German Journal on Artificial Intelligence (KI). He has worked in natural language processing, multimodal interface design, dialogue systems, and knowledge representation for over 17 years and leads a team of more than 10 full-time researchers and engineers at DFKI, and 6 (PhD/Msc) students. His research interests include common-sense and machine learning methods for multimodal human computer interfaces and knowledge discovery, mobile interface design, cognitive modelling with ontologies, and usability for the Internet of the Future. He is particularly interested in practical methods for Augmented Cognition and Mixed Reality, Compensating Cognitive Impairments, (Medical) Cyber-Physical Systems, and Human-Machine Collaboration in Production.

**Klaus-Peter Engelbrecht** is working as a Postdoc at the Quality and Usability Lab, Telekom Innovation Laboratories, Technische Universität Berlin. He studied Communication Research and Musicology and received his Magister degree in 2006 from Technische Universität Berlin. In July 2011 he successfully defended his Dissertation thesis "Estimating Spoken Dialog System Quality with User Models". His more recent work deals with modeling how users experience interactions with automated dialog partners, and how this impacts their quality judgments. From 2013 through 2014, he managed the Forschungscampus project "Interaktion & Sensoren" funded by the Federal Ministry of Education and Research of Germany. In this project, novel ways of interacting with technology in the smart home were designed and evaluated.
**Thomas Olsson** is an adjunct professor of User Experience of Socio-Technical Systems at the Department of Pervasive Computing in Tampere University of Technology (TUT), Finland. He received the Dr. Tech. degree in 2012 with a thesis addressing user experience and user expectations of future mobile augmented reality systems. Currently he leads a research team focusing on social technologies that aim to enhance social interaction and collaboration between co-located people, utilizing technologies like proximity sensing, wearables and smart networked objects. His other research interests include user experience in various ubiquitous computing systems, covering, e.g., context awareness, internet of things and smart environments, as well as user-centered design and user expectations of new interactive technology.

Dr. **Dirk Schnelle-Walka** lead the "Talk&Touch" group at the Telecooperation Lab at TU Darmstadt until end of 2014. Since then, he works as a function owner speech&dialog for S1nn in the automotive industry to take his research portfolio to an industrial level. His research focus is on voice-centric multimodal interaction in smart spaces. He authored more than 50 book chapters, journal article and conference papers and is chairing the IUI workshop on Interacting Smart Objects and the EICS Workshop on Engineering interactive Systems with SCXML. He is also the head behind several open source projects around speech technology, e.g. the open source voice browser JVoiceXML

**Andrés Lucero** is an Associate Professor of interaction design at the University of Southern Denmark in Kolding. His recent work at Nokia focused on the design and evaluation of novel interaction techniques for mobile phones and other interactive surfaces. He got his Masters degree in Visual Communication Design from Universidad Tecnológica Metropolitana (UTEM), Santiago, Chile. In 2004 he received a Professional Doctorate in Engineering (PDEng) degree in User-System Interaction from the Eindhoven University of Technology (TU/e) in the Netherlands, which included a one-year project in Philips Research. In 2009 he completed his PhD at the TU/e on co-designing mixed reality support tools for industrial designers. As part of his PhD work, he was a visiting researcher at the University of Art and Design Helsinki (TAIK) in Finland. His interests lie in the areas of mobile human-computer interaction, co-design, and design research.