# Visualizing Information Retrieved from (Large) WHAT Networks

Gerrit van Der Veer, Achim Ebert, Nahum Gershon, Peter Dannenmann

► To cite this version:

Gerrit van Der Veer, Achim Ebert, Nahum Gershon, Peter Dannenmann. Visualizing Information Retrieved from (Large) WHAT Networks. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.735-740, 10.1007/978-3-030-29390-1_72. hal-02878635

HAL Id: hal-02878635

https://inria.hal.science/hal-02878635

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Visualizing Information Retrieved from (large) WHAT
Networks

Gerrit van der Veer<fnref n="1" />, Achim Ebert<fnref n="2" />, Nahum Gershon<fnref n="3" />, and Peter Dannenmann<fnref n="4" />

**Abstract.** At INTERACT 2017 the organizers of this workshop had organized a very successful workshop called “Beyond Computers: Wearables, Humans, And Things – WHAT!<fnref n="1" />” Following that workshop, which focused on new and emerging WHAT technologies themselves, in this workshop, we want to address networks of WHATs as well as visualization- and interaction-technologies to deal with data emerging from these networks. Using WHATs as sensors, large networks of these provide large amounts of information. Such networks can range e.g. from smartwatches monitoring and transmitting the health status of participants in a fun run via smart tags monitoring and transmitting the storage conditions of perishable foods up to large networks of smart sensors monitoring environmental conditions in some urban or industrial environment. The large amounts of data generated from such networks need new paradigms for retrieving relevant information. In this workshop, we want to discuss such new paradigms for interacting with data generated from large networks of WHATs as well as new methods of information visualization to utilize these sensor data. Furthermore, utilizing WHATs themselves as means for interaction with and visualizing information contained in these large amounts of data will be a topic of this workshop.

**Keywords:** Wearable Devices, Implantable Devices, Internet of Things, Interaction, Information Visualization.

# 1 Introduction

In INTERACT 2017’s workshop “Beyond Computers: Wearables, Humans, And Things – WHAT!!”[1] the changing relationship between humans and computers was taken up. In the workshop, the participants discussed interaction techniques that arose

<footnote n="1">¹ Department of Computer Science, Vrije Universiteit Amsterdam, The Netherlands</footnote>

<footnote n="2">² Computer Graphics and HCI Group, University of Kaiserslautern, Germany</footnote>

<footnote n="3">³ The MITRE Corporation, United States of America</footnote>

<footnote n="4">⁴ Department of Engineering, RheinMain University of Applied Sciences, Germany</footnote>

<footnote n="1">¹gerrit@acm.org, ²ebert@cs.uni-kl.de,</footnote>

<footnote n="3">³schmooz@mac.com, ⁴peter.dannenmann@hs-rm.de</footnote>

<footnote n="1">¹ The terms WHAT or WHAT! (Wearables, Humans, And Things) were coined by Nahum Gershon and Steve Mann [e.g., see "Wearables, Humans, And Things: The Veillance Games People Play", IEEE-GEM 2015, or "Wearables, Humans, and Things as a Single Ecosystem!", IEEE Internet of Things 2015]</footnote>
from the fact that computer chips migrated from the computer internal organs to many
other devices - to things, wearables, and even onto the skin (skinnables) and into the
human body (implantables). Participants discussed the effects of this revolution on the
way we look at the relationships between humans and among humans, human ele-
ments and computing devices and what should be done to improve these interactions
and “entanglements” and to understand them better.

This workshop enhances the ideas developed in the previous workshop by discuss-
ing new interaction paradigms and mechanisms that arise from the large amounts of
data that emerge from numerous WHATs that increasingly populate our environment,
our skin, and even our bodies. These WHATs include smart textiles [2], smartwatches
[3], smart implants [4], and of course the huge variety of items related to the Internet
of Things.

With such WHATs becoming more and more popular as unobtrusive aides in our
daily lives, these devices create huge amounts of data. In order to provide good sup-
port to our daily routines, information, which is relevant for these routines and that is
contained in these data can be analyzed and visually presented to the users. The deal-
ing with these large amounts of data, the retrieval and visualization of the information
contained in these data and even the use of WHATs to interact with these data will be
the topic in the focus of this workshop.

## 2 Objectives

This workshop is intended to act as a platform for discussions about

1. interacting with large amounts of data generated by wearable and implantable devices embedded into our environment. Typically, such devices are connected to some network and typically, such devices exchange data among each other. Wearable and implantable devices enhance the physiological and / or mental capabilities of their wearers and by interacting with other devices, they provide valuable information to them. However, this information typically is hidden in large amounts of data. Relevant information has to be retrieved from the data and subsequently has to be visualized in a suitable manner in order to be of use for the wearer of the wearable or implantable device. In this workshop, new paradigms for interacting with these large amounts of data generated by networked WHATs shall be discussed in conjunction with new paradigms for retrieving and visualizing the information contained in these data.

2. interacting with the same kind of data generated by elements of the Internet of Things. In addition to wearable of implantable devices, such objects provide various kinds of data (and information) about our environment. Although the objects generating the data and the type of information contained in them differ from wearable and implantable devices, the methods to interact with the data and to retrieve and visualize information contained in the data is quite similar to the aforementioned methods.

3. using wearable and implantable devices as means of interaction and for visualizing information retrieved from the data generated by networks of wearable or implant-
able devices or data generated from devices connected to the Internet of Things. When using wearable or implantable devices, it is quite natural to use them also for interacting with or visualizing the information contained in the data generated by the large sensor networks made up of various connected devices.

Examples of such large sensor networks are connected smartwatches during e.g. some fun run, transmitting the health status of the run's participants. Giving medical personnel along the track relevant information about the health status of the runners, they can take quick action in case of emerging medical problems.

Another example can be smart tags on perishable food goods. Giving shoppers access to information provided by such smart tags can help them making informed decisions about which goods to buy. On the other hand, shop employees know which goods have not been stored in proper conditions and can prevent these goods from being sold.

As a third example, smart sensors in some environmental or industrial environment gather a lot of data about the status of the environment, may it be the amount of pollution or may it be the health status of the industrial process. Providing relevant information about the environment to decision makers can vastly improve the quality of their decisions.

Wearable or implantable devices can provide completely new paradigms of presenting relevant information about the above-mentioned situations to decision makers. Interacting with these data to filter out relevant information is greatly changed when using such devices as tools for quickly alerting personnel in case of any problems or just for plain information about current states of the system or situation under supervision.

In this workshop, we want to facilitate discussions among scholars, practitioners, and students to develop these needed new paradigms for interacting with such large amounts of sensor data created by networks of WHATs. These new paradigms include new methods for retrieving relevant information from the huge amounts of data made available by large networks of WHATs. We are especially interested in methods for defining which information is relevant and how new visualization and interaction paradigms can be tailored to fit this definition of “relevant”

We want to encourage the workshop's participants to think together about how to realize such the new interaction, engagement, and relationship between technology, humans, “modified” humans, the new reality and the data and information emerging from these elements.

We encourage researchers and practitioners to share their ideas and experience for these new realities of interaction, engagement, and interface mechanisms with the community. Researchers and practitioners from the areas of the Internet of things (IoT), wearables, implantables, skinnables, and embedded computing are in particular encouraged to participate.
## 3 Topics

In this workshop, we want to discuss topics related to interaction mechanisms suitable for large amounts of data emerging from networks of WHATs and the information contained therein. Additionally, we want to find new methods for an efficient information retrieval from these data as well as visualization methods suitable for such retrieved information. Especially we want to discuss, how WHATs themselves can help us to interact with these data and facilitate an easy search for and visualization of information in these data that is relevant for the user's task at hand. Therefore, topics to be discussed by the participants of this workshop include but are not limited to:

* What kinds of data generated by WHAT networks do we consider as sources of information that should be shared?

* Which different kinds of user groups should have access to and interact with data generated from WHAT networks?

* How can we define what information contained in data generated by WHATs is relevant to a user?

* Do we need an interactive process for information retrieval from WHAT-generated data or can such a process be automated?

* Is there a common interaction paradigm for shared WHAT network data or may there be several independent interaction paradigms? Especially: Is there a difference in interaction needs depending on the source of WHAT-generated data (i.e. is there a difference in interaction needs between data generated by wearables, implantables, or by elements of the Internet of Things)?

* How is the privacy of data ensured when a user takes part in a WHAT network?

* How can relationships among WHATs (e.g. mere interaction vs. symbiosis) be described?

* Could we define a scale from minimal interaction to full symbiosis?

* Holistic views: When does a group of WHATs become a team, group of organisms, or agents?

* What new interaction mechanisms are necessary to use wearable and implantable devices to interact with information retrieved from large networks of WHATs?

* How can users browse large amounts of data using WHATs?

* Which senses (e.g. touch and smell) can be used to interact with such information?

* What is the role of system thinking and practice in dealing and managing arrays of WHATs?

## 4 Target Audience

Interacting with WHATs is a truly interdisciplinary task. On the one hand, WHATs are applied in a variety of scientific disciplines and on the other, quite a number of disciplines use data generated by WHATs and the information retrieved from these. Moreover, various disciplines are involved in gathering the data, processing them and finally in the information retrieval.
This workshop should act as a platform for discussions among the different scientific disciplines that are involved in this kind of interaction. Therefore, we encourage researchers, scholars, and students interested in interdisciplinary discussions and unconventional solutions that cover several domains, to participate in this workshop. We especially expect that participants from the following domains might be interested in presenting and discussing their position papers in this workshop:

* Interaction Design

* Cognitive Science / Cognitive Psychology

* Visualization and Multimedia

* Artificial Intelligence and Robotics

* Developers and practitioners of wearables, implantables, skinnables and the Internet of Things

* Engineers applying networks of IoT-elements e.g. for SCADA purposes

* Common sense practitioners

# 5 Expected Outcome

Participants in the workshop will present their ideas in position papers that are made available to the workshop attendants prior to the workshop. During the workshop, the participants will present their ideas in brief workshop talks, followed by an immediate discussion with the other participants. Position papers and talks are grouped according to specific topics like “user modelling”, “interaction among WHATs”, or “interaction paradigms for large amounts of Data from WHAT networks”. For every topic we expect several position papers or talks, respectively. In order to foster an extensive discussion of the presented topics, a specific discussion block will be held after the presentations related to one topic. In this discussion block, the workshop participants will reflect on all the presentations of the respective block. The position papers will be discussed openly. By taking this approach, we expect the workshop participants to develop new insights in interaction mechanisms among WHATS and with large amounts of data and information gathered from networks of WHATs. Specifically, by grouping the position papers by topics and discussing the papers of a whole block in general, we expect a more focused discussion where the single participants’ ideas will complement each other providing a creative environment to develop new ideas for interaction mechanisms with WHATs.

We expect these insights to further the dissemination of WHATs by offering more intuitive interaction mechanisms with and among them. The results of the discussions will be noted down already during the workshop and edited in such a way that they can be presented to the main conference later during a session where all workshop results will be discussed.

In preparation of the workshop we encourage potential participants already to join discussions in advance in the Facebook page: Wearables, Humans And Things - WHAT (https://www.facebook.com/WHAT2016/

After the Workshop, we will keep in touch with the participants in order to continue the discussions started at the workshop. For this purpose, again we want to use the
Facebook page and we will encourage the participants to send extended and revised versions of their position papers to the organizers for publishing e.g. in a Springer LNCS book or in a special issue of a suitable journal/magazine.

## References

1. Gershon, N., Ebert, A, van der Veer, G., Dannenmann, P.: Beyond Computers: Wearables, Humans, And Things - WHAT! In: Bernhaupt, R., Dalvi, G., Joshi, A., Balkrishan, D., O'Neill, J., Winckler, M. (eds.) Human-Computer Interaction - INTERACT 2017, 16th IFIP TC 13 International Conference, Mumbai, India, September 25-29, 2017, Proceedings, Part IV, LNCS, Vol. 10516, pp. 515-517. Springer, Heidelberg (2017).

2. Fraunhofer Institute for Reliability and Microintegration IZM: Smart Textiles, https://www.izm.fraunhofer.de/en/abteilungen/systemIntegrationinterconnectiontechnologien/arbeitsgebiete/smart_textiles.html, accessed May 30th, 2019

3. Zhang, C., Yang, J., Southern, C., Starner, T., Abowd, G.: WatchOut: extending interactions on a smartwatch with inertial sensing. In: Beigl, M.: Proceedings of the 2016 ACM International Symposium on Wearable Computers, ACM Digital Library, New York (2016)

4. Ledet, E. H., Liddle, B, Kradinova, K, Harper, S.: Smart implants in orthopedic surgery, improving patient outcomes: a review. In: Innovation and Entrepreneurship in Health, Volume 5, pp. 41-51, Dovepress (2018)