# Initial Steps Towards Infrastructuring Body-Centric Computing

Amir Zare Pashaei, Ilja Smorgun, David Lamas, Vladimir Tomberg

► To cite this version:

Amir Zare Pashaei, Ilja Smorgun, David Lamas, Vladimir Tomberg. Initial Steps Towards Infrastructuring Body-Centric Computing. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.640-643, 10.1007/978-3-030-29390-1_53. hal-02878648

HAL Id: hal-02878648

https://inria.hal.science/hal-02878648

Submitted on 23 Jun 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Initial Steps Towards Infrastructuring
Body-Centric Computing

Amir Zare Pashaei¹[0000-0003-3780-1242], Ilja Smorgun¹[0000-0002-5564-8416],
David Lamas¹[0000-0003-0295-453X], and Vladimir
Tomberg¹[0000-0003-3187-2955]

Tallinn University amirzp@tlu.ee

**Abstract.** Digital technologies are moving towards the human body. Designing for the body is commonly dubbed as body-centric computing (BCC) and includes two sides: 1) the neuro-science side; 2) the technical side mainly focuses on establishing and enabling the interactions. A conceptual framework enabling the systematic design and development of body-centric applications is introduced. Its novelty and importance are assessed within a set of application scenarios.

**Keywords:** Body-centric computing · framework · implementation · scenarios.

# 1 Introduction

Body-centric computing has proven to be quite difficult to implement. According to Mueller et al. [8], such difficulty was observed while they were doing the study around the topic of BCC. Additionally, some difficulties have affected the implementation of BCC because of technological barriers [11].

According to Svanaes et al. [10] it is a design challenge to understand how it would be possible to design such an artefact, which would make it easy for the body to learn to use it so that it becomes a natural part of the body and therefore can benefit from the users *"bodily-kinesthetic intelligence"*.

Kistler proposes a generic framework "FUBI" [6] which recognises full-body gestures and postures in real-time from the data of a depth sensor integrated using OpenNI or the Kinect SDK. The Smart-Its project [2, 3] proposes a generic layered architecture for sensor-based context computation, providing a programming abstraction that separates layers for raw sensor data, features extracted from sensors (cues), and abstract contexts derived from cues. However, previous work does not provide adequate support for organising contexts in a formal structured format.

Our work tries to establish a modular infrastructure for BCC, which has not been addressed before. This modular platform:

- Would not require developers to perform additional programming

- Would support flexible interaction modalities

- Would have a structured format
- Would have a modular mechanism for querying and reasoning context and information

- Would offer the designers and researchers a foundation for the developments of their BCC applications.

## 2 Related Work

Kistler et al. [7] describe that people can interact with each other on several channels, such as speech, gestures or postures. For example, Microsoft has paved the way to controller-free user interaction by releasing Kinect [1]. Other vendors have since then adopted these interaction modalities in their own products, such as Animoji in Apple's iOS, which has been used as Realtime Performance-Based Facial Avatars [9] or BMW's HoloActive touch [4], which enables users to interact with their car's system in hands-free scenarios.

Digital technologies are moving towards having the human body at the centre of focus and all the computations should happen either on or around the body. The human body can be the core for in-body or on-body interactions, which as the result would enable novel interaction scenarios.

Mueller et al. [8] describe that the core application domains for BCC are entertainment, health, well-being and sport. It can be either improving one's bodily motions and interactions or enhancing resilience.

## 3 Conceptual Framework

Nowadays we know that a number of research is being done on smart environments, but we see that in the future the smart environment is going to go from the scale of the room to the scale of the body. The person's body is going to become the smart environment. This is what we call an on-body network of vibro-tactile interfaces (VTI).

To facilitate the connection between the interfaces in smart environments, e.g. smart homes, there are off-the-shelf products, such as smart home information hubs. These days, they are mostly provided in the form of a speaker, which controls the music at home and also controls all the other smart devices at home. We want to create a similar infrastructure, but for the body.

We are introducing a scalable, reliable and modular body-centric infrastructure. This infrastructure encapsulates underlying technologies with our integration of interaction modalities into well-established micro services that can be used as a foundation for building BCC applications. This infrastructure will have different characteristics, such as a set of predefined interaction patterns, which are going to be used for the purpose of communication in different scenarios, such as sense of presence in remote relationships, or behaviour change in environments such as classroom and others. On the other hand, the infrastructure includes interaction modalities such as vibration with specific pattern, which would be used in different interfaces such as VTIs.
The infrastructure has three layers. The first layer includes a set of VTIs that are going to be put on the person's body. These need to be designed in a way that they can communicate through a number of protocols. At the moment we are considering several communication protocols, such as WiFi, Bluetooth Low Energy, or USB. An individual VTI can be designed for:

- Input collecting data from the user's body and the surrounding environment through a set of sensors

- Output providing on-body feedback to the user through a set of actuators

- Both input and output, working as a combination of sensors and actuators.

The second layer is the central processing unit. The purpose of having this device is to facilitate the connection between the VTIs by having a ruling engine in its core. This device gathers information from the input devices and sends the correct values to the corresponding output devices, by adopting the "publish-subscribe" pattern. The central device would include software specifications on a number of levels. It is going to include a set of patterns for sending and receiving instructions and a repository for publishing vendor-specific modules. This way each VTI will be recognised by the central computing device. Otherwise the VTI will be registered and recognised as a generic device, supporting only a basic level of functionality. The third layer is the back-end cloud platform, which can be implemented on top of Microsoft Azure IoT Hub, or alternative configurations.

# 4 Application Scenarios

Body-centric wearable technology, enabled by the proposed BCC infrastructure will facilitate a variety of application scenarios in different contexts, such as social interactions in remote relationships, well-being, health care and entertainment. For instance, sense of presence in remote relationships can be enabled by a VTI placed on the body. The sense of presence, in this case, is achieved by perceiving mediated touch by skin receptors[5].

# 5 Current Status

So far, a number of initial prototypes have been developed. The current prototypes are of two types, input device and output device. The input device has built-in sensors (e.g. pressure sensors) and a processing unit, which supports Bluetooth Low Energy connection. The output device has built-in actuators (such as vibration motors/LED) and a similar processing unit for the input device. A central device has been set up to aid the sensor and actuator communication.

# 6 Future Work

The aim of this research is to enable BCC interaction scenarios by proposing an infrastructure, which would be used as the foundation for BCC developments.
The purpose is to enable the BCC developments not only in controlled lab envi-
ronments, but also to make it possible outside the lab so that everyone regardless
of their location would be able to use it. In the future it would be possible to
analyse how using this infrastructure would change people's behaviour and how
people would use this infrastructure for different purposes. It is also important
to understand how this infrastructure would affect people's lives and what would
be the challenge for designers in their BCC solution.

7 Closing Remarks

An open infrastructure is being proposed to aid the design of BCC solutions. At the moment, the infrastructure is in its early stage and needs improvements. the infrastructure can be used in both lab and outside the lab.

References

1. Gabel, M., Gilad-Bachrach, R., Renshaw, E., Schuster, A.: Full body gait analysis with kinect. In: Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE. pp. 1964–1967. IEEE (2012)

2. Gellersen, H., Kortuem, G., Schmidt, A., Beigl, M.: Physical prototyping with smart-its. IEEE Pervasive computing **3**(3), 74–82 (2004)

3. Gellersen, H.W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Networks and Applications **7**(5), 341–351 (2002)

4. Hermann, D.S.: Automotive displays-trends, opportunities and challenges. In: 2018 25th International Workshop on Active-Matrix Flatpanel Displays and Devices (AM-FPD). pp. 1–6. IEEE (2018)

5. Israr, A., Abnousi, F.: Towards pleasant touch: Vibrotactile grids for social touch interactions. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. p. LBW131. ACM (2018)

6. Kistler, F.: Full body interaction: Design, implementation, and user support (2016)

7. Kistler, F., Endrass, B., Damian, I., Dang, C.T., André, E.: Natural interaction with culturally adaptive virtual characters. Journal on Multimodal User Interfaces **6**(1-2), 39–47 (2012)

8. Mueller, F., Andres, J., Marshall, J., Svanæs, D., Gerling, K., Tholander, J., Martin-Niedecken, A.L., Márquez Segura, E., van den Hoven, E., Graham, N., et al.: Body-centric computing: results from a weeklong dagstuhl seminar in a german castle. interactions **25**(4), 34–39 (2018)

9. Pauly, M.: Realtime performance-based facial avatars for immersive gameplay. In: Proceedings of Motion on Games. pp. 23–28. ACM (2013)

10. Svanaes, D., Solheim, M.: Wag your tail and flap your ears: The kinesthetic user experience of extending your body. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 3778–3779. ACM (2016)

11. Yun, J., Ahn, I.Y., Choi, S.C., Kim, J.: Tteo (things talk to each other): Programming smart spaces based on iot systems. Sensors **16**(4), 467 (2016)