<figure><img src="image_1.png" /></figure>

HAL
open science

# Multimodality in Embodied Experience Design:
Workshop at INTERACT 2017, Mumbai, INDIA

Mehul Bhatt, Clayton Lewis

<figure><img src="image_2.png" /></figure>

## ► To cite this version:

Mehul Bhatt, Clayton Lewis. Multimodality in Embodied Experience Design: Workshop at INTERACT 2017, Mumbai, INDIA. 16th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2017, Bombay, India. pp.533-534. hal-01680414

HAL Id: hal-01680414

https://inria.hal.science/hal-01680414

Submitted on 10 Jan 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Multimodality in Embodied Experience Design
Workshop at INTERACT 2017, Mumbai, INDIA

Mehul Bhatt<fnref n="1" />,<fnref n="2" />✉ and Clayton Lewis<fnref n="3" />

http://www.mehulbhatt.org

http://spot.colorado.edu/clayton/

**Abstract.** The workshop on Multimodality in Embodied Experience Design addresses the role of multimodality and mediated interaction for the analysis and design of human-centred, embodied, cognitive user experiences. Research topics being addressed encompass formal, computational, cognitive, design, engineering, empirical, and philosophical perspectives at the interface of artificial intelligence, cognitive science, and interaction design.

**Keywords:** Multimodality · Embodiment · Cognitive science · Artificial intelligence · Design · Media

## 1 About the Workshop

This workshop addresses the role of **multimodality and mediated interaction** for the analysis and design of human-centred, embodied, cognitive user experiences.

The workshop focusses on multimodality studies aimed at the semantic interpretation of human behaviour, and the empirically-driven synthesis of embodied interactive experiences in real world settings. In focus are narrative media design, architecture & built environment design, product design, cognitive media studies (film, animation, VR, sound & music), and user interaction studies. In these and other design contexts, the workshop emphasises evidence-based multimodality studies from the viewpoints of visual (e.g., attention and recipient effects), visuo-locomotive (e.g., movement, wayfinding), and visuo-auditory (e.g., narrative media) cognitive experiences. Modalities being investigated include, but are not limited to:

1. Visual attention (e.g., by eye-tracking), gesture, speech, language, facial expressions, tactile interactions, olfaction

<footnote n="1">¹ Örebro University, Orebro, Sweden</footnote>

<footnote n="2">² University of Bremen, Bremen, Germany</footnote>

<footnote n="3">³ University of Colorado, Boulder, USA</footnote>
2. Human expert guided event segmentation (e.g. coming from behavioural or environmental psychologists, designers, annotators, crowd-sensing)

3. Deep analysis based on dialogic components, think-aloud protocols

The **scientific agenda** of the workshop focusses on the multi-modality of the embodied visuo-spatial thinking involved in “problem-solving” for the design of objects, artefacts, and interactive people-experiences emanating therefrom.

**Universality and inclusion** in design are of overarching focus in all design contexts relevant to this workshop; here, the implications of multimodality studies for inclusive design, e.g., creation of presentations of the same content in different modalities, are also of interest.

The workshop brings together experts in:

- Human-Computer Interaction

- Spatial Cognition and Computation

- Cognitive Science and Psychology

- Artificial intelligence

- Neuroscience

- Communications and Media

- Design Studies

The workshop provides a platform to discuss the development of next-generation embodied interaction design systems, practices, and (human-centered) assistive frameworks & technologies encompassing the multi-faceted nature of embodied interaction design conception and synthesis. Contributions addressing the workshop themes from formal, computational, cognitive, design, engineering, empirical, and philosophical perspectives are most welcome.

## 2 Workshop Chairs

Mehul Bhatt is Professor at Örebro University (Sweden), and Professor at the University of Bremen, Germany. He leads the Human-Centred Cognitive Assistance Lab at the University of Bremen, Germany (http://hcc.uni-bremen.de/), and is co-founder of the research and consulting group DesignSpace (www.design-space.org). Mehul’s research encompasses the areas of artificial intelligence, cognitive science, and human-computer interaction.

Clayton Lewis is Professor of Computer Science and Fellow of the Institute of Cognitive Science at the University of Colorado, Boulder (United States). He is well known for his research on evaluation methods in user interface design, and contributions in the thinking aloud method and cognitive walkthrough methods. He was elected to the ACM CHI Academy in 2009, recognizing his contributions to the field of human-computer interaction. In 2011 he was further recognized by the ACM CHI Social Impact Award, for his work on technology for people with cognitive, language, and learning disabilities.