# Jammify: Interactive Multi-sensory System for Digital Art Jamming

Sachith Muthukumarana, Don Samitha Elvitigala, Qin Wu, Yun Suen Pai, Suranga Nanayakkara

► To cite this version:

Sachith Muthukumarana, Don Samitha Elvitigala, Qin Wu, Yun Suen Pai, Suranga Nanayakkara. Jammify: Interactive Multi-sensory System for Digital Art Jamming. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.23-41, 10.1007/978-3-030-85607-6_2. hal-04291189

HAL Id: hal-04291189

https://inria.hal.science/hal-04291189v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Jammify: Interactive Multi-Sensory System for Digital Art Jamming

Sachith Muthukumarana¹, Don Samitha Elvitigala¹, Qin Wu¹,
Yun Suen Pai², and Suranga Nanayakkara¹

¹ Augmented Human Lab, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand
{firstname}@ahlab.org¹

² Empathic Computing Laboratory, Auckland Bioengineering Institute, The University of Auckland, Auckland, New Zealand
yspai1412@gmail.com²

**Abstract.** As social distancing is becoming the new normal, technology holds the potential to bridge this societal gap through novel interaction modalities that allow multiple users to collaborate and create content together. We present Jammify, an interactive multi-sensory system that focuses on providing a unique digital art-jamming experience with a visual display and a wearable arm-sleeve. The ‘jamming-canvas’ visual display is a two-sided LED light wall (2m × 6m) where users can draw free-hand gestures on either side and switch between two view modes: own-view and shared-view. The arm-sleeve uses shape-memory-alloy integrated fabric to sense and re-create a subtle and natural touch sensation on each other’s hands. We describe the details of the design and interaction possibilities based on the diverse combinations of both input and output modalities of the system, as well as findings from a user study with ten participants.

**Keywords:** Digital Drawings · Large Displays · Collaborative Drawing · Crowd and Creativity · I/O Interface · Haptics · Touch

## 1 Introduction

Dynamic media jamming interfaces allow users to engage and compose improvised artworks [11]. Collaborative drawing can be considered as one potential jamming application that promotes creativity between individuals [38]. When two individuals are engaged in collaborative drawing, the shared-canvas is converted into a common-platform in which they can communicate their creativity and emotions [7, 44, 75]. Previous studies reveal that large displays enable interactions from afar [61], as well as interactions that are simple [66, 30]. Therefore, enabling collaborative drawings in a large display could be used to provide a unique art-jamming experience for users [65].
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Jammify comprised of a jamming-canvas based on a two-sided large LED display (a,b) that enables collaborative drawing with free-hand gestures (c). A wearable forearm augmentation (d) provides additional feedback by recreating natural touch sensations.</figcaption></figure>

Building upon prior work on large displays towards collective playful interactions [45, 14] and the current state of the world, we present Jammify. Our system comprises a shared drawing canvas based on a two-sided large-scale LED display. Two individuals can use Jammify to draw with free-handed gestures on each side, creating a novel system for playful, open-ended interactions, and casual creativity for everyday pedestrians in public spaces. Both users are interconnected through a wearable arm-sleeve which can sense and recreate a natural touch sensation on the skin. The arm-sleeve, that extends prior work [52], acts as an integrated [49] communication channel between the two users, allowing them to sense and share subtle affective touch reactions on each other's forearms while maintaining appropriate social distancing. For the visual modality, two users can switch between different modes: own-view and shared-view, through the two conversely connected displays. We see this as an important aspect to consider in collaboration, given that shifts to using remote-based systems largely lack a multi-sensory experience. Although remote tools have vastly improved over the years, we are looking towards an assistive augmentation [34], that also facilitates multi-sensory feedback among users for shared content creation.

In this paper, we describe the technical implementation and the interaction possibilities of the system. In addition, we discuss how Jammify enables multiple interaction possibilities, such as collaborative, competitive, and exploratory tasks. Finally, we evaluated the effects of Jammify on user perception and experience across various proposed interaction modalities. In summary, we contribute with: 1) a system that allows user to experience a unique digital art-jamming experience based on visual feedback and the sensation of touch. 2) user evaluation of the Jammify with ten users to understand users' perception of the digital art jamming experience.

## 2 Related Work

Our research is based upon prior work on (1) haptic-affect interfaces that empower person-to-person interaction, (2) jamming interfaces that allow users to
engage in improvised collaborative tasks, as well as (3) interactive large displays that enable multi-user interactions in-the-wild.

## 2.1 Haptic-Affect Interfaces

There has been a significant amount of work focusing on haptic interfaces that enable interactions between multiple users to express affections [50, 21]. Particularly, interacting with multi-modal affective feedback methods, such as visual and haptic modalities, has been proven to be more influential in enabling an immersive user experience [2, 36, 74]. Previous research [9, 8] imply the possibility of making people co-present by sharing and communicating touch feelings through abstract interfaces. Technological advancements with vibration motors renders vibrotactile feedback as one of the most utilised modes to convey affect-related information [71, 57, 58]. However, non-vibrating interfaces, such as Shape-Memory Alloy (SMA) based wearable interfaces [28, 52, 42, 67, 31, 51, 54] are becoming a popular alternative for vibrotactile interfaces as they are more natural and expressive [31]. Moreover, SMA demonstrates multiple advantages, such as efficiency in large-amplitude actuation [35], competitive Power-to-Weight ratio [41], and a smaller form factor. Some of the early explorations concerning SMAs such as Sprout I/O [16], HapticClench [29], and Tickler [43] exhibit insights about using SMAs in the research domain. Nakao et al. proposed ShareHaptics [55], an SMA-based haptic system that allowed multiple remote participants to share haptic sensations via the hands and feet depending on the application. Recently, *Springlets* [31] explored six expressive sensations on the skin using a single flexible sticker based on SMA springs. *Touch me Gently* [52] proposes a forearm augmentation that can mimic the natural touch sensations on the forearm by applying shear-forces on the skin using an SMA matrix configuration. In this work, we build upon *Touch me Gently* [52] to sense touch and share emotive information between a pair of users.

## 2.2 Art Jamming Interfaces

Dynamic media jamming interfaces allow multiple users to engage with generative multimedia systems to compose impromptu artworks, particularly in the audiovisual domain [11]. They mainly focus on multi-user interaction and the interfaces associated with collaborative interactions. These have gained an increasingly important role over single-user multimodal interfaces in human-centered designs, as they facilitate multiple users to work in group tasks [6, 20]. For example, VJing is a format of real-time visual performance art that conducts concurrently with music sessions as a visual backdrop, allowing users to manipulate different video streams in real-time [69]. Meanwhile, drawing can be a potential collaborative platform for users to perform jamming sessions. Even before the invention of languages, drawing was one of the earliest ways in which humans conveyed expressions and ideas [12, 24]. Nowadays, users generally have the tools to become digital artists by adjusting several parameters in the particular system using graphical or physical controller interfaces [11, 33]. For instance,
*BodyDiagrams* [37] proposes an interface that enables the communication of pain
symptoms via drawing. Similarly, considerable work has been done in the medic-
inal domain to evaluate the expressiveness of drawing in aiding interpretations,
diagnosis, and recovery [26, 27, 64]. Drawing has also been used as a tool to inter-
pret certain emotions of users [72, 48, 25], essentially demonstrating the influence
of drawing in affective communication. Our work adds to this body of work by
having the display act as a bridge between socially distanced individuals and as
an interactive medium to connect them.

2.3 Interactive Large Displays

Amongst the multiple advantages of large displays, recent past work has also considerably highlighted performance benefits [62] and user satisfaction [17]. Additionally, psychological gains are also among the significant advantages of large displays as they aid memory improvement [18], peripheral awareness [17], and spatial knowledge [5], etc. Alternatively, large displays have been employed in shared surfaces across multiple users as they trigger and boost communication among unknown persons [46, 10], enhance co-located collaborative activities [53, 63], and facilitate the sense of community [63, 15]. For instance, Zadow et al. [70] explored the multi-user interaction between personal devices and large displays. In addition, Graffito [65] presents a case study that explored the ways crowd-based performative interaction happens with large-scale displays. Generally, interactive displays invite users to creatively express themselves through the display. Although most interaction methodologies share texts or images over a personalised device, such as a mobile phone [70, 3, 14, 45, 56, 73, 23], there has been considerable interest in the HCI community to blend the natural gestures of users with large display interactions [39, 1]. Sensing mechanisms such as motion tracking systems [47], mid-air gestures [10], and gaze estimations [40] have already been deployed in large displays. In this work, we combine sensing with natural touch sensation feedback for a multi-sensory experience.

3 Jammify

The goal of Jammify was to encourage users to engage in spontaneous and
impromptu interactions, either with each other or with the system itself. We
see this being especially useful currently, given that the ongoing pandemic has
negatively affected human interaction. Jammify facilitates these interactions by
being a bridge that provides both visual and haptic modalities while maintaining
social distance. Moreover, one of our primary considerations while designing
Jammify was to prevent users from physically contacting the wall or with each
other while interacting with the system.

The overall design architecture of Jammify has two main components, the
jamming canvas (public visual feedback) and the arm-sleeve (private haptic feed-
back). The jamming CANVAS is the platform where two users can perform digital
drawings using simple mid-air hand gestures. The augmented arm sleeve (see
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Jammify System Architecture: Two large displays visualize the illustrations users draw. Light controllers control the two large displays according to the gesture classifier and hand tracker sensed by two depth cameras. The augmented arm-sleeve incorporates an 8-layer design; a) Electrical insulation layer, b) Copper layer 1, c) Velostat sheet, d) Copper layer 2, e) Thermal dissipation layer, f) SMA wire layout, g) Adhesive and stretchable textile, h) Thermal and electrical insulation layer.</figcaption></figure>

*Figure 1d)* connects users with a channel to share remote touch sensations between them.

**3.1 Jamming Canvas**

The jamming-canvas consists of a large LED display and a pair of depth cameras that can track and detect free-hand gestures. The light display controllers and the depth cameras are controlled by a central computer.

**Light Display:** The Jammify canvas is a two sided 2m × 6m large canvas made using ten light panels. Each panel has four strands, illuminating 180 iColor Flex LMX LEDs installed to a plywood panel as shown in *Figure 1a*. Altogether, the entire canvas has 1800 individually addressable, full-color LED nodes. Each node produces a maximum of 6.56 candela of light output while consuming just 1W of power. The light strands were connected to 8 Philips color kinetic light controllers with an inbuilt Ethernet controller that allowed individual nodes to be addressed. We deployed the jamming-canvas right beside the main entrance of a building, facilitating impromptu interactions with Jammify among people who pass through. Initial observations show that users were curious to examine the inside of the wall to see how it is made. Therefore, we integrated a ‘behind-the-scene’ into our design by keeping an opening for the light display (see *Figure 1b*) for people to see inside.

The shared jamming-canvas operates in two modes: (1) *Own-View (OV)* allows two users to draw on the light display individually as two separate drawings, and (2) *Shared-View (SV)* enables both users to work on the same drawing simultaneously, as shown in *Figure 3e and 3f*. In the *OV* setting, two users work on their own illustration independently on the two-sided light displays. For instance, the *OV* mode is more suitable for a competitive task, whereas the *SV*
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Jammify enables multi-user interaction possibilities based on the diverse combinations of the following modes; (a) *One-to-One (O2O)*, (b) *Gestures-to-Shapes (GTS)*, (c) *Touch-to-Touch (T2T)*, (d) *Gestures-to-Gestures (G2G)*, (e) *Own-View (OV)*, (f) *Shared-View (SV)*, (g) *Feeling-User (FU)*, and (h) *Feeling-Drawing (FD)*.</figcaption></figure>

mode is more suitable for a collaborative task. When no participants are present, all the lights of the display start fading in and out slowly, creating an appealing effect [32] for the people passing by.

**Depth Camera:** To control lights using hand gestures, we mounted two Intel RealSense D435 depth cameras [13] on top of both sides of the canvas (see Figure 2). To track hand movements, we wrote a Unity application using Unity track SDK [59]. The position of the hand is mapped to the corresponding pixel coordinates of the light display. To enable drawings on the canvas, we interfaced the Intel depth cameras with iColor Flex LMX LEDs by using a software controller implemented using ‘color kinetics’ Java Processing API, along with a TCP library to interface with Unity.

We propose two ways of transforming the interaction in front of the display (see Figure 3a and 3b) based on the geometrical location of the hand: (1) *One-to-One (O2O)* maps the position of the hand to the one-to-one pixel coordinate of the display, and (2) *Gestures-to-Shapes (G2S)* maps the free-hand gestures, namely *swipe-up*, *swipe-down*, *swipe-left*, and *swipe-right* to approximations of pre-defined shapes, namely circles, triangles, squares, and lines respectively to create collage type illustrations. Both users can switch between different modes by performing a *hand-waving* gesture. Holding the hand in the same spatial location for a period of three seconds would allow the user to choose different colours for the drawing. Furthermore, users can reset the jamming-canvas by performing a *push* gesture. Jammify supports one user per side of the set-up. We marked a specific location in front of the display where users can interact. When a user interacts with the system, other people, such as onlookers, are filtered out based on the horizontal distance from the display.
## 3.2 Augmented Arm Sleeve

We propose the inclusion of arm sleeves for two key reasons. Firstly, we wanted to pair a haptic modality with the jamming canvas to transmit subtle touch sensations among users. Secondly, we are looking towards an easy-to-put-on wearable device with a relatively large surface area for interaction with the hands. Arm sleeves [52] is one such device that enables users to pass subtle emotive touch responses among each other. Furthermore, it is easy to wear, as the forearm is a suitably large surface area without being obtrusive. The arm sleeve comprises of two main components; (1) *Sensing Layer Stack* to sense and classify the touch gestures performed on the forearm using a Force-Sensitive Resistor (FSR) matrix, and (2) *Actuation Layer Stack* to recreate the touch sensation of the skin by actuating an SMA-based plaster matrix, placed on top of each other on the forearm of the user.

**Sensing Layer Stack:** The sensing layer stack is based on 15 FSRs in a 5 × 3 matrix that covers most of the forearm area. A pressure sensitive conductive sheet (*Velostat*) was placed in between two non-attached copper layers (see Figure 2a to 2d). Depending on the applied force on the node, the Velostat changes the conductivity and the connectivity of the two non-attached copper nodes. An overall sampling rate of 50Hz was achieved by switching between five rows of digital outputs and three analog inputs. The topmost layer is an electric insulation layer to eliminate direct contact between the copper nodes and the skin.

The sensing layer stack of the augmented arm-sleeve incorporates the real-time 2D coordinates associated with the touch gestures performed on the forearm. Based on these touch locations, Jammify allows users to interact in two ways (see Figure 3c and 3d): (1) *Touch-to-Touch (T2T)* that maps the touch locations of the hand to the exact SMA patch on the augmented arm-sleeve of the other user, and (2) * Gestures-to-Gestures (G2G) that classify six touch gestures performed on the forearm (stroking down the arm, stroking up the arm, grabbing the arm, grabbing the wrist, encircling on the arm, and encircling on the wrist) and activates the same actuation pattern in the SMA matrix.*

**Actuation Layer Stack:** We developed the actuation part of the forearm-sleeve based on recent research work that demonstrated recreating the perception of natural touch sensation on the skin [52]. This was implemented with an SMA-based flexible plaster matrix design with four layers, as shown in Figure 2e to h, to generate shear-forces on the skin. In Jammify, we added a top layer to allow both sensing and feedback.

The arm-sleeve interface of Jammify also has two operating modes: (1) *Feeling-User (FU) mode* that allows users to share touch responses via the forearm augmentation, and (2) *Feeling-Drawings (FD) mode* that enables users to feel the artworks drawn on the jamming-canvas (see Figure 3g and 3h). In FU, a user can receive haptic responses in the form of feedback, guidance, or reaction for the drawn elements shared by the other user. We identified this mode as most
<table><caption>**Fig. 4.** Jammify's Interaction space composes two dimensions: Number of users (*Single-user Interactions* versus *Two-user Interactions*) and the employment of the augmented arm-sleeve (*With the Augmented Arm-sleeve* versus *Without the Augmented Arm-Sleeve*).</caption><thead><tr><th rowspan="2"></th><th colspan="2">Single-user Interaction</th><th colspan="2">Two-user Interaction</th></tr><tr><th>Depth Camera<br>One-to-One (O2O)<br>Gestures-to-Shapes (G2S)<br>FSR matrix<br>-</th><th>Light Display<br>Own-View (OV)<br>SMA matrix<br>Feeling-Drawing (FD)</th><th>Depth Camera<br>One-to-One (O2O)<br>Gestures-to-Shapes (G2S)<br>FSR matrix<br>Touch-to-Touch (T2T)<br>Gestures-to-Gestures (G2G)</th><th>Light Display<br>Own-View (OV)<br>Shared-View (SV)<br>SMA matrix<br>Feeling-Drawing (FD)<br>Feeling-User (FU)</th></tr></thead><tbody><tr><th>With the Augmented<br>Arm-sleeve</th><td></td><td></td><td></td><td></td></tr><tr><th>Without the Augmented<br>Arm-sleeve</th><td></td><td></td><td></td><td></td></tr></tbody></table>

suitable for a collaborative task where users can share several responses, such as emotions via haptic reactions. In the *FD* mode, we activate the SMA plaster matrix when the user hovers the hand over the boundaries (edges) of the drawn elements. This mode is more suited for single-user interactions.

## 3.3 Interaction Space

Jammify incorporates a combination of two types of inputs to the system (depth camera and FSR matrix), as well as two output modalities controlled by the system (light display and SMA matrix), as mentioned above. In this section, we suggest how these interactions can be combined: *Single-user Interactions* versus *Two-user Interactions* in terms of the number of users, and *With the Augmented Arm-sleeve* versus *Without the Augmented Arm-Sleeve* in terms of the utilization of the Augmented Arm-sleeve. Depending on the design requirements, Jammify can refer to different combinations of operation modes (as shown in Figure 4) to facilitate various types of applications. We choose three potential types of applications that Jammify can assist; Collaborative tasks where users can work together or a single user can collaborate with Jammify, Competitive tasks in which two users can compete with each other or a single user can compete with Jammify, and Exploratory tasks where single users can free-play with Jammify.

**Collaborative Tasks:** Research work on collaborative drawing runs back several decades [68, 60], and its potential to support creativity is well established [19, 76]. In Jammify, when a pair of users interact with the system in a collaborative task, we propose *One-to-One (O2O)* as the depth camera mapping mechanism to draw on the canvas freely, *Shared-View (SV)* as the display mode for the shared view, *Gestures-to-Gestures (G2G)* mapping of the FSR matrix, and *Feeling-User (FU)* mode as the SMA matrix configuration to share real-time haptic responses. Similarly, a single user can also accomplish a collaborative task with Jammify. In this case, Jammify provides a prepared low-resolution image,
allowing the user to modify the drawing as needed. In contrast to the two-user condition, we suggest maintaining the light display mode as the *Own-View (OV)* mode and the depth camera mapping as *One-to-One (O2O)* in the single-user mode. When a single-user uses an augmented arm-sleeve, it can only operate in the *Feeling-Drawing (FD)* mode.

**Competitive Tasks:** Competition tends to have positive effects on creativity [4, 22]. For a competitive task, such as two users engaging in a drawing competition with each other, we propose *Gestures-to-Shapes (G2S)* as the mapping mechanism to collage abstract illustration, *Own-View (OV)* as the display mode for having two independent views, and *Feeling-Drawing (FD)* mode as the SMA matrix setting for users to feel the drawing in an abstract way independently. One person can compete with Jammify to complete a given abstract illustration as quickly as possible. The combination of *One-to-One (O2O)* depth camera mapping mechanism, *Own-View (OV)* display mode, and *Feeling-Drawing (FD)* mode is an exemplar for a single user interaction in this case.

**Exploratory Tasks:** Exploratory interactions associated with drawings have been shown to be applicable in playful interactions [23, 77]. Likewise, in the exploratory mode of Jammify, such as a free-hand drawing interaction, users can free-play with the system without a particular objective. We propose *One-to-One (O2O)* as the depth camera configuration, *Own-View (OV)* as the display mode for the individual view, and *Feeling-Drawing (FD)* mode as the SMA matrix setting for the user to feel the drawing on the forearm. In the same way, users can engage in a simpler interplay with *Gestures-to-Shapes (G2S)* depth camera mode to create more abstract artworks by performing free-hand gestures mentioned above.

# 4 Evaluation

The goal of this user study is to understand users' perception of the digital art jamming experience Jammify provides. Ten participants (8 males and 2 females) aged between 22 and 38 years ($M = 29.7, SD = 4.29$) were recruited, and among them, four participants had previous experience with interactive displays. The jamming-canvas was deployed right beside the entrance of our institute and the user study was also conducted at the same location. At the beginning of the study, we explained the study details, instructions, and objectives to each participant. We explore several configurations including: *Arm-sleeve availability (With the Augmented Arm-sleeve, Without the Augmented Arm-sleeve)*, and *Types of the task (Collaborative, Competitive, Exploratory)*. Altogether, each participant went through six different scenarios; 2 *Arm-sleeve availability* × 3 *Types of the tasks*. In *Collaborative* tasks, an experimenter acted as the external person behind the screen. However, participants were unaware of this. We only informed them that an anonymous external person would be collaborating with them. In
<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Participants were engaged in three types of tasks: *Collaborative*, *Competitive*, and *Exploratory*. a) In the *Collaborative* task, users were given a partly completed drawing to complete as a collaborator. b) We provided a monochrome image to fill with colours in 30 seconds in *Competitive* tasks. c) The *Exploratory* task allowed the participants to interact with the jamming-canvas as they wish. d) All of the above mentioned tasks were conducted in two conditions With and Without the arm-sleeve.</figcaption></figure>

*Competitive* tasks, participants were asked to fill a monochrome image with colours within 30 seconds. To increase the competitiveness, we told them that their colouring will be evaluated. In *Exploratory* tasks, participants were allowed to interact with the display freely. The study took approximately one-hour per participant. After each condition, the participants filled out a questionnaire, asking about the emotions expressed through the interactions, negative and positive experiences, and comments on drawn illustrations. The quotes were collected using a questionnaire where participants were asked to select the most preferred mode and the reason for the selection at the end of the user study.

## 4.1 Results and Discussion

**Exploratory mode preferred by most participants:** When we asked participants about their favourite digital art jamming mode, the majority (nine out of ten) declared that they enjoyed the *Exploratory* task with the augmented arm-sleeve the most. They reported the reason as being the *"freedom"* they had during the task. For example, P1: *"I had complete freedom"*, and P4: *"It gave me freedom of drawing, so I was able to draw what came into my mind"*. The freedom that the *Exploratory* mode gives may also have awakened their creativity, which the majority of participants enjoyed. P8: *"I like exploratory mode. It doesn't have an explicit task. So no pressure, more creativity"*. Only one participant felt that the *Exploratory* task required more effort since they needed to complete the drawing solely by themselves. The majority of participants preferred the *Collaborative* task, as it was less difficult. They specifically mentioned that users could acquire more support for the drawings in this mode, especially for those who lack the confidence in their artistic skill. P2: *"I'm not good at drawing. Therefore, drawing with someone else gave me confidence and help me to do a better job at drawing"*. None of the participants mentioned that they liked *Competitive* mode. The reason behind this could be the induced temporal cognitive load created by asking them to finish the task within 30 seconds, which was not a lot of time to awaken their creativity, compared to *Exploratory* mode.
or *Collaborative* (For example, refer P8’s statement above). Furthermore, many participants found that they could not enjoy the touch feedback during the *Competitive mode*. P4: “Competitive felt hurried so I really was not able to respond to the touch feedback, but I could respond accordingly for the other modes. I enjoyed it”.

**Augmented Arm-sleeve enhanced the experience:** In terms of the user experience with the augmented arm-sleeve combined with digital art jamming, all participants identified this as the most preferred mode compared to *Without the Augmented Arm-sleeve* setting. This was also reflected during the questionnaire when participants used terms such as “*nice haptic feedback*” (P8), “*great combination*” (P5), and “*amusing feedback*” (P7) to describe the haptic sensations generated by the arm-sleeve. Participants also noted that the haptic sensation blended with the visual stimulations induced a unique interactive experience. For example, P6: “*It felt more interactive. I felt I can explore stuff in a new way. It allowed me to be more creative and play around*”, P7: “*amusing feedback generated by the arm sleeve made it more interactive*”. Moreover, the participant who liked the *Collaborative* task mentioned that the arm sleeve gave him a feeling of the presence of his counterpart, who interacts with him from the other side of the wall (in this case, experimenter). This shows that the augmented arm sleeve enhanced the participants’ experience in two main ways: increased the liveliness of the jamming experience, and increased the sense of presence of a partner interacting from the other side of the wall. Therefore, the participants’ comments further confirm that the augmented arm-sleeve did not help them during the *Competitive* task (P2, P5). The greater focus on the competitive visual task caused users not to perceive the haptic sensation, thus making Jammify less entertaining. Moreover, some of the participants enjoyed the warmth generated in the arm-sleeve as a pleasing add-on. P6: “*warmth generated in the arm-sleeve felt pleasant as the environment was cold*”. This could also be due to users perceiving the warm sensation as a touch from a physical hand, allowing us to close the perceived gap between users, even with social distancing.

**Other Observations:** We observed several notable incidents while implementing the jamming-canvas at our institute's entrance and during the user studies. As the researchers in our institute passed by, they approached us with many queries about the set-up and the technological aspects of Jammify. These inquiries led us to believe that, despite the fact researchers generally lean towards science and technology as opposed to art, Jammify could nevertheless spark a sense of curiosity regarding its implementation and deployment. After the deployment of the set-up, even individuals from non-technical backgrounds, such as cleaners, delivery persons, and security guards tried to interact with the jamming-canvas. Although we thought participants would be overwhelmed after the one-hour user study, most were still interested in playing with the system post-study. These instances showcase that peoples' desire to interact with Jammify can potentially bridge two important gaps; the first is the gap of social
distancing by allowing multi-sensory interaction between users, and the second
being the gap between users who are and are not artistically or technologically
inclined.

**Design Implications:** We discuss the design implications derived using the observations and results extracted during the design process and user study of Jammify. We learned that users did not have to physically touch the display necessarily; however, it is essential to support multiple users and show the resulting interaction between the users. Thus, when designing multi-user performative interactions [65], sharing the resulting outcome among all the users would improve the overall interaction depending on the context. Also, the illustrations projected on the large display need not be realistic. Greater abstraction in patterns are suitable as long as users can make a connection to their actions. With simple, yet never ending abstract patterns, users will find the system playful, and it would support attracting users to engage with the system. Similarly, we found that any form of pre-assigned tasks for the interaction should be minimised; rather, free-form interplay should be prioritised. Also, a system such as an interactive public wall should have easy access and allow for simple casual creativity among multiple users. The interaction should reflect this.

Nevertheless, employing an augmented arm-sleeve to enable a touch interface showed that users prefer a multi-modal feedback approach. However, there are challenges when incorporating a wearable device that users share in a crowd-based system. An ideal solution would be providing the device in the form of a disposable layer, such as a skin wrap. Such an alternative approach would remove hygiene concerns, necessity of sanitizing hands, and cleaning the device each time. Although it has practical challenges, haptics, in particular, provide enormously beneficial opportunities amidst the pandemic.

**Limitations and Future work:** We acknowledge that our current number of participants is lacking for an in-depth quantitative analysis, largely due to the pandemic situation. Thus, we plan to reserve that for future works, including a study on long-term deployment. With the forearm augmentation, we employed the same pair of forearm sleeves for all the participants to wear. We spent a significant amount of time ensuring they were sufficiently hygienic. Hygiene concerns made us realise the challenging task of re-designing the system to be either easily sterilisable or disposable. Furthermore, users may sense a slight warmth on the skin from the heat generated in the SMA wires, although there is a thermal protective layer in the forearm augmentation just above the skin. Additionally, some of participants commented on the brightness of the jamming-canvas in low light conditions. In the future, this can be resolved by employing an ambient light sensor in the implementation. Currently, the system does not restrict the drawing privileges of users. Users are permitted to draw anything in public, potentially resulting in obscene drawings. In the future, these concerns can be avoided using a machine learning algorithm.
The current implementation of the arm-sleeve can be problematic for hy-
giene, yet from an interaction design standpoint, we find the subtle touch sen-
sation from the sleeves to be quite crucial while social distancing. Although
this proof-of-concept augmented arm-sleeve provides many advantages, improv-
ing the system usability in the real-world context requires more work. Those
limitations can be overcome by manufacturing and designing the device in a
sterilisable or disposable manner. In the user study, we ensured that our current
implementation was sufficiently disinfected for users to interact with Jammify.

5 Conclusion

We developed Jammify, a multi-sensory system that enable users to share their remote touch sensations via an augmented arm-sleeve, while they engage in a collaborative drawing activity on a jamming-canvas that consists of a two-sided light display. The overall system provides multi-user interaction possibilities by enabling users to interact with free-hand gestures on each side of the wall in several modes, such as own-view and shared-view. The user interactions were promising, as this unique art-jamming system delivers an assertive experience to the users. We evaluated Jammify with ten participants in three different modes: Collaborative, Competitive, and Exploratory. We found that the Exploratory mode was most preferred by participants. Also, based on the findings, we concluded that the augmented arm sleeve enhanced the user experience with Jammify. As it maintains appropriate social distancing, we envision the concept of Jammify as a blend of haptic and visual modalities in large scale public interactive designs so that users may still create content together in these troubling times.

6 Acknowledgement

This work was supported by Assistive Augmentation research grant under the
Entrepreneurial Universities (EU) initiative of New Zealand. We thank Peter
Cleveland, Robyn Walton, and Prasanth Sasikumar for their assistance in con-
structing the LED display.

References

1. Ackad, C., Tomitsch, M., Kay, J.: Skeletons and silhouettes: Comparing user representations at a gesture-based large display. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. p. 2343–2347. CHI ’16, Association for Computing Machinery, New York, NY, USA (2016). https://doi.org/10.1145/2858036.2858427

2. Akshita, Alagarai Sampath, H., Indurkhya, B., Lee, E., Bae, Y.: Towards multimodal affective feedback: Interaction between visual and haptic modalities. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. p. 2043–2052. CHI ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2702123.2702288
3. Ananny, M., Strohecker, C., Biddick, K.: Shifting scales on common ground: developing personal expressions and public opinions. International Journal of Continuing Engineering Education and Life Long Learning **14**(6), 484–505 (2004)

4. Baer, M., Leenders, R.T.A., Oldham, G.R., Vadera, A.K.: Win or lose the battle for creativity: The power and perils of intergroup competition. Academy of Management Journal **53**(4), 827–845 (2010)

5. Bakdash, J.Z., Augustyn, J.S., Proffitt, D.R.: Large displays enhance spatial knowledge of a virtual environment. In: Proceedings of the 3rd Symposium on Applied Perception in Graphics and Visualization. p. 59–62. APGV ’06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1140491.1140503

6. Barthelmess, P., Kaiser, E., Lunsford, R., McGee, D., Cohen, P., Oviatt, S.: Human-centered collaborative interaction. In: Proceedings of the 1st ACM International Workshop on Human-Centered Multimedia. p. 1–8. HCM ’06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1178745.1178747

7. Boehner, K., DePaula, R., Dourish, P., Sengers, P.: Affect: From information to interaction. In: Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility. p. 59–68. CC ’05, Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1094562.1094570

8. Bonanni, L., Vaucelle, C.: Affective touchcasting. In: ACM SIGGRAPH 2006 Sketches. p. 35–es. SIGGRAPH ’06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1179849.1179893

9. Brave, S., Dahley, A.: Intouch: A medium for haptic interpersonal communication. In: CHI ’97 Extended Abstracts on Human Factors in Computing Systems. p. 363–364. CHI EA ’97, Association for Computing Machinery, New York, NY, USA (1997). https://doi.org/10.1145/1120212.1120435

10. Brignull, H., Rogers, Y.: Enticing people to interact with large public displays in public spaces. In: Proceedings of INTERACT. vol. 3, pp. 17–24. Brighton, UK (2003)

11. Brown, A.R., Sorensen, A.: Dynamic media arts programming in impromptu. In: Proceedings of the 6th ACM SIGCHI conference on Creativity & Cognition. pp. 245–246 (2007)

12. Burkitt, E., Barrett, M., Davis, A.: Effects of different emotion terms on the size and colour of children’s drawings. International Journal of Art Therapy **14**(2), 74–84 (2009)

13. depth camera, I.R.: https://www.intelrealsense.com/depth-camera-d435/ (2020), accessed: 2020-04-10

14. Cheok, A.D., Fernando, O.N.N., Wijesena, J.P., Mustafa, A.u.R., Shankar, R., Barthoff, A.K., Tosa, N., Choi, Y., Agarwal, M.: Blogwall: social and cultural interaction for children. Advances in Human-Computer Interaction **2008** (2008)

15. Churchill, E.F., Nelson, L., Denoue, L., Helfman, J., Murphy, P.: Sharing multimedia content with interactive public displays: a case study. In: Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques. pp. 7–16 (2004)

16. Coelho, M., Maes, P.: Sprout i/o: A texturally rich interface. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction. p. 221–222. TEI ’08, Association for Computing Machinery, New York, NY, USA (2008). https://doi.org/10.1145/1347390.1347440
17. Czerwinski, M., Robertson, G., Meyers, B., Smith, G., Robbins, D., Tan, D.: Large display research overview. In: CHI '06 Extended Abstracts on Human Factors in Computing Systems. p. 69–74. CHI EA '06, Association for Computing Machinery, New York, NY, USA (2006). https://doi.org/10.1145/1125451.1125471

18. Czerwinski, M., Smith, G., Regan, T., Meyers, B., Robertson, G.G., Starkweather, G.K.: Toward characterizing the productivity benefits of very large displays. In: Interact. vol. 3, pp. 9–16 (2003)

19. Davis, N., Hsiao, C.P., Singh, K.Y., Lin, B., Magerko, B.: Creative sense-making: Quantifying interaction dynamics in co-creation. In: Proceedings of the 2017 ACM SIGCHI Conference on Creativity and Cognition. p. 356–366. C&amp;C '17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3059454.3059478, https://doi.org/10.1145/3059454.3059478

20. Duarte, L., Carriço, L.: The collaboration platform: A cooperative work course case-study. In: 2010 Eighth International Conference on Creating, Connecting and Collaborating through Computing. pp. 19–25. IEEE (2010)

21. Eid, M.A., Al Osman, H.: Affective haptics: Current research and future directions. IEEE Access 4, 26–40 (2015)

22. Eisenberg, J., Thompson, W.F.: The effects of competition on improvisers’ motivation, stress, and creative performance. Creativity Research Journal 23(2), 129–136 (2011)

23. Elvitigala, S., Chan, S.W.T., Howell, N., Matthies, D.J., Nanayakkara, S.: Doodle daydream: An interactive display to support playful and creative interactions between co-workers. In: Proceedings of the Symposium on Spatial User Interaction. p. 186. SUI '18, Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3267782.3274681

24. Fan, J.E., Dinculescu, M., Ha, D.: Collabdraw: An environment for collaborative sketching with an artificial agent. In: Proceedings of the 2019 on Creativity and Cognition. p. 556–561. Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3325480.3326578

25. Fan, L., Yu, C., Shi, Y.: Guided social sharing of emotions through drawing art therapy: Generation of deep emotional expression and helpful emotional responses. In: Proceedings of the Seventh International Symposium of Chinese CHI. p. 65–78. Chinese CHI '19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3332169.3333571

26. Felix, E.R., Galoian, K.A., Aarons, C., Brown, M.D., Kearing, S.A., Heiss, U.: Utility of quantitative computerized pain drawings in a sample of spinal stenosis patients. Pain Medicine 11(3), 382–389 (2010)

27. Ghinea, G., Gill, D., Frank, A., De Souza, L.: Using geographical information systems for management of back-pain data. Journal of Management in Medicine (2002)

28. Gupta, A., Irudayaraj, A.A.R., Balakrishnan, R.: Hapticclench: Investigating squeeze sensations using memory alloys. In: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. p. 109–117. UIST '17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3126594.3126598

29. Gupta, A., Irudayaraj, A.A.R., Balakrishnan, R.: Hapticclench: Investigating squeeze sensations using memory alloys. In: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. pp. 109–117 (2017)
30. Gutwin, C., Greenberg, S.: A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work (CSCW) 11(3-4), 411–446 (2002)

31. Hamdan, N.A.h., Wagner, A., Voelker, S., Steimle, J., Borchers, J.: Springlets: Expressive, flexible and silent on-skin tactile interfaces. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. CHI ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3290605.3300718

32. Harrison, C., Horstman, J., Hsieh, G., Hudson, S.: Unlocking the expressivity of point lights. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 1683–1692. CHI ’12, Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2207676.2208296

33. Hettiarachchi, A., Nanayakkara, S., Yeo, K.P., Shilkrot, R., Maes, P.: Finger-draw: More than a digital paintbrush. In: Proceedings of the 4th Augmented Human International Conference. p. 1–4. AH ’13, Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2459236.2459237, https://doi.org/10.1145/2459236.2459237

34. Huber, J., Shilkrot, R., Maes, P., Nanayakkara, S.: Assistive Augmentation. Springer Publishing Company, Incorporated, 1st edn. (2017)

35. Ilievski, F., Mazzeo, A.D., Shepherd, R.F., Chen, X., Whitesides, G.M.: Soft robotics for chemists. Angewandte Chemie International Edition 50(8), 1890–1895 (2011)

36. Indurkhya, B.: Towards multimodal affective stimulation: Interaction between visual, auditory and haptic modalities. In: International Conference on ICT Innovations. pp. 3–8. Springer (2016)

37. Jang, A., MacLean, D.L., Heer, J.: Bodydiagrams: Improving communication of pain symptoms through drawing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 1153–1162. CHI ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2556288.2557223

38. Journeaux, J., Gorrill, H. (eds.): Collective and Collaborative Drawing in Contemporary Practice. Cambridge Scholars Publishing, United Kingdom (12 2017)

39. Kettebekov, S., Sharma, R.: Toward natural gesture/speech control of a large display. In: IFIP International Conference on Engineering for Human-Computer Interaction. pp. 221–234. Springer (2001)

40. Khamis, M., Hoesl, A., Klimczak, A., Reiss, M., Alt, F., Bulling, A.: Eyescout: Active eye tracking for position and movement independent gaze interaction with large public displays. In: Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. p. 155–166. UIST ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3126594.3126630

41. Kheirikhah, M.M., Rabiee, S., Edalat, M.E.: A review of shape memory alloy actuators in robotics. In: Robot Soccer World Cup. pp. 206–217. Springer (2010)

42. Knoop, E., Rossiter, J.: The tickler: A compliant wearable tactile display for stroking and tickling. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. p. 1133–1138. CHI EA ’15, Association for Computing Machinery, New York, NY, USA (2015). https://doi.org/10.1145/2702613.2732749

43. Knoop, E., Rossiter, J.: The tickler: a compliant wearable tactile display for stroking and tickling. In: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. pp. 1133–1138 (2015)
44. Leahu, L., Schwenk, S., Sengers, P.: Subjective objectivity: negotiating emotional meaning. In: Proceedings of the 7th ACM conference on Designing interactive systems. pp. 425–434 (2008)

45. Martin, K., Penn, A., Gavin, L.: Engaging with a situated display via picture messaging. In: CHI’06 extended abstracts on Human Factors in Computing Systems. pp. 1079–1084 (2006)

46. McCarthy, J.: Using public displays to create conversation opportunities. In: Proceedings of Workshop on Public, Community, and Situated Displays at CSCW’02 (2002)

47. Michelis, D., Müller, J.: The audience funnel: Observations of gesture based interaction with multiple large displays in a city center. Intl. Journal of Human-Computer Interaction **27**(6), 562–579 (2011)

48. Moon, B.L.: The role of metaphor in art therapy: Theory, method, and experience. Charles C Thomas Publisher (2007)

49. Mueller, F.F., Lopes, P., Strohmeier, P., Ju, W., Seim, C., Weigel, M., Nanayakkara, S., Obrist, M., Li, Z., Delfa, J., Nishida, J., Gerber, E.M., Svanaes, D., Grudin, J., Greuter, S., Kunze, K., Erickson, T., Greenspan, S., Inami, M., Marshall, J., Reiterer, H., Wolf, K., Meyer, J., Schiphorst, T., Wang, D., Maes, P.: Next steps for human-computer integration. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. p. 1–15. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376242, https://doi.org/10.1145/3313831.3376242

50. Mullenbach, J., Shultz, C., Colgate, J.E., Piper, A.M.: Exploring affective communication through variable-friction surface haptics. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 3963–3972. CHI ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2556288.2557343

51. Muthukumarana, S., Elvitigala, D.S., Cortes, J.P.F., Matthies, D.J., Nanayakkara, S.: Phantomtouch: Creating an extended reality by the illusion of touch using a shape-memory alloy matrix. In: SIGGRAPH Asia 2019 XR. p. 29–30. SA ’19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3355355.3361877

52. Muthukumarana, S., Elvitigala, D.S., Forero Cortes, J.P., Matthies, D.J., Nanayakkara, S.: Touch me gently: Recreating the perception of touch using a shape-memory alloy matrix. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. p. 1–12. CHI ’20, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376491

53. Mynatt, E.D., Igarashi, T., Edwards, W.K., LaMarca, A.: Flatland: New dimensions in office whiteboards. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 346–353. CHI ’99, Association for Computing Machinery, New York, NY, USA (1999). https://doi.org/10.1145/302979.303108

54. Nakao, T., Kunze, K., Isogai, M., Shimizu, S., Pai, Y.S.: Fingerflex: Shape memory alloy-based actuation on fingers for kinesthetic haptic feedback. In: 19th International Conference on Mobile and Ubiquitous Multimedia. p. 240–244. MUM 2020, Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3428361.3428404

55. Nakao, T., Santana, S.K., Isogai, M., Shimizu, S., Kimata, H., Kunze, K., Pai, Y.S.: Sharehaptics: A modular haptic feedback system using shape memory alloy for mixed reality shared space applications. In: ACM SIG-
GRAPH 2019 Posters. SIGGRAPH '19, Association for Computing Machinery, New York, NY, USA (2019). https://doi.org/10.1145/3306214.3338597, https://doi.org/10.1145/3306214.3338597

56. Nanayakkara, S., Schroepfer, T., Wyse, L., Lian, A., Withana, A.: Sonicsg: From floating to sounding pixels. In: Proceedings of the 8th Augmented Human International Conference. AH '17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3041164.3041190

57. Nanayakkara, S., Taylor, E., Wyse, L., Ong, S.H.: An enhanced musical experience for the deaf: Design and evaluation of a music display and a haptic chair. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 337–346. CHI ’09, Association for Computing Machinery, New York, NY, USA (2009). https://doi.org/10.1145/1518701.1518756, https://doi.org/10.1145/1518701.1518756

58. Nanayakkara, S.C., Wyse, L., Ong, S.H., Taylor, E.A.: Enhancing musical experience for the hearing-impaired using visual and haptic displays. Human-Computer Interaction **28**(2), 115–160 (2013). https://doi.org/10.1080/07370024.2012.697006, https://www.tandfonline.com/doi/abs/10.1080/07370024.2012.697006

59. Nuitrack: https://nuitrack.com/ (2020), accessed: 2020-02-10

60. Peng, C.: Survey of collaborative drawing support tools. Computer Supported Cooperative Work (CSCW) **1**(3), 197–228 (1992)

61. Reetz, A., Gutwin, C., Stach, T., Nacenta, M.A., Subramanian, S.: Superflick: a natural and efficient technique for long-distance object placement on digital tables. In: Graphics Interface. vol. 2006, pp. 163–170 (2006)

62. Robertson, G., Czerwinski, M., Baudisch, P., Meyers, B., Robbins, D., Smith, G., Tan, D.: The large-display user experience. IEEE computer graphics and applications **25**(4), 44–51 (2005)

63. Rogers, Y., Lindley, S.: Collaborating around vertical and horizontal large interactive displays: which way is best? Interacting with Computers **16**(6), 1133–1152 (2004)

64. Shaballout, N., Neubert, T.A., Boudreau, S., Beissner, F.: From paper to digital applications of the pain drawing: Systematic review of methodological milestones. JMIR mHealth and uHealth **7**(9), e14569 (2019)

65. Sheridan, J., Bryan-Kinns, N., Reeves, S., Marshall, J., Lane, G.: Graffito: Crowd-based performative interaction at festivals. In: CHI ’11 Extended Abstracts on Human Factors in Computing Systems. p. 1129–1134. CHI EA ’11, Association for Computing Machinery, New York, NY, USA (2011). https://doi.org/10.1145/1979742.1979725, https://doi.org/10.1145/1979742.1979725

66. Shoemaker, G., Tang, A., Booth, K.S.: Shadow reaching: A new perspective on interaction for large displays. In: Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology. p. 53–56. UIST ’07, Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1294211.1294221

67. Suhonen, K., Väänänen-Vainio-Mattila, K., Mäkelä, K.: User experiences and expectations of vibrotactile, thermal and squeeze feedback in interpersonal communication. In: The 26th BCS Conference on Human Computer Interaction 26. pp. 205–214 (2012)

68. Tang, J.C., Minneman, S.L.: Videodraw: A video interface for collaborative drawing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. p. 313–320. CHI ’90, Association for Computing
Machinery, New York, NY, USA (1990). https://doi.org/10.1145/97243.97302
https://doi.org/10.1145/97243.97302

69. Taylor, S., Izadi, S., Kirk, D., Harper, R., Garcia-Mendoza, A.: Turning the tables: an interactive surface forvjing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1251–1254 (2009)

70. Von Zadow, U.: Using personal devices to facilitate multi-user interaction with large display walls. In: Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology. pp. 25–28 (2015)

71. Wilson, G., Brewster, S.A.: Multi-moji: Combining thermal, vibrotactile and visual stimuli to expand the affective range of feedback. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. p. 1743–1755. CHI ’17, Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025614

72. Yalom, I.D.: The gift of therapy: An open letter to a new generation of therapists and their patients. Journal of Social Work Practice (2002)

73. Yeo, K.P., Nanayakkara, S.: Speechplay: Composing and sharing expressive speech through visually augmented text. In: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration. p. 565–568. OzCHI ’13, Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2541016.2541061

74. Yoo, T., Yoo, Y., Choi, S.: An explorative study on crossmodal congruence between visual and tactile icons based on emotional responses. In: Proceedings of the 16th International Conference on Multimodal Interaction. p. 96–103. ICMI ’14, Association for Computing Machinery, New York, NY, USA (2014). https://doi.org/10.1145/2663204.2663231

75. Zhao, M., Wang, Y., Redmiles, D.: Using collaborative online drawing to build up distributed teams. In: 2017 IEEE 12th International Conference on Global Software Engineering (ICGSE). pp. 61–65. IEEE (2017)

76. Zhao, M., Wang, Y., Redmiles, D.: Using collaborative online drawing to build up distributed teams. In: 2017 IEEE 12th International Conference on Global Software Engineering (ICGSE). pp. 61–65 (2017). https://doi.org/10.1109/ICGSE.2017.3

77. Zhao, M., Wang, Y., Redmiles, D.: Using playful drawing to support affective expressions and sharing in distributed teams. In: 2017 IEEE/ACM 2nd International Workshop on Emotion Awareness in Software Engineering (SEmotion). pp. 38–41 (2017). https://doi.org/10.1109/SEmotion.2017.3