# Study Marbles: A Wearable Light for Online Collaborative Learning in Video Meetings

Yanhong Li, Bill Bapisch, Jenny Phu, Thomas Weber, Heinrich Hußmann

► To cite this version:

Yanhong Li, Bill Bapisch, Jenny Phu, Thomas Weber, Heinrich Hußmann. Study Marbles: A Wearable Light for Online Collaborative Learning in Video Meetings. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.712-721, 10.1007/978-3-030-85623-6_40. hal-04331780

HAL Id: hal-04331780

https://inria.hal.science/hal-04331780v1

Submitted on 8 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Study Marbles: A Wearable Light for Online Collaborative Learning in Video Meetings

Yanhong Li * yanhong.li@ifi.lmu.de, Bill Bapisch *
bill.bapisch@campus.lmu.de, Jenny Phu * jenny.phu@campus.lmu.de,
Thomas Weber thomas.weber@ifi.lmu.de, and Heinrich Hußmann
hussmann@ifi.lmu.de

LMU Munich, Frauenlobstr. 7a, 80337 Munich, Germany
http://www.medien.ifi.lmu.de

**Abstract.** Video meetings gained popularity for remote communica-
tion, both for work and education. As a result, collaborative online learn-
ing has become increasingly widespread. However, it is a challenge to
make students feel engaged and connected during video meetings. In
this study, we addressed this problem with a prototype for a wearable
user interface called *Study Marbles*. It aimed to create a more social and
active sense of remote, collaborative learning in video conferences. Our
device is a tangible necklace with attachable, illuminated marbles that
can be worn during video meetings. It could visualize students' learning
status, moderate group discussions, and enable voting. The user study
showed that participants perceived our prototype as a good way to create
a more active and connected environment and to improve the interaction
between group members in video conferences.

**Keywords:** tangible user interfaces, TUI, tangible learning, online learning, remote tangible interaction, group learning, collaboration, distance learning

# 1 Introduction

Video meetings have become increasingly common in the past years, especially during the COVID-19 pandemic. Online video conferences have made it feasible for people to communicate and collaborate from distributed locations. This also applies to the learning domain where collaborative online learning with video conferencing (VC) systems, such as Zoom, has become common. However, compared to face-to-face classroom learning, VC was found to be less interactive and less encouraging to discussions [6]. Therefore, there is an issue of how to make remote video communication more engaging. Andel et al. [2] argue that it is “more important now than ever [...] to effectively optimize and enhance the online learning experience”.

<footnote>* The first three authors contributed equally to this study and were parallel first author</footnote>
Research has shown that Tangible User Interfaces (TUIs) can be helpful to increase motivation by offering playful interaction with tangible objects [11,13]. TUIs show a good potential for online learning, but currently there are few studies of TUIs for remote collaborative learning, and even fewer for use in video conferences.

In this paper, we present and evaluate a TUI prototype named “*Study Marbles*”, which is a tangible necklace worn by each call member in a video meeting for augmenting communication with tangible, illuminated marbles. The reason why we chose a tangible, light-based interface for communication instead of traditional digital video overlays is to achieve a stronger coupling of bits and atoms [9] and interaction independent from video conferencing software. *Study Marbles* has three main functions: 1) status visualisation, 2) discussion moderation and 3) multiple choice voting. We aim to design a tangible tool to create a more involving, motivating and social environment for group video meetings.

## 2 Related Work

Online learning has grown rapidly over the last years. There are many online learning platforms enabling people in different locations to study together, for instance to access learning materials, attend online lectures and communicate in video conferences. However, there is a concern that virtual learning might reduce the learning quality [1]. For example, students in remote learning environments might feel a lack of focus, lower commitment and support [8]. In addition, a study showed that the majority of students thought that the use of Video Conference (VC) technology “discouraged classroom discussions” and was a “barrier to their interaction with the instructor” [6]. Therefore, online education needs engaging and connected learning environments.

One solution to achieve this is social presence. Indicating social presence in video-central online learning environments through features like comments was shown to result in higher motivation to participate and better performance [2]. In addition, there are different approaches to make people feel more connected in remote group settings. One study showed that visualisations help collaboration in remote synchronous group work [3]. Another solution was BuddyWall [14], a tangible ambient wall to visualise “an awareness of group presence”. Lastly, wearables in this area were also studied, like a tangible apps bracelet [7] for non-verbal communication and reminders. Successful conversational learning requires more than a communication channel, it takes a mutual understanding and methods to make a conversation successful and efficient. One way to facilitate this is through specialized learning devices [16]. These devices need to be “highly portable”, “unobtrusive”, “available anywhere”, “useful, suited to everyday needs” and “intuitive to use” [15].

Even though it is possible to use digital overlays to visualise different states in video meetings, the goal of specialised tangible learning devices is to counteract the strong influence of digital communication in online learning. In the following, we combine wearable devices with the ability of haptic interaction and
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Illuminated marbles in the video call show the current status of each call member using traffic light color coding. The participant in the lower left corner has not selected a marble and the necklace remains empty as the marbles are detachable.</figcaption></figure>

ambient light in the real world. Thereby we want to use the approach "People, Bits and Atoms" [9] to bridge the gap between the digital and the real world. For this purpose we associate digitally available information with physical objects (marbles), similar to Durrell Bishop's Marble Answering Machine [12]. We extend this idea to represent information using light in the real environment and then send it through the digital channel (VC).

# 3 Study Marbles

In the following section we share several concept ideas and walk through the design process, which led to our final prototype of a wearable light-based interface to enhance communication in video meetings.

## 3.1 Concept Ideas

Our process for gathering concept ideas started with the question: "How might we support groups collaborating and learning remotely with a tangible light-based tool?". We considered different aspects of learning, such as learning through acquisition, inquiry, discussion, practice and collaboration [10]. Based on these, we examined the following common use cases in more detail: communicating availability, calling a meeting, leading a discussion, collaborative voting, requesting breaks and asking a teacher for help, with the use of an ambient light on the desk or in the periphery. In parallel, we collected light-based interactions for communication such as turning light on/off, dimming, flashing, changing light temperature, brightness, colors, color gradients, light and shadow or using light stencils or projections.

In the end, we decided on a wearable necklace to enhance collaboration and group communication in video meetings because of its good visibility in the view of the video camera. At the same time, we were inspired by Bishop's Marble Answering Machine [12], which used marbles as physical visualisations of messages. Therefore, we developed the idea of light-emitting marbles attachable to the
necklace through tangible interactions (see Fig. 1). More specifically, our prototype has the following functions: (1) providing status information with different colors associated with different meanings; (2) moderating the group discussion with tangible interactions, for example attaching and detaching marbles to the necklace; (3) supporting learning through multiple-choice voting.

One of our aims was to design intuitive tangible interactions with this wearable interface, so we integrated the following actions: touching buttons on sides of the hexagon to change color, shaking/swinging the necklace (detected by accelerometer) and attaching and detaching marbles with magnetic latches. Finally, we planned a central database and a mobile app for further development.

## 3.2 Design Process

An important aspect of the design process was the overall shape of the wearable object. The size needed to be small in order to be light enough to be worn, however, it should also be large enough to be visible in a video meeting. Because of the relatively small size of the prototype, it was important to select the right components early on. The ESP32 D1 mini was chosen for its compact size and important functionalities for our prototype, such as WiFi. Other components included the MPU-6050 accelerometer and RGB LEDs based on the WS2811.

The hardest challenge was to design and build a detachable marble that contains an LED and provided reliable electronic contacts when inserted and removed. Therefore, we split our design process into two parts: necklace design and marbles design. For designing the marbles, the goal was to create a connection that prevented improper electronic connection and at the same time guaranteed a strong and stable connection. To differentiate between different types of marbles and accordingly control the color of the RGB LED, we used a voltage divider to detect different voltage levels for different marbles, with resistors between the marble and micro-controller. In total, we needed four leads connecting the lamp to the body (ground, 5 volt, LED color control and a lead for the resistors).

The design decision that turned out to be very effective was to use pogo pins as electrical contacts between the marbles and the necklace. These were built into the necklace and the marbles were fitted with matching concave connectors. In order to facilitate effortless and firm attachment of the marbles to the pogo pins, there are two strong magnets in the lid of the necklace and two smaller ones hidden inside each marble.

The hexagonal design was chosen to allow up to six capacitive touch buttons on each side. We decided to only use four in order to have space at the bottom and top to grip the device and also for the micro-USB port to upload new firmware. In order to fabricate a reliable and sturdy prototype, as shown in Fig. 2, the entire assembly was first designed as a 3D model and later 3D printed.

## 3.3 Prototype

The finished model for the hexagonal necklace and marbles can be seen in Fig. 2. Each pin is used to set one of the colors: red, yellow, green and blue. We included
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Left: 3D model render of components; Right: *Study Marbles* prototype with an inserted illuminated color marble.</figcaption></figure>

two different types of marbles with the same design. However, there was one
difference: the *color marble* lights up when it is attached to the necklace; while
the *action marble* used for the discussion and vote mode does not, it lights up
only when it's your turn or you have voted. We designed three use cases for it:

(1) **Status:** To visualise the users’ current status, for example their feeling regarding a task or the status of their progress on a worksheet in a learning context, users can insert the *color marble* into the necklace. The color marble lights up in a color, which can be changed by pressing one of the four touch pins on the sides of the hexagon. Additionally, we implemented light effects: By *shaking* the necklace, a flashing/blinking light can be triggered, for example to signal distress. *Swinging* the hexagon on the necklace string can trigger a breathing/fading light effect to show a relaxed state.

(2) **Discussion:** The *Study Marbles* can moderate group discussion by light-
ing up the marble of the current speaker. The discussion mode can be started,
after everyone inserts the *action marble*. The action marble turns off at first. One
person can start the discussion mode in the app. The system then chooses one
random person whose marble turns on as a random speaker. After the speaker
removes his/her marble or when the Skip button is pressed in the app, the next
random speaker is chosen. This ensures that everyone gets a turn to speak.

(3) **Multiple-choice voting:** The vote mode is also started by inserting the *action marble*, which is off in the beginning. One person can start the voting in the app. Subsequently participants can vote by pressing one of the four sides of the hexagon, which represent a multiple choice answer (a, b, c, d). When someone has voted, their marble lights up in white. After everyone has voted, the votes can be revealed by pressing a button in the app and the colors of all marbles change to the answers chosen by their users.
The discussion and vote mode can be started with a smartphone app, which we built with Thunkable <fnref n="1" />, a tool for fast prototyping of cross-platforms apps. The communication between the smartphone app and the ESP32 inside the necklace works by accessing variables in a cloud database and polling it for updates, e.g. to know the start of a new mode or the detachment of a marble.

# 4 Findings

In order to know users' practical impressions and suggestions for *Study Marbles*, we conducted an online survey. The online format of the survey (due to the pandemic situation) is a limitation of our user study, however, we included detailed videos to show all the functions, together with text descriptions. Finally, we got 41 participants (19 females and 22 males). Eighteen of the 41 participants were 18-24 years old and 10 were 26-34 years old. The rest were 35-74 years old. More than half of them were students. 35 participants had a higher education and 31 used video meetings multiple times a week or more often. The most common reasons for using VC were "attending university lectures, classes and work meetings".

We asked participants to rate *Study Marbles'* usability with the *System Usability Scale* (SUS) [4]. The average of the result SUS scores was 69.15, with a standard deviation of 13.16, which shows a good usability rating [5].

In addition, we have ten items grouped in three areas, to know participants' feeling and opinions of *Study Marbles*, which can be seen in Fig. 3.

*Wearable*: 63% of participants liked our wearable design and 83% thought that the light of the marbles was a good way to convey information. On the other hand, participants' opinions were more divided when asked if it is desirable to have a wearable tool for online meetings.

*Video meetings*: 76% of participants thought they would pay more attention to the video when using *Study Marbles*, however only 39% thought they would feel more comfortable to turn on their video when using *Study Marbles*. The last two questions about video meetings asked if *Study Marbles* could be distracting during video calls. This was approved by almost half of the participants. This raises the question if *Study Marbles* is more likely to distract the users than help. Here, the participants were rather uncertain, as this is still a very new area and they lacked experience with the technology.

*Collaborative*: Regarding *Study Marbles'* effect on collaboration, the majority of participants thought that it could make them more willing to participate in group work and would help to communicate with group members more effectively. Most agreed that it could make them feel more connected with group members. All in all, *Study Marbles* was rated to have a positive impact on collaboration.

<footnote n="1">¹ https://thunkable.com/</footnote>
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** Users' experience about *Study Marbles'* wearable, collaborative and video meeting characteristics</figcaption></figure>

Participants commented: "I think this is a good way to enforce participation in a lecture setting", "Study Marbles is especially advantageous in video conferences with many people to make the participants feel more included", "Good idea to make everyone more active in the meeting".

# 5 Discussion

In this section, we will discuss challenges, possibilities of using *Study Marbles* and limitations of this work. One of the disadvantages of *Study Marbles* is accessibility for (partially) color-blind people, as the color of the light is used for the status and vote mode. The visibility of the light could be another issue, which could be dependent on video quality and the posture of the wearer. Some people commented *Study Marbles* could be distracting from the meeting itself and that it would be more suited for smaller groups. Two people thought that there could be misinterpretations of the light meanings, which have to be clarified before using the system. Some mentioned the concern that the necklace might be too heavy. Since the participants could only watch a video of the *Study Marbles*, this was rather difficult to estimate, but the necklace actually weighs only 70g. The vote mode could also be improved by displaying a chart of the vote results and one person noted they would like to keep votes hidden to not be under
pressure while voting. Some participants commented that *Study Marbles* could be integrated into a video conferencing tool to reduce the number of devices, for example combining it with the “hand-raising”- function in Zoom.

Study Marbles can also be extended with different functions, such as a timer or reminder before a meeting, group allocation or used in completely different settings, such as real-life meetings or classrooms. As alternative use cases, participants suggested using it for games, in Agile teams, in combination with task boards or as a timer for Pomodoro learning. Nonverbal light signals can also be useful to ask for help in a classroom or when talking is not possible, for example one participant suggested it could be used in hospitals.

In the survey, we asked the participants what other kinds of wearables they would like to use instead of a necklace. Most people thought that the necklace was the best option. Arguments for a necklace design was portability and visibility during the video meeting. Only one person said that they would never wear a necklace for that. Other ideas for wearables included a bracelet, a wristband or a watch, pin-buttons, a cap, hat or headband. Some unique ideas were lights attached to headphones or a modified pair of glasses.

Limitations of our survey were that it was just an online survey, in which the participants could not try the prototype in real life. Also, we did not test the prototype in a learning or teaching scenario, so this could be investigated more. We did not specifically search for participants with a background of teaching or studying, however most of our participants were students. Future work could recruit teachers to evaluate our prototype and also children and teenagers, since all our participants were older than 18.

# 6 Conclusion

In this paper we presented *Study Marbles*, a concept idea for a tangible, wear- able interface with illuminated marbles to support online, remote collaboration, particularly for online learning. So far, the concept is still in its early stages and there is still a lot of potential for future research. Our review of related work showed that tangible interaction with light-based interfaces is still a new area, which is promising, not only for collaborative remote learning. Main results of our online survey were that the usability of *Study Marbles* was mostly rated positively, and that it could help to make group members feel more included and motivated to participate.

Beyond a field evaluation, future work could explore if users prefer tangible interaction vs. software-integrated interaction in common digital devices (PC, tablet, smartphone) to control the marbles and if there are any advantages or performance differences between both. Variations of *Study Marbles* using dif- ferent wearables or other types of tangible objects and alternative interactions with illuminating marbles could also be studied in the future. Since *Study Mar- bles* does not have to be limited to virtual online meetings or learning contexts, there are still numerous possible use cases that can be investigated, for example real-life meetings, classrooms, games or work settings.
**Acknowledgements** This research is funded by the Elite Network of Bavaria (K-GS-2012-209).

## References

1. Amelung, C., Laffey, J., Turner, P.: Supporting collaborative learning in online higher education through activity awareness. pp. 75–77 (01 2007). https://doi.org/10.3115/1599600.1599611

2. Andel, S., de Vreede, T., Spector, P., Padmanabhan, B., Singh, V., de Vreede, G.J.: Do social features help in video-centric online learning platforms? a presence perspective. Computers in Human Behavior **113**, 106505 (07 2020). https://doi.org/10.1016/j.chb.2020.106505

3. Balakrishnan, A., Fussell, S., Kiesler, S.: Do visualizations improve synchronous remote collaboration? pp. 1227–1236 (01 2008). https://doi.org/10.1145/1357054.1357246

4. Brooke, J.: Sus: A quick and dirty usability scale. Usability Eval. Ind. **189** (11 1995)

5. Brooke, J.: Sus: a retrospective. Journal of Usability Studies **8**, 29–40 (01 2013)

6. Doggett, M.: The videoconferencing classroom: What do students think? Architectural and Manufacturing Sciences Faculty Publications (01 2007)

7. Fortmann, J., Root, E., Boll, S., Heuten, W.: Tangible apps bracelet: Designing modular wrist-worn digital jewellery for multiple purposes. pp. 841–852 (06 2016). https://doi.org/10.1145/2901790.2901838

8. Hussein, E., Daoud, S., Alrabaiah, H., Badawi, R.: Exploring undergraduate students' attitudes towards emergency online learning during covid-19: A case from the uae. Children and Youth Services Review **119**, 105699 (2020). https://doi.org/10.1016/j.childyouth.2020.105699

9. Ishii, H., Ullmer, B.: Tangible bits: Towards seamless interfaces between people, bits and atoms. Conference on Human Factors in Computing Systems - Proceedings (09 1998). https://doi.org/10.1145/258549.258715

10. Laurillard, D.: Teaching as a design science: Building pedagogical patterns for learning and technology. Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology pp. 1–258 (01 2012). https://doi.org/10.4324/9780203125083

11. O'Malley, C., Stanton Fraser, D.: Literature Review in Learning with Tangible Technologies (2004), https://telearn.archives-ouvertes.fr/hal-00190328, a NESTA Futurelab Research report - report 12

12. Polynor, R.: The hand that rocks the cradle (1995)

13. Price, S., Rogers, Y., Scaife, M., Stanton, D., Neale, H.: Using ‘tangibles’ to promote novel forms of playful learning. Interacting with Computers **15**, 169–185 (04 2003). https://doi.org/10.1016/S0953-5438(03)00006-7

14. Quintanilha, M.: Buddywall: a tangible user interface for wireless remote communication. pp. 3711–3716 (01 2008). https://doi.org/10.1145/1358628.1358918

15. Sharples, M.: The design of personal mobile technologies for lifelong learning. Computers & Education **34**, 177–193 (04 2000). https://doi.org/10.1016/S0360-1315(99)00044-5

16. Sharples, M.: Disruptive devices: Mobile technology for conversational learning. International Journal of Continuing Engineering Education and Lifelong Learning **12** (01 2002). https://doi.org/10.1504/IJCEELL.2002.002148