# GrouPen: A Tangible User Interface to Support Remote Collaborative Learning

Yanhong Li, Yu Sun, Tianyang Lu, Thomas Weber, Heinrich Hußmann

► To cite this version:

Yanhong Li, Yu Sun, Tianyang Lu, Thomas Weber, Heinrich Hußmann. GrouPen: A Tangible User Interface to Support Remote Collaborative Learning. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.590-598, 10.1007/978-3-030-85610-6_33. hal-04215503

HAL Id: hal-04215503

https://inria.hal.science/hal-04215503v1

Submitted on 22 Sep 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# GrouPen: A Tangible User Interface to Support Remote Collaborative Learning

Yanhong Li * yanhong.li@ifi.lmu.de, Yu Sun * Yu.Sun@campus.lmu.de, Tianyang Lu * Tianyang.Lu@campus.lmu.de, Thomas Weber thomas.weber@ifi.lmu.de, and Heinrich Hußmann hussmann@ifi.lmu.de

LMU Munich, Frauenlobstr. 7a, 80337 Munich, Germany
http://www.medien.ifi.lmu.de

**Abstract.** Tangible User Interfaces (TUIs) have been used in many learning contexts. However, its application in the remote collaborative learning area remains relatively unexplored. In this study, we contribute to this use case with *GrouPen*. It is a TUI prototype embedded in a regular pen, thus allowing easy-to-use tangible interaction. We showed how it could be used to enable and enhance collaborative learning and engage students. *GrouPen* uses natural gestures to show statuses in several learning phases. We evaluated the prototype using a survey that yielded positive feedback and supported the hypothesis that TUIs like *GrouPen* could facilitate learning connectedness and engagement.

**Keywords:** Tangible User Interface, tangible learning, remote collaborative learning, human-centered design, ubiquitous computing

## 1 Introduction

Online or distance learning is becoming an important part of formal education, particularly – but not exclusively – during the COVID-19 pandemic. It is often facilitated via video conferencing tools such as Zoom or Microsoft Teams.¹ These tools are convenient for verbal communication, but often fall short in engaging learners to collaborate effectively. Even though direct communication via video is fairly easy, secondary communication via implicit cues is much more challenging, and can quickly cause information overload of the visual and auditory senses. Tangible User Interfaces (TUIs), which provide a physical interface and can help learners interact in a natural way, can alleviate this and thus support remote collaborative learning. Manipulating objects physically and tangibly requires less cognitive effort,² and provides an embodied way to interact with the world. However, as far as we know, there are few studies about the use of TUIs to facilitate remote collaborative learning. In this paper, we therefore introduce *GrouPen*, a TUI designed to help learners to be more engaged in collaborative group learning (Sect. 3). It allows students to see their peers' progress, get

<footnote>* The first three authors contributed equally to this study and were parallel first author</footnote>
matched with their peers and also lets teachers get an overview of their students'
learning progresses. A first evaluation (Sect. 4) shows *GrouPen* as a TUI can help
solve the communication problems that are caused by a spatially asynchronous
learning scenario. Our work thus makes a contribution to explore how TUIs can
make students more connected and engaged in a remote collaborative learning
environment.

## 2 Related work

### 2.1 Remote collaborative learning

Compared with individual learning or competitive learning, collaborative learning was found to be a better learning strategy to develop students' critical thinking skills and make them have a higher achievement. 3,4 With the development of communication technology, collaborative learning happens more often remotely. It is good for making students collaborate and communicate with group members regardless of location. However, engaging all group members and making them feel connected with others is challenging. Except peer-to-peer communication, previous studies found technology and teachers were the most valuable and important factors for students in remote collaborative learning. 5,6 Therefore, a remote collaboration tool should not only consider to facilitate group communication, but also could help teachers understand their students' learning progresses and provide them timely feedback.

### 2.2 TUI for learning

As a part of Weiser's "ubiquitous computing",⁷ technology becomes ubiquitous and embedded in our daily life objects, which can be naturally interacted with, like grabbing. Tangible learning involves gesture, motion or full-body interaction and "emphasizes the use of the body in educational practice".⁸ TUI for learning emphasizes physical activities and manipulation of physical objects for learning.⁹ In the field of education, TUI has been applied to multiple projects as well, aiming to help students better understand abstract concepts like mathematical problems or programming. Danli et al.¹⁰ proposed *T-Maze*, a tangible programming environment designed to allow children to build computer programs by manipulating a set of wooden blocks which are interconnected by magnets. By using physical manipulation, learners could benefit more from *T-Maze*. To help students understand abstract probability problems, Bertrand et al.¹¹ designed and developed an interactive interface for collaborative learning. Students could rearrange physical tokens to see the effects of various constraints on the problem space. However, these TUIs are not embedded in everyday objects and can not be seamlessly integrated into everyday usage and compute ubiquitously. Moreover, current TUIs focus little on remote learning, where students can cooperate with others remotely with TUIs. This is a gap especially in the time of the pandemic. Therefore, we designed and developed *GrouPen* to explore the possibilities of TUI for remote collaborative learning.
# 3 GrouPen

## 3.1 Concept ideas

To design an idea that is suitable for remote collaborative learning, we were inspired by the ubiquitous physical tools in daily life. TUI would work better if it can be integrated with computing ability. The pen is an essential tool in learning activities, thus it's worth making a pen as a TUI that users can directly interact with. Our target users are students who learn and discuss in a small group with up to 6 students. Possible user scenarios could be solving quiz-based group learning tasks, e.g. mathematics and physics, etc. GrouPen is equipped with one micro-controller, five sensors, two actuators, and one pen base to hold the pen. Students can use the LED colors on the pen to show their learning progresses, such as finish the quiz, have a question, or get matched with their peers. As shown in Fig. 1, collaborative learning processes are divided into five stages, which are working, having questions, discussing, finished, offline. Correspondingly, it can be identified with five different colors: yellow, red, blue, green, and grey. The color codes are consistent with daily conventions such as red is for urgency, and in our case red is for questioning. This is, of course, adaptable to individual preferences or circumstances.

<table><caption>Fig. 1. GrouPen's color coding concepts</caption><tr><td>Color</td><td>Yellow</td><td>Red</td><td>Blue</td><td>Green</td><td>Grey</td></tr><tr><td>Status</td><td>In progress</td><td>Having questions/ needing help</td><td>Discussing with other students</td><td>Finished/ available for discussion</td><td>Offline</td></tr></table>

To change the light color, learners can interact with the pen by performing specific gestures: (1) take the pen from the pen base; (2) raise the pen vertically; (3) shake the pen; and (4) put the pen back to the pen base. As the light colors are triggered by gesture interactions, information will be conveyed in a tangible way.

## 3.2 Design process

We came up with the idea that GrouPen should have an embodiment of a real pen, because a pen is the most common tool used in our learning activities. It fits well to some remote learning situations. For example, when it is not convenient to have an audio chat in the online virtual room, because other students are still working. In addition, when the student wants to know the availability of his or her teammates, he or she needs to frequently check the notifications or
chats. It is easy to cause information overload. Therefore, we designed a color-
changeable pen with LEDs on it to indicate the status of each group member.
More specifically, *GrouPen* is attached with LED strips, each of which represents
the status of a team member. It avoids students to frequently check notification
and be distracted by irrelevant messages. Meanwhile, it is also suitable for remote
collaboration without accessing communication device such as laptops. Fig. 2
shows how gesture interactions can realise the transition between various states.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Gesture interactions with *GrouPen*: a) Take the *GrouPen* from pen base; b) Raise it vertically; c) Shake it; d) Put it back to the pen base. The first row represents the user itself, and the other rows represent the team mates.</figcaption></figure>

These gestures are intuitive to understand and have consistency with daily
learning habits. There are four main interactions with *GrouPen*: As the pen
base is suitable to be designed as the “started” and “finished” trigger, the first
interaction is to *take the pen out of the pen base*. In the meantime, the LED,
which indicates the users’ status, will show a yellow light to convey the working
status. The second interaction is to *raise the pen vertically* to switch to a ques-
tioning state, which was inspired by raising our hands when we have a question
in the face-to-face (F2F) classroom. By *shaking the GrouPen*, students are able
to connect with other group members for discussion. The inspiration was from
the WeChat<fnref n="1" /> “shake to get paired” function for social activity. As it is easy to
become a habit, we adopted “shake” as the trigger for matching. After shaking,
the system will automatically search for other students who have shaken the pen
and are ready for discussion. If there is no one to match, the user will be in the
waiting queue until someone else shakes to start a discussion or raise vertically
to ask a question. After completing the learning task, students can *put the pen
back in the pen base*, their light will thus turn to green, which means they have
finished the task and are available for discussion.

Except the functions above, we added the following other four features.
**Progress bar:** While working on a learning task, students might want to see
how much time they have spent and how much time they have left, therefore,
we decided to design a progress bar on the pen base to enable time manage-

<footnote n="1">¹ WeChat is a popular Chinese social media application</footnote>
ment. **Focus mode**: students can turn off light status to concentrate on the task without being disturbed. In this context, *GrouPen* becomes an ordinary writing pen. **Show emotion**: A study of Botzer et al.¹² showed that grip force could be thought as a measure in joystick-controlled tasks and the participants who were under higher stress had a significant higher grip force. Therefore, we used grip force as a design element and made *GrouPen* emotion-aware. It has a pressure sensor to perceive how tight the student grip the pen and shows it through the brightness of LEDs. The information of stress level can not only be timely shared with peers and teachers, but also can be stored for the analysis of learning behavior. *GrouPen* can help the teacher in the collaborative learning through: (1) enhance peer-to-peer discussion so that there are fewer students in the waiting queue; (2) let the teacher know if students have questions. *GrouPen* can also be used in a self-organised learning environment without the presence of teachers.

## 3.3 Prototype

For the hardware, all used sensors and actuators are demonstrated in Fig. 3. We used Teensy board as the micro-controller, which was much smaller than Arduino Uno board to fit in the plastic bottle. The LEDs we used were WS2812B strips, which could be cut and soldered to indicate the progress and status of group members. For gesture recognition, a 3-axis gyroscope and a 3-axis accelerometer MPU6050 were used. They could obtain the value of the pen in the x-y-z axis, determine the orientation of the pen, and recognize the user's gestures. In addition, we used the digital shake sensor SKU:SEN0289. During the function testing, it worked more accurately than MPU6050 to detect users' shaking movement. The light sensor LM393 was placed in the pen base to detect users' behavior of picking up/dropping down the *GrouPen*. The TTP223B digital capacitive touch sensor was implemented on the top of the pen, which was designed to be touched by the students for turning on and off the lights. In order to show the learning emotion, a Force Sensing Resistor (FSR) MD30-60 was implemented to detect students' grip force. The brightness of the LED will then be changed according to the pressure value. Finally, a vibration module is mounted in our prototype for providing tactile feedback to indicate the changes of status or functions. For the software, we used the Teensyduino IDE for programming and testing. A web server was built on an ESP 32 micro-controller to realise communication among group members.

In order to show our proof-of-concept, we need to make it look like a real pen. We used a 300ml size plastic bottle as the shell of the *GrouPen*. We cut the dip from a pencil with the electric cutter and inserted it into the plastic bottle, so that *GrouPen* can have a replaceable and usable dip for writing.

## 3.4 User study

In order to test *GrouPen*'s usability, user experience, and willingness to use for remote collaborative learning, we conducted user studies with 7 participants (4
<figure><img src="image_4.png" /><figcaption>Fig. 3. Hardware details in the GrouPen</figcaption></figure>

females, 3 males). Their average age was 33 (M = 24, SD = 14.93). Six of them learn daily in the remote situation, the other one learns weekly in the remote situation.

Our user study was conducted in two forms, online with Zoom platform and F2F, both of which lasted for about 30 minutes. Online study had four steps: first, participants filled the consent and demographic questionnaire to get their personal information (e.g. age and gender) and remote learning experience and habits. Then, we played the video of the *GrouPen* and had a verbal explanation of its usage and functions. After that, participants were asked to fill the System Usability Scale (SUS)¹³ questionnaire (a ten-item attitude Likert scale to understand *GrouPen*'s effectiveness, efficiency, and satisfaction) and rate the user experience of for collaboration (i.e. "Using this *GrouPen* would help me engage in remote collaborative learning", and "Using the *GrouPen* could make me more aware of my group members"). Finally, an interview was conducted to know more about their opinions, for example, what they thought was the biggest problem in remote learning and suggestions to improve *GrouPen*. The interview was audio recorded for interview analysis. The F2F study was conducted in a similar process, except we showed them our physical prototype and demonstrated the features lively. However, participants for online just watched our functional video.

# 4 Findings

From the results of SUS, we found users' overall attitude towards *GrouPen* was 70, which was above average score of 68. According to the general guideline on the interpretation of SUS score,¹³ a score above 80.3 is excellent, between 68 and 80.3 is good.

According to their responses, *GrouPen* was thought to be more effective than non-physical learning tools to help students engage in collaborative learning. More specifically, users found *GrouPen* was interesting to use and strongly agreed it could help them for remote collaborative learning. All participants agree that
it would be a good idea to use *GrouPen* for remote learning, which indicated a high user acceptance of TUI. They also agreed that *GrouPen* could be used as an enhancement and complement to the existing collaboration tool. In addition, their said *GrouPen* could help them be more aware of group members. We also found that most respondents agreed *GrouPen* could make them feel more connected with group members.

In the interview, we got more in-depth feedback from users. We summarized it from three dimensions: (1) What remote learning problems can be solved? (2) What are possible application scenarios of *GrouPen*? (3) How do users feel while using it? Many remote learning problems can be solved by *GrouPen*, such as “mitigate the problem that students can not communicate F2F due to the pandemic”, “better understand if my teammates have problems”, “*GrouPen* makes the status update easier”. Some talked about it from the second dimension: what are the suitable remote learning scenarios that *GrouPen* could be applied to. “suitable for solving mathematical problems”. As for personal feelings, we got some feedback such as “the feeling of binding together is increased”, “feel more motivated”, “want to help him/her actively if the light changed to red ” etc.

# 5 Discussion

From the results, we found that users had a positive reception and were willing to use *GrouPen* in the remote learning environment. They thought our prototype was useful to solve their remote collaboration problems, such as convey and receive the progress and availability information. *GrouPen*, as a TUI learning tool, could make the information more noticeable, make students pay more attention to their teammates, and be more aware of their peers' needs. Participants feel more active to take some actions when they saw someone's color has changed. As the information that someone needs help can be conveyed so directly, a more active connection can be made.

Participants were enthusiastic of using TUI, because the current tools they used for remote learning were all non-physical software, such as Zoom or Microsoft Teams. In comparison to digital tools, TUI provides opportunities that students can directly interact with their gestures and get immediate visual and haptic feedback, which facilitates their emotional connections with the teacher and classmates.

A biggest advantage of *GrouPen* is that it only conveys the most basic and essential information to the learners. Therefore, users do not need to use digital devices on a regular basis and feel more similar like learning in the F2F classroom. However, *GrouPen* also has limitations. For example, it can only support non-verbal and easy communication (i.e. status, availability, and group members' emotions). When it comes to general or discussion situations, audio calls and video conferencing are still needed.

In addition, four suggestions could be considered for future development. Some of these suggestions come from the feedback of the participants in our
study, the other derive from our reflections. First, hardware enhancement: it
might be better to integrate a function of audio recording, which can make it
more independent with computer; Second, writing experience: *GrouPen* could
be “thinner, compacter”, which could make users have a real writing experience;
Third, personalisation: concerns and ideas about force-sensing were expressed
in the interview, for example, “the individual baseline for the force of holding
pens should be measured before”. Thus, future work needs to consider users’
diversities and adjust for their differences; Finally, learning habits analysis:
*GrouPen* could be a good medium to trace learners’ behavior or habits. Like
one of our participant mentioned, “the data of force level should be sent to the
teacher”.

6 Conclusion

By combining computing technology and everyday objects into TUIs, like *GrouPen*, we hope to create engagement in scenarios such as remote collaborative learning. This is a particular important learning context but also challenging. Our early prototype in this study is a first step to enhance group connectedness in this situation. Integrating the interaction into everyday objects, which are being used anyway, ensures that overhead is kept to a minimum and allows for a number of natural interactions (as we described in our prototype). Receiving information about the status of group members or tasks via such a TUI should help keep mental load and distraction low. This in turn would benefit students’ learning. The positive feedback from our evaluation corroborated this hypothesis, because participants saw the benefits of our interface.

Certainly, even with natural gestures and everyday objects, there will be a
learning curve, which needs a larger and long-term study to see the effectiveness.
These should particularly take place in real-world learning scenarios with differ-
ent learners ranging from elementary school to university students. Beyond that,
*GrouPen* should also be adapted into new usage scenarios such as project-based
learning tasks.

**Acknowledgements** This research is funded by the Elite Network of Bavaria
(K-GS-2012-209).
Bibliography

[1] Kristóf, Z. *Central European Journal of Educational Research 2020*, **2**, 84-92.

[2] Hallam, J.; Zheng, C.; Posner, N.; Ericson, H.; Swarts, M.; Do, E. The Light Orchard: An Immersive Display Platform for Collaborative Tangible Interaction. 2017; pp 245–248.

[3] Roseth, C. J.; Johnson, D. W.; Johnson, R. T. *Psychological bulletin* **2008**, *134*, 223.

[4] Gokhale, A. A. **1995**,

[5] Davies, R.; Yeung, E.; Mori, B.; Nixon, S. A. *Medical teacher* **2012**, *34*, e676–e683.

[6] Mallon, M.; Bernsten, S. USA: *Association of College and Research Libraries and American Library Association* **2015**,

[7] Weiser, M. *SIGMOBILE Mob. Comput. Commun. Rev. 1999*, **3**, 3–11.

[8] Eisenberg, M.; Pares, N. *Tangible and Full-body Interfaces in Learning*; 2014; pp 339–357.

[9] Bakker, S.; Antle, A.; van den Hoven, E. *Personal and Ubiquitous Computing* **2011**, *16*, 433–449.

[10] Wang, D.; Wang, T.; Liu, Z. *The Scientific World Journal* **2014**, *2014*, 428080.

[11] Schneider, B.; Blikstein, P.; Mackay, W. Combinatorix: a Tangible User Interface that Supports Collaborative Learning of Probabilities. 2012; pp 129–132.

[12] Botzer, A.; Sahar, Y.; Wagner, M.; Elbaum, T. *Behaviour & Information Technology* **2021**, *40*, 476–482.

[13] Brooke, J. *Sus – a Quick and Dirty Usability Scale*; 1996; pp 189–194.