Exploring Collective Physiology Sharing as Social Cues toSupport Engagement in Online Learning
Jiawen Han

hanjiawen@kmd.keio.ac.jp
Keio University
Japan

Zhuoqi Fu

fzq1996@kmd.keio.ac.jp
Keio University
Japan

Chi-Lan Yangchilan.yang@cyber.t.u-tokyo.ac.jp
The University of TokyoJapan

Reiya Horii

rhorii018@kmd.keio.ac.jp
Keio University
Japan

George Chernyshov

chernshov@kmd.keio.ac.jp
Keio University
Japan

Takuji Narumi

narumi@cyber.t.u-tokyo.ac.jp
The University of TokyoJapan

Kai Kunze

kai@kmd.keio.ac.jp
Keio University
Japan

Figure 1: Streaming system used in the field study. Left is the device for streaming blood volume pressure (BVP) signal. Rightis the interface consisted of class content area, streaming area, and camera area.The visualization in the streaming area wasgenerated based on two features: (1) beats per minute (BPM) (2) the percentage of adjacent normal sinus beats’ interbeat
intervals that differ from each other by more than 50 ms (pNN50). The red and blue lines represented collective BPM and
pNN50 respectively and both were calculated from rolling means of all data contributors every one minute.

ABSTRACTInsufficient social cues between distributed learners in online learning could result in lack of engagement and social bonds. With thedevelopment of wearable sensing, sharing physiological data can beused to enhance mutual understanding and connectedness amongsharers. Our work aims to explore the potential of sharing heart
rate (HR) and heart rate variability (HRV) collected from distributed
learners to enhance their online learning experiences. We implemented a physiological streaming system and conducted a field
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
MUM 2021, December 5–8, 2021, Leuven, Belgium© 2021 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8643-2/21/12.
https://doi.org/10.1145/3490632.3497827

192

study with 11 learners in online classes. This paper describes thestudy and discusses our interview findings by contrasting the influence of visualized collective physiological data from viewpointsof data contributors and viewers. Our exploratory results suggest
streaming collective HR and HRV from multiple distributed learners
could be used in online classes to improve engagement and senseof community.

CCS CONCEPTS• Human-centered computing → Empirical studies in collaborative and social computing.

KEYWORDS
physiological sensing, online learning system,engagement, visualization

MUM 2021, December 5–8, 2021, Leuven, BelgiumHan, et al.

ACM Reference Format:
Jiawen Han, Chi-Lan Yang, George Chernyshov, Zhuoqi Fu, Reiya Horii,Takuji Narumi, and Kai Kunze. 2021. Exploring Collective Physiology Sharing as Social Cues to Support Engagement in Online Learning. In 20thInternational Conference on Mobile and Ubiquitous Multimedia (MUM 2021),
December 5–8, 2021, Leuven, Belgium. ACM, New York, NY, USA, 3 pages.
https://doi.org/10.1145/3490632.3497827

1

BACKGROUND AND MOTIVATION

Lacking of engagement and sense of community has been identified
challenging for distributed learners in online learning [8, 14, 17]
due to insufficient socials cues [17]. Various approaches have been
proposed to enhance social presence in online learning. For example, visualizing historical social comments from prior online peers
to support learner engagements [9], or applying scenario-based
interaction design to facilitate social interactions among distributed
learners [16]. However, most research focused on mediating usergenerated contents to engage distributed learners, which requires
learners to actively spend extra efforts while attending the lecture.
Different from these related works, our research aims to use physiological data as social cues to express distributed learners’ implicit
feedback during online learning.
Physiological signal such as electroencephalography (EEG) [4, 5,10], galvanic skin response (GSR) [15], and heart rate (HR) [1, 12]
can be a substitute for social cues to interpret others engagement
and emotional situations. We selected HR and heart rate variability (HRV) as our key physiological data because HR is relatively
intuitive to interpret and the streaming system is feasible to be setup in large scale even remotely. Slovak et al. proved HR feedback
is a strong connectedness cue affecting interactions especially inremote scenarios [12]. There are also some practical applications inwhich HR and HRV sharing alleviated stress in remote collaborations [15] and augment social games [1]. Inspired by these works,
our study aims to explore how sharing cardiovascular activities
could influence distributed learners’ engagement and sense of community. Our exploratory results show the feasibility of streamingcollective HR and HRV measurements from distributed learners
and the potential to enhance social bonds in online learning.

2

FIELD STUDYWe conducted a field study to investigate how distributed learners
react to the streaming system that presents collective HR and HRV
measurements in real time. Our system tracked Blood Volume Pulse(BVP) from self-built wrist-worn devices with an optical sensor
placed on the fingertip referring to the set-up in previous work [2, 6,13] (Fig. 1 (left)). The device sampled the BVP at 50Hz and streamed
to our system server via User Datagram Protocol (UDP), which
supported distributed learners to stream their data without location
restrictions. We implemented glance-able line charts right below
the presenter’s slide content (Fig. 1 (right)) to avoid distracting
learners from the class content. BPM was selected as an intuitive
HR indicator of excitement and anxiety. While pNN50 was adopted
as an established HRV feature to reflect relaxation and sustained
attention [7, 11]. We briefly introduced the features we used in thebeginning of the class.
The streaming session was embedded in a lecture series lasting
about 40 minutes. 48 learners in total attended the online lecture

193

over Zoom. The field study had ethic board approval from Keio
University. 8 learners volunteered to stream and share their physiological data. After the streaming session, we organized interviews
on 11 learners and 6 of them were data contributors while theothers were viewers. During the interview, we asked participants
about their perception and experience while viewing the streaming
visual in the class, and to compare their experiences in class without the streaming event. We also asked participants to share theirinterpretations about the visuals including the meanings behindand sources of the data. We also questioned the learners who contributed streaming data about their personal feelings when wearing
the device and sharing their physiological data to others.
We delivered surveys after the classes with and without thestreaming visualization to quantify its impact on learners. Besides
demographic questions, the survey asked questions about senseof community [14] and perceived psychological engagement [3]in 7-point Likert scales (1:Strongly Disagree, 7:Strongly Agree).
17 learners answered the survey after the class without seeing
physiological data streaming, and 19 learners answered the survey
after the class implemented with physiological data streaming. We
notified students that joining the study was not compulsory and allof the answers would be kept confidential and not related to any
type of grading.

3RESULTS AND DISCUSSIONSIn this section, we summarized the themes emerged from the qualitative analysis and identified feedback from data contributors andviewers. We also report survey results to triangulate the interview
findings.
All the participants reported they had the motivation to lookat the streaming visual. Main reasons were the curiosity in others’ reactions ( "I find it very interesting to see how people developtogether."(C1)) and the awareness of what’s going on in the class"Whether I intentionally looked the streaming was related to theinstructor and the content of the course itself."(C3).
In accordance with our concept design, participants reported
increasing engagement and sense of community with the streaming system. As data contributors, they could involve in the class
more actively ("The change of the data visualization was like themore spontaneous reaction, what’s happening in class [...] when theinstructor was asking us something, then I was curious if it goes up
because people feel involved and then they’re like, oh, I need to actnow."(C1)), and feel more connected to other distributed learners
by seeing the data visualization ("Putting aside how accurate thevisualization were or what it meant, I felt it really cool to see that
we are somehow affecting this class and visuals. [...] I felt like I waspart of a class and felt like there were people like in this class that
there was a presence rather than me sitting alone in the room andwhatever." (C2)). Moreover, data contributors did not report privacy
concerns with the sharing mostly because of the visual was shownin an aggregated manner and perceived anonymity (" Because it
feels like so many people’s data and not my data alone. It’s something
more objective and doesn’t feel violating." (C3)).
Regarding viewers’ experience, they mentioned it was a little
hard to understand the system setup, such as how was the dataExploring Collective Physiology Sharing as Social Cues to Support Engagement in Online Learning

recorded, integrated, and transformed. However, they were still interested in the trend and fluctuations of the streaming visualization("I can get a rough idea of what it is, but I don’t understand what
it is. But looking at the trend, I can feel the change." (V2)). Viewers
also mentioned the streaming visual enhanced their engagement
and connection especially when most of the people turned off thecameras in the online class, which was a common practice of theclass we studied ("I could see the visuals like moving. So I can seethat people are listening or people are actually there attending theclass and will sort of indirectly causing me to feel like feel connectedto the students." (V1)).
Survey results also showed perceived psychological engagement
was slightly higher in the streaming session (M = 3.99, SD = 0.38)
than that without streaming (M = 3.73, SD = 0.4), with a marginal
significant difference (t[32]= 1.95 , (.05<p<.10)). Although there
was no significant difference of sense of community between twoconditions (t[33] = 0.92, n.s., without streaming: M = 4.29, SD = 1.01;
with streaming: M = 4.63, SD = 1.13). We will further explore incontrolled lab settings to clarify the effect of streaming visualizationon learners’ perception of social bonds.

4CONCLUSION AND FUTURE WORKThis field work presents the feasibility and potential of streamingcollective HR and HRV remotely from multiple users in real time.
Data contributors and viewers reported slightly different experiencein terms of participation. However, both of their feedback indicatesthat sharing HR and HRV during the online class could serve as
an extra channel to provide social cues for enhancing distributed
learners’ engagement and sense of community. In the future, we willfurther explore physiological, psychological, and behavior changes
among distributed learners during the feedback loop triggered byvisualizing shared physiological data.

ACKNOWLEDGMENTS
This work is conducted under the Cybernetic Being project supported by JST Moonshot R&D Program Grant Number JPMJMS2013
and the Keio University Doctorate Student Grant-in-Aid Programfrom Ushioda Memorial Fund.

REFERENCES[1] Jérémy Frey. 2016. Remote heart rate sensing and projection to renew traditionalboard games and foster social interactions. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 1865–1871.
[2] Zhuoqi Fu, Jiawen Han, Dingding Zheng, Moe Sugawa, Taichi Furukawa,
Chernyshov George, Hynds Danny, Padovani Marcelo, Marky Karola, Kouta
Minamizawa, et al. 2021. Boiling Mind-A Dataset of Physiological Signals during an Exploratory Dance Performance. In Augmented Humans Conference 2021.
301–303.
[3] Chad Harms and Frank Biocca. 2004. Internal consistency and reliability of thenetworked minds measure of social presence. (2004).
[4] Mariam Hassib, Mohamed Khamis, Susanne Friedl, Stefan Schneegass, and FlorianAlt. 2017. Brainatwork: Logging cognitive engagement and tasks in the workplaceusing electroencephalography. In Proceedings of the 16th international conference
on mobile and ubiquitous multimedia. 305–310.
[5] Mariam Hassib, Stefan Schneegass, Philipp Eiglsperger, Niels Henze, Albrecht
Schmidt, and Florian Alt. 2017. EngageMeter: A system for implicit audienceengagement sensing using electroencephalography. In Proceedings of the 2017
Chi conference on human factors in computing systems. 5114–5119.
[6] Yan He, Dingding Zheng, George. Chernyshov, Ragnar. Thomsen, Jiawen. Han,Danny Hynds, Yun Suen. Pai, Kai. Kunze, and Kouta. Minamizawa. 2021. Frisson
Waves: Sharing Frisson to Create Collective Empathetic Experiences for Music194

MUM 2021, December 5–8, 2021, Leuven, Belgium

Performances. In 2021 IEEE World Haptics Conference (WHC). 591–591. https://doi.org/10.1109/WHC49131.2021.9517258[7] Jacob B Holzman and David J Bridgett. 2017. Heart rate variability indices as
bio-markers of top-down self-regulatory mechanisms: A meta-analytic review.
Neuroscience & biobehavioral reviews 74 (2017), 233–255.
[8] René F Kizilcec, Chris Piech, and Emily Schneider. 2013. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In
Proceedings of the third international conference on learning analytics and knowledge. 170–179.
[9] Meng-Yun Liao, Ching-Ying Sung, Hao-Chuan Wang, Wen-Chieh Lin, and Fu-Yin
Cherng. 2019. Embodying historical learners’ messages as learning companions
in a VR classroom. In Extended Abstracts of the 2019 CHI Conference on Human
Factors in Computing Systems. 1–6.
[10] Fannie Liu, Laura Dabbish, and Geoff Kaufman. 2017. Can biosignals be expressive? how visualizations affect impression formation from shared brain activity.
Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1–21.
[11] Fred Shaffer and James P Ginsberg. 2017. An overview of heart rate variabilitymetrics and norms. Frontiers in public health 5 (2017), 258.
[12] Petr Slovák, Joris Janssen, and Geraldine Fitzpatrick. 2012. Understanding heart
rate sharing: towards unpacking physiosocial space. In Proceedings of the SIGCHI
conference on human factors in computing systems. 859–868.
[13] Moe Sugawa, Taichi Furukawa, George Chernyshov, Danny Hynds, Jiawen Han,
Marcelo Padovani, Dingding Zheng, Karola Marky, Kai Kunze, and Kouta Minamizawa. 2021. Boiling Mind: Amplifying the Audience-Performer Connection
through Sonification and Visualization of Heart and Electrodermal Activities. InProceedings of the Fifteenth International Conference on Tangible, Embedded, andEmbodied Interaction. 1–10.
[14] Na Sun, Mary Beth Rosson, and John M Carroll. 2018. Where is community
among online learners? Identity, efficacy and personal ties. In Proceedings of the2018 chi conference on human factors in computing systems. 1–13.
[15] Chiew Seng Sean Tan, Johannes Schöning, Kris Luyten, and Karin Coninx. 2014.
Investigating the effects of using biofeedback as visual stress indicator during
video-mediated collaboration. In Proceedings of the SIGCHI Conference on HumanFactors in Computing Systems. 71–80.
[16] Saijing Zheng, Mary Beth Rosson, Patrick C Shih, and John M Carroll. 2015.
Designing MOOCs as interactive places for collaborative learning. In Proceedingsof the Second (2015) ACM Conference on Learning@ Scale. 343–346.
[17] Saijing Zheng, Mary Beth Rosson, Patrick C Shih, and John M Carroll. 2015.
Understanding student motivation, behaviors and perceptions in MOOCs. InProceedings of the 18th ACM conference on computer supported cooperative work &social computing. 1882–1895.