Tranquillity at Home: Designing Plant-mediated Interaction forFatigue Assessment
Michi Kanda

Kai Kunzemichikanda@kmd.keio.ac.jp
Keio University Graduate School of Media DesignYokohama, Japan

Keio University Graduate School of Media DesignYokohama, Japan
kai@kmd.keio.ac.jp

ABSTRACTThis paper presents a human-plant interaction system that helps
observe daily mental fatigue levels by enabling the synchronisationof human eye blink data with plant health. We want to facilitate
introspection on our subjective well-being by leveraging plants’organic growth as a reflective medium. Based on users’ daily fatigue
levels assessed from their eye blink data, the system controls thequality and intensity of the grow LED installed on the sensoraugmented plant. Another LED placed in front of the planter also
displays the plant’s health, combining light intensity and daily
average eye blink frequency. We aim to design a novel means toconnect human biosignals with plant health and introduce humanplant interaction as a reflection on subjective well-being.

CCS CONCEPTS• Human-centered computing → Systems and tools for interaction design.

KEYWORDS
Human-Plant Interaction, Human-Plant Synchronisation, Bio-DigitalHybrid Systems, Positive Computing
ACM Reference Format:
Michi Kanda and Kai Kunze. 2021. Tranquillity at Home: Designing Plantmediated Interaction for Fatigue Assessment. In Augmented Humans International Conference 2021 (AHs ’21), February 22–24, 2021, Rovaniemi, Finland.
ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3458709.3458978

1Figure 1: Browser-based eye blink detection using TensorFlow’s face landmarks detection library (top). The augmented plant with a grow LED at the top of the planter andplant health LED indicator in front of the planter (bottom)

INTRODUCTION

Due to the prolonged social distancing measures and fear of theCOVID-19, people are increasingly spending a significant amountof time at home during the pandemic. This imposed restriction has
inevitably deprived many of one’s agency, putting further strain onmental health. As a way to combat growing concerns about deteriorating mental health, we present a novel interaction system that
supports reflection on one’s subjective well-being while offering
an opportunity to reconnect with nature at home through plants.
The therapeutic aspect of nature, including houseplants, hasbeen well researched [12, 15, 18]. However, there has been little
research into employing plants as a support medium for improving
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
AHs ’21, February 22–24, 2021, Rovaniemi, Finland© 2021 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8428-5/21/02.
https://doi.org/10.1145/3458709.3458978

292

subjective well-being in the HCI field. Some works have attempted
to facilitate healthy habits using plants as motifs [3, 8, 14]. Kaneret al., for instance, visualises an artificial plant on a water bottle toencourage users to drink more water [8]. While it is a rare example
that incorporates a plant to promote well-being, a plant remains a
motif rather than an interaction medium in this case.
There are also instances which incorporate physical plants as
information displays [5–7]. Yet, they are mostly interested in plantmovements for notifications or treating each plant as pixels indisplays. Only a few have considered incorporating personal digital
data into human-plant interaction [10, 11]. However, to the best of
our knowledge, we are not aware of other attempts to synchronisehuman biosignals with plants.
The contributions of this work are as follows:• Implementing an initial prototype that connects humanbiosignals with plant health to synchronise eye blink data
(i.e. fatigue levels) from a user with lights to increase/inhibit
plant growth.

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

Michi Kanda and Kai Kunze• Introducing and discussing human-plant interaction as areflective medium for subjective well-being.

2APPROACH

One important characteristic of the human mind is that it has significant fluctuations in productivity and capacity. Our mind has
ebb and flow and is affected by various factors, some of which we
do not even realise. For example, when it comes to judging one’s
fatigue levels, people seem to be quite unreliable [22]. There are avariety of technologies that can be used to assist fatigue assessment
by measuring biosignals [2, 17, 21, 22]; however, the examples thatsuccessfully integrate such technologies into human-plant interaction systems are close to none. Thus, we devised a human-plantinteraction system to synchronise human eye blink data (as a proxyof fatigue levels) with plant health. To assist people with effectively
assessing their fatigue levels, we link users’ cognitive state to plant
attributes in their environment by providing a more visible representation of the fluctuations in productivity and capacity. We focus
on assessing fatigue and mapping fatigue levels to plant health in
this prototype.

3PROTOTYPE DESIGNThe key parts of the system involve effectively measuring mental
fatigue levels and visualising them with plants to create a senseof human-plant synchronisation. To implement these parts, a reliable biosignal sensing method and intuitive visual feedback arenecessary. We used a browser-based eye blink data collection toassess human fatigue levels and use the data to control the growLED installed on top of a plant. We will elaborate on the setup andinteraction design in the next section.

3.1

Setup

The system is comprised of following components: 1) an augmented
plant with sensors (light, humidity and temperature sensors) andactuators for monitoring and visualising plant health, 2) a grow light(LED) to regulate plant growth, 3) and the software for computing
human fatigue levels based on daily eye blink data and sending it to
the augmented plant. An ivy plant was chosen for this setup since
it is well suited for growing indoor and can develop a discernible
growth within a relatively short period time (e.g. two leaves perweek [20]).

3.2

Measuring Mental Fatigue Levels

For measuring mental fatigue levels, we built a browser-based eye
blink detection system using face landmarks detection library fromTensorFlow.js. Since the library does not compute eye blinks, weapplied the algorithm proposed by Soukupová and Čech [19]. The
algorithm utilises a metric called the eye aspect ratio. It computes
the absolute distance between each set of vertical eye landmarks(the difference between P2 and P6, P3 and P5 in the figure 1) anddivide the sum of each vertical distance by the multiple of 2 of theabsolute distance of horizontal eye landmarks (P1 and P4 in thesame figure). The equation for the eye aspect ratio is as follows:
EAR =Figure 2: Implementation of eye blink ratio based blink detection (based on Soukupová et al.[19])3.3

Synchronising Fatigue Levels with Plant
HealthTo communicate mental fatigue levels with a plant, we employed agrow LED in two colours: 1) purple (the combination of blue and red)
for no or minimal fatigue, and 2) blue for increased fatigue levels.
The rationale for choosing these colours is that they are known toinfluence plant growth; purple facilitates plant growth [13, 16], andblue inhibits it [4, 9]. In other words, an optimal cognitive statusis linked to facilitating plant growth, and a deteriorating cognitive
status is linked to inhibiting plant growth. Upon determining thecolour of the grow LED, the threshold was set to 15 blinks per
minute since it has been reported as normal spontaneous blinkrate [1, 17]. If the average blink rate per minute of a user is higher
than 15 (indicating fatigue), the LED turns to blue (inhibiting plantgrowth). If the blink rate is equal or below the blink rate threshold,the LED turns to purple (promoting plant growth). The blink rate
threshold can be adapted for individual users in future work.
As for determining plant health, we employed pseudo metrics tovisualise changes in daily plant health observable to human eyes.
In addition to representing human fatigue levels, the two grow LEDalso influence plant health in this prototype. Based on the growLED intensity read from the light sensor and eye blink frequency
thresholds, the augmented plant visualises its health status in green,
yellow, and red, indicating good health, diminishing health, andneed for attention respectively.
In short, by showing the direct link between human fatigue
levels and plant health, the system offers a means to reflect on
one’s subjective well-being and aims to show how plants and human well-being can improve together as a result of human-plantsynchronisation.

4CONCLUSION & FUTURE WORKIn this paper, we introduced the concept of human-plant synchronisation and how an interaction system supported by this concept
can visualise subjective well-being. This research delved into approaches that enhance a symbiotic relationship between humans
and plants through visualising the connection between humanbiosignals (i.e. eye blink data) and plant health. Future work will
involve an in-the-wild evaluation study to observe the changes inboth human subjective well-being and plant growth over the long
period of time. It will also evaluate how a degree of human-plantsynchronisation will affect human subjective well-being.

||p2 − p6|| + ||p3 − p5||2||p1 − p4||

293Tranquillity at Home

AHs ’21, February 22–24, 2021, Rovaniemi, Finland

REFERENCES[1] Abusharha A. A. 2017. Changes in blink rate and ocular symptoms during
different reading tasks. Clinical optometry, 133–138. https://doi.org/10.2147/
OPTO.S142718[2] Jie Bao, Jiawen Han, Akira Kato, and Kai Kunze. 2020. Sleepy watch: towards
predicting daytime sleepiness based on body temperature. In Adjunct Proceedingsof the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2020 ACM International Symposium on WearableComputers. 9–12.
[3] Sunny Consolvo, David W. McDonald, Tammy Toscos, Mike Y. Chen, Jon
Froehlich, Beverly Harrison, Predrag Klasnja, Anthony LaMarca, Louis LeGrand,Ryan Libby, Ian Smith, and James A. Landay. 2008. Activity Sensing in theWild: A Field Trial of Ubifit Garden. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy) (CHI ’08). Association for Computing Machinery, New York, NY, USA, 1797–1806. https://doi.org/10.1145/1357054.1357335[4] Daniel J. Cosgrove. 1981. Rapid Suppression of Growth by Blue Light: Occurrence,
Time Course, and General Characteristics. Plant Physiology 67, 3 (1981), 584–590.
http://www.jstor.org/stable/4266691[5] Vito Gentile, Salvatore Sorce, Ivan Elhart, and Fabrizio Milazzo. 2018. Plantxel: Towards a plant-based controllable display. In Proceedings of the 7th ACM
International Symposium on Pervasive Displays. 1–8.
[6] Phillip Gough, Larissa Pschetz, Naseem Ahmadpour, Leigh-Anne Hepburn, ClareCooper, Carolina Ramirez-Figueroa, and Oron Catts. 2020. The Nature of biodesigned systems: Directions for HCI. In Companion Publication of the 2020 ACM
Designing Interactive Systems Conference. 389–392.
[7] Jan Hammerschmidt, Thomas Hermann, Alex Walender, and Niels Krömker. 2015.
InfoPlant: Multimodal augmentation of plants for enhanced human-computerinteraction. In 2015 6th IEEE International Conference on Cognitive Infocommunications (CogInfoCom). IEEE, 511–516.
[8] Gül Kaner, Hüseyin Uğur Genç, Salih Berk Dinçer, Deniz Erdoğan, and AykutCoşkun. 2018. GROW: a smart bottle that uses its surface as an ambient display
to motivate daily water intake. In Extended Abstracts of the 2018 CHI Conferenceon Human Factors in Computing Systems. 1–6.
[9] Sun-Ja Kim, Eun-Joo Hahn, Jeong-Wook Heo, and Kee-Yoeup Paek. 2004. Effectsof LEDs on net photosynthetic rate, growth and leaf stomata of chrysanthemum
plantlets in vitro. Scientia Horticulturae 101, 1 (2004), 143–151. https://doi.org/
10.1016/j.scienta.2003.10.003[10] Satoshi Kuribayashi, Yusuke Sakamoto, and Hiroya Tanaka. 2007. I/O plant: atool kit for designing augmented human-plant interactions. In CHI’07 extended
abstracts on Human factors in computing systems. 2537–2542.
[11] Satoshi Kuribayashi and Akira Wakita. 2006. PlantDisplay: Turning Houseplants
into Ambient Display. In Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (Hollywood, California,USA) (ACE ’06). Association for Computing Machinery, New York, NY, USA,40–es. https://doi.org/10.1145/1178823.1178871[12] Ms. Lee, J. Lee, and BJ. et al Park. 2015. Interaction with indoor plants may reduce
psychological and physiological stress by suppressing autonomic nervous system
activity in young adults: a randomized crossover study. J Physiol Anthropol 34(April 2015). https://doi.org/10.1186/s40101-015-0060-8[13] Kuan-Hung Lin, Meng-Yuan Huang, Wen-Dar Huang, Ming-Huang Hsu, Zhi-Wei
Yang, and Chi-Ming Yang. 2013. The effects of red, blue, and white light-emitting
diodes on the growth, development, and edible quality of hydroponically grownlettuce (Lactuca sativa L. var. capitata). Scientia Horticulturae 150 (2013), 86–91.
https://doi.org/10.1016/j.scienta.2012.10.002[14] Yikun Liu, Yuan Jia, Wei Pan, and Mark S. Pfaff. 2014. Supporting Task Resumption
Using Visual Feedback. In Proceedings of the 17th ACM Conference on Computer
Supported Cooperative Work & Social Computing (Baltimore, Maryland, USA)
(CSCW ’14). Association for Computing Machinery, New York, NY, USA, 767–777.
https://doi.org/10.1145/2531602.2531710[15] A. Sachs Naomi. 2020. Access to Nature Has Always Been Important; With
COVID-19, It Is Essential. Health Environments Research & Design Journal,242–244. https://doi.org/10.1177/1937586720949792[16] D.T. Nhut, T. Takamura, Hiroko Watanabe, A. Murakami, K. Murakami, andM. Tanaka. 2002. Sugar-free micropropagation of Eucalyptus citriodora using
light-emitting diode (LEDs) and film-rockwool culture system. Environ Control
Biol 40 (01 2002), 147–155.
[17] Soha Rostaminia, Addison Mayberry, Deepak Ganesan, Benjamin Marlin, andJeremy Gummeson. 2017. Ilid: low-power sensing of fatigue and drowsinessmeasures on a computational eyeglass. Proceedings of the ACM on interactive,
mobile, wearable and ubiquitous technologies 1, 2 (2017), 1–26.
[18] Seiji Shibata and N. Suzuki. 2001. Effects of indoor foliage plants on subjects’
recovery from mental fatigue. North American Journal of Psychology 3 (01 2001),385–396.
[19] Tereza Soukupová and Jan Čech. 2016. Real-Time Eye Blink Detection using
Facial Landmarks.

294[20] Otto L. Stein and Elizabeth B. Fosket. 1969. COMPARATIVE DEVELOPMENTAL
ANATOMY OF SHOOTS OF JUVENILE AND ADULT HEDERA HELIX. American
Journal of Botany 56, 5 (1969), 546–551. https://doi.org/10.1002/j.1537-2197.1969.
tb07568.x arXiv:https://bsapubs.onlinelibrary.wiley.com/doi/pdf/10.1002/j.15372197.1969.tb07568.x[21] Benjamin Tag, Ryan Mannschreck, Kazunori Sugiura, George Chernyshov, Naohisa Ohta, and Kai Kunze. 2017. Facial thermography for attention tracking
on smart eyewear: An initial study. In Proceedings of the 2017 CHI Conference
Extended Abstracts on Human Factors in Computing Systems. 2959–2966.
[22] Benjamin Tag, Andrew W Vargo, Aman Gupta, George Chernyshov, Kai Kunze,
and Tilman Dingler. 2019. Continuous alertness assessments: Using EOG glasses
to unobtrusively monitor fatigue levels In-The-Wild. In Proceedings of the 2019
CHI Conference on Human Factors in Computing Systems. 1–12.