Work in Progress

TEI 2017, March 20–23, 2017, Yokohama, Japan

Eyewear to Make Me Smile – Can
Electric Muscle Stimulation increase
Happiness?
Lai Yen-Chin
Keio Media Design
Keio University
Yokohama, Japanlaiyen@kmd.keio.ac.jp

Junichi Shimizu
Keio Media Design
Keio University
Yokohama, Japan
jun.shimi@kmd.keio.ac.jp

YuanLing Feng
Keio Media Design
Keio University
Yokohama, Japanyfeng@kmd.keio.ac.jp

Takuro Nakao
Keio Media Design
Keio University
Yokohama, Japana8111140@kmd.keio.ac.jp

Abstract
This paper suggests a prototype system making users
smile using electrical muscle stimulation (EMS). We present
design considerations, our approach, and the current system setup. We continue to describe our first insights into
using an unobtrusive EMS setup to stimulate smiles andshow first findings from our entry in this year’s UIST student
innovation competition. We also outline our approach andfollow-up experiments to potentially use this technology toimprove user’s mood.

Author Keywords
Kai Kunze
Keio Media Design
Keio University
Yokohama, Japan
kai@kmd.keio.ac.jp

Electric Muscle Stimulation; Smile; Smart Glasses

ACM Classification Keywords
H.5.m [Information interfaces and presentation (e.g., HCI)]:
Miscellaneous

Introduction
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
TEI ’17 , March 20-23, 2017, Yokohama, Japan
ACM 978-1-4503-4676-4/17/03.
http://dx.doi.org/10.1145/3024969.3025097

With the increases in mental issues and disorders on therise, we want to explore if we can change or better enhance
our mood using technology [3, 6]. Given James-Lange Theory of Emotion that suggests that physiological reactions
can induce feelings and emotions, people can feel happierjust by smiling more throughout the day [2]. We want to create technologies to manage better our affect. This paper
presents an initial small step towards that direction.

579

Work in Progress

TEI 2017, March 20–23, 2017, Yokohama, Japan

Our contributions are as follows: (1) we evaluate if it’s possible to induce smiles using a relative unobtrusive ElectricMuscle Stimulation (EMS) setup, (2) we discuss a electrodeplacements and other issues suitable for inducing smiles
with the potential to integrate them in a smart glasses design, (3) we show insights learned from participating in theUIST student innovation competition with our prototype system, (4) we present an initial experimental design to evaluate if induces smiles from EMS can actually change theuser’s emotion.
Overall, we believe it’s an important first step to investigate
EMS for emotion control.Given we will get clearance fromthe ethics board of our institution, participants at TEI can try
an excellent demonstration of our technology at TEI (aftersigning a consent form).

Approach

Figure 1: Setup and pictures fromthe student competition

There are already technologies utilizing James-Lange Theory of emotion; most prominently, the Smile-encouraging
digital appliances by Tsujita and Rekimoto [9]. However,their approach is to "force" the user to smile for interactionswith digital devices. For example, to turn your alarm clock
off in the morning you have to smile to your alarm clock.
Our approach is different. We are following embodiment
theory and some indication of psychology that it might bepossible to induce emotions by physically changing yourappearance using external stimuli. In our case by making you smile using electric muscle stimulation. Of course,
the movement is very different from a voluntary or "forced"
smile (like in Tusjita’s work): only a few muscles can be activated etc. Yet, there is interesting work from Barrett et al.
in constrained psychology lab studies that it might be feasible [1].

In this paper, we are introducing our first step to inducesmiles with unobtrusive hardware. In a next step, in future
work, we will evaluate the mental effects of EMS.

Student Competition Entry: FaceCook
We participated in a Student Innovation Competition atUIST a large international HCI conference with an application to induce smiles using EMS. The basic idea for ourcompetition entry was to create a game between 2 users.
One user is in a virtual environment and gets different smile
intensities (smirk, smile, stronger smile). See Figure 1 forpictures from the competition. A user outside of the virtual environment has to guess which smile intensity theother user sees. Our entry won the "Innovative UI Award" inthe competition. In the following we will go over the systemsetup as well as the initial trials during the competition.
Experimental Setup
We use a modification1 to the EMS toolkit prototype by Peiffer et al. [7].
To generates a smiling facial expression by actuating theusersâĂŹ facial muscles with an electrical muscle stimulation (EMS) unit. This unit provides 2 independent channels
with an intensity of up to 70 mA. Each channel has 2 roundelectrode pads sized 32mm in diameter.
Electrode Placement
The electrodes were placed on specific locations of thecheek to maximize the cheek muscle stimulation. These
locations were selected due to the nerve position on the human face, as shown in figure. In order to generate a smiling
facial expression, we put electrode pads around zygomatic
branches and buccal branches.

1

http://plopes.org/ems580

Work in Progress

TEI 2017, March 20–23, 2017, Yokohama, Japan

We conducted an experiment on 23 participants in the Student Innovation Contest held at the large international HCI
conference.
Initial Experiments
As inducing smiles is highly dependent on the right electrode placement and time for the placement was limited (to1-2 minutes per participant), it was unfortunately unpracticalto play the game as introduced above. Therefore, in eachtrial, we set up a virtual environment using the Oculus Rifthead mount display(HMD) with 3 videos presenting different levels of smiling/laughing faces. Each video triggers a
different intensity of electrical stimulation which generated
different facial muscles movements on the participants.

Initial Results

Figure 2: The top schematicshows the facial nerve.
cz 2015-2016
TeachMeAnatomy.com[CC-BY-NC-ND 4.0] The second
picture shows the electrodeplacement used in our trials: weput electrode pads at thezygomatic branches and buccal
branches to induce the smile

8 participants had their cheek muscles stimulated accurately to create a smiling expression of three levels. For 14
participants we could see a "smiling effect" (twitching) forthe correct facial muscles, however the 3 stages for smiling were not distinguishable. For one participant we could
not induce an effect at all. This seems at the first glance
not so encouraging, however given the short setup phase(1-2 minutes) to find the right muscles. We believe that thiscan be improved. After the competition we found also another electrode placement (placing the second electrode on
the lower cheek underneath the mouth) that seems to work
more reliable for a quick setup (and for some applicationfield). Yet, we will continue to use the previous setup as it isbetter integratabtle in a smart glasses design.
In the following we give a couple of qualitative statementsfrom the participants. The participants stated that "Withthis system, I feel it is easier for me to smile." And one ofthe participants also described the feeling as "It is tricky
because I did feel that I was smiling more when my facialmuscles moved but the stimulation also made me feel uneasy." For the content in our experiment design, some participants thought "It is good that the stimulation and the content were showing the same level of smiling/laughing. It
made the system easier to understand." However, one ofthe participants commented that it was difficult to differentiate between the effectiveness of the system, or the generalidea of the proposed work that was amusing, that caused
him or her to smile.

Next Steps
The major next steps are for us to make the smile stimulation over EMS more reliable. Here we made already acouple of improvements, as it’s easier for us to find the rightpositions for the electrodes. Also just increasing the time forelectrode adjustments to around 5 minutes already helps(not possible during the contest).
The most interesting follow up research is of course if changing a user’s face using EMS actually can influence their perceived happiness over a short or longer time. Here we will
follow the controlled experimental setups from psychology
first and assessing the effects for shorter durations [1].
If these experiments go well, we will integrate the technology in a smart eyewear design and will explore long term,
more unconstrained mood effects using an EMS glasses
design (as shown in the sketches in Figure 3).

Related WorkAs mentioned before the closest to our work is Tsujita etal.’s research using smiles as an interaction modality and"happiness counter" [9].
Regarding EMS, there is also exiting research using it forinteractive systems as an output modality from pedestriancruise control to guiding drawings [8, 5].

581

Work in Progress

TEI 2017, March 20–23, 2017, Yokohama, Japan

There are a couple of psychologists, most prominently Barrett and Ekman, assessing the effects of inducing smiles
over EMS [1, 4]. We see these studies as complimentary
as they strengthen our hypothesis about the emotional influence of smiling, however they are conducted in highly
constrained lab environments with more obtrusive setups(more electrodes etc.).

3. Alain Ehrenberg. 2016. The weariness of the self:
Diagnosing the history of depression in thecontemporary age. McGill-Queen’s Press-MQUP.

We are unaware of any work that uses EMS in the face as ainteraction technology.

5. Pedro Lopes, Doăa Yüksel, François Guimbretière, andPatrick Baudisch. 2016. Muscle-plotter: An Interactive
System based on Electrical Muscle Stimulation thatProduces Spatial Output. In Proceedings of the 29th
Annual Symposium on User Interface Software andTechnology. ACM, 207–217.

Conclusion
We have presented our first insights in how to use EMS tostimulate smiles and showed first findings from the student
innovation competition at a large international HCI conference.
We believe that EMS is an interesting technology to evaluate in terms of embodiment and would like to show ourfindings and spark discussions demonstrating our system
for the TEI community.

Acknowledgements
This work is partly supported by the JST CREST Project onExperience Supplements.
Figure 3: Concept: fromelectrodes to a smart eyewear
device to induce smiles

REFERENCES
1. Lisa Feldman Barrett. 2012. Emotions are real.
Emotion 12, 3 (2012), 413.
2. Walter B Cannon. 1927. The James-Lange theory ofemotions: A critical examination and an alternativetheory. The American journal of psychology 39, 1/4(1927), 106–124.

4. Paul Ekman, Richard J Davidson, and Wallace V
Friesen. 1990. The Duchenne smile: Emotional
expression and brain physiology: II. Journal ofpersonality and social psychology 58, 2 (1990), 342.

6. Mark Olfson, Benjamin G Druss, and Steven C Marcus.
2015. Trends in mental health care among children andadolescents. New England Journal of Medicine 372, 21(2015), 2029–2038.
7. Max Pfeiffer, Tim Duente, and Michael Rohs. 2016. Let
your body move: a prototyping toolkit for wearable force
feedback with electrical muscle stimulation. In
Proceedings of the 18th International Conference on
Human-Computer Interaction with Mobile Devices andServices. ACM, 418–427.
8. Max Pfeiffer, Tim Dünte, Stefan Schneegass, FlorianAlt, and Michael Rohs. 2015. Cruise Control for
Pedestrians: Controlling Walking Direction using
Electrical Muscle Stimulation. In Proceedings of the33rd Annual ACM Conference on Human Factors in
Computing Systems. ACM, 2505–2514.
9. Hitomi Tsujita and Jun Rekimoto. 2013.
Smile-encouraging digital appliances. IEEE Pervasive
Computing 4, 12 (2013), 5–7.

582