2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth)

Nekoze! – Monitoring and Detecting Head Posture while
Working with Laptop and Mobile Phone
Katsuma Tanaka, Shoya Ishimaru, Koichi Kise

Kai Kunze, Masahiko Inami

Osaka Prefecture University, Japan{katsuma, ishimaru}@m.cs.osakafu-u.ac.jp
kise@cs.osakafu-u.ac.jp

Graduate School of Media Design
Keio University, Yokohama, Japan
Email: {kai,inami}@kmd.keio.ac.jp

Abstract—Neck pain and other spine related injuries are on
the rise. One potential cause is bad head posture while using
digital devices. We present two systems to monitor and improvehead posture for two of the common problematic cases: laptop
and mobile phone use. The laptop system uses the front-facing
camera. The mobile system the sensor data of a smart glasses
prototype. Both systems work reasonable in an initial user study
with 10 and 8 participants (72 % detection on the laptop system,
100% for the mobile one). From the discussion, it seems users
are especially interested in the smart glasses based monitoring
implementation.
Keywords—eye wear computing; head posture; smart glasses

I.

I NTRODUCTION

As more and more people spend time on digital devices,
injuries and chronic conditions related to this increased usage
are also on the rise. Using computing devices with bad posture
can lead to neck pain, spine problems, repetitive stress injuries,
just to name a few.
However, so far most researchers focus on recommending
regular breaks and to correct their posture over using altering
their office (e.g. monitor adjustments, standing desks). They
disregard the emerging ergonomic problems related to smart
phones and tablets[5]. Especially, with increase of smartphone
usage for longer periods of time, users tend to stay in unhealthy
postures while writing text messages or playing mobile games.
They bend their head forward too much. With the ease ofaccess to smart phones, there is the concern of over-useage.
This combined with the unhealthy head positions can lead tochronic neck pain, musculoskeletal disorders, repetitive stressinjuries (RSI) or worse device-usage-related injuries[14].

Fig. 1.

User wearing a J!NS MEME smart glasses prototype.

II.

R ELATED W ORK

A couple of researchers work on dedicated Sensors. Denmans et al. use a system based on a bend sensor to detectbad head postures and give feedback to the users[4]. Sonne
et al. embed sensors into a ”Habit aware mouse”[12] focusing
mostly on hand strain.

We believe we need a new approach towards posturemonitoring and improvement focusing on unobtrusive sensing
and seamless interactions with the user to prevent bad postureand increase quality of life.

There is a lot of research focusing on detecting headpostures using accelerometers [9], [1]. Yet, most focuses on
augmented reality or other applications. The others, we see ascomplementary work as the hardware/algorithms are usuallymore complex using bigger dedicated sensing devices.

The contributions of this paper are as follows: (1) we
describe an approach and a system to monitor head posturesusing a laptop and a mobile system using smart glasses.
(2) We detect bad head posture (over an 19 degree sholder
angle) for both systems. (3) We present a user study andexpert evaluation for both system. Feedback indicates that oursystems are already usable as is. The laptop system detected
72 of 100 bad posture states. The mobile system detected all
100 during our initial user tests. (4) We discuss results andpresent recommendations for a head posture monitoring andimprovement system.

The focus of the research is also mostly on tracking rehabilitation progress instead of working on prevention. Arteaga
et al. developed a system to help stroke patients from having
bad posture[1]. They implemented a monitoring system placing
accelerometers and actuators at different the body parts. They
warn the patient over several modalities and also log bad
postures for the doctor to check later. Yet, they did not focus
on head posture. Along this lines, Daian et al. present an
intelligent chair to measure posture in the office using force
resistive sensors[3] focusing on body posture while sitting.

237

978-1-63190-045-7 © 2015 ICSTDOI 10.4108/icst.pervasivehealth.2015.260226

There is also an Android app called ”Text Neck Indicator”1 .
Yet it just monitors the angle of the smart phone not the head.
Close to our work is the research by Lee et al. and Shinet al. [10], [11]. Lee et al. present an algorithm to monitor theneck angle using a front facing camera on the smart phone.
Yet, they don’t present an evaluation for the system. Shin et al.
use smartphone usage patterns (e.g. the ratio of SMSs to calls,
the number of event-initiated sessions) to identify problematicusage patterns impacting social and heath aspects of users.

Fig. 2.

Notification of the desktop Nekoze application

There are also a couple posture monitoring/improvement
systems using computer vision. Jaimes et al. uses a camera
to detect activities and related postures. They show a small
icon on the computer screen to indicate the quality of posture
related to the current activity[8], [7].
Posture monitoring systems have been proposed by Jaimes
and Liu [5, 6]. Their systems use computer vision to determine the user’s activities through his/her posture. By creating
postural awareness Jaimes helps the user to improve his/her
posture by continually indicating posture quality via an icon
[5].
Related to using smart glasses, Hong et al. use smart
earphones with integrated accelerometer and gryroscope tomonitor the posture [6]. Yet their goal is also not headangle determination. They monitor how long the user is notmoving/staying in the same postures. The user can get a recordof posture shifts during the day.
III.

A PPROACH

We decided to focus on head posture as it seems to be
an increasing problem with the use of laptops and especiallysmart phones/tablets. The first question for us was what constitues a bad head position. Consulting ergonomics research,straining (or bad) head postures are defined in ergonomics
over a head/shoulder angle higher than 19 degrees[2]. We
focus on monitoring this head angle. Additionally, working
with increases in neck angle (over 19 degrees) for over longer
time are associated with loss in height, torque increases on
the cervical spine, and increased discomfort[2], [13]. As seenfrom this work as well, most cases of bad posture happen
during laptop use due to their bad ergonomic design and during
mobile phone/tablet use. Therefore, we want to tackle theses
cases and decided to explore both cases by designing initialsystems for head posture detection.
Our goal is to do monitoring as unobtrusive as possible
best with the devices themselves. That’s why we decided firstto use the front-facing camera on a laptop/tablet/phone only.
Initial tests confirmed that on a laptop computer it’s feasible.
Without much trouble we could detect bad head postures using
straight forward Open CV face detection libraries.
However, with early tests we had really bad results on thephones and tablets. Compared to the laptop for some users theface is not always full in the view of the camera. Detecting
distances and posture is also more difficult due to the varying
phone positions and angles to the face. Therefore we decided
to include smart glasses. A lot of people already wear glasses
for work on a computer or mobile screen. We did not want to
1 https://play.google.com/store/apps/details?id=com.textneckbeta.gui&hl=de238

Fig. 3. The vector sum of the raw acceleration axis during a bad postureincident (head angle over 19 degrees). The change is easily visible only fromthe raw values.

use dedicated posture detection devices as some related workand head phones seemed to be too limited in use ( e.g. need
to be taken out for social interactions). In the following wewill discuss our first implementations of the two systems. It
is to note that these are exploratory implementations to gather
feedback.
IV.

I MPLEMENTATIONSAs mentioned, we focus on the laptop and mobile use cases.
We implemented two Nekoze! prototypes one for each use casedescribed in the following. Nekoze is Japanese meaning ”toarch one’s back like a cat”. The word is used to refer to bad
posture when working on a computer or mobile device.
A. Nekoze! Laptop Prototype
For the laptop system, we use an implementation thataccess the front-facing camera available on most systems. It
applies face detection and detects if the face gets too close to
the screen. The prototype is implemented Mac OS X.
As soon as the user gets too close to the screen, we have
3 stimuli: a notification on the top bar (see Fig 2), notificationand sound, just sound, or a ”screen flash” (screen brightness
increases slightly and contrast is reduced ).
For the user study later, we had a calibration at the
beginning of the session, with the user sitting straight.
B. Nekoze! Mobile Prototype
For the mobile prototype we use a J!NS MEME prototype
and an iPhone. J!NS MEME are sensing glasses includingAdditionally, We ask 5 users to hold their head a specificangle (5,15,20,25,30,35) 10 times and record data. We determine the angle using a protractor. After applying the low passfilter and training the support vector regression we get an errorof roughly 5 degrees with a standard deviation of 3.
B. Expert Evaluations
Fig. 4. A participant holding the phone in a straining posture with a headangle over 19 degrees (left) and in a neutral postion (right) under 19 degrees.

electrodes to detect eye movements and inertial motion sensors(accelerometer/gyroscope). For the system we just use theaccelerometer. MEME streams the sensor data to a laptop or
smartphone with a sampling rate of 60 Hz for the acceleration.
Battery runtime for the current prototype is around 8 hours.
We use the raw accelerometer data from the glasses toestimate the head angle. Already simple thresholding performs
quite well. Yet, we calculate the vector sum of x,y,z acceleration afterwards we apply a low pass filter (cutoff 1.5 Hz) and
use the acceleration vector sum value as input for a support
vector regression to determine the head angle.
Fig 3 shows the raw accelerometer vector sum for a bad
posture incident.
If the head angle is over 19 degrees, we use the following
feedback mechanisms: Standard notification shown at the topof the mobile screen or a screen flash (changing brightness
and contrast).
V.

I NITIAL E VALUATION

A. User Studies
Both studies were conducted in several offices and school
environments indoors at desks where people tend to work withcomputers/laptops and for the mobile study standing on thefloor in the kitchen, office or living rooms.
We had 10 participants (4 female, mean age 27, std 10)
for the laptop study. We introduce the laptop system to theparticipants, show them the functionality and let them try it
while playing a browser game and web surfing (between 5-15min.). For the test we use notifications with sound. The users
perform 10 x straining posture (head angle over 19 degrees)and 10 x normal posture positions in front of the system. Afterthe test we interviewed them and presented them the alternative
feedback mechanisms.
For the mobile study we had 8 participants (3 female, mean
age 34, std 8). We ask the users to put the smartphone into
their pocket, take it out and use it as they regularly would.
Measuring the head angle 5 of the 8 users showed a bad posture(head angle over 19 degrees).
After this test we introduce them to the background of ourstudy and the mobile system. The users can try the systemusing a game (a puzzle game called 2045) or a longer text toread. We also record 10 x straining posture (head angle over
19 degrees) and 10 x normal posture positions from the users(see Fig. 4).
239

For expert evaluation we discussed with two medical
professional working in physical rehabilitation, specializing inneck and back related injuries.
After showing them the two systems, we conducted an
informal interview and discussed with them. They liked theoverall system designs and think it could already be used forpatients. They prefer the mobile smart glasses solution as it
is able to detect bad posture not only while sitting in frontof a computer. However, for both systems they criticized thenotifications. They informed us that chronic neck pain andsimilar symptoms are seldom from a couple of short bad
posture incidents. What is potentially worse is one bad postureincident held for a long time (e.g. a user playing a game for
10 -20 minutes with bad head posture). Therefore, repeated
notifications after 2-5 minutes of bad posture make more sense.
Showing them the different feedback mechanisms both alsopreferred the screen flash. One professional mentioned that it
might be too easy to ignore. She would introduce a more andmore severe feedback (heavier flashing) the longer the posture
was not changed. The other professional mentioned he would
like to see a summary of a patients posture changes duringday, to give better guidance and advice to his patients.
Regarding user interface and system, we interviewed to
two HCI researchers. They also didn’t like the notification
type feedback too much, screen flash was also their preferred
feedback. They also mentioned haptics (vibration) or an led in
the glasses as other modalities to inform the user. One Expert
remarked that it might be worth exploring if there are feedback
possibilities so that the user does not actively need to change
the position, but the system induces the change subconsciously(e.g. muscle stimulation).
VI.

D ISCUSSION

For both user studies, users remarked they would like to
use a system to track their posture during everyday life. 15 of
the 18 users have trouble with the neck pain or a stiff necksometimes.
Discussing the laptop system first, the notification or soundwere not so attractive to 6 users, they preferred the screen
flash. 2 users preferred the notifications without sound and2 were indifferent. 7 users could imagine to use the system
as is, for 2 any presented feedback was too annoying to use
everyday. Yet, they could not give suggestions on what type of
feedback they wanted instead. Our laptop system could detect
72 of the 100 bad posture states. For one user the system
did not perform well as there were some lighting issues (sunbacklight). 3 users voiced some frustration that the system was
not working optimally (didn’t detect their bad posture). One
user mentioned he is using MacBreakZ 2 , a personal ergonomic
assistant program designed to promote healthy and productive
2 http://www.publicspace.net/MacBreakZ/computer use. It monitors the typing behavior and provides
exercise breaks accordingly. He could imagine Nekoze! to do
the same for back pain (not only for hands).
About the mobile system, all 8 users liked the idea ofusing smart glasses to detect. 5 users already wear everyday.
Also here the users preferred the screen flash (all 8) comparedto the notifications. Still 3 users found even the screen flash
to irritating especially when reading. One user mentioned hewould like to get just a vibration on the phone or better on thesmart glasses. All straining posture states could be correctlyclassified by the system. One user asked if the glasses could
also be used to monitor/analyze her steps, as she had trouble
a couple of years ago related to uneven gait.
VII.

C ONCLUSION AND F UTURE W ORK

Next we will start an exploratory user study to explore
potential feedback ideas more. In term we will conduct alsoa larger experimental setup to validate the posture detection
on the smart glasses. Several users and experts suggested tobuild out the long term monitoring for later review and custom
interventions. This is also a direction we will explore more.
Other important future work include improving the recognition
system for the latop Nekoze! validating the detection forboth systems on a larger population, implementing log/better
feedback mechanisms and identifying how/when to intervene.
ACKNOWLEDGMENT
This work is partly supported by Kakenhi Grant-in-Aid B
No. 26730095 and the CREST project from Japan Society for
the Promotion of Science (JSPS).
R EFERENCES

[2]
[3]

[4]

[5]

[7]

[8]

[9]

[10]

In this paper we presented 2 systems to detect straining had
positions and designed initial user studies to collect feedback
for further development as well as to evaluate the current
systems performance. The results of our first posture monitoring and improvement systems seems encouraging. Especiallysmart glasses seem an interesting approach to track posture,as they can be used not only while working on the laptop but
also during other everyday situations. e.g. reading a hardcover
book.

[1]

[6]

Sonia Arteaga, Jessica Chevalier, Andrew Coile, Andrew William Hill,
Serdar Sali, Sangheeta Sudhakhrisnan, and Sri H. Kurniawan. Low-costaccelerometry-based posture monitoring system for stroke survivors. In
Proceedings of the 10th International ACM SIGACCESS Conference on
Computers and Accessibility, Assets ’08, pages 243–244, New York,NY, USA, 2008. ACM.
RA Bonney and EN Corlett. Head posture and loading of the cervicalspine. Applied Ergonomics, 33(5):415–417, 2002.
Ioana Daian, AM Van Ruiten, Albertine Visser, and Senka Zubic. Sensitive chair: a force sensing chair with multimodal real-time feedback
via agent. In Proceedings of the 14th European conference on Cognitive
ergonomics: invent! explore!, pages 163–166. ACM, 2007.
Carrie Demmans, Sriram Subramanian, and Jon Titus. Posture monitoring and improvement for laptop use. In CHI ’07 Extended Abstracts on
Human Factors in Computing Systems, CHI EA ’07, pages 2357–2362,
New York, NY, USA, 2007. ACM.
KENNETH K HANSRAJ. Assessment of stresses in the cervicalspine caused by posture and position of the head. Surgical technology
international, 25:277–279, 2014.

240

[11]

[12]

[13]

[14]

Dezhi Hong, Ben Zhang, Qiang Li, Shahriar Nirjon, Robert Dickerson,
Guobin Shen, Xiaofan Jiang, and John Stankovic. Septimu: continuous
in-situ human wellness monitoring and feedback using sensors embedded in earphones. In Proceedings of the 11th international conferenceon Information Processing in Sensor Networks, pages 159–160. ACM,
2012.
Alejandro Jaimes. Sit straight (and tell me what i did today): a humanposture alarm and activity summarization system. In Proceedings of the
2nd ACM workshop on Continuous archival and retrieval of personal
experiences, pages 23–34. ACM, 2005.
Alejandro Jaimes. Posture and activity silhouettes for self-reporting,interruption management, and attentive interfaces. In Proceedings ofthe 11th international conference on Intelligent user interfaces, pages
24–31. ACM, 2006.
Soonmook Jeong, Taehoun Song, Hyungmin Kim, Miyoung Kang,
Keyho Kwon, and Jae Wook Jeon. Human necks posture measurement
using a 3-axis accelerometer sensor. In Computational Science and Its
Applications-ICCSA 2011, pages 96–109. Springer, 2011.
Hosub Lee, Young Sang Choi, and Sunjae Lee. Mobile posturemonitoring system to prevent physical health risk of smartphone users.
In Proceedings of the 2012 ACM Conference on Ubiquitous Computing,
UbiComp ’12, pages 592–593, New York, NY, USA, 2012. ACM.
Choonsung Shin and Anind K. Dey. Automatically detecting problematic use of smartphones. In Proceedings of the 2013 ACM International
Joint Conference on Pervasive and Ubiquitous Computing, UbiComp’13, pages 335–344, New York, NY, USA, 2013. ACM.
Tobias Sonne and Kaj Grønbæk. Exploring new potentials in preventing
unhealthy computer habits. In CHI ’13 Extended Abstracts on Human
Factors in Computing Systems, CHI EA ’13, pages 487–492, New York,NY, USA, 2013. ACM.
Leon Straker, Kerry J Jones, and Jenni Miller. A comparison of the
postures assumed when using laptop computers and desktop computers.
Applied Ergonomics, 28(4):263–268, 1997.
Jens Wahlström. Ergonomics, musculoskeletal disorders and computer
work. Occupational Medicine, 55(3):168–176, 2005.