2019 IEEE 8th Global Conference on Consumer Electronics (GCCE)

HumTouch: Finger gesture recognition onhydrogel-painted paper using hum-driven signals
Kyoichi Tachi, Shogo Okamoto, Yasuhiro Akiyama, and Yoji Yamada
Department of Mechanical Systems Engineering, Nagoya University, Nagoya, Japan

200 mm

Ch. 1

Ch. 2yx

200 mmAbstract—We propose a passive touch sensor that can be implemented on semi-conductive materials, such as paper. This sensor
detects electric currents leaked from a human body when touchedwith bare fingers. Such electric currents are mostly produced byAC power lines in the environment. We tested the potential of thesensor for gesture recognition by using a paper sheet painted witha hydrogel ink. A gesture classification algorithm combining thek-means method and Mahalanobis’ generalized distance could
successfully classify 90% of the gestures drawn on the paperwith a fingertip. Gestures including multi-directional lines andcircles were nearly perfectly classified whereas digits, such as 2and 8 exhibited difficulties with classification. The hydrogel ink
can turn other non-conductive materials comprising woods andplastic into touch sensors, and such hum-based touch sensors are
expected to find practical applications in gesture recognition.
Index Terms—Hum noise, touch sensor, gesture

Ch. 3Ch. 4Fig. 1. Paper after being painted with hydrogel ink. Each of the four electrodes
was connected to an oscilloscope. Top right: Touch with a bare finger. Bottom
right: Dry and flexible semi-conductive paper.

I. I NTRODUCTION
Most commercially popular touch sensing devices detectcapacitive or resistive changes of surfaces when they are
touched by humans. Although these principles are used forspecific or designed surfaces, camera-based or force-sensingbased methods [1] are potentially used for turning any surface
into a touch sensing medium. Nonetheless, they require structured environments and camera-based interfaces suffer from
occlusion problems. Unlike these touch sensing techniques,
hum-based touch sensors [2], [3] can be implemented on semiconductive materials, such as stone and paper that commonly
exist in our living spaces without large modification of environments. When a human touches a semi-conductive surface, acurrent in the human body caused by the electromagnetic fieldof AC power lines in the environment flows on the surface. It isthen detected by an electrode placed on the surface. Hence, thissensor is a passive-type sensor and does not apply any energy
to object surfaces. In addition, by using multiple electrodes
placed on the surface, the touch position can be localized [4].
When using a hum-based touch sensor on a large area
surface, applications involving gesture recognition can berealized. However, localization of the contact point using ahum-based touch sensor is not precise due to the environmental
electromagnetic noises and non-uniformity of the material
properties of the surface. Therefore, it is necessary to evaluate the gesture identification abilities of a hum-based touch
sensor. In this study, we verified the feasibility of gesturerecognition using a paper and hum-touch sensing method.
This was achieved through the identification of one-stroke
This study was in part supported by the Mazda Foundation.

gestures that are frequently used for gesture interfaces by using
a conventional gesture classification algorithm.
II. M ATERIAL AND A PPARATUS
A. Paper with hydrogel ink
We used a paper towel of 20 × 20 cm that was soaked intoa polymer ink and dried. The formulation and constitution of
the ink were chosen according to [5]. The same ink can turnother non-conductive materials such as woods and plastics into
touch-sensitive surfaces. Although we used the paper in thepresent study, similar gesture performances are expected forwoods and plastics.
B. Signal measurement
We placed four electrodes, one at each corner of the paper
towel, as shown in Fig. 1, and measured the voltages detected using an oscilloscope. The measured voltage amplitudes
changed according to the distance between the finger andelectrodes [4] and the signal frequency was 60 Hz, whichis equal to that of the AC power supply. We refer to theseamplitude envelopes detected at the four electrode channels
as vi (t) (i = 1, 2, 3, 4), as shown in Fig. 2.
III. E XPERIMENT
We recorded the voltage outputs from the four electrodes
when seven types of gestures were drawn with a fingertip
on the paper. The gestures comprised circles (CW, CCW), a
vertical line from top to bottom, horizontal line from left toright, diagonal line from top left to bottom right, and Arabic
numerals: 2 and 8. If straight line gestures in one direction

157

978-1-7281-3575-5/19/$31.00 ©2019 IEEEPeak voltage (V)

3.6

Ch. 1v1(t)3.4

Ch. 2v2(t)Ch. 3v3(t)TABLE IC ORRECT GESTURE CLASSIFICATION RATES

Ch. 4v4(t)Type of gesture
Circle (CW)
Circle (CCW)
Vertical line
Horizontal line
Diagonal line
Number 2
Number 8Total

3.23
2.8
0.40

0Time (s)Fig. 2. Envelopes of 60-Hz voltages recorded at four electrodes on the paper
when a circle was drawn.

B. Algorithm and parameters used for classification
We used four parameters for classification of gestures:mean velocities along x and y axes and two types of sumsof differential arguments. The latter two parameters werecomputed by

0.04
0.02
y (-)

Recognition rates
18/20
19/20
19/20
19/19
19/19
15/19
14/20
123/13709∑∠(aj+1 − aj )(3)∠(|aj+1 − aj |)

(4)

-0.02
j=19∑

-0.04
-0.08

-0.04

0x (-)

0.04

0.08

j=1Fig. 3. Velocity vectors on the dimensionless and deformed Cartesian spacewhen a circle was drawn. Red curve is the trajectory of the circle.

can be identified, those in the opposite direction can be also
identified using the same algorithm. Hence, in order to avoid
an apparent increase in the classification rate, we measured
these gestures in only one direction. The most complex onestroke gestures tested for were Arabic numerals because more
complex gestures are unlikely to be used in practice. Eachgesture was drawn 20 times by using a bare finger in anoffice environment; however, a few trials were judged invalid
and were not used for the latter analysis. In these invalid
trials, the one-stroke gesture was failed and the finger was
not continuously in touch with the paper during drawing.
IV. G ESTURE CLASSIFICATION ALGORITHM
A. Calculation of velocity vectors of gesture
We converted vi (t) to dimensionless Cartesian coordinates
by using following equations:v2 + v3 − v1 − v4
Σviv1 + v2 − v3 − v4
y=.
Σvi
x=(1)
(2)

The trajectory of a gesture was then acquired, as shown inFig. 3. The red line is the trajectory in the dimensionlesscoordinate space when a circular gesture was drawn; however,as mentioned earlier, it is distorted due to the characteristicsof the hum-based touch sensor. We calculated velocity vectors
at each of the ten points that equally divide the trajectory interms of time intervals.

where ai is the i-th velocity vector among the ten vectors
computed for each gesture. The first equation sums up thedifference between two neighboring velocity vectors whereas
the second equation sums up the absolute difference.
We used the k-means method with Mahalanobis’ generalized distances to classify the gestures, each of which is definedby a combination of four-parameters. The centroid of eachof the seven types of gestures was allocated on the fourparameter space. Then, when a new gesture was acquired, it
was classified as the nearest gesture in terms of Mahalanobis’
generalized distance.
V. R ESULTS
Table I shows the ratios of correct gesture recognition by aleave-one-out cross validation. The mean correct recognition
rate was approximately 90%. The rates for Arabic digits (2and 8) were remarkably low.
Potential causes of miss-classification include distortionof the gesture trajectory on the dimensionless coordinates.
Especially when the finger touches near the electrode, thecoordinates computed using (1) and (2) nonlinearly correspondto the physical coordinates on the paper. Such problems canbe solved by application of a nonlinear coordinate transform
as a preprocessing before the use of linear classification
algorithms. In addition, we only used four types of parameters
for classification; a greater number of parameters would leadto more successful classification.
VI. C ONCLUSIONSIn this study, we verified the feasibility of gesture recognition using a hum-based touch sensing technique. Simple
gestures, such as lines and circles drawn on a hydrogel-painted
paper were classified at approximately 95% whereas digits of158

2 and 8 were classified at approximately 70%. By adopting
appropriate classification algorithms and feature parameters,
the hum-based touch sensing technique can be employed inpractical application.
R EFERENCES[1] T. Tsuji, N. Kurita, and S. Sakaino, “Development of a desk-type tactileinterface using force sensors,” in Proceedings of IEEE International
Conference on Industrial Electronics, Control and Instrumentation, 2014,
pp. 5248–5253.
[2] H. Elfekey, H. A. Bastawrous, and S. Okamoto, “A touch sensing
technique using the human body extremely low frequency fields,” Sensors,
vol. 16, no. 12, p. 2049, 2016.
[3] H. Elfekey and S. Okamoto, Turning surfaces into touch panels:
A Granite-touch pad, ser. Lecture Notes in Electrical Engineering.
Springer, 2016, vol. 432, pp. 301–304.
[4] H. Elfekey, S. Okamoto, and Y. Yamada, “Localization of touch on granite
based on ac hum noise,” in Proceedings of IEEE Global Conference onConsumer Electronics, 2017, pp. 1–2.
[5] K. Ramamohan, V. B. S. Achari, A. K. Sharma, and L. Xiuyang,“Electrical and structural characterization of PVA/PEG polymer blend
electrolyte films doped with NaClO4 ,” Ionics, vol. 21, pp. 1333–1340,
2015.

159