MagneTrack: Magnetic Field Separation Method for Continuousand Simultaneous 1-DOF Tracking of Two-magnets
Takehiro Abe

Daisuke Sakamoto

Independent
Kyoto, Kyoto, Japan

Hokkaido University
Sapporo, Hokkaido, Japan

Figure 1: Tracking two magnets individually using a common smartphone on straight lines.

ABSTRACTThis paper presents a method to track two magnets using one magnetometer on a smart device (e.g., smartphone), which detects thelocations of the magnets around the device. A magnetometer measures the overlapping data signals generated by the two magnets.
Thus, a data-separation technique is required to separately track thecontinuous and simultaneous signals generated by the two magnets.
In this study, we develop a separation algorithm based on nonnegative matrix factorization to continuously and simultaneously track
two magnets, and then conduct an experiment to validate our concept. We then compare our method in both simulated and actual
environments. In the actual environment, we observe real data signals arising from the two magnets on a magnetometer. Finally, we
present example applications to demonstrate the use cases of ourmodel in human–computer interaction systems.

CCS CONCEPTS• Human-centered computing → Smartphones; User interface design; Human computer interaction (HCI).

KEYWORDS
magnetic user interface, magnetic feld separation, algorithm, twomagnet tracking
Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor proft or commercial advantage and that copies bear this notice and the full citationon the frst page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specifc permission
and/or a fee. Request permissions from permissions@acm.org.
MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France© 2021 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-8328-8/21/09. . . $15.00
https://doi.org/10.1145/3447526.3472052

ACM Reference Format:
Takehiro Abe and Daisuke Sakamoto. 2021. MagneTrack: Magnetic Field
Separation Method for Continuous and Simultaneous 1-DOF Tracking of
Two-magnets. In 23rd International Conference on Mobile Human-Computer
Interaction (MobileHCI ’21), September 27-October 1, 2021, Toulouse & Virtual,
France. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3447526.
3472052

1

INTRODUCTION

Magnets are important materials used in human–computer interaction (HCI) research; they are small and cheap, but powerful enoughto demonstrate various user interactions. Researchers have used
this material to track fnger movements around a smartphone andsmartwatch [10, 15], tangible and physical interactions [24, 27], andmidair interaction [22]. In addition to interaction variations, there
are various confgurations of magnets and magnetometers, suchas interaction with magnetometer arrays [25], multiple tangible
objects with magnets [13], and spinning magnets [3]. In this paper,
we focus on the confguration of one magnetometer and two magnets, and propose a method to continuously and simultaneously
track the magnets’ positions along straight lines around a common
smart device (Fig. 1).
Continuous and simultaneous two-magnet tracking will greatly
enhance physical interactions on smart devices. The input rangeof the touchscreen used as an input interface for smart devices
depends on its size. The larger the screen, the larger the input
range; however, this reduces the device’s portability. In contrast,
the smaller the screen, the better the portability; however, thismakes touch interactions difcult. Therefore, an auxiliary input
widget, independent of the screen size, can be added to a smart
device to make it more user friendly. In particular, continuous input widgets, such as sliders and levers, are highly compatible with

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

a conventional swiping operation (e.g., screen scrolling); consequently, they are easier to use than discrete input components,
such as buttons. Furthermore, continuous input devices are well
suited to tangible user interfaces and game controllers, and canthus enhance the functionality of smart devices. One advantage ofusing magnets with smart devices is that many such devices havebuilt-in magnetometers; therefore, our concept of continuous andsimultaneous two-magnet tracking does not require any hardware
modifcation.
To develop a continuous and simultaneous two-magnet tracking
method, we employ a magnetic feld-separation approach. Somemachine-learning algorithms widely used in HCI, such as random
forest and support vector regression, are suitable candidates formagnet-tracking problems. However, these algorithms are usedwith either discontinuous positional relations or single continuous
magnet tracking. For magnetic feld separation, we use a supervised learning method based on nonnegative matrix factorization(NMF). NMF is often used for tasks involving both image and audio
signal processing [21, 31], and it is also efective for magnetic feld
separation, which has similar properties.
We formulate a method to separate the overlapping magnetic
felds from one magnetometer to track two magnets on straightlines. Our model tracks magnets on one-dimensional (1D) straightlines on two sides of the magnetometer. Although this method isonly 1D, it is the frst approach proposed to separate overlapping
magnetic felds using NMF. We develop an algorithm to realize magnetic feld separation and develop a prototype system to establish
our concept.
The contributions of our work are summarized as follows:(1) We develop an algorithm to realize continuous and simultaneous two-magnet tracking by separating the overlapping
magnetic felds. For this purpose, only a three-point calibration process is necessary; our method does not require muchdata for learning.
(2) We validate our method by comparing the results in both
simulated and actual environments. In the actual environment, we observe the data obtained from a magnetometerin response to the two magnets.
(3) We demonstrate example applications to illustrate the usecase of our technique.

2

RELATED WORK: MAGNETIC USERINTERFACE

Various magnetic user interfaces have been proposed to date forboth input to output. For output interfaces, due to the attraction andrepulsion properties of magnetic materials, some researchers have
explored the application of magnets to haptic feedback [5, 27, 30, 36–
38, 42]. In contrast, we focus on the application of magnets as input
interfaces that operate by continuously and simultaneously tracking
two magnets placed around a smart device.

2.1

Single-Magnet Tracking

Most modern smart devices are equipped with a magnetometer;
thus, many input interfaces using the magnetometer have been
proposed. Harrison and Hudson proposed an approach to start

Abe and Sakamoto

software apps by detecting the diference in the magnetic feld generated by a magnet attached to the user’s fnger [10]. Ketabdar etal. also proposed a gesture input system for smart devices usinga magnet, which was shown to be applicable to various software
apps [17, 18]. Kadomura and Siio proposed user interaction via
a magnet attached to a fngernail, which was intended to allow
users to operate smart devices naturally [15]. Zalmai et al. proposeda gesture-recognition method by ftting the trajectory model, assuming straight movement gestures [41]. Yoon et al. demonstrated
an interesting approach that combined a magnet with an inertial
measurement unit (IMU) for three-dimensional (3D) pen- and fngerbased interactions [39, 40]. McIntosh et al. used a magnet not only
for gesture input, but also for haptic feedback by placing a coilon a smart device [26]. They also utilized IMU to apply geomagnetism cancellation to improve the tracking accuracy. By usinga magnetic ring, Park et al. identifed which fnger was used on
the touchscreen [29]. They achieved high accuracy by applying
geomagnetism cancellation and further used the touched position
as a feature of machine learning.

2.2

Multimagnet Identifcation

Hwang and Bianchi proposed the use of tangible magnets to interact with smart devices [2, 13], and Hwang et al. demonstrated
a tangible interaction system with multiple tangible objects using multiple magnets [13]. However, their six-page paper did notpresent the system implementation adequately; hence, we wereunable to reconstruct it. Moreover, they neither conducted an experiment to demonstrate the system’s accuracy nor provided adequate
metrics to understand the system’s operation. Hwang et al. also
tried to detect the movement of a pair of magnets by using a decision tree [12]. However, this approach could only handle the static
positional relation of magnets, and not continuous position tracking. To resolve this issue, Bianchi and Oakley proposed a model torecognize multiple tangibles that spin a magnet based on the frequency component of the magnetic feld [4]. Although recognition
based on the frequency component can work robustly, even when
the number of tangibles increases, it requires special devices to spin
the magnet. Chan and Gollakota proposed an interaction method
using magnetic patterns generated from a conductive thread [6].
As the interference of each magnetic pattern is trivial, the problemof overlapping magnetic felds can be avoided.

2.3

Advanced Magnet Tracking Using Multiple
Magnetometers

Some researchers have increased the number of magnetometers inorder to detect the exact movement of a magnet. Chen et al. achieved
3D point tracking of a fngertip using two magnetometers, whichcannot be achieved using a single magnetometer [7]. Moreover,Chen et al. tracked fve fngers independently by increasing thenumber of magnetometers from two to four and using AC-driven
electromagnets operating at unique frequencies [8]. Similarly, Parizi
et al. embedded three magnetometers on a smartwatch and precisely
tracked an index fnger wearing an electromagnet as a ring [28].
Liang et al. and Kuo et al. recognized both the precise locationand type of magnet tangibles in a smart device in which a Hall
efect sensor grid was embedded [20, 23–25]. Even though a simple

MagneTrack: Continuous and Simultaneous 1-DOF Tracking of Two-magnets

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

algorithm can estimate the exact position of a magnet by increasing
the number of sensors, this approach requires a special device, suchas a magnetometer grid.

2.4

Continuous and Simultaneous Two-Magnet
Tracking: Our Contribution

Various studies were conducted on the use of magnets for userinterfaces in the feld of HCI. These studies demonstrated various
applications using single–, double–, or multiple–magnet tracking
and/or identifcation techniques. Among these, we focus on continuously and simultaneously tracking two magnets with one degreeof freedom. Similarly, MagGetz [13], demonstrated some example
applications of continuous and simultaneous magnet tracking using
a similar method to ours. Our method shows advantages over theirs
in terms of ease of calibration, ease of geomagnetic cancellation,and stability with respect to the relative position change of thesensor and magnet. To achieve facile calibration, we develop an
algorithm to separate the overlapping magnetic felds and track two
magnets using only three-point calibration. Similarly, we implemented geomagnetic cancellation so that the smart device can beused with two magnets while being able to rotate. This mechanism
contributes to the stable detection of both magnets.

3

MAGNETIC FIELD-SEPARATION METHOD

Owing to the additive property of magnetic felds, overlapped magnetic felds from multiple magnets can be separated using a linear
model. This property is also applicable to complex spectra, and thus
the methods used in sound source separation can be applied to magnetic feld separation by extending the two-dimensional spectrum
components (i.e., real and imaginary parts) to a 3D magnetic feld.
The complex NMF proposed by Kameoka [16] et al., which can
express a complex spectrum by using a compact base model, is asuitable candidate for magnetic feld separation. Another candidate
is that proposed by Ahuja et al., in which a complex spectrum isdivided into positive and negative components by NMF [1]. Our
method is based on both of these methods.
We developed a method to track the positions of two magnets
on straight lines using one magnetometer. The key concept forcontinuous and simultaneous two-magnet tracking is magnetic
feld separation. As shown in Fig. 2, in magnetic feld separation,
the input has nine values measured from IMU (i.e., accelerometer,gyroscope and magnetometer). Each of those three sensors outputs
three real numbers of (x, y and z). The output value is the intensity
ratio of each magnetic feld diference Hi . i is the index of themagnets. We use two magnets in this work, so i should be 0 and 1.
As a result, we get the two output values with our method. Prior to
the separation process, we registered the magnetic feld obtained(b)when all magnets are located at the reference position di and(e)when each magnet is located at the boundary position di forcalibration. By interpolating both positions as corner points,(b)

di = (1 − Hi )di(e)+ Hi di ,we estimated the position of each magnet di .

(1)

Figure 2: Overview of magnetic feld separation. (a) Our separation system is calibrated with two magnetic feld diferences between a magnetic feld observed when both magnets
are located at a reference position (i.e., case A) and those observed when each magnet is located at a boundary position(i.e., case B and case C). (b) Our geomagnetism cancellation
system is calibrated with the observed IMU data while thesmart device with magnets is swung from side to side, similar to drawing a fgure 8. (c) The output is an intensity ratiothat expresses where each magnet is located between the reference and the boundary position.

3.1

Magnetic Field Diference Model

If the 3-axis indices of the magnetic feld are defned as ϕ ∈ {x, y, z},then the diference model of the magnetic feld generated fromeach magnet can be expressed by the product of the preregisteredmagnetic feld diference X i,ϕ (basis vector) and the intensity ratioof each current magnetic feld Hi , as follows:
Mi,ϕ = Hi X i,ϕ ,(2)

where Hi satisfes the condition 0 ≤ Hi . Using the magnetic feld
diference model Mi,ϕ and the magnetic feld observed when all
magnets are located at the reference position Eϕ , the magnetic feld
measured from a magnetometer Bϕ is established as follows:ÕBϕ =Mi,ϕ + Eϕ + ϵϕ ,(3)iwhere ϵϕ is the error due to the model error and noise.

3.2

Geomagnetism Cancellation

Eϕ comprises the geomagnetic feld and magnetic feld generated(b)when both magnets are located at the reference position di . Whenthe orientation of the smart device is tilted, the positional relation between the two magnets and the magnetometer is constant,whereas the geomagnetic feld component rotates depending on
device orientation. Therefore, we apply a simple approach of geomagnetism cancellation, which is quite similar to that proposedMobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, Franceby [26]. Eϕ is expressed asÕ
Eϕ =ϕ ′ ∈ {x,y,z }Abe and Sakamoto

(G)
(H )
Rϕϕ ′ Eϕ + Eϕ ,(4)using the rotation matrix of the device orientation Rϕϕ ′ , which
can be obtained from its accelerometer and its gyroscope, the geomagnetic feld observed when the smart device is not tilted (i.e.,(G)absolute orientation) Eϕ , and the magnetic feld generated from
(H )

r

2

2

2

of a sphere with center Eϕ and radius E ϕ(G) (= Ex(G ) + Ey(G ) + Ez(G) ).
Therefore, we sampled the magnetic feld at various orientationsof the smart device with two magnets for calibration, and thenftted the sphere to the sampled data Bk,ϕ . Next, we applied a leastsquares method where the following was used as a cost function to(H )(G)obtain Eϕ and Eϕ :
J≡

ÕÕ
(H )
(G) 2
( (Bk,ϕ − Eϕ )2 − Eϕ )2 .
k

(5)

ϕ

We obtained the magnetic feld diference generated from thetwo magnets Yϕ by applying geomagnetism calibration with theobservation data:(6)
Yϕ = Bϕ − Eϕ .

3.3

Separation Process

Based on NMF, we separated the magnetic feld diference of twomagnets Yϕ into those of each magnet Mi,ϕ . For a real given magnetic feld diference, we separated the observation data and thebasis vector into positive and negative components applied to NMF.
(+)

Yϕ

= max(0, Yϕ ),(+)

X i,ϕ = max(0, X i,ϕ ),

(−)

Yϕ

= min(0, Yϕ )(7)

X i,ϕ = min(0, X i,ϕ )

(8)

(−)

Each component is assigned to the following NMF update formulas:
(+)

(+)

βi,ϕ = Í

Hi X i,ϕ(+)
n Hn X n,ϕ

(−)

,

(−)

βi,ϕ = Í

Hi X i,ϕ(−)
n Hn X n,ϕ

,

(+)
(+)
(+) (+)(+)
Yˆi,ϕ = Hi X i,ϕ + βi,ϕ (Yϕ − Hi X i,ϕ ),
(−)

(−)

(−)

(−)

4

EXPERIMENT

We conducted an experiment to validate our method by comparing
the results obtained in simulated and real environments. In the realenvironment, we obtained data from a magnetometer in responseto two magnets.

(H )the two magnets and an environment Eϕ . The variable Eϕ iswell known as the hard iron efect in the task of geomagnetism
cancellation [19, 33], which indicates an observed geomagnetic feld
distortion produced by a paramagnetic material, such as a permanent magnet. As shown in Eq. (4), Eϕ is distributed on the surface(H )of each magnetic feld Hi obtained through this iteration and theposition of each magnet di were derived from Eq. (1).

(−)

Ŷi,ϕ = Hi X i,ϕ + βi,ϕ (Yϕ − Hi X i,ϕ ),Í(+) (+) (+)(−) (−) (−)ϕ (Ŷi,ϕ X i,ϕ /βi,ϕ + Ŷi,ϕ X i,ϕ /βi,ϕ )Hi = Í
.
(+) 2 (+)(−) 2 (−)ϕ ((X i,ϕ ) /βi,ϕ + X i,ϕ ) /βi,ϕ )

(9)

(10)

(11)

Eq. (9) derives the component ratio of the magnetic feld generated
from each magnet βi,ϕ , Eq. (10) distributes the diference between
the observation data and the magnetic feld diference model tothe magnetic feld model with the component ratio, and Eq. (11)calculates the intensity ratio Hi from the values obtained in Eq. (9)and Eq. (10). By iteratively updating Eq. (9), (10), and (11), thesolution converges to a stationary point. Finally, the intensity ratio

4.1

Condition

We compared two conditions in this experiment: the simulated andactual environments.
4.1.1 Simulated Ideal Environment (Simulation). We tested our
method in a simulated magnetic feld that represented an ideal
environment without any noise. All parameters in this scenario
were fabricated by a Java program, and all trials were performed in
the simulated environment.
4.1.2 Actual Environment (Actual). In this environment, we tested
our method in a real magnetic feld, which was monitored using amagnetometer, which varied in response to the movements of twomagnets. All trials were performed with a magnetometer installed
on a smartphone and two magnets. We collected data using various
combinations of parameters in an actual environment, and thensupplied the collected data to be analyzed by our method.
The following sections mainly describe the experimental setup
and parameters in the actual environment. The simulated environment was created to mimic the actual environment.

4.2

Setup and Apparatus

This experiment employed two magnets, each of which comprised
four small magnets (13 mm in diameter and 2.5 mm thick with afux density of 180 mT by Seria Co., Ltd.). All four magnets werestacked to form a single cylindrical magnet. We used a Nexus 5X
smartphone with an installed magnetometer as a data-recording
device. The exact position of the magnetometer on board was notofcially available, so we exploratively searched for its position. The
yellow circle on the smartphone in Fig. 3 indicates the estimatedposition of the sensor. The error margin was approximately ±1 mm.
Fig. 3 shows the experimental data-collection setup, where the
Nexus 5X is located in the center as the data-recording device. We
adjusted its position to align the magnetometer with the center ofthe experimental setup. The two magnets were designed to movealong straight lines on the left and right sides of the recording device.
The experimental parameters are discussed in the next subsection.

4.3

Parameters4.3.1 Direction of the Magnetic Field. A magnet creates a directional magnetic feld around itself comprising three spatial dimensions (i.e., x, y and z) (Fig. 4). In this experiment, we used threevariations of the parameters to determine the direction of the magnetic feld. There are two lines on the z plane in the experimental
setup, on the left and right sides of the device. The same direction
is used for the two magnets; thus, there are nine variations of theparameters: lx − rx (left-x and right-x), lx − ry, lx − rz, ly − rx,MagneTrack: Continuous and Simultaneous 1-DOF Tracking of Two-magnets

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France(1) SensorMIDDLE: A magnetometer is located at the middle ofthe magnets’ movement range. SensorMIDDLE is a condition in which the distance between the magnet and magnetometer is equal when the magnet is placed at the referenceor boundary position.
(2) SensorTOP: A magnetometer is located at the top of the magnets’ movement range. SensorTOP is a condition in which
the distance between the magnet and magnetometer simply
decreases as the magnet moves from the reference position
to the boundary position.

Figure 3: Environmental(Sensor MI DDLE)

setup

for

data

collection.

ly − ly, ly − rz, lz − rx, lz − ry and lz − rz. We maintained this pairof straight lines as the two magnets moved along them.
We synthesized the magnetic feld created by the two magnets
and analyzed it using a magnetometer by employing the Biot–Savart
law.


mϕ
µ 0 3⟨m, r⟩r ϕ
Bϕ =
−
(12)
4π
∥r∥ 5
∥r∥ 3

We selected those two conditions to investigate how the line andmagnetometer positions afect the tracking results. The Sensor MI DDLE
condition is a balanced case where the distances between the sensor and both the boundary and reference positions are equal, andSensorTOP condition is an unbalanced case. These conditions are
considered because the distances between the magnets and thesensor are important factors in this experiment. Our hypothesis isthat the unbalanced case will more robustly track magnets thanthe balanced case because the position is not uniquely determinedwhen the distance is symmetrical. Note that we only select one
condition for the unbalanced case, and omit the corresponding SensorBOTTOM case because the SensorTOP and SensorBOTTOM aresymmetric in terms of the distance between the magnets and sensor.
Thus, their results should be the same.

SensorMIDDLE.

SensorTOP.

Figure 5: Positional conditions of magnetometer.

Figure 4: Direction of magnet.

4.3.2 Position of the Magnets. In this experiment, two magnets
moved along straight lines. A 100-mm line was divided into 10
intervals to create 11 data points. All intervals were 10 mm wideand equally spaced (Fig. 3). We collected all combinations of left
and right data points on two lines and arrived at 11 × 11 = 121 data
points for each pair of magnet directions.
4.3.3 Position of the Magnetometer. Not only the direction of eachmagnet, but also the reference and the boundary positions determine the basis vector. As shown in Fig. 5, we conducted this experiment for the two positional conditions of the magnetometer
described below.

4.4

Procedure

The experimental procedure is as follows:(1) The magnetometer location was selected based on the positional conditions SensorMIDDLE and SensorTOP.
(2) A pair of directions was selected for the two magnets.
(3) The device was calibrated by frst placing the two magnets
on the reference positions, and then recording the positions
on the device. Second, the left magnet was moved to theboundary position on the left line and the two magnet positions were recorded. Third, the left magnet was returned to
the reference position on the left line, the right magnet was
moved to the boundary position on the right line, and thetwo magnet positions were recorded.
(4) Data collection commenced from the reference positions. We
collected 11 × 11 = 121 data points by fxing the directionsof the magnets while moving them along the straight lines.

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France(5) Steps 1–4 of the procedure were repeated until all two positional conditions and all nine pairs of direction of the magnets were completed. Note that the order of selection in thesesteps did not afect the experimental result.
We performed the above procedure for data collection in a realenvironmental setup. Data collection was completed in four hours.
In contrast, the magnet position tracking was conducted within, at
most, 10 ms.

4.5

Abe and Sakamoto

Figure 7: Visualization method for the collected data.

Results and Discussions

4.5.1 Results. Fig. 6 shows the results of the experiments conducted in the simulated and actual environments. Two results wereobtained for each dataset because there are two sets of magnets
and lines. The orange color represents the results of the left side of
the magnet, and the navy color represents the results of the rightside of the magnet. The color strength signifes the proximity ofthe magnet on the line to the reference position, where white represents 0 mm from the reference position and the full color represents
100 mm from the reference position. The results are represented asa matrix, which shows all combinations of the positions of the twomagnets (Fig. 7).
Fig. 8 shows the mean absolute errors between the two resultsfrom the simulated and real environments. The error indicates thediference between the simulated and actual positions; the lower
the error, the better the results. The minimum error is zero. In
Sensor MI DDLE, there are two pairs that show a small error in boththe left and right lines: lx −ry and ly −rx. In contrast, in SensorTOP,
all pairs show smaller errors along both the left and right lines
compared to those of the SensorMIDDLE condition. These results
indicate that our method successfully calculates the actual positions from the observation of a magnetometer, depending on the
combination of the magnet direction and positional conditions.

SensorMIDDLE.

SensorTOP.
Figure 6: Results of the simulated (top) and actual (bottom)
environments

4.5.2 Discussion. Here, we briefy discuss the reason for the diferences between the SensorMIDDLE and SensorTOP conditions. The
key diference arises from the relationship between each magnet
and the magnetometer. Fig. 9 shows the diferences in the magnetic
feld when each magnet is moving along a straight line in the simulated environment. As we can see from the two basis vectors of thez direction in SensorMIDDLE (Fig. 9 left side), all the components
of magnetic feld at the boundary position are zero; therefore, thebasis vector X i,ϕ is zero. This is because the distances between themagnetometer and both the reference and boundary positions are
the same in the case of SensorMIDDLE. The relative position froma magnet to a magnetometer on the z-plane is r = (x 0 , y0 , 0), andthe moment of a magnet which is directed to z is m = (0, 0, m 0 );thus, the inner product of the two vectors is zero: ⟨m, r⟩ = 0. Subm
stituting this zero value into Eq. (12) yields Bϕ = − ∥r ϕ∥ 3 , which only
includes the moment of the magnet and the distance between themagnet and magnetometer. Because the magnet moment is always
constant, the magnetic feld diference at the reference positionis equal to that at the boundary position (i.e., Yϕ = 0 when eachmagnet is placed at either of the two positions). In contrast, thedistance between the magnet and the magnetometer is not the sameat every point between the reference and boundary positions in thecase of SensorTOP; therefore, the basis vector X i,ϕ is not zero, andthe algorithm more accurately separates the magnetic feld than in
the case of SensorMIDDLE.
In addition to the case of the z direction in SensorMI DDLE, when
two magnets point to the same direction (i.e., lx − rx, ly − ry andlz − rz), our method could not track each magnet at all. This is because the observed magnetic feld diference Yϕ cannot be uniquely
recombined from the basis vector X i,ϕ . As shown in “direction: x”
of Fig. 9, the basis vectors of two magnets in SensorMI DDLE are
X l ef t,ϕ = (0, By , 0) and X r iдht,ϕ = (0, −By , 0), respectively. Thus,
the basis vectors are parallel. For example, if a magnetic feld difference Yϕ = (0, By , 0) is observed, this can induce a nonuniqueintensity ratio: (Hl ef t , Hr iдht ) = (1.0+α, α). This is also applicable
to “direction: y” and “direction: z” (note that frst of all, the basis
vector of the “direction: z” becomes zero).
From the above discussion, we conclude that the key to accurate tracking is ensuring that the basis vectors are asymmetric.
As can be seen from the two basis vectors of the x direction in
SensorTOP (Fig. 9 right side), the basis vectors need not be fully
orthogonal to achieve accurate tracking. This is because the error oflx − rx in SensorTOP is not high, even though there is only a smalldiference in the y component between these basis vectors. This isMagneTrack: Continuous and Simultaneous 1-DOF Tracking of Two-magnets

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

Figure 8: Average and standard deviation (SD) of the errors.

Case of moving left magnet for SensorMIDDLE.

Case of moving left magnet for SensorTOP.

Case of moving right magnet for SensorMIDDLE.

Case of moving right magnet for SensorTOP.

Figure 9: Magnetic feld diference simulated while each magnet is moved between the reference and boundary positions.

likely to be satisfed in SensorTOP based on these results. In addition, as mentioned above, the distance between the magnetometerand a magnet in SensorTOP simply decreases in response to themagnet’s movement. Therefore, SensorTOP is a position condition
where the magnetic feld changes more linearly with respect tomagnet movement than in SensorMIDDLE. NMF is a model thatassumes linearity, and SensorTOP also meets this assumption better than Sensor MI DDLE. Therefore, we recommend placing thesensor in an unbalanced location in terms of the distance between
the magnets and the sensor. Theoretically, a sensor can be placed at
SensorRIGHT or SensorLEFT, because the position is geometricallysame as SensorTOP in this setup. If the positional relationship between the magnets and sensor becomes more unbalanced, then thesystem can robustly and accurately track the magnets.
It has been demonstrated clearly that our algorithm performs
well in the continuous and simultaneous tracking of two magnets
in SensorTOP, regardless of the combination of magnet directions,but not in Sensor MIDDLE, except in the case where the combination of magnet directions is x and y. The proposed magnetic
feld-separation method was the frst algorithm to track two magnets in this manner; we implemented and demonstrated its viability
for the HCI applications in this study. Previous studies on magnet
tracking for user interface applications (e.g., [13]) did not aim toevaluate how their systems work. In contrast, our work is the frst
attempt to evaluate the continuous and simultaneous two magnet
tracking algorithm.

5

EXAMPLE APPLICATIONS

We created several example applications to examine the efectiveness of our method. Our algorithm tracks two magnets on twolines with a magnetometer, which has three axes (x, y and z). Our
experiment uses the x and y axes on the z plane, but the result
is not limited to them. We can also use this method with otheraxes (i.e., x and z or y and z) via simple calibration. Similarly, ouralgorithm works if the magnets move in two orthogonal lines; thetwo magnets do not necessarily need to be on the same plane. The
system will separate the magnetic feld for two orthogonal lines, aswell as for lines on the same planar surface. The examples discussed
below use various axes.

5.1

Smartphone Case with Input AssistanceOur method can be applied to a smartphone case that has mechanical widgets, such as that presented in [34]. We created a simple
web browsing app as a use case. As shown in Fig. 10, the smartphone case has two physical sliders: one that can move verticallyand another that can move horizontally. The user is expected toMobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

Abe and Sakamoto

Figure 13: Archery game.

Figure 10: Web browsing app: (a) the smartphone case has a
top slider and a side slider that each include a magnet unit,(b) the back of a smartphone with the case, (c) pulling the top
slider, (d) scrolling a website down by pulling the top slider,(e) scrolling a website up by pressing the side slider, and (f)zooming a website by pulling the top slider and pressing theside slider simultaneously.

magnets, this smartphone case was made entirely of a 3D-printed
flament. When the vertical slider is pulled, the app screen scrolls
down. When the horizontal slider is pushed, the app screen scrolls
up. Furthermore, when both sliders are triggered, the app screen
zooms in to the center. Although these operations are also feasible
through gestures on a touch screen, such as swiping and pinching,
the smartphone case can adjust the scrolling speed continuously
without occlusion.

5.2

Figure 11: Music and dance game (©SAN-X CO., LTD).

5.3

Figure 12: Ambient light control: (a) the smartphone standhas a top slider and a right slider that each includes a magnet unit, (b) moving the right slider down to make the illuminance dark, (c) moving the top slider to the left side to
make the ambient color blue, (d) moving the right slider tothe bottom side to turn the ambient light of.

manipulate these sliders with the index fnger and thumb, respectively. Each slider has a 3D-printed spring that is similar to themechanism proposed in [35]. In essence, except for the attachedGame Controller: Stufed Toy Arms

We created an example application that detects the vertical positionof a stufed toy’s arms (Fig. 11), with which the users can manipulate
two objects on the screen. The arms can regulate the input fromthe users through vertical movements along straight lines; this is
an intuitive example of using continuous and simultaneous twomagnet tracking in practical usage. Beyond the simple manipulationof a stufed toy, we also created a music and dance app in which auser makes the stufed toy dance. Here, musical notes are displayed
based on the movement of the toy’s arms. Our method can the
track continuous positions of the hands of the stufed toy, thus
allowing a user to input two movements more easily than by using
a conventional game controller. It is also possible to imagine various
other applications for games, such as fghting virtual enemies andmoving virtual objects by manipulating a real stufed toy.

Remote for Ambient Light Control

To demonstrate more accurate tracking, we created another example application for controlling ambient light (Fig. 12). A smartphone
stand is placed with two straight lines around it, and on which twomagnets are placed. One magnet moves up and down along thevertical line, and the other moves left and right, like a slider, along
the horizontal line. This is an orthogonal setup. Once a smartphone
is set on the stand, the user frst calibrates the magnets. Then, theuser can control the color and illuminance level by moving themagnets along the straight lines. Our algorithm accurately tracks
the position of the magnet so that the user can control the colorand illuminance level intuitively. The advantage of this method isthat the user does not need to calibrate the sensor frequently, as the
positional relationship between the magnets and the sensor does
not change frequently in everyday use.

MagneTrack: Continuous and Simultaneous 1-DOF Tracking of Two-magnets

5.4

Game Controller: Archery

Our method can be applied not only to a stufed toy, but also to any
object that allows the magnet position to be changed. As shownin Fig. 13, we created an example application in which a user canmake a game character move and shoot an arrow in a virtual world
corresponding to input from a bow controller. This bow controller
has a grip to move one magnet to the front and a structure to moveanother magnet to the back by drawing the installed string. Whena user does not exert a force on the controller, these magnets returnto their original location via the movement of a spring and rubber
thread. The virtual character’s movement speed (i.e., walking, running) corresponds to the user’s grip force, and the arrow action (e.g.,fying distance or launch cancellation) corresponds to the user’s
draw force. In addition, the perspective in the virtual world canbe controlled by the controller’s tilt angle, which is measured by agyroscope.

6

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

distortion by updating the premeasured geomagnetic feld constantly when no magnets are nearby [29], the model of the geomagnetic feld is essentially spherical, similar to the case of ourmodel. In contrast, it is known that the geomagnetic feld distribution in (x, y and z) can become an ellipsoid due to distortion
arising from the soft iron efect from ferromagnetic materials, suchas steel [19, 33]. In our preliminary experiments, we also modeled
the geomagnetic feld as an ellipsoid, but the tracking accuracy did
not improve signifcantly, whereas the increased parameters (e.g.,
three elliptic radii) increased the overftting. Therefore, we adopted
the simpler method proposed by [26, 28]. In robotics, to improve
the accuracy of a robot’s position recognition, machine learningbased methods, such as the Bayesian nonparametric model [32],neural networks, and support vector regression [9], have been proposed to better model the distribution. The complexity of thesemodels makes them unsuitable for input user interfaces because
they require considerable data input for calibration (i.e., training).

DISCUSSION

We summarize the experiment as follows:• We confrmed that our method can stably track two magnets
that move along straight lines on the z plane continuously
and simultaneously in the case where the combination ofmagnet directions is x and y (i.e., lx − ry or ly − rx) in the
Sensor MI DDLE case.
• When the magnetometer is located outside the range ofmagnet movement (i.e., SensorTOP in our experiment), theaccuracy of tracking remains stable, regardless of the combination of magnet directions.
Here, we briefy discuss the geomagnetism cancellation andcalibration process used, and the limitations of our method.

6.1

Geomagnetism Cancellation

The purpose of geomagnetism cancellation is to mitigate the infuence of a geomagnetic feld. Recently, geomagnetism cancellation
has become a standard technique in HCI-related research felds,
and our method adopts a simple method that is equivalent to thosein [26, 28, 29]. Thus, the tracking accuracy is preserved, even user
are likely to tilt their smart devices. This means that geomagnetism
cancellation contributes to the practical application of continuous
two-magnet tracking. In conventional research [13], geomagnetism
cancellation was not considered because the placement of the smartphone was assumed to be fxed. However, we assume that users
will tilt and swing their smart devices, even while our method isbeing applied, so the simple method of geomagnetism cancellation,
which has been deemed efective in [26], is integrated into our
method. Although there was no detailed procedure was provided
for geomagnetism cancellation calibration, we used a well-known
calibration procedure used on Google Maps: sampling IMU data
while the smart device is swung from side to side, similar to drawing
a fgure 8.
In geomagnetism cancellation, we and [26, 28] assume that there
is no distortion in the observed geomagnetic feld, and simply rotate the premeasured geomagnetic feld according to the tilt of thesmart device. Thus, the distribution of geomagnetism in (x, y and z)
is assumed to be spherical. Although Park et al. account for the

6.2

Calibration

We used a simple three-point calibration technique in order todefne the two straight line segments where each magnet moves
within the tracking range. Such calibration is required, even in
most other magnet-tracking methods. For the separation process,calibration is essential to accurately track a magnet before usinga system. It is used not only for two-magnet tracking, but also forsingle-magnet tracking. Frequent calibration is costly for users. The
total number of registrations in our method is only three: one casewhere both magnets are located at the reference positions, andtwo cases where each magnet is located at a boundary position.
This number of registrations is the same as that used in otherconventional research [13].
As shown in Fig. 2, the calibration of our method consists of two
mutually independent parts: one for the separation process andthe other for geomagnetism cancellation. That means there is noneed to calibrate the separation process again when geomagnetism
cancellation is recalibrated, and vice versa. The separation process
requires recalibration when either the range of magnet movement,
magnet orientation, or type of magnet to be tracked is changed.
For example, a user needs to recalibrate the separation process
when they change their application from a game controller (Section
5.2) to a smartphone case (Section 5.1). Because the observed geomagnetism is susceptible to building materials (e.g., ferromagnetic
materials) [32], the geomagnetism cancellation is recommended tobe recalibrated when changing rooms.
The deep neural network (DNN) is a major option for the sourceseparation task that has been researched previously [11, 14]. DNN
is a more widely used technique than NMF; however, it requires
a considerable amount of training data to track the magnets accurately and stably. In contrast, our source-separation algorithm
based on the NMF requires only three data for continuous andsimultaneous magnet tracking, which is signifcantly less than that
required by the DNN. This simplifes the calibration process, whichis also better for the end-users. We consider that the calibration
process of our technique is easy enough for practical use.

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

6.3

Limitations

Observing the magnetic feld generated from magnets far awayfrom a sensor is difcult because the intensity of a magnetic feld
is inversely proportional to the square of the distance. A strong
magnet can help reduce this limitation; however, it is difcult touse due to safety concerns, as strong magnets can break electric
appliances. Therefore, it is difcult to apply our method to largescale, full-body interactions. Moreover, this limitation does not only
apply to our algorithm and method, but also to all models using
magnets in general. Similarly, our method cannot track magnets
if they stray from the line between the reference and boundary
positions.
Our model tracks two magnets along two straight lines with 1 degree of freedom (DoF), which is simply a theoretical constraint thatis the same as in previous research conducted in the feld of HCI.
As the value measured from the magnetometer of a smart device
comprises three continuous values (i.e., 3 axes), a single magnetometer can only identify the status of a magnet with less than three
DoFs. In theory, tracking three magnets on straight lines seems to
be feasible; however, it requires a rigid positional condition to befound where each basis vector of the three magnets is nonzero anddiferent in value. We and previous researchers [4, 13], who haveproposed models to identify or track multiple magnets using only asingle magnetometer, are subjected to this limitation. Therefore, in
multimagnet tracking using a common smart device, the capabilityof each magnet-tracking task should be 1 DoF. Regardless of thislimitation, Hwang et al. demonstrated many example applications
using the combination of either one slider, one rotation knob, or
two buttons [13]. Additionally, we demonstrate three example applications of continuous two-magnet tracking using two sliders or
two game controller levers. Similarly, our method can reproduce
all the example applications presented in [13] by simply adding athreshold for the tracking result obtained as continuous values.
Because our method uses one magnetometer to track two magnets, the efect when a magnet strays from a straight line is signifcant; therefore, it afects the tracking results of not only themagnet that the user is moving, but also the other. For example, inthe example using a stufed toy’s arms (Fig. 11), when the left armof the toy strays from a straight line in an unintended direction, thesystem still assumes that the magnet on the line. As a result, thesystem misunderstands the position of both the left and right sides.
This suggests that the position of the left magnet afects the positionof the right magnet. Hence, our method requires either a physicalmechanism to keep the movement of each magnet on a straightline or a design that guides the user to move each magnet along a
straight line. A magnetometer array, as proposed in [24, 25], canalso track each magnet moving along a straight line. This method
is capable of tracking a magnet moving on a plane as well as on astraight line, but it is also superior in terms of tracking each magnet
independently; therefore, if a magnet moves away from a straight
line, the other magnet is not afected.
Thus, our work has a limitation in that the system may misunderstand the position of both magnets because we use a single
sensor for our method, which is otherwise an advantage and a
key contribution of our work. It is currently difcult to install twomagnetometer sensors on a smartphone. Although our method

Abe and Sakamoto

requires large constraints to use, the algorithm used in our method
continuously and simultaneously tracks two magnets stably while
they remain on the straight lines. Our example applications demonstrated the possibility of utilizing our single-magnet method, andwe believe that our work provides new insights into magnetic HCI
research.

7

CONCLUSION

We presented a method to continuously and simultaneously track
the positions of two magnets around a common smart device. The
key concept underlying our method is using NMF to separate theoverlapping magnetic felds observed by the magnetometer on asmart device. We conducted an experiment to validate our methodby comparing a simulated ideal environment and an actual environment. The two pairs of magnet directions were successfully separated, and their positions were accurately tracked by our method
in an actual environment with a smart device.

REFERENCES[1] Chaitanya Ahuja, Karan Nathwani, and Rajesh M. Hegde. 2014. A Complex
Matrix Factorization approach to Joint Modeling of Magnitude and Phase forSource Separation. CoRR abs/1411.6741 (2014). arXiv:1411.6741 http://arxiv.org/
abs/1411.6741[2] Andrea Bianchi and Ian Oakley. 2013. Designing Tangible Magnetic Appcessories.
In Proceedings of the 7th International Conference on Tangible, Embedded and
Embodied Interaction (Barcelona, Spain) (TEI ’13). ACM, New York, NY, USA,
255–258. https://doi.org/10.1145/2460625.2460667[3] Andrea Bianchi and Ian Oakley. 2015. MagnID: Tracking Multiple Magnetic
Tokens. In Proceedings of the Ninth International Conference on Tangible, Embedded,and Embodied Interaction (Stanford, California, USA) (TEI ’15). ACM, New York,NY, USA, 61–68. https://doi.org/10.1145/2677199.2680582[4] Andrea Bianchi and Ian Oakley. 2015. MagnID: Tracking Multiple Magnetic
Tokens. In Proceedings of the Ninth International Conference on Tangible, Embedded,and Embodied Interaction (Stanford, California, USA) (TEI ’15). ACM, New York,NY, USA, 61–68. https://doi.org/10.1145/2677199.2680582[5] Roger Boldu, Sambhav Jain, Juan Pablo Forero Cortes, Haimo Zhang, and Suranga
Nanayakkara. 2019. M-Hair: Creating Novel Tactile Feedback by Augmenting the
Body Hair to Respond to Magnetic Field. In Proceedings of the 32nd Annual ACM
Symposium on User Interface Software and Technology (New Orleans, LA, USA)(UIST ’19). Association for Computing Machinery, New York, NY, USA, 323–328.
https://doi.org/10.1145/3332165.3347955[6] Justin Chan and Shyamnath Gollakota. 2017. Data Storage and Interaction Using
Magnetized Fabric. In Proceedings of the 30th Annual ACM Symposium on User
Interface Software and Technology (Qu&#233;bec City, QC, Canada) (UIST ’17).
ACM, New York, NY, USA, 655–663. https://doi.org/10.1145/3126594.3126620[7] Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3DInput Using Two Magnetic Sensors. In Proceedings of the 26th Annual ACM
Symposium on User Interface Software and Technology (St. Andrews, Scotland,
United Kingdom) (UIST ’13). ACM, New York, NY, USA, 237–244. https://doi.
org/10.1145/2501988.2502035[8] Ke-Yu Chen, Shwetak N. Patel, and Sean Keller. 2016. Finexus: Tracking Precise
Motions of Multiple Fingertips Using Magnetic Sensing. In Proceedings of the 2016
CHI Conference on Human Factors in Computing Systems (San Jose, California,
USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA,
1504–1514. https://doi.org/10.1145/2858036.2858125[9] L. Christensen, M. Krell, and F. Kirchner. 2017. Learning magnetic feld distortioncompensation for robotic systems. In 2017 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS). 3516–3521.
[10] Chris Harrison and Scott E. Hudson. 2009. Abracadabra: Wireless, High-precision,and Unpowered Finger Input for Very Small Mobile Devices. In Proceedings ofthe 22Nd Annual ACM Symposium on User Interface Software and Technology(Victoria, BC, Canada) (UIST ’09). ACM, New York, NY, USA, 121–124. https://doi.org/10.1145/1622176.1622199[11] Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, and Paris Smaragdis.
2014. Deep learning for monaural speech separation. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1562–1566.
https://doi.org/10.1109/ICASSP.2014.6853860[12] Sungjae Hwang, Myungwook Ahn, and Kwangyun Wohn. 2013. Magnetic Marionette: Magnetically Driven Elastic Controller on Mobile Device. In Proceedingsof the Companion Publication of the 2013 International Conference on Intelligent

MagneTrack: Continuous and Simultaneous 1-DOF Tracking of Two-magnets

User Interfaces Companion (Santa Monica, California, USA) (IUI ’13 Companion).
ACM, New York, NY, USA, 75–76. https://doi.org/10.1145/2451176.2451207[13] Sungjae Hwang, Myungwook Ahn, and Kwang-yun Wohn. 2013. MagGetz:
Customizable Passive Tangible Controllers on and Around Conventional Mobile
Devices. In Proceedings of the 26th Annual ACM Symposium on User Interface
Software and Technology (St. Andrews, Scotland, United Kingdom) (UIST ’13).
ACM, New York, NY, USA, 411–416. https://doi.org/10.1145/2501988.2501991[14] Andreas Jansson, Eric J. Humphrey, Nicola Montecchio, Rachel M. Bittner, Aparna
Kumar, and Tillman Weyde. 2017. Singing Voice Separation with Deep U-Net
Convolutional Networks.. In Proceedings of the 18th International Society forMusic Information Retrieval Conference. ISMIR, Suzhou, China, 745–751. https://doi.org/10.5281/zenodo.1414934[15] Azusa Kadomura and Itiro Siio. 2015. MagNail: User Interaction with Smart
Device Through Magnet Attached to Fingernail. In Adjunct Proceedings of the2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing
and Proceedings of the 2015 ACM International Symposium on Wearable Computers(Osaka, Japan) (UbiComp/ISWC’15 Adjunct). ACM, New York, NY, USA, 309–312.
https://doi.org/10.1145/2800835.2800859[16] Hirokazu Kameoka, Nobutaka Ono, Kunio Kashino, and Shigeki Sagayama. 2009.
Complex NMF: A new sparse representation for acoustic signals. In 2009 IEEE
International Conference on Acoustics, Speech and Signal Processing. 3437–3440.
https://doi.org/10.1109/ICASSP.2009.4960364[17] Hamed Ketabdar, Amirhossein Jahanbekam, Kamer Ali Yuksel, Tobias Hirsch, andAmin Haji Abolhassani. 2011. MagiMusic: Using Embedded Compass (Magnetic)
Sensor for Touch-less Gesture Based Interaction with Digital Music Instruments
in Mobile Devices. In Proceedings of the Fifth International Conference on Tangible,
Embedded, and Embodied Interaction (Funchal, Portugal) (TEI ’11). ACM, New
York, NY, USA, 241–244. https://doi.org/10.1145/1935701.1935749[18] Hamed Ketabdar, Kamer Ali Yüksel, and Mehran Roshandel. 2010. MagiTact: Interaction with Mobile Devices Based on Compass (Magnetic) Sensor.
In Proceedings of the 15th International Conference on Intelligent User Interfaces(Hong Kong, China) (IUI ’10). ACM, New York, NY, USA, 413–414. https://doi.org/10.1145/1719970.1720048[19] M. Kok and T. B. Schön. 2016. Magnetometer Calibration Using Inertial Sensors.
IEEE Sensors Journal 16, 14 (2016), 5679–5689.
[20] Han-Chih Kuo, Rong-Hao Liang, Long-Fei Lin, and Bing-Yu Chen. 2016. GaussMarbles: Spherical Magnetic Tangibles for Interacting with Portable Physical Constraints. In Proceedings of the 2016 CHI Conference on Human Factors in Computing
Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 4228–4232. https://doi.org/10.1145/2858036.2858559[21] Daniel D. Lee and H. Sebastian Seung. 1999. Learning the parts of objects bynonnegative matrix factorization. Nature 401 (1999), 788–791.
[22] Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN: Mid-air Tangible Interaction
Enabled by Computer Controlled Magnetic Levitation. In Proceedings of the24th Annual ACM Symposium on User Interface Software and Technology (Santa
Barbara, California, USA) (UIST ’11). ACM, New York, NY, USA, 327–336. https://doi.org/10.1145/2047196.2047239[23] Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang,
De-Nian Yang, and Bing-Yu Chen. 2014. GaussBricks: Magnetic Building Blocks
for Constructive Tangible Interactions on Portable Displays. ACM SIGCHI
Conference on Human Factors in Computing Systems (ACM CHI 2014) (04 2014).
https://doi.org/10.1145/2556288.2557105[24] Rong-Hao Liang, Kai-Yin Cheng, Liwei Chan, Chuan-Xhyuan Peng, Mike Y.
Chen, Rung-Huei Liang, De-Nian Yang, and Bing-Yu Chen. 2013. GaussBits:
Magnetic Tangible Bits for Portable and Occlusion-free Near-surface Interactions.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(Paris, France) (CHI ’13). ACM, New York, NY, USA, 1391–1400. https://doi.org/
10.1145/2470654.2466185[25] Rong-Hao Liang, Kai-Yin Cheng, Chao-Huai Su, Chien-Ting Weng, Bing-Yu Chen,
and De-Nian Yang. 2012. GaussSense: Attachable Stylus Sensing Using Magnetic
Sensor Grid. In Proceedings of the 25th Annual ACM Symposium on User Interface
Software and Technology (Cambridge, Massachusetts, USA) (UIST ’12). ACM, New
York, NY, USA, 319–326. https://doi.org/10.1145/2380116.2380157[26] Jess McIntosh, Paul Strohmeier, Jarrod Knibbe, Sebastian Boring, and Kasper
Hornbæk. 2019. Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI
’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300638[27] Masa Ogata. 2018. Magneto-Haptics: Embedding Magnetic Force Feedback forPhysical Interactions. In Proceedings of the 31st Annual ACM Symposium on User
Interface Software and Technology (Berlin, Germany) (UIST ’18). ACM, New York,NY, USA, 737–743. https://doi.org/10.1145/3242587.3242615[28] Farshid Salemi Parizi, Eric Whitmire, and Shwetak Patel. 2019. AuraRing: Precise
Electromagnetic Finger Tracking. Proc. ACM Interact. Mob. Wearable UbiquitousTechnol. 3, 4, Article 150 (Dec. 2019), 28 pages. https://doi.org/10.1145/3369831[29] Keunwoo Park, Daehwa Kim, Seongkook Heo, and Geehyuk Lee. 2020. MagTouch:
Robust Finger Identifcation for a Smartwatch Using a Magnet Ring and a Built-in

MobileHCI ’21, September 27-October 1, 2021, Toulouse & Virtual, France

Magnetometer. In Proceedings of the 2020 CHI Conference on Human Factors in
Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing
Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376234[30] Fabrizio Pece, Juan Jose Zarate, Velko Vechev, Nadine Besse, Olexandr Gudozhnik,
Herbert Shea, and Otmar Hilliges. 2017. MagTics: Flexible and Thin Form Factor
Magnetic Actuators for Dynamic and Wearable Haptic Feedback. In Proceedingsof the 30th Annual ACM Symposium on User Interface Software and Technology(Québec City, QC, Canada) (UIST ’17). Association for Computing Machinery,
New York, NY, USA, 143–154. https://doi.org/10.1145/3126594.3126609[31] P. Smaragdis and J. C. Brown. 2003. Non-negative matrix factorization for
polyphonic music transcription. In 2003 IEEE Workshop on Applications of Signal
Processing to Audio and Acoustics (IEEE Cat. No.03TH8684). 177–180. https://doi.org/10.1109/ASPAA.2003.1285860[32] A. Solin, M. Kok, N. Wahlström, T. B. Schön, and S. Särkkä. 2018. Modelingand Interpolation of the Ambient Magnetic Field by Gaussian Processes. IEEE
Transactions on Robotics 34, 4 (2018), 1112–1127.
[33] J. F. Vasconcelos, G. Elkaim, C. Silvestre, P. Oliveira, and B. Cardeira. 2011. Geometric Approach to Strapdown Magnetometer Calibration in Sensor Frame. IEEE
Trans. Aerospace Electron. Systems 47, 2 (2011), 1293–1306.
[34] Chang Xiao, Karl Bayer, Changxi Zheng, and Shree K. Nayar. 2019. Vidgets:
Modular Mechanical Widgets for Mobile Devices. ACM Trans. Graph. 38, 4,
Article 100 (July 2019), 12 pages. https://doi.org/10.1145/3306346.3322943[35] Wataru Yamada, Hiroyuki Manabe, and Daizo Ikeda. 2018. CamTrackPoint:
Camera-Based Pointing Stick Using Transmitted Light through Finger. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology(Berlin, Germany) (UIST ’18). Association for Computing Machinery, New York,NY, USA, 313–320. https://doi.org/10.1145/3242587.3242641[36] Kentaro Yasu. 2017. Magnetic Plotter: A Macrotexture Design Method Using
Magnetic Rubber Sheets. In Proceedings of the 2017 CHI Conference on Human
Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for
Computing Machinery, New York, NY, USA, 4983–4993. https://doi.org/10.1145/
3025453.3025702[37] Kentaro Yasu. 2019. Magnetact: Magnetic-Sheet-Based Haptic Interfaces for
Touch Devices. In Proceedings of the 2019 CHI Conference on Human Factors in
Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing
Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3290605.3300470[38] Kentaro Yasu. 2020. MagneLayer: Force Field Fabrication by Layered Magnetic
Sheets. In Proceedings of the 2020 CHI Conference on Human Factors in Computing
Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery,
New York, NY, USA, 1–9. https://doi.org/10.1145/3313831.3376552[39] Sang Ho Yoon, Ke Huo, and Karthik Ramani. 2016. TMotion: Embedded 3D
Mobile Input Using Magnetic Sensing Technique. In Proceedings of the TEI ’16:
Tenth International Conference on Tangible, Embedded, and Embodied Interaction(Eindhoven, Netherlands) (TEI ’16). ACM, New York, NY, USA, 21–29. https://doi.org/10.1145/2839462.2839463[40] Sang Ho Yoon, Yunbo Zhang, Ke Huo, and Karthik Ramani. 2016. TRing: Instantand Customizable Interactions with Objects Using an Embedded Magnet anda Finger-Worn Device. In Proceedings of the 29th Annual Symposium on User
Interface Software and Technology (Tokyo, Japan) (UIST ’16). ACM, New York, NY,
USA, 169–181. https://doi.org/10.1145/2984511.2984529[41] Zalmai, N., Kaeslin, C., Bruderer, L., Nef, S., and Loeliger, H. 1999. Gesturerecognition from magnetic feld measurements using a bank of linear state spacemodels and local likelihood fltering. In Proc. ICASSP ’15. 2569–2573.
[42] Clement Zheng, Jeeeun Kim, Daniel Leithinger, Mark D. Gross, and Ellen YiLuen Do. 2019. Mechamagnets: Designing and Fabricating Haptic and Functional Physical Inputs with Embedded Magnets. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (Tempe, Arizona, USA) (TEI ’19). ACM, New York, NY, USA, 325–334.
https://doi.org/10.1145/3294109.3295622