  To make off-screen interaction without specialized hardware practical, we
investigate using deep learning methods to process the common built-in IMU
sensor (accelerometers and gyroscopes) on mobile phones into a useful set of
one-handed interaction events. We present the design, training, implementation
and applications of TapNet, a multi-task network that detects tapping on the
smartphone. With phone form factor as auxiliary information, TapNet can jointly
learn from data across devices and simultaneously recognize multiple tap
properties, including tap direction and tap location. We developed two datasets
consisting of over 135K training samples, 38K testing samples, and 32
participants in total. Experimental evaluation demonstrated the effectiveness
of the TapNet design and its significant improvement over the state of the art.
Along with the datasets,
(https://sites.google.com/site/michaelxlhuang/datasets/tapnet-dataset), and
extensive experiments, TapNet establishes a new technical foundation for
off-screen mobile input.
