2013 Conference on Technologies and Applications of Artificial Intelligence

Tap Model to Improve Input Accuracy of Touch Panels
Takahisa Tani
The Graduate University for Advanced Studies
Tokyo, Japan
Email: tani@nii.ac.jp

Seiji Yamada
National Institute of Informatics
The Graduate University for Advanced Studies
Tokyo Institute of Technology
Tokyo, Japan
Email: seiji@nii.ac.jp

Abstract—In recent years, devices that use touch panels asinterfaces, such as smart phones and tablet PCs, have spread.
These devices have many advantages. For example, operating
the panel can be done more intuitively in comparison with using
conventional physical buttons, and the devices are quite more
ﬂexible than those that use a traditional ﬁxed UI. However,
mistakes frequently occur when inputting with a touch panel
because the buttons have no physical boundaries and users
cannot get tactile feedback with their ﬁngers because thepanels never change physically. Thus, the input accuracy of
touch-panel devices is lower than that of devices with physicalbuttons. There are studies on improving input accuracy. Mostof them use language models for typing natural language orprobabilistic models to describe the errors made when users
tap their ﬁngers. However, these models are not practical,
and the experiments are preliminary. Thus, in this paper, we
propose a more practical model for improving input accuracy,
in which the relative relationships between a target object
and neighbor object that might inﬂuence error making whentouching the target are tested. We consider that our model
can describe important properties for designing various UIs
depending on practical applications. We then make a plan toconduct experiments in order to build our model in a calibrated
way and discuss our evaluation of the model.

Figure 1.

screens for input is called the ”fat ﬁnger problem”[1]. This
problem is a problem of accuracy in pointing manipulation.
In the future, input devices will progress, and pointing
accuracy beomes more important. Thus, improving accuracy
improvement is important.
There are many studies such as on improving the accuracyof software keyboards (Figure 1). The software keyboard
needs to place a lot of keys in a small area. Hence, thisis a typical example of the fat ﬁnger problem because thekeys are too small for a user to correctly tap them. Most ofthese studies have used the following two methods. The ﬁrst
one uses language models[2][3]. The model has language
information such as a dictionary. The system can predict thenext character by using the pattern of inputted charactersand the dictionary. For example, when the ﬁrst part of aword is input, the system can predict the next character by
matching the input part with words recorded in a dictionary.
Although this approach is quite effective for key-typing
like tapping a software keyboard, it cannot be applied toother input UIs, including tapping simple buttons, not ona software keyboard. Thus, the applicable coverage of thislanguage model is signiﬁcantly restricted, so we need todevelop more general models to improve input accuracy forvarious concrete UIs.
The second one uses tap models[4][5]. The model has
information on the difference between the locations ofbuttons and the points where a user taps. The system revises
points on the panel of the screen.

Keywords-user interface, touch panel, tap model, touchaccuracy

I. I NTRODUCTION
In recent years, devices that use touch panels as interfaces,such as smart phones and tablet PCs, has spread. These have
many advantages. For example, operating the panel can bedone more intuitively in comparison with using conventional
physical buttons, and the devices are quite more ﬂexible thanthose that use a traditional ﬁxed UI.
However, mistakes frequently occur when inputting with atouch panel because the buttons have no physical boundaries
and users cannot get tactile feedback because the panels
never physically change when being tapped. Thus, the input
accuracy of touch-panel devices is signiﬁcantly lower thanthose using physical buttons in a traditional way. In addition,
users often make unintentional mistakes when using thepanels for input. In particular, smartphones usually have asmaller screen and smaller UI in comparison with conventional large UIs on a computer display. Thus, the lack of highinput accuracy is serious, and this problem with using small
978-1-4799-2528-5/13 $31.00 © 2013 IEEE
DOI 10.1109/.58

Example of software keyboard on smartphone266265

There are also studies that use both methods[6], [7], [8].
There are fundamental studies in which limited and concrete applications to practical UIs are not assumed[9][10].
Although these studies might provide novel knowledge in a
general aspect, it is very difﬁcult to use this knowledge todesign UIs practically. Thus, these models are not practical,
and the experiments are preliminary. This means that themodels may be inﬂuenced by more complex factors such as
the shape or color of the interface.
Thus, in this study, we choose a method that predicts atouch point from multiple sensors that was proposed by [10].
Also, we propose a more practical model that includes theinﬂuence of neighbor objects, e.g., buttons and links.

II. M ETHOD OF E STIMATING TAP P OINTS WITHM ULTIPLE S ENSORS .

The previous studies[10] proposed the following method.
Let s be the input of the multiple sensors and (x, y) be theintended location of a user. Here, multiple sensors means
the output of the touch panel (e.g., location, time, size ofarea, pressure), accelerator sensor, and so on. Then, theycalculate the function (x, y) = f (s) by regression, and thesystem estimates the intended location from the sensor input
by f .
Next, we extend this touch model to a more practicalone by introducing the relationship between a target object
and a neighbor object. Furthermore, we try to introduce
incremental learning to improve the touch model throughuser execution of the UI.

Figure 2.

Tap target and tap location

to be applied to the same as conventional touch models.
Furthermore, this proposed touch model is very practicaland precise because it effectively introduces the inﬂuence ofneighbor objects in contrast with traditional touch models.
III. E XPERIMENT P LAN
We are currently planning the experiments to obtain alarge number of training data and to evaluate the accuracyof our touch model as follows.

A. Inﬂuence of Interface Shape on Tap Model
The tap location in practical use changes with variousfactors. In particular, it is known that the tap model significantly changes with the differences in how the device isheld and how the ﬁngers operate the device. This inﬂuence
may be solved by estimating these factors with sensors likeacceleration sensors.
However, the tap model may change with the interface
shape. In ﬁgure 2, let the blue square be a target. The tap
location has a distribution like the blue line (a), where a
Gaussian distribution is assumed for the touch model. If
there is another object (the green square) on the right handside, the distribution will move to the left side because theuser is aware of the green square. In addition, there might
be an inﬂuence from the color, size, or shape of the object.
The previous studies do not consider this kind of inﬂuencebecause the sensor inputs s do not include this information.
Therefore, we propose adding interface shape i as a variableof f . We consider this information makes a touch model
quite more practical. Since our touch model with interface
shape is basically characterized with (x, y) coordinationon the touch panel, it can be applicable to the various
UIs independent of the properties of tappable objects likebuttons and icons. Thus, this model has a wide coverage

A. Method
We evaluate the inﬂuence of the interface shape on thetap model and the method of obtaining the training data
implicitly. Participants perform a task in which they tap amarker on a touch panel. Figure 3 shows the task windows.
The marker will disappear, and another one will appear
in another location when it is tapped. A marker does notdisappear until being tapped. We instruct participants to tap
as quickly as possible.
The target color is blue or yellow, and the size is 3
∼ 10mm. In addition, some additional markers appear, as
shown in Figure 3(b) ∼ 3(d). All parameters of a tap arerecorded and considered. We show a sufﬁcient number ofmarkers randomly in order to analyze the learning effect.
B. Evaluation
1) Relationship Between Interface Shape and Tap Parameters: The differences in an interface are considered for eachparameter of the obtained tap data.

266267

[3] K. Al Faraj, M. Mojahid, and N. Vigouroux, “Bigkey: A virtual keyboard for mobile devices,” in Proceedings of the 13th
International Conference on Human-Computer Interaction.
Part III: Ubiquitous and Intelligent Interaction. SpringerVerlag, 2009, pp. 3–10.
[4] L. Findlater and J. Wobbrock, “Personalized input: improving
ten-ﬁnger touchscreen typing through automatic adaptation,”in Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems, ser. CHI ’12. ACM, 2012, pp. 815–824.
[5] J. Himberg, J. Häkkilä, P. Kangas, and J. Mäntyjärvi, “Online personalization of a touch screen based keyboard,” inProceedings of the 8th international conference on Intelligent
user interfaces, ser. IUI ’03. ACM, 2003, pp. 77–84.
(a) Target marker only(b) One additional marker[6] A. Gunawardana, T. Paek, and C. Meek, “Usability guided
key-target resizing for soft keyboards,” in Proceedings of the15th international conference on Intelligent user interfaces,
ser. IUI ’10. ACM, 2010, pp. 111–118.
[7] D. Rudchenko, T. Paek, and E. Badger, “Text text revolution:a game that improves text entry on mobile touchscreen keyboards,” in Proceedings of the 9th international conference
on Pervasive computing, ser. Pervasive’11. Springer-Verlag,
2011, pp. 206–213.
[8] M. Goel, A. Jansen, T. Mandel, S. N. Patel, and J. O.
Wobbrock, “Contexttype: using hand posture information toimprove mobile touch screen text entry,” in Proceedings ofthe SIGCHI Conference on Human Factors in Computing
Systems, ser. CHI ’13. ACM, 2013, pp. 2795–2798.

(c) Two additional markers
Figure 3.

[9] C. Holz and P. Baudisch, “Understanding touch,” in Proceedings of the SIGCHI Conference on Human Factors inComputing Systems, ser. CHI ’11. ACM, 2011, pp. 2501–2510.

(d) Eight additional markers
Task windows[10] D. Weir, S. Rogers, R. Murray-Smith, and M. Löchtefeld,“A user-speciﬁc machine learning approach for improving
touch accuracy on mobile devices,” in Proceedings of the25th annual ACM symposium on User interface software andtechnology, ser. UIST ’12. ACM, 2012, pp. 465–476.

2) Accuracy of Input: Input data is corrected by using
the tap model obtained by using the proposed technique, andinput accuracy is evaluated. As a candidate for comparison,
the result of not correcting and the model that does not takeinterface form into consideration are examined.
IV. C ONCLUSION
In this study, we considered that our model can describe
important properties for designing various UIs depending onpractical applications. We therefore made a plan to conduct
experiments in order to build our model in a calibrated way
and discuss the evaluation of our model. In the future, wewill experiment on and evaluate our method.
R EFERENCES[1] K. A. Siek, Y. Rogers, and K. H. Connelly, “Fat ﬁnger
worries: How older and younger users physically interact withpdas,” In Proc. of Interact 2005, Tech. Rep., 2005.
[2] J. Goodman and G. Venolia, “Language modeling for soft
keyboards,” in Proc. AAAI 2002. AAAI Press, 2002, pp.
419–424.

267
268