An Empirical Study of Typing Rateson mini-QWERTY Keyboards
Edward Clarkson, James Clawson, Kent Lyons, and Thad Starner
College of Computing and GVU Center
Georgia Institute of Technology
801 Atlantic Drive, Atlanta, GA 30332-0280{edcclark, jamer, kent, thad}@cc.gatech.edu
ABSTRACT

We present a longitudinal study of mini-QWERTY
keyboard use, examining the learning rates of novice miniQWERTY users. The study consists of 20 twenty-minutetyping sessions using two different-sized keyboard models.
Subjects average over 31 words per minute (WPM) for thefirst session and increase to an average of 60 WPM by thetwentieth. Individual subjects also exceed the upper boundof 60.74 WPM suggested by MacKenzie and Soukoreff’s
model of two-thumb text entry [5]. We discuss our resultsin the context of this model.
Author Keywords

Text entry, keypad input, mobile input, mini-QWERTY,
thumb keyboard.
ACM Classification Keywords

H.5.2 User Interfaces: Input devices and strategies.
INTRODUCTION

Mobile computing devices’ popularity has increased
rapidly—some estimates place global mobile phone use at
more than 1.52 billion [1].
Along with this rapid
expansion, the development of advanced two-way pagers,
personal digital assistants (PDAs) and hybrid
phone/PDA/pager devices has also contributed to a similar
growth in mobile text messaging: 135 billion messages
were sent in just the first quarter of 2004 [1].
Considering the amount of text being input on mobileplatforms, research has targeted the development andevaluation of mobile text entry methods. One technique isto use a miniature version of the traditional desktop
QWERTY keyboard (referred to as ‘mini-QWERTY’ or‘thumb’ keyboards).
Despite the presence of miniQWERTY keyboards in the mobile computing marketplace,there is very little published data on user typing rates withthese devices. We present the results of a longitudinal
study of novice mini-QWERTY users, the results of which

Copyright is held by the author/owner(s).
CHI 2005, April 2–7, 2005, Portland, Oregon, USA.
ACM 1-59593-002-7/05/0004.

indicate users can reach typing rates that meet or exceed
many other current mobile input technologies. These firsthand observations can provide a comparative basis for
future mobile text-input methods.
RELATED WORK

Several examples of commercial mini-QWERTY devices
are shown in Figure 1. The RIM Blackberry mobile
information device has included a mini-QWERTY
keyboard since 1997, and it continues to attract a loyal
customer following. The Danger HipTop (also known asthe Sidekick) is a similar, newer device that includes asizeable mini-QWERTY keyboard under a flip-up screen.
Nokia has taken a somewhat different approach with its
6800 series of mobile phones. Its front face can flip open toreveal a split mini-QWERTY layout, with the screen set in
the middle of the keyboard.
MacKenzie and Soukoreff have created a theoretical modelof two-thumb text entry on miniature keyboards [5]. Using
English language letter frequency distributions and Fitts’
Law calculations, they predicted a peak expert rate of 60.74
WPM on mini-QWERTY keyboards. A sensitivity analysisof the model to various parameters (e.g., Fitts’ Law
coefficients, word corpus effect, etc.) yielded no more thana +/-10% variation from their original figure. Validation of
the model was left for future work.
Researchers at Canesta, Inc. reported a study that included
mini-QWERTY typing speeds at CHI 2003 [8]. In
evaluating their virtual projection keyboard, they tested itagainst a desktop QWERTY keyboard, Grafitti pen input
and a thumb keyboard. They recruited 11 subjects whoused each method in random order, typing a phrase
repeatedly for 2 minutes. Subjects achieved an average of
27.6 WPM on the thumb keyboards, 64.8 WPM on theconventional keyboard, 46.6 WPM on the Canesta
keyboard, and 14.0 WPM with Grafitti. The authors statetheir subjects included both novice and expert Canesta
keyboard users but do not mention participants’ experience
with any of the other input devices.
For a morecomprehensive review of mobile input technologies, we
direct the reader to the review of text entry techniques
found in MacKenzie and Soukoreff [6].

triangular space keys set below the rest of the keys. The
Targus keys measure 6.73 mm. along their major axes and4.83 mm. along their minor axes. The Dell keys are 5.99 x
4.06 mm. (by comparison, the Danger Sidekick keys areapproximately 8 x 6 mm.). We chose to use two similar
keyboards to test more than one point in the keyboard
design space.

Figure 1. Nokia 6820 (top), RIM Blackbery (bottom left) andDanger/T-Mobile Sidekick (bottom right).
RESEARCH METHOD

The study occurred in our usability lab with each of the two
keyboards connected to a separate Pentium III workstation.
We employed the Twidor software package (used in theseries of studies on the Twiddler chording keyboard [2, 3])
and adapted it to accept data from our modified keyboards.
The Twidor software was configured to use the MacKenzie
and Soukoreff phrase set [7], a set of 500 phrases
representative of the English language. The phrases range
from 16 to 43 characters with an average length of 28
characters. The phrase set was modified to use only
American English spellings and display only lower caseletters and spaces (no punctuation or capitalization). Eachtest phrase was shown on the screen and remained there
while the subject typed.

A number of aspects of our design are directly influenced
by other work. In particular, the compensation arrangement
and session structure follow those in Lyons, et al. [3].
Participants

We recruited 21 subjects, none of whom had ever used amini-QWERTY keyboard more than once before. Eachsubject was randomly assigned one of two different miniQWERTY keyboards to use throughout the study (Dell andTargus branded, discussed further below). Similar to theLyons, et al. study [3], participants were compensated at arate proportional to their WPM x Accuracy over the entire
session: $0.125 x WPM x Accuracy, with a $4 minimum
for each session. Participants were asked to complete 20
twenty-minute sessions over the course of 11 days. Foursubjects did not complete all 20 sessions, leaving 10subjects in the Targus group and 7 in the Dell. We
excluded the data from the last three subjects in the Targus
group in order to match the number of subjects in the Dellgroup.
This change allows an analysis of variance(ANOVA) calculation, and all statistics reported are drawn
from this smaller sample. Excluding the three subjects’
data resulted in a mean change of less than 1 WPM on thelast session. Our final sample size was 14 subjects (7 ineach group), all ages 18-34. Six participants were female,eight male, and all but one was right-handed.
Equipment and Software

We used two mini-QWERTY keyboards in the experiment,
one by Dell (for the Dell Axim) and one by Targus (for thePalm m505). We modified each keyboard to connect to a
standard desktop computer serial port. The Dell and Targus
keyboards transmit at 4800 and 9600 baud, respectively.
The letter keys on both keyboards are oval-shaped as shownin Figure 2. The Dell keyboard has a single space key inthe middle of the bottom key row, while the Targus has twoFigure 2. The modified Targus (top) and Dell (bottom)
keyboards.

Design and Procedure

We structured the study as a 2 x 20 factorial mixed design,with the keyboard as the between-subjects factor and the 20
sessions as the within-subjects factor. Sessions werecompleted in pairs with a 5-minute break after the first 20minute session. Each session pair was separated by at least
two hours and no more than two days. As mentioned, eachsubject was randomly assigned a keyboard that was usedthroughout his or her participation.
Each session was preceded by a warm-up phase, which
consisted of the phrases “abcd efgh ijkl mnop” and “qrstuvwx yz” repeated twice. The warm-up phase was notcounted in the session statistics. The remainder of thesession consisted of a number of trial blocks, containing ten
randomly selected phrases each.
Each participant
completed as many blocks as he or she could in a twentyminute session. The subjects were instructed to type using
only their two thumbs and to type as quickly and accurately
as possible. The test software provided statistical feedbackin the form of WPM and accuracy data for the most recent
sentence typed and the current session average.
In addition to mini-QWERTY rates, we also collected
desktop QWERTY typing speeds averaged over 20 phrases
at each subject’s first and last sessions. Participants
completed a demographics survey before the first session
and a debriefing survey after the end of the last session.
RESULTSThe 14 participants typed 33,945 sentences across all
sessions, encompassing over 950,000 individual characters.
Session statistics are weighted by the number of charactersin each sentence, and error rates were calculated using
Soukoreff and MacKenzie’s total error metric [10].
Averaged over both keyboards, participants had a mean
first session typing rate of 31.72 WPM (σ = 7.00) and asession twenty mean of 60.03 WPM (σ = 8.40). The
average error rate for session one was 6.12% (σ = 3.46%),rising gradually to 8.32% (σ = 4.13%) by session twenty.
The slight upward trend in error rates is consistent with ourprevious mobile text entry research [3].
Learning Rates

We derived an exponential regression curve for eachparticipant and the group averages. The per-group data
correlated closely with the derived curves (R2 > 0.98; seeFigure 4). Power curves fitted to individual users showed
high correlation, with R2 values ranging from 0.76 to 0.97.
Keyboard Differences

The Targus group typed faster than the Dell group. The
mean session one rates were 34.33 WPM (σ = 8.15) on theTargus and 29.12 WPM (σ = 4.88) on the Dell; session
twenty average speeds were 61.44 (σ = 8.16) on the Targus
and 58.61 WPM (σ = 9.03) on the Dell (see Figures 3 and4).
We performed a two-way analysis of variance(ANOVA) with the session number and the keyboard typeas our factors. The ANOVA showed significant effects forboth the session and keyboard variables with extremely
high confidences (p < 0.0001). Interaction between the two
was not present (p > 0.9999). These results indicate there
was a statistically significant difference between the twokeyboards—the Targus was faster than the Dell—and,unsurprisingly, typing speeds improved over time.
The mean desktop QWERTY typing speed for the Targus
group was 84.99 WPM (σ = 19.84) before the study and98.31 WPM (σ = 19.25) at the end and 77.38 WPM (σ =15.03) and 86.79 WPM (σ = 14.20) at the start and end forthe Dell. We measured the correlations of the starting
desktop QWERTY speeds with session one mini-QWERTY
rates, and the end desktop QWERTY speeds with session
twenty mini-QWERTY rates. The correlations were mild
(R2 = 0.31 and 0.57, respectively). The increase in desktop
QWERTY speeds may be due to increased familiarity withthe test environment and is consistent with previous work
[3].

80.00

80.00

70.00

70.00

60.00

60.00

50.00

50.00

0.1923y = 35.915x
2
R = 0.9818

0.2216WPM

WPM

y = 29.268x2
R = 0.9814

40.00

40.00

30.00

30.00

20.00

20.00

10.00

10.00

0.00

0.00

12

3

4

5

6

7

8

9

10

11

12

13

14

15

16

1718

1920Session

Figure 3. Individual session average WPM curves. Dellcurves are dashed; Targus are solid.

12

3

4

5

6

78

910111213

14

15

16

1718

19

2021

2223

2425

2627

28

29Session

Figure 4. Per-group session average WPM with regression
curves, equations and correlations. Dell curves are on thebottom; Targus curves on top.

30

Survey Results

After the end of the last session, each participant completed
a debriefing survey, answering a number of free-form and7-point Likert scale questions regarding how they used thekeyboards and their comfort level. Users found the miniQWERTY keyboards marginally comfortable overall (mean4.00, σ = 1.41) and much less comfortable than a full-size
keyboard (mean 2.29, σ = 0.99). There was no statisticaldifference in comfort ratings between the two keyboards (p= 0.36). Subjects also reported they were less likely to lookat the screen after the last session than at the start; however,this was a marginally significant result (p = 0.056). This
result conforms to some anecdotal responses (e.g., “Notlooking at the screen vastly improved my speeds!”).
DISCUSSION

MacKenzie and Soukoreff reported a predicted peak expert
mini-QWERTY rate of 60.74 WPM. We cannot makedirect comparisons of our results to their model, since themodel prediction was derived from a keyboard withsignificantly different properties. To enable an accuratecomparison, we replicated MacKenzie and Soukoreff’s
calculations [5, 9] using measurements from our Dell
keyboard. We used the Dell as our reference since it mostclosely resembled the model assumptions. The resulting
calculations on the MacKenzie phrase set yielded a model
prediction of 57.89 WPM. Three subjects in the Dell groupexceeded the two-thumb model’s predicted maximum forour keyboard on at least one session. A total of 23 Dellsessions exceeded the predicted speed, and the single fastest
session speed was 74.69 WPM.
One potential reason for the empirical deviation from themodel relates to key width measurement. MacKenzie andSoukoroff noted that the relative width of the thumb could
confound the width measurement.
In addition, ourkeyboards are built with oval keys with major axes inclined
45° from horizontal, putting the major axis of each key
parallel to the natural movement of the thumb. As a result,using the width of the keys along the line of approach(denoted W′ [4]) may be more appropriate for similar
keyboards.
FUTURE WORK

Several avenues exist for future work in this area and on thedata we have presented in particular. We wish to fullyreconcile the two-thumb model with our empirical results.
Determining what key width measurement method is mostappropriate for mini-QWERTY keyboards is necessary, as
is clarifying the degree to which thumb size confounds
those measurements. Other model assumptions to consider
include the minimum time required between key presses
and the Fitts’ Law coefficients for our keyboard.
Our anecdotal and qualitative data suggest that users
increase the time they spend looking at the keyboard as theygain more experience. Yet the ability to type effectively

without looking at the input device (“blind typing”) can bebeneficial when mobile; we wish to examine how well ourexperts can type without visual feedback.
CONCLUSIONS

We have presented a longitudinal study of mini-QWERTY
keyboard use, finding significant differences between our
two keyboard groups and expert typing rates exceeding 60
WPM. These rates compare well with other mobile devices
in terms of both novice and expert typing rates, and providea performance benchmark for further research in this area.
ACKNOWLEDGMENTS

Many thanks go to our study participants for their time andpatience and to Brian Gane for his significant help with thestatistical analysis.
Thanks also to the anonymous
reviewers for their suggestions regarding this and futurework. This work is funded in part by NSF Career Grant#0093291 and the NIDRR Wireless RERC.
REFERENCES

1.

CellularOnline, "Latest Mobile, GSM, Global,Handset, Base Station, & Regional Cellular Statistics".
http://www.cellular.co.za/, September, 2004.
2. Lyons, K., Plaisted, D. and Starner, T., Expert
Chording Text Entry on the Twiddler One-Handed
Keyboard. In Proc. ISWC '04.
3. Lyons, K., et al., Twiddler Typing: One-Handed
Chording Text Entry for Mobile Phones. In Proc. CHI'04.
4. MacKenzie, I. S. and Buxton, W., Extending Fitts' Law
to Two-Dimensional Tasks. In Proc. CHI '92, pp. 219226.
5. MacKenzie, I. S. and Soukoreff, R. W., A Model of
Two-Thumb Text Entry. In Proceedings of Graphics
Interface 2002, pp. 117-124.
6. MacKenzie, I. S. and Soukoreff, R. W. Text Entry for
Mobile Computing: Models and Methods, Theory andPractice. Human-Computer Interaction, Vol. 17, pp.
147-198.
7. MacKenzie, I. S. and Soukoreff, R. W. Phrase Sets for
Evaluating Text Entry Techniques. In Proc. CHI '03,
pp. 754-755.
8. Roeber, H., Bacus, J. and Tomasi, C., Typing in Thin
Air: The Canesta Projection Keyboard - a New Methodof Interaction with Electronic Devices. In CHI '03
Extended Abstracts pp. 712-713.
9. Soukoreff, R. W., Text Entry for Mobile Systems:
Models, Measures, and Analyses for Text Entry
Research. M. Sc. Thesis, Department of Computer
Science, York University, 2002.
10. Soukoreff, R. W. and MacKenzie, I. S., Metrics for
Text Entry Research: An Evaluation of MSD andKSPC, and a New Unified Error Metric. In Proc. CHI'03, pp. 113-120.