# Design of a Template for Handwriting Based Hindi Text Entry in Handheld Devices

Diya Gangopadhyay, Ityam Vasal, Pradeep Yammiyavar

► To cite this version:

Diya Gangopadhyay, Ityam Vasal, Pradeep Yammiyavar. Design of a Template for Handwriting Based Hindi Text Entry in Handheld Devices. Second IFIP WG 13.6 Conference on Human Work Interaction Design: Usability in Social, Cultural and Organizational Contexts (HWID), Oct 2009, Pune, India. pp.132-143, 10.1007/978-3-642-11762-6_11. hal-01056247

HAL Id: hal-01056247

https://inria.hal.science/hal-01056247

Submitted on 18 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Design of a Template for Handwriting Based Hindi Text Entry in Handheld Devices

Diya Gangopadhyay¹, Ityam Vasal², Pradeep Yammiyavar³

¹ Carnegie Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213

² FICO India, Titanium Building, Kodihalli, Bangalore-560017, India

³ Professor, IIT Guwahati, Guwahati, Assam-781039, India

{dgangopa@andrew.cmu.edu, ityam.vasal@gmail.com, pradeep@iitg.ernet.in}

**Abstract.** **Abstract.** Mobile phones, in the recent times, have become affordable and accessible to a wider range of users including the hitherto technologically and economically under-represented segments. Indian users are a gigantic consumer base for mobile phones. With Hindi being one of the most widely spoken languages in the country and the primary tool of communication for about a third of its population, an effective solution for Hindi text entry in mobile devices is expected to be immensely useful to the non English speaking users. This paper proposes a mobile phone handwriting based text entry solution for Hindi language, which allows for an easy text entry method, while facilitating better recognition accuracy.

**Keywords:** Keypad based entry, Hindi text entry, multi keystroke, mobile phones, usability evaluation, handwriting recognition.

## 1 Introduction

Mobile phones are fast emerging as the primary mode of communication for increasingly large sections of population with diverse needs. This has resulted in the creation of several distinct user groups with vastly different needs. Ample opportunities are available to the mobile service providers, to increase their market share by catering to the specific needs of each segment of these diversified user groups. New services need to be designed and tested for user groups with a common cultural base, in order to be able to hold the market. Researchers like Rose [1] have found that the earlier the factors of localization are considered, the better the acceptance ratio of the service and hence, the device.

India has emerged as the second largest mobile phone market in the world after China, in April 2008 with the subscriber base already crossing the 250-million mark [2]. The increasing mobile penetration in India and particularly in the semi-urban and rural parts has resulted in a large user base for mobile interface in regional languages; Hindi being one of the most widely spoken amongst them. The Hindi speaking user base constitutes 41% of the Indian population, with a major percentage of it not being
proficient in the use of English language. The focus of this paper is to find the limitations of the existing solutions for Hindi text input in mobile phones, and propose an alternative handwriting based input solution.

## 2 Hindi Letterforms

Hindi is written in a script called Nagari or Devanagari. Hindi is spoken using a combination of 52 sounds. These sounds are represented in the Devanagari script by 52 symbols: for 10 vowels (Fig.1), 2 vowel modifiers (Fig.1) and 40 consonants (Fig. 2). Vowels and consonants together are called *Akshars*. Along with pure consonants the language consists of partial consonants which are called *ardha-akshar* and conjunct consonants. Besides, for every vowel, there is a corresponding *Matra* (Fig. 1) which is combined with consonants to impart the sound of that vowel.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Image showing the vowels (first 10 symbols) and vowel modifiers (last 2 symbols) and their corresponding Matras present below each vowel</figcaption></figure>

Fig. 2 shows 36 consonants. Apart from these there are four other compound consonants which are not very frequently used. Structurally, the letters in the Hindi alphabet can be said to be spatially composed of five parts. The main component of each letter or the principal character (characters in case of conjuncts) occupies the central position while the Matras can occupy space either above, below or to the sides of the principal character. The Matras usually take up a smaller proportion of the space as compared to the principal character. Therefore written Hindi script is essentially a combination of multiple strokes.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Image showing the consonants</figcaption></figure>

## 3 Existing Hindi Text Input Solutions in Mobile Phones

Mapping the Hindi alphabet to a mobile keypad is difficult due to the sheer number of letters (52) in the alphabet. Moreover, the existence of Matras that occupy space above or below the alphabet and the partial consonants make the problem of mapping existing key boards, all the more complex. The current mobile keypads mapping Hindi letters can broadly be classified into the following schemes [3]:

1) Multi-keystroke

2) Single stroke followed by navigation

3) Two key-strokes

For the purpose of this study, we refer only to the multi-keystroke scheme as implemented by Nokia 1100. We chose the multi-keystroke method for our study since it is the most common method of text entry in mobile phones [11]. We used the Nokia interface because Nokia is by far the most commonly used mobile phone in India, with a market share of about 60% [11]

Presently the mobile phones of different brands (Nokia, Samsung, etc) use different keypad mapping schemes [3] and have their respective patents on the same. However, there exists no standard scheme for mobile keypad mapping for Hindi text entry [3]. This implies that each time the mobile handset is changed, user has to learn an entirely new mapping scheme for Hindi text entry. This makes it even more
difficult to learn, understand and remember these mappings, demanding a new
method to reduce the cognitive load of the users and improving usability.

The multiple mappings result in requiring more number of keystrokes per key than the user can keep track of. Each key is mapped to 2-5 characters, resulting in high error rate [3]. It also requires users to remember the sequence of letters in the alphabet, resulting in higher cognitive load. In addition to all this, a vast majority of the users in emerging markets are yet to adopt fully to the usage of keypads and typing from the conventional pen-paper writing. It follows that for a user base accustomed to hand-written message, transmission stylus or pen based text input is more suited.

4 Handwriting Recognition and its Limitations

Handwriting recognition is defined as the ability of a computer to translate human
writing into displayed text. The image of the written text may be sensed "off line"
from a piece of paper by optical scanning (optical character recognition) or intelligent
word recognition. Alternatively, the movements of the pen tip may be sensed "on
line", for example by a pen-based screen surface [13].

Handwriting based text entry solution for mobile devices, has been extensively
researched in the recent times, some of the areas of focus being recognition
algorithms [6] and interface issues [4] [6].

Bharath and Madhavanath [7] propose a solution for continuous handwriting based input in small writing surfaces using pen or finger. The technique used is allowing over-writing in the writing area in a continuous manner rather than enter one character at a time. Usability issues concerning text input in small surface are discussed. These which include requiring the user's attention to the characters being written and switching focus between the writing surface and the screen. A gesture-recognition based quick text entry solution called SHARK (shorthand aided rapid keyboarding) [7], is proposed in which uses shorthand gesturing in order to speed up stylus based keyboard entry. The results indicate that users were able to learn to write correct, recognizable gestures. It can be deduced that learning a slightly modified form of writing (as required by our proposed design) is not likely to be too difficult.

In the field of text input in Indian Languages, Aparna, K. et al [4] presents a
method for online recognition mechanism in the case of hand-written Tamil text. It
describes character recognition as recognizing the constituent strokes and matching
them against a pre-defined database. The developments of similar systems for other
Indian languages as outlined have been used as assumptions for proposed template.

In MacKenzie et al, [8] the tradeoff between the recognition accuracy of
handwriting, memory requirement of the device and forcing constrained writing to
ease recognition is discussed. Allowing natural handwriting requires a very advanced
recognizer with larger memory, which may be a problem for handheld devices. An
interface that forces some constraints to make recognition easy without adversely
affecting the usability of the device is therefore a necessity. From the users' usability
point of view simplified inputs methods with minimum strokes become imperative.

## 4.1 Hindi Script and Handwriting Recognition

Online pen based surface recognition is what we are proposing as a potential solution for Hindi text input in mobile phones. The target user group for this solution comprises rural and semi-urban Hindi speaking population, who are better versed with writing than with keyboard entry. This method supports the familiar act of writing which our target user groups are used to. It also reduces the burden of high learnability required in the existing keypad based solutions, hence minimizing the barrier of familiarity to the solution being proposed.

Moreover, Hindi keypad entry, as it stands now, has several usability issues and no
standardized interface or keypad mapping scheme [3]. The current interfaces require
more number of successive pressing of a single key than users can keep track of [3].
This is a result of mapping too many letters on a single key. This problem can be
partially solved by using a full keyboard instead of a 12 keys keypad. However,
problems related to mapping the Matras and the partial consonants will still remain,
since no standard, easy to use solution exists for this issue [5].

The major issue here is the low accuracy of handwriting recognition for Hindi,
especially for free form writing. Poor accuracy level severely hampers usability of
this technology.

Character recognition is done in the following steps [4]:

1) Stroke identification- where the input stroke is compared with the database of existing defined strokes

2) Character recognition- grouping already identified stroke labels and converting them to suitable character codes. For Indian alphabet, the ISCII (Indian Script Code for Information Interchange) is used. ISCII is a phonetic code, which represents composite characters in terms of component consonants and vowels.

Stroke recognition has been identified as the crucial factor that determines the
accuracy of the handwriting recognition algorithm.

The ideal input conditions for high recognition accuracy include [4]:

1. Consistent size of the hand written strokes constituting the letters

2. Constant orientation of the letters

## 5 Proposed Handwriting Based Solution

The solution we are proposing combines handwriting (stylus or pen) based input with
intelligent suggestion, as described in a subsequent section. To improve the
recognition accuracy we conceived the notion of providing the users with a template to write on. The template would facilitate the written strokes to be closer to the ideal condition, as described in the previous section. We discuss the two key components of our solution; the template and the intelligent suggestion in the following two subsections.

## 5.1 The Template

Our literature review shows that recognition accuracy for hand-written text improves considerably if the size of the letterform (glyph) remains consistent and the orientation, vertical [4].

In order to constrain the hand-written input text in this manner, we propose a writing template (Fig.3). This template is a square block that is partitioned into nine sub-divisions as shown in Fig. 4. The users are required to constraint their letters in such a way that the *Akshara* (main component of the letter) fits within the inner block and the *Matras* in one of the side or top or bottom blocks (Fig. 5). Further, users are encouraged to take up as much space as possible within each block without moving out of them. The upper and the lower blocks have a grey area within them. This is the area within which the strokes should touch the upper and the lower edges of the template as in Fig. 5. This ensures both a lower and an upper limit on the size of each component and yet provide wide margin of freedom to the users finger action. The ideal way of writing inside the template has been shown in Fig. 5.

By separating regions for the *akshara* and the *matras*, the template allows for *region specific coding*. That is, anything written within the central square is necessarily an *akshara* whereas anything written in the rectangles at the sides is necessarily a *matra*. This helps in using separate database for the identification of the written letter-form depending on the location.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Image showing the dimensions of the template used</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>Fig. 4. Image showing the dimensions of the template used</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5. Image showing the writing format inside the proposed template using three different Matras occupying right, left and bottom positions, respectively with the same letter. The Matras should touch the edge of the boxes (above and below) within the grey areas.</figcaption></figure>

To allow natural writing to the extent possible, we designed our interface to consist
of two rows of touch-pad based writing space with eight templates each (Fig. 6).

<figure><img src="image_7.png" /><figcaption>Fig. 6. Image showing the mapping of template on a touch based mobile phone</figcaption></figure>
## 5.2 Intelligent Suggestion

While the template is likely to have a significant positive impact on recognition accuracy, it is also likely to reduce the speed of writing as the users have to be mindful about where they are placing each stroke. To counteract this, we have incorporated Intelligent Suggestion of the most probable letters. As the user writes the glyphs, the system tries to guess what letter it corresponds to. Once the system is reasonably confident of its guesses, it suggests the three most probable letters that the user is trying to write. The user can then disambiguate by choosing on the correct option or simply ignore the suggestions and continue to write. This feature is similar to the "Approximate String Matching Technique" as discussed in [14].

This feature makes sure that the user does not need to write each letter to completion. Once a written glyph is specified enough for the system to suggest it as one of the three possible letters, the user can simply click on the right suggestion to select it.

This feature is thus expected to increase the overall speed of input especially if the suggestions are accurate most of the time.

# 6 Evaluation of the Proposed Solution

Our purpose behind designing the writing template was to ensure consistency in the written letterforms, in order to facilitate an enhanced recognition precision rate. However, it is important to ensure that while doing so we are not constraining the users to write in a specific way which they need to learn. We are also interested in comparing the speed of input using our template with the current state of the art solution with keypad entry. To ensure that our template allows for a text entry rate higher than existing Hindi keypads, and that it is easy to work with, we conducted a usability test [9].

The metrics we were interested in are:

1) Effectiveness of our template (if users can complete a task correctly)

2) Ease compared to keypad entry (number of errors made)

3) Text entry rate as compared to keypad

We first conducted a benchmark usability test with the Nokia 1100 Hindi keypad. This model represents one of the three most prominent mapping schemes for Hindi keypads.

**Description of Mapping Scheme in Nokia 1100.** The Nokia 1100 keypad is based on the multi-keystroke model. Multiple letters are mapped onto each key and are disambiguated by the number of times the key is pressed in quick succession. The mappings follow the sequence of the letters in the Hindi alphabet. Each key has a set of consecutive letters mapped onto it. This scheme is not made explicit by the
interface. Only the first and the last letters mapped onto a key are displayed and the user is expected to know the range of letters covered. The number of letters mapped onto each key varies between 2 and 5 characters.

## 6.1 The Benchmark Usability Testing

We conducted a task analysis [10] involving on ten subjects.. The subjects were divided into two groups of six and four. To one group the mapping was explained and to the other group it was not. This was done to "artificially" create a mix of expert and novice users since we had no access to actual expert users of Hindi keypad based text entry.

The participant users were in the age range of 18 to 22, tech savvy and frequent users of mobile phone. They were conversant with the Hindi (Devanagari) alphabet and the language. However, they had little or no experience with Hindi keypads. Though we could not access users from the emerging user base of mobile phones due to limited resources, we ensured that the chosen participant's familiarity with Hindi keypads and expertise in typing Hindi letters were comparable to our target user segment. Since these are the two predominant factors in this study, we expect to see similar results with a more representative sample.

The task assigned to them was to type in the following sentence in Hindi:

मैं दीपावली पर घर आ रहा हूँ

The sentence comprises 26 characters, including the Matras and the spaces between the words. We counted the spaces since those also need to be typed using the space key on a keypad. Conjunct letters were excluded from the task to avoid making it too complex. The observations from the test are enlisted in Tables 1 and 2.

<table><caption>**Table 1.** Observations for the group to whom the mapping scheme was explained</caption><thead><tr><td>S. No</td><td>Task Completed</td><td>Number of errors</td><td>Total time taken</td><td>Time per character (in sec)</td></tr></thead><tbody><tr><td>1</td><td>Yes</td><td>1</td><td>240</td><td>9.23</td></tr><tr><td>2</td><td>Yes</td><td>1</td><td>117</td><td>4.5</td></tr><tr><td>3</td><td>Yes</td><td>4</td><td>215</td><td>8.27</td></tr><tr><td>4</td><td>Yes</td><td>0</td><td>109</td><td>4.19</td></tr><tr><td>5</td><td>Yes</td><td>1</td><td>180</td><td>6.92</td></tr><tr><td>6</td><td>Yes</td><td>3</td><td>160</td><td>6.15</td></tr><tr><td>Mean</td><td></td><td>1.66</td><td>170.17</td><td>6.54</td></tr></tbody></table>
<table><caption>**Table 2.** Observations for the group to whom the mapping scheme was not explained</caption><thead><tr><th>S. No</th><th>Task Completed</th><th>Number of errors</th><th>Total time taken</th><th>Time per character (in sec)</th></tr></thead><tbody><tr><td>1</td><td>Yes</td><td>1</td><td>210</td><td>8.08</td></tr><tr><td>2</td><td>Yes</td><td>5</td><td>320</td><td>12.31</td></tr><tr><td>3</td><td>No</td><td>4</td><td>--</td><td>--</td></tr><tr><td>4</td><td>Yes</td><td>3</td><td>315</td><td>12.11</td></tr><tr><td>Mean</td><td></td><td>3.25</td><td>281.67</td><td>10.83</td></tr></tbody></table>

## 6.2 Usability Testing of the Proposed Template

To measure the speed and ease of handwriting based text input, we asked five users to write the same Hindi sentence on a paper with our template printed on it. They were explained the specifications of writing within the template and asked to adhere to them. This test was conducted with the assumption that speed and overall ease of handwriting will not vary significantly between pen-paper and stylus. However, there are certain differences between pen-paper interaction and stylus based interaction on a handheld device, which need to be kept in mind.

First of all, the test was done with the paper with the templates placed on a table, giving a stable writing surface. This enabled the participants to devote their attention fully on the act of writing, since they did not have to hold the base they were writing on, which can otherwise make the act of writing more difficult. Since the real users would be required to hold the handheld device while writing on its screen using a stylus, it would require them to simultaneously keep the writing surface stable by avoiding movement, and write using the template, adversely affecting its usability. We propose usability tests with the actual interface, in order to get a more accurate picture of the interaction.

Secondly, it is important to keep in mind that while writing, the interaction with the template will require the writing area to be the focus of the user with frequent shift of focus to the screen to get feedback that the correct letters are being written, a factor discussed in [6]. This aspect is expected to slower down the rate of writing and also need some time to get familiarized with.

Ergonomic factors such as enabling users to hold the handheld device in such a way so as to make the writing surface as stable as possible are also to be considered. The holding area should be close to the writing area for this purpose.

The profile of the participants used was kept the same as the benchmark test to be able to make valid comparisons. We were interested in the overall time and the number of errors made as a measure of ease. The results are enumerated in table 3.
<table><caption>**Table 3.** Results of usability testing of our template</caption><thead><tr><th>S. No</th><th>Task Completed</th><th>Number of errors</th><th>Time to write in free handwriting (in sec)</th><th>Time to write in template (in sec)</th><th>Time per character (in sec)</th></tr></thead><tbody><tr><td>1</td><td>Yes</td><td>2</td><td>11</td><td>89</td><td>3.4</td></tr><tr><td>2</td><td>Yes</td><td>0</td><td>15</td><td>58</td><td>2.23</td></tr><tr><td>3</td><td>Yes</td><td>0</td><td>16</td><td>47</td><td>1.8</td></tr><tr><td>4</td><td>Yes</td><td>0</td><td>21</td><td>51</td><td>1.91</td></tr><tr><td>5</td><td>Yes</td><td>1</td><td>17</td><td>46</td><td>1.77</td></tr><tr><td>Mean</td><td></td><td>0.6</td><td>16</td><td>58.2</td><td>2.24</td></tr></tbody></table>

# 7 Discussion

Our task analysis results show that handwriting based input for Hindi using our proposed template is about 3 times faster than keypad entry when the keypad mapping scheme is explained to the users and 5 times faster when it is not.

All the participants got used to the template with some basic instructions and were not seen to have any difficulty with following the specifications. It did not require them to alter the order in which they wrote the strokes; however it did require them to consciously control the proportions of the component strokes of each letter, which slowed them down. They reported to not have any problem doing this once they got used to it after writing the first few letters.

The fact that all participants could complete the task and made very few errors show that writing in the template is easy to learn. The error rate is a considerable improvement over the keypad mode of entry. The proposed template thus provides a structure to handwritten letterforms in order to improve recognition accuracy while scoring over current keypad based solutions in terms of speed and ease (lower error rate) of input.

Constraining users to write within the structure of the template slows down their writing to a large extent (Table 3) which can impact the overall experience of writing. However, using intelligent prediction and allowing users to choose from the most probable options is likely to improve speed considerably.
## 7.1 Effectiveness of the Proposed Solution

The template addresses the key issue of recognition accuracy by enforcing consistency of orientation and size of the various components of Hindi letters. It does so by reducing variation in the spatial composition of the letterforms. If a hand-writing recognition based system is trained using samples of letters written using this template, the recognition system can learn the specific inter-relationships (or size and orientation) of the components of the letterforms as enforced by the template. Since these inter-relationships remain consistent for all letters written using the template, they can act as cues for recognition and afford higher accuracy.

While it structures written letters, it does not make the act of writing strenuous, nor does it require considerable amount of learning as indicated by the results of task analysis. Since it leverages on the act of writing and not requiring searching for letters in a set of keys, it is suitable for the non-tech savvy emerging user base of mobile phones. Although the template slows down the speed of writing considerably it is still an improvement over key-pad based solutions currently available. It attains higher speed by not requiring users to search for letters or remember their sequence in order to enter text.

## 7.2 Limitations of the Study and Future Work

While the results of our task analysis are promising, we should keep in mind its shortcomings which could have affected the results. Since, the tasks were tested on a small sample of 5 users, the results should not be over-generalized. Moreover, the sample users did not include users with low expertise with technology, who are an important target user base for our solution. The solution thus needs to be tested with a larger set of users, adequately representing the emerging user base to get statistically significant results.

Lastly, the evaluation of the template was done on a paper simulation and requires validation with an actual hand-writing recognition system. The most important step is to validate that the template actually improves the accuracy of hand-writing recognition.

## 8 Conclusions

The proposed template in conjugation with an adequately trained hand-writing recognition system is likely to be an effective solution to the problem of Hindi text entry in mobile devices by improving the recognition accuracy and allowing a reasonable speed of input. Results from this study show that it is easy to learn and quicker than keypad based solutions. For novice users who belongs to the emerging
new segments it is not likely to put strain on the user in terms of use ability and
learning ability.

Acknowledgments.

We would like to acknowledge all our users who cooperated in the usability testing
process.

9 References

1. Rose, K. and D. Zuhlke.: Localization of user-interface design for mainland China: Empirical study results and their description in a localization model and language issues, Evolution of the human-computer interaction. Nova Science Publishers, New York (2005)

2. TRAI (an ISO 9001: 2000 Certified Organization) Annual Report (2007-08)

3. Katre D. S.: Cross-cultural Usability Issues of Bilingual (Hindi & English) Mobile Phones. In: Indo-Danish HCI Research Symposium (2006)

4. Aparna, K. H., Vidhya Subramanian, M. Kasirajan, G. Vijay Prakash, V. S. Chakravarthy, Sriganesh Madhvanath.: Online Handwriting Recognition for Tamil. In: Ninth International Workshop on Frontiers in Handwriting Recognition - IWFHR (2004)

5. Rathod, A. &Joshi, A.: A Dynamic text Input Scheme for phonetic scripts like Devnagari. In: (2002)

6. Bharath A. and Madhvanath, S.: A Novel Handwriting-based Text Input for Pen and Touch Interfaces. In: FreePad, HPL-2007-203 (2007)

7. Zhai, S. & Kristensson, P.: Shorthand Writing on Stylus Keyboard. In: CHI (2003)

8. MacKenzie, I. and Soukoreff, R. William.: Text Entry for Mobile Computing: Models and Methods,Theory and Practice. In: Human Computer Interaction (2002).

9. Jacob Nielsen's Alert Box, http://www.useit.com/alertbox/20030825.html

10. Description on UsabilityNet, http://www.usabilitynet.org/tools/taskanalysis.htm

11. Rajeshkannan R., Nareshkumar M., Ganesan R. and Balakrishnan R.: Language Localization for Mobile Phones. In: Mobile and Pervasive Computing (2008)

12. Mobile Handset Market Share, http://www.pluggd.in/mobile/mobile-handset-market-share-india-nokia-leads-while-samsung-beats-motorola-1684/

13. Handwriting Recognition, http://en.wikipedia.org/wiki/Handwriting_recognition

14. Masui T.: An Efficient Text Input Method for Pen based Computers. In: Proceedings of the ACM Conference on Human Factors in Computing Systems – CHI (1998)