# Ustaad: A Mobile Platform for Teaching Illiterates

Syed Tirmizi, Yashfa Iftikhar, Sarah Ali, Ahmed Ehsan, Ali Ehsan, Suleman Shahid

► To cite this version:

Syed Tirmizi, Yashfa Iftikhar, Sarah Ali, Ahmed Ehsan, Ali Ehsan, et al.. Ustaad: A Mobile Platform for Teaching Illiterates. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.788-796, 10.1007/978-3-030-29384-0_47. hal-02544602

HAL Id: hal-02544602

https://inria.hal.science/hal-02544602

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Ustaad: A mobile platform for teaching illiterates

Syed Ali Umair Tirmizi¹, Yashfa Iftikhar², Sarah Ali³, Ahmed Ehsan⁴, Ali Ehsan⁵,
Suleman Shahid⁶

Lahore University of Management Sciences, Lahore, Pakistan
{18030004¹, 17030013², 17030026³, 17030009⁴, 17030008⁵, suleman.shahid⁶}@lums.edu.pk

**Abstract.** According to a recent statistical analysis conducted in 2018, more than 40% of the population has no reading or writing skills especially in rural areas of Pakistan. On the contrary, the mobile phone users have grown at a very steep rate even with a stagnant literacy rate. We formed a user-driven approach to research, develop and test a prototype mobile application that could be used to teach illiterates basic reading, writing and counting skills without using traditional schooling techniques. This first of a kind application provided the user the ability to customize their own learning plan. Focusing on native language Urdu, the application teaches them the required skill they need for daily life activities such as writing their own name, scenario-based calculations, identifying commonly used words.

**Keywords:** User-centered design, illiteracy, mobile phones, human-computer-interaction

## 1 Introduction

Pakistan is in the throes of an educational crisis, the literacy rate in Pakistan is lowest among the South Asian countries, with a current literacy rate of 55 percent[2] in the most populated province; Punjab, has witnessed a significant decline in literacy rate from 64 percent to 59 percent during the years of 2016-2017[3]. The definition of literacy in Pakistan was recently redefined as: “The ability to read and understand a simple text in any language from a newspaper or magazine, write a simple letter and perform a basic mathematical calculation (i.e. counting and addition/subtraction)” after the Population and Housing Census 2017[1]. However, technology is making huge strides in the country with a growing number of mobile phone subscribers each year. Recent statistics by telecommunication authority of the country, PTA revealed that over 150 million mobile subscribers are present in Pakistan who may have access to 4G and 3G technology and currently more than 67 million already on high speed mobile internet [20]. These internet-enabled smartphones provide a special medium for education in native language Urdu among illiterates especially in adults who cannot start or resume schooling due to different social reasons.

Using the definition of literacy described in the Population and Housing Census 2017 [1] in our study, we examined the illiterate adults in rural parts of Lahore. Through user-centered design, we develop a prototype application called Ustaad to
teach basic reading, writing and counting skills through mobile phone. We tested the
application with adults from different backgrounds having little or no formal educa-
tion and showed that self-learning can be increased through tailor-made application
for illiterates. This presents a framework for teaching native language to users belong-
ing to low literacy areas by developing skills useful in day to day life in a short
amount of time.

## 2 Related Work

Recently, work done in ICT4D has provided more useful applications to people who have received little or no formal education. Improvising techniques such as pictures, emoji, voice and sometimes haptic inputs for text have been included in applications as aids for illiterate users. A certain study was done to help illiterate people combat one of the most frequent problems they face which is texting someone [6]. Using techniques such as speech-to-text, pictures and reusing previous messages the study concluded that illiterate people use mobile quite frequently but avoid text-based applications as such SMS applications, chat messengers and when required use some traditional tricks to combat this problem such as asking a relative for help or stickers/emoji. The prototype of the study focused on providing such a solution based on common interaction techniques using interactive and text-free design.

Such strides are also part of medical and emergency services where an emphatic
research and development project resulted in the first aid iOS application that teaches
illiterates first aid procedures in Mardan area of KPK province in Pakistan [7]. The
study focused on the usability of a sketch-based interface vs an image-based interface
for teaching first aid activities in low literacy area showing sketch based far more
effective in illiterate people. With the introduction of mobile computing and the rapid
growth of mobile users, the need to develop an interface that can be used by both illit-
erates and literates has also taken strides. HCI introduces a new aspect to mobile
computing where interfaces are being designed to facilitate more illiterates. The key
factor in such research is a combination of Graphics and voice I/Os [8, 9] such as
done by IBM India. Although voice-based technology is making great strides in help-
ing less educated interact more with computers and a mobile device such as Web
Spoken technology by IBM Research India [10] but it still isn't as effective as visual
aids due to complex nature of speech synthesis and speech generation. Technologies
such as pattern recognition, deep learning have made visual inputs a lot easier to im-
plement in computer systems making human-computer interaction closer to human
interaction. Hand gestures also shown in studies [11] is shown to be an effective tool
for recognition and gesture-based inputs. Research conducted in Switzerland among
illiterate people ended up recommending icons, colors, and symbols are very im-
portant feature of design for mobile phone interfaces for illiterate and semi-literate
users [12].

Most of the previous work done focuses on children [17] [18] [19]. These applica-
tions allowed pre-decided learning plans with a fixed amount of words without any
support for customized input. Few works which targeted illiterate adults focused on
facilitating rather than actually teaching illiterates in reading writing and calculation
[16][5]. This is the first systematic study on illiterates with focus on Pakistan's native language with customized learning and curriculum planning with the ability to input their own choice of words through voice.

# 3 User Research

We conducted user research using a blend of contextual inquiry and interviews. Since our candidates are illiterate, other user research methods like surveys and questionnaires cannot be applied. Around 20-25 Semi-structured interviews were conducted in different rural areas of Lahore that collected information regarding education background technology used and what problems they face in day to day tasks due to illiteracy. This would fuel our prototype design.

## 3.1 Participants

We recruited the participants for our user research from Lahore where around 60% of the population is illiterate. We conducted 23 interviews of people for the user research phase, with age 18 and above, around which 21 were male while around 2 were female.

## 3.2 Interview Questions

Users were asked 24-26 questions in total and some questions were added or dropped according to a response from the user. First few questions were used to gather data about user such as age, profession, and formal education. The second part focused on a situation where the user were required to read and write words or sentences and the last part of the interview focused on the user's view of technology. Users were asked if they use technology and how much skill they already have with it. Users were asked if that technology required them to touch screen or use tracer pen. Some interactive questions were added to see how much knowledge the user has common signs and symbols.

## 3.3 Findings

The set of findings can be divided into three parts. The first part of the findings is related to the level of literacy and causes. The second part focuses on technology and if they use it and third focusing on how they cope with a situation involving literacy. The user interviewed often seemed to be disengaged with the idea of learning from start and often showed disapproval of the idea of formal education if given chance now. In our interview, we had a blend of people, some who hadn't studied at all, some who had been to school for some time. 73.9% of our candidates were the ones who had never been to any school. 87.5% of them refused when inquired whether they could read or write a small sentence in any language. Similarly, 47.8% of people knew and could recognize numbers when shown to them. 54.5% of our interviewee's told us that they can perform some small basic calculations of addition and subtraction, however, in reality, 23.5% of our candidates answered the sum of 35 and 25 correctly (For their ease we told them the numbers both in English and Urdu). We
showed them some basic symbols of a tick, cross, forward and backward arrows which are generally used in many places. 65.25% of the candidates weren't able to identify any one of the symbols. In order to evaluate the extent to which our candidates are technology literate, we asked them whether they could operate mobile phones, computers. 47.8% of our candidates could use a simple mobile or smart mobile. 21.7% of people couldn't operate any mobile phone. None could operate a computer/laptop.

A question where the user was asked how they do contact finding and adding on their mobile phone received a lot of people saying they use someone for help or use pictures. Some said they use icons or shape of letters and numbers. A common practice among illiterate smartphone user was saving contact name along with a picture. One user said, "I have a smart mobile and for contacts, I have saved, I also have pictures along". Some user also said they use voice recording services on the messaging application. A user said "I don't do messages. I am on WhatsApp and I do voice message. My friends know that since I cannot read, they only reply using voice message".

This developed the base on how to thinking and designing something that could be used to teach these basic tasks to illiterates by technologies they use most. It should have visual and audio cues as well as provide important educational perspective to the user in their daily life with minimal effort required.

## 4 Design And Development

The design phase incorporated the findings in the user research phase with certain features; the solution was an Android mobile application that only uses large icons and voice for interaction. Audio input kept minimalistic and targeted common words and names. Use of color, icons and common cultural signs also incorporated. Another application called Ustaad Tracker was used for tracking the progress of the user and allowing an expert to set the education plan. We added the shortlisted sub-tasks derived from user research in the application. Those subtasks are reading/writing common words, reading/writing numbers, reading/writing custom words, and performing scenario/story based calculation. The major target of the study was to make the user able to read and write his name and the learning was evaluated automatically using accuracy.

To have a better look at the requirements of users, we designed a low fidelity prototype and tested it on 5 participants. Users found the screen flow simple and easy to navigate but some users pointed out repeating voice instruction if no action has been performed for a while.

Inspired by the lessons from the testing with prototype and interview results, we here describe the final android application called "Ustaad" which provides the facility to learn basic and customized words using the phone. The solution is divided into two mobile based applications that would be running on commonly available mobile OS (android). Main Application would be labeled Ustaad, A standalone application meant to be designed for the user. The objective of this application is to make user practice all three tasks which are reading, writing and counting that will help them transition into literate phase. A companion application called Ustaad tracker would be a com-
plimentary application that can be downloaded by the helper of a user to track the process and set an educational plan for the user to follow. Helper is someone related to the user or someone user often goes to for help regarding literacy tasks. The helper can devise an educational plan for the user, picking section they want the user to attempt before moving to next as shown in figure 4. It can be linked to the main Ustaad application through the setting's button.

The Ustaad application (figure 1) was based on a prototype developed before. It would again have 3 sections of reading, writing, and calculation. Each section will have certain subsections. For reading section user will have each icon for learning to read number, words, and name. Each subsection will have a testing phase where the user will have to identify the spoken word by selecting options. For the writing section, a special tracer would be developed. The user would be able to see words, numbers, and name in the form of the dotted line as shown in fig 9 and would be asked to practice. The performance of user will measure and accuracy will be shown after. For calculation, section user will be shown a scenario teaching number based on previous sections. Here the user will be able to see calculations are to be performed and then he will be tested as shown in figure 3.

<figure><img src="image_2.png" /></figure>

## 5 Evaluation

Our evaluation phase consisted of 13 participants with age ranging from 19 to 60 plus with all belonging from Lahore. The participants recruited had no or little formal education with the same social background as participants from the user research phase. A testing methodology followed to find the difficulty faced by users and flaws highlighted by them. The facial expressions of the users and their activity on the application were recorded. We also conducted pre-test and post-test using questionnaires to measure the time spends on each task by the user, recorded the emotional response of user for each task, measure the accuracy of steps user performs in each task and
measure the user's accuracy of tracing words. We recorded the user's facial expressions as well as their activity on mobile application through two different cameras. The user will be asked to sign a consent form and will be informed about the amount of time it will take for completing the test. We briefed them different scenarios and ask them to perform a list of tasks for each scenario by doing so we can evaluate the usability, effectiveness, efficiency and enjoyable of our application. Afterward, a post-test questionnaire will be asked which include question based on the experience of the user from the application and how much learning they achieved.

We measured both quantitative data along with emotional and subjective response of users. The first measurement was the time taken by the user (TTU) to perform certain sections and subsections in minutes. Second important measurement was the user's facial expression during testing. We also calculated how accurately the user was able to perform the given tasks. For this, we divide each task into the fine-grained steps and calculate accuracy by aggregating the correctly performed steps then dividing it by the total number of steps needed to perform that task mathematically shown in figure 5.

$$V_{\text{Tr.usr}}^{\text{acc}} = \frac{\sum_{j=1}^{n} T_{ij}}{\sum_{j=1}^{n} T_{ij}}$$

Where:

$T_i$ = Task i

$n$ = Total number of tasks

$T_{ij}$ = Each step j in task i

$n_i$ = Total number of steps in a T_i

$T_{ak}$ = Each correctly performed step k in task i

$m_i$ = Total number of correct steps in a T_i

$$Accuracy = \frac{I}{max(W, T)}$$

Where:

$T$ = Total numbers of pixels in the filled text

$W$ = Total numbers of pixels in canvas that user traced on

$I$ = Intersection of $T$ and $W$

<caption>Fig. 5.</caption>

<caption>Fig. 6.</caption>

Moreover, we also measured the accuracy of the Urdu text traced by the user when he/she is provided an outline of the text. This will be calculated and logged by the application. The formula for accuracy is shown in figure 6. Accuracy results are shown in table 1.

<figure><img src="image_3.png" /><figcaption>Table 1: Tracing Accuracy Examples</figcaption></figure>

## 6 Results and Discussion

The result section can be divided into 2 major sections: quantitative data gathered by mobile facility based on measurements and user feedback from the post-test interview.
**6.1 Quantitative results**

An average TTU was 30 minutes for each testing session because we had to explain the concept of application testing to the user beforehand and their unfamiliarity with technology added even more time to the equation. For reading time more than 60% of users on average took a minute or less. Majority of user took more time for names and common words than numbers. In the writing section, around 75% of the people were able to complete this task with average TTU less than 2 minutes while the remaining 25% failed to perform the task at all. The calculation section was completed by most users in 2 minutes with average TTU less than a minute.

In emotional responses, the most dominant emotional reactions were Neutral, Confusion, and Joy. Neutral reaction meant that they were able to understand the task pretty well e.g. Learn calculation scenario had 4 neutral reactions. Joy meant that they were able to really enjoy the task at hand; participants really enjoyed writing their own name, reading common words with pictures, and learning calculation with scenarios.

The confusion mostly occurred when the user gets lost in the navigation but some users were confused from the very start till the very end. These totally confused users are those users who seldom use a mobile phone.

In terms of accuracy; the percentage of steps correctly taken by the user to complete a particular task, around 55.6% of the people were able to perform the writing task with 100% accuracy, 11.1% performed it with 75% accuracy, another 11.1% performed it with 50% accuracy and 22.2% were not able to perform the task at all. In reading most of the users were able to identify written numbers correctly but navigation issues were most dominant in reading custom words and common words. More often the user got lost. Around 55.6% of the people were able to perform the calculation task with 100% accuracy, 11.1% performed it with 75% accuracy, another 11.1% performed it with 50% accuracy and 22.2% were not able to perform the task at all.

**6.2 User Feedback**

When we asked the candidate that from a scale of 1 (worst) to 5 (best), how much did the like our application, 23.1% of them rated 5, 38.5% rated 4, 30.8% of them rated 3, while only 7.7% of them rated 2. When asked about the interface of the application we received an extremely positive response as 84.6% categorized the interface in Good and Very Good category. The most user found that writing own name was the most interesting section of the application. Around 91% of users found the application to be of the daily user while some also said there was room for improvement. When it comes to benefits, we had a very positive response as 92.3% believed that they will be able to write their name after using our app, while a 100% believed that they will be able to read and write basic words, numbers, and perform basic calculations. When asked which task they found the most difficult to understand interestingly enough 23.1% of them said write Your Name task. This might be because it involved tracing of their own name which is naturally difficult to convey and perform. In general, around 70% of user found the application easy to use while 30% found it difficult to navigate.
We clearly observed different candidates found the application more suited to them. A user said "Yes if I get a chance, I will dedicate some time for it. I will give 1 to 2 hours daily". People found application as an alternative for formal education which is expensive as well as time-consuming for an adult doing a job to support a family. To the question would they like to learn in future one user replied "Yes, but only if it is limited to a smartphone. If I have to go to a university, then it will be difficult for me because I have kids so I don't have time". His reply explained that for a worker with a family to support, a smartphone application is far suitable as time spent and learning will be according to their affordance.

# 7 Discussion

The inability to read, write, and calculate not only makes illiterates dependent on others but influences their self-esteem and increase their feeling of distress. Our statistics suggest that a significant amount (87%) of the candidates considered literacy as important and are willing to spend some time given the opportunity to learn. At the same time, these individuals are busy with jobs and families so, as some candidates pointed out, formal institution offering years of formal education won't work for them. Unfortunately, not much attention is being paid towards these individuals although they represent a significant part of society.

The Ustaad application is the first of its kind, as targets illiterate adults and teaches them how to read, write and calculate without following any formal education curriculum. It skips the initial alphabet learning and helps the user learn the required words in a more intuitive manner. This method of learning has proven to be very effective as almost all the interviewed candidates agreed that using Ustaad will successfully teach them basic reading, writing, and calculations skills.

# 8 Conclusion and Future Work

All of our candidates agreed that the application will help them get better in reading, writing, and calculation. Most of them found the application delightful to use and would use the app if installed on their mobile phones. Some of the candidates did highlight some usability issues including finding the navigation difficult and confusion in mapping intention to action. Our evaluation process proves that after solving these usability issues, our app can make illiterate less dependent on others as they themselves will be able to read, write and calculate through repetition. The study offers a framework for teaching any language using technology to users belonging to countries with low literacy and a similar socio-economic situation like Pakistan.

Although the work offers a detailed insight to literacy problems in Pakistan along with a possible solution, further work is required in the form a longitudinal study to see the effect of the application on illiterates in a longer period of use.
References

1. Govt redefines literacy for count (Tribune) [Website]. Retrieved from
https://tribune.com.pk/story/1341159/govt-redefines-literacy-count/

2. Literacy Rate of Education in Pakistan 2016 (Archivist Online) [Website]
http://www.archivistonline.pk/literacy-rate-in-pakistan/

3. Literacy Rate in Pakistan Drops by 2% in 2016-17 (Pro Pakistani)[Website]
https://propakistani.pk/2017/05/26/literacy-rate-pakistan-drops-2-2016-17/

4. Pakistan Demographics Profile 2018 (Index Mundi) [Website]
https://www.indexmundi.com/pakistan/demographics_profile.html

5. Wang, Xindi & Karthik Kota, Kesava & Reddy, Kolli & Baran, Denise & Bhatia, Nalin. (2018). Litebox: Design for Adult Literacy. 1-6. 10.1145/3170427.3180654.

6. Friscira, Elsa, Hendrik Knoche, and Jeffrey Huang. "Getting in touch with text: Designing a mobile phone application for illiterate users to harness SMS." In Proceedings of the 2nd ACM Symposium on Computing for Development, p. 5. ACM, 2012.

7. Shah, Syed Zafar Ali, Iftikhar Ahmed Khan, Imran Maqsood, Taimoor Ali Khan, and Yasir Khan. "First-Aid Application for Illiterates and Its Usability Evaluation." In Frontiers of Information Technology (FIT), 2015 13th International Conference on, pp. 125-131. IEEE, 2015.

8. Agarwal, Sheetal & Grover, Jyoti & Kumar, Arun & Puri, Monia & Singh, Meghna & Remy, Christian. (2013). Visual Conversational Interfaces to Empower Low-Literacy Users. 8120. 729-736. 10.1007/978-3-642-40498-6_67. A

9. Remy, Christian & Agarwal, Sheetal & Kumar, Arun & Srivastava, Saurabh. (2013). Supporting Voice Content Sharing among Underprivileged People in Urban India. 8120. 10.1007/978-3-642-40498-6_38.

10. Kumar, Arun, and Sheetal K. Agarwal. "Spoken web: using voice as an accessibility tool for disadvantaged people in developing regions." ACM SIGACCESS Accessibility and Computing 104 (2012): 3-11.

11. Pavlovic, Vladimir I., Rajeev Sharma, and Thomas S. Huang. "Visual interpretation of hand gestures for human-computer interaction: A review." IEEE Transactions on Pattern Analysis & Machine Intelligence 7 (1997): 677-695.

12. Knoche, Hendrik, and Jeffrey Huang. "Text is not the enemy-How illiterates use their mobile phones." In NUIs for new worlds: new interaction forms and interfaces for mobile applications in developing countries-CHI 2012 workshop. 2012.

13. El-Seoud, M. Samir Abou, Islam ATF Taj-Eddin, and Ann Nosseir. "Using handheld mobile system for teaching illiterates." In Interactive Collaborative Learning (ICL), 2014 International Conference on, pp. 446-449. IEEE, 2014.

14. Ahmed, Syed Ishtiaque, Steven J. Jackson, Maruf Zaber, Mehrab Bin Morshed, Md Habibullah Bin Ismail, and Sharmin Afrose. "Ecologies of use and design: individual and social practices of mobile phone use within low-literate rickshawpuller communities in urban Bangladesh." In Proceedings of the 4th Annual Symposium on Computing for Development, p. 14. ACM, 2013.

15. Al-Alaoui, Mohammad Adnan, Mohammad Amin Abou Harb, Zeid Abou Chahine, and Elias Yaacoub. "A new approach for arabic offline handwriting recognition." IEEE multidisciplinary engineering education magazine 4, no. 3 (2009): 89-97.

16. Xindi Wang, Kesava Karthik Kota, Kolli Reddy, Denise Baran, and Nalin Bhatia. 2018. Litebox: Design for Adult Literacy. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18). ACM, New York, NY, USA, Paper SDC12, 6 pages. DOI: https://doi.org/10.1145/3170427.3180654
10 A. Tirmizi et al.

17. Learn Urdu. 2018. A simple and interesting app to learn reading and writing Urdu language. Retrieved Sep 02, 2018 from https://play.google.com/store/apps/details?id=com.LearnUrdu.Free

18. Kids Urdu Qaida. 2015 An application to learn and recognize Urdu Language Alphabets Retrieved Sep 04, 2018 from https://play.google.com/store/apps/details?id=com.suave.urduqaida

19. Kids Urdu Learner: Urdu Qaida, Games and Poems 2018. Urdu learning application for kids. Retrieved Nov 01, 2018 from http://www.overleaf.com/

20. Telecom Indicators (2019) | PTA. Available at: https://www.pta.gov.pk/en/telecom-indicators [Accessed 5 Oct. 2018].