Changing data practices for community health workers:
Introducing digital data collection in West Bengal, India
Joyojeet Pal1 , Anjuli Dasika1 , Ahmad Hasan1 , Jackie Wolf1 , Nick Reid1 , Vaishnav
Kameswaran1 , Purva Yardi1 , Allyson Mackay1 , Abram Wagner1 , Bhramar Mukherjee 1 ,Sucheta Joshi1 , Sujay Santra2 , Priyamvada Pandey3
1 Universityof Michigan, Ann Arbor, MI, USA{joyojeet, adasika, hasanah, jgjwolf, nickreid, vaikam, purvak, atmackay, awag, bhramar, suchetaj}@umich.edu
2 iKure{sujay@ikure.in}, 3 Independent Researcher {priyamvadaonline@gmail.com}

ABSTRACT

Information M anagement, Android, Tablets

In this paper, we present our findings on the experiences of West
Bengal Community Health Workers (CHWs) in transitioning frompaper to tablet- and mobile-based data collection. Throughqualitative interviews, usability testing and timed observations,
we found that efficiency and quality of data collected werecomparable between the use of tablet devices and traditional paper
methods, but data collection performed on smaller mobile phoneinterfaces was less efficient compared to paper. There was no
significant difference in the quality of data collected across all
three modes. In terms of work practices, we found that while
initial interactions with CHWs suggested positive feelings aboutswitching to digital devices, in their actual practices they retained
and preferred the use of paper, and had workarounds tocircumvent the digital data collection process. While there were
foreseeable challenges around individual user experience, such asdevice familiarity, and application interface flexibility, the morecompelling challenge in transitioning CHWs to digital datacollection was organizational. The agency of CHWs withinorganizations, the levels of training with both data practices anddevices themselves, and the sense of comfort that the datacollectors felt with the overall project emerge as important factorsof attention for implementers of new data management practices.

1

CCS CONCEPTS• Human computer interaction (HCI) ➝ HCI design andevaluation methods • Computing methodologies➝ Userstudies.

KEYWORDSODK, M aternal Child Health (M CH), India, Pregnancy,
Permission to make digital or hard copies of part or all of this wo r k f o r
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial a dv a nta ge a n d t ha t
copies bear this notice and the full citation on the first page. Copyrights
for third-party components of this work must be honored. For all o th e ruses, contact the owner/author(s).
ICTD '17, November 16–19, 2017, Lahore, Pakistan© 2017 Copyright is held by the owner/author(s). Pu blic ation rig hts
licensed to ACM.
ACM ISBN 978-1-4503-5277-2/17/11$15.00https://doi.org/10.1145/3136560.3136582

INTRODUCTION

In many parts of the world with health provider shortages,
Community Health Workers (CHWs) play a central role in healthcare provision and often serve as the sole interface between
patients and health services [1, 2]. CHWs typically have limited
formal training, and resource-strapped health care systems
struggle to provide CHWs with the right tools and incentives to beeffective.
Research has shown that in settings where the CHWs are the maininterface to the formal health system, these providers facesignificant stress in their line of work. Problems consistently cited
include difficult work conditions, inconsistent access to medical
supplies and training, communities’ resistance to their counsel,
and lack of career growth [3, 4, 5, 6, 7].
CHWs tend to operate in rural or low-income urbanneighborhoods, are frequently from the communities in which
they work, and often have minimal formal education [8]. As
liaisons between these communities and the formal health system,
CHWs have increasingly taken on the roles of gathering andcommunicating population health data, particularly through digitalmeans. This has resulted in a range of efforts to get CHWs
familiar with utilizing devices such as mobiles, tablets, andcustom-made devices in conducting data collection work [9]. In
this paper, we focus on the technology-aided data collection
aspect of community health work. We present a case study of
CHWs in rural West Bengal in East India, and explore thechallenges and opportunities of introducing a digital datacollection system into their existing workflow. It is worth notingthat these health workers are involved in maternal and child health(M CH) data gathering, which has shown slow progress in thisregion of India [10], particularly among rural communities [11,12].
Introducing CHWs to digital data collection poses challenges, as
is true with any major workplace practice change. But CHWs are
somewhat unique in that their practices are a lot less regulated
than regular clinicians or nurses. They also tend to have lower
education and they typically lack organized labor options or
longer-term contracting, making them susceptible to variable
work practices being imposed upon them. This can lead to serious
consequences, both for the professional work of community
health management, and specifically in the case of digital datacollection, in the quality of data emerging from their newly
instituted practices [6, 13].
This paper builds on the findings of several key studies of digital
data collection in low-resource settings with an evaluation ofchallenges of interface comfort and data integrity situated within
the context of the daily workflow management by CHWs working
for a grassroots health care organization working on M CH issues.
Our work took place in a real-world setting in which the datacollection in the process of this research was part of public healthpractices. We documented the issues in performing digital datacollection, ranging from common user experience challenges
relating to training and comfort with touchscreens, to structuralissues such as CHWs’ agency within organizations they work in.
From a two-year field project with the CHWs, we assessed datacollection accuracy and user experiences through qualitativeinterviews, a quantitative usability experiment, and semistructured survey conducted on digital devices and paper.

2 RELATED WORK
There is a large body of literature examining the role of mobile
devices in CHW programs [9]. Such work proposes thattechnology has the potential to assist both CHW workflows andthe data eventuating from their community interactions, and thateven with limited or no prior technology training, digitaltechnologies simplify data collection at both the point of care andat the aggregative administrative levels [14] and reduce data
falsification [15]. As such, there is a growing expectation for theuse of phones and tablets with mobile device capabilities incommunity health work [16]. Studies of mobile-based datacollection by CHWs have also shown that being able to use
mobile devices in general may influence how empowered CHWs
are to use these devices in their work [17]. DeRenzi et al.
identified six CHW workflows that mobile device implementation
could improve — data collection, training, communication,
decision support, oversight, and promotion of healthy behaviors.
They specifically noted the lack of work evaluating suchinterventions [18]. Electronic forms for mobile data collection byhealth workers have been evaluated for accuracy in other studies
in a lab setting [19] or even for user experience in the field forcertain tasks [20] in addition to general studies that evaluate
multiple mobile textual interfaces for low-literate users [21].
In the field of Information and Communication Technologies andDevelopment (ICTD), an important early thread in the work on
digitization of paper records came through the use of cameraequipped mobile phones (CAM s). One such CAM study
examined the use of mobile devices for data collection in the
microfinance sector and found that mobile phones can be aneffective platform for rural computing applications [22]. The
study also started the conversation on the trainability of rural
workers with limited literacy for the purposes of data collection.
Around the same time, personal digital assistants (PDAs) wereseen as having a role in improving data quality from various
regions of the world through simplified survey management [23].
One such study explored the use of PDAs to collect data for astandardized pediatric care module in Tanzania [24]. The results
showed promising possibilities — using devices for datacollection was easy for people with no prior device experience;and from a data perspective, it reduced deviations by assisting
CHWs through a pre-programmed logic. In addition to assistingwith data collection, research suggests that mobile technology canbe used to increase CHW performance adherence. De Renzi et al.
demonstrated how web- and voice-based feedback offered CHWs

the opportunity to view individualized performance metrics andcompare their data with other CHWs [25]. This feedback systemshowed an increase in CHW performance adherence, measured asincreased client visits [26]. In other words, there is much work toshow that when digital practices are incorporated properly into thepractices of community health work, they are likely to makehealth delivery processes more efficient.
A second body of related work is on digital data collection, andprojects that occurred when consumer mobile devices were justbeginning to expand in computing capability. Open Data Kit(ODK) [27] was one such project that allowed for an open
framework to enable data collection. ODK has led not only to aseries of academic testbeds using text entry, scan functionalities[28], and sensing [29], but also to a number of real-world
deployments and creation of commercial products in this broader
space [30]. ICTD projects have used ODK within various domains
including microfinance, mapping [31] and health caremanagement, ranging from outreach to clinical practice [32, 33]
and general workflow management [34].
In the field of health informatics, and particularly in industrialized
nations with high-resource health facilities, the use of healthtechnology and information systems has been studied closely [35].
The implementation of health technologies for data collection has
been shown to improve health outcomes and care quality [36] andreduce complications as a result of human error [37]. One
common finding relevant to our work of implementing digital datacollection systems in low-resource settings is that organizational
management is, in many cases, the defining factor for successful
technology adoption [38]. Nembhard et al. identified six key
considerations for successful technology implementation — create
opportunities for experimentation and adoption, frame forlearning,
promoteorganizational
identification,usetransformational
leadership,measure performance anddevelopment, and reward implementation efforts [39].
Empirical results on real-world mobile use in clinicaleffectiveness, particularly for CHWs in low- and middle-income
countries, has been relatively spotty, and research is inconclusive
about the actual effectiveness of mobile use at increasing either
the outreach of CHWs or the quality of data coming back from thefield [40, 41].
Despite much work on digital means of information management,
paper is, in many cases, still preferred. A study by Ghosh et al.
[42] highlighted the benefits of paper — including its
conduciveness to easy inscriptions with a physical trace, which
offers persistent visual feedback. Paper offers visibility andtransparency without additional exertion by the user, and is aknown and trusted medium. One perspective is also that it isunlikely that automation of workflows will help an already flawed
CHW system [15]. Ramachandran et al. argued that ICTs are
more useful to CHWs as a means of persuasion and motivation
rather than the tasks that they are meant to automate [43].
M ultiple studies touch upon such organizational and social factorsthat affect adoption and effectiveness of mobile data collection
andinformation
dissemination
toprovidedesignrecommendations [15, 19, 20, 44] or general policy implications[45]. We take a deep dive into such issues, specifically in thecontext of data collection of complex information throughqualitative interviews and observations.

3 WORK SETTING
Our implementation involved creating and deploying a digital datacollection tool to be used by CHWs, and to collect primary dataon usability and data quality.

3.1 Organizations
The research presented here was conducted as a partnership
between the University of M ichigan — represented by researchers
from pediatric neurology, biostatistics, and human–computerinteraction — and iKure, a grassroots health care group operatingin West Bengal. iKure manages a network of community healthproviders throughout the state. It provides medical consultations
and gathers health information, and does its outreach primarily incommunities that do not have immediate access to institutional
health facilities. The two organizations had intersecting goals. The
M ichigan group was seeking to document the prevalence of healthrisk in a low-resource community in the context of maternal andchild health. iKure, which works with M CH data in West Bengal,
India, was interested in exploring the feasibility of incorporating
digital tools for health data collection as part of their work.
A mobile-based digital collection tool was built for the project,
but the quality of data was not compromised for the experimentwith the technical artifact. Separate checks and balances were
designed for the integrity of data for the health risk study, theresults of which have not been published.
We worked primarily in six locations around West Bengal,
detailed in Table 1. All of these locations exist within a 2- to 4hour driving distance from the metropolitan city of Kolkata.
Location
Bhowanipore
Bhai Nagar

Institutional health facility accessDisconnected by riverbed, nearest hospitalthrough single vehicle dirt road in Debra
Access to temporary clinic with two
government nurses, nearest hospital in Debra

Jagannathpur

Local clinic

Godapiasal

CSR health camps in town, access togovernment hospital by bicycle

ChengailCommunity training hospital

Clinics, one medical college plus full-fledged
hospital
Table 1: Locations and Access to Health Facilities

M idnapore

3.2 Tools
From a clinical perspective, the goal of our work was to examine
the viability of digital data entry for M CH community healthprograms. Our team decided to use two survey instruments, apregnancy baseline survey and the Ages and Stages Questionnaire(ASQ), in the usability studies because these tools are frequently
used for clinical purposes in India. CHWs would normallyadminister the questionnaires to assess the health of pregnant
women and to document child development every 2 months
between 12 months and 24 months.
The pregnancy baseline survey consisted of 42 questions inquiring
about pregnant women’s demographic information, reproductive
history, antenatal care, delivery plans, and health status. The
survey was broadly divided into two modules: generaldemographic questions and specific pregnancy-related questions.
The latter module included questions regarding health careexperiences during pregnancy, blood test measurements, and postdelivery information, which add a longitudinal component to thedata collection process. The ASQ was a series of evidence-based
health surveys that have been used in a variety of studies within
the U.S. and India, and have been integrated into clinical practice[46]. ASQ surveys, when administered on paper, are supposed tobe administered exactly as printed, with a very specific flow andlogic that CHWs are expected to follow precisely.
We used the ODK form-building tools to design and implement
these surveys for the CHWs to use on Android tablets andsmartphones. We found the ODK suite to be most desirable forthis research study because the survey forms — once downloaded
onto the ODK Collect application — run completely offline,
which was necessary given the unreliable connectivity in ruralWest Bengal.
Seven child development surveys (one for each 2-month stage
between 12 months and 24 months) were created directly from thethird edition of the Ages and Stages Questionnaire (ASQ-3)
materials. Questions were grouped and color-coded according toperformance-related activities the ASQ desired to assess in thechildren: communication skills, gross motor skills, fine motor
skills, problem-solving and personal–social skills.
The scoring scheme classified children as “at-risk” or “normal” interms of proper growth and development. Although these surveys
had been tested in an Indian context, the researchers realized fromcreating the mobile versions of these surveys that familiarizing
CHWs with these forms would be rather challenging. The childdevelopment surveys had more complex terms and grammaticalconstructions, and were lengthier than the pregnancy baseline
survey.
ODK forms were built in Excel. In developing the surveys, it was
important to ensure that the content was understandable to the
CHWs (linguistically appropriate and containing basic
terminology) and that the survey question flow matched the users’
current understanding of how to collect patient data. The survey
language was iteratively tailored between site visits to improve
the navigability of these forms when the CHWs tested them on themobile devices.
The data collection was conducted on mobile phones, tablets, andpaper. The mobile phones were M icromax Bolts with 4-inch
screens, running Android v4.4 (KitKat), and had 512 M B RAM
and 4 GB internal memory. These phones retail at approximately
US $75 each. The tablets used were iBall Slide 3G 7803Q tabletswith 7.85-inch screens, running Android v4.2 (JellyBean) and had
16 GB internal memory. These tablets retail at approximately US$60 each. An important goal was to study the feasibility of mobile
data collection on low-end, off-the-shelf commercial devices,which would be affordable in low-resource settings. The ODK
forms were built in M arch 2014.

3.3 Community Health Workers
The majority of pregnant women and children from our sample
are served by CHWs as their primary point of contact with thehealth system. CHWs are typically from the communities inwhich they work. Their work involves traveling door-to-door invillages, collecting health metrics, spreading health-related
messages, facilitating referrals, and in the case of certain
infectious diseases — particularly leprosy and tuberculosis —providing medicines and ensuring that the patients take them. The

work is fairly physical in nature because it involves difficult
commutes, and the women must carry all materials they would
need for their visits, including medicines, basic vitals equipment,
and documentation materials such as files and notepads. Data
gathered during field visits are relayed back to community healthcenters located at nodal points. Prior to our field testing, CHWs
were required to keep records of conversations, which they did byhand in ruled books with carbon paper, copying each patient’s
data onto a new page. They did not audio-record their interactions
with clients. The typical method was to do shorthand field notes
and then do proper write-ups later at home.

4

RESEARCH DESIGN

Our research design involved primary research with 10 CHWs
who were involved in this project. We also interviewed staff
members at iKure and ASHA workers. The data collection
methods also involved the recruitment of infants and pregnant
women for the study.

4.1 Preliminary Interviews
We first conducted qualitative research, which occurred in twophases. The first phase of preliminary interviews was with theiKure management staff, ASHA workers, and CHWs tounderstand their motivations and concerns in using mobile
technology for community health work.
We conducted three interviews with leaders of iKure at their
headquarters in Kolkata to gain an understanding of theorganizational motivations for technology implementation.
Following this, we interviewed seven ASHA workers for an
overview of the field setting, because they had been formallydesignated by the government to work in that region and had an
in-depth understanding of the health care concerns for the fieldsites. The ASHAs’ training and experience with rural health careand information management made it easier for them to talk about
the ways that technology had become part of their practices. Allinterviews with ASHAs were semi-structured and oriented toward
understanding their practices, routines, and concerns. Allinterviews were conducted in Bengali, video-recorded,transcribed, and analyzed for key themes. This work took place inM ay 2015.

4.2 CHW interviews and observations
Figure 1: CHWs share instructional materials that help themwith performing health screenings in the field (photo credit
Jackie Wolf)
iKure employs its own CHWs — who are in some cases
Accredited Health Activists (ASHAs). ASHAs are formallytrained Community Health Workers through the government ofIndia who receive comprehensive training in health education andpromotion [47]. In our sample, ASHAs were distinct from CHWs
in that they were trained professionals who had been working
extensively in their respective communities for a number of years,
and were also required, as part of their work, to undergo regularrefresher training provided by the government. iKure’ CHWs on
the other hand, were community members who had been hired for
health care work. For all the CHWs we spoke with, theiremployment with iKure was the first time they had been formally
hired to work in the health care sector (Fig. 1). The CHWs
working with iKure in our sample were hired through snowballed
connections, and were relatively younger than the ASHAs. The
ASHA workers in our study ranged from 27 to 42 years of age.
The CHWs were between 18 and 38 years. CHWs had no priorexperience with using smartphones, but they all had experience
using keypad-based feature phones. All the health workers were
given 1 day of training to get accustomed to the smartphone and
ODK interface and were able to navigate through the forms
several times. The CHWs were required to return the devices forthe data collection work when they were not in use; this meant
they were not able to “play with” the mobiles to gain comfort with
the interface and its affordances, which other studies havesuggested offers important learning benefits to technology users incomparable situations [45].

The second phase of qualitative work involved interviews with
CHWs at three field sites — Bhowanipore, Bhai Nagar, andJagannathpur — which were visited over 3 weeks. Each of theCHWs was an employee of iKure. At the time of our interviews,
the CHWs had undergone a day of training in recent weeks to
become familiarized with the ODK application. This work took
place in June 2015.
We conducted a basic usability test providing design probes andactivities to confirm basic understanding of and ability to use themobile- and tablet-based applications (Fig. 2). CHWs wereintroduced to the differences between paper and electronic
surveys. They were asked to perform sample tasks of registering apatient, completing forms, and more advanced features such as
messaging physicians. Each CHW was assigned a task, following
which we observed their completion of the task and asked fortheir impressions of the ease or difficulty of the task.

Figure 2: CHWs conducting usability tests after being given
written instructions (photo credit Jackie Wolf)

Although there were some minor differences in the tasks tested ateach location for logistical convenience, all the CHWs had thesame pre-test training with the mobile application, and all were
tested on the all the tasks listed in Table 2.
S ite

Tasks
Paper vs. electronic survey
Bhowanipore
Patient registration
Vitals and symptoms recording
Categorizing information into forms
Bhai Nagar
Cancel, Save, Next button logic
End-to-end test with patient registration
Jagannathpur
Case entry workflow
Physician referral logic
CHW-physician messaging features
Table 2: Field Visits for Observations in Chronological Order

4.3 Patient Sampling
CHWs were asked to recruit infants and pregnant women in theircircles — in this case through the six locations surveyed, during
the summer of 2015. All respondents to our survey were restricted
to pregnant women in their first, second, or third trimesters, andchildren ages 12–24 months. These included a total of 279 womenand 368 children, for a total of 647 respondents in the study.
All of the pregnant women were offered Nutrimix, a locally
produced protein and iron food supplement, as a token of
appreciation for their participation. A physician was available at
all times for CHWs to consult as part of the study. Interactions
with the respondents were conducted in Bengali. None of theinteractions with the mothers or children was audio- or videorecorded. In this study, we do not report the outcomes of theextended study, which were used for clinical purposes, but instead
focus on a subset of 150 people for timed observations. This work
took place between July and November 2015 and was conductedby a total of 10 CHWs spread throughout the locations.

4.4 Timed Observations
The timed observations took place after the completion of morethan 200 surveys on mobiles and tablets, to ensure that the
respondents were comfortable filling out forms on a device. Each
CHW was assigned to 15 data collection tasks. For each task, a
CHW completed two or three infant and corresponding adultforms, such that by the end of the test, each individual had sevenof one and eight of another form completed. Thus each CHW
completed five tablet forms, five mobile forms, and five paper
forms. One CHW did one extra form on a tablet. All CHWs had
the same M icromax mobile devices or tablets (Table 3). This
work took place in November 2015.
Data Collection TypeWe conducted two forms of analysis on the timed observations.
First, we wanted to see which of the three data collection
mediums was the most time efficient. Second, we wanted to see
whether there was a difference among the three formats in termsof data quality. We collected the fully completed form in eachcase to examine the results within the larger set of 647 surveys toanalyze the consistency of data quality with the observed cases.

4.5 Second-round User Research
To mitigate potential bias with the qualitative data from CHWs,
when both the international researchers and iKure staff weredeeply involved in the day-to-day functioning of the work, we
conducted a second round of interviews and usability tests withseven of the 10 CHWs who worked on the project, the others
having left for other opportunities. This took place in September
2015. During this round of work, only the primary researcher (alocal Bengali speaker unaffiliated with iKure) and one local
usability tester interacted with the CHWs. Our goal at this point
was to revisit the field site to test the veracity of the first round of
results and conduct the usability study in a low-pressure
environment.
Unlike the observational study in phase 1 (section 4.2 of thispaper), which was conducted in the field, during our second visit
to the field we conducted a structured usability test using both thechild development and pregnancy baseline questionnaires andfollowed up with interviews. The tasks that were tested included
opening the correct form, filling out the form, and saving, editing,and reviewing it.

4.6 Analysis
We used basic descriptive statistics to analyze the results from thetimed observations. We compared average (mean) values througha t-test, and descriptively present medians in box plots. We used agrounded theory approach to work through our qualitative dataand create themes for discussion and theory-building [48]. The
transcripts of interviews and observations were read by threecoders independently and annotated for themes, which werecollated and discussed as a group. Following this, we selected
quotes from the interviews that adequately represent these themes.

5

RESULTS

5.1 Timed Performance across modes
While the sample sizes (n=15 for each CHW) are small and do notallow for a deep comparison, we can see in Table 4 that themobile method was the slowest in average time taken for all the
CHWs except CHW 07. M oreover, the ASQ form for children
was generally longer than the pregnancy baseline questionnaire.

Forms Collected

Infant ASQ (M obile)

20

Infant ASQ (Paper)

29

Infant ASQ (Tablet)

21

Adult Pregnancy Baseline (M obile)

30

Adult Pregnancy Baseline (Paper)

20

Adult Pregnancy Baseline (Tablet)
30
Table 3: Types and Quantities of Data Collection

Looking at the time taken in each mode by the type of form, wefound that the differences between the three survey modes was
highly significant (p=.05) among groups. The box plot in Figure 3
shows that tablets and paper outperformed mobiles for both thepregnancy baseline questionnaire and the ASQ.
The mobile input was the slowest on average for both thepregnancy baseline and children’s surveys. We also found that
paper and tablets were roughly comparable. While in the meantime taken, tablets performed the best, there was a higher variancein the tablets in terms of time taken to complete both thepregnancy baseline and the ASQ surveys. The differences
between the groups was significant (p<0.01).
To examine whether the quality of data is comparable among thethree modes of data collection, we ran a series of tests on key
variables to see whether there was any distinction among thesamples from satisficing behavior by the survey enumerator or
any other form of data entry behavior that might cause the data to
be compromised.
For this, we compared means among the three data collection
types — mobile, paper, and tablet. To do this, three scalevariables were selected, two at the start and one at the end of thesurvey, and their means were compared. Our hypothesis was that
if the data were reliable, there should be no significant difference
in means among mobile, paper, and tablet (unless there were othervariables that caused these differences). The choice of spreadingout the selected variables through the survey was to control for
any fatigue on the part of the survey administrators that mightlead to satisficing toward the end of the survey. Table 5 shows
results for the ASQ surveys.

5.2 Data integrity
We checked the data for accuracy to study whether working on
the tablet or mobile environment led to errors from factors such as
interface unfamiliarity or the pressure of working on a new device
while being observed. We found no effects. None of thedifferences among the means of the variables in Table 5 wassignificant. As we see, gestational age and birth weight are
extremely close among all three categories. This is consistent with
our hypothesis that birth weights and weeks of gestation would
have no differences across large enough samples.
Variable
Gestational Age (weeks)

Mobile(n=318)Paper(n=30)

Tablet(n=20)

37.9

38.2

38.1

Birth Weight (kg)
2.8
2.7
2.7
Current Head Circumference
(cm)
45.2
45.0
45.6
Table 5: Means for S elected Variables Compared by Format(Children’s AS Q Forms)

With the sample of pregnant women, we again sought three-scalevariables spread through the questionnaire that we could examine
for consistency (see Table 6).

S erial No.

MobilePaper

Tablet

CHW01 (n=5,5,5)

21:02

14:11

16:56

CHW02 (n=5,5,5)

19:55

17:49

15:21

CHW03 (n=5,5,5)

18:23

17:58

17:37

Variable

CHW04 (n=5,5,5)

17:43

14:11

12:22

CHW05 (n=5,5,5)

24:27

17:23

CHW06 (n=5,5,5)

23:16

CHW07 (n=5,5,5)

Mobile(n=229)Paper(n=20)

Tablet(n=30)

Woman’s Age (years)

22.0

22.4

21.3

21:58

Weight (kg)

49.7

50.1

47.8

19:21

21:30

16:04

16:43

13:38

Pulse rate (bpm)
81.3
82.5
82.2
Table 6: Means for S elected Variables Compared by Format(Pregnancy Baseline)

CHW08 (n=5,4,6)

21:18

20:56

18:03

CHW09 (n=5,5,5)

20:04

13:59

14:28

CHW10 (n=5,5,5)
20:33
17:31
16:30
Table 4: Mean Time Taken by CHW for Forms – both AS Q
and Pregnancy Baseline combined (Min)
Timed mobile tests were compared to timed paper and tablet tests.

We found no significant difference among these variables acrossthe three input formats. We conclude therefore that the quality ofdata was not inconsistent across the three modes.

5.3 User Experience
When we visited the site in September 2015 to check the progress
on ODK usage, we learned that the CHWs still primarily relied on
paper forms to take health information on site (Fig. 4). This
information would then be input in the mobile forms at a latertime or directly input into Excel sheets at the iKure office by adata entry operator working on a desktop computer. We foundthat the CHWs were concerned about data integrity and preferred
the tangible paper-based forms. This was exemplified during thetimed tablet- and mobile-based data collection tests, where theseven CHWs universally preferred paper as a means for datacollection over tablet and mobile means.

Figure 3: Box plot depicts the differences in median time
taken to complete the surveys in various formats (n=150).

CHWs typically had significant domestic responsibilities at home,having to redo something done at work was difficult — the entire
interaction needed to be captured effectively on the spot.

5.3.2 Training
We found in our preliminary stakeholder interviews andsubsequent first-round CHW observations that iKure’ focus wason swift training since they had to deal with multiple competing
projects where CHWs were expected to serve primary roles.
While training was supposed to include some elements of generaldevice familiarity, we found that CHWs were mainly trained withthe front-end data collection tasks.

Figure 4: CHWs used paper-based data collection when they
had concerns with data integrity (photo credit S andip Dutta)

5.3.1 Data collection logistics
The health data collected by the CHWs is complex in nature. In
parts of the forms, they have to collect contextual data about thepatient and note it in text fields. The form has fields for patient
information, health history, and many yes-or-no questions about
patient health. There are also fields that ask the CHW to mark thecurrent condition of the pregnant women. This makes the forms
complex, and the cost of getting something wrong high, thus
CHWs preferred paper, validating Ghosh et al.’s [42] findings.
M argins on paper forms were used to explain issues; these in turn
also became the CHWs’ defense for claiming that they had fully
reported the issues.
If mistake on paper I have to cut and write again and forphone cross it and write. On paper I can see things which
are correct and which are incorrect. For phone I can also
see but it will take time but for paper I can see it at a glance.
— CHW 3A related problem was the logistics of typing in English,
especially given the CHWs’ limited command over the language.
CHWs were Bengali speakers, and all the forms were eventuallycreated in Bengali script. However, where text responses (insteadof check buttons) were required, these had to be typed in English.
Given the limited extent of English literacy, as well as limited
awareness of digital artifacts built for English-language users suchas position of letters on virtual keypads, we found that CHWs
experienced a significant cognitive load in translating this largely
contextual data into English — resulting in slow typing tasks(mean time = 18.5 secs for 10 characters) and more spelling errors(mean number of spelling errors = 2.8 errors per CHW per form).
Clicking all options are quick in mobile … but in mobile
while typing error occurs and after reaching home if you see
the errors then you have to go again to the house you went
before. English alphabets are main problem while using
mobile. — CHW7
In trials, we found that frequently data had to be repeated multiple
times for the CHWs to capture it effectively or correctly because
there were no auto-complete or pre-filled terms, units of measure,or dates, further adding to the cognitive load of spelling out terms
or means for form validation to check for internal inconsistencies.

This created two problems — first, not including advanced ODKfunctionalities into training meant that the CHWs did not know
how to address application-level problems, or make minor
alterations to the package. Some ability to make minor
amendments to the forms would have been helpful in adding
unexpected responses to questions. We found during the secondround user experience tests that none of the seven participants
could save the form in the current state, rename the form or go to
the start of the form. Only one participant could send thequestionnaire to the server, and only two could delete thepregnancy baseline form. All the CHWs at this point had pastexperience with using ODK on some devices, and yet, working on
something outside of the very precise steps one used in an
application was challenging.
The more important challenge, however, was that of generaldevice familiarity. None of the CHWs owned a smartphone during
the first phase of the test, and one purchased one in the process of
the work. CHWs were unfamiliar with the basic functioning of thedevice. For instance, P2 did not know how to switch the keyboard
from alphabets to alphanumeric. Fine gestures, like the tap andhold to move the cursor to a specific location, were unknown tofive of the seven CHWs. In the case of errors, all five CHWs
deleted all the characters before re-typing them instead of justretyping the erroneous ones.
The limited training with device use led to uncertainty on whether
one could recover from errors. We found on multiple occasions
that basic touchscreen operations would confound the CHWs,
leading either to long-winded solutions or resets. The iKurephones for the project were taken away from the CHWs at the endof each day, and they were allowed to use them only on days
when they had data collection tasks. We found that outside of our
joint project, the CHWs were going about their regular work using
paper, so the only time they entered data on a mobile device wason the days they were working on this project.
Forms like ASQ are carefully designed to have an optimal flow on
paper. CHWs had familiarized themselves with the forms and thelinear flow of questions in the pregnancy and child questionnaires,but did not have a greater understanding of the form logic.
Building expertise with a specific type of data collection wastranslated as learning and anticipate flow on a form and moving
along accordingly. This meant that periods without practice using
a specific form led them to fumble during the first few iterations
back. Since CHWs also work with different populations and notjust mothers and children, remembering the logic and flow ofindividual forms is not an optimal strategy of technology
familiarization.

In fact, when I was continuously using it earlier during the 3months of the survey, I had gotten used to it with practiceand was becoming comfortable. Now I am out of touch using
it so I am having problems again. — CHW2
Knowing an application’s functions just at the surface level has
negative consequences on interface comfort since the individual
does not have a deeper understanding of the entire process. We
see this with patient matching, which ODK forms do not allow.
Once the data are collected and sent to the server they no longer
reside on the phone. When the CHWs collect the informationfrom the same women at different junctures, someone in the iKure
office has to spend time matching the new data against what were
captured previously. This matching is hard, especially if the nameis spelled differently in each instance, which is a common
problem with typing Indian names in English.
During the usability test, this was logged as a minor error.
However, in the health care context, this is a very serious error.
From iKure’s perspective, this was a major problem because of
the lack of patient-matching functionality and the potential of
mismatched data records. Training (or ongoing practice) did notgive CHWs an intuitive sense of what errors to watch out for, nor
did they internalize what errors were more important than others.
Consequently, turning to paper collection allowed a material formof verification in which the CHW could hand over to higher
authority the work they had done, wherefore checking for errors
as a responsibility had been passed on.

no iKure representatives in their immediate vicinity, the CHWs
were concerned about making mistakes, and whether these may
get reported to their employers and the impact that may have on
their employment. The refrain of “we are testing the system, notyou” was insufficient in this cultural context.
iKure also supported the practice of having paper as a backup. As
a company working with larger health corporations or
governments involved in health initiatives, their priority was
reliable data rather than the means by which data were gathered.
Validation of data input in the forms was intermediated bysomeone at the iKure office. It was verified against information
captured in paper forms and then sent to the server to avoid
incorrect data. However, the reasoning behind this process was
not clearly communicated to the CHWs. This further prevented
the CHWs from developing confidence in their data entry abilities
and reinforced their reliance on paper forms.
We want both mobile and copy ... I can write in copy at fieldand can enter all details in mobile at home ... what is very
helpful when you use mobile ... I don't have write ... I justhave to tap on phone screen ... copy is important aspermanent entry as it won't delete and for mobile it's quick toenter the data. — CHW7
From the interviews, it was evident that the CHWs had some
intrinsic motivation to use the ODK application and forms. They
had a desire to be able to use smartphones and practice English,
possibly to help them get hired as ASHAs.

5.3.3 Organizational issues
The CHWs working with iKure are typically hired throughconnections, are relatively younger than the formally trained
ASHAs. CHWs are typically from extremely poor families, nearor below the poverty line themselves, often heavily dependent on
their income. In our early interviews, we found that ASHAs
typically had both a deeper understanding of the health carescenario and a more fundamental appreciation of the nature andintent of health information gathering. Lacking an appreciation of
the bigger picture impacted both how the CHWs approached thework, and their ability to appreciate their own value in theprocess. While ASHAs thought of themselves as healthprofessionals providing a service to the community, CHWs saw
themselves as employees of a company carrying out data tasks
assigned to them.
During the first training session, several of the CHWs had to bereassured that they could not break a tablet by touching it. Even in
our second phase of user experience work, the CHWs were
extremely cautious with the devices, not doing anything outside of
the scripted work of form filling, concerned they may cause
damage the devices.
Along with the fear of technology, there was also deference toauthority, which we had noted earlier, prompting a second roundof user experience research. We found in both rounds of work thatwhenever interviews or usability tests were conducted in theiKure office, CHWs were wary of authority figures. This is not
unusual behavior, since usability tests of any kind can be difficult
to participate in when being watched over by one’s supervisors.
Even outside of those in-office interactions, it was hard toconvince the CHWs that we were attempting to understand thefunctioning of the systems, and disaggregating their employers
from our presence. Consequently, even in usability tests that had

It would be better to use the phone since we are learning to
use this new instrument and wish to learn more and use thistechnology. — CHW4
The patient perceptions of the phone being used also affected
CHWs’ inclinations to use the ODK application. On one hand,validating Ramachandran et al. [43], phones granted them moreauthority in the eyes of the patients. However, because thepatients were themselves not too familiar with phones, they could
not review the data that had been captured to make sure it wascorrect. One CHW also noted feeling pressured if she took longer
to fill out the mobile ODK form in front of the patient.
In the paper method, the person being interviewed, thevillager, is familiar with it and sometimes they go throughwhat we have written and give us a feedback or confirmationthat the entry is correct. Most of the people being
interviewed is not very comfortable/familiar with the phone
so they cannot go through what we do. — CHW46DISCUSSION

The ecosystem of digital data entry for CHWs in West Bengal
involves altering existing practices and, in many ways,
fundamentally changing the CHW workflow and relationships
with communities. CHWs across West Bengal agreed that fieldvisits take long to complete and are physically taxing, particularly
given the need to carry around large stacks of paper for datacollection. While implementations of digital data collection
systems are often driven by the need for better data, we foundhere that a significant driver and area of concern for CHWs intheir successful use of these tools is empowerment.

6.1 Empowerment
We define empowerment based on whether CHWs felt more orless confident in their work with digital data collection. Our
results indicate a multitude of factors played a role in CHWs’
empowerment (or lack thereof) to use digital devices in their
work.
CHWs’ beliefs about technology drove expectations about what
mobiles and tablets could do for them. In conducting the CHWinterviews (section 4.2), we observed how the general use oftablet devices increased the feeling of empowerment among
CHWs, as noted by Ramachandran et al. [43]. For instance,
CHWs repeatedly stated that one of the potential benefits of thedevices was the ability to go back to any record in time and haveendless storage spaces, thereby enabling the CHW to better
manage their data deliverables to the organization. All 19 healthworkers (CHWs and ASHAs) that were interviewed indicatedvery positive views of technology, and we believe this was partly
due to their understanding of technology usage as a connection tobetter employment opportunities.
Results from our timed observations (section 5.1) indicate thattablets were just as efficient as paper-based data collection andthat CHWs performed faster on tablets than on mobile devices.
The larger interface allowed easier navigation of forms. Although
we found higher variance in the time taken for data collection,
tablet-based data collection was slightly more time efficient thanpaper-based collection. An additional factor to consider withpaper data collection is the time taken to input the data into asystem, such as a server, which was not a factor with the mobile
or tablet devices. We also found here that the data accuracy
coming from all three mediums in the first round of testing was
highly comparable. Data quality was consistent between tabletand paper-based collection, and seemed to support tablet
implementation as the primary method of data collection, topositively impact on CHW workflow.
Despite the high potential for tablet implementation after the firstphase of research, the second round of user research revealed that
CHWs felt less empowered with tablet-based data collection, for
two major reasons. The two themes, presented next, bring forthnuance into our initial determination that tablet-based datacollection is the obvious next step for digital data collection inrural settings.

6.2 Organizational Structure and Management
We found that the position of CHWs within the iKureorganization significantly impacted their motivation and ability tofeel empowered in their adoption of digital data collection. In
conducting our second-round user research (section 4.5), we
found that an important challenge during this stage ofinterviewing was the influencing presence of the research teamand stakeholders from iKure. This was not due to any intent fromthe iKure staff to control the CHWs, but likely attributable to arange of issues that we were unable to examine in depth such as
the work culture between employees and management, the urban–
rural relations between CHWs with limited formal education andthe urban officers, or the contractual nature of work that createdgreater exposure to employment termination. The addition offoreign researchers apparently studying every move during
usability tests only added to the nervousness of CHWs.
Power issues were evident during the start of the field trials. The
CHWs were deferential in the presence of administrators fromiKure during interactions in both Kolkata and in the field. As aresult, we found that during early field interviews, after the CHWs
had received training, they were uncomfortable holding thedevices, particularly when there was a supervisor from the head
office on location, irrespective of supervisors’ instructions to
CHWs to be more comfortable with the devices.
An example of how these factors around the work culture andpower relations created biased responses from CHWs was
revealed upon conducting our second-round user research. As
mentioned, we were surprised to learn that CHWs circumvented
the intended workflow of using tablets for data collection. M ore
specifically, all CHWs reverted to using paper for data collection
when out in the field, and manually typed the data into the tablets
upon returning to the iKure office — essentially doubling theirown workload.
We would not have made this observation had we elected to not
conduct the second user study — and we, therefore, credit themasking of this issue to CHW power dynamics. However, we
believe the core of this issue is rooted in a problem with CHW
training, which we discuss in the next section. What this case does
highlight is the problem of short-term field research engagements,
which are often common in HCI4D or ICTD projects that are
constrained by time and a disconnect between the on-the-groundworkers and the larger organizations that fund them. This scenariopresents a case for deeper engagement of design research inlonger-term changes that are made to communities through design
innovation.
As it pertains to organizational management, we noticed that usercentered design principles were not a high priority for iKure in
designing and implementing technology into CHWs’ daily roles.
Design-wise, CHWs were not involved in the creation of the ODKapplication and its interface, and there was significant discussion
within the iKure team about whether the application needed to be
written in Bengali script. Initially there was an interest to build thetools in English for easier transferability to other non-Bengali
speaking states where iKure is active as well as to ensure thestandardization of medical terminology. However, CHWs hadvery limited English-language skill beyond numerical literacy andnot sufficiently familiar with medical terminology to collect allnecessary data. Herein lies a challenge between designing themost usable product, and ensuring the right compliance with data
and terminologies that are essential in a healthcare data collection
exercise. Although the question and instructions of the forms were
eventually implemented in Bengali, input was still done with aRoman script keypad, thus there were challenges with free-text
entry in English, as mentioned. While we expected this to posechallenges, we felt having the forms in a script the CHWs could
read would minimize the cognitive overload of working in alanguage in which one had only basic familiarity.
We found that the frustrations experienced by CHWs in copingwith language barriers were significant. As for the rollout of thetechnology, minimally involving CHWs in the decision-making
process to implement technology, and preventing them from using
the devices on a regular basis, left them separated from the larger
logic of what was driving the organization’s design decisions.
While this may seem unfamiliar to the Western context, in which
new data practices in the health sector often include an involved
process of consultation, the separation of such decisions from theimplementers is not uncommon in India.
iKure’s concerns with safeguarding the devices came from pastexperience of damage or loss, and that CHWs are hired oncontract — with the expectation of a certain among of churn ofemployees. Recovering devices from CHWs, who are primarilyrural residents, involved a significant effort overhead for theorganization. Yet, this process of limiting device access restrictedbuy-in from the CHWs to use the technology, which Creswell and
Sheikh pinpointed as a key element for successful technology
adoption [38]. Rollout of the application took place in a top-dow n
fashion, and CHWs were introduced to the technology as though
it were a new requirement of their job, rather than an innovative
tool that CHWs could use to transform their work and ease theirworkload.

6.3 Training
CHW training was limited due to resource shortages, and could beinconsistent depending on whether CHWs were hired as a groupand trained together or whether they came in later in the processof a data collection project.
Our interview results indicated that the iKure tablets were many
CHWs’ first exposure to any form of mobile technology. M any
actions, such as typing, swiping, and scrolling on a mobileinterface were challenging. CHWs also encountered problems
when they clicked on the wrong text (on the text for “Age” insteadof the text box), and faced unfamiliar drop-down features.
There were also nonverbal indications that the health workers
were unfamiliar with mobile devices: during each test, the women
would crowd closely around the device to observe each healthworker go through the usability test, and whenever the researchteam instructed the participant to “swipe,” “scroll” or type on themobile device screen, the health workers surrounding her would
make the gestures in the air, either to guide the participant in
performing these actions, or to practice scrolling and swiping
themselves. In other words, the social elements of learning a newtechnology are valuable in early technology training, since theCHWs appeared to learn from one another.
We also found that CHWs were uneasy when there was no
response (N/A or N/R) because they felt that N/A responses wereakin to leaving the form incomplete. They were afraid of being
penalized by their supervisors for incomplete work.
Adaptation to the interface took a while, and common errors werecreated by the lack of exposure to forms through other digital
media such as computers or touchscreens. By extension, some of
the typical “intuitive” elements such as “forward,” “back,”“home” and “error” buttons, or features such as auto-correct, werelikely to be entirely new to the users. During the first training
session, several of the CHWs had to be reassured that they could
not break a tablet by touching it. Digital keyboard responsivenesswas among one of the early familiarity challenges; many CHWs
accidentally deleted entries entirely by going “back” when they
were not sure they had typed material in.
CHWs also expressed nervousness about carrying the device bythemselves to the field, particularly because they felt a decreased
sense of control over errors. As mentioned in 5.3, CHWs
preferred paper because they could view corrections and makeedits for later consideration, but felt they could not do so using
tablets. This, coupled with the fact that CHWs were only given
front-end application training demonstrates how CHWs were
never provided a comprehensive understanding of how to use
these tools confidently in their fieldwork. Furthermore, we
observed that iKure did not offer consistent training and that thestaff seemed to rarely check in on CHWs to inquire how theirwork was going. Because of the lack of control CHWs felt as aresult of insufficient training, there was an overall sense of
disempowerment expressed in using tablets for digital datacollection.
In spite of these obstacles, all of the health workers who weinteracted with in the study felt that they would be able to manage
the technical elements of the data collection once properly trained.
We know this to be accurate based on the follow-up timed
observations where we found that at least on tablets, CHWs
performed at par or better than on paper. Furthermore, we foundthat the vast majority of the technical familiarity challenges werequickly resolved with some practice. We also found that thefamiliarity issues with M CH medical jargon, specific in the ASQ
forms and the Pregnancy Baseline questionnaires, were
sufficiently resolved as CHWs gained familiarity with the terms.
In the second-round study, CHWs had reverted to paper-based
data collection while in the field, which we understand was due to
CHWs’ distrust in the data being safely and completely
delivered through the cloud if the devices were used in the field.
We believe this distrust could have been a direct result of
insufficient training.
On one hand, CHWs could have minimal trust in cloud-based datadelivery simply because of their limited understanding of how it
works. From conducting our interviews, we learned that CHWs
did not have a nuanced understanding of device capacity,
network, and storage.
A second reason for mistrust of cloud-based data delivery is thelack of a delivery receipt or a tangible confirmation that the
information was saved. Without knowing with confidence
whether it was successfully uploaded to the cloud, CHWs felt
uneasy using tablets. In comparison, CHWs felt more comfortableusing paper and being able to physically submit documentation
containing patient health information. Due to insufficient training,
however, CHWs were not aware that receipt or confirmation
functionality could be built into further versions of theapplication. From the De Renzi et al. studies [24, 25], we
understand that CHWs are willing to utilize a distinct web-based
platform to monitor and improve their performance, whichsuggests that CHWs would not only be willing to learn how tocheck for delivery confirmation should it become available in an
updated version of the ODK application but would also utilize thisfeature to improve their performance in data collection.
This further demonstrates that consistent and comprehensive
training could serve as a vehicle of empowerment in CHWs’ useof tablets for data collection.

7

CONCLUSION & LIMITATIONS

iKure's model of engagement with the communities where theirhealthcare interventions take place is to leverage existing
community members rather than bring in external healthprofessionals. This creates new forms of health care access thatwere previously non-existent in many of the rural settings studied,but also brings new challenges for examination on managing
processes that are new for the citizen, the community healthworker, and the system that manages it all.
The work presented here is in the limited scope of a M aternalChild Health (M CH) data use scenario. Other scenarios in which
the data to be collected are either harder to do without additional
aids (visuals) or expertise on part of the CHW (training,awareness of symptoms and follow-up questions), or involve
conditions that have a different set of social sanctions (sexuallytransmitted diseases) present unique study scenarios. Thus, theresults from this study are proposed as relevant primarily to the
M CH community.
This project was aimed at sampling pregnancy and earlychildhood health concerns in rural West Bengal and involved
digital technology as data collection tools. The mixed-methods
approach of studying the benefits and challenges of collecting thedata on mobile devices gives us a look at the ways in which
technology impacts community health work. The work alsoenables contextual understanding of some of the motivations andconcerns of CHWs that impact implementation in the short run,but reveals potential consequences for the uptake and use oftechnology over the long term. Studies of longer duration could
help better elucidate acceptability of different data collection
methods, along with the quality of data collected.
The study shows that initial concerns of CHWs not being able tonavigate technical interfaces can be offset with training. It isreasonable to expect that CHWs can match the speed of datacollection on paper without the loss of data quality if training is
consistent. However, the study also underlines the perils ofbuilding technology without adequately gaining the buy-in of theintended end users. While running projects by fiat may workwhen organizations can exercise sufficient agency over theiremployees, such approaches also risk unraveling when newtechnologies are not implemented or designed for those who use
them.
Finally, this work reiterates a growing concern among design
professionals that for new technologies to be successful in realworld settings, designers need to step up and dig past participantbias or satisficing responses on the value of technology by theirusers [49]. Understanding the value of new technology in doing
an existing task more efficiently requires examining closely how
it affects users beyond the short scope of a usability study andwithin their ecosystem of practices [50].

8ACKNOWLEDGMENTS

This study was funded by the Trehan Foundation through the
University of M ichigan M Cubed Diamond program. We are
grateful to the support of the community health workers whocompleted the interviews. Authors would further like to thankSandip Dutta, Aparna Ghosh, Sharmila Chatterjee, Tirumala
Santra, and Sreyashi Dey for their inputs and assistance at various
points during this project.

9

REFERENCES[1] L. Chen et al. 2004. Human resources for health:
Overcoming the crisis. Lancet 364, 9449, 1984-1990.
[2] A. Haines et al. 2007. Achieving child survival goals:
Potential contribution of community health workers. Lancet
369, 9579, 2121-2131.
[3] Z. Haq, Z. Iqbal, and A. Rahman. 2008. Job stress amongcommunity health workers: A multi-method study from
Pakistan. Int. J. Mental Health Syst. 2, 1, 1.
[4] M . Kok and S. M uula. 2013. M otivation and job satisfactionof health surveillance assistants in M wanza, M alawi: An
explorative study. Malawi Med. J. 25, 1, 5-11.
[5] J. M . Zulu et al. 2014. Integrating national community-based
health worker programmes into health systems: A systematic
review identifying lessons learned from low-and middleincome countries. BMC Publ. Health 14, 1, 1.

[6] A. Brunie et al. 2014. Keeping community health workers in
Uganda motivated: Key challenges, facilitators, and preferred
program inputs. Glob. Health Sci. Pract. 2, 1, 103-116.
[7] E. Sarin et al. 2016. Identification of challenges and needs toimprove community health workers performance: Narrativesof Accredited Social Health Activists (ASHA) from two
Indian districts. J. Publ. Health Dev. Count. 2, 2, 173-182.
[8] A. K. Rowe et al. 2005. How can we achieve and maintainhigh-quality performance of health workers in low-resource
settings? Lancet 366, 9490, 1026-1035.
[9] R. Braun et al. 2013. Community health workers and mobile
technology: A systematic review of the literature. PloS One
8, 6, e65772.
[10] S. K. Bhattacharyya et al. 2008. M aternal mortality in India:
A 20‐year study from a large referral medical college
hospital, West Bengal. J. Obstet. Gynaecol. Res. 34, 4, 499503.
[11] A. Biswas et al. 2003. Status of maternal and new born careat first referral units in the state of West Bengal. Ind. J. Publ.
Health 48, 1, 21-26.
[12] A. Dutta, R. Khasnabis, and S. Banerjee. 2016. Healthreform and infant mortality in West Bengal, India: An untold
story. J. Asian Publ. Pol. 2016, 1-17.
[13] J. Tendler and S. Freedheim. 1994. Trust in a rent-seeking
world: Health and government transformed in Northeast
Brazil. World Dev. 22, 12, 1771-1791.
[14] W. Karlen and C. Scheffer. 2014. Design of an interactive
medical guideline application for community health workers.
In Proceedings of the 36th Annual International Conferenceof the IEEE Engineering in Medicine and Biology Society.
IEEE.
[15] M . Tomlinson et al. 2009. The use of mobile phones as adata collection tool: A report from a household survey inSouth Africa. BMC Med. Inform. Dec. Mak. 9, 1, 51.
[16] P. Codyre. 2014. Will an app fill the gap? Innovativetechnology to provide point-of-care information. Front. Publ.
Health, 2.
[17] A. Strauss and J. Corbin. 1998. Basics of Qualitative
Research: Techniques and Procedures for Developing
Grounded Theory. Sage Publications, Thousand Oaks, CA.
[18] B. DeRenzi et al. 2011. M obile phone tools for field‐based
health care workers in low‐income countries. Mt. Sinai J.
Med. 78, 3, 406-418.
[19] S. Patnaik, E. Brunskill, and W. Thies. 2009. Evaluating theaccuracy of data collection on mobile phones: A study offorms, SM S, and voice. In Information and Communication
Technologies and Development (ICTD), 2009 International
Conference on. IEEE, 74-84.
[20] I. M edhi, M . Jain, A. Tewari, M . Bhavsar, M . M athekeFischer, and E. Cutrell. 2012. Combating rural childmalnutrition through inexpensive mobile phones.
In Proceedings of the 7th Nordic Conference on HumanComputer Interaction: Making Sense Through Design. ACM
Press, New York, NY, 635-644.
[21] I. M edhi, S. Patnaik, E. Brunskill, S. N. Gautama, W. Thies,
and K. Toyama. 2011. Designing mobile interfaces fornovice and low-literacy users. ACM Transactions on
Computer-Human Interaction (TOCHI), 18, 1, 2.
[22] T. S. Parikh et al. 2006. M obile phones and paper
documents: Evaluating a new approach for capturing
microfinance data in rural India. In Proceedings of the
SIGCHI conference on Human Factors in computing
systems. ACM Press, New York, NY.

[23] K. Shirima et al. 2007. The use of personal digital assistantsfor data entry at the point of collection in a large household
survey in southern Tanzania. Emerg. Themes Epidem. 4, 1, 1.
[24] B. DeRenzi et al. 2008. E-IM CI: Improving pediatric healthcare in low-income countries. In Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems. ACM
Press, New York, NY.
[25] B. DeRenzi et al. 2017. Supporting community healthworkers in India through voice-and web-based feedback. In
Proceedings of the 2017 CHI Conference on Human Factors
in Computing Systems. ACM Press, New York, NY.
[26] B. DeRenzi et al. 2016. Closing the feedback Loop: A 12month evaluation of ASTA, a self-tracking application for
ASHAs. In Proceedings of the Eighth International
Conference on Information and Communication
Technologies and Development. ACM Press, New York, NY.
[27] C. Hartung et al. 2010. Open data kit: Tools to build
information services for developing regions. In Proceedingsof the 4th ACM/IEEE International Conference on
Information and Communication Technologies andDevelopment. ACM Press, New York, NY.
[28] N. Dell et al. 2013. Integrating ODK scan into thecommunity health worker supply chain in M ozambique. In
Proceedings of the Sixth International Conference on
Information and Communication Technologies andDevelopment: Full Papers-Volume 1. ACM Press, New
York, NY.
[29] R. Chaudhri et al. 2012. Open data kit sensors: M obile datacollection with wired and wireless sensors. In Proceedings ofthe 2nd ACM Symposium on Computing for Development.
ACM Press, New York, NY.
[30] K. Chen et al. 2012. Shreddr: Pipelined paper digitization for
low-resource organizations. In Proceedings of the 2nd ACM
Symposium on Computing for Development. ACM Press,
New York, NY.
[31] A. Signore. 2016. M apping and sharing agro-biodiversity
using Open Data Kit and Google Fusion Tables. Comp.
Elect. Agri. 127, 87-91.
[32] Y. Anokwa et al. 2012. Design of a phone-based clinicaldecision support system for resource-limited settings. In
Proceedings of the Fifth International Conference on
Information and Communication Technologies andDevelopment. ACM Press, New York, NY.
[33] E. W. Zegura, E. Derkits, and J. Cooper. 2015. See my work:
Sustaining a data reporting practice by mental healthclinicians in Liberia. In Proceedings of the SeventhInternational Conference on Information andCommunication Technologies and Development. ACM Press,
New York, NY.
[34] W. Brunette et al. 2013. Customizing and improving medical
workflows using ODK survey. In Proceedings of the 4th
Annual Symposium on Computing for Development. ACM
Press, New York, NY.
[35] C. Free et al. 2013. The effectiveness of mobile-healthtechnologies to improve health care service delivery
processes: A systematic review and meta-analysis. PLoS
Med. 10, 1, e1001363.
[36] B. Chaudhry et al. 2006. Systematic review: Impact of healthinformation technology on quality, efficiency, and costs ofmedical care. Ann. Intern. Med. 144, 10, 742-752.

[37] R. Kaushal, K. N. Barker, and D. W. Bates. 2001. How caninformation technology improve patient safety and reduce
medication errors in children's health care? Arch. Pediatr.
Adolesc. Med. 155, 9, 1002-1007.
[38] K. Cresswell and A. Sheikh. 2013. Organizational issues in
the implementation and adoption of health information
technology innovations: An interpretative review. Int. J.
Med. Inform. 82, 5, e73-e86.
[39] I. M . Nembhard et al. 2009. Why does the quality of healthcare continue to lag? Insights from management research.
Acad. Manag. Persp. 2009, 24-42.
[40] D. Peiris et al. 2014. Use of mHealth systems and tools for
non-communicable diseases in low-and middle-income
countries: A systematic review. J. Cardiovasc. Translat. Res.
7, 8, 677-691.
[41] S. Agarwal et al. 2015. Evidence on feasibility and effective
use of mHealth strategies by frontline health workers indeveloping countries: Systematic review. Trop. Med. Int.
Health 20, 8, 1003-1014.
[42] I. Ghosh et al. 2015. The persistence of paper: A case study
in microfinance from Ghana. In Proceedings of the SeventhInternational Conference on Information andCommunication Technologies and Development. ACM Press,
New York, NY.
[43] D. Ramachandran et al. 2010. M obile-izing health workers inrural India. In Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems. ACM Press, New
York, NY.
[44] M . M olapo, M . Densmore, and B. DeRenzi. 2017. Video
consumption patterns for first time smartphone users:
Community health workers in Lesotho. In Proceedings of the2017 CHI Conference on Human Factors in Computing
Systems. ACM Press, New York, NY, 6159-6170.
[45] A. Schwartz, M . Bhavsar, E. Cutrell, E., J. Donner, and M .
Densmore. 2013. Balancing burden and benefit: Nonprescribed use of employer-issued mobile devices.
In Proceedings of the Sixth International Conference on
Information and Communications Technologies andDevelopment: Notes-Volume 2. ACM Press, New York, NY,
140-143.
[46] S. B. M ukherjee et al. 2014. Incorporating developmental
screening and surveillance of young children in officepractice. Indian Pediatr. 51, 8, 627-635.
[47] P. Hota. 2006. National rural health mission. Indian J.
Pediatr. 73, 3, 193-195.
[48] Strauss, A. and J. Corbin, Basics of qualitative research:
Techniques and procedures for developing grounded theory.
1998: Sage Publications, Inc.
[49] N. Dell et al. 2012. Yours is better!: Participant response bias
in HCI. In Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems. ACM Press, New York, NY.
[50] J. Pal 2017. CHI4Good or Good4CHI? In Proceedings of the2017 CHI Conference Extended Abstracts on Human Factors
in Computing Systems. ACM Press, New York, NY.