JMIR MEDICAL INFORMATICS

Kallakuri et al

Implementation Report

Learnings From Implementation of Technology-Enabled Mental
Health Interventions in India: Implementation Report
Sudha Kallakuri1, MSc; Sridevi Gara1, BE; Mahesh Godi1, BE; Sandhya Kanaka Yatirajula1, PhD; Srilatha Paslawar1,
MPH; Mercian Daniel1, PhD; David Peiris2,3, MBBS, MIPH, PhD; Pallab Kumar Maulik1,3,4,5,6, MSc, MD, PhD
1George Institute for Global Health, New Delhi, India

2

George Institute for Global Health, Sydney, Australia

3

Faculty of Medicine, University of New South Wales, Sydney, Australia

4

Department of Brain Sciences, Imperial College London, London, United Kingdom

5

Prasanna School of Public Health, Manipal Academy of Higher Education, Manipal, India

6

George Institute for Global Health, London, United Kingdom

Corresponding Author:
Sudha Kallakuri, MSc
George Institute for Global Health308 Elegance Tower, Third Floor
Plot No 8, Jasola District Centre
New Delhi, 110025
India
Phone: 91 11 4158 8091
Email: skallakuri1@georgeinstitute.org.in

Abstract
Background: Recent years have witnessed an increase in the use of technology-enabled interventions for delivering mental
health care in different settings. Technological solutions have been advocated to increase access to care, especially in primary
health care settings in low- and middle-income countries, to facilitate task-sharing given the lack of trained mental healthprofessionals.
Objective: This report describes the experiences and challenges faced during the development and implementation oftechnology-enabled interventions for mental health among adults and adolescents in rural and urban settings of India.
Methods: A detailed overview of the technological frameworks used in various studies, including the Systematic Medical
Appraisal and Referral Treatment (SMART) Mental Health pilot study, SMART Mental Health cluster randomized controlled
trial, and Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums (ARTEMIS) study, is provided. This
includes the mobile apps that were used to collect data and the use of the database to store the data that were collected. Based on
the experiences faced, the technological enhancements and adaptations made at the mobile app and database levels are describedin detail.
Implementation (Results): Development of descriptive analytics at the database level; enabling offline and online data storage
modalities; customizing the Open Medical Record System platform to suit the study requirements; modifying the encryption
settings, thereby making the system more secure; and merging different apps for simultaneous data collection were some of theenhancements made across different projects.
Conclusions: Technology-enabled interventions prove to be a useful solution to cater to large populations in low-resource
settings. The development of mobile apps is subject to the context and the area where they would be implemented. This paper
outlines the need for careful testing using an iterative process that may support future research using similar technology.
Trial Registration:
SMART Mental Health trial: Clinical Trial Registry India CTRI/2018/08/015355;
https://ctri.nic.in/Clinicaltrials/pmaindet2.php?EncHid=MjMyNTQ=&Enc=&userName=CTRI/2018/08/015355. ARTEMIS
trial:
Clinical
Trial
Registry
India
CTRI/2022/02/040307;
https://ctri.nic.in/Clinicaltrials/pmaindet2.php?EncHid=NDcxMTE=&Enc=&userName=CTRI/2022/02/040307(JMIR Med Inform 2024;12:e47504) doi: 10.2196/47504

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 1
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

KEYWORDS
mental health; technological interventions; digital health; community intervention; implementation; eHealth; India; Asia;
development; health technology

Introduction
The burden of mental disorders [1] and the treatment gap due
to untreated mental disorders in low- and middle-income
countries (LMICs) such as India is estimated to range between
75% and 85% [2], with 1 in every 27 individuals being treatedfor depression [3]. Technological solutions have been advocatedto increase access to care, especially in primary health caresettings in LMICs, to facilitate task-sharing, given the lack oftrained mental health professionals. Research has indicated theeffectiveness of employing technologies for addressing complex
health concerns among people with mental illnesses. However,the cost-effectiveness of technology-enabled interventions
compared to in-person interventions has not yet been established[4].
Technology-enabled service delivery models have increased
access to care and facilitated service monitoring, with mobile
health (mHealth) being one such strategy. The World HealthOrganization (WHO) defines mHealth as “a medical and publichealth practice that is supported by mobile devices, such as
mobile phones, patient monitoring devices, and other wirelessdevices” [5]. mHealth in the form of electronic decision support
systems (EDSSs) has been widely adopted by service users andproviders for monitoring health status and for diagnosing andmanaging a range of health conditions, including mental
disorders and substance use [6]. mHealth use has increased withincreasing penetration of mobile network connectivity [7].
This paper highlights the processes involved in the development
and implementation of technology-enabled interventions
employed in three projects across rural and urban settings inIndia among adults and adolescents: the Systematic Medical
Appraisal and Referral Treatment (SMART) Mental Health(SMH) Pilot project [8], and two cluster randomized controlled
trials (cRCTs), SMH trial [9] and the Adolescents’ Resilienceand Treatment Needs for Mental Health in Indian Slums(ARTEMIS) trial [10].

All three projects used EDSSs to facilitate the identification,
diagnosis, and management of common mental disorders(CMDs), including depression, anxiety, psychological distress,
and increased suicide risk. A task-sharing approach was used
where nonphysician health workers known as Accredited Social
Health Activists (ASHAs) and primary care doctors worked
together to support people at high risk of CMDs [8-10].
This implementation report describes the experiences of using
technology in implementing these three mental health projects,
following implementation reporting guidelines [11].

Methods
Aims and Objectives
This paper highlights the processes involved in the development
and implementation of technology-enabled interventions
employed in three projects across rural and urban settings among
adults and adolescents in India.

Blueprint Summary
The overall technological framework of the SMH pilot study
has two main components: a mobile app and a database.
Different mobile apps were developed to collect data atdivergent phases of the study (Figure 1). All apps were installedon 7-inch Android tablets for use by ASHAs/community womenvolunteers (CWVs), or primary health center (PHC) doctors.
ASHAs are local women trained from the community with
8th-10th–grade education levels to support the implementationof health programs. While ASHAs work contractually, they are
incentivized for their involvement in other projects. CWVs are
women who reside in the same community where the study is
being done. These CWVs were chosen from the slums andwould have similar education level as ASHAs. They weretrained on basic knowledge about mental health, along with thestigma and care of individuals with stress, depression, andincreased suicide risk.

Figure 1. Different phases of the study.

The three studies underwent a formative phase, testing study
tools and mobile apps while gauging user acceptance [12,13].
Iterations were made based on user feedback before theintervention phase. The technical team assessed the app andreleased a test version for research team testing. Once confirmed,
a definitive version was used for data collection.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

In the preintervention phase, geographical mapping anddemarcation of the village boundaries were performed, followedby house listing to obtain accurate census data. Custom apps
were developed for each step, including population screening
for identifying individuals at risk of CMDs, which involved
data collectors and ASHAs using specific screening tools. Afterscreening, baseline data on various variables were collected
before the intervention was implemented (Figure 1).
JMIR Med Inform 2024 | vol. 12 | e47504 | p. 2
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Technical Framework Design

algorithm-based diagnoses and evidence-based treatment
recommendations, including comorbidities. Doctors followed
these recommendations, entering the type of care provided(pharmacological, psychological, referral, or combinations
thereof) into their app. Doctors input the data to generate a
traffic light–coded priorities list for ASHAs, indicating the statusof screen-positive individuals in their area. Using color coding
due to the low education levels of ASHAs, the list included
pertinent questions on treatment adherence, social support, andstressors for each color category. The list was dynamic, changing
based on doctors’ updates during patient follow-up visits.

The key components of the EDSS included the ASHA app,
doctors app, and priority listing app (Table 1). Each ASHA hada finite set of individuals who lived in the geographical location
covered by her. The tablets had encrypted, password-protectedindividual logins unique for every ASHA. Individuals screening
positive were referred to primary care doctors for clinicaldiagnosis and treatment based on predetermined cut-off scores.
The doctors used the WHO mhGAP-IG tool (version 1.0) [14]for diagnosing and treating people with CMDs, offering

Table 1. Details of the apps used for the three studies and the target of the intervention.
App

Phase of the study

Users

SMART MHa (Pilot) and SMART MH trial focused on rural adults
Listing app

Listing (household census data collection)

Data collectors

Screening app

Household screening for common mental disorders

ASHAsbBaseline data collection app

Baseline: collected data on different variables andstressors triggering anxiety/depression

Data collectors

Intervention (ASHA app)

Intervention: for regular follow up of adults at high

ASHAs

c

risk of CMDs who sought care from the doctor orhave yet to seek careIntervention (doctor app)

Intervention: diagnosis and treatment for CMDs among Primary care doctors
adults

3M, 6M, and 12M app

Assessments at 3, 6, and 12 months of the intervention Data collectors

ARTEMISd trial focused on adolescentsa

Listing app

Listing (household census data collection)

Data collectors

Screening app

Household screening for common mental disorders

Data collectors

Baseline data collection app

Baseline: collected data on different variables andstressors triggering anxiety/depression

Data collectors

Intervention (ASHA app)

Intervention: for regular follow up of adolescents who ASHAs
are at high risk of CMDs who sought care from thedoctor or have yet to seek careIntervention (doctor app)

Intervention: diagnosis and treatment for CMDs

3M, 6M, and 12 M app

Assessments at 3, 6, and 12 months of the intervention Data collectors

Primary care doctors

SMART MH: Systematic Medical Appraisal and Referral Treatment (SMART) Mental Health.

b

ASHA: Accredited Social Health Activist.

c

CMD: common mental disorder.

d

ARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums.

Identifying an Electronic Medical Record System
All three projects utilized apps based on the Open Medical
Record System (OpenMRS) [15], a community-driven
open-source software for medical record storage and processing.
OpenMRS is robust, scalable for large interventions, andcustomizable to study workflows and data collection needs.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

OpenMRS was chosen for these projects as it is freely available.
Based on our earlier experience, the functionalities were suitablefor our mental health projects [16]. Data collected on tablets
underwent authentication and were transferred to the application
programming interface (API) server, which were then sent tothe application server housing the central OpenMRS database(Figure 2).

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 3
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Figure 2. Workflow of data. API: application programming interface; Asha: Accredited Social Health Activist.

Phases of App Development
The apps went through different phases of development,enhancement, and adaptations across the three projects to suit
the specific requirements of each project (Figure 3).
Figure 3. Phases of app development. API: application programming interface; MRS: Medical Record System; UI: user interface; UX: user experience.

Based on the scope of work (ie, a detailed document or
description that outlines the specific tasks, activities, objectives,and deliverables associated with a particular project) received,
the technological team assessed requirements and checked thefeasibility of incorporating them.

The next step involved configuring project-specific technicaldetails such as concepts, encounter types, visit frequencies, user
roles, and API settings within OpenMRS. Additionally, custom
tables were created to facilitate real-time reporting and analytics,
ensuring efficient data management and analysis for the project.

The next step involved the design of the mobile app userinterface (UI)/user experience (UX) prototype, which was aninteractive mock-up of the mobile app. The prototype contained
key UIs, screens, and simulated functions without any working
code or final design elements. This provided a better
understanding of the real-time UI and UX before production.

The final step was the development of the mobile app and APIs,
which was carried out as a multistage process. The set up
followed the sequence of development, test stage, andproduction environments. The final prototype for the mobile
app involved integrating the EDSS into the app. The SMH apps
supported online/offline features. Standard security integrations
were enabled while developing the mobile app in the local
database in the three different environments. In the test
environment, the integrated feature was assessed with test data
to evaluate the impact of the load of data and the performanceof the app. In the stage environment, this phase included anexact replica of a production environment for testing. In the
production environment, the software or products were madelive for use. Once the development of the app was complete
and certified by the quality assurance team, it was deployed forthe production environment. Screenshots of the app are provided
in Figures 4-6.

Subsequently, the EDSSs were designed according to standard
existing diagnosis and management guidelines, which were
programmed to develop the most appropriate apps. To identify
encounter data fields, individual interactions by ASHAs/doctors
were recorded as separate encounters in OpenMRS. Different
study phases had distinct data points, necessitating a logicalflow of questions. Specific roles were assigned, tailoring thedata collection tools to individual responsibilities. For instance,
the follow-ups for ASHAs used priority-listing questions,whereas the doctors app incorporated mhGAP tool queries. This
ensured targeted and relevant data capture for each study
participant.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

Continuous modifications and maintenance of the app wereapplied across the projects’ lifetimes.

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 4
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Figure 4. App screenshot 1.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 5
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Figure 5. App screenshot 2.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 6
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Figure 6. App screenshot 3.

TargetThe SMH Pilot was implemented in 42 villages across rural andtribal areas of Andhra Pradesh [8,17] with the goal ofunderstanding the feasibility and acceptability of using mobile
technology and task-sharing approaches to address CMDs. This
project covered approximately 50,000 adults and informed thesubsequent SMH Trial, which took place in villages acrossHaryana and Andhra Pradesh, screening 165,000 adults in 133
villages and 44 PHCs. Currently, ARTEMIS is being
implemented among 70,000 adolescents (10-19 years old) in
60 urban slum clusters in Vijayawada (Andhra Pradesh) andNew Delhi.

Ethical Considerations
All collected data are securely stored on central servers inHyderabad, with restricted access limited to the project team.
Participants provided written consent and received detailed
https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

information about data collection at various time points. The
SMART Mental Health pilot study was approved by the
Independent Ethics Review Committee of the Centre for Chronic
DiseaseControl(IRB00006330)
forstudies
CCDC_IEC_03_2014 and CCDC_IEC_02_2014 on October
1, 2014; the SMART Mental Health cluster randomized
controlled trial was approved by the George Institute Ethics
Committee (009/2018) on April 27, 2018; and the ARTEMIS
trial was approved by the George Institute Ethics Committee(17/2020) on September 4, 2020. The study tools were approvedby The George Institute Ethics Committee, and each participant
was assigned a unique identification number at the study’s
outset. Data were consistently deidentified before any sharing,and only research staff and the study’s implementation andstatistical teams had access to the data, ensuring that
confidentiality and ethical standards were maintained throughout
the research process.
JMIR Med Inform 2024 | vol. 12 | e47504 | p. 7
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Participating Entities
The studies have received funding from various internationalorganizations such as Wellcome Trust/Department of
Biotechnology (India Alliance), National Health and Medical
Research Council Australia, and the UK Medical ResearchCouncil. Importantly, these funders are not involved in datacollection or analysis and do not have access to the data.
Government agencies, although collaborators, also do not
manage or analyze the data. The SMH app is under intellectual
property rights of the developer, The George Institute India.
Local government consultation occurred for support, but they
have no role in data governance.

Budget Planning
A predefined budget was allocated to the development andimplementation of the technological interventions. The main
costs incurred included the cost of the server (INR 500,000=US$6862) and the time cost of an Android developer and atechnical lead (INR 200,000=US $2868/month for the initial 6months for development and then a 25% time cost for
maintenance). The other costs included the procurement oftablets for data collection.

and mobile app in JavaScript Object Notation (JSON) format.
HL7 has also developed other standards, including the HL7
Clinical Document Architecture. We used FHIR in our apps as
it was designed to facilitate interoperability of health caresystems, allowing different health care apps and devices toeasily exchange and share data. As the FHIR standard is basedon modern web technologies such as Representational State
Transfer principles, JSON, and Extensible Markup Language,
it provides a flexible and scalable approach to health care dataexchange, making it easier for developers to build interoperable
apps.

Sustainability
The study was developed and implemented in collaboration
with the Andhra Pradesh and Haryana governments. The tool
has been previously utilized in two studies with adults while
undergoing several phases of enhancements and is currently
being used in the ARTEMIS study with adolescents. Poststudy,
the tool will be shared with government and othernongovernmental organizations interested in using it.

Implementation (Results)

Interoperability

Coverage

The apps used in the three studies followed the Health Level 7(HL7)/Fast Healthcare Interoperability Resources (FHIR)
standards for exchanging patient information between a server

The overall coverage of the number of study participants,
ASHAs/CWVs, and doctors reached in the three studies isdetailed in Table 2.

Table 2. Coverage of participants across the three projects.

aProject

Study participants reached,n

ASHAsa/CWVsb included,nDoctors included, n

SMART MHc Pilot (2014 to 2019)50,000 adults

40

14

SMART MH Trial (2018 to 2022)

165,000 adults

175

50

ARTEMISd (2020-2024)

69,600 adolescents (10-19
years old)

104

27

ASHA: Accredited Social Health Activist.

b

CWV: community woman volunteer.

c

SMART MH: Systematic Medical Appraisal and Referral Treatment Mental Health.

d

ARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums.

Outcomes and Technical Amendments Made to the
Data Repository
OpenMRS indicated the overall number of instances that data
were collected for each individual participant at different time
points in the study. As OpenMRS has a report generation model,
it was difficult to compare different data points for the same
person or between participants across the same time point.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

Hence, an intermediary database was developed in house tofacilitate the process of running customized reports, whichenabled comparison of data at different time points. This process
evolved following considerable testing at the backend to obtain
the desired output in terms of data visualization. Somecustomizations were made to OpenMRS to suit study
requirements (Table 3).

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 8
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Table 3. Steps of configurations made to the Open Medical Record System (OpenMRS).
Configuration of OpenMRS modules

Features for the study

Creation of a concept dictionary

Every data point to be used for the study was created as a concept and given a short nameRole management

The roles of each user were fixed and were restricted based on the type of activity they were expected
to do; for example, the project manager was only given access to user data management and downloading
reports

User management

As per our project flow, the different users were allocated to each role, such as ASHAsa, doctors, field
staff/data collector, project manager, and administrator

Encounter management

Each entry into the tab for a specific user (ie, ASHA, doctor, data collector) was recorded as an encounter
with a unique encounter ID, which helped to differentiate the number of encounters that had taken placefor each study participant

Managing encounter types

Based on the different phases of the study, each phase was also considered as a separate encounter, suchas the screening, rescreening, ASHA follow-up, and doctor follow-up phases

Manage observations

Each data point was considered as a separate observation

Managing persons

Demographic data for every app user (ASHA, doctor) or participant were stored as person details

Managing patients

In this feature, any additional personal identifiers/demographic details identified could be modified/configured

Cohort management

Specific cohorts were created for every phase of the project, matched to the user. This enabled the users
to access data of people who were in their own cohort. This helped them to identify and follow up theindividuals easily. This was done both for ASHAs and doctors, with each doctor in a particular PHCb
having a defined set of ASHAs, who in turn had a defined set of high-risk individuals

Multilocation data management

This was a custom development made to the system to ensure the data of one location (state) were notmerged with data from another location. This was relevant to the SMHc and ARTEMISd trials, which
involved two different geographical locations.

aASHA: Accredited Social Health Activist.

b

PHC: primary health center.

c

SMH: Systematic Medical Appraisal and Referral Treatment Mental Health.

d

ARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums.

In India, internet connectivity varies, particularly in rural
regions. To address this, the quality of connectivity was assessedin each study area and hotspots were identified. Offline andonline data storage methods were implemented, allowing local
storage on tablets in areas with poor connectivity. Data could
later be uploaded to the central server once connectivity was
restored. Additionally, networks at certain PHCs were improved,
increasing the bandwidth to enable ASHAs and doctors to
upload data when in proximity to these PHCs.

Lessons Learned
There were several lessons learned while designing andimplementing these interventions, which resulted in several
enhancements to the systems for improving UX and achieve
the study outcomes. Following the SMH Pilot, issues wereidentified in the EDSS that needed to be corrected for the SMH
and ARTEMIS trials (Table 4). During the project, unforeseenchallenges arose due to the COVID-19 pandemic. Face-to-facehttps://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

training for health workers was impossible, leading to the
preparation of training materials delivered with the assistanceof field staff. Additionally, some tablets used by health workers
broke down, necessitating replacements and revealing bugs in
the app. The SMH cRCT project faced difficulties because of
COVID-19, and different mitigation strategies were adopted toensure implementation of the different stages of the project.
However, due to the rapidly changing situation, those also had
to be modified quickly. Considering all the issues encounteredearlier, we tried to mitigate all these challenges encounteredduring the SMH pilot study and cRCT, leading to enhancementof the apps developed for the ARTEMIS project. To have asmooth transition from the test environment to the live
environment, the technical team performed additional checks
by testing the apps by the field staff and creating data that were
uploaded to the server to confirm whether all the fields are being
populated correctly. This helped in reducing the errors whiledata were being captured in live scenarios.

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 9
(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Table 4. Enhancements made to the electronic decision support system.
Issues that needed amendments

Solutions for the problems/issues

Daily monitoring of data at the field level
and comparison of data across sites, localities, and users was very difficult. Monitoring of clinical data of patients was also difficult

Development of descriptive analytics at the database level while implementing the SMHa trial was doneto ease monitoring of data. There were many enhancements made at that level, in terms of representing
real-time data from different aspects of the study. This included identification of mental health service
use, the burden of different mental health conditions, and comparison of different conditions, among
other factors. These analytics could be viewed by comparisons made across regions, gender, and age
groups. These were represented through pictorial modes such as graphs and pie charts (see Figure 7 forexamples)Monitoring an individual’s mental healthstatus over time was not possible

Analytics were developed to track the PHQ9b and GAD7c scores of an individual in the different phasesof the study. Data captured periodically during monitoring could be viewed as graphs and charts basedon the longitudinal data at the backend using analytics.

The performance of ASHAsd could not be
tracked well

There were enhancements made to the ASHA app, which tracked the performance of each ASHA andprovided data about the numbers of screenings and follow-ups performed, including the time taken for
each. Random audio recordings of their interactions were also captured to ensure quality checks.

As the database is encrypted and stored in The app is protected with multifactor authentication using a password and lock pin as an enhancementa password-protected, secure location, it is to the existing setup.
hard to gain access to data by reverse engineering or decoding
User interface and functioning of the app
were not clear

Several changes were made to the user interface, including a change of font size, color, and creating
different section headers using attractive symbols/pictures, for better user experience

Enabling online training during COVID-19 Some of the training materials were embedded in the mobile apps to enable easier access for trainees
using virtual modes during COVID-19.
Real-time monitoring of the activities ofRandom audio recording of interaction of field staff with study participants or high-risk individuals was
field staff was required to ensure increased enabled. The time taken for each screening was also made available at the database level for these audio
data quality
recordings. This helped the implementation team to monitor data collection and quality.
Merging of two apps, namely household
listing and participant screening, into one
appaThis merger made it possible for simultaneous data collection for both listing and screening, which saved
time for both the participant and field staff and reduced multiple visits to the same household for datacollection.

SMH: Systematic Medical Appraisal and Referral Treatment Mental Health.

b

PHQ9: Patient Health Questionnaire-9.

c

GAD7: Generalized Anxiety Disorder-7.

dASHA: Accredited Social Health Activist.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 10(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Figure 7. Snapshot of the analytics developed to monitor the progress of the study outcomes. +Ve: positive; -Ve: negative.

Discussion
Principal Findings
This paper outlines the experience of employing technology inmental health service delivery across rural and urban India inthree projects. We have highlighted the implementation
challenges and app adaptations based on user feedback, offering
insights valuable for technology-based mental health projects
in resource-limited settings. Technology-enabled interventions
have shown effectiveness in diagnosing, treating, and following
up on various health conditions [18,19]. Most mHealthinterventions used in India have been disease-specific and do
not involve a health systems approach. One example of a more
health system–focused app is the Government to Government
web-based monitoring information system that has been set upby the Ministry of Health & Family Welfare, Government ofIndia, to monitor the National Health Mission and other healthprograms. To increase effectiveness, these innovations should
focus on creating new avenues to integrate tools that haveencouraging and sustainable outcomes related to access, equity,
quality, and responsiveness. The SMH app can be integrated
with government systems after specific modifications. The useof electronic medical record systems and telemedicine areexamples of some of the interventions implemented and foundto be beneficial for health care delivery for large populations,
https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

especially in LMICs [16,19,20]. However, there is a need tounderstand the local context and setting while developing or
enhancing any existing app, as some of the original features
may not be relevant to the local context, making further
adaptations critical.
One way to enhance the functional capabilities of apps such as
SMH is to link the app with telemedicine facilities that amplify
the ability to connect to remotely located consumers withspecialists located in larger cities [13]. For example, machinelearning has been applied for suicide prediction, matching
patients to appropriate treatment, improving the efficacy ofmental health care by clinicians, and monitoring patients fortreatment adherence with the help of smartphones and sensors[21].
Another way to leverage technology in mental health is by using
artificial intelligence. A recent systematic review recommended
the use of artificial intelligence technologies as accurate andeffective strategies in the diagnosis and treatment of mental
health conditions [22]. Virtual reality technology has proven to
be a useful and powerful tool in addiction research [23]. The
user interacts with the virtual reality environment, offering anenvironment close to real life that is dynamic in nature andrequires active participation. These environments can be used
to develop psychotherapeutic interventions by adding a personal
JMIR Med Inform 2024 | vol. 12 | e47504 | p. 11(page number not for citation purposes)

JMIR MEDICAL INFORMATICS
touch, having predictable conditions with additional features
such as embodiment, eye tracking, and other biological factors[24].
There is still substantial work to be done in terms of scaling up
these interventions and understanding their feasibility andacceptability across different settings and populations. Use ofnovel strategies such as videogaming can be explored to
implement mental health interventions that can be customized
to specific populations [25]. Such techniques should beconsidered in future iterations of the technology platform[26,27].

Limitations
There were a few limitations in our apps. First, the mobile apps
developed were limited to stress, depression, anxiety, andincreased suicide risk; however, the principles of including othermental health conditions would be quite similar. Second,although the projects had a system of referring participantsrequiring specialist care to mental health professionals, it was

Kallakuri et al
beyond the scope of the projects to track the care provided by
the mental health professionals through our app. This was
because our app was developed through primary healthsystem–focused application for use in low-resource settings andwas not linked to any central electronic health record system as
is possible in more developed health systems with more robust
data capture and record-sharing capabilities, such as the National
Health Service in the United Kingdom or health systems inAustralia. Third, the current apps are compatible on Android
platforms and could not be expanded to other operating systems.
Finally, the apps developed were specifically created following
consultations with local stakeholders; hence, theirgeneralizability across other settings will need to be assessed
after adaptation is complete.

Future Recommendations
Given our experiences, we have compiled a set of suggested
recommendations for technology-based interventions in similar
settings, which are presented in Textbox 1.

Textbox 1. Recommendations for technology-based interventions.
•

Inclusion of the technical team from the outset when study protocols are being developed.

•

All study-related tools and database designing should be finalized in consultation with relevant experts.

•

A protocol that details the process of server support in terms of setup/maintenance needs should be developed and followed.

•

The server needs to factor in the size of the data set and latest versions of operating systems in reducing any issues faced.

•App user interface/user experience should be designed and assessed for acceptability by targeted populations. The use of reports or data analytics
for the study must be discussed and finalized as per study needs.

•

Develop systems that can be used across any kind of device, are compatible for software or version upgrades, and are web-based and easily
programmable.

•

A technical guide with frequently asked questions outlining the various aspects of technology, such as navigation, problem-solving, and reportingof issues, should be developed to facilitate staff training.

•

The infrastructure and the architecture of the app should be flexible for making modifications or scaling up the app. The scalability is measured
by the number of requests an app can manage and support the app effectively. A decision needs to be taken in terms of adding resources to thecomputing system for scaling either horizontally (adding more machines to the existing pool) or vertically (adding more power to the existingmachines). Both types of scaling are similar as they add computing resources to the infrastructure; however, there are distinct differences between
the two in terms of implementation and performance.

Conclusion
In conclusion, the development of any health-related app is
subject to the context and the area where it would be

implemented. There is a need for careful testing using iterative
processes, allocate human and budgetary resources that are
adequate, and integrate apps with larger electronic health record
systems that inform health systems.

Acknowledgments
We would like to acknowledge the entire Systematic Medical Appraisal and Referral Treatment (SMART) Mental Health teamat the Andhra Pradesh, Haryana site and the staff of Adolescents’ Resilience and Treatment Needs for Mental Health in IndianSlums (ARTEMIS) at the Vijayawada and Delhi sites who provided input while assessing the apps, which greatly helped in
making specific enhancements to the apps. This work was supported by ARTEMIS (to SG, MG, and SK), Australian National
Health and Medical Research Council (NHMRC) Global Alliances for Chronic Disease (GACD) (SMART for Common Mental
Disorders in India; grant APP1143911), and UK Research and Innovation (UKRI)/Medical Research Council (MRC) grant(ARTEMIS; MR/S023224/1 to PKM). DP is partially or wholly supported through the SMART Mental Health NHMRC/GACD
grant. PKM is the principal investigator on the ARTEMIS Project and coprincipal investigator on the SMART Mental Health
Project and is partially supported by both projects. DP is supported by fellowships from the NHMRC of Australia (1,143,904)
and the Heart Foundation of Australia (101,890). SKY is supported by the ARTEMIS Project (UKRI/MRC grant MR/S023224/1),
SP is supported by the ARTEMIS Project and another project titled MentAL Health Risk Factors among Older AdolesceNts
living in Urban Slums: An InTervention to Improve ResIlience (ANUMATI) funded by the Indian Council of Medical Research(grant 2019-0531). MD is supported by SMART Mental Health funded by NHMRC Australia (grant APP1143911) and thehttps://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 12(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

International Study of Discrimination and Stigma Outcomes (INDIGO) Partnership Research Programme funded by the UK MRC(MR/R023697/1). The funding bodies played no role in the design of the study and in the conceptualization and writing of themanuscript.

Authors' Contributions
SK, SG, PKM: conceptualization. SK and SG: writing of first draft. MG, SK, SKY, SP, MD, DP, and PM: review & editing. All
authors read and approved the final manuscript.

Conflicts of Interest
All authors are employees of The George Institute, which has a part-owned social enterprise, George Health Enterprises, withcommercial relationships involving digital health innovations.

References
1.
2.
3.

4.

5.
6.

7.
8.

9.

10.

11.

12.

13.

14.
15.
16.

Vigo D, Thornicroft G, Atun R. Estimating the true global burden of mental illness. Lancet Psychiatry. Feb 2016;3(2):171-178.
[doi: 10.1016/S2215-0366(15)00505-2] [Medline: 26851330]
Kohn R, Saxena S, Levav I, Saraceno B. The treatment gap in mental health care. Bull World Health Organ. Nov
2004;82(11):858-866. [FREE Full text] [Medline: 15640922]
Thornicroft G, Chatterji S, Evans-Lacko S, Gruber M, Sampson N, Aguilar-Gaxiola S, et al. Undertreatment of people withmajor depressive disorder in 21 countries. Br J Psychiatry. Feb 02, 2017;210(2):119-124. [FREE Full text] [doi:10.1192/bjp.bp.116.188078] [Medline: 27908899]
Naslund JA, Marsch LA, McHugo GJ, Bartels SJ. Emerging mHealth and eHealth interventions for serious mental illness:a review of the literature. J Ment Health. May 28, 2015;24(5):321-332. [FREE Full text] [doi:10.3109/09638237.2015.1019054] [Medline: 26017625]
World Health Organization. mHealth: new horizons for health through mobile technologies: second global survey on
eHealth. Geneva. World Health Organization; 2011.
Kumar S, Nilsen WJ, Abernethy A, Atienza A, Patrick K, Pavel M, et al. Mobile health technology evaluation: the mHealthevidence workshop. Am J Prev Med. Aug 2013;45(2):228-236. [FREE Full text] [doi: 10.1016/j.amepre.2013.03.017][Medline: 23867031]
Agrawal N. Telephone network and internet penetration in India: a pragmatic study using data analytics. Global J EnterpriseInf Syst. 2021;13(1):42-48. [FREE Full text]
Maulik PK, Devarapalli S, Kallakuri S, Bhattacharya A, Peiris D, Patel A. The systematic medical appraisal referral andtreatment mental health project: quasi-experimental study to evaluate a technology-enabled mental health services delivery
model implemented in rural India. J Med Internet Res. Feb 27, 2020;22(2):e15553. [FREE Full text] [doi: 10.2196/15553][Medline: 32130125]
Daniel M, Maulik PK, Kallakuri S, Kaur A, Devarapalli S, Mukherjee A, et al. An integrated community and primary
healthcare worker intervention to reduce stigma and improve management of common mental disorders in rural India:
protocol for the SMART Mental Health programme. Trials. Mar 02, 2021;22(1):179. [FREE Full text] [doi:10.1186/s13063-021-05136-5] [Medline: 33653406]
Yatirajula SK, Kallakuri S, Paslawar S, Mukherjee A, Bhattacharya A, Chatterjee S, et al. An intervention to reduce stigma
and improve management of depression, risk of suicide/self-harm and other significant emotional or medically unexplainedcomplaints among adolescents living in urban slums: protocol for the ARTEMIS project. Trials. Jul 29, 2022;23(1):612.
[FREE Full text] [doi: 10.1186/s13063-022-06539-8] [Medline: 35906663]
Perrin Franck C, Babington-Ashaye A, Dietrich D, Bediang G, Veltsos P, Gupta PP, et al. iCHECK-DH: Guidelines andChecklist for the Reporting on Digital Health Implementations. J Med Internet Res. May 10, 2023;25:e46694. [FREE Fulltext] [doi: 10.2196/46694] [Medline: 37163336]
Maulik PK, Tewari A, Devarapalli S, Kallakuri S, Patel A. The Systematic Medical Appraisal, Referral and Treatment(SMART) Mental Health Project: development and testing of electronic decision support system and formative research
to understand perceptions about mental health in rural India. PLoS One. Oct 12, 2016;11(10):e0164404. [FREE Full text][doi: 10.1371/journal.pone.0164404] [Medline: 27732652]
Daniel M, Kaur A, Mukherjee A, Bhattacharya A, Tewari A, Sagar R, et al. The systematic medical appraisal, referral andtreatment (SMART) mental health programme: formative research informing a cluster randomized controlled trial. SSM
Mental Health. Dec 2023;3:100223. [doi: 10.1016/j.ssmmh.2023.100223]
Scaling up care for mental, neurological and substance use disorders: mhGAP. World Health Organization. 2008. URL:
https://www.who.int/activities/scaling-up-mental-health-care [accessed 2024-01-14]
OpenMRS. URL: https://openmrs.org/ [accessed 2024-01-22]
Peiris D, Praveen D, Mogulluru K, Ameer MA, Raghu A, Li Q, et al. SMARThealth India: a stepped-wedge, cluster
randomised controlled trial of a community health worker managed mobile health intervention for people assessed at highhttps://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 13(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

17.

18.

19.

20.
21.
22.

23.

24.
25.
26.
27.

Kallakuri et al

cardiovascular disease risk in rural India. PLoS One. Mar 26, 2019;14(3):e0213708. [FREE Full text] [doi:10.1371/journal.pone.0213708] [Medline: 30913216]
Maulik PK, Kallakuri S, Devarapalli S, Vadlamani VK, Jha V, Patel A. Increasing use of mental health services in remoteareas using mobile technology: a pre-post evaluation of the SMART Mental Health project in rural India. J Glob Health.
Jun 2017;7(1):010408. [FREE Full text] [doi: 10.7189/jogh.07.010408] [Medline: 28400954]
Bassi A, John O, Praveen D, Maulik PK, Panda R, Jha V. Current status and future directions of mHealth interventions forhealth system strengthening in India: systematic review. JMIR Mhealth Uhealth. Oct 26, 2018;6(10):e11440. [FREE Fulltext] [doi: 10.2196/11440] [Medline: 30368435]
Koppar AR, Sridhar V. A workflow solution for electronic health records to improve healthcare delivery efficiency in rural
India. Presented at: 2009 International Conference on eHealth, Telemedicine, and Social Medicine; February 1-7, 2009,2009; Cancun, Mexico. [doi: 10.1109/etelemed.2009.30]
Acharya R, Rai J. Evaluation of patient and doctor perception toward the use of telemedicine in Apollo Tele Health Services,
India. J Family Med Prim Care. 2016;5(4):798-803. [FREE Full text] [doi: 10.4103/2249-4863.201174] [Medline: 28348994]
Haggerty E. Healthcare and digital transformation. Network Security. Aug 2017;2017(8):7-11. [doi:10.1016/s1353-4858(17)30081-8]
Graham S, Depp C, Lee EE, Nebeker C, Tu X, Kim H, et al. Artificial intelligence for mental health and mental illnesses:
an overview. Curr Psychiatry Rep. Nov 07, 2019;21(11):116. [FREE Full text] [doi: 10.1007/s11920-019-1094-0] [Medline:
31701320]
Segawa T, Baudry T, Bourla A, Blanc J, Peretti C, Mouchabac S, et al. Virtual reality (VR) in assessment and treatmentof addictive disorders: a systematic review. Front Neurosci. Jan 10, 2019;13:1409. [FREE Full text] [doi:10.3389/fnins.2019.01409] [Medline: 31998066]
Cavalcante Passos I, Mwangi B, Kapczinski F. Personalized psychiatry: Big data analytics in mental health. New York,NY. SpringerLink; 2019.
Hamari J, Keronen L. Why do people play games? A meta-analysis. Int J Inf Manag. Jun 2017;37(3):125-141. [doi:10.1016/j.ijinfomgt.2017.01.006]
Wilkinson N, Ang RP, Goh DH. Online video game therapy for mental health concerns: a review. Int J Soc Psychiatry. Jul
01, 2008;54(4):370-382. [doi: 10.1177/0020764008091659] [Medline: 18720897]
Li J, Theng Y, Foo S. Game-based digital interventions for depression therapy: a systematic review and meta-analysis.
Cyberpsychol Behav Soc Netw. Aug 2014;17(8):519-527. [FREE Full text] [doi: 10.1089/cyber.2013.0481] [Medline:
24810933]

Abbreviations
API: application programming interface
ARTEMIS: Adolescents’ Resilience and Treatment Needs for Mental Health in Indian Slums
ASHA: Accredited Social Health Activist
CMD: common mental disorder
CWV: community woman volunteer
cRCT: cluster randomized controlled trial
EDSS: electronic decision support system
FHIR: Fast Healthcare Interoperability Resource
HL7: Health Level 7
JSON: JavaScript Object Notation
LMIC: low- and middle-income country
mHealth: mobile health
OpenMRS: Open Medical Record System
PHC: primary health center
SMH: Systematic Medical Appraisal and Referral Treatment (SMART) Mental Health
UI: user interface
UX: user experience
WHO: World Health Organization

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 14(page number not for citation purposes)

JMIR MEDICAL INFORMATICS

Kallakuri et al

Edited by C Perrin; submitted 22.03.23; peer-reviewed by N Mungoli, E Korshakova; comments to author 24.04.23; revised version
received 21.05.23; accepted 29.11.23; published 15.02.24
Please cite as:
Kallakuri S, Gara S, Godi M, Yatirajula SK, Paslawar S, Daniel M, Peiris D, Maulik PK
Learnings From Implementation of Technology-Enabled Mental Health Interventions in India: Implementation Report
JMIR Med Inform 2024;12:e47504
URL: https://medinform.jmir.org/2024/1/e47504
doi: 10.2196/47504
PMID: 38358790©Sudha Kallakuri, Sridevi Gara, Mahesh Godi, Sandhya Kanaka Yatirajula, Srilatha Paslawar, Mercian Daniel, David Peiris,
Pallab Kumar Maulik. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 15.02.2024. This is an
open-accessarticledistributed
under
the
terms
of
the
Creative
Commons
Attribution
License(https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium,provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information,a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included.

https://medinform.jmir.org/2024/1/e47504

XSL• FO
RenderX

JMIR Med Inform 2024 | vol. 12 | e47504 | p. 15(page number not for citation purposes)