JMIR FORMATIVE RESEARCH

Shea et al

Original Paper

Assessment of App-Based Versus Conventional Survey Modalities
for Reproductive Health Research in India, South Africa, and theUnited States: Comparative Cross-Sectional Study
Amanda A Shea1*, PhD; Jonathan Thornburg2*, PhD; Virginia J Vitzthum3,4*, PhD
1Biowink GmbH, Berlin, Germany

2

Department of Astronomy & Center for Spacetime Symmetries, Indiana University, Bloomington, IN, United States

3

Centre for Menstrual Cycle and Ovulation Research, Division of Endocrinology/Medicine, The University of British Columbia, Vancouver, BC, Canada

4

Department of Anthropology, Indiana University, Bloomington, IN, United States*

all authors contributed equally

Corresponding Author:
Virginia J Vitzthum, PhD
Centre for Menstrual Cycle and Ovulation Research
Division of Endocrinology/Medicine
The University of British Columbia
2775 Laurel Street
Vancouver, BC, V5Z 1M9
CanadaPhone: 1 250 984 3033
Email: dr.vj.vitzthum@gmail.com

Abstract
Background: There is a widely acknowledged global need for more research on reproductive health (including contraception,
menstrual health, sexuality, and maternal morbidities) and its impact on overall well-being. However, several factors—notably,high costs, considerable effort, and the sensitivity of these topics—impede the collection of the necessary data, especially in lessaccessible and lower-income populations. The burgeoning ownership of smartphones and growing use of menstrual tracking apps
(MTAs) may present an opportunity to conduct reproductive health research with fewer impediments than those associated withconventional survey methods.
Objective: The main objective was to ascertain the feasibility, potential usefulness, and limitations of conducting reproductive
health research using a mainstream MTA.
Methods: In each of the 3 countries, we evaluated questionnaire responses from (1) current users of an MTA (Clue) and (2)participants surveyed using conventional survey modalities (in-person interviews, SMS text messaging, and web-basedquestionnaires). We compared these responses with published data collected from large nationally representative benchmark
samples (the United States Census and the Demographic and Health Surveys for South Africa and India).
Results: Given a sufficiently large user base, app-distributed surveys were able to quickly capture large samples on par withother methods and at low cost, with the additional advantage of being able to deploy remotely and simultaneously across countries.
In each country, neither the app nor the conventional modality sample emerged as a consistently closer match to the distributionsof the demographic attributes and the patterns of contraceptive use reported for the respective benchmark sample. Despite efforts
to obtain representative samples, the conventional modality samples sometimes over- and other times underrepresented some
subgroups (eg, underrepresentation of married persons in the United States and overrepresentation of rural residents in India). In
all 3 countries, app users were younger, more educated, more likely to be urban residents, and more likely to use nonhormonal
rather than hormonal contraceptive methods compared with the respective national benchmark. App users, compared with theconventional modality samples, consistently reported being more comfortable discussing their menstrual periods with otherpersons (eg, family, friends, and health care providers), suggesting that MTA users may be more likely to respond truthfully toquestions on sensitive or taboo health topics. The app samples’ consistency across countries regarding users’ demographic profiles,
contraceptive choices, and personal attitudes toward menstruation supports the validity of making cross-country comparisons ofsurvey findings for a given app’s users.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Conclusions: MTAs such as Clue can provide a quick, scalable, and cost-effective method for collecting health data, including
on sensitive topics, across a wide variety of settings and countries. With expanding global access to technology and the increasinguse of these tools, consumer MTAs can be a viable survey modality to strengthen reproductive health research.
(JMIR Form Res 2023;7:e44705) doi: 10.2196/44705
KEYWORDS
mobile health; mHealth; femtech; reproductive health; menstrual health; sexual health; survey modalities; menstrual tracking
app; India; South Africa; United States; mobile phone

Introduction
Background
The reproductive system is an integral component of bodilyfunctioning and overall well-being. Reproductive healthencompasses the proper functioning of the reproductive system
at all developmental stages throughout life and includes theability to have a satisfying and safe sex life and the freedom todecide whether and how to reproduce [1,2]. It is widely
acknowledged by governments and international organizations,health care researchers and providers, and the general populationthat there is a pressing global need to improve reproductive
health for all people. Accurate comprehensive data areinvaluable for effectively addressing this challenge. However,in most countries, there are surprisingly few data on the
reproductive health of the population or on the availability ofhealth services and other resources necessary to achieve
population-wide satisfactory reproductive health. The acquisitionof such data can be daunting.
A crucial component of efforts to reduce maternal and infantmortality and morbidity over the past few decades has been andis the collection of nationally representative health data, often
conducted by a nation’s government, sometimes in collaboration
with nongovernmental agencies (eg, the Demographic and
Health Survey [DHS] Program [3]). These data are used toassess a population’s current health status, evaluate the impacts
of programs, and inform future efforts to improve health. Carefulplanning, technically sophisticated statistical sampling, andpretested standardized questionnaires (customized to a
population’s cultural and socioeconomic attributes) administeredin face-to-face interviews by trained staff underpin the reliability
and representativeness of DHS data [4]. Therefore, it was andis possible to accurately monitor the gains and setbacks inmaternal and infant health over several decades even in thepoorest populations.
However, DHS and other nationally representative studies
require considerable effort, expense, and time, all of which limit
the coverage of different health issues and preclude frequent
surveys or modifying questionnaires in response to unanticipateddevelopments (eg, new diseases and novel trends affecting
population health). For example, during the COVID-19
pandemic, rapid changes in access to contraceptives and healthcare services occurring at different times and to different degrees
across locations were difficult to document and unlikely to be
captured in a country’s DHS survey. Such unanticipated andrapid shifts in health status and services can fall under the radarof surveys and censuses that take a year or more to organize
and are typically separated by several years.
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

Smaller-scale targeted surveys, which are more rapidly andeasily deployed, are a widely used approach for gathering datathat complement those from national studies. Small-scale
surveys can gather useful subsets of the same data types as
national studies, target a single subpopulation, or explore newhealth issues. There are several conventional modalities
commonly used in such surveys, including one-on-one
interviews (in person or by telephone), mailed questionnaires
to be completed and returned by a respondent, web-basedquestionnaires distributed via social media and other websites,
and SMS text messages to cell phones.
Each of these conventional modalities has its own strengths andweaknesses in terms of representativeness, generalizability,
costs, sample size, and collection speed [5,6]. For example,
meetings between a respondent and trained personnel who
administer a questionnaire may be difficult to arrange.
Spontaneous telephone calls may be rebuffed, especially if thepotential respondent is busy. However, if respondents arecompensated (as is typically the case for commercial survey
providers), completion rates can be very good, although thecosts can be high.
Using smartphones and associated mobile apps for datacollection can be a more rapid, cost-effective, and scalable
modality than other conventional modality surveys (CMSs).
The rapid growth of smartphone use makes it particularly
suitable for collecting data from a broad swath of a population.
Although smartphone ownership remains more widespread inhigher-income countries and among younger, more educatedpeople, as well as among men as compared with women,smartphone ownership is increasing across the board, with thesegaps narrowing over the past decade [7]. Smartphone ownershipis now expected to expand to more than two-thirds of the global
population by 2025 [7].
With the increasing ubiquity of mobile devices, there has beena parallel increase in their use for public health and medical
practice, referred to as mobile health (mHealth), which has beenused in various settings from medication adherence in clinicalpractice to behavior change interventions in large populations[8]. Among the advantages over most CMSs, mobile app datacollection can be implemented remotely and completed at theconvenience of the respondents. Thus, participants do not needto live near or visit a research site to take part, increasing theability to include those limited by travel burdens or even those
who simply want the convenience of participating whenever
and wherever they prefer [9]. Similarly, researchers do not needto travel to conduct data collection. These various advantages
reduce staff and participant costs, thereby reducing researchexpenses. Owing to the widespread use of smartphones,
JMIR Form Res 2023 | vol. 7 | e44705 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
researchers can more easily reach a larger and potentially morediverse population, yielding greater sample sizes and moregeneralizable results. The possibility of continued directcommunication with participants via mobile apps also makes
this modality suitable for follow-up or for use in longitudinalstudies. App-based data collection may also provide respondents
with greater anonymity and increased privacy and comfort,
which, in turn, could improve respondent accuracy even for
culturally taboo topics. In contrast, completion rates may be
low and data quality may be poor if respondents are not
motivated to provide truthful, complete answers to questions.
Theoretically, almost any mainstream smartphone app with alarge user base can serve as a conduit to a pool of potential
survey respondents. However, health apps are likely to be aparticularly productive choice for reaching people who are
motivated to provide truthful answers to questions regarding
their own health as these users have demonstrated an interestin the topic. As menstruation is a practically universal
experience in female individuals from puberty to menopause,
we hypothesized that menstrual tracking apps (MTAs), which
are already widely used by both adolescent girls and adultwomen [10-12], are exceptionally well positioned for collecting
reproductive health data from this segment of the population.
Furthermore, menstrual health is a key component of overall
health as well as reproductive health [13-15]. Proper functioningof the menstrual cycle is obviously essential to fertility and isa key factor in contraceptive decision-making [16]. Moreover,menstrual cycling affects nearly all aspects of female health,including immune function [17,18], mental health [19,20],
cardiovascular health [21], and bone health [22]. Many MTAs
provide support for a range of use cases, including menstruation,
contraceptive use, conception, pregnancy, and perimenopause,with the ability to track a broad range of experiences such as
periods, energy, mood, pain, sex, sleep, exercise, illness, andmedications. Clearly, there is an expansive array of health topics
that could be addressed by incorporating MTAs into research
designs.
However, there is currently little information on the feasibilityof conducting surveys via MTAs or information on the
respondent characteristics of those who have been recruitedusing these platforms.

Objectives
Our study aimed to help address this research gap. Specifically,in each of the 3 countries (India, South Africa, and the UnitedStates), we evaluated the comparability of questionnaire
responses collected using a CMS and an MTA to highly reliablebenchmark data collected by the respective national
governments. The study questionnaire included sections on
individual demographic attributes and current contraceptive use(topics that are typically included in national-level reproductive
health surveys) and a section on the respondents’ attitudes
regarding menstrual bleeding (which potentially differ between
MTA users and the general population and which has emerged
as a significant but understudied factor affecting contraceptive
choice [16]). We also assessed differences in performance,
including ease of use and ability to achieve desired sample sizes,
between the CMS and the MTA.
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

Shea et al

Methods
Clue is a smartphone app for tracking menstrual bleeding andother aspects of health (eg, pain, sleep, and emotions). Owing
to its large global user base (10 million active users worldwide),
we hypothesized that the Clue app [23] would be a suitablechoice for research in different cultural and demographic
contexts.

Ethical Considerations
This project underwent ethics review in each of the 3 study
countries. It was deemed exempt by the Western Institutional
Review Board in the United States, approved by the Sigma
institutional review board in India (10008/IRB/20-21), andapproved by the Research Ethics Committee at the Universityof Pretoria in South Africa (reference 05079421[HUM042/0620]). In accordance with each CMS provider’spolicy and the approved protocol, CMS respondents received
nonmonetary gifts (such as reward points and phone top-ups)
but did not receive direct monetary compensation. App-based
survey participants were not paid or otherwise compensated fortaking part in this study.
Only persons aged 18 to 45 years were eligible to participate
in the study. All prospective participants were provided withstudy information, including the study goals, how study data
would be processed and stored (and that survey responses would
be archived for other researchers to use to answer additional
research questions), and the contact information of theresearchers for any questions or concerns. All participation wascompletely voluntary, and informed consent was obtained before
participants could begin the survey. Except for Clue users in
the United States, where app survey data were combined withan individual’s app-tracked data (with the consent of theparticipant), all surveys were taken completely anonymously.

Analytical Strategy
In each country, we compared data that were collected using
different modalities and that, by the nature of their datacollection protocols, represented different segments of eachcountry’s population:
1.
2.

3.

A questionnaire sent to a random sample of current Clue
app users
A shortened version of the same questionnaire distributedby commercial survey companies using conventional
modalities (in-person interviews, web-based questionnaires,and mobile phone SMS text message surveys)
Publicly available, nationally representative demographic
databases (these data provided a reliable benchmark against
which to assess the demographic composition of the app
and CMS samples)

Study Countries
In total, 3 countries were selected for this study based on thefollowing considerations:
1.
2.

Presence of country-specific, recent, publicly available, andnationally representative demographic survey data
Sufficiently large Clue user base such that, given typicaluser click-through rates for app-based surveys, an adequate
JMIR Form Res 2023 | vol. 7 | e44705 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

3.
4.

sample size for the intended analyses was likely to beachievable
Representation from 3 distinct global regions
Representation of different population profiles ofcontraceptive use

The United States, India, and South Africa met these criteria.
These 3 countries were also selected as India was a Family
Planning 2020 target country [24], South Africa had greater useof long-acting injectables than either the United States or India,
and previous studies have reported that cultural norms andbeliefs regarding menstrual bleeding differ greatly between
these populations [25-28].

Questionnaire Development and TestingRespondents were asked questions about their demographic
profile, current and past contraceptive use (if any), and attitudes
and preferences regarding menstrual bleeding. The number ofquestions varied slightly across countries to accommodate local
differences in demographic categories and the limitations of thedifferent modalities. All versions of the study questionnaire areprovided with the archived data.
For each country, the goal was to collect data from a CMS
participant sample that would resemble a sample that could
have been recruited using standard research methods in thatspecific setting. This approach allowed for the country-specificcomparison of CMS, app, and benchmark data. However,
because of the necessary differences in recruitment criteria for
the CMS (a reflection of already known sociodemographic
differences between the countries), the study design was not
specifically intended to collect CMS data that were directlycomparable across all 3 countries. In contrast, befitting the lackof information before the study on the attributes of the app
population, data collection from the app samples did not use
targeting strategies or quotas. This approach for this group
allowed for an evaluation of the distribution of the characteristicsof the current app user base.
The study questionnaire was first developed for English speakers
in the United States and tested via in-person interviews and a
web-based testing platform, UserTesting. A native Spanish
consultant followed the same protocols for testing thequestionnaire with Spanish speakers in the United States.
Subsequent user testing in South Africa and India was conductedunder the direct guidance of in-country experienced research
consultants. The country-specific questionnaires wereextensively tested for local suitability (eg, income and ethnicity
categories) and comprehension (eg, locally colloquial language
for types of contraceptives and for attitudes regarding menstrual
bleeding).

App-Based Data Collection
Data were collected from November 9 to 29, 2020, in the UnitedStates and from November 12, 2020, to February 17, 2021, in
India and South Africa. Invitations with a link to thequestionnaire were distributed via in-app message in English
to randomly selected Clue users in each of the 3 study countries
and also in Spanish to those users in the United States whose
phone language was set to Spanish.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

Shea et al
As people responding to app-based questionnaires may enter
false or impossible answers (perhaps to pass the time or for
personal amusement), it was explained to the app users at thestart that the survey was for research purposes. As a final quality
check, the respondent was asked to answer yes or no to thefollowing statement: “I took the survey seriously.” Only those
who answered yes were included in the analyses.
The target sample sizes for each country reflected thecountry-specific Clue app user base and likely completion rates,
and the sample size needed for the planned analyses. For India
and South Africa, the target sample sizes were set at 2500
completed questionnaires each, equal to the target set for the
CMSs described in the following section. Taking advantage ofthe larger Clue user base in the United States, this target sample
size was set at 10,000 completed questionnaires so as to provide
adequate statistical power for future subgroup analyses. Data
collection ended when the target was reached or when the daily
response rate had decreased to near 0.

CMS Data Collection
A commercial third-party research company in each country
collected the questionnaire data using various CMSs [29-34].
The specific approach used in each country was selected basedon cost, ease of use under local conditions, and likelihood ofaccessing low-income participants. The questionnaires weresuitably modified to fit the data collection method. The target
was 2500 completed surveys for each country; this sample size
would be both sufficient for the planned analyses and affordable.
The CMSs used were (1) web-based questionnaires delivered
to a research panel representative of the broader population in
the United States; (2) in-person interviews administered byhealth workers at the respondents’ homes in the states of Bihar
and Uttar Pradesh in India; and (3) questionnaires administered
via SMS text messaging to mobile phones nationally in South
Africa, which were supplemented with tablet surveys
administered in person at respondents’ homes to have a better
representation of the lowest socioeconomic group.
Multimedia Appendix 1 provides information on the commercial
survey providers and the details of each country’s CMS.

Nationally Representative Demographic Data
Demographic data collected from the app surveys and CMSs
were subsequently compared with publicly available benchmark
data (sources discussed in this section) to assess the
locale-specific representativeness of the respondent samples.
The CMS and app surveys included only persons aged 18 to 45
years. Published statistical analyses for the benchmark samples
are, for some variables, based on different age ranges and, inthe case of sensitive data (eg, contraceptive use), may be
restricted to persons who are currently married. The specificcharacteristics of the benchmark and other modality samples
are provided in the country-specific tables in the Results section.
When necessary for correct cross-modality comparison, thereported data were weighted, and this was noted in the tables
when applicable. For example, the South African DHS reports
the percentage of the entire population (of all ages) for each of
the age bins listed in the table reporting South African
JMIR Form Res 2023 | vol. 7 | e44705 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

participant demographics in the Results section. The column
for the app-based survey reports the percentage of the sample
for each of these age bins. To compare the 2 age distributions,each age bin for the DHS sample was weighted to the summed
percentage for the bins from the ages of 18 to 45 years.
Benchmarks that only reported a variable’s distribution for those
aged 15 to 49 years were not weighted.
The selected benchmarks for the United States were the US
2020 census [35] for the demographic variables and the US
Centers for Disease Control and Prevention National Survey of
Family Growth (approximately N=6000; 2017-2019) for
contraceptive use variables [36].
For the India and South Africa comparisons, the selectedbenchmarks for demographic and contraceptive use variables
were the most recently available DHS [3] data in each country(2016 for South Africa, 2019-2021 for India and Bihar, and2015-2016 for Uttar Pradesh). All DHS data were collected via

in-person interviews. The sampling method included aprobability-based multistage stratified random cluster sampleof households nationwide with poststratification weights applied
to correct for minor selection and nonresponse biases. The total
DHS sample sizes for women aged 18 to 45 years wereapproximately 8514 in South Africa and 521,300 in India. As
approximately half of the Indian CMS sample came from eachof 2 Indian states (Bihar and Uttar Pradesh), the benchmark for“Current Contraceptive Use” in the CMS sample was a 50/50
weighted average of the DHS-reported findings for these 2
states. The Indian app-based survey was nationwide; therefore,
the benchmark for this sample was India’s national DHS report.

Results
Recruitment
Table 1 reports recruitment statistics for the app-based survey
in each country.

Table 1. App-based subsamples from initial contact to completed data collection. Total number of survey invitations (in-app messages and push
notifications) sent to app users, user click-through rates, and final sample sizes eligible for the analyses in this report.

aCountryIn-app message(number of users)

Push notifications(number of users)

User click-through rate (ofthose contacted), n/N (%)

Ineligible users (screened Completed and eligibleout by survey)
for these analyses

India

104,890

66,243

10,392/171,133a (6.1)

2680

1246

South Africa

30,72019,3586,151/50,078a (12.3)

1049

879

United States

438,183

N/Ab

29,187/438,183a (6.7)

7776

10,244

Denominator = in-app message + push notifications.

b

N/A: not applicable.

Demographic Characteristics of the Study Samples
United States
Of the 2500 respondents who completed the CMS (ie, Dynata
web-based questionnaire [29]), 2464 (98.56%) answeredquestions regarding demographic attributes, contraceptive
method use, and menstrual bleeding attitudes. A total of 10,244
app respondents answered these same questions and also
explicitly affirmed that they had taken the survey seriously.
Table 2 presents the distribution of respondents in the UnitedStates by age, educational level, marital status, and residence
locale, and Table 3 presents race or ethnicity of women aged
18 to 45 years from each of the 3 data collection methods:
conventional web-based survey, app-based survey, and the 2020
US census (the benchmark) [35].
By design, the CMS sample was intended to be a good
demographic match for the benchmark. As expected, thedistribution of this survey sample’s age categories was within
3% of that of the benchmark except for overrepresentation of
the oldest age group. The app survey sample was within 3% of
the benchmark in the ages of 18 to 19 years and 30 to 34 years
but substantially overrepresented those aged 20 to 29 years andunderrepresented those aged 35 to 45 years.
Education categories in the CMS sample were within 2% of thebenchmark. The app sample was also within 2% of thebenchmark for the proportion who held high school degrees but
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

overrepresented those with higher education by 9%(7340/10,244, 71.65%). There were very few app respondents
who had not completed high school. The proportion of ruralparticipants in both the CMS and app survey was within 3% of
the benchmark. Both surveys underrepresented suburban
residents; the app survey substantially overrepresented urban
residents. The proportion of women in the United States who
had never married at the age of 33 years (the median age of theconventional survey sample) was 30%, and at the age of 26
years (the median age of the app sample), it was 62%. The CMS
substantially overrepresented never-married women by 11%(1004/2464, 40.75%); the app sample (6189/10,244, 60.42%
never married) was comparable with the benchmark.
The app and CMS samples were similar in ethnicity and racecomposition, and each sample was reasonably similar in the
distribution of the racial and ethnic minority categories to thosereported in the US census (Table 3). However, both modalities,
more so the CMS, had a lower percentage of White respondents
than did the benchmark. The CMS provider had set numericquotas for each ethnicity category. Although the intended total
sample size was achieved, the total for White respondents was
15% less than its preset quota. These differences in outcomesare at least partially attributable to the different approaches
taken to collecting race and ethnicity data. The US census asked
2 separate questions (“race” and “ethnicity”), and the survey
questionnaires asked only 1 question but allowed for multiple
responses.
JMIR Form Res 2023 | vol. 7 | e44705 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Table 2. Demographics of participants in the United States—composition of the conventional modality survey (CMS) and app-based survey samples(comprising respondents from unknown locales across the United States) compared with the composition of the US 2020 census.
CMS survey sample(total n=2464), n (%)

App-based survey sample Benchmark survey (US 2020
(total n=10,244), n (%)
census; %a)18-19

128 (5.19)

1036 (10.11)

7

20-24

340 (13.8)

3395 (33.14)b

17

25-29

378 (15.34)2526 (24.66)b

18

30-34

492 (19.97)

1794 (17.51)

19

Age (years)

35-45

1126 (45.7)bc1493 (14.57)

Median (1st quartile-3rd quartile)

33.0 (27-39)

26.0 (22-31)—d

No formal education or only primary education

175 (7.1)

38 (0.37)c

9Secondary (high school) or equivalent

702 (28.49)

2848 (27.8)

26.5

Tertiary (at least some postsecondary education)

1587 (64.41)

7340 (71.65)b63

No answer

0 (0)18 (0.18)

1.5

Rural

409 (16.6)

1243 (12.13)

14

Suburban

1195 (48.5)c4523 (44.15)c

55

Urban

860 (34.9)4464 (43.58)b

31

No answer

0 (0)14 (0.14)

0

Never married

1004 (40.75)b

6189 (60.42)62 at the age of 26 years; 30 at
the age of 33 years

Married or engaged

1202 (48.78)c

3621 (35.35)—

Divorced or separated

258 (10.47)c

421 (4.11)—

No answer

0 (0)13 (0.13)—

39

Educational level

LocaleRelationship statusa

Percentage of those aged 18-45 years for all demographics except education level, which is percentage of those aged ≥25 years.

bOverrepresentation compared with benchmark.

c

Underrepresentation compared with benchmark.

d

Not available.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Table 3. Race or ethnicity of participants in the United States—composition of the conventional modality survey (CMS) and app-based survey samples(comprising respondents from unknown locales across the United States) compared with the composition of the US 2020 census.
Selected response

CMS sample (n=2769a), n (%) App-based sample (n=11,676a), n (%) US 2020 census of the population (%)

Asian

248 (9)b

698 (6)b

5.9c

Black

392 (14.2)b1040 (8.9)b

13.4c

Hawaiian or Pacific Island Native

22 (0.8)b

49 (0.4)b

0.3c

Hispanic

550 (19.9)b1884 (16.1)b

18.9d

American Indian or Alaska Native

76 (2.7)b339 (2.9)b

1.3c

White

1436 (51.9)b7510 (64.3)b76.3c

No answer or “other”

45 (1.6)

156 (1.3)—eaResponse total is greater than the number of respondents as respondents may select more than one of the listed options. For each modality, the tableshows the percentage of responses in each category.
b

Includes those who chose (1) this category alone and (2) this category in addition to any other category.

cIncludes those who chose (1) this category alone and (2) this category and Hispanic.

dIncludes those who chose Hispanic and one or more of the other categories.

e

Not available.

India
The CMS sample, after meeting quality controls, comprised
2515 respondents from 2 states, Bihar and Uttar Pradesh. The
final app-based sample, after meeting quality controls,
comprised 1246 respondents from across India.
Compared with the Indian national DHS sample (Table 4), theapp sample overrepresented persons aged <30 years andhttps://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

underrepresented those aged ≥30 years. The disparities between
the app and benchmark samples were greater for the othervariables. The app sample substantially underrepresented thosewith less education and those belonging to historicallydisadvantaged groups (“caste”) [37]. Although 67.5% of Indians
live in rural communities, only 3.85% (48/1246) of the app
sample did. Similarly, 72% of the benchmark sample were
married or engaged, but only 30.58% (381/1246) of the app
sample reported this status.

JMIR Form Res 2023 | vol. 7 | e44705 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Table 4. India participant demographics—the app-based survey sample (respondents from unknown locales across India) compared with the national
Demographic and Health Survey (DHS) sample, and the conventional modality survey (CMS) sample (collected in Bihar and Uttar Pradesh) comparedwith the DHS samples from these same 2 states.
App-based sam- India DHS 2019-2021
ple (total(table 3.1), % of sample
n=1246), n (%) (weighted %)a

CMS sample(total n=2515),n (%)

Bihar DHS 2019-2021 Uttar Pradesh DHS(tables 3, 17), % of2015-2016 (tables 3,
sample (weighted %)a 10), % of sample

18-19

167 (13.4)b

6.8 (8.4)

72 (2.9)c

4.5 (11.7)

2.2 (5.7)

20-24

518 (41.6)b

16.5 (20.5)

718 (28.5)b

8.8 (23)

9.7 (25.5)

25-29

345 (27.7)b

16.2 (20.1)

837 (33.3)b

7.5 (19.6)

7.8 (20.5)

30-34

146 (11.7)c

13.9 (17.2)

402 (16)

6.2 (16.1)

6.5 (17.1)

35-45

70 (5.6)c27.2 (33.7)

486 (19.3)c

11.3 (29.3)

11.9 (31.2)—d

26 (24-32)——(weighted %)a

Age (years)

Median (1st quartile-3rd quartile) 24 (21-28)
Educational level
No formal education

0 (0)c22.6

550 (21.9)c

38

36

Primary

8 (0.6)c

18.6 (1-7 yearse)

1011 (40.2)b33 (1-9 yearse)

31 (1-9 yearse)

Secondary (high school)

200 (16.1)c

33 (8-11 yearse)

704 (28)b13 (10-11 yearse)

10 (10-11 yearse)

Tertiary (college or above)

1031 (82.7)b26 (≥12 yearse)

250 (9.9)c

16 (≥12 yearse)

23 (≥12 yearse)

No answer

7 (0.6)

—

0 (0)

—

—

Rural

48 (3.9)c

67.5

2482 (98.7)b84

74

Suburban

176 (14.1)

Not included

24 (1)

Not included

Not included

Urban

1012 (81.2)b

32.5

9 (0.4)c

16

26

No answer

10 (0.8)

—

0 (0)

—

—

Never married

837 (67.2)b

23.6

39 (1.6)c23

29

Married or engaged

381 (30.6)c722457 (97.7)b75

68

Divorced or separated or widowed 20 (1.6)

1

17 (0.7)

2

3

No answer

8 (0.6)—2 (0.1)——

Scheduled caste

78 (6.3)c

21.9

848 (33.7)b24

23

Scheduled tribe

23 (1.8)c

9.3

160 (6.4)

4

1

Other backward class149 (12)c

42.9

1232 (49)54

54None of the above (general catego- 961 (77.1)bry castes)

25.2

271 (10.8)c17

22

Other or no answer—

4 (0.2)1

0.2

LocaleRelationship status

Social groupa43 (3.5)

Percentage of those aged 15-49 years (weighted to match this study's age categories).

bOverrepresentation compared with benchmarks.

c

Underrepresentation compared with benchmarks.

d

Not available.

eYears of schooling.

Compared with benchmark samples from the 2 Indian statesthat were surveyed in this study (Table 4), the CMS sample had
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

a comparable percentage of those aged 30 to 34 years,
overrepresented those in their 20s, and substantially
JMIR Form Res 2023 | vol. 7 | e44705 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et alunderrepresented the youngest and oldest cohorts. The CMS
sample underrepresented those with no formal schooling or witha college education and substantially overrepresented those with
1 to 11 years of schooling, those who were married, and thosein rural communities.

substantially underrepresented those aged ≥30 years. The CMS
underrepresented those with a primary education andoverrepresented those with secondary education. The app sample
substantially overrepresented those with postsecondary
education and underrepresented those with less education.

In sum, compared with their respective benchmarks, the app
and CMS survey samples both overrepresented those in their
20s and substantially underrepresented those aged ≥35 years.
The app sample overrepresented those aged 18 to 19 years; theCMS sample underrepresented this cohort. Both modalities
underrepresented those without formal education; the app sample
dramatically overrepresented those who had at least some higher
education.

Compared with the benchmark, the CMS and app samples
overrepresented never-married persons. The CMS also
overrepresented married or engaged persons and urban residents.
However, these discrepancies likely reflect, at least to someextent, the differences between these instruments and the
response options used in the South African DHS, which did notinclude “suburban” as an option for locale but did include“living together” as an option for relationship status.

South Africa

The ethnicity or race distribution (referred to as “populationgroup” in the South African DHS) in the CMS was comparablewith that of the benchmark. In the app sample, Black African
individuals were severely underrepresented, and all other groups
were overrepresented, especially White individuals.

Compared with the South African national DHS (the benchmark;
Table 5), the CMS overrepresented those aged 25 to 29 years
and underrepresented those aged 35 to 45 years. In contrast, theapp sample overrepresented those aged <25 years andhttps://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Table 5. South Africa participant demographics—composition of the aggregated conventional modality survey (CMS) sample and the app-based sample(respondents from unknown locales across South Africa) compared with the 2016 national Demographic and Health Survey (DHS).
Aggregated CMS sample(total n=2523), n (%)

App-based sample (total
n=879), n (%)

South Africa DHS 2016 (table 3.1), % of

18-19

75 (3)

209 (23.8)b

3.4 (4.3)

20-24

559 (22.2)

303 (34.5)b

16.6 (21.2)

25-29

687 (27.2)b179 (20.4)

17 (21.7)

30-34

566 (22.4)

110 (12.5)c

15.7 (20)35-45

636 (25.2)c78 (8.9)c

25.7 (32.8)

Median (1st quartile-3rd quartile)

29.0 (24-35)

23.0 (20-28)—d

No formal education

39 (1.5)0 (0)c2

Primary education

67 (2.7)c

9 (1)c

9.1

Secondary (high school)

2184 (86.6)b

340 (38.7)c

77.1

Tertiary (college or above)

233 (9.2)

527 (60)b

11.8

No answer

0 (0)3 (0.3)0

Rural1081 (42.8)b54 (6.1)c

32.7 (“not urban”)

Suburban0 (0)453 (51.5)b

Not included

Urban

1442 (57.2)c

369 (42)c

67.3

No answer

0 (0)3 (0.3)—

Never married

1648 (65.3)b662 (75.3)b

58.5“Living together”

Not included

Not included

12.5

Married or engaged

779 (30.9)b

192 (21.8)

23.3

Divorced or separated or widowed

96 (3.8)

20 (2.3)

5.6

No answer

0 (0)5 (0.6)—

Black African

2301 (91.2)

254 (28.9)c

86.8

Colored128 (5.1)

97 (11)

8.6

Indian or Asian

38 (1.5)

47 (5.3)

sample (weighted %)a

Age (years)

Educational level

LocaleRelationship status

Population group

a

White

51 (2)

484 (55.1)

Other or no answer

6 (0.2)34 (3.9)

1.5
b

3.1
0.1

Percentage of those aged 15-49 years (weighted to match this study's age categories).

bOverrepresentation compared with benchmark.

c

Underrepresentation compared with benchmark.

d

Not available.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Contraceptive Use
United States
The app and CMS samples had similar proportions of current
contraceptive users (7748/10,244, 75.63% and 1778/2464,
72.16%, respectively), which were much higher than that of the
US benchmark sample (47%; Table 6). Of these current
contraceptive users, the CMS and benchmark respondents had

similar proportions of hormonal (approximately two-thirds;
1184/1778, 66.59% and 60%, respectively) and nonhormonal(approximately one-third; 570/1778, 32.06% and 29%,
respectively) method users. In contrast, the app sample had
more nonhormonal than hormonal method users (4147/7748,
53.52% vs 3033/7748, 39.15%), principally because of the lower
use of the pill and higher use of nonhormonal methods than in
the CMS and benchmark samples.

Table 6. Current contraceptive use in the United States—conventional modality survey (CMS) and app-based samples compared with the 2020 US
National Survey of Family Growth (NSFG) 2017 to 2019 [38].
Responses

CMS sample (n=2464), n (%) App-based sample (n=10,244), n (%) NSFG (aged 15-49 years, % of sample)

Current contraceptive users

1778 (72.2a,b)7748 (75.6a,b)

Weighted %c (47a)Hormonal methods (total)

1184 (66.6c)

3033 (39.1c,d)60c (28.4a)

LARCe methods

421 (23.7c)

1216 (15.7c)

26c (12.4a)

IUDf or IUSg

192 (10.8c,d)827 (10.7c,d)18c (8.4a)

Injection

143 (8c)43 (0.6c)

4c (2a)Implant

86 (4.8c)

346 (4.5c)

4c (2a)763 (42.9c)

1757 (22.5c)

34c (16a)

Pill

763 (42.9b,c)1586 (20.5c,d)30c (14a)Ring or patch

0 (0)171 (2.2c)

4c (2a)Morning-after pill24 (1.3c)

60 (0.8c)

N/Ah

Nonhormonal methods (total)

570 (32.1c)4147 (53.5b,c)29c (13.8a)

Condoms

354 (19.9)c

1894 (24.4)c

18c (8.4a)

Natural methods

128 (7.2)c

1598 (20.6)b,c

11c (5.4a)Copper IUD

31 (1.7)c655 (8.5)c0 (0)Other methods

57 (3.2c,d)568 (7.3)c,d12c (5.6a)686 (27.8a)2496 (24.4a)53aNone used in the last 3 months

510 (20.7a,d)1688 (16.5)a,d

34.7a

Female sterilization

N/A

N/A

18a

No answer or N/A

176 (7.1a)808 (7.9a)

N/A

Non-LARC methods

Other responsesaPercentage of survey sample.

bOverrepresentation compared with benchmark.

cPercentage of current contraceptive users (samples do not include sterilized respondents).

d

Underrepresentation compared with benchmark.

e

LARC: long-acting reversible contraception.

f

IUD: intrauterine device.

g

IUS: intrauterine system.

h

N/A: not applicable.

India
The rates of current contraceptive use (Table 7) in thebenchmark samples were much lower (24.5% in the DHS
Bihar/Uttar Pradesh sample and 29% in the DHS national
sample) than those in the Bihar/Uttar Pradesh CMS sample(2217/2515, 88.15%) or the nationwide app sample (637/1246,
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

51.12%). Nonetheless, in each sample, the proportion of current
contraceptive users relying on nonhormonal methods was clearly
the highest proportion of such users and approximately similar(71%-88%) across all 4 samples. App respondents wereapproximately twice as likely to use condoms (387/637, 60.8%)
as the other samples, a practice that largely accounts for thehigh reliance on nonhormonal methods in the app sample. More
JMIR Form Res 2023 | vol. 7 | e44705 | p. 11(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et althan half (56%) of the Bihar/Uttar Pradesh DHS sample relied
on nonhormonal methods; only 27.5% (610/2217) relied onthese methods in the Bihar/Uttar Pradesh CMS sample.

Half of the Bihar/Uttar Pradesh DHS sample but only 6.96%(175/2515) of the CMS sample in these states reported nothaving used contraception during the previous 3 months. The
rates of not using contraceptives were similar in the app(359/1246, 28.81%) and DHS (33%) national samples.

Table 7. Current contraceptive use in India—conventional modality survey (CMS) sample (collected in Bihar: n=1234; Uttar Pradesh: n=1281) comparedwith Demographic and Health Survey (DHS) aggregated samples from these same 2 states (aged 15-49 years and currently married) and app-based
sample (respondents from unknown locales across India) compared with 2019 to 2021 national DHS sample (figure 5.1 and table 5.5 in India DHS
report; aged 15-49 years and currently married).
Responses

Bihar and Uttar Pradesh
CMS sample(n=2515), n (%)

DHS (Bihar and Uttar Pradesh, App-based sample(n=1246), n (%)
approximately n=98,000)a

DHS 2019-2021 (approximately n=521,300)

2217 (88.2)b,c24.5b

637 (51.1)b,c

29b

Hormonal methods (total)

581 (26.2)c,d

15.6d

76 (11.9)d,e

27.3d

LARCf methods

176 (7.9)d

7.4d

3 (0.5)d9d

IUDg or IUSh

0

4.2d

3 (0.5)d,e

7d (2.1b)

Injection

176 (7.9)d

3.2d

0

2d (0.6b)

Implant

0

0

0

0

395 (17.8)d

8.2d

48 (7.5)

18.3d

Pill

395 (17.8)c,d

8.2d

44 (6.9)d,e

18d (5.1b)

Patch or Ring00

4 (0.6)d0

Morning-after pill

10 (0.5)d0

25 (3.9)d

0.3d (0.1d)

Nonhormonal methods (total)

1636 (73.8)d,e85d

560 (87.9)c,d71dCondoms

777 (35)d29d

387 (60.8)c,d

33d (9.5b)

Natural methods

610 (27.5)d,e56d

151 (23.7)d,e

38d (11b)

Copper IUDg

76 (3.4)d0

9 (1.4)d0

Other methods

173 (7.8)d0

13 (2)d1d (0.3b)298 (11.8)b75.5b

609 (48.9)b71bNone used in last 3 months

175 (7)b,e49.5b359 (28.8)b33bFemale sterilization

N/Ai

26b

N/A

38b

No answer or N/A

123 (4.9)b

N/A

250 (20.1)b

N/A

Current users

Non-LARC methods

Other responsesaNational

Equal weighting of Bihar and Uttar Pradesh.

b

Percentage of survey sample.

cOverrepresentation compared with benchmark.

dPercentage of current contraceptive users (samples do not include sterilized respondents).

e

Underrepresentation compared with benchmark.

f

LARC: long-acting reversible contraception.

g

IUD: intrauterine device.

h

IUS: intrauterine systemi

N/A: not applicable.

South Africa
The CMS and benchmark samples (Table 8) had similar rates
of current contraceptive use (1130/2523, 44.79% and 47%,https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

respectively), and the app sample overrepresented current users(586/879, 66.7%).
However, the CMS sample underrepresented hormonal methods
compared with the benchmark (703/1130, 62.21% vs 73%,
JMIR Form Res 2023 | vol. 7 | e44705 | p. 12(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

respectively), especially LARC methods (519/1130, 45.93% vs62%, respectively). Among those using hormonal contraception
in the CMS and benchmark samples, the respective rates of useof the pill (139/1130, 12.3% and 11%) and intrauterine devices(11/1130, 0.97% and 2%) were similar.

Table 8. Current contraceptive use in South Africa—conventional modality survey (CMS) and app-based samples compared with 2016 Demographicand Health Survey (DHS) sample (aged 15-44 years).
Responses

CMS sample (n=2523), n (%)

App sample (n=879), n (%)

DHS 2016 (n=7632)

Current users

1130 (44.8)a586 (66.7)a,b

Weighted %c (47a)Hormonal methods (total)

703 (62.2)c,d203 (34.6)c

73c

LARCe methods

519 (45.9)c

56 (9.6)c

62c

IUDf or IUSg (coil)

11 (1)c33 (5.6)c2c (1a)Injection

411 (36.4)c

14 (2.4)c53c (24a)

Implant

97 (8.6)c

9 (1.5)c7c (3a)139 (12.3)c

133 (22.7)c

11c

Pill

139 (12.3)c

125 (21.3)b,c

11c (5a)Patch or Ring

0 (0)8 (1.4)c0 (0)Morning-after pill45 (4)c14 (2.4)c<1c (<1a)Nonhormonal methods (total)

427 (37.8)b,c

289 (49.3)b,c27c

Condoms

379 (33.5)b,c

192 (32.8)b,c

26c (12a)Natural methods

48 (4.2)c

26 (4.4)c<1c (<1a)Copper IUD

0 (0)39 (6.7)c0 (0)Other methods

0 (0)32 (5.5)c<1c (<1a)1393 (55.2)a293 (33.3)a53aNone used in last 3 months

1393 (55.2)a196 (22.3)a50a

Female sterilization

N/Ah

N/A

3a

No answer or N/A

0 (0)97 (11)aN/A

Non-LARC methods

Other responsesaPercentage of survey sample.

bOverrepresentation compared with benchmark.

cPercentage of current contraceptive users (samples do not include sterilized respondents).

d

Underrepresentation compared with benchmark.

e

LARC: long-acting reversible contraception.

f

IUD: intrauterine device.

g

IUS: intrauterine system.

h

N/A: not applicable.

Although current contraceptive users were overrepresented in
the app sample (586/879, 66.7%) relative to the benchmark, theuse of hormonal methods was much lower (203/586, 34.6%).
Nonetheless, compared with the benchmark, the app sample
overrepresented pill users (125/586, 21.3%).
Injection was the most commonly used current method in South
Africa (53%); however, only 36.37% (411/1130) of the CMS
respondents and 2.4% (14/586) of the app respondents wereusing this method. In contrast, condoms were overrepresented
in both the CMS and app samples.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

Attitudes Regarding Menstrual Bleeding
For each country, we compared the data on menstrual bleeding
attitudes collected using the app survey and the CMS (Table
9). Benchmark data for these variables are not available ascomparable questions have not been included in the DHS or,
to our knowledge, in any other nationally representative survey
study in these countries. However, menstrual bleeding attitudes,
including whether people feel menstruation is a burden or abenefit (or a mix of both), can have important implications forcontraceptive decision-making and overall health [39]. As MTA
users have self-selected to track their menstrual cycles,
ascertaining whether their perspectives on menstruation differ
JMIR Form Res 2023 | vol. 7 | e44705 | p. 13(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

from that of the broader population can be important for

designing research studies and intervention protocols.

Table 9. Menstrual bleeding attitudes—conventional modality survey (CMS) and app samples.
United States
CMS sample(n=2469), n (%)

India

South Africa

App sample(n=10,244), n (%)

CMS sample(n=2515), n (%)

App sample
CMS sample(n=1246), n (%) (n=2523), n (%)

App sample(n=879), n (%)

I need to have periods to be in good health
Positive

1030 (42)4254 (42)2288 (91)936 (75)1649 (65)472 (54)

Neutral

683 (28)2455) (24)100 (4)131 (11)325 (13)

192 (22)

Negative

756 (31)3530 (34)46 (2)

173 (14)549 (22)213 (24)

No answer

0 (0)

5 (<0.5)

81 (3)

6 (<0.5)0 (0)2 (<0.5)

I wish I could take a break from having my period
Positive

1570 (64)6842 (67)1054 (42)639 (51)1124 (45)551 (63)

Neutral506 (20)1773 (17)550 (22)236 (19)283 (11)151 (17)

Negative393 (16)1624 (16)830 (33)364 (29)1116 (44)175 (20)

No answer

0 (0)

5 (<0.5)

81 (3)

7 (1)

0 (0)

2 (<0.5)

I don’t want to change my natural menstrual cycle
Positive

1036 (42)4727 (46)1110 (44)1003 (80)1638 (65)515 (59)

Neutral747 (30)2073 (20)489 (19)101 (8)302 (12)156 (18)

Negative

686 (28)

3439 (34)835 (33)

135 (11)583 (23)206 (23)

No answer

0 (0)

5 (<0.5)

81 (3)

7 (1)

0 (0)

2 (<0.5)

How often would you like to have your period?
Regular periods953 (43)a4265 (42)2169 (86)811 (65)1344 (53)448 (51)

Regular periods but able toskip sometimes

586 (26)a2241 (22)135 (5)207 (17)605 (24)

185 (21)

Not have periods except
when I want to become
pregnant

301 (14)a

1752 (17)77 (3)

123 (10)349 (14)131 (15)

Never have a period

381 (17)a

1973 (19)53 (2)

102 (8)225 (9)111 (13)

No answer

248 (10)13 (<0.5)

81 (3)

3 (<0.5)0 (0)4 (<0.5)

Who would you be comfortable talking to about your period? (check all that apply)aAnyone566 (23)3878 (38)177 (7)440 (35)521 (21)320 (36)

Some or all friends

1373 (56)9275 (91)488 (19)1034 (83)448 (18)744 (85)

Family members

1278 (52)7891 (77)851 (34)901 (72)380 (15)633 (72)

My partner or partners

1396 (57)9152 (89)2033 (81)1115 (89)640 (25)759 (86)

Physician or nurse

1665 (67)9764 (95)1178 (47)1071 (86)977 (39)779 (89)

Pharmacist

791 (32)6201 (61)160 (6)574 (46)

182 (7)554 (63)

Health worker

770 (31)7939 (77)347 (14)791 (63)410 (16)631 (72)

None of these327 (13)91 (1)

95 (4)

27 (2)107 (4)14 (2)

Percentage of all those (n=2221) who answered this question.

In the United States, the app and CMS samples had similar
distributions of responses to the first 4 questions. Pluralities
agreed that periods are necessary for good health, that they did
not want to change their natural menstrual cycle, and that theywanted regular periods. However, a clear majority of eachsample (CMS: 1570/2469, 64%; app: 6842/10,244, 67%) also
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

agreed with the following statement: “I wish I could take a breakfrom having my period.” Notably, a much larger proportion of
the Clue app sample than of the CMS sample was comfortable
talking about their periods with a variety of people (medical
personnel, partners, friends, and family). Only somewhat morethan half of the CMS sample was comfortable speaking to these
JMIR Form Res 2023 | vol. 7 | e44705 | p. 14(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
categories of persons; less than a third were comfortablespeaking to pharmacists and health care workers.
In India, most of the CMS and app samples agreed that periods
are necessary for good health and wanted regular periods (Table
9). However, the CMS sample was more likely to prefer to haveregular periods (86%), and only 42% selected that they wanted
to take a break from their periods compared with 65% and 51%,
respectively, for the app sample. In contrast, most (80%) of theapp sample selected that they did not want to change theirnatural cycle compared with only 44% of the CMS sample. A
high percentage of the national sample of app users were open
to speaking about their period with medical staff, partners,
friends, and family; slightly less than half were comfortablespeaking to pharmacists. The CMS conducted in Bihar and Uttar
Pradesh found high comfort in respondents regarding speaking
with their husbands but much less comfort speaking with anyone
else.

Shea et alsize was quickly met using the web-based CMS in the UnitedStates, the SMS text messaging–based CMS in South Africa
was not able to meet the target quotas and required
supplementation using another method (in-person tablet
surveys). In India, in-person interviews were able to meet thedesired sample size, but because of COVID-19 pandemic
challenges, the protocols and time frame had to be adjusted to
do so.
Thus, when user numbers were sufficient, app-distributed
surveys were able to quickly capture large samples on par withother methods and at low cost. An additional advantage of app
distribution, as with other digital platforms with a global reach,
was that recruitment could be deployed remotely andsimultaneously across countries. In addition, respondents wereable to participate from the safety and comfort of the locationof their choosing, which enhanced ease of participation during
the COVID-19 pandemic.

In South Africa, the app and CMS samples were very similarin the distributions of preferred frequency of menstrual bleeding
and maintaining the natural menstrual cycle (majorities wanted
regular periods and did not want to change their cycles). Mostof both South African samples also agreed that periods areneeded to be in good health, although the proportion of those
who agreed with this opinion was higher in the CMS sample.
The app sample was much more likely to want a break fromperiods than the CMS sample (65% vs 45%). High proportionsof the app sample were comfortable talking about their periods
with medical staff, partners, and friends; a smaller but still
majority percentage of app users would speak with family,
health care workers, and pharmacists. Although 39% of the
CMS sample were comfortable speaking to physicians andnurses, a quarter or less was comfortable speaking to anyone
else.

Demographic Variables

Discussion

When comparing the app users to the respective benchmarks
across all countries, the app samples consistently
overrepresented those who were younger, were urban residents,and had more education. In particular, postsecondary education
was higher in the app user group than in the broader population,although this disparity was more marked in India and South
Africa than in the United States. These patterns are consistent
with younger, more educated persons being more likely to own
a smartphone and use health apps [10,40]. In India and South
Africa, the app samples also overrepresented never-marriedpersons from more socially or economically advantaged groups,
whereas in the US app sample, relationship status and ethnicity
or race representation were more comparable with thebenchmark.

Principal Findings
Overview
The goal of this study was to investigate the feasibility of
conducting reproductive health surveys using mainstream
MTAs. The expanding use of smartphones and related apps
provides a novel means of reaching large and diverse pools ofpotential research participants, but little is known about thesuitability of using this novel modality to gather reproductive
health data. In this study, we evaluated ease of use; ability toachieve desired sample sizes; and representativeness with regard
to demographic variables, contraceptive use, and menstrual
bleeding attitudes of the responses collected using the Clue
MTA compared with responses collected using CMSs or
nationally representative benchmark data in 3 target countries.

Performance
In the United States, the target sample size of 10,000 completed
questionnaires collected via the Clue app was reached in <3weeks, whereas targets were not met in India or South Africa,
where the number of active users was lower. Similar trends
were observed for the use of CMSs. Although the target sample
https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

In each of the 3 countries, there were clear demographic
differences between the samples of the different modalities,
with neither the app samples nor the CMS samples emerging
as a consistently better match to the distribution of attributes
observed in the respective national-level benchmarks. Despite
the efforts to achieve reasonable demographic representativenessin the CMS samples, it is notable that this goal was not achieved.
The CMS samples deviated from the benchmarks incountry-specific ways, sometimes over- and other times
underrepresenting demographic subgroups. For example, the
CMS sample underrepresented married individuals in the UnitedStates, overrepresented married individuals and rural residents
in India (to the near exclusion of any other locales or relationshipstatuses), and skewed toward midrange age groups in South
Africa.

Of the 3 countries, app users in the United States were the mostdiverse sample, likely a result of the larger user base and higher
market penetration. In the United States, it is estimated that
one-third of women use MTAs [11]. Although MTA use iscommon worldwide, users in countries with more limited
smartphone and data access such as India and South Africa may
be more representative of more advantaged early adopters of
the technology, a group that may have unique characteristics.
However, as current trends continue and the gender and
JMIR Form Res 2023 | vol. 7 | e44705 | p. 15(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
socioeconomic divide narrows with regard to access totechnology, these differences will lessen.
It should also be noted that, for any platform, the user base isreflective of the app’s current features and the company’smarketing strategies. For example, when this study was
conducted, the primary functionality of the Clue app was general
period tracking. However, with additional features such as
potential modes for trying to conceive or tracking
perimenopause symptoms, the expected user base would besignificantly different. Thus, platform functionality and otherfactors that could influence user base characteristics should be
evaluated when selecting a research platform and developing
strategies intended to achieve study samples that are suitable
for the specific research questions.

Current Contraceptive Use
The app samples in all 3 countries had substantially higher
proportions of current contraceptive users than the respective
benchmark samples. In the CMSs, the United States and India
also had higher proportions of current contraceptive users,
whereas the South African sample was similar to the benchmark.
Although the proportion of contraceptive types varied acrosssurvey modalities, in all 3 countries, the app sample
overrepresented nonhormonal method users. This is to be
expected as, at the time of the study, the Clue app was mostcommonly used for menstrual tracking and period predictions.
Thus, those who do not have menstrual periods (because of, forexample, hormonal contraceptive–induced amenorrhea or
hormonal regimens that produce predictable periods) may beless likely to find value in such an app and, therefore, are lesslikely to be represented in the data sample. Although Clue does
have features for hormonal contraceptive users, such as
contraceptive tracking and reminders, these are less frequently
used than features for tracking monthly bleeding by
nonhormonal contraceptive users.
The CMS samples deviated from the benchmarks in ways thatare likely reflective of the unique circumstances (ie, the
COVID-19 pandemic) at the time of the research. For example,
in the CMS sample in India, there was a higher current use ofcontraceptives, including hormonal methods, compared withthe benchmark. In this instance, the CMS interviews wereconducted by local health workers (with extra safety protocols
in place because of the COVID-19 pandemic). The refusal rates
to participate were much higher than in typical research in thearea, likely because of pandemic fears. In addition, there werelocal initiatives to increase access to contraceptives, including
long-acting injectables, during the pandemic. Women who
agreed to participate in the research may have been more
comfortable with local health workers and local initiatives,potentially biasing the sample toward certain characteristics.
Thus, a comprehensive evaluation of the sample and external
factors that could influence research results should always be
considered during research planning and data interpretation
regardless of method.

Menstrual Bleeding Attitudes

Shea et al
the broader population with regard to attitudes toward menstrual
bleeding (such as whether they wanted to have regular cycles
vs preferring cessation of menstruation or whether they felt theyneeded to have periods to be in good health), which led toself-selection for using MTAs. However, in each of the 3
countries, bleeding attitudes from the respective app survey and
CMS samples were reasonably similar.
The one exception was for the following question: “Who would
you be comfortable talking to about your period?” Although
CMS respondents in all countries were generally not comfortablediscussing their periods with a wide array of others, most Clue
app respondents were comfortable talking with their partners,
friends, family, and health care providers. Furthermore, there
was little difference between the Clue users’ responses acrosscountries.
These patterns suggest that, although attitudes toward
menstruation are generally similar between MTA users and thegeneral population, app users may be more receptive to
responding to questions on sensitive topics regarding women’s
health. In addition, having an interest in the subject matter(demonstrated by their interest in tracking aspects of their own
health) indicates that app users may be more invested, comparedwith a general audience, in contributing to research on a
personally meaningful topic. Furthermore, entering responses
via a smartphone app may be perceived as more private thansome CMS methods, a factor that may prompt MTA respondents
to answer questions regarding reproductive health moretruthfully and comprehensively.

Study Limitations
Different CMSs and strategies were used to optimize the CMS
protocols in each of the 3 countries. These decisions were basedon several factors, including (most importantly) local experts’
advice on which modalities could be expected to achieve a
sufficient sample of diverse respondents at a low cost. Although
the same base survey questions were used for all CMSs, different
modifications were used to meet the requirements of each. For
example, the SMS text message questions and answers in South
Africa had to be modified to fit very low limits on the numberof characters that could be used. This modification was
acceptable for the goals of this study, but these differences inwording may limit the comparability of CMS findings acrossthe 3 countries.
In addition, this research was conducted during the COVID-19
pandemic, which affected data collection. Some protocols were
adjusted, such as the modification of interview protocols in
India to ensure the safety of participants and interviewers, andsome timelines for data collection were shifted, again,
particularly in India as the interviews were conducted
face-to-face. These unavoidable changes resulted in slightly
different time frames for data collection. As the impact of thepandemic also varied by country and time frame, variables suchas willingness to participate, current contraceptive methods
used, and the resulting needs and preferences of respondents
were differentially affected. These factors further limited the
direct comparability of CMS data across samples and countries.

Given the specific purpose of MTAs for tracking menstrual
cycles, it was anticipated that these app users could differ fromhttps://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 16(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
When evaluating the Clue samples specifically, there was likely
selection bias in who chose to participate in surveys sent via
the app. As demographic information is not routinely collectedby the Clue app, it is not possible to ascertain whether thesesurvey participants were a reasonable representation, at the timeof data collection, of the overall Clue user base. In addition, thesmaller size of the Clue samples from India and South Africa
may have limited our ability to draw conclusions about app
users in these countries. However, as with any commercial
platform, user demographic characteristics are only a snapshotof the current user population. As a user base likely changes
over time, the specific demographic profile of any researchsample is best assessed concurrently with the collection of thedata of principal interest in a given study.

Conclusions
This study found that MTAs with a sufficiently large user baseprovide a quick and inexpensive means of reaching potential
participants for reproductive health research. In all 3 target
countries, Clue app survey respondents were younger, moreeducated, more likely to be urban residents, and more likely touse nonhormonal contraceptive methods than the national
benchmark samples. Within each country, the app-based and
CMS samples expressed comparable attitudes toward
menstruation. However, MTA users were more comfortable

Shea et al
discussing their menstrual cycles with others, which suggeststhat MTA users may be more inclined to respond truthfully andfully to sensitive reproductive health questions.
These demographic and attitudinal consistencies in the data, as
well as the ability to distribute surveys remotely andsimultaneously across countries using MTAs, lends strength tocross-country comparisons of app-based survey findings. In
contrast, CMS samples deviated in country-specific ways,
sometimes over- and other times underrepresenting demographic
and contraceptive use subgroups compared with the benchmark
samples. Thus, regardless of the research method, it is important
to define the desired participant sample and collectsociodemographic data to confirm the composition of theobtained sample. Nonetheless, in choosing a research modality,
factors such as cost, target sample size, or the need for a specifictarget group may be more important than demographic
representativeness. A benefit of mainstream MTAs forreproductive health data collection are the large-scale, globally
distributed user groups with a heightened interest in this aspectof health and who likely have greater comfort in discussingthese and related (often taboo) issues. Thus, apps such as Clue
offer an alternative platform for data collection and an exciting
opportunity to understand the role of population-specificsociocultural beliefs and practices in achieving satisfactory
reproductive health for all.

Acknowledgments
This paper is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation,
distribution, and reproduction in any medium or format as long as appropriate credit is given to the original authors and the source,
a link is provided to the Creative Commons license, and it is indicated whether changes were made. The images or other third-party
material in this paper are included in its Creative Commons license unless indicated otherwise in a credit line to the material. Ifmaterial is not included in the paper’s Creative Commons license and the intended use is not permitted by statutory regulations
or exceeds the permitted use, permission will need to be obtained directly from the copyright holder. To view a copy of thislicense, visit the website [41]. The Creative Commons Public Domain Dedication waiver [42] applies to the data made available
in this paper unless otherwise stated in a credit line to the data. Financial support for this work was provided by the Bill andMelinda Gates Foundation (INV-009978). The funding body collaborated in the development of the project’s general goals. It
played no role in the collection, analyses and interpretation of data, nor in the preparation of this manuscript and the decision tosubmit this paper for publication.

Data Availability
The deidentified data sets used and analyzed during this study are available at the Dryad Digital Repository [39].

Conflicts of Interest
AAS is an employee of Biowink GmbH. JT is a paid consultant for Biowink GmbH. VJV is a paid consultant for Biowink GmbH.

Multimedia Appendix 1
Conventional modality survey methods for each study country.
[DOCX File , 17 KB-Multimedia Appendix 1]

References
1.
2.

Reproductive health. National Institute of Environmental Health Sciences. 2022. URL: https://www.niehs.nih.gov/health/
topics/conditions/repro-health/index.cfm [accessed 2022-12-12]
Reproductive health. World Health Organization. URL: https://www.who.int/westernpacific/health-topics/reproductive-health[accessed 2022-12-12]https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 17(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
3.

4.
5.

6.

7.

8.

9.
10.
11.
12.
13.
14.

15.

16.

17.
18.

19.

20.

21.

22.

23.
24.

The Demographic and Health Surveys (DHS) Program has collected, analyzed, and disseminated accurate and representative
data on population, health, HIV, and nutrition through more than 400 surveys in over 90 countries. The Demographic and
Health Surveys. URL: https://dhsprogram.com/ [accessed 2023-08-14]
Krieger N. A century of census tracts: health and the body politic (1906-2006). J Urban Health. 2006 May 14;83(3):355-361[FREE Full text] [doi: 10.1007/s11524-006-9040-y] [Medline: 16739037]
Greenleaf AR, Gibson DG, Khattar C, Labrique AB, Pariyo GW. Building the evidence base for remote data collection in
low- and middle-income countries: comparing reliability and accuracy across survey modalities. J Med Internet Res. 2017
May 05;19(5):e140 [FREE Full text] [doi: 10.2196/jmir.7331] [Medline: 28476728]
Greenleaf AR, Gadiaga A, Choi Y, Guiella G, Turke S, Battle N, et al. Automated and interviewer-administered mobile
phone surveys in Burkina Faso: sociodemographic differences among female mobile phone survey respondents andnonrespondents. JMIR Mhealth Uhealth. 2020 Jul 14;8(7):e17891 [FREE Full text] [doi: 10.2196/17891] [Medline:
32673250]
Poushter J. Smartphone ownership and internet usage continues to climb in emerging economies. Pew Research Center,
Global Attitudes Project. 2016 Feb 22. URL: https://www.pewresearch.org/global/2016/02/22/
smartphone-ownership-and-internet-usage-continues-to-climb-in-emerging-economies/ [accessed 2023-01-24]
LeFevre AE, Shah N, Bashingwa JJ, George AS, Mohan D. Does women's mobile phone ownership matter for health?
Evidence from 15 countries. BMJ Global Health. 2020 May 17;5(5):e002524 [FREE Full text] [doi:10.1136/bmjgh-2020-002524] [Medline: 32424014]
Dorsey ER, Chan YF, McConnell MV, Shaw SY, Trister AD, Friend SH. The use of smartphones for health research. AcadMed. 2017 Feb;92(2):157-160 [doi: 10.1097/ACM.0000000000001205] [Medline: 27119325]
Fox S, Duggan M. Mobile health 2012. Pew Research Center: Internet Science & Tech. 2012 Nov 08. URL: https://www.
pewresearch.org/internet/2012/11/08/mobile-health-2012/ [accessed 2021-09-02]
Health apps and information survey. Henry J. Kaiser Family Foundation. 2019 Sep 10. URL: https://www.kff.org/other/poll-finding/kff-health-apps-and-information-survey/ [accessed 2023-03-12]
Wartella E, Rideout V, Montague H, Beaudoin-Ryan L, Lauricella A. Teens, health and technology: a national survey.
Media Commun. 2016 Jun 16;4(3):13-23 [FREE Full text] [doi: 10.17645/mac.v4i3.515]
Alvergne A, Högqvist Tabor V. Is female health cyclical? Evolutionary perspectives on menstruation. Trends Ecol Evol.
2018 Jun;33(6):399-414 [doi: 10.1016/j.tree.2018.03.006] [Medline: 29778270]
Hennegan J, Winkler IT, Bobel C, Keiser D, Hampton J, Larsson G, et al. Menstrual health: a definition for policy, practice,and research. Sex Reprod Health Matters. 2021 Dec 29;29(1):31-38 [FREE Full text] [doi: 10.1080/26410397.2021.1911618][Medline: 33910492]
Shea AA, Vitzthum VJ. The extent and causes of natural variation in menstrual cycles: integrating empirically-based modelsof ovarian cycling into research on women’s health. Drug Discov Today Dis Models. 2020;32:41-49 [FREE Full text] [doi:10.1016/j.ddmod.2020.11.002]
Shea AA, Kulkarni M, Thornburg J, Ventola C, Vitzthum VJ. A bother or a benefit? How contraceptive users balance thetrade-offs between preferred menstrual bleeding patterns and preferred contraceptive methods in India, South Africa, andthe United States. Women Reprod Health. 2023 [FREE Full text] [doi: 10.1080/23293691.2023.2267533]
Bouman A, Heineman MJ, Faas MM. Sex hormones and the immune response in humans. Hum Reprod Update. 2005
Aug;11(4):411-423 [FREE Full text] [doi: 10.1093/humupd/dmi008] [Medline: 15817524]
Lorenz TK, Worthman CM, Vitzthum VJ. Links among inflammation, sexual activity and ovulation: evolutionary trade-offs
and clinical implications. Evol Med Public Health. 2015 Dec 16;2015(1):304-324 [FREE Full text] [doi:10.1093/emph/eov029] [Medline: 26675298]
Barsom SH, Mansfield PK, Koch PB, Gierach G, West SG. Association between psychological stress and menstrual cycle
characteristics in perimenopausal women. Women's Health Issues. 2004 Nov;14(6):235-241 [doi: 10.1016/j.whi.2004.07.006][Medline: 15589774]
Reilly TJ, Sagnay de la Bastida VC, Joyce DW, Cullen AE, McGuire P. Exacerbation of psychosis during the perimenstrualphase of the menstrual cycle: systematic review and meta-analysis. Schizophr Bull. 2020 Jan 04;46(1):78-90 [FREE Fulltext] [doi: 10.1093/schbul/sbz030] [Medline: 31071226]
Hamelin BA, Méthot J, Arsenault M, Pilote S, Poirier P, Plante S, et al. Influence of the menstrual cycle on the timing ofacute coronary events in premenopausal women. Am J Med. 2003 May;114(7):599-602 [doi:10.1016/s0002-9343(03)00051-2] [Medline: 12753884]
Prior JC. Women’s reproductive system as balanced estradiol and progesterone actions—a revolutionary, paradigm-shifting
concept in women’s health. Drug Discov Today Dis Models. 2020;32:31-40 [FREE Full text] [doi:10.1016/j.ddmod.2020.11.005]
Clue. Biowink. URL: https://helloclue.com/ [accessed 2020-03-15]
Cahill N, Sonneveldt E, Stover J, Weinberger M, Williamson J, Wei C, et al. Modern contraceptive use, unmet need, anddemand satisfied among women of reproductive age who are married or in a union in the focus countries of the Family
Planning 2020 initiative: a systematic analysis using the Family Planning Estimation Tool. Lancet. 2018 Mar
03;391(10123):870-882 [FREE Full text] [doi: 10.1016/S0140-6736(17)33104-5] [Medline: 29217374]

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

Shea et al

JMIR Form Res 2023 | vol. 7 | e44705 | p. 18(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
25.

26.
27.

28.

29.
30.
31.
32.
33.
34.

35.
36.
37.
38.
39.

40.

41.
42.

Shea et al

Laher F, Todd CS, Stibich MA, Phofa R, Behane X, Mohapi L, et al. A qualitative assessment of decisions affecting
contraceptive utilization and fertility intentions among HIV-positive women in Soweto, South Africa. AIDS Behav. 2009
Jun 24;13 Suppl 1(S1):47-54 [doi: 10.1007/s10461-009-9544-z] [Medline: 19308719]
Ndwamato NN, Ogunbanjo GA. The beliefs and practices of Tshivenda-speaking multiparous women on contraception: aqualitative study. S Afr Fam Pract. 2009 Jul;51(4):340-342 [FREE Full text] [doi: 10.1080/20786204.2009.10873876]
Shakya HB, Dasgupta A, Ghule M, Battala M, Saggurti N, Donta B, et al. Spousal discordance on reports of contraceptive
communication, contraceptive use, and ideal family size in rural India: a cross-sectional study. BMC Women's Health.
2018 Sep 04;18(1):147 [FREE Full text] [doi: 10.1186/s12905-018-0636-7] [Medline: 30180845]
Todd CS, Stibich MA, Laher F, Malta MS, Bastos FI, Imbuki K, et al. Influence of culture on contraceptive utilization
among HIV-positive women in Brazil, Kenya, and South Africa. AIDS Behav. 2011 Feb 26;15(2):454-468 [doi:10.1007/s10461-010-9848-z] [Medline: 21110078]
World’s largest first party data platform. Dynata. URL: https://www.dynata.com/ [accessed 2023-08-14]
Sigma Social Research. Sigma. URL: https://www.sigma-india.in [accessed 2023-08-15]
GeoPoll research services and mobile surveys in emerging markets. GeoPoll. URL: https://www.geopoll.com/ [accessed2023-08-14]
Home page. M-Ploy. URL: http://www.mployresearch.co.za/ [accessed 2023-08-14]
Geopoll audience measurement (GAM) methodology and interface overview. GeoPoll. 2019. URL: https://cdn2.hubspot.net/
hubfs/325431/GAM_Methodology_and_Interface_Overview_August2019.pdf [accessed 2023-11-06]
Ntloedibe M, Ngqinani P. Understanding the living standards measure segmentation in South Africa. United States
Department of Agriculture & Global Agricultural Information Network. URL: https://apps.fas.usda.gov/newgainapi/api/
Report/DownloadReportByFileName?fileName=Understanding%20the%20Living%20Standards%20Measure%20
Segmentation%20in%20South%20Africa_Pretoria_South%20Africa%20-%20Republic%20of_11-13-2020 [accessed2023-12-03]
U.S. Census Bureau QuickFacts: United States. United States Census Bureau. 2021 Jul 01. URL: https://www.census.gov/
quickfacts/fact/table/US/PST045221 [accessed 2023-01-24]
NSFG - Listing C - Key Statistics from the National Survey of Family Growth. National Center for Health Statistics. 2022
Jun 29. URL: https://www.cdc.gov/nchs/nsfg/key_statistics/c-keystat.htm [accessed 2022-10-10]
Sahgal N, Evans J, Salazar AM, Starr KJ, Corichi M. Religion in India: tolerance and segregation. Pew Research Center.
2021 Jun. URL: https://www.pewresearch.org/religion/2021/06/29/attitudes-about-caste/ [accessed 2022-10-31]
NCHS Data Brief, number 288. Centers for Disease Control and Prevention. 2020 Oct. URL: https://www.cdc.gov/nchs/
data/databriefs/db388-tables-508.pdf#page=2 [accessed 2023-01-24]
Shea AA, Wever F, Ventola C, Thornburg J, Vitzthum VJ. More than blood: app-tracking reveals variability in heavy
menstrual bleeding construct. BMC Women's Health. 2023 Apr 11;23(1):170 [FREE Full text] [doi:10.1186/s12905-023-02312-4] [Medline: 37041503]
Carroll JK, Moorhead A, Bond R, LeBlanc WG, Petrella RJ, Fiscella K. Who uses mobile phone health apps and does use
matter? A secondary data analytics approach. J Med Internet Res. 2017 Apr 19;19(4):e125 [FREE Full text] [doi:10.2196/jmir.5604] [Medline: 28428170]
CC by 4.0 deed: attribution 4.0 international. Creative Commons. URL: https://creativecommons.org/licenses/by/4.0/[accessed 2023-11-07]
CC0 1.0 deed: CC0 1.0 universal. Creative Commons. URL: https://creativecommons.org/publicdomain/zero/1.0/ [accessed2023-11-07]

Abbreviations
CMS: conventional modality survey
DHS: Demographic and Health Survey
mHealth: mobile health
MTA: menstrual tracking app

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 19(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Shea et al

Edited by A Mavragani; submitted 17.03.23; peer-reviewed by M Stevens-Uninsky, V Jennings, N Levitz; comments to author 12.06.23;
revised version received 30.08.23; accepted 11.09.23; published 01.12.23
Please cite as:Shea AA, Thornburg J, Vitzthum VJAssessment of App-Based Versus Conventional Survey Modalities for Reproductive Health Research in India, South Africa, and theUnited States: Comparative Cross-Sectional StudyJMIR Form Res 2023;7:e44705
URL: https://formative.jmir.org/2023/1/e44705
doi: 10.2196/44705
PMID: 38039064©Amanda A Shea, Jonathan Thornburg, Virginia J Vitzthum. Originally published in JMIR Formative Research(https://formative.jmir.org), 01.12.2023. This is an open-access article distributed under the terms of the Creative Commons
Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete
bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license
information must be included.

https://formative.jmir.org/2023/1/e44705

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e44705 | p. 20(page number not for citation purposes)