Title: An Evaluation of Menstrual Health Apps’ Functionality, Inclusiveness, and Health
Education Information
Version: 0 Date: 02 Mar 2024
Reviewer's report:
The authors provide an interesting and constructive evaluation of 14 freely available apps.  The
manuscript is mostly clear, and the framework used (with some further clarification) could
provide a helpful tool for future research, developers, and practitioners to consider the functions
and utility of MH apps. I appreciated the emphasis on understanding the symptoms across the
menstrual cycle, and the need for apps to better account for severity. The findings may also
benefit users and healthcare providers and help decision-making, and promote further discussion
on app choice and thinking needs. Some changes should be made to make the results more
reliable and valid (further details on definitions, reviewer team etc.)
Abstract:
Abstract states search terms:
“period pain,” “period app,” and “menstrual app” - search terms.
But in the methods, “period pain, period app, and menstrual cramp”
Also, any reason why not period tracker or menstrual cycle tracker?
Intro:
The introduction is brief but clear. Some references might need to be included (just suggestions),
but if I remember correctly, Moglia et al. used a similar scoring method.
Moglia ML, Nguyen HV, Chyjek K, Chen KT, Castaño PM. Evaluation of Smartphone
Menstrual Cycle Tracking Applications Using an Adapted APPLICATIONS Scoring System.
Obstet Gynecol. 2016 Jun;127(6):1153-1160. doi: 10.1097/AOG.0000000000001444. PMID:
27159760. >.
Eschler, Jordan PhD; Menking, Amanda MS; Fox, Sarah PhD; Backonja, Uba PhD, RN.
Defining Menstrual Literacy With the Aim of Evaluating Mobile Menstrual Tracking
Applications. CIN: Computers, Informatics, Nursing 37(12):p 638-646, December 2019. | DOI:
10.1097/CIN.0000000000000559
Pichon et al – you mention this study, but it is contains a review/screening of apps (revised: You
mention clearly in discussion, could make clearer in intro too).
Tylstedt B, Normark M and Eklund L (2023) Reimagining the cycle: interaction in self-
tracking period apps and menstrual empowerment. Front. Comput. Sci. 5:1166210. doi: 10.3389/
fcomp.2023.1166210
“thereby reinforcing the stigma that symptoms are part of a menstrual cycle that people should
accept [11, 12]”
I’m not sure stigma is the correct word here, common belief? Misconception?
“Thus, it is important to determine whether menstrual health apps meet users’ diverse needs and
preferences.”
- Might be good to share how you decided on these preferences. There isn’t much reference to
studies that explore preferences/needs/user experiences, or maybe a reflexive note is needed...?
OR simply reframe as I am not sure this design can determine whether apps meet needs and
preferences, without further research.
“The findings from this study will help inform app users, healthcare providers, and app
developers.” ….to ?
Could you add a sentence on how and what it can inform from the start?
Methods:
Some criteria need to be more detailed/clearer to show how you decided on the scores–
particularly for readers who may not be familiar with the functions of MH apps.
Irregularity:
“The inclusiveness of cycle lengths and regularities was determined by the ability to enter cycle
length and tailor them to irregular cycles (i.e., cycle lengths other than the typical 28-day cycle)”
cont.
- Could you provide more/clearer detail on how you defined ability to tailor irregular cycles. Was
there a focus on cycle length beyond “assumed standard” 28 days? This is what the description
implies. Or were you able to test for compatibility with various menstrual irregularities beyond
longer or short cycle length. People with menstrual irregularities have reported finding apps
challenging, and the inability to cater to irregularity has been cited as a reason for stopping use,
or feeling frustrated or even self-blame/sense of abnormality in some of the user experience
literature (Patel et al. 2024; Levy & Romo-Aviles 2019) and others have found that apps don’t/
can’t (really) make sense of irregularity (Pichon et al. 2021) So, that 100% off apps can tailor
for this, is encouraging but also surprising – important to make the parameters clear particularly
as this is a 0 -1 scale.
"Accessibility"
You use the term accessibility, which I think many readers may link to accessibility features for
people with disabilities. There is no mention of MH app accessibility for people with disabilities.
Rather it seems like you mean connectivity, localisation etc.
It would be good to specify if (or why it wasn’t feasible to) test for usability/functionality of
accessibility features across the sample of apps (voiceover, haptics, etc.) Or maybe you could
use a more specific term? Or define why you use accessibility.
MH information:
“First, we looked at the reliability of the menstrual health information, the comprehensiveness of
the information, if the information came from a credible source, and at the clarity and accuracy
of the charts, graphs, images, and/or videos”
- Needs more detail or a table on how these were evaluated. What are the criteria for a
‘comprehensive” or a “credible source”? How did you decide on scores for clarity and accuracy?
Could you be more explicit on the expertise of the review team or framework?
"Second, we assessed if apps had a presence of additional health information. We did this by
seeing if each app provided information on hormonal changes throughout the menstrual cycle."
- Could you make this a bit clearer, I am not sure what you mean by additional health information.
Results:
- Also consider the above suggestions in the results.
Fertility Goals. "All but one (i.e., Flo Period Tracker and Calendar) app had an ovulation
prediction function. However, some apps (14.2%) required the user to go into settings and turn
on ovulation prediction and fertility planning functions, depending on the user’s personal fertility
goals."
- Is there a typo somewhere?? States 85.7% in the graph, But here you’re saying 95%+ (all but
one)? Does Flo not predict ovulation? Or are the 14% (n=2) those that don’t, as they require
going into settings? If so, would this still count as predicting ovulation… could you clarify this
section?
"Information on When to Seek Professional Care. Cycle and Period Tracker-Femia and Period
Tracker Eve were the only two apps that encouraged users to seek medical attention for their
menstrual symptoms. And Information on Menstrual Symptom Treatments."
- Could you provide more detail (maybe in the methods/screen section) on how you did this? Did
you enter specific symptoms,  for how long, etc? (apologies if I missed this!) This could apply
for some other sections as I imagine to test some of these functions you would need to enter
data and become a “user” as some apps will track symptoms/patterns over time, or presence of
symptoms during a certain phase of the cycle.
Discussion
“It is worth noting that none of the reviewed apps involved health professionals to develop their
symptom trackers. “
That is a concerning key finding! Maybe in the findings where you state this, could you clarify
how you determined this, or possible suggestion to reframe some of these claims based on your
data, for example, “no evidence of health professional involvement was found”
“In addition, future studies should analyze how apps cater to different ages and cultures”
- Why? How?