Title: Effect of mentorship and a mHealth application in updating provider skills and knowledge
in maternal and newborn care in two informal settlements of Nairobi
Version: 2 Date: 19 Aug 2023
Reviewer's report:
The topic and focus of the study are relevant and important for understanding how support
for HCPs especially in more challenging settings like informal settlements in LMICs can be
tackled to improve the quality of provision of maternal and newborn services and therefore
health outcomes of women and babies. The intervention programme itself is an interesting one
to study both given the competence improvement approach and methods and the circumstances
(Covid) under it which it was implemented.
However, the methods to generate findings (assessment of competencies under a before-after
framework) is somewhat unoriginal and while it provides insight and confirms learnings from
other studies in terms of factors that affect the effectiveness of interventions, in its current
format the manuscript does not really provide as much original learning. With no longitudinal
measurement of retention, or a more rigorous assessment the effect of continued learning (from
the DELTA or MENTORS)  gives somewhat limited insight into the role of the two platforms
in improving competencies and the sustainability of the change (leading to behaviour change).
Below are some suggestions for what may be possible to explore by the authors to develop
a more in-depth understanding from the programme and therefore increase the impact of the
manuscript.
- The setting of informal settlements being the context for the work at present is not really
addressed other that by flagging the importance of understanding such contexts better. It would
be useful to include more specific insights and provide learnings from the programme as
compared to existing literature specifically to identify how this work helps to address the gaps.
- The methodology for the assessment may not be able sufficiently complex for more insight
than what is currently provided but that means the interpretation of the findings is what needs to
be improved to elevate the manuscript to a higher level for publishing. This needs to incorporate
self-learning v supervised learning, ability to practice (this is mentioned in the manuscript in
the statement that those from dispensaries with less exposure to certain services less confident
in the those skills); challenges such as shortages of various types, staff rapport and teamwork,
therefore 'soft skills' aspect of healthcare service provision. If there is any data from the study
which can provide more insight into the nuances of what can be used to understand and explain
the results from the knowledge assessment it would be greatly beneficial for the manuscript.
- Is there any more insight from the cross analysis of the effect of the two interventions (DELTA
and MENTORS) that may be of benefit to include? This may offer more insight into what works
or does not work to provide guidance for future programmes?
- Would the authors have any data on the learning experience from the qualitative study? That
may provide useful insights from the platform users on what worked what did not for them?
There are elements of that included in the manuscript, so it may be perhaps an idea to focus on
this aspect for the manuscript more directly?
- Would it be possible to provide more insight from health outcomes data from the facilities
included in the programme to assess if the additional training did change the care provision?
(Although this may be confounded by the effect of Covid on service provision overall therefore
may not offer as much insight.)
- The presentation of the results at present reads quite compartmentalised, especially the
quantitative and the qualitative parts. Is there a way the authors could revise the presentation to
make for a more original and insightful narrative to guide through the manuscript?
In terms of the current manuscript here are some suggestions for revisions:
- p. 11 'Private sector providers also reported having been exposed to updates via mentorship
with an average score of 10.5/16 at baseline and 9.7/16 at endline (p=0.545).': What do 'updates'
mean? Please clarify to make more self-explanatory.
- p. 11: It is reported that 27 HCPs did not use the DELTA platform - Do their knowledge test
offer insight into how their scores differ from those who did use the platform?
- Given the effect of mentorship on competencies seems impossible to prove based on the results
presented - how do the authors interpret that? What do these findings mean?
- Under limitations it may be prudent to include the measure of self-reported competence being
somewhat problematic as it is not objectively measured or assessed. As such it is subject to bias
nad needs to be treated with a degree of caution.
- It is mentioned in the manuscript that other programmes were running concurrently to the
one being studied - any reflections on what that means for te attribution? Given the multitude
of intervention programmes with similar aims, and yet the limited effect on the competencies
of healthcare providers, would the authors be able to offer any insights as to how to what that
means and if there are any recommendations stemming from the programme to provide some
guidance for future work?
- p24 Quality improvement teams: This is the first time this intervention is listed - please
update the methods and findings section or provide an explanation that will help integrate this
information earlier, as well as incorporate it better in the assessment of the programme.
Some formatting suggestions and other edits:
- Spacing: superfluous and missing spaces found throughout the manuscript - please review and
amend as needed.
- American v British spelling to be standardised, e.g. labour v labor.
- Use of full stops at the end of headings to be standardised - preferable removed but if to be
used then to be used throughout.
- Standardise use of quotation marks - various styles used at present.
- Check for use of bold font and upper-case lettering where is it not necessary.
- Check for tense use - past tense should be used for consistency in the Findings section.
- p.21: [Mentor_0]1. - please check formatting and amend as needed.
- Note: breech not breach delivery - please check and amend throughout.
- Please review the tables for consistency in presenting information.