General disclaimers. The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of WHO concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted and dashed lines on maps represent approximate border lines for which there may not yet be full agreement.
The mention of specific companies or of certain manufacturers' products does not imply that they are endorsed or recommended by WHO in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.
All reasonable precautions have been taken by WHO to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall WHO be liable for damages arising from its use.

# Future research
This chapter on future research highlights crosscutting evidence gaps observed across a range of interventions in relation to effectiveness, resource use and cost-effectiveness, and gender, equity and rights. In addition, specific research questions are provided for each of the interventions, based on the gaps identified through the evidence-to-decision framework and GDG.

# Overarching research gaps
The following sections describe the overarching research priorities identified through this guideline process. These reflect the general areas in which the available evidence was found to be of low or very low certainty or confidence, or where no direct evidence was identified. Where studies were available, in some cases the certainty or confidence of the evidence was affected by poor reporting of outcomes, studies with small numbers of participants, and limited representation across different settings.
Annex 6 maps the state of evidence and its gaps based on the findings from the effectiveness reviews for the included digital health interventions.
For many of the interventions, the available evidence on effectiveness was sparse. Future research should measure health system process improvements that may immediately result from the digital intervention, such as health workers' adherence to recommended practice, as well as related distal health outcomes. Researchers should be realistic about the extent to which digital health interventions can impact on distal health outcomes, which are often affected by a variety of factors beyond the interaction with the digital intervention. Additionally, effectiveness studies need to include ways of concurrently monitoring technological performance (for example, do messages reach intended individuals?) and behavioural performance or user engagement (e.g. do individuals who get messages listen to or read them, and subsequently act on them?).

# Resource use and cost-effectiveness
The studies included in the systematic reviews of the effectiveness of the digital interventions considered by the guideline identified limited evidence on the resources used to implement these interventions. Costing studies should assess costs over a longer period, with appropriate accounting of amortization and maintenance of equipment and the continuous user support required. Future research should explore the cost-effectiveness, and potential for cost savings of the identified intervention and additional savings achieved through combining interventions.

# Gender, equity and rights
Further research needs to encompass a wider range of contexts and populations, including populations with poor access to digital or conventional health services, in order to better understand and mitigate any potential negative impacts on gender, equity and rights. Key research questions include how digital health interventions can help to reduce disparities in linking to the wider health system and whether these interventions may create further inequities in some settings as a consequence of poor network coverage, limited control of mobile devices, or a lack of other resources. Research should also explore unintentional exacerbation of inequities based on who has access to digital devices, and who has access to network connectivity.

# Implementation research
Due to the strong focus on integrated health systems and interoperability, future research should also examine the synergies across different combinations of digital health interventions to determine which packages of interventions are most effective and cost-effective. Addressing this question is important given the potential complexity of implementing packages of digital interventions and the costs of establishing and maintaining these systems. Specific questions include the following.

# Considerations for the design of future evaluations
The GDG also identified several issues related to the design of future evaluations of digital health interventions, including the following:
 Health system focused digital interventions, such as stock management and birth and death notification, are often complex in the number of components, behaviours targeted, and organizational levels involved (89). These factors may make designs such as randomized controlled trials for evaluating the effectiveness of these interventions difficult to apply. Other designs may therefore need to be considered, such as controlled before-and-after studies, stepped-wedge randomized controlled trials and interrupted time series studies.
 While there is value in evaluating changes in client/patient health outcomes, intermediate outcomes are also critical for the evaluation of digital health interventions. For example, the effect of decision support on client/patient health outcomes are influenced not only by the information delivered through the digital system, but also by a host of other factors, including access to medicines, their cost, family support, and biomedical factors such as whether the individual responds appropriately to recommended treatments or has comorbidities. A logical framework of how the digital intervention functions may be helpful in understanding the pathways through which the intervention influences a targeted behaviour or health system challenge and in selecting appropriate outcomes along these pathways.
 Digital technologies provide new opportunities to capture research data for measuring the effectiveness of implementations in real time, thus facilitating the ability to conduct evaluations more rapidly. Incorporating the research data collection needs for primary and secondary outcomes of interest at the design stage can ensure that the data needed to measure these outcomes is captured alongside the implementation.
 Rapid changes in digital technologies and the iterative approaches often used for software development may force digital health interventions to evolve during evaluation periods, which may pose challenges for the evaluation process. Detailed process evaluations running alongside impact evaluations may be helpful in understanding the effects of incremental changes in the digital interventions over time.
 Future research efforts should establish common metrics and tools for assessing the effectiveness and cost-effectiveness of digital health interventions

# Evidence maps and illustrative research questions
The tables below illustrates the general trends in the evidence found in the effectiveness reviews, demonstrating low and very low certainty evidence across most interventions. For more details on the specific interventions and outcomes, please review the summary of findings in Web Supplement 1.
In addition, specific research gaps and accompanying illustrative research questions are listed Table A5.4. These questions should be addressed using rigorous methods.

# Has benefits (high certainty evidence)

# Leads to harm (high certainty evidence) no incidence
Size of bubbles reflects the number of studies contributing to the outcome 1-3 7-10 4-6
Intervention-specific research gaps Table A5.4 outlines the specific research gaps, with illustrative research questions, identified for each of the interventions included in the guideline. These research questions should be addressed using rigorous methods.  Gender, equity and rights  How does this intervention increase or decrease health-related disparities? Are there population groups or settings that may not be able to be able to benefit from this intervention, and how can this be addressed?
1 Although WHO's Classification of digital health interventions v1.0 uses the term "client" (13), the terms "individual" and "patient" may be used interchangeably, where appropriate.

# Intervention Evidenceto-decision domain Research gaps and illustrative research questions
Providerto-provider telemedicine Effectiveness  What are the conditions that contribute to the effectiveness of provider-toprovider telemedicine?  Future research should include the following outcomes:
 health worker performance and adherence to recommended practice, quality of care provision  health behaviour, status and well-being  health worker and client satisfaction  unintended consequences, including the specific risks and safety concerns for implementing telemedicine different health domains or conditions.
 How is provider-to-provider telemedicine perceived by health workers to influence inter-professional interactions and collaboration?
Feasibility  What are the potential barriers to implementing these interventions, and how can these be mitigated? Such barriers include, for example, challenges in connectivity and its resulting consequences on the quality of information exchange (e.g. loss of quality of image files, interrupted connections).
Other  What mechanisms can be used to ensure that decision support tools evolve with new clinical evidence and subsequent policy changes? The development of the clinical algorithms used within decision support systems is presently an inexact science. Further research is needed to identify best practice, to develop and refine these algorithms both in terms of their clinical effectiveness and their ease of use and acceptability for health workers and clients. The use of artificial intelligence for the development of decision support systems is an emerging area that may help to refine algorithms, but more research is needed on acceptability, feasibility and ethics.

# Intervention Evidenceto-decision domain Research gaps and illustrative research questions mLearning
Effectiveness  What are the health system conditions that contribute to the effectiveness of mLearning? Research should include these outcomes:  health worker skills and attitudes, including long-term effects on these outcomes  health worker performance and adherence to recommended practice; quality of care provision  client health behaviours  unintended consequences.