# Key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol

Linda Nyanchoka, Catrin Tudur-Smith, Raphaël Porcher, Darko Hren

► To cite this version:

Linda Nyanchoka, Catrin Tudur-Smith, Raphaël Porcher, Darko Hren. Key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol. BMJ Open, 2019, 9 (8), pp.e027926. 10.1136/bmjopen-2018-027926. hal-02947778

HAL Id: hal-02947778

https://hal.inrae.fr/hal-02947778

Submitted on 8 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol

Linda Nyanchoka, Catrin Tudur-Smith, Raphaël Porcher, Darko Hren

► To cite this version:

Linda Nyanchoka, Catrin Tudur-Smith, Raphaël Porcher, Darko Hren. Key stakeholders’ perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol. BMJ Open, 2019, 9 (8), pp.e027926. 10.1136/bmjopen-2018-027926. hal-02947778

HAL Id: hal-02947778

https://hal.inrae.fr/hal-02947778v1

Submitted on 8 Jan 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# BMJ Open Key stakeholders' perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol

Linda Nyanchoka,¹,²,³ Catrin Tudur-Smith,²,³ Raphaël Porcher,¹ Darko Hren⁴

**To cite:** Nyanchoka L, Tudur-Smith C, Porcher R, et al. Key stakeholders' perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol. *BMJ Open* 2019;9:e027926. doi:10.1136/bmjopen-2018-027926

► Prepublication history and additional material for this paper are available online. To view these files, please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2018-027926).

Received 15 November 2018
Revised 27 June 2019
Accepted 02 August 2019

<figure><img src="image_2.png" /></figure>

Check for updates

© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. Published by BMJ.

¹Université de Paris, CRESS, INSERM, INRA, F-75004, Paris, France

²Institute of Translational Medicine, University of Liverpool, Liverpool, UK

³Department of Biostatistics, University of Liverpool, Liverpool, UK

⁴School of Humanities and Social Sciences, University of Split, Split, Croatia

**Correspondence to**
Linda Nyanchoka;
Inyanchoka@gmail.com

## ABSTRACT

**Introduction** Identifying research gaps can inform the design and conduct of health research, practice and policies by informing the current body of evidence. Audiences including researchers, clinical guideline developers, clinicians, policymakers, research regulatory bodies, funders and patients/the public can also benefit from understanding the status of research and research gaps to make informed choices. This study aims to explore how key informants define research gaps and characterise methods/practices used to identify and display gaps in health research to inform future research practice and policies.

**Methods and analysis** This is an exploratory qualitative study using semi-structured in-depth interviews. The participants will be recruited by purposive sampling from initiatives and organisations previously identified in a scoping review on methods to identify, prioritise and display gaps in health research. We anticipate performing up to 28 interviews with the different key informant groups who are involved in using evidence to inform health policy, practice and research. Interviews will be thematically analysed as outlined by Braun and Clarke. The qualitative data-analysis software NVivo V.12 Pro will be used to aid data management and analysis.

**Discussion** This is the protocol for a follow-up study that aims to complement and enrich the findings of the scoping review on methods to identify, prioritise and display gaps in health research. The overall project aims to develop methodological guidance for describing, identifying and displaying gaps in health research.

**Ethics and dissemination** The research obtained ethical approval from the University of Liverpool, UK. The findings will be disseminated via conferences, meetings (organised by the Methods in Research on Research project), peer-reviewed publications and lay magazines because the study participants will include the public/patients.

## BACKGROUND

Identifying research gaps can help inform the design and conduct of health research, practice and policies by providing a better understanding of the current body of evidence. The term 'research gap' is not well defined, and its meaning can differ depending on the researcher and research context. A recent

## Strengths and limitations of this study

► The qualitative nature of this study provides an in-depth understanding of key informants' perspectives and experiences in describing, identifying and displaying gaps in health research.

► This study is embedded in a larger study aiming to develop methodological guidance to identify and display gaps in health research.

► This study would have benefited from including patient/public perspectives in designing the study to be able to improve the importance and relevance of the findings for this population.

scoping review on methods used to identify, prioritise and display gaps in health research reported 12 different definitions related to gaps in health research (eg, population, theoretical and methodology gaps), each describing research gaps differently.¹ This finding shows the ambiguity of the term 'research gaps' and the different practices it may be related to.

As a basis for further exploring and understanding 'research gaps', we start from the definition given by the National Collaborating Centre for Methods and Tools (NCCMT) in Canada based on the work of Robinson *et al*, whereby a research gap is defined as a topic or area for which missing or insufficient information limits the ability to reach a conclusion for a question.² Given the different meanings and definitions of research gaps found in the scoping review,¹ we consider it important to further explore definitions rather than just adopt or modify the NCCMT definition. Clearly defining the type of research gap can help determine how to better identify, characterise, prioritise and address research gaps.

Different methods for identifying research gaps have been reported; for example, scoping reviews and umbrella reviews are emerging methods for mapping and
# BMJ Open Key stakeholders' perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol

Linda Nyanchoka,¹,²,³ Catrin Tudur-Smith,²,³ Raphaël Porcher,¹ Darko Hren⁴

**To cite:** Nyanchoka L, Tudur-Smith C, Porcher R, et al. Key stakeholders' perspectives and experiences with defining, identifying and displaying gaps in health research: a qualitative study protocol. *BMJ Open* 2019;9:e027926. doi:10.1136/bmjopen-2018-027926

► Prepublication history and additional material for this paper are available online. To view these files, please visit the journal online (http://dx.doi.org/10.1136/bmjopen-2018-027926).

Received 15 November 2018
Revised 27 June 2019
Accepted 02 August 2019

<figure><img src="image_2.png" /></figure>

Check for updates

© Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY. Published by BMJ.

¹Université de Paris, CRESS, INSERM, INRA, F-75004, Paris, France

²Institute of Translational Medicine, University of Liverpool, Liverpool, UK

³Department of Biostatistics, University of Liverpool, Liverpool, UK

⁴School of Humanities and Social Sciences, University of Split, Split, Croatia

**Correspondence to**
Linda Nyanchoka;
Inyanchoka@gmail.com

## ABSTRACT

**Introduction** Identifying research gaps can inform the design and conduct of health research, practice and policies by informing the current body of evidence. Audiences including researchers, clinical guideline developers, clinicians, policymakers, research regulatory bodies, funders and patients/the public can also benefit from understanding the status of research and research gaps to make informed choices. This study aims to explore how key informants define research gaps and characterise methods/practices used to identify and display gaps in health research to inform future research practice and policies.

**Methods and analysis** This is an exploratory qualitative study using semi-structured in-depth interviews. The participants will be recruited by purposive sampling from initiatives and organisations previously identified in a scoping review on methods to identify, prioritise and display gaps in health research. We anticipate performing up to 28 interviews with the different key informant groups who are involved in using evidence to inform health policy, practice and research. Interviews will be thematically analysed as outlined by Braun and Clarke. The qualitative data-analysis software NVivo V.12 Pro will be used to aid data management and analysis.

**Discussion** This is the protocol for a follow-up study that aims to complement and enrich the findings of the scoping review on methods to identify, prioritise and display gaps in health research. The overall project aims to develop methodological guidance for describing, identifying and displaying gaps in health research.

**Ethics and dissemination** The research obtained ethical approval from the University of Liverpool, UK. The findings will be disseminated via conferences, meetings (organised by the Methods in Research on Research project), peer-reviewed publications and lay magazines because the study participants will include the public/patients.

## BACKGROUND

Identifying research gaps can help inform the design and conduct of health research, practice and policies by providing a better understanding of the current body of evidence. The term 'research gap' is not well defined, and its meaning can differ depending on the researcher and research context. A recent

## Strengths and limitations of this study

► The qualitative nature of this study provides an in-depth understanding of key informants' perspectives and experiences in describing, identifying and displaying gaps in health research.

► This study is embedded in a larger study aiming to develop methodological guidance to identify and display gaps in health research.

► This study would have benefited from including patient/public perspectives in designing the study to be able to improve the importance and relevance of the findings for this population.

scoping review on methods used to identify, prioritise and display gaps in health research reported 12 different definitions related to gaps in health research (eg, population, theoretical and methodology gaps), each describing research gaps differently.¹ This finding shows the ambiguity of the term 'research gaps' and the different practices it may be related to.

As a basis for further exploring and understanding 'research gaps', we start from the definition given by the National Collaborating Centre for Methods and Tools (NCCMT) in Canada based on the work of Robinson *et al*, whereby a research gap is defined as a topic or area for which missing or insufficient information limits the ability to reach a conclusion for a question.² Given the different meanings and definitions of research gaps found in the scoping review,¹ we consider it important to further explore definitions rather than just adopt or modify the NCCMT definition. Clearly defining the type of research gap can help determine how to better identify, characterise, prioritise and address research gaps.

Different methods for identifying research gaps have been reported; for example, scoping reviews and umbrella reviews are emerging methods for mapping and
summarising evidence. These methods have an explicit
aim of identifying research gaps in a broad area as
compared with systematic reviews that focus on answering
a specific research question. ³⁻⁷ Robinson *et al* developed
a framework using systematic reviews to identify research
gaps² in which they classified the reasons for the existence
of research gaps and used the population, intervention,
comparison, outcome and setting process to charac-
terise them. Scoping, umbrella and systematic reviews are
reported to specifically identify research gaps, but other
methods are being used, and further exploring these
methods can optimise their definition, methodological
scrutiny and practice. ⁸⁻¹⁸ Furthermore, the aforemen-
tioned methods focus on the use of secondary research
methods to identify research gaps. However, a recent
scoping review showed that other methods have been used
to identify gaps, including primary and both primary and
secondary research methods. ¹ The scoping review showed
a lack of consensus on what constitutes the best method-
ological approaches to identify research gaps, determine
research priorities and display research gaps or prior-
ities. ¹ ⁵ ⁷ Therefore, to better understand the different
methods and ongoing practices, we aimed to conduct a
qualitative study to further explore more in-depth key
stakeholder experiences in describing research gaps and
the methods used to identify and display gaps in health
research.

This study is part of larger ongoing efforts to avoid waste
in producing and reporting research evidence, with a
focus on the identification of research gaps.¹⁹ Healthcare
decisions for individual patients, public health policies
and clinical guidelines should be informed by the best
available research evidence while taking into consider-
ation research gaps. Investigating experiences with prac-
tices/methods used to identify research gaps can inform
explicit methodological approaches in identifying and
describing research gaps. This investigation can enhance
practices of different stakeholder groups (ie, health
professionals, commissioners, researchers, patients/the
public and decision-makers) when addressing areas of
uncertainty within the research problem and topic area.²⁰
Initiatives such as the James Lind Alliance, UK Data-
base of Uncertainties about the Effects of Treatments,
Cochrane Agenda and Priority Setting Methods Group
and Evidence-based Research Network are some exam-
ples of existing efforts to identify and prioritise research
gaps in health.¹

This study is nested in a larger project aimed at devel-
oping methodological guidance for identifying gaps in
health research. The first step in the project was a scoping
review describing methods used to identify, prioritise and
display gaps in health research in scientific literature. The
scoping review mapped evidence on different definitions
reported for the term 'research gap' as well as methods
used to identify research gaps and determine research
priorities and display research gaps or research priori-
ties. ¹ The second step is the qualitative study described
in this protocol. The aim of the study is to investigate the

experience of key stakeholders (ie, researchers, funders,
clinicians, clinical guideline developers, public health
professionals, commissioners, patients/the public and
policymakers) with defining research gaps and practices/
methods used to identify and display research gaps. The
final step will be an integration and overview combining
findings from the scoping review and qualitative study to
provide a comprehensive overview of methods used to
identify and display research gaps. These study findings
will be used to inform the methodological guidance on
identifying research gaps.

The specific objectives of the study are to (1) investi-
gate key stakeholders' knowledge, perceptions and expe-
riences with defining research gaps and (2) characterise
methods/practices used for identifying and displaying
gaps in health research.

METHODS AND ANALYSIS

Qualitative study design

This study is an exploratory qualitative study using semi-structured interviews. This method will provide in-depth insight into key stakeholders’ perspectives, experiences, and practices with defining, identifying and displaying research gaps. Investigating perspectives of different key stakeholders will ensure that the issue is not explored through one lens but rather a variety of lenses. This will allow for revealing and better understanding multiple facets of research gaps including definitions and methodological approaches/practices to identify and display gaps.²¹

Study sample and recruitment

The study sample will include the following stakeholder
groups (ie, researchers, funders, clinicians, clinical guide-
line developers, public health professionals, commis-
sioners, patients/the public and policymakers). The
stakeholder groups will be organised in three main cate-
gories focusing on the use of evidence to inform health
policy, health practice and health research. These cate-
gories (policy, practice and research) are determined
from the scoping review findings.¹ More information
and examples of organisations are given in table 1. Study
participants will be recruited via contacts and organisa-
tions identified in the scoping review, relevant scientific
publications, existing professional networks (eg, Horizon
2020 (H2020) Project Methods in Research on Research
(MiRoR)) and contacts from conference attendance (eg,
Evidence Live and Cochrane Colloquium).

This study will also include patients or members of
the public as key informants, which will allow for better
understanding participants' perceived needs and priori-
ties in identifying research gaps to make informed
health decisions. Patients/the public will be recruited
and identified via patient support groups online, commu-
nity centres and public involvement websites such as the
peopleinresearch.org platform that involves the public in
health research.
summarising evidence. These methods have an explicit
aim of identifying research gaps in a broad area as
compared with systematic reviews that focus on answering
a specific research question. ³⁻⁷ Robinson *et al* developed
a framework using systematic reviews to identify research
gaps² in which they classified the reasons for the existence
of research gaps and used the population, intervention,
comparison, outcome and setting process to charac-
terise them. Scoping, umbrella and systematic reviews are
reported to specifically identify research gaps, but other
methods are being used, and further exploring these
methods can optimise their definition, methodological
scrutiny and practice. ⁸⁻¹⁸ Furthermore, the aforemen-
tioned methods focus on the use of secondary research
methods to identify research gaps. However, a recent
scoping review showed that other methods have been used
to identify gaps, including primary and both primary and
secondary research methods. ¹ The scoping review showed
a lack of consensus on what constitutes the best method-
ological approaches to identify research gaps, determine
research priorities and display research gaps or prior-
ities. ¹ ⁵ ⁷ Therefore, to better understand the different
methods and ongoing practices, we aimed to conduct a
qualitative study to further explore more in-depth key
stakeholder experiences in describing research gaps and
the methods used to identify and display gaps in health
research.

This study is part of larger ongoing efforts to avoid waste
in producing and reporting research evidence, with a
focus on the identification of research gaps.¹⁹ Healthcare
decisions for individual patients, public health policies
and clinical guidelines should be informed by the best
available research evidence while taking into consider-
ation research gaps. Investigating experiences with prac-
tices/methods used to identify research gaps can inform
explicit methodological approaches in identifying and
describing research gaps. This investigation can enhance
practices of different stakeholder groups (ie, health
professionals, commissioners, researchers, patients/the
public and decision-makers) when addressing areas of
uncertainty within the research problem and topic area.²⁰
Initiatives such as the James Lind Alliance, UK Data-
base of Uncertainties about the Effects of Treatments,
Cochrane Agenda and Priority Setting Methods Group
and Evidence-based Research Network are some exam-
ples of existing efforts to identify and prioritise research
gaps in health.¹

This study is nested in a larger project aimed at devel-
oping methodological guidance for identifying gaps in
health research. The first step in the project was a scoping
review describing methods used to identify, prioritise and
display gaps in health research in scientific literature. The
scoping review mapped evidence on different definitions
reported for the term ‘research gap’ as well as methods
used to identify research gaps and determine research
priorities and display research gaps or research priori-
ties.¹ The second step is the qualitative study described
in this protocol. The aim of the study is to investigate the

experience of key stakeholders (ie, researchers, funders,
clinicians, clinical guideline developers, public health
professionals, commissioners, patients/the public and
policymakers) with defining research gaps and practices/
methods used to identify and display research gaps. The
final step will be an integration and overview combining
findings from the scoping review and qualitative study to
provide a comprehensive overview of methods used to
identify and display research gaps. These study findings
will be used to inform the methodological guidance on
identifying research gaps.

The specific objectives of the study are to (1) investi-
gate key stakeholders’ knowledge, perceptions and expe-
riences with defining research gaps and (2) characterise
methods/practices used for identifying and displaying
gaps in health research.

METHODS AND ANALYSIS

Qualitative study design

This study is an exploratory qualitative study using semi-structured interviews. This method will provide in-depth insight into key stakeholders’ perspectives, experiences, and practices with defining, identifying and displaying research gaps. Investigating perspectives of different key stakeholders will ensure that the issue is not explored through one lens but rather a variety of lenses. This will allow for revealing and better understanding multiple facets of research gaps including definitions and methodological approaches/practices to identify and display gaps.²¹

Study sample and recruitment

The study sample will include the following stakeholder
groups (ie, researchers, funders, clinicians, clinical guide-
line developers, public health professionals, commis-
sioners, patients/the public and policymakers). The
stakeholder groups will be organised in three main cate-
gories focusing on the use of evidence to inform health
policy, health practice and health research. These cate-
gories (policy, practice and research) are determined
from the scoping review findings.¹ More information
and examples of organisations are given in table 1. Study
participants will be recruited via contacts and organisa-
tions identified in the scoping review, relevant scientific
publications, existing professional networks (eg, Horizon
2020 (H2020) Project Methods in Research on Research
(MiRoR)) and contacts from conference attendance (eg,
Evidence Live and Cochrane Colloquium).

This study will also include patients or members of
the public as key informants, which will allow for better
understanding participants’ perceived needs and priori-
ties in identifying research gaps to make informed
health decisions. Patients/the public will be recruited
and identified via patient support groups online, commu-
nity centres and public involvement websites such as the
peopleinresearch.org platform that involves the public in
health research.
<figure><img src="image_3.png" /></figure>

<table><caption>**Table 1** Key informants</caption><thead><tr><th>Categories</th><th>Key informants</th><th>Examples</th><th>Expected number of interviews</th></tr></thead><tbody><tr><td>Health policy</td><td>Policymakers</td><td>Ministry of health officials</td><td>2-4</td></tr><tr><td rowspan="4">Health practice</td><td>Clinicians</td><td>Healthcare professionals (doctors, nurses)</td><td>2-4</td></tr><tr><td>Clinical guideline developers</td><td>UK National Institute for Health and Care Excellence</td><td>2-4</td></tr><tr><td>Public health professionals, commissioners</td><td>National public health bodies</td><td>2-4</td></tr><tr><td>Public/patients</td><td>Patient forums/groups</td><td>2-4</td></tr><tr><td rowspan="2">Health research</td><td>Researchers</td><td>Research institutes/universities<br>Knowledge synthesis research groups<br>Belgian Health Care Knowledge Centre<br>Africa Evidence Network<br>Student forums</td><td>2-4</td></tr><tr><td>Funding bodies</td><td>UK National Institute for Health Research<br>European Union</td><td>2-4</td></tr></tbody></table>

We will use purposive sampling to ensure that the perspectives of all identified stakeholder groups are represented. Purposeful sampling is widely used in qualitative research for identifying and selecting information-rich cases, and in this study, further elaboration of the term research gap is needed to better understand the context of the research gaps and methods/practices used to identify and display the research gaps.²² ²³

We anticipate performing about 14–28 interviews. This number of interviews will provide for data saturation (ie, the point when new data do not add to a better understanding of the studied phenomenon but rather repeat what was previously expressed²⁴) and also obtain a scope of responses from each stakeholder group. This estimation of interview participants is based on a study involving 60 interviews that showed saturation with 12 interviews, with broader themes apparent after only 6 interviews.²⁵ The authors noted that factors such as heterogeneity of the sample affect how many interviews are required but concluded that to understand common perceptions and experiences among a group of relatively homogeneous individuals, 12 interviews should suffice.²⁵ Another study, after examining 25 in-depth interviews, found code saturation after interviews, with the range of thematic issues identified; the authors proposed 16–24 interviews to reach saturation (ie, a richly textured understanding of issues²⁶). Therefore, we aim to gather 14–28 interviews for our three main categories (health policy, practice and research).

Saturation will be guided by the seven parameters identified by Hennink *et al*,²⁶ ²⁷ including the study purpose, population, sampling strategy, data quality, type of codes, code book and saturation goal, and focus retrieved from the study. Each of these parameters will be considered throughout the study.

## Data collection and recording

Semi-structured interviews will be used for this study. The main reason for selecting semi-structured interviews is to allow for specific areas to be addressed while giving the interviewees the opportunity to reflect on their experiences and perspectives related to defining, identifying and presenting research gaps that are relevant to them and that may not have been explored or anticipated by the researcher(s).²⁸

We will conduct interviews in-person and using teleconference, according to the participant's availability and preference. In-person interviews will be conducted primarily with participants residing or reachable in London, UK, and other participants will be interviewed via teleconference (see online supplementary appendix 1 for the interview guide for both the in-person and teleconference interviews). The interviews will be recorded on a digital recorder for face-to-face interviews and electronically for teleconference interviews.

The guide was developed by focusing on exploring key stakeholder perspectives and experiences with the following key areas:

1. Participant background information.

2. Definitions of research gaps.

3. Knowledge, perceptions and experiences on methods/practices used to identify and display gaps in health research to inform further health policy, practice and research.

These three domains were developed with information from the scoping review to guide the questions. The interview topic guide will be piloted before data collection. It will also be adapted according to key stakeholder groups to ensure that it is meaningful to their background and to gather more relevant information based on their experiences and knowledge.²⁹
<figure><img src="image_3.png" /><figcaption>**Table 1** Key informants</figcaption></figure>

<table><thead><tr><th>Categories</th><th>Key informants</th><th>Examples</th><th>Expected number of interviews</th></tr></thead><tbody><tr><td>Health policy</td><td>Policymakers</td><td>Ministry of health officials</td><td>2-4</td></tr><tr><td rowspan="4">Health practice</td><td>Clinicians</td><td>Healthcare professionals (doctors, nurses)</td><td>2-4</td></tr><tr><td>Clinical guideline developers</td><td>UK National Institute for Health and Care Excellence</td><td>2-4</td></tr><tr><td>Public health professionals, commissioners</td><td>National public health bodies</td><td>2-4</td></tr><tr><td>Public/patients</td><td>Patient forums/groups</td><td>2-4</td></tr><tr><td rowspan="2">Health research</td><td>Researchers</td><td>Research institutes/universities<br>Knowledge synthesis research groups<br>Belgian Health Care Knowledge Centre<br>Africa Evidence Network<br>Student forums</td><td>2-4</td></tr><tr><td>Funding bodies</td><td>UK National Institute for Health Research<br>European Union</td><td>2-4</td></tr></tbody></table>

We will use purposive sampling to ensure that the perspectives of all identified stakeholder groups are represented. Purposeful sampling is widely used in qualitative research for identifying and selecting information-rich cases, and in this study, further elaboration of the term research gap is needed to better understand the context of the research gaps and methods/practices used to identify and display the research gaps.²² ²³

We anticipate performing about 14–28 interviews. This number of interviews will provide for data saturation (ie, the point when new data do not add to a better understanding of the studied phenomenon but rather repeat what was previously expressed²⁴) and also obtain a scope of responses from each stakeholder group. This estimation of interview participants is based on a study involving 60 interviews that showed saturation with 12 interviews, with broader themes apparent after only 6 interviews.²⁵ The authors noted that factors such as heterogeneity of the sample affect how many interviews are required but concluded that to understand common perceptions and experiences among a group of relatively homogeneous individuals, 12 interviews should suffice.²⁵ Another study, after examining 25 in-depth interviews, found code saturation after interviews, with the range of thematic issues identified; the authors proposed 16–24 interviews to reach saturation (ie, a richly textured understanding of issues²⁶). Therefore, we aim to gather 14–28 interviews for our three main categories (health policy, practice and research).

Saturation will be guided by the seven parameters identified by Hennink *et al*,²⁶ ²⁷ including the study purpose, population, sampling strategy, data quality, type of codes, code book and saturation goal, and focus retrieved from the study. Each of these parameters will be considered throughout the study.

## Data collection and recording

Semi-structured interviews will be used for this study. The main reason for selecting semi-structured interviews is to allow for specific areas to be addressed while giving the interviewees the opportunity to reflect on their experiences and perspectives related to defining, identifying and presenting research gaps that are relevant to them and that may not have been explored or anticipated by the researcher(s).²⁸

We will conduct interviews in-person and using teleconference, according to the participant's availability and preference. In-person interviews will be conducted primarily with participants residing or reachable in London, UK, and other participants will be interviewed via teleconference (see online supplementary appendix 1 for the interview guide for both the in-person and teleconference interviews). The interviews will be recorded on a digital recorder for face-to-face interviews and electronically for teleconference interviews.

The guide was developed by focusing on exploring key stakeholder perspectives and experiences with the following key areas:

1. Participant background information.

2. Definitions of research gaps.

3. Knowledge, perceptions and experiences on methods/practices used to identify and display gaps in health research to inform further health policy, practice and research.

These three domains were developed with information from the scoping review to guide the questions. The interview topic guide will be piloted before data collection. It will also be adapted according to key stakeholder groups to ensure that it is meaningful to their background and to gather more relevant information based on their experiences and knowledge.²⁹
The semi-structured interview guide contains two levels of questions: main themes and follow-up questions. The main themes cover the general content of the research gaps aimed to encourage participants to speak freely about their perceptions, experiences and practices. Follow-up questions are prompts and probes aimed at following respondents' answers and investigating the raised issues more in-depth. The interview guide covers the main topics of the study, providing a focused structure for the discussion during the interviews. However, it does not need to be strictly followed — the main focus is on providing a setting that encourages respondents to share their perceptions and experiences with research gaps as thoroughly as possible within the constraints of our study aims.³⁰

All interviews will be transcribed verbatim and anonymised. The lead researcher (LN) will transcribe two interviews to help inform the analytical process, and the other audio files will be transcribed by a professional transcription agency licensed from the University of Liverpool.

## Data analysis

We will use analytical categories to describe and explain definitions, experiences and practices reported among the groups of participants. All data relevant to each category (describing research gaps, experience with identifying and displaying research gaps) will be identified and examined to ensure that each data item is checked accordingly.

Our approach is based on the thematic analysis outlined by Braun and Clarke.³¹ The steps include the following: (1) transcription and checking transcripts with recordings for accuracy; (2) open coding from interview responses to be performed by two researchers independently (LN and DH); (3) agreement of initial codes to be discussed among the researchers and an initial codebook developed; (4) the code structure to be used for analysing the remaining responses with openness to including new codes and refining existing ones; and (5) themes and subthemes to be identified from the final code structure and their relationships presented.³¹

The initial coding framework for our analysis will start from broad categories identified in the previous scoping review, on which the interviews were structured. Within these broad categories (ie, describing research gaps, experience with identifying and displaying research gaps), analytic categories will be inductively derived from the data. In this sense, our approach includes both top-down and bottom-up development of analytic categories and themes.

Trustworthiness during thematic data analysis will be ensured by storing raw data systematically, documenting detailed notes about the development and hierarchies of concepts and themes, establishing consensus on themes, providing detailed descriptions of context and describing the process of coding and analysis.⁸ ⁹ NVivo V.12 Pro, a qualitative data analysis software, will be used for data management and analysis.

## Ensuring study quality

To further ensure rigour and trustworthiness, this study will be guided by Guba and Lincoln's concepts for defining and investigating quality in qualitative research that can be considered parallel to quantitative research concepts of validity and reliability.²⁷ ³² ³³ The concepts include credibility, transferability, dependability, confirmability, audit trails and reflectivity. They are inter-related, and thinking through them from the onset and incorporating them in a study will improve the study rigour.

Credibility is defined as the confidence that can be placed in the truth of the research findings³⁴⁻³⁶; it is considered the most important criterion to ensure rigour and trustworthiness. To ensure credibility of our study, we will use peer debriefing, which will entail the qualitative lead researcher (LN) seeking support from the senior researcher (DH) to provide scholarly guidance. The feedback will help improve the quality of the inquiry findings.³⁶ Transferability refers to the extent to which findings of qualitative research can be transferred to other contexts and are useful to people in other settings.²¹ ³⁶⁻³⁸ We aim to address transferability by reporting a rich, detailed description of the key stakeholders' context and location.³⁶ ³⁸ Dependability is related to whether the research questions are clear and logically connected to the research purpose and design.³⁷ We aim to achieve dependability by first drafting this protocol to guide our study and future studies with a similar purpose. Confirmability has been related to objectivity or neutrality for establishing that the data and interpretations of the findings are not figments of the inquirer's imagination but are clearly derived from the data, that data collection and interpretations of the study are clearly deliberated from the data and not misinterpreted.³⁷ We aim to address confirmability by documenting the justification of methodological and analytical choices to illustrate how the data were derived in relation to the study objectives and transparently describing the research steps taken from the start of the project to the development and reporting of the findings. Records of the research path will be kept throughout the study, and debriefing sessions will be held between the main researcher (LN) and senior researcher (DH). Finally, reflexivity includes examining one's own conceptual lens, explicit and implicit assumptions, preconceptions and values and how these affect research decisions in all phases of qualitative studies. Reflexivity will be achieved by ensuring transparency of the study process by maintaining clear documentation.

## Patient or public involvement

There is no patient or public involvement in the design or analysis of this study. However, we plan to involve patients/the public in findings that pertain to them and in disseminating study findings. This will be achieved by using patient/public online platforms such as peoplein-research.org.
The semi-structured interview guide contains two levels of questions: main themes and follow-up questions. The main themes cover the general content of the research gaps aimed to encourage participants to speak freely about their perceptions, experiences and practices. Follow-up questions are prompts and probes aimed at following respondents' answers and investigating the raised issues more in-depth. The interview guide covers the main topics of the study, providing a focused structure for the discussion during the interviews. However, it does not need to be strictly followed — the main focus is on providing a setting that encourages respondents to share their perceptions and experiences with research gaps as thoroughly as possible within the constraints of our study aims.³⁰

All interviews will be transcribed verbatim and anonymised. The lead researcher (LN) will transcribe two interviews to help inform the analytical process, and the other audio files will be transcribed by a professional transcription agency licensed from the University of Liverpool.

## Data analysis

We will use analytical categories to describe and explain definitions, experiences and practices reported among the groups of participants. All data relevant to each category (describing research gaps, experience with identifying and displaying research gaps) will be identified and examined to ensure that each data item is checked accordingly.

Our approach is based on the thematic analysis outlined by Braun and Clarke.³¹ The steps include the following: (1) transcription and checking transcripts with recordings for accuracy; (2) open coding from interview responses to be performed by two researchers independently (LN and DH); (3) agreement of initial codes to be discussed among the researchers and an initial codebook developed; (4) the code structure to be used for analysing the remaining responses with openness to including new codes and refining existing ones; and (5) themes and subthemes to be identified from the final code structure and their relationships presented.³¹

The initial coding framework for our analysis will start from broad categories identified in the previous scoping review, on which the interviews were structured. Within these broad categories (ie, describing research gaps, experience with identifying and displaying research gaps), analytic categories will be inductively derived from the data. In this sense, our approach includes both top-down and bottom-up development of analytic categories and themes.

Trustworthiness during thematic data analysis will be ensured by storing raw data systematically, documenting detailed notes about the development and hierarchies of concepts and themes, establishing consensus on themes, providing detailed descriptions of context and describing the process of coding and analysis.⁸ ⁹ NVivo V.12 Pro, a qualitative data analysis software, will be used for data management and analysis.

## Ensuring study quality

To further ensure rigour and trustworthiness, this study will be guided by Guba and Lincoln's concepts for defining and investigating quality in qualitative research that can be considered parallel to quantitative research concepts of validity and reliability.²⁷ ³² ³³ The concepts include credibility, transferability, dependability, confirmability, audit trails and reflectivity. They are inter-related, and thinking through them from the onset and incorporating them in a study will improve the study rigour.

Credibility is defined as the confidence that can be placed in the truth of the research findings³⁴⁻³⁶; it is considered the most important criterion to ensure rigour and trustworthiness. To ensure credibility of our study, we will use peer debriefing, which will entail the qualitative lead researcher (LN) seeking support from the senior researcher (DH) to provide scholarly guidance. The feedback will help improve the quality of the inquiry findings.³⁶ Transferability refers to the extent to which findings of qualitative research can be transferred to other contexts and are useful to people in other settings.²¹ ³⁶⁻³⁸ We aim to address transferability by reporting a rich, detailed description of the key stakeholders' context and location.³⁶ ³⁸ Dependability is related to whether the research questions are clear and logically connected to the research purpose and design.³⁷ We aim to achieve dependability by first drafting this protocol to guide our study and future studies with a similar purpose. Confirmability has been related to objectivity or neutrality for establishing that the data and interpretations of the findings are not figments of the inquirer's imagination but are clearly derived from the data, that data collection and interpretations of the study are clearly deliberated from the data and not misinterpreted.³⁷ We aim to address confirmability by documenting the justification of methodological and analytical choices to illustrate how the data were derived in relation to the study objectives and transparently describing the research steps taken from the start of the project to the development and reporting of the findings. Records of the research path will be kept throughout the study, and debriefing sessions will be held between the main researcher (LN) and senior researcher (DH). Finally, reflexivity includes examining one's own conceptual lens, explicit and implicit assumptions, preconceptions and values and how these affect research decisions in all phases of qualitative studies. Reflexivity will be achieved by ensuring transparency of the study process by maintaining clear documentation.

## Patient or public involvement

There is no patient or public involvement in the design or analysis of this study. However, we plan to involve patients/the public in findings that pertain to them and in disseminating study findings. This will be achieved by using patient/public online platforms such as peoplein-research.org.
## DISCUSSION

This study will provide insights into issues related to defining research gaps and methods used to identify and display gaps in health research from perspectives of key stakeholders involved in the process. This is a follow-up study of a wider project; the first study was a scoping review exploring methods used to identify and display research gaps reported in scientific publications.¹ The scoping review showed variation and ambiguity in how research gaps are described as well as the methods used to identify and prioritise research gaps. Several of the articles described the development of a framework or tool for identifying and prioritising research gaps and applying it to a specific topic area as an example for application.¹,²,⁷,³⁹ There were no evaluations of reproducibility of the method framworks identified in the scoping review.¹,⁷ Furthermore, despite articles highlighting the existence of research gaps in their studies, very few specifically described the gaps and the causes or the method of identification, so fully understanding the relevance and importance of the research gap to adequately address it is difficult. Our scoping review also primarily found the use of secondary research methods such as systematic reviews and scoping reviews as the most commonly used methods to identify gaps; although other methods were identified, they were inadequately described. The scoping review also showed that besides researchers, different audiences including clinicians, policymakers, funders and patients or the public can benefit from understanding gaps and methods/practices on how to identify and display gaps in health research. This qualitative study aims to go beyond the scientific literature in describing, identifying and displaying gaps in health research and directly talk to people about their understanding and practices. Given the nature of this topic that is not fully explored, there is a need to investigate real practices to be able to develop methodological guidance, taking into consideration the existing literature and ongoing practices.

This study has some limitations; one is not including patients/the public in designing the study. Including patients/public perspectives would have benefited the study design by being able to improve the importance and relevance of the findings for this population. One of the main strengths of the study is improving the definition of research gaps and subsequently improving the accurate reporting of research gaps to clearly elucidate the characteristics, which can help in making evidence-based decisions. For example, making a decision based on a research gap contributing to lack of primary research on a specific health problem can differ from a research gap related to lack of secondary research summarising the research. Hence, all these factors regarding research gaps need to be highlighted if they are known and made explicit when disseminating and communicating research. In addition, providing more information on what the gap represents may inform users of evidence on more specific information about the research gap and how it can be addressed

more accurately. We anticipate that this study will advance efforts in research and practice on this topic area.

## ETHICS AND DISSEMINATION

Informed consent will be obtained in accordance with the University of Liverpool Ethics Committee board requirements. Verbal consent will be sought for phone interviews and written consent for in-person interviews. Confidentiality and data protection will be ensured in accordance with the University of Liverpool Ethics Committee board. All participant information will be anonymised, and hard-copy data will be stored in a locked unit. Soft-copy material will be stored in a password-protected file. On completion of the study and publication of the study results, all study material will be stored and disposed of according to the rules and regulations of the University of Liverpool. The study protocol will be stored in the data repository Zenodo.

At the end of this research project, the results will be presented at conferences and relevant meetings (eg, H2020 Project MiRoR). They will also be published in a peer-reviewed journal and as part of a doctoral thesis of the PhD fellow (LN) as well as in professional and lay magazines and presented in workshops at professional events for stakeholder groups and as online materials with good practice examples.

**Acknowledgements** We acknowledge the support of Daniela Lai and Cristian R. Montenegro in providing feedback on the interview guide. We thank Laura Smales (BioMedEditing, Toronto, ON) for editing the manuscript.

**Contributors** LN and DH conceived the study with guidance and feedback from RP and CT-S. All authors read and approved the final manuscript.

**Funding** This project is a part of a MiRoR (Methods in Research on Research)-funded PhD being undertaken by LN. MiRoR received funding from the European Union's Horizon 2020 research and innovation programme under a Marie Skłodowska-Curie grant (agreement no. 676207).

**Competing interests** None declared.

**Patient consent for publication** Not required.

**Provenance and peer review** Not commissioned; externally peer reviewed.

**Open access** This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

## REFERENCES

1. Nyanchoka L, Tudur-Smith C, Thu VN, et al. A scoping review describes methods used to identify, prioritize and display gaps in health research. *J Clin Epidemiol* 2019;109:99–110.

2. Robinson KA, Saldanha IJ, Mckoy NA. Development of a framework to identify research gaps from systematic reviews. *J Clin Epidemiol* 2011;64:1325–30.

3. Snilstveit B, Vojtkova M, Bhavsar A, Birte S, Martina V, Ami B, et al. Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas. *J Clin Epidemiol* 2016;79:120.

4. Pham MT, Rajić A, Greig JD, et al. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. *Research Synthesis Methods* 2014;5:371–85.

5. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. *Int J Soc Res Methodol* 2005;8:19–32.
<figure><img src="image_4.png" /></figure>

## DISCUSSION

This study will provide insights into issues related to defining research gaps and methods used to identify and display gaps in health research from perspectives of key stakeholders involved in the process. This is a follow-up study of a wider project; the first study was a scoping review exploring methods used to identify and display research gaps reported in scientific publications.¹ The scoping review showed variation and ambiguity in how research gaps are described as well as the methods used to identify and prioritise research gaps. Several of the articles described the development of a framework or tool for identifying and prioritising research gaps and applying it to a specific topic area as an example for application.¹,²,⁷,³⁹ There were no evaluations of reproducibility of the method framworks identified in the scoping review.¹,⁷ Furthermore, despite articles highlighting the existence of research gaps in their studies, very few specifically described the gaps and the causes or the method of identification, so fully understanding the relevance and importance of the research gap to adequately address it is difficult. Our scoping review also primarily found the use of secondary research methods such as systematic reviews and scoping reviews as the most commonly used methods to identify gaps; although other methods were identified, they were inadequately described. The scoping review also showed that besides researchers, different audiences including clinicians, policymakers, funders and patients or the public can benefit from understanding gaps and methods/practices on how to identify and display gaps in health research. This qualitative study aims to go beyond the scientific literature in describing, identifying and displaying gaps in health research and directly talk to people about their understanding and practices. Given the nature of this topic that is not fully explored, there is a need to investigate real practices to be able to develop methodological guidance, taking into consideration the existing literature and ongoing practices.

This study has some limitations; one is not including patients/the public in designing the study. Including patients/public perspectives would have benefited the study design by being able to improve the importance and relevance of the findings for this population. One of the main strengths of the study is improving the definition of research gaps and subsequently improving the accurate reporting of research gaps to clearly elucidate the characteristics, which can help in making evidence-based decisions. For example, making a decision based on a research gap contributing to lack of primary research on a specific health problem can differ from a research gap related to lack of secondary research summarising the research. Hence, all these factors regarding research gaps need to be highlighted if they are known and made explicit when disseminating and communicating research. In addition, providing more information on what the gap represents may inform users of evidence on more specific information about the research gap and how it can be addressed

more accurately. We anticipate that this study will advance efforts in research and practice on this topic area.

## ETHICS AND DISSEMINATION

Informed consent will be obtained in accordance with the University of Liverpool Ethics Committee board requirements. Verbal consent will be sought for phone interviews and written consent for in-person interviews. Confidentiality and data protection will be ensured in accordance with the University of Liverpool Ethics Committee board. All participant information will be anonymised, and hard-copy data will be stored in a locked unit. Soft-copy material will be stored in a password-protected file. On completion of the study and publication of the study results, all study material will be stored and disposed of according to the rules and regulations of the University of Liverpool. The study protocol will be stored in the data repository Zenodo.

At the end of this research project, the results will be presented at conferences and relevant meetings (eg, H2020 Project MiRoR). They will also be published in a peer-reviewed journal and as part of a doctoral thesis of the PhD fellow (LN) as well as in professional and lay magazines and presented in workshops at professional events for stakeholder groups and as online materials with good practice examples.

**Acknowledgements** We acknowledge the support of Daniela Lai and Cristian R. Montenegro in providing feedback on the interview guide. We thank Laura Smales (BioMedEditing, Toronto, ON) for editing the manuscript.

**Contributors** LN and DH conceived the study with guidance and feedback from RP and CT-S. All authors read and approved the final manuscript.

**Funding** This project is a part of a MiRoR (Methods in Research on Research)-funded PhD being undertaken by LN. MiRoR received funding from the European Union's Horizon 2020 research and innovation programme under a Marie Skłodowska-Curie grant (agreement no. 676207).

**Competing interests** None declared.

**Patient consent for publication** Not required.

**Provenance and peer review** Not commissioned; externally peer reviewed.

**Open access** This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

## REFERENCES

1. Nyanchoka L, Tudur-Smith C, Thu VN, et al. A scoping review describes methods used to identify, prioritize and display gaps in health research. *J Clin Epidemiol* 2019;109:99–110.

2. Robinson KA, Saldanha IJ, Mckoy NA. Development of a framework to identify research gaps from systematic reviews. *J Clin Epidemiol* 2011;64:1325–30.

3. Snilstveit B, Vojtkova M, Bhavsar A, Birte S, Martina V, Ami B, et al. Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas. *J Clin Epidemiol* 2016;79:120.

4. Pham MT, Rajić A, Greig JD, et al. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. *Research Synthesis Methods* 2014;5:371–85.

5. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. *Int J Soc Res Methodol* 2005;8:19–32.
6. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. *Implementation Science* 2010;5.

7. Carey T, Yon A, Beadles C, et al. Prioritizing future research through examination of research gaps in systematic reviews. Prepared for the Patient-Centered Outcomes Research Institute, 2012.

8. Whittemore R, Chao A, Jang M, et al. Methods for knowledge synthesis: an overview. *Heart & Lung* 2014;43:453-61.

9. Kastner M, Antony J, Soobiah C, et al. Conceptual recommendations for selecting the most appropriate knowledge synthesis method to answer research questions related to complex evidence. *J Clin Epidemiol* 2016;73:43-9.

10. Manson H. Systematic reviews are not enough: policymakers need a greater variety of synthesized evidence. *J Clin Epidemiol* 2016;73:11-14.

11. Perrier L, Lightfoot D, Kealey MR, et al. Knowledge synthesis research: a bibliometric analysis. *J Clin Epidemiol* 2016;73:50-7.

12. Pluye P, Hong QN, Bush PL, et al. Opening-up the definition of systematic literature review: the plurality of worldviews, methodologies and methods for reviews and syntheses. *J Clin Epidemiol* 2016;73:2-5.

13. Sales A. Generating and using evidence: reflections from the perspective of implementation. *J Clin Epidemiol* 2016;73:6-7.

14. Straus SE, Kastner M, Soobiah C, et al. Introduction: engaging researchers on developing, using, and improving knowledge synthesis methods: a series of articles describing the results of a scoping review on emerging knowledge synthesis methods. *J Clin Epidemiol* 2016;73:15-18.

15. Tricco AC, Antony J, Soobiah C, et al. Knowledge synthesis methods for generating or refining theory: a scoping review reveals that little guidance is available. *J Clin Epidemiol* 2016;73:36-42.

16. Tricco AC, Antony J, Soobiah C, et al. Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. *J Clin Epidemiol* 2016;73:29-35.

17. Tricco AC, Soobiah C, Antony J, et al. A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method. *J Clin Epidemiol* 2016;73:19-28.

18. Wong G. Knowledge synthesis approaches-spoilt for choice? *J Clin Epidemiol* 2016;73:8-10.

19. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. *The Lancet* 2009;374:86-9.

20. Bouma GD. *The research process*. USA: Oxford University Press, 2004.

21. Yin RK. *Case study research: design and methods (applied social research methods)*. London and Singapore: Sage, 2009.

22. Setia M. Methodology series module 5: sampling strategies. *Indian J Dermatol* 2016;61:505-9.

23. Patton MQ. *Qualitative evaluation and research methods*. SAGE Publications, 1990.

24. Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. *Qual Quant* 2018;52.

25. Guest G, Bunce A. How many interviews are enough?: an experiment with data saturation and variability. *Field Methods* 2006;18:59-82.

26. Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? *Qual Health Res* 2017;27:591-608.

27. Shenton A. *Strategies for ensuring Trustworthiness in qualitative research projects*. 22, 2004.

28. Britten N. Qualitative interviews in medical research. BMJ 1995;311:251-3.

29. Kvale S, Brinkmann S. *Interviews: learning the craft of qualitative research interviewing*. Sage, 2009.

30. Gill P, Stewart K, Treasure E, et al. Methods of data collection in qualitative research: interviews and focus groups. *Br Dent J* 2008;204:291-5.

31. Braun V, Clarke V. Using thematic analysis in psychology. *Qual Res Psychol* 2006;3:77-101.

32. Guba EG, Lincoln YS. *Naturalistic inquiry*, 1985.

33. Glonti K, Hren D. Editors' perspectives on the peer-review process in biomedical journals: protocol for a qualitative study. *BMJ Open* 2018;8:e020568.

34. Macnee CL, McCabe S. *Understanding nursing research: using research in evidence-based practice*. Lippincott Williams & Wilkins, 2008.

35. Holloway I, Wheeler S. Ensuring trustworthiness and quality. In: Holloway I, Wheeler S, eds. *Research in nursing*. 2nd edn. India: Blackwell Publishing, 2002: 250-63.

36. AnneyVN. Ensuring the quality of the findings of qualitative research: looking at trustworthiness criteria. *JETERAPS*;5:272-81.

37. Tobin GA, Begley CM. Methodological rigour within a qualitative framework. *J Adv Nurs* 2004;48:388-96.

38. Connelly LM. Trustworthiness in qualitative research. *Medsurg Nursing* 2016;25:435-7.

39. Robinson KA, Saldanha IJ, McKoy NA. Identification of research gaps from evidence-based guidelines: a pilot study in cystic fibrosis. *Int J Technol Assess Health Care* 2011;27:247-52.
6. Levac D, Colquhoun H, O'Brien KK. Scoping studies: advancing the methodology. *Implementation Science* 2010;5.

7. Carey T, Yon A, Beadles C, et al. Prioritizing future research through examination of research gaps in systematic reviews. Prepared for the Patient-Centered Outcomes Research Institute, 2012.

8. Whittemore R, Chao A, Jang M, et al. Methods for knowledge synthesis: an overview. *Heart & Lung* 2014;43:453-61.

9. Kastner M, Antony J, Soobiah C, et al. Conceptual recommendations for selecting the most appropriate knowledge synthesis method to answer research questions related to complex evidence. *J Clin Epidemiol* 2016;73:43-9.

10. Manson H. Systematic reviews are not enough: policymakers need a greater variety of synthesized evidence. *J Clin Epidemiol* 2016;73:11-14.

11. Perrier L, Lightfoot D, Kealey MR, et al. Knowledge synthesis research: a bibliometric analysis. *J Clin Epidemiol* 2016;73:50-7.

12. Pluye P, Hong QN, Bush PL, et al. Opening-up the definition of systematic literature review: the plurality of worldviews, methodologies and methods for reviews and syntheses. *J Clin Epidemiol* 2016;73:2-5.

13. Sales A. Generating and using evidence: reflections from the perspective of implementation. *J Clin Epidemiol* 2016;73:6-7.

14. Straus SE, Kastner M, Soobiah C, et al. Introduction: engaging researchers on developing, using, and improving knowledge synthesis methods: a series of articles describing the results of a scoping review on emerging knowledge synthesis methods. *J Clin Epidemiol* 2016;73:15-18.

15. Tricco AC, Antony J, Soobiah C, et al. Knowledge synthesis methods for generating or refining theory: a scoping review reveals that little guidance is available. *J Clin Epidemiol* 2016;73:36-42.

16. Tricco AC, Antony J, Soobiah C, et al. Knowledge synthesis methods for integrating qualitative and quantitative data: a scoping review reveals poor operationalization of the methodological steps. *J Clin Epidemiol* 2016;73:29-35.

17. Tricco AC, Soobiah C, Antony J, et al. A scoping review identifies multiple emerging knowledge synthesis methods, but few studies operationalize the method. *J Clin Epidemiol* 2016;73:19-28.

18. Wong G. Knowledge synthesis approaches-spoilt for choice? *J Clin Epidemiol* 2016;73:8-10.

19. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. *The Lancet* 2009;374:86-9.

20. Bouma GD. *The research process*. USA: Oxford University Press, 2004.

21. Yin RK. *Case study research: design and methods (applied social research methods)*. London and Singapore: Sage, 2009.

22. Setia M. Methodology series module 5: sampling strategies. *Indian J Dermatol* 2016;61:505-9.

23. Patton MQ. *Qualitative evaluation and research methods*. SAGE Publications, 1990.

24. Saunders B, Sim J, Kingstone T, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. *Qual Quant* 2018;52.

25. Guest G, Bunce A. How many interviews are enough?: an experiment with data saturation and variability. *Field Methods* 2006;18:59-82.

26. Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? *Qual Health Res* 2017;27:591-608.

27. Shenton A. *Strategies for ensuring Trustworthiness in qualitative research projects*. 22, 2004.

28. Britten N. Qualitative interviews in medical research. BMJ 1995;311:251-3.

29. Kvale S, Brinkmann S. *Interviews: learning the craft of qualitative research interviewing*. Sage, 2009.

30. Gill P, Stewart K, Treasure E, et al. Methods of data collection in qualitative research: interviews and focus groups. *Br Dent J* 2008;204:291-5.

31. Braun V, Clarke V. Using thematic analysis in psychology. *Qual Res Psychol* 2006;3:77-101.

32. Guba EG, Lincoln YS. *Naturalistic inquiry*, 1985.

33. Glonti K, Hren D. Editors' perspectives on the peer-review process in biomedical journals: protocol for a qualitative study. *BMJ Open* 2018;8:e020568.

34. Macnee CL, McCabe S. *Understanding nursing research: using research in evidence-based practice*. Lippincott Williams & Wilkins, 2008.

35. Holloway I, Wheeler S. Ensuring trustworthiness and quality. In: Holloway I, Wheeler S, eds. *Research in nursing*. 2nd edn. India: Blackwell Publishing, 2002: 250-63.

36. AnneyVN. Ensuring the quality of the findings of qualitative research: looking at trustworthiness criteria. *JETERAPS*;5:272-81.

37. Tobin GA, Begley CM. Methodological rigour within a qualitative framework. *J Adv Nurs* 2004;48:388-96.

38. Connelly LM. Trustworthiness in qualitative research. *Medsurg Nursing* 2016;25:435-7.

39. Robinson KA, Saldanha IJ, McKoy NA. Identification of research gaps from evidence-based guidelines: a pilot study in cystic fibrosis. *Int J Technol Assess Health Care* 2011;27:247-52.