A Scoping Review Protocol on the Use of Artificial Intelligence Techniques in Nursing Data Systems
Creators: name: Singla, Shina
name: Medeiros, Christina
name: Howitt, Lyndsay
name: Burt, Amy
name: Nizum, Nafsin
name: Naik, Shanoja
name: Grinspun, Doris
Publisher: name: Open Science Framework
Year: 2023
Subjects: subject: Statistics and Probability
subject: Physical Sciences and Mathematics
subject: Medicine and Health Sciences
subject: Computer Sciences
subject: Nursing
subject: FOS: Health sciences
subject: Artificial Intelligence and Robotics
subject: Databases and Information Systems

description: Introduction In 2003, the Registered Nurses’ Association of Ontario (RNAO) launched the Best Practice Spotlight Organization® (BPSO®) designation as a key knowledge-translation strategy to support the implementation of best practice guidelines (BPGs), rapid learning and sustainability of evidence-based practice changes[1]. RNAO has since developed two data systems to monitor and evaluate the impact of BPG implementation: Nursing Quality Indicators for Reporting and Evaluation® (NQuIRE®) and the MyBPSO reporting system. The NQuIRE data system collects aggregated de-identified data from BPSOs related to their BPG implementation efforts based on quality indicators. Recently, the RNAO has launched an artificial intelligence (AI) and machine learning (ML) initiative to assess data obtained from the two data systems and predict patterns in BPG implementation across organizations. AI and ML uses various computer techniques and advanced analytics to unwrap patterns of hidden information[2]. AI and ML techniques also aid in the performance of different tasks via statistical and algorithmic calculations that conventionally required human intervention in a comparatively faster way[2]. ML algorithms can be used to manage Big data and help with nursing assessments, interventions, clinical decision-making and documentation[3]. AI-powered techniques can also play a vital role in contributing to the understanding of BPG implementation. To better understand the impact of BPG implementation, RNAO intends to develop AI and ML techniques applicable to the NQuIRE and MyBPSO data systems. This makes it imperative to know which AI and ML methods exist to further support clinical decision-making and BPG implementation, which provided the basis for our scoping review. The goals of this scoping review are to gain a deeper understanding of implementation strategies and practice changes that improve clinical, organizational and health-system outcomes, and to advance BPSOs capacity for rapid learning and continuous quality improvement which can be uncovered via AI and ML techniques. Objectives The purpose of this scoping review is to understand the adaptation and implementation of various AI and ML techniques in a variety of health-care contexts utilizing different data systems which house nursing-related aggregated data. Keywords and subject headings associated with the following terms were combined: “health care”, “machine learning and deep learning”, and “data systems” (Appendix). Methods Standardized reporting guidelines stipulate certain items to include in research reports to improve methodological transparency and the uptake of research findings[4]. Specifically, scoping reviews follow systematic approaches to search and gather evidence while identifying knowledge gaps. Unfortunately, there is no existing scoping review protocol guidance. However, The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) provides a checklist of items for scoping review development[4], used as a guiding document for this review. The summary of RNAO’s scoping review is registered in the Open Science Framework database (registration doi: 10.17605/OSF.IO/ZP6CG)[5]. This scoping review will map out current available evidence on AI and ML techniques. This may include identifying core concepts related to AI and ML, relevant terminology or techniques associated with AI and ML, knowledge gaps and summative techniques that were potentially adapted to build AI. Members of the Monitoring &amp; Evaluation team at RNAO (SS, CM and SN) initiated this scoping review by conceptualizing the application of AI and ML techniques in nursing databases and determining the need for further research. The RNAO Guideline Development and Research Team (LH, AB and NN), provided support and aided in development of the methodology of the scoping review. RNAO’s Chief Executive Officer (DG) and Associate Director and Chief Scientist, RNAO (SN) informed the scope and direction of the project and will provide feedback on the results. This scoping review followed the Arksey and O'Malley framework[6] which has been further advanced by Levac, Colquhoun and O’Brien[7]. The five steps of the methodological framework include: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing and reporting the results. Step 1: Identifying the research question This scoping review was conducted to build on evidence that supports RNAO’s AI and ML initiative. First, a research question was constructed to guide the scoping review[7]: What type of AI and ML techniques have been used in nursing data systems that are built on the use of aggregated data? In this protocol, nursing data systems refer to any data systems used to store clinical data related to the nursing scope of practice in health service organizations for future use or analysis. The term “aggregated data” refers to de-identified organizational information and aggregated data from quality indicators collected on a monthly basis. This does not include data registries, which only store data and are not part of data analysis at the organization level. Step 2: Identifying relevant studies A broad search strategy was developed to ensure the comprehensiveness of the evidence retrieved, which is a common aspect of all scoping reviews[6]. The MEDLINE, Cumulative Index of Nursing and Allied Health Literature (CINAHL) and Embase databases were searched for peer-reviewed literature. The search terms (Appendix) were drafted by SS and CM and were reviewed by LH and SN. Search terms included “nursing”, “artificial intelligence”, “data systems”, “statistics”, and “aggregated data”. Relevant Medical Subject Heading (MeSH) terms, CINAHL Subject Headings and keywords were added and combined using Boolean operators “AND” and “OR”. One reviewer ran the final search strategy (SS). Inclusion and exclusion criteria were developed through consultations with all authors. Grey literature was excluded from this scoping review because the focus was on scientific peer-reviewed literature and the fact that grey literature may be limited due to incomplete or incorrect identification. Grey literature is more difficult to process than other types of open-source citations owing to its nonstandard formats. The literature search was limited to articles published in the last two decades to ensure the evidence on AI and ML techniques was current. Our inclusion/exclusion criteria are described below: Inclusion criteria Quantitative studies All included citations must be: • Peer-reviewed • Published in the year 2000 or after • Available in English • Relevant to AI/ML in nursing • Based on aggregated data • Utilizing nursing data systems Exclusion criteria Qualitative studies and mixed-method studies • Grey literature such as theses, editorials, white paper publications • Conference abstracts • Discussion papers • Case studies • Published literature reviews, systematic/scoping reviews Step 3: Study selection Citations extracted from three databases were imported into EndNote[8] for de-duplication and then uploaded into DistillerSR[9] for screening. Two reviewers (SS and CM) independently screened citations in two stages: 1) a title and abstract screening and 2) full text review. The title and abstract screening was performed to determine relevancy of studies to the “use of AI and ML in the field of nursing or delivery of health care” while adhering to the inclusion criteria. For the full text review, the inclusion/exclusion criteria were even further refined to determine the utilization and analysis of aggregated data and nursing data systems. The full text articles were downloaded to DistillerSR[9], independently reviewed by SS and CM. Any disagreements were resolved through discussions by both reviewers. Step 4: Charting the data Data from included studies are extracted at this stage[7]. The data charting form will be created by the two reviewers (SS and CM) in Microsoft Excel 2019 (Microsoft Corp) and will be reviewed by SN and LH. Since data charting is an iterative process[7], the data charting form will be refined concurrently with data extraction. SS and CM will extract data from half of the citations and will then verify the half of the charted data they did not input. For the purpose of this scoping review, the proposed information to be extracted is as follows: • RefID (from DistillerSR) • First author (Year) • Study design • Country • Study setting • Eligibility criteria • Objective of study • Type of AI tool used • Description of AI method • Quantitative outcomes • Clinical outcomes • Additional comments/notes Step 5: Collating, summarizing and reporting the results There are three distinct steps to enhance the consistency of reporting: analyzing the data, reporting results, and applying meaning to the results[7]. A summary of the study findings will be presented in a manuscript following completion of the scoping review. Results Electronic database searches were conducted in March 2022 and a total of 2627 articles were retrieved and downloaded into EndNote[8]. After the removal of duplicates, 1518 articles remained. RNAO researchers aim to complete title and abstract screening, full text review, data charting and publishing a journal article by spring 2023. Conclusion and Future Directions The objective of this scoping review is to summarize the evidence available on current AI technologies used by data systems containing aggregated nursing-specific data. These insights will assist in ensuring that RNAO’s data systems are up to date with the most current AI and ML techniques. Preliminary findings indicate that AI and ML has been applied to the fields of oncology, pediatrics, neurology, and the use of Big data in nursing and health care. Recent publications also discuss the use of AI and ML techniques in COVID-19-related research. These findings will be presented in a final manuscript, and will also contain opportunities for implication of the findings and gaps for further research. Funding This work is funded by the Government of Ontario. All work produced by RNAO is editorially independent from its funding source. Conflicts of Interest None declared References 1. Registered Nurses’ Association of Ontario. Best Practice Spotlight Organizations (BPSO) (2023). Source: https://rnao.ca/bpg/bpso 2. Registered Nurses’ Association of Ontario. Nursing &amp; Compassionate Care in the Age of Artificial Intelligence: Engaging the Emerging Future (2020). Source: https://rnao.ca/sites/rnao-ca/files/RNAO-AMS_Report-Nursing_and_Compassionate_Care_in_the_Age_of_AI_Final_For_Media_Release_10.21.2020.pdf 3. Ahmad, S., Jenkins, M. Artificial intelligence for nursing practice and management: Current and potential research and education. CIN Plus, 40(3), 139-144, 2022. 4. Tricco, A.C., Lillie, E., Zarin, W., O'Brien, K.K., Colquhoun, H., Levac, D., et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med 2018 Oct 02;169(7):467-473. https://doi.org/10.7326/M18-0850 5. Singla, S., Medeiros, C., Howitt, L., Burt, A., Nizum, N., Naik, S., &amp; Grinspun, D. (2022, November 24). A Scoping Review on the Use of Artificial Intelligence Techniques in Nursing Data Systems. https://doi.org/10.17605/OSF.IO/ZP6CG 6. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology 2005 Feb;8(1):19-32. https://doi.org/10.1080/1364557032000119616 7. Levac D, Colquhoun H, O'Brien K. Scoping studies: advancing the methodology. Implement Sci 2010; 5:69. https://doi.org/10.1186/1748-5908-5-69 8. EndNote. Available: https://endnote.com/ 9. DistillerSR. Available: https://www.distillersr.com/