# Supporting Knowledge Synthesis Methods Training: Review of the Evidence for Online Systematic Review Instruction

**Robin M. N. Parker, Sarah M. Visintini, Leah M. N. Boulos, Krista Ritchie, and Jill A. Hayden**

*Evidence Synthesis and Information Services Librarian, WK Kellogg Health Sciences Library, Dalhousie University, Halifax, NS; Evidence Synthesis Coordinator, Maritimes SPOR Support Unit and Nova Scotia Site of Cochrane Canada, Halifax, NS; MLIS Candidate and Student Intern, WK Kellogg Health Sciences Library Dalhousie University, Halifax, NS; Assistant Professor, Educational Psychology, Faculty of Education, Dalhousie University, Halifax, NS; Associate Professor, Dept. of Community Health and Epidemiology, Dalhousie University, Halifax, NS*

**Introduction:** Health librarians' support for systematic reviews includes directing reviewers to high-quality educational resources. Extensive research into online learning exists, but is it used to guide online delivery of systematic review training? In order to recommend high-quality training this project addresses the question: Are online approaches for teaching systematic review methodology based on best practices for online instruction? **Methods:** We conducted a comprehensive review of published educational and biomedical literature to identify reports of online instruction in systematic review methods. The paucity of academic literature precipitated an environmental scan to inventory existing web-based approaches. Online training resources were located using strategic web searches. We included instructional material for which we could access the course content freely online or by contacting the creators. After screening to ensure relevance to systematic review methods we extracted data on course characteristics. Using a detailed evaluation rubric developed by Foster et al. (2014) for online evidence-based practice instruction, two reviewers assessed the included courses and tools using the following criteria: 1) design; 2) format; 3) content; 4) degree and type of interactivity; 5) general usability. This evalua-

tion revealed the extent to which the training resources followed accepted best practices in online instruction. **Results:** We present the results of our evaluation of online courses, modules, and videos that provide instruction on some or all of the steps of conducting systematic reviews. Resources assessed varied in means of delivery, type of access (free or for-fee), and intended audience. The content was similarly diverse, with some courses or series of modules covering all steps of the systematic review process, while others, particularly video tutorials, frequently addres- sing only a portion of the stages of conducting a review and having minimal interactivity. **Discussion:** There is a range of resources available those learning how to conduct system- atic reviews or other knowledge synthesis projects. The most appropriate training will depend on the needs and resources of the individual researcher: freely available videos and training modules may give a helpful overview of the process or a reminder of common challenges while online systematic review courses offered by research or academic institutions provide more in-depth and interactive coverage of each step and will help reviewers complete a systematic review in real- time, however they are frequently costly or require a more significant commitment of time.

# Locating Systematic Reviews Comprehensively and Efficiently

**Robert W. Sandieson and Rachel M. Sandieson**

*Professor, Faculty of Education, Western University, London, ON; Senior Library Assistant, Allyn and Betty Taylor Library, Western University, London, ON*

**Introduction:** Systematic reviews have proliferated and evolved recently to include many types of research methods. Difficulties result locating reviews due to index-

ing, terminology, methods or textual context. The present investigation tested a search filter development theory, known as Pearl Harvesting, in order to find all of the
unique linguistic markers related to systematic reviews in order to create comprehensive searching. **Methods:** Two texts on systematic reviews, one quantitative and one qualitative, were analyzed for linguistic content pertinent to systematic reviews. Some terms were noted for the lack of precision, e.g., "review" so were disambiguated to create a specific rich text filter. The terms were validated as necessary to the filter by searching each term separately and subtracting citations produced by all the other terms to see if any remaining were unique and relevant. Existing search strategies for systematic review use low precision polysemic terms, such as "review". Precision is measured at the level of individual term not as used in real world searches that contain a number of search filters. Therefore, a further analysis was done to create the complete set of polysemic terms for systematic review as a second, equivalent filter. The question was how extensive was the loss of precision at the real world search level where population and intervention filters were included. **Results:** There were 110 specific linguistic markers found for a

specific, rich text filter. Another filter contained three polysemic terms, i.e., review*, synthesi*, systemat*, plus search*, and variations of meta-analysis. Studies that developed search strategies for systematic reviews vary considerably and do not contain all of these terms; either the specific or polysemic. The two filters here were equivalent in finding systematic reviews. An analysis of real world searches found a minimal loss of precision using the polysemic filter in some cases, but not in others if the number of citations found was extensive. Further, an analysis of a Cochrane review of reviews demonstrated that the filters developed here could locate its 78 primary systematic reviews. **Discussion:** The Pearl Harvesting Information Retrieval Theory was valuable in developing two equivalent search filters for systematic reviews. The polysemic filter has an ease of use. The rich text filter is more precise and of value if the search yields a high volume of citations. This study indicated that search precision is a relative phenomena depending on context.

# Information Literacy on the Move: Delivering Library Training to Hospital Staff with a Mobile Computer Lab

Katie D. McLean, Vivien Gorham, and Katie Quinn

Librarian Educator, Nova Scotia Health Authority, Halifax, NS; Library Technician, Hospital Library Services, Halifax, NS; Library Technician, Hospital Library Services, Halifax, NS

**Introduction:** Reliable access to well-maintained computer labs and a dependable Internet connection can be hard to come by in the hospital environment. With the majority of information resources being online, and local resources failing to support easy access, the library needed a better option for delivering instruction. Would a mobile computer lab maintained by library staff provide a practical solution?

**Methods:** Library staff researched other libraries using mobile labs and purchased four laptops, an Internet Hot-spot, four mice, and a carrying case on wheels. The mobile lab was used for all standard library training over a period of one year. Training attendance, evaluations and overall cost were used to measure the success of the intervention.

**Results:** Prior to delivery of training events through the mobile lab, clients consistently provided negative feedback regarding the state of hospital computers. One year after mobile lab implementation, negative feedback citing computer issues dropped by over 90%. Numbers of persons trained remained relatively consistent with previous years, at approximately 720 persons, during the first year of mobile lab use. Cost per person trained was about $6.00 in the first year. **Discussion:** Use of a mobile computer lab increased client satisfaction with library training, and was found to be well worth the initial cost. Other hospital libraries may find this approach a worthwhile investment.
# Health and Literacy: The Mini-Med School and the Literacy Foundation Working Together to Reach Native Children

Monique Clar, Sophie Chiasson, Éric Drouin, Bianca Seminaro, and Maryse Fagnant

Health Sciences Librarian, Bibliothèque de la santé, Université de Montréal, Montréal, QC; Library technician, Service des acquisition, Université de Montréal, Montréal, QC; First Nations Program Advisor, Département de pédiatrie, Université de Montréal, Montréal, QC; Student, École de bibliothéconomie et des sciences de l'information, Université de Montréal, Montréal, QC; Medical student, Medical School, Université de Montréal, Montréal, QC

The Mini-Med School (MMS) aims to motivate Native children to stay in school, introduce them to the health professions and encourage them to have healthy living practices. As reading skills play a major role in student retention, the MMS, in collaboration with the Health Library and the Literacy Foundation, is taking actions to facilitate children's access to books. The MMS is a University program where health sciences students visit Native communities' schools. The Health Library participates in those visits, develops book collections with the school libraries and recruits LIS students for the MMS book booth. The Literacy Foundation's The Gift of

Reading program aims to prevent reading and writing difficulties by giving books to underprivileged children. The MMS and the Library are joining forces with the Foundation to give children a better access to books. The medical librarian will continue working with the schools to upgrade the donated collections according to data on book usage and appreciation. MMS book collections will also be provided to three additional Native schools. On the other hand, The Gift of Reading program will now reach many Native children at home in those communities, by allowing them to receive a brand new book.

# PRESS: Peer Review of Electronic Search Strategies 2015 Updated Guideline Statement

Margaret Sampson, Jessie McGowan, Douglas M. Salzwedel, Elise Cogo, and Carol Lefebvre

Manager, Library Services, Children's Hospital of Eastern Ontario, Ottawa, ON; Health information consultant / Adjunct professor, School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, Ottawa, ON; Trials Search Coordinator / Information Specialist, University of British Columbia, Vancouver, BC; Research Consultant and Information Specialist, Independent, Scarborough, ON; Independent Information Consultant, Lefebvre Associates Ltd, Oxfordshire, United Kingdom

**Introduction:** To update an evidence-based guideline for peer review of searches for evidence syntheses, including systematic reviews and health technology assessments, and to assess the evidence of impact of peer review on search performance. **Methods:** Evidence-based elements related to search quality or search errors were identified through a systematic review, web-based survey and international consensus development forum. Eligible evidence for the systematic review related either to individual search elements or checklists of elements and must have reported impact on recall or precision. Evidence was summarized and integrated with expert opinion obtained from the survey and forum. A guideline statement was published including an assessment form, guidance on its use and recommendations

for librarians implementing PRESS in their practice. An elaboration and explanation document was also prepared documenting complete methods for the update as well as systematic review findings. **Results:** Several studies showed that peer review checklists were effective at improving search quality. Six existing PRESS elements were retained and their peer review will be briefly described: translation of the research question; Boolean and proximity operators; subject headings; text word search; limits and filters, syntax and line numbers; and spelling. No new elements were identified for addition to the existing PRESS Evidence Based Checklist (2010) and no evidence refuted any of the existing elements. Survey respondents and forum participants recommended that peer review should occur after the MEDLINE search
has been developed but before other database searches are
developed. The element related to translation of the search
for other databases was removed, with the recommendation
that translations be peer reviewed at the discretion of the
search developer. **Discussion:** PRESS provides tools to guide
the peer review of electronic search strategies and improve
their effectiveness. Elements presented here are detailed in
the PRESS: Peer Review of Electronic Search Strategies

2015 Guideline Statement (http://www.sciencedirect.com/
science/article/pii/S0895435616000585) including the Assess-
ment Form, guidance in its use and recommendations for
librarians implementing PRESS in their practice. The PRESS
Explanation andamp; Elaboration (https://www.cadth.ca/
press-2015-guideline-explanation-and-elaboration) reports
complete methods for the update as well as results of the
systematic review.

eSRAP: A System for Collaborative Monitoring of
Latest Trends in Patient-Oriented Research

Vera Granikov, David Li Tang, France Bouthillier, and Pierre Pluye

Research-embedded health information specialist, Family Medicine, McGill University, Montreal, QC; Adjunct Professor, Family Medicine, McGill University, Montreal, QC; Associate Professor, School of Information Studies, McGill University, Montreal, QC; Full Professor, FRQS Senior Research Scholar, Family Medicine, McGill University, Montreal, QC

**Introduction:** Despite numerous literature surveillance tools and applications, users continue to be challenged by time constraints, information overload, and insufficient appraisal skills. eSRAP is a collaborative research trend monitoring system for Patient Oriented Research (POR). Through crowdsourcing, its purpose is to help POR communities to quickly identify relevant high quality studies, to support teaching, continuing education, preparation of protocols, and publications. **Methods:** eSRAP is an open access system funded by the ‘Methodological Developments’ platform of the Quebec SPOR-SUPPORT Unit. Using search strategies developed by a librarian and subject experts, the system retrieves new publications as soon as

they are indexed in bibliographic databases (e.g., Scopus).
Users then read and rate abstracts according to community-
determined appraisal criteria. The main advantages of
eSRAP are the collaborative “filtering” of most relevant
high quality studies and user-based customization. To
identify the barriers and the facilitators associated with
system use, we will use a mixed methods design, combining a
prospective observational study with a qualitative multiple
case study. Evaluation results will be used to improve
eSRAP and will contribute to knowledge on research trend
monitoring and literature surveillance tools. Finally, lessons
learned may be transferable to other professional groups
facing challenges associated with staying current.

Librarians! Let's Leverage Our Role to Raise the
Quality of Biomedical Research

Heather Cunningham, Erica Lenton, Ana Patricia Ayala, and Shona Kirtley

Assistant Director, Research and Innovation Unit, Gerstein Science Information Centre, University of Toronto,
Toronto, ON; Faculty Liaison and Instruction Librarian, Gerstein Science Information Centre, University of Toronto,
Toronto, ON; Instruction and Faculty Liaison Librarian, Gerstein Science Information Centre, University of Toronto,
Toronto, ON; EQUATOR Knowledge and Information Manager/Senior Research Information Specialist, Centre for
Statistics in Medicine, University of Oxford, Oxford, United Kingdom

**Objective:** Reducing research waste through adherence to protocols and reporting guidelines is increasingly important in health science research. This presentation examines how librarians can leverage our participation on research teams to raise awareness of, and compliance with, reporting guidelines and protocols for biomedical research. **Brief Description:** Dedication to research reproducibility is a key aspect of our library's knowledge synthesis service. We see

collaborating on protocol registration and publication as
crucial. Incorporating protocol adherence in the early stages
of synthesis studies will result in a more robust manuscript
and will ease the journal submission process. **Program
Evaluation:** A mixed methods approach of this long term
study will measure protocol compliance, knowledge and
experience of research teams regarding reporting guidelines.
**Anticipated Outcomes:** Reproducibility of the knowledge
synthesis model for other libraries/settings; librarians use
their unique position in the knowledge synthesis cycle to
raise awareness about reporting guidelines; open dialogue in
the medical librarianship community on the ways they can

positively impact the quality of biomedical research; deter-
mining incentives and barriers to reporting guideline
compliance; demonstrating improved quality in the conduct
and reporting of knowledge syntheses.

Dare to Know: Sharing the Value of the Library

Susan Baer

Director of Libraries and Archives, Regina Qu'Appelle Health Region, Regina, SK

**Introduction:** To raise awareness of the library's value to the organization to avoid cuts to the library's budget, a multi-faceted promotional campaign was designed featuring the library's new brand. Under the provincial library week theme, five champions shared their insights regarding the value of the library to their practice, providing excellent, impartial quotes for promotion with other clinical staff. **Methods:** Champions were selected based on frequency of library use or from large research projects. The campaign also included comments from the CEO regarding the library's alignment with the organization's goals. Interviews consisted of five questions of 15 minutes, which were taped

to capture all comments. Quotes were selected to integrate
into the library's rebranding project. During library week,
the new pamphlet was launched, and five promotional
messages were broadcast using digital signage, websites,
posters, the organization's e-newsletter and email in an
attempt to capture the attention of clinical staff. Statistics
from previous research studies connecting the library's
value to patient care reinforced the messaging. Snapshots of
library statistics before library week were compared with
statistics for the following two months. Along with the
increase in usage, the advocacy component was an immea-
surable benefit of the campaign.

Finding the Middle Ground Between a Web
Specialist and the Wild West: A New Web
Governance Model for an Academic Health
Sciences Library

Susanna Galbraith and Stephanie Sanger

Virtual Services Librarian, McMaster University, Hamilton, ON; Clinical Services Librarian, Health Sciences Library, McMaster University, Hamilton, ON

**Purpose:** This project investigated current web governance practices within and outside libraries to develop an internal web governance model for an academic health sciences library. **Setting/Participants/Resources:** Librarians in collaboration with the management committee at an academic health sciences library. **Brief Description:** This project was initiated by the director of the library to solve existing issues with content management and web management at the library. This talk describes the process, final outcome successes and challenges encountered in developing a new web governance model. Web governance for this project has been defined as a framework for establishing

accountability, roles and decision-making authority for an
organization's web presence. Challenges we foresee with
implementing the proposed governance model include
working with existing organizational structures, change
management and buy-in from library staff. **Results/Outcome:** The new web governance model has been
presented to the management committee at the library.
The recommendations are being finalized for implementa-
tion. **Evaluation Method:** The recommendations will be
piloted for one year. Library staff will be surveyed before,
during and again at the end of the year to assess the project's
success.