# Product Review: Covidence (Systematic Review Software)

**Product:** Covidence (Systematic Review Software)

**Purpose:** Web-based systematic review tool designed to facilitate the process of screening, data extraction, and analysis.

**URL:** www.covidence.org

**Intended Audience:** Researchers, participants in systematic review processes

**Bottom line:** Covidence is an inexpensive product with a simple design, enabling multiple reviewers to work more efficiently through the steps of a systematic review. If used effectively, and with strong foresight, it could help to streamline and facilitate a review process. However, users should be aware of its limitations in citation uploading and the lack of customizability of the screening tools. A strong understanding of the nuances of systematic review processes and a strong protocol would be essential for the effective use of Covidence.

## Product description

Covidence is a relatively new web-based systematic review program that aims to make evidence synthesis a more proficient process. Covidence is a not-for-profit service started in 2013, and it is run by a team in Melbourne, Australia.

Elements of the systematic review that can be conducted via Covidence are: citation importing and screening, full-text review, study selection, quality assessment, data extraction, and data exporting. Covidence enables users to work through the steps of the systematic review process more fluidly, theoretically avoiding the time-consuming logistics and organizational headaches often associated with systematic reviews.

## Cost and subscription

Covidence requires users to register and login with an email address. Registered users can participate in their first systematic review free of charge. With a paid account (costing $20 per month), members can create and manage an unlimited amount of reviews.

One of the benefits of this web-based systematic review program is that it allows several researchers to work together on one project in real time. Moreover, it is possible to invite others to participate in a review; however,

by joining an already created review, new members will be using up their first free review.

## User support

Covidence boasts a simple and minimalist interface. The company's confidence in its easy to use interface has led to the omission of a “Help” page. Instead, the site provides a “Support” tab, enabling users to message its support team (personal experience has been that they are very quick to respond!). The site's instructions are brief and exist for certain icons only (e.g., brief instructions are provided for uploading citations near the “Upload” icon). There are also short demonstrations of some tasks available on their YouTube channel.

## Features

### Importing citations

Prior to screening citations, users must first import citations to the Covidence website. Citation searching and retrieval are often the responsibility of the health sciences librarian. Thus, this process is given more emphasis in this review than other aspects of the software. From an information management perspective, Covidence provides few pathways to import citations prior to screening for reviews, and some import methods are more valuable than others.

The first option allows users to conduct searches from within Covidence through an embedded PubMed search. Users can copy and paste a PubMed search strategy into the “Settings” section of their review. They can choose to run the embedded PubMed search strategy, which will automatically import the results into the screening section. This method keeps track of the date the user ran the search and allows them to run updated searches at a later date; the aforementioned searches are date-limited so as not to import articles preceding the last run date. By running the same PubMed search within Covidence and via the PubMed database, users attain the same number of results. Although this search functionality is an interesting feature, it can only be done via PubMed. Systematic reviews should consider more than one database.

A second importing option is to import records via RIS, CRS, or Endnote XML formats (bibliographic file formats commonly used to transfer bibliographies from one reference manager to another or to other applications
that handle bibliographic references). Users can do this from the “Screen” section, where there is a small icon called “Import citations” (Figure 1).

There is no option to search for duplicates in Covidence. Any and all citations that are imported into the program are immediately sent to the “Screen” section for screening. It is also not possible to delete or remove citations. In addition, users cannot view any unique identifiers for citations.

## Screening citations

Reviewers are provided with a section titled “Citations needing your vote.” In the settings section, it is possible to determine whether you want to have one or two reviewers only. Records to review cannot be allocated to specific reviewers (if you have three reviewers available, any two of these three could review any citation). There is assumed dual reviewer agreement (i.e., users cannot have expedited screening where it would only need one person to move an article forward, but two people to reject it).

It is not possible to develop forms or questions for the screening process. Users can only vote “Yes”, “No”, or “Unsure” for articles that are in their screening list. Members are provided with all of the available citation information for each record (title, authors, year, journal, issue, volume, and abstract) (Figure 1).

After articles are reviewed and conflicts are resolved, included citations will be moved to the next screening process: the full-text screen. Citations that are not included are moved into a section called “Irrelevant.”

Covidence allows for only one screening prior to the full-text screen. This screening allows reviewers to view all of the available information for this record. This can be

limiting, as some reviewers may want to consider two or more screening processes of varying levels of information (e.g., title only, title and abstract only). Users are unable to remove or hide additional information such as journal title, year, and authors from reviewers during the screening process.

## Full-text screening

Full-text screening is done in a similar fashion to the first-level screening. Users can determine whether one or two reviewers are required to review the article through the “Settings” tab. If a reviewer chooses to exclude a citation, they are given a drop-down box with reasons for exclusion. There is a prepopulated list, but it is also possible to add additional reasons for exclusion.

Users can manually attach one PDF file to each citation record. No other document types are accepted; nor can the user attach more than one PDF file to one record.

## Quality assessment and data extraction

For both quality assessment and data extraction, Covidence has predeveloped forms that are ideal if you are extracting data from clinical trials. This can be problematic for reviews that include a wider or different range of study types.

An impressive feature of Covidence is its ability to extract information directly from a PDF file. Users can highlight sections of a PDF file, identify which section of the form it is relevant to, and add discussion notes (Figure 2). Doing this in the PDF file will transfer information to your quality assessment or data extraction form.

<figure><img src="image_1.png" /><figcaption>**Fig. 1.** Covidence screening page where users can manually import and screen citations.</figcaption></figure>
<figure><img src="image_2.png" /><figcaption>**Fig. 2.** Covidence enables users to perform data extraction and quality assessment by highlighting and classifying PDF portions. Once highlighted, data are automatically entered in the appropriate form.</figcaption></figure>

The quality assessment form will auto save changes, whereas there is a save icon on data extraction pages. Users must save their work. Data extraction will not be auto saved.

## Data export

The Covidence export function is designed to work with Review Manager (RevMan). Covidence has the ability to populate a pre-existing RevMan file with citation information. The characteristics of included/excluded studies tables will be populated along with data tables and references. Data extraction information can also be exported to a comma-separated values (CSV) file.

## Strengths

* Enables several reviewers to screen and review articles in real time

* Low cost

* Simple navigation

* Quick response time from support team

* Manages inclusion and exclusion of citations, ensuring accurate reporting numbers

* Direct import from Endnote and direct export to RevMan

* Innovative data extraction features

## Weaknesses

* Lack of static help pages

* Internal searching mechanism limited to PubMed

* Inability to screen records for duplicates

* Does not provide unique identifiers to records

* Inability to customize screening processes

* Limited reviewer screening options

* Focused on review of clinical trials

## Conclusion

Covidence is still working its way through the systematic review facilitator world. The practice of knowledge synthesis has greatly expanded from traditional intervention study reviews, and Covidence will need to become more flexible to work effectively for a greater number of reviewers.

To effectively use this tool, users should have a solid understanding of systematic review processes, and the related information management issues that can arise. From an information management perspective, I would not recommend using the embedded literature searching features of the program, but rather start working from within a citation management system and import records
only after duplicate screening. The lack of customizability at the screening levels will require reviewers to have a very sound understanding of their inclusion and exclusion criteria prior to screening. Before beginning a review, it is essential for a user to explore the program to ensure that the limitations in customization still enable Covidence to meet reviewers' needs. Although Covidence is designed to streamline the screening and extraction processes of a review, not considering its limitations could lead to increasing a workload rather than easing it.

**Jessica Babineau**

*Information Specialist*
*Library and Information Services*
*University Health Network*
550 University Avenue
Toronto Ontario
M5G 2A2
Email: Jessica.babineau@uhn.ca