DOI: 10.3886/icpsr39615.v1
title: Develop, Test, and Disseminate a New Technology to Modernize Data Abstraction in Systematic Reviews [Methods Study], United States, 2013-2019
title: Version 1

description: Systematic reviews combine the results of many studies. In health research, these reviews can help determine which treatments or types of care work best. As part of a systematic review, researchers find and record important study information, such as design and results, from published journal articles. This process, called abstraction, takes time. If researchers make errors during this process, the systematic review may come to incorrect conclusions, which can affect healthcare decisions. Researchers abstract information in different ways. In single abstraction and verification, one person abstracts information and a second person reviews it for accuracy. In dual abstraction, two people abstract information on their own and compare the results.
In this study, the research team created and tested a new software program to help with abstraction. In the new software, researchers place flags within a journal article, displayed next to a data collection form on a computer screen, to easily find abstracted information. The team compared three approaches for abstracting information:

Single abstraction and verification with the new software
Single abstraction and verification without the new software
Dual abstraction without the new software

The research team looked at how accurate the abstractions were and how much time it took to do them.
 To access the software and methods, please visit the DAA Bitbucket.
description: To compare the accuracy and efficiency of data abstraction supported by new Data Abstraction Assistant (DAA) software with two traditional methods for data abstraction in systematic reviews.
description: This randomized crossover trial compared the accuracy and efficiency of single abstraction plus verification using DAA (DAA verification), single abstraction plus verification without DAA (regular verification), and independent dual abstraction without DAA (independent abstraction). The research team developed DAA for this study; DAA creates a permanent link between relevant information in the journal article and the electronic data collection form displayed side-by-side on a computer screen used for abstraction.
The study included 52 participants with prior data abstraction experience. Participants worked in 26 pairs; each pair included a more experienced abstractor who had authored three or more published systematic reviews and a less experienced abstractor. Each pair abstracted six articles on different topics, such as preventing falls or depression treatment. Pairs used each of the three approaches, in a random order, on two articles:

DAA verification. The less experienced abstractor used DAA to complete abstractions. The more experienced abstractor reviewed for accuracy and, if needed, changed the abstracted content or discussed changes with the original abstractor to reach consensus.
Regular verification. The less experienced abstractor completed abstractions without using DAA. As with DAA verification, the more experienced abstractor reviewed for accuracy and worked with the original abstractor to reach consensus.
Independent abstraction. Each of the two abstractors worked independently without using DAA. The pair then compared responses and discussed any differences to reach consensus.
To determine error proportions, the research team compared abstracted information with an answer key developed by two experienced systematic reviewers. The abstractors also recorded the time spent on data abstraction.

Patients, policy makers, health industry representatives, and researchers with experience in systematic reviews, clinical trials, and clinical practice guidelines contributed to all aspects of the study.
description: Datasets: 
DS1: Develop, Test, and Disseminate a New Technology to Modernize Data Abstraction in Systematic Reviews [Methods Study], United States, 2013-2019
description: Researchers from Johns Hopkins University and Brown University.