**Warning:**	The NCBI web site requires JavaScript to function. 	more...

An official website of the United States government

**The .gov means it's official.**							Federal government websites often end in .gov or .mil. Before							sharing sensitive information, make sure you're on a federal							government site.						

**The site is secure.**							The **https://** ensures that you are connecting to the							official website and that any information you provide is encrypted							and transmitted securely.						

Log in

#### Account

                          - Dashboard
                          - Publications
                          - Account settings
                          - Log out

Access keysNCBI HomepageMyNCBI HomepageMain ContentMain Navigation

# Bookshelf

## 

                -  Browse Titles 
                -  Advanced 
                -  Help 
                -  Disclaimer 

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

                        - This publication is provided for historical reference only and the information may be out of date.

**This publication is provided for historical reference only and the information may be out of date.**

< PrevNext >

# Systems to Rate the Strength Of Scientific Evidence

*Evidence Reports/Technology Assessments, No. 47*

Suzanne West, PhD, MPH, Valerie King, MD, MPH, Timothy S Carey, MD, MPH, Kathleen N Lohr, MD, Nikki McKoy, BS, Sonya F Sutton, BSPH, and Linda Lux, MPA.

Agency for Healthcare Research and Quality (US)2002 Apr

                            - Copyright and        Permissions

Bookshelf Copyright Notice

## Structured Abstract

#### Objectives:

            Health care decisions are increasingly being made on research-based evidence,						rather than on expert opinion or clinical experience alone. This report						examines systematic approaches to assessing the strength of scientific						evidence. Such systems allow evaluation of either individual articles or						entire bodies of research on a particular subject, for use in making						evidence-based health-care decisions. Identification of methods to assess						health care research results is a task that Congress directed the Agency for						Healthcare Research and Quality to undertake as part of the Healthcare						Research and Quality Act of 1999.

#### Search Strategy:

            The authors built on an earlier project concerning evaluating evidence for						systematic reviews. They expanded this work by conducting a MEDLINE search						(covering the years 1995 to mid-2000) for relevant articles published in						English on either rating the quality of individual research studies or on						grading a body of scientific evidence. Information from other Evidence-based						Practice Centers (EPCs) and other groups involved in evidence-based medicine						(such as the Cochrane Collaboration Methods Group) was used to supplement						these sources.

#### Selection of Studies:

            The initial MEDLINE search for systems for assessing study quality identified						704 articles, while the search on strength of evidence identified 679						papers. Each abstract was assessed by two reviewers to determine						eligibility. An additional 219 publications were identified from other						sources The first 100 Abstracts in each group were used to develop a coding						system for categorizing the publications.

#### Data Collection and Analysis:

            From the 1,602 titles and abstracts reviewed for the report, 109 were						retained for further analysis. In addition, the authors examined 12 reports						from various AHRQ-supported EPCs. To account for differences in study						designs -- systematic reviews and meta-analyses, randomized controlled						trials (RCTs), observational studies, and diagnostic studies -- the authors						developed four Study Quality Grids whose columns denote evaluations domains						of interest, and whose rows are the individual systems, checklists, scales,						or instruments. Taken together, the grids form "evidence tables" that						document the characterisitics (strengths and weaknesses) of these different						systems.

#### Main Results:

            The authors separately analyzed systems found in the literature and those in						use by the EPCs. Four non-EPC checklists for use with systematic reviews or						meta-analyses accounted for at least six of seven domains needed to be						considered high-performing. For analysis of RCTs, the authors concluded that						eight systems represent acceptable approaches that could be used without						major modifications. Six high-performing systems were identified to evaluate						observational studies. Five non-EPC checklists adequately dealt with studies						of diagnostic tests. For assessment of the strength of a body of evidence,						seven systems fully addressed the quality, quantity, and consistency of the						evidence.

#### Conclusions:

           Overall, the authors identified 19 generic systems that fully address their						key quality domains for a particular type of study. The authors also						identified seven systems that address all three quality domains grading the						strength of a body of evidence. The authors also recommended future research						areas to bridge gaps where information or empirical documentation is needed.						The authors hope that these systems will prove useful to those developing						clinical practice guidelines or other health-related policy advice.

## Contents

                          - Expand All
                          - Collapse All

                        - Preface
                        - Acknowledgments
                        - Summary
                        - 1. Introduction                           - Motivation for and Goals of the Present Study
                          - Systematic Reviews of Scientific Evidence
                          - Quality Assessments in Systematic Reviews
                          - Concepts of Study Quality and Strength of Evidence
                          - Organization of This Report
                        - 2. Methods                           - Solicitation of Input and Data
                          - Literature Search
                          - Development of Study Quality Grids
                          - Development of Evidence Strength Grid
                          - Abstraction of Data
                          - Preparation of Final Report
                        - 3. Results                           - Data Collection Efforts
                          - Findings for Systems to Rate The Quality of Individual Studies
                          - Findings for Systems to Rate the Strength Of a Body of Evidence
                        - 4. Discussion                           - Data Collection Challenges
                          - Conceptualization of the Project
                          - Study Quality
                          - Strength of a Body of Evidence
                          - Limitations of the Research
                          - Selecting Systems for Use Today: A "Best Practices" Orientation
                          - Epc Systems
                          - Recommendations for Future Research
                          - Summary and Conclusion
                        - Appendixes                           - Appendix A: Approaches to Grading Quality and Rating Strength of Evidence					Used by Evidence-based Practice Centers
                          - Appendix B: Quality of Evidence Grids
                          - Appendix C: Strength of Eviendce Grids Acronyms
                          - Appendix D: An Annotated Bibliography of Empiric Evidence Used in Grid					Development
                          - Systematic Reviews and Meta-Analyses
                          - Randomized Controlled Trials
                          - Observational Studies
                          - Diagnostic Studies
                          - Appendix E: Excluded Articles
                          - Appendix F: Abstraction Forms
                          - Appendix G: Glossary
                        - References

                          - Expand All
                          - Collapse All

Prepared for: Agency for Healthcare Research and                            Quality, U.S. Department of Health and Human Services.1             Contract No. 290-97-0011. Prepared by: Research Triangle Institute-University of North Carolina Evidence-based Practice Center, Research Triangle Park, North Carolina.

#### Suggested citation:

West S, King V, Carey TS, et al. Systems to Rate the Strength of Scientific                Evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research                Triangle Institute-University of North Carolina Evidence-based Practice Center                under Contract No. 290-97-0011). AHRQ Publication No. 02-E016. Rockville, MD:                Agency for Healthcare Research and Quality. April 2002.

The authors of this report are responsible for its content.            Statements in the report should not be construed as endorsement            by the Agency for Healthcare Research and Quality or the U.S.            Department of Health and Human Services of a particular drug,            device, test, treatment, or other clinical service.

                        - 1
                        - 
2101 East Jefferson Street, Rockville, MD 20852. www​.ahrq.gov

Copyright Notice

Bookshelf ID: NBK33881

< PrevNext >

Share

### Views

                    - PubReader
                    - Print View
                    - Cite this Page West S, King V, Carey TS, et al. Systems to Rate the Strength Of Scientific Evidence. Rockville (MD): Agency for Healthcare Research and Quality (US); 2002 Apr. (Evidence Reports/Technology Assessments, No. 47.)

### Other titles in these collections

                    - AHRQ Evidence Reports
                    - Health Services/Technology Assessment Text                            (HSTAT)

### Related publications

                    - Summary

### Similar articles in PubMed

                    - Rating the strength of scientific evidence: relevance for quality improvement programs. [Int J Qual Health Care. 2004] Rating the strength of scientific evidence: relevance for quality improvement programs.*Lohr KN.**Int J Qual Health Care. 2004 Feb; 16(1):9-18.*
                    - [How to develop evidence-based ophthalmology]. [Zhonghua Yan Ke Za Zhi. 2004] [How to develop evidence-based ophthalmology].*.**Zhonghua Yan Ke Za Zhi. 2004 Apr; 40(4):280-3.*
                    - Evidence-based clinical biochemistry. [Ann Clin Biochem. 1997] Evidence-based clinical biochemistry.*Moore RA.**Ann Clin Biochem. 1997 Jan; 34 ( Pt 1):3-7.*
                    - Review Concepts and principles of evidence-based laboratory medicine. [Am Clin Lab. 1999] Review Concepts and principles of evidence-based laboratory medicine.*Moore RA.**Am Clin Lab. 1999 Jan-Feb; 18(1):24-5.*
                    - Review Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses. [Health Technol Assess. 1999] Review Assessing the quality of reports of randomised trials: implications for the conduct of meta-analyses.*Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, Pham B, Klassen TP.**Health Technol Assess. 1999; 3(12):i-iv, 1-98.*
See reviews...See all...

### Recent Activity

ClearTurn OffTurn On
                      - Systems to Rate the Strength Of Scientific Evidence Systems to Rate the Strength Of Scientific Evidence 
                      - Management of Prolonged Pregnancy Management of Prolonged Pregnancy 
                      - Endoscopic Retrograde Cholangiopancreatography Endoscopic Retrograde Cholangiopancreatography 
                      - Management of Chronic Central Neuropathic Pain Following Traumatic Spinal Cord I... Management of Chronic Central Neuropathic Pain Following Traumatic Spinal Cord Injury 
                      - Management of Chronic Asthma Management of Chronic Asthma 

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on
See more...

TwitterFacebookLinkedInGitHubNCBI Insights Blog

Connect with NLM
                      - Twitter
                      - Facebook
                      - Youtube

National Library of Medicine8600 Rockville Pike							Bethesda, MD 20894

Web PoliciesFOIAHHS Vulnerability Disclosure

HelpAccessibilityCareers

-  NLM 
-  NIH 
-  HHS 
-  USA.gov