A network analysis of the propagation of evidence regarding the effectiveness of fat-controlled diets in the secondary prevention of coronary heart disease (CHD): Selective citation in reviews
Authors: Rhodri Ivor Leng
Journal: PLoS ONE (2018)
Discipline: Multidisciplinary
Keywords: randomised controlled trials; cholesterol-lowering diets; secondary prevention; coronary heart disease; literature reviews
Affiliations: Department of Science, Technology and Innovation Studies, School of Social and Political Science, University of Edinburgh, Edinburgh, United Kingdom.

Abstract:  Objective: To examine how the first randomised controlled trials (RCTs) evaluating the efficacy of cholesterol-lowering diets in the secondary prevention of coronary heart disease were interpreted in reviews of the literature prior to the National Institutes of Health consensus conference in 1984. Design: Claim-specific citation network analysis was used to study the network of citations between reviews and RCTs over a defined period (1969-1984). RCTs were identified and classified according to whether their conclusions supported or opposed the use of dietary fat modification/restriction in the secondary prevention of coronary heart disease. Each review published in this period that cited any of the RCTs was classified as supportive, neutral, or unsupportive to the use of dietary fat modification based on a quotation analysis of its evaluation of the findings of these RCTs. Citation bias and underutilisation were detected by applying a comparative density measure, in-degree centrality, and out-degree in a series of sub-graph analyses. Results: In total, 66 unique publications were identified (four RCTs-one supportive, three unsupportive; 62 reviews-28 supportive, 17 neutral, 17 unsupportive). On average, supportive reviews underutilised the available RCTs to a greater degree than other reviews. Amongst the supportive group, citation bias was common-23 (82%) reviews cited only the one RCT that was supportive. Conclusion: Most reviews that disseminated a supportive evaluation of the results of RCTs in the context of secondary prevention cited only data that supported this position. 

Retracted: No

PubPeer comments:
 The concept of citation network analysis is not new, but it is important. In this manuscript, Dr. Leng uses Web of Science to analyze a subfield of papers about radomized controlled trials (RCTs) of coronay heart disease between 1965-1984. While this topic seems limited in scope, Dr. Leng argues the opposite is true: this example provides insight into how scientists resolve conflict through manuscript publciation. He finds clear citation bias toward positive studies and reviews. In general, the scientific literature reports an overwhelming % of positive results, and it is unsurprising this study finds that reviews of the RCT literature was biased toward studies that presented positive data. Notably,  one finding of this paper is that reviews in this field were poor at considering all the available empirical evidence . This makes sense when one considers the limitations of review articles and the narrative-centric motivations of review authors. In sum, this paper aims to emphasize the limitations of citation metrics in evaluating the impact of individual manuscripts and the direction of research fields. ,