The quest for excellence (part 2): reviewing a clinical trial for the<i>British Journal of Dermatology</i>
Authors: J.R. Ingram
Journal: British Journal of Dermatology (2014)

Consider the scenario of dermatologists in a clinic, weighing up an important therapeutic decision for their patient. Being aware that the evidence is limited they perform a quick PubMed search and are delighted to discover a randomized controlled trial (RCT) in the field. The RCT evidence tips their position of clinical equipoise and leads them to recommend a particular intervention to the patient. But was the RCT a robust study, with high standards of reporting, providing evidence that really is relevant to the patient? It all comes down to the quality of peer review of the clinical trial by the particular journal. Following on from a previous editorial covering general aspects of reviewing for the BJD,1 in this article I will focus on how to review a clinical trial in the context that publication of high-quality dermatology clinical trial evidence is one of the BJD's core missions. Most of my comments will relate to RCTs, because this is the methodology favoured by the BJD to minimize bias when assessing the beneficial and harmful effects of an intervention. The reviewer's job is made easier by the BJD requiring all RCTs to conform to CONSORT standards of reporting,2 including a CONSORT checklist and a flowchart of participants from screening to study end. The TIDieR checklist is an extension of item 5 of the CONSORT 2010 statement and is a useful guide for ensuring that trial interventions are accurately and fully reported.3 In addition, prospective trial registration in a clinical trial database will soon be a requirement of the BJD.4 Many clinical trials are already registered, and reviewers should access the electronic trial registration document using the unique trial identifier, if provided. It is important to ensure that the trial was indeed registered prospectively, rather than at a later date, and to check that the nature and timing of primary outcomes are declared. Trial databases can also be used to detect publication bias by identifying trials that were registered but never published and to check for ongoing trials in the field. When assessing the methods section of an RCT, it is important to consider all the main potential sources of bias, namely selection, performance, detection, attrition, reporting and other bias. Selection bias includes the method of randomization, where computer-generated randomization is considered robust but randomization based on the day of the week is not. Allocation concealment is also included here, an example being the use of nonopaque envelopes to conceal treatment assignment, which is open to abuse by well-meaning recruiters who might wish particular patients to receive the active treatment. Blinding affects performance bias, which considers whether all trial participants are looked after in the same way apart from the intervention itself. Adequate blinding can be preserved by identical placebos or sham treatments, and can be lost by differences in the adverse-effect profiles of the interventions under investigation. Detection bias, which is concerned with outcomes assessment, is also influenced by blinding. If adequate blinding is not possible, detection bias can be reduced by employing an independent outcomes assessor in each trial centre, or ensuring blinded assessment of disease severity from clinical photographs. Attrition bias becomes a potential issue when outcomes data are incomplete and an intention-to-treat analysis is not performed. Trials should employ a recognized method to deal with missing data, for example last observation carried forward or imputation, and ideally this should be prospectively declared in a published protocol or clinical trials database. Reporting bias is concerned with inclusion of all measured outcomes in the trial report, which can be checked from the clinical trial registration document if available, and inclusion of all primary outcomes in the abstract (Table 1). Any harmful effects of the intervention should be prominently reported. Other potential sources of bias include concomitant treatments or an imbalance in baseline participant characteristics between groups, the latter being minimized by stratified randomization. Critique of the results of a clinical trial should begin with consideration of the particular outcomes chosen. If there is consensus about the core outcome set for the condition, for example the HOME initiative for eczema,5, 6 have these outcomes been measured? Are the efficacy outcomes validated and likely to be relevant to patients and clinicians? Ideally, a minimal clinically important difference should be available for the primary efficacy outcome to help define treatment success or failure. The absolute effect size of the intervention compared with the control should be made clear, rather than providing percentage reductions that can mask clinically insignificant benefits. Effect size should be distinguished from the precision of the result, which in the case of a continuous variable demonstrates whether the mean outcome values for the study arms are likely to differ. Confidence intervals are preferred to P-values because these allow the reviewer and reader to assess the maximum and minimum difference in mean effect size between the intervention and control at the stated level of confidence. Statistics is an area that requires careful review because authors may make errors inadvertently, or use statistics to overemphasize treatment effects. The potential for bias can be mitigated by blinding of the study statistician to treatment allocation. In parallel-group RCTs, analysis of results should focus on the differences between study arms, rather than within one study arm before and after an intervention, which may reflect regression to the mean for an active disease.7 The BJD has a statistics editor to call upon if a reviewer is in any doubt about statistical issues. Finally, the reviewer should assess whether the study authors’ conclusions are congruent with their findings, and check that there is no extension of conclusions beyond the study findings. The discussion should include a comment about study limitations, particularly the degree of external generalizability of the findings. Returning to the dermatologists in the clinic weighing up an important therapeutic decision, they discover a number of RCTs on further interrogation of the literature. Not all are relevant to their patient and not all are of high quality. They weigh up the evidence, communicate this to their patient and agree on a course of action. The clinical encounter could end here, however an important element of practising evidence-based dermatology is to disseminate the results of evidence searches to help guide other clinicians and avoid repetition of effort. This is where incorporation of a critical appraisal session within the local journal club becomes very useful. Evidence searches can be presented in the journal club forum to assess whether the clinical problem and the evidence uncovered are of interest to a wider audience. The best presentations can be worked up as Critically Appraised Topics (CATs) to allow submission to the BJD's Putting Papers into Practice section. Publication of CATs ensures that the BJD remains at the forefront of supporting clinical decision making. For the dermatology trainee, CATs provide a good chance to practice critical appraisal skills and gain a BJD publication. Furthermore, because reviewing a clinical trial requires the same critical appraisal toolkit, authorship of a CAT is an ideal way for a trainee to put themselves forward as a BJD reviewer. The next tricky decision in the clinic could be converted into a very helpful opportunity! Phyllis Spuls, Jonathan Batchelor and Alex Anstey kindly reviewed a previous version of the manuscript. None declared.