Quality of randomized trials in solid organ transplantation.

T randomized controlled trial is regarded as the best level of evidence that we have in trying to decide on the value of a particular treatment or intervention or on the value of a particular diagnostic procedure. Systematic reviews and meta-analyses can provide an enhanced level of evidence, but these too depend on the validity of the randomized trials on which they are based. As part of the initial steps of establishing a library of randomized trials, systematic reviews and metaanalyses in solid organ transplantation for the transplant community, the Centre for Evidence in Transplantation will provide as a regular feature of the Journal a record of recent randomized trials and an assessment of the quality of the trial. A record of trials published in the first six months of 2004 appears in the following pages together with a brief description of the aim of the trial and an assessment of quality. Further details are on the Journal web site. The evaluation of the quality of randomized trials is subject to considerable debate. A number of scoring systems have been used to assess quality (1), all of which have defects, while others feel that the quality of a trial is best assessed by considering the important individual components that make a high quality trial without using a summary score. We do not pretend that the evaluation we have used in this initial analysis is perfect but it does indicate that a substantial number of randomized trials in organ transplantation leave much to be desired. Recently Fritsche et al. (2) have reviewed 63 reports of large multicenter trials of immunosuppression in transplantation published since 1990 and have found serious defects in study quality and the quality of outcome reporting. Thus the randomized trial on which we place so much weight may distort the results of systematic reviews and meta-analyses and so have limited value as a source of evidence for the transplant practitioner. The purpose of a randomized trial is to exclude any source of bias that would influence the analysis and interpretation of the outcome. In this respect there are a number of critical components of any trial and these include appropriate randomization of the experimental and control populations, adequate concealment of the randomization sequence, blinding of the investigators and patients to the intervention (not always possible) and the outcome assessment, details of the attrition rate in the trial arms, and an analysis of outcomes based on intention to treat. In this initial evaluation we have used the Jadad scale (a commonly used method) to which we have added two other key components, namely an adequate description of allocation concealment and an analysis based on intention to treat, for the absence of these two components can result in exaggerated treatment effects. We are currently evaluating methods of assessing the quality of outcome reporting in randomized trials in organ transplantation, not only of efficacy but also of safety, as obviously this too is important. Indeed recently Chan and Altman have noted that incomplete reporting of outcomes in published articles is common (3). However the value of outcome data is critically dependent on the quality of the trial methodology. In subsequent reports we will also attempt to give an assessment of the quality of outcome reporting. The Jadad scale is based on scoring for several of the items listed above and may produce a score from 0 to 5, with 5 being the top of the scale (4). A score of 3 or above is considered to represent a high quality trial but as this does not include allocation concealment or intention to treat analysis, a high quality trial on the Jadad scale could still be defective. However if we take these two other important components of good trial methodology into account as described above as well, then it is still apparent that a majority of trials reported in the first six months of 2004 were inadequate with respect to their methodology. What can be done to improve the design of trials and the quality of the reporting of trials? I would recommend the use of the CONSORT (Consolidated Standards of Reporting Trials) statement (5, 6) for this purpose which has been endorsed by many prominent major general journals including The Lancet, Annals of Internal Medicine and the Journal of the American Medical Association. CONSORT provides a check list and a flow diagram to ensure a standard way of reporting trials but attention to many of these details in the design of a trial will help to ensure that the final analysis and report of a trial will be adequate to provide a valued source of evidence for the transplant practitioner. It is intended that in the future instructions to authors in the Journal will include the recommendation that reports of randomized trials in organ transplantation follow the guidelines of reporting provided by the CONSORT statement.