Issues in the design, conduct and reporting of clinical trials that impact on the quality of decision making

I executive summary Background Clinical trial investigators and research ethics committees are obliged to ensure the that clinical trials are methodologically sound, and that the results of those trials are made publicly available in an open and honest fashion that is not misleading and does not misrepresent the data. This will ensure that only the best research is conducted, and that clinical decision making is based on the best possible,evidence. Objective The objective of this thesis was to explore issues in the design, conduct and reporting of clinical trials that impact on the quality of decision making by research ethics committees, as well as health care providers and consumers. Methods Three studies were conducted, the first of which was a follow-up study of 103 randomised trials submitted to a human research ethics committee in central Sydney. Information in the trial protocol was compared with that reported in the trial publication, the intention being to identify discrepancies between the two documents, particularly in relation to the primary outcomes. The second study reported is a systematic review of published studies evaluating the impact of shared scientific or ethical review of multi-centre clinical trial protocols on the quality of clinical research and the research process. The third is a prospective cohort study oftrials submitted to a central committee for scientific review, and the impact that review had on the functioning and decision-making of human research ethics committees. Results Selective reporting of primary outcomes encompasses selection of which outcomes are reported (discrepancy in identity), how the outcome is defined (discrepancy in definition) and selection of the amount of information reported for an outcome (completeness of reporting). Selective reporting of the primary outcome existed in some form in a significant proportion of trial publications. 17% of outcomes declared as a primary in the protocol were not reported as the primary outcome in the publication, and 15% of outcomes declared as a primary in the publication were not declared as primary outcomes in the protocol. Lack of adequate outcome definition in protocols and publications meant it was not possible to assess discrepancy in definition. 76% of trials completely reported all of their comparisons. Statistically significant comparisons were more likely to be completely reported. Trials with a completely or partially documented sample size calculation in the protocol were significantly less likely to selectively report the primary outcome, and this variable may be a proxy measure for the quality of the trial based on the protocol. The quality of the trial based on information obtained from its protocol is a fundamental component in the decision-making process of an ethics committee to approve or reject an application. Evidence for this is reflected by the nature of the changes requested by ethics in committees as a condition of approval. There is insufficient evidence provided by the identifiable published studies to be able to determine whether centralised scientific or ethical review improves the quality of trials or the quality of decisions made by RECs. The prospective study revealed that the 22 ethics committees affiliated with NSW Health spend a considerable amount of time each year on scientific review: an estimated 2,315 hours equivalent to approximately 1.4 full-time equivalent positions. RECs found the information provided by the central committee useful, and believed that it reduced the overall time taken to consider the trial at their meeting and improved the level of confidence they had in their decision making. Discussion Reporting of clinical trials should be consistent with original trial design as outlined in the clinical trial protocol. In the follow-up study of randomised trials, incomplete reporting of the primary outcome was a particularly problem among trials with outcomes that were not conventionally significant statistically. This may compromise the abil ity to undertake unbiased systematic reviews of all relevant trials. The research also identified that there can be changes in one or more aspects of a primary outcome between the protocol and the publication, such as a change in the definition of a positive test, or a change in the time frame for measurement. These