On a mission from GodFighting publication bias (2002)

As medical development accelerates and more and more methods are developed for diagnosing and following, and particularly for treating or preventing disease, all physicians are faced with an increasing and often bewildering multitude of different and partially conflicting methods for managing their patients. Commercial companies have legitimate interests in selling what they have developed and do what they can to retrieve their development costs through marketing of their new products. Not everything that is new is necessarily good, however, and even less is better than what we presently use. It is obviously necessary that all new drugs, and medical procedures and instruments be subjected to appropriate testing, so that their value or lack of value is assessed and compared to that of other available alternatives for treatment or diagnosis. New drugs are subjected to approval processes by regulatory agencies, but such processes are less strict – or in many countries nonexisting – for surgical and diagnostic procedures. While the drug approval process should provide at least a minimum test of safety and efficacy, this usually is not enough information to allow doctors to decide what role a new drug should assume relative to the other available alternatives. In approval processes surrogate endpoints must oftentimes be accepted. One example where a surrogate endpoint has been the rule is approval for new glaucoma drugs. Regulatory agencies have never required proof that new glaucoma drugs actually slow the progression of glaucoma or reduce the incidence of glaucoma damage in patients with ocular hypertension; instead it has always been sufficient to show that such drugs lower intraocular pressure. Furthermore, there may be more efficient alternatives, or alternatives that are preferable because of fewer side-effects or lower cost. Adding to this complexity are the facts that approved drugs may be adopted as standard treatments for indications that have not been approved, and that drug marketing may include claims that have not been evaluated in the approval process. Again, using glaucoma drugs as an example, most of these were certainly not initially approved for registration based on results showing improvements of ocular blood flow or oxygenation or neuroprotective properties, yet such effects may exist and regularly come up in discussions of the various therapeutic choices today. Modern evidence-based medicine is striving to provide a solid foundation for medical management by systematic reviews of all available results from the scientific literature in meta-analyses, where each study is given a weight according to the quality and the relevance of results for the problem at hand. Randomised trials provide the best evidence for systematic reviews and are thus given great weight. The need to be as complete as possible when assembling data for such analyses has rightly been stressed (Moher et al. 1996). Publication bias is the tendency on the parts of investigators, of reviewers, and of editors to submit or accept manuscripts on the basis on the direction or the strength of study results (Dickersin 1990). Publication bias first became known in the social sciences, but the concept has become increasingly recognised in medicine since the 1980s. Positive publication bias is the rule. The results of many studies are never published, but studies showing high efficacy are much more likely to be reported than those in which the observed efficacy is average or poor (Begg 1985). This results in an unbalance of published results – positive publishing bias. Thus an interesting retrospective study of 487 research projects that had been approved by the Central Oxford Research Ethics Committee, found that studies with statistically significant results were much more likely to be published than those that found no difference between study groups (the adjusted odds ratio was 2.32!), and that these differences were larger for observational and laboratory-based studies than for randomised trials (Easterbrook et al. 1991). Commercial interests in medicine are of course strong, and may drive the willingness or unwillingness to publish results. It has thus, e.g. been shown that pharmaceutical company sponsorship of economic analyses of new drugs has been associated with reduced likelihood of reporting unfavourable results (Friedberg et al. 1999). Publishing bias can be detrimental for conclusions drawn from the literature, and perhaps particularly so for systematic reviews. Therefore, in this era of evidence-based medicine publishing bias may be particularly dangerous, since meta-analyses cannot overcome the bias of missing evidence (Buffler 1989). Usually, it is not possible to obtain information on all clinical studies performed on a certain subject. The knowledge of the problems resulting from publication bias is now large and widespread, and failure to publish is considered inappropriate scientific conduct (Dickersin 1997). Selective reporting of clinical research results is described as not only unscientific but also unethical. Arguments against publishing negative results, such as that full-length reports devaluate the quality of literature, because the data are usually not so important or are of little interest to readers, are readily countered by the strong argument that such negative results provide at least some evidence and balance against the overwhelming power of positive data that are readily accepted for publication (Cleophas and Cleophas 1999). Researchers are also increasingly aware that nonreporting is a breach of the promise to patients, and that it is in fact unethical to take advantage of individual patients' willingness to participate in medical research while not reporting the results of the study. If the results are left unpublished, the risk and personal inconvenience of patients participating in the study become meaningless. It is therefore appropriate and desirable that different measures to fight this bias are seriously considered. One such measure is the recommendation by the International Committee of Medical Journal Editors to routinely request from authors a disclosure of their own and the sponsors' role in the study, and to sign a document that he or she has had access to the data, and controlled the decision to publish. Increasingly, requests also are being made that medical research ethics committees should verify whether approved studies are in fact conducted, and if so, that they are published. Publishing bias may also be reduced by legislation being passed, e.g. right now in Sweden, which makes ethics committee approval mandatory and increases representation by members of the surrounding community on such committees. Positive publishing bias also reduces medicine's ability to weed out old myths. Absent the weeding out process, we would still believe that earth is flat, that venesection helps against all diseases and that glaucoma is identical to elevated intraocular pressure. Publication of negative results must happen if we are to question and attack well-accepted but erroneous beliefs. Four papers with negative results are published in this issue of the journal, and they all make important contributions challenging previous views, hopes or published results. Friström reports a prospective study of colour contrast sensitivity developed in the 1980s, a test that has often been suggested as a tool for screening for early glaucoma. The colour contrast test did not reveal glaucomatous damage before conventional computerised static perimetry in patients with ocular hypertension, nor did it predict those patients who subsequently developed glaucoma during a five-year follow-up. Marchini and coworkers report an evaluation of the new Ocuton S home tonometer in healthy subjects and glaucoma patients. They found that results were correlated to those of the Goldmann tonometer, but that Ocuton tonometry constantly gave higher readings. The clinical consequences seem clear. Conclusions that patients often have elevated IOP levels in their homes, despite the fact that pressures are normal in the office are thus not supported if the Ocuton is used outside the office. An article by Bergstrand and coworkers is a randomised placebo-controlled trial of topical dorzolamide treatment on ocular blood flow in glaucoma. The material was unique in the sense that all 47 patients in the study were detected in a large population-based survey and were thus untreated before the study. Ocular blood flow was measured both with colour Doppler of retrobulbar vessels and scanning laser ophthalmoscope fluorescein angiograms of retinal vessels. This is the largest study to date on this subject, and it showed entirely negative results; dorzolamide gave no measurable effects. This is in contrast with several earlier studies, all smaller than the one published here, and may be particularly interesting today, when properties other than IOP lowering, i.e. changes of blood flow and neuroprotection, are so often discussed. Tests of stereo vision have been suggested as an alternative to visual acuity testing when screening children for amblyopia and strabismus. Visual acuity testing is difficult in children and gives large numbers of overreferrals. In this issue Ohlsson and coworkers report a study of the effectiveness of the Lang II stereo card for this purpose. Neither sensitivity nor specificity was acceptable. While this may be an undesired result, it certainly is informative and of direct clinical value. Acta Ophthalmologica is a nonprofit publication, legally a foundation and controlled by the five Nordic Ophthalmological Societies. It must be Acta's role to serve medical science, authors, and patients. We therefore invite properly conducted studies also if they have negative results, and maybe particularly if those results may be in conflict with current views. This will not mean that every negative study will be able to find its way into our columns, but here Acta will try to be ‘on a mission from God’, promoting balanced reporting by striving to avoid publication bias.