Website of the Week: Medical errors
Authors: Alex Williamson
Journal: BMJ (2003)

Imagine this scenario. A group of university researchers gets funding to study the causes of a well known human genetic disease (disease X). Disease X is fatal in a small percentage of cases, but there is a range of effective, but expensive, treatments for its symptoms.

After six months of the four year study many of the patients who have been tested seem to have the same rare combination of changes in a particular gene. The researchers also discover that some of the patients with this rare combination of genetic changes also had the same rare childhood illness (illness Y). Illness Y is fatal in about a half of cases, and it is not known how to cure or prevent it. Illness Y produces very painful symptoms, for which there is no effective treatment. It was not previously known that disease X and illness Y might be linked.

The researchers' initial conclusion is that the rare combination of genetic changes may be responsible for both disease X and illness Y. Any treatments for disease X, they think, may be effective against illness Y.

By coincidence, the research study is partially funded by the drug company that makes some of the treatments for disease X. The rest of the funding comes from a medical charity.

The researchers need to decide whether they should go public with their initial conclusion at this early stage of the study. They believe that if they make their preliminary conclusion public, some people will want the expensive treatments for disease X to be tested on children with illness Y.

The researchers consider these options:



Hold a press conference and announce their initial conclusion to journalists within a few days


Present their initial conclusion at a scientific conference within the next three months to gain the views of other researchers in the field and simultaneously to announce their initial conclusion to journalists


Prepare a scientific paper describing their initial conclusion and submit it to a journal to be assessed by other researchers in the same field and, if approved, published within 12 months and simultaneously announced to journalists.






Does this sound familiar? If you were one of these researchers, what would you do?

A Royal Society working party is using the above hypothetical scenario to assist in its consultation on best practice in communicating the results of scientific research to the public. Patrick Bateson, vice president of the Royal Society, chairs the working party, which also includes scientists, journal editors, publishers, journalists, and experts in consumer affairs. I agreed to join the working party because, as a medical publisher, I have first hand experience of the flaws in the peer review process. I have also seen the distress caused to patients when a new piece of research is misrepresented or exaggerated by the authors or by the lay media.​media.




Figure 1



Bateson: research has profound effects on public opinion

Credit: THE ROYAL SOCIETY




At a recent press conference Bateson said that the results of scientific research have profound effects on public opinion and policy, so it is important that scientists practise the highest standards of professionalism and integrity when communicating their results. At present, scientists rely almost exclusively on the practice of peer review, in which other experts check the quality of their results, analysis, and interpretation before they are made public. However, we know that peer review is not perfect, and some papers published in leading peer reviewed journals are afterwards found to be based on false or poorly interpreted results.

Two recent and well publicised retractions have fuelled anxieties about the credibility and quality of peer review. In September Science retracted a paper describing severe effects in monkeys that were given ecstasy. Two of the 10 animals died. The results shocked many and were seized on by the lay press in their campaigns against recreational drugs. It was then discovered that the monkeys had been mistakenly given methamphetamine, not ecstasy. In another case Nature Medicine retracted a paper that had reported favourable outcomes of the first human applications of a hybrid cell vaccine. The report had received much hype and raised the hopes of cancer patients. After publication, however, doubts were raised about the scientific rigour of the study.

Researchers themselves may raise the public's awareness of science and their anxieties by publicising questionable results that may or may not have been subjected to peer review. We are all only too familiar with the debate over the measles, mumps, and rubella vaccine stimulated by Andrew Wake-field's research and the claims about the birth of human clones.

The working party expects to report its findings to the council of the Royal Society early in the New Year. We hope that this will provide advice to researchers, journalists, and the wider public.