Methodologic Issues in the Clinical Trial Study of Depression
Berger, D., Pharma Japan, 2083; 18-19. March 24, 2008.

A recent study by Kirsch et al. entitled, Initial Severity and AntidepressantBenefits: A Meta-Analysis of Data Submitted to the Food and Drug
Administration, published by the Public Library of Science (PLoS Med February26, 2008) caused a stir in the mass media who chimed in with sound bites like,“Antidepressants don’t work much better than placebos for many depressedpatients.” This study obtained data on all clinical trials submitted to the FDA forthe licensing of four new-generation antidepressants for which full datasets wereavailable (fluoxetine, venlafaxine, nefazodone, and paroxetine) and usedmeta-analytic techniques to assess linear and quadratic effects of initial severityon improvement scores for drug and placebo groups and on drug–placebo
difference scores.
While the study is a poignant reminder of the difficulty to prove statistical efficacy
for antidepressants, its message that antidepressants do not work much better
than placebo except for the most severely depressed patients will leave many
patients with significant depressive symptoms without proper medical care.
This is a summary of the major findings of this study:
1. Efficacy reaches clinical significance for only the most extremely
depressed patients, and this is due to a decrease in the response toplacebo rather than an increase in the response to medication.
2. A substantial response to placebo was seen in moderately depressedgroups and in groups with very severe levels of depression. It decreased
somewhat, but was still substantial, in groups with the most-severe levelsof depression.
3. Given these data, there seems little evidence to support the prescription ofantidepressant medication to any but the most severely depressedpatients.
How can we interpret these findings in light of our believing in the incredible
power of these drugs to help depression? The reason is that Clinical practice,especially for depression, is not practiced like a clinical trial, and that for anumber of methodologic reasons, the pharmacodynamics of depression is noteasy to study in a clinical trial. The result is that only modest improvement may
be seen in the active treatment, and the results of trials of some
antidepressants that later obtain market approval are negative.

First, let me enumerate some of the differences between a clinical trial andclinical practice:
1. There are few exclusion criteria, inclusion criteria are wider, co-morbid andcomplex patients are treated.
2. Flexible dosing and multiple medications are employed.
3. In clinical practice, one treats an individual patient, treatments are notbased solely on the results of a clinical trial. Many depressed patientsclearly remit when drug is given and relapse when drug is removed. Also,
it is common for patients to respond only to certain types of drugs, e.g.,
serotonin promoting drugs but not norepinephrine promoting drugs orvice-versa.
Next, we need to review why the study of efficacy of depression is so difficult:
1. Symptoms of depression are not an easily measurable and clearly reactive
pharmacodynamic parameter (The action of antidepressants on
depression are not as easily measurable as the action of reserpine on
blood pressure).
2. Depression is difficult to treat. Close to 50% of subjects, whether on activeor placebo will not have much response in a clinical trial. This shows thatplacebo itself is not exactly a favorable treatment for depression.
3. Depression assessment is subjective, and while there has been much
criticism of the use of the Hamilton Depression Rating Scale (HDRS) for aclinical trial, this scale is still seen by the regulatory agencies to be the sinequa non for depression assessment. The Montgomery-Asberg Depression
Rating Scale (MADRS) is though by many experts be an improvement overthe HDRS, however, the Kirsch et at. study did not analyze MADRS data,and in general depression rating scales may not be as insightful as a goodclinical interview.
4. Many patients will receive sleep medications even on placebo, and thismay improve some of the symptoms that are assessed making it seem likethe depression has also lifted.
5. Many symptoms rated as part of depression may improve in a therapeutic
environment with no drug (hopelessness, low self-esteem), and this may
cloud the issue of whether the depression itself had responded as many
patients with mild depression respond drug or not.
6. The sponsors of the study need to have some response endpoint that theycan measure within a 6-8 week time-frame of the study so that “responder”
is usually defined as a 50% reduction on the HDRS. So even if you are aresponder on placebo, you may still have significant depressive symptoms
left over. In other words, many subjects on both sides are “responders”. Ifthey have mild non-melancholic depression their depression is not really
being treated but they feel better. Even if they have major depression withmelancholia, the symptoms that can get better do, but the depression is notbetter; e.g., 50% reduction should not be considered response.

7. Entry of subjects with only mild symptoms of depression is one of thebiggest problems in a clinical trial (Fig 1.). Melancholic depression, or amorbid-type of depression is the most likely type of depression to respond
only to active medication. The study sites are motivated to rapidly enter as
many subjects as possible, thus many mild (non-melancholic) cases areentered. Non-melancholic patients have enough symptoms of depression
to score above the cut-off on the HDRS, but they do not have the loss ofappetite, worthlessness, and loss of ability to enjoy life like the melancholic
cases, and are more likely to have a number of symptoms that improve in atherapeutic environment under placebo treatment and be coded as a“responder”. Psychotic depression should also not be entered in the study
as they require an antipsychotic in addition to an antidepressant-this is
usually not a problem in a clinical trial.

Fig 1.
Spread of symptoms widens but isinversely proportional to number of patients.
Psychotic Depression
Melancholic Depression
Non-Melancholic Depression

Depressive
Mood Disorder Axis

8. Drug companies want to do the studies efficiently and at low cost and will
set the number of subjects at the lowest number needed based on previous
data in order to obtain the estimated statistical power. Because of themethodologic difficulties inherent in the nature of depression studies as
described here, this number may not be enough to get wide significance vsplacebo.
9. The meta-analysis assumes that all studies were of the same quality, that
all the protocols and subject numbers were adequate, that proper andconsistent orientation of the sites was comparable, etc. It is well known thatinter-site variability in the discrimination of subjects and in the quality of thestudy is often large in depression studies.
While the conclusion of the Kirsch study may not be valid for the real world ofdepression treatment, what can we take home from this study to help the
medical care and clinical trial construct for depression treatment?

First, we need to improve the methods for clinical trials of depression and theknowledge of these difficulties at the regulatory agency level. We need toshow the agencies what makes sense, work with the experts in the fields, andlarge Pharma needs to take the lead in creating sounder-trials.
Next, I think the response of the industry must be more than a sound bite thatsounds like, “sorry you are wrong” message. One large Pharma company, forexample, was reported to say that the authors of the study had “failed toacknowledge” the very positive benefits of SSRIs and their conclusions were“at odds with the very positive benefits seen in actual clinical practice”. I think
we must use these opportunities to educate the media and the consumer witha message that describes the complexities involved. For example,“Antidepressants have shown efficacy in depression in spite of the hurdles ofstudying this complex illness in a clinical trial-and we can provide you with thedetails, suicide rates decrease in societies in proportion to use ofantidepressants, and the medical community has studied and proven theability of antidepressants to greatly improve the quality of life of millions ofpersons with this debilitating illness”, is perhaps a more well-rounded initialcomment.
Finally, both the industry and the medical profession needs to make clear andconcise responses to these kinds of reports. There should be a response thatis written in clear wordage for the lay person, and submitted to the major
media organs. A technical response for the professional should also beprepared which should be submitted to a major medical journal. An initialsummary response should be placed on the internet on a new website
dedicated to this response so that is comes up on a search term (ie.,“antidepressants don’t work”), and the lay and professional responses placed
there later after publication. We should never just say, “of courseantidepressants work”, we must explain why. The patients who need the care,and the medical professionals and industry teams who put in the sweat todeliver this care deserve this much.

Doug Berger, M.D., Ph.D.
Japan Pharmaceutical Consultant
www.japanpsychiatrist.com/DrugDev.htm.