Pitfalls of pharmacoepidemiology
Authors: R. Farmer, T. Williams, A. Nightingale
Journal: BMJ (2000)

Editor—Skegg1 is concerned about the differences between our study2 and that of Jick et al.3 We share his concerns but are disappointed that he did not discuss the crucial differences between the studies and how these might affect the findings.

The general practice research database comprises data from general practitioners that are collated, quality controlled, and distributed by the Medicines Control Agency. We normally use all available data or exceptionally use well-defined subsets. Our recent study focused on practices that contributed data continuously between 1993 and 1998.2 Jick et al used an undefined subset.3 They asserted that “over the past 12 years we have removed over half of the original practices because of inadequate data quality” without defining the criteria used. We estimate that they have used at most 55% of the database. Studies by Jick et al are difficult to interpret because they use different subsets in different studies. Their paper on venous thromboembolism and combined oral contraceptives reported on two studies.4 The first used a population of 303 470 women and the second 238 470 women, and the reasons for this have never been disclosed. Their 1998 paper reporting on the changes in rates of pregnancies after the 1995 warning “reviewed the experience of all women.”5 Unlike national statistics, they found little change in the frequencies of pregnancies associated with the pill scare. They also reported “little change in the proportion of women who stopped using the pill,” which is inconsistent with sales figures of combined oral contraceptives.

Skegg is right to call for the Medicines Control Agency to conduct a thorough investigation. Some confusion could be resolved immediately. Either the database is quality controlled and all the data, or relevant well-defined subsets, should be used for all studies or the database is flawed and the substandard practices should be specified by the agency.

Jick et al misrepresent our identification of potential cases and the methods used to exclude those with an identifiable proximal cause.2 We searched the database for women with a record of a deep vein thrombosis or pulmonary embolism and for women who died while using combined oral contraceptives. All records of potential cases were reviewed individually—the same method used by Jick et al. Cases were excluded using the same criteria as Jick et al. All our cases had records of treatment with oral anticoagulants. We did not require cases to have a coded record admission as a criterion because we have shown that a significant proportion of the verified cases that had been admitted had no coded record of admission.6 Jick et al's claim that the coding of admission is 90% complete is not supported by the references they cite. Moreover, admission for deep vein thrombosis is not inevitable in the United Kingdom. It is possible that the very low incidence rates of venous thromboembolism that Jick et al report resulted from using admission as a primary search criterion and that some of the discrepancy between the studies is due to their underidentification of cases.7 We wrote “the methods and case identification are described elsewhere.” The “elsewhere” was referenced.8 We have never said that the cases were identified using an automated computer search and are astounded that Jick et al should misrepresent our methods.

The studies have other differences. Jick et al restricted their investigation to women aged 15 to 39 years, whereas 11% of our cases were aged 40 to 49 years. They restricted their study to women exposed to levonorgestrel, gestodene, or desogestrel combined oral contraceptives; 10% of our cases before the pill scare and 28.9% of those after were exposed to other combined oral contraceptives. Had we used the same constraints as they did, we would have discarded 27.3% of all the cases of venous thromboembolism. The Committee on Safety of Medicine's warning was neither limited to age nor “third generation” and levonorgestrel combined oral contraceptives.9

In the hypothetical calculation of the expected numbers of cases we assumed that third generation combined oral contraceptivess had twice the risk of the second generation ones. The absolute incidence rates were derived from period 1 and then applied to period 2 given the age and product specific utilisation during that period. We used 5 year age bands, not year of age and happily point out this correction.

We are criticised for inadequate control for confounding. In their cohort investigation Jick et al used similar methods to us; in both studies rates were adjusted by year of age. Had adjustment for factors such as body mass index and smoking reduced the post pill scare rates to a significant extent this would have meant that combined oral contraceptives were preferentially prescribed to higher risk women after the scare. The adjustment for factors other than age that Jick et al reported was from a case-control study. We did not publish the results of a case-control study.

We have confidence in our results; the rate of venous thromboembolism among users of combined oral contraceptives was unaffected by the dramatic change in patterns of use after the pill scare. Our findings are consistent with those of Goldacre et al with the Oxford morbidity and mortality database.10