Choice of hormone replacement therapy in young women with ovarian failure

Sirs, Endocrinologists working at my hospital have recently drawn my attention to the article `Sustained reduction in circulating cholesterol in adult hypopituitary patients given lowdose titrated growth hormone replacement therapy: a two-year study' by Florakis et al. According to the Summary, this open uncontrolled study showed that `total and low density lipoprotein (LDL)-cholesterol levels were significantly lower at 6 months and were maintained throughout the study'. Furthermore, the authors say that `the decrease in total cholesterol was confined to patients with pretreatment total cholesterol levels above 5 ́8 mmol/l'; this result is emphasized in the first sentence of the Conclusions and also in the text, with Table 3 showing the pattern of mean total cholesterol values in four `cholesterol categories' at baseline, 6 and 24 months (end of the study). It is also commented on as being relevant in the Discussion. I was very surprised to read that a clear demonstration of the `regression to the mean' can be considered a scientific result. Regression to the mean is very well known to statisticians, who have dedicated a number of papers in the statistical literature to it and, more particularly, to attempts to obtain valid estimates of a change (before±after) even in its presence. It is also a subject that has been raised in journals aimed at a medical readership (Bland & Altman, 1994a, 1994b; Cole, 2000) and explained very clearly in a language that medical researchers must surely be able to understand (Bland & Altman, 1994a). For example, Bland & Altman say `people with extreme values of the measurement, such as high blood pressure (the previous sentence also includes weight and serum cholesterol concentrations), may be treated to bring their values closer to the mean. '1⁄4 and `If they are measured again we will observe that the mean of the extreme group is now closer to the mean of the whole population, that is reduced. This should not be interpreted as showing the effect of the treatment. ' There are also some other, less fundamental, points that can be criticized in the paper: X It is hardly possible to consider a 24-month result in 24 patients (only 26 ́7% of those enrolled at baseline) as relevant. The drop-out rate is too high. It needs to be remembered that data obtained from less than 80% of the starting population (as a very minimal threshold) have to be discarded as unreliable and biased by patient selection. X There is a wrong use of the standard error of the mean (SEM). The variability of recorded data is appropriately given by the standard deviation; readers have no interest in the variability of `all of the possible means obtained on samples with the same number of patients', i.e. the variability of the mean. This has been called a `definite error' (Altman, 1998). X In the statistical analysis paragraph, it is stated that `paired t-test or Wilcoxon's signed rank test' were used for both normally and non-normally distributed data and, rather surprisingly, one-factor repeated measures analysis of variance (ANOVA) was used to analyse the differences before and after 2 years' GH treatment. Do the authors think that non-abnormally distributed data became normally distributed when differences are considered, thus allowing the use of the analysis of variance? ; X The claimed `sustained reduction' (at 24 months) is based on a result obtained in 24 patients at a borderline significance level; however, this is much more than a subgroup analysis and so has to be considered biased by definition and consequently rejected. Furthermore, no data are shown in Table 3 concerning LDL cholesterol at 24 months and this is also disturbing when reading the paper. X After 6 months, the decrease in total cholesterol and LDL cholesterol is, respectively, 6 ́4% and 10 ́9%; at 24 months, the decrease in total cholesterol is 6 ́6%. Although statistically significant because of the relatively large number of patients, all of these decreases are of limited entity and it seems somewhat questionable to consider them relevant from the clinical point of view; of course, the well-known effect of GH on lipid metabolism is not under discussion. In conclusion, studies of `treatment effect' should be planned and carried out on the basis of sound clinical trial methodology (controlled by means of a randomization procedure, complete follow-up of patients, etc.) and, if uncontrolled studies have to be performed for ethical reasons (for example, hypopituitary patients need GH replacement therapy), more sophisticated statistical techniques have to be used (Davis, 1976; George et al., 1997). Doctors frequently analyse their data by copying the procedures they find in the literature if they have (or think they have) the same problems of data analysis without the necessary criticism. I strongly suggest submitting every article to a Statistical Consultant in order to reduce the use of incorrect statistical analysis and result reporting.