Reviewer's report
Title:Validation of a risk prediction tool for coronary heart disease in middle-aged
women
Version:1Date:26 January 2015
Reviewer: Giovanni Veronesi
Reviewer's report:
In this paper, De Vito and colleagues approached an important topic such as
cardiovascular risk prediction in women. In particular, they aim to validate a risk
tool based on self-reported information on lifestyle as well as clinical risk factors,
using a population of middle-aged, mostly Caucasian and highly-educated
women. However, to this reviewer the manuscript in its present form suffers of a
number of weaknesses requiring Major Compulsory Revisions.
1. The study rationale is to validate a risk score incorporating information on diet
and physical activity in a population of middle aged women, possibly to be used
in the primary prevention setting. However, none of the food questionnaire items
was statistically associated with the CHD endpoint in this population, except for
alcohol intake. Physical activity was reducing the risk by 15%, as compared to a
45% from the consensus RR. So in this population CHD predictors essentially
reduced to the same list of RFs of any other existing score, blood lipids and
blood pressure in their self-reported version, plus family history of CHD (which is
however included for instance in the Raynold's risk score for women). We have
no information on how the c-statistic of 0.71 compares to the AUC of other more
widely used risk tools when used in this population, nor we have hints on the
clinical utility of using the YDR score in middle-aged women. All we know at the
end of the reading is that the AUC of the YDR in this population is 0.71, reducing
to 0.59 in women older than 60. All considered, the sentence on the "adequate
validity" of the YDR in the conclusions paragraph is quite overstated, to this
reviewer.
Major Compulsory Revisions required on the statistical analysis:
2. Model calibration is essential in a validation study (see for instance Collins et
al. BMC Medical Research Methodology 2014, 14:40), yet authors lack to provide
any calibration assessment for the YDR tool in this population, for instance by
comparing observed and expected number of events by YDR relative risk
categories. Given the logistic modelling, the Hosmer–Lemeshow test should be
appropriate.
Without information on calibration, even the c-statistic cannot be adequately
interpreted.
3. In addition, it is not clear how the YDR can estimate the 10-year risk of CHD
"compared to an average person of the same age and time". In table 1 and in the
formula at page 7, neither the prevalence nor the consensus RRs seems to be
age- or sex-dependent, except the RRs for the BMI and WC. If the YDR was
already taking age into account, then I would not expect the RRs in table 2 to
change once adjusting for age; nevertheless, the RRs for the last two YDR
relative risk category increased, potentially suggesting that the average age in
these two categories may be lower than in the reference group.
4. While authors acknowledged in the limitations the exclusion of women with
missing data, it would be helpful to have more information, such as for instance
mean age, number and rates of CHD events compared to the complete case
women.
Level of interest: An article of insufficient interest to warrant publication in a
scientific/medical journal
Quality of written English: Acceptable
Statistical review: Yes, and I have assessed the statistics in my report.
Declaration of competing interests:
I declare that I have no competing interests