Assessment of Fracture Risk: Population Association Versus Individual Prediction (2017)

Remember that the only constant in life is change —Gautama Buddha Assessment of fracture risk in most clinical settings is primarily based on a bone mineral density (BMD) measurement in combination with various risk factors. BMD is the most robust (and most reproducible) determinant of fracture risk.1 Several studies have estimated that for each standard deviation lower BMD there is a 1.5-fold to 2.5-fold increase in fracture risk2-4—a magnitude of association equivalent to the association between blood pressure and stroke.5 In an elegant analysis, Black and colleagues6 have reported that a single BMD measurement could predict fracture risk over 20 to 25 years. One could casually infer from this finding that repeat BMD measurement is not necessary for assessing fracture risk in asymptomatic individuals. However, among the sea of data, there are several subtle messages that deserve to be re-emphasized within the context of population versus individual prediction. It is worthwhile to consider some conceptual issues related to population-based association and individual-based prediction. The distinction between association and prediction is necessary for understanding a relationship,7 but this is not widely appreciated. Association is a population relationship, and the primary goal of association analysis is to identify factors that explain the relationship in a group of individuals. Prediction, on the other hand, is concerned with an individual's outcome; here, the goal is to find factors that forecast specific outcomes for an individual. Metrics for assessing an association are odds ratio, risk ratio, hazard ratio, and correlation coefficient. Statistical measures for assessing the performance of a predictor include area under the receiver operating characteristic curve (AUC, reflecting the ability of discrimination), deviation between observed and predicted values (ie, calibration), positive predictive value, and negative predictive value. Although a robust association can translate into better prediction, they are not synonymous, and we seek to clarify the distinctions of meaning, aim and metrics between association and prediction. Critically, a statistically significant association in a group of individuals does not necessarily translate into good prediction for an individual.8, 9 Statistical significance as indicated by common metrics such as p value and 95% confidence interval are not the same as prediction that is measured by calibration and discrimination. A risk factor may achieve statistical significance (eg, p < 0.05) when the sample size is large enough, but that same factor can be a poor predictor of future outcome. Moreover, a risk factor or a set of risk factors may be statistically significantly associated with an outcome because of their larger effect on a small number of events in the population, yet provide poor prediction for individuals in the population. For example, an association between baseline femoral neck BMD and hip fracture characterized by a hazard ratio of 1.8 (p < 0.0001) translates to an AUC of ∼0.67, well below the acceptable level of 0.80. This also translates to only 8% of the variation in risk being attributable to differences in BMD. To achieve the AUC of 0.80, the hazard ratio should be greater than four, and this has not been found in any large-scale epidemiologic studies. Still, even if a risk factor or a model achieves the AUC of 0.80, it may still be a poor individual predictor because it may underestimate or overestimate the actual risk for an individual. For a factor or group of factors to give good prediction, they should show good discrimination and good calibration. In the study by Black and colleagues6 in this issue of the JBMR, the average hazard ratio (per standard deviation lower in femoral neck BMD) for hip fracture was 2.6 over the period of 0 to 5 years, and progressively decreased to 1.8 over the period of 20 to 25 years. For nonvertebral, nonhip fractures, the hazard ratio remained at the level of 1.5 for the period of 0 to 25 years.6 Although the authors did not report the discriminatory value of baseline BMD, it can be inferred from simulation that that it was in the range of 0.6 to 0.7. These data might appear to suggest that the magnitude of association between femoral BMD and fracture risk does not substantially change with time. However, contrary to the implication of the title of the paper ‘The Ability of a Single BMD and Fracture History Assessment to Predict Fracture Over 25 Years in Postmenopausal Women’, these data do not necessarily suggest that a single measurement of BMD is adequate for discriminating or predicting the risk of fracture for an individual over the period of 25 years. Indeed, as Black and colleagues6 point out, additional clinical data and time-related risk factors, including BMD measurements, are needed for better fracture prediction. Predicting fracture for an individual is much harder than predicting the risk of fracture for a group of individuals. This is not just because of the random nature of fracture at the individual level, but more importantly, the time-dependent nature of predictors. Black and colleagues6 showed that the magnitude of association between BMD and fracture tended to be higher when BMD was measured close to the event than when BMD was measured distant prior to the event, suggesting the need for more BMD measurements and other clinical data for optimizing the risk assessment. BMD, just like other measured markers such as bone turnover markers and hormones, changes with age. In most postmenopausal women and older men, BMD declines with advancing age. Although the average rate of decline (less than 1% per year) gives a false impression that BMD does not substantially change, the individual age-related rates of bone loss are highly heterogeneous between individuals.10, 11 Although there is no consensus on the quantitative criteria for defining bone loss status, qualitatively some individuals lose BMD rapidly (so-called “fast losers”) and some lose BMD in a gradual fashion (“slow losers”). We12 and others13-18 have shown that excessive bone loss was a strong risk factor for fracture and fracture-related mortality, independent of baseline BMD. In general, each standard deviation increase in the rate of bone loss at the femoral neck is associated with a 40% increase in the risk of subsequent fracture.12 The rate of loss of BMD tends to be higher among women with low baseline BMD (even after accounting for the regression toward the mean phenomenon) than those with higher baseline BMD.19 Thus, an individual with baseline BMD close to the osteoporotic level (ie, T-score –2.5) has a greater risk of fracture than an individual with a baseline T-score of –1.5, not simply because the former has lower BMD, but because the former is likely to have a greater rate of bone loss than the latter. This study,6 together with previous studies,20, 21 renews an important question whether repeating BMD measurement is necessary for the assessment of fracture risk in asymptomatic individuals.19, 21-24 In contrast to a majority of studies,13-18 some studies20, 21 did not observe an incremental contribution of bone loss to fracture risk. If a single BMD measurement could predict fracture for an individual over the next 25 years, then there would be little justification for repeat BMD measurements. However, the evidence for the relationship between BMD and fracture risk so far is mainly concerned with association, and this is not strong enough for discounting the value of repeat BMD measurements. Indeed, for a measured baseline BMD measurement of an individual, the time to reach osteoporosis and/or fracture is dependent on gender, age, and current BMD measurement. For instance, we have shown that a 75-year-old man with a baseline BMD T-score of –1.5 (osteopenia), the time to reach 10% risk of osteoporosis or fracture is 2.7 years, but for an 80-year-old woman with BMD T-score of –2, the time to reach 10% risk of fracture or osteoporosis is 2 years.12 Therefore, the decision of whether to repeat BMD measurements cannot be made without some knowledge of an individual's initial risk related to age and other factors. These other risk parameters include history of prior fractures and falls, comorbidities, and medication use3, 4, 25 and estimates of bone biomechanical properties including trabecular bone score26 and bone strength analyses from finite element models,27 and microstructure of bone28 from quantitative computed tomography. At the heart of fracture risk assessment is the conflict between population and individual, and between association and prediction.29 The risk of fracture is derived from a group of individuals, but it is used as an intrinsic estimate for an individual. However, theoretically, each individual is likely to have a unique risk of fracture that is determined by the individual's profile of environmental and genetic risk factors.30 In reality, an individual's profile is dependent on the sample of risk factors considered; thus, the predicted risk for the individual is not necessarily unique. It should also be noted that the predicted probability of fracture for an individual is actually an average which can be viewed as the “wisdom of crowds” as postulated by Francis Galton.31 This delicate paradox explains why different predictive models, although each “valid” overall, can yield substantially different risks for an individual. We suggest that fracture risk assessment should move beyond association analysis and embrace prediction analysis. Instead of asking whether a factor is associated with fracture, we should ask whether the factor is predictive of fracture. Fracture is caused by a constellation of factors—many of which are likely interrelated and interactional—and current statistical models striving for elegance and parsimony are almost always inadequate to delineate their separate contributions or to capture their etiologic complexity. In prediction analysis, statistical models may be less elegant but they help identify potential highly predictive factors that are ignored by traditional association analysis. Machine learning approaches such as artificial neural network models can be very useful in fracture prediction.32, 33 From an association perspective, a single BMD measurement is statistically associated with fracture risk over two decades for a group of individuals, but from a prediction perspective, repeat BMD measurements (and other factors) are necessary for a more accurate prediction of fracture for an individual. To put the Buddha's maxim “the only constant in life is change” in perspective, the change in bone density—and for that matter, any risk factor—is more than a quantitative concept; it is a means to help us understand the clinical realm of osteoporosis. The authors have no conflict of interests regarding the publication of this article.