Individual risk prediction using data beyond the medical clinic.

© 2018 Joule Inc. or its licensors CMAJ | AUGUST 13, 2018 | VOLUME 190 | ISSUE 32 E947 A t their cores, the disciplines of public health and clinical medicine are both based on decisions. A rational approach to decision-making relies on weighing the predicted benefits and risks of one alternative versus those of others. Accordingly, the quest for accurate predictions has consumed considerable effort for generations of researchers. Although the general framework for medical decision-making is decades old and many robust prediction models exist, we are entering an era in which the amount, quality and latency of data have changed dramatically. And as more data become available faster, many more analysts are crafting predictions pertinent to individuals and populations. In linked research, Manuel and colleagues advance the field by showing that information from population surveys can produce estimates of cardiovascular disease risk of comparable accuracy to estimates created from data collected within the health care system using more traditional medical risk measures.1 This approach offers much less expensive data collection and is more easily repeatable than detailed measures that require a physical visit to a clinic and often an evaluation by a specialist. Although in-depth symptoms, physical examinations and specialized medical tests would likely refine such predictions, the survey-based model is efficient and could be used to better rationalize the ordering of medical tests. Manuel and colleagues’ model particularly used smoking history in a more powerful way than most previous models have done. One downside of the approach is its oversimplification of measures that require physical assessment and are continuous, such as information about blood pressure; however, the authors should be congratulated on their exceptionally thorough and transparent analysis using state-of-the-art statistical modelling. An important element of the linked research is an in-depth analy sis of the operating characteristics of the predictive model. Although appropriate methods for characterizing predictions are well-described,2 many published models fail to consider essential elements. At a basic level, those developing a predictive model should define its ability to discriminate between people with and without the outcome of interest, and its calibration with the actual event proportions in the relevant population. A major strength of the present model is that the researchers were able to assess the calibration within 205 subpopulations that would be of interest for decision-making. In fields with an advanced understanding of decision thresholds, further information about whether the information is useful in moving beyond a decision point is also advantageous. Knowing about a change in risk when nothing would be done differently may not be useful to patients or clinicians. The only notable fault we find with the authors’ analysis is that they, like many others, used percentiles of risk estimates in some characterizations of model performance. Grouping participants into percentiles would be appropriate in settings in which groups influence individual outcomes, but, in general, “percentiling” and the variable heterogeneity of absolute risk within intervals it entails obscures the fact that an individual’s risk is a function of her own characteristics and not of some group to which she may belong (i.e., interval boundaries are functions of how many participants have similar predicted risks). Continuous calibration curves showing absolute accuracy over the entire risk spectrum without subgrouping provide a better way of understanding the characteristics of a model. As we look forward, we must recognize the changes now occurring in the health information environment. Electronic health records COMMENTARY