Prediction models in respiratory medicine

Clinicians need to be able to predict which patients will develop specific diseases or complications, and what outcomes are likely in the short or long term. The Framingham and other cohort prediction equations are widely used to predict the risk of coronary heart disease in the next 10 years. However, respiratory medicine does not yet have such widely accepted predictive models. For example, Pinart et al. conducted a systematic review of childhood asthma prediction models. The authors searched for studies of early respiratory symptoms in children of 4 years old or younger as a predictor of asthma between the ages of 6 and 12 years. They identified 14 predictive models mostly using family history, comorbidities or precursors of asthma and severity of early symptoms. Nine models also included clinical tests such as specific immunoglobulin E (IgE) sensitization. However, the predictive ability of all these models was quite poor and there was no single best performing model. Occupational asthma is one of the few preventable phenotypes of asthma in adults. High-risk industries routinely conduct medical examinations of prospective employees, typically including spirometry. Some authorities have advocated the measurement of nonspecific bronchial hyper-responsiveness (BHR) as a part of the pre-employment or pre-placement medical examination. We conducted an inception cohort study in the Australian aluminium industry including a modified Medical Research Council (MRC) questionnaire, spirometry and methacholine challenge test. Baseline BHR was weakly associated with incident chest tightness, but not wheeze or cough over a median of 4 years follow-up. Although the specificity of BHR was 70–77%, the sensitivity was only 24–39% for these asthma symptoms. We concluded that by itself, methacholine challenge was unlikely to be a useful pre-employment or pre-placement screening test in this industry. Another systematic review examined prediction models for the development of chronic obstructive pulmonary disease (COPD). Matheson et al. searched the literature to 2016, selected 30 articles for full-text review, but eventually incorporated only four studies. All of these models included smoking, the strongest known risk factor for COPD, and sex. Some of the models also included age, ethnicity, socio-economic status, height, weight, personal or family history of other lung diseases such as asthma or chronic bronchitis, early life factors, environmental pollution and biomarkers. Perhaps surprisingly, only one model included a measurement of lung function. Some of the models were able to discriminate between those correctly or incorrectly classified as COPD; however, none were particularly accurate at predicting future risk of COPD. Thus, clearly better predictive models are required to be clinically useful for common respiratory diseases such as asthma and COPD. How should the readers or authors of Respirology evaluate or write future publications on predictive models? Fortunately, the editors of 31 journals in respiratory, sleep and critical care medicine have recently published some helpful guidelines. Careful consideration needs to be given to selecting predictor variables, operationalizing these variables, specifying the outcome, dealing with missing data and validating the model, preferably in an external data set. The guidelines describe the metrics available to evaluate the performance of a model. Checklists are also available to ensure that all the key features of a prediction model are accurately reported (e.g. https://www.tripod-statement.org/). We have introduced the statistical considerations for prediction modelling previously in this journal. The new guidelines provide another lens with useful references, and a recent textbook edited by Riley et al. adds substantially to the relevant literature. A key point, highlighted in Figure 1, is that the overall model performance has two distinct aspects: calibration and discrimination. These feed into model utility along with considerations such as disease prevalence or incidence rates, cost-effectiveness of preventive or treatment regimens and so on. We take this opportunity to expand on a point raised by the guidelines, the issues of optimism-corrected internal validation and external validation. Internal validation of a new model is an essential step, confirming its predictive potential with the data source used to develop it. If data are precious, that is, the data source has a limited sample size, then cross-classification is an ideal approach to take for internal validation. The split-sample approach is also valid (if the split is random), but because of its requirement to quarantine a large proportion of the available data, it should only be