Authors’ response

would be treated as a high risk. A 1/11 chance of a cost of 10 for a false negative is balanced by a 10/11 chance of a cost of 1 for a false positive result. The likely cost is 10/11 of the cost of a false positive result in each case. It is possible to measure the performance of any model in units of the cost of one false positive result. By this criterion, SLRF performs poorly in both the training and the validation sets. In the training set the total cost is 1764 + 10 × 185 = 3614, outperformed by the poorly calibrated logistic regression model that they used as a benchmark: total cost 321 × 10 + 1 = 3211. In the validation set (prevalence 348=33190/, SLRF fails to identify 198 future homicides and misclassifies 2193 non-recidivists; total cost 2193+10×198=4173 times the cost of a false positive result. The simpler ‘method’ of treating everyone as negative, and letting them all go free, would give a substantially lower total cost of 3450 times the cost of a false positive conviction. However, I am not actually suggesting that as public policy. In this example, the cost ratio may be set too low. With a cost ratio of 15:1, the SLRF model would be as good as treating everyone as low risk; with one of 100:1, it would reduce costs by 38%. This sort of saving in costs to society would justify the positive reaction that has resulted from using the model. It would be interesting to compare the confusion diagrams for the logistic model, based on a cut point at 9.09%, with that from the preferred method of Berk and his colleagues. However, this would only be a fair comparison if every attempt was made to fit the best possible logistic regression model, as this was done with SLRF. In particular, I would be interested to see the use of a stepwise method of variable selection, based perhaps on maximizing the Akaike information criterion. Continuous and ordered variables (such as current age, age at first offence and number of offences) could be fitted by using ‘traffic light’ dummy variables, defined (for instance) as (current age 16), (current age 17), (current age 18) etc. With this system, the exclusion of a single dummy will have the effect of combining adjacent groups. In addition, first-order interaction terms might be used, as there are a relatively large number of cases. Akaike’s information criterion is implemented within STATA by the -swaiccommand (Wang, 2000; Wang and Steichen, 2005). However, if Akaike’s information criterion is not available, stepwise regression with a relatively modest inclusion criterion . p< 0:1/ might be used. Other approaches to logistic regression are certainly possible; these are simply the ones that I prefer. There are several advantages to a logistic regression approach: the software is generally available. The modelling is relatively fast and simple once the dummy variables have been defined. The model terms may reveal something about the causes of the bad outcome (whether recidivism or disease). The model can be implemented as a simple formula and made generally available in a standard spreadsheet. If there is a change in the relative importance of false positive and false negative results, it is easy to allow for this.