A reply (1998)

Dr Atherton criticises the use of the diagnostic test statistics of sensitivity and specificity in our prospective evaluation of the whoosh test. I do not agree with his criticisms and I would like to provide further statistical evidence. Fluoroscopic imaging provides a gold standard against which to measure other tests for caudal epidural needle placement; statistically, the treatment of needle placement status is no different to that of ‘disease’ status more commonly associated with diagnostic tests. Here is a fuller analysis of the diagnostic test results concerned: For the study as a whole: prevalence (pre-test likelihood of correct needle placement) = 92%. For clinical impression: + ve predictive (post-test likelihood of incorrect needle placement) = 93%; − ve predictive value A (post-test likelihood of correct needle placement) = 78%; − ve predictive value B (post-test likelihood of correct needle placement despite − ve test) = 22% sensitivity (true positive rate) = 94%; specificity (true negative rate) = 20%; likelihood ratio (and 95% confidence interval) for + ve test = 1.18 (0.98–1.92); likelihood ratio (and 95% confidence interval) for − ve test = 0.29 (0.08–1.18). For whoosh test: + ve predictive (post-test likelihood of incorrect needle placement) = 96%; − ve predictive value A (post-test likelihood of correct needle placement) = 80%; − ve predictive value B (post-test likelihood of correct needle placement despite − ve test) = 20% sensitivity (true positive rate) = 80%; specificity (true negative rate) = 60%; likelihood ratio (and 95% confidence interval) for + ve test = 2.00 (1.15 to 4.78); likelihood ratio (and 95% confidence interval) for − vs test = 0.33 (0.19 to 0.68). It is perhaps more meaningful, in a clinical sense, to think in terms of odds in this situation. The post-test odds of correct needle placement equals the pre-test odds of correct needle placement multiplied by the likelihood ratio. You can consider the likelihood ratio as indicating the value of the test for increasing certainty about correct needle placement. The likelihood ratios for whoosh test suggest its superiority (in statistical terms) to the clinical impression. The confidence intervals for the likelihood ratios for clinical impression include unity and therefore one cannot make robust inference from them; ideally larger numbers would be sought. The greater variation here is explained by the greater subjectivity of clinical impression compared with the whoosh test. Analysis of the value of the whoosh test when indicated by clinical impression compared with routine use of the whoosh test would, as Dr Atherton suggests, be useful. This was not part of the current study design and therefore it would not have been appropriate to perform a retrospective subgroup analysis. Any future larger study of the whoosh test should aim to eliminate as much bias and confounding as possible in the study design if the real effect of whoosh test directed by clinical impression is to be resolved. For example, the intention to use a whoosh test could bias the clinical impression; therefore randomisation to whoosh, no whoosh or clinically indicated whoosh should happen after the clinical impression is recorded. Dr Atherton suggests working on a null hypothesis of no association between test and fluoroscopy results either in discrete paired comparisons of binomial proportions or in a stratified analysis of the contingency tables. One would hope that the ethics committee would not have allowed the study to proceed had the answer to this test of hypothesis not been determined in advance. The most challenging part of the diagnostic test evaluation is to weigh the clinical consequences of a false positive result against those of a false negative result. To consider these elements in separate statistical analyses with independent probabilities would be specious. The diagnostic test approach facilitates clinical interpretation and maintains statistical robustness.