# Modernising Receiver Operating Characteristic (ROC) Curves +
## 4 Results and Discussions
### Limitations  
In assessing potential limitations in the present study, known limitations in both approaches can be noted. Rasch logistic regression depends on the assumption that a single dimension underlies the explanatory variable determining classification task difficulty and there are standard tests for establishing the unidimensionality, such as an examination of clustering in principal component analyses (PCA), of the residuals of Rasch logistic regressions. It is conceivable that task difficulty at the highest hCG concentration levels belongs to a different dimension than that of determining classification difficulty at low concentrations, which warrants further investigation. In this work, we have tested two approaches (Section 3.4.2) to Rasch logistic regression of ROC curves where the probability of successful classification, \(P_{success}\), is calculated (i) for each device and concentration level individually and (ii) at each concentration level, es  
Figure 8: ROC curves with uncertainty intervals (uncertainty coverage factors, \(k=2\)) for device type 1 used for pregnancy testing with (**a**) classic ROC and (**b**) \(z_{ROC}\). timating the cumulative rates for TPR and TNR. These alternative calculations yielded different estimates of device ability, as plotted in Figure 3.  
The preliminary PCA results indicate for all four modes of calculation only small (less than 1%) unexplained variances. So-called construct alleys are another tool to assess the assumptions--such as scale sensitivity--behind regular Rasch logistic regressions [8]. Such effects may influence both calculations (Section 3.4.2) according to method (i)--that is, conventional Rasch regression--but arguably perhaps even more in method (ii), since the sums included in the ROC expressions for TPR and TNR cover not only scores for the device at the target value of interest, but also for all other target values. Potential limitations in the two methods of calculation, such as the multidimensionality in task difficulty estimation, could also influence estimates of device ability since the Rasch model (Equation (1)) solves for difficulty and ability conjointly. Such effects, which would indicate a breakdown in specific objectivity (Appendix A.2), might be revealed by inspecting plots of device ability with the two alternative methods, as shown in Figure 9. In Figure 9a, for TPR, the device abilities with a cumulative Rasch-based analysis (\(x\)-axis) are unable to resolve individual device abilities, while conventional Rasch methodology (\(y\)-axis) shows clear differences in ability between the two groups of devices. (Higher logit values correspond to higher device ability). In Figure 9b, for TNR, a corresponding separation of the two groups of devices is shown while there are fewer measurement uncertainties for the cumulative Rasch-based analysis. Interestingly, there are clearly two devices (no. 4 and 5) performing much better with the cumulative Rasch-based analysis of TNR (\(x\)-axis of Figure 9b) compared with a conventional Rasch methodology (\(y\)-axis of Figure 9b). This could be explained by this device having fewer data compared to the other devices (Table 1), both in terms of tested target values and the number of tests per target values. [PERSON] and [PERSON] [19] have emphasised that two parameters are key to explaining task difficulty: (i) the estimate of \(SL\) and (ii) the classification uncertainty, \(u\). As already presented, the fitting procedure (Section 4.2) used for task difficulty over the range \(6<x<\) SL (IU/L) in Figure 4 yields estimates of SL close to the diagnostic value. However, in contrast, the new \(z_{ROC}\) plots (Section 4.4) (as well as [PERSON]'s plots) reveal only approximate linear relations, with estimated \(SL\) further from the diagnostic value, when spanning the whole range of target concentration values \(6<x<60\) (IU/L), that is, beyond the specification limit, \(SL\), to match the range adopted when plotting traditional ROC curves.