# Modernising Receiver Operating Characteristic (ROC) Curves +
## 5 Conclusions  
In this paper, we have argued for and demonstrated the added value of a modernisation of the traditional ROC curve methodology, bringing it up to date with modern measurement theory. Inherent and often unacknowledged challenges implicit to the ROC tool as part of CTT, such as non-linearity, effects of ordinality, and confounding task difficulty and instrument ability, are met here with a novel approach combining measurement system analysis and item response theory (IRT), using data from pregnancy testing as an example.  
This methodology should be competitive with other alternatives, such as rankits (Appendix A), and also goes some way to meeting earlier criticism that separate information about TPR (sensitivity) and FPR (1--specificity, fall-out) was lost, and that no distinctions were made for differences in device-specific SL when plotting traditional ROC curves [10].  
We have re-examined and built on earlier studies of ROC curves for the assessment of device type classification performance using an approach combining MSA and IRT. The proposed modernised ROC goes some way to meeting the challenges posed by [PERSON] [12] in his original criticism of traditional ROC curves, including handling the non-linear % metrics TPR and FPR. The values of device ability from separate Rasch IRT regressions for each axis of ROC curves were found to perform significantly better and are recommended to replace other approaches for device assessment, with correlation coefficients with traditional area-under-curve metrics of at least 0.92, which exceeded that of linearised ROC plots, such as [PERSON]'s (Section 4.5). Measurement uncertainties in ROC curves were evaluated (Section 4.6), which also constitute an important part of assuring the measurement quality of each ROC curve.  
The limitations in the present study have been identified (Section 4.7). Future studies remain to be made and will be applied to other sets of data, including an extension of the univariate studies presented here to include multivariate decision risks and more complete data sets. We also recommend further investigations of the appropriateness of using a cumulative Rasch-based analysis in order to not diminish full specific objectivity (Appendix A.2).  
The new methodology should enable more reliable decision-making in conformity assessment in the presence of measurement uncertainty applicable to many decision-making scenarios, ranging from pregnancy testing (as studied here) to machine learning, where use of the ROC curve as a metric for assessing machine learning algorithms has become almost indispensable [9].  
Conceptualisation, L.R.P. and J.M.; methodology, L.R.P.; validation, J.M., A.S. and G.N.; formal analysis, L.R.P.; data curation, A.S. and G.N.; writing--original draft preparation, L.R.P.; writing--review and editing, J.M., A.S. and G.N.; visualisation, J.M. All authors have read and agreed to the published version of the manuscript.  
Part of the work reported has also been part of the 15 HLT04 NeuroMET and 18 HLT09 NeuroMET2 projects which received funding (2016-2022) from the EMPIR programme co-financed by the Participating States and from the European Union's Horizon 2020 research and innovation programme. Hence, we would like to express our great appreciation to collaborators and partners for our valuable and constructive work together.  
InCG data from proficiency testing has been kindly provided by: Equalis Uppsala (SE) and Noklus Bergen (NO).  
This work is based on learnings and outcomes in several projects during the past years, including the RISE internal platform for category-based measurements.  
The authors declare no conflict of interest.