# Modernising Receiver Operating Characteristic (ROC) Curves +
###### Abstract  
The justification for making a measurement can be sought in asking what decisions are based on measurement, such as in assessing the compliance of a quality characteristic of an entity in relation to a specification limit, _SL_. The relative performance of testing devices and classification algorithms used in assessing compliance is often evaluated using the venerable and ever popular receiver operating characteristic (ROC). However, the ROC tool has potentially all the limitations of classic test theory (CTT) such as the non-linearity, effects of ordinality and confounding task difficulty and instrument ability. These limitations, inherent and often unacknowledged when using the ROC tool, are tackled here for the first time with a modernised approach combining measurement system analysis (MSA) and item response theory (IRT), using data from pregnancy testing as an example. The new method of assessing device ability from separate Rasch IRT regressions for each axis of ROC curves is found to perform significantly better, with correlation coefficients with traditional area-under-curve metrics of at least 0.92 which exceeds that of linearised ROC plots, such as [PERSON]'s, and is recommended to replace other approaches for device assessment. The resulting improved measurement quality of each ROC curve achieved with this original approach should enable more reliable decision-making in conformity assessment in many scenarios, including machine learning, where its use as a metric for assessing classification algorithms has become almost indispensable.  
os 2023 10 March 2023 9 May 2023 10 May 2023  
since limitations--an inherent non-linearity, the effects of ordinality [5] and a general confounding of task difficulty and instrument ability--are often unacknowledged, making ROC metrics for rating testing performance, such as area under curve (_AUC_), difficult and unreliable to interpret. This has much to do with the fact that ROC curves belong to classic test theory (CTT, Appendix A.1), whose limitations are becoming increasingly recognised in diverse fields such as psychometry, customer satisfaction studies and compositional data analysis [4]. As in those other fields, ROCs are expected to be prone to unnecessarily large uncertainties leading to substantial risks of incorrect decisions with potentially serious consequences in many fields of application.  
These limitations in the venerable ROC tool, inherent and often unacknowledged, are tackled here for the first time with a modernised approach combining measurement system analysis (MSA) and item response theory (IRT). The performance of the new method of assessing device ability will be compared with previous ROC methodology, and recommendations will be made regarding replacing other approaches for device assessment.  
The proposed methodology to modernise the ROC has general applicability. Examples include studies where the instrument-making classifications in support of diagnosis can either be a traditional engineering device (e.g., lateral flow assaying [6] such as pregnancy tests--as studied here--diabetes testing [7] and covid 19 screening tests [8]), or where a human acts as a measurement instrument [4], such as when the abilities of different clinicians to make correct diagnoses are to be assessed. ROC curves are also gaining popularity in modern technological areas such as machine learning, where their use as a metric for assessing machine learning algorithms has become almost indispensable [9].