# Best Practices for Applying and Interpreting the Total Operating Characteristic
## 1 Introduction  
The Total Operating Characteristic (TOC) measures the degree to which presence observations for a binary variable correspond to earlier-ranked observations for a rank variable [1]. The TOC has many applications in Geo-Information and beyond. TOC curves have become popular for comparing machine learning algorithms and remotely sensed indices that rank observations in terms of the predicted presence of the binary variable. A TOC curve plots the cumulative presence of the binary variable versus the ranks of the rank variable. Thus, the TOC curve shows a tremendous amount of potentially useful information. A threshold applied to any particular rank creates a point on the TOC curve. Each point reveals the sizes in the corresponding 2-by-2 confusion matrix's four entries: hits, false alarms, misses, and correct rejections. Those four entries are also known, respectively, as true positives, false positives, false negatives, and true negatives. Those four entries can generate a variety of potentially relevant metrics, depending on the research question. In this sense, the TOC gives the total information in each confusion matrix for each point on the TOC curve. The Relative Operating Characteristic (ROC) is the predecessor of the TOC [2; 3]. The computation of the TOC follows a logic similar to the logic of the ROC, meaning that both the TOC and ROC compare a binary variable to multiple thresholds of the rank variable. However, the ROC shows less information than the TOC. The TOC shows the size of binary presence and the size of the extent, which the ROC does not show. For each threshold, the ROC plots sensitivity versus 1 minus specificity, where sensitivity = hits/(hits + misses) and 1 minus specificity = false alarms/(false alarms + correct rejections). These two ratios do not reveal the sizes of the four entries. Some authors [4] have proposed plots of precision versus recall where precision = hits/(hits + false alarms) and recall = hits/(hits + misses), which also fail to reveal the sizes of the four entries that the TOC reveals. The TOC, ROC, and precision versus recall curves generate a metric called the Area Under the Curve (AUC), which integrates the strength of association across the plotted thresholds. The AUC is a unitless metric that ranges from zero to one. An AUC of zero indicates that absence is associated perfectly with early ranks; an AUC of a half is the association expected from random ranks; an AUC of one indicates that presence is associated perfectly with early ranks. The AUC can be remarkably misleading when interpreted without the context of additional types of information [5; 6; 7; 8]. Any single metric expresses a single aspect of the results, while TOC curves show many aspects of the results. Early adopters have used TOC curves to reveal helpful information that ROC curves would not have shown. However, some authors have missed important concepts concerning the computation, presentation, and interpretation of TOC curves. Therefore, Table 1 offers guidance on best practices for using the TOC. Many of these ideas apply also to the ROC. These best practices are ideas that authors should consider when using the TOC. Several of these best practices relate to when scientists use maps to calibrate a simulation model with an earlier time interval then predict change during a subsequent validation time interval. The list begins with the most important step, which is to specify the purpose, which can be tricky. An initial purpose might be to assess the classification accuracy, but upon further consideration, this purpose might be insufficiently precise. Several conversations might be necessary to develop a sufficiently precise purpose that a quantitative method can assess. Practices 2-4 allow readers to gain insight into the data quality and spatial