[20] Mohammad Ali Kadampur and Sulaiman Al Riyaee. Skin cancer detection: Applying a deep learning based model driven architecture in the cloud for classifying dermal cell images. Informatics in Medicine Unlocked, 18:100282, 2020. ISSN 2352-9148.
[21] Gavin Kerrigan, Padhraic Smyth, and Mark Steyers. Combining human predictions with model probabilities via confusion matrices and calibration. In NeurIPS, 2021.
[22] Donald E Knuth. Two notes on notation. The American Mathematical Monthly, 99(5):403-422, 1992.
[23] Oluwasanmi Koyejo, Nagarajan Natarajan, Pradeep Ravikumar, and Inderjit S. Dhillon. Consistent multilabel classification. In NeurIPS, pages 3321-3329, 2015.
[24] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[25] Maksim Lapin, Matthias Hein, and Bernt Schiele. Analysis and optimization of loss functions for multiclass, top-k, and multilabel classification. IEEE Trans. Pattern Anal. Mach. Intell., 40 (7):1533-1554, 2018.
[26] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. Focal loss for dense object detection. IEEE International Conference on Computer Vision (ICCV), 2017.
[27] Daniel Link, Bernd Hellingrath, and Jie Ling. A human-is-the-loop approach for semi-automated content moderation. In ISCRAM, 2016.
[28] David Madras, Toniann Pitassi, and Richard S. Zemel. Predict responsibly: Improving fairness and accuracy by learning to defer. In NeurIPS, 2018.
[29] Aditya Krishna Menon and Robert C. Williamson. Bayes-optimal scorers for bipartite ranking. In  $COLT$ , volume 35 of JMLR Workshop and Conference Proceedings, pages 68-106. JMLR.org, 2014. URL http://proceedings.mlr.press/v35/menon14.html.
[30] Hussein Mozannar and David A. Sontag. Consistent estimators for learning to defer to an expert. In ICML, 2020.
[31] Hussein Mozannar, Arvind Satyanarayan, and David A. Sontag. Teaching humans when to defer to a classifier via exemplars. In AAAI, 2022.
[32] Hussein Mozannar, Hunter Lang, Dennis Wei, Prasanna Sattigeri, Subhro Das, and David A. Sontag. Who should predict? exact algorithms for learning to defer to humans. CoRR, abs/2301.06197, 2023.
[33] Harikrishna Narasimhan, Wittawat Jitkrittum, Aditya Krishna Menon, Ankit Singh Rawat, and Sanjiv Kumar. Post-hoc estimators for learning to defer to an expert. In NeurIPS, 2022.
[34] Chenri Ni, Nontawat Charoenphakdee, Junya Honda, and Masashi Sugiyama. On the calibration of multiclass classification with rejection. In NeurIPS, 2019.
[35] Nastaran Okati, Abir De, and Manuel Gomez-Rodriguez. Differentiable learning under triage. In NeurIPS, 2021.
[36] Joshua C. Peterson, Ruairidh M. Battleday, Thomas L. Griffiths, and Olga Russakovsky. Human uncertainty makes classification more robust. In ICCV, pages 9616-9625, 2019.
[37] Bernardo Ávila Pires and Csaba Szepesvári. Multiclass classification calibration functions. CoRR, abs/1609.06385, 2016. URL http://arxiv.org/abs/1609.06385.
[38] Harish G. Ramaswamy and Shivani Agarwal. Convex calibration dimension for multiclass loss matrices. J. Mach. Learn. Res., 17:14:1-14:45, 2016.
[39] Harish G. Ramaswamy and Shivani Agarwal. Convex calibration dimension for multiclass loss matrices. J. Mach. Learn. Res., 17:14:1-14:45, 2016. URL http://jmlr.org/papers/v17/14-316.html.