SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this paper, we characterize the learnability of a multilabel ranking hypothesis class in both the batch and online setting for a wide range of practical ranking losses. In all cases, we show that a ranking hypothesis class is learnable if and only if a sufficient number of its binary-valued threshold restrictions are learnable. Our paper studies two families of ranking loss functions and leaves it open to characterize the learnability of other natural ranking loss functions. One loss function not captured by our families is recall@p.
While we do establish quantitative bounds on the sample complexity and regret, our bounds are not optimal. It may be difficult to improve the sample complexity and regret bound at the highest level of generality for all losses in the families considered here. However, for natural losses such as sum loss, it is an interesting future direction to derive the optimal sample complexity and regret bounds in both the realizable and agnostic settings. In addition, our bounds depend on the number of labels  $K$ . Recently,  $K$ -free bounds have been achieved for multiclass classification problems in both batch and online settings [Brukhim et al., 2022; Hanneke et al., 2023]. An interesting future direction is to study whether  $K$ -free bounds are possible for multilabel ranking.
Finally, the focus of this paper is on characterizing learnability, and thus our algorithms are not computationally efficient. A natural future direction is to construct computationally efficient algorithms for multilabel ranking. Along this direction, since ERM is the most common algorithm used in practice, it is an important future direction to tightly quantify the sample complexity of ERM in the batch setting. Moreover, in learning theory, combinatorial dimensions play an important role in providing a tight quantitative characterization of learnability. Thus, it is an interesting future direction to identify combinatorial dimensions that characterize multilabel ranking learnability for specific loss functions.
We acknowledge the support of NSF via grant IIS-2007055. VR acknowledges the support of the NSF Graduate Research Fellowship.
Noga Alon, Amos Beimel, Shay Moran, and Uri Stemmer. Closure properties for private classification and online prediction. In Conference on Learning Theory, pages 119-152. PMLR, 2020.
Ivo M Baltruschat, Hannes Nickisch, Michael Grass, Tobias Knopp, and Axel Saalbach. Comparison of deep learning approaches for multi-label chest x-ray classification. Scientific reports, 9(1):1-10, 2019.
Young Hun Jung and Ambuj Tewari. Online boosting algorithms for multi-label ranking. In International Conference on Artificial Intelligence and Statistics, pages 279-287. PMLR, 2018.
Anna Korba, Alexandre Garcia, and Florence d'Alché Buc. A structured prediction approach for label ranking. Advances in Neural Information Processing Systems, 31, 2018.
Oluwasanmi O Koyejo, Nagarajan Natarajan, Pradeep K Ravikumar, and Inderjit S Dhillon. Consistent multilabel classification. Advances in Neural Information Processing Systems, 28, 2015.
Michel Ledoux and Michel Talagrand. Probability in Banach Spaces: isoperimetry and processes, volume 23. Springer Science & Business Media, 1991.
Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning, 2:285-318, 1987.
Tie-Yan Liu et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3):225-331, 2009.
Andrew Kachites McCallum. Multi-label text classification with a mixture model trained by em. In AAAI'99 workshop on text learning, 1999.
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari. Sequential complexities and uniform martingale laws of large numbers. Probability theory and related fields, 161:111-153, 2015.
Vinod Raman, Unique Subedi, and Ambuj Tewari. A characterization of multioutput learnability. arXiv.cs.LG, 2023. preprint arXiv:2303.17716.
Pradeep Ravikumar, Ambuj Tewari, and Eunho Yang. On ndcg consistency of listwise ranking methods. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 618-626. JMLR Workshop and Conference Proceedings, 2011.
Robert E Schapire and Yoram Singer. Boostexter: A boosting-based system for text categorization. Machine learning, 39:135-168, 2000.
Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, USA, 2014.
Leslie G. Valiant. A theory of the learnable. In Symposium on the Theory of Computing, 1984.
V. Vapnik and A. Chervonenkis. Theory of Pattern Recognition [in Russian]. 1974.
Xi Wang and Gita Sukthankar. Multi-label relational neighbor classification using social context features. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 464-472, 2013.
Hao Yang, Joey Tianyi Zhou, Yu Zhang, Bin-Bin Gao, Jianxin Wu, and Jianfei Cai. Exploit bounding box annotations for multi-label object recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 280-288, 2016.
Min-Ling Zhang and Zhi-Hua Zhou. A review on multi-label learning algorithms. IEEE transactions on knowledge and data engineering, 26(8):1819-1837, 2013.