Package: mlrl-boomer
Important links:Documentation|Issue Tracker|Changelog|Contributors|Code of ConductThis software package provides the official implementation ofBOOMER - an algorithm for learning gradient boosted multi-label classification rulesthat integrates with the popularscikit-learnmachine learning framework.The goal ofmulti-label classificationis the automatic assignment of sets of labels to individual data points, for example, the annotation of text documents with topics. The BOOMER algorithm usesgradient boostingto learn an ensemble of rules that is built with respect to a given multivariate loss function. To provide a versatile tool for different use cases, great emphasis is put on theefficiencyof the implementation. To ensure itsflexibility, it is designed in a modular fashion and can therefore easily be adjusted to different requirements.ReferencesThe algorithm was first published in the followingpaper. A preprint version is publicly availablehere.Michael Rapp, Eneldo Loza MencÃ­a, Johannes FÃ¼rnkranz Vu-Linh Nguyen and Eyke HÃ¼llermeier. Learning Gradient Boosted Multi-label Classification Rules. In: Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML-PKDD), 2020, Springer.If you use the algorithm in a scientific publication, we would appreciate citations to the mentioned paper. An overview of publications that are concerned with the BOOMER algorithm, together with information on how to cite them, can be found in the section"References"of the documentation.FunctionalitiesThe algorithm that is provided by this project currently supports the following core functionalities to learn an ensemble of boosted classification rules:Label-wise decomposable or non-decomposable loss functionscan be minimized in expectation.L1 and L2 regularizationcan be used.Single-label, partial, or complete headscan be used by rules, i.e., they can predict for an individual label, a subset of the available labels, or all labels. Predicting for multiple labels simultaneously enables rules to model local dependencies between labels.Various strategies for predicting regression scores, labels or probabilitiesare available.Isotonic regression models can be used to calibrate marginal and joint probabilitiespredicted by a model.Rules can be constructed via a greedy search or a beam search.The latter may help to improve the quality of individual rules.Sampling techniques and stratification methodscan be used to learn new rules on a subset of the available training examples, features, or labels.Shrinkage (a.k.a. the learning rate) can be adjustedto control the impact of individual rules on the overall ensemble.Fine-grained control over the specificity/generality of rulesis provided via hyper-parameters.Incremental reduced error pruningcan be used to remove overly specific conditions from rules and prevent overfitting.Post- and pre-pruning (a.k.a. early stopping)allows to determine the optimal number of rules to be included in an ensemble.Sequential post-optimizationmay help to improve the predictive performance of a model by reconstructing each rule in the context of the other rules.Native support for numerical, ordinal, and nominal featureseliminates the need for pre-processing techniques such as one-hot encoding.Handling of missing feature values, i.e., occurrences of NaN in the feature matrix, is implemented by the algorithm.Runtime and Memory OptimizationsIn addition, the following features that may speed up training or reduce the memory footprint are currently implemented:Unsupervised feature binningcan be used to speed up the evaluation of a rule's potential conditions when dealing with numerical features.Gradient-based label binning (GBLB)can be used to assign the available labels to a limited number of bins. This may speed up training significantly when minimizing a non-decomposable loss function using rules with partial or complete heads.Sparse feature matricescan be used for training and prediction. This may speed up training significantly on some data sets.Sparse label matricescan be used for training. This may reduce the memory footprint in case of large data sets.Sparse prediction matricescan be used to store predicted labels. This may reduce the memory footprint in case of large data sets.Sparse matrices for storing gradients and Hessianscan be used if supported by the loss function. This may speed up training significantly on data sets with many labels.Multi-threadingcan be used to parallelize the evaluation of a rule's potential refinements across several features, to update the gradients and Hessians of individual examples in parallel, or to obtain predictions for several examples in parallel.DocumentationAn extensive user guide, as well as an API documentation for developers, is available athttps://mlrl-boomer.readthedocs.io. If you are new to the project, you probably want to read about the following topics:Instructions forinstalling the software packageorbuilding the project from source.Examples of how touse the algorithmin your own Python code or how to use thecommand line API.An overview of availableparameters.A collection of benchmark datasets that are compatible with the algorithm are provided in a separaterepository.For an overview of changes and new features that have been included in past releases, please refer to thechangelog.LicenseThis project is open source software licensed under the terms of theMIT license. We welcome contributions to the project to enhance its functionality and make it more accessible to a broader audience. A frequently updated list of contributors is availablehere.All contributions to the project and discussions on theissue trackerare expected to follow thecode of conduct.