Package: mlrl-common
"MLRL-Common": Building-Blocks for Multi-label Rule Learning AlgorithmsThis software package provides common modules to be used by different types ofmulti-label rule learning (MLRL)algorithms that integrate with the popularscikit-learnmachine learning framework.The goal ofmulti-label classificationis the automatic assignment of sets of labels to individual data points, for example, the annotation of text documents with topics.The library serves as the basis for the implementation of the following rule learning algorithms:BOOMER (Gradient Boosted Multi-label Classification Rules): A state-of-the art algorithm that usesgradient boostingto learn an ensemble of rules that is built with respect to a given multivariate loss function.FunctionalitiesThis package follows a unified and modular framework for the implementation of different types of MLRL algorithms. In the following, we provide an overview of the individual modules an instantiation of the framework must implement.Rule InductionA module for rule induction that is responsible for the construction of individual rules. Currently, the following modules of this kind are implemented:A module forgreedy rule inductionthat conducts a top-down search, where rules are constructed by adding one condition after the other and adjusting its prediction accordingly.Rule induction based on abeam search, where a top-down search is conducted as described above. However, instead of focusing on the best solution at each step, the algorithm keeps track of a predefined number of promising solutions and picks the best one at the end.All of the above modules supportnumerical, ordinal, and nominal featuresand can handlemissing feature values. They can also be combined with methods forunsupervised feature binning, where training examples with similar features values are assigned to bins in order to reduce the training complexity. Moreover,multi-threadingcan be used to speed up training.Model AssemblageA module for the assemblage of a rule model that consists of several rules. Currently, the following strategies can be used for constructing a model:Sequential assemblage of rule models, where one rule is learned after the other.Sampling MethodsA wide variety of sampling methods, includingsampling with and without replacement, as well asstratified sampling techniques, is provided by this package. They can be used to learn new rules on a subset of the available training examples, features, or labels.(Label Space) StatisticsSo-called label space statistics serve as the basis for assessing the quality of potential rules and determining their predictions. The notion of the statistics heavily depend on the rule learning algorithm at hand. For this reason, no particular implementation is currently included in this package.Post-ProcessingPost-processing methods can be used to alter the predictions of a rule after it has been learned. Whether this is desirable or not heavily depends on the rule learning algorithm at hand. For this reason, no post-processing methods are currently provided by this package.Pruning MethodsRule pruning techniques can optionally be applied to a rule after its construction to improve its generalization to unseen data and prevent overfitting. The following pruning techniques are currently supported by this package:Incremental reduced error pruning (IREP)removes overly specific conditions from a rule if this results in an increase of predictive performance (measured on a holdout set of the training data).Stopping CriteriaOne or several stopping criteria can be used to decide whether additional rules should be added to a model or not. Currently, the following criteria are provided out-of-the-box:Asize-based stopping criterionthat ensures that a certain number of rules is not exceeded.Atime-based stopping criterionthat stops training as soon as a predefined runtime was exceeded.Pre-pruning (a.k.a. early stopping)aims at terminating the training process as soon as the performance of a model stagnates or declines (measured on a holdout set of the training data).Post-OptimizationPost-optimization methods can be employed to further improve the predictive performance of a model after it has been assembled. Currently, the following post-optimization techniques can be used:Sequential post-optimizationreconstructs each rule in a model in the context of the other rules.Post-pruningmay remove trailing rules from a model in this increases the models performance (as measured on a holdout set of the training data).Prediction algorithmA prediction algorithm is needed to derive predictions from the rules in a previously assembled model. As prediction methods heavily depend on the rule learning algorithm at hand, no implementation is provided by this package out-of-the-box. However, it defines interfaces for the prediction ofregression scores, binary predictions, or probability estimates.LicenseThis project is open source software licensed under the terms of theMIT license. We welcome contributions to the project to enhance its functionality and make it more accessible to a broader audience. A frequently updated list of contributors is availablehere.