Package: example-wise-f1-maximizer
Example-wise F1 MaximizerImportant links:Issue Tracker|Changelog|Code of ConductThis software package provides an implementation of a meta-learning algorithm for multi-label classification that aims to maximize the example-wise F1-measure. It integrates with the popularscikit-learnmachine learning framework and can also be used with frameworks for multi-label classification likescikit-multilearn.The goal ofmulti-label classificationis the automatic assignment of sets of labels to individual data points, for example, the annotation of text documents with topics. The example-wiseF1-measureis a particularly relevant evaluation measure for this kind of predictions, as it requires a classifier to achieve a good balance between labels predicted as relevant or irrelevant for an example, i.e., it must neither be to conservative nor to aggressive when it comes to predicting labels as relevant.MethodologyThe algorithm implemented by this project transforms an original multi-label problem withnlabels into a series ofn * n + 1binary classification problems. A probabilistic base estimator is then fit to each of these independent sub-problems as described in the followingpaper:Mingyuan Zhan, Harish G. Ramaswamy, and Shivani Agarwal. Convex Calibrated Surrogates for the Multi-Label F-Measure. In: Proceedings of the International Conference on Machine Learning (ICML), 2020.The probabilities predicted by the individual base estimators for unseen examples consitute an x nprobability matrixp, as well as an additional probabilityp_0. Whereasp_0corresponds to the prior probability of the null vector, i.e., a label vector that does not contain any relevant labels, each probabilityp_ikat thei-th row andk-th column ofpcorresponds to the conditional probability of a label vector withkrelevant labels, where thei-th label is relevant. In order to identify the label vector that maximizes the F1-measure in expectation, these probabilities are used as inputs to the "General F-Measure maximizer" (GFM), as proposed in the followingpaper:Krzysztof Dembczyński, Willem Waegeman, Weiwei Cheng, and Eyke Hüllermeier. An Exact Algorithm for F-Measure Maximization. In: Advances in Neural Information Processing Systems, 2011.Please note that this implementation has not been written by any of the authors shown above.DocumentationInstallationThe software package is available atPiPyand can easily be installed via PIP using the following command:pip install example-wise-f1-maximizerUsageTo use the classifier in your own Python code, you need to import the classExampleWiseF1Maximizer. It can be instantiated and used as shown below:fromexample_wise_f1_maximizerimportExampleWiseF1Maximizerfromsklearn.linear_modelimportLogisticRegressionclf=ExampleWiseF1Maximizer(estimator=LogisticRegression())x=[[1,2,3],# Two training examples with three features[11,12,13]]y=[[1,0],# Ground truth labels of each training example[0,1]]clf.fit(x,y)pred=clf.predict(x)The fit method accepts two inputs,xandy:A two-dimensional feature matrixx, where each row corresponds to a training example and each column corresponds to a particular feature.A two-dimensional binary label matrixy, where each row corresponds to a training examples and each column corresponds to a label. If an element in the matrix is unlike zero, it indicates that respective label is relevant to an example. Elements that are equal to zero denote irrevant labels.Both,xandy, are expected to benumpy arraysor equivalentarray-likedata types. In particular, the use ofscipy sparse matricesis supported.In the previous example, logistic regression as implemented by the classLogisticRegressionfrom the scikit-learn framework is used as a base estimator. Alternatively, you can use any probabilistic estimator for binary classification that is compatible with the scikit-learn framework and implements thepredict_probafunction.LicenseThis project is open source software licensed under the terms of theMIT license. We welcome contributions to the project to enhance its functionality and make it more accessible to a broader audience.All contributions to the project and discussions on theissue trackerare expected to follow thecode of conduct.