Package: kditransform
kditransformThe kernel-density integral transformation(McCarter, 2023, TMLR), likemin-max scalingandquantile transformation, maps continuous features to the range[0, 1].
It achieves a happy balance between these two transforms, preserving the shape of the input distribution like min-max scaling, while nonlinearly attenuating the effect of outliers like quantile transformation.
It can also be used to discretize features, offering a data-driven alternative to univariate clustering orK-bins discretization.You can tune the interpolation $\alpha$ between 0 (quantile transform) and $\infty$ (min-max transform), but a good default is $\alpha=1$, which is equivalent to usingscipy.stats.gaussian_kde(bw_method=1). This is an easy way to improves performance for a lot of supervised learning problems. Seethis notebookfor example usage and thepaperfor a detailed description of the method.Accuracy on IrisrMSE on CA HousingInstallationInstallation from PyPIpip install kditransformInstallation from sourceAfter cloning this repo, install the dependencies on the command-line, then install kditransform:pip install -r requirements.txt
pip install -e .
pytestUsagekditransform.KDTransformeris a drop-in replacement forsklearn.preprocessing.QuantileTransformer. Whenalpha(defaults to 1.0) is small, our method behaves like the QuantileTransformer; whenalphais large, it behaves likesklearn.preprocessing.MinMaxScaler.import numpy as np
from kditransform import KDITransformer
X = np.random.uniform(size=(500, 1))
kdt = KDITransformer(alpha=1.)
Y = kdt.fit_transform(X)kditransform.KDIDiscretizeroffers an API based onsklearn.preprocessing.KBinsDiscretizer. It encodes each feature ordinally, similarly toKBinsDiscretizer(encode='ordinal').from kditransform import KDIDiscretizer
rng = np.random.default_rng(1)
x1 = rng.normal(1, 0.75, size=int(0.55*N))
x2 = rng.normal(4, 1, size=int(0.3*N))
x3 = rng.uniform(0, 20, size=int(0.15*N))
X = np.sort(np.r_[x1, x2, x3]).reshape(-1, 1)
kdd = KDIDiscretizer()
T = kdd.fit_transform(X)Initialized asKDIDiscretizer(enable_predict_proba=True), we can also output one-hot encodings and probabilistic one-hot encodings of single-feature input data.kdd = KDIDiscretizer(enable_predict_proba=True).fit(X)
P = kdd.predict(X)  # one-hot encoding
P = kdd.predict_proba(X)  # probabilistic one-hot encodingCiting this methodIf you use this tool, please cite KDITransform
using the following reference to ourTMLR paper:In Bibtex format:@article{mccarter2023the,title={The Kernel Density Integral Transformation},author={Calvin McCarter},journal={Transactions on Machine Learning Research},issn={2835-8856},year={2023},url={https://openreview.net/forum?id=6OEcDKZj5j},note={}}Usage with TabPFNTabPFNis a meta-learned Transformer model for tabular classification. In the TabPFN paper, features are preprocessed with the concatenation of z-scored & power-transformed features. After simplyadding KDITransform'ed features, I observedimprovementson the reported benchmarks. In particular, on the 30 test datasets in OpenML-CC18, mean AUC OVO increases from 0.8943 to 0.8950; on the subset of 18 numerical datasets in Table 1 of the TabPFN paper, mean AUC OVO increases from 0.9335 to 0.9344.