Package: krrpm
KRRPM (Repository under construction)Kernel Ridge Regression Predictive MachineThis repository contains an implementation of a Kernel Ridge Regression
Predictive Machine, a non-parametric method for the probabilistic prediction of
continuous values.The implementation is available as a package that can be installed withpippip install krrpmKRRPMKRRPM[1] is a form of Conformal Predictive Distribution[2] which is a framework for
non-parametric prior-free probabilistic prediction.
The prediction for a test object is provided in the form of Cumulative
Distribution Function for the label of test objects.A key advantage is that, under minimal assumptions, the KRRPM producescalibratedpredictions, i.e. the probabilities assigned to events correspond
to the relative frequencies of those events, within statistical
fluctuation. The only assumption is that training data and test data bei.i.d., that is, independent and identically distributed.A gentle introduction to Conformal Predictive Distributions can be found in the
Tutorial at [https://cml.rhul.ac.uk/people/ptocca/HomePage/COPA2020___Tutorial_on_Predictive_Distributions.pdf]KRRPM ImplementationThe KRRPM.py file implements the Kernel Ridge Regression Predictive Machine.
The API it exposes has been kept as similar as possible to that ofscikit-learnregressors. This makes it interoperable with thescikit-learnframework, allowing in particular the use of its parameter optimization
facilities (e.g.GridSearchCV).An instance of the KRRPM regressor is first created with the constructorKRRPM(), which takes a scalaraas regularization parameter and a choice ofkernel.     The regression can be thenfit()on the training data.     The
predictions are obtained with thepredict()method; in keeping with thescikit-learnconventions, thepredict()method returns a vector with one
scalar (as opposed to a distribution) for each supplied test object. The value
corresponds to the median in the distribution computed for the test object. The
predictive distributions are in a data attribute (predicted_distributions) of
the regressor, which is populated when the predict() method is called. Each
predictive distribution (which corresponds in a sense to a Cumulative  Distribution Function, as mentioned
above) is expressed as a step-wise function specified by an array of 'n' floats.
The steps have all "height"1/nand the elements of the array specify their
location.The implementation usesO(n^2)memory andO(n^3)time.Some effort has been put into minimizing the use of memory.
As a reference, a training set of 80,000 objects was used successfully to train
a KRRPM model on a server with 132GB of RAM.KRRPM ExampleThe notebookKRRPM-Example.ipynbin this repository provides a complete example of the application of KRRPM.
It can be run by clicking on the badge below.AcknowledgementsThis implementation was developed while at the Centre for Reliable Machine Learning, Royal Holloway, University of London.
The author is grateful to AstraZeneca for the grant R10911 "Automated Chemical Synthesis".References[1]: "Conformal Predictive Distributions  with Kernels", V. Vovk et al., 2018 [https://link.springer.com/chapter/10.1007/978-3-319-99492-5_4][2]: "Nonparametric Predictive Distributions Based on Conformal Prediction", V. Vovk, 2019, [https://doi.org/10.1007/s10994-018-5755-8]