Package: approx-cp
Approximate full Conformal PredictionThis repository contains the Python implementation ofApproximating Full Conformal Prediction at Scale via Influence Functions.OverviewContentsThird-party softwareUsageTutorial NotebookExperimentsReferenceOverviewApproximate full Conformal Prediction (ACP) outputs a prediction set that contains the true label with at least a probability specified by the practicioner. In large datasets, ACP inherits the statistical power of the highly efficient full Conformal Prediction. The method works as a wrapper for any differentiable ML model.ContentsThis repository is organized as follows. In the foldersrc/acpyou can find the following modules:methods.pyPython implementation of the ACP algorithms.others.pyPython implementation of the comparing methods (SCP, APS, RAPS, CV+, JK+).wrapper.pyPython implementation of ACP as a wrapper for any differentiable PyTorch model. Seemodels.pyfor examples.models.pyExamples of models compatible withwrapper.py(e.g., logistic regression, neural network, convolutional neural network).experiments.pyPython file to run the experiments from the command line.models/Saved models.The foldersrc/third_party/contains additional third-party software.Third-party softwareWe include the following third-party packages for comparison with ACP:RAPSAPS, CV+, JK+UsageInstallationACP can be utilized as a fully-independentpippackage. You can download the framework by running the following command in the terminal:pipinstallapprox-cpIn order to use ACP in your own models, just include the following imports in your file:fromacp.wrapperimportACP_D,ACP_O#Deleted scheme (ACP_D) and ordinary scheme (ACP_O)Alternatively, you can clone this repo by running:gitclonehttps://github.com/cambridge-mlg/acpcdacpAnd install the ACP Python package in a customizable conda environment:condacreate-nmyenvpython=3.9
condaactivatemyenv
pipinstall--upgradepip
pipinstall-e.Now, just include the import:fromacp.wrapperimportACP_D,ACP_OConstructing prediction sets with ACPACP works as a wrapper for any PyTorch model with.fit()and.predict()methods. Once you instantiate your model, you can generate tight prediction sets that contain the true label with a specified probability. Here is an example with synthetic data:fromsklearn.datasetsimportmake_classification
fromsklearn.model_selectionimporttrain_test_split
fromacp.modelsimportNeuralNetwork
fromacp.wrapperimportACP_D

X,Y=make_classification(n_samples=1100,n_features=10,n_classes=2,n_clusters_per_class=1,n_informative=3,random_state=42)Xtrain,Xtest,Ytrain,Ytest=train_test_split(X,Y,test_size=100,random_state=42)model=NeuralNetwork(input_size=10,num_neurons=[20,10],out_size=2,seed=42,l2_reg=0.01)ACP=ACP_D(Xtrain,Ytrain,model,seed=42,verbose=True)sets=ACP.predict(Xtest,epsilon=0.1,out_file="results/test")ReferenceJ. Abad Martinez, U. Bhatt, A. Weller and G. Cherubin. Approximating Full Conformal Prediction at Scale via Influence Functions. Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence (AAAI), 2023.BiBTeX:@inproceedings{Abad2023ApproximatingFC,
  title={Approximating Full Conformal Prediction at Scale via Influence Functions},
  author={Javier Abad and Umang Bhatt and Adrian Weller and Giovanni Cherubin},
  booktitle={AAAI Conference on Artificial Intelligence},
  year={2023}
}