Package: puncc
Puncc(short forPredictiveuncertaintycalibration andconformalization) is an open-source Python library. It seamlessly integrates a collection of state-of-the-art conformal prediction algorithms and associated techniques for diverse machine learning tasks, including regression, classification and anomaly detection.Puncccan be used with any predictive model to provide rigorous uncertainty estimations.
Under data exchangeability (ori.i.d), the generated prediction sets are guaranteed to cover the true outputs within a user-defined error $\alpha$.Documentation is availableonline.📚 Table of contents🐾 Installation📖 Documentation👨‍🎓 Tutorials🚀 QuickStart📚 Citation💻 Contributing🙏 Acknowledgments👨‍💻 Creators📝 License🐾 Installationpunccrequires a version of python higher than 3.8 and several libraries including Scikit-learn and Numpy. It is recommended to installpunccin a virtual environment to not mess with your system's dependencies.You can directly install the library using pip:pipinstallpuncc📖 DocumentationFor comprehensive documentation, we encourage you to visit theofficial documentation page.👨‍🎓 TutorialsWe highly recommand following the introduction tutorials to get familiar with the library and its API:Introduction tutorialAPI tutorialYou can also familiarize yourself with the architecture ofpunccto build more efficiently your own conformal prediction methods:Architecture overview🚀 QuickstartConformal prediction enables to transform point predictions into interval predictions with high probability of coverage. The figure below shows the result of applying the split conformal algorithm on a linear regressor.Many conformal prediction algorithms can easily be applied usingpuncc.  The code snippet below shows the example of split conformal prediction with a pretrained linear model:fromdeel.puncc.api.predictionimportBasePredictorfromdeel.puncc.regressionimportSplitCP# Load calibration and test data# ...# Pretrained regression model# trained_linear_model = ...# Wrap the model to enable interoperability with different ML librariestrained_predictor=BasePredictor(trained_linear_model)# Instanciate the split conformal wrapper for the linear model.# Train argument is set to False because we do not want to retrain the modelsplit_cp=SplitCP(trained_predictor,train=False)# With a calibration dataset, compute (and store) nonconformity scoressplit_cp.fit(X_calib=X_calib,y_calib=y_calib)# Obtain the model's point prediction y_pred and prediction interval# PI = [y_pred_lower, y_pred_upper] for a target coverage of 90% (1-alpha).y_pred,y_pred_lower,y_pred_upper=split_cp.predict(X_test,alpha=0.1)The library provides several metrics (deel.puncc.metrics) and plotting capabilities (deel.puncc.plotting) to evaluate and visualize the results of a conformal procedure. For a target error rate of $\alpha = 0.1$, the marginal coverage reached in this example on the test set is higher than $90$% (seeIntroduction tutorial):90% Prediction Interval with Split Conformal Prediction.More flexibility with the APIPunccprovides two ways of defining and using conformal prediction wrappers:A direct approach to run state-of-the-art conformal prediction procedures. This is what we used in the previous conformal regression example.Low-level API: a more flexible approach based of full customization of the prediction model, the choice of nonconformity scores and the split between fit and calibration datasets.A quick comparison of both approaches is provided in theAPI tutorialfor a regression problem.📚 CitationIf you use our library for your work, please cite our paper:@inproceedings{mendil2023puncc,
  title={PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization},
  author={Mendil, Mouhcine and Mossina, Luca and Vigouroux, David},
  booktitle={Conformal and Probabilistic Prediction with Applications},
  pages={582--601},
  year={2023},
  organization={PMLR}
}Puncchas been used to support the work presented in our COPA 2022 paper on conformal prediction for time series.@inproceedings{mendil2022robust,
  title={Robust Gas Demand Forecasting With Conformal Prediction},
  author={Mendil, Mouhcine and Mossina, Luca and Nabhan, Marc and Pasini, Kevin},
  booktitle={Conformal and Probabilistic Prediction with Applications},
  pages={169--187},
  year={2022},
  organization={PMLR}
}🙏 AcknowledgmentsThis project received funding from the French ”Investing for the Future – PIA3” program within the Artificial and Natural Intelligence Toulouse Institute (ANITI). The authors gratefully acknowledge the support of theDEELproject.👨‍💻 About the DevelopersPuncc's development team is a group of passionate scientists and engineers who are committed to developing a dependable and user-friendly open-source software. We are always looking for new contributors to this initiative. If you are interested in helping us develop puncc, please feel free to get involved.💻 ContributingContributions are welcome! Feel free to report an issue or open a pull
request. Take a look at our guidelineshere.🔑 LicenseThe package is released underMITlicense.