Package: ferret-xai
ferret is Python library that streamlines the use and benchmarking of interpretability techniques on Transformers models.Documentation:https://ferret.readthedocs.ioPaper:https://arxiv.org/abs/2208.01575Demo:https://huggingface.co/spaces/g8a9/ferretferretis meant to integrate seamlessly with 🤗transformersmodels, among which it currently supports text models only.
We provide:🔍 Four established interpretability techniques based onToken-level Feature Attribution. Use them to find the most relevant words to your model output quickly.⚖️ SixFaithfulness and Plausibility evaluation protocols. Benchmark any token-level explanation against these tests to guide your choice toward the most reliable explainer.📝 ExamplesAll around tutorial (to test all explainers, evaluation metrics, and interface with XAI datasets):ColabText ClassificationIntent Detection with Multilingual XLM RoBERTa:ColabGetting StartedInstallationpipinstall-Uferret-xaiOur main dependencies are 🤗tranformersanddatasets.ImportantSome of our dependencies might use the package name forscikit-learnand that breaks ferret installation.If your pip install command fails, try:SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=Truepipinstall-Uferret-xaiThis is hopefully a temporary situation!Explain & BenchmarkThe code below provides a minimal example to run all the feature-attribution explainers supported by ferret and benchmark them on faithfulness metrics.We start from a common text classification pipelinefromtransformersimportAutoModelForSequenceClassification,AutoTokenizerfromferretimportBenchmarkname="cardiffnlp/twitter-xlm-roberta-base-sentiment"model=AutoModelForSequenceClassification.from_pretrained(name)tokenizer=AutoTokenizer.from_pretrained(name)Usingferretis as simple as:bench=Benchmark(model,tokenizer)explanations=bench.explain("You look stunning!",target=1)evaluations=bench.evaluate_explanations(explanations,target=1)bench.show_evaluation_table(evaluations)Be sure to run the code in a Jupyter Notebook/Colab: the cell above will produce a nicely-formatted table to analyze the saliency maps.Featuresferretoffers apainlessintegration with Hugging Face models and
naming conventions. If you are already using thetransformerslibrary, you
immediately get access to ourExplanation and Evaluation API.Post-Hoc ExplainersGradient (plain gradients or multiplied by input token embeddings) (Simonyan et al., 2014)Integrated Gradient (plain gradients or multiplied by input token embeddings) (Sundararajan et al., 2017)SHAP (via Partition SHAP approximation of Shapley values) (Lundberg and Lee, 2017)LIME (Ribeiro et al., 2016)Evaluation MetricsFaithfulnessmeasures:AOPC Comprehensiveness (DeYoung et al.,
2020)AOPC Sufficiency (DeYoung et al.,
2020)Kendall's Tau correlation with Leave-One-Out token removal. (Jain
and Wallace, 2019)Plausibilitymeasures:Area-Under-Precision-Recall-Curve (soft score) (DeYoung et al., 2020)Token F1 (hard score) (DeYoung et al., 2020)Token Intersection Over Union (hard score) (DeYoung et al., 2020)See ourpaperfor details.VisualizationTheBenchmarkclass exposes easy-to-use table
visualization methods (e.g., within Jupyter Notebooks)bench=Benchmark(model,tokenizer)# Pretty-print feature attribution scores by all supported explainersexplanations=bench.explain("You look stunning!")bench.show_table(explanations)# Pretty-print all the supported evaluation metricsevaluations=bench.evaluate_explanations(explanations)bench.show_evaluation_table(evaluations)Dataset EvaluationsTheBenchmarkclass has a handy method to compute and
average our evaluation metrics across multiple samples from a dataset.importnumpyasnpbench=Benchmark(model,tokenizer)# Compute and average evaluation scores one of the supported datasetsamples=np.arange(20)hatexdata=bench.load_dataset("hatexplain")sample_evaluations=bench.evaluate_samples(hatexdata,samples)# Pretty-print the resultsbench.show_samples_evaluation_table(sample_evaluations)Planned DevelopementSeethe changelog filefor further
details.✅ GPU acceleartion support for inference (v0.4.0)✅ Batched Inference for internal methods's approximation steps (e.g., LIME or SHAP) (v0.4.0)⚙️ Simplified Task API to support NLI, Zero-Shot Text Classification, Language Modeling (branch).⚙️ Multi-sample explanation generation and evaluation⚙️ Support to explainers for seq2seq and autoregressive generation throughinseq.⚙️ New evaluation measure: Sensitivity, Stability (Yin et al.)⚙️ New evaluation measure: Area Under the Threshold-Performance Curve (AUC-TP) (Atanasova et al.)⚙️ New explainer: Sampling and Occlusion (SOC) (Jin et al., 2020)⚙️ New explainer: Discretized Integrated Gradient (DIG) (Sanyal and Ren, 2021)⚙️ New explainer: Value Zeroing (Mohebbi et al, 2023)⚙️ Support additional form of aggregation over embeddings' hidden dimension.AuthorsGiuseppe AttanasioEliana PastorDebora NozzaChiara Di BonaventuraCreditsThis package was created with Cookiecutter and theaudreyr/cookiecutter-pypackageproject template.Cookiecutter:https://github.com/audreyr/cookiecutteraudreyr/cookiecutter-pypackageLogo and graphical assets made byLuca Attanasio.If you are usingferretfor your work, please consider citing us!@inproceedings{attanasio-etal-2023-ferret,title="ferret: a Framework for Benchmarking Explainers on Transformers",author="Attanasio, Giuseppe and Pastor, Eliana and Di Bonaventura, Chiara and Nozza, Debora",booktitle="Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations",month=may,year="2023",publisher="Association for Computational Linguistics",}