NIST AI RMF Playbook
AI RMF Playbook > Manage > MANAGE 2.2 (part 3)

### Interpretability and Explainability Approaches

Chaofan Chen, Oscar Li, Chaofan Tao, et al. 2019. This Looks Like That: Deep Learning for Interpretable Image Recognition. arXiv:1806.10574. [URL](https://doi.org/10.48550/arXiv.1806.10574)

Cynthia Rudin. 2019. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. arXiv:1811.10154. [URL](https://doi.org/10.48550/arXiv.1811.10154)

Daniel W. Apley, Jingyu Zhu. 2019. Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models. arXiv:1612.08468. [URL](https://doi.org/10.48550/arXiv.1612.08468)

David A. Broniatowski. 2021. Psychological Foundations of Explainability and Interpretability in Artificial Intelligence. National Institute of Standards and Technology (NIST) IR 8367. National Institute of Standards and Technology, Gaithersburg, MD. [URL](https://doi.org/10.6028/NIST.IR.8367)

Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, et al. 2021. Manipulating and Measuring Model Interpretability. arXiv:1802.07810. [URL](https://doi.org/10.48550/arXiv.1802.07810)

Hongyu Yang, Cynthia Rudin, Margo Seltzer. 2017. Scalable Bayesian Rule Lists. arXiv:1602.08610. [URL](https://doi.org/10.48550/arXiv.1602.08610)

P. Jonathon Phillips, Carina A. Hahn, Peter C. Fontana, et al. 2021. Four Principles of Explainable Artificial Intelligence. National Institute of Standards and Technology (NIST) IR 8312. National Institute of Standards and Technology, Gaithersburg, MD. [URL](https://doi.org/10.6028/NIST.IR.8312)

Scott Lundberg, Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. arXiv:1705.07874. [URL](https://doi.org/10.48550/arXiv.1705.07874)

Susanne Gaube, Harini Suresh, Martina Raue, et al. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. npj Digital Medicine 4, Article 31 (2021). [URL](https://doi.org/10.1038/s41746-021-00385-9)

Yin Lou, Rich Caruana, Johannes Gehrke, et al. 2013. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD '13), August 2013. Association for Computing Machinery, New York, NY, USA, 623-631. [URL](https://doi.org/10.1145/2487575.2487579)

### Post-Decommission

Upol Ehsan, Ranjit Singh, Jacob Metcalf and Mark O. Riedl. "The Algorithmic Imprint." Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (2022). [URL] (https://arxiv.org/pdf/2206.03275v1)

### Privacy Resources

National Institute for Standards and Technology (NIST). 2022. Privacy Framework. [URL](https://www.nist.gov/privacy-framework)

### Data Governance

Marijn Janssen, Paul Brous, Elsa Estevez, Luis S. Barbosa, Tomasz Janowski, Data governance: Organizing data for trustworthy Artificial Intelligence, Government Information Quarterly, Volume 37, Issue 3, 2020, 101493, ISSN 0740-624X. [URL](https://doi.org/10.1016/j.giq.2020.101493)

### Software Resources

- [PiML](https://github.com/SelfExplainML/PiML-Toolbox) (explainable models, performance assessment)
- [Interpret](https://github.com/interpretml/interpret) (explainable models)
- [Iml](https://cran.r-project.org/web/packages/iml/index.html) (explainable models)
- [Drifter](https://github.com/ModelOriented/drifter) library (performance assessment)
- [Manifold](https://github.com/uber/manifold) library (performance assessment)
- [SALib](https://github.com/SALib/SALib) library (performance assessment)
- [What-If Tool](https://pair-code.github.io/what-if-tool/index.html#about) (performance assessment)
- [MLextend](http://rasbt.github.io/mlxtend/) (performance assessment)
- AI Fairness 360:
- [Python](https://github.com/Trusted-AI/AIF360) (bias testing and mitigation)
- [R](https://github.com/Trusted-AI/AIF360/tree/master/aif360/aif360-r) (bias testing and mitigation)
- [Adversarial-robustness-toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox) (ML security)
- [Robustness](https://github.com/MadryLab/robustness) (ML security)
- [tensorflow/privacy](https://github.com/tensorflow/privacy) (ML security)
- [NIST De-identification Tools](https://www.nist.gov/itl/applied-cybersecurity/privacy-engineering/collaboration-space/focus-areas/de-id/tools) (Privacy and ML security)
- [Dvc](https://dvc.org/) (MLops, deployment)
- [Gigantum](https://github.com/gigantum) (MLops, deployment)
- [Mlflow](https://mlflow.org/) (MLops, deployment)
- [Mlmd](https://github.com/google/ml-metadata) (MLops, deployment)
- [Modeldb](https://github.com/VertaAI/modeldb) (MLops, deployment)