NIST AI RMF Playbook
AI RMF Playbook > Manage > MANAGE 2.2 (part 1)

MANAGE 2.2: Mechanisms are in place and applied to sustain the value of deployed AI systems.

About:
System performance and trustworthiness may evolve and shift over time, once an AI system is deployed and put into operation. This phenomenon, generally known as drift, can degrade the value of the AI system to the organization and increase the likelihood of negative impacts. Regular monitoring of AI systems' performance and trustworthiness enhances organizations' ability to detect and respond to drift, and thus sustain an AI system's value once deployed. Processes and mechanisms for regular monitoring address system functionality and behavior - as well as impacts and alignment with the values and norms within the specific context of use. For example, considerations regarding impacts on personal or public safety or privacy may include limiting high speeds when operating autonomous vehicles or restricting illicit content recommendations for minors.

Regular monitoring activities can enable organizations to systematically and proactively identify emergent risks and respond according to established protocols and metrics. Options for organizational responses include 1) avoiding the risk, 2)accepting the risk, 3) mitigating the risk, or 4) transferring the risk. Each of these actions require planning and resources. Organizations are encouraged to establish risk management protocols with consideration of the trustworthiness characteristics, the deployment context, and real world impacts.

Suggested Actions:
- Establish risk controls considering trustworthiness characteristics, including:
- Data management, quality, and privacy (e.g. minimization, rectification or deletion requests) controls as part of organizational data governance policies.
- Machine learning and end-point security countermeasures (e.g., robust models, differential privacy, authentication, throttling).
- Business rules that augment, limit or restrict AI system outputs within certain contexts
- Utilizing domain expertise related to deployment context for continuous improvement and TEVV across the AI lifecycle.
- Development and regular tracking of human-AI teaming configurations.
- Model assessment and test, evaluation, validation and verification (TEVV) protocols.
- Use of standardized documentation and transparency mechanisms.
- Software quality assurance practices across AI lifecycle.
- Mechanisms to explore system limitations and avoid past failed designs or deployments.
- Establish mechanisms to capture feedback from system end users and potentially impacted groups while system is in deployment.
-Establish mechanisms to capture feedback from system end users and potentially impacted groups about how changes in system deployment (e.g., introducing new technology, decommissioning algorithms and models, adapting system, model or algorithm) may create negative impacts that are not visible along the AI lifecycle.
- Review insurance policies, warranties, or contracts for legal or oversight requirements for risk transfer procedures.
- Document risk tolerance decisions and risk acceptance procedures.

Transparency and Documentation:
### Organizations can document the following

- To what extent can users or parties affected by the outputs of the AI system test the AI system and provide feedback?
- Could the AI system expose people to harm or negative impacts? What was done to mitigate or reduce the potential for harm?
- How will the accountable human(s) address changes in accuracy and precision due to either an adversary's attempts to disrupt the AI or unrelated changes in the operational or business environment?

### AI Transparency Resources

- GAO-21-519SP - Artificial Intelligence: An Accountability Framework for Federal Agencies & Other Entities. [URL](https://www.gao.gov/products/gao-21-519sp)
- Artificial Intelligence Ethics Framework For The Intelligence Community. [URL](https://www.intelligence.gov/artificial-intelligence-ethics-framework-for-the-intelligence-community)

References:
### Safety, Validity and Reliability Risk Management Approaches and Resources

AI Incident Database. 2022. AI Incident Database. [URL](https://incidentdatabase.ai/)

AIAAIC Repository. 2022. AI, algorithmic and automation incidents collected, dissected, examined, and divulged. [URL](https://www.aiaaic.org/aiaaic-repository)

Alexander D'Amour, Katherine Heller, Dan Moldovan, et al. 2020. Underspecification Presents Challenges for Credibility in Modern Machine Learning. arXiv:2011.03395. [URL](https://arxiv.org/abs/2011.03395)

Andrew L. Beam, Arjun K. Manrai, Marzyeh Ghassemi. 2020. Challenges to the Reproducibility of Machine Learning Models in Health Care. Jama 323, 4 (January 6, 2020), 305-306. [URL](https://doi.org/10.1001/jama.2019.20866)

Anthony M. Barrett, Dan Hendrycks, Jessica Newman et al. 2022. Actionable Guidance for High-Consequence AI Risk Management: Towards Standards Addressing AI Catastrophic Risks. arXiv:2206.08966. [URL](https://doi.org/10.48550/arXiv.2206.08966)

Debugging Machine Learning Models, In Proceedings of ICLR 2019 Workshop, May 6, 2019, New Orleans, Louisiana. [URL](https://debug-ml-iclr2019.github.io/)

Jessie J. Smith, Saleema Amershi, Solon Barocas, et al. 2022. REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. arXiv:2205.08363. [URL](https://arxiv.org/abs/2205.08363)

Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, et al. 2020. Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program) arXiv:2003.12206. [URL](https://doi.org/10.48550/arXiv.2003.12206)

Kirstie Whitaker. 2017. Showing your working: a how to guide to reproducible research. (August 2017). [LINK](https://github.com/WhitakerLab/ReproducibleResearch/blob/master/PRESENTATIONS/Whitaker_ICON_August2017.pdf), [URL](https://doi.org/10.6084/m9.figshare.4244996.v2)

Netflix. Chaos Monkey. [URL](https://netflix.github.io/chaosmonkey/)