# Planning for AI governance #

Plan how to use watsonx\.governance to accelerate responsible, transparent, and explainable AI workflows with an AI governance solution that provides end\-to\-end monitoring for machine learning and generative AI models\.

## Governance capabilities ##

Note:To govern metadata from foundation models, you must have watsonx\.ai provisioned\.

Consider these watsonx\.governance capabilities as you plan your governance strategy:

<!-- <ul> -->

 *  Collect metadata in factsheets about machine learning models and prompt templates for large language models\.
 *  Customize the metadata facts that are captured in factsheets for machine learning and foundation models\.
 *  Monitor machine learning deployments for fairness, drift, and quality to ensure that your models are meeting specified standards\.
 *  Monitor foundation models for breaches of toxic language thresholds or detection of personal identifiable information\.
 *  Evaluate prompt templates with metrics designed to measure performance and to test for the presence of prohibited content, such as hateful speech\.
 *  Collect model health data including data size, latency, and throughput to help you assess performance issues and manage resource consumption\.
 *  Assign a single risk score to tracked models to indicate the relative impact of the associated model\. For example, a model that predicts sensitive information such as a credit score might be assigned a higher risk score than a model that projects ice cream sales\.
 *  Use the automated transaction analysis tools to improve transparency and explainability for your AI assets\. For example, see how a feature contributes to a prediction and test what\-if scenarios to explore different outcomes\.

<!-- </ul> -->

## Planning for governance ##

Consider these governance strategies:

<!-- <ul> -->

 *  [Build your governance team](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#people)
 *  [Set up your governance structures](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#structure)
 *  [Manage collaboration with roles and access control](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#collab)
 *  [Develop a communication plan](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#communicate)\.
 *  [Implement a simple solution](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#simple)
 *  [Plan for more complex solutions](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-plan.html?context=cdpaas&locale=en#complex)

<!-- </ul> -->

### Build a governance team ###

Consider the expertise that you need on your governance team\. A typical governance plan might include the following roles\. In some cases, the same person might fill multiple roles\. In other cases, a role might represent a team of people\.

<!-- <ul> -->

 *  **Model owner**: The owner creates an AI use case to track a solution to a business need\. The owner requests the model or prompt template, manages the approval process, and tracks the solution through the AI Lifecycle\.
 *  **Model developer/Data scientist**: The developer works with the data in a data set or a large language model (LLM) and creates the machine learning model or LLM prompt template\.
 *  **Model validator**: the validator tests the solution to determine whether it meets the goals that are stated in the AI use case\.
 *  **Risk and compliance manager**: The risk manager determines the policies and compliance thresholds for the AI use case\. For example, the risk manager might determine the rules to apply for testing a solution for fairness or for screening output for hateful and abusive speech\.
 *  **MLOps engineer**: The MLOps engineer moves a solution from a pre\-production (test) environment, to a production environment when a solution is deemed ready to be fully deployed\.
 *  **App developer**: Following deployment, and app developer runs evaluations against the deployment to monitor how the solution performs against the metric threshold set by the risk and compliance owner\. If performance drops below specified thresholds, the app developer works with the other stakeholders to address problems and update the model or prompt template\.

<!-- </ul> -->

### Set up a governance structure ###

After identifying roles and assembling a team, plan your governance structure\.

<!-- <ol> -->

1.  Create an inventory for storing AI use cases\. An inventory is where you store and view AI use cases and the factsheets that are associated with the assets being governed\. Depending on your governance requirements, store all use cases in a single inventory, or create multiple inventories for your governance efforts\.
2.  Create projects for collaboration\. If you are using IBM tools, create a Watson Studio project\. The project can hold the data that is required to train or test the AI solution and the model or prompt template being governed\. Use the access control to restrict access to the approved collaborators\.
3.  Create a pre\-production deployment space\. Use the space to test your model or prompt template by using test data\. Like a project, a space provides access control features so you can include the required collaborators\.
4.  Configure test and validation evaluations\. Provide the model or prompt template details and configure a set of evaluations to test the performance of your solution\. For example, you might test a machine learning model for dimensions such as fairness, quality, and drift, and test a prompt template against metrics such as perplexity (how accurate the output is), or toxicity (whether the output contains hateful or abusive speech\.) By testing on known (labeled) data, you can evaluate the performance before moving a solution to production\.
5.  Configure a production space\. When the model or prompt template is ready to be deployed to a production environment, move the solution and all dependencies to a production space\. A production space typically has a tighter access control list\.
6.  Configure evaluations for the deployed model\. Provide the model details and configure evaluations for the solution\. You now test against live data rather than test data\. It is important to monitor your solution so that you are alerted if thresholds are crossed, indicating a potential problem with the deployed solution\.

<!-- </ol> -->

### Manage collaboration for governance ###

Watsonx\.governance is built on a collaborative platform to allow for all approved team members to contribute to the goals of solving business problems\.

To plan for collaboration, consider how to manage access to the inventories, projects, and spaces you use for governance\.

Use roles along with access control features to ensure that your team has appropriate access to meet goals\.

### Develop a communication plan ###

Some of the workflow around defining an AI use case and moving assets through the lifecycle rely on effective communication\. Decide how your team will communicate and establish the details\. For example,:

<!-- <ul> -->

 *  Will you use email for decision\-making or a messaging tool such as Slack?
 *  Is there a formal process for adding comments to an asset as it moves through a workflow?

<!-- </ul> -->

Create your communication plan and share it with your team\.

### Implement a simple governance solution ###

As you roll out your governance strategy, start with a simple implementation, then consider how to build incrementally to a more comprehensive solution\. The simplest implementation requires an AI use case in an inventory, with an asset moving from request to production\.

For the most straightforward implementation of AI governance, you can use a IBM Knowledge Catalog to track and inventory models\. An AI use case in an inventory consists of a set of factsheets containing lineage, history, and other relevant information about a model's lifecycle\. A watsonx administrator must create an inventory and add data scientists, data engineers, and other users as collaborators\.

![Inventories store factsheets with metadata about governed assets](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/xgov-simple.svg)

AI use case owners can request and track assets:

<!-- <ul> -->

 *  Business users create AI use cases in the inventory to request machine\-learning models or LLM prompt templates\.
 *  Data scientists associate the trained asset with an AI use case to create AI factsheets\.

<!-- </ul> -->

AI factsheets accumulate information about the model or prompt templates in the following ways:

<!-- <ul> -->

 *  All actions that are associated with the tracked asset are automatically saved, including deployments and evaluations\.
 *  All changes to input data assets are automatically saved\.
 *  Data scientists can add tags, business terms, supporting documentation, and other information\.
 *  Data scientists can associate challenger models with the AI use cases to compare model performance\.

<!-- </ul> -->

Validators and other stakeholders review AI factsheets to ensure compliance and certify asset progress from development to production\. They can also generate reports from the factsheets to print, share, or archive details\.

### Plan for more complex solutions ###

You can extend your AI governance implementation at any time\. Consider these options to extend governance:

<!-- <ul> -->

 *  MLOps engineers can extend model tracking to include external models that are created with third\-party machine learning models\.
 *  MLOps engineers can add custom properties to factsheets to track more information\.
 *  Compliance analysts can customize the default report templates to generate tailored reports for the organization\.
 *  Record the results of IBM Watson OpenScale evaluations for fairness and other metrics as part of model tracking\.

<!-- </ul> -->

## Governing assets that are created locally or externally ##

Watsonx\.governance provides the tools for you to govern assets you created using IBM tools, such as machine learning models created by using AutoAI or foundation model prompt templates created in a watsonx project\. You can also govern machine learning models that are created by using non\-IBM tools, such as Microsoft Azure or Amazon Web Services\. As you develop your governance plan, consider these differences:

<!-- <ul> -->

 *  IBM assets developed with tools such as Watson Studio are available for governance earlier in the lifecycle\. You can track the factsheet for a local asset from the Development phase, and have visibility into details such as the training data and creation details from an earlier stage\.
 *  An inventory owner or administrator must enable governance for external models\.
 *  When governance is enabled for external models, they can be added to an AI use case explicitly, or automatically, when they are evaluated with Watson OpenScale\.

<!-- </ul> -->

For a list of supported machine learning model providers, see [Supported machine learning providers in Watson OpenScale](https://dataplatform.cloud.ibm.com/docs/content/wsj/model/wos-frameworks-ovr.html)\.

## Next steps ##

To begin governance, follow the steps in [Provisioning and launching IBM watsonx\.governance](https://dataplatform.cloud.ibm.com/docs/content/wsj/model/wos-provision-launch.html) to provision Watson OpenScale with AI Factsheets\.

**Parent topic:**[Watsonx\.governance overview](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/xgov-overview.html)

<!-- </article "role="article" "> -->