Chapter 4. Monitoring model bias As a data scientist, you might want to monitor your machine learning models for bias. This means monitoring for algorithmic deficiencies that might skew the outcomes or decisions that the model produces. Importantly, this type of monitoring helps you to ensure that the model is not biased against particular protected groups or features. Red Hat OpenShift AI provides a set of metrics that help you to monitor your models for bias. You can use the OpenShift AI interface to choose an available metric and then configure model-specific details such as a protected attribute, the privileged and unprivileged groups, the outcome you want to monitor, and a threshold for bias. You then see a chart of the calculated values for a specified number of model inferences. For more information about the specific bias metrics, see Supported bias metrics . 4.1. Creating a bias metric To monitor a deployed model for bias, you must first create bias metrics. When you create a bias metric, you specify details relevant to your model such as a protected attribute, privileged and unprivileged groups, a model outcome and a value that you want to monitor, and the acceptable threshold for bias. For information about the specific bias metrics, see Supported bias metrics . For the complete list of TrustyAI metrics, see TrustyAI service API . You can create a bias metric for a model by using the OpenShift AI dashboard or by using the OpenShift command-line interface (CLI). 4.1.1. Creating a bias metric by using the dashboard You can use the OpenShift AI dashboard to create a bias metric for a model. Prerequisites You are familiar with the bias metrics that OpenShift AI supports and how to interpret them. You are familiar with the specific data set schema and understand the names and meanings of the inputs and outputs. Your OpenShift cluster administrator added you as a user to the OpenShift cluster and has installed the TrustyAI service for the data science project that contains the deployed models. You set up TrustyAI for your data science project, as described in Setting up TrustyAI for your project . Procedure Optional: To set the TRUSTY_ROUTE variable, follow these steps. In a terminal window, log in to the OpenShift cluster where OpenShift AI is deployed. Set the TRUSTY_ROUTE variable to the external route for the TrustyAI service pod. In the left menu of the OpenShift AI dashboard, click Model Serving . On the Deployed models page, select your project from the drop-down list. Click the name of the model that you want to configure bias metrics for. On the metrics page for the model, click the Model bias tab. Click Configure . In the Configure bias metrics dialog, complete the following steps to configure bias metrics: In the Metric name field, type a unique name for your bias metric. Note that you cannot change the name of this metric later. From the Metric type list, select one of the metrics types that are available in OpenShift AI. In the Protected attribute field, type the name of an attribute in your model that you want to monitor for bias. Tip You can use a curl command to query the metadata endpoint and view input attribute names and values. For example: curl -H "Authorization: Bearer USDTOKEN" USDTRUSTY_ROUTE/info | jq ".[0].data.inputSchema" In the Privileged value field, type the name of a privileged group for the protected attribute that you specified. In the Unprivileged value field, type the name of an unprivileged group for the protected attribute that you specified. In the Output field, type the name of the model outcome that you want to monitor for bias. Tip You can use a curl command to query the metadata endpoint and view output attribute names and values. For example: curl -H "Authorization: Bearer USDTOKEN" USDTRUSTY_ROUTE/info | jq ".[0].data.outputSchema" In the Output value field, type the value of the outcome that you want to monitor for bias. In the Violation threshold field, type the bias threshold for your selected metric type. This threshold value defines how far the specified metric can be from the fairness value for your metric, before the model is considered biased. In the Metric batch size field, type the number of model inferences that OpenShift AI includes each time it calculates the metric. Ensure that the values you entered are correct. Note You cannot edit a model bias metric configuration after you create it. Instead, you can duplicate a metric and then edit (configure) it; however, the history of the original metric is not applied to the copy. Click Configure . Verification The Bias metric configuration page shows the bias metrics that you configured for your model. step To view metrics, on the Bias metric configuration page, click View metrics in the upper-right corner. 4.1.2. Creating a bias metric by using the CLI You can use the OpenShift command-line interface (CLI) to create a bias metric for a model. Prerequisites You are familiar with the bias metrics that OpenShift AI supports and how to interpret them. You are familiar with the specific data set schema and understand the names and meanings of the inputs and outputs. Your OpenShift cluster administrator added you as a user to the OpenShift cluster and has installed the TrustyAI service for the data science project that contains the deployed models. You set up TrustyAI for your data science project, as described in Setting up TrustyAI for your project . Procedure In a terminal window, log in to the OpenShift cluster where OpenShift AI is deployed. Set the TRUSTY_ROUTE variable to the external route for the TrustyAI service pod. Optionally, get the full list of TrustyAI service endpoints and payloads. Use POST /metrics/group/fairness/spd/request to schedule a recurring bias monitoring metric with the following syntax and payload structure: Syntax : Payload structure : modelId The name of the model to query. protectedAttribute The name of the feature that distinguishes the groups that you are checking for fairness. privilegedAttribute The suspected favored (positively biased) class. unprivilegedAttribute The suspected unfavored (negatively biased) class. outcomeName The name of the output that provides the output you are examining for fairness. favorableOutcome The value of the outcomeName output that describes the favorable or desired model prediction. batchSize The number of inferences to include in the calculation. For example: Verification The bias metrics request should return output similar to the following: The specificDefinition field helps you understand the real-world interpretation of these metric values. For this example, the model is fair over the Is Male-Identifying? field, with the rate of positive outcome only differing by about -0.3%. 4.1.3. Duplicating a bias metric If you want to edit an existing metric, you can duplicate (copy) it in the OpenShift AI interface and then edit the values in the copy. However, note that the history of the original metric is not applied to the copy. Prerequisites You are familiar with the bias metrics that OpenShift AI supports and how to interpret them. You are familiar with the specific data set schema and understand the names and meanings of the inputs and outputs. There is an existing bias metric that you want to duplicate. Procedure In the left menu of the OpenShift AI dashboard, click Model Serving . On the Deployed models page, click the name of the model with the bias metric that you want to duplicate. On the metrics page for the model, click the Model bias tab. Click Configure . On the Bias metric configuration page, click the action menu (...) to the metric that you want to copy and then click Duplicate . In the Configure bias metric dialog, follow these steps: In the Metric name field, type a unique name for your bias metric. Note that you cannot change the name of this metric later. Change the values of the fields as needed. For a description of these fields, see Creating a bias metric by using the dashboard . Ensure that the values you entered are correct, and then click Configure . Verification The Bias metric configuration page shows the bias metrics that you configured for your model. step To view metrics, on the Bias metric configuration page, click View metrics in the upper-right corner. 4.2. Deleting a bias metric You can delete a bias metric for a model by using the OpenShift AI dashboard or by using the OpenShift command-line interface (CLI). 4.2.1. Deleting a bias metric by using the dashboard You can use the OpenShift AI dashboard to delete a bias metric for a model. Prerequisites You have logged in to Red Hat OpenShift AI. There is an existing bias metric that you want to delete. Procedure In the left menu of the OpenShift AI dashboard, click Model Serving . On the Deployed models page, click the name of the model with the bias metric that you want to delete. On the metrics page for the model, click the Model bias tab. Click Configure . Click the action menu (...) to the metric that you want to delete and then click Delete . In the Delete bias metric dialog, type the metric name to confirm the deletion. Note You cannot undo deleting a bias metric. Click Delete bias metric . Verification The Bias metric configuration page does not show the bias metric that you deleted. 4.2.2. Deleting a bias metric by using the CLI You can use the OpenShift command-line interface (CLI) to delete a bias metric for a model. Prerequisites You have installed the OpenShift CLI ( oc ). You have a user token for authentication as described in Authenticating the TrustyAI service . There is an existing bias metric that you want to delete. Procedure Open a new terminal window. Follow these steps to log in to your OpenShift cluster: In the upper-right corner of the OpenShift web console, click your user name and select Copy login command . After you have logged in, click Display token . Copy the Log in with this token command and paste it in the OpenShift command-line interface (CLI). In the OpenShift CLI, get the route to the TrustyAI service: Optional: To list all currently active requests for a metric, use GET /metrics/{{metric}}/requests . For example, to list all currently scheduled SPD metrics, type: Alternatively, to list all currently scheduled metric requests, use GET /metrics/all/requests . To delete a metric, send an HTTP DELETE request to the /metrics/USDMETRIC/request endpoint to stop the periodic calculation, including the id of periodic task that you want to cancel in the payload. For example: Verification Use GET /metrics/{{metric}}/requests to list all currently active requests for the metric and verify the metric that you deleted is not shown. For example: 4.3. Viewing bias metrics for a model After you create bias monitoring metrics, you can use the OpenShift AI dashboard to view and update the metrics that you configured. Prerequisite You configured bias metrics for your model as described in Creating a bias metric . Procedure In the OpenShift AI dashboard, click Model Serving . On the Deployed models page, click the name of a model that you want to view bias metrics for. On the metrics page for the model, click the Model bias tab. To update the metrics shown on the page, follow these steps: In the Metrics to display section, use the Select a metric list to select a metric to show on the page. Note Each time you select a metric to show on the page, an additional Select a metric list appears. This enables you to show multiple metrics on the page. From the Time range list in the upper-right corner, select a value. From the Refresh interval list in the upper-right corner, select a value. The metrics page shows the metrics that you selected. Optional: To remove one or more metrics from the page, in the Metrics to display section, perform one of the following actions: To remove an individual metric, click the cancel icon (✖) to the metric name. To remove all metrics, click the cancel icon (✖) in the Select a metric list. Optional: To return to configuring bias metrics for the model, on the metrics page, click Configure in the upper-right corner. Verification The metrics page shows the metrics selections that you made. 4.4. Supported bias metrics Red Hat OpenShift AI supports the following bias metrics: Statistical Parity Difference Statistical Parity Difference (SPD) is the difference in the probability of a favorable outcome prediction between unprivileged and privileged groups. The formal definition of SPD is the following: y = 1 is the favorable outcome. Du and Dp are the unprivileged and privileged group data. You can interpret SPD values as follows: A value of 0 means that the model is behaving fairly for a selected attribute (for example, race, gender). A value in the range -0.1 to 0.1 means that the model is reasonably fair for a selected attribute. Instead, you can attribute the difference in probability to other factors, such as the sample size. A value outside the range -0.1 to 0.1 indicates that the model is unfair for a selected attribute. A negative value indicates that the model has bias against the unprivileged group. A positive value indicates that the model has bias against the privileged group. Disparate Impact Ratio Disparate Impact Ratio (DIR) is the ratio of the probability of a favorable outcome prediction for unprivileged groups to that of privileged groups. The formal definition of DIR is the following: y = 1 is the favorable outcome. Du and Dp are the unprivileged and privileged group data. The threshold to identify bias depends on your own criteria and specific use case. For example, if your threshold for identifying bias is represented by a DIR value below 0.8 or above 1.2 , you can interpret the DIR values as follows: A value of 1 means that the model is fair for a selected attribute. A value of between 0.8 and 1.2 means that the model is reasonably fair for a selected attribute. A value below 0.8 or above 1.2 indicates bias.

Commands:
login
TRUSTY_ROUTE=https://USD(oc get route/trustyai-service --template={{.spec.host}})
login
TRUSTY_ROUTE=https://USD(oc get route/trustyai-service --template={{.spec.host}})
curl -H "Authorization: Bearer USDTOKEN" --location USDTRUSTY_ROUTE/q/openapi
curl -sk -H "Authorization: Bearer USDTOKEN" -X POST --location USDTRUSTY_ROUTE/metrics/group/fairness/spd/request --header 'Content-Type: application/json' --data <payload>
curl -sk -H "Authorization: Bearer USDTOKEN" -X POST --location USDTRUSTY_ROUTE /metrics/group/fairness/spd/request --header 'Content-Type: application/json' --data "{ \"modelId\": \"demo-loan-nn-onnx-alpha\", \"protectedAttribute\": \"Is Male-Identifying?\", \"privilegedAttribute\": 1.0, \"unprivilegedAttribute\": 0.0, \"outcomeName\": \"Will Default?\", \"favorableOutcome\": 0, \"batchSize\": 5000 }"
{ "timestamp":"2023-10-24T12:06:04.586+00:00", "type":"metric", "value":-0.0029676404469311524, "namedValues":null, "specificDefinition":"The SPD of -0.002968 indicates that the likelihood of Group:Is Male-Identifying?=1.0 receiving Outcome:Will Default?=0 was -0.296764 percentage points lower than that of Group:Is Male-Identifying?=0.0.", "name":"SPD", "id":"d2707d5b-cae9-41aa-bcd3-d950176cbbaf", "thresholds":{"lowerBound":-0.1,"upperBound":0.1,"outsideBounds":false} }
oc login --token= <token> --server= <openshift_cluster_url>
TRUSTY_ROUTE=https://USD(oc get route/trustyai-service --template={{.spec.host}})
curl -H "Authorization: Bearer USDTOKEN" -X GET --location "USDTRUSTY_ROUTE/metrics/spd/requests"
curl -H "Authorization: Bearer USDTOKEN" -X GET --location "USDTRUSTY_ROUTE/metrics/all/requests"
curl -H "Authorization: Bearer USDTOKEN" -X DELETE --location "USDTRUSTY_ROUTE/metrics/spd/request" -H "Content-Type: application/json" -d "{ \"requestId\": \"3281c891-e2a5-4eb3-b05d-7f3831acbb56\" }"
curl -H "Authorization: Bearer USDTOKEN" -X GET --location "USDTRUSTY_ROUTE/metrics/spd/requests"