# watsonx\.governance model health monitor evaluation metrics #

watsonx\.governance enables model health monitor evaluations by default to help you understand your model behavior and performance\. You can use model health metrics to determine how efficiently your model deployment processes your transactions\.

## Supported model health metrics ##

The following metric categories for model health evaluations are supported by watsonx\.governance\. Each category contains metrics that provide details about your model performance:

<!-- <ul> -->

 *  Scoring requests
    
    watsonx.governance calculates the number of scoring requests that your model deployment receives during model health evaluations. This metric category is supported for traditional machine learning models and foundation models.

<!-- </ul> -->

<!-- <ul> -->

 *  Records
    
    watsonx.governance calculates the **total**, **average**, **minimum**, **maximum**, and **median** number of transaction records that are processed across scoring requests during model health evaluations. This metric category is supported for traditional machine learning models and foundation models.

<!-- </ul> -->

<!-- <ul> -->

 *  Token count
    
    watsonx.governance calculates the number of tokens that are processed across scoring requests for your model deployment. This metric category is supported for foundation models only. watsonx.governance calculates the following metrics to measure token count during evaluations: - **Input token count**: Calculates the **total**, **average**, **minimum**, **maximum**, and **median** input token count across multiple scoring requests during evaluations - **Output token count**: Calculates the **total**, **average**, **minimum**, **maximum**, and **median** output token count across scoring requests during evaluations

<!-- </ul> -->

<!-- <ul> -->

 *  Throughput and latency
    
    watsonx.governance calculates latency by tracking the time that it takes to process scoring requests and transaction records per millisecond (ms). Throughput is calculated by tracking the number of scoring requests and transaction records that are processed per second. To calculate throughput and latency, watsonx.governance uses the `response_time` value from your scoring requests to track the time that your model deployment takes to process scoring requests. For Watson Machine Learning deployments, Watson OpenScale automatically detects the `response_time` value when you configure evaluations. For external and custom deployments, you must specify the `response_time` value when you send scoring requests to calculate throughput and latency as shown in the following example from the Watson OpenScale [Python SDK](https://client-docs.aiopenscale.cloud.ibm.com/html/index.html): `python from ibm_watson_openscale.supporting_classes.payload_record import PayloadRecord client.data_sets.store_records( data_set_id=payload_data_set_id, request_body=[ PayloadRecord( scoring_id=<uuid>, request=openscale_input, response=openscale_output, response_time=<response_time>, user_id=<user_id>) ] )` watsonx.governance calculates the following metrics to measure thoughput and latency during evaluations: - **API latency**: Time taken (in ms) to process a scoring request by your model deployment. - **API throughput**: Number of scoring requests processed by your model deployment per second - **Record latency**: Time taken (in ms) to process a record by your model deployment - **Record throughput**: Number of records processed by your model deployment per second This metric category is supported for traditional machine learning models and foundation models.

<!-- </ul> -->

<!-- <ul> -->

 *  Users
    
    watsonx.governance calculates the number of users that send scoring requests to your model deployments. This metric category is supported for traditional machine learning models and foundation models. To calculate the number of users, watsonx.governance uses the `user_id` from scoring requests to identify the users that send the scoring requests that your model receives. For external and custom deployments, you must specify the `user_id` value when you send scoring requests to calculate the number of users as shown in the following example from the Watson OpenScale [Python SDK](https://client-docs.aiopenscale.cloud.ibm.com/html/index.html): `python from ibm_watson_openscale.supporting_classes.payload_record import PayloadRecord client.data_sets.store_records( data_set_id=payload_data_set_id, request_body=[ PayloadRecord( scoring_id=<uuid>, request=openscale_input, response=openscale_output, response_time=<response_time>, user_id=<user_id>). --> value to be supplied by user ] )` When you view a summary of the **Users** metric in watsonx.governance, you can use the real-time view to see the total number of users and the aggregated views to see the average number of users.

<!-- </ul> -->

<!-- <ul> -->

 *  Payload size
    
    watsonx.governance calculates the **total**, **average**, **minimum**, **maximum**, and **median** payload size of the transaction records that your model deployment processes across scoring requests in kilobytes (KB). watsonx.governance does not support payload size metrics for image models. This metric category is supported for traditional machine learning models only.

<!-- </ul> -->

<!-- </article "role="article" "> -->