Chapter 6. Working with accelerator profiles To configure accelerators for your data scientists to use in OpenShift AI, you must create an associated accelerator profile. An accelerator profile is a custom resource definition (CRD) on OpenShift that has an AcceleratorProfile resource, and defines the specification of the accelerator. You can create and manage accelerator profiles by selecting Settings Accelerator profiles on the OpenShift AI dashboard. For accelerators that are new to your deployment, you must manually configure an accelerator profile for each accelerator. If your deployment contains an accelerator before you upgrade, the associated accelerator profile remains after the upgrade. You can manage the accelerators that appear to your data scientists by assigning specific accelerator profiles to your custom notebook images. This example shows the code for a Habana Gaudi 1 accelerator profile: --- apiVersion: dashboard.opendatahub.io/v1alpha kind: AcceleratorProfile metadata: name: hpu-profile-first-gen-gaudi spec: displayName: Habana HPU - 1st Gen Gaudi description: First Generation Habana Gaudi device enabled: true identifier: habana.ai/gaudi tolerations: - effect: NoSchedule key: habana.ai/gaudi operator: Exists --- The accelerator profile code appears on the Instances tab on the details page for the AcceleratorProfile custom resource definition (CRD). For more information about accelerator profile attributes, see the following table: Table 6.1. Accelerator profile attributes Attribute Type Required Description displayName String Required The display name of the accelerator profile. description String Optional Descriptive text defining the accelerator profile. identifier String Required A unique identifier defining the accelerator resource. enabled Boolean Required Determines if the accelerator is visible in OpenShift AI. tolerations Array Optional The tolerations that can apply to notebooks and serving runtimes that use the accelerator. For more information about the toleration attributes that OpenShift AI supports, see Toleration v1 core . Additional resources Toleration v1 core Understanding taints and tolerations Managing resources from custom resource definitions 6.1. Viewing accelerator profiles If you have defined accelerator profiles for OpenShift AI, you can view, enable, and disable them from the Accelerator profiles page. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. Your deployment contains existing accelerator profiles. Procedure From the OpenShift AI dashboard, click Settings Accelerator profiles . The Accelerator profiles page appears, displaying existing accelerator profiles. Inspect the list of accelerator profiles. To enable or disable an accelerator profile, on the row containing the accelerator profile, click the toggle in the Enable column. Verification The Accelerator profiles page appears appears, displaying existing accelerator profiles. 6.2. Creating an accelerator profile To configure accelerators for your data scientists to use in OpenShift AI, you must create an associated accelerator profile. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. Procedure From the OpenShift AI dashboard, click Settings Accelerator profiles . The Accelerator profiles page appears, displaying existing accelerator profiles. To enable or disable an existing accelerator profile, on the row containing the relevant accelerator profile, click the toggle in the Enable column. Click Create accelerator profile . The Create accelerator profile dialog appears. In the Name field, enter a name for the accelerator profile. In the Identifier field, enter a unique string that identifies the hardware accelerator associated with the accelerator profile. Optional: In the Description field, enter a description for the accelerator profile. To enable or disable the accelerator profile immediately after creation, click the toggle in the Enable column. Optional: Add a toleration to schedule pods with matching taints. Click Add toleration . The Add toleration dialog opens. From the Operator list, select one of the following options: Equal - The key/value/effect parameters must match. This is the default. Exists - The key/effect parameters must match. You must leave a blank value parameter, which matches any. From the Effect list, select one of the following options: None NoSchedule - New pods that do not match the taint are not scheduled onto that node. Existing pods on the node remain. PreferNoSchedule - New pods that do not match the taint might be scheduled onto that node, but the scheduler tries not to. Existing pods on the node remain. NoExecute - New pods that do not match the taint cannot be scheduled onto that node. Existing pods on the node that do not have a matching toleration are removed. In the Key field, enter a toleration key. The key is any string, up to 253 characters. The key must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores. In the Value field, enter a toleration value. The value is any string, up to 63 characters. The value must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores. In the Toleration Seconds section, select one of the following options to specify how long a pod stays bound to a node that has a node condition. Forever - Pods stays permanently bound to a node. Custom value - Enter a value, in seconds, to define how long pods stay bound to a node that has a node condition. Click Add . Click Create accelerator profile . Verification The accelerator profile appears on the Accelerator profiles page. The Accelerator list appears on the Start a notebook server page. After you select an accelerator, the Number of accelerators field appears, which you can use to choose the number of accelerators for your notebook server. The accelerator profile appears on the Instances tab on the details page for the AcceleratorProfile custom resource definition (CRD). Additional resources Toleration v1 core Understanding taints and tolerations Managing resources from custom resource definitions 6.3. Updating an accelerator profile You can update the existing accelerator profiles in your deployment. You might want to change important identifying information, such as the display name, the identifier, or the description. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. The accelerator profile exists in your deployment. Procedure From the OpenShift AI dashboard, click Settings Notebook images . The Notebook images page appears. Previously imported notebook images are displayed. To enable or disable a previously imported notebook image, on the row containing the relevant notebook image, click the toggle in the Enable column. Click the action menu (...) and select Edit from the list. The Edit accelerator profile dialog opens. In the Name field, update the accelerator profile name. In the Identifier field, update the unique string that identifies the hardware accelerator associated with the accelerator profile, if applicable. Optional: In the Description field, update the accelerator profile. To enable or disable the accelerator profile immediately after creation, click the toggle in the Enable column. Optional: Add a toleration to schedule pods with matching taints. Click Add toleration . The Add toleration dialog opens. From the Operator list, select one of the following options: Equal - The key/value/effect parameters must match. This is the default. Exists - The key/effect parameters must match. You must leave a blank value parameter, which matches any. From the Effect list, select one of the following options: None NoSchedule - New pods that do not match the taint are not scheduled onto that node. Existing pods on the node remain. PreferNoSchedule - New pods that do not match the taint might be scheduled onto that node, but the scheduler tries not to. Existing pods on the node remain. NoExecute - New pods that do not match the taint cannot be scheduled onto that node. Existing pods on the node that do not have a matching toleration are removed. In the Key field, enter a toleration key. The key is any string, up to 253 characters. The key must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores. In the Value field, enter a toleration value. The value is any string, up to 63 characters. The value must begin with a letter or number, and may contain letters, numbers, hyphens, dots, and underscores. In the Toleration Seconds section, select one of the following options to specify how long a pod stays bound to a node that has a node condition. Forever - Pods stays permanently bound to a node. Custom value - Enter a value, in seconds, to define how long pods stay bound to a node that has a node condition. Click Add . If your accelerator profile contains existing tolerations, you can edit them. Click the action menu (...) on the row containing the toleration that you want to edit and select Edit from the list. Complete the applicable fields to update the details of the toleration. Click Update . Click Update accelerator profile . Verification If your accelerator profile has new identifying information, this information appears in the Accelerator list on the Start a notebook server page. Additional resources Toleration v1 core Understanding taints and tolerations Managing resources from custom resource definitions 6.4. Deleting an accelerator profile To discard accelerator profiles that you no longer require, you can delete them so that they do not appear on the dashboard. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. The accelerator profile that you want to delete exists in your deployment. Procedure From the OpenShift AI dashboard, click Settings Accelerator profiles . The Accelerator profiles page appears, displaying existing accelerator profiles. Click the action menu ( ... ) beside the accelerator profile that you want to delete and click Delete . The Delete accelerator profile dialog opens. Enter the name of the accelerator profile in the text field to confirm that you intend to delete it. Click Delete . Verification The accelerator profile no longer appears on the Accelerator profiles page. Additional resources Toleration v1 core Understanding taints and tolerations Managing resources from custom resource definitions 6.5. Configuring a recommended accelerator for notebook images To help you indicate the most suitable accelerators to your data scientists, you can configure a recommended tag to appear on the dashboard. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. You have existing notebook images in your deployment. You have enabled GPU support in OpenShift AI. This includes installing the Node Feature Discovery operator and NVIDIA GPU Operators. For more information, see Installing the Node Feature Discovery operator and Enabling NVIDIA GPUs . Procedure From the OpenShift AI dashboard, click Settings Notebook images . The Notebook images page appears. Previously imported notebook images are displayed. Click the action menu (...) and select Edit from the list. The Update notebook image dialog opens. From the Accelerator identifier list, select an identifier to set its accelerator as recommended with the notebook image. If the notebook image contains only one accelerator identifier, the identifier name displays by default. Click Update . Note If you have already configured an accelerator identifier for a notebook image, you can specify a recommended accelerator for the notebook image by creating an associated accelerator profile. To do this, click Create profile on the row containing the notebook image and complete the relevant fields. If the notebook image does not contain an accelerator identifier, you must manually configure one before creating an associated accelerator profile. Verification When your data scientists select an accelerator with a specific notebook image, a tag appears to the corresponding accelerator indicating its compatibility. 6.6. Configuring a recommended accelerator for serving runtimes To help you indicate the most suitable accelerators to your data scientists, you can configure a recommended accelerator tag for your serving runtimes. Prerequisites You have logged in to OpenShift AI as a user with OpenShift AI administrator privileges. You have enabled GPU support in OpenShift AI. This includes installing the Node Feature Discovery operator and NVIDIA GPU Operators. For more information, see Installing the Node Feature Discovery operator and Enabling NVIDIA GPUs . Procedure From the OpenShift AI dashboard, click Settings > Serving runtimes . The Serving runtimes page opens and shows the model-serving runtimes that are already installed and enabled in your OpenShift AI deployment. By default, the OpenVINO Model Server runtime is pre-installed and enabled in OpenShift AI. Edit your custom runtime that you want to add the recommended accelerator tag to, click the action menu (...) and select Edit . A page with an embedded YAML editor opens. Note You cannot directly edit the OpenVINO Model Server runtime that is included in OpenShift AI by default. However, you can clone this runtime and edit the cloned version. You can then add the edited clone as a new, custom runtime. To do this, click the action menu beside the OpenVINO Model Server and select Duplicate . In the editor, enter the YAML code to apply the annotation opendatahub.io/recommended-accelerators . The excerpt in this example shows the annotation to set a recommended tag for an NVIDIA GPU accelerator: metadata: annotations: opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]' Click Update . Verification When your data scientists select an accelerator with a specific serving runtime, a tag appears to the corresponding accelerator indicating its compatibility.

Commands:
--- apiVersion: dashboard.opendatahub.io/v1alpha kind: AcceleratorProfile metadata: name: hpu-profile-first-gen-gaudi spec: displayName: Habana HPU - 1st Gen Gaudi description: First Generation Habana Gaudi device enabled: true identifier: habana.ai/gaudi tolerations: - effect: NoSchedule key: habana.ai/gaudi operator: Exists ---
metadata: annotations: opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]'