Chapter 3. Technology Preview features Important This section describes Technology Preview features in Red Hat OpenShift AI 2.18. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope . Guardrails Orchestrator Framework Red Hat introduces the Guardrails Orchestrator as Technology Preview, enabling flexible AI content filtering with support for multiple detection methods and scalable integration options. This initial release launches advanced response handling capabilities, including blocking and masking of inappropriate content. Distributed InstructLab training InstructLab is an open-source project for enhancing large language models (LLMs) in generative artificial intelligence (gen AI) applications. It fine-tunes models using synthetic data generation (SDG) techniques and a structured taxonomy to create diverse, high-quality training datasets. There are two ways to run the InstructLab training flow: the InstructLab pipeline, which is the recommended approach, and the standalone script: InstructLab pipeline The InstructLab pipeline is now available as a Technology Preview feature, enabling you to run the full InstructLab workflow through a data science pipeline in OpenShift AI. For prerequisites and setup instructions to run this pipeline, see Running InstructLab Pipeline with Data Science Pipelines on RHOAI . Important You must have NVIDIA GPU Operator 24.6 installed to use the InstructLab pipeline in OpenShift AI 2.18. Standalone script Distributed InstructLab training is available as a Technology Preview feature, offering enhanced performance for training tasks in distributed environments compared to single-node setups. By orchestrating multi-node training jobs, this feature improves efficiency and scalability and allows users to leverage distributed resources for more effective AI model development. To implement the full InstructLab training flow with the standalone script, including data preparation, transfer, and distributed execution, see Distributed InstructLab Training on RHOAI . Mandatory Kueue local-queue labeling policy for Ray cluster creation Cluster administrators can use the Validating Admission Policy feature to enforce the mandatory labeling of Ray cluster resources with Kueue local-queue identifiers. This labeling ensures that workloads are properly categorized and routed based on queue management policies, which prevents resource contention and enhances operational efficiency. When the local-queue labeling policy is enforced, Ray clusters are created only if they are configured to use a local queue, and the Ray cluster resources are then managed by Kueue. The local-queue labeling policy is enforced for all projects by default, but can be disabled for some or all projects. For more information about the local-queue labeling policy, see Enforcing the use of local queues . Note This feature might introduce a breaking change for users who did not previously use Kueue local queues to manage their Ray cluster resources. RStudio Server notebook image With the RStudio Server notebook image, you can access the RStudio IDE, an integrated development environment for R. The R programming language is used for statistical computing and graphics to support data analysis and predictions. To use the RStudio Server notebook image, you must first build it by creating a secret and triggering the BuildConfig , and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images . Important Disclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench. CUDA - RStudio Server notebook image With the CUDA - RStudio Server notebook image, you can access the RStudio IDE and NVIDIA CUDA Toolkit. The RStudio IDE is an integrated development environment for the R programming language for statistical computing and graphics. With the NVIDIA CUDA toolkit, you can enhance your work by using GPU-accelerated libraries and optimization tools. To use the CUDA - RStudio Server notebook image, you must first build it by creating a secret and triggering the BuildConfig , and then enable it in the OpenShift AI UI by editing the rstudio-rhel9 image stream. For more information, see Building the RStudio Server workbench images . Important Disclaimer: Red Hat supports managing workbenches in OpenShift AI. However, Red Hat does not provide support for the RStudio software. RStudio Server is available through rstudio.org and is subject to their licensing terms. You should review their licensing terms before you use this sample workbench. The CUDA - RStudio Server notebook image contains NVIDIA CUDA technology. CUDA licensing information is available in the CUDA Toolkit documentation. You should review their licensing terms before you use this sample workbench. Model Registry OpenShift AI now supports the Model Registry Operator. The Model Registry Operator is not installed by default in Technology Preview mode. The model registry is a central repository that contains metadata related to machine learning models from inception to deployment. OCI containers for model storage You can use OCI storage as an alternative to cloud storage services for model serving. First, you create an OCI container image to contain the model. The image is uploaded to an OCI-compatible registry, such as Quay. Later, when deploying a model, the model serving platform references the repository of the containerized model. Using an OCI container can provide the following advantages: Reduced startup times, because the cluster keeps a cache of downloaded images. Restarting the model pod does not download the model again. Lower disk space usage, because the model is not downloaded on each pod replica, assuming pods are scheduled on the same node. Enhanced performance when pre-fetching images or asynchronous loading. Compatibility and integration, because it can be easily integrated with KServe. No additional dependencies are required and the infrastructure might already be available. Support for multinode deployment of very large models Serving models over multiple graphical processing unit (GPU) nodes when using a single-model serving runtime is now available as a Technology Preview feature. Deploy your models across multiple GPU nodes to improve efficiency when deploying large models such as large language models (LLMs). For more information, see Deploying models across multiple GPU nodes .

Commands: