Creating an Endpoint in Generative AI

 Important



Pre-General Availability: 2023-09-18

The following legal notice applies to Oracle pre-GA (Beta) releases. For copyright and other applicable notices, see Oracle Legal Notices.

Pre-General Availability Draft Documentation Notice

This documentation is in pre-General Availability status and is intended for demonstration and preliminary use only. It may not be specific to the hardware on which you are using the software. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to this documentation and will not be responsible for any loss, costs, or damages incurred due to the use of this documentation.

This documentation is not a commitment by Oracle to deliver any material, code, functionality or services. This documentation, and Oracle Pre-GA programs and services are subject to change at any time without notice and, accordingly, should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality for Oracle’s Pre-GA programs and services remains at the sole discretion of Oracle. All release dates or other predictions of future events are subject to change. The future availability of any future Oracle program or service should not be relied on in entering into any license or service agreement with Oracle.

 Note

You must create a custom model, before you can create an endpoint for it. After a custom model is active, which can take up to several hours, perform the following steps.
In the navigation bar of the Console, choose a region that hosts Generative AI, for example, US Midwest (Chicago). If you don't know which region to choose, see Regions with Generative AI.
Open the navigation menu and click Analytics & AI. Under AI Services, click Generative AI.
In the left navigation, choose the compartment that contains the custom model that you want to add an endpoint for.
Click Custom models.
Click the name of the custom model that you want to add an endpoint for.
Click Create endpoint.
 Tip

In the Create endpoint panel, don't change the compartment for the endpoint. To have the endpoint listed in the custom model's detail page, ensure that you create the endpoint in the same compartment as the custom model.
(Optional) Enter a name and description. If you leave the name blank, the system generates a name that you can change later.

The generated name has the format generativeaiendpoint<timestamp>. For example:

generativeaiendpoint20230904025637

Choose a dedicated AI cluster to host the endpoint by performing one of the following actions:
Click a Dedicated AI cluster from the drop-down list.
 Note


Only active hosting dedicated AI clusters are listed in the dedicated AI cluster drop-down list. If you just created a hosting dedicated AI cluster, wait a few minutes for that cluster to become active and show up in the list.
If the hosting dedicated AI cluster is in a different compartment than the custom model's compartment, change to the cluster's compartment and choose that cluster.
Click Create new dedicated AI cluster and perform the following steps:
(Optional) Enter a name and description. If you leave the name blank, the system generates a name with the generativeaidedicatedaicluster<timestamp> format, that you can change later.
For Type, click Hosting-beta.
(Required) Read the terms and click to agree with the beta testing use of the dedicated AI cluster.
Click Create.
Wait a few minutes for the cluster to be created.
From the Dedicated AI cluster drop-down list, choose the dedicated AI cluster that you created.
If the model that you're creating an endpoint for is in a different compartment than the endpoint's compartment, click Change compartment and choose the compartment that hosts the model.
Choose the custom model and the model version that you want to create an endpoint for.
 Tip

Only models with an active state are listed in the Model list. If the custom model that you're looking for isn't listed, click Cancel. Then under Generative AI, click Custom models and ensure that your custom model is in an active state.
Click Create endpoint.
You're directed to the endpoint details page where you can track the state of the endpoint.
After the endpoint is active, click View in Playground and start using the custom model.