Repository: skypilot-org/skypilot
Pull request #5463: [Nebius] Add support config file and remove hardcode
Author: login: SalikovAlex
Created: 2025-04-30T20:55:41Z | Merged: 2025-05-05T19:29:56Z

The Nebius project ID file support has been removed in favor of configuration in `~/.sky/config.yaml`, which now includes `project_id` and `fabric` options. Enhancements include streamlined GPU cluster creation and clearer documentation for Nebius configurations.

Tested (run the relevant ones):

- [x] Code formatting: install pre-commit (auto-check on commit) or `bash format.sh`
- [x] Any manual or new tests for this PR (please specify below)
- [x] All smoke tests: `/smoke-test` (CI) or `pytest tests/test_smoke.py` (local)




Diff:
diff --git a/docs/source/cloud-setup/cloud-permissions/nebius.rst b/docs/source/cloud-setup/cloud-permissions/nebius.rst
index ee167b585cc..1824b655591 100644
--- a/docs/source/cloud-setup/cloud-permissions/nebius.rst
+++ b/docs/source/cloud-setup/cloud-permissions/nebius.rst
@@ -37,4 +37,3 @@ To use *Service Account* authentication, follow these steps:
 **Important Notes:**
 
 * The `NEBIUS_IAM_TOKEN` file, if present, will take priority for authentication.
-* Service Accounts are restricted to a single region. Ensure you configure the Service Account for the appropriate region during creation.
diff --git a/docs/source/getting-started/installation.rst b/docs/source/getting-started/installation.rst
index c0ee40fbaf1..4fd3d4ce841 100644
--- a/docs/source/getting-started/installation.rst
+++ b/docs/source/getting-started/installation.rst
@@ -544,11 +544,8 @@ Nebius
   nebius iam get-access-token > ~/.nebius/NEBIUS_IAM_TOKEN.txt
   nebius --format json iam whoami|jq -r '.user_profile.tenants[0].tenant_id' > ~/.nebius/NEBIUS_TENANT_ID.txt
 
-**Optional**: You can specify a preferable project ID, which will be used if a project ID is required in the designated region.
 
-.. code-block:: shell
-
-  echo $NEBIUS_PROJECT_ID > ~/.nebius/NEBIUS_PROJECT_ID.txt
+**Optional**: You can specify specific project ID and fabric in `~/.sky/config.yaml`, see :ref:`Configuration project_id and fabric for Nebius <config-yaml-nebius>`.
 
 Alternatively, you can also use a service account to access Nebius, see :ref:`Using Service Account for Nebius <nebius-service-account>`.
 
@@ -568,8 +565,8 @@ In the prompt, enter your Nebius Access Key ID and Secret Access Key (see `instr
 
   aws configure set aws_access_key_id $NB_ACCESS_KEY_AWS_ID --profile nebius
   aws configure set aws_secret_access_key $NB_SECRET_ACCESS_KEY --profile nebius
-  aws configure set region eu-west1 --profile nebius
-  aws configure set endpoint_url https://storage.eu-west1.nebius.cloud:443  --profile nebius
+  aws configure set region <REGION> --profile nebius
+  aws configure set endpoint_url <ENDPOINT>  --profile nebius
 
 Request quotas for first time users
 --------------------------------------
diff --git a/docs/source/reference/config.rst b/docs/source/reference/config.rst
index b6bb2810eee..518617e43b2 100644
--- a/docs/source/reference/config.rst
+++ b/docs/source/reference/config.rst
@@ -122,6 +122,13 @@ Below is the configuration syntax and some example values. See detailed explanat
     :ref:`us-ashburn-1 <config-yaml-oci>`:
       vcn_ocid: ocid1.vcn.oc1.ap-seoul-1.amaaaaaaak7gbriarkfs2ssus5mh347ktmi3xa72tadajep6asio3ubqgarq
       vcn_subnet: ocid1.subnet.oc1.iad.aaaaaaaafbj7i3aqc4ofjaapa5edakde6g4ea2yaslcsay32cthp7qo55pxa
+  :ref:`nebius <config-yaml-nebius>`:
+    :ref:`eu-north1 <config-yaml-nebius>`:
+      project_id: project-e00xxxxxxxxxxx
+      fabric: fabric-3
+    :ref:`eu-west1 <config-yaml-nebius>`:
+      project_id: project-e01xxxxxxxxxxx
+      fabric: fabric-5
 
 Fields
 ----------
@@ -1048,6 +1055,40 @@ Example:
           vcn_ocid: ocid1.vcn.oc1.ap-seoul-1.amaaaaaaak7gbriarkfs2ssus5mh347ktmi3xa72tadajep6asio3ubqgarq
           vcn_subnet: ocid1.subnet.oc1.iad.aaaaaaaafbj7i3aqc4ofjaapa5edakde6g4ea2yaslcsay32cthp7qo55pxa
 
+.. _config-yaml-nebius:
+
+``nebius``
+~~~~~~~~~~
+
+Advanced Nebius configuration (optional).
+
+``project_id``
+    Identifier for the Nebius project (optional)
+    Default: Uses first available project if not specified
+
+``fabric``
+    GPU cluster configuration identifier (optional)
+    Optional: GPU cluster disabled if not specified
+
+
+The configuration must be specified in region-specific sections.
+
+Example:
+
+.. code-block:: yaml
+
+    nebius:
+        # Region-specific configuration
+        eu-north1:
+            # Project identifier for this region
+            # Optional: Uses first available project if not specified
+            project_id: project-e00......
+            # GPU cluster fabric identifier
+            # Optional: GPU cluster disabled if not specified
+            fabric: fabric-3
+        eu-west1:
+            project_id: project-e01...
+            fabric: fabric-5
 
 .. toctree::
    :hidden:
diff --git a/sky/adaptors/nebius.py b/sky/adaptors/nebius.py
index acc20e5abe9..6a4a5537a47 100644
--- a/sky/adaptors/nebius.py
+++ b/sky/adaptors/nebius.py
@@ -8,11 +8,9 @@
 
 NEBIUS_TENANT_ID_FILENAME = 'NEBIUS_TENANT_ID.txt'
 NEBIUS_IAM_TOKEN_FILENAME = 'NEBIUS_IAM_TOKEN.txt'
-NEBIUS_PROJECT_ID_FILENAME = 'NEBIUS_PROJECT_ID.txt'
 NEBIUS_CREDENTIALS_FILENAME = 'credentials.json'
 NEBIUS_TENANT_ID_PATH = '~/.nebius/' + NEBIUS_TENANT_ID_FILENAME
 NEBIUS_IAM_TOKEN_PATH = '~/.nebius/' + NEBIUS_IAM_TOKEN_FILENAME
-NEBIUS_PROJECT_ID_PATH = '~/.nebius/' + NEBIUS_PROJECT_ID_FILENAME
 NEBIUS_CREDENTIALS_PATH = '~/.nebius/' + NEBIUS_CREDENTIALS_FILENAME
 
 DEFAULT_REGION = 'eu-north1'
@@ -92,16 +90,6 @@ def is_token_or_cred_file_exist():
             os.path.exists(os.path.expanduser(NEBIUS_CREDENTIALS_PATH)))
 
 
-@annotations.lru_cache(scope='request')
-def get_project_id():
-    try:
-        with open(os.path.expanduser(NEBIUS_PROJECT_ID_PATH),
-                  encoding='utf-8') as file:
-            return file.read().strip()
-    except FileNotFoundError:
-        return None
-
-
 @annotations.lru_cache(scope='request')
 def get_tenant_id():
     try:
diff --git a/sky/clouds/nebius.py b/sky/clouds/nebius.py
index bfac7c3040b..aeec3a154b6 100644
--- a/sky/clouds/nebius.py
+++ b/sky/clouds/nebius.py
@@ -17,7 +17,6 @@
     # credential files for Nebius
     nebius.NEBIUS_TENANT_ID_FILENAME,
     nebius.NEBIUS_IAM_TOKEN_FILENAME,
-    nebius.NEBIUS_PROJECT_ID_FILENAME,
     nebius.NEBIUS_CREDENTIALS_FILENAME
 ]
 
diff --git a/sky/provision/nebius/utils.py b/sky/provision/nebius/utils.py
index 29997cf14c8..43936f3e15a 100644
--- a/sky/provision/nebius/utils.py
+++ b/sky/provision/nebius/utils.py
@@ -4,6 +4,7 @@
 import uuid
 
 from sky import sky_logging
+from sky import skypilot_config
 from sky.adaptors import nebius
 from sky.utils import common_utils
 
@@ -35,44 +36,22 @@ def get_project_by_region(region: str) -> str:
     service = nebius.iam().ProjectServiceClient(nebius.sdk())
     projects = service.list(nebius.iam().ListProjectsRequest(
         parent_id=nebius.get_tenant_id())).wait()
-    # To find a project in a specific region, we rely on the project ID to
-    # deduce the region, since there is currently no method to retrieve region
-    # information directly from the project. Additionally, there is only one
-    # project per region, and projects cannot be created at this time.
-    # The region is determined from the project ID using a region-specific
-    # identifier embedded in it.
-    # Project id looks like project-e00xxxxxxxxxxxxxx where
-    # e00 - id of region 'eu-north1'
-    # e01 - id of region 'eu-west1'
-    region_ids = {'eu-north1': 'e00', 'eu-west1': 'e01'}
-    # TODO(SalikovAlex): fix when info about region will be in projects list
-    # Currently, Nebius cloud supports 2 regions. We manually enumerate
-    # them here. Reference: https://docs.nebius.com/overview/regions
 
     #  Check is there project if in config
-    preferable_project_id = nebius.get_project_id()
-    if preferable_project_id is not None:
-        if preferable_project_id[8:11] == region_ids[region]:
-            return preferable_project_id
-        logger.warning(
-            f'Can\'t use customized NEBIUS_PROJECT_ID ({preferable_project_id})'
-            f' for region {region}. Please check if the project ID is correct.')
+    project_id = skypilot_config.get_nested(('nebius', region, 'project_id'),
+                                            None)
+    if project_id is not None:
+        return project_id
     for project in projects.items:
-        if project.metadata.id[8:11] == region_ids[region]:
+        if project.status.region == region:
             return project.metadata.id
     raise Exception(f'No project found for region "{region}".')
 
 
-def get_or_create_gpu_cluster(name: str, region: str) -> str:
+def get_or_create_gpu_cluster(name: str, project_id: str, fabric: str) -> str:
     """Creates a GPU cluster.
-    When creating a GPU cluster, select an InfiniBand fabric for it:
-
-    fabric-2, fabric-3 or fabric-4 for projects in the eu-north1 region.
-    fabric-5 for projects in the eu-west1 region.
-
     https://docs.nebius.com/compute/clusters/gpu
     """
-    project_id = get_project_by_region(region)
     service = nebius.compute().GpuClusterServiceClient(nebius.sdk())
     try:
         cluster = service.get_by_name(nebius.nebius_common().GetByNameRequest(
@@ -80,14 +59,7 @@ def get_or_create_gpu_cluster(name: str, region: str) -> str:
             name=name,
         )).wait()
         cluster_id = cluster.metadata.id
-    except nebius.request_error() as no_cluster_found_error:
-        if region == 'eu-north1':
-            fabric = 'fabric-4'
-        elif region == 'eu-west1':
-            fabric = 'fabric-5'
-        else:
-            raise RuntimeError(
-                f'Unsupported region {region}.') from no_cluster_found_error
+    except nebius.request_error():
         cluster = service.create(nebius.compute().CreateGpuClusterRequest(
             metadata=nebius.nebius_common().ResourceMetadata(
                 parent_id=project_id,
@@ -196,15 +168,23 @@ def launch(cluster_name_on_cloud: str, node_type: str, platform: str,
 
     disk_name = 'disk-' + instance_name
     cluster_id = None
+    project_id = get_project_by_region(region)
     # 8 GPU virtual machines can be grouped into a GPU cluster.
     # The GPU clusters are built with InfiniBand secure high-speed networking.
     # https://docs.nebius.com/compute/clusters/gpu
     if platform in ('gpu-h100-sxm', 'gpu-h200-sxm'):
         if preset == '8gpu-128vcpu-1600gb':
-            cluster_id = get_or_create_gpu_cluster(cluster_name_on_cloud,
-                                                   region)
+            #  Check is there fabric in config
+            fabric = skypilot_config.get_nested(('nebius', region, 'fabric'),
+                                                None)
+            if fabric is None:
+                logger.warning(
+                    f'Set up fabric for region {region} in ~/.sky/config.yaml '
+                    'to use GPU clusters.')
+            else:
+                cluster_id = get_or_create_gpu_cluster(cluster_name_on_cloud,
+                                                       project_id, fabric)
 
-    project_id = get_project_by_region(region)
     service = nebius.compute().DiskServiceClient(nebius.sdk())
     disk = service.create(nebius.compute().CreateDiskRequest(
         metadata=nebius.nebius_common().ResourceMetadata(
diff --git a/sky/utils/schemas.py b/sky/utils/schemas.py
index 9dd4d669c79..3b201917ef5 100644
--- a/sky/utils/schemas.py
+++ b/sky/utils/schemas.py
@@ -963,6 +963,24 @@ def get_config_schema():
                 }
             },
         },
+        'nebius': {
+            'type': 'object',
+            'required': [],
+            'properties': {},
+            'additionalProperties': {
+                'type': 'object',
+                'required': [],
+                'additionalProperties': False,
+                'properties': {
+                    'project_id': {
+                        'type': 'string',
+                    },
+                    'fabric': {
+                        'type': 'string',
+                    },
+                }
+            },
+        }
     }
 
     admin_policy_schema = {


Reviews:
user: login: cg505
body: Is there any way we can throw an error if the user typos a region name in the config? E.g. if I accidentally write `ue-north1`, it should have an error.
Otherwise, I think this looks pretty good. Thanks!
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-01T21:24:59Z
user: login: cg505
body: Great, thank you! Going to run some tests but otherwise looks good to merge.
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-05T18:27:08Z

Review comments:
diff_hunk: @@ -544,11 +544,16 @@ Nebius
   nebius iam get-access-token > ~/.nebius/NEBIUS_IAM_TOKEN.txt
   nebius --format json iam whoami|jq -r '.user_profile.tenants[0].tenant_id' > ~/.nebius/NEBIUS_TENANT_ID.txt
 
-**Optional**: You can specify a preferable project ID, which will be used if a project ID is required in the designated region.
 
-.. code-block:: shell
+**Optional**: You can specify a preferable project ID and fabric in `~/.sky/config.yaml`
path: docs/source/getting-started/installation.rst
user: login: cg505
body: Question about "preferable" - does this mean that the project id / fabric can be ignored in some cases, or is it always respected?
created_at: 2025-05-01T21:17:04Z
author_association: COLLABORATOR
diff_hunk: @@ -1048,6 +1055,40 @@ Example:
           vcn_ocid: ocid1.vcn.oc1.ap-seoul-1.amaaaaaaak7gbriarkfs2ssus5mh347ktmi3xa72tadajep6asio3ubqgarq
           vcn_subnet: ocid1.subnet.oc1.iad.aaaaaaaafbj7i3aqc4ofjaapa5edakde6g4ea2yaslcsay32cthp7qo55pxa
 
+.. _config-yaml-nebius:
+
+``nebius``
+~~~~~~~~~~
+
+Advanced Nebius configuration (optional).
+
+``project_id``
+    Identifier for the Nebius project (optional)
+    Default: Uses first available project if not specified
+
+``fabric``
+    GPU cluster configuration identifier (optional)
+    Optional: GPU cluster disabled if not specified
+
+
+The configuration can be specified in region-specific sections.
path: docs/source/reference/config.rst
user: login: cg505
body: ```suggestion
The configuration must be specified in region-specific sections.
```
created_at: 2025-05-01T21:20:14Z
author_association: COLLABORATOR
diff_hunk: @@ -544,11 +544,16 @@ Nebius
   nebius iam get-access-token > ~/.nebius/NEBIUS_IAM_TOKEN.txt
   nebius --format json iam whoami|jq -r '.user_profile.tenants[0].tenant_id' > ~/.nebius/NEBIUS_TENANT_ID.txt
 
-**Optional**: You can specify a preferable project ID, which will be used if a project ID is required in the designated region.
 
-.. code-block:: shell
+**Optional**: You can specify a preferable project ID and fabric in `~/.sky/config.yaml`
+
+.. code-block:: yaml
 
-  echo $NEBIUS_PROJECT_ID > ~/.nebius/NEBIUS_PROJECT_ID.txt
+  # ~/.sky/config.yaml
+  nebius:
+      eu-north1:
+        project_id: project-e00xxxxxxxxxxxx
+        fabric: fabric-2
path: docs/source/getting-started/installation.rst
user: login: cg505
body: I think this should be moved to cloud-setup/cloud-permissions/nebius.rst. We can just have a link in the "Optional:" part to that page.
created_at: 2025-05-01T21:21:46Z
author_association: COLLABORATOR
diff_hunk: @@ -573,8 +565,8 @@ In the prompt, enter your Nebius Access Key ID and Secret Access Key (see `instr
 
   aws configure set aws_access_key_id $NB_ACCESS_KEY_AWS_ID --profile nebius
   aws configure set aws_secret_access_key $NB_SECRET_ACCESS_KEY --profile nebius
-  aws configure set region eu-west1 --profile nebius
-  aws configure set endpoint_url https://storage.eu-west1.nebius.cloud:443  --profile nebius
+  aws configure set region REGION --profile nebius
+  aws configure set endpoint_url ENDPOINT  --profile nebius
path: docs/source/getting-started/installation.rst
user: login: cg505
body: ```suggestion
  aws configure set region <REGION> --profile nebius
  aws configure set endpoint_url <ENDPOINT>  --profile nebius
```
Just to clarify that these values should be replaced
created_at: 2025-05-05T18:25:48Z
author_association: COLLABORATOR

PR comments:
user: login: SalikovAlex
created_at: 2025-05-02T12:05:55Z
body: > Is there any way we can throw an error if the user typos a region name in the config? E.g. if I accidentally write ue-north1, it should have an error.

Unfortunately, there is no method to receive all available regions, and I don't want to hardcode regions.
As soon as this method is available, I can add this check in the future. 
author_association: CONTRIBUTOR
user: login: cg505
created_at: 2025-05-05T18:32:15Z
body: Also, just to clarify, it's now REQUIRED to set the fabric in config in order to use GPU clusters, right?
author_association: COLLABORATOR
user: login: cg505
created_at: 2025-05-05T18:33:13Z
body: /quicktest-core
author_association: COLLABORATOR
user: login: cg505
created_at: 2025-05-05T18:33:17Z
body: /smoke-test -k test_basic
author_association: COLLABORATOR