Repository: Snapchat/GiGL
Pull request #61: Introduce Bootstrap Resource Config script for testing/developing locally
Author: login: svij-sc
Created: 2025-05-21T16:11:39Z | Merged: 2025-05-23T04:39:21Z

**Scope of work done**

<!-- Description of PR goes here -->

1. Changed `Makefile` to allow us to override compute resources that are used for unit and integration tests.
2. Subsequently introduced a script that allows new users to help instantiate those values
3. Updated tests which were failing as a result of ^ change i.e. they may have been hardcoded to use a specific bucket, etc. 

Added ability to run easily override the default test resource config for unit and integration tests:
<img width="940" alt="Screenshot 2025-05-21 at 9 12 40 AM" src="https://github.com/user-attachments/assets/c7016349-bc8a-4dca-95de-74b7449ef6ab" />

See updated docs: 
https://snapchat.github.io/GiGL/index.html#tests

<img width="771" alt="Screenshot 2025-05-21 at 9 13 03 AM" src="https://github.com/user-attachments/assets/cd56a85b-6b5a-478f-8bff-0b174b437041" />

Doc:
https://snapchat.github.io/GiGL/docs/user_guide/getting_started/quick_start.html#config-setup



<!-- Relevant screenshots go here (optional) -->

Where is the documentation for this feature?: N/A

Did you add automated tests or write a test plan?

- [x] Create an instance using script introduced in this PR: https://github.com/Snapchat/GiGL/pull/58
- [x] Follow updated instructions introduced in this draft PR to setup necessary resources permissions, et al: [here](https://github.com/Snapchat/GiGL/pull/53/files#diff-192172726bbaa34fe5f06fcaeb241823a7ea09c9d758d523067bfb2f9ab49029). Instructions also deployed here: https://snapchat.github.io/GiGL/docs/user_guide/getting_started/quick_start.html#config-setup
- [x] Ran the script introduced in this PR
- [x] Ran unit/integration tests
- [x] Able to run a successful e2e pipeline as per example provided here: https://snapchat.github.io/GiGL/docs/user_guide/getting_started/quick_start.html#running-an-end-to-end-gigl-pipeline
    ```
    export GIGL_CORA_NABLP_TASK_CONFIG="gigl/src/mocking/configs/e2e_node_anchor_based_link_prediction_template_gbml_config.yaml"
    make \
      job_name="$(whoami)_gigl_hello_world_cora_nalp" \
      task_config_uri="$GIGL_CORA_NABLP_TASK_CONFIG" \
      resource_config_uri="$GIGL_TEST_DEFAULT_RESOURCE_CONFIG" \
      start_at="config_populator" \
      run_dev_gnn_kubeflow_pipeline
    ```


***Updated Changelog.md?*** NO

***Ready for code review?:*** YES



Diff:
diff --git a/Makefile b/Makefile
index 6fd83b5ed..5fe597207 100644
--- a/Makefile
+++ b/Makefile
@@ -8,13 +8,15 @@ DATE:=$(shell /bin/date "+%Y%m%d-%H%M")
 
 # GIT HASH, or empty string if not in a git repo.
 GIT_HASH?=$(shell git rev-parse HEAD 2>/dev/null || "")
+PWD=$(shell pwd)
 
 # You can override GIGL_PROJECT by setting it in your environment i.e.
 # adding `export GIGL_PROJECT=your_project` to your shell config (~/.bashrc, ~/.zshrc, etc.)
 GIGL_PROJECT?=external-snap-ci-github-gigl
-DOCKER_IMAGE_DATAFLOW_RUNTIME_NAME:=us-central1-docker.pkg.dev/${GIGL_PROJECT}/gigl-base-images/src-cpu-dataflow
-DOCKER_IMAGE_MAIN_CUDA_NAME:=us-central1-docker.pkg.dev/${GIGL_PROJECT}/gigl-base-images/src-cuda
-DOCKER_IMAGE_MAIN_CPU_NAME:=us-central1-docker.pkg.dev/${GIGL_PROJECT}/gigl-base-images/src-cpu
+GIGL_DOCKER_ARTIFACT_REGISTRY?=us-central1-docker.pkg.dev/${GIGL_PROJECT}/gigl-base-images
+DOCKER_IMAGE_DATAFLOW_RUNTIME_NAME:=${GIGL_DOCKER_ARTIFACT_REGISTRY}/src-cpu-dataflow
+DOCKER_IMAGE_MAIN_CUDA_NAME:=${GIGL_DOCKER_ARTIFACT_REGISTRY}/src-cuda
+DOCKER_IMAGE_MAIN_CPU_NAME:=${GIGL_DOCKER_ARTIFACT_REGISTRY}/src-cpu
 
 DOCKER_IMAGE_DATAFLOW_RUNTIME_NAME_WITH_TAG:=${DOCKER_IMAGE_DATAFLOW_RUNTIME_NAME}:${DATE}
 DOCKER_IMAGE_MAIN_CUDA_NAME_WITH_TAG:=${DOCKER_IMAGE_MAIN_CUDA_NAME}:${DATE}
@@ -22,6 +24,9 @@ DOCKER_IMAGE_MAIN_CPU_NAME_WITH_TAG:=${DOCKER_IMAGE_MAIN_CPU_NAME}:${DATE}
 
 PYTHON_DIRS:=examples python shared scripts
 PY_TEST_FILES?="*_test.py"
+# You can override GIGL_TEST_DEFAULT_RESOURCE_CONFIG by setting it in your environment i.e.
+# adding `export GIGL_TEST_DEFAULT_RESOURCE_CONFIG=your_resource_config` to your shell config (~/.bashrc, ~/.zshrc, etc.)
+GIGL_TEST_DEFAULT_RESOURCE_CONFIG?=${PWD}/deployment/configs/unittest_resource_config.yaml
 
 GIT_BRANCH:=$(shell git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "")
 
@@ -32,6 +37,7 @@ GIT_BRANCH:=$(shell git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "")
 # Thus, we only want to format the Markdown files that we explicitly include in our repo.
 MD_FILES:=$(shell if [ ! ${GIT_BRANCH} ]; then echo "."; else git ls-tree --name-only -r ${GIT_BRANCH} . | grep ".md"; fi;)
 
+
 get_ver_hash:
 	# Fetches the git commit hash and stores it in `$GIT_COMMIT`
 	git diff --quiet || { echo Branch is dirty, please commit changes and ensure branch is clean; exit 1; }
@@ -139,7 +145,7 @@ unit_test_py: clean_build_files_py type_check
 	( cd python ; \
 	python -m tests.unit.main \
 		--env=test \
-		--resource_config_uri ../deployment/configs/unittest_resource_config.yaml \
+		--resource_config_uri=${GIGL_TEST_DEFAULT_RESOURCE_CONFIG} \
 		--test_file_pattern=$(PY_TEST_FILES) \
 	)
 
@@ -183,7 +189,7 @@ integration_test:
 	cd python ;\
 	python -m tests.integration.main \
 	--env=test \
-	--resource_config_uri ../deployment/configs/unittest_resource_config.yaml \
+	--resource_config_uri=${GIGL_TEST_DEFAULT_RESOURCE_CONFIG} \
 	--test_file_pattern=$(PY_TEST_FILES) \
 	)
 
diff --git a/deployment/configs/unittest_resource_config.yaml b/deployment/configs/unittest_resource_config.yaml
index adf177657..2b6666ec3 100644
--- a/deployment/configs/unittest_resource_config.yaml
+++ b/deployment/configs/unittest_resource_config.yaml
@@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100  # set to `ACCELERATOR_TYPE_UNSPECIFIED` for cpu training
+    gpu_limit: 1  # set to 0 for cpu training
     num_replicas: 2
 inferencer_config:
   num_workers: 1
diff --git a/python/tests/integration/common/file_loader_test.py b/python/tests/integration/common/file_loader_test.py
index f8d2e98b2..d1a9cf8f7 100644
--- a/python/tests/integration/common/file_loader_test.py
+++ b/python/tests/integration/common/file_loader_test.py
@@ -6,8 +6,8 @@
 import gigl.common.utils.local_fs as local_fs
 from gigl.common import GcsUri, LocalUri, Uri
 from gigl.common.utils.gcs import GcsUtils
+from gigl.env.pipelines_config import get_resource_config
 from gigl.src.common.utils.file_loader import FileLoader
-from tests.test_assets.uri_constants import TEST_DATA_GCS_BUCKET
 
 
 class FileLoaderTest(unittest.TestCase):
@@ -15,38 +15,43 @@ def setUp(self) -> None:
         self.file_loader = FileLoader()
         self.gcs_utils = GcsUtils()
         test_uuid = str(uuid.uuid4())
-        # TODO (svij): Refactor name and how the location of tests is derived
-        # Also need to investigate whether or not we need to clean up these assets
         self.test_asset_directory: LocalUri = LocalUri.join(".test_assets", test_uuid)
+        resource_config = get_resource_config()
         self.gcs_test_asset_directory: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, test_uuid
+            resource_config.shared_resource_config.common_compute_config.temp_assets_bucket,
+            test_uuid,
         )
 
-    # def test_local_temp_file(self):
-    #     local_file_path_src: LocalUri = LocalUri.join(
-    #         self.test_asset_directory, "test_local_temp_file.txt"
-    #     )
+    def tearDown(self) -> None:
+        self.gcs_utils.delete_files_in_bucket_dir(
+            gcs_path=self.gcs_test_asset_directory
+        )
+
+    def test_local_temp_file(self):
+        local_file_path_src: LocalUri = LocalUri.join(
+            self.test_asset_directory, "test_local_temp_file.txt"
+        )
 
-    #     local_fs.remove_file_if_exist(local_path=local_file_path_src)
+        local_fs.remove_file_if_exist(local_path=local_file_path_src)
 
-    #     # Create files and ensure they exist
-    #     local_fs.create_empty_file_if_none_exists(local_path=local_file_path_src)
-    #     self.assertTrue(local_fs.does_path_exist(local_file_path_src))
-    #     with open(local_file_path_src.uri, "w") as f:
-    #         f.write("Hello")
+        # Create files and ensure they exist
+        local_fs.create_empty_file_if_none_exists(local_path=local_file_path_src)
+        self.assertTrue(local_fs.does_path_exist(local_file_path_src))
+        with open(local_file_path_src.uri, "w") as f:
+            f.write("Hello")
 
-    #     temp_f = self.file_loader.load_to_temp_file(file_uri_src=local_file_path_src)
-    #     with open(temp_f.name, "r") as f:
-    #         msg = f.read()
-    #     self.assertTrue(msg == "Hello")
-    #     temp_f.close()
+        temp_f = self.file_loader.load_to_temp_file(file_uri_src=local_file_path_src)
+        with open(temp_f.name, "r") as f:
+            msg = f.read()
+        self.assertTrue(msg == "Hello")
+        temp_f.close()
 
     def test_gcs_temp_file(self):
         local_file_path_src: LocalUri = LocalUri.join(
             self.test_asset_directory, "test_gcs_temp_file.txt"
         )
         gcs_file_path_src: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, "test_gcs_temp_file.txt"
+            self.gcs_test_asset_directory, "test_gcs_temp_file.txt"
         )
 
         local_fs.remove_file_if_exist(local_path=local_file_path_src)
@@ -101,7 +106,7 @@ def test_local_to_gcs_file(self):
             self.test_asset_directory, "test_local_to_gcs.txt"
         )
         gcs_file_path_dst: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, "test_local_to_gcs.txt"
+            self.gcs_test_asset_directory, "test_local_to_gcs.txt"
         )
 
         local_fs.remove_file_if_exist(local_path=local_file_path_src)
@@ -122,7 +127,7 @@ def test_gcs_to_local_file(self):
         )
 
         gcs_file_path_src: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, "test_gcs_to_local.txt"
+            self.gcs_test_asset_directory, "test_gcs_to_local.txt"
         )
 
         local_file_path_dst: LocalUri = LocalUri.join(
@@ -149,10 +154,10 @@ def test_gcs_to_local_file(self):
 
     def test_gcs_to_gcs_file(self):
         gcs_file_path_src: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, "test_gcs_to_gcs_src.txt"
+            self.gcs_test_asset_directory, "test_gcs_to_gcs_src.txt"
         )
         gcs_file_path_dst: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, "test_gcs_to_gcs_dst.txt"
+            self.gcs_test_asset_directory, "test_gcs_to_gcs_dst.txt"
         )
         with self.assertRaises(TypeError):
             self.file_loader.load_files(
@@ -189,7 +194,7 @@ def test_local_to_gcs_dir(self):
         local_files = ["a.txt", "b.txt", "c.txt", "d.txt"]
         local_src_dir: LocalUri = LocalUri.join(self.test_asset_directory, "src")
         gcs_dst_dir: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, self.test_asset_directory, "dst"
+            self.gcs_test_asset_directory, self.test_asset_directory, "dst"
         )
 
         local_file_paths_src: List[LocalUri] = [
@@ -213,14 +218,16 @@ def test_local_to_gcs_dir(self):
         for gcs_file in gcs_file_paths_dst:
             self.assertTrue(self.gcs_utils.does_gcs_file_exist(gcs_path=gcs_file))
         self.gcs_utils.delete_files_in_bucket_dir(
-            gcs_path=GcsUri.join(TEST_DATA_GCS_BUCKET, self.test_asset_directory)
+            gcs_path=GcsUri.join(
+                self.gcs_test_asset_directory, self.test_asset_directory
+            )
         )
 
     def test_gcs_to_local_dir(self):
         local_files = ["a.txt", "b.txt", "c.txt", "d.txt"]
         local_src_dir: LocalUri = LocalUri.join(self.test_asset_directory, "src")
         gcs_src_dir: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, self.test_asset_directory, "src"
+            self.gcs_test_asset_directory, self.test_asset_directory, "src"
         )
         local_dst_dir: LocalUri = LocalUri.join(self.test_asset_directory, "dst")
 
@@ -262,15 +269,17 @@ def test_gcs_to_local_dir(self):
         for file in local_file_paths_dst:
             self.assertTrue(local_fs.does_path_exist(file))
         self.gcs_utils.delete_files_in_bucket_dir(
-            gcs_path=GcsUri.join(TEST_DATA_GCS_BUCKET, self.test_asset_directory)
+            gcs_path=GcsUri.join(
+                self.gcs_test_asset_directory, self.test_asset_directory
+            )
         )
 
     def test_gcs_to_gcs_dir(self):
         gcs_src_dir: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, self.test_asset_directory, "src"
+            self.gcs_test_asset_directory, self.test_asset_directory, "src"
         )
         gcs_dst_dir: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, self.test_asset_directory, "dst"
+            self.gcs_test_asset_directory, self.test_asset_directory, "dst"
         )
         dir_uri_map: Dict[Uri, Uri] = {gcs_src_dir: gcs_dst_dir}
 
@@ -295,6 +304,3 @@ def test_can_file_loader_check_existance_and_delete_uris(self):
         # Ensure both files are deleted
         self.assertFalse(file_loader.does_uri_exist(uri=tmp_local_file))
         self.assertFalse(file_loader.does_uri_exist(uri=tmp_gcs_file))
-
-    def tearDown(self) -> None:
-        pass
diff --git a/python/tests/integration/pipeline/split_generator/split_generator_pipeline_test.py b/python/tests/integration/pipeline/split_generator/split_generator_pipeline_test.py
index e7ab96332..ce2db4fcc 100644
--- a/python/tests/integration/pipeline/split_generator/split_generator_pipeline_test.py
+++ b/python/tests/integration/pipeline/split_generator/split_generator_pipeline_test.py
@@ -771,7 +771,11 @@ def test_split_generator_transductive_node_anchor_link_prediction(
         resource_config_uri = UriFactory.create_uri(
             uri=get_resource_config().get_resource_config_uri
         )
-        assert isinstance(resource_config_uri, LocalUri)
+
+        if not isinstance(resource_config_uri, LocalUri):
+            file_loader = FileLoader()
+            temp_file = file_loader.load_to_temp_file(file_uri_src=resource_config_uri)
+            resource_config_uri = LocalUri(uri=temp_file.name)
 
         self._compile_and_run_splitgen_pipeline_locally(
             frozen_gbml_config_uri=frozen_gbml_config_uri,
@@ -818,7 +822,10 @@ def test_split_generator_transductive_supervised_node_classification(
         resource_config_uri = UriFactory.create_uri(
             uri=get_resource_config().get_resource_config_uri
         )
-        assert isinstance(resource_config_uri, LocalUri)
+        if not isinstance(resource_config_uri, LocalUri):
+            file_loader = FileLoader()
+            temp_file = file_loader.load_to_temp_file(file_uri_src=resource_config_uri)
+            resource_config_uri = LocalUri(uri=temp_file.name)
 
         self._compile_and_run_splitgen_pipeline_locally(
             frozen_gbml_config_uri=frozen_gbml_config_uri,
@@ -867,7 +874,10 @@ def test_split_generator_inductive_supervised_node_classification(
         resource_config_uri = UriFactory.create_uri(
             uri=get_resource_config().get_resource_config_uri
         )
-        assert isinstance(resource_config_uri, LocalUri)
+        if not isinstance(resource_config_uri, LocalUri):
+            file_loader = FileLoader()
+            temp_file = file_loader.load_to_temp_file(file_uri_src=resource_config_uri)
+            resource_config_uri = LocalUri(uri=temp_file.name)
 
         self._compile_and_run_splitgen_pipeline_locally(
             frozen_gbml_config_uri=frozen_gbml_config_uri,
diff --git a/python/tests/integration/pipeline/subgraph_sampler/subgraph_sampler_test.py b/python/tests/integration/pipeline/subgraph_sampler/subgraph_sampler_test.py
index 1f509b1fd..5e2fdb6d5 100644
--- a/python/tests/integration/pipeline/subgraph_sampler/subgraph_sampler_test.py
+++ b/python/tests/integration/pipeline/subgraph_sampler/subgraph_sampler_test.py
@@ -124,7 +124,10 @@ def __compile_and_run_sgs_pipeline_locally(
         resource_config_uri = UriFactory.create_uri(
             uri=get_resource_config().get_resource_config_uri
         )
-        assert isinstance(resource_config_uri, LocalUri)
+        if not isinstance(resource_config_uri, LocalUri):
+            file_loader = FileLoader()
+            temp_file = file_loader.load_to_temp_file(file_uri_src=resource_config_uri)
+            resource_config_uri = LocalUri(uri=temp_file.name)
 
         compile_and_run_sgs_pipeline_locally(
             frozen_gbml_config_uri=frozen_gbml_config_uri,
diff --git a/scripts/bootstrap_resource_config.py b/scripts/bootstrap_resource_config.py
new file mode 100644
index 000000000..dca1e4c5e
--- /dev/null
+++ b/scripts/bootstrap_resource_config.py
@@ -0,0 +1,327 @@
+import argparse
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+from dataclasses import dataclass
+from typing import Dict, Optional
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+@dataclass
+class Param:
+    default: Optional[str]
+    description: str
+    long_description: str = ""
+    required: bool = True
+
+
+class SupportedParams:
+    defaults: Dict[str, Param]
+
+    def __init__(self):
+        try:
+            project = subprocess.check_output(
+                ["gcloud", "config", "get-value", "project"], text=True
+            ).strip()
+        except subprocess.CalledProcessError as e:
+            print(
+                "Error retrieving active project name; is your gcloud SDK configured correctly?",
+                e,
+            )
+            raise
+        self.defaults: Dict[str, Param] = {
+            "project": Param(
+                default=project,
+                description="GCP Project name that you are planning on using",
+            ),
+            "region": Param(
+                default="us-central1",
+                description="The GCP region where you created your resources",
+            ),
+            "gcp_service_account_email": Param(
+                default=None, description="The GCP Service account"
+            ),
+            "docker_artifact_registry_path": Param(
+                default=None,
+                description="The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`",
+            ),
+            "temp_assets_bq_dataset_name": Param(
+                default=None,
+                description="`temp_assets_bq_dataset_name` - Dataset name used for temporary assets",
+            ),
+            "embedding_bq_dataset_name": Param(
+                default=None,
+                description="`embedding_bq_dataset_name` - Dataset used of output embeddings",
+            ),
+            "temp_assets_bucket": Param(
+                default=None,
+                description="`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`",
+            ),
+            "perm_assets_bucket": Param(
+                default=None,
+                description="`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`",
+            ),
+        }
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path: str,
+    gigl_test_default_resource_config: str,
+    gigl_project: str,
+    gigl_docker_artifact_registry_path: str,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by GiGL/scripts/bootstrap_resource_config.py.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command `{command}` failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command `{command}` failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"Confirmed BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command `{command}` failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"Confirmed GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    supported_params = SupportedParams()
+    parser = argparse.ArgumentParser(
+        description="Bootstrap GiGL resource config. All parameters can be provided as CLI args or interactively."
+    )
+    for key, param in supported_params.defaults.items():
+        help_text = (
+            f"{param.description} (default: {param.default})"
+            if param.default
+            else param.description
+        )
+        parser.add_argument(f"--{key}", type=str, help=help_text)
+    args = parser.parse_args()
+
+    values: Dict[str, str] = {}
+    for key, param in supported_params.defaults.items():
+        # Check if value is provided in command line arguments
+        if getattr(args, key):
+            values[key] = getattr(args, key)
+            continue
+        # If not, prompt for input
+        else:
+            input_question: str
+            long_description_clause = (
+                f"\n{param.long_description}" if param.long_description else ""
+            )
+            if param.default is None:
+                required_clause = "(required)" if param.required else "(optional)"
+                input_question = f"-> {param.description}{long_description_clause} {required_clause}: "
+            else:
+                input_question = f"-> {param.description}{long_description_clause}\nDefaults to: [{param.default}]: "
+
+            values[key] = input(input_question).strip() or param.default  # type: ignore
+            if not values[key] and param.required:
+                raise ValueError(
+                    f"Missing required value for {key}. Please provide a value."
+                )
+
+    # Validate existence of resources
+    assert_gcp_project_exists(values["project"])
+    assert values["region"], "Region cannot be empty"
+    assert values["gcp_service_account_email"], "GCP Service account cannot be empty"
+    assert values[
+        "docker_artifact_registry_path"
+    ], "Docker Artifact Registry path cannot be empty"
+    assert_bq_dataset_exists(
+        dataset_name=values["temp_assets_bq_dataset_name"], project=values["project"]
+    )
+    assert_bq_dataset_exists(
+        dataset_name=values["embedding_bq_dataset_name"], project=values["project"]
+    )
+    assert_gcs_bucket_exists(bucket_name=values["temp_assets_bucket"])
+    assert_gcs_bucket_exists(bucket_name=values["perm_assets_bucket"])
+
+    curr_datetime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+    curr_username = getpass.getuser()
+    default_resource_config_dest_path = f"{values['perm_assets_bucket']}/{curr_username}/{curr_datetime}/gigl_test_default_resource_config.yaml"
+
+    destination_file_path = (
+        input(
+            f"Output path for resource config (default: {default_resource_config_dest_path}); can be GCS or Local path: "
+        ).strip()
+        or default_resource_config_dest_path
+    )
+    assert destination_file_path and destination_file_path.startswith(
+        "gs://"
+    ), "Destination file path must be a GCS path starting with 'gs://'"
+
+    print("=======================================================")
+    print(f"Will now create the resource config file @ {destination_file_path}.")
+    print("Using the following values:")
+    update_fields_dict = {
+        "project": values["project"],
+        "region": values["region"],
+        "gcp_service_account_email": values["gcp_service_account_email"],
+        "temp_assets_bucket": values["temp_assets_bucket"],
+        "temp_regional_assets_bucket": values["temp_assets_bucket"],
+        "perm_assets_bucket": values["perm_assets_bucket"],
+        "temp_assets_bq_dataset_name": values["temp_assets_bq_dataset_name"],
+        "embedding_bq_dataset_name": values["embedding_bq_dataset_name"],
+    }
+    for key, value in update_fields_dict.items():
+        print(f"  {key}: {value}")
+
+    with open(TEMPLATE_SOURCE_RESOURCE_CONFIG, "r") as file:
+        config = yaml.safe_load(file)
+
+    # Update the YAML content
+    common_compute_config: dict = config.get("shared_resource_config").get(
+        "common_compute_config"
+    )
+    common_compute_config.update(update_fields_dict)
+
+    tmp_file = tempfile.NamedTemporaryFile(delete=False)
+    with open(tmp_file.name, "w") as file:
+        yaml.safe_dump(config, file)
+
+    file_loader = FileLoader(project=values["project"])
+    file_uri_src = UriFactory.create_uri(uri=tmp_file.name)
+    file_uri_dest = UriFactory.create_uri(uri=destination_file_path)
+    file_loader.load_file(file_uri_src=file_uri_src, file_uri_dst=file_uri_dest)
+
+    print(f"Updated YAML file saved at '{destination_file_path}'")
+
+    # Update the user's shell configuration
+    should_update_shell_config = (
+        input(
+            "Do you want to update your shell configuration file so you can use this new resource config for tests? [y/n] (Default: y): "
+        )
+        .strip()
+        .lower()
+        or "y"
+    )
+    if should_update_shell_config == "y":
+        shell_config_path: str = infer_shell_file()
+        update_shell_config(
+            shell_config_path=shell_config_path,
+            gigl_test_default_resource_config=destination_file_path,
+            gigl_project=values["project"],
+            gigl_docker_artifact_registry_path=values["docker_artifact_registry_path"],
+        )
+
+        print(
+            f"Please restart your shell or run `source {shell_config_path}` to apply the changes."
+        )
+    else:
+        print(
+            "Skipping shell configuration update. Please remember to set the environment variables manually "
+            + "if you want `make unit_test | integration_test` commands to work correctly."
+        )
diff --git a/scripts/setup_environment.sh b/scripts/setup_environment.sh
deleted file mode 100644
index 2549cb210..000000000
--- a/scripts/setup_environment.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-echo "Initializing environment and installing dependencies"
-
-apt-get update && apt-get upgrade -y && apt-get install -y wget unzip cmake
-
-if [ -d "../miniconda" ]
-then
-  echo "miniconda already installed"
-else
-  wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ../miniconda.sh
-  bash ../miniconda.sh -b -p ../miniconda
-fi
-
-source ../miniconda/etc/profile.d/conda.sh
-
-conda init bash
-
-make initialize_environment
-
-# for some reason this needs to be run again
-source ../miniconda/etc/profile.d/conda.sh
-conda activate gnn
-
-# hack to fix a bug when running in GCE
-export BOTO_CONFIG=null
-
-CURR_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-
-make install_dev_deps
-
-source ~/.profile


Reviews:
user: login: kmontemayor2-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-21T16:35:23Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-21T16:37:00Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-21T16:37:21Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T20:22:29Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T20:22:39Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T20:23:51Z
user: login: svij-sc
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T20:24:31Z
user: login: kmontemayor2-sc
body: Thanks Shubham, was some update commit not pushed here?
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-22T20:45:56Z
user: login: svij-sc
body: > Thanks Shubham, was some update commit not pushed here?

Correct - whoops. I have pushed changes and addressed the new comments.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-23T02:10:58Z
user: login: mkolodner-sc
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-23T02:22:27Z

Review comments:
diff_hunk: @@ -83,45 +89,45 @@ install_deps:
 generate_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong.
+# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong
 generate_dev_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cuda_hashed_requirements:
 	pip-compile  -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra experimental \
+	--extra torch25-cuda-121 --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cuda_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra dev --extra experimental \
+	--extra torch25-cuda-121 --extra transform --extra dev \
 	./python/pyproject.toml
 
-# These are a collection of tests that are run before anything is installed using tools available on host.
+# These are a collection of tests that are run before anything is installed using tools abialable on host.
path: Makefile
user: login: kmontemayor2-sc
body: revert?
created_at: 2025-05-21T16:27:09Z
author_association: COLLABORATOR
diff_hunk: @@ -368,27 +368,23 @@ run_all_e2e_tests: job_name_prefixes_str:=\
 		"cora_nalp_test_on" \
 		"cora_snc_test_on" \
 		"dblp_nalp_test_on" \
-		"cora_glt_udl_test_on" \
-		"dblp_glt_test_on"
+		"cora_glt_udl_test_on"
 # Removed UDL due to transient issue:
 # "gigl/src/mocking/configs/e2e_udl_node_anchor_based_link_prediction_template_gbml_config.yaml"
 run_all_e2e_tests: task_config_uris_str:=\
 		"gigl/src/mocking/configs/e2e_node_anchor_based_link_prediction_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/e2e_supervised_node_classification_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/dblp_node_anchor_based_link_prediction_template_gbml_config.yaml" \
-		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml" \
-		"examples/distributed/configs/e2e_dblp_glt_task_config.yaml"
+		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml"
 run_all_e2e_tests: resource_config_uris_str:=\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
-		"deployment/configs/e2e_glt_resource_config.yaml"\
 		"deployment/configs/e2e_glt_resource_config.yaml"
 run_all_e2e_tests: should_compile_then_run_str:=\
 		"true" \
 		"false" \
 		"false" \
-		"false" \
 		"false"
 run_all_e2e_tests: _run_e2e_kfp_test
path: Makefile
user: login: kmontemayor2-sc
body: revert?
created_at: 2025-05-21T16:27:33Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
+    gpu_limit: 1  # set to 0 for cpu training
path: deployment/configs/unittest_resource_config.yaml
user: login: kmontemayor2-sc
body: qq why this change?
created_at: 2025-05-21T16:27:50Z
author_association: COLLABORATOR
diff_hunk: @@ -6,47 +6,33 @@
 import gigl.common.utils.local_fs as local_fs
 from gigl.common import GcsUri, LocalUri, Uri
 from gigl.common.utils.gcs import GcsUtils
+from gigl.env.pipelines_config import get_resource_config
 from gigl.src.common.utils.file_loader import FileLoader
-from tests.test_assets.uri_constants import TEST_DATA_GCS_BUCKET
 
 
 class FileLoaderTest(unittest.TestCase):
     def setUp(self) -> None:
         self.file_loader = FileLoader()
         self.gcs_utils = GcsUtils()
         test_uuid = str(uuid.uuid4())
-        # TODO (svij): Refactor name and how the location of tests is derived
-        # Also need to investigate whether or not we need to clean up these assets
         self.test_asset_directory: LocalUri = LocalUri.join(".test_assets", test_uuid)
+        resource_config = get_resource_config()
         self.gcs_test_asset_directory: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, test_uuid
+            resource_config.shared_resource_config.common_compute_config.temp_assets_bucket,
+            test_uuid,
         )
 
-    # def test_local_temp_file(self):
-    #     local_file_path_src: LocalUri = LocalUri.join(
-    #         self.test_asset_directory, "test_local_temp_file.txt"
-    #     )
-
-    #     local_fs.remove_file_if_exist(local_path=local_file_path_src)
-
-    #     # Create files and ensure they exist
-    #     local_fs.create_empty_file_if_none_exists(local_path=local_file_path_src)
-    #     self.assertTrue(local_fs.does_path_exist(local_file_path_src))
-    #     with open(local_file_path_src.uri, "w") as f:
-    #         f.write("Hello")
-
-    #     temp_f = self.file_loader.load_to_temp_file(file_uri_src=local_file_path_src)
-    #     with open(temp_f.name, "r") as f:
-    #         msg = f.read()
-    #     self.assertTrue(msg == "Hello")
-    #     temp_f.close()
path: python/tests/integration/common/file_loader_test.py
user: login: kmontemayor2-sc
body: btw do you know why this test was disabled and can we re-enable it?
created_at: 2025-05-21T16:28:29Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: nit. types?
created_at: 2025-05-21T16:29:23Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: why not create if it doesn't exist?
created_at: 2025-05-21T16:29:45Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: is there a bug here if there are multiple blocks? eg we would update all blocks?
created_at: 2025-05-21T16:30:48Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: ```suggestion
        "# This section is auto-generated by the GiGL scripts/bootstrap_resource_config.py script.\n",
```

nit?
created_at: 2025-05-21T16:31:29Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: btw can we also use `argparse` here e.g. I can provide some things via cli like `python bootstrap_resource_config.py --project my-project` and then it asks me for the non-project info?
created_at: 2025-05-21T16:33:34Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
+
+    curr_datetime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+    curr_username = getpass.getuser()
+    resource_config_dest_path = f"{perm_assets_bucket}/{curr_username}/{curr_datetime}/gigl_test_default_resource_config.yaml"
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: Why not also allow local paths here?
created_at: 2025-05-21T16:34:39Z
author_association: COLLABORATOR
diff_hunk: @@ -83,45 +89,45 @@ install_deps:
 generate_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong.
+# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong
 generate_dev_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cuda_hashed_requirements:
 	pip-compile  -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra experimental \
+	--extra torch25-cuda-121 --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cuda_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra dev --extra experimental \
+	--extra torch25-cuda-121 --extra transform --extra dev \
 	./python/pyproject.toml
 
-# These are a collection of tests that are run before anything is installed using tools available on host.
+# These are a collection of tests that are run before anything is installed using tools abialable on host.
path: Makefile
user: login: svij-sc
body: oops; thanks
created_at: 2025-05-21T16:37:00Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
+    gpu_limit: 1  # set to 0 for cpu training
path: deployment/configs/unittest_resource_config.yaml
user: login: svij-sc
body: it seems that configuration is just not supported anymore 
created_at: 2025-05-21T16:37:20Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
+
+    curr_datetime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+    curr_username = getpass.getuser()
+    resource_config_dest_path = f"{perm_assets_bucket}/{curr_username}/{curr_datetime}/gigl_test_default_resource_config.yaml"
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: more  so folks don't accidentally leak their private resource configs; accidentally committing them to the repo.
For intermediary users if they wanted they can do this manually and store it locally.
created_at: 2025-05-22T20:22:28Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: sure, this is a good idea - let me update
created_at: 2025-05-22T20:22:39Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: This block is tied to this script.
If there are multiple blocks we would just delete them both and append a new block in its place.

I think this is expected behaviour. 
created_at: 2025-05-22T20:23:51Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: User should provide a working shell  - we shouldnt just create one for them i.e. this is likely a mistype.
created_at: 2025-05-22T20:24:31Z
author_association: COLLABORATOR
diff_hunk: @@ -83,45 +89,45 @@ install_deps:
 generate_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong.
+# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong
 generate_dev_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cuda_hashed_requirements:
 	pip-compile  -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra experimental \
+	--extra torch25-cuda-121 --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cuda_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra dev --extra experimental \
+	--extra torch25-cuda-121 --extra transform --extra dev \
 	./python/pyproject.toml
 
-# These are a collection of tests that are run before anything is installed using tools available on host.
+# These are a collection of tests that are run before anything is installed using tools abialable on host.
path: Makefile
user: login: kmontemayor2-sc
body: not reverted?
created_at: 2025-05-22T20:38:41Z
author_association: COLLABORATOR
diff_hunk: @@ -368,27 +368,23 @@ run_all_e2e_tests: job_name_prefixes_str:=\
 		"cora_nalp_test_on" \
 		"cora_snc_test_on" \
 		"dblp_nalp_test_on" \
-		"cora_glt_udl_test_on" \
-		"dblp_glt_test_on"
+		"cora_glt_udl_test_on"
 # Removed UDL due to transient issue:
 # "gigl/src/mocking/configs/e2e_udl_node_anchor_based_link_prediction_template_gbml_config.yaml"
 run_all_e2e_tests: task_config_uris_str:=\
 		"gigl/src/mocking/configs/e2e_node_anchor_based_link_prediction_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/e2e_supervised_node_classification_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/dblp_node_anchor_based_link_prediction_template_gbml_config.yaml" \
-		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml" \
-		"examples/distributed/configs/e2e_dblp_glt_task_config.yaml"
+		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml"
 run_all_e2e_tests: resource_config_uris_str:=\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
-		"deployment/configs/e2e_glt_resource_config.yaml"\
 		"deployment/configs/e2e_glt_resource_config.yaml"
 run_all_e2e_tests: should_compile_then_run_str:=\
 		"true" \
 		"false" \
 		"false" \
-		"false" \
 		"false"
 run_all_e2e_tests: _run_e2e_kfp_test
path: Makefile
user: login: kmontemayor2-sc
body: likewise?
created_at: 2025-05-22T20:38:57Z
author_association: COLLABORATOR
diff_hunk: @@ -487,9 +483,6 @@ stop_toaster:
 release_gigl:
 	@echo "This needs to be implemented"
 
-publish_docs:
-	@echo "This needs to be implemented"
-
path: Makefile
user: login: kmontemayor2-sc
body: I think this was brought up in another pr but why remove this?
created_at: 2025-05-22T20:39:24Z
author_association: COLLABORATOR
diff_hunk: @@ -8,20 +8,25 @@ DATE:=$(shell /bin/date "+%Y%m%d-%H%M")
 
 # GIT HASH, or empty string if not in a git repo.
 GIT_HASH?=$(shell git rev-parse HEAD 2>/dev/null || "")
+PWD=$(shell pwd)
path: Makefile
user: login: kmontemayor2-sc
body: nit. Is this only used once, any reason to not inline it?
created_at: 2025-05-22T20:39:32Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
+    gpu_limit: 1  # set to 0 for cpu training
path: deployment/configs/unittest_resource_config.yaml
user: login: kmontemayor2-sc
body: odd, somewhat weird that this worked at all then?
created_at: 2025-05-22T20:40:30Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
path: deployment/configs/unittest_resource_config.yaml
user: login: kmontemayor2-sc
body: should we also note that this should be removed for cpu training? what is the vai behavior when you give a gpu type and gpu_limit: 0?
created_at: 2025-05-22T20:41:18Z
author_association: COLLABORATOR
diff_hunk: @@ -6,47 +6,33 @@
 import gigl.common.utils.local_fs as local_fs
 from gigl.common import GcsUri, LocalUri, Uri
 from gigl.common.utils.gcs import GcsUtils
+from gigl.env.pipelines_config import get_resource_config
 from gigl.src.common.utils.file_loader import FileLoader
-from tests.test_assets.uri_constants import TEST_DATA_GCS_BUCKET
 
 
 class FileLoaderTest(unittest.TestCase):
     def setUp(self) -> None:
         self.file_loader = FileLoader()
         self.gcs_utils = GcsUtils()
         test_uuid = str(uuid.uuid4())
-        # TODO (svij): Refactor name and how the location of tests is derived
-        # Also need to investigate whether or not we need to clean up these assets
         self.test_asset_directory: LocalUri = LocalUri.join(".test_assets", test_uuid)
+        resource_config = get_resource_config()
         self.gcs_test_asset_directory: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, test_uuid
+            resource_config.shared_resource_config.common_compute_config.temp_assets_bucket,
+            test_uuid,
         )
 
-    # def test_local_temp_file(self):
-    #     local_file_path_src: LocalUri = LocalUri.join(
-    #         self.test_asset_directory, "test_local_temp_file.txt"
-    #     )
-
-    #     local_fs.remove_file_if_exist(local_path=local_file_path_src)
-
-    #     # Create files and ensure they exist
-    #     local_fs.create_empty_file_if_none_exists(local_path=local_file_path_src)
-    #     self.assertTrue(local_fs.does_path_exist(local_file_path_src))
-    #     with open(local_file_path_src.uri, "w") as f:
-    #         f.write("Hello")
-
-    #     temp_f = self.file_loader.load_to_temp_file(file_uri_src=local_file_path_src)
-    #     with open(temp_f.name, "r") as f:
-    #         msg = f.read()
-    #     self.assertTrue(msg == "Hello")
-    #     temp_f.close()
path: python/tests/integration/common/file_loader_test.py
user: login: kmontemayor2-sc
body: bump.
created_at: 2025-05-22T20:41:42Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: bump?
created_at: 2025-05-22T20:42:33Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: I think shells can function without a rc file right? Though I agree this is unusual and maybe throwing is fine.
created_at: 2025-05-22T20:43:22Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: ack thanks!
created_at: 2025-05-22T20:43:33Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: ```suggestion
        print(f"Command `{command}` failed with error: {result.stderr}")
```

nit?
created_at: 2025-05-22T20:44:11Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: ditto generally. 
created_at: 2025-05-22T20:44:30Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: did you mean to update this?
created_at: 2025-05-22T20:44:58Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
+
+    curr_datetime = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
+    curr_username = getpass.getuser()
+    resource_config_dest_path = f"{perm_assets_bucket}/{curr_username}/{curr_datetime}/gigl_test_default_resource_config.yaml"
path: scripts/bootstrap_resource_config.py
user: login: kmontemayor2-sc
body: makes sense, thanks for explaining :)
created_at: 2025-05-22T20:45:24Z
author_association: COLLABORATOR
diff_hunk: @@ -8,20 +8,25 @@ DATE:=$(shell /bin/date "+%Y%m%d-%H%M")
 
 # GIT HASH, or empty string if not in a git repo.
 GIT_HASH?=$(shell git rev-parse HEAD 2>/dev/null || "")
+PWD=$(shell pwd)
path: Makefile
user: login: svij-sc
body: It just seemed like a good thing to expose.
Since its a nit Ill keep it this way - I prefer it.
created_at: 2025-05-23T02:04:12Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
path: deployment/configs/unittest_resource_config.yaml
user: login: svij-sc
body: thanks updated
created_at: 2025-05-23T02:05:03Z
author_association: COLLABORATOR
diff_hunk: @@ -46,8 +46,8 @@ split_generator_config:
 trainer_config:
   vertex_ai_trainer_config:
     machine_type: "n1-highmem-8"
-    gpu_type: nvidia-tesla-k80
-    gpu_limit: 1
+    gpu_type: nvidia-tesla-p100
+    gpu_limit: 1  # set to 0 for cpu training
path: deployment/configs/unittest_resource_config.yaml
user: login: svij-sc
body: i dont think we used this every for orchestration so it was never called i.e. its used for individual integration tests.
Although i tried using it for orchestration and it failed.
created_at: 2025-05-23T02:05:47Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: ack addressed.
created_at: 2025-05-23T02:06:17Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: addressed
created_at: 2025-05-23T02:08:54Z
author_association: COLLABORATOR
diff_hunk: @@ -83,45 +89,45 @@ install_deps:
 generate_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong.
+# Can only be run on an arm64 mac, otherwise generated hashed req file will be wrong
 generate_dev_mac_arm64_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_darwin_arm64_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra experimental \
+	--extra torch25-cpu --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cpu_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cpu_requirements_unified.txt \
-	--extra torch25-cpu --extra transform --extra dev --extra experimental \
+	--extra torch25-cpu --extra transform --extra dev \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_linux_cuda_hashed_requirements:
 	pip-compile  -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra experimental \
+	--extra torch25-cuda-121 --extra transform \
 	./python/pyproject.toml
 
-# Can only be run on linux, otherwise generated hashed req file will be wrong.
+# Can only be run on linux, otherwise generated hashed req file will be wrong
 generate_dev_linux_cuda_hashed_requirements:
 	pip-compile -v --allow-unsafe --generate-hashes --no-emit-index-url --resolver=backtracking \
 	--output-file=requirements/dev_linux_cuda_requirements_unified.txt \
-	--extra torch25-cuda-121 --extra transform --extra dev --extra experimental \
+	--extra torch25-cuda-121 --extra transform --extra dev \
 	./python/pyproject.toml
 
-# These are a collection of tests that are run before anything is installed using tools available on host.
+# These are a collection of tests that are run before anything is installed using tools abialable on host.
path: Makefile
user: login: svij-sc
body: sorry I didnt push when I left comments - it should be addressed now.
created_at: 2025-05-23T02:09:20Z
author_association: COLLABORATOR
diff_hunk: @@ -368,27 +368,23 @@ run_all_e2e_tests: job_name_prefixes_str:=\
 		"cora_nalp_test_on" \
 		"cora_snc_test_on" \
 		"dblp_nalp_test_on" \
-		"cora_glt_udl_test_on" \
-		"dblp_glt_test_on"
+		"cora_glt_udl_test_on"
 # Removed UDL due to transient issue:
 # "gigl/src/mocking/configs/e2e_udl_node_anchor_based_link_prediction_template_gbml_config.yaml"
 run_all_e2e_tests: task_config_uris_str:=\
 		"gigl/src/mocking/configs/e2e_node_anchor_based_link_prediction_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/e2e_supervised_node_classification_template_gbml_config.yaml" \
 		"gigl/src/mocking/configs/dblp_node_anchor_based_link_prediction_template_gbml_config.yaml" \
-		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml" \
-		"examples/distributed/configs/e2e_dblp_glt_task_config.yaml"
+		"examples/distributed/configs/e2e_cora_udl_glt_task_config.yaml"
 run_all_e2e_tests: resource_config_uris_str:=\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
 		"deployment/configs/e2e_cicd_resource_config.yaml"\
-		"deployment/configs/e2e_glt_resource_config.yaml"\
 		"deployment/configs/e2e_glt_resource_config.yaml"
 run_all_e2e_tests: should_compile_then_run_str:=\
 		"true" \
 		"false" \
 		"false" \
-		"false" \
 		"false"
 run_all_e2e_tests: _run_e2e_kfp_test
path: Makefile
user: login: svij-sc
body: sorry I didnt push when I left comments - it should be addressed now.
created_at: 2025-05-23T02:09:29Z
author_association: COLLABORATOR
diff_hunk: @@ -6,47 +6,33 @@
 import gigl.common.utils.local_fs as local_fs
 from gigl.common import GcsUri, LocalUri, Uri
 from gigl.common.utils.gcs import GcsUtils
+from gigl.env.pipelines_config import get_resource_config
 from gigl.src.common.utils.file_loader import FileLoader
-from tests.test_assets.uri_constants import TEST_DATA_GCS_BUCKET
 
 
 class FileLoaderTest(unittest.TestCase):
     def setUp(self) -> None:
         self.file_loader = FileLoader()
         self.gcs_utils = GcsUtils()
         test_uuid = str(uuid.uuid4())
-        # TODO (svij): Refactor name and how the location of tests is derived
-        # Also need to investigate whether or not we need to clean up these assets
         self.test_asset_directory: LocalUri = LocalUri.join(".test_assets", test_uuid)
+        resource_config = get_resource_config()
         self.gcs_test_asset_directory: GcsUri = GcsUri.join(
-            TEST_DATA_GCS_BUCKET, test_uuid
+            resource_config.shared_resource_config.common_compute_config.temp_assets_bucket,
+            test_uuid,
         )
 
-    # def test_local_temp_file(self):
-    #     local_file_path_src: LocalUri = LocalUri.join(
-    #         self.test_asset_directory, "test_local_temp_file.txt"
-    #     )
-
-    #     local_fs.remove_file_if_exist(local_path=local_file_path_src)
-
-    #     # Create files and ensure they exist
-    #     local_fs.create_empty_file_if_none_exists(local_path=local_file_path_src)
-    #     self.assertTrue(local_fs.does_path_exist(local_file_path_src))
-    #     with open(local_file_path_src.uri, "w") as f:
-    #         f.write("Hello")
-
-    #     temp_f = self.file_loader.load_to_temp_file(file_uri_src=local_file_path_src)
-    #     with open(temp_f.name, "r") as f:
-    #         msg = f.read()
-    #     self.assertTrue(msg == "Hello")
-    #     temp_f.close()
path: python/tests/integration/common/file_loader_test.py
user: login: svij-sc
body: sorry I didnt push when I left comments - I have re-enabled the test
created_at: 2025-05-23T02:09:34Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: addressed
created_at: 2025-05-23T02:09:59Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,244 @@
+import datetime
+import getpass
+import os
+import pathlib
+import subprocess
+import tempfile
+
+import yaml
+
+from gigl.common import UriFactory
+from gigl.src.common.utils.file_loader import FileLoader
+
+GIGL_ROOT_DIR = pathlib.Path(__file__).resolve().parent.parent
+TEMPLATE_SOURCE_RESOURCE_CONFIG = (
+    GIGL_ROOT_DIR / "deployment" / "configs" / "unittest_resource_config.yaml"
+)
+
+
+def infer_shell_file() -> str:
+    """Infers the user's default shell configuration file."""
+    shell = os.environ.get("SHELL", "")
+    shell_config_map = {
+        "zsh": "~/.zshrc",
+        "bash": "~/.bashrc",
+    }
+
+    for key, config_file in shell_config_map.items():
+        if key in shell:
+            print(f"Detected shell: {key}. Using config file: {config_file}")
+            return config_file
+
+    print(
+        "Could not infer the default shell. Please specify the shell configuration file manually."
+    )
+    return input(
+        "Enter the path to your shell configuration file (e.g., ~/.bashrc): "
+    ).strip()
+
+
+def update_shell_config(
+    shell_config_path,
+    gigl_test_default_resource_config,
+    gigl_project,
+    gigl_docker_artifact_registry_path,
+):
+    """Updates the shell configuration file with the environment variables in an idempotent way."""
+    shell_config_path = os.path.expanduser(shell_config_path)
+    start_marker = "# ====== GiGL ENV Config - Begin ====="
+    end_marker = "# ====== GiGL ENV Config - End ====="
+    export_lines = [
+        start_marker + "\n",
+        "# This section is auto-generated by the GiGL bootstrap_resource_config.py script.\n",
+        f'export GIGL_TEST_DEFAULT_RESOURCE_CONFIG="{gigl_test_default_resource_config}"\n',
+        f'export GIGL_PROJECT="{gigl_project}"\n',
+        f'export GIGL_DOCKER_ARTIFACT_REGISTRY="{gigl_docker_artifact_registry_path}"\n',
+        end_marker + "\n",
+    ]
+
+    # Read the existing shell config file
+    if not os.path.exists(shell_config_path):
+        raise FileNotFoundError(
+            f"Shell config file '{shell_config_path}' does not exist."
+        )
+    shell_config_lines: list[str]
+    with open(shell_config_path, "r") as shell_config:
+        shell_config_lines = shell_config.readlines()
+
+    inside_block = False
+    updated_shell_config_lines = []
+    for line in shell_config_lines:
+        if line.strip().startswith(start_marker):
+            inside_block = True
+        elif line.strip().startswith(end_marker):
+            inside_block = False
+            continue
+        if not inside_block:
+            updated_shell_config_lines.append(line)
+
+    # Add the new GiGL config block
+    updated_shell_config_lines.extend(export_lines)
+
+    # Write back to the shell config file
+    with open(shell_config_path, "w") as shell_config:
+        shell_config.writelines(updated_shell_config_lines)
+
+    print(f"Updated {shell_config_path} with:\n{export_lines}.")
+
+
+def assert_gcp_project_exists(project_id: str):
+    command = f"gcloud projects describe {project_id}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+
+    print(result.stdout)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"Project '{project_id}' does not exist or you do not have access to it."
+        )
+
+
+def assert_bq_dataset_exists(dataset_name: str, project: str):
+    command = f"bq show --project_id {project} {dataset_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"BigQuery dataset '{dataset_name}' does not exist in project '{project}' or you do not have access to it."
+        )
+    print(f"BigQuery dataset '{dataset_name}' exists.")
+
+
+def assert_gcs_bucket_exists(bucket_name: str):
+    assert bucket_name.startswith("gs://"), "Bucket name must start with 'gs://'"
+    command = f"gsutil ls {bucket_name}"
+    result = subprocess.run(command, shell=True, capture_output=True, text=True)
+    if result.returncode != 0:
+        print(f"Command failed with error: {result.stderr}")
+        raise ValueError(
+            f"GCS bucket '{bucket_name}' does not exist or you do not have access to it."
+        )
+    print(f"GCS bucket '{bucket_name}' exists.")
+
+
+if __name__ == "__main__":
+    print("Welcome to the GiGL Cloud Environment Configuration Script!")
+    print("This script will help you set up your cloud environment for GiGL.")
+    print(
+        "Before running this script, please ensure you have followed the GiGL Cloud Setup Guide:"
+    )
+    print(
+        "https://snapchat.github.io/GiGL/docs/user_guide/getting_started/cloud_setup_guide"
+    )
+    print("======================================================")
+
+    print(
+        f"We will use `{TEMPLATE_SOURCE_RESOURCE_CONFIG}` as the template for your resource config YAML file."
+    )
+
+    # Default values
+    print("Please provide us Project information.")
+    project: str = input("GCP Project name that you are planning on using: ").strip()
+    assert_gcp_project_exists(project)
+    region: str = input(
+        f"The GCP region where you created your resources i.e. `us-central1`: "
+    ).strip()
+    assert region, "Region cannot be empty"
+    gcp_service_account_email: str = input("The GCP Service account: ").strip()
+    assert gcp_service_account_email, "GCP Service account cannot be empty"
+    docker_artifact_registry_path: str = input(
+        f"The Docker Artifact Registry path where your source code images will be stored i.e. `us-central1-docker.pkg.dev/YOUR_PROJECT_NAME/YOUR_REPO_NAME`: "
+    ).strip()
+    assert (
+        docker_artifact_registry_path
+    ), "Docker Artifact Registry path cannot be empty"
+
+    print("\n\nPlease provide us the information on the BQ Datasets you want to use")
+    temp_assets_bq_dataset_name: str = input(
+        "`temp_assets_bq_dataset_name` - Dataset name used for temporary assets: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=temp_assets_bq_dataset_name, project=project)
+    embedding_bq_dataset_name: str = input(
+        f"`embedding_bq_dataset_name` - Dataset used of output embeddings: "
+    ).strip()
+    assert_bq_dataset_exists(dataset_name=embedding_bq_dataset_name, project=project)
+
+    print("\n\nPlease provide us the information on the GCS Buckets you want to use")
+    temp_assets_bucket: str = input(
+        f"`temp_assets_bucket` - GCS Bucket for storing temporary assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=temp_assets_bucket)
+    perm_assets_bucket: str = input(
+        f"`perm_assets_bucket` - GCS Bucket for storing permanent assets i.e. `gs://YOUR_BUCKET_NAME`: "
+    ).strip()
+    assert_gcs_bucket_exists(bucket_name=perm_assets_bucket)
path: scripts/bootstrap_resource_config.py
user: login: svij-sc
body: sorry I didnt push when I left comments. We now support args being passed into the script at runtime
created_at: 2025-05-23T02:10:23Z
author_association: COLLABORATOR

PR comments:
user: login: svij-sc
created_at: 2025-05-23T02:13:27Z
body: /unit_test
author_association: COLLABORATOR
user: login: svij-sc
created_at: 2025-05-23T02:13:32Z
body: /integration_test
author_association: COLLABORATOR
user: login: github-actions[bot]
created_at: 2025-05-23T02:13:42Z
body: # GiGL Automation

@ _02:13:42UTC_ : 🔄 `Unit Test` [started](https://github.com/Snapchat/GiGL/actions/runs/15200774845).

@ _02:37:10UTC_ : ✅ Workflow completed successfully.

author_association: CONTRIBUTOR
user: login: github-actions[bot]
created_at: 2025-05-23T02:13:44Z
body: # GiGL Automation

@ _02:13:44UTC_ : 🔄 `Integration Test` [started](https://github.com/Snapchat/GiGL/actions/runs/15200775689).

@ _03:14:10UTC_ : ✅ Workflow completed successfully.

author_association: CONTRIBUTOR