Repository: BerriAI/litellm

Issue:
[Bug]: Vertex AI context caching fails with "global" location - 404 Not Found
### What happened?

## Bug Description

When using `vertex_location: "global"` with Vertex AI context caching (cachedContents API), LiteLLM generates an incorrect API endpoint URL, resulting in a 404 Not Found error.

## Current Behavior

**Error**:
```
litellm.NotFoundError: Vertex_aiException - 404 Not Found
The requested URL /v1/projects/project-03/locations/global/cachedContents was not found
```

**Incorrect URL generated by LiteLLM**:
- Settings logs to verbose we can see that
```
https://global-aiplatform.googleapis.com/v1/projects/project-03/locations/global/cachedContents
         ^^^^^^
       ❌ This domain doesn't exist
```

**Correct URL should be**:
```
https://aiplatform.googleapis.com/v1/projects/project-03/locations/global/cachedContents
    (no location prefix for "global")
```

## Expected Behavior

When `vertex_location` is set to `"global"`, LiteLLM should construct the URL without the location prefix, matching Google's global endpoint format.

## Comparison with Regular Completion

**Regular completion** (without cache_control) **works correctly**:
```python
# ✅ Correct URL for generateContent with "global"
POST https://aiplatform.googleapis.com/v1/projects/project-03/locations/global/publishers/google/models/gemini-2.5-flash:generateContent
```

**Context caching** (with cache_control) **fails**:
```python
# ❌ Incorrect URL for cachedContents with "global"
POST https://global-aiplatform.googleapis.com/v1/projects/ts-dev-03/locations/global/cachedContents
```

## Reproduction Steps

```python
from litellm import Router

model_list = [{
    "model": "vertex_ai/gemini-2.5-flash",
    "vertex_credentials": '{"type": "service_account", ...}',
    "vertex_location": "global",  # Using global location
    "vertex_project": "your-project-id",
}]

router = Router(model_list=model_list)

# This works fine (regular completion)
response = router.completion(
    model="gemini-2.5-flash",
    messages=[{"role": "user", "content": "Hi"}]
)
print("✅ Regular completion works!")

# This fails with 404 (context caching)
response = router.completion(
    model="gemini-2.5-flash",
    messages=[{
        "role": "system",
        "content": [{
            "type": "text",
            "text": "Long system prompt",
            "cache_control": {"type": "ephemeral"}  # Triggers caching
        }]
    }, {
        "role": "user",
        "content": "Hello"
    }]
)
# ❌ Fails with 404 Not Found
```

## Environment

- **LiteLLM Version**: 1.79.3
- **Python Version**: 3.12
- **Provider**: Vertex AI (GCP)
- **Model**: gemini-2.5-flash
- **Location**: global

## Root Cause

**File**: `litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py`

**Method**: `_get_token_and_url_context_caching()`

**Issue**: The URL construction doesn't handle "global" location specially:

```python
# Current (buggy) code:
elif custom_llm_provider == "vertex_ai":
    url = f"https://{vertex_location}-aiplatform.googleapis.com/v1/..."
    # When vertex_location="global", this creates: global-aiplatform.googleapis.com (404)
```

**Note**: Regular completion calls (in `vertex_llm_base.py`) already handle "global" correctly, but context caching doesn't.

## Proposed Fix

Add conditional check for "global" location:

```python
elif custom_llm_provider == "vertex_ai":
    if vertex_location == "global":
        # Global endpoint without location prefix
        url = f"https://aiplatform.googleapis.com/v1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
    else:
        # Regional endpoints with location prefix
        url = f"https://{vertex_location}-aiplatform.googleapis.com/v1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
```

## Related Issues

- #11190 - Similar fix needed for non-google models (fixed in PR #11194)
- #9234 - Global region support request
- PR #10658 - Fixed global endpoint for regular completion

## Additional Context

According to [Google's documentation](https://cloud.google.com/vertex-ai/generative-ai/docs/context-cache/context-cache-create), context caching supports the global endpoint. LiteLLM's regular completion already handles this correctly, but context caching module missed this fix.

### Relevant log output

```shell
NotFoundError: litellm.NotFoundError: Vertex_aiException - <!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 404 (Not Found)!!1</title>
  <style>
    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}@media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>404.</b> <ins>That’s an error.</ins>
  <p>The requested URL <code>/v1/projects/project-03/locations/global/cachedContents</code> was not found on this server.  <ins>That’s all we know.</ins>
. Received Model Group=gemini-2.5-flash
Available Model Group Fallbacks=None
```

### Are you a ML Ops Team?

Yes

### What LiteLLM version are you on ?

1.80.5

### Twitter / LinkedIn details

https://www.linkedin.com/in/prawaan-singh-1965b2104/

Discussion:

Fix (patch):
diff --git a/litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py b/litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py
index 70b068b5a4d9..26be4d3c2b88 100644
--- a/litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py
+++ b/litellm/llms/vertex_ai/context_caching/vertex_ai_context_caching.py
@@ -64,11 +64,17 @@ def _get_token_and_url_context_caching(
         elif custom_llm_provider == "vertex_ai":
             auth_header = vertex_auth_header
             endpoint = "cachedContents"
-            url = f"https://{vertex_location}-aiplatform.googleapis.com/v1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
+            if vertex_location == "global":
+                url = f"https://aiplatform.googleapis.com/v1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
+            else:
+                url = f"https://{vertex_location}-aiplatform.googleapis.com/v1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
         else:
             auth_header = vertex_auth_header
             endpoint = "cachedContents"
-            url = f"https://{vertex_location}-aiplatform.googleapis.com/v1beta1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
+            if vertex_location == "global":
+                url = f"https://aiplatform.googleapis.com/v1beta1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
+            else:
+                url = f"https://{vertex_location}-aiplatform.googleapis.com/v1beta1/projects/{vertex_project}/locations/{vertex_location}/{endpoint}"
 
 
         return self._check_custom_proxy(


Test changes:
diff --git a/tests/test_litellm/llms/vertex_ai/context_caching/test_vertex_ai_context_caching.py b/tests/test_litellm/llms/vertex_ai/context_caching/test_vertex_ai_context_caching.py
index 0320092c7a3d..88d1b59c5b55 100644
--- a/tests/test_litellm/llms/vertex_ai/context_caching/test_vertex_ai_context_caching.py
+++ b/tests/test_litellm/llms/vertex_ai/context_caching/test_vertex_ai_context_caching.py
@@ -785,3 +785,64 @@ async def test_async_check_and_create_cache_tools_popped_from_optional_params(
 
             # But original tools should still be available for comparison
             assert original_tools == self.sample_tools
+
+
+class TestVertexAIGlobalLocation:
+    """Test global location handling in context caching."""
+
+    def test_global_location_url_construction_v1(self):
+        """Test that global location uses correct URL (no location prefix) for v1 API."""
+        caching = ContextCachingEndpoints()
+
+        # Mock the _check_custom_proxy to return the URL unchanged
+        with patch.object(caching, '_check_custom_proxy', side_effect=lambda **kwargs: (kwargs.get('auth_header'), kwargs.get('url'))):
+            auth_header, url = caching._get_token_and_url_context_caching(
+                gemini_api_key=None,
+                custom_llm_provider="vertex_ai",
+                api_base=None,
+                vertex_project="test-project",
+                vertex_location="global",
+                vertex_auth_header="Bearer test-token",
+            )
+
+            # Assert correct URL format for global
+            expected_url = "https://aiplatform.googleapis.com/v1/projects/test-project/locations/global/cachedContents"
+            assert url == expected_url, f"Expected {expected_url}, got {url}"
+            assert "global-aiplatform" not in url, "URL should not contain 'global-aiplatform' prefix"
+
+    def test_regional_location_url_construction_v1(self):
+        """Test that regional location uses correct URL (with location prefix) for v1 API."""
+        caching = ContextCachingEndpoints()
+
+        with patch.object(caching, '_check_custom_proxy', side_effect=lambda **kwargs: (kwargs.get('auth_header'), kwargs.get('url'))):
+            auth_header, url = caching._get_token_and_url_context_caching(
+                gemini_api_key=None,
+                custom_llm_provider="vertex_ai",
+                api_base=None,
+                vertex_project="test-project",
+                vertex_location="us-central1",
+                vertex_auth_header="Bearer test-token",
+            )
+
+            # Assert correct URL format for regional
+            expected_url = "https://us-central1-aiplatform.googleapis.com/v1/projects/test-project/locations/us-central1/cachedContents"
+            assert url == expected_url, f"Expected {expected_url}, got {url}"
+
+    def test_global_location_url_construction_v1beta1(self):
+        """Test that global location uses correct URL for v1beta1 API."""
+        caching = ContextCachingEndpoints()
+
+        with patch.object(caching, '_check_custom_proxy', side_effect=lambda **kwargs: (kwargs.get('auth_header'), kwargs.get('url'))):
+            auth_header, url = caching._get_token_and_url_context_caching(
+                gemini_api_key=None,
+                custom_llm_provider="vertex_ai_beta",
+                api_base=None,
+                vertex_project="test-project",
+                vertex_location="global",
+                vertex_auth_header="Bearer test-token",
+            )
+
+            # Assert correct URL format for global with beta API
+            expected_url = "https://aiplatform.googleapis.com/v1beta1/projects/test-project/locations/global/cachedContents"
+            assert url == expected_url, f"Expected {expected_url}, got {url}"
+            assert "global-aiplatform" not in url, "URL should not contain 'global-aiplatform' prefix"
\ No newline at end of file