Repository: UKGovernmentBEIS/inspect_ai
Issue: HttpResponseError on Azure
Root cause: Environment Incompatibility

Not sure if this should go here on on the inspect_evals repo.

When running an eval on Azure, I get 
```
HttpResponseError: Operation returned an invalid status 'Failed Dependency'
Content: no healthy upstream

Task interrupted (40 of 120,320 total samples logged before interruption). Resume task with:

inspect eval-retry logs/2025-03-27T16-56-42+01-00_mmlu-pro_G33nxPiq2SfKtid3h9ocW4.eval                                                                                                                                                                                          

Log: logs/2025-03-27T16-56-42+01-00_mmlu-pro_G33nxPiq2SfKtid3h9ocW4.eval
```
Full stacktrace below. This happens with different model deployments and benchmarks. If I relaunch the task with the command provided, the error appears again. It does not seem to be related to the stage of the evaluation.

What could it be related to? Is is related to the Azure infrastructure setup?

In my `.env` file I set
```
AZURE_API_KEY="key"
AZUREAI_OPENAI_API_KEY="key"
INSPECT_EVAL_MODEL_API_KEY="key"
 
INSPECT_EVAL_MODEL_BASE_URL="url"
AZUREAI_OPENAI_BASE_URL="url"
 
AZUREAI_BASE_URL="url"
AZUREAI_API_KEY="key"
INSPECT_EVAL_MODEL="model"
```

The full stacktrace is
```
│ ╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_util/_async.py:57 in tg_collect                     │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py:767 in __aexit__                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │    764 │   │   │   │                                                                                                 │                                                                                                                                                        │
│ │    765 │   │   │   │   self._active = False                                                                          │                                                                                                                                                        │
│ │    766 │   │   │   │   if self._exceptions:                                                                          │                                                                                                                                                        │
│ │ ❱  767 │   │   │   │   │   raise BaseExceptionGroup(                                                                 │                                                                                                                                                        │
│ │    768 │   │   │   │   │   │   "unhandled errors in a TaskGroup", self._exceptions                                   │                                                                                                                                                        │
│ │    769 │   │   │   │   │   )                                                                                         │                                                                                                                                                        │
│ │    770 │   │   │   │   elif exc_val:                                                                                 │                                                                                                                                                        │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯                                                                                                                                                        │
│ ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)                                                                                                                                                                                                               │
│                                                                                                                                                                                                                                                                                 │
│ During handling of the above exception, another exception occurred:                                                                                                                                                                                                             │
│                                                                                                                                                                                                                                                                                 │
│ ╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_eval/task/run.py:302 in task_run                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_util/_async.py:72 in tg_collect                     │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_util/_async.py:60 in run_task                       │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_eval/task/run.py:809 in task_run_sample             │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_eval/task/run.py:630 in task_run_sample             │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/solver/_plan.py:106 in __call__                      │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/solver/_multiple_choice.py:302 in solve              │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_eval/task/run.py:249 in generate                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/_eval/task/generate.py:33 in task_generate           │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_model.py:346 in generate                      │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_model.py:564 in _generate                     │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/asyncio/__init__.py:189 in async_wrapped               │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/asyncio/__init__.py:111 in __call__                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/asyncio/__init__.py:153 in iter                        │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/_utils.py:99 in inner                                  │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/__init__.py:398 in <lambda>                            │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /usr/lib/python3.10/concurrent/futures/_base.py:451 in result                                                        │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │   448 │   │   │   │   if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]:                                         │                                                                                                                                                        │
│ │   449 │   │   │   │   │   raise CancelledError()                                                                     │                                                                                                                                                        │
│ │   450 │   │   │   │   elif self._state == FINISHED:                                                                  │                                                                                                                                                        │
│ │ ❱ 451 │   │   │   │   │   return self.__get_result()                                                                 │                                                                                                                                                        │
│ │   452 │   │   │   │                                                                                                  │                                                                                                                                                        │
│ │   453 │   │   │   │   self._condition.wait(timeout)                                                                  │                                                                                                                                                        │
│ │   454                                                                                                                │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /usr/lib/python3.10/concurrent/futures/_base.py:403 in __get_result                                                  │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │   400 │   def __get_result(self):                                                                                    │                                                                                                                                                        │
│ │   401 │   │   if self._exception:                                                                                    │                                                                                                                                                        │
│ │   402 │   │   │   try:                                                                                               │                                                                                                                                                        │
│ │ ❱ 403 │   │   │   │   raise self._exception                                                                          │                                                                                                                                                        │
│ │   404 │   │   │   finally:                                                                                           │                                                                                                                                                        │
│ │   405 │   │   │   │   # Break a reference cycle with the exception in self._exception                                │                                                                                                                                                        │
│ │   406 │   │   │   │   self = None                                                                                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/tenacity/asyncio/__init__.py:114 in __call__                    │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_model.py:504 in generate                      │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_providers/azureai.py:203 in generate          │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_providers/azureai.py:278 in                   │                                                                                                                                                        │
│ │ handle_azure_error                                                                                                   │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/inspect_ai/model/_providers/azureai.py:191 in generate          │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │ /home/franzia/test_venv/lib/python3.10/site-packages/azure/ai/inference/aio/_patch.py:676 in complete                │                                                                                                                                                        │
│ │                                                                                                                      │                                                                                                                                                        │
│ │    673 │   │   │   if _stream:                                                                                       │                                                                                                                                                        │
│ │    674 │   │   │   │   await response.read()  # Load the body in memory and close the socket                         │                                                                                                                                                        │
│ │    675 │   │   │   map_error(status_code=response.status_code, response=response, error_map=err                      │                                                                                                                                                        │
│ │ ❱  676 │   │   │   raise HttpResponseError(response=response)                                                        │                                                                                                                                                        │
│ │    677 │   │                                                                                                         │                                                                                                                                                        │
│ │    678 │   │   if _stream:                                                                                           │                                                                                                                                                        │
│ │    679 │   │   │   return _models.AsyncStreamingChatCompletions(response)                                            │                                                                                                                                                        │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ 
```


I think there is something amiss with your Azure model deployment. All of the following currently work for me on `main` and PyPI latest:

```bash
export AZUREAI_API_KEY=<key>
export AZUREAI_BASE_URL=<base-url>
inspect eval popularity.py --model azureai/Llama-3.3-70B-Instruct
```

```bash
export AZUREAI_OPENAI_API_KEY=<key>
export AZUREAI_OPENAI_BASE_URL=<base-url>
inspect eval popularity.py --model openai/azure/gpt-4o-mini
```

```bash
export AZUREAI_MISTRAL_API_KEY=<key>
AZUREAI_MISTRAL_BASE_URL=<base-url>
export inspect eval popularity.py --model mistral/azure/Mistral-Large-2411
```

Note that as-per the [AzureAI docs](https://inspect.aisi.org.uk/providers.html#azure-ai) these models are all deployed on [Azure AI Foundry](https://ai.azure.com/) which is their new service that replaces various other legacy services.

I've been running some tests on different Azure servers, it seems to be working fine. Thanks.