Repository: UKGovernmentBEIS/inspect_ai
Issue: Is there an implementation for Streamable HTTP mcp servers?
Root cause: Resource Mishandling

## Current behavior:
We can only create clients for remote sse mcp servers and not the newer streamable-http endpoints

## Desired behavior:
Create client for streamablehttp mcps

## Steps I tried to make  it work:
I noticed an implementation of streamable http in mcp/client/streamable_http.py, so I just propagated that up to inspect_ai.tool module basically copying the mcp_server_sse interface

## /tool/_mcp/_mcp.py:
```python
def create_server_streamablehttp(
        url:str,
        headers: dict[str, Any] | None = None,
        timeout: float = 5,
        sse_read_timeout: float = 60 * 5,
)->MCPServer:
    return MCPServerImpl(
        lambda: streamablehttp_client(url=url, headers=headers, timeout=timeout, sse_read_timeout=sse_read_timeout),
        name=url,
        events=True
    )

```


## tool/mcp/server.py
```python
def mcp_server_streamablehttp(
            *,
    url: str,
    headers: dict[str, Any] | None = None,
    timeout: float = 5,
    sse_read_timeout: float = 60 * 5,
) -> MCPServer:
    """MCP Server (Streamable HTTP).

   Streamable HTTP interface to MCP server.  Use this for MCP servers available via a URL endpoint.

    Args:
        url: URL to remote server
        headers: Headers to send server (typically authorization is included here)
        timeout: Timeout for HTTP operations
        sse_read_timeout: How long (in seconds) the client will wait for a new
            event before disconnecting.

    Returns:
        McpClient: Client for MCP Server
    """
    verfify_mcp_package()
    from ._mcp import create_server_streamablehttp

    return create_server_streamablehttp(url, headers, timeout, sse_read_timeout)

```

However, this does not work for some reason and the following error happens:

```bash
INFO:     Started server process [29624]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     127.0.0.1:56454 - "POST /schedule-eval HTTP/1.1" 200 OK
[+] Pulling 1/1
 ✔ default Pulled                                                                                                   1.4s 

╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│task (1 sample): none/none                                                                                             │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭───────────────────────────────────────── Traceback (most recent call last) ──────────────────────────────────────────╮ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_eval/task/run.py:371 in   │ 
│ task_run                                                                                                             │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_util/_async.py:74 in      │ 
│ tg_collect                                                                                                           │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_util/_async.py:62 in      │ 
│ run_task                                                                                                             │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_eval/task/run.py:367 in   │ 
│ run_sample                                                                                                           │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_eval/task/run.py:330 in   │ 
│ run                                                                                                                  │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_eval/task/run.py:934 in   │ 
│ task_run_sample                                                                                                      │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/_eval/task/run.py:697 in   │ 
│ task_run_sample                                                                                                      │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/inspect_ai/util/_limit.py:583 in      │ 
│ __exit__                                                                                                             │ 
│                                                                                                                      │ 
│ /home/michaelwaves/repos//backend/.venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py:465 in    │ 
│ __exit__                                                                                                             │ 
│                                                                                                                      │ 
│    462 │   │   assert self._host_task is not None                                                                    │ 
│    463 │   │   host_task_state = _task_states.get(self._host_task)                                                   │ 
│    464 │   │   if host_task_state is None or host_task_state.cancel_scope is not self:                               │ 
│ ❱  465 │   │   │   raise RuntimeError(                                                                               │ 
│    466 │   │   │   │   "Attempted to exit a cancel scope that isn't the current tasks's "                            │ 
│    467 │   │   │   │   "current cancel scope"                                                                        │ 
│    468 │   │   │   )                                                                                                 │ 
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ 
RuntimeError: Attempted to exit a cancel scope that isn't the current tasks's current cancel scope                       
                                                                                                                         
Task interrupted (no samples completed before interruption)                                                              
                                                                                                                         
Log: logs/10/2025-08-02T02-11-44-04-00_task_2STNLhRTsBAqRgQSCsabro.json                                                  
                                                                                                                         
[08/02/25 02:11:58] ERROR    an error occurred during closing of asynchronous generator               base_events.py:1785
                             <async_generator object streamablehttp_client at 0x7f2798204640>                            
                             asyncgen: <async_generator object streamablehttp_client at                                  
                             0x7f2798204640>                                                                             
                               + Exception Group Traceback (most recent call last):                                      
                               |   File                                                                                  
                             "/home/michaelwaves/repos//backend/.venv/lib/python3.11/site-p                    
                             ackages/anyio/_backends/_asyncio.py", line 772, in __aexit__                                
                               |     raise BaseExceptionGroup(                                                           
                               | BaseExceptionGroup: unhandled errors in a TaskGroup (1                                  
                             sub-exception)                                                                              
                               +-+---------------- 1 ----------------                                                    
                                 | Traceback (most recent call last):                                                    
                                 |   File                                                                                
                             "/home/michaelwaves/repos//backend/.venv/lib/python3.11/site-p                    
                             ackages/mcp/client/streamable_http.py", line 498, in                                        
                             streamablehttp_client                                                                       
                                 |     yield (                                                                           
                                 | GeneratorExit                                                                         
                                 +------------------------------------                                                   
                                                                                                                         
                             During handling of the above exception, another exception occurred:                         
                                                                                                                         
                             Traceback (most recent call last):                                                          
                               File                                                                                      
                             "/home/michaelwaves/repos//backend/.venv/lib/python3.11/site-p                    
                             ackages/mcp/client/streamable_http.py", line 474, in                                        
                             streamablehttp_client                                                                       
                                 async with anyio.create_task_group() as tg:                                             
                               File                                                                                      
                             "/home/michaelwaves/repos//backend/.venv/lib/python3.11/site-p                    
                             ackages/anyio/_backends/_asyncio.py", line 778, in __aexit__                                
                                 if self.cancel_scope.__exit__(type(exc), exc, exc.__traceback__):                       
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                        
                               File                                                                                      
                             "/home/michaelwaves/repos//backend/.venv/lib/python3.11/site-p                    
                             ackages/anyio/_backends/_asyncio.py", line 457, in __exit__                                 
                                 raise RuntimeError(                                                                     
                             RuntimeError: Attempted to exit cancel scope in a different task than it                    
                             was entered in                                                                              
```

Is there some fundamental architectural reason streamable http client does not work yet? Any pointers to implement this functionality? Or maybe it's just because I'm running with fastapi as well?

@epatey Could you take a look at this?

We have added this feature here: https://github.com/UKGovernmentBEIS/inspect_ai/pull/2221

Note that our simple test for streamablehttp passes. The failure you are seeing is likely a failure to connect with the underlying streamable http server -- you should make sure it is running and that you have the right URL, etc.