Repository: UKGovernmentBEIS/inspect_ai
Issue: Allow specifying parallel_tool_calls=False in ToolDef
Root cause: Unimplemented Feature Gap

While the _execution_ of tools in parallel is available via `ToolDef.parallel`, actually specifying that the model itself is not allowed to call a tool in parallel (i.e. multiple times in a single turn) like https://platform.openai.com/docs/guides/function-calling/configuring-parallel-function-calling is not currently available.

As a concrete example, if a model is given a `select_option` tool, we only allow it to call `select_option` once per response.

For example, an implementation using a hypothetical `ToolDef.multiple_tool_calls_per_response_allowed` in `call_model` (name arbitrarily chosen for illustration).

```python
async def call_model(state: TaskState, tools: Sequence[ToolDef], no_tool_call_message: str | None = None) -> TaskState:
    tool_infos = [def_to_info(tool_def) for tool_def in tools]
    names_to_tools = {tool_def.name: tool_def for tool_def in tools}

    state.output = await get_model().generate(state.messages, tools=tool_infos)
    state.messages.append(state.output.message)

    if state.output.message.tool_calls is None:
        if no_tool_call_message is not None:
            # Anthropic models do not allow 2 consecutive assistant messages
            message: ChatMessage = ChatMessageUser(content=no_tool_call_message)
            state.messages.append(message)
        return state

    tool_calls: list[ToolCall] = state.output.message.tool_calls

    tool_call_count_per_tool = collections.Counter(tool_call.function for tool_call in tool_calls)

    for tool_call in tool_calls:
        tool = names_to_tools[tool_call.function]

        # check for disallowed parallel tool calls, creating one error per tool call in case there's
        # apis which enforce that each tool call gets a response
        if not tool.multiple_tool_calls_per_response_allowed and tool_call_count_per_tool[tool_call.function] > 1:
            content = (
                f"Only one tool call at a time allowed for {tool.name} but "
                f"{tool_call_count_per_tool[tool_call.function]} found"
            )
            tool_call_error = ToolCallError(type="parsing", message=content)
            message = ChatMessageTool(
                tool_call_id=tool_call.id, function=tool.name, content=content, error=tool_call_error
            )
            state.messages.append(message)
            continue
```

We actually had this but took it out (thinking that our making the tool calls sequential was enough). But if you think we should we can bring this code back:

https://github.com/UKGovernmentBEIS/inspect_ai/commit/c2c9190a9aa6844a0aa82597c3bd94e936cfd527#diff-7c1c5a97261e2f87c1c3fd17c1cd3f6e68e66a1fb26e5701ed02b7f395866bbbL229

Does that cover this concern? 

I think that would cover it. There might be a requirement to control them separately but I don't think we have that yet. 

In any case I think we should make it clear in the docs that there are 2 things:
1. models calling a tool multiple times in one message
2. executing multiple tools in parallel

there is a world where you want to allow the model call multiple tools in a single message but ensure they still get executed serially 

okay, resolved here: https://github.com/UKGovernmentBEIS/inspect_ai/commit/7394bd9b1ccbf3d114e9da5f8c150fc0b55599b2

Hey, sorry, I think I actually misunderstood something here.

IIRC our requirement is to have a way to tell the model "you can't call this tool more than once permessage"

But the diff above allows multiple calls to the same tool per message, and just executes them serially.

Would it be possible to support this? No biggie if not -- we can just do this validation on our end before calling `call_tools`


If you mark a tool w/ `parallel=False` (https://inspect.ai-safety-institute.org.uk/tools.html#parallel-tool-calls) then we will turn off all parallel tool calling for model providers that support it (currently OpenAI and Grok). They don't have a further level of granularity which says "don't call this specific tool more than once" (you'd probably have to do that via prompting). To be fully general though for all providers you would need to do some manual handling before invoking `call_tools()`