Repository: UKGovernmentBEIS/inspect_ai
Issue: How do we deal with multiple inputs to prompt_template()?
Root cause: Unimplemented Feature Gap

## Problem

Currently, we are sort of limited to having a single 'input' to a prompt; how do we deal with situations where we perhaps want to format a prompt with multiple inputs from a sample? for example, we may have a 'body_of_text' and a 'question' that we want to populate into a formatted prompt template, e.g in the below example I would want 2 inputs, **{prompt}** and an additional input **{question}**


> 

system

> 
> You are a question answering bot.

> 
> 
user

> Here is the job description:
> **{prompt}**
> 
> Here are the Questions:
> 
> Questions
> 
> Question: **{question}**
> 

> 
> 
assistant

> 



I notice in the prompt_template solver there are extra params that are accepted, but these are a separate argument; what if we want to store, and use additional arguments from the current TaskState? e.g in the above example where we may have multiple 'columns' in our dataset that should be populated into our prompt.

## my current hacky solution
Currently to do this I am just unpacking the metadata of the state and it works fine for me.

```diff
@solver
def prompt_template(template: str, **params: dict[str, Any]) -> Solver:
    """Parameterized prompt template.

    Prompt template containing a `{prompt}` placeholder and any
    number of additional `params`.

     Args:
      template (str | list[Message]):
          The conversation template to use. A simple string or
          a list of messages
      **params (dict[str,Any]):
          A mapping of the parameters to fill into the template
          excluding the `{prompt}` parameter which is taken
          from the input.

    Returns:
      A solver that uses the specified prompt template.
    """
    # determine the prompt template
    prompt_template = resource(template)

    async def solve(state: TaskState, generate: Generate) -> TaskState:
          prompt = state.user_prompt
+        additional_inputs = state.metadata
-        prompt.text = prompt_template.format(prompt=prompt.text, **params)
+        prompt.text = prompt_template.format(prompt=prompt.text, **params,**additional_inputs)
        return state

    return solve
```

## request
 It would be cool to have perhaps the ability to add 'additional_inputs'  when defining a dataset, perhaps it can be stored in the TaskState?

```
dataset=csv_dataset(
    dataset,
    FieldSpec(input="JobDescription",additional_inputs=['Question'], metadata=["Question"]),
)

```

## EDIT

sorry, this is starting to get long.. I incured the same issue in the graders, the grader prompts for the in-built scorers assume just the main args; input,answer,criteria; there are cases where you would want to add extra information directly from the sample, it would be great to do that here. I used the same workaround; i.e when create a score i just added the metadata too, so if any of those fields are in the prompt, presumably we want to populate them...

```diff
   async def score(state: TaskState, target: Target) -> Score:
        # format the scoring template
        score_prompt = grading_template.format(
            question=state.input_text,
            answer=state.output.completion,
            criterion=target.text,
            instructions=instructions,
+            **state.metadata,
        )
```



Yes, these custom fields should indeed be handled by metadata fields. So the question is should we forward all of the metadata automatically (as in your second example) or create some special designation as in your first example. My initial take is that making metadata automatically available in the template would be a sound and generally less confusing thing to do.



agreed, keep it simple :D 

Okay, PR for this is here: https://github.com/UKGovernmentBEIS/inspect_ai/pull/203

LMK if this conforms to your expectations or if you see any other problems with it!

awesome, thanks for this - will take a look this morning

Thanks again for this it definitely solves my problems. I left some comments on the PR.