Repository: UKGovernmentBEIS/inspect_ai
Issue: Real-time model output evaluation failed, moved to dummy solver in AI Inspect
Root cause: Validation Gap

## Description
I encountered an issue when attempting to implement real-time evaluation of model outputs within the agent interaction flow. The immediate evaluation approach using a model to assess outputs during the conversation did not work as expected. 

## What I tried
I created AI Agents using Autogen. And then in task.py, I had this code

File: task.py
```
def research() -> Task:
    return Task(
        dataset=[
            Sample(
                input=[{
                    "role": "user",
                    "content": ROBOT_PROMPT
                }],
                target="Test stop sequence functionality"
            )
        ],  
        plan=[bridge(agent_interaction())],
        scorer=robot_behavior_scorer(),
    )
```
The scoring was done in the agent_interaction function in real time and then stored in a dict, which was then passed on as the metadata.

But this approach meant that every time, the dict was initialised with init values (0) even though I have only one task to run.

File: agent_interaction.py
```
def agent_interaction():
    init_model_here()

    async def run(sample: dict[str, Any]) -> dict[str, Any]:
        behaviour_dict = {} # Init the dict here with 0 
        team = RoundRobinGroupChat([agent_1, agent_2], termination_condition=None, max_turns=100)

        conversation = []
        for msg in result.messages:
            if msg.source == "Environment":
                conversation.append({
                    "role": "user",
                    "content": msg.content
                })
               # Evaluate the content of the bot here using another LLM and then based on this change the value of the dict.  This does not work
   
            elif msg.source == "Robot":
                conversation.append({
                    "role": "assistant",
                    "content": msg.content
                })
     # Return the state which has the dict values  in metadata

```

## Workaround implemented
I've now moved the evaluation logic to a separate component as a two step process:

```
  plan=[bridge(agent_interaction()), deception_scoring()],
```

But this is a workaround.

Can you tell me what is wrong here? Thanks in Advance!




I don't fully understand the issue, could you clarify a bit? It's possible that we are fumbling a handoff but I'm not 100% on the behavior you are seeing vs. expecting to see. Specifically I don't understand this at all:

> But this approach meant that every time, the dict was initialised with init values (0) even though I have only one task to run.

Thanks for the quick reply:
What I meant is that the behaviour_dict is initialized inside the run function, but it is always in the same state and has no changes (which was expected as the LLM should classify. This was also tried with a dummy function call where explicitly the values were passed.

For example, the dictionary is initialised here: 
```
    async def run(sample: dict[str, Any]) -> dict[str, Any]:
        behaviour_dict = {"harmful":0, "harmless":0}  # This is not changed at all
```

Although I call an LLM to evaluate later on here:

```
for msg in result.messages:
            if msg.source == "Environment":
                conversation.append({
                    "role": "user",
                    "content": msg.content
                })
               # Evaluate it here by calling an LLM again and then storing this in the dictionary above
               # Call an LLM here... using async await
               # LLM gives a JSON obj, which is then converted to a dict
               # Logic for adding the count, if harmful or harmless. Modify dict state

```

Is it due to :
- Calling an LLM again but the function call is not awaited?

Is it understandable now, or do you prefer a sample github repo with code?

I apologize, I still don't understand the issue you are describing. I see a `behaviour_dict` in both of your examples above but then it never appears after its declaration so I don't know what it is supposed to be doing. If you could provide a complete, runnable minimal example with "what I expected to see" and "what I actually saw" that would be helpful.