Repository: explodinggradients/ragas
Issue: TopicAdherence throws ValueError: operands could not be broadcast together with shapes (22,) (20,)
Root cause: Validation Gap

[x] I have checked the [documentation](https://docs.ragas.io/) and related resources and couldn't resolve my bug.

**Describe the bug**
I'm running a LlamaIndex AgentWorkflow, a human-in-the-loop workflow which has access to several tools. I record all the events and user/assistant messages. I then use `convert_to_ragas_messages(events)` to convert messages to Ragas format and compute metrics.

NOTE: I'm simulating a multi-turn conversation between the Llamaindex workflow assistant and another LLM playing the human role. 
There isn't a lot of variation in the multi-turn conversation, both assistant and humer messages are more or less always the same.

The Ragas evaluation works sometimes, other times it throws the error below.

Ragas version: 0.4.1
Python version: 3.13.11

**Code to Reproduce**
Record all Llamaindex events as described in the documentation: https://docs.ragas.io/en/stable/howtos/integrations/llamaindex_agents/#agent-goal-accuracy

Then, once the events have been collected in  `events: list[Event]`:

```
    client = AsyncOpenAI(
        api_key=os.environ.get("OPENAI_API_KEY"),
    )
    evaluator_llm = llm_factory(model="gpt-4o-mini", provider="openai", client=client) 

    ragas_messages = convert_to_ragas_messages(events)
    
    ### Goal accuracy
    goal_accuracy = AgentGoalAccuracyWithReference(llm=evaluator_llm)
    goal_score = await goal_accuracy.ascore(
        ragas_messages,
        reference="The user has defined a job description for a backend engineer role."
    ) 
    print(f"Agent goal accuracy score: {goal_score.value}")
    if goal_score.reason:
        print(f"Agent goal accuracy reason: {goal_score.reason}")

    ### Topic adherence
    topic_adherence_scorer = TopicAdherence(llm=evaluator_llm, mode="precision")
    topic_score = await topic_adherence_scorer.ascore(ragas_messages, reference_topics=["recruitment"]) 
    print(f"Topic adherence score: {topic_score.value}")
    if topic_score.reason:
        print(f"Topic adherence reason: {topic_score.reason}")
```

**Error trace**
```
Agent goal accuracy score: 0.0
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/ariel/Projects/tekne/platform/backend/llm_eval/test_recruiter_workflow_ragas.py", line 318, in <module>
    asyncio.run(main())
    ~~~~~~~~~~~^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.11/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 195, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.11/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/opt/homebrew/Cellar/python@3.13/3.13.11/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 725, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/ariel/Projects/tekne/platform/backend/llm_eval/test_recruiter_workflow_ragas.py", line 314, in main
    await evaluate_with_ragas(test_cases, events)
  File "/Users/ariel/Projects/tekne/platform/backend/llm_eval/test_recruiter_workflow_ragas.py", line 268, in evaluate_with_ragas
    topic_score = await topic_adherence_scorer.ascore(ragas_messages, reference_topics=["recruitment"]) # type: ignore
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ariel/Projects/tekne/platform/backend/.venv/lib/python3.13/site-packages/ragas/metrics/collections/topic_adherence/metric.py", line 125, in ascore
    score = self._compute_score(topic_answered, topic_classifications)
  File "/Users/ariel/Projects/tekne/platform/backend/.venv/lib/python3.13/site-packages/ragas/metrics/collections/topic_adherence/metric.py", line 194, in _compute_score
    true_positives = np.sum(topic_answered & topic_classifications)
                            ~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
ValueError: operands could not be broadcast together with shapes (22,) (20,) 
```

**Expected behavior**
I would expect to see the metrics. I'm not sure why Agent goal accuracy score is sometimes 0.0 and other times 1.0.


**Additional context**
I can attach the output of the entire multi-turn simulation if needed.

<!-- PS: bugs suck but is also part of the process. We sincerely apologies for breaking your flow because of it, but don't worry, we got your back ❤️. We will get this fixed as fast as we can and thanks for helping us out by reporting it 🙏. -->


Thanks @arielfaur for reporting this. Handled it now in PR #2503. This got unnoticed even in the legacy version. 

Could you check it out and let me know if it works for you now? 

@anistark Just tested your PR and the error is gone! I'm getting a Topic adherence score of 0.0, however. I did get scores in the past (before I started seeing this error). It's not entirely clear what to set in `reference_topics`. How many topics am I supposed to set? The longer the better? Should each topic be a single word or more of a concept?

Thank you!

@anistark I figured out the `reference_topics` parameter. By the way, there's also an error with `ToolCallF1`. Should I open another ticket?

> [@anistark](https://github.com/anistark) I figured out the `reference_topics` parameter. By the way, there's also an error with `ToolCallF1`. Should I open another ticket?

That's great. Yes, please feel free to open another issue for it.