Repository: confident-ai/deepeval
Issue: HallucinationMetric example failed
Root cause: Interface Contract Mismatch

**Describe the bug**

I tried to follow `HallucinationMetric` example in the readme https://github.com/confident-ai/deepeval#evaluating-a-dataset--test-cases-in-bulk

However, it failed. Any ideas? Thank you! ☺️

**To Reproduce**

```py
import pytest
from deepeval import assert_test
from deepeval.dataset import EvaluationDataset
from deepeval.metrics import HallucinationMetric
from deepeval.test_case import LLMTestCase

def get_refund_policy(question: str, refund_day_number: int) -> str:
    return f"You have {refund_day_number} days to get a full refund at no extra cost."

question = "What if these shoes don't fit?"
first_test_case = LLMTestCase(
    input=question,
    actual_output=get_refund_policy(question, refund_day_number=30),
    context=[
        "All customers are eligible for a 30 day full refund at no extra costs.",
    ],
)
dataset = EvaluationDataset(test_cases=[first_test_case])


@pytest.mark.parametrize(
    "test_case",
    dataset,
)
def test_customer_chatbot(test_case: LLMTestCase):
    hallucination_metric = HallucinationMetric(threshold=0.3)
    assert_test(test_case, [hallucination_metric])
```

```sh
deepeval set-litellm anthropic/claude-sonnet-4 --api-base=https://litellm.hongbomiao.com --api-key=xxx
deepeval test run src
```

**Expected behavior**

Expected no error. But got error

```sh
deepeval test run src -vv
===================================================== test session starts ======================================================
platform darwin -- Python 3.13.3, pytest-8.4.1, pluggy-1.6.0 -- /Users/hongbo-miao/Clouds/Git/hongbomiao.com/llm-evaluation/hm-deepeval/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/hongbo-miao/Clouds/Git/hongbomiao.com/llm-evaluation/hm-deepeval
configfile: pytest.ini
plugins: deepeval-3.2.3, xdist-3.7.0, repeat-0.9.4, anyio-4.9.0, rerunfailures-12.0, cov-6.2.1, asyncio-1.0.0
asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 1 item                                                                                                               

src/utils/test_get_refund_policy.py::test_customer_chatbot[test_case0] Evaluating 1 test case(s) in parallel ━━━━━━━━━━━━━━━━━━━
Evaluating 1 test case(s) in parallel ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0:00:04
    🎯 Evaluating test case #0        ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% 0:00:04
FAILEDRunning teardown with pytest sessionfinish...


=========================================================== FAILURES ===========================================================
______________________________________________ test_customer_chatbot[test_case0] _______________________________________________

test_case = LLMTestCase(input="What if these shoes don't fit?", actual_output='You have 30 days to get a full refund at no extra c...ne, tools_called=None, comments=None, expected_tools=None, token_cost=None, completion_time=None, name=None, tags=None)

    @pytest.mark.parametrize(
        "test_case",
        dataset,
    )
    def test_customer_chatbot(test_case: LLMTestCase):
        hallucination_metric = HallucinationMetric(threshold=0.3)
>       assert_test(test_case, [hallucination_metric])

src/utils/test_get_refund_policy.py:75: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.13/site-packages/deepeval/evaluate/evaluate.py:116: in assert_test
    test_result = loop.run_until_complete(
../../../../../.local/share/uv/python/cpython-3.13.3-macos-aarch64-none/lib/python3.13/asyncio/base_events.py:719: in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/deepeval/evaluate/execute.py:542: in a_execute_test_cases
    await asyncio.gather(*tasks)
.venv/lib/python3.13/site-packages/deepeval/evaluate/execute.py:425: in execute_with_semaphore
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.13/site-packages/deepeval/evaluate/execute.py:661: in a_execute_llm_test_cases
    await measure_metrics_with_indicator(
.venv/lib/python3.13/site-packages/deepeval/metrics/indicator.py:230: in measure_metrics_with_indicator
    await asyncio.gather(*tasks)
.venv/lib/python3.13/site-packages/deepeval/metrics/indicator.py:243: in safe_a_measure
    await metric.a_measure(
.venv/lib/python3.13/site-packages/deepeval/metrics/hallucination/hallucination.py:108: in a_measure
    await self._a_generate_verdicts(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <deepeval.metrics.hallucination.hallucination.HallucinationMetric object at 0x1346f6210>
actual_output = 'You have 30 days to get a full refund at no extra cost.'
contexts = ['All customers are eligible for a 30 day full refund at no extra costs.']

    async def _a_generate_verdicts(
        self, actual_output: str, contexts: List[str]
    ) -> List[HallucinationVerdict]:
        verdicts: List[HallucinationVerdict] = []
        prompt = self.evaluation_template.generate_verdicts(
            actual_output=actual_output, contexts=contexts
        )
        if self.using_native_model:
>           res, cost = await self.model.a_generate(prompt, schema=Verdicts)
            ^^^^^^^^^
E           ValueError: not enough values to unpack (expected 2, got 1)

.venv/lib/python3.13/site-packages/deepeval/metrics/hallucination/hallucination.py:195: ValueError
===================================================== slowest 10 durations =====================================================
5.53s call     src/utils/test_get_refund_policy.py::test_customer_chatbot[test_case0]

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
=================================================== short test summary info ====================================================
FAILED src/utils/test_get_refund_policy.py::test_customer_chatbot[test_case0] - ValueError: not enough values to unpack (expected 2, got 1)
================================================ 1 failed, 4 warnings in 5.68s =================================================
No test cases found, please try again.
Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x1348716a0>
Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x11769d670>, 1549115.865479291)])']
connector: <aiohttp.connector.TCPConnector object at 0x1348712b0>
```

**Desktop (please complete the following information):**
 - OS: macOS 15.5
 - Browser Brave
 - Version 1.80.115

**Additional context**
Add any other context about the problem here.


Hey @hongbo-miao seems like its an litellm integration issue, we'll take a look at it. In the meantime can you just wrap your litellm into a custom LLM wrapper? https://deepeval.com/guides/guides-using-custom-llms