Repository: confident-ai/deepeval
Issue: AttributeError: 'SummarizationAlignmentVerdict' object has no attribute 'verdicts' -- Async False
Root cause: Interface Contract Mismatch

**Describe the bug**
When `run_async=False`, `SummarizationMetric` has an attribute error when it is evaluated.

**To Reproduce**
Steps to reproduce the behavior:

1. Run `pip install -qU deepeval==3.7.4`
2. Run the following code:

```py
from dotenv import load_dotenv
from deepeval import evaluate
from deepeval.metrics import SummarizationMetric
from deepeval.evaluate import AsyncConfig
from deepeval.test_case import LLMTestCase

load_dotenv('.env')

tests = [
    LLMTestCase(
        input='Marley was dead: to begin with...',
        actual_output="The text opens by firmly establishing that Marley is dead...",
        expected_output=None,
        context=None,
        retrieval_context=None,
        additional_metadata=None,
        tools_called=None,
        comments=None,
        expected_tools=None,
        token_cost=None,
        completion_time=None,
        name=None,
        tags=None,
        mcp_servers=None,
        mcp_tools_called=None,
        mcp_resources_called=None,
        mcp_prompts_called=None
    )
]

metric = SummarizationMetric(model='gpt-4.1-nano')
results = evaluate(tests_41, [metric], async_config=AsyncConfig(run_async=False))
```

3. View error:

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_32063/195509519.py in <cell line: 34>()
     32 
     33 metric = SummarizationMetric(model='gpt-4.1-nano')
---> 34 results = evaluate(tests_41, [metric], async_config=AsyncConfig(run_async=False))

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/evaluate.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/evaluate.py) in evaluate(test_cases, metrics, metric_collection, hyperparameters, identifier, async_config, display_config, cache_config, error_config)
    246                 )
    247             else:
--> 248                 test_results = execute_test_cases(
    249                     test_cases,
    250                     metrics,

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py) in execute_test_cases(test_cases, metrics, error_config, display_config, cache_config, identifier, test_run_manager, _use_bar_indicator, _is_assert_test)
    553                 total=len(test_cases),
    554             )
--> 555             evaluate_test_cases(progress=progress, pbar_id=pbar_id)
    556     else:
    557         evaluate_test_cases()

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py) in evaluate_test_cases(progress, pbar_id)
    476                                 update_pbar(progress, pbar_test_case_id)
    477 
--> 478                 run_sync_with_timeout(_run_case, deadline_timeout)
    479             except (asyncio.TimeoutError, TimeoutError):
    480                 msg = (

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/models/retry_policy.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/models/retry_policy.py) in run_sync_with_timeout(func, timeout_seconds, *args, **kwargs)
    619     if result["exc"] is not None:
    620         exc = result["exc"]
--> 621         raise exc.with_traceback(getattr(exc, "__traceback__", None))
    622     return result["value"]
    623 

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/models/retry_policy.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/models/retry_policy.py) in target()
    582     def target():
    583         try:
--> 584             result["value"] = context.run(func, *args, **kwargs)
    585         except BaseException as e:
    586             result["exc"] = e

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py) in _run_case()
    408 
    409                                 if metric_data is None:
--> 410                                     res = _execute_metric(
    411                                         metric=metric,
    412                                         test_case=test_case,

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/evaluate/execute.py) in _execute_metric(metric, test_case, show_metric_indicator, in_component, error_config)
   3212 ) -> Optional[str]:
   3213     try:
-> 3214         metric.measure(
   3215             test_case,
   3216             _show_indicator=show_metric_indicator,

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/metrics/summarization/summarization.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/metrics/summarization/summarization.py) in measure(self, test_case, _show_indicator, _in_component, _log_metric_to_confident)
    103                 )
    104                 self.alignment_verdicts: List[SummarizationAlignmentVerdict] = (
--> 105                     self._generate_alignment_verdicts()
    106                 )
    107                 alignment_score = self._calculate_score(ScoreType.ALIGNMENT)

[~/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/metrics/summarization/summarization.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/deepeval/metrics/summarization/summarization.py) in _generate_alignment_verdicts(self)
    494             )
    495             self.evaluation_cost += cost
--> 496             verdicts = [item for item in res.verdicts]
    497             return verdicts
    498         else:

[~/anaconda3/envs/python3/lib/python3.10/site-packages/pydantic/main.py](https://gai-1101-rwzk.notebook.us-west-2.sagemaker.aws/home/ec2-user/anaconda3/envs/python3/lib/python3.10/site-packages/pydantic/main.py) in __getattr__(self, item)
   1024                     else:
   1025                         # this is the current error
-> 1026                         raise AttributeError(f'{type(self).__name__!r} object has no attribute {item!r}')
   1027 
   1028         def __setattr__(self, name: str, value: Any) -> None:

AttributeError: 'SummarizationAlignmentVerdict' object has no attribute 'verdicts'
```

**Expected behavior**
Evaluate to complete successfully.

**Screenshots**
N/A

**Desktop (please complete the following information):**
N/A

**Smartphone (please complete the following information):**
N/A

**Additional context**
Code was run in a Jupyter Notebook hosted on AWS SageMaker AI.


Hey @dgrahn, thank you for taking the time to report this issue. You will be happy to know that this issue is fixed by https://github.com/confident-ai/deepeval/pull/2367. Once that PR is merged and released your script will run without error.

all the best!