Repository: openai/evals
Issue: TypeError: unsupported operand type(s) for +: 'int' and 'CompletionTokensDetail
Root cause: Validation Gap

### Describe the bug

When trying to run out of the box evals test, getting the following stack trace:

```
(venv) <USER>@<HOSTNAME> <PROJECT_PATH> % python3 -m evals.cli.oaieval gpt-3.5-turbo test-match
[2025-04-16 22:26:45,558] [registry.py:271] Loading registry from <PATH_TO_VENV>/lib/python3.10/site-packages/evals/registry/evals
[2025-04-16 22:26:46,128] [registry.py:271] Loading registry from /Users/<USER>/.evals/evals
[2025-04-16 22:26:46,131] [oaieval.py:215] Run started: <RUN_ID>
[2025-04-16 22:26:46,261] [eval.py:36] Evaluating 3 samples
[2025-04-16 22:26:46,277] [eval.py:144] Running in threaded mode with 10 threads!
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  3.36it/s]
[2025-04-16 22:26:47,189] [oaieval.py:275] Found 3/3 sampling events with usage data
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/Cellar/python@3.10/3.10.12_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "<PROJECT_PATH>/venv/lib/python3.10/site-packages/evals/cli/oaieval.py", line 308, in <module>
    main()
  File "<PROJECT_PATH>/venv/lib/python3.10/site-packages/evals/cli/oaieval.py", line 304, in main
    run(args)
  File "<PROJECT_PATH>/venv/lib/python3.10/site-packages/evals/cli/oaieval.py", line 227, in run
    add_token_usage_to_result(result, recorder)
  File "<PROJECT_PATH>/venv/lib/python3.10/site-packages/evals/cli/oaieval.py", line 278, in add_token_usage_to_result
    total_usage = {
  File "<PROJECT_PATH>/venv/lib/python3.10/site-packages/evals/cli/oaieval.py", line 279, in <dictcomp>
    key: sum(u[key] if u[key] is not None else 0 for u in usage_events)
TypeError: unsupported operand type(s) for +: 'int' and 'CompletionTokensDetails'
```

### To Reproduce

Running:
```
python3 -m evals.cli.oaieval gpt-3.5-turbo test-match
```

### Code snippets

```Python

```

### OS

macOS

### Python version

Python v3.10.12

### Library version

openai-evals 3.0.1

I have fixed this issue in my local environment. The problem was caused by the summing logic in the add_token_usage_to_result function, which assumed all usage values were integers. In some cases, the usage values were objects (such as CompletionTokensDetails), which caused a TypeError.

I updated the function to safely extract numeric values from these objects before summing. Here is the relevant code change:


```  def get_token_count(obj):
    if obj is None:
        return 0
    if isinstance(obj, dict) and "count" in obj:
        return obj["count"]
    if hasattr(obj, "count"):
        return obj.count
    if isinstance(obj, (int, float)):
        return obj
    return 0
``` 

total_usage = {
    key: sum(get_token_count(u.get(key)) for u in usage_events)
    for key in set().union(*(u.keys() for u in usage_events))
}



After this change, the evaluation runs successfully and all usage values are logged correctly, with no TypeError.





![Image](https://github.com/user-attachments/assets/068b61f6-0fbb-4d0e-b046-fd351f57a18d)