Repository: openai/evals
Issue: Running an evaluation can lead to circular import error
Root cause: Resource Mishandling

### Describe the bug

```console
$ oaieval gpt-3.5-turbo test-match
[2023-12-06 19:14:25,433] [registry.py:249] Loading registry from /home/abhin/.local/lib/python3.9/site-packages/evals/registry/evals
[2023-12-06 19:14:25,529] [registry.py:249] Loading registry from /home/abhin/.evals/evals
[2023-12-06 19:14:25,530] [oaieval.py:110] Run started: 231206191425YO46H5EF
[2023-12-06 19:14:25,531] [data.py:75] Fetching test_match/samples.jsonl
[2023-12-06 19:14:25,531] [eval.py:34] Evaluating 3 samples
[2023-12-06 19:14:25,536] [eval.py:153] Running in threaded mode with 10 threads!
  0%|                                                                                | 0/3 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/abhin/.local/bin/oaieval", line 8, in <module>
    sys.exit(main())
  File "/home/abhin/.local/lib/python3.9/site-packages/evals/cli/oaieval.py", line 164, in main
    run(args)
  File "/home/abhin/.local/lib/python3.9/site-packages/evals/cli/oaieval.py", line 141, in run
    result = eval.run(recorder)
  File "/home/abhin/.local/lib/python3.9/site-packages/evals/elsuite/basic/match.py", line 53, in run
    self.eval_all_samples(recorder, samples)
  File "/home/abhin/.local/lib/python3.9/site-packages/evals/eval.py", line 155, in eval_all_samples
    idx_and_result = list(tqdm(iter, total=len(work_items), disable=not show_progress))
  File "/home/abhin/.local/lib/python3.9/site-packages/tqdm/std.py", line 1182, in __iter__
    for obj in iterable:
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 870, in next
    raise value
  File "/usr/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/abhin/.local/lib/python3.9/site-packages/evals/eval.py", line 140, in worker_thread
    executor = concurrent.futures.ThreadPoolExecutor(max_workers=1)
  File "/usr/lib/python3.9/concurrent/futures/__init__.py", line 49, in __getattr__
    from .thread import ThreadPoolExecutor as te
ImportError: cannot import name 'ThreadPoolExecutor' from partially initialized module 'concurrent.futures.thread' (most likely due to a circular import) (/usr/lib/python3.9/concurrent/futures/thread.py)
```

### To Reproduce

1. `mkdir tmp`
2. `cd tmp`
3. `virtualenv -p python3.9 .venv`
4. `source .venv/bin/activate`
5. `pip install evals==1.0.3.post1 openai==0.28.1`
6. `oaieval gpt-3.5-turbo test-match`
7. It should break with the exception mentioned above.

### Code snippets

_No response_

### OS

debian-11

### Python version

Python 3.9.2

### Library version

1.0.3.post1

I'm able to temporarily work around it by setting `EVALS_SEQUENTIAL=1`, but this does slow down the evals.

Can you try rerunning this on main and let us know if there's still an issue? We also plan to release a new version of the library soon, so if main is working you should be able to expect this fixed in the pypi package soon

Closing as i think this should work if you use evals 2.0.0

@etr2460 I am having this same issue with: version = "2.0.0.post1"
Some evals work as expected but some like "oaieval gpt-3.5-turbo test-match" are giving me this error:

```
Traceback (most recent call last):
  File "/Users/juanmenendez/PycharmProjects/evals444/.venv/bin/oaieval", line 8, in <module>
    sys.exit(main())
  File "/Users/juanmenendez/PycharmProjects/evals444/evals/cli/oaieval.py", line 300, in main
    run(args)
  File "/Users/juanmenendez/PycharmProjects/evals444/evals/cli/oaieval.py", line 158, in run
    eval_spec.args.update(extra_eval_params)
AttributeError: 'NoneType' object has no attribute 'update'
```

Any idea?