Repository: openai/human-eval
Issue: AttributeError: Can't pickle local object 'check_correctness.<locals>.unsafe_execute'
Root cause: Resource Mishandling

When I run "evaluate_functional_correctness sample.jsonl --problem_file=problem.jsonl", it has the following problem.

Can u help me? thx
Detail log.

-------------------------------------------------

Reading samples...
1it [00:00, 2118.34it/s]
Running test suites...
  0%|                                                                                                                            | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 33, in <module>
    sys.exit(load_entry_point('human-eval', 'console_scripts', 'evaluate_functional_correctness')())
  File "/Users/user/opt/anaconda3/envs/py38/bin/evaluate_functional_correctness", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 28, in <module>
    sys.exit(main())
  File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 25, in main
    fire.Fire(entry_point)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 475, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/site-packages/fire/core.py", line 691, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluate_functional_correctness.py", line 20, in entry_point
    results = evaluate_functional_correctness(sample_file, k, n_workers, timeout, problem_file)
  File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/evaluation.py", line 75, in evaluate_functional_correctness
    result = future.result()
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/user/Desktop/Program/LLM/human-eval/human_eval/execution.py", line 73, in check_correctness
    p.start()
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/user/opt/anaconda3/envs/py38/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'check_correctness.<locals>.unsafe_execute'


你可以参考18楼的回复

Seeing the same error, what is the fix?

Same problem

@tianzhaotju @emrgnt-cmplxty  I have fixed this problem in my repo with pull request https://github.com/openai/human-eval/pull/30

Hello Devs, i have encountered this issue too. Well for me the suggested fix does not work at all. My workaround is to use a Linux Distro under WSL-2 or just by itself. For me it straight clicks, and everything works fine. I am not sure why the problem is present only on my windows system. Hope I could help.

Also getting this error on macOS Sonoma, but it runs fine in a Windows VM.