load_dataset("livecodebench/code_generation_lite", version_tag="release_v2")  TypeError: 'NoneType' object is not callable
Opened by dshwei (open)

### Describe the bug

from datasets import load_dataset
lcb_codegen = load_dataset("livecodebench/code_generation_lite", version_tag="release_v2")

or
configs = get_dataset_config_names("livecodebench/code_generation_lite", trust_remote_code=True)

both error:
Traceback (most recent call last):
File "", line 1, in
File "/workspace/miniconda/envs/grpo/lib/python3.10/site-packages/datasets/load.py", line 2131, in load_dataset
builder_instance = load_dataset_builder(
File "/workspace/miniconda/envs/grpo/lib/python3.10/site-packages/datasets/load.py", line 1888, in load_dataset_builder
builder_instance: DatasetBuilder = builder_cls(
TypeError: 'NoneType' object is not callable

### Steps to reproduce the bug

from datasets import get_dataset_config_names

configs = get_dataset_config_names("livecodebench/code_generation_lite", trust_remote_code=True)
OR 
lcb_codegen = load_dataset("livecodebench/code_generation_lite", version_tag="release_v2")

### Expected behavior

load  datasets  livecodebench/code_generation_lite


### Environment info

 import  datasets
version '3.3.2'

Comments:
> datasets

Hi, have you solved this bug? Today I also met the same problem about `livecodebench/code_generation_lite` when evaluating the `Open-R1` repo. I am looking forward to your reply!

![Image](https://github.com/user-attachments/assets/02e92fbf-da33-41b3-b8d4-f79b293a54f1)

Hey guys,
I tried to reproduce the issue and it works fine. I used google colab as enviroment.

![Image](https://github.com/user-attachments/assets/024dd8e1-bd10-470b-9a6d-60759ffdb984)

> Hey guys, I tried to reproduce the issue and it works fine. I used google colab as enviroment.
> 
> ![Image](https://github.com/user-attachments/assets/024dd8e1-bd10-470b-9a6d-60759ffdb984)

Thanks for your kind reply! I wonder which Python version do you use? My Python version is 3.11.11 and datasets version is 3.3.2 but I still met this bug.

<img width="1121" alt="Image" src="https://github.com/user-attachments/assets/7c2c5007-ee55-4030-94b9-01fcdea0bf4a" />

@zwxandy It's Python 3.11.11

@Serzhanov @zwxandy I have met the same problem, have this problem be solved?

> [@Serzhanov](https://github.com/Serzhanov) [@zwxandy](https://github.com/zwxandy) I have met the same problem, have this problem be solved?

I try to downgrade datasets version to 2.20.0，and it works for me @Serzhanov @dshwei , hope this work for you too :)

> > datasets
> 
> Hi, have you solved this bug? Today I also met the same problem about `livecodebench/code_generation_lite` when evaluating the `Open-R1` repo. I am looking forward to your reply!
> 
> ![Image](https://github.com/user-attachments/assets/02e92fbf-da33-41b3-b8d4-f79b293a54f1)

Hi, have you resolved this problem? I meet the same bug when evaluating the ’Open-R1’, too. Looking forward to your reply!

> > [@Serzhanov](https://github.com/Serzhanov) [@zwxandy](https://github.com/zwxandy) I have met the same problem, have this problem be solved?
> 
> I try to downgrade datasets version to 2.20.0，and it works for me [@Serzhanov](https://github.com/Serzhanov) [@dshwei](https://github.com/dshwei) , hope this work for you too :)

I still met the same bug after downgrading datasets version to 2.20.0. Moreover, it is not friendly to Open-R1 since there can be another bug: `open-r1 0.1.0.dev0 requires datasets>=3.2.0` with datasets==2.20.0

> > > datasets
> > 
> > 
> > Hi, have you solved this bug? Today I also met the same problem about `livecodebench/code_generation_lite` when evaluating the `Open-R1` repo. I am looking forward to your reply!
> > ![Image](https://github.com/user-attachments/assets/02e92fbf-da33-41b3-b8d4-f79b293a54f1)
> 
> Hi, have you resolved this problem? I meet the same bug when evaluating the ’Open-R1’, too. Looking forward to your reply!

Hi, I still cannot solve this bug introduced from datasets version. Downgrading datasets version to 2.20.0 cannot work for me and it introduces another problem `open-r1 0.1.0.dev0 requires datasets>=3.2.0` in Open-R1.

Luckily, there is a tricky way to enable you to run Open-R1. You can remove or comment the code related to `lcb` in `~/anaconda3/envs/openr1/lib/python3.11/site-packages/lighteval/tasks/extended/__init__.py`. I have reproduce the results of DeepSeek-R1-Distill-Qwen-1.5B and 7B on MATH-500, GPQA, and AIME24.

You can have a try~

The issue was resolved .
because the file` livecodebench/code_generation_lite/code_generation_lite.py `was not downloaded. Manually downloading it fixed the problem.