THUDM

/

chatglm2-6b

like 2.02k

Follow

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University 1,110

THUDM/chatglm2-6b accuracy=0 on lambada_openai dataset with lm-eval.
Here is command, Could you have a try?

git clone https://github.com/EleutherAI/lm-evaluation-harness.git
cd lm-evlauation-harness
python setup.py install

python main.py --model hf-causal --model_args pretrained=THUDM/chatglm2-6b,trust_remote_code=True  --tasks lambada_openai --limit 10 --batch_size 1 --no_cache