Model surprisingly good at EvalPlus

#17

by cristian-rodriguez - opened Apr 29

Apr 29

Hi,
I am checking the following benchmarks :

EvalPlus
LiveCodeBench
CodeQwen performs top-5 in the first but top-25 in the second. LiveCodeBench is supposed to avoid data contamination.

I wonder if we can then (probably) assume some data contamination on the training ? Or does anybody have a reasonable explanation for this gap?

Thanks

Qwen org May 1

We strictly enforced data contamination during the training.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment