What is the humaneval? and humaneval plus scores?
#2
by
rombodawg
- opened
You say its trained on code but you havnt posted any coding benchmarks, or even compared it to codellama
How to download the models?
You say its trained on code but you havnt posted any coding benchmarks, or even compared it to codellama
We did not focus on its coding capability. At 2T checkpoint, we did benchmark on Humaneval and MBPP. It scored 17.7 on human eval and 26 on MBPP, respectively.
No further benchmark on code was carried out there after.
weitianwen
changed discussion status to
closed