deepseek-ai/deepseek-coder-7b-base-v1.5
Text Generation
•
Updated
•
1.57k
•
31
collecting models I want to evaluate on shadereval-task2: https://github.com/bigcode-project/bigcode-evaluation-harness/pull/173 at fp16!!
Note currently #1 with error rate of 0.353
Note # previous #1 error rate 0.38
Note has noteable difference between fp16 and fp32, will need to run bf16 likely contaminated
Note likely contaminated
Note needs rerun with incomplete_generation tag
Note performs the worst with error rate of 0.79
Note this space holds the evaluation metric that is used. It also has a usually up to date leaderboard. check for updates: https://huggingface.co/spaces/Vipitis/shadermatch/blob/main/result_preview.png