Evaluation of Dolphin Mixtral series failed

#517
by Dampfinchen - opened
deleted

@Dampfinchen I was wondering the same about Dolphin so I went to its model card and it says "trust_remote_code is required". I've heard evaluations fail if remote code is required.

Open LLM Leaderboard org

Hi, the models failed during download, probably because it took too long to download, I re-added them to the queue.

SaylorTwift changed discussion status to closed
Dampfinchen changed discussion status to open
Open LLM Leaderboard org
edited Jan 11

Hi!
These models all failed at download again. Could you make sure their weights are in safetensors?
(As requested by the submit form :) )

That's good to know. These are not my models but it seems they are not in the safetensors format. I guess that explains it then, thank you.

Dampfinchen changed discussion status to closed
deleted
edited Jan 11

@Dampfinchen Perhaps try submitting Dolphin's GGUF versions. They should score virtually the same as their unquantized versions and safetensors shouldn't be an issue.

https://huggingface.co/TheBloke/dolphin-2.7-mixtral-8x7b-GGUF

Edit: Or GPTQ

https://huggingface.co/TheBloke/dolphin-2.7-mixtral-8x7b-GPTQ

Sign up or log in to comment