TRI-ML/mamba-7b-rw failed

#704
by devingulliver - opened

Once again requesting logs for a failed model. Hopefully together we'll be able to find the issue and rerun it.
(Sorry for the increased activity recently, I'm studying recurrent LLMs and reproducible evals like these are a godsend)

Open LLM Leaderboard org

Hi @devingulliver !

Could you please provide me with the request file of the failed model?

Open LLM Leaderboard org

Hi @devingulliver !

I've rescheduled your model, according to logs, the model just got stuck on loading, but that was a problem on our end, it should be fine now

I'll close this issue now, please write here / reopen it in case of any problems with this model

alozowski changed discussion status to closed

It appears to have failed again (same request file). Is it for the same reason or a different reason?

alozowski changed discussion status to open
Open LLM Leaderboard org

Hi @devingulliver ,

The error for this model was connected with the last week backend update โ€“ we fixed it and I've resubmitted your model now, it should be fine

alozowski changed discussion status to closed

Somehow it's failed a third time. Is there a third issue with the backend or is something wrong with the model this time?

alozowski changed discussion status to open

If there is an issue with the model, feel free to post the logs. The request file was linked earlier in the thread.

Open LLM Leaderboard org

Hi @devingulliver ,

We are investigating the error on our side, we'll be back when we figure out it

It's been nearly a month, wanted to bump this once more to see if any progress could be shared

Sign up or log in to comment