Is it 30B? In the description it says 13B

#2
by rafa9 - opened

This is wizard-vicuna-13b trained with a subset of the dataset - responses that contained alignment / moralizing were removed. ....

Cognitive Computations org

Yeah it's 30b

The original wizard-vicuna model was never published in 30b

Hey @ehartford
Appreciate your work here πŸ™
Can you please elaborate? is it a README typo? if Vicuna was never released with 30B, what is the base model used to train Wizard-Vicuna-30B?
And how it relates to Vicuna (only by dataset but not by weights)?
Thank you in advance

Cognitive Computations org

The readme is not a typo.
Vicuna's code was used to finetune.
Wizard-Vicuna's dataset (with refusals and bias removed) was used as the training dataset.
llama-30b was used as the base model.

Sign up or log in to comment