Have you considered using the Vicuna v1.1 version for training?
#5
by
QuantumBolt
- opened
Vicuna has released a new version v1.1 and it performs better than the v0 version. And training on Vicuna v1.1 may provide better performance.
Major updates of weights v1.1
- Refactor the tokenization and separator. In Vicuna v1.1, the separator has been changed from "###" to the EOS token "". This change makes it easier to determine the generation stop criteria and enables better compatibility with other libraries.
- Fix the supervised fine-tuning loss computation for better model quality.
Also seen at:
https://huggingface.co/lmsys/vicuna-7b-delta-v1.1#major-updates-of-weights-v11
We're rapidly improving StableVicuna. A new version is on the horizon. We're already internally testing it at Carper.
LouisStability
changed discussion status to
closed