Tensoic

/

TinyLlama-1.1B-3T-openhermes

like 8

Sorry for the late reply. The prompt template is alpaca. Please check axolotl docs.

Ps. The base model is unusable and sucks. This fine tune too. Outputs are utter garbage for sm reason

I'm training him with SFT and then with DPO, do you think it would also go wrong?

yep. I feel its a waste of time lmao

But did you train the model first in SFT and then DPO?

This is just SFT. Haven't tried dpo