Finetune of Yi-34B-200K (the version with better ctx, Yi-34B-200K v2 or Yi-34B-200K-XLCTX (my preffered name)) on adamo1139/rawrr_v2_2_stage1 dataset via ORPO and GaLore on 4-bit (bnb) weights.
This is not a chat model!! It's meant to serve as base for further finetuning that has less behaviour inherited from being trained on OpenAI etc. AI generated content. If you don't want your finetune to sound like an AI model, using this as a base should be a good idea.
- Downloads last month
- 231