Prompting syntax?
Was the data modified to use the Vicuna syntax, or is it still LimaRP's Alpaca-ish format?
This uses the limarpv3 lora trained on llama-fied Yi, so it uses the Alpaca-ish format with the length prompts. I'll add the full card when I'm not busy with my actual job.
Oh I see. You mean this lora? https://huggingface.co/Doctor-Shotgun/limarpv3-yi-llama-34b-lora
Or an unlisted one?
The one you linked to, yes.
Could you share your settings with Alpaca-LimaRP format here? Neither that nor the original Alpaca works for me with this model. Vicuna doesn't work too. It isn't coherent at all for me: either impersonates the user, Speaks In All Capitals, repeats itself work for word, or outputs random symbols.
The only format that worked, and I gotta admit, worked really well in my case, was ChatML. Are you sure the LoRA merge changed the formatting if the model?
Could you share your settings with Alpaca-LimaRP format here? Neither that nor the original Alpaca works for me with this model. Vicuna doesn't work too. It isn't coherent at all for me: either impersonates the user, Speaks In All Capitals, repeats itself work for word, or outputs random symbols.
The only format that worked, and I gotta admit, worked really well in my case, was ChatML. Are you sure the LoRA merge changed the formatting if the model?
This model is quite literally just https://huggingface.co/NousResearch/Nous-Capybara-34B + (https://huggingface.co/Doctor-Shotgun/limarpv3-yi-llama-34b-lora x0.5), so there is no ChatML training on it at all.
Can't attest to the quality of the results (as it looks like I didn't even find it notable enough to write a model card), but in theory it only contains training in Vicuna and in Alpaca-LimaRP.
Koboldcpp console log:
llm_load_print_meta: general.name = doctor-shotgun_nous-capybara-limarpv3-34b
llm_load_print_meta: BOS token = 1 '<|startoftext|>'
llm_load_print_meta: EOS token = 2 '<|endoftext|>'
Those tokens are similar to the ChatML format: <|im_start|> <|im_end|>
These are the default BOS and EOS tokens used by Yi. Doesn't have anything to do with ChatML.