Training data vs Context Lenght
Hi
@jeiku
.
It's possible that as Aura and Luna datasets have been trained up to 2048 tokens, if I set the context lenght at 4096, once 2048 tokens are surpassed eventually it runs out of training data and I get refusals from it no matter what jailbreak I have in the character card?
It's a problem that I had with a pretty long chat.
Neither Aura nor Luna includes the uncensoring data. Both of these LoRA's are trained on text completions including my own writing and some futa erotica. The reason they are trained at 2048 is because no single chunk of data is larger than that as I have included hard stop sequences which are converted to stop tokens after each paragraph. Whatever issue you are having is not a result of these datasets.
The uncensored aspect of this model is derived from Undi95's Unholy which is Toxicqa and Toxic DPO trained at 8192 context.
I am not the cause of your issue and I suggest you try another model if you are unsatisfied with this one as I have already released several successor models.
The model is uncensored. Under the correct circumstances it will provide any information you ask of it. I have tested my models to full context and at varying levels of NSFW/NSFL/Illegal content and they work as intended.
I recommend you review your generation settings and the amount of context you have set in your backend.
I love your model, I was just trying to understand what was the problem so I could fix it. I had context lenght at 4096. As it seems the problem is in my end, I will keep trying different setups, although in Layla I don't have as many things to tweak like in Kobold or ST.
Keep up the good work.