Request: jeiku/Average_Normie_v3.69_8B
Model name:
Average_Normie_v3.69_8B
Model link:
https://huggingface.co/jeiku/Average_Normie_v3.69_8B
Brief description:
Another average normie just like you and me... or is it? NSFW focused and easy to steer with editing, this model aims to please even the most hardcore LLM enthusiast. Built upon a foundation of the most depraved models yet to be released, some could argue it goes too far in that direction. Whatever side you land on, at least give it a shot, what do you have to lose?
An image/direct image link to represent the model (square shaped):
Additonal quants (if you want any):
Q2_K for my weak wittle phone processor (last one worked great, imatrix seems to make a huge difference.)
The art is a masterpiece.
Built upon a foundation of the most depraved models yet to be released, some could argue it goes too far in that direction.
Now, that's a selling pitch!
As soon as I saw the art I knew I had a winner on my hands. I tested it pretty thoroughly. It has a habit of taking on whatever style you set down for it. This is the model I'll be using moving forward, so I hope the folks like it.
It has a habit of taking on whatever style you set down for it.
Exciting news. Fingers crossed.
What are nightmares made of?
Answer:
@jeiku -- Which presets/template you used for testing? Virt's? ST built-in Llama-3? Nitral's?
I'm running all my models in Kobold Lite since I switched to mobile. I did test in Instruct mode with Llama 3 template, Temp 1.2, Min P 0.05, Rep Pen 1.1, Rep Pen Range 512. Personally, I set \n as a stopping string, but that may not be what most people want, although I tested both ways.
Kobold, eh. Is there a problem with the SillyTavern interface on Mobile? I pretty much only use it on mobile, with the server running on my computer, and it works fine?
I'm running the model locally on mobile and ime koboldcpp->kobold lite is anywhere from 1.5-2x faster than relaying to sillytavern. I've also tested several apk's including layla and chatterui (fuck mlc,) and my koboldcpp build beats them all for some reason.
I'm getting single sentence responses in 30-40 seconds on a Note10. I plan on upgrading next month to an SD 8 gen 3 phone tho, so i may switch to something more convenient when i have more speed.
:O
I didn't know you could infer with KCPP locally. Curious...
https://huggingface.co/Lewdiculous/Average_Normie_v3.69_8B-GGUF-IQ-Imatrix
Q2_K-imat should be uploaded now. Since you use that on mobile then, might be handy, so it's up first. The rest will come later.
Ah, I see, you can just use Termux to use the Linux version of KCPP. Nice huh.
thx b0ss
I plan on upgrading next month to an SD 8 gen 3 phone tho, so i may switch to something more convenient when i have more speed.
Huge.
kobold lite is anywhere from 1.5-2x faster than relaying to sillytavern
Are you talking about the latency to get inference started compared to sending it through SillyTavern? I can't say I ever noticed it. I don't really care for saving 500ms, unless you're talking about something else.
Are you talking about the latency to get inference started compared to sending it through SillyTavern? I can't say I ever noticed it. I don't really care for saving 500ms, unless you're talking about something else.
Honestly, It's probably just due to me bloating the prompt with AN time macros and other injections like world info, etc. Another issue I came up against was running out of RAM and losing persistence of the model due to only having 8GB of RAM. This caused me to have to re-ingest the prompt which takes multiple minutes at a full 2048 tokens. But also, due to an unrelated issue i had to wipe my termux install and i just never re-installed sillytavern because i'm lazy.
Honestly I wouldn't be surprised if part of your problem there was just because of Sillytavern, it's very shoddily engineered at points and it wouldn't surprise me at all that kobold's UI is much lighter than it in general
Another issue I came up against was running out of RAM and losing persistence of the model due to only having 8GB of RAM.
Okay, 8GB is pretty low for that, yeah... I'm assuming you can't easily upgrade it in your system, since that was an issue.