jukofyork/creative-writing-control-vectors-v3.0 · *** llmixer's ExllamaV2 wrapper! ***

Oct 31

Hi jukofyork,
Great job with these vectors! For those who prefer using ExLlamaV2 instead of llama.cpp, I've created a wrapper that integrates these control vectors.
You can check it out here: llmixer/ExllamaV2-Control-Vectors

jukofyork

Owner Oct 31

Hi jukofyork,
Great job with these vectors! For those who prefer using ExLlamaV2 instead of llama.cpp, I've created a wrapper that integrates these control vectors.
You can check it out here: llmixer/ExllamaV2-Control-Vectors

Awesome! People have been asking for an ExLlamaV2 version and @gghfez managed to hack them in using a .bias weight, but it wasn't possible to change the scale factor.

I'll post a link to this in the "thread of doom" :

https://huggingface.co/jukofyork/creative-writing-control-vectors-BETA-v0.1/discussions/2

Most people interested will see it better there I think.

jukofyork pinned discussion Oct 31

jukofyork changed discussion title from ExllamaV2 to *** See here for llmixer's ExllamaV2 wrapper! *** Oct 31

jukofyork changed discussion title from *** See here for llmixer's ExllamaV2 wrapper! *** to *** llmixer's ExllamaV2 wrapper! *** Oct 31

jukofyork

Owner Oct 31

I pinned and renamed the thread to hopefully get more attention.

I will have to give ExllamaV2 a go myself now, as it does look way easier to experiment with compared to llama.cpp!

llmixer

Oct 31

Thanks! Yeah, absolutely, that's why I mostly use exl2 myself.

jukofyork unpinned discussion 19 days ago