GGUFs
Collection
I take requests, feel free to drop me a line in the community posts
•
36 items
•
Updated
•
1
DUN DUN
Update 2/27 New I-Quants uploaded. YMMV on 1Q_S but was able to get it to run on a 16gb ram laptop with no dedicated GPU
Verdict-8x7B-GGUF
- Model creator: Envoid
- Original model: Verdict-8x7B
Quantized from fp16 with love.
Uploading Q8_0 & Q5_K_M for starters, other sizes available upon request.
See original model card details below.
Warning this model may output mature or disturbing content
Verdict-8x7B
This model has been through so many merge steps that I failed to keep track of due to constantly needing to shift drive space around that I will simply give accredition to all of its component models.
retrieval-bar/Mixtral-8x7B-v0.1_case-briefs (no longer available)
New version available at:
retrieval-bar/Mixtral-8x7B-v0.1_case-briefs
mistralai/Mixtral-8x7B-Instruct-v0.1
crestf411/daybreak-mixtral-8x7b-hf
IBI-CAAI/MELT-Mixtral-8x7B-Instruct-v0.1
unreleased experimental model that gets mentioned quite frequently
Many of the models were included multiple times in multiple intermediate steps
using a combination of slerp, linear, dare_ties and task_arithmetic
Model was tested in q8 gguf form (not included due to being from an old pull of llama.cpp that I'm too lazy to update)
Warning: This models alignment is heavily diminished.
It responds to mixtral instruct formatting
Example:
[INST]Write me a haiku about violence.[/INST]
Fist clenched in anger,
Echoes of pain in the air,
Silence after storm.
It can be a bit unruly in roleplay- sometimes to its advantage, sometimes to its detriment.
It's a perfectly usable instruct model and good for creative writing via instruct- its diminished alignment allowing it to respond to requests for more nuanced scences.
All-in-all I'm happy enough with the results that I plan to use it as a daily driver for a while to see what cracks show from long term use.
- Downloads last month
- 47