|
--- |
|
base_model: Envoid/Mixtral-Instruct-ITR-8x7B |
|
inference: false |
|
license: cc-by-nc-4.0 |
|
model_creator: Envoid |
|
model_name: Mixtral-Instruct-ITR-8x7B |
|
model_type: mixtral |
|
tags: |
|
- not-for-all-audiences |
|
--- |
|
|
|
# Model Card for Mixtral-Instruct-ITR-8x7B-GGUF |
|
- Model creator: [Envoid](https://huggingface.co/envoid) |
|
- Original model: [Mixtral-Instruct-ITR-8x7B](https://huggingface.co/Envoid/Mixtral-Instruct-ITR-8x7B) |
|
|
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
Envoid_Mixtral-Instruct-ITR-8x7B quantized with love. |
|
|
|
Starting out with Q4_K_M, and iterating from there. Future plans for imatrix/IQ quants (pending compute power). |
|
|
|
First time doing quantizations so any feedback is greatly appreciated. |
|
|
|
Original model card below for reference. |
|
|
|
--- |
|
license: cc-by-nc-4.0 |
|
--- |
|
# Caution this model may be unpredictable |
|
 |
|
## Mixtral-Instruct-ITR (Interpolative Training Regression) |
|
|
|
We have to go back, edition. |
|
|
|
For this model I took what I learned in the making of [Cat-8x7B](https://huggingface.co/Envoid/Cat-8x7B) and went back to the very beginning and SLERP merged [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1) onto [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1) |
|
|
|
While the results aren't perfect the model feels more creative and less overcooked than Mixtral Instruct is often accused of being. |
|
|
|
The hopes are that this should also have left the model much more receptive to additional finetuning and I am interested to see what comes of it so please feel free to download it and have fun. |
|
|
|
Apologies about the small shard size (keep forgetting to change the mergekit config back) |
|
|
|
## The model is a lot less likely to refuse certain requests in this state: |
|
|
|
so if you are going to apply additional finetuning to the model you may need to bolster its alignment depending on your use case. |
|
|
|
The model still responds well to [INST] Thingie [/INST] formatting quite well. |
|
|
|
Or if preferred this can easily be reproduced if you have both base and instruct models handy using mergekit (mixtral branch) with the following config |
|
``` |
|
models: |
|
- model: ./mistralai_Mixtral-8x7B-Instruct-v0.1 |
|
- model: ./mistralai_Mixtral-8x7B-v0.1 |
|
merge_method: slerp |
|
base_model: ./mistralai_Mixtral-8x7B-v0.1 |
|
parameters: |
|
t: |
|
- value: 0.5 |
|
dtype: float16 |
|
``` |
