Text Generation
Transformers
Safetensors
mistral
Safetensors
text-generation-inference
Merge
7b
mistralai/Mistral-7B-Instruct-v0.1
fblgit/una-cybertron-7b-v2-bf16
juanako
UNA
cybertron
fbl
dataset:fblgit/tree-of-knowledge
dataset:Open-Orca/SlimOrca-Dedup
dataset:allenai/ultrafeedback_binarized_cleaned
Inference Endpoints
has_space
conversational
license: apache-2.0 | |
tags: | |
- Safetensors | |
- mistral | |
- text-generation-inference | |
- merge | |
- mistral | |
- 7b | |
- mistralai/Mistral-7B-Instruct-v0.1 | |
- fblgit/una-cybertron-7b-v2-bf16 | |
- transformers | |
- safetensors | |
- mistral | |
- text-generation | |
- juanako | |
- UNA | |
- cybertron | |
- fbl | |
- dataset:fblgit/tree-of-knowledge | |
- dataset:Open-Orca/SlimOrca-Dedup | |
- dataset:allenai/ultrafeedback_binarized_cleaned | |
- license:apache-2.0 | |
- autotrain_compatible | |
- endpoints_compatible | |
- has_space | |
- text-generation-inference | |
- region:us | |
# una-cybertron-7b-v2-bf16-Mistral-7B-Instruct-v0.1 | |
una-cybertron-7b-v2-bf16-Mistral-7B-Instruct-v0.1 is a merge of the following models: | |
* [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) | |
* [fblgit/una-cybertron-7b-v2-bf16](https://huggingface.co/fblgit/una-cybertron-7b-v2-bf16) | |
## 🧩 Configuration | |
```yaml | |
slices: | |
- sources: | |
- model: mistralai/Mistral-7B-Instruct-v0.1 | |
layer_range: [0, 32] | |
- model: fblgit/una-cybertron-7b-v2-bf16 | |
layer_range: [0, 32] | |
merge_method: slerp | |
base_model: mistralai/Mistral-7B-Instruct-v0.1 | |
parameters: | |
t: | |
- filter: self_attn | |
value: [0, 0.5, 0.3, 0.7, 1] | |
- filter: mlp | |
value: [1, 0.5, 0.7, 0.3, 0] | |
- value: 0.5 | |
dtype: bfloat16 | |
``` | |
## 💻 Usage | |
```python | |
!pip install -qU transformers accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "MaziyarPanahi/una-cybertron-7b-v2-bf16-Mistral-7B-Instruct-v0.1" | |
messages = [{"role": "user", "content": "What is a large language model?"}] | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
torch_dtype=torch.float16, | |
device_map="auto", | |
) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` |