File size: 5,207 Bytes
d5c378a f611c0d ff84d57 d5c378a f611c0d ff84d57 d5c378a f611c0d ff84d57 d5c378a f611c0d d5c378a f611c0d d5c378a f611c0d 575a07a d5c378a ff84d57 f611c0d d5c378a f611c0d d5c378a 575a07a f611c0d 575a07a f611c0d 575a07a d5c378a f611c0d ff84d57 575a07a f611c0d 575a07a f611c0d d5c378a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
---
license: apache-2.0
tags:
- moe
- frankenmoe
- merge
- mergekit
- lazymergekit
- Locutusque/TinyMistral-248M-v2.5-Instruct
- Locutusque/TinyMistral-248M-v2.5-Instruct
- Locutusque/TinyMistral-248M-v2.5-Instruct
- jtatman/tinymistral-samantha-chatml-lora-v2
base_model:
- Locutusque/TinyMistral-248M-v2.5-Instruct
- Locutusque/TinyMistral-248M-v2.5-Instruct
- Locutusque/TinyMistral-248M-v2.5-Instruct
- jtatman/tinymistral-samantha-chatml-lora-v2
---
# TinyMistral-248m-v2.5-4x-Moe
TinyMistral-248m-v2.5-4x-Moe is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [Locutusque/TinyMistral-248M-v2.5-Instruct](https://huggingface.co/Locutusque/TinyMistral-248M-v2.5-Instruct)
* [Locutusque/TinyMistral-248M-v2.5-Instruct](https://huggingface.co/Locutusque/TinyMistral-248M-v2.5-Instruct)
* [Locutusque/TinyMistral-248M-v2.5-Instruct](https://huggingface.co/Locutusque/TinyMistral-248M-v2.5-Instruct)
* [jtatman/tinymistral-samantha-chatml-lora-v2](https://huggingface.co/jtatman/tinymistral-samantha-chatml-lora-v2)
## 🧩 Configuration
```yaml
base_model: Locutusque/TinyMistral-248M-v2.5-Instruct
experts:
- source_model: Locutusque/TinyMistral-248M-v2.5-Instruct
positive_prompts:
- "Write me a Python program that calculates the factorial of n."
- "Help me debug this code."
- "Optimize this C++ program."
negative_prompts:
- "How do you"
- "Explain the concept of"
- "Give an overview of"
- "Compare and contrast between"
- "Provide information about"
- "Help me understand"
- "Summarize"
- "Make a recommendation on"
- "Answer this question"
- "Craft me a list of some nice places to visit around the world."
- "Write me a story"
- "Write me an essay"
- "How do I incorporate visual elements into my writing?"
- source_model: Locutusque/TinyMistral-248M-v2.5-Instruct
positive_prompts:
- "What is the product of 2 x 5 x 18?"
- "How do I guess the value of x for the function f(x) = x^4 - 2x^2 - 1?"
negative_prompts:
- "Help me debug this code."
- "Optimize this C# script."
- "Implement this feature using JavaScript."
- "Convert this HTML structure into a more efficient design."
- "Assist me with writing a program that"
- "Craft me a list of some nice places to visit around the world. "
- "Write me a story"
- "Write me an essay"
- "How do I incorporate visual elements into my writing?"
- source_model: Locutusque/TinyMistral-248M-v2.5-Instruct
positive_prompts:
- "How do I incorporate fewer visual elements into my art but retain impact?"
negative_prompts:
- "Help me debug this code."
- "Optimize this C# script."
- "Implement this feature using JavaScript."
- "Convert this HTML structure into a more efficient design."
- "Help me debug this code."
- "Optimize this C# script."
- "Implement this feature using JavaScript."
- "Convert this HTML structure into a more efficient design."
- "Compare and contrast between"
- "Provide information about"
- "Help me understand"
- "Summarize"
- "Make a recommendation on"
- "Answer this question"
- "Craft me a list of some nice places to visit around the world. "
- "Write me a story"
- "Write me an essay"
- source_model: jtatman/tinymistral-samantha-chatml-lora-v2
positive_prompts:
- "Craft me a list of some nice places to visit around the world. "
- "Write me a story"
- "Write me an essay"
- "Create a fantasy story about"
- "Tell me about the wild fjords."
negative_prompts:
- "Help me debug this code."
- "Optimize this C# script."
- "Implement this feature using JavaScript."
- "Convert this HTML structure into a more efficient design."
- "Help me debug this code."
- "Optimize this C# script."
- "Implement this feature using JavaScript."
- "Convert this HTML structure into a more efficient design."
- "Compare and contrast between"
- "Provide information about"
- "Help me understand"
- "Summarize"
- "Make a recommendation on"
- "Answer this question"
- "How do I incorporate visual elements into my writing?"
gate_mode: hidden
```
## 💻 Usage
```python
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "jtatman/TinyMistral-248m-v2.5-4x-Moe"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
``` |