Text Generation
Transformers
Safetensors
mixtral
Mixture of Experts
frankenmoe
Merge
mergekit
lazymergekit
M4-ai/TinyMistral-248M-v2-cleaner
Locutusque/TinyMistral-248M-Instruct
jtatman/tinymistral-v2-pycoder-instuct-248m
Locutusque/TinyMistral-248M-v2-Instruct
Eval Results
text-generation-inference
Inference Endpoints
license: apache-2.0 | |
tags: | |
- moe | |
- frankenmoe | |
- merge | |
- mergekit | |
- lazymergekit | |
- M4-ai/TinyMistral-248M-v2-cleaner | |
- Locutusque/TinyMistral-248M-Instruct | |
- jtatman/tinymistral-v2-pycoder-instuct-248m | |
- Locutusque/TinyMistral-248M-v2-Instruct | |
base_model: | |
- M4-ai/TinyMistral-248M-v2-cleaner | |
- Locutusque/TinyMistral-248M-Instruct | |
- jtatman/tinymistral-v2-pycoder-instuct-248m | |
- Locutusque/TinyMistral-248M-v2-Instruct | |
model-index: | |
- name: TinyMistral-248Mx4-MOE | |
results: | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: AI2 Reasoning Challenge (25-Shot) | |
type: ai2_arc | |
config: ARC-Challenge | |
split: test | |
args: | |
num_few_shot: 25 | |
metrics: | |
- type: acc_norm | |
value: 29.52 | |
name: normalized accuracy | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: HellaSwag (10-Shot) | |
type: hellaswag | |
split: validation | |
args: | |
num_few_shot: 10 | |
metrics: | |
- type: acc_norm | |
value: 25.71 | |
name: normalized accuracy | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: MMLU (5-Shot) | |
type: cais/mmlu | |
config: all | |
split: test | |
args: | |
num_few_shot: 5 | |
metrics: | |
- type: acc | |
value: 24.82 | |
name: accuracy | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: TruthfulQA (0-shot) | |
type: truthful_qa | |
config: multiple_choice | |
split: validation | |
args: | |
num_few_shot: 0 | |
metrics: | |
- type: mc2 | |
value: 48.66 | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: Winogrande (5-shot) | |
type: winogrande | |
config: winogrande_xl | |
split: validation | |
args: | |
num_few_shot: 5 | |
metrics: | |
- type: acc | |
value: 51.78 | |
name: accuracy | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
- task: | |
type: text-generation | |
name: Text Generation | |
dataset: | |
name: GSM8k (5-shot) | |
type: gsm8k | |
config: main | |
split: test | |
args: | |
num_few_shot: 5 | |
metrics: | |
- type: acc | |
value: 0.0 | |
name: accuracy | |
source: | |
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=222gate/TinyMistral-248Mx4-MOE | |
name: Open LLM Leaderboard | |
# TinyMistral-248Mx4-MOE | |
TinyMistral-248Mx4-MOE is a Mixure of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing): | |
* [M4-ai/TinyMistral-248M-v2-cleaner](https://huggingface.co/M4-ai/TinyMistral-248M-v2-cleaner) | |
* [Locutusque/TinyMistral-248M-Instruct](https://huggingface.co/Locutusque/TinyMistral-248M-Instruct) | |
* [jtatman/tinymistral-v2-pycoder-instuct-248m](https://huggingface.co/jtatman/tinymistral-v2-pycoder-instuct-248m) | |
* [Locutusque/TinyMistral-248M-v2-Instruct](https://huggingface.co/Locutusque/TinyMistral-248M-v2-Instruct) | |
## 🧩 Configuration | |
```yaml | |
base_model: Locutusque/TinyMistral-248M-v2-Instruct | |
gate_mode: hidden | |
dtype: bfloat16 | |
experts: | |
- source_model: M4-ai/TinyMistral-248M-v2-cleaner | |
positive_prompts: | |
- "versatile" | |
- "helpful" | |
- "factual" | |
- "integrated" | |
- "adaptive" | |
- "comprehensive" | |
- "balanced" | |
negative_prompts: | |
- "specialized" | |
- "narrow" | |
- "focused" | |
- "limited" | |
- "specific" | |
- source_model: Locutusque/TinyMistral-248M-Instruct | |
positive_prompts: | |
- "creative" | |
- "chat" | |
- "discuss" | |
- "culture" | |
- "world" | |
- "expressive" | |
- "detailed" | |
- "imaginative" | |
- "engaging" | |
negative_prompts: | |
- "sorry" | |
- "cannot" | |
- "factual" | |
- "concise" | |
- "straightforward" | |
- "objective" | |
- "dry" | |
- source_model: jtatman/tinymistral-v2-pycoder-instuct-248m | |
positive_prompts: | |
- "analytical" | |
- "accurate" | |
- "logical" | |
- "knowledgeable" | |
- "precise" | |
- "calculate" | |
- "compute" | |
- "solve" | |
- "work" | |
- "python" | |
- "javascript" | |
- "programming" | |
- "algorithm" | |
- "tell me" | |
- "assistant" | |
negative_prompts: | |
- "creative" | |
- "abstract" | |
- "imaginative" | |
- "artistic" | |
- "emotional" | |
- "mistake" | |
- "inaccurate" | |
- source_model: Locutusque/TinyMistral-248M-v2-Instruct | |
positive_prompts: | |
- "instructive" | |
- "clear" | |
- "directive" | |
- "helpful" | |
- "informative" | |
negative_prompts: | |
- "exploratory" | |
- "open-ended" | |
- "narrative" | |
- "speculative" | |
- "artistic" | |
``` | |
## 💻 Usage | |
```python | |
!pip install -qU transformers bitsandbytes accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "222gate/TinyMistral-248Mx4-MOE" | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True}, | |
) | |
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}] | |
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` | |
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) | |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_222gate__TinyMistral-248Mx4-MOE) | |
| Metric |Value| | |
|---------------------------------|----:| | |
|Avg. |30.08| | |
|AI2 Reasoning Challenge (25-Shot)|29.52| | |
|HellaSwag (10-Shot) |25.71| | |
|MMLU (5-Shot) |24.82| | |
|TruthfulQA (0-shot) |48.66| | |
|Winogrande (5-shot) |51.78| | |
|GSM8k (5-shot) | 0.00| | |