|
--- |
|
license: apache-2.0 |
|
tags: |
|
- moe |
|
- merge |
|
- abideen/NexoNimbus-7B |
|
- mlabonne/NeuralMarcoro14-7B |
|
language: |
|
- en |
|
library_name: transformers |
|
--- |
|
|
|
# NexoNimbus-MoE-2x7B |
|
|
|
 |
|
|
|
NexoNimbus-MoE-2x7B is a Mixure of Experts (MoE) made with the following models: |
|
* [abideen/NexoNimbus-7B](https://huggingface.co/abideen/NexoNimbus-7B) |
|
* [mlabonne/NeuralMarcoro14-7B](https://huggingface.co/mlabonne/NeuralMarcoro14-7B) |
|
|
|
🏆 Evaluation |
|
NexoNimbus-MoE-2x7B is the 10th best-performing 13B LLM on the Open LLM Leaderboard: |
|
|
|
|
|
 |
|
|
|
|
|
| Task |Version| Metric |Value| |Stderr| |
|
|-------------|------:|--------|----:|---|-----:| |
|
|arc_challenge| 0|acc |62.28|± | 1.41| |
|
| | |acc_norm|66.80|± | 1.37| |
|
|hellaswag | 0|acc |66.83|± | 0.46| |
|
| | |acc_norm|85.66|± | 0.34| |
|
|gsm8k | 0|acc |53.52|± | 1.37| |
|
|winogrande | 0|acc |81.53|± | 1.09| |
|
|mmlu | 0|acc |64.51|± | 1.00| |
|
|
|
Average: 67.51% |
|
|
|
### TruthfulQA |
|
| Task |Version|Metric|Value| |Stderr| |
|
|-------------|------:|------|----:|---|-----:| |
|
|truthfulqa_mc| 1|mc1 |35.98|± | 1.68| |
|
| | |mc2 |53.05|± | 1.53| |
|
|
|
|
|
## 🧩 Configuration |
|
|
|
```yaml |
|
base_model: teknium/OpenHermes-2.5-Mistral-7B |
|
gate_mode: hidden |
|
dtype: bfloat16 |
|
experts: |
|
- source_model: abideen/NexoNimbus-7B |
|
positive_prompts: |
|
- "Mathematics" |
|
- "Physics" |
|
- "Chemistry" |
|
- "Biology" |
|
- "Medicine" |
|
- "Engineering" |
|
- "Computer Science" |
|
|
|
negative_prompts: |
|
- "History" |
|
- "Philosophy" |
|
- "Linguistics" |
|
- "Literature" |
|
- "Art and Art History" |
|
- "Music Theory and Composition" |
|
- "Performing Arts (Theater, Dance)" |
|
|
|
- source_model: mlabonne/NeuralMarcoro14-7B |
|
positive_prompts: |
|
- "Earth Sciences (Geology, Meteorology, Oceanography)" |
|
- "Environmental Science" |
|
- "Astronomy and Space Science" |
|
- "Psychology" |
|
- "Sociology" |
|
- "Anthropology" |
|
- "Political Science" |
|
- "Economics" |
|
negative_prompts: |
|
- "Education" |
|
- "Law" |
|
- "Theology and Religious Studies" |
|
- "Communication Studies" |
|
- "Business and Management" |
|
- "Agricultural Sciences" |
|
- "Nutrition and Food Science" |
|
- "Sports Science" |
|
``` |
|
|
|
## 💻 Usage |
|
|
|
Here's a [Colab notebook](https://colab.research.google.com/drive/1B1Q7vO95cDkEJbKIPhOWr6exB9-Q_lr-?usp=sharing) to run NexoNimbus-MoE-2x7B in 4-bit precision on a free T4 GPU. |
|
|
|
```python |
|
!pip install -qU transformers bitsandbytes accelerate |
|
|
|
from transformers import AutoTokenizer |
|
import transformers |
|
import torch |
|
|
|
model = "abideen/NexoNimbus-MoE-2x7B" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model) |
|
pipeline = transformers.pipeline( |
|
"text-generation", |
|
model=model, |
|
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True}, |
|
) |
|
|
|
messages = [{"role": "user", "content": "Explain what is machine learning."}] |
|
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) |
|
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) |
|
print(outputs[0]["generated_text"]) |
|
``` |
