abideen commited on
Commit
c9af20d
1 Parent(s): afe7dee

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -27,21 +27,21 @@ NexoNimbus-MoE-2x7B is the 10th best-performing 13B LLM on the Open LLM Leaderbo
27
 
28
  | Task |Version| Metric |Value| |Stderr|
29
  |-------------|------:|--------|----:|---|-----:|
30
- |arc_challenge| 0|acc |68.25|± | 1.36|
31
- | | |acc_norm|70.81|± | 1.38|
32
- |hellaswag | 0|acc |70.86|± | 0.45|
33
- | | |acc_norm|87.86|± | 0.32|
34
- |gsm8k | 0|acc |70.35|± | 1.25|
35
- |winogrande | 0|acc |84.84|± | 1.00|
36
- |mmlu | 0|acc |64.69|± | 1.00|
37
 
38
- Average: 73.5%
39
 
40
  ### TruthfulQA
41
  | Task |Version|Metric|Value| |Stderr|
42
  |-------------|------:|------|----:|---|-----:|
43
- |truthfulqa_mc| 1|mc1 |46.26|± | 1.74|
44
- | | |mc2 |62.42|± | 1.54|
45
 
46
 
47
  ## 🧩 Configuration
@@ -93,7 +93,7 @@ experts:
93
 
94
  ## 💻 Usage
95
 
96
- Here's a [Colab notebook](https://colab.research.google.com/drive/1F9lzL1IeZRMgiSbY9UbgCR__RreIflJh?usp=sharing) to run NexoNimbus-MoE-2x7B in 4-bit precision on a free T4 GPU.
97
 
98
  ```python
99
  !pip install -qU transformers bitsandbytes accelerate
@@ -111,7 +111,7 @@ pipeline = transformers.pipeline(
111
  model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
112
  )
113
 
114
- messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
115
  prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
116
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
117
  print(outputs[0]["generated_text"])
 
27
 
28
  | Task |Version| Metric |Value| |Stderr|
29
  |-------------|------:|--------|----:|---|-----:|
30
+ |arc_challenge| 0|acc |62.28|± | 1.41|
31
+ | | |acc_norm|66.80|± | 1.37|
32
+ |hellaswag | 0|acc |66.83|± | 0.46|
33
+ | | |acc_norm|85.66|± | 0.34|
34
+ |gsm8k | 0|acc |53.52|± | 1.37|
35
+ |winogrande | 0|acc |81.53|± | 1.09|
36
+ |mmlu | 0|acc |64.51|± | 1.00|
37
 
38
+ Average: 67.51%
39
 
40
  ### TruthfulQA
41
  | Task |Version|Metric|Value| |Stderr|
42
  |-------------|------:|------|----:|---|-----:|
43
+ |truthfulqa_mc| 1|mc1 |35.98|± | 1.68|
44
+ | | |mc2 |53.05|± | 1.53|
45
 
46
 
47
  ## 🧩 Configuration
 
93
 
94
  ## 💻 Usage
95
 
96
+ Here's a [Colab notebook](https://colab.research.google.com/drive/1B1Q7vO95cDkEJbKIPhOWr6exB9-Q_lr-?usp=sharing) to run NexoNimbus-MoE-2x7B in 4-bit precision on a free T4 GPU.
97
 
98
  ```python
99
  !pip install -qU transformers bitsandbytes accelerate
 
111
  model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
112
  )
113
 
114
+ messages = [{"role": "user", "content": "Explain what is machine learning."}]
115
  prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
116
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
117
  print(outputs[0]["generated_text"])