abideen commited on
Commit
d824911
1 Parent(s): 066f714

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +38 -20
README.md CHANGED
@@ -27,7 +27,8 @@ model-index:
27
  value: 68.52
28
  name: normalized accuracy
29
  source:
30
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
31
  name: Open LLM Leaderboard
32
  - task:
33
  type: text-generation
@@ -43,7 +44,8 @@ model-index:
43
  value: 87.3
44
  name: normalized accuracy
45
  source:
46
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
47
  name: Open LLM Leaderboard
48
  - task:
49
  type: text-generation
@@ -60,7 +62,8 @@ model-index:
60
  value: 64.65
61
  name: accuracy
62
  source:
63
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
64
  name: Open LLM Leaderboard
65
  - task:
66
  type: text-generation
@@ -76,7 +79,8 @@ model-index:
76
  - type: mc2
77
  value: 61.21
78
  source:
79
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
80
  name: Open LLM Leaderboard
81
  - task:
82
  type: text-generation
@@ -93,7 +97,8 @@ model-index:
93
  value: 80.19
94
  name: accuracy
95
  source:
96
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
97
  name: Open LLM Leaderboard
98
  - task:
99
  type: text-generation
@@ -110,16 +115,42 @@ model-index:
110
  value: 65.13
111
  name: accuracy
112
  source:
113
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
 
114
  name: Open LLM Leaderboard
 
 
 
115
  ---
116
 
117
  # MonarchCoder-7B
118
 
119
- MonarchCoder-7B is a merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
 
 
 
120
  * [Syed-Hasan-8503/Tess-Coder-7B-Mistral-v1.0](https://huggingface.co/Syed-Hasan-8503/Tess-Coder-7B-Mistral-v1.0)
121
  * [mlabonne/AlphaMonarch-7B](https://huggingface.co/mlabonne/AlphaMonarch-7B)
122
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
123
  ## 🧩 Configuration
124
 
125
  ```yaml
@@ -165,16 +196,3 @@ pipeline = transformers.pipeline(
165
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
166
  print(outputs[0]["generated_text"])
167
  ```
168
- # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
169
- Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_abideen__MonarchCoder-7B)
170
-
171
- | Metric |Value|
172
- |---------------------------------|----:|
173
- |Avg. |71.17|
174
- |AI2 Reasoning Challenge (25-Shot)|68.52|
175
- |HellaSwag (10-Shot) |87.30|
176
- |MMLU (5-Shot) |64.65|
177
- |TruthfulQA (0-shot) |61.21|
178
- |Winogrande (5-shot) |80.19|
179
- |GSM8k (5-shot) |65.13|
180
-
 
27
  value: 68.52
28
  name: normalized accuracy
29
  source:
30
+ url: >-
31
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
32
  name: Open LLM Leaderboard
33
  - task:
34
  type: text-generation
 
44
  value: 87.3
45
  name: normalized accuracy
46
  source:
47
+ url: >-
48
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
49
  name: Open LLM Leaderboard
50
  - task:
51
  type: text-generation
 
62
  value: 64.65
63
  name: accuracy
64
  source:
65
+ url: >-
66
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
67
  name: Open LLM Leaderboard
68
  - task:
69
  type: text-generation
 
79
  - type: mc2
80
  value: 61.21
81
  source:
82
+ url: >-
83
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
84
  name: Open LLM Leaderboard
85
  - task:
86
  type: text-generation
 
97
  value: 80.19
98
  name: accuracy
99
  source:
100
+ url: >-
101
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
102
  name: Open LLM Leaderboard
103
  - task:
104
  type: text-generation
 
115
  value: 65.13
116
  name: accuracy
117
  source:
118
+ url: >-
119
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=abideen/MonarchCoder-7B
120
  name: Open LLM Leaderboard
121
+ language:
122
+ - en
123
+ library_name: transformers
124
  ---
125
 
126
  # MonarchCoder-7B
127
 
128
+
129
+ ![image/jpeg](https://cdn-uploads.huggingface.co/production/uploads/64e380b2e12618b261fa6ba0/oJN8_xoMOq2RlIc799m-x.jpeg)
130
+
131
+ MonarchCoder-7B is a slerp merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
132
  * [Syed-Hasan-8503/Tess-Coder-7B-Mistral-v1.0](https://huggingface.co/Syed-Hasan-8503/Tess-Coder-7B-Mistral-v1.0)
133
  * [mlabonne/AlphaMonarch-7B](https://huggingface.co/mlabonne/AlphaMonarch-7B)
134
 
135
+ The main aim behind creating this model is to create a model that performs well in reasoning, conversation, and coding. AlphaMonarch pperforms amazing on reasoning and conversation tasks. Merging AlphaMonarch with a coding model yielded MonarchCoder-7B which performs better on OpenLLM, Nous, and HumanEval benchmark. Although [MonarchCoder-2x7B](abideen/MonarchCoder-MoE-2x7B) performs better than MonarchCoder-7B.
136
+
137
+
138
+
139
+
140
+ # [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
141
+ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_abideen__MonarchCoder-7B)
142
+
143
+ | Metric |Value|
144
+ |---------------------------------|----:|
145
+ |Avg. |71.17|
146
+ |AI2 Reasoning Challenge (25-Shot)|68.52|
147
+ |HellaSwag (10-Shot) |87.30|
148
+ |MMLU (5-Shot) |64.65|
149
+ |TruthfulQA (0-shot) |61.21|
150
+ |Winogrande (5-shot) |80.19|
151
+ |GSM8k (5-shot) |65.13|
152
+
153
+
154
  ## 🧩 Configuration
155
 
156
  ```yaml
 
196
  outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
197
  print(outputs[0]["generated_text"])
198
  ```