Commit
•
f5ddd6f
1
Parent(s):
24d656d
Update README.md
Browse files
README.md
CHANGED
|
@@ -22,19 +22,23 @@ license: cc-by-nc-sa-4.0
|
|
| 22 |
COKAL_merged_test-v1-13B is an auto-regressive language model based on the LLaMA2 transformer architecture.
|
| 23 |
|
| 24 |
|
| 25 |
-
|
|
|
|
|
|
|
| 26 |
|
| 27 |
[HumanF-MarkrAI/COKAL-DPO-13b-v2](https://huggingface.co/HumanF-MarkrAI/COKAL-DPO-13b-v2)
|
| 28 |
|
| 29 |
[MarkrAI/DopeorNope-maestro-v2-DPO-13b](https://huggingface.co/MarkrAI/DopeorNope-maestro-v2-DPO-13b)
|
| 30 |
|
| 31 |
-
|
|
|
|
| 32 |
|
| 33 |
I utilized `slerp merging` to smoothly blend the gradients of the base models to create my model.
|
| 34 |
|
| 35 |
The merging approach relies on luck, but at the same time, if I have an accurate understanding of my model's performance, I can carefully select models that excel in each aspect to develop a well-balanced model.
|
| 36 |
|
| 37 |
|
|
|
|
| 38 |
|
| 39 |
# **Model Benchmark**
|
| 40 |
|
|
@@ -51,9 +55,11 @@ The merging approach relies on luck, but at the same time, if I have an accurate
|
|
| 51 |
| [DopeorNope-maestro-v2-DPO-13b🐻❄️](https://huggingface.co/MarkrAI/DopeorNope-maestro-v2-DPO-13b) | 49.42 | 45.14 | 56.69 | 41.37 | 42.26 | 61.63 |
|
| 52 |
|
| 53 |
|
|
|
|
| 54 |
|
| 55 |
# Implementation Code
|
| 56 |
|
|
|
|
| 57 |
## Load model
|
| 58 |
```python
|
| 59 |
|
|
|
|
| 22 |
COKAL_merged_test-v1-13B is an auto-regressive language model based on the LLaMA2 transformer architecture.
|
| 23 |
|
| 24 |
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## **Base Model**
|
| 28 |
|
| 29 |
[HumanF-MarkrAI/COKAL-DPO-13b-v2](https://huggingface.co/HumanF-MarkrAI/COKAL-DPO-13b-v2)
|
| 30 |
|
| 31 |
[MarkrAI/DopeorNope-maestro-v2-DPO-13b](https://huggingface.co/MarkrAI/DopeorNope-maestro-v2-DPO-13b)
|
| 32 |
|
| 33 |
+
|
| 34 |
+
## **Implemented Method**
|
| 35 |
|
| 36 |
I utilized `slerp merging` to smoothly blend the gradients of the base models to create my model.
|
| 37 |
|
| 38 |
The merging approach relies on luck, but at the same time, if I have an accurate understanding of my model's performance, I can carefully select models that excel in each aspect to develop a well-balanced model.
|
| 39 |
|
| 40 |
|
| 41 |
+
---
|
| 42 |
|
| 43 |
# **Model Benchmark**
|
| 44 |
|
|
|
|
| 55 |
| [DopeorNope-maestro-v2-DPO-13b🐻❄️](https://huggingface.co/MarkrAI/DopeorNope-maestro-v2-DPO-13b) | 49.42 | 45.14 | 56.69 | 41.37 | 42.26 | 61.63 |
|
| 56 |
|
| 57 |
|
| 58 |
+
---
|
| 59 |
|
| 60 |
# Implementation Code
|
| 61 |
|
| 62 |
+
|
| 63 |
## Load model
|
| 64 |
```python
|
| 65 |
|