Update README.md
Browse files
README.md
CHANGED
|
@@ -5,9 +5,12 @@ base_model:
|
|
| 5 |
tags:
|
| 6 |
- mergekit
|
| 7 |
- merge
|
| 8 |
-
|
| 9 |
---
|
| 10 |
-
#
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 13 |
|
|
@@ -45,4 +48,4 @@ parameters:
|
|
| 45 |
- value: 0.5
|
| 46 |
dtype: bfloat16
|
| 47 |
|
| 48 |
-
```
|
|
|
|
| 5 |
tags:
|
| 6 |
- mergekit
|
| 7 |
- merge
|
| 8 |
+
license: apache-2.0
|
| 9 |
---
|
| 10 |
+
# Should be working.
|
| 11 |
+
* Test merge of two extremely large MoE models using SLERP. Don't know if it's working correctly yet, haven't had the time or hardware to test.
|
| 12 |
+
|
| 13 |
+
# merge
|
| 14 |
|
| 15 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 16 |
|
|
|
|
| 48 |
- value: 0.5
|
| 49 |
dtype: bfloat16
|
| 50 |
|
| 51 |
+
```
|