LilyWinter
commited on
Update README.md
Browse files
README.md
CHANGED
@@ -115,10 +115,16 @@ A merge using a custom algorithm (NearSwap) of:
|
|
115 |
|
116 |
<img src=https://huggingface.co/alchemonaut/BoreanGale-70B/resolve/main/bg.png>
|
117 |
|
|
|
|
|
|
|
|
|
|
|
118 |
NearSwap retains most of the weights of the base model (Miqu), but when a weight is similar between the two, it is interpolated to the secondary model (WinterGoddess) value. A parameter *t* specifies the sameness threshold. When the distance between two values is below *t*, the weight from the secondary model (WinterGoddess) is used.
|
119 |
|
120 |
This version of the model uses *t* = 0.001. At this *t*, about 10% of weights are fully switched to WinterGoddess. Model quality rapidly degrades above *t* = 0.0025:
|
121 |
|
|
|
122 |
- *t* = 0.0003 (~2% full swap)
|
123 |
- *t* = 0.001 (~10% full swap): This model
|
124 |
- *t* = 0.0025 (~18% full swap): Generates one paragraph okay, but then reverts to garbage
|
|
|
115 |
|
116 |
<img src=https://huggingface.co/alchemonaut/BoreanGale-70B/resolve/main/bg.png>
|
117 |
|
118 |
+
<br/>
|
119 |
+
<br/>
|
120 |
+
|
121 |
+
# NearSwap Algorithm
|
122 |
+
|
123 |
NearSwap retains most of the weights of the base model (Miqu), but when a weight is similar between the two, it is interpolated to the secondary model (WinterGoddess) value. A parameter *t* specifies the sameness threshold. When the distance between two values is below *t*, the weight from the secondary model (WinterGoddess) is used.
|
124 |
|
125 |
This version of the model uses *t* = 0.001. At this *t*, about 10% of weights are fully switched to WinterGoddess. Model quality rapidly degrades above *t* = 0.0025:
|
126 |
|
127 |
+
- *t* = 0.0001 (~0.8% full swap): [QuartetAnemoi-70B-t0.0001](https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001)
|
128 |
- *t* = 0.0003 (~2% full swap)
|
129 |
- *t* = 0.001 (~10% full swap): This model
|
130 |
- *t* = 0.0025 (~18% full swap): Generates one paragraph okay, but then reverts to garbage
|