LilyWinter commited on
Commit
d7d8ba8
·
verified ·
1 Parent(s): c4df438

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -0
README.md CHANGED
@@ -115,10 +115,16 @@ A merge using a custom algorithm (NearSwap) of:
115
 
116
  <img src=https://huggingface.co/alchemonaut/BoreanGale-70B/resolve/main/bg.png>
117
 
 
 
 
 
 
118
  NearSwap retains most of the weights of the base model (Miqu), but when a weight is similar between the two, it is interpolated to the secondary model (WinterGoddess) value. A parameter *t* specifies the sameness threshold. When the distance between two values is below *t*, the weight from the secondary model (WinterGoddess) is used.
119
 
120
  This version of the model uses *t* = 0.001. At this *t*, about 10% of weights are fully switched to WinterGoddess. Model quality rapidly degrades above *t* = 0.0025:
121
 
 
122
  - *t* = 0.0003 (~2% full swap)
123
  - *t* = 0.001 (~10% full swap): This model
124
  - *t* = 0.0025 (~18% full swap): Generates one paragraph okay, but then reverts to garbage
 
115
 
116
  <img src=https://huggingface.co/alchemonaut/BoreanGale-70B/resolve/main/bg.png>
117
 
118
+ <br/>
119
+ <br/>
120
+
121
+ # NearSwap Algorithm
122
+
123
  NearSwap retains most of the weights of the base model (Miqu), but when a weight is similar between the two, it is interpolated to the secondary model (WinterGoddess) value. A parameter *t* specifies the sameness threshold. When the distance between two values is below *t*, the weight from the secondary model (WinterGoddess) is used.
124
 
125
  This version of the model uses *t* = 0.001. At this *t*, about 10% of weights are fully switched to WinterGoddess. Model quality rapidly degrades above *t* = 0.0025:
126
 
127
+ - *t* = 0.0001 (~0.8% full swap): [QuartetAnemoi-70B-t0.0001](https://huggingface.co/alchemonaut/QuartetAnemoi-70B-t0.0001)
128
  - *t* = 0.0003 (~2% full swap)
129
  - *t* = 0.001 (~10% full swap): This model
130
  - *t* = 0.0025 (~18% full swap): Generates one paragraph okay, but then reverts to garbage