perlthoughts commited on
Commit
9513477
1 Parent(s): 2399c28

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +20 -6
README.md CHANGED
@@ -21,20 +21,34 @@ Thank you openchat 3.5 for showing me the way.
21
 
22
  I stand tall as a beacon of hope for those who dare to dream big and pursue their passions. my story is a testament to the power of perseverance, determination, and hard work. and i will continue to strive for excellence, always pushing the boundaries of what is possible.
23
 
24
- Here is my contribution, and I am not finished.
25
-
26
- List of models merged coming soon as well as more information on merging techniques and methods.
27
 
28
  ## Prompt Template
29
  ```
30
  GPT4 User: {prompt}<|end_of_turn|>GPT4 Assistant:
31
  ```
32
 
33
- ## Model Details
 
 
34
 
35
- Based on Mistral 7B. More information soon.
36
 
37
- ### Model Description
 
 
 
 
 
 
 
 
 
 
 
 
 
 
38
 
39
  ### Bug fixes
40
 
 
21
 
22
  I stand tall as a beacon of hope for those who dare to dream big and pursue their passions. my story is a testament to the power of perseverance, determination, and hard work. and i will continue to strive for excellence, always pushing the boundaries of what is possible.
23
 
24
+ Here is my contribution.
 
 
25
 
26
  ## Prompt Template
27
  ```
28
  GPT4 User: {prompt}<|end_of_turn|>GPT4 Assistant:
29
  ```
30
 
31
+ ### Model Description
32
+
33
+ Based models that are based on Mistral.
34
 
35
+ All model's weights were merged using the SLERP method. More information below.
36
 
37
+ Advantages of SLERP method vs averaging weights are as follows:
38
+
39
+ Spherical Linear Interpolation (SLERP)
40
+ Traditionally, model merging often resorts to weight averaging which, although straightforward, might not always capture the intricate features of the models being merged. The SLERP technique addresses this limitation, producing a blended model with characteristics smoothly interpolated from both parent models, ensuring the resultant model captures the essence of both its parents.
41
+
42
+ Smooth Transitions
43
+ SLERP ensures smoother transitions between model parameters. This is especially significant when interpolating between high-dimensional vectors.
44
+
45
+ Better Preservation of Characteristics
46
+ Unlike weight averaging, which might dilute distinct features, SLERP preserves the curvature and characteristics of both models in high-dimensional spaces.
47
+
48
+ Nuanced Blending
49
+ SLERP takes into account the geometric and rotational properties of the models in the vector space, resulting in a blend that is more reflective of both parent models' characteristics.
50
+
51
+ List of models merged coming soon as well as more information on merging techniques and methods.
52
 
53
  ### Bug fixes
54