DavidAU commited on
Commit
68c2691
1 Parent(s): c099520

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -0
README.md CHANGED
@@ -100,6 +100,58 @@ of these quants will function better.
100
 
101
  Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
102
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
103
  <B>Models Used:</b>
104
 
105
  This is a high precision "DARE TIES" merge at the layer level (each layer per model adjusted - 168 points of adjustment over the 4 models) comprised of these models:
 
100
 
101
  Rep pen adjustments may also be required to get the most out of this model at this/these quant level(s).
102
 
103
+ <B>Brainstorm 5x</B>
104
+
105
+ The BRAINSTORM process was developed by David_AU.
106
+
107
+ Some of the core principals behind this process are discussed in this <a href="https://arxiv.org/pdf/2401.02415">
108
+ scientific paper : Progressive LLaMA with Block Expansion </a>.
109
+
110
+ However I went in a completely different direction from what was outlined in this paper.
111
+
112
+ What is "Brainstorm" ?
113
+
114
+ The reasoning center of an LLM is taken apart, reassembled, and expanded.
115
+
116
+ In this case for this model: 5 times
117
+
118
+ Then these centers are individually calibrated. These "centers" also interact with each other.
119
+ This introduces subtle changes into the reasoning process.
120
+ The calibrations further adjust - dial up or down - these "changes" further.
121
+ The number of centers (5x,10x etc) allow more "tuning points" to further customize how the model reasons so to speak.
122
+
123
+ The core aim of this process is to increase the model's detail, concept and connection to the "world",
124
+ general concept connections, prose quality and prose length without affecting instruction following.
125
+
126
+ This will also enhance any creative use case(s) of any kind, including "brainstorming", creative art form(s) and like case uses.
127
+
128
+ Here are some of the enhancements this process brings to the model's performance:
129
+
130
+ - Prose generation seems more focused on the moment to moment.
131
+ - Sometimes there will be "preamble" and/or foreshadowing present.
132
+ - Fewer or no "cliches"
133
+ - Better overall prose and/or more complex / nuanced prose.
134
+ - A greater sense of nuance on all levels.
135
+ - Coherence is stronger.
136
+ - Description is more detailed, and connected closer to the content.
137
+ - Simile and Metaphors are stronger and better connected to the prose, story, and character.
138
+ - Sense of "there" / in the moment is enhanced.
139
+ - Details are more vivid, and there are more of them.
140
+ - Prose generation length can be long to extreme.
141
+ - Emotional engagement is stronger.
142
+ - The model will take FEWER liberties vs a normal model: It will follow directives more closely but will "guess" less.
143
+ - The MORE instructions and/or details you provide the more strongly the model will respond.
144
+ - Depending on the model "voice" may be more "human" vs original model's "voice".
145
+
146
+ Other "lab" observations:
147
+
148
+ - This process does not, in my opinion, make the model 5x or 10x "smarter" - if only that was true!
149
+ - However, a change in "IQ" was not an issue / a priority, and was not tested or calibrated for so to speak.
150
+ - From lab testing it seems to ponder, and consider more carefully roughly speaking.
151
+ - You could say this process sharpens the model's focus on it's task(s) at a deeper level.
152
+
153
+ The process to modify the model occurs at the root level - source files level. The model can quanted as a GGUF, EXL2, AWQ etc etc.
154
+
155
  <B>Models Used:</b>
156
 
157
  This is a high precision "DARE TIES" merge at the layer level (each layer per model adjusted - 168 points of adjustment over the 4 models) comprised of these models: