fernandofernandes commited on
Commit
df30192
1 Parent(s): 91e0a33

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -2
README.md CHANGED
@@ -2,10 +2,14 @@
2
  license: cc-by-nc-2.0
3
  ---
4
 
 
 
5
  An experimentation regarding 'lasering' each expert to denoise and enhance model capabilities.
6
 
7
- This follows the implementation of laserRMT @ https://github.com/cognitivecomputations/laserRMT
 
 
8
 
9
  Here, we are controlling layers checking which ones have lower signal to noise ratios (which are more subject to noise), to apply Laser interventions, still using Machenko Pastur to calculate this ratio.
10
 
11
- We are looking forward to measure its evaluation over the most important benchmarks.
 
2
  license: cc-by-nc-2.0
3
  ---
4
 
5
+ by David, Fernando and Eric
6
+
7
  An experimentation regarding 'lasering' each expert to denoise and enhance model capabilities.
8
 
9
+ This model has half size in comparison to the Mixtral 8x7b Instruct. And it basically has the same level of performance (we are working to get a better MMLU score).
10
+
11
+ It follows the implementation of laserRMT @ https://github.com/cognitivecomputations/laserRMT
12
 
13
  Here, we are controlling layers checking which ones have lower signal to noise ratios (which are more subject to noise), to apply Laser interventions, still using Machenko Pastur to calculate this ratio.
14
 
15
+ We intend to be the first of a family of experimentations being carried out @ Cognitive Computations.