Update README.md

README.md

@@ -7,6 +7,4 @@ An untrained precursor MoE created from Cosmo using mergekit.
 
 Gate routing initialized using prompt hidden state method. Five are based on the visualized topic clusters of Cosmopedia data, three are task-oriented.
 
-Degenerate layers are 0, 1, and 2 (I believe this means experts will be underutilized for the lowest-level features).
-Best I could do with test-and-try prompt-based routing.
-Further research might start from the reversed direction, if available in some interpretability tool (activating layer into prompts).
+Degenerate layers were 0, 1, and 2. Expert gates for layers 0, 1, and 2 have been randomly initialized to with luck mitigate this.