Perfect MoE's my write up, and help to you for making MoE's
#6
by
rombodawg
- opened
Ive created a write up google doc for perfecting the formula for creating MoE models using mergkit. You can find it in the link bellow, Im calling the community (at the very least the people who intend to create models) to read it, use the knowledge to experiment and come back and leave comments on the write up doc for the whole community to share the knowledge and improve our ability to create better ai merged and MoE models.
https://docs.google.com/document/d/1_vOftBnrk9NRk5h10UqrfJ5CDih9KBKL61yvrZtVWPE/edit?usp=sharing