How does the MoE work?

by PacmanIncarnate - opened Jan 8

Jan 8

Can you explain how the gate picks between the two included models? Was there any additional merge system involved as well, or simple MoE?

Jan 8

Also, did you fine-tuned after the merge?

Jan 9

Owner Jan 9

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment