mlabonne

/

phixtral-4x2_8

like 204

Could you share your custom merge kit moe? Or the one you are currently using, I tried both merge kit 0.3.2 and merge kit 0.4.0 and both with and with out the old/new phi code and both gave me a ‘model.embed_tokens.weight” or a (model.layer)error. And on their github the owner of the repository said “phi is currently incompatible and their are no future plans to change that”