Hugging Face's logo

TehVenom
/
MPT-7b-Chat-Instruct-LongCTX-Merge

Text Generation
Transformers
PyTorch
mpt
custom_code
text-generation-inference
Inference Endpoints
Model card Files Community
4

weighted average?

#4
by winglian - opened
Discussion
winglian

Is this simply a weighted average of each parameter? Could you share the merging code please? thanks!

Henk717

https://github.com/TehVenomm/LM_Transformers_BlockMerge

Sign up or log in to comment