Preforms really well, what did you use?
This model does really well, better than the intel default in my tests. What did you use do to the merge?
Thats nice! I will try and merge some more in the future.
And i used the mergekit here: https://github.com/cg123/mergekit
Didn't know it supported mistral yet, thanks :)
is it heavy on hardware the merging ?
I would say a bit, but it depends on what we are working with. I merged this 11b with my RTX 2060 mobile and it worked fine.
I would say a bit, but it depends on what we are working with. I merged this 11b with my RTX 2060 mobile and it worked fine.
You did it with an RTX 2060 Mobile?
S4sch, do you mind writing a step by step tutorial about how you did that, beginner friendly of course (And maybe send me through Discord) I would love to do some too. But I am utterly useless with coding and whatnot.
I would say a bit, but it depends on what we are working with. I merged this 11b with my RTX 2060 mobile and it worked fine.
Wow man that's superb.
You did it with an RTX 2060 Mobile?
S4sch, do you mind writing a step by step tutorial about how you did that, beginner friendly of course (And maybe send me through Discord) I would love to do some too. But I am utterly useless with coding and whatnot.
I would also love a tutorial of how you do it please😁
Yes, i could do that. Right now i even a students project in which i will present merging, so currently i am designing an .ipynb file which automatizes the entire process for merging, quantization and the testing and does all the necessary installs. The goal is that at the end, you only would have to define the models you want to merge and everything should be done. I can tell you as soon as im closer to finishing that.
Yes, i could do that. Right now i even a students project in which i will present merging, so currently i am designing an .ipynb file which automatizes the entire process for merging, quantization and the testing and does all the necessary installs. The goal is that at the end, you only would have to define the models you want to merge and everything should be done. I can tell you as soon as im closer to finishing that.
That would be awesome, if you literally search for mergekit or merging models and everything else on Youtube, Google and etc... You cannot find a single tutorial about it, we could have a bunch of more people growing the entire ecosystem if these process could be easily tutored or like you're trying to do automated.
Yeah i feel you, i also realized at the beginning when i came into it, how little there is about it and that it was quite difficult to go into. The best forum i would recommend is https://www.reddit.com/r/LocalLLaMA/. There you find atleast some information for some of the things in the open source model space. I'm not sure whether there is anything specific about how to merge though, but there might be.