Add a bitnet step to the workflow
You make a lot of models, I download a lot of quants from you....... some of these models are really large though, and people don't necessarily have the hardware spec to run them..... but you and other quantinizers could just add the bitnet workflow to your workflows, which would reduce the ram required to run the models, potentially opening up more ai accessibility since the models could then run on more devices with less ram.
thx
I'd do it myself but there are so many that need it done and I have a 3050 laptop 4gb vram 😒 plz brudder, be the one the start making ai more accessible
Hi, to do bitnet we will need to train models. We cannot train thousands of models. We are like you just a few people running this project on our own computers. Noone is paying us, it's more like we are paying for this with our hardware and electricity. We quant with main llamacpp, which provides a large range of different precisions, including very small ones
I am aware of that, however SOME folk do have access to what we need to get this done, they just for whatever reason are not doing it, when all models should be attempted to be bitnet'd and output compared to original...... what sense does it make to require everyone to need 10s to 100s of gbs of vram to run models, when we know each model could be ran faster with minimal difference just by bitnetting it..... also @mradermacher himself might have access from his job, I'd say start with the largest possible and work your way down, base and selected finetunes only.....
Hi, to do bitnet we will need to train models. We cannot train thousands of models. We are like you just a few people running this project on our own computers. Noone is paying us, it's more like we are paying for this with our hardware and electricity. We quant with main llamacpp, which provides a large range of different precisions, including very small ones
I just looked at the alleged hardware you have listed as having on your profile 💀 brudder wtf are you talking about get to work you have hella flops allegedly for this kind of project (note, all hardware listed on my page are cloud assets, for seeing what models I can run on vast.ai and Google colab)
also @mradermacher himself might have access from his job, I'd say start with the largest possible and work your way down, base and selected finetunes only.....
Im a part of the team, and I must tell you we dont have hundreds of B200 gpu hours per model to retrain each single model to be bitnet.
I just looked at the alleged hardware you have listed as having on your profile 💀
bro I put it as pi (you know, the 3.1415926535897932385 thingy?). It is not a real counter that you have to verify, you can set it to be whatever you want lol
the difference between normal quant and bitnet is the fact that you have to literally train the model. for quants it's way faster as it's just a mathematical conversion and can be done even on a raspberry pi if you are brave enough, while training a bitnet... well good luck with that, even if it's just a post training quantization aware training it's still going to take a reallyyy long time with a bunch of resources. technically possible, but it will take us years to train a bunch of models
bro I put it as pi (you know, the 3.1415926535897932385 thingy?). It is not a real counter that you have to verify, you can set it to be whatever you want lol
🗿 why would one lie about their gpus online
bro I put it as pi (you know, the 3.1415926535897932385 thingy?). It is not a real counter that you have to verify, you can set it to be whatever you want lol
🗿 why would one lie about their gpus online
Hey, not part of team mradermacher.
They're not lying, they just don't put the real number (if there is one).
Also, they do quants, not bitnet, so not sure what you want since you've already been told they don't do that.
Really strange conversation, bro.
why would one lie about their gpus online
Just a stupid counter lol
Really strange conversation, bro.
Agreed