Could you make a smaller version?
I don't know if this is a wild question, but is it possible to create a smaller version of the model that could, for example, run on a mobile phone?
Or are there perhaps smaller models you'd recommend for this purpose?
Interesting, I don't see any Phi2 variant in HF. I don't mind doing that but are you sure you can't get a heavily quantized version of this model to run on your phone? This model has a Q2_K GGUF and that might be worth trying still.
A Phi2 version would be great! If you make it, I will make it part of my current project, which should make it useful for a lot of people.
It's a 100% browser-based, local way to run smaller AI's, all designed/parameterized for specific tasks.
I will soon :) https://huggingface.co/microsoft/Phi-3-mini-128k-instruct
Fantastic :-)
Any progress?
@victunes I hope you'll still pursue this, it would be wonderful to have a smaller model that can run in the browser.