Is it possible to host this in a mobile device locally? or have anyone tried it already?
When running on Colab it takes around 15Gb of vRAM, taking around 1 min per response. I don't think phones now days can handle that, but I could be wrong.
· Sign up or log in to comment
