Possible to run in CPU in offline mode?
#6
by manged - opened
Would it be possible to run the model in offline mode in CPU? sample code shows it needs hf token. I'd prefer to download the model in local machine and not fetch data every time.
Yes, it is absolutely possible to run locally in offline mode.
The first time you run the code it saves the model in your system cache. The next time you run the code, it fetches from the same cache if internet is off.
It will definitely run on CPU, and we were even able to run it on mobile phones.