Does this work with HF Inference API?
#18
by
sagardesai
- opened
I was met with this when I tried running replit-code-v1-3b via the Inference API:
Loading replit/replit-code-v1-3b requires you to execute the configuration file in that repo on your local machine. Make sure you have read the code there to avoid malicious use, then set the option trust_remote_code=True to remove this error.
I was wondering if/why it would be necessary to do anything locally , haven't had to do that for any other models like Google's flan-t5-xxl.
Thanks!
You will need a custom handler. I have requested that the HF team write an example of one using the MPT architecture, so hopefully that will be added to the docs soon.
