How to use it in an app?
#30
by
damc
- opened
If I would like to use Starcoder model in an application, what is the best way to run it in the production environment?
Also, is there some API that I can call or do I need to host it myself?
@damc You may be interested in this project: https://github.com/bigcode-project/starcoder.cpp, which provides a demo on IOS application of StarCoder.
You can also call the API endpoint of the model https://api-inference.huggingface.co/models/bigcode/starcoder/ and use text-generation-inference library like it's done in this playground
loubnabnl
changed discussion status to
closed