serving the model
#13
by
prudant
- opened
Hi there! is there any way to serve your great model considering all the model fetures, embeddings, scores for rerank purposes, open source options or paid options for serving ?
you might consider using vespa: https://github.com/vespa-engine/pyvespa/blob/master/docs/sphinx/source/examples/mother-of-all-embedding-models-cloud.ipynb
thanks, Vespa is overkilling for now for my use cases and testing purposes, I (with the help of the gpt) build a simple but robust usable server for FAST local testing / developing purposes, this is the link:
https://github.com/puppetm4st3r/baai_m3_simple_server
feel free to share or comment
sugestions are welcome.
regards, and congrats for the great job and model :)