Inference Endpoints #12

by nynewco - opened

Hi so I wanted to add two model used in this space to Inference Endpoints and I wanted to know what is the process?
like did you make 2 separate endpoint? since your script is using just one endpoint how does that work? also did you use any custom handler?

hi @nynewco tagging @philschmid for the Inference Endpoints question

Hello @nynewco,

Depending on how big the models are you could create a single endpoint with a customer handler see:

hi @philschmid thanks for your info. could you send me a sample of this space multi model list?
should i just replace models in multi model dict to blips 2 model? can you give me a reference to this space

The attached blog posts shows you how you can do this

Sign up or log in to comment