Spaces:
Paused
JSON response format when using custom model?
I followed https://github.com/huggingface/chat-ui to deploy a version in gcp, and I love the chat interface.
I would love to hook it up to one of my custom models, so I specified the
"endpoints": [{"url": "http://127.0.0.1:8000"}]
}
]`
for MODELS as suggested.
I receive the message that has been posted in the web interface at my endpoint, but I am unable to send back the proper json response. So far, in python, I do:
response_content = [
{
"generated_text": "Please show this response."
}
]
response = make_response(jsonify(response_content))
return response
It is received in the chat-ui code (confirmed by injecting console.log statements), but it doesn't show in the browser conversation.
Can someone please clarify what json (content, headers, whatever is needed) I need to send from my custom model endpoint as a response to the chat-ui interface? Or if this is the wrong place to ask, tell me where I should ask?
I'm also having a hard time figuring out how to do this. It seems they are using an event stream and there is poor error handling so if you get the format wrong the only information you get is a stream error like the following:
TypeError [ERR_INVALID_STATE]: Invalid state: Controller is already closed
at new NodeError (node:internal/errors:393:5)
at ReadableStreamDefaultController.enqueue (node:internal/webstreams/readablestream:1008:13)
at update (/Users/alfred/Projects/ProjectElephant/chat-ui/src/routes/conversation/[id]/+server.ts:161:20)
at eval (/Users/alfred/Projects/ProjectElephant/chat-ui/src/routes/conversation/[id]/+server.ts:171:13)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.start (/Users/alfred/Projects/ProjectElephant/chat-ui/src/routes/conversation/[id]/+server.ts:222:7) {
code: 'ERR_INVALID_STATE'
}