Spaces:
Running
on
Zero
Running
on
Zero
Investigate better queue
#1
by
adamelliotfields
- opened
Right now we have a single generate
function that takes a model name. Under the hood, this function is assigned a concurrency_id
which is used by Gradio's queue to form concurrency groups. It also results in a simple REST API with only one /generate
endpoint.
It might make more sense for the model to be tied to a button so multiple groups can be formed. This would also create multiple API endpoints and API requests for a model would be grouped accordingly.
For the UI, try toggling the visibility of the other buttons so it always appears as if there is a single button. Leave the current model dropdown and this button visibility is toggled on change.