Anyway to enable longer prompts for this Space?
I'm getting
"""
Response is : <Response [500]>
output is : {'error': "This is a very long prompt, we're temporarily not accepting these"}
"""
Is it possible to enable longer prompts for this Space? LG is interesting problem and I'm sure it will give new insights
We're working on it.
We need to optimize the model and the API running it so we can run longer prompts and longer generation without blocking every other user.
This should land soon.
Stay tuned.
Cheers.
@nasril would love to learn about how the inference team handles system design for various LLM's of different sizes, can you provide pointers to some resources ?
Hi @muhtasham ,
Stay tuned on our blog https://huggingface.co/blog.
More public information is coming (Probably of the course of several blog posts in the coming month).
By the way, we already increased the size of prompts you could send. Please let us know if you find anything weird.