Spaces:
Running
on
A100
Running
on
A100
How is the model so fast and accurate?
#17
by
Saugatkafley
- opened
I am really impressed by how fast it can generate excellent answers almost instantly. What was used behind this low latency inference?
This demo uses TGI https://github.com/huggingface/text-generation-inference as a backend.
Thank you so much! @olivierdehaene . It is really fast!