GitHub repository: phospho-app/fastassert
Primary language: Jupyter Notebook
Created: 2024-02-13T04:23:29Z
Description: Dockerized LLM inference server with constrained output (JSON mode), built on top of vLLM. Faster, cheaper and without rate limits. Compare the quality and latency to your current LLM API provider.