flexgen (pypi): Running large language models like OPT-175B/GPT-3 on a single GPU. Focusing on high-throughput large-batch generation.
Keywords: deep-learning|gpt-3|high-throughput|large-language-models|machine-learning|offloading|opt