GitHub repository: microsoft/batch-inference
Primary language: Python
Created: 2023-04-17T07:15:40Z
Description: Dynamic batching library for Deep Learning inference. Tutorials for LLM, GPT scenarios.