GitHub repository: softmaxer/batchollama
Primary language: Go
Created: 2024-03-07T10:18:51Z
Description: Run Ollama inference in batches, where each batch executes as prompts concurrently.