GitHub repository: perk11/large-model-proxy
Primary language: Go
Created: 2024-07-21T04:42:50Z
Description: Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand