GitHub repository: James-QiuHaoran/LLM-serving-with-proxy-models
Primary language: Python
Created: 2024-04-12T14:10:07Z
Description: Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction