hao-ai-lab/lookaheaddecoding: Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
Category: llm
Language: Python
Topics: decoding, lookahead