PROMTEC : Fast LLM Inference Decoding using Prompt Multi-Lookup with Template Database and Common Sequences (2025)

Abstract: We propose PROMTEC, a novel multi-faceted approach to accelerate the inference of large language models (LLMs) by leveraging three key techniques: Prompt Multi-Lookup, Template Datastore, and Common Sequences methods. Prompt Multi-Lookup enhances the autoregressive decoding efficiency by generating multiple candidate sequences from context. Template Datastore exploits structured patterns, particularly in mathematical and code generation tasks, to enable fast and accurate candidate generation. Common Sequences optimize inference by precomputing frequent short sequences in specialized domains. For mathematical generation, PROMTEC achieves a 3.91 \times speedup on the miniF2F benchmark. For code generation, it achieves up to a 4.23 \times speedup on the HumanEval benchmark. This work highlights the potential of integrated candidate generation to accelerate LLM inference while maintaining high-quality outputs.

## introduction large language models (llms) are crucial in numerous applications, including question answering, program synthesis, and task automation @xcite @xcite . however, the significant inference costs and time-consuming autoregressive decoding processes pose substantial challenges. the need for efficient token generation is heightened by inference-time scaling, which requires generating longer outputs for complex tasks @xcite . multi-agent and pipelined llm systems improve accuracy and reliability but suffer from long response times due to sequential processing stages @xcite . speculative decoding has emerged as a promising technique to accelerate llm output speed * corresponding author @xcite @xcite @xcite . this method allows llms to verify multiple tokens in a single forward pass using small draft models or additional decoding heads, reducing latency. however, it faces limitations such as the need for high-quality draft models, increased gpu memory usage, and orchestration complexity @xcite . recent advancements have integrated additional decoding heads directly into the llm, utilizing its final hidden representations @xcite . while addressing some challenges of separate draft models, this approach still requires fine-tuning and additional gpu memory, which can be substantial for large models @xcite . complementing speculative decoding, tree attention modifies traditional attention mechanisms to verify multiple sequences more efficiently @xcite . by structuring tokens hierarchically, tree attention allows parallel verification of multiple speculative paths, enhancing decoding efficiency. despite these advancements, there remains substantial room for improvement in the speed and efficiency of llm inference. we propose promtec, which integrates several novel methods to further accelerate the autoregressive decoding process. our contributions include: ## conclusion this study introduced a novel approach to enhancing the efficiency of large language models (llms) by integrating prompt multi-lookup, template datastore, and common sequences methods. these techniques collectively improve the speed of autoregressive decoding by generating multiple candidate sequences efficiently. our evaluations on the minif2f benchmark demonstrate significant speedups, achieving up to a 3.91× improvement over standard methods. additionally, on the humaneval benchmark for code generation, our methods achieve up to a 4.23× speedup. this approach outperforms existing speculative decoding techniques, offering a scalable solution for faster llm inference while maintaining high-quality outputs. moreover, we performed an ablation study to understand the individual contributions of each component to the overall performance improvement. future work can explore further optimizations and applications to larger models and other domains.