The Impact of Hyperparameters on Large Language Model Inference Performance: An Evaluation of vLLM and HuggingFace Pipelines
Paper
•
2408.01050
•
Published
•
8
Inference Optimization
-
-
Efficient Inference of Vision Instruction-Following Models with Elastic Cache
Paper • 2407.18121 • Published • 15 -
LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
Paper • 2407.14057 • Published • 44 -
Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
Paper • 2407.10969 • Published • 20 -
Inference Performance Optimization for Large Language Models on CPUs
Paper • 2407.07304 • Published • 52