Clarify vLLM CUDA graph sizing and refine model card details
#3
by oliverholworthy - opened
Clarifies the vLLM deployment guidance and makes a small set of model card corrections.
- Recommends a maximum CUDA graph capture size of 4,096 as a conservative default and 8,192 as a practical upper limit with automatic buckets.
- Documents the use of workload-specific capture sizes above 8,192 and provides illustrative cold-start measurements.
- Improves the vLLM examples and clarifies the expected configuration warning.
- Notes that QAD primarily targeted accuracy recovery for long input sequences.
- Corrects “Nepalese” to “Nepali.”
oliverholworthy changed pull request title from Clarify CUDA graph cold-start measurements to Clarify vLLM CUDA graph sizing and refine model card details
ronay-nv changed pull request status to merged