Repository: vllm-project/guidellm
Issue: Guidellm fails with SGLang 0.4.6 due to unlimited output token generation
Root cause: External Dependency Breakage

**Describe the bug**
Guidellm benchmarking fails when testing against SGLang 0.4.6 servers. While the same configuration works with SGLang 0.5.5.post3, requests sent to SGLang 0.4.6 result in extremely long generation sequences (hundreds to thousands of tokens), causing requests to hang and constraints like --max-requests to not work properly.

**Expected behavior**
guidellm can normally access sglang's OpenAI API interface

**Environment**
Include all relevant environment information:
1. OS [e.g. Ubuntu 22.04]:
2. Python version [e.g. 3.13.7]:

**To Reproduce**
Exact steps to reproduce the behavior:
   - Guidellm version: latest (from source)
   - Working SGLang version: 0.5.5.post3
   - Non-working SGLang version: 0.4.6
   - Model: Qwen3-32B
   - Test command:

   1   guidellm benchmark \
   2     --target "http://ip:port" \
   3     --processor "/root/qwen-model/tokenizer/qwen3-32b" \
   4     --rate-type "concurrent" \
   5     --rate 1 \
   6     --max-requests 1 \
   7     --data "prompt_tokens=32,output_tokens=32,samples=1"
  When using guidellm to benchmark against SGLang 0.4.6, the following issues occur:

   1. Unlimited token generation: Requests with output_tokens=64 generate hundreds or thousands of tokens instead of respecting the specified limit
   2. Request state management failure: The generated requests don't complete properly in guidellm, causing --max-requests constraints to fail
   
**Errors**

<img width="3014" height="1160" alt="Image" src="https://github.com/user-attachments/assets/cdaaaacd-f256-4841-af1a-70d64fcd3354" />

<img width="2260" height="1454" alt="Image" src="https://github.com/user-attachments/assets/cd7bd032-a9cb-4fe3-82cd-c0dde28047b4" />
Parameter mapping issue: Guidellm's output_tokens=64 parameter does not properly map to SGLang 0.4.6's max_tokens parameter

**Additional context**
Add any other context about the problem here. Also include any relevant files.


To me that sounds like something was broken in SGLang 0.4.6 and has been fixed in-between 0.4.6 and 0.5.5.post3. You can try switching to the `/v1/completions` endpoint with `--request-type text_completions` or omitting `output_tokens` to see if the server behaves better.

Switching to the /v1/completions endpoint is working normally.Configuring output_tokens as empty resulted in an error. It seems to be a required parameter.

guidellm benchmark \
  --target "http://127.0.0.1:30000" \
  --processor "/root/data/Qwen3-32B" \
  --rate-type "concurrent" \
  --rate 1 \
  --max-requests 1 \
  --request-type text_completions \
  --data "prompt_tokens=32,output_tokens=32,samples=1"
✔ OpenAIHTTPBackend backend validated with model /root/data/Qwen3-32B
  {'target': 'http://127.0.0.1:30000', 'model': None, 'timeout': 60.0, 'http2': True, 'follow_redirects': True, 'verify': False, 'openai_paths': {'health': 'health', 'models':            
  'v1/models', 'text_completions': 'v1/completions', 'chat_completions': 'v1/chat/completions', 'audio_transcriptions': 'v1/audio/transcriptions', 'audio_translations':                   
  'v1/audio/translations'}, 'validate_backend': {'method': 'GET', 'url': 'http://127.0.0.1:30000/health'}}                                                                                 
✔ Processor resolved
  Using processor '/root/data/Qwen3-32B'                                                                                                                                                   
✔ Request loader initialized with inf unique requests
  {'data': "['prompt_tokens=32,output_tokens=32,samples=1']", 'data_args': '[]', 'data_samples': -1, 'preprocessors': ['GenerativeColumnMapper',                                           
  'GenerativeTextCompletionsRequestFormatter'], 'collator': 'GenerativeRequestCollator', 'sampler': 'None', 'num_workers': 1, 'random_seed': 42}                                           
✔ Resolved transient phase configurations
  Warmup: percent=None value=None mode='prefer_duration'                                                                                                                                   
  Cooldown: percent=None value=None mode='prefer_duration'                                                                                                                                 
  Rampup (Throughput/Concurrent): 0.0                                                                                                                                                      
✔ ConcurrentProfile profile resolved
  {'str': "type_='concurrent' completed_strategies=[] constraints={'max_requests': 1} rampup_duration=0.0 streams=[1] strategy_types=['concurrent']", 'type': 'ConcurrentProfile', 'class':
  'ConcurrentProfile', 'module': 'guidellm.benchmark.profiles', 'attributes': {'type_': 'concurrent', 'completed_strategies': [], 'constraints': {'max_requests': 1}, 'rampup_duration':   
  0.0, 'streams': [1]}}                                                                                                                                                                    
✔ Output formats resolved
  {'json': "output_path=PosixPath('/root/data/jxj/guidellm')", 'csv': "output_path=PosixPath('/root/data/jxj/guidellm')", 'html': "output_path=PosixPath('/root/data/jxj/guidellm')"}      
✔ Setup complete, starting benchmarks...



╭─ Benchmarks ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [08:15:35]   100% concurrent@1 (complete)   Req:    1.3 req/s,    0.79s Lat,     1.0 Conc,       1 Comp,        0 Inc,        0 Err                                                     │
│                                             Tok:   43.9 gen/s,   87.9 tot/s,  66.2ms TTFT,   23.5ms ITL,    32 Prompt,       32 Gen 
------------------------------------model log
[2025-11-24 08:15:35] INFO:     127.0.0.1:55108 - "POST /v1/completions HTTP/1.1" 200 OK
[2025-11-24 08:15:35 TP0] Prefill batch. #new-seq: 1, #new-token: 32, #cached-token: 0, token usage: 0.00, #running-req: 0, #queue-req: 0
[2025-11-24 08:15:35 TP0] Decode batch. #running-req: 1, #token: 40, token usage: 0.00, gen throughput (token/s): 0.01, #queue-req: 0

> Configuring output_tokens as empty resulted in an error. It seems to be a required parameter.

My bad, we have a future planned improvement to synthetic data where this should be possible.

As for `/v1/chat/completions` I quickly glanced through SGLang's release notes and found that `max_completion_tokens` is not supported until v0.4.7 ([ref](https://github.com/sgl-project/sglang/pull/5857)). Not sure why it hangs the request rather than ignoring the field, but that is a question to ask the SGLang community.