AttributeError: 'Qwen2_5OmniConfig' object has no attribute 'num_attention_heads'
Author: jieguolove
Pull request: no
Labels: name: bug

### System Info

root@445d74596699:/vllm-workspace# transformers-cli env 

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.52.0.dev0
- Platform: Linux-5.15.0-43-generic-x86_64-with-glibc2.35
- Python version: 3.12.9
- Huggingface_hub version: 0.30.2
- Safetensors version: 0.5.3
- Accelerate version: 1.5.2
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA L20


`(base) root@node15:/disk2/Qwen2.5-Omni-7B# more docker-compose.yml 
#version: '3.3'
services:
  # vllm
  vllm-openai:
    image: vllm/vllm-openai:v0.8.2
    container_name: Qwen2.5-Omni-7B
    restart: unless-stopped
    runtime: nvidia
    ports:
      - 8007:8000
    volumes:
      - /disk2:/models
    command: >
      --model /models/Qwen2.5-Omni-7B
      --tokenizer_mode="auto"
      --trust-remote-code
      --dtype=bfloat16
      --max_num_seqs=256
      --tensor_parallel_size=1
      --gpu-memory-utilization=0.9
      --max-model-len=65536
      --served-model-name=Qwen2.5-Omni-7B
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: [gpu]
              device_ids: [ "1" ]
    ipc: host
networks:
        vllm:
(base) root@node15:/disk2/Qwen2.5-Omni-7B# docker commit 445d74596699 vllm/vllm-openai:v0.8.2
sha256:fdf1171c4bc4edc473bb3857597124ae73176c1691a27befccb4360c81ff0d60
(base) root@node15:/disk2/Qwen2.5-Omni-7B# docker compose -f docker-compose.yml up -d
[+] Running 2/2
 ✔ Network qwen25-omni-7b_default  Created                                                                                                                     0.0s 
 ✔ Container Qwen2.5-Omni-7B       Started                                                                                                                     0.6s 
(base) root@node15:/disk2/Qwen2.5-Omni-7B# docker logs -f Qwen2.5-Omni-7B 
INFO 04-15 00:06:11 [__init__.py:239] Automatically detected platform cuda.
INFO 04-15 00:06:13 [api_server.py:981] vLLM API server version 0.8.2
INFO 04-15 00:06:13 [api_server.py:982] args: Namespace(host=None, port=8000, uvicorn_log_level='info', disable_uvicorn_access_log=False, allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, enable_ssl_refresh=False, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_request_id_headers=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/models/Qwen2.5-Omni-7B', task='auto', tokenizer=None, hf_config_path=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=True, allowed_local_media_path=None, download_dir=None, load_format='auto', config_format=<ConfigFormat.AUTO: 'auto'>, dtype='bfloat16', kv_cache_dtype='auto', max_model_len=65536, guided_decoding_backend='xgrammar', logits_processor_pattern=None, model_impl='auto', distributed_executor_backend=None, pipeline_parallel_size=1, tensor_parallel_size=1, enable_expert_parallel=False, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=None, enable_prefix_caching=None, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=None, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_partial_prefills=1, max_long_partial_prefills=1, long_prefill_token_threshold=0, max_num_seqs=256, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, hf_overrides=None, enforce_eager=False, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, disable_mm_preprocessor_cache=False, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, use_tqdm_on_load=True, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_config=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['Qwen2.5-Omni-7B'], qlora_adapter_name_or_path=None, show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', scheduler_cls='vllm.core.scheduler.Scheduler', override_neuron_config=None, override_pooler_config=None, compilation_config=None, kv_transfer_config=None, worker_cls='auto', worker_extension_cls='', generation_config='auto', override_generation_config=None, enable_sleep_mode=False, calculate_kv_scales=False, additional_config=None, enable_reasoning=False, reasoning_parser=None, disable_cascade_attn=False, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False, enable_server_load_tracking=False)
Unrecognized keys in `rope_scaling` for 'rope_type'='default': {'mrope_section'}
INFO 04-15 00:06:22 [config.py:585] This model supports multiple tasks: {'reward', 'generate', 'classify', 'score', 'embed'}. Defaulting to 'generate'.
INFO 04-15 00:06:22 [config.py:1697] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 04-15 00:06:24 [core.py:54] Initializing a V1 LLM engine (v0.8.2) with config: model='/models/Qwen2.5-Omni-7B', speculative_config=None, tokenizer='/models/Qwen2.5-Omni-7B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.bfloat16, max_seq_len=65536, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=False, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar', reasoning_backend=None), observability_config=ObservabilityConfig(show_hidden_metrics=False, otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=None, served_model_name=Qwen2.5-Omni-7B, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=True, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"level":3,"custom_ops":["none"],"splitting_ops":["vllm.unified_attention","vllm.unified_attention_with_output"],"use_inductor":true,"compile_sizes":[],"use_cudagraph":true,"cudagraph_num_of_warmups":1,"cudagraph_capture_sizes":[512,504,496,488,480,472,464,456,448,440,432,424,416,408,400,392,384,376,368,360,352,344,336,328,320,312,304,296,288,280,272,264,256,248,240,232,224,216,208,200,192,184,176,168,160,152,144,136,128,120,112,104,96,88,80,72,64,56,48,40,32,24,16,8,4,2,1],"max_capture_size":512}
WARNING 04-15 00:06:25 [utils.py:2321] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x7fabea685df0>
INFO 04-15 00:06:26 [parallel_state.py:954] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0
ERROR 04-15 00:06:26 [core.py:343] EngineCore hit an exception: Traceback (most recent call last):
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 335, in run_engine_core
ERROR 04-15 00:06:26 [core.py:343]     engine_core = EngineCoreProc(*args, **kwargs)
ERROR 04-15 00:06:26 [core.py:343]                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 290, in __init__
ERROR 04-15 00:06:26 [core.py:343]     super().__init__(vllm_config, executor_class, log_stats)
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/engine/core.py", line 60, in __init__
ERROR 04-15 00:06:26 [core.py:343]     self.model_executor = executor_class(vllm_config)
ERROR 04-15 00:06:26 [core.py:343]                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/executor_base.py", line 52, in __init__
ERROR 04-15 00:06:26 [core.py:343]     self._init_executor()
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 46, in _init_executor
ERROR 04-15 00:06:26 [core.py:343]     self.collective_rpc("init_device")
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/executor/uniproc_executor.py", line 56, in collective_rpc
ERROR 04-15 00:06:26 [core.py:343]     answer = run_method(self.driver_worker, method, args, kwargs)
ERROR 04-15 00:06:26 [core.py:343]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/utils.py", line 2255, in run_method
ERROR 04-15 00:06:26 [core.py:343]     return func(*args, **kwargs)
ERROR 04-15 00:06:26 [core.py:343]            ^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/worker/worker_base.py", line 604, in init_device
ERROR 04-15 00:06:26 [core.py:343]     self.worker.init_device()  # type: ignore
ERROR 04-15 00:06:26 [core.py:343]     ^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_worker.py", line 120, in init_device
ERROR 04-15 00:06:26 [core.py:343]     self.model_runner: GPUModelRunner = GPUModelRunner(
ERROR 04-15 00:06:26 [core.py:343]                                         ^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/v1/worker/gpu_model_runner.py", line 106, in __init__
ERROR 04-15 00:06:26 [core.py:343]     self.num_kv_heads = model_config.get_num_kv_heads(parallel_config)
ERROR 04-15 00:06:26 [core.py:343]                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/config.py", line 884, in get_num_kv_heads
ERROR 04-15 00:06:26 [core.py:343]     total_num_kv_heads = self.get_total_num_kv_heads()
ERROR 04-15 00:06:26 [core.py:343]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/vllm/config.py", line 876, in get_total_num_kv_heads
ERROR 04-15 00:06:26 [core.py:343]     return self.hf_text_config.num_attention_heads
ERROR 04-15 00:06:26 [core.py:343]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343]   File "/usr/local/lib/python3.12/dist-packages/transformers/configuration_utils.py", line 211, in __getattribute__
ERROR 04-15 00:06:26 [core.py:343]     return super().__getattribute__(key)
ERROR 04-15 00:06:26 [core.py:343]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ERROR 04-15 00:06:26 [core.py:343] AttributeError: 'Qwen2_5OmniConfig' object has no attribute 'num_attention_heads'
ERROR 04-15 00:06:26 [core.py:343] 
CRITICAL 04-15 00:06:26 [core_client.py:269] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

![Image](https://github.com/user-attachments/assets/d13588fb-b462-487a-9ebf-1afb8024bf7f)

### Expected behavior

no error

Comments:
hey @jieguolove !

The `num_attention_heads` is present in the config, but is located under text config which is `Qwen2_5OmniTextConfig`. I believe vLLM tries to infer text config from `config.get_text_config()` which is causing issues since the config is deeply nested. In that case the best would be for vLLM to infer text config with `config.thinker_config.get_text_config()`

Tagging @hmellor here from vLLM team

vLLM does not have model specific config loading. Ideally we would not have to have special cases like this.

Is there a way to reliably get the text config from the top level pretrained config?

Update from internal conversation: we'll be fixing it on vLLM side if possible and add `config.get_text_config()` support for deeply nested configs as qwen-omni in transformers