Repository: triton-lang/triton

Issue #6080: Cannot find the class AttrsDescriptor
Created: 2025-03-01T04:28:16Z
Describe the bug
I encountered an error while manually compiling the triton project on ARM architecture and deploying Qwen-7B-Instruct-8B using VLLM. The compilation process went smoothly.The error message indicates that the class AttrsDescriptor cannot be found. I also couldn't find this class in the code I downloaded from GitHub, specifically in the path triton/python/triton/compiler/compiler.py. My command execution is as follows:
pip install ninja cmake wheel pybind11 # build-time dependencies
pip install -e python
Error reported as follows:
RROR 03-01 04:14:45 registry.py:327] Error in inspecting model architecture 'Qwen2ForCausalLM'
ERROR 03-01 04:14:45 registry.py:327] Traceback (most recent call last):
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 522, in _run_in_subprocess
ERROR 03-01 04:14:45 registry.py:327]     returned.check_returncode()
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/lib/python3.10/subprocess.py", line 457, in check_returncode
ERROR 03-01 04:14:45 registry.py:327]     raise CalledProcessError(self.returncode, self.args, self.stdout,
ERROR 03-01 04:14:45 registry.py:327] subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'vllm.model_executor.models.registry']' returned non-zero exit status 1.
ERROR 03-01 04:14:45 registry.py:327]
ERROR 03-01 04:14:45 registry.py:327] The above exception was the direct cause of the following exception:
ERROR 03-01 04:14:45 registry.py:327]
ERROR 03-01 04:14:45 registry.py:327] Traceback (most recent call last):
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 325, in _try_inspect_model_cls
ERROR 03-01 04:14:45 registry.py:327]     return model.inspect_model_cls()
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 287, in inspect_model_cls
ERROR 03-01 04:14:45 registry.py:327]     return _run_in_subprocess(
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 525, in _run_in_subprocess
ERROR 03-01 04:14:45 registry.py:327]     raise RuntimeError(f"Error raised in subprocess:\n"
ERROR 03-01 04:14:45 registry.py:327] RuntimeError: Error raised in subprocess:
ERROR 03-01 04:14:45 registry.py:327] /usr/lib/python3.10/runpy.py:126: RuntimeWarning: 'vllm.model_executor.models.registry' found in sys.modules after import of package 'vllm.model_executor.models', but prior to execution of 'vllm.model_executor.models.registry'; this may result in unpredictable behaviour
ERROR 03-01 04:14:45 registry.py:327]   warn(RuntimeWarning(msg))
ERROR 03-01 04:14:45 registry.py:327] Traceback (most recent call last):
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/runtime/hints.py", line 47, in 
ERROR 03-01 04:14:45 registry.py:327]     from triton.backends.compiler import AttrsDescriptor
ERROR 03-01 04:14:45 registry.py:327] ImportError: cannot import name 'AttrsDescriptor' from 'triton.backends.compiler' (/workspace/triton/python/triton/backends/compiler.py)
ERROR 03-01 04:14:45 registry.py:327]
ERROR 03-01 04:14:45 registry.py:327] During handling of the above exception, another exception occurred:
ERROR 03-01 04:14:45 registry.py:327]
ERROR 03-01 04:14:45 registry.py:327] Traceback (most recent call last):
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
ERROR 03-01 04:14:45 registry.py:327]     return _run_code(code, main_globals, None,
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
ERROR 03-01 04:14:45 registry.py:327]     exec(code, run_globals)
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 546, in 
ERROR 03-01 04:14:45 registry.py:327]     _run()
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 539, in _run
ERROR 03-01 04:14:45 registry.py:327]     result = fn()
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 288, in 
ERROR 03-01 04:14:45 registry.py:327]     lambda: _ModelInfo.from_model_cls(self.load_model_cls()))
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/registry.py", line 291, in load_model_cls
ERROR 03-01 04:14:45 registry.py:327]     mod = importlib.import_module(self.module_name)
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/lib/python3.10/importlib/init.py", line 126, in import_module
ERROR 03-01 04:14:45 registry.py:327]     return _bootstrap._gcd_import(name[level:], package, level)
ERROR 03-01 04:14:45 registry.py:327]   File "", line 1050, in _gcd_import
ERROR 03-01 04:14:45 registry.py:327]   File "", line 1027, in _find_and_load
ERROR 03-01 04:14:45 registry.py:327]   File "", line 1006, in _find_and_load_unlocked
ERROR 03-01 04:14:45 registry.py:327]   File "", line 688, in _load_unlocked
ERROR 03-01 04:14:45 registry.py:327]   File "", line 883, in exec_module
ERROR 03-01 04:14:45 registry.py:327]   File "", line 241, in _call_with_frames_removed
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/models/qwen2.py", line 40, in 
ERROR 03-01 04:14:45 registry.py:327]     from vllm.model_executor.layers.logits_processor import LogitsProcessor
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/layers/logits_processor.py", line 11, in 
ERROR 03-01 04:14:45 registry.py:327]     from vllm.model_executor.layers.vocab_parallel_embedding import (
ERROR 03-01 04:14:45 registry.py:327]   File "/workspace/vllm/vllm/model_executor/layers/vocab_parallel_embedding.py", line 137, in 
ERROR 03-01 04:14:45 registry.py:327]     def get_masked_input_and_mask(
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/init.py", line 2506, in fn
ERROR 03-01 04:14:45 registry.py:327]     return compile(
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/init.py", line 2535, in compile
ERROR 03-01 04:14:45 registry.py:327]     return torch._dynamo.optimize(
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 852, in optimize
ERROR 03-01 04:14:45 registry.py:327]     return _optimize(rebuild_ctx, *args, **kwargs)
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 906, in _optimize
ERROR 03-01 04:14:45 registry.py:327]     backend = get_compiler_fn(backend)
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 795, in get_compiler_fn
ERROR 03-01 04:14:45 registry.py:327]     from .repro.after_dynamo import wrap_backend_debug
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/repro/after_dynamo.py", line 16, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._dynamo.debug_utils import (
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/debug_utils.py", line 25, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._dynamo.testing import rand_strided
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/testing.py", line 28, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._dynamo.backends.debugging import aot_eager
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/backends/debugging.py", line 10, in 
ERROR 03-01 04:14:45 registry.py:327]     from functorch.compile import min_cut_rematerialization_partition
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/functorch/compile/init.py", line 2, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._functorch.aot_autograd import (
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/aot_autograd.py", line 38, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._inductor.output_code import OutputCode
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/output_code.py", line 46, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._inductor.cudagraph_utils import (
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_utils.py", line 10, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._inductor.utils import InputType
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/utils.py", line 49, in 
ERROR 03-01 04:14:45 registry.py:327]     from torch._inductor.runtime.hints import DeviceProperties
ERROR 03-01 04:14:45 registry.py:327]   File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/runtime/hints.py", line 68, in 
ERROR 03-01 04:14:45 registry.py:327]     from triton.compiler.compiler import AttrsDescriptor
ERROR 03-01 04:14:45 registry.py:327] ImportError: cannot import name 'AttrsDescriptor' from 'triton.compiler.compiler' (/workspace/triton/python/triton/compiler/compiler.py)
ERROR 03-01 04:14:45 registry.py:327]
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 774, in 
uvloop.run(run_server(args))
File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 82, in run
return loop.run_until_complete(wrapper())
File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 61, in wrapper
return await main
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 740, in run_server
async with build_async_engine_client(args) as engine_client:
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 118, in build_async_engine_client
async with build_async_engine_client_from_engine_args(
File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
return await anext(self.gen)
File "/workspace/vllm/vllm/entrypoints/openai/api_server.py", line 210, in build_async_engine_client_from_engine_args
engine_config = engine_args.create_engine_config()
File "/workspace/vllm/vllm/engine/arg_utils.py", line 1044, in create_engine_config
model_config = self.create_model_config()
File "/workspace/vllm/vllm/engine/arg_utils.py", line 970, in create_model_config
return ModelConfig(
File "/workspace/vllm/vllm/config.py", line 337, in init
self.multimodal_config = self._init_multimodal_config(
File "/workspace/vllm/vllm/config.py", line 392, in _init_multimodal_config
if ModelRegistry.is_multimodal_model(architectures):
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 461, in is_multimodal_model
model_cls, _ = self.inspect_model_cls(architectures)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 421, in inspect_model_cls
return self._raise_for_unsupported(architectures)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 378, in _raise_for_unsupported
raise ValueError(
ValueError: Model architectures ['Qwen2ForCausalLM'] failed to be inspected. Please check the logs for more details.
^CException ignored in atexit callback: <function _exit_function at 0xfffec7fc3ac0>
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/util.py", line 357, in _exit_function
p.join()
File "/usr/lib/python3.10/multiprocessing/process.py", line 149, in join
res = self._popen.wait(timeout)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 43, in wait
return self.poll(os.WNOHANG if timeout == 0.0 else 0)
File "/usr/lib/python3.10/multiprocessing/popen_fork.py", line 27, in poll
pid, sts = os.waitpid(self.pid, flag)
KeyboardInterrupt:
ERROR 03-01 04:14:54 engine.py:366]
Traceback (most recent call last):
File "/workspace/vllm/vllm/engine/multiprocessing/engine.py", line 357, in run_mp_engine
engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
File "/workspace/vllm/vllm/engine/multiprocessing/engine.py", line 114, in from_engine_args
engine_config = engine_args.create_engine_config(usage_context)
File "/workspace/vllm/vllm/engine/arg_utils.py", line 1044, in create_engine_config
model_config = self.create_model_config()
File "/workspace/vllm/vllm/engine/arg_utils.py", line 970, in create_model_config
return ModelConfig(
File "/workspace/vllm/vllm/config.py", line 337, in init
self.multimodal_config = self._init_multimodal_config(
File "/workspace/vllm/vllm/config.py", line 392, in _init_multimodal_config
if ModelRegistry.is_multimodal_model(architectures):
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 461, in is_multimodal_model
model_cls, _ = self.inspect_model_cls(architectures)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 417, in inspect_model_cls
model_info = self._try_inspect_model_cls(arch)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 397, in _try_inspect_model_cls
return _try_inspect_model_cls(model_arch, self.models[model_arch])
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 325, in _try_inspect_model_cls
return model.inspect_model_cls()
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 287, in inspect_model_cls
return _run_in_subprocess(
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 515, in _run_in_subprocess
returned = subprocess.run(
File "/usr/lib/python3.10/subprocess.py", line 505, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/lib/python3.10/subprocess.py", line 1154, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python3.10/subprocess.py", line 2021, in _communicate
ready = selector.select(timeout)
File "/usr/lib/python3.10/selectors.py", line 416, in select
fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt
Process SpawnProcess-1:
Traceback (most recent call last):
File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/workspace/vllm/vllm/engine/multiprocessing/engine.py", line 368, in run_mp_engine
raise e
File "/workspace/vllm/vllm/engine/multiprocessing/engine.py", line 357, in run_mp_engine
engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
File "/workspace/vllm/vllm/engine/multiprocessing/engine.py", line 114, in from_engine_args
engine_config = engine_args.create_engine_config(usage_context)
File "/workspace/vllm/vllm/engine/arg_utils.py", line 1044, in create_engine_config
model_config = self.create_model_config()
File "/workspace/vllm/vllm/engine/arg_utils.py", line 970, in create_model_config
return ModelConfig(
File "/workspace/vllm/vllm/config.py", line 337, in init
self.multimodal_config = self._init_multimodal_config(
File "/workspace/vllm/vllm/config.py", line 392, in _init_multimodal_config
if ModelRegistry.is_multimodal_model(architectures):
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 461, in is_multimodal_model
model_cls, _ = self.inspect_model_cls(architectures)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 417, in inspect_model_cls
model_info = self._try_inspect_model_cls(arch)
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 397, in _try_inspect_model_cls
return _try_inspect_model_cls(model_arch, self.models[model_arch])
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 325, in _try_inspect_model_cls
return model.inspect_model_cls()
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 287, in inspect_model_cls
return _run_in_subprocess(
File "/workspace/vllm/vllm/model_executor/models/registry.py", line 515, in _run_in_subprocess
returned = subprocess.run(
File "/usr/lib/python3.10/subprocess.py", line 505, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "/usr/lib/python3.10/subprocess.py", line 1154, in communicate
stdout, stderr = self._communicate(input, endtime, timeout)
File "/usr/lib/python3.10/subprocess.py", line 2021, in _communicate
ready = selector.select(timeout)
File "/usr/lib/python3.10/selectors.py", line 416, in select
fd_event_list = self._selector.poll(timeout)
Environment details
system：ubuntu 22.04 and aarch64
triton:
Version: 3.2.0+git23753f4f
Location: /usr/local/lib/python3.10/dist-packages
Editable project location: /workspace/triton/python
Requires: setuptools
torch: Version: 2.6.0.dev20241230+cu126,nightly version
cuda: Cuda compilation tools, release 12.2, V12.2.140.Build cuda_12.2.r12.2/compiler.33191640_0
ninja： 1.11.1.1
cmake: 3.27.6
wheel:  0.41.2
pybind11: 2.11.1

Duplicate issue #6157: 3D `tl.dot` crash with `TRITON_F32_DEFAULT = "ieee"`
Created: 2025-03-09T19:30:34Z
I usually disable TF32 to check numerics of the computation, but I noticed that 3D dot crashes when TF32 is turned off. The following code runs fine, but throws following message when using TRITON_F32_DEFAULT = "ieee"
(You might have to use TRITON_F32_DEFAULT=ieee TRITON_ALWAYS_COMPILE=1 since the latest PyPI version of Triton does not trigger recompile when disabling TF32)
python3.11: /root/.triton/llvm/llvm-86b69c31-almalinux-x64/include/llvm/ADT/SmallVector.h:291: reference llvm::SmallVectorTemplateCommon<mlir::Value>::operator[](size_type) [T = mlir::Value]: Assertion `idx < size()' failed.
Aborted (core dumped)

import torch
import triton
import triton.language as tl

@triton.jit
def kernel(
    output_ptr,
    M: tl.constexpr,
    N: tl.constexpr,
    K: tl.constexpr,
    L: tl.constexpr,
):
    # Create 3D tensors [L, M, K] and [L, K, N]
    a = tl.zeros([L, M, K], dtype=tl.float32)
    b = tl.zeros([L, K, N], dtype=tl.float32)
    
    # Try to perform dot product along K dimension
    c = tl.dot(a, b)  # Should result in [L, M, N]
    
    # Store result using block pointer
    p_out = tl.make_block_ptr(
        output_ptr, 
        shape=(L, M, N),
        strides=(M * N, N, 1),
        offsets=(0, 0, 0),
        block_shape=(L, M, N),
        order=(2, 1, 0)
    )
    
    tl.store(p_out, c)

def func():
    # Test dimensions
    L, M, N, K = 2, 16, 16, 16
    
    # Create output tensor
    output = torch.empty((L, M, N), device='cuda', dtype=torch.float32)
    
    # Launch kernel
    grid = (1,)
    kernel[grid](
        output,
        M=M,
        N=N,
        K=K,
        L=L,
    )
    
    print("Output shape:", output.shape)
    print("Output sum:", output.sum().item())

if __name__ == "__main__":
    func()

Closing comment (2025-03-09T22:51:41Z): ^looks like this is related to #6080, so I guess there isn't really a solution other than waiting for the official 3.3.0 release (or compiling PyTorch from scratch)?

Yes