Repository: intel/intel-xpu-backend-for-triton
Pull request #4276: Separate TTGPU to LLVM and TritonGEN to LLVM Lowering
Author: login: alexbaden
Created: 2025-05-22T16:49:47Z | Merged: 2025-05-28T00:31:04Z

Changes the Intel lowering pipeline during `make_llir` to first convert TritonIntelGPU to LLVM Dialect, then convert TritonGEN to LLVM Dialect. This is analogous to the NVIDIA backend, allows us to run the GEN verifier on all GEN ops, and allows us to write lit tests against TTGPUIR looking for specific gen ops - e.g. 2d block loads with specific tile sizes. 

To lower GEN to LLVM Dialect we need to add the GEN To SPIRV and SPIRV to LLVM patterns to the lowering step. I chose to do that in the existing TritonGENToLLVM pass, but we could introduce a new pass too (`TritonGENToLLVMViaSPIRV`?). 

Finally, I had to disable the tensor of pointer -> 2d block io lit test. The test is failing Gen validation because the pitch passed to the TritonGEN op is 0. This likely wouldn't work in practice, but it is now being caught during lit testing because we are running the verifier. I have opened a separate issue to fix this: https://github.com/intel/intel-xpu-backend-for-triton/issues/4275

I also discovered several places where the DPAS layout was using the incorrect `opsPerChannel` parameter for the data type - I had initially opened https://github.com/intel/intel-xpu-backend-for-triton/issues/4270 to track this, but fixed the issue in this PR. However, we should consider layout validation to ensure DPAS type-specific parameters match the data type.

close #4269

Linked issue #4269: Separate TritonGEN to LLVM from TritonIntelGPU to LLVM
Currently TritonIntelGPU lowers to LLVM in a single shot using the TritonGEN dialect ops which are immediately lowered to llvm spirv/ocl function calls. 

With the change from OCL functions to SPIRV intrinsics many parameters that were previously part of the function signature are now function arguments. Adapting the lit test validation to read the function arguments is pretty messy, so we started looking at alternatives. The NVIDIA GPU dialect separates lowering of TTGPU to LLVM from lowering of NVGPU to LLVM (even though both happen in the same major step):
```
        nvidia.passes.ttgpuir.add_to_llvmir(pm, capability, ptx_version)
        passes.common.add_canonicalizer(pm)
        passes.common.add_cse(pm)
        nvidia.passes.ttnvgpuir.add_nvgpu_to_llvm(pm)
```

If we do something similar with GEN, we can write lit tests against the TTGPU -> GEN lowering and against the TTGPU -> GEN -> LLVM lowering, which lets us preserve the existing behavior without diverging from upstream or not using SPIR-V builtins. 

An ancillary benefit is the verifier for TritonGEN ops will now be automatically invoked when the op is created. Currently, the verifier is skipped because the op is immediately lowered again to LLVM.

Diff:
diff --git a/test/Conversion/intel/sub-group-transpose.mlir b/test/Conversion/intel/sub-group-transpose.mlir
index 34d3fbc4c8..a35da0125d 100644
--- a/test/Conversion/intel/sub-group-transpose.mlir
+++ b/test/Conversion/intel/sub-group-transpose.mlir
@@ -1,4 +1,4 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm | FileCheck %s
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm | FileCheck %s
 
 // Basic 16x16 transpose test
 
diff --git a/test/Conversion/intel/tritongpu_to_gen.mlir b/test/Conversion/intel/tritongpu_to_gen.mlir
index 22bd38886a..06ba04b205 100644
--- a/test/Conversion/intel/tritongpu_to_gen.mlir
+++ b/test/Conversion/intel/tritongpu_to_gen.mlir
@@ -1,4 +1,4 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory  --convert-triton-intel-gpu-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm
 
 module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
   // CHECK: llvm.func spir_kernelcc @test_empty_kernel(%arg0: i64, %arg1: !llvm.ptr<1>)
@@ -1026,11 +1026,11 @@ module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
 
 // -----
 
-#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
+#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 1, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
 #shared = #ttg.swizzled_shared<{vec = 1, perPhase = 1, maxPhase = 1, order = [1, 0], CTAsPerCGA = [1, 1], CTASplitNum = [1, 1], CTAOrder = [1, 0]}>
 #blocked = #ttg.blocked<{sizePerThread = [1, 4], threadsPerWarp = [2, 16], warpsPerCTA = [1, 4], order = [1, 0], CTAsPerCGA = [1, 1], CTASplitNum = [1, 1], CTAOrder = [1, 0]}>
 #dot_operand_a = #ttg.dot_op<{opIdx=0, parent=#dpas, kWidth=1}>
-#dot_operand_b = #ttg.dot_op<{opIdx=1, parent=#dpas, kWidth=2}>
+#dot_operand_b = #ttg.dot_op<{opIdx=1, parent=#dpas, kWidth=1}>
 #smem = #ttg.shared_memory
 module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
   // CHECK-LABEL: matmul_tf32dot
@@ -1040,7 +1040,7 @@ module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
     %a_mat = ttg.local_load %a : !ttg.memdesc<32x16xf32, #shared, #smem> -> tensor<32x16xf32, #dot_operand_a>
     %b_mat = ttg.local_load %b : !ttg.memdesc<16x32xf32, #shared, #smem> -> tensor<16x32xf32, #dot_operand_b>
 
-    // CHECK-COUNT-2: llvm.call spir_funccc @_Z45__spirv_SubgroupMatrixMultiplyAccumulateINTELiDv8_fS_S_i(%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}) {{.*}} : (i32, vector<8xf32>, vector<8xf32>, vector<8xf32>, i32) -> vector<8xf32>
+    // CHECK-COUNT-2: llvm.call spir_funccc @_Z45__spirv_SubgroupMatrixMultiplyAccumulateINTELiDv4_fDv8_fS0_i(%{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}, %{{.*}}) {{.*}} : (i32, vector<4xf32>, vector<8xf32>, vector<8xf32>, i32) -> vector<8xf32>
     %28 = tt.dot %a_mat, %b_mat, %cst, inputPrecision = tf32 : tensor<32x16xf32, #dot_operand_a> * tensor<16x32xf32, #dot_operand_b> -> tensor<32x32xf32, #dpas>
     %38 = ttg.convert_layout %28 : tensor<32x32xf32, #dpas> -> tensor<32x32xf32, #blocked>
 
@@ -1485,11 +1485,11 @@ module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 8 : i32} {
 
 // -----
 
-#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
+#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 1, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
 #shared = #ttg.swizzled_shared<{vec = 1, perPhase = 1, maxPhase = 1, order = [1, 0], CTAsPerCGA = [1, 1], CTASplitNum = [1, 1], CTAOrder = [1, 0]}>
 #blocked = #ttg.blocked<{sizePerThread = [1, 4], threadsPerWarp = [2, 16], warpsPerCTA = [1, 4], order = [1, 0], CTAsPerCGA = [1, 1], CTASplitNum = [1, 1], CTAOrder = [1, 0]}>
 #dot_operand_a = #ttg.dot_op<{opIdx=0, parent=#dpas, kWidth=1}>
-#dot_operand_b = #ttg.dot_op<{opIdx=1, parent=#dpas, kWidth=2}>
+#dot_operand_b = #ttg.dot_op<{opIdx=1, parent=#dpas, kWidth=1}>
 module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
   // CHECK-LABEL: matmul_tf32_cst_b
   tt.func @matmul_tf32_cst_b(%ptr:!tt.ptr<f32> {tt.divisibility = 16 : i32},
diff --git a/test/Conversion/intel/tritongpu_to_gen_dot.mlir b/test/Conversion/intel/tritongpu_to_gen_dot.mlir
index 0bcb388ee6..aee786d6b6 100644
--- a/test/Conversion/intel/tritongpu_to_gen_dot.mlir
+++ b/test/Conversion/intel/tritongpu_to_gen_dot.mlir
@@ -1,5 +1,5 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory  --convert-triton-intel-gpu-to-llvm --cse -canonicalize | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,NO-AGGRESSIVE-REUSE
-// RUN: env TRITON_INTEL_AGGRESSIVE_DPAS_REUSE=1 triton-opt %s -split-input-file --allocate-shared-memory  --convert-triton-intel-gpu-to-llvm --cse -canonicalize | FileCheck %s --implicit-check-not=llvm.inline_asm  --check-prefixes=CHECK,AGGRESSIVE-REUSE
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse -canonicalize | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,NO-AGGRESSIVE-REUSE
+// RUN: env TRITON_INTEL_AGGRESSIVE_DPAS_REUSE=1 triton-opt %s -split-input-file --allocate-shared-memory  --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse -canonicalize | FileCheck %s --implicit-check-not=llvm.inline_asm  --check-prefixes=CHECK,AGGRESSIVE-REUSE
 
 #dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [1, 1], repCluster = [1, 1]}>
 #dot_operand_a = #ttg.dot_op<{opIdx=0, parent=#dpas, kWidth=1}>
diff --git a/test/Conversion/intel/tritongpu_to_llvm_intel_advanced_path.mlir b/test/Conversion/intel/tritongpu_to_llvm_intel_advanced_path.mlir
index ed129cfe04..59553b2171 100644
--- a/test/Conversion/intel/tritongpu_to_llvm_intel_advanced_path.mlir
+++ b/test/Conversion/intel/tritongpu_to_llvm_intel_advanced_path.mlir
@@ -1,4 +1,4 @@
-// RUN: env TRITON_INTEL_ADVANCED_PATH=1 triton-opt %s --convert-triton-intel-gpu-to-llvm --split-input-file | FileCheck %s
+// RUN: env TRITON_INTEL_ADVANCED_PATH=1 triton-opt %s --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --split-input-file | FileCheck %s
 
 module attributes {"ttig.support_sg_2d_block", "ttig.support_dpas", "ttg.num-warps" = 32 : i32, "ttg.threads-per-warp" = 1 : i32} {
   // CHECK-DAG: llvm.func spir_funccc @_Z45__spirv_SubgroupMatrixMultiplyAccumulateINTELiDv8_sDv8_iDv8_fi(i32, vector<8xi16>, vector<8xi32>, vector<8xf32>, i32) -> vector<8xf32> attributes {convergent, memory_effects = #llvm.memory_effects<other = none, argMem = none, inaccessibleMem = none>, no_unwind, will_return}
diff --git a/test/TritonIntelGPU/blockptr_load.mlir b/test/TritonIntelGPU/blockptr_load.mlir
index 840a3e40f0..7cacd0b1c5 100644
--- a/test/TritonIntelGPU/blockptr_load.mlir
+++ b/test/TritonIntelGPU/blockptr_load.mlir
@@ -1,5 +1,5 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,LARGE-BLOCK-SIZE-TRANS-B
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm=one_matrix_per_load_for_bt=1 | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,SMALL-BLOCK-SIZE-TRANS-B
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,LARGE-BLOCK-SIZE-TRANS-B
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm=one_matrix_per_load_for_bt=1 --convert-tritongen-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm --check-prefixes=CHECK,SMALL-BLOCK-SIZE-TRANS-B
 
 // CHECK-DAG: llvm.func spir_funccc @_Z45__spirv_SubgroupMatrixMultiplyAccumulateINTELiDv8_sDv8_iDv8_fi(i32, vector<8xi16>, vector<8xi32>, vector<8xf32>, i32) -> vector<8xf32> attributes {convergent, memory_effects = #llvm.memory_effects<other = none, argMem = none, inaccessibleMem = none>, no_unwind, will_return}
 // CHECK-DAG: llvm.func spir_funccc @_Z32__spirv_Subgroup2DBlockLoadINTELiiiiPU3AS1viiiDv2_iPv(i32, i32, i32, i32, !llvm.ptr<1> {llvm.nonnull, llvm.readonly}, i32, i32, i32, vector<2xi32>, !llvm.ptr {llvm.nonnull, llvm.writeonly}) attributes {no_unwind, will_return}
diff --git a/test/TritonIntelGPU/blockptr_store.mlir b/test/TritonIntelGPU/blockptr_store.mlir
index 504820f71c..93b7bbeff0 100644
--- a/test/TritonIntelGPU/blockptr_store.mlir
+++ b/test/TritonIntelGPU/blockptr_store.mlir
@@ -1,4 +1,4 @@
-// RUN: triton-opt %s -split-input-file --convert-triton-intel-gpu-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm | FileCheck %s --implicit-check-not=llvm.inline_asm
 
 // CHECK: llvm.func spir_funccc @_Z33__spirv_Subgroup2DBlockStoreINTELiiiiPvPU3AS1viiiDv2_i(i32, i32, i32, i32, !llvm.ptr {llvm.nonnull, llvm.readonly}, !llvm.ptr<1> {llvm.nonnull, llvm.writeonly}, i32, i32, i32, vector<2xi32>) attributes {no_unwind, will_return}
 #dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [4, 2], repCluster = [1, 1], A = [8, 16], B = [16, 16], C = [8, 16]}>
diff --git a/test/TritonIntelGPU/prefetch-to-llvm.mlir b/test/TritonIntelGPU/prefetch-to-llvm.mlir
index 2e60c7ce1a..e8146142b4 100644
--- a/test/TritonIntelGPU/prefetch-to-llvm.mlir
+++ b/test/TritonIntelGPU/prefetch-to-llvm.mlir
@@ -1,4 +1,4 @@
-// RUN: triton-opt %s -split-input-file --convert-triton-intel-gpu-to-llvm --cse -canonicalize | FileCheck %s
+// RUN: triton-opt %s -split-input-file --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse -canonicalize | FileCheck %s
 
 // CHECK: llvm.func spir_funccc @_Z36__spirv_Subgroup2DBlockPrefetchINTELiiiiPU3AS1viiiDv2_i(i32, i32, i32, i32, !llvm.ptr<1> {llvm.nonnull}, i32, i32, i32, vector<2xi32>) attributes {memory_effects = #llvm.memory_effects<other = none, argMem = read, inaccessibleMem = none>, no_unwind}
 module attributes {"ttg.num-warps" = 8 : i32, "ttg.threads-per-warp" = 16 : i32} {
diff --git a/test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir b/test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
index fbe82b5c7c..ebab107fcf 100644
--- a/test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
+++ b/test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
@@ -1,4 +1,4 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
 
 // CHECK:   llvm.func spir_funccc @_Z32__spirv_Subgroup2DBlockLoadINTELiiiiPU3AS1viiiDv2_iPv
 #mma = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [2, 4], repCluster = [4, 2], A = [32, 16], B = [16, 32], C = [32, 32]}>
diff --git a/third_party/intel/backend/compiler.py b/third_party/intel/backend/compiler.py
index d84fddfbeb..c377ca319a 100644
--- a/third_party/intel/backend/compiler.py
+++ b/third_party/intel/backend/compiler.py
@@ -327,6 +327,7 @@ def make_llir(src, metadata, options):
         passes.ttgpuir.add_allocate_global_scratch_memory(pm)
         intel.passes.ttgpuir.add_to_llvmir(pm, options.advanced_path, options.one_matrix_per_load_for_bt,
                                            options.enable_tile_load_linear_layout)
+        intel.passes.ttgpuir.add_gen_to_llvm(pm)
         intel.passes.ttgpuir.add_rewrite_stack_ptr(pm)
         passes.common.add_canonicalizer(pm)
         passes.common.add_cse(pm)
diff --git a/third_party/intel/include/Dialect/TritonIntelGPU/IR/TritonIntelGPUDialect.td b/third_party/intel/include/Dialect/TritonIntelGPU/IR/TritonIntelGPUDialect.td
index 3bf2ab8f3e..cc239c79ff 100644
--- a/third_party/intel/include/Dialect/TritonIntelGPU/IR/TritonIntelGPUDialect.td
+++ b/third_party/intel/include/Dialect/TritonIntelGPU/IR/TritonIntelGPUDialect.td
@@ -16,6 +16,7 @@ def TritonIntelGPU_Dialect : Dialect {
     "triton::TritonDialect",
     "triton::gpu::TritonGPUDialect",
     "mlir::gpu::GPUDialect",
+    "mlir::triton::TritonGEN::TritonGENDialect",
   ];
 
   let extraClassDeclaration = [{
diff --git a/third_party/intel/include/TritonGENToLLVM/Passes.td b/third_party/intel/include/TritonGENToLLVM/Passes.td
index 84a0a14186..70a09bd68a 100644
--- a/third_party/intel/include/TritonGENToLLVM/Passes.td
+++ b/third_party/intel/include/TritonGENToLLVM/Passes.td
@@ -11,11 +11,12 @@
 
 include "mlir/Pass/PassBase.td"
 
-def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm"> {
+def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm", "mlir::ModuleOp"> {
   let summary = "Convert the Triton GEN dialect to the LLVM dialect";
   let description = [{
     This pass converts the TritonGEN dialect operations to LLVM dialect operations.
   }];
+
   let dependentDialects = ["mlir::LLVM::LLVMDialect"];
 }
 
diff --git a/third_party/intel/lib/TritonGENToLLVM/TritonGENToLLVMPass.cpp b/third_party/intel/lib/TritonGENToLLVM/TritonGENToLLVMPass.cpp
index 1774314da5..e643dffde2 100644
--- a/third_party/intel/lib/TritonGENToLLVM/TritonGENToLLVMPass.cpp
+++ b/third_party/intel/lib/TritonGENToLLVM/TritonGENToLLVMPass.cpp
@@ -12,6 +12,7 @@
 #include "mlir/Conversion/ConvertToLLVM/ToLLVMInterface.h"
 #include "mlir/Conversion/LLVMCommon/ConversionTarget.h"
 #include "mlir/Conversion/LLVMCommon/Pattern.h"
+#include "mlir/Conversion/SPIRVToLLVM/SPIRVToLLVM.h"
 #include "mlir/Dialect/LLVMIR/FunctionCallUtils.h"
 #include "mlir/Dialect/LLVMIR/LLVMAttrs.h"
 #include "mlir/Dialect/LLVMIR/LLVMDialect.h"
@@ -37,6 +38,7 @@
 
 #include "intel/include/Dialect/TritonGEN/IR/TritonGENDialect.h"
 #include "intel/include/TritonGENToLLVM/TritonGENToLLVMPass.h"
+#include "intel/include/TritonGENToSPIRV/TritonGENToSPIRVPass.h"
 
 namespace mlir::triton {
 #define GEN_PASS_DEF_CONVERTTRITONGENTOLLVM
@@ -818,15 +820,19 @@ struct ConvertTritonGENToLLVM
 
   void runOnOperation() override {
     MLIRContext *ctx = &getContext();
-    RewritePatternSet pattern(ctx);
+    RewritePatternSet patterns(ctx);
     LowerToLLVMOptions options(ctx);
-    LLVMTypeConverter converter(ctx, options);
+    LLVMTypeConverter typeConverter(ctx, options);
     LLVMConversionTarget target(*ctx);
 
-    populateTritonGENToLLVMConversionPatterns(converter, pattern);
+    populateTritonGENToLLVMConversionPatterns(typeConverter, patterns);
+
+    populateTritonGENToSPIRVConversionPatterns(patterns);
+    populateSPIRVToLLVMConversionPatterns(typeConverter, patterns,
+                                          spirv::ClientAPI::OpenCL);
 
-    if (failed(
-            applyPartialConversion(getOperation(), target, std::move(pattern))))
+    if (failed(applyPartialConversion(getOperation(), target,
+                                      std::move(patterns))))
       signalPassFailure();
   }
 };
diff --git a/third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp b/third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
index 79f994f841..5ef4c16ad0 100644
--- a/third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
+++ b/third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
@@ -557,8 +557,10 @@ struct PrefetchOpConversion
             /*v_blocks*/ vBlocks,
             /*cache_opt*/ TritonGEN::LoadCacheControl::L1C_L3C);
         if (failed(newOp.verify())) {
-          // Explicitly invoke verifier because `triton_gen` ops are immediately
-          // lowered further to a builtin call.
+          // delete the op so that the verifier will not abort the pass
+          // pipeline later, as we can fail this path and try a different
+          // approach.
+          rewriter.eraseOp(newOp);
           return failure();
         }
       }
@@ -757,8 +759,10 @@ struct PrefetchOpConversion
                 /*v_blocks*/ vBlocks,
                 /*cache_opt*/ TritonGEN::LoadCacheControl::L1C_L3C);
             if (failed(newOp.verify())) {
-              // Explicitly invoke verifier because `triton_gen` ops are
-              // immediately lowered further to a builtin call.
+              // delete the op so that the verifier will not abort the pass
+              // pipeline later, as we can fail this path and try a different
+              // approach.
+              rewriter.eraseOp(newOp);
               return failure();
             }
           }
@@ -1573,8 +1577,10 @@ struct LoadOpConversion
                   /*transpose*/ false,
                   /*vnni_transform*/ false);
               if (failed(load2dOp.verify())) {
-                // Explicitly invoke verifier because `triton_gen` ops are
-                // immediately lowered further to a builtin call.
+                // delete the op so that the verifier will not abort the pass
+                // pipeline later, as we can fail this path and try a different
+                // approach.
+                rewriter.eraseOp(load2dOp);
                 return failure();
               }
 
@@ -2086,8 +2092,10 @@ struct LoadOpConversion
               (usePackedType && !isOperandA && !isTransposeRequired &&
                originalElemBits != 32));
           if (failed(load2dOp.verify())) {
-            // Explicitly invoke verifier because `triton_gen` ops are
-            // immediately lowered further to a builtin call.
+            // delete the op so that the verifier will not abort the pass
+            // pipeline later, as we can fail this path and try a different
+            // approach.
+            rewriter.eraseOp(load2dOp);
             return failure();
           }
           LLVM_DEBUG(llvm::dbgs() << "Generated load op: " << load2dOp << "\n");
@@ -2520,8 +2528,10 @@ struct StoreOpConversion
                 /*stored_val*/ b.bitcast(storeVal, store2DGenXType));
 
             if (failed(newOp.verify())) {
-              // Explicitly invoke verifier because `triton_gen` ops are
-              // immediately lowered further to a builtin call.
+              // delete the op so that the verifier will not abort the pass
+              // pipeline later, as we can fail this path and try a different
+              // approach.
+              rewriter.eraseOp(newOp);
               return failure();
             }
           }
diff --git a/third_party/intel/lib/TritonIntelGPUToLLVM/PipelineManager.h b/third_party/intel/lib/TritonIntelGPUToLLVM/PipelineManager.h
index e0a23baeff..c7ec9ae077 100644
--- a/third_party/intel/lib/TritonIntelGPUToLLVM/PipelineManager.h
+++ b/third_party/intel/lib/TritonIntelGPUToLLVM/PipelineManager.h
@@ -254,10 +254,8 @@ class TritonGPUToLLVMPipelineManager {
     // to help convert scalar expression to LLVM.
     arith::populateArithToLLVMConversionPatterns(typeConverter, patterns);
     populateMathToLLVMConversionPatterns(typeConverter, patterns);
-    triton::populateTritonGENToLLVMConversionPatterns(typeConverter, patterns);
     triton::populateGPUToTritonGENConversionPatterns(typeConverter, patterns);
     cf::populateControlFlowToLLVMConversionPatterns(typeConverter, patterns);
-    populateTritonGENToSPIRVConversionPatterns(patterns);
     populateGpuToLLVMSPVConversionPatterns(typeConverter, patterns);
     populateSPIRVToLLVMConversionPatterns(typeConverter, patterns,
                                           spirv::ClientAPI::OpenCL);
diff --git a/third_party/intel/lib/TritonIntelGPUToLLVM/TritonGPUToLLVM.cpp b/third_party/intel/lib/TritonIntelGPUToLLVM/TritonGPUToLLVM.cpp
index 7f243ef0c6..7d45ef3335 100644
--- a/third_party/intel/lib/TritonIntelGPUToLLVM/TritonGPUToLLVM.cpp
+++ b/third_party/intel/lib/TritonIntelGPUToLLVM/TritonGPUToLLVM.cpp
@@ -51,7 +51,7 @@ class TritonLLVMConversionTarget : public ConversionTarget {
   explicit TritonLLVMConversionTarget(MLIRContext &ctx)
       : ConversionTarget(ctx) {
     addLegalDialect<LLVM::LLVMDialect>();
-    addIllegalDialect<triton::TritonGEN::TritonGENDialect>();
+    addLegalDialect<triton::TritonGEN::TritonGENDialect>();
     addIllegalDialect<triton::TritonDialect>();
     addIllegalDialect<triton::gpu::TritonGPUDialect>();
     addIllegalDialect<triton::gpu::intel::TritonIntelGPUDialect>();
diff --git a/third_party/intel/triton_xpu.cc b/third_party/intel/triton_xpu.cc
index b886c9f06b..387ac71045 100644
--- a/third_party/intel/triton_xpu.cc
+++ b/third_party/intel/triton_xpu.cc
@@ -16,6 +16,7 @@
 #include "intel/include/Target/LLVMIR/Dialect/TritonGEN/TritonGENToLLVMIRTranslation.h"
 #include "intel/include/Target/LLVMIR/PostProcess.h"
 #include "intel/include/TritonAnnotateModule/Passes.h"
+#include "intel/include/TritonGENToLLVM/Passes.h"
 #include "intel/include/TritonIntelGPUToLLVM/Passes.h"
 #include "intel/include/TritonRaiseBlockPointer/Passes.h"
 #include "intel/include/TritonToTritonGPUWarp/Passes.h"
@@ -67,6 +68,7 @@ void init_triton_intel_passes_ttgpuir(py::module &&m) {
   ADD_PASS_OPTION_WRAPPER_3("add_to_llvmir",
                             gpu::intel::createConvertTritonIntelGPUToLLVM, bool,
                             bool, bool);
+  ADD_PASS_WRAPPER_0("add_gen_to_llvm", createConvertTritonGENToLLVM);
   ADD_PASS_WRAPPER_0("add_accelerate_matmul",
                      gpu::intel::createTritonIntelGPUAccelerateMatmul);
   ADD_PASS_WRAPPER_0("add_rewrite_stack_ptr",


Reviews:
user: login: whitneywhtsang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T18:13:32Z
user: login: alexbaden
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T18:36:04Z
user: login: etiotto
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T18:52:01Z
user: login: alexbaden
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T18:54:14Z
user: login: whitneywhtsang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-22T22:02:10Z
user: login: whitneywhtsang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T00:45:00Z
user: login: whitneywhtsang
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T00:56:35Z
user: login: kurapov-peter
body: LGTM!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T08:28:11Z
user: login: alexbaden
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T12:47:52Z
user: login: whitneywhtsang
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-23T14:24:48Z
user: login: sommerlukas
body: Looks good to me, just two small questions.
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T14:03:37Z
user: login: alexbaden
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T14:06:38Z
user: login: alexbaden
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T14:07:49Z
user: login: sommerlukas
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T14:19:59Z
user: login: sommerlukas
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-27T21:33:28Z

Review comments:
diff_hunk: @@ -11,11 +11,14 @@
 
 include "mlir/Pass/PassBase.td"
 
-def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm"> {
+def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm", "mlir::ModuleOp"> {
   let summary = "Convert the Triton GEN dialect to the LLVM dialect";
   let description = [{
     This pass converts the TritonGEN dialect operations to LLVM dialect operations.
   }];
+
+  let constructor = "mlir::triton::createConvertTritonGENToLLVM()";
path: third_party/intel/include/TritonGENToLLVM/Passes.td
user: login: whitneywhtsang
body: I think there is no need to do this explicitly. For example, `TritonIntelTensorDescToBlockPointer` doesn't need to define constructor, and it has `createTritonIntelTensorDescToBlockPointer`.
created_at: 2025-05-22T18:09:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -26,6 +26,8 @@ void populateTritonGENToLLVMConversionPatterns(LLVMTypeConverter &converter,
 
 void registerConvertTritonGENToLLVMInterface(DialectRegistry &registry);
 
+std::unique_ptr<OperationPass<ModuleOp>> createConvertTritonGENToLLVM();
path: third_party/intel/include/TritonGENToLLVM/TritonGENToLLVMPass.h
user: login: whitneywhtsang
body: may not need.
created_at: 2025-05-22T18:10:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -11,11 +11,14 @@
 
 include "mlir/Pass/PassBase.td"
 
-def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm"> {
+def ConvertTritonGENToLLVM : Pass<"convert-tritongen-to-llvm", "mlir::ModuleOp"> {
   let summary = "Convert the Triton GEN dialect to the LLVM dialect";
   let description = [{
     This pass converts the TritonGEN dialect operations to LLVM dialect operations.
   }];
+
+  let constructor = "mlir::triton::createConvertTritonGENToLLVM()";
path: third_party/intel/include/TritonGENToLLVM/Passes.td
user: login: alexbaden
body: You're right, removed.
created_at: 2025-05-22T18:36:04Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,4 +1,6 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
+// XFAIL: *
path: test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
user: login: etiotto
body: Can you elaborate on the reason this tests  were passing before ?  XFAILing them all is worrisome, we should find a way to fix the compiler before landing this PR. 
created_at: 2025-05-22T18:49:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,4 +1,6 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
+// XFAIL: *
path: test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
user: login: alexbaden
body: They were passing because the compiler was generating invalid IR, not verifying it, and not running it - you cannot call 2d block load with a pitch value of 0, but that was considered a "pass" prior to this PR enabling verification of all GEN ops during the lowering. 
created_at: 2025-05-22T18:54:14Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,4 +1,6 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
+// XFAIL: *
path: test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
user: login: whitneywhtsang
body: Should be fixed by #4279.
created_at: 2025-05-22T22:02:10Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2520,8 +2524,9 @@ struct StoreOpConversion
                 /*stored_val*/ b.bitcast(storeVal, store2DGenXType));
 
             if (failed(newOp.verify())) {
-              // Explicitly invoke verifier because `triton_gen` ops are
-              // immediately lowered further to a builtin call.
+              // delete the op so that the verifier will not abort the pass
+              // pipeline later. fail this path and retry with gather load.
path: third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
user: login: whitneywhtsang
body: please fix the comments, should not be gather load for all of them.
created_at: 2025-05-23T00:45:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,4 +1,6 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
+// XFAIL: *
+// COM: Failing validation due to 0 stride returned from axis info
path: test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
user: login: whitneywhtsang
body: ```suggestion
```
created_at: 2025-05-23T00:56:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1,4 +1,6 @@
-// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --cse | FileCheck %s --implicit-check-not=llvm.inline_asm
+// RUN: triton-opt %s -split-input-file --allocate-shared-memory --convert-triton-intel-gpu-to-llvm --convert-tritongen-to-llvm --cse  | FileCheck %s --implicit-check-not=llvm.inline_asm
+// XFAIL: *
path: test/TritonIntelGPU/tensor-pointer-load-block-2d.mlir
user: login: whitneywhtsang
body: #4279 is merged now.
created_at: 2025-05-23T00:56:27Z
author_association: CONTRIBUTOR
diff_hunk: @@ -2520,8 +2524,9 @@ struct StoreOpConversion
                 /*stored_val*/ b.bitcast(storeVal, store2DGenXType));
 
             if (failed(newOp.verify())) {
-              // Explicitly invoke verifier because `triton_gen` ops are
-              // immediately lowered further to a builtin call.
+              // delete the op so that the verifier will not abort the pass
+              // pipeline later. fail this path and retry with gather load.
path: third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
user: login: alexbaden
body: done
created_at: 2025-05-23T12:47:52Z
author_association: CONTRIBUTOR
diff_hunk: @@ -557,8 +557,9 @@ struct PrefetchOpConversion
             /*v_blocks*/ vBlocks,
             /*cache_opt*/ TritonGEN::LoadCacheControl::L1C_L3C);
         if (failed(newOp.verify())) {
-          // Explicitly invoke verifier because `triton_gen` ops are immediately
-          // lowered further to a builtin call.
+          // delete the op so that the verifier will not abort the pass
+          // pipeline later, as we can fail this path and retry.
path: third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
user: login: sommerlukas
body: What does "retry" mean here and in the other cases in this file? IIUC, the prefetch operation is deleted, so we wouldn't pass through here anymore.
created_at: 2025-05-27T14:01:35Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1026,11 +1026,11 @@ module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
 
 // -----
 
-#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
+#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 1, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
path: test/Conversion/intel/tritongpu_to_gen.mlir
user: login: sommerlukas
body: Was this test/layout wrong before? Or is there another reason to change this?
created_at: 2025-05-27T14:03:22Z
author_association: CONTRIBUTOR
diff_hunk: @@ -557,8 +557,9 @@ struct PrefetchOpConversion
             /*v_blocks*/ vBlocks,
             /*cache_opt*/ TritonGEN::LoadCacheControl::L1C_L3C);
         if (failed(newOp.verify())) {
-          // Explicitly invoke verifier because `triton_gen` ops are immediately
-          // lowered further to a builtin call.
+          // delete the op so that the verifier will not abort the pass
+          // pipeline later, as we can fail this path and retry.
path: third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
user: login: alexbaden
body: "do something else" 
The initial comment was arguably also wrong - what is really happening is we explicitly invoke the verifier so that if the triton_gen op fails verification we can abort this function and "try something else" i.e. retry. 
created_at: 2025-05-27T14:06:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -1026,11 +1026,11 @@ module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32} {
 
 // -----
 
-#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 2, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
+#dpas = #ttig.dpas<{repeatCount = 8, systolicDepth = 8, executionSize = 16, opsPerChan = 1, threadsPerWarp = 16, warpsPerCTA = [2, 2], repCluster = [1, 1]}>
path: test/Conversion/intel/tritongpu_to_gen.mlir
user: login: alexbaden
body: yes, see https://github.com/intel/intel-xpu-backend-for-triton/issues/4270

I think we need to do layout verification on DPAS hence why the issue is still open (though that may not be possible if we don't have the tensor type when we would do the verification), but I need to fix it here because the Gen ops are now verified and will fail verification and the test otherwise.
created_at: 2025-05-27T14:07:49Z
author_association: CONTRIBUTOR
diff_hunk: @@ -557,8 +557,9 @@ struct PrefetchOpConversion
             /*v_blocks*/ vBlocks,
             /*cache_opt*/ TritonGEN::LoadCacheControl::L1C_L3C);
         if (failed(newOp.verify())) {
-          // Explicitly invoke verifier because `triton_gen` ops are immediately
-          // lowered further to a builtin call.
+          // delete the op so that the verifier will not abort the pass
+          // pipeline later, as we can fail this path and retry.
path: third_party/intel/lib/TritonIntelGPUToLLVM/LoadStoreOpToLLVM.cpp
user: login: sommerlukas
body: Maybe `try a different approach` would be more appropriate then if we don't attempt to prefetch and lower again (which is what I understand by `retry`)?
created_at: 2025-05-27T14:19:59Z
author_association: CONTRIBUTOR

PR comments:
user: login: whitneywhtsang
created_at: 2025-05-22T18:00:44Z
body: > Changes the Intel lowering pipeline during make_llir to first convert TritonIntelGPU to LLVM Dialect, then convert TritonGEN to LLVM Dialect.

Is it a typo? Do you mean `first convert TritonIntelGPU to TritonGEN Dialect, then convert TritonGEN to LLVM Dialect.`
author_association: CONTRIBUTOR
user: login: alexbaden
created_at: 2025-05-22T18:07:22Z
body: Not a typo, because the TTGPU ops are still lowered to LLVM dialect ops. But the entire module is not lowered to LLVM dialect ops all at once anymore. 
author_association: CONTRIBUTOR
user: login: whitneywhtsang
created_at: 2025-05-22T18:11:56Z
body: > Not a typo, because the TTGPU ops are still lowered to LLVM dialect ops. But the entire module is not lowered to LLVM dialect ops all at once anymore.

icic, to be explicit it would be TritonIntelGPU to LLVM or TritonGEN, then TritonGEN to LLVM.
author_association: CONTRIBUTOR
user: login: alexbaden
created_at: 2025-05-23T00:33:38Z
body: Benchmarks show no change: https://github.com/intel/intel-xpu-backend-for-triton/actions/runs/15195454763/job/42738278903
author_association: CONTRIBUTOR