Repository: pulp-platform/Deeploy
Pull request #45: Fix L3 Bugs: DMA Struct Datatype and Maxpool Margin Error
Author: login: runwangdl
Created: 2025-02-26T15:58:33Z | Merged: 2025-03-20T08:05:39Z

## Fixed
- **Maxpool Tile Calculation Error**: The last dimension padding was incorrectly calculated due to L3 wraptiling solution. This has been fixed by updating `serializeTilingSolution` of Maxpool to avoid incorrect padding of Maxpool and prevent potential DMA 3D transfer issues of Maxpool.
- **DMA 1D Copy Assertion Issue**: Updated the DMA length datatype from `uint16` to `uint32` to avoid assertion failures when dealing with large block transfers.

## Added
- **New Test Cases**: Added and passed tests for `16×16 64` and `16×16 128` configurations to validate correctness.

## PR Merge Checklist
1. [x] The PR is rebased on the latest `devel` commit and pointing to `devel`.
2. [x] Your PR reviewed and approved.
3. [x] All checks are passing.
4. [x] The `CHANGELOG.md` file has been updated.


Diff:
diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml
index a94e95b71e..5b710a8a95 100644
--- a/.github/workflows/CI.yml
+++ b/.github/workflows/CI.yml
@@ -429,7 +429,7 @@ jobs:
           },
           {
             "name": "testFloatMaxPool",
-            "L1": [2000]
+            "L1": [5000]
           },
           {
             "name": "testFloatMatmul",
@@ -507,6 +507,10 @@ jobs:
             L1: [60000, 10000, 5000]
           - name: "CCT/CCT_1_32_32_8"
             L1: [64000]
+          - name: "CCT/CCT_1_16_16_64"
+            L1: [64000]
+          - name: "CCT/CCT_1_16_16_128"
+            L1: [64000]
         num-cores:
           - 8
         default-memory-level:
@@ -543,6 +547,10 @@ jobs:
             L1: [60000, 20000, 10000]
           - name: "CCT/CCT_1_32_32_8"
             L1: [64000]
+          - name: "CCT/CCT_1_16_16_64"
+            L1: [64000]
+          - name: "CCT/CCT_1_16_16_128"
+            L1: [64000]
         num-cores:
           - 8
         double-buffer:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 317f752580..25af0071c8 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -208,9 +208,19 @@ Change main.c to use OUTPUTTYPE instead of float
 - C template implementation in `DequantTemplate.py` for efficient dequantization
 - Type checker implementation in `TypeCheckers.py` to handle bit-width and signedness
 
+## Fix L3 Bugs: DMA Struct Datatype and Maxpool Margin Error
+
+### Added
+- New Test Cases: Added and passed tests for 16×16 64 and 16×16 128 configurations to validate correctness.
+
+### Fixed
+- Maxpool Tile Calculation Error: The last dimension padding was incorrectly calculated due to L3 wraptiling solution. This has been fixed by updating serializeTilingSolution of Maxpool to avoid incorrect padding of Maxpool and prevent potential DMA 3D transfer issues of Maxpool.
+
+- DMA 1D Copy Assertion Issue: Updated the DMA length datatype from uint16 to uint32 to avoid assertion failures when dealing with large block transfers.
+
 ## Implemented Updates for handling Quantized Linear DNN
 
 ### Added
 - New `_sanitizeGraphNames` function to sanitize the names of the nodes and tensors of the graph
 - Implementation for both Generic and Siracusa targets in the Deeploy framework
-- Modified the binding of dequant in `Bindings.py` to handle int32 after GEMM operation
\ No newline at end of file
+- Modified the binding of dequant in `Bindings.py` to handle int32 after GEMM operation
diff --git a/Deeploy/Targets/PULPOpen/TileConstraints/MatMulTileConstraint.py b/Deeploy/Targets/PULPOpen/TileConstraints/MatMulTileConstraint.py
index db236953dc..2b5d284159 100644
--- a/Deeploy/Targets/PULPOpen/TileConstraints/MatMulTileConstraint.py
+++ b/Deeploy/Targets/PULPOpen/TileConstraints/MatMulTileConstraint.py
@@ -99,9 +99,6 @@ def addPolicyConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkCo
         tilerModel.addConstraint(ASecondDimVar == parseDict['N'])
         tilerModel.addConstraint(BFirstDimVar == parseDict['N'])
 
-        # VIC: For now we tile only one of the inputs
-        tilerModel.addConstraint(BSecondDimVar == parseDict['O'])
-
         return tilerModel
 
     @classmethod
diff --git a/Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py b/Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py
index d2ecf5a568..695ba15812 100644
--- a/Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py
+++ b/Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py
@@ -33,10 +33,11 @@
 from Deeploy.TilingExtension.MemoryConstraints import NodeMemoryConstraint
 from Deeploy.TilingExtension.TileConstraint import TileConstraint
 from Deeploy.TilingExtension.TilerModel import TilerModel
-from Deeploy.TilingExtension.TilingCodegen import AbsoluteHyperRectangle, TilingSchedule, VariableReplacementScheme
+from Deeploy.TilingExtension.TilingCodegen import AbsoluteHyperRectangle, HyperRectangle, TilingSchedule, \
+    VariableReplacementScheme
 
 
-class MaxPoolTileConstraint(TileConstraint):
+class MaxPoolHWTileConstraint(TileConstraint):
 
     @staticmethod
     def addGeometricalConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
@@ -173,3 +174,89 @@ def serializeTilingSolution(
         variableReplacementSchedule = VariableReplacementScheme(replacements, replacementTypes)
 
         return variableReplacementSchedule, tilingSchedule
+
+
+# RW: This constraint tiles the channels of maxpool, which avoids issues with padding margin calculations
+#  when the default memory level is L3.
+
+
+class MaxPoolCTileConstraint(TileConstraint):
+
+    @staticmethod
+    def addGeometricalConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for bufferName in [inputBufferName, outputBufferName]:
+            tilerModel.addTensorDimToModel(ctxt, bufferName)
+
+        # RW: Apply constraints only to the Channel dimension
+        tilerModel.addConstraint(
+            tilerModel.getTensorDimVar(tensorName = outputBufferName, dimIdx = numDims - 1) ==
+            tilerModel.getTensorDimVar(tensorName = inputBufferName, dimIdx = numDims - 1))
+
+        return tilerModel
+
+    @staticmethod
+    def addPolicyConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for idx in range(numDims):
+            if idx != numDims - 1:  # RW: Keep all dimensions except C (index 1 in NCHW) fixed
+                tilerModel.addConstraint(
+                    tilerModel.getTensorDimVar(tensorName = outputBufferName, dimIdx = idx) == ctxt.lookup(
+                        outputBufferName).shape[idx])
+                tilerModel.addConstraint(
+                    tilerModel.getTensorDimVar(tensorName = inputBufferName, dimIdx = idx) == ctxt.lookup(
+                        inputBufferName).shape[idx])
+
+        return tilerModel
+
+    @classmethod
+    def serializeTilingSolution(
+            cls, tilingSolution: NodeMemoryConstraint, absoluteOutputCubes: List[AbsoluteHyperRectangle],
+            targetMemLevel: str, ctxt: NetworkContext,
+            operatorRepresentation: OperatorRepresentation) -> Tuple[VariableReplacementScheme, TilingSchedule]:
+        outputCubes = [cube.rectangle for cube in absoluteOutputCubes]
+
+        addrNames = ['data_in', 'data_out']
+        inputBaseOffsets, outputBaseOffsets = cls.extractBaseAddr(tilingSolution, targetMemLevel,
+                                                                  operatorRepresentation, addrNames)
+
+        inputInCubes = []
+        replacementTypes = {}
+        replacements: Dict[str, List[int]] = {}
+
+        numDims = len(ctxt.lookup(operatorRepresentation['data_in']).shape)
+        replacementTypes["ch_im_in"] = PointerClass(uint16_t)
+        replacements["ch_im_in"] = []
+
+        input_shape = ctxt.lookup(operatorRepresentation['data_in']).shape
+        output_shape = ctxt.lookup(operatorRepresentation['data_out']).shape
+
+        for cube in outputCubes:
+            input_offset = list(cube.offset)
+            input_dims = list(input_shape)
+            input_dims[-1] = cube.dims[-1]
+            InCube = HyperRectangle(tuple(input_offset), tuple(input_dims))
+            inputInCubes.append(InCube)
+
+            replacements["ch_im_in"].append(input_dims[-1])
+
+        inputLoadSchedule = []
+        outputLoadSchedule = []
+
+        for a in inputInCubes:
+            inputLoadSchedule.append({"data_in": a})
+
+        for out in outputCubes:
+            outputLoadSchedule.append({"data_out": out})
+
+        tilingSchedule = TilingSchedule(inputBaseOffsets, outputBaseOffsets, inputLoadSchedule, outputLoadSchedule)
+        variableReplacementSchedule = VariableReplacementScheme(replacements, replacementTypes)
+
+        return variableReplacementSchedule, tilingSchedule
diff --git a/Deeploy/Targets/PULPOpen/Tiler.py b/Deeploy/Targets/PULPOpen/Tiler.py
index b5cad3676a..1037d94ac1 100644
--- a/Deeploy/Targets/PULPOpen/Tiler.py
+++ b/Deeploy/Targets/PULPOpen/Tiler.py
@@ -53,7 +53,7 @@
 from Deeploy.Targets.PULPOpen.TileConstraints.iSoftmaxTileConstraint import iSoftmaxTileConstraint
 from Deeploy.Targets.PULPOpen.TileConstraints.LayernormTileConstraint import LayernormTileConstraint
 from Deeploy.Targets.PULPOpen.TileConstraints.MatMulTileConstraint import MatMulTileConstraint
-from Deeploy.Targets.PULPOpen.TileConstraints.MaxPoolTileConstraint import MaxPoolTileConstraint
+from Deeploy.Targets.PULPOpen.TileConstraints.MaxPoolTileConstraint import MaxPoolCTileConstraint
 from Deeploy.Targets.PULPOpen.TileConstraints.RequantShiftTileConstraint import RequantShiftTileConstraint
 from Deeploy.TilingExtension.TilerExtension import TilingReadyNodeBindings
 
@@ -98,7 +98,7 @@
                                                          tileConstraint = NOPTileConstraint())
 
 PULPMaxPool2DTilingReadyBindings = TilingReadyNodeBindings(nodeBindings = PULPMaxPool2DBindings,
-                                                           tileConstraint = MaxPoolTileConstraint())
+                                                           tileConstraint = MaxPoolCTileConstraint())
 
 PULPRQSTilingReadyBindings = TilingReadyNodeBindings(nodeBindings = PULPRQSBindings,
                                                      tileConstraint = RequantShiftTileConstraint())


Reviews:
user: login: Victor-Jung
body: LGTM, a few minor comments. After addressing comment, I will merge this PR as soon as [this PR](https://github.com/pulp-platform/Deeploy/pull/47) is merged.
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T08:03:41Z
user: login: runwangdl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T16:23:02Z
user: login: runwangdl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T16:23:13Z
user: login: runwangdl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-03T16:23:18Z
user: login: Victor-Jung
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T10:44:01Z
user: login: runwangdl
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-18T11:03:45Z
user: login: Victor-Jung
body: Perfect, I will merge once it's rebased :D
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-19T15:10:57Z

Review comments:
diff_hunk: @@ -0,0 +1,34 @@
+# ----------------------------------------------------------------------
+#
+# File: FloatSoftmaxTemplate.py
+#
+# Last edited: 23.1.2025
+#
+# Copyright (C) 2021, ETH Zurich and University of Bologna.
+#
+# Author: Run Wang, ETH Zurich
+#
+# ----------------------------------------------------------------------
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the License); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from Deeploy.DeeployTypes import NodeTemplate
+
+referenceTemplate = NodeTemplate("""
+// Softmax (Name: ${nodeName}, Op: ${nodeOp})
+int8_t ${nodeName}_core_id = pi_core_id();
+if (${nodeName}_core_id == 0) {
+    Softmax_fp${data_in_type.referencedType.typeWidth}_fp${data_out_type.referencedType.typeWidth}(${data_in}, ${data_out}, ${size}, ${lastDimLength});
+}
+""")
path: Deeploy/Targets/PULPOpen/Templates/FloatSoftmaxTemplate.py
user: login: Victor-Jung
body: Why not using the SINGLE_CORE macro from the previous PR?
created_at: 2025-03-03T07:59:13Z
author_association: MEMBER
diff_hunk: @@ -173,3 +174,85 @@ def serializeTilingSolution(
         variableReplacementSchedule = VariableReplacementScheme(replacements, replacementTypes)
 
         return variableReplacementSchedule, tilingSchedule
+
+
+class MaxPoolCTileConstraint(TileConstraint):
+
+    @staticmethod
+    def addGeometricalConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for bufferName in [inputBufferName, outputBufferName]:
+            tilerModel.addTensorDimToModel(ctxt, bufferName)
+
+        # RW: Apply constraints only to the Channel dimension
+        tilerModel.addConstraint(
+            tilerModel.getTensorDimVar(tensorName = outputBufferName, dimIdx = numDims - 1) ==
+            tilerModel.getTensorDimVar(tensorName = inputBufferName, dimIdx = numDims - 1))
+
+        return tilerModel
+
+    @staticmethod
+    def addPolicyConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for idx in range(numDims):
+            if idx != numDims - 1:  # RW: Keep all dimensions except C (index 1 in NCHW) fixed
path: Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py
user: login: Victor-Jung
body: Explain a bit better why we tile only on the C dimension.
created_at: 2025-03-03T08:00:06Z
author_association: MEMBER
diff_hunk: @@ -42,4 +42,7 @@
 
 #include "DeeployBasicMath.h"
 
+#define BEGIN_SINGLE_CORE if (pi_core_id() == 0) {
+#define END_SINGLE_CORE }
+#define SINGLE_CORE if (pi_core_id() == 0)
path: TargetLibraries/PULPOpen/inc/DeeployMath.h
user: login: Victor-Jung
body: In the previous PR, you created these macros in a separate file (`pulp_core.h`). If possible, I prefer these macro to be in `DeeployMath.h` so remove them from `pulp_core.h`.
created_at: 2025-03-03T08:02:05Z
author_association: MEMBER
diff_hunk: @@ -0,0 +1,34 @@
+# ----------------------------------------------------------------------
+#
+# File: FloatSoftmaxTemplate.py
+#
+# Last edited: 23.1.2025
+#
+# Copyright (C) 2021, ETH Zurich and University of Bologna.
+#
+# Author: Run Wang, ETH Zurich
+#
+# ----------------------------------------------------------------------
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the License); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an AS IS BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from Deeploy.DeeployTypes import NodeTemplate
+
+referenceTemplate = NodeTemplate("""
+// Softmax (Name: ${nodeName}, Op: ${nodeOp})
+int8_t ${nodeName}_core_id = pi_core_id();
+if (${nodeName}_core_id == 0) {
+    Softmax_fp${data_in_type.referencedType.typeWidth}_fp${data_out_type.referencedType.typeWidth}(${data_in}, ${data_out}, ${size}, ${lastDimLength});
+}
+""")
path: Deeploy/Targets/PULPOpen/Templates/FloatSoftmaxTemplate.py
user: login: runwangdl
body: Fixed in commit [e4f1b7](https://github.com/pulp-platform/Deeploy/commit/e4f1b7e5a7fc5e50f55f58ec0c35379cfc96a1c0) . 
created_at: 2025-03-03T16:23:02Z
author_association: CONTRIBUTOR
diff_hunk: @@ -173,3 +174,85 @@ def serializeTilingSolution(
         variableReplacementSchedule = VariableReplacementScheme(replacements, replacementTypes)
 
         return variableReplacementSchedule, tilingSchedule
+
+
+class MaxPoolCTileConstraint(TileConstraint):
+
+    @staticmethod
+    def addGeometricalConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for bufferName in [inputBufferName, outputBufferName]:
+            tilerModel.addTensorDimToModel(ctxt, bufferName)
+
+        # RW: Apply constraints only to the Channel dimension
+        tilerModel.addConstraint(
+            tilerModel.getTensorDimVar(tensorName = outputBufferName, dimIdx = numDims - 1) ==
+            tilerModel.getTensorDimVar(tensorName = inputBufferName, dimIdx = numDims - 1))
+
+        return tilerModel
+
+    @staticmethod
+    def addPolicyConstraint(tilerModel: TilerModel, parseDict: Dict, ctxt: NetworkContext) -> TilerModel:
+        inputBufferName = parseDict['data_in']
+        outputBufferName = parseDict['data_out']
+        numDims = len(ctxt.lookup(inputBufferName).shape)
+
+        for idx in range(numDims):
+            if idx != numDims - 1:  # RW: Keep all dimensions except C (index 1 in NCHW) fixed
path: Deeploy/Targets/PULPOpen/TileConstraints/MaxPoolTileConstraint.py
user: login: runwangdl
body: Fixed in commit [e4f1b7](https://github.com/pulp-platform/Deeploy/commit/e4f1b7e5a7fc5e50f55f58ec0c35379cfc96a1c0) .
created_at: 2025-03-03T16:23:13Z
author_association: CONTRIBUTOR
diff_hunk: @@ -42,4 +42,7 @@
 
 #include "DeeployBasicMath.h"
 
+#define BEGIN_SINGLE_CORE if (pi_core_id() == 0) {
+#define END_SINGLE_CORE }
+#define SINGLE_CORE if (pi_core_id() == 0)
path: TargetLibraries/PULPOpen/inc/DeeployMath.h
user: login: runwangdl
body: Fixed in commit [e4f1b7](https://github.com/pulp-platform/Deeploy/commit/e4f1b7e5a7fc5e50f55f58ec0c35379cfc96a1c0) .
created_at: 2025-03-03T16:23:18Z
author_association: CONTRIBUTOR
diff_hunk: @@ -297,6 +297,7 @@ def _rectToDMAStruct(cls, ctxt: NetworkContext, rectangle: HyperRectangle, direc
         if number_of_1d_copies > 1 or number_of_2d_copies > 1:
             mchan_flags += 0x4
         mchan_cmd = length_2d_copy + (mchan_flags << 17)
+        assert length_2d_copy <= 2**17, f"The DMA transfer size for mchan should be representable with 17 bits, current value is {length_2d_copy}"
path: Deeploy/Targets/PULPOpen/CodeTransformationPasses/PULPClusterTilingSB.py
user: login: Victor-Jung
body: Something is wrong here. This line should no be an addition and it should be `assert length_2d_copy <= 2**17, f"The DMA transfer size for mchan should be representable with 17 bits, current number of bits required is {np.ceil(np.log2(length_2d_copy))}"`
created_at: 2025-03-18T10:44:00Z
author_association: MEMBER
diff_hunk: @@ -297,6 +297,7 @@ def _rectToDMAStruct(cls, ctxt: NetworkContext, rectangle: HyperRectangle, direc
         if number_of_1d_copies > 1 or number_of_2d_copies > 1:
             mchan_flags += 0x4
         mchan_cmd = length_2d_copy + (mchan_flags << 17)
+        assert length_2d_copy <= 2**17, f"The DMA transfer size for mchan should be representable with 17 bits, current value is {length_2d_copy}"
path: Deeploy/Targets/PULPOpen/CodeTransformationPasses/PULPClusterTilingSB.py
user: login: runwangdl
body: Fixed in commit [8bfdb13](https://github.com/pulp-platform/Deeploy/pull/45/commits/8bfdb13acd8463dd4b405e9c96f6cb843815fcf6)
created_at: 2025-03-18T11:03:45Z
author_association: CONTRIBUTOR

PR comments:
user: login: Victor-Jung
created_at: 2025-03-19T14:31:56Z
body: LGTM, you added two `.sh` files, most likely by accident. Please remove them.
author_association: MEMBER
user: login: runwangdl
created_at: 2025-03-19T15:04:45Z
body: > LGTM, you added two `.sh` files, most likely by accident. Please remove them.
Fixed in Commit [031dc79](https://github.com/pulp-platform/Deeploy/pull/45/commits/031dc79c224f81f175ae191cfcd27a112d7e6e4f)

author_association: CONTRIBUTOR
user: login: runwangdl
created_at: 2025-03-19T15:19:01Z
body: > Perfect, I will merge once it's rebased :D

Conflicts merged
author_association: CONTRIBUTOR