Repository: onnx/onnx
Pull request: Add shape inference function for dynamic quantize linear
Signed-off-by: Anna Jung (VMware) <antheaj@vmware.com>

**Description**
- Add shape inference function to dynamic quantize linear

**Motivation and Context**
- Why is this change required? What problem does it solve? 
explicitly add shape inference function

- If it fixes an open issue, please link to the issue here.
fixes #3505 

Problem statement:
DynamicQuantizeLinear function does not have shape inference function
# Bug Report

### Is the issue related to model conversion? No

### Describe the bug
DynamicQuantizeLinear function op does not have shape inference function defined. In absence of shape inference, function body is used to get the shape inference for the function op and although it works as a fallback option it hurts perf.

### Expected behavior
Add shape inference function for DynamicQuantizeLinear 

### Notes

Any additional information

Hints from issue discussion:
@annajung : Can you pick this up?
Yeah, I'll give it a try! :D 
Thanks we are targeting this for ONNX 1.10 release which is planned for July end.
Hi @askhade, started working on this and have a question about if there is a good way to set the dim value for the outputs? Is the right way is to calculate each output in the shape inference function?


Each dim value represents the shape for that dimension in TensorShapeProto. Depending on the kind of operator the way output dims are calculated can be different... in some cases we simply propagate input shape to output shape, in some an attribute or input to op explicitly states the output shape and in others we explicitly calculate the value for every dimension.

Commit message: Merge branch 'master' into master

Patch to review:
diff --git a/onnx/defs/quantization/defs.cc b/onnx/defs/quantization/defs.cc
index 69ed7d87e7a..7b2f21f6838 100644
--- a/onnx/defs/quantization/defs.cc
+++ b/onnx/defs/quantization/defs.cc
@@ -179,6 +179,20 @@ ONNX_OPERATOR_SET_SCHEMA(
            {{"Zeropoint"}, "Cast", {"Rounded_ZeroPoint_FP"}, {MakeAttribute("to", int64_t(2))}},
            {{"y_scale"}, "Identity", {"Scale"}},
            {{"y_zero_point"}, "Identity", {"Zeropoint"}},
-           {{"y"}, "QuantizeLinear", {"x", "Scale", "Zeropoint"}}})));
+           {{"y"}, "QuantizeLinear", {"x", "Scale", "Zeropoint"}}}))
+        .TypeAndShapeInferenceFunction(
+            [](ONNX_NAMESPACE::InferenceContext& ctx) {
+              updateOutputElemType(ctx, 0, TensorProto::UINT8);
+              updateOutputElemType(ctx, 1, TensorProto::FLOAT);
+              updateOutputElemType(ctx, 2, TensorProto::UINT8);
+
+              if (!hasInputShape(ctx, 0))
+                return;
+
+              auto& input_shape = getInputShape(ctx, 0);
+              updateOutputShape(ctx, 0, input_shape);
+              ctx.getOutputType(1)->mutable_tensor_type()->mutable_shape();
+              ctx.getOutputType(2)->mutable_tensor_type()->mutable_shape();
+            }));
 
 } // namespace ONNX_NAMESPACE
diff --git a/onnx/test/shape_inference_test.py b/onnx/test/shape_inference_test.py
index 37ec58d2c9b..bf089e9eb9e 100644
--- a/onnx/test/shape_inference_test.py
+++ b/onnx/test/shape_inference_test.py
@@ -2317,6 +2317,15 @@ def test_dequantizelinear(self):  # type: () -> None
             [])
         self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.FLOAT, (30, 4, 5))])
 
+    def test_dynamicquantizelinear(self):  # type: () -> None
+        graph = self._make_graph(
+            [('x', TensorProto.FLOAT, (30, 4, 5))],
+            [make_node('DynamicQuantizeLinear', ['x'], ['y', 'y_scale', 'y_zero_point'])],
+            [])
+        self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.UINT8, (30, 4, 5)),
+                                      make_tensor_value_info('y_scale', TensorProto.FLOAT, ()),
+                                      make_tensor_value_info('y_zero_point', TensorProto.UINT8, ())])
+
     def test_reversesequence(self):  # type: () -> None
         graph = self._make_graph(
             [('x', TensorProto.FLOAT, (4, 5, 6)),


Code review comments:
diff_hunk: @@ -179,6 +179,20 @@ ONNX_OPERATOR_SET_SCHEMA(
            {{"Zeropoint"}, "Cast", {"Rounded_ZeroPoint_FP"}, {MakeAttribute("to", int64_t(2))}},
            {{"y_scale"}, "Identity", {"Scale"}},
            {{"y_zero_point"}, "Identity", {"Zeropoint"}},
-           {{"y"}, "QuantizeLinear", {"x", "Scale", "Zeropoint"}}})));
+           {{"y"}, "QuantizeLinear", {"x", "Scale", "Zeropoint"}}}))
+        .TypeAndShapeInferenceFunction(
+            [](ONNX_NAMESPACE::InferenceContext& ctx) {
+              updateOutputElemType(ctx, 0, TensorProto::UINT8);
+              updateOutputElemType(ctx, 1, TensorProto::FLOAT);
+              updateOutputElemType(ctx, 2, TensorProto::UINT8);
+
+              if (!hasInputShape(ctx, 0))
+                return;
+
+              auto& input_shape = getInputShape(ctx, 0);
+              updateOutputShape(ctx, 0, input_shape);
+              ctx.getOutputType(1)->mutable_tensor_type()->mutable_shape();
path: onnx/defs/quantization/defs.cc
text: @user1:
I think that these two lines for scale and zero-point can be moved up before line 189 (since this is valid even if we don't know the input shape).

@author:
Sure that makes sense. Will update

@author:
done! PTAL