Repository: onnx/onnx
Pull request: Enhance shape inference: ParseData/Transpose/QuantizeLinear
**Description**
1. Enhance  ParseData check: having raw_data cannot be String
2 Transpose should check repeated perm to prevent issues
3 If third input (y_zero_point) of QuantizeLinear is an empty string, shape inference should still work because y_zero_point is an optional input

**Motivation and Context**
1. Continues a sub task from https://github.com/onnx/onnx/pull/3787#discussion_r733854228.
2. Closes #3709 
3. Closes #3744

Problem statement:
ONNX checker doesn't check repeated permutation for Transpose operator
# Bug Report

### Is the issue related to model conversion?
If the ONNX checker reports issues with this model then this is most probably related to the converter used to convert the original framework model to ONNX. Please create this bug in the appropriate converter's GitHub repo (pytorch, tensorflow-onnx, sklearn-onnx, keras-onnx, onnxmltools) to get the best help.

### Describe the bug
The ONNX checker doesn't check if `perm` attribute contains duplicate entry, which is checked in numpy. The sample code below repeats 0 axis and passed the check.

### System information
- OS Platform and Distribution (*e.g. Linux Ubuntu 16.04*): Ubuntu 20.04
- ONNX version (*e.g. 1.7*):  1.10.1
- Python version: 3.8.10


### Reproduction instructions
- Describe the code to reproduce the behavior.
```
from onnx import helper
from onnx import TensorProto as tp
from onnx import checker
from onnx import save
from onnx import shape_inference
inputs = [
    ("input", tp.FLOAT, [2, 3, 4]),
]
weights = []
outputs = [
    ("output", tp.FLOAT, [4, 2, 2]),
]
node = helper.make_node('Transpose',
                        inputs=[input[0] for input in (inputs + weights)],
                        outputs=[output[0] for output in outputs],
                        perm=[2, 0, 0])
graph = helper.make_graph(
    [node], 'test_graph',
    [helper.make_tensor_value_info(*input) for input in inputs],
    [helper.make_tensor_value_info(*output) for output in outputs],
    [helper.make_tensor(*weight) for weight in weights])
model = helper.make_model(graph, producer_name='model')
checker.check_model(model, full_check=True)
inferred_shape = shape_inference.infer_shapes(model)
print(inferred_shape)
```
- Attach the ONNX model to the issue (where applicable)

### Expected behavior
A clear and concise description of what you expected to happen.

### Notes

Any additional information


ONNX shape inference for QuantizeLinear fails on empty string as zero_point
# Bug Report

### Describe the bug
An ONNX model with QuantizeLinear (opset 13) passes shape inference when it has no input for zero_point, but fails if the input is the empty string.  Trailing optional inputs should be able to be expressed in either way.

onnx version 1.10.1

Hints from issue discussion:
Hi @GuanLuo,
Thank you for identifying this issue. I can verify it -- ONNX shape inference for Transpose does not check whether any perm value is repeated: https://github.com/onnx/onnx/blob/master/onnx/defs/tensor/defs.cc#L1157, but NumPy and [ONNXRuntime](https://github.com/microsoft/onnxruntime/blob/0367e1f1c27ab097b4272f29e880843568c29e95/onnxruntime/core/providers/cpu/tensor/transpose.h#L55) do check. It should be improved and anyone interested in enhancing this check is welcome. Thanks!
Hi @TomWildenhain-Microsoft,
Do you have any handy model/script to repro it? Thanks

Commit message: quant-10 as well

Signed-off-by: Chun-Wei Chen <jacky82226@gmail.com>

Patch to review:
diff --git a/onnx/defs/quantization/defs.cc b/onnx/defs/quantization/defs.cc
index a5d69a2ed93..ca20b931082 100644
--- a/onnx/defs/quantization/defs.cc
+++ b/onnx/defs/quantization/defs.cc
@@ -55,14 +55,14 @@ ONNX_OPERATOR_SET_SCHEMA(
         .SetDoc(QuantizeLinear_ver13_doc)
         .TypeAndShapeInferenceFunction(
             [](ONNX_NAMESPACE::InferenceContext& ctx) {
-              if (ctx.getNumInputs() == 3) {
+              if (ctx.getNumInputs() == 3 && ctx.getInputType(2) != nullptr) {
                 propagateElemTypeFromInputToOutput(ctx, 2, 0);
               } else {
                 updateOutputElemType(ctx, 0, TensorProto::UINT8);
               }
-
-              if (!hasInputShape(ctx, 0))
+              if (!hasInputShape(ctx, 0)) {
                 return;
+              }
 
               auto& input_shape = getInputShape(ctx, 0);
               updateOutputShape(ctx, 0, input_shape);
diff --git a/onnx/defs/quantization/old.cc b/onnx/defs/quantization/old.cc
index 88a2ec1bcde..211cbfb2575 100644
--- a/onnx/defs/quantization/old.cc
+++ b/onnx/defs/quantization/old.cc
@@ -47,10 +47,14 @@ ONNX_OPERATOR_SET_SCHEMA(
         .SetDoc(QuantizeLinear_ver10_doc)
         .TypeAndShapeInferenceFunction(
             [](ONNX_NAMESPACE::InferenceContext& ctx) {
-              propagateElemTypeFromInputToOutput(ctx, 2, 0);
-
-              if (!hasInputShape(ctx, 0))
+              if (ctx.getNumInputs() == 3 && ctx.getInputType(2) != nullptr) {
+                propagateElemTypeFromInputToOutput(ctx, 2, 0);
+              } else {
+                updateOutputElemType(ctx, 0, TensorProto::UINT8);
+              }
+              if (!hasInputShape(ctx, 0)) {
                 return;
+              }
 
               auto& input_shape = getInputShape(ctx, 0);
               updateOutputShape(ctx, 0, input_shape);
diff --git a/onnx/defs/tensor/defs.cc b/onnx/defs/tensor/defs.cc
index 34b46c69fc1..31c28fa5f98 100644
--- a/onnx/defs/tensor/defs.cc
+++ b/onnx/defs/tensor/defs.cc
@@ -1158,7 +1158,8 @@ ONNX_OPERATOR_SET_SCHEMA(
               perm.push_back(i);
           } else if (!perm.empty()) {
             // check if every index is valid
-            for (int64_t fromDimIndex : perm)
+            std::vector<bool> seen(shape.dim_size(), false);
+            for (int64_t fromDimIndex : perm) {
               if (!(0 <= fromDimIndex && fromDimIndex < shape.dim_size())) {
                 std::ostringstream oss;
                 oss << "Invalid attribute perm {" << perm[0];
@@ -1174,7 +1175,14 @@ ONNX_OPERATOR_SET_SCHEMA(
                   oss << "}";
                 }
                 fail_type_inference(oss.str());
+              } else {
+                // check if any perm is repeated
+                if (seen[fromDimIndex]) {
+                  fail_type_inference("Attribute perm for Transpose cannot be repeated: ", fromDimIndex);
+                }
+                seen[fromDimIndex] = true;
               }
+            }
           }
 
           propagateElemTypeFromInputToOutput(ctx, 0, 0);
diff --git a/onnx/defs/tensor/old.cc b/onnx/defs/tensor/old.cc
index 0e23b21ada2..549b7d4b52d 100644
--- a/onnx/defs/tensor/old.cc
+++ b/onnx/defs/tensor/old.cc
@@ -981,7 +981,8 @@ ONNX_OPERATOR_SET_SCHEMA(
               perm.push_back(i);
           } else if (!perm.empty()) {
             // check if every index is valid
-            for (int64_t fromDimIndex : perm)
+            std::vector<bool> seen(shape.dim_size(), false);
+            for (int64_t fromDimIndex : perm) {
               if (!(0 <= fromDimIndex && fromDimIndex < shape.dim_size())) {
                 std::ostringstream oss;
                 oss << "Invalid attribute perm {" << perm[0];
@@ -997,7 +998,14 @@ ONNX_OPERATOR_SET_SCHEMA(
                   oss << "}";
                 }
                 fail_type_inference(oss.str());
+              } else {
+                // check if any perm is repeated
+                if (seen[fromDimIndex]) {
+                  fail_type_inference("Attribute perm for Transpose cannot be repeated: ", fromDimIndex);
+                }
+                seen[fromDimIndex] = true;
               }
+            }
           }
 
           propagateElemTypeFromInputToOutput(ctx, 0, 0);
diff --git a/onnx/defs/tensor_proto_util.cc b/onnx/defs/tensor_proto_util.cc
index abb8caa5932..e9de0fffec2 100644
--- a/onnx/defs/tensor_proto_util.cc
+++ b/onnx/defs/tensor_proto_util.cc
@@ -64,6 +64,10 @@ namespace ONNX_NAMESPACE {
       res.insert(res.end(), data.begin(), data.end());                         \
       return res;                                                              \
     }                                                                          \
+    if (tensor_proto->data_type() == TensorProto_DataType_STRING) {            \
+      fail_shape_inference("The data_type of tensor: ", tensor_proto->name(),  \
+      " cannot be string because it has raw_data which disallows string.");    \
+    }                                                                          \
     /* The given tensor does have raw_data itself so parse it by given type */ \
     /* make copy as we may have to reverse bytes */                            \
     std::string raw_data = tensor_proto->raw_data();                           \


Code review comments:
diff_hunk: @@ -1174,7 +1175,14 @@ ONNX_OPERATOR_SET_SCHEMA(
                   oss << "}";
                 }
                 fail_type_inference(oss.str());
+              } else {
+                // check if any perm is repeated
+                if (seen[fromDimIndex]) {
+                  fail_type_inference("Attribute perm for Transpose cannot be repeated: ", fromDimIndex);
path: onnx/defs/tensor/defs.cc
text: @user1:
nit: Attribute perm for Transpose has repeated entry for value fromIndex? or something similar?

diff_hunk: @@ -64,6 +64,10 @@ namespace ONNX_NAMESPACE {
       res.insert(res.end(), data.begin(), data.end());                         \
       return res;                                                              \
     }                                                                          \
+    if (tensor_proto->data_type() == TensorProto_DataType_STRING) {            \
+      fail_shape_inference("The data_type of tensor: ", tensor_proto->name(),  \
+      " cannot be string because it has raw_data which disallows string.");    \
path: onnx/defs/tensor_proto_util.cc
text: @user1:
nit: tensor_proto->name() data type is string. string content is required to be stored in repeated bytes string_data field. raw_data type cannot be string. 

@author:
Both updated for improving error messages. Thanks