Repository: onnx/onnx
Pull request: Support dynamic 'pads' and 'value' in Pad operator
Change description:

This change is to make two static attributes of the Pad op - 'pads' and 'value' into dynamic inputs to support conversion of some TensorFlow models into ONNX which allow these 2 values to be fed in dynamically to the op. This change is to basically support conversion of such TF models into ONNX.

Summary of changes:

* Make interface change in Pad operator as described in title
* Rename 'value' to 'constant_value' to reflect its usage only for 'constant' mode
* Make related node test case changes
* Make related shape inference test changes
* Make related changes in optimizer passes
* Make related optimizer test changes

Issues that are resolved:
Resolves #1765 - Adds description and examples for 'Edge' and 'Reflect' mode
Resolves #2313 - Allows testing optimizer(s) for previous opset specs of ops/nodes

Problem statement:
Meaning of edge and reflect in Pad operator?
The meaning of edge and reflect are not described in the documentation for the Pad operator. There is an example, but please tell us the intended meaning in English if you would.

Thanks

Allow creation of older opset ops/nodes 
It will be great if the onnx helper's `make_node()` abstraction helps us create older opset nodes/ops that we can use to cover shape inference tests and support testing the optimizer passesfor the older version of the op's spec. 

This will help guard against shape inference regressions for older opsets and test to make sure optimizer passes can handle correctly all versions of an op's spec.

Hints from issue discussion:
let's keep this issue open to track the pad op spec clarification.

Commit message: Update Changelog.md

Patch to review:
diff --git a/docs/Changelog.md b/docs/Changelog.md
index 14e7621e7b9..e2d04e4430b 100644
--- a/docs/Changelog.md
+++ b/docs/Changelog.md
@@ -11461,6 +11461,81 @@ This version of the operator has been available since version 11 of the default
 <dd>Constrain to any tensor type.</dd>
 </dl>
 
+### <a name="Pad-11"></a>**Pad-11**</a>
+
+  Given a tensor containing the data to be padded (`data`), a tensor containing the number of start and end pad values for axis (`pads`), (optionally) a `mode`, and (optionally) `value`, 
+  a padded tensor (`output`) is generated.
+  
+  The three supported `modes` are (similar to corresponding modes supported by `numpy.pad`):
+  
+  1) `constant`(default) - pads with a given constant value as specified by `value` (which defaults to 0)
+  
+  2) `reflect` - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis
+  
+  3) `edge` - pads with the edge values of array
+  
+  Example (`constant` mode):
+    Insert 0 pads to the beginning of the second dimension.
+  
+    data = 
+    [
+        [1.0, 1.2],
+        [2.3, 3.4],
+        [4.5, 5.7],
+    ] 
+  
+    pads = [0, 2, 0, 0]
+  
+    mode = 'constant'
+  
+    value = 0.0
+  
+    output = 
+    [
+        [
+            [0.0, 0.0, 1.0, 1.2],
+            [0.0, 0.0, 2.3, 3.4],
+            [0.0, 0.0, 4.5, 5.7],
+        ],
+    ]
+  
+
+#### Version
+
+This version of the operator has been available since version 11 of the default ONNX operator set.
+
+#### Attributes
+
+<dl>
+<dt><tt>mode</tt> : string (default is constant)</dt>
+<dd>Supported modes: `constant`(default), `reflect`, `edge`</dd>
+</dl>
+
+#### Inputs (2 - 3)
+
+<dl>
+<dt><tt>data</tt> : T</dt>
+<dd>Input tensor.</dd>
+<dt><tt>pads</tt> : tensor(int64)</dt>
+<dd>Tensor of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. `pads` should be a 1D tensor of shape [2 * input_rank]. `pads` format should be: [x1_begin, x2_begin,...,x1_end, x2_end,...], where xi_begin is the number of pad values added at the beginning of axis `i` and xi_end, the number of pad values added at the end of axis `i`.</dd>
+<dt><tt>value</tt> (optional) : T</dt>
+<dd>(Optional) A scalar value to be used if the mode chosen is `constant` (by default it is 0.0).</dd>
+</dl>
+
+#### Outputs
+
+<dl>
+<dt><tt>output</tt> : T</dt>
+<dd>Tensor after padding.</dd>
+</dl>
+
+#### Type Constraints
+
+<dl>
+<dt><tt>T</tt> : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double)</dt>
+<dd>Constrains input and output to only numeric types.</dd>
+</dl>
+
 ### <a name="Range-11"></a>**Range-11**</a>
 
   Generate a tensor containing a sequence of numbers that begin at `start` and extends by increments of `delta` 
diff --git a/docs/Operators.md b/docs/Operators.md
index f820681fc5c..32b5b287313 100644
--- a/docs/Operators.md
+++ b/docs/Operators.md
@@ -9934,45 +9934,65 @@ expect(node, inputs=[x, slope], outputs=[y],
 
 ### <a name="Pad"></a><a name="pad">**Pad**</a>
 
-  Given `data` tensor, pads, mode, and value.
-  Example:
+  Given a tensor containing the data to be padded (`data`), a tensor containing the number of start and end pad values for axis (`pads`), (optionally) a `mode`, and (optionally) `value`, 
+  a padded tensor (`output`) is generated.
+  
+  The three supported `modes` are (similar to corresponding modes supported by `numpy.pad`):
+  
+  1) `constant`(default) - pads with a given constant value as specified by `value` (which defaults to 0)
+  
+  2) `reflect` - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis
+  
+  3) `edge` - pads with the edge values of array
+  
+  Example (`constant` mode):
     Insert 0 pads to the beginning of the second dimension.
-    data = [
+  
+    data = 
+    [
         [1.0, 1.2],
         [2.3, 3.4],
         [4.5, 5.7],
-    ]
+    ] 
+  
     pads = [0, 2, 0, 0]
-    output = [
+  
+    mode = 'constant'
+  
+    value = 0.0
+  
+    output = 
+    [
         [
             [0.0, 0.0, 1.0, 1.2],
             [0.0, 0.0, 2.3, 3.4],
             [0.0, 0.0, 4.5, 5.7],
         ],
     ]
+  
 
 #### Version
 
-This version of the operator has been available since version 2 of the default ONNX operator set.
+This version of the operator has been available since version 11 of the default ONNX operator set.
 
-Other versions of this operator: <a href="Changelog.md#Pad-1">Pad-1</a>
+Other versions of this operator: <a href="Changelog.md#Pad-1">Pad-1</a>, <a href="Changelog.md#Pad-2">Pad-2</a>
 
 #### Attributes
 
 <dl>
 <dt><tt>mode</tt> : string (default is constant)</dt>
-<dd>Three modes: constant(default), reflect, edge</dd>
-<dt><tt>pads</tt> : list of ints (required)</dt>
-<dd>List of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D it is the number of pixels. `pads` rank should be double of the input's rank. `pads` format should be as follow [x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels added at the beginning of axis `i` and xi_end, the number of pixels added at the end of axis `i`.</dd>
-<dt><tt>value</tt> : float (default is 0.0)</dt>
-<dd>One float, indicates the value to be filled.</dd>
+<dd>Supported modes: `constant`(default), `reflect`, `edge`</dd>
 </dl>
 
-#### Inputs
+#### Inputs (2 - 3)
 
 <dl>
 <dt><tt>data</tt> : T</dt>
 <dd>Input tensor.</dd>
+<dt><tt>pads</tt> : tensor(int64)</dt>
+<dd>Tensor of integers indicating the number of padding elements to add or remove (if negative) at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. `pads` should be a 1D tensor of shape [2 * input_rank]. `pads` format should be: [x1_begin, x2_begin,...,x1_end, x2_end,...], where xi_begin is the number of pad values added at the beginning of axis `i` and xi_end, the number of pad values added at the end of axis `i`.</dd>
+<dt><tt>value</tt> (optional) : T</dt>
+<dd>(Optional) A scalar value to be used if the mode chosen is `constant` (by default it is 0.0).</dd>
 </dl>
 
 #### Outputs
@@ -9985,8 +10005,8 @@ Other versions of this operator: <a href="Changelog.md#Pad-1">Pad-1</a>
 #### Type Constraints
 
 <dl>
-<dt><tt>T</tt> : tensor(float16), tensor(float), tensor(double)</dt>
-<dd>Constrain input and output types to float tensors.</dd>
+<dt><tt>T</tt> : tensor(uint8), tensor(uint16), tensor(uint32), tensor(uint64), tensor(int8), tensor(int16), tensor(int32), tensor(int64), tensor(float16), tensor(float), tensor(double)</dt>
+<dd>Constrains input and output to only numeric types.</dd>
 </dl>
 
 
@@ -9998,21 +10018,21 @@ Other versions of this operator: <a href="Changelog.md#Pad-1">Pad-1</a>
 ```python
 node = onnx.helper.make_node(
     'Pad',
-    inputs=['x'],
+    inputs=['x', 'pads', 'value'],
     outputs=['y'],
-    mode='constant',
-    value=1.2,
-    pads=[0, 0, 1, 3, 0, 0, 2, 4],
+    mode='constant'
 )
 x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-y = np.pad(
+pads = np.array([0, 0, 1, 3, 0, 0, 2, 4]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+value = np.float32(1.2)
+y = pad_impl(
     x,
-    pad_width=((0, 0), (0, 0), (1, 2), (3, 4)),
-    mode='constant',
-    constant_values=1.2,
+    ((0, 0), (0, 0), (1, 2), (3, 4)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+    'constant',
+    1.2
 )
 
-expect(node, inputs=[x], outputs=[y],
+expect(node, inputs=[x, pads, value], outputs=[y],
        name='test_constant_pad')
 ```
 
@@ -10026,19 +10046,19 @@ expect(node, inputs=[x], outputs=[y],
 for mode in ['edge', 'reflect']:
     node = onnx.helper.make_node(
         'Pad',
-        inputs=['x'],
+        inputs=['x', 'pads'],
         outputs=['y'],
-        mode=mode,
-        pads=[0, 0, 1, 1, 0, 0, 1, 1]
+        mode=mode
     )
-    x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-    y = np.pad(
+    x = np.random.randn(1, 3, 4, 5).astype(np.int32)
+    pads = np.array([0, 0, 1, 1, 0, 0, 1, 1]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+    y = pad_impl(
         x,
-        pad_width=((0, 0), (0, 0), (1, 1), (1, 1)),
-        mode=mode,
+        ((0, 0), (0, 0), (1, 1), (1, 1)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+        mode
     )
 
-    expect(node, inputs=[x], outputs=[y],
+    expect(node, inputs=[x, pads], outputs=[y],
            name='test_{}_pad'.format(mode))
 ```
 
diff --git a/docs/TestCoverage.md b/docs/TestCoverage.md
index 99c410d7b1b..20f0839801c 100644
--- a/docs/TestCoverage.md
+++ b/docs/TestCoverage.md
@@ -5434,21 +5434,21 @@ There are 2 test cases, listed as following:
 ```python
 node = onnx.helper.make_node(
     'Pad',
-    inputs=['x'],
+    inputs=['x', 'pads', 'value'],
     outputs=['y'],
-    mode='constant',
-    value=1.2,
-    pads=[0, 0, 1, 3, 0, 0, 2, 4],
+    mode='constant'
 )
 x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-y = np.pad(
+pads = np.array([0, 0, 1, 3, 0, 0, 2, 4]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+value = np.float32(1.2)
+y = pad_impl(
     x,
-    pad_width=((0, 0), (0, 0), (1, 2), (3, 4)),
-    mode='constant',
-    constant_values=1.2,
+    ((0, 0), (0, 0), (1, 2), (3, 4)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+    'constant',
+    1.2
 )
 
-expect(node, inputs=[x], outputs=[y],
+expect(node, inputs=[x, pads, value], outputs=[y],
        name='test_constant_pad')
 ```
 
@@ -5460,19 +5460,19 @@ expect(node, inputs=[x], outputs=[y],
 for mode in ['edge', 'reflect']:
     node = onnx.helper.make_node(
         'Pad',
-        inputs=['x'],
+        inputs=['x', 'pads'],
         outputs=['y'],
-        mode=mode,
-        pads=[0, 0, 1, 1, 0, 0, 1, 1]
+        mode=mode
     )
-    x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-    y = np.pad(
+    x = np.random.randn(1, 3, 4, 5).astype(np.int32)
+    pads = np.array([0, 0, 1, 1, 0, 0, 1, 1]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+    y = pad_impl(
         x,
-        pad_width=((0, 0), (0, 0), (1, 1), (1, 1)),
-        mode=mode,
+        ((0, 0), (0, 0), (1, 1), (1, 1)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+        mode
     )
 
-    expect(node, inputs=[x], outputs=[y],
+    expect(node, inputs=[x, pads], outputs=[y],
            name='test_{}_pad'.format(mode))
 ```
 
diff --git a/onnx/backend/test/case/node/pad.py b/onnx/backend/test/case/node/pad.py
index ab173fc3191..2b6c2f9c32f 100644
--- a/onnx/backend/test/case/node/pad.py
+++ b/onnx/backend/test/case/node/pad.py
@@ -10,27 +10,46 @@
 from . import expect
 
 
+def pad_impl(data, pad_width, mode, constant_values=0.0):
+    # type: (np.ndarray, tuple, str, float) -> np.ndarray
+    if mode == 'constant':
+        y = np.pad(
+            data,
+            pad_width=pad_width,
+            mode=mode,
+            constant_values=constant_values,
+        )
+        return y
+
+    y = np.pad(
+        data,
+        pad_width=pad_width,
+        mode=mode,
+    )
+    return y
+
+
 class Pad(Base):
 
     @staticmethod
     def export_constant_pad():  # type: () -> None
         node = onnx.helper.make_node(
             'Pad',
-            inputs=['x'],
+            inputs=['x', 'pads', 'value'],
             outputs=['y'],
-            mode='constant',
-            value=1.2,
-            pads=[0, 0, 1, 3, 0, 0, 2, 4],
+            mode='constant'
         )
         x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-        y = np.pad(
+        pads = np.array([0, 0, 1, 3, 0, 0, 2, 4]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+        value = np.float32(1.2)
+        y = pad_impl(
             x,
-            pad_width=((0, 0), (0, 0), (1, 2), (3, 4)),
-            mode='constant',
-            constant_values=1.2,
+            ((0, 0), (0, 0), (1, 2), (3, 4)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+            'constant',
+            1.2
         )
 
-        expect(node, inputs=[x], outputs=[y],
+        expect(node, inputs=[x, pads, value], outputs=[y],
                name='test_constant_pad')
 
     @staticmethod
@@ -38,17 +57,17 @@ def export_reflection_and_edge_pad():  # type: () -> None
         for mode in ['edge', 'reflect']:
             node = onnx.helper.make_node(
                 'Pad',
-                inputs=['x'],
+                inputs=['x', 'pads'],
                 outputs=['y'],
-                mode=mode,
-                pads=[0, 0, 1, 1, 0, 0, 1, 1]
+                mode=mode
             )
-            x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-            y = np.pad(
+            x = np.random.randn(1, 3, 4, 5).astype(np.int32)
+            pads = np.array([0, 0, 1, 1, 0, 0, 1, 1]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+            y = pad_impl(
                 x,
-                pad_width=((0, 0), (0, 0), (1, 1), (1, 1)),
-                mode=mode,
+                ((0, 0), (0, 0), (1, 1), (1, 1)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
+                mode
             )
 
-            expect(node, inputs=[x], outputs=[y],
+            expect(node, inputs=[x, pads], outputs=[y],
                    name='test_{}_pad'.format(mode))
diff --git a/onnx/backend/test/data/node/test_constant_pad/model.onnx b/onnx/backend/test/data/node/test_constant_pad/model.onnx
index 60fb751fd56..c4edfc6ca3e 100644
Binary files a/onnx/backend/test/data/node/test_constant_pad/model.onnx and b/onnx/backend/test/data/node/test_constant_pad/model.onnx differ
diff --git a/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_1.pb b/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_1.pb
new file mode 100644
index 00000000000..14c2aba5f41
Binary files /dev/null and b/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_1.pb differ
diff --git a/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_2.pb b/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_2.pb
new file mode 100644
index 00000000000..61cc724a896
--- /dev/null
+++ b/onnx/backend/test/data/node/test_constant_pad/test_data_set_0/input_2.pb
@@ -0,0 +1 @@
+BvalueJ���?
\ No newline at end of file
diff --git a/onnx/backend/test/data/node/test_edge_pad/model.onnx b/onnx/backend/test/data/node/test_edge_pad/model.onnx
index 64708345a8d..57f2e783a0e 100644
Binary files a/onnx/backend/test/data/node/test_edge_pad/model.onnx and b/onnx/backend/test/data/node/test_edge_pad/model.onnx differ
diff --git a/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_0.pb b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_0.pb
index 09d5a14a4bf..f625fc015a6 100644
Binary files a/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_0.pb and b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_0.pb differ
diff --git a/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_1.pb b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_1.pb
new file mode 100644
index 00000000000..131ea5f957c
Binary files /dev/null and b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/input_1.pb differ
diff --git a/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/output_0.pb
index ddbb36d2ff1..12dd0a3bab0 100644
Binary files a/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/output_0.pb and b/onnx/backend/test/data/node/test_edge_pad/test_data_set_0/output_0.pb differ
diff --git a/onnx/backend/test/data/node/test_reflect_pad/model.onnx b/onnx/backend/test/data/node/test_reflect_pad/model.onnx
index bf496ba08c2..55c1776a1cb 100644
Binary files a/onnx/backend/test/data/node/test_reflect_pad/model.onnx and b/onnx/backend/test/data/node/test_reflect_pad/model.onnx differ
diff --git a/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_0.pb b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_0.pb
index 64a85e419fa..3737f7c7506 100644
Binary files a/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_0.pb and b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_0.pb differ
diff --git a/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_1.pb b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_1.pb
new file mode 100644
index 00000000000..131ea5f957c
Binary files /dev/null and b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/input_1.pb differ
diff --git a/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/output_0.pb b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/output_0.pb
index 0f809b59e23..1827c788068 100644
Binary files a/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/output_0.pb and b/onnx/backend/test/data/node/test_reflect_pad/test_data_set_0/output_0.pb differ
diff --git a/onnx/common/platform_helpers.h b/onnx/common/platform_helpers.h
new file mode 100644
index 00000000000..14c6c5f3776
--- /dev/null
+++ b/onnx/common/platform_helpers.h
@@ -0,0 +1,16 @@
+// Copyright (c) ONNX Project Contributors.
+// Licensed under the MIT license.
+
+#pragma once
+
+namespace ONNX_NAMESPACE {
+
+// Determine if the processor is little endian or not
+inline bool is_processor_little_endian() {
+  int num = 1;
+  if (*(char*)&num == 1)
+    return true;
+  return false;
+}
+
+} // namespace ONNX_NAMESPACE
diff --git a/onnx/defs/operator_sets.h b/onnx/defs/operator_sets.h
index 29c1cc6a841..2bcb961d87d 100644
--- a/onnx/defs/operator_sets.h
+++ b/onnx/defs/operator_sets.h
@@ -617,16 +617,17 @@ class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, LogSoftmax);
 class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Softmax);
 class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Scan);
 class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Split);
+class ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Pad);
 
 // Iterate over schema from ai.onnx version 11
 class OpSet_Onnx_ver11 {
  public:
   static void ForEachSchema(std::function<void(OpSchema&&)> fn) {
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Loop)>());
+    fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, BitShift)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Unique)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, CumSum)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Round)>());
-    fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, BitShift)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, TopK)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(
            Onnx, 11, DepthToSpace)>());
@@ -670,10 +671,11 @@ class OpSet_Onnx_ver11 {
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Softmax)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Scan)>());
     fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Split)>());
+    fn(GetOpSchema<ONNX_OPERATOR_SET_SCHEMA_CLASS_NAME(Onnx, 11, Pad)>());
   }
 };
 
-inline void RegisterOnnxOperatorSetSchema() {
+  inline void RegisterOnnxOperatorSetSchema() {
   RegisterOpSetSchema<OpSet_Onnx_ver1>();
   RegisterOpSetSchema<OpSet_Onnx_ver2>();
   RegisterOpSetSchema<OpSet_Onnx_ver3>();
diff --git a/onnx/defs/tensor/defs.cc b/onnx/defs/tensor/defs.cc
index 81040a8b6ab..18e8bc9fa30 100644
--- a/onnx/defs/tensor/defs.cc
+++ b/onnx/defs/tensor/defs.cc
@@ -292,7 +292,8 @@ ONNX_OPERATOR_SET_SCHEMA(
     11,
     OpSchema()
         .Attr(
-            "axis", "Which axis to concat on. A negative value means counting dimensions from the back. "
+            "axis",
+            "Which axis to concat on. A negative value means counting dimensions from the back. "
             "Accepted range is [-r, r-1] where r = rank(inputs)..",
             AttributeProto::INT)
         .SetDoc("Concatenate a list of tensors into a single tensor")
@@ -905,7 +906,11 @@ ONNX_OPERATOR_SET_SCHEMA(
         .SetDoc(ScatterND_ver11_doc)
         .Input(0, "data", "Tensor of rank r >= 1.", "T")
         .Input(1, "indices", "Tensor of rank q >= 1.", "tensor(int64)")
-        .Input(2, "updates", "Tensor of rank q + r - indices_shape[-1] - 1.", "T")
+        .Input(
+            2,
+            "updates",
+            "Tensor of rank q + r - indices_shape[-1] - 1.",
+            "T")
         .Output(0, "output", "Tensor of rank r >= 1.", "T")
         .TypeConstraint(
             "T",
@@ -1336,7 +1341,7 @@ ONNX_OPERATOR_SET_SCHEMA(
           ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
           const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
           const auto input_ndim = input_shape.dim_size();
-          for(size_t (i) = 0; i < axes.size(); ++i) {
+          for (size_t(i) = 0; i < axes.size(); ++i) {
             if (axes[i] < 0)
               axes[i] += input_ndim;
           }
@@ -1370,96 +1375,6 @@ ONNX_OPERATOR_SET_SCHEMA(
           }
         }));
 
-static const char* Pad_ver2_doc = R"DOC(
-Given `data` tensor, pads, mode, and value.
-Example:
-  Insert 0 pads to the beginning of the second dimension.
-  data = [
-      [1.0, 1.2],
-      [2.3, 3.4],
-      [4.5, 5.7],
-  ]
-  pads = [0, 2, 0, 0]
-  output = [
-      [
-          [0.0, 0.0, 1.0, 1.2],
-          [0.0, 0.0, 2.3, 3.4],
-          [0.0, 0.0, 4.5, 5.7],
-      ],
-  ]
-)DOC";
-
-ONNX_OPERATOR_SET_SCHEMA(
-    Pad,
-    2,
-    OpSchema()
-        .Attr(
-            "pads",
-            "List of integers indicating the number of padding elements to add or remove (if negative) "
-            "at the beginning and end of each axis. For 2D it is the number of pixels. "
-            "`pads` rank should be double of the input's rank. `pads` format should be as follow "
-            "[x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels "
-            "added at the beginning of axis `i` and xi_end, the number of pixels added at "
-            "the end of axis `i`.",
-            AttributeProto::INTS)
-        .Attr(
-            "mode",
-            "Three modes: constant(default), reflect, edge",
-            AttributeProto::STRING,
-            std::string("constant"))
-        .Attr(
-            "value",
-            "One float, indicates the value to be filled.",
-            AttributeProto::FLOAT,
-            0.0f)
-        .SetDoc(Pad_ver2_doc)
-        .Input(0, "data", "Input tensor.", "T")
-        .Output(0, "output", "Tensor after padding.", "T")
-        .TypeConstraint(
-            "T",
-            {"tensor(float16)", "tensor(float)", "tensor(double)"},
-            "Constrain input and output types to float tensors.")
-        .TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
-          propagateElemTypeFromInputToOutput(ctx, 0, 0);
-          if (!hasNInputShapes(ctx, 1)) {
-            return;
-          }
-
-          auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
-
-          std::vector<int64_t> pads;
-          if (!getRepeatedAttribute(ctx, "pads", pads))
-            fail_shape_inference("Attribute value for pads is required");
-          if (pads.size() != static_cast<size_t>(input_shape.dim_size() * 2)) {
-            fail_shape_inference("Attribute pads has incorrect length");
-            ;
-          }
-
-          ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
-
-          for (size_t i = 0; (int64_t)i < input_shape.dim_size(); ++i) {
-            auto* newdim = ctx.getOutputType(0)
-                               ->mutable_tensor_type()
-                               ->mutable_shape()
-                               ->add_dim();
-            if (ctx.getInputType(0)
-                    ->tensor_type()
-                    .shape()
-                    .dim((int)i)
-                    .has_dim_value()) {
-              newdim->set_dim_value(
-                  ctx.getInputType(0)
-                      ->tensor_type()
-                      .shape()
-                      .dim((int)i)
-                      .dim_value() +
-                  pads[i] + pads[input_shape.dim_size() + i]);
-            } else if (pads[i] + pads[input_shape.dim_size() + i] == 0) {
-              *newdim = input_shape.dim((int)i);
-            }
-          }
-        }));
-
 static const char* SpaceToDepth_ver1_doc =
     R"DOC(SpaceToDepth rearranges blocks of spatial data into depth. More specifically,
 this op outputs a copy of the input tensor where values from the height and width dimensions
@@ -1771,7 +1686,8 @@ ONNX_OPERATOR_SET_SCHEMA(
             Resize_attr_coordinate_transformation_mode_doc,
             AttributeProto::STRING,
             std::string("half_pixel"))
-        .Attr("nearest_mode",
+        .Attr(
+            "nearest_mode",
             "Four modes: round_prefer_floor (default, as known as round half down), round_prefer_ceil (as known as round half up), floor, ceil. Only used by nearest interpolation. It indicates how to get \"nearest\" pixel in input tensor from x_original, so this attribute is valid only if \"mode\" is \"nearest\".",
             AttributeProto::STRING,
             std::string("round_prefer_floor"))
@@ -1871,17 +1787,17 @@ ONNX_OPERATOR_SET_SCHEMA(
           propagateElemTypeFromInputToOutput(ctx, 0, 0);
           if (hasInputShape(ctx, 0)) {
             const TensorShapeProto& indices_shape =
-              ctx.getInputType(0)->tensor_type().shape();
+                ctx.getInputType(0)->tensor_type().shape();
             int r = indices_shape.dim_size();
             if (r < 1) {
               fail_shape_inference("Indices tensor must have rank >= 1");
             }
-          auto axisAttr = ctx.getAttribute("axis");
-          if (axisAttr) {
-            int axis = static_cast<int>(axisAttr->i());
+            auto axisAttr = ctx.getAttribute("axis");
+            if (axisAttr) {
+              int axis = static_cast<int>(axisAttr->i());
               if (axis < -r || axis >= r) {
                 fail_shape_inference(
-                  "'axis' must be in [-rank(indices), rank(indices)-1]");
+                    "'axis' must be in [-rank(indices), rank(indices)-1]");
               }
               if (axis < 0) {
                 axis += r;
@@ -2438,9 +2354,9 @@ ONNX_OPERATOR_SET_SCHEMA(
                 xTensorProto->tensor_type().shape();
             int rank = input_shape.dim_size();
             if (axis < 0)
-                axis += rank;
+              axis += rank;
             if (axis < 0 || axis >= rank)
-                fail_shape_inference("Invalid value for attribute axis");
+              fail_shape_inference("Invalid value for attribute axis");
 
             // 'Y' has the same shape as 'X' except in the 'axis' dimension
             // which is unknown.
@@ -2529,8 +2445,12 @@ ONNX_OPERATOR_SET_SCHEMA(
     OpSchema()
         .SetDoc(GatherND_ver11_doc)
         .Input(0, "data", "Tensor of rank r >= 1.", "T")
-        .Input(1, "indices", "Tensor of rank q >= 1. All index values are expected to be within bounds [-s, s-1] "
-            "along axis of size s. It is an error if any of the index values are out of bounds.", "tensor(int64)")
+        .Input(
+            1,
+            "indices",
+            "Tensor of rank q >= 1. All index values are expected to be within bounds [-s, s-1] "
+            "along axis of size s. It is an error if any of the index values are out of bounds.",
+            "tensor(int64)")
         .Output(
             0,
             "output",
@@ -2594,4 +2514,122 @@ ONNX_OPERATOR_SET_SCHEMA(
           }
         }));
 
-} // namespace ONNX_NAMESPACE
+static const char* Pad_ver11_doc = R"DOC(
+Given a tensor containing the data to be padded (`data`), a tensor containing the number of start and end pad values for axis (`pads`), (optionally) a `mode`, and (optionally) `value`, 
+a padded tensor (`output`) is generated.
+
+The three supported `modes` are (similar to corresponding modes supported by `numpy.pad`):
+
+1) `constant`(default) - pads with a given constant value as specified by `value` (which defaults to 0)
+
+2) `reflect` - pads with the reflection of the vector mirrored on the first and last values of the vector along each axis
+
+3) `edge` - pads with the edge values of array
+
+Example (`constant` mode):
+  Insert 0 pads to the beginning of the second dimension.
+
+  data = 
+  [
+      [1.0, 1.2],
+      [2.3, 3.4],
+      [4.5, 5.7],
+  ] 
+
+  pads = [0, 2, 0, 0]
+
+  mode = 'constant'
+
+  value = 0.0
+
+  output = 
+  [
+      [
+          [0.0, 0.0, 1.0, 1.2],
+          [0.0, 0.0, 2.3, 3.4],
+          [0.0, 0.0, 4.5, 5.7],
+      ],
+  ]
+
+)DOC";
+
+ONNX_OPERATOR_SET_SCHEMA(
+    Pad,
+    11,
+    OpSchema()
+        .Attr(
+            "mode",
+            "Supported modes: `constant`(default), `reflect`, `edge`",
+            AttributeProto::STRING,
+            std::string("constant"))
+        .SetDoc(Pad_ver11_doc)
+        .Input(0, "data", "Input tensor.", "T")
+        .Input(
+            1,
+            "pads",
+            "Tensor of integers indicating the number of padding elements to add or remove (if negative) "
+            "at the beginning and end of each axis. For 2D input tensor, it is the number of pixels. "
+            "`pads` should be a 1D tensor of shape [2 * input_rank]. "
+            "`pads` format should be: [x1_begin, x2_begin,...,x1_end, x2_end,...], "
+            "where xi_begin is the number of pad values added at the beginning of axis `i` and "
+            "xi_end, the number of pad values added at the end of axis `i`.",
+            "tensor(int64)")
+        .Input(
+            2,
+            "value",
+            "(Optional) A scalar value to be used if the mode chosen is `constant` (by default it is 0.0).",
+            "T",
+            OpSchema::Optional)
+        .Output(0, "output", "Tensor after padding.", "T")
+        .TypeConstraint(
+            "T",
+            OpSchema::all_numeric_types(),
+            "Constrains input and output to only numeric types.")
+        .TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
+          // Type inference
+          propagateElemTypeFromInputToOutput(ctx, 0, 0);
+          // Shape inference needs the input data shape
+          if (!hasNInputShapes(ctx, 1)) {
+            return;
+          }
+          const auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
+          const auto input_rank = input_shape.dim_size();
+
+          // Infer output shape if 'pads' tensor is available
+          const auto* pads_initializer = ctx.getInputData(1);
+          if (nullptr != pads_initializer) {
+            const auto& pads_shape = ctx.getInputType(1)->tensor_type().shape();
+            if (pads_initializer->dims_size() != 1 ||
+                pads_initializer->data_type() != TensorProto::INT64)
+              fail_shape_inference(
+                  "'pads' input must be a 1D (shape: [2 * input_rank]) tensor of type int64");
+
+            const auto& pads_data = ParseData<int64_t>(pads_initializer);
+
+            if (pads_data.size() != static_cast<size_t>(2 * input_rank))
+              fail_shape_inference("Pads has incorrect number of values");
+
+            auto* output_shape =
+                ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
+            for (size_t i = 0; static_cast<int64_t>(i) < input_rank; ++i) {
+              const auto& input_dim = input_shape.dim((int)i);
+              auto* output_dim = output_shape->add_dim();
+              if (input_dim.has_dim_value()) {
+                output_dim->set_dim_value(
+                    input_dim.dim_value() + pads_data[i] +
+                    pads_data[i + input_rank]);
+              } else if (pads_data[i] + pads_data[i + input_rank] == 0) {
+                *output_dim = input_dim;
+              }
+            }
+          } else {
+            // Infer output shapes' rank in any case
+            auto* output_shape_0 = getOutputShape(ctx, 0);
+            for (size_t i = 0; static_cast<int64_t>(i) < input_rank; ++i) {
+              output_shape_0->add_dim();
+            }
+          }
+          return;
+        }));
+
+} // namespace ONNX_NAMESPACE
\ No newline at end of file
diff --git a/onnx/defs/tensor/old.cc b/onnx/defs/tensor/old.cc
old mode 100755
new mode 100644
index e34eb3cc79b..cae0800cf21
--- a/onnx/defs/tensor/old.cc
+++ b/onnx/defs/tensor/old.cc
@@ -462,11 +462,11 @@ ONNX_OPERATOR_SET_SCHEMA(
           if (output_shape->dim_size() > 0) {
             if (output_shape->dim_size() != input_shape.dim_size()) {
               fail_shape_inference(
-                "Ranks inferred (",
-                input_shape.dim_size(),
-                ") is not equal to the existing rank value (",
-                output_shape->dim_size(),
-                ").");
+                  "Ranks inferred (",
+                  input_shape.dim_size(),
+                  ") is not equal to the existing rank value (",
+                  output_shape->dim_size(),
+                  ").");
             }
           } else { // Infer the rank of output anyway
             for (int i = 0; i < input_shape.dim_size(); ++i) {
@@ -478,21 +478,19 @@ ONNX_OPERATOR_SET_SCHEMA(
             // Infer output shape's dimension value if 'scales' is known.
             if (scales->type() == AttributeProto_AttributeType_FLOATS) {
               const std::vector<float> scales_data(
-                scales->floats().begin(), 
-                scales->floats().end()
-              );
-              if (scales_data.size() != static_cast<size_t>(input_shape.dim_size())) {
+                  scales->floats().begin(), scales->floats().end());
+              if (scales_data.size() !=
+                  static_cast<size_t>(input_shape.dim_size())) {
                 fail_shape_inference(
-                  "Number of elements of attribute 'scales' must be same as rank of input 'X'");
+                    "Number of elements of attribute 'scales' must be same as rank of input 'X'");
               }
-              resizeShapeInferenceHelper(input_shape, scales_data, output_shape);
+              resizeShapeInferenceHelper(
+                  input_shape, scales_data, output_shape);
             } else {
-              fail_shape_inference(
-                "Attribute 'scales' must have floats type.");
+              fail_shape_inference("Attribute 'scales' must have floats type.");
             } // scales->type() == float
           } else {
-            fail_shape_inference(
-              "Attribute 'scales' is required.");
+            fail_shape_inference("Attribute 'scales' is required.");
           } // nullptr != scales
         }));
 
@@ -526,8 +524,7 @@ ONNX_OPERATOR_SET_SCHEMA(
         .SetDoc(Upsample_ver9_doc)
         .TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
           resizeShapeInference(ctx, false);
-        }
-));
+        }));
 
 static const char* Resize_ver10_doc = R"DOC(
 Resize the input tensor.
@@ -1041,7 +1038,8 @@ ONNX_OPERATOR_SET_SCHEMA(
           }
         }));
 
-static const char* DepthToSpace_ver1_doc = R"DOC(DepthToSpace rearranges (permutes) data from depth into blocks of spatial data.
+static const char* DepthToSpace_ver1_doc =
+    R"DOC(DepthToSpace rearranges (permutes) data from depth into blocks of spatial data.
 This is the reverse transformation of SpaceToDepth. More specifically, this op outputs a copy of
 the input tensor where values from the depth dimension are moved in spatial blocks to the height
 and width dimensions.
@@ -1202,7 +1200,7 @@ ONNX_OPERATOR_SET_SCHEMA(
                                             : // i - axis < q
                     data_shape.dim(i - q + 1); // i < out_rank < q + r - 1
           }
-}));
+        }));
 
 static const char* Squeeze_ver1_doc = R"DOC(
 Remove single-dimensional entries from the shape of a tensor.
@@ -1247,20 +1245,20 @@ ONNX_OPERATOR_SET_SCHEMA(
 
           for (int i = 0, j = 0; i < input_shape.dim_size(); ++i) {
             if (static_cast<size_t>(j) < axes.size() && axes[j] == i) {
-                if(input_shape.dim(i).has_dim_value() && input_shape.dim(i).dim_value() != 1) {
-                    fail_shape_inference(
-                        "Dimension of input ",
-                        i,
-                        " must be 1 instead of ",
-                        input_shape.dim(i).dim_value());
-                }
+              if (input_shape.dim(i).has_dim_value() &&
+                  input_shape.dim(i).dim_value() != 1) {
+                fail_shape_inference(
+                    "Dimension of input ",
+                    i,
+                    " must be 1 instead of ",
+                    input_shape.dim(i).dim_value());
+              }
               ++j;
             } else {
               *ctx.getOutputType(0)
                    ->mutable_tensor_type()
                    ->mutable_shape()
-                   ->add_dim() =
-                  input_shape.dim(i);
+                   ->add_dim() = input_shape.dim(i);
             }
           }
         }));
@@ -1415,17 +1413,20 @@ ONNX_OPERATOR_SET_SCHEMA(
             fail_type_inference("OneHot node must have three inputs.");
           }
           // Input 'depth' must be a scalar or a single-element vector.
-          // TODO: Ideally to match spec for this input only allow Scalar should be allowed.
-          // Making this change now can affect backward compatibility for this op.
-          // Since this does not seem like a good justification to update version for this op,
-          // allowing both scalar and 1 element vector for now. In future when version update
-          // for this op is done we should only allow scalar or chage the spec to allow both.
+          // TODO: Ideally to match spec for this input only allow Scalar should
+          // be allowed. Making this change now can affect backward
+          // compatibility for this op. Since this does not seem like a good
+          // justification to update version for this op, allowing both scalar
+          // and 1 element vector for now. In future when version update for
+          // this op is done we should only allow scalar or chage the spec to
+          // allow both.
           if (hasInputShape(ctx, 1)) {
             auto& depth_shape = getInputShape(ctx, 1);
-            if (depth_shape.dim_size() != 0 && depth_shape.dim_size() !=1) {
-              fail_type_inference("Input 'depth' must be a scalar or rank 1 tensor.");
+            if (depth_shape.dim_size() != 0 && depth_shape.dim_size() != 1) {
+              fail_type_inference(
+                  "Input 'depth' must be a scalar or rank 1 tensor.");
             }
-            if (depth_shape.dim_size() ==1 &&
+            if (depth_shape.dim_size() == 1 &&
                 depth_shape.dim((int)0).has_dim_value() &&
                 depth_shape.dim((int)0).dim_value() != 1) {
               fail_type_inference(
@@ -1597,4 +1598,94 @@ ONNX_OPERATOR_SET_SCHEMA(
           }
         }));
 
-} // namespace ONNX_NAMESPACE
+static const char* Pad_ver2_doc = R"DOC(
+Given `data` tensor, pads, mode, and value.
+Example:
+  Insert 0 pads to the beginning of the second dimension.
+  data = [
+      [1.0, 1.2],
+      [2.3, 3.4],
+      [4.5, 5.7],
+  ]
+  pads = [0, 2, 0, 0]
+  output = [
+      [
+          [0.0, 0.0, 1.0, 1.2],
+          [0.0, 0.0, 2.3, 3.4],
+          [0.0, 0.0, 4.5, 5.7],
+      ],
+  ]
+)DOC";
+
+ONNX_OPERATOR_SET_SCHEMA(
+    Pad,
+    2,
+    OpSchema()
+        .Attr(
+            "pads",
+            "List of integers indicating the number of padding elements to add or remove (if negative) "
+            "at the beginning and end of each axis. For 2D it is the number of pixels. "
+            "`pads` rank should be double of the input's rank. `pads` format should be as follow "
+            "[x1_begin, x2_begin...x1_end, x2_end,...], where xi_begin the number of pixels "
+            "added at the beginning of axis `i` and xi_end, the number of pixels added at "
+            "the end of axis `i`.",
+            AttributeProto::INTS)
+        .Attr(
+            "mode",
+            "Three modes: constant(default), reflect, edge",
+            AttributeProto::STRING,
+            std::string("constant"))
+        .Attr(
+            "value",
+            "One float, indicates the value to be filled.",
+            AttributeProto::FLOAT,
+            0.0f)
+        .SetDoc(Pad_ver2_doc)
+        .Input(0, "data", "Input tensor.", "T")
+        .Output(0, "output", "Tensor after padding.", "T")
+        .TypeConstraint(
+            "T",
+            {"tensor(float16)", "tensor(float)", "tensor(double)"},
+            "Constrain input and output types to float tensors.")
+        .TypeAndShapeInferenceFunction([](InferenceContext& ctx) {
+          propagateElemTypeFromInputToOutput(ctx, 0, 0);
+          if (!hasNInputShapes(ctx, 1)) {
+            return;
+          }
+
+          auto& input_shape = ctx.getInputType(0)->tensor_type().shape();
+
+          std::vector<int64_t> pads;
+          if (!getRepeatedAttribute(ctx, "pads", pads))
+            fail_shape_inference("Attribute value for pads is required");
+          if (pads.size() != static_cast<size_t>(input_shape.dim_size() * 2)) {
+            fail_shape_inference("Attribute pads has incorrect length");
+            ;
+          }
+
+          ctx.getOutputType(0)->mutable_tensor_type()->mutable_shape();
+
+          for (size_t i = 0; (int64_t)i < input_shape.dim_size(); ++i) {
+            auto* newdim = ctx.getOutputType(0)
+                               ->mutable_tensor_type()
+                               ->mutable_shape()
+                               ->add_dim();
+            if (ctx.getInputType(0)
+                    ->tensor_type()
+                    .shape()
+                    .dim((int)i)
+                    .has_dim_value()) {
+              newdim->set_dim_value(
+                  ctx.getInputType(0)
+                      ->tensor_type()
+                      .shape()
+                      .dim((int)i)
+                      .dim_value() +
+                  pads[i] + pads[input_shape.dim_size() + i]);
+            } else if (pads[i] + pads[input_shape.dim_size() + i] == 0) {
+              *newdim = input_shape.dim((int)i);
+            }
+          }
+        }));
+
+} // namespace ONNX_NAMESPACE
\ No newline at end of file
diff --git a/onnx/defs/tensor_proto_util.cc b/onnx/defs/tensor_proto_util.cc
index 5c0fc8b74e3..f6b366f0c34 100644
--- a/onnx/defs/tensor_proto_util.cc
+++ b/onnx/defs/tensor_proto_util.cc
@@ -3,6 +3,7 @@
 
 #include "tensor_proto_util.h"
 #include <vector>
+#include "onnx/common/platform_helpers.h"
 
 namespace ONNX_NAMESPACE {
 
@@ -27,13 +28,6 @@ namespace ONNX_NAMESPACE {
     return t;                                                   \
   }
 
-inline bool is_platform_little_endian() {
-  int num = 1;
-  if (*(char*)&num == 1)
-    return true;
-  return false;
-}
-
 #define DEFINE_PARSE_DATA(type, typed_data_fetch)                          \
   template <>                                                              \
   const std::vector<type> ParseData(const TensorProto* tensor_proto) {     \
@@ -48,7 +42,7 @@ inline bool is_platform_little_endian() {
     /* okay to remove const qualifier as we have already made a copy */    \
     char* bytes = const_cast<char*>(raw_data.c_str());                     \
     /*onnx is little endian serialized always-tweak byte order if needed*/ \
-    if (!is_platform_little_endian()) {                                    \
+    if (!is_processor_little_endian()) {                                   \
       const size_t element_size = sizeof(type);                            \
       const size_t num_elements = raw_data.size() / element_size;          \
       for (size_t i = 0; i < num_elements; ++i) {                          \
diff --git a/onnx/defs/tensor_util.cc b/onnx/defs/tensor_util.cc
new file mode 100644
index 00000000000..8cb15c8ab66
--- /dev/null
+++ b/onnx/defs/tensor_util.cc
@@ -0,0 +1,52 @@
+// Copyright (c) ONNX Project Contributors.
+// Licensed under the MIT license.
+
+#include "tensor_util.h"
+#include <vector>
+#include "onnx/common/platform_helpers.h"
+
+namespace ONNX_NAMESPACE {
+
+#define DEFINE_PARSE_DATA(type, typed_data_fetch)                          \
+  template <>                                                              \
+  const std::vector<type> ParseData(const Tensor* tensor) {                \
+    std::vector<type> res;                                                 \
+    if (!tensor->is_raw_data()) {                                          \
+      const auto& data = tensor->typed_data_fetch();                       \
+      res.insert(res.end(), data.begin(), data.end());                     \
+      return res;                                                          \
+    }                                                                      \
+    /* make copy as we may have to reverse bytes */                        \
+    std::string raw_data = tensor->raw();                                  \
+    /* okay to remove const qualifier as we have already made a copy */    \
+    char* bytes = const_cast<char*>(raw_data.c_str());                     \
+    /*onnx is little endian serialized always-tweak byte order if needed*/ \
+    if (!is_processor_little_endian()) {                                   \
+      const size_t element_size = sizeof(type);                            \
+      const size_t num_elements = raw_data.size() / element_size;          \
+      for (size_t i = 0; i < num_elements; ++i) {                          \
+        char* start_byte = bytes + i * element_size;                       \
+        char* end_byte = start_byte + element_size - 1;                    \
+        /* keep swapping */                                                \
+        for (size_t count = 0; count < element_size / 2; ++count) {        \
+          char temp = *start_byte;                                         \
+          *start_byte = *end_byte;                                         \
+          *end_byte = temp;                                                \
+          ++start_byte;                                                    \
+          --end_byte;                                                      \
+        }                                                                  \
+      }                                                                    \
+    }                                                                      \
+    res.insert(                                                            \
+        res.end(),                                                         \
+        reinterpret_cast<const type*>(bytes),                              \
+        reinterpret_cast<const type*>(bytes + raw_data.size()));           \
+    return res;                                                            \
+  }
+
+DEFINE_PARSE_DATA(int32_t, int32s)
+DEFINE_PARSE_DATA(int64_t, int64s)
+DEFINE_PARSE_DATA(float, floats)
+DEFINE_PARSE_DATA(double, doubles)
+
+} // namespace ONNX_NAMESPACE
diff --git a/onnx/defs/tensor_util.h b/onnx/defs/tensor_util.h
new file mode 100644
index 00000000000..9c535fd0f61
--- /dev/null
+++ b/onnx/defs/tensor_util.h
@@ -0,0 +1,13 @@
+// Copyright (c) ONNX Project Contributors.
+// Licensed under the MIT license.
+
+#pragma once
+
+#include "onnx/common/ir.h"
+
+namespace ONNX_NAMESPACE {
+
+template <typename T>
+const std::vector<T> ParseData(const Tensor* tensor);
+
+} // namespace ONNX_NAMESPACE
diff --git a/onnx/examples/make_model.ipynb b/onnx/examples/make_model.ipynb
index d1bde4b9178..5517aaec2d4 100644
--- a/onnx/examples/make_model.ipynb
+++ b/onnx/examples/make_model.ipynb
@@ -16,6 +16,7 @@
       "The graph in model:\n",
       "node {\n",
       "  input: \"X\"\n",
+      "  input: \"Pads\"\n",    
       "  output: \"Y\"\n",
       "  op_type: \"Pad\"\n",
       "  attribute {\n",
@@ -23,21 +24,17 @@
       "    s: \"constant\"\n",
       "    type: STRING\n",
       "  }\n",
-      "  attribute {\n",
-      "    name: \"pads\"\n",
-      "    ints: 0\n",
-      "    ints: 1\n",
-      "    ints: 0\n",
-      "    ints: 1\n",
-      "    type: INTS\n",
-      "  }\n",
-      "  attribute {\n",
-      "    name: \"value\"\n",
-      "    f: 1.5\n",
-      "    type: FLOAT\n",
-      "  }\n",
       "}\n",
       "name: \"test-model\"\n",
+      "initializer {\n",
+      "  dims: 4\n", 
+      "  data_type: 7\n",
+      "  int64_data: 0\n",
+      "  int64_data: 0\n",
+      "  int64_data: 1\n",
+      "  int64_data: 1\n",
+      "  name: \"Pads\"\n",
+      "}\n",      
       "input {\n",
       "  name: \"X\"\n",
       "  type {\n",
@@ -54,6 +51,19 @@
       "    }\n",
       "  }\n",
       "}\n",
+      "input {\n",
+      "  name: \"Pads\"\n",
+      "  type {\n",
+      "    tensor_type {\n",
+      "      elem_type: 7\n",
+      "      shape {\n",
+      "        dim {\n",
+      "          dim_value: 4\n",
+      "        }\n",
+      "      }\n",
+      "    }\n",
+      "  }\n",
+      "}\n",  
       "output {\n",
       "  name: \"Y\"\n",
       "  type {\n",
@@ -93,25 +103,27 @@
     "# Create one input (ValueInfoProto)\n",
     "X = helper.make_tensor_value_info('X', TensorProto.FLOAT, [1, 2])\n",
     "\n",
+    "# Create second input (ValueInfoProto)\n",
+    "Pads = helper.make_tensor_value_info('Pads', TensorProto.INT64, [4])\n",
+    "\n",  
     "# Create one output (ValueInfoProto)\n",
     "Y = helper.make_tensor_value_info('Y', TensorProto.FLOAT, [1, 4])\n",
     "\n",
     "# Create a node (NodeProto)\n",
     "node_def = helper.make_node(\n",
     "    'Pad', # node name\n",
-    "    ['X'], # inputs\n",
+    "    ['X', 'Pads'], # inputs\n",
     "    ['Y'], # outputs\n",
     "    mode='constant', # Attributes\n",
-    "    value=1.5,\n",
-    "    pads=[0, 1, 0, 1],\n",
     ")\n",
     "\n",
     "# Create the graph (GraphProto)\n",
     "graph_def = helper.make_graph(\n",
     "    [node_def],\n",
     "    \"test-model\",\n",
-    "    [X],\n",
+    "    [X, Pads],\n",
     "    [Y],\n",
+    "    [helper.make_tensor('Pads', TensorProto.INT64, [4,], [0, 0, 1, 1,])],\n",
     ")\n",
     "\n",
     "# Create the model (ModelProto)\n",
diff --git a/onnx/optimizer/passes/eliminate_nop_pad.h b/onnx/optimizer/passes/eliminate_nop_pad.h
index caafb08ac94..dc85bac697b 100644
--- a/onnx/optimizer/passes/eliminate_nop_pad.h
+++ b/onnx/optimizer/passes/eliminate_nop_pad.h
@@ -3,6 +3,7 @@
 
 #pragma once
 
+#include "onnx/defs/tensor_util.h"
 #include "onnx/optimizer/pass.h"
 
 namespace ONNX_NAMESPACE {
@@ -18,20 +19,47 @@ struct EliminateNopPad final : public PredicateBasedPass {
   std::string getPassName() const override {
     return "eliminate_nop_pad";
   }
-  static bool is_nop_pad(const std::vector<int64_t>& pads) {
-    for (size_t i = 0; i < pads.size(); i++)
-      if (pads[i] > 0)
+
+  static bool is_nop_pad(Node* node, Graph& graph) {
+    if (node->hasAttribute(kpads)) {
+      // opset 10 and below
+      const auto& pads = node->is(kpads);
+      for (size_t i = 0; i < pads.size(); i++)
+        if (pads[i] > 0)
+          return false;
+      return true;
+    } else {
+      // opset 11 and above
+      const auto pads_name = node->inputs()[1]->uniqueName();
+      const auto pads_initializer = graph.getInitializer(pads_name);
+      // 'pad' node has the 'pads' input which has not been initialized -
+      // can't proceed with elimination
+      if (pads_initializer == graph.initializers().end())
         return false;
-    return true;
+
+      // validate values within 'pads'
+      if (pads_initializer->elem_type() == TensorProto::INT64) {
+        const auto& pads = ParseData<int64_t>(&*pads_initializer);
+        for (const auto& val : pads) {
+          if (val > 0)
+            return false;
+        }
+        return true;
+      }
+    }
+
+    return false;
   }
 
   bool patternMatchPredicate(Node* node) override {
-    return (node->kind() == kPad && node->hasAttribute(kpads)) &&
-        is_nop_pad(node->is(kpads));
+    return node->kind() == kPad;
   }
-  bool runTransform(Node* node, Graph&, NodeDestroyType& destroy_current)
+
+  bool runTransform(Node* node, Graph& graph, NodeDestroyType& destroy_current)
       override {
-    node->output()->replaceAllUsesWith(node->input());
+    if (!is_nop_pad(node, graph))
+      return false;
+    node->output()->replaceAllUsesWith(node->inputs()[0]);
     destroy_current = NodeDestroyType::DestroyOne;
     return true;
   }
diff --git a/onnx/optimizer/passes/fuse_pad_into_conv.h b/onnx/optimizer/passes/fuse_pad_into_conv.h
index 19344acc179..ea2485cdc2e 100644
--- a/onnx/optimizer/passes/fuse_pad_into_conv.h
+++ b/onnx/optimizer/passes/fuse_pad_into_conv.h
@@ -4,17 +4,17 @@
 #pragma once
 
 // Before:
-//   P = Pad(X)
-//   Z = Conv(P, Y)
+//   P = Pad(X) - opset 10 and below (or) Pad(X, Pads, [Value]) - opset 11 and
+//   above Z = Conv(P, Y)
 // After:
 //   Z = Conv(X, Y) with "pads" attribute set
 //
 // the pass handles the case when Pad is zero-padding the input
-// (i.e. mode=constant and value=0)
+// (i.e. mode=constant and Value=0)
 
 #include <numeric>
 
-#include "onnx/common/assertions.h"
+#include "onnx/defs/tensor_util.h"
 #include "onnx/optimizer/pass.h"
 
 namespace ONNX_NAMESPACE {
@@ -32,46 +32,102 @@ struct FusePadIntoConv final : public PredicateBasedPass {
   bool patternMatchPredicate(Node* node) override {
     return node->kind() == kConv && node->inputs()[0]->node()->kind() == kPad;
   }
-  bool runTransform(Node* n, Graph& /*graph*/, NodeDestroyType& destroy_current)
+  bool runTransform(Node* n, Graph& graph, NodeDestroyType& destroy_current)
       override {
     destroy_current = NodeDestroyType::DestroyZero;
 
     // check if Pad is only used by Conv
     if (n->inputs()[0]->uses().size() > 1) {
-        return false;
+      return false;
     }
 
     Node* conv = n;
     Node* pad = n->inputs()[0]->node();
 
+    // Process 'pads' data
+    std::vector<int64_t> pads;
+    if (pad->hasAttribute(kpads)) {
+      // opset 10 and below
+      pads = pad->is(kpads);
+    } else {
+      // opset 11 and above - first check if 'pad' node has 'pads' input
+      // initialized
+      const auto pads_name = pad->inputs()[1]->uniqueName();
+      const auto pads_initializer = graph.getInitializer(pads_name);
+      // 'pad' node has the 'pads' input which has not been initialized -
+      // can't proceed with fusing
+      if (pads_initializer == graph.initializers().end()) {
+        return false;
+      }
+
+      // make sure the type of 'pads' is INT64
+      if (pads_initializer->elem_type() != TensorProto::INT64) {
+        return false;
+      }
+
+      // parse 'pads' data from the initialized input
+      pads = ParseData<int64_t>(&*pads_initializer);
+    }
+
+    // Process 'mode'
     std::string pad_mode;
     if (pad->hasAttribute(kmode)) {
       pad_mode = pad->s(kmode);
     } else {
       pad_mode = "constant";
     }
-    float value = 0.0;
+
+    // cannot fuse if the pad mode is not "Constant"
+    if (pad_mode != "constant") {
+      return false;
+    }
+
+    // Process 'value'
+    double value = 0.0;
     if (pad->hasAttribute(kvalue)) {
-      value = static_cast<float>(pad->f(kvalue));
+      // opset 10 and below
+      value = static_cast<double>(pad->f(kvalue));
+    } else if (pad->inputs().size() == 3) {
+      // opset 11 and above - check if the 'pad' node has the optional 'value'
+      // input check if it has data initialized
+      const auto value_name = pad->inputs()[2]->uniqueName();
+      const auto value_initializer = graph.getInitializer(value_name);
+
+      // 'pad' node has the 'value' input which has not been initialized -
+      // can't proceed with fusing
+      if (value_initializer == graph.initializers().end())
+        return false;
+
+      // parse 'value' data from the initialized input
+      if (value_initializer->elem_type() == TensorProto::FLOAT) {
+        const auto& data = ParseData<float>(&*value_initializer);
+        value = static_cast<double>(data[0]);
+      } else if (value_initializer->elem_type() == TensorProto::DOUBLE) {
+        const auto& data = ParseData<double>(&*value_initializer);
+        value = data[0];
+      } else {
+        // either float16 or not relevant data type for this input - no fusing
+        return false;
+      }
     }
 
     // check if Pad is used to zero-pad the input
-    if (pad_mode != "constant" || value != 0.0) {
+    if (value != 0.0) {
       return false;
     }
 
-    std::vector<int64_t> pads = pad->is(kpads);
     int pads_size = static_cast<int>(pads.size());
 
     // check if padding is applied only on feature dims
-    if (pads[0] != 0 || pads[1] != 0 ||
-        pads[pads_size / 2] != 0 || pads[pads_size / 2 + 1] != 0) {
+    if (pads[0] != 0 || pads[1] != 0 || pads[pads_size / 2] != 0 ||
+        pads[pads_size / 2 + 1] != 0) {
       return false;
     }
 
     // check if padding is only positive
-    if (std::any_of(pads.begin(), pads.end(),
-        [](int64_t local_value) { return local_value < 0; })) {
+    if (std::any_of(pads.begin(), pads.end(), [](int64_t local_value) {
+          return local_value < 0;
+        })) {
       return false;
     }
 
diff --git a/onnx/test/optimizer_test.py b/onnx/test/optimizer_test.py
index 7cfac2f5925..a4aa12421c1 100644
--- a/onnx/test/optimizer_test.py
+++ b/onnx/test/optimizer_test.py
@@ -177,12 +177,17 @@ def test_nop_transpose_default(self):  # type: () -> None
         assert optimized_model.graph.node[0].op_type == "Transpose"
 
     def test_nop_pad(self):  # type: () -> None
-        nodes = [helper.make_node("Pad", ["X"], ["Y"], pads=[0, 0])]
+        nodes = [helper.make_node("Pad", ["X", "Pads"], ["Y"])]
         graph = helper.make_graph(
             nodes,
             "test",
-            [helper.make_tensor_value_info("X", TensorProto.FLOAT, (2, 3))],
-            [helper.make_tensor_value_info("Y", TensorProto.FLOAT, (2, 3))])
+            [helper.make_tensor_value_info("X", TensorProto.FLOAT, (2, 3)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (4,))],
+            [helper.make_tensor_value_info("Y", TensorProto.FLOAT, (2, 3))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+                dims=(4,),
+                vals=np.array([0, 0, 0, 0]).astype(np.int64).tobytes(),
+                raw=True)])
         assert len(graph.node) == 1
         optimized_model = self._optimized(graph, ["eliminate_nop_pad"])
 
@@ -194,12 +199,17 @@ def check_pad(node):  # type: (NodeProto) -> None
         assert len(optimized_model.graph.node) == 0
 
     def test_nop_pad_default(self):  # type: () -> None
-        trans = helper.make_node("Pad", ["X"], ["Y"], pads=[0, 1])
+        trans = helper.make_node("Pad", ["X", "Pads"], ["Y"])
         graph = helper.make_graph(
             [trans],
             "test",
-            [helper.make_tensor_value_info("X", TensorProto.FLOAT, (2, 3))],
-            [helper.make_tensor_value_info("Y", TensorProto.FLOAT, (2, 4))])
+            [helper.make_tensor_value_info("X", TensorProto.FLOAT, (2, 3)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (4,))],
+            [helper.make_tensor_value_info("Y", TensorProto.FLOAT, (2, 4))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+                dims=(4,),
+                vals=np.array([0, 1, 0, 0]).astype(np.int64).tobytes(),
+                raw=True)])
         optimized_model = self._optimized(graph, ["eliminate_nop_pad"])
 
         assert len(list(optimized_model.graph.node)) == 1
@@ -766,19 +776,22 @@ def test_fuse_matmul_add_bias_into_gemm_multiple_use_no_fuse(self):  # type: ()
     def test_fuse_pad_into_conv(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="constant",
-            pads=[0, 0, 0, 0, 0, 0, 1, 1]
+            mode="constant"
         )
         conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
         graph = helper.make_graph(
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 2, 2)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (8,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 3, 3))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(8,),
+             vals=np.array([0, 0, 0, 0, 0, 0, 1, 1]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert len(list(optimized_model.graph.node)) == 1
@@ -786,22 +799,25 @@ def test_fuse_pad_into_conv(self):  # type: () -> None
         assert optimized_model.graph.node[0].attribute[0].name == "pads"
         assert list(optimized_model.graph.node[0].attribute[0].ints) == [0, 0, 1, 1]
 
-    def test_fuse_pad_into_con_1d(self):  # type: () -> None
+    def test_fuse_pad_into_conv_1d(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="constant",
-            pads=[0, 0, 1, 0, 0, 1]
+            mode="constant"
         )
         conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
         graph = helper.make_graph(
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 30)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (6,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 32))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(6,),
+             vals=np.array([0, 0, 1, 0, 0, 1]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert len(list(optimized_model.graph.node)) == 1
@@ -812,10 +828,9 @@ def test_fuse_pad_into_con_1d(self):  # type: () -> None
     def test_fuse_pad_into_conv_existing_conv_pad(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="constant",
-            pads=[0, 0, 0, 0, 0, 0, 1, 1]
+            mode="constant"
         )
         conv = helper.make_node(
             "Conv",
@@ -827,9 +842,13 @@ def test_fuse_pad_into_conv_existing_conv_pad(self):  # type: () -> None
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 2, 2)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (8,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 4, 4))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(8,),
+             vals=np.array([0, 0, 0, 0, 0, 0, 1, 1]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert len(list(optimized_model.graph.node)) == 1
@@ -840,19 +859,22 @@ def test_fuse_pad_into_conv_existing_conv_pad(self):  # type: () -> None
     def test_fuse_pad_into_conv_pad_feature_no_fuse(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="constant",
-            pads=[0, 1, 0, 0, 0, 0, 0, 0]
+            mode="constant"
         )
         conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
         graph = helper.make_graph(
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 4, 3, 3)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (8,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 3, 3))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(8,),
+             vals=np.array([0, 1, 0, 0, 0, 0, 0, 0]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert optimized_model.graph == graph
@@ -860,19 +882,22 @@ def test_fuse_pad_into_conv_pad_feature_no_fuse(self):  # type: () -> None
     def test_fuse_pad_into_conv_negative_pad_no_fuse(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="constant",
-            pads=[0, 0, 0, 0, 0, 0, -1, -1]
+            mode="constant"
         )
         conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
         graph = helper.make_graph(
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 4, 4)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (8,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 3, 3))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(8,),
+             vals=np.array([0, 0, 0, 0, 0, 0, -1, -1]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert optimized_model.graph == graph
@@ -880,19 +905,22 @@ def test_fuse_pad_into_conv_negative_pad_no_fuse(self):  # type: () -> None
     def test_fuse_pad_into_conv_reflection_pad_no_fuse(self):  # type: () -> None
         pad = helper.make_node(
             "Pad",
-            ["X"],
+            ["X", "Pads"],
             ["P"],
-            mode="reflect",
-            pads=[0, 0, 0, 0, 0, 0, 1, 1]
+            mode="reflect"
         )
         conv = helper.make_node("Conv", ["P", "Y"], ["Z"])
         graph = helper.make_graph(
             [pad, conv],
             "test",
             [helper.make_tensor_value_info("X", TensorProto.FLOAT, (1, 5, 2, 2)),
+             helper.make_tensor_value_info("Pads", TensorProto.INT64, (8,)),
              helper.make_tensor_value_info("Y", TensorProto.FLOAT, (16, 5, 3, 3))],
-            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))]
-        )
+            [helper.make_tensor_value_info("Z", TensorProto.FLOAT, (1, 16, 1, 1))],
+            [helper.make_tensor("Pads", TensorProto.INT64,
+             dims=(8,),
+             vals=np.array([0, 0, 0, 0, 0, 0, 1, 1]).astype(np.int64).tobytes(),
+             raw=True)])
         optimized_model = self._optimized(graph, ["fuse_pad_into_conv"])
 
         assert optimized_model.graph == graph
diff --git a/onnx/test/shape_inference_test.py b/onnx/test/shape_inference_test.py
index 014b27f4852..50d4c78702c 100644
--- a/onnx/test/shape_inference_test.py
+++ b/onnx/test/shape_inference_test.py
@@ -615,20 +615,6 @@ def test_slice_variable_input_types(self):  # type: () -> None
                          make_tensor('axes', TensorProto.INT32, (2,), (0, 1))])
         self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.DOUBLE, (2, 2))])
 
-    def test_pad(self):  # type: () -> None
-        graph = self._make_graph(
-            [('x', TensorProto.FLOAT, (1, None, 2))],
-            [make_node('Pad', 'x', 'y', pads=[1, 3, 1, 1, 0, 1])],
-            [])
-        self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.FLOAT, (3, None, 4))])  # type: ignore
-
-    def test_constant_pad_2d(self):  # type: () -> None
-        graph = self._make_graph(
-            [('x', TensorProto.FLOAT, (2, 3, 4, 4))],
-            [make_node('Pad', 'x', 'y', pads=[0, 0, 3, 1, 0, 0, 4, 2], mode="constant", value=2.0)],
-            [])
-        self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.FLOAT, (2, 3, 11, 7))])
-
     def test_conv(self):  # type: () -> None
         graph = self._make_graph(
             [('x', TensorProto.FLOAT, (3, 4, 5, 6, 7)),
@@ -2202,6 +2188,15 @@ def test_cumsum(self):  # type: () -> None
             [])
         self._assert_inferred(graph, [make_tensor_value_info('z', TensorProto.FLOAT, (2, 3))])
 
+    def test_pad(self):  # type: () -> None
+        graph = self._make_graph(
+            [('x', TensorProto.FLOAT, (1, None, 2)),
+             ('pads', TensorProto.INT64, (6,))],
+            [make_node('Pad', ['x', 'pads'], 'y')],
+            [],
+            initializer=[make_tensor('pads', TensorProto.INT64, (6,), (1, 3, 1, 1, 0, 1,))])
+        self._assert_inferred(graph, [make_tensor_value_info('y', TensorProto.FLOAT, (3, None, 4))])  # type: ignore
+
 
 if __name__ == '__main__':
     unittest.main()


Code review comments:
diff_hunk: @@ -10026,19 +10046,19 @@ expect(node, inputs=[x], outputs=[y],
 for mode in ['edge', 'reflect']:
     node = onnx.helper.make_node(
         'Pad',
-        inputs=['x'],
+        inputs=['x', 'pads'],
         outputs=['y'],
-        mode=mode,
-        pads=[0, 0, 1, 1, 0, 0, 1, 1]
+        mode=mode
     )
-    x = np.random.randn(1, 3, 4, 5).astype(np.float32)
-    y = np.pad(
+    x = np.random.randn(1, 3, 4, 5).astype(np.int32)
+    pads = np.array([0, 0, 1, 1, 0, 0, 1, 1]).astype(np.int64)  # pad order [x1_begin, x2_begin, ..., x1_end, x2_end, ...]
+    y = pad_impl(
         x,
-        pad_width=((0, 0), (0, 0), (1, 1), (1, 1)),
-        mode=mode,
+        ((0, 0), (0, 0), (1, 1), (1, 1)),  # re-order to np.pad accepted order ((x1_begin, x1_end), (x2_begin, x2_end), ...)
path: docs/Operators.md
text: @user1:
`pad_impl` is a reference implementation of ONNX `Pad`. It should accept ONNX `pads` directly. We need to modify `pad_impl` to consume `pads` in ONNX's format.

@author:
Sure. Will do.

@author:
Fixed.