Repository: apple/coremltools

Issue:
Todays' beta2 checkin breaks  old  issue #759  F.max_pool2d where stride is NOT explicitly specified
Am comparing the old test case testPool from issue #759 with today's git version
commit 37e619d99bf603d2cb9ea0839fa3ebe649996b0a (HEAD -> master, tag: 4.0b2, origin/master, origin/HEAD)

versus from a few days ago July 22
commit 705244e2be26c3fb7881fd7a731d25a55f5e4765

test case from issue #759 

now fails again.
(I cannot attach it atm. Seems AWS issue)

reproducible:---------------

yes

log:---------------

Torch version : 1.5.1
CoreML tools version : 4.0b2
Converting Frontend ==> MIL Ops:   0%|                                                                                                                                                                                                                                                             | 0/6 [00:00<?, ? ops/s]Converting op 13 : constant : shape = n/a
Converting op 14 : constant : shape = n/a
Converting op 15 : constant : shape = n/a
Converting op 16 : constant : shape = n/a
Converting op 11 : constant : shape = n/a
Converting op 12 : max_pool2d : shape = (1, 1, 28, 28)
Converting Frontend ==> MIL Ops:  83%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                        | 5/6 [00:00<00:00, 4629.47 ops/s]
Traceback (most recent call last):
  File "testPool.py", line 94, in <module>
    inputs=[ ct.TensorType(name="input1", shape=dummy_input.shape) ],
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/_converters_entry.py", line 299, in convert
    **kwargs
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/converter.py", line 120, in _convert
    prog = frontend_converter(model, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/converter.py", line 62, in __call__
    return load(*args, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 86, in load
    raise e
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 76, in load
    prog = converter.convert()
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 221, in convert
    convert_nodes(self.context, self.graph)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 64, in convert_nodes
    _add_op(context, node)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 584, in max_pool2d
    name=node.name,
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
    return cls._add_op(op_cls, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/builder.py", line 188, in _add_op
    new_op = op_cls(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/pool.py", line 180, in __init__
    super(max_pool, self).__init__(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/pool.py", line 25, in __init__
    super(Pooling, self).__init__(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 148, in __init__
    self._validate_and_set_inputs(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 360, in _validate_and_set_inputs
    parsed_inputs = self.input_spec.parse_inputs(kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/input_type.py", line 67, in parse_inputs
    raise TypeError(msg)
TypeError: Input strides has type <class 'coremltools.converters.mil.mil.types.type_tensor.tensor.<locals>.tensor'> not compatible with expected type IntTensorInputType
Todays' beta2 checkin breaks  old  issue #759  F.max_pool2d where stride is NOT explicitly specified
Am comparing the old test case testPool from issue #759 with today's git version
commit 37e619d99bf603d2cb9ea0839fa3ebe649996b0a (HEAD -> master, tag: 4.0b2, origin/master, origin/HEAD)

versus from a few days ago July 22
commit 705244e2be26c3fb7881fd7a731d25a55f5e4765

test case from issue #759 

now fails again.
(I cannot attach it atm. Seems AWS issue)

reproducible:---------------

yes

log:---------------

Torch version : 1.5.1
CoreML tools version : 4.0b2
Converting Frontend ==> MIL Ops:   0%|                                                                                                                                                                                                                                                             | 0/6 [00:00<?, ? ops/s]Converting op 13 : constant : shape = n/a
Converting op 14 : constant : shape = n/a
Converting op 15 : constant : shape = n/a
Converting op 16 : constant : shape = n/a
Converting op 11 : constant : shape = n/a
Converting op 12 : max_pool2d : shape = (1, 1, 28, 28)
Converting Frontend ==> MIL Ops:  83%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                        | 5/6 [00:00<00:00, 4629.47 ops/s]
Traceback (most recent call last):
  File "testPool.py", line 94, in <module>
    inputs=[ ct.TensorType(name="input1", shape=dummy_input.shape) ],
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/_converters_entry.py", line 299, in convert
    **kwargs
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/converter.py", line 120, in _convert
    prog = frontend_converter(model, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/converter.py", line 62, in __call__
    return load(*args, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 86, in load
    raise e
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/load.py", line 76, in load
    prog = converter.convert()
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/converter.py", line 221, in convert
    convert_nodes(self.context, self.graph)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 64, in convert_nodes
    _add_op(context, node)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/frontend/torch/ops.py", line 584, in max_pool2d
    name=node.name,
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/registry.py", line 62, in add_op
    return cls._add_op(op_cls, **kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/builder.py", line 188, in _add_op
    new_op = op_cls(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/pool.py", line 180, in __init__
    super(max_pool, self).__init__(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/ops/defs/pool.py", line 25, in __init__
    super(Pooling, self).__init__(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 148, in __init__
    self._validate_and_set_inputs(**kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/operation.py", line 360, in _validate_and_set_inputs
    parsed_inputs = self.input_spec.parse_inputs(kwargs)
  File "~/Library/Python/3.7/lib/python/site-packages/coremltools/converters/mil/mil/input_type.py", line 67, in parse_inputs
    raise TypeError(msg)
TypeError: Input strides has type <class 'coremltools.converters.mil.mil.types.type_tensor.tensor.<locals>.tensor'> not compatible with expected type IntTensorInputType


Discussion:
As AWS does not allow me to attach a test case, here is the code to verify with 4.0b2 vs previous

import torch
import torch.nn as nn
import torch.nn.functional as F
import coremltools as ct

class small_model(nn.Module):
    def __init__(self):
        super(small_model, self).__init__()
        
        self.pool1  = nn.MaxPool2d(2,stride=1) # Pool 2x2, explicit stride
        self.pool1b = nn.MaxPool2d(2) # Pool 2x2, implicit stride
        
    def forward(self, x):
        
        # Functional pooling
        y = F.max_pool2d(x, 2) # BOOM, BUG,  implicit stride=1
        #y = F.max_pool2d(x, 2, stride=1) # OK, explicit stride=1
        
        # Layer-based pooling
        #y = self.pool1(x)  # OK
        #y = self.pool1b(x) # OK

        return y

if __name__ == '__main__':
    print ("Torch version : " + str(torch.__version__))
    print ("CoreML tools version : " + str(ct.__version__))

    model = small_model()
    model.eval()
    dummy_input = torch.randn(1,1,28,28)
    traced_model = torch.jit.trace(model, dummy_input)
    model = ct.convert(
        traced_model,
        inputs=[ ct.TensorType(name="input1", shape=dummy_input.shape) ],
    )

Maybe a tentative fix and put on a branch. Will push shortly for review.
In the meantime, new test case 
[testPool.txt](https://github.com/apple/coremltools/files/5017284/testPool.txt)

As I have difficulty with git pushing the branch (error 403), here is the tentative patch for review

[patch.txt](https://github.com/apple/coremltools/files/5017298/patch.txt)

PS: To be clear, I cannot generate the PR as I cannot push my branch at the moment. As the patch has only 4 lines of code, maybe someone else can carry it forward. Thx.
As AWS does not allow me to attach a test case, here is the code to verify with 4.0b2 vs previous

import torch
import torch.nn as nn
import torch.nn.functional as F
import coremltools as ct

class small_model(nn.Module):
    def __init__(self):
        super(small_model, self).__init__()
        
        self.pool1  = nn.MaxPool2d(2,stride=1) # Pool 2x2, explicit stride
        self.pool1b = nn.MaxPool2d(2) # Pool 2x2, implicit stride
        
    def forward(self, x):
        
        # Functional pooling
        y = F.max_pool2d(x, 2) # BOOM, BUG,  implicit stride=1
        #y = F.max_pool2d(x, 2, stride=1) # OK, explicit stride=1
        
        # Layer-based pooling
        #y = self.pool1(x)  # OK
        #y = self.pool1b(x) # OK

        return y

if __name__ == '__main__':
    print ("Torch version : " + str(torch.__version__))
    print ("CoreML tools version : " + str(ct.__version__))

    model = small_model()
    model.eval()
    dummy_input = torch.randn(1,1,28,28)
    traced_model = torch.jit.trace(model, dummy_input)
    model = ct.convert(
        traced_model,
        inputs=[ ct.TensorType(name="input1", shape=dummy_input.shape) ],
    )

Maybe a tentative fix and put on a branch. Will push shortly for review.
In the meantime, new test case 
[testPool.txt](https://github.com/apple/coremltools/files/5017284/testPool.txt)

As I have difficulty with git pushing the branch (error 403), here is the tentative patch for review

[patch.txt](https://github.com/apple/coremltools/files/5017298/patch.txt)

PS: To be clear, I cannot generate the PR as I cannot push my branch at the moment. As the patch has only 4 lines of code, maybe someone else can carry it forward. Thx.

Resolving patch:
diff --git a/coremltools/converters/mil/frontend/torch/ops.py b/coremltools/converters/mil/frontend/torch/ops.py
--- a/coremltools/converters/mil/frontend/torch/ops.py
+++ b/coremltools/converters/mil/frontend/torch/ops.py
@@ -551,6 +551,8 @@ def max_pool2d(context, node):
     x = inputs[0]
     kernel_sizes = inputs[1]
     strides = inputs[2]
+    if strides.op.op_type == "const"  and (not list(strides.val)):
+        strides = mb.const(val=kernel_sizes.val, name=strides.name)
     pad_type = "custom"
 
     # Need to explicity state L-R, T-B pad
@@ -1527,6 +1529,8 @@ def _avg_pool(context, node, inputs):
     x = inputs[0]
     kernel_sizes = inputs[1]
     strides = inputs[2]
+    if strides.op.op_type == "const"  and (not list(strides.val)):
+        strides = mb.const(val=kernel_sizes.val, name=strides.name)
     pad_type = "custom"
     # Need to explicity state L-R, T-B pad
     pad = inputs[3]


Test patch: