Repository: pytorch
Issue: Conversion Error in pytorch mobile with metal
Labels: module: memory format,oncall: mobile

### 🐛 Describe the bug

Hi! I'm trying to convert my model to . However, I can't do it. There were a lot of problems, but mostly they are indicative, for example "Could not run 'aten::native_batch_norm' with arguments from the 'Metal' backend.". It is clear that there is no such operator at the moment (InstanceNorm2d). However, in some operators I get this: 
__'memory_format' argument is incompatible with Metal tensor__

```python3
import torch
import torch.nn as nn
from torch.utils.mobile_optimizer import optimize_for_mobile


class TEST(nn.Module):
    def __init__(self):
        super().__init__()

    def forward(self, x):
        return torch.pow(x, 2)


rand_input = torch.randn(1, 3, 256, 256)
with torch.no_grad():
    export_model = TEST()
    export_model = export_model.eval()
    export_model = export_model.cpu()
    print(export_model(rand_input).shape)
    print(export_model(rand_input).dtype)

    traced_script_module = torch.jit.trace(export_model, rand_input.cpu()) ## both options cannot work
    #traced_script_module = torch.jit.script(export_model) ## both options cannot work
    optimized_model = optimize_for_mobile(traced_script_module, backend='metal')
    print(torch.jit.export_opnames(optimized_model))
    optimized_model._save_for_lite_interpreter("traced_test.ptl")
```
output 
```
torch.Size([1, 3, 256, 256])
torch.float32
['aten::pow.Tensor_Scalar']
```

Then i try to run model on Iphone X
```objectivec
at::Tensor rand_input = torch::randn({1,3,256,256}).metal();
std::cout<<" sizes  "<<rand_input.size(0)<<" "<<rand_input.size(1)<<" "<<rand_input.size(2)<<" "<<rand_input.size(3)<<std::endl;
NSTimeInterval start = [NSDate timeIntervalSinceReferenceDate];
auto outputTensor = _impl.forward({ rand_input }).toTensor().cpu();
NSTimeInterval end = [NSDate timeIntervalSinceReferenceDate];
NSLog(@">>>> MODEL FPS: %f", 1.0f/(end - start));
```
out 
```
2022-02-01 17:39:59.369543+0300 Proj-iOS[25555:24850162] Metal API Validation Enabled
 sizes  1 3 256 256
2022-02-01 17:39:59.450054+0300 Proj-iOS[25555:24850162] 'memory_format' argument is incompatible with Metal tensor
  
  Debug info for handle(s): debug_handles:{-1}, was not found.
  
Exception raised from empty at /Users/distiller/project/aten/src/ATen/native/metal/MetalAten.mm:84 (most recent call first):
frame #0: _ZN3c106detail14torchCheckFailEPKcS2_jS2_ + 92 (0x101cecdd8 in Proj-iOS)
frame #1: _ZN2at6native5metal5emptyEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE + 284 (0x10168fa2c in Proj-iOS)
frame #2: _ZN2at4_ops19empty_memory_format10redispatchEN3c1014DispatchKeySetENS2_8ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEENS6_INS2_12MemoryFormatEEE + 200 (0x1009f6e04 in Proj-iOS)
frame #3: _ZNK3c1010Dispatcher4callIN2at6TensorEJNS_8ArrayRefIxEENS_8optionalINS_10ScalarTypeEEENS6_INS_6LayoutEEENS6_INS_6DeviceEEENS6_IbEENS6_INS_12MemoryFormatEEEEEET_RKNS_19TypedOperatorHandleIFSG_DpT0_EEESJ_ + 228 (0x100a45e60 in Proj-iOS)
frame #4: _ZN2at4_ops19empty_memory_format4callEN3c108ArrayRefIxEENS2_8optionalINS2_10ScalarTypeEEENS5_INS2_6LayoutEEENS5_INS2_6DeviceEEENS5_IbEENS5_INS2_12MemoryFormatEEE + 144 (0x1009f6c64 in Proj-iOS)
frame #5: _ZN2at5emptyEN3c108ArrayRefIxEENS0_13TensorOptionsENS0_8optionalINS0_12MemoryFormatEEE + 276 (0x100c3ae70 in Proj-iOS)
frame #6: _ZN2at12_GLOBAL__N_155structured_pow_Tensor_Scalar_default_backend_functional10set_outputExN3c108ArrayRefIxEES4_NS2_13TensorOptionsENS3_INS_7DimnameEEE + 152 (0x100c61770 in Proj-iOS)
frame #7: _ZN2at18TensorIteratorBase11fast_set_upERKNS_20TensorIteratorConfigE + 224 (0x10168d230 in Proj-iOS)
frame #8: _ZN2at18TensorIteratorBase5buildERNS_20TensorIteratorConfigE + 152 (0x10168a6e8 in Proj-iOS)
frame #9: _ZN2at18TensorIteratorBase47build_output_borrowing_argument_owning_unary_opERKNS_10TensorBaseES3_ + 140 (0x10168b720 in Proj-iOS)
frame #10: _ZN2at4meta28structured_pow_Tensor_Scalar4metaERKNS_6TensorERKN3c106ScalarE + 176 (0x10150246c in Proj-iOS)
frame #11: _ZN2at12_GLOBAL__N_125wrapper_pow_Tensor_ScalarERKNS_6TensorERKN3c106ScalarE + 116 (0x100c1c58c in Proj-iOS)
frame #12: _ZN3c104impl34call_functor_with_args_from_stack_INS0_6detail31WrapFunctionIntoRuntimeFunctor_IPFN2at6TensorERKS5_RKNS_6ScalarEES5_NS_4guts8typelist8typelistIJS7_SA_EEEEELb0EJLm0ELm1EEJS7_SA_EEENSt3__15decayINSD_21infer_function_traitsIT_E4type11return_typeEE4typeEPNS_14OperatorKernelENS_14DispatchKeySetEPNSI_6vectorINS_6IValueENSI_9allocatorISU_EEEENSI_16integer_sequenceImJXspT1_EEEEPNSF_IJDpT2_EEE + 96 (0x100bb80c0 in Proj-iOS)
frame #13: _ZN3c104impl31make_boxed_from_unboxed_functorINS0_6detail31WrapFunctionIntoRuntimeFunctor_IPFN2at6TensorERKS5_RKNS_6ScalarEES5_NS_4guts8typelist8typelistIJS7_SA_EEEEELb0EE4callEPNS_14OperatorKernelERKNS_14OperatorHandleENS_14DispatchKeySetEPNSt3__16vectorINS_6IValueENSP_9allocatorISR_EEEE + 40 (0x100bb8004 in Proj-iOS)
frame #14: _ZNK3c1010Dispatcher9callBoxedERKNS_14OperatorHandleEPNSt3__16vectorINS_6IValueENS4_9allocatorIS6_EEEE + 164 (0x10176c044 in Proj-iOS)
frame #15: _ZN5torch3jit6mobile16InterpreterState3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 4580 (0x101777584 in Proj-iOS)
frame #16: _ZN5torch3jit6mobile8Function3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 108 (0x10176a4cc in Proj-iOS)
frame #17: _ZNK5torch3jit6mobile6Method3runERNSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 560 (0x101779e28 in Proj-iOS)
frame #18: _ZNK5torch3jit6mobile6MethodclENSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 24 (0x10177a9d4 in Proj-iOS)
frame #19: _ZN5torch3jit6mobile6Module7forwardENSt3__16vectorIN3c106IValueENS3_9allocatorIS6_EEEE + 148 (0x101918f04 in Proj-iOS)
frame #20: -[InferenceModule + + (0x101918528 in Proj-iOS)
frame #21: -[ViewController + + (0x101a950f0 in Proj-iOS)
frame #22: -[CvVideoCamera + + (0x10203daf4 in Proj-iOS)
frame #23: CF54A5DB-6EE5-3E73-B7E1-D2D0AB7598C8 + 147436 (0x1c7081fec in AVFCapture)
frame #24: CF54A5DB-6EE5-3E73-B7E1-D2D0AB7598C8 + 146764 (0x1c7081d4c in AVFCapture)
frame #25: 2F509455-C380-3E7C-AFB5-CB0461F08A60 + 143420 (0x1c713f03c in CMCapture)
frame #26: 2F509455-C380-3E7C-AFB5-CB0461F08A60 + 2651332 (0x1c73a34c4 in CMCapture)
frame #27: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 397976 (0x1ae03f298 in libdispatch.dylib)
frame #28: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 236144 (0x1ae017a70 in libdispatch.dylib)
frame #29: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 303900 (0x1ae02831c in libdispatch.dylib)
frame #30: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 250384 (0x1ae01b210 in libdispatch.dylib)
frame #31: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 253484 (0x1ae01be2c in libdispatch.dylib)
frame #32: 03AD11F9-67AE-3219-ACA3-DF0A9AF629D4 + 292460 (0x1ae02566c in libdispatch.dylib)
frame #33: _pthread_wqthread + 272 (0x1f68b55bc in libsystem_pthread.dylib)
frame #34: start_wqthread + 8 (0x1f68b886c in libsystem_pthread.dylib)
2022-02-01 17:39:59.450868+0300 Proj-iOS[25555:24850162] >>>> FPS: 5.739442
```

Also, i have this problem on:
* nn.AvgPool2d(256)
* torch.exp()
* nn.LeakyReLU(0.1)
* nn.Upsample(scale_factor=2, mode='bilinear') BUT!! nn.Upsample(scale_factor=2) works correctly

Please help me understand what the error is mean and how it can be fixed.

linked q: #62872
https://discuss.pytorch.org/t/ios-metal-memory-format-argument-is-incompatible-with-metal-tensor/138562/3
https://discuss.pytorch.org/t/memory-format-argument-is-incompatible-with-metal-tensor/128706
but this is ops not quantized

### Versions

Collecting environment information...
PyTorch version: 1.11.0.dev20220114
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 12.1 (x86_64)
GCC version: Could not collect
Clang version: 13.0.0 (clang-1300.0.29.30)
CMake version: version 3.20.5
Libc version: N/A

Python version: 3.9.9 | packaged by conda-forge | (main, Dec 20 2021, 02:41:37)  [Clang 11.1.0 ] (64-bit runtime)
Python platform: macOS-12.1-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.2
[pip3] torch==1.11.0.dev20220114
[pip3] torchvision==0.12.0.dev20220119
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               9.0                  h41a26b3_0  
[conda] libblas                   3.9.0              12_osx64_mkl    conda-forge
[conda] libcblas                  3.9.0              12_osx64_mkl    conda-forge
[conda] liblapack                 3.9.0              12_osx64_mkl    conda-forge
[conda] liblapacke                3.9.0              12_osx64_mkl    conda-forge
[conda] mkl                       2021.4.0           hecd8cb5_637  
[conda] mkl-service               2.4.0            py39h9ed2024_0  
[conda] mkl_fft                   1.3.1            py39h4ab4a9b_0  
[conda] mkl_random                1.2.2            py39hb2f4e1b_0  
[conda] numpy                     1.21.2           py39h4b4dc7a_0  
[conda] numpy-base                1.21.2           py39he0bd621_0  
[conda] pytorch                   1.11.0.dev20220114         py3.9_0    pytorch-nightly
[conda] torchvision               0.12.0.dev20220119        py39_cpu    pytorch-nightly

Install LibTorch (demo nets work correctly):  
pod 'LibTorch-Lite-Nightly'

cc @VitalyFedyunin @jamesr66a