Repository: pytorch
Issue: Mobile Android: Could not run 'aten::quantize_per_tensor' with arguments from the 'Vulkan' backend
Labels: triaged,module: android,oncall: mobile

## 🐛 Bug

When running a model with vulkan on android (converted using the mobile recipes on PyTorch Mobile page), this error happens on forward pass. Works fine when inferencing on CPU.

## To Reproduce

### Mobile Code:
```Java
try {
                    bitmap = ((BitmapDrawable)imageView.getDrawable()).getBitmap();
                    image_h = bitmap.getHeight();
                    image_w = bitmap.getWidth();

                    //resize to 416 x 416
                    bitmap = Bitmap.createScaledBitmap(bitmap, 416, 416, true);

                    module = Module.load(fetchModelFile(MainActivity.this, "mobile.pts"), Device.VULKAN);
                } catch (IOException e) {
                    finish();
                }

                //Input Tensor
                float[] mean = {0.5f, 0.5f, 0.5f};
                float[] std = {0.5f, 0.5f, 0.5f};
                final Tensor input = TensorImageUtils.bitmapToFloat32Tensor(
                        bitmap,
                        mean,
                        std
                );

//                //Calling the forward of the model to run our input
                final Tensor output = module.forward(IValue.from(input)).toTensor(); 
```

At forward this is raised:
```
E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.example.uwudemo, PID: 18203
    java.lang.RuntimeException: The following operation failed in the TorchScript interpreter.
    Traceback of TorchScript, serialized code (most recent call last):
      File "code/__torch__/unet_aligner_trainer/models/unet.py", line 43, in forward
        _13 = self.encoder_layer2
        _14 = self.encoder_layer1
        _15 = (self.conv1).forward((self.quant).forward(X, ), )
                                    ~~~~~~~~~~~~~~~~~~~ <--- HERE
        _16 = (_14).forward(_15, )
        _17 = (_13).forward(_16, )
      File "code/__torch__/torch/nn/quantized/modules.py", line 10, in forward
      def forward(self: __torch__.torch.nn.quantized.modules.Quantize,
        X: Tensor) -> Tensor:
        input = torch.quantize_per_tensor(X, 0.0066144033335149288, 104, 13)
                ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
        return input
    class DeQuantize(Module):

// stack trace of python calls //

    RuntimeError: Could not run 'aten::quantize_per_tensor' with arguments from the 'Vulkan' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::quantize_per_tensor' is only available for these backends: [CPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, Tracer, Autocast, Batched, VmapMode].
    
    CPU: registered at aten/src/ATen/RegisterCPU.cpp:5925 [kernel]
    BackendSelect: fallthrough registered at ../aten/src/ATen/core/BackendSelectFallbackKernel.cpp:3 [backend fallback]
    Named: registered at ../aten/src/ATen/core/NamedRegistrations.cpp:7 [backend fallback]
    AutogradOther: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:35 [backend fallback]
    AutogradCPU: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:39 [backend fallback]
    AutogradCUDA: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:43 [backend fallback]
    AutogradXLA: fallthrough registered at ../aten/src/ATen/core/VariableFallbackKernel.cpp:47 [backend fallback]
    Tracer: fallthrough registered at ../torch/csrc/jit/frontend/tracer.cpp:999 [backend fallback]
    Autocast: fallthrough registered at ../aten/src/ATen/autocast_mode.cpp:250 [backend fallback]
    Batched: registered at ../aten/src/ATen/BatchingRegistrations.cpp:1016 [backend fallback]
    VmapMode: fallthrough registered at ../aten/src/ATen/VmapModeRegistrations.cpp:33 [backend fallback]
    
    
        at org.pytorch.NativePeer.forward(Native Method)
        at org.pytorch.Module.forward(Module.java:49)
        at com.example.uwudemo.MainActivity$2.onClick(MainActivity.java:106)
        at android.view.View.performClick(View.java:7448)
        at com.google.android.material.button.MaterialButton.performClick(MaterialButton.java:967)
        at android.view.View.performClickInternal(View.java:7425)
        at android.view.View.access$3600(View.java:810)
        at android.view.View$PerformClick.run(View.java:28305)
        at android.os.Handler.handleCallback(Handler.java:938)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:223)
        at android.app.ActivityThread.main(ActivityThread.java:7660)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)
```

## Expected behavior

As per the mobile recipe, optimizing for mobile with backend set to "Vulkan" should apply all the layer optimizations needed.

## Environment
 - PyTorch Version (e.g., 1.0): 1.8.1
 - OS (e.g., Linux): Ubuntu 18.04.3
 - How you installed PyTorch (`conda`, `pip`, source): pip
 - Build command you used (if compiling from source):
 - Python version: 3.6
 - CUDA/cuDNN version: 10.2 / 7.6.5
 - GPU models and configuration: 4x RTX2080Ti
 - Any other relevant information: Android Build version 30

## Additional context

<!-- Add any other context about the problem here. -->