Repository: pytorch
Issue: Runtime error in Libtorch cpp project (Didn't find engine for operation quantized::conv2d_prepack NoQEngine)
Labels: oncall: quantization,triaged

### 🐛 Describe the bug

I am working on cpp project having quantization process using qnnpack. However, I encounter the error when I execute the build result. Building with torchscript models is successful, but executing the build result gives me this error.
```
terminate called after throwing an instance of 'std::runtime_error'
  what():  The following operation failed in the TorchScript interpreter.
Traceback of TorchScript, serialized code (most recent call last):
  File "code/__torch__/torch/nn/quantized/modules/conv.py", line 72, in __setstate__
    self.groups = (state)[8]
    self.padding_mode = (state)[9]
    _7 = (self).set_weight_bias((state)[10], (state)[11], )
          ~~~~~~~~~~~~~~~~~~~~~ <--- HERE
    self.scale = (state)[12]
    self.zero_point = (state)[13]
  File "code/__torch__/torch/nn/quantized/modules/conv.py", line 94, in set_weight_bias
      _13 = [_11, _12]
      _14, _15, = dilation
      _16 = ops.quantized.conv2d_prepack(w, b, _10, _13, [_14, _15], groups)
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
      self._packed_params = _16
    else:

Traceback of TorchScript, original code (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/quantized/modules/conv.py", line 176, in __setstate__
        self.groups = state[8]
        self.padding_mode = state[9]
        self.set_weight_bias(state[10], state[11])
        ~~~~~~~~~~~~~~~~~~~~ <--- HERE
        self.scale = state[12]
        self.zero_point = state[13]
  File "/opt/conda/lib/python3.8/site-packages/torch/nn/quantized/modules/conv.py", line 432, in set_weight_bias
    def set_weight_bias(self, w: torch.Tensor, b: Optional[torch.Tensor]) -> None:
        if self.padding_mode == 'zeros':
            self._packed_params = torch.ops.quantized.conv2d_prepack(
                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
                w, b, self.stride, self.padding, self.dilation, self.groups)
        else:
RuntimeError: Didn't find engine for operation quantized::conv2d_prepack NoQEngine

Aborted (core dumped)
```

When I check `torch.backends.quantized.supported_engines`, qnnpack comes out to be supported.
```
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.backends.quantized.supported_engines
['qnnpack', 'none']
```

### Machine
`Linux 674db9b68c77 5.15.0-1013-aws #17-Ubuntu SMP Fri Jun 10 10:43:12 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux`

What can be the causes of this error?

### Versions

```
python collect_env.py 
Collecting environment information...
PyTorch version: 1.11.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.4 LTS (aarch64)
GCC version: (Ubuntu 10.3.0-1ubuntu1~20.04) 10.3.0
Clang version: Could not collect
CMake version: version 3.16.3
Libc version: glibc-2.31

Python version: 3.8.10 (default, Mar 15 2022, 12:22:08)  [GCC 9.4.0] (64-bit runtime)
Python platform: Linux-5.15.0-1013-aws-aarch64-with-glibc2.29
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.5
[pip3] torch==1.11.0
[pip3] torchdata==0.3.0
[pip3] torchtext==0.12.0a0+d7a34d6
[pip3] torchvision==0.12.0
[conda] Could not collect
```

### Additional context
I have built the torchscript model in `x86_64` machine and am trying to compile the model in `aarch64` machine for deployment. Does it have possibility of causing this issue?

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel