Repository: pytorch
Issue: [Mobile] Valid scale ratio with invalid error
Labels: oncall: mobile

## 🐛 Bug


Hi, I try to generate torchScript and use dummy data to inference.
However, I encounter error.

Description
---
First, I inference the model on PC and everything is fine. 
Here is the code.
```python
import torch

model = torch.jit.load('mymodel_int8.pt')
input = torch.rand(1, 3, 144, 144)
model(input)
```
However, I just run the code on phone, 
error occurs. 
Here is my code. Just run the code on `onCreate`. 
```java
public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        Executor executor = Executors.newSingleThreadExecutor();
        executor.execute(new Runnable() {
            @Override
            public void run() {
                try {
                    Module module = Module.load(assetFilePath(MainActivity.this, "mymodel_int8.pt"));

                    FloatBuffer mInputTensorBuffer = Tensor.allocateFloatBuffer(3 * 144 * 144);
                    Tensor mInputTensor = Tensor.fromBlob(mInputTensorBuffer, new long[]{1, 3, 144, 144});
                    Tensor outputTensor = module.forward(IValue.from(mInputTensor)).toTensor();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        });
    }

    public static String assetFilePath(Context context, String assetName) throws IOException {
        File file = new File(context.getFilesDir(), assetName);
        if (file.exists() && file.length() > 0) {
            return file.getAbsolutePath();
        }

        try (InputStream is = context.getAssets().open(assetName)) {
            try (OutputStream os = new FileOutputStream(file)) {
                byte[] buffer = new byte[4 * 1024];
                int read;
                while ((read = is.read(buffer)) != -1) {
                    os.write(buffer, 0, read);
                }
                os.flush();
            }
            return file.getAbsolutePath();
        }
    }
}
```

And here is the error:
```
2021-05-08 14:42:05.014 14446-14509/com.example.jittest E/QNNPACK: failed to create add operator with 5.304397e-05 A-to-output scale ratio: scale ratio must be in [2**-14, 2**8) range
2021-05-08 14:42:05.021 14446-14509/com.example.jittest E/AndroidRuntime: FATAL EXCEPTION: pool-1-thread-1
    Process: com.example.jittest, PID: 14446
    java.lang.RuntimeException: The following operation failed in the TorchScript interpreter.
    Traceback of TorchScript, serialized code (most recent call last):
      File "code/__torch__.py", line 32, in forward
        _18 = getattr(_0, "17")
        out = (_1).forward(x0, )
        out0 = (_2).forward(out, )
                ~~~~~~~~~~~ <--- HERE
        out1 = (_3).forward(out0, )
        out2 = (_4).forward(out1, )
      File "code/__torch__.py", line 104, in forward
        _30 = (self.f_op).add_scalar((self.f_op).mul_scalar(y1_h, 2., ), -1., )
        self.s = (_29).mul_scalar(_30, self.clamp, )
        y2 = (self.f_op).add((self.f_op).mul(x2, self.s, ), (self.G).forward(y1, ), )
              ~~~~~~~~~~~~~~ <--- HERE
        return torch.cat([y1, y2], 1)
    class DenseBlock(Module):
      File "code/__torch__/torch/nn/quantized/modules/functional_modules/___torch_mangle_0.py", line 18, in add
        x: Tensor,
        y: Tensor) -> Tensor:
        r = ops.quantized.add(x, y, self.scale, self.zero_point)
            ~~~~~~~~~~~~~~~~~ <--- HERE
        r0 = (self.activation_post_process).forward(r, )
        return r0
    
    Traceback of TorchScript, original code (most recent call last):
      File "fuse_model_practice.py", line 226, in forward
            #         out = op.forward(out, rev)
            for op in self.operations:
                    out = op.forward(out)
                          ~~~~~~~~~~ <--- HERE
        
            out = self.dequant(out)
      File "/home/sunner/anaconda3/envs/torch1.7/lib/python3.6/site-packages/torch/nn/quantized/modules/functional_modules.py", line 186, in add
        def add(self, x: Tensor, y: Tensor) -> Tensor:
            r = ops.quantized.add(x, y, scale=self.scale, zero_point=self.zero_point)
                ~~~~~~~~~~~~~~~~~ <--- HERE
            r = self.activation_post_process(r)
            return r
    RuntimeError: createStatus == pytorch_qnnp_status_success INTERNAL ASSERT FAILED at "../aten/src/ATen/native/quantized/cpu/qadd.cpp":179, please report a bug to PyTorch. failed to create QNNPACK Add operator
    
        at org.pytorch.NativePeer.forward(Native Method)
        at org.pytorch.Module.forward(Module.java:49)
        at com.example.jittest.MainActivity$1.run(MainActivity.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
        at java.lang.Thread.run(Thread.java:760)
```

## Environment
#### PC
Collecting environment information...
PyTorch version: 1.8.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 16.04.7 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1

Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration: GPU 0: GeForce GTX 1080 Ti
Nvidia driver version: 440.118.02
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.19.2
[pip3] torch==1.8.1
[pip3] torchvision==0.9.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2020.2                      256  
[conda] mkl-service               2.3.0            py36he8ac12f_0  
[conda] mkl_fft                   1.3.0            py36h54f3939_0  
[conda] mkl_random                1.1.1            py36h0573a6f_0  
[conda] numpy                     1.19.2           py36h54aff64_0  
[conda] numpy-base                1.19.2           py36hfa32c7d_0  
[conda] pytorch                   1.8.1           py3.6_cuda10.2_cudnn7.6.5_0    pytorch
[conda] torchvision               0.9.1                py36_cu102    pytorch

#### Gradle on my project
```gradle
dependencies {
    implementation 'org.pytorch:pytorch_android:1.8.0'
    implementation 'org.pytorch:pytorch_android_torchvision:1.8.0'
}
```