Repository: pytorch
Issue: CppException in Android Studio with faster_rcnn_fbnetv3g_fpn.yaml Configuration
Labels: oncall: mobile

### 🐛 Describe the bug

**Issue Description:**
- **Model Configurations Tested:**
  - `faster_rcnn_fbnetv3a_C4.yaml` - Works perfectly in both Python and Android environments.
  - `faster_rcnn_fbnetv3g_fpn.yaml` - Works in Python (with slightly reduced accuracy), but fails in the Android environment.

- **Python Environment Behavior:**
  - With `faster_rcnn_fbnetv3g_fpn.yaml`, the model works but does not achieve the same level of object detection accuracy as before exporting to mobile.

- **Android Environment Behavior:**
  - A fatal exception occurs when using the model trained with `faster_rcnn_fbnetv3g_fpn.yaml`.

**Steps to Reproduce:**
1. Train, test, and export an object detection model with D2Go and configuration `faster_rcnn_fbnetv3a_C4.yaml`. Observe successful functionality.
2. Repeat the process with `faster_rcnn_fbnetv3g_fpn.yaml`. Notice reduced performance in Python and a fatal error in Android.

**Code Snippets:**
python
```
import torch
import torch.nn as nn
from typing import Tuple

class Wrapper(torch.nn.Module):
    def __init__(self, model):
        super().__init__()
        self.model = model

    def forward(self, inputs: torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]:
        x = inputs.unsqueeze(0) * 255
        scale = 320.0 / min(x.shape[-2], x.shape[-1])
        x = torch.nn.functional.interpolate(x, scale_factor=scale, mode="bilinear", align_corners=True, recompute_scale_factor=True)
        out = self.model(x[0])
        return out[0] / scale, out[1], out[2], out[3]

# Additional model export and optimization code
predictor_path = convert_and_export_predictor(
    copy.deepcopy(cfg),
    copy.deepcopy(pytorch_model),
    "torchscript_int8@tracing",
    '/data/outlet/',
    data_loader
)
path_ = os.path.join(predictor_path, "model.jit")
torchmodel = torch.jit.load(path_)
wrapped_model = Wrapper(torchmodel)
scripted_model = torch.jit.script(wrapped_model)
optimized_model = optimize_for_mobile(scripted_model)
optimized_model._save_for_lite_interpreter("/data/outlet/d2go1.pt")
```


**Error Observed in Android:**
```
FATAL EXCEPTION: Thread-2
Process: org.pytorch.demo.objectdetection, PID: 4544
com.facebook.jni.CppException: Output channel size of weight and bias must match.

Debug info for handle(s): debug_handles:{-1}, was not found.

Exception raised from apply_impl at /home/agunapal/pytorch/aten/src/ATen/native/quantized/cpu/qconv.cpp:899 (most recent call first):
(no backtrace available)
at org.pytorch.LiteNativePeer.forward(Native Method)
at org.pytorch.Module.forward(Module.java:52)
at org.pytorch.demo.objectdetection.MainActivity.run(MainActivity.java:333)
at java.lang.Thread.run(Thread.java:1012)
```

**Expected Behavior:**
- The model with `faster_rcnn_fbnetv3g_fpn.yaml` should function correctly in the Android environment, similar to its behavior in Python and like the `faster_rcnn_fbnetv3a_C4.yaml` model.




### Versions

PyTorch version: 2.2.0.dev20231016+cu118
Is debug build: False
CUDA used to build PyTorch: 11.8
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.2 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.8.89
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3060
Nvidia driver version: 531.61
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      39 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             24
On-line CPU(s) list:                0-23
Vendor ID:                          GenuineIntel
Model name:                         13th Gen Intel(R) Core(TM) i7-13700F
CPU family:                         6
Model:                              183
Thread(s) per core:                 2
Core(s) per socket:                 12
Socket(s):                          1
Stepping:                           1
BogoMIPS:                           4224.01
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm serialize flush_l1d arch_capabilities
Virtualization:                     VT-x
Hypervisor vendor:                  Microsoft
Virtualization type:                full
L1d cache:                          576 KiB (12 instances)
L1i cache:                          384 KiB (12 instances)
L2 cache:                           24 MiB (12 instances)
L3 cache:                           30 MiB (1 instance)
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Not affected
Vulnerability Retbleed:             Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.24.1
[pip3] pytorch-lightning==1.8.6
[pip3] pytorch-triton==2.1.0+6e4932cda8
[pip3] torch==2.2.0.dev20231016+cu118
[pip3] torchaudio==2.2.0.dev20231016+cu118
[pip3] torchmetrics==1.2.0
[pip3] torchvision==0.17.0.dev20231016+cu118
[conda] numpy                     1.24.1                   pypi_0    pypi
[conda] pytorch-lightning         1.8.6                    pypi_0    pypi
[conda] pytorch-triton            2.1.0+6e4932cda8          pypi_0    pypi
[conda] torch                     2.2.0.dev20231016+cu118          pypi_0    pypi
[conda] torchaudio                2.2.0.dev20231016+cu118          pypi_0    pypi
[conda] torchmetrics              1.2.0                    pypi_0    pypi
[conda] torchvision               0.17.0.dev20231016+cu118          pypi_0    pypi