Index out of bounds Error with PerChannel Quantization 
Labels: oncall: quantization, triaged

### 🐛 Index out of bounds Error with PerChannel Quantization

Hello,

I have encountered this problem while trying to perform per-channel quantization on weights with ch_axis=1 quantization parameter. It causes the “index out of bounds error” when dimention of axis 1 of the weight tensor is smaller than dimention of axis 0(in the following example 100 is smaller than 110(note that 100 will be axis 1 in weight matrix)). If the axis 0 dimention is smaller(when changing 110 to 90) the error doesn’t occure. It is not reproducible with ch_axis=0 quantization parameter. With ch_axis=0 it doesn’t matter if one axis dimention is bigger or smaller then the other.
Here is a minimal example that fails:
```
import torch
import torch.nn as nn
from torch.quantization.observer import PerChannelMinMaxObserver, MinMaxObserver

class QTestNet1(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear = nn.Linear(100,110, bias=False)
        
    def set_qconfig(self):

        self.linear.qconfig = torch.quantization.QConfig(
            weight=PerChannelMinMaxObserver.with_args(dtype=torch.qint8, ch_axis=1),
            activation=MinMaxObserver.with_args(qscheme=torch.per_tensor_affine)
        )
    
    def forward(self, x):
        x = self.linear(x)
        return x
        
model = QTestNet1()
model.set_qconfig()
model_prepared = torch.quantization.prepare_qat(model, inplace=False)

input_x = torch.randn(1,100) 
model_prepared(input_x).shape # just checking that forward doesn't fail

model_int8 = torch.quantization.convert(model_prepared.eval()) # error
```

This is a full error:
```
/usr/local/lib/python3.7/dist-packages/torch/ao/quantization/quantize.py in convert(module, mapping, inplace, remove_qconfig, is_reference, convert_custom_config_dict)
    519     _convert(
    520         module, mapping, inplace=True, is_reference=is_reference,
--> 521         convert_custom_config_dict=convert_custom_config_dict)
    522     if remove_qconfig:
    523         _remove_qconfig(module)

/usr/local/lib/python3.7/dist-packages/torch/ao/quantization/quantize.py in _convert(module, mapping, inplace, is_reference, convert_custom_config_dict)
    557             _convert(mod, mapping, True,  # inplace
    558                      is_reference, convert_custom_config_dict)
--> 559         reassign[name] = swap_module(mod, mapping, custom_module_class_mapping)
    560 
    561     for key, value in reassign.items():

/usr/local/lib/python3.7/dist-packages/torch/ao/quantization/quantize.py in swap_module(mod, mapping, custom_module_class_mapping)
    590                 new_mod = qmod.from_float(mod, weight_qparams)
    591             else:
--> 592                 new_mod = qmod.from_float(mod)
    593             swapped = True
    594 

/usr/local/lib/python3.7/dist-packages/torch/nn/quantized/modules/linear.py in from_float(cls, mod)
    271                       mod.out_features,
    272                       dtype=dtype)
--> 273         qlinear.set_weight_bias(qweight, mod.bias)
    274         qlinear.scale = float(act_scale)
    275         qlinear.zero_point = int(act_zp)

/usr/local/lib/python3.7/dist-packages/torch/nn/quantized/modules/linear.py in set_weight_bias(self, w, b)
    232 
    233     def set_weight_bias(self, w: torch.Tensor, b: Optional[torch.Tensor]) -> None:
--> 234         self._packed_params.set_weight_bias(w, b)
    235 
    236     @classmethod

/usr/local/lib/python3.7/dist-packages/torch/nn/quantized/modules/linear.py in set_weight_bias(self, weight, bias)
     25     def set_weight_bias(self, weight: torch.Tensor, bias: Optional[torch.Tensor]) -> None:
     26         if self.dtype == torch.qint8:
---> 27             self._packed_params = torch.ops.quantized.linear_prepack(weight, bias)
     28         elif self.dtype == torch.float16:
     29             self._packed_params = torch.ops.quantized.linear_prepack_fp16(weight, bias)

/usr/local/lib/python3.7/dist-packages/torch/_ops.py in __call__(self, *args, **kwargs)
    141         # We save the function ptr as the `op` attribute on
    142         # OpOverloadPacket to access it here.
--> 143         return self._op(*args, **kwargs or {})
    144 
    145     # TODO: use this to make a __dir__

IndexError: select(): index 100 out of range for tensor of size [100] at dimension 0
```

I am using torch==1.12.1+cu113 from google colab.

### Versions

Collecting environment information...
PyTorch version: 1.12.1+cu113
Is debug build: False
CUDA used to build PyTorch: 11.3
ROCM used to build PyTorch: N/A

OS: Ubuntu 18.04.6 LTS (x86_64)
GCC version: (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Clang version: 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
CMake version: version 3.22.6
Libc version: glibc-2.26

Python version: 3.7.13 (default, Apr 24 2022, 01:04:09)  [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
Is CUDA available: False
CUDA runtime version: 11.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.6.5
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.0.5
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.0.5
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.21.6
[pip3] torch==1.12.1+cu113
[pip3] torchaudio==0.12.1+cu113
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.13.1
[pip3] torchvision==0.13.1+cu113
[conda] Could not collect

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @vkuzo @jgong5 @Xia-Weiwen @leslie-fang-intel