Repository: pytorch
Issue: quantization: error message when using `convert_fx` on a model on cuda should be better
Labels: oncall: quantization,triaged

### 🐛 Describe the bug

When a user tries to use `convert_fx` on a model which is on cuda, the error message doesn't make sense.  We should either throw an error message which asks the user to move the model to cuda, or we should lower successfully.

Repro script:

```
import torch
import torch.ao.quantization.quantize_fx as quantize_fx

qconfig_mapping = torch.ao.quantization.get_default_qconfig_mapping('fbgemm')
m = torch.nn.Sequential(torch.nn.Linear(1, 1)).to('cuda')                    
mp = quantize_fx.prepare_fx(m, qconfig_mapping, (torch.randn(1),))           
mq = quantize_fx.convert_fx(mp)                                              
```

Error message:

```
import torch.ao.quantization.quantize_fx as quantize_fx
(pytorch) [vasiliy@devgpu032.ftw6 ~/local/pytorch (master)]$ python ../tmp/test.py
/data/users/vasiliy/pytorch/torch/ao/quantization/observer.py:214: UserWarning: Please use quant_min and quant_max to spe
cify the range for observers.                     reduce_range will be deprecated in a future release of PyTorch.
  warnings.warn(
/data/users/vasiliy/pytorch/torch/ao/quantization/observer.py:1204: UserWarning: must run observer before calling calcula
te_qparams.                                    Returning default scale and zero point 
  warnings.warn(
Traceback (most recent call last):
  File "/data/users/vasiliy/pytorch/../tmp/test.py", line 18, in <module>
    mq = quantize_fx.convert_fx(mp)
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/quantize_fx.py", line 622, in convert_fx
    return _convert_fx(
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/quantize_fx.py", line 548, in _convert_fx
    quantized = convert(
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/fx/convert.py", line 766, in convert
    model = lower_to_fbgemm(model, node_name_to_qconfig, node_name_to_scope)
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/fx/lower_to_fbgemm.py", line 16, in lower_to_fbgemm
    return _lower_to_native_backend(model, qconfig_map, node_name_to_scope)
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/fx/_lower_to_native_backend.py", line 928, in _lower_to_native_
backend
    _lower_static_weighted_ref_module(model, qconfig_map)
  File "/data/users/vasiliy/pytorch/torch/ao/quantization/fx/_lower_to_native_backend.py", line 499, in _lower_static_wei
ghted_ref_module
    q_module = q_class.from_reference(ref_module, output_scale, output_zero_point)
  File "/data/users/vasiliy/pytorch/torch/ao/nn/quantized/modules/linear.py", line 298, in from_reference
    qlinear.set_weight_bias(qweight, ref_qlinear.bias)
  File "/data/users/vasiliy/pytorch/torch/ao/nn/quantized/modules/linear.py", line 240, in set_weight_bias
    self._packed_params.set_weight_bias(w, b)
  File "/data/users/vasiliy/pytorch/torch/ao/nn/quantized/modules/linear.py", line 32, in set_weight_bias
    self._packed_params = torch.ops.quantized.linear_prepack(weight, bias)
  File "/data/users/vasiliy/pytorch/torch/_ops.py", line 446, in __call__
    return self._op(*args, **kwargs or {})
RuntimeError: Unsupported qscheme: per_channel_affine
```

### Versions

master

cc @jerryzh168 @jianyuh @raghuramank100 @jamesr66a @jgong5 @Xia-Weiwen @leslie-fang-intel