Issue 1:
(This is a feature that I've been working on (see #21493 (comment)). I open this issue so that relevant PRs can be better tracked.)

🚀 Feature and Motivation

Better CMake integration. Currently there are a lot of inconsistency and redundancy between the Python setup scripts and CMake scripts (duplicated env check, overriding env check, inconsistent treatment of which results of checking will be used, etc.). I would like to improve the integration.

Pitch

The improvement I would like to implement includes:

Allowing build options to be directly specified using options specified in CMakeLists.txt (without explicitly being processed in Python setup scripts).

Allowing users to adjust build options directly via cmake-gui/ccmake/editing CMakeCache.txt (currently users must take care of both options in CMakeCache.txt and environmental variables passed to setup.py and ensure they are consistent upon rebuild).

removing a lot of system and library environment checks in Python setup scripts while CMake can already handle them pretty well.

...

Existing PRs (including those that were made to clear the path)

#21034

#21296

#21417

#21493

#21653

#21877

#22153

#22154

#22215

#22360

Issue 2:
🐛 Bug

I am getting above error while doing inference after quantization of my model.

a. This happens after 62nd epoch. If I quantize using any weights till 62nd epoch, every thing is fine.

b. If I use partial (1/3) data to train my model, I see no issue.

c. Only with "qnnpack". No issue with 'fbgemm'

d. After executing operation in model that cause this error, the output becomes random (due to "qnnp_delete_operator(convolution);" in qnnpack/convolution.c)

e. Location of error (in model) changes with weights.

To Reproduce

Steps to reproduce the behavior:

Train float model

Load trained model

Quantize float model with "qnnpack' backend: (my quantization script)

def static_quantize(model, backend):
if backend not in torch.backends.quantized.supported_engines:
raise RuntimeError("Quantized backend not supported ")
model.eval()
if backend == 'fbgemm':
torch.backends.quantized.engine = 'fbgemm'
model.qconfig = torch.quantization.get_default_qconfig('fbgemm')
elif backend == 'qnnpack':
torch.backends.quantized.engine = 'qnnpack'
model.qconfig = torch.quantization.get_default_qconfig('qnnpack')

print(model.qconfig)
model.fuse_model()
torch.quantization.prepare(model, inplace=True)
# Calibrate first
print("Calibrating model...")
epochs = 10
validate(model, cfg, epochs)
# calibration done
torch.quantization.convert(model, inplace=True)
# Test post quantization accuracy
validate(model, cfg, epochs) <------- **ERROR** seen here

Quantized model operation causing this issue

(3): Sequential(
(0): QuantBlock(
(skip_add): QFunctional(scale=0.11691034585237503, zero_point=0)
(conv): Sequential(
(0): ConvBN(
(0): QuantizedConv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), scale=0.2153073400259018, zero_point=62, padding=(1, 1), groups=256)
(1): Identity()
)
(1): ConvBNReLU(
(0): QuantizedConvReLU2d(256, 256, kernel_size=(1, 1), stride=(1, 1), scale=0.06376750022172928, zero_point=0)
(1): Identity()
(2): Identity()
)
)
)

## Expected behavior
output of qnnpack and fbgemm should be same.

## Environment

- PyTorch Version (e.g., 1.0): '1.5.0.dev20200316' (nightly)
- OS (e.g., Linux): Ubuntu 18.04
- How you installed PyTorch (`conda`, `pip`, source): Conda
- Build command you used (if compiling from source): NA
- Python version: 3.7
- CUDA/cuDNN version: 10.1
- GPU models and configuration: RTX 2060
- Any other relevant information:

## Additional context

Working weights for Conv layer.
tensor([[[[-9.8111e-02, -2.3818e-01, -1.0097e-01],
[-8.6748e-02, -1.5611e+00, -3.7088e-02],
[-7.2189e-02, 9.3585e-03, -2.1457e-01]]],
[[[-9.0585e-01, -1.4491e+00, -1.0290e+00],
[ 6.0444e-02, 1.7321e-02, 4.5470e-02],
[ 2.0289e-01, 1.7305e-01, 2.3026e-01]]],
[[[-3.9760e-01, -1.7248e-01, -4.0785e-01],
[ 5.5625e-03, 7.6404e-01, -7.1765e-02],
[-5.1368e-01, -3.6436e-01, -5.2025e-01]]],
[[[-1.2394e-01, -4.8440e-01, -4.7897e-01],
[-4.6179e-01, -4.1148e-01, -4.7571e-01],
[-3.6709e-01, -2.5656e-01, -3.8935e-01]]],
[[[ 3.8117e-01, 2.7804e-01, 4.4873e-01],
[ 4.8555e-01, -2.8842e-01, 2.9044e-01],
[ 2.5739e-01, 6.3275e-01, 3.8192e-01]]],
[[[-2.5214e-01, 3.7439e-01, -3.5673e-01],
[ 3.0839e-01, 1.0794e+00, 4.3154e-01],
[-2.5715e-01, 3.6404e-01, -3.6204e-01]]],
[[[-5.4990e-01, -2.2252e-01, -5.0191e-01],
[-1.7313e-01, 3.8547e-01, -1.1505e-01],
[-6.0288e-01, -2.6323e-01, -6.4934e-01]]],
[[[ 2.3457e-01, 2.6993e-01, 3.5980e-01],
[ 1.4922e-01, 1.1058e+00, 4.5924e-01],
[ 5.3947e-01, 1.6402e-01, 1.4880e-01]]],
[[[ 5.9202e-01, 5.5206e-02, 5.2480e-01],
[ 4.3953e-02, -1.1449e-01, 1.7012e-01],
[ 5.4099e-01, -1.3535e-01, 4.9393e-01]]],
[[[-3.6607e-01, 2.1279e-01, -3.4666e-01],
[ 1.9444e-01, 1.1031e+00, 2.0701e-01],
[-3.1785e-01, 2.0595e-01, -3.7019e-01]]],
[[[ 9.1770e-02, -2.8905e-02, 1.8759e-01],
[-1.2641e-01, -1.3670e+00, 1.0820e-01],
[ 3.7640e-01, 2.5282e-02, 1.6139e-01]]],
[[[ 6.6392e-02, 4.7435e-01, 2.2360e-01],
[ 1.1640e-01, 4.4550e-02, 1.0114e-01],
[ 7.5970e-01, -2.6593e-02, 7.4978e-01]]],
[[[-2.4952e-01, -2.9136e-01, -4.3275e-01],
[-4.3836e-01, 5.7868e-01, -4.1796e-01],
[-3.1188e-01, -4.1504e-01, -6.0209e-01]]],
[[[-7.6740e-03, -1.2746e-01, 7.3953e-02],
[ 1.1245e-01, 2.4729e+00, 2.5669e-01],
[ 2.8247e-02, 1.6268e-01, -7.0686e-04]]],
[[[ 6.0586e-01, -1.2779e-02, 6.1001e-01],
[-9.3175e-02, -3.9671e-01, -1.0740e-01],
[ 6.3908e-01, -1.3216e-01, 5.9539e-01]]],
[[[ 3.3967e-01, -4.4094e-01, 3.3106e-01],
[-2.6672e-01, 1.2313e+00, -2.3391e-01],
[ 2.3279e-01, -2.8770e-01, 3.1637e-01]]],
[[[-3.4574e-01, -1.9476e-01, -4.8610e-01],
[-2.2992e-01, -1.4230e-02, -1.9174e-01],
[-5.4567e-01, -3.5120e-01, -6.8289e-01]]],
[[[ 4.5144e-01, -2.5136e-02, 5.3568e-01],
[ 3.1801e-02, -5.8331e-01, -3.4960e-02],
[-9.1540e-04, -4.8772e-01, -1.8636e-01]]],
[[[ 1.0150e-01, -1.2134e-02, 1.6102e-02],
[-4.1484e-02, 4.3275e-01, -6.8248e-02],
[-7.1185e-01, -7.1240e-01, -7.0851e-01]]],
[[[-3.6291e-01, 1.8500e-01, -4.4183e-01],
[ 8.4609e-02, 5.8029e-01, 1.4827e-01],
[-5.9297e-01, 1.8420e-02, -6.8263e-01]]],
[[[-5.8049e-01, -3.5364e-01, -7.2523e-01],
[-3.7166e-01, -3.1462e-01, -5.6026e-01],
[-2.5259e-01, -2.0127e-01, -4.1163e-01]]],
[[[-4.1470e-01, -2.7154e-01, -3.6831e-01],
[-1.1561e-01, -3.8793e-01, -2.6043e-01],
[-6.1500e-01, -1.8974e-01, -6.4628e-01]]],
[[[-4.7443e-01, -3.7602e-01, -4.7216e-01],
[-3.7285e-01, 2.5685e-01, -1.8080e-01],
[-3.2134e-01, -4.7332e-01, -4.5040e-01]]],
[[[-2.4336e-01, -2.8814e-01, -2.2455e-01],
[-2.7187e-01, 9.4440e-01, -3.6674e-01],
[-2.5768e-01, -2.6512e-01, -2.8699e-01]]],
[[[ 5.1769e-01, -3.3097e-02, 5.4375e-01],
[-5.0911e-02, 5.1187e-01, -2.4701e-01],
[ 5.3332e-01, -1.9179e-01, 6.5832e-01]]],
[[[ 8.7317e-02, -2.7439e-01, -2.3370e-01],
[-1.5015e-02, -1.7981e+00, -2.3003e-01],
[-1.8798e-01, -2.9342e-01, -1.4052e-01]]],
[[[-4.0463e-01, -5.8371e-01, -6.1616e-01],
[-6.3553e-01, 2.8725e-01, -4.3908e-01],
[ 4.6381e-03, -4.6432e-01, 7.3069e-02]]],
[[[-6.9479e-01, -2.8610e-01, -6.3922e-01],
[-3.1537e-02, 1.7567e-01, -4.9486e-04],
[-3.4923e-01, -5.3215e-01, -3.3615e-01]]],
[[[-6.9439e-01, -5.9017e-01, -7.6260e-01],
[-1.4265e-01, -8.5820e-01, -1.9734e-01],
[ 2.0202e-01, 2.4088e-01, 3.1906e-01]]],
[[[ 4.8009e-01, -2.1032e-01, 4.4050e-01],
[ 1.6932e-01, 3.0352e-01, -5.3674e-02],
[ 5.3025e-01, 1.7536e-01, 6.8803e-01]]],
[[[ 2.6775e-01, 8.2845e-01, 1.2976e-01],
[ 4.5156e-01, 2.0780e-01, 2.3832e-01],
[ 4.6654e-01, 2.3515e-01, 3.9078e-01]]],
[[[ 4.9575e-01, -4.8354e-02, 6.2080e-01],
[ 1.2933e-01, 4.5557e-01, -1.2404e-01],
[ 5.6906e-01, 4.3322e-01, 4.7213e-01]]]])

Not working weights of Conv layer
tensor([[[[ 3.3279e-01, -2.0911e-01, 4.0976e-01],
[-1.9482e-01, -2.8142e+00, -2.2299e-01],
[ 3.9004e-01, -1.7411e-01, 4.2168e-01]]],
[[[ 2.7489e-01, 9.7963e-01, 2.2939e-02],
[ 7.0204e-01, 1.3131e+00, 5.6977e-01],
[ 2.9443e-01, 8.0951e-01, 4.5278e-01]]],
[[[ 8.4607e-01, 1.9752e-02, 1.0065e+00],
[ 3.2082e-02, -2.1234e-01, 2.6094e-01],
[ 1.0338e+00, 1.6526e-01, 9.6839e-01]]],
[[[-2.5110e-01, -2.7901e-02, -2.9815e-01],
[ 3.5116e-02, 2.2587e+00, 1.0436e-01],
[-3.0516e-01, -1.5374e-02, -2.8045e-01]]],
[[[-5.4234e-01, 7.3581e-02, -3.2796e-01],
[ 4.3465e-02, 2.4696e+00, -1.4038e-02],
[-2.6090e-01, -6.6863e-02, -2.2878e-01]]],
[[[ 7.7577e-01, 1.6263e-01, 8.0620e-01],
[ 1.2166e-01, -1.3349e+00, 2.2095e-02],
[ 7.8446e-01, 1.8965e-01, 7.7734e-01]]],
[[[ 2.8460e-01, 2.8625e-01, 2.8036e-01],
[ 2.8995e-01, -2.0631e-01, 1.6038e-01],
[ 2.2620e-01, 2.3574e-01, 4.5563e-01]]],
[[[-8.2636e-01, 3.0475e-01, -4.1388e-01],
[ 3.1593e-01, 1.9472e+00, 1.1006e-01],
[-6.6613e-01, 2.9975e-01, -4.6172e-01]]],
[[[-5.3745e-02, -1.6191e-01, -5.6559e-02],
[-6.6030e-02, -5.9331e+00, -1.3726e-01],
[-5.5640e-02, -1.1951e-01, -9.8351e-02]]],
[[[-4.5139e-01, 3.0925e-01, -4.2166e-01],
[ 1.5557e-01, 2.0291e+00, 9.6838e-02],
[-5.0419e-01, 1.4905e-01, -4.8405e-01]]],
[[[-5.9708e-02, -5.9127e-01, 2.7119e-01],
[-7.0508e-01, -1.6482e+00, -6.5309e-01],
[ 7.8688e-02, -1.0563e+00, 9.6375e-02]]],
[[[ 7.2531e-01, 2.1859e-01, 8.1715e-01],
[ 1.4592e-01, -1.3310e+00, 2.8370e-02],
[ 7.2177e-01, 3.3575e-01, 7.5479e-01]]],
[[[-6.9039e-01, -1.6698e-01, -8.5761e-01],
[-7.3998e-02, -2.5820e-01, -3.2171e-02],
[-9.9621e-01, -2.2918e-01, -1.1667e+00]]],
[[[ 1.4400e-01, 1.5582e-01, 4.7007e-01],
[ 6.2186e-03, 1.5049e+00, 1.3891e-01],
[ 5.8765e-01, -9.2874e-02, 1.2079e+00]]],
[[[-4.7601e-01, 1.7371e-01, -3.9373e-01],
[-1.4278e-03, 2.3851e+00, 3.5437e-02],
[-3.5845e-01, 7.5070e-02, -4.3677e-01]]
[[[-3.9381e-01, -2.0094e-01, -3.3219e-01],
[ 1.1169e-01, 1.8181e+00, -3.7194e-01],
[-5.1897e-02, 1.7572e-01, -5.5085e-01]]],
[[[ 8.1767e-01, 5.8831e-01, 7.3225e-01],
[ 4.0319e-01, -8.8078e-01, 5.0771e-01],
[ 8.4507e-01, 7.1296e-01, 7.6765e-01]]],
[[[-6.0087e-02, -1.2139e+00, -1.1047e-03],
[-6.6237e-01, -4.8200e-01, -8.6077e-01],
[-1.8411e-01, -1.1920e+00, -7.4986e-02]]],
[[[-9.4947e-01, -9.8611e-01, -1.8920e-01],
[-8.7728e-02, 9.9390e-01, -3.6914e-01],
[-7.8076e-01, -1.1616e-01, -6.4246e-01]]],
[[[ 5.5130e-01, -3.4770e-01, 6.2945e-01],
[-3.8659e-01, -1.4704e+00, -4.9395e-01],
[ 5.3765e-01, -3.4119e-01, 6.7366e-01]]],
[[[-4.4788e-01, -6.4880e-02, 5.1750e-01],
[-6.9337e-01, -2.2426e+00, -1.9765e-02],
[-6.0051e-01, -2.1296e-01, 2.9482e-01]]],
[[[-2.9910e-01, -5.7485e-01, -5.7792e-01],
[-6.2466e-01, -7.3714e-01, -7.8880e-01],
[-7.6090e-01, -7.5296e-01, -1.1582e+00]]],
[[[-9.3275e-01, -3.5397e-01, -9.9876e-01],
[-2.0837e-01, -2.4731e-01, -2.6931e-01],
[-9.3229e-01, -3.3563e-01, -8.7554e-01]]],
[[[-7.8048e-01, -1.7040e-01, -8.7957e-01],
[-1.0868e-01, 1.3689e+00, -1.6108e-01],
[-8.2380e-01, -2.1541e-01, -8.2356e-01]]],
[[[ 6.6214e-01, -2.8046e-01, 7.4298e-01],
[-2.4776e-01, 2.3627e+00, -1.6467e-01],
[ 5.6969e-01, -2.3224e-01, 6.5499e-01]]],
[[[ 1.0392e+00, 7.8446e-02, 1.1745e+00],
[-1.9321e-01, -5.8134e-01, -2.2323e-01],
[ 7.3575e-01, 6.3480e-02, 1.2036e+00]]],
[[[-7.4549e-01, 3.5171e-01, -7.4779e-01],
[ 2.7267e-01, -1.0939e+00, 3.1744e-01],
[-8.6094e-01, 1.8053e-01, -9.5986e-01]]],
[[[-7.4761e-02, -7.4965e-02, -4.8395e-03],
[-7.0916e-02, -9.4117e-02, 6.8479e-03],
[-7.0671e-02, -5.7122e-02, 1.1384e-03]]],
[[[ 2.0594e-02, -2.6332e-01, -2.0066e-01],
[ 3.5086e-01, 2.8736e+00, 2.2054e-01],
[ 1.2846e-01, 1.2652e-01, -2.1748e-01]]],
[[[ 4.8172e-01, 4.0160e-01, 5.4478e-01],
[ 4.1767e-01, -1.5756e+00, 4.4751e-01],
[ 4.5373e-01, 4.2261e-01, 4.6070e-01]]],
[[[-2.4105e-01, 6.2362e-02, 2.6139e-01],
[-4.1651e-01, -1.4579e-01, -6.3829e-01],
[-5.3446e-01, -7.7817e-01, -7.7269e-01]]],
[[[ 9.1111e-02, 7.7725e-01, 7.9571e-01],
[ 4.4018e-01, 1.0821e+00, 2.7830e-01],
[ 5.1079e-01, 5.2820e-01, 4.4008e-01]]]])

cc @jerryzh168 @jianyuh @dzhulgakov @raghuramank100 @jamesr66a