Issue 1:
🐛 Bug

Running a 2d convolution (no padding, stride=1) on a 3x1 input, sliced from a 3x3 tensor, and a 3x1 weight, sliced from a 3x3 tensor. When trying to run it on CPU the convolution result is wrong, but when moving to GPU (by adding .cuda() after the tensors), the result is correct!

To Reproduce

Run the code below to reproduce the bug. It's pretty self explanatory.

import torch
import torch.nn.functional as F

x = torch.Tensor([[[[0., 2., 2.],
[0., 0., 1.],
[2., 0., 0.]]]])
w = torch.Tensor([[[[1., 1., 0.],
[1., 0., 3.],
[0., 3., 1.]]]])

x1 = x[:,:,:,2:] # Using only the last column from x, containing [2,1,0]
w1 = w[:,:,:,2:] # Using only the last column from w, containing [0,3,1]
#The result should be ((2*0) + (1*3) + (0*1)) = 3

print(F.conv2d(x1, w1, None, padding=0, stride=1)) # tensor([[[[1.]]]]) WRONG!!!!
print(F.conv2d(x1.cuda(), w1.cuda(), None, padding=0, stride=1)) # tensor([[[[3.]]]], device='cuda:0') as expected

Expected behavior

As seen in the code above, the CPU gives the result of 1, when the actual result should be 3.

Environment

PyTorch Version (e.g., 1.0): 1.0

OS (e.g., Linux): Ubuntu 16.04

How you installed PyTorch (conda, pip, source): pip

Build command you used (if compiling from source):

Python version: 3.5.2

CUDA/cuDNN version: 9.0

GPU models and configuration: GeForce GTX 1060

Any other relevant information:

Additional context

Issue 2:
🐛 Bug

If I load convolution weights from a NumPy float32 array, the result of the forward pass does not match that of a layer with the same weights loaded from a NumPy float64 array or of a Keras convolution layer with the same weights. Casting the layer to double fixes the discrepancy, even after casting back to float. Comparing the convolution weights before and after casting shows that they have not changed.

To Reproduce

import numpy as np
import torch
from torch import nn

d_in = 25
d_out = 64
k = 7
B = 2
L = 10

conv = nn.Conv1d(d_in, d_out, k, stride=2, padding=0)
pad = nn.ConstantPad1d((2, 3), 0)
np.random.seed(1034)
X = np.random.random((B, L, d_in))
X = torch.as_tensor(X).transpose(-1, -2)
X = pad(X)
w = np.random.random((k, d_in, d_out))
w = w.astype('float32')
b = np.random.random((d_out, ))

w = torch.as_tensor(w).permute(2, 1, 0).float()
b = torch.as_tensor(b).float()

conv.weight = nn.Parameter(w)
conv.bias = nn.Parameter(b)

out1 = conv(X.float()).detach().numpy()
w1 = conv.weight.detach().numpy()
out2 = conv.double()(X).detach().numpy()
w2 = conv.weight.detach().numpy()
out3 = conv.float()(X.float()).detach().numpy()
w3 = conv.weight.detach().numpy()
print('w == w1?')
print(np.allclose(w.detach().numpy(), w1))
print('w1 == w2?')
print(np.allclose(w1, w2))
print('w2 == w3?')
print(np.allclose(w2, w3))
print('out1 == out2?')
print(np.allclose(out1, out2, atol=1e-6))
print('out2 == out3?')
print(np.allclose(out2, out3, atol=1e-6))

Result:

w == w1?
True
w1 == w2?
True
w2 == w3?
True
out1 == out2?
False
out2 == out3?
True

Some notes:

The type for the bias doesn't seem to matter.

out2 and out3 match the result for a Keras convolution layer with the same weights.

Expected behavior

out1 should be within machine precision of out2 and out3.

Environment

PyTorch version: 1.0.0

Is debug build: No

CUDA used to build PyTorch: None

OS: Mac OSX 10.14.2

GCC version: Could not collect

CMake version: Could not collect

Python version: 3.7

Is CUDA available: No

CUDA runtime version: No CUDA

GPU models and configuration: No CUDA

Nvidia driver version: No CUDA

cuDNN version: No CUDA

Versions of relevant libraries:

[pip] Could not collect

[conda] blas 1.0 mkl

[conda] mkl 2019.1 144

[conda] mkl-service 1.1.2 py37hfbe908c_5

[conda] mkl_fft 1.0.6 py37h27c97d8_0

[conda] mkl_random 1.0.2 py37h27c97d8_0

[conda] pytorch 1.0.0 py3.7_1 pytorch

[conda] torchvision 0.2.1 py_2 pytorch

[conda] numpy 1.15.4