Issue 1:
🐛 Bug

Product of ones is zero on cuda. This seems like a rather common issue, but i found no mentions of it here after a brief keyword search.

To reproduce

import torch
print(torch.__version__)
print(torch.prod(torch.ones(10 ** 6, device='cuda')))

prints

1.0.1.post2
tensor(0., device='cuda:0')

Expected behavior

I'd expect it to be 1 :)

Environment

Collecting environment information...
PyTorch version: 1.0.1.post2
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.1 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0 20160609
CMake version: version 3.5.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration:
GPU 0: Tesla M40
GPU 1: Tesla M40
GPU 2: Tesla M40
GPU 3: Tesla M40

Nvidia driver version: 396.37
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.6.0.21
/usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.4

Versions of relevant libraries:
[pip] numpy==1.16.3
[pip] numpydoc==0.7.0
[pip] torch==1.0.1.post2
[pip] torchvision==0.2.1
[conda] blas 1.0 mkl
[conda] faiss-gpu 1.4.0 py36_cuda8.0.61_1 pytorch
[conda] mkl 2018.0.1 h19d6760_4
[conda] mkl-service 1.1.2 py36h17a0993_4
[conda] pytorch 1.0.1 py3.6_cuda8.0.61_cudnn7.1.2_2 pytorch
[conda] torch 1.0.1.post2 <pip>
[conda] torchvision 0.2.1 py_2 pytorch

Additional context

This does not seem to be a floating point precision issue:

print(torch.prod(torch.ones(130_560, device='cuda'), dim=0))
print(torch.prod(torch.ones(130_561, device='cuda'), dim=0))

prints

tensor(1., device='cuda:0')
tensor(0., device='cuda:0')

Issue 2:
🐛 Bug

Potentially relates to #15602.

To Reproduce

Steps to reproduce the behavior:

>>> import torch
>>> x = torch.ones([20, 30, 400], dtype=torch.uint8, device='cpu')
>>> x.prod()
tensor(1)
>>> x = torch.ones([20, 30, 400], dtype=torch.uint8, device='cuda')
>>> x.prod()
tensor(0, device='cuda:0')

The results on small Tensors are correct:

>>> x = torch.ones([2, 3, 4], dtype=torch.uint8, device='cuda')
>>> x.prod()
tensor(1, device='cuda:0')

Expected behavior

>>> x = torch.ones([20, 30, 400], dtype=torch.uint8, device='cuda')
>>> x.prod()
tensor(1, device='cuda:0')

Environment

PyTorch version: 1.1.0a0+a0d22b6

Is debug build: No

CUDA used to build PyTorch: 9.2.88

OS: CentOS Linux 7 (Core)

GCC version: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)

CMake version: version 3.12.2

Python version: 3.7

Is CUDA available: Yes

CUDA runtime version: 9.2.88

GPU models and configuration:

GPU 0: Tesla M40

GPU 1: Tesla M40

Nvidia driver version: 396.26

cuDNN version: Probably one of the following:

/usr/local/fbcode/gcc-5-glibc-2.23/lib/libcudnn.so.6

/usr/local/fbcode/gcc-5-glibc-2.23/lib/libcudnn.so.6.0.20

/usr/local/fbcode/gcc-5-glibc-2.23/lib/libcudnn.so.6.0.21

/usr/local/fbcode/gcc-5-glibc-2.23/lib/libcudnn.so.7.1.2

/usr/local/fbcode/gcc-5-glibc-2.23/lib/libcudnn.so.7.1.4

/usr/local/fbcode/platform007/lib/libcudnn.so.7.1.4

Versions of relevant libraries:

[pip] numpy (1.15.4)

[pip] torch (1.1.0a0+a0d22b6, /home/shenli/project/pytorch)

[conda] blas 1.0 mkl

[conda] mkl 2019.1 144

[conda] mkl-include 2019.1 144

[conda] mkl_fft 1.0.6 py37hd81dba3_0

[conda] mkl_random 1.0.2 py37hd81dba3_0

[conda] torch 1.1.0a0+a0d22b6