Issue 1:
🐛 Bug

Pytorch-1.0-Cuda-10.0 with ONNX-1.2.1 and 1.3.0. Export onnx model core dumps on Ubuntu on GPU instance

To Reproduce

Steps to reproduce the behavior:

Spin up an Ubuntu with GPU instance like EC2 p2 or g3

install Cuda-10, cudnn-7.4.1, NCCL-2.3.7

anaconda

conda create -n pytorch_p27 python=2.7

conda install pytorch=1.0.0=py2.7_cuda10.0.130_cudnn7.4.1_1 torchvision=0.2.1 cuda100=1.0 -c pytorch

pip install -U onnx==1.2.1 or 1.3.0

example used to replicate:

https://github.com/onnx/tutorials/blob/master/tutorials/PytorchOnnxExport.ipynb

Error:

Illegal instruction (core dumped)

Expected behavior

It should have exported a .onnx file with the model

Environment

Please copy and paste the output from our

environment collection script

PyTorch version: 1.0.0

Is debug build: No

CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 16.04.5 LTS

GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609

CMake version: version 3.5.1

Python version: 2.7

Is CUDA available: Yes

CUDA runtime version: 10.0.130

GPU models and configuration:

GPU 0: Tesla M60

GPU 1: Tesla M60

GPU 2: Tesla M60

GPU 3: Tesla M60

Nvidia driver version: 410.79

cuDNN version: Probably one of the following:

/usr/local/cuda-10.0/lib64/libcudnn.so.7.4.1

/usr/local/cuda-10.0/lib64/libcudnn_static.a

/usr/local/cuda-8.0/lib64/libcudnn.so.6.0.21

/usr/local/cuda-8.0/lib64/libcudnn_static.a

/usr/local/cuda-9.0/lib64/libcudnn.so.7.3.1

/usr/local/cuda-9.0/lib64/libcudnn_static.a

/usr/local/cuda-9.2/lib64/libcudnn.so.7.3.1

/usr/local/cuda-9.2/lib64/libcudnn_static.a

Versions of relevant libraries:

[pip] Could not collect

[conda] blas 1.0 mkl

[conda] cuda100 1.0 0 pytorch

[conda] mkl 2018.0.3 1

[conda] mkl_fft 1.0.6 py27h7dd41cf_0

[conda] mkl_random 1.0.1 py27h4414c95_1

[conda] pytorch 1.0.0 py2.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch

[conda] torchvision 0.2.1 py_2 pytorch

Additional context

complete list of packages

packages in environment:

Name Version Build Channel

blas 1.0 mkl

ca-certificates 2018.03.07 0

certifi 2018.10.15 py27_0

cffi 1.11.5 py27he75722e_1

cuda100 1.0 0 pytorch

freetype 2.9.1 h8a8886c_1

intel-openmp 2019.1 144

jpeg 9b h024ee3a_2

libedit 3.1.20170329 h6b74fdf_2

libffi 3.2.1 hd88cf55_4

libgcc-ng 8.2.0 hdf63c60_1

libgfortran-ng 7.3.0 hdf63c60_0

libpng 1.6.35 hbc83047_0

libstdcxx-ng 8.2.0 hdf63c60_1

libtiff 4.0.9 he85c1e1_2

mkl 2018.0.3 1

mkl_fft 1.0.6 py27h7dd41cf_0

mkl_random 1.0.1 py27h4414c95_1

ncurses 6.1 he6710b0_1

ninja 1.8.2 py27h6bb024c_1

numpy 1.15.4 py27h1d66e8a_0

numpy-base 1.15.4 py27h81de0dd_0

olefile 0.46 py27_0

onnx 1.3.0

openssl 1.1.1a h7b6447c_0

pillow 5.3.0 py27h34e0f95_0

pip 18.1 py27_0

protobuf 3.6.1

pycparser 2.19 py27_0

python 2.7.15 h9bab390_4

pytorch 1.0.0 py2.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch

readline 7.0 h7b6447c_5

setuptools 40.6.2 py27_0

six 1.11.0 py27_1

sqlite 3.25.3 h7b6447c_0

tk 8.6.8 hbc83047_0

torchvision 0.2.1 py_2 pytorch

typing 3.6.6

typing-extensions 3.6.6

wheel 0.32.3 py27_0

xz 5.2.4 h14c3975_4

zlib 1.2.11 h7b6447c_3

Issue 2:
🐛 Bug

Pytorch-1.0-Cuda-10.0 with ONNX-1.2.1 and 1.3.0. Export onnx model core dumps on Ubuntu on GPU instance

To Reproduce

Steps to reproduce the behavior:

Spin up an Ubuntu with GPU instance like EC2 p2 or g3

install Cuda-10, cudnn-7.4.1, NCCL-2.3.7

anaconda

conda create -n pytorch_p27 python=2.7

conda install pytorch=1.0.0=py2.7_cuda10.0.130_cudnn7.4.1_1 torchvision=0.2.1 cuda100=1.0 -c pytorch

pip install -U onnx==1.2.1 or 1.3.0

example used to replicate:

https://github.com/onnx/tutorials/blob/master/tutorials/PytorchOnnxExport.ipynb

Error:

Illegal instruction (core dumped)

Expected behavior

It should have exported a .onnx file with the model

Environment

Please copy and paste the output from our

environment collection script

PyTorch version: 1.0.0

Is debug build: No

CUDA used to build PyTorch: 10.0.130

OS: Ubuntu 16.04.5 LTS

GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609

CMake version: version 3.5.1

Python version: 2.7

Is CUDA available: Yes

CUDA runtime version: 10.0.130

GPU models and configuration:

GPU 0: Tesla M60

GPU 1: Tesla M60

GPU 2: Tesla M60

GPU 3: Tesla M60

Nvidia driver version: 410.79

cuDNN version: Probably one of the following:

/usr/local/cuda-10.0/lib64/libcudnn.so.7.4.1

/usr/local/cuda-10.0/lib64/libcudnn_static.a

/usr/local/cuda-8.0/lib64/libcudnn.so.6.0.21

/usr/local/cuda-8.0/lib64/libcudnn_static.a

/usr/local/cuda-9.0/lib64/libcudnn.so.7.3.1

/usr/local/cuda-9.0/lib64/libcudnn_static.a

/usr/local/cuda-9.2/lib64/libcudnn.so.7.3.1

/usr/local/cuda-9.2/lib64/libcudnn_static.a

Versions of relevant libraries:

[pip] Could not collect

[conda] blas 1.0 mkl

[conda] cuda100 1.0 0 pytorch

[conda] mkl 2018.0.3 1

[conda] mkl_fft 1.0.6 py27h7dd41cf_0

[conda] mkl_random 1.0.1 py27h4414c95_1

[conda] pytorch 1.0.0 py2.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch

[conda] torchvision 0.2.1 py_2 pytorch

Additional context

complete list of packages

packages in environment:

Name Version Build Channel

blas 1.0 mkl

ca-certificates 2018.03.07 0

certifi 2018.10.15 py27_0

cffi 1.11.5 py27he75722e_1

cuda100 1.0 0 pytorch

freetype 2.9.1 h8a8886c_1

intel-openmp 2019.1 144

jpeg 9b h024ee3a_2

libedit 3.1.20170329 h6b74fdf_2

libffi 3.2.1 hd88cf55_4

libgcc-ng 8.2.0 hdf63c60_1

libgfortran-ng 7.3.0 hdf63c60_0

libpng 1.6.35 hbc83047_0

libstdcxx-ng 8.2.0 hdf63c60_1

libtiff 4.0.9 he85c1e1_2

mkl 2018.0.3 1

mkl_fft 1.0.6 py27h7dd41cf_0

mkl_random 1.0.1 py27h4414c95_1

ncurses 6.1 he6710b0_1

ninja 1.8.2 py27h6bb024c_1

numpy 1.15.4 py27h1d66e8a_0

numpy-base 1.15.4 py27h81de0dd_0

olefile 0.46 py27_0

onnx 1.3.0

openssl 1.1.1a h7b6447c_0

pillow 5.3.0 py27h34e0f95_0

pip 18.1 py27_0

protobuf 3.6.1

pycparser 2.19 py27_0

python 2.7.15 h9bab390_4

pytorch 1.0.0 py2.7_cuda10.0.130_cudnn7.4.1_1 [cuda100] pytorch

readline 7.0 h7b6447c_5

setuptools 40.6.2 py27_0

six 1.11.0 py27_1

sqlite 3.25.3 h7b6447c_0

tk 8.6.8 hbc83047_0

torchvision 0.2.1 py_2 pytorch

typing 3.6.6

typing-extensions 3.6.6

wheel 0.32.3 py27_0

xz 5.2.4 h14c3975_4

zlib 1.2.11 h7b6447c_3