Issue 1:
Issue description

Hello! I am trying to install pytorch from source in conda, and I got these error message:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:31:17: error: ‘ncclInt8’ was not declared in this scope

{at::kChar, ncclInt8},

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:32:17: error: ‘ncclUint8’ was not declared in this scope

{at::kByte, ncclUint8},

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:35:16: error: ‘ncclInt32’ was not declared in this scope

{at::kInt, ncclInt32},

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:38:1: error: could not convert ‘{{at::kChar, }, {at::kByte, }, {at::kFloat, ncclFloat}, {at::kDouble, ncclDouble}, {at::kInt, }, {at::kLong, ncclInt64}, {at::kHalf, ncclHalf}}’ from ‘’ to ‘std::unordered_map<at::ScalarType, ncclDataType_t>’

};

^

In file included from /home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:1:0:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘void thd::DataChannelNccl::_destroyNcclResources(THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:187:40: error: void value not ignored as it ought to be

NCCL_CHECK(ncclCommDestroy(comm));

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘thd::NcclResourcePair thd::DataChannelNccl::_getNcclResourcePair(std::vectorat::Tensor&, THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:295:29: error: ‘ncclGroupStart’ was not declared in this scope

NCCL_CHECK(ncclGroupStart());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:302:27: error: ‘ncclGroupEnd’ was not declared in this scope

NCCL_CHECK(ncclGroupEnd());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::allReduce(std::vectorat::Tensor&, THDReduceOp, THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:417:29: error: ‘ncclGroupStart’ was not declared in this scope

NCCL_CHECK(ncclGroupStart());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:432:27: error: ‘ncclGroupEnd’ was not declared in this scope

NCCL_CHECK(ncclGroupEnd());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::allGather(std::vectorat::Tensor&, std::vectorat::Tensor&, THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:466:29: error: ‘ncclGroupStart’ was not declared in this scope

NCCL_CHECK(ncclGroupStart());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:473:27: error: invalid conversion from ‘void*’ to ‘int’ [-fpermissive]

output[i].data_ptr(),

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:474:23: error: invalid conversion from ‘int64_t {aka long int}’ to ‘ncclDataType_t’ [-fpermissive]

input[i].numel(),

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:477:15: error: cannot convert ‘ncclDataType_t’ to ‘void*’ for argument ‘4’ to ‘ncclResult_t ncclAllGather(const void*, int, ncclDataType_t, void*, ncclComm_t, cudaStream_t)’

stream));

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:480:27: error: ‘ncclGroupEnd’ was not declared in this scope

NCCL_CHECK(ncclGroupEnd());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::reduce(std::vectorat::Tensor&, THDReduceOp, thd::rank_type, THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:516:29: error: ‘ncclGroupStart’ was not declared in this scope

NCCL_CHECK(ncclGroupStart());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:532:27: error: ‘ncclGroupEnd’ was not declared in this scope

NCCL_CHECK(ncclGroupEnd());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp: In member function ‘virtual void thd::DataChannelNccl::broadcast(std::vectorat::Tensor&, thd::rank_type, THDGroup)’:

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:568:29: error: ‘ncclGroupStart’ was not declared in this scope

NCCL_CHECK(ncclGroupStart());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd;

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.cpp:582:27: error: ‘ncclGroupEnd’ was not declared in this scope

NCCL_CHECK(ncclGroupEnd());

^

/home/lin/pytorch/torch/lib/THD/base/data_channels/DataChannelNccl.hpp:16:26: note: in definition of macro ‘NCCL_CHECK’

ncclResult_t error = cmd; \

I have searched on google. And I find it may be the problem of nccl. I run

export NO_SYSTEM_NCCL=1

but it didn't work.

System Info

PyTorch

How you installed PyTorch (conda, pip, source):conda

Build command you used (if compiling from source):python setup install

OS: Ubuntu 16.04

PyTorch version: 1.0.0

Python version: 3.7.1

CUDA/cuDNN version: Cuda8 cudnn 7.0.5

GPU models and configuration: 1080Ti

GCC version (if compiling from source): gcc 5.3.1

CMake version: 3.12.2

Issue 2:
We currently only support int varargs when they are the only argument in the Python torch wrapping. This limits, for example, the use of varargs in einsum.

The root cause is in the pattern for using PythonArgParser with the templated ParsedArgs as documented in the comment at the top of torch/csrc/utils/python_arg_parser.h.

The fact that ParsedArgs are accessible without PythonArgParser keeps us from doing Python object allocations (for a slice of arguments to assign to the vararg) that could be cleaned up in the destructor of PythonArgParser without feeling dirty.

I see two obivous ways out:

move the ParsedArgs structure into PythonArgParser and make that templated (but I would guess that there was a reason to not do that all along - possibly duplicated functions)

add a std::vector<PyObject*> varargs to ParsedArgs that gets the varargs.

Would either of these be acceptable?

Best regards

Thomas