Problem compiling pytorch in WITH_DISTRIBUTED=1 mode

dvaldes: Hello everyone, I am using a debian 8.6.0, cmake 3.7.0, and python-3.5.2 and  I’m really very lost … THE ERROR: <SCODE>/usr/include/string.h:66:14: note:   ‘memset’
 extern void *memset (void *__s, int __c, size_t __n) __THROW __nonnull ((1));
              ^
CMakeFiles/THD.dir/build.make:398: recipe for target 'CMakeFiles/THD.dir/master_worker/master/THDStorage.cpp.o' failed
make[2]: *** [CMakeFiles/THD.dir/master_worker/master/THDStorage.cpp.o] Error 1
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp: In member function ‘virtual void thd::DataChannelMPI::send(const thd::Scalar&, int)’:
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:329:52: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
            MPI_UINT8_T, dst_rank, 0, MPI_COMM_WORLD);
                                                    ^
In file included from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.hpp:5:0,
                 from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:1:
/usr/lib/openmpi/include/mpi.h:1384:20: note: initializing argument 1 of ‘int MPI_Send(void*, int, MPI_Datatype, int, int, MPI_Comm)’
 OMPI_DECLSPEC  int MPI_Send(void *buf, int count, MPI_Datatype datatype, int dest,
                    ^
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp: In member function ‘virtual void thd::DataChannelMPI::send(thpp::Tensor&, int)’:
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:340:52: error: invalid conversion from ‘const void*’ to ‘void*’ [-fpermissive]
            MPI_UINT8_T, dst_rank, 0, MPI_COMM_WORLD);
                                                    ^
In file included from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.hpp:5:0,
                 from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:1:
/usr/lib/openmpi/include/mpi.h:1384:20: note: initializing argument 1 of ‘int MPI_Send(void*, int, MPI_Datatype, int, int, MPI_Comm)’
 OMPI_DECLSPEC  int MPI_Send(void *buf, int count, MPI_Datatype datatype, int dest,
                    ^
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp: In member function ‘virtual THDGroup thd::DataChannelMPI::newGroup(const std::vector<int>&)’:
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:476:56: error: invalid conversion from ‘const int*’ to ‘int*’ [-fpermissive]
   MPI_Group_incl(world_group, ranks.size(), ranks.data(), &ranks_group);
                                                        ^
In file included from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.hpp:5:0,
                 from /soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:1:
/usr/lib/openmpi/include/mpi.h:1269:20: note: initializing argument 3 of ‘int MPI_Group_incl(MPI_Group, int, int*, ompi_group_t**)’
 OMPI_DECLSPEC  int MPI_Group_incl(MPI_Group group, int n, int *ranks,
                    ^
/soft/pytorch-dist/torch/lib/THD/base/data_channels/DataChannelMPI.cpp:479:66: error: ‘MPI_Comm_create_group’ was not declared in this scope
   MPI_Comm_create_group(MPI_COMM_WORLD, ranks_group, 0, &new_comm);
                                                                  ^
CMakeFiles/THD.dir/build.make:422: recipe for target 'CMakeFiles/THD.dir/master_worker/master/THDTensor.cpp.o' failed
make[2]: *** [CMakeFiles/THD.dir/master_worker/master/THDTensor.cpp.o] Error 1
CMakeFiles/THD.dir/build.make:158: recipe for target 'CMakeFiles/THD.dir/base/data_channels/DataChannelMPI.cpp.o' failed
make[2]: *** [CMakeFiles/THD.dir/base/data_channels/DataChannelMPI.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/THD.dir/all' failed
make[1]: *** [CMakeFiles/THD.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
<ECODE> I follow this instructions: Using Python 3 (Python 3.4) Install build dependencies Essentials <SCODE>sudo apt-get update
sudo apt-get install git build-essential
ccache

sudo apt-get install ccache
export CC="ccache gcc"
export CXX="ccache g++"
<ECODE> CMake The default CMake version in Debian’s repositories is too old. Ubuntu 16.10 has version 3.5.2 and it works fine. <SCODE>wget https://cmake.org/files/v3.7/cmake-3.7.0.tar.gz
tar xf cmake-3.7.0.tar.gz
rm cmake-3.7.0.tar.gz
cd cmake-3.7.0
./bootstrap
make
sudo make install
cd ..
<ECODE> Install THD dependencies Asio C++ Library <SCODE>sudo apt-get install libasio-dev
<ECODE> MPI implementation <SCODE>sudo apt-get install mpich
<ECODE> Set up Python <SCODE>sudo apt-get install python3-dev python3-pip
<ECODE> Set up virtual environment <SCODE>sudo pip3 install virtualenv
virtualenv venv
source venv/bin/activate
<ECODE> Install PyTorch <SCODE>export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$HOME/pytorch-dist/torch/lib"

git clone https://github.com/apaszke/pytorch-dist/
cd pytorch-dist
pip3 install -r requirements.txt
WITH_DISTRIBUTED=1 python3 setup.py build_deps
WITH_DISTRIBUTED=1 python3 setup.py develop
<ECODE> Thanks a lot for your help. Dani

apaszke: Thanks, we’ll look into it. However, note that the distributed package is still in pre-alpha and will be likely slow or can break in weird ways. We’ll notify everyone once it’s ready for use.

dvaldes: Thanks for your answer, Adam . Exactly, which are the requirements for pytorch to work? I mean, versions of the operating system, gcc, cmake, cuda, etc. etc. We are a little lost, but we think that the problem in compilation comes from cuda… Thanks a lot!

smth: 

apaszke: The problem is because of MPI. Installing it might help.

kirk86: Thank you guys for all your work. If I may, I would like to remind you also of users like me where in most of the cases we don’t have root privileges on the machines we operate upon (e.g. clusters). So, whenever the pytorch.distribute package rolls out, I would like to request, if possible to also make it available for easy installation via anaconda or pip which might take care also of the dependencies. Thanks again, and keep up the good work! Cheers.