Change:
- [NVIDIA CUDA](https://developer.nvidia.com/cuda-downloads) 7.5 or above - [NVIDIA CuDNN](https://developer.nvidia.com/cudnn) v5.x <ins> If you want to disable CUDA support, export environment variable `NO_CUDA=1`. </ins> #### Install optional dependencies On Linux

Discussion:
Caffe and TensorFlow supports both CPUONLY and CUDA mode. The meaning of CPUONLY mode is to enable users to run pytorch on architectures except for AMD64 and PPC64EL (Seems that CUDA is only available on AMD64 and PPC64EL). For example, the CPU_ONLY version of is available for (not only) amd64, armel, i386, ppc64el and s390x, while the CUDA version only supports amd64 and ppc64el.
I think we don't plan to invest too much time in supporting many different architectures. The vast majority of the users is on x86 and this will remain our main focus. If someone is willing to spend their time supporting other archs, we'll happily accept PRs, but we have a lot of other priorities. We also support CPU only and CUDA mode. You can use or build pytorch without any GPU. We also have CPU-only binaries (see the website). You can force building without CUDA with env variable, as I pointed out in another PR (it's broken because of THPP at the moment, but I'll fix it soon). The only limitation for supporting different archs that I see is what our backends are limited to, and the fact that you need a C++11 compiler to build the extension. Apart from this, I don't think we depend on x86 anywhere. Can you please elaborate on what kind of guidance would you like to see? More detailed instructions on how to build pytorch somewhere in the docs?
As reported in [1], Pytorch's backend library (libTH) is now healthy in portability. I suggest that the environment variable should be referred in . [1]

I'm trying to implement distributed adversarial training in PyTorch. Thus, in my program pipeline I need to forward the output of one DDP model to another one. When I run the code in distributed setting, the following error is throwed: frame : c10::Error::Error(c10::SourceLocation, std::string const&) + 0x45 (0x7fa248f906d5 in /export/home/haoran/anaconda3/envs/torch1.2/lib/python3.6/site-) frame : c10d::Reducer::prepareforbackward(std::vector<torch::autograd::Variable, std::allocator<torch::autograd::Variableconst&) + 0x666 (0x7fa26f9d0896 in /export/home/haoran/anaconda3/envs/torch1.2/lib/python3.6/site-) frame : <unknown function+ 0x6df3b8 (0x7fa26f9c43b8 in /export/home/haoran/anaconda3/envs/torch1.2/lib/python3.6/site-) frame : <unknown function+ 0x1fc7f0 (0x7fa26f4e17f0 in /export/home/haoran/anaconda3/envs/torch1.2/lib/python3.6/site-) frame : main + 0x16d (0x400c1d in /export/home/haoran/anaconda3/envs/torch1.2/bin/python) frame : _libcstartmain + 0xf5 (0x7fa285e06c05 in ) frame : /export/home/haoran/anaconda3/envs/torch1.2/bin/python() [0x4009e9] Running the following code with DistributedDataParallel <!-- If you have a code sample, error messages, stack traces, please provide it here as well --The code could run successfully without throwing error. Please copy and paste the output from our (or fill out the checklist below manually).

You can get the script and run it with: PyTorch version: 1.2.0.dev20190701 Is debug build: No CUDA used to build PyTorch: 9.0.176 OS: CentOS Linux 7 (Core) GCC version: (GCC) 5.4.0 CMake version: version 3.6.2 Python version: 3.6 Is CUDA available: Yes CUDA runtime version: 9.0.176 GPU models and configuration: GPU 0: Tesla P40 GPU 1: Tesla P40 GPU 2: Tesla P40 GPU 3: Tesla P40 Nvidia driver version: 390.25 cuDNN version: Could not collect Versions of relevant libraries: [pip] numpy==1.16.4 [pip] torch==1.2.0.dev20190701 [conda] blas 1.0 mkl defaults [conda] mkl 2019.4 243 defaults [conda] mklfft 1.0.12 py36ha843d7b0 defaults [conda] mklrandom 1.0.2 py36hd81dba30 defaults [conda] pytorch-nightly 1.2.0.dev20190701 py3.6cuda9.0.176cudnn7.5.1_0 pytorch <!-- Add any other context about the problem here. --
Please, try one of the suggested options: (1) passing the keyword argument findunusedparameters=True to ; (2) making sure all forward function outputs participate in calculating loss. If you already have done the above two steps, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's forward function. Please include the loss function and the structure of the return value of forward of your module when reporting this issue (e.g. list, dict, iterable). If the issue will still exist, please, reopen this issue.
I double checked your repro to figure out if this is a real issue or not and found that even setting won't help you here. It's a simple example of course, but you're not using the variable in your loss computation. The model's forward function returns it which is an indication for DDP that it will receive a gradient.

Then when it doesn't, and you try to run the next iteration, it raises an error, because the previous iteration didn't finish. I recommend you return only the tensor you plan to use in the loss computation.
Is there any way to see which parameters on earth are unused? It is useless to simply give this RuntimeError or set findunusedparameters to True.
I had the same problem with just one GPUI have modified the code according to what you said
I met the same problem and I found this topic. It works for me. (though it's been a long time for you)