Change:
I'm trying to run a variant of ERFNet on an NVIDIA TX-2 running Jetpack 3.2 (CUDA 9.0 and CuDNN 7). I get the following error: RuntimeError: cuda runtime error (7) : too many resources requested for launch at ../../../pytorch/torch/lib/THCUNN/im2col.h:120 This error indicates the size of the model + overhead is too large for the GPU? But this is roughly a 700mb model trying to perform inference on a single 640x512 grayscale image on a GPU that has roughly 6.5Gb of free space. I even tried training a new model on images at half that resolution and get the same error. Any tips/feedback is appreciated. PyTorch or Caffe2: PyTorch How you installed PyTorch (conda, pip, source): Source Build command you used (if compiling from source): Followed jetson-reinforcement github OS: Ubuntu 16.04 on Nvidia TX2 PyTorch version: 0.3.0 Python version: 3.6 CUDA/cuDNN version: CUDA 9.0 , CUDNN 7 GPU models and configuration: Tegra X2 GCC version (if compiling from source): 5.4 CMake version: 3.5.1 Versions of any other relevant libraries:
we should fix the launch parameters. I presume we cant use as many threads per block on the TX2 as we use on desktop GPUs. can you tell us what the limits of TX2 GPUs are for the fix to im2col
Threads per block and maximum blocks in the grid are actually the same for TX2 as they are for desktop GPUs , only number of registers is smaller. do you know what might be causing this?
as said, the number of threads per block is constant across different NVIDIA GPUs, the Tegra series included. Is it possible that it was compiled to require more registers than is available to the TX2, and maybe the kernel invocation of im2col requires some sort of launch_bounds() command?
TX2 is arm64 platform. I presume pytorch is compiled from source. In this case, there isn't a chance that it was compiled to requre more registers than available on TX2 I think. The build log will be helpful to see.
thanks for the quick response. I will try dig up the build log tomorrow and post it here.
if there are no launch bounds, it is in fact possible that a kernel is compiled to request more registers than available. At the compile time, compiler does not know how many threads you'll want to launch with, so, potentially, it can use too many registers per thread to later satisfy runtime requirements, and, e.g. trying to launch with 512 or 1024 threads could fail (not even a single block can be put on an SM), whereas launching with 256 would succeed.
is there a way where we can audit that GPU constraints on our server setting (i.e. not actually sitting and compiling TX2 PyTorch)
Adding launch_bounds with the max number of threads the kernel is going to be launched with will cause compiler not to overuse registers. We had to do it e.g. for interp kernels when cuda 9 started using more registers
Ahh, that was as I suspected Thank you for clearing this up. do you guys think a fix will be released in the near future (week?). Thanks for all the help
Works now, thanks!
hello, I met the same error on tx2, complied pytorch 0.3.0.. But the error is this: 'cuda runtime error(7): too many resources requested for launch at '. The file is different with '', is that I need add this line 'launch_bounds(1024)' to every function in file ''? Thank you!
Hello I am also facing a similar issue: Here's my setup: How you installed PyTorch (conda, pip, source): Source Build command you used (if compiling from source): Followed jetson-reinforcement github OS: Ubuntu 16.04 on Nvidia TX2 PyTorch version: 0.5.0a0+ (torchvision 0.2.1) Python version: 3.5 CUDA/cuDNN version: CUDA 9.0 , CUDNN 7 GPU models and configuration: Tegra X2 GCC version (if compiling from source): 5.4 CMake version: 3.12.2 Source code: Full log: Do you know what it could be?
I know that '' without launch_bounds(1024) leads to this error, but I don't know how to fix it... Tried this solution , but still not working (after compilation)
My issue was solved by following
Hi, I have the same problem. Can this problem solved by gitting clone the new pytorch, than changing CUDANUMTHREADS =256 in the below two file and compiling it? !
that appears to fix some of the functions, but not perhaps all - not sure. For more info, see:

Discussion:
// Kernel for fast unfold+copy // (borrowed from Caffe: https://github.com/BVLC/caffe/blob/master/src/caffe/layers/conv_layer.cu) template <typename Dtype> <ins> __launch_bounds__(CUDA_NUM_THREADS) </ins> __global__ void im2col_kernel(const int n, const Dtype* data_im, const int height, const int width, const int ksize_h, const int ksize_w,

} template <typename Dtype, typename Acctype> <ins> __launch_bounds__(CUDA_NUM_THREADS) </ins> __global__ void col2im_kernel(const int n, const Dtype* data_col, const int height, const int width, const int channels, const int kernel_h, const int kernel_w,