Using libtorch with Cuda

username: Andre4102
cooked: <p>Hi, I want to do fault injection on a neural network for a project and I chose Pytorch C++ as a framework.  Since I want to use nvbitFI as fault injector, so I need an explicit calls to a CUDA kernel function. The idea is to write only the forward function since fault injection will be done in the inference phase and not during training. I have already tried using the -&gt;to(device) function of pytorch, but the fault injector does not detect and kernel operation.<br>
I wrote my own class that extends conv2dimpl and calls a cuda function I wrote, but when I try to compile it, I get the following error.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/4/74c6e802c61aa0d58bbf3228b8faa5022d187a48.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/74c6e802c61aa0d58bbf3228b8faa5022d187a48" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74c6e802c61aa0d58bbf3228b8faa5022d187a48_2_690x114.jpeg" alt="image" data-base62-sha1="gF3xyU04qSS3te2uC2haTnwQjEc" width="690" height="114" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74c6e802c61aa0d58bbf3228b8faa5022d187a48_2_690x114.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/7/4/74c6e802c61aa0d58bbf3228b8faa5022d187a48_2_1035x171.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/4/74c6e802c61aa0d58bbf3228b8faa5022d187a48.jpeg 2x" data-dominant-color="401C35"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1280×212 81.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>the code is the following:</p>
<blockquote>
<p>torch::Tensor forward_cuda(torch::Tensor weights, torch::Tensor input, torch::Tensor bias, torch::Tensor output);</p>
<p>class Conv2dCudaImpl : public torch::nn::Conv2dImpl{<br>
public:<br>
Conv2dCudaImpl(<br>
int kernelSize, int inChannels, int outChannels,<br>
int padding, int dilation = 1, int stride = 1,<br>
int groups = 1, std::string padding_mode = “zeros”,<br>
bool useBias = true) : torch::nn::Conv2dImpl(torch::nn::Conv2dOptions(inChannels, outChannels, kernelSize).padding(padding).dilation(dilation).stride(stride)){<br>
this-&gt;kernelSize = kernelSize;<br>
this-&gt;padding = padding;<br>
this-&gt;dilation = dilation;<br>
this-&gt;stride = stride;<br>
this-&gt;inChannels = inChannels;<br>
this-&gt;outChannels = outChannels;</p>
<pre><code>  	max = groups / (inChannels*kernelSize*kernelSize);
  	min = -1*max;
  	
  	weight = (max-min)*torch::rand({outChannels, inChannels, kernelSize, kernelSize}, torch::requires_grad(true)) + min; 
  	
  	if(useBias == false){
  		bias = 0 * torch::rand({outChannels});
  	}else{
  		bias = (max-min)*torch::rand({outChannels}, torch::requires_grad(true)) + min;
  	}
  }
  
  torch::Tensor forward(const torch::Tensor&amp; input){
  	//height of input matrix, padding, dilation, kernel height, stride
  	out_h = (input.data().size(2)+2*padding-dilation*(weight.data().size(2)-1)-1)/stride+1;
  	
  	//width of input matrix, padding, dilation, kernel width, stride
  	out_w = (input.data().size(3)+2*padding-dilation*(weight.data().size(3)-1)-1)/stride+1;
  	
  	//batch size, number of output channels, height, width, just to initialize the tensor to the correct size
  	output = torch::randn({input.data().size(0), outChannels, out_h, out_w});
  	
  	return forward_cuda(weight, input, bias, output);
  }
</code></pre>
<p>private:<br>
variable declaration<br>
};<br>
TORCH_MODULE(Conv2dCuda);</p>
</blockquote>
<p>I found another code for UNET <a href="https://github.com/divideconcept/PyTorch-libtorch-U-Net" rel="noopener nofollow ugc">here</a>, which I used for reference alongside the tutorial I found <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="noopener nofollow ugc">on the Pytorch website</a> and when I try to compile it does not give me this error.</p>