How to convert PyTorch code into equivalent Cuda code?

username: Ajinkya.Bankar
cooked: <p>I need Cuda codes of the pre-trained AlexNet, GoogleNet, and VGG. Is there any utility that converts PyTorch code into equivalent Cuda code? Kindly help. Thanks.</p>

username: ptrblck
cooked: <p>PyTorch code cannot be directly translated to CUDA code, as the CUDA kernels are only a part of the overall model.<br>
What is your use case?</p>

username: Ajinkya.Bankar
cooked: <p>Thank you for your response. I want to use AlexNet and VGG inferencing on the GPGPU Simulator. But the simulator does not support PyTorch directly. So I want to convert pre-trained models from the PyTorch into Cuda codes and then use them on the simulator for inferencing. Is there any other possible way? Kindly help.</p>

username: ptrblck
cooked: <p>I’m not familiar with the GPGPU Simulator, but from a quick look at it, I’m not sure, if this library can be used to simulate a complete application or if it’s rather used for specific kernels.</p>

username: EssamWisam1
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> could you elaborate further. Like why is it possible to get Triton code from <code>torch.compile</code> but not CUDA code?</p>

username: ptrblck
cooked: <p>You can read the native CUDA kernels in the code base, but note my questions:</p>
<aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="116015">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>PyTorch code cannot be directly translated to CUDA code, as the CUDA kernels are only a part of the overall model.</p>
</blockquote>
</aside>
<aside class="quote no-group" data-username="ptrblck" data-post="4" data-topic="116015">
<div class="title">
<div class="quote-controls"></div>
<img alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>if this library can be used to simulate a complete application or if it’s rather used for specific kernels.</p>
</blockquote>
</aside>
<p>Exporting the host as well as device code to CUDA won’t be possible as you need the execute the host code by definition.<br>
If you are only asking about native CUDA kernels, you can find them in the repository.<br>
CUDA Math libs (e.g. cuBLAS and cuDNN) are closed-source and you won’t be able to read the code.</p>

username: EssamWisam1
cooked: <p>Thank you so much.</p>
<p>I am pondering why is it not possible to convert a PyTorch nn.Module into its equivalent CUDA code since my understanding is that this (or something similar) happens under the hood to run on my Nvidia GPU. Or otherwise why torch.compile does not have this option.</p>
<blockquote>
<p>as the CUDA kernels are only a part of the overall model.<br>
Could you clarify further? I thought:</p>
</blockquote>
<pre><code class="lang-auto">import torch

class MyModel(torch.nn.Module):
    def forward(self, x):
        return torch.relu(x @ x.T + x)
</code></pre>
<p>would purely equivalent to a pipeline of one or more kernels written in CUDA.</p>