Get the underlying function calls for a function call and locate them in PyTorch code base

username: sabiha_afroz
cooked: <p>I want to see the following function calls happening for Tensor.to(device) function and how the moving of a Tensor is working from CPU to GPU or GPU to CPU. I used the following code to get an idea of function calls.</p>
<pre><code class="lang-auto">import torch

# Enable the profiler
with torch.autograd.profiler.profile() as prof:
    # Your code snippet involving Tensor.to() here
    tensor = torch.randn(3, 3).to(device="cuda")
    tensor = tensor.to(device="cpu")

# Print the profiler results
print(prof)
</code></pre>
<p>The output is<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/b/6b9eb9fb5b0eab8c19c41d9afdb4fe98079382ca.png" data-download-href="https://discuss.pytorch.org/uploads/default/6b9eb9fb5b0eab8c19c41d9afdb4fe98079382ca" title="Screenshot (553)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/6/b/6b9eb9fb5b0eab8c19c41d9afdb4fe98079382ca.png" alt="Screenshot (553)" data-base62-sha1="fm39Sgp3VLaQzwoBLPhg6HomM0q" width="690" height="322" data-dominant-color="252525"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot (553)</span><span class="informations">1126×527 15 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So, for the Tensor.to(device) function, aten::to, atten::<em>to_copy, cudaDeviceGetStreamPriorityRange, cudaMalloc, aten::copy</em>  functions are getting called.<br>
I want to track these functions in PyTorch code base. I am new to PyTorch. So, I fail to locate and navigate the PyTorch code base. I do not know if there are any other tools to get more detailed information about classes that contain the functions. Any kind of help will be appreciated. Thanks in advance.</p>

username: ptrblck
cooked: <p><a href="https://github.com/pytorch/pytorch/blob/fb492f7ca14ee429a522589f9c71da53f56fc323/aten/src/ATen/native/Copy.cpp#L315"><code>copy_</code></a> will call into <a href="https://github.com/pytorch/pytorch/blob/fb492f7ca14ee429a522589f9c71da53f56fc323/aten/src/ATen/native/Copy.cpp#L135"><code>copy_impl</code></a> and dispatch to the <code>stub</code> which will eventually call into e.g. <a href="https://github.com/pytorch/pytorch/blob/3e21c785a4be06a75791bcae5179f830cc62d557/aten/src/ATen/native/cuda/Copy.cu#L173"><code>copy_device_to_device</code></a> and there into the actual <code>memcpy</code> kernel.</p>

username: sabiha_afroz
cooked: <p>Thanks for linking the class. Can you give some guidelines on how to navigate through the PyTorch source code and how to locate the functions in the source code?</p>

username: ptrblck
cooked: <p>I usually check the <a href="https://github.com/pytorch/pytorch/blob/637cf4a3f2cfdd364005681636ca885bdc4d5887/aten/src/ATen/native/native_functions.yaml"><code>native_functions.yml</code></a> first to check for dispatching into other internal methods. Once I’ve found the kernel name I <code>grep -r</code> for it in the source code repository and narrow down where it’s defined.</p>