The cuda kernel used by pytorch matrix multiplication

username: MalaJeans
cooked: <p>When testing matrix multiplication with pytorch，<br>
If  the scale of matrix multiplication is m=10240,n=5120,k=5120.The cuda kernel used by pytorch matrix multiplication is：<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e.png" data-download-href="https://discuss.pytorch.org/uploads/default/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e" title="微信图片_20230707195411"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e_2_690x89.png" alt="微信图片_20230707195411" data-base62-sha1="rmO04mxmUMGj8QF4zLgK09AhuNM" width="690" height="89" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e_2_690x89.png, https://discuss.pytorch.org/uploads/default/original/3X/b/f/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/b/f/bfcead69d80bad9d1882d4b179f8fcc99f1dad1e.png 2x" data-dominant-color="0A0A0A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20230707195411</span><span class="informations">831×108 36.1 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
but when the scale of matrix multiplication is m=40960,n=20480,k=10240,the result is:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72.png" data-download-href="https://discuss.pytorch.org/uploads/default/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72" title="微信图片_20230707195516"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/e/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72_2_690x74.png" alt="微信图片_20230707195516" data-base62-sha1="8V9e9EePj44YinDHjwS3UePEqGK" width="690" height="74" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/e/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72_2_690x74.png, https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/e/3e872e7a7a3d6251c6a72408afdfa61e5f8edf72.png 2x" data-dominant-color="080808"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">微信图片_20230707195516</span><span class="informations">831×90 24.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Question:<br>
when m=40960,n=20480,k=10240,the cuda kernel not in use?</p>
<p>the code is:</p>
<p>import torch<br>
import time</p>
<p>torch.backends.cuda.matmul.allow_tf32 = True</p>
<p>m = 40960<br>
n = 20480<br>
k = 5120</p>
<p>input = torch.randn(m, k, dtype=torch.float32,device=‘cuda’)<br>
weight = torch.randn(k, n, dtype=torch.float32,device=‘cuda’)</p>
<p>output = torch.matmul(input, weight)</p>

username: ptrblck
cooked: <p>The second output doesn’t show any matmul kernel execution.<br>
Which device are you using and were you running into an error (e.g. OOM)?</p>

username: MalaJeans
cooked: <p>My device is RTX A5000.<br>
What is the function of the cuda kernel “distribution elementwise grid stride kernel” ？<br>
I looked at it again. I think it was OOM. Thanks~</p>

username: ptrblck
cooked: <p>This kernel should be called by the <code>randn</code> operation.</p>

username: MalaJeans
cooked: <p>I get it！<br>
Thank you your reply！</p>