akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How Can I Know Which Convolution Algo Was Used When I Set "torch.backends.cudnn.benchmark = True"?

HITLCX: <p>It is validated that convolution process can be accelerated in Pytorch by setting “torch.backends.cudnn.benchmark = True”.I want to know which convolution algorithm(FFT, Winograd and GEMM)was selected. Moreover, if it is possible to set the convolution algo manually?<br>
I wonder if there is any way to know which convolution algo was used, and how to set the specific algo manually.</p>
Memory leak on CPU.How to release explicitly?

long_story: <p>Hi all.<br>
Is there a way like torch.cuda.empty_cache() to release CPU memory explicitly? I meet memory leak on CPU.<br>
My code is like:</p>
<pre><code class="lang-auto">def func(model):
    torch.jit.trace(model)
    out = model(input)

for i in range(10):
     model = resnet()
     func(model)
     printsomething
</code></pre>
<p>Every iteration,  run at <em>printsomething</em>,the memory is increase about 2G. The next iteration, when <em>torch.jit.trace</em> is executed, the memory will be released.</p>
<p>A simpler code snippet is:</p>
<pre><code class="lang-auto">def sub_fun1():
    from torchvision.models.resnet import resnet50
    model = resnet50(num_classes=10)
    del model

def sub_fun2():
    from torchvision.models.resnet import resnet50
    model = resnet50(num_classes=10)
    model = model.cuda()
    del model

@profile(precision=4)
def main():
    sub_fun1()
    sub_fun2()
    gc.collect()

</code></pre>
<p>The memory usage is like:</p>
<pre><code class="lang-auto">Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    61 215.7070 MiB 215.7070 MiB           1   @profile(precision=4)
    62                                         def main():
    63 237.2070 MiB  21.5000 MiB           1       sub_fun1()
    64 2988.8672 MiB 2751.6602 MiB           1       sub_fun2()
    65 2988.8672 MiB   0.0000 MiB           1       gc.collect()
</code></pre>
<p>So the memory is not released after sub_fun2 is called.Why this happen?</p>
Comparable C++ Function to the PyTorch torch.index_select()

bjaeger: <p>I have a tensor of shape <code>300 x 500 x 40 x 40</code> and a list of indices (e.g. length of list is 100 with values between 0 and 499)</p>
<p>With <code>tensor = torch.index_select(tensor, 1, torch.tensor(index_list))</code> I reduce my tensor to a shape of <code>300 x 100 x 40 x 40</code> by only selecting the values for the indices at dimension 1 (500 → 100).</p>
<p>What is the comparable function in the C++ API to achieve the same?</p>
<p>So far my workaround is the following:</p>
<pre><code class="lang-auto">torch::Tensor features;
torch::Tensor features_reduced;
for (int i = 0; i &lt; indices_list.sizes()[0]; ++i) {
  if (i == 0) { // get first
    features_reduced = features.index({torch::indexing::Slice(), indices_list[i] , torch::indexing::Slice(), torch::indexing::Slice()}).unsqueeze(1);
  }
  else { // concatenate to the previous tensor
    features_reduced = torch::cat({features_reduced, features.index({torch::indexing::Slice(), indices_list[i] , torch::indexing::Slice(), torch::indexing::Slice()}).unsqueeze(1)}, 1);
  }
}
</code></pre>
I want to use vulkan for pytorch module in android

user21: <p>When loading a module<br>
mModule = Module.load(assetFilePath(context, “vulkan.pt”), Device.VULKAN)</p>
<p>to do this</p>
<p>1.7 version in android studio<br>
implementation ‘org.pytorch:pytorch_android:1.7.0’<br>
implementation ‘org.pytorch:pytorch_android_torchvision:1.7.0’<br>
I did.</p>
<p>outputTuple = mModule!!.forward(IValue.from(inputTensor)).toTuple()<br>
An error occurs during forwarding.</p>
<p>the error message is<br>
com.facebook.jni.CppException: PyTorch is not linked with support for vulkan devices<br>
Exception raised from getDeviceGuardImpl at … /c10/core/impl/DeviceGuardImplInterface.h:216 (most recent call first):</p>
<p>This is it.</p>
<p>Why?</p>

vjayd: <p>Hi <a class="mention" href="/u/user21">@user21</a> ,</p>
<ol>
<li>Can you check if the .so files that you have are vulkan compatible?<br>
If not then create a Vulkan compatible PyTorch Android libs</li>
<li>Make sure .pt file is generated by providing backend=‘vulkan’ during export</li>
</ol>
How does "*" work in multiplication?

Litchilitchy: <p>I know <code>*</code> could work as <code>Z = X * Y</code> in matrix element-wise multiplication and <code>Y</code> could also be a scalar here.</p>
<p>However, where is this overload implemented? I tried to debug in Python on this line <code>Z = X * Y</code> but I could not step in any deeper. I want to know how this is overloaded.</p>
<p>PS: I tried to multiply a quantized weight of <code>QInt8</code> dtype and a float scalar and it directly raised<br>
<code>promoteTypes with quantized numbers is not handled yet; figure out what the correct rules should be, offending types: QInt8 Float</code><br>
, which is in C++ code. But the debugger could not show any detail. I think understanding this implementation of multiplication would be benificial for this issue.</p>
Register_forward_hook alternative or workaround

Longbowman: <p>As <a href="https://github.com/pytorch/pytorch/issues/25883" rel="noopener nofollow ugc">this topic</a> says register_forward_hook function is not implemented in C++ API.<br>
I desperately need to implement this one. This hook in python version allows a function to be called multiple times, but the model to be ‘clean’ at the end of each run for other uses. Is there any way to do?</p>

bjaeger: <p>I would also need this function in the C++ API, however I haven’t found any solution/workaround so far…</p>
Model.to(device) hangs

SCUTVK: <p>Our school’s slurm server’s device is v100 with cuda 10.1. Lack of root user, I can not upgrade nvidia driver. The code able to run successfully in my pc(tesla p4 with cuda 11.6) can not run in server. Besides, I have tried kinds of torch versions and cuda version (&lt;=10.1), but it always hangs when running model.to device.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/f/cf03b001c8d7341aa64d170b8cc8d235944c68fb.png" alt="image" data-base62-sha1="txkJGxC0yMhNdH9mCtQ9o4AxsT1" width="554" height="382"></p>
<pre><code class="lang-auto">self.bert_regression_by_word_document.to(device=self.args['device'])
</code></pre>

ptrblck: <p>Based on your output 3/4 GPUs seem to work while GPU0 reports an uncorrectable ECC error, so you might want to check the RAM of this device.</p>

SCUTVK: <p>Oh! I can’t believe I didn’t notice the ecc problem in this GPU, causing me to spend days troubleshooting environmental issues. Thx very much.</p>

ptrblck: <p>Just to confirm my understanding: are you able to use the other GPUs in this PyTorch environment and it only fails on GPU0 (the one showing the ECC error)?</p>

SCUTVK: <p>No, I can not choose which gpu to use. It is assigned by the slurm system.</p>

SCUTVK: <p>I think you are right, because I use the same environment to run my code successfully weeks before.</p>