akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Ensembles of NN's

username: AKT_TARAFDER
cooked: <pre><code class="lang-auto">IN=7
HU=35
model=nn.Sequential(nn.Linear(IN,HU),nn.Dropout(p=0.1),
                    nn.ReLU(),
                    nn.Linear(HU,HU),
                    nn.ReLU(),
                    nn.Linear(HU,HU),
                    nn.ReLU(),
                    nn.Linear(HU,IN),nn.Dropout(p=0.1),
                    nn.ReLU(),
                    nn.Linear(IN,1))

def create_ensembles(num):
    ensembles=[model for i in range(num)]
    
    return ensembles

ensembles=create ensembles(10)
</code></pre>
<p>Does this create 10 isolated neural networks that can be trained by looping? Or these are connected? I want to train these and use majority voting for classification.</p>

username: KFrank
cooked: <p>Hi Akt!</p>
<aside class="quote no-group" data-username="AKT_TARAFDER" data-post="1" data-topic="117272" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/akt_tarafder/48/35960_2.png" class="avatar"> AKT_TARAFDER:</div>
<blockquote>
<pre><code class="lang-auto">def create_ensembles(num):
    ensembles=[model for i in range(num)]
    
    return ensembles

ensembles=create ensembles(10)
</code></pre>
<p>Does this create 10 isolated neural networks that can be trained by looping?</p>
</blockquote>
</aside>
<p>Your code creates only a single neural network.</p>
<p>This is really a python question rather than about pytorch in particular.</p>
<p>Your <code>create_ensembles()</code> creates a list (using a list comprehension)<br>
that contains <code>num</code> references <em>that all refer to the same</em> model.</p>
<p>You first instantiate a <em>single</em> instance of a <code>Sequential</code> (and then<br>
assign it to the <em>reference</em> <code>model</code>).  <code>create_ensembles()</code> then makes<br>
10 copies of that same <em>reference</em> and puts them in a list.</p>
<p>Here is some pure python (no pytorch) that illustrates what is going on:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; dict = {}     # make an instance of something
&gt;&gt;&gt; l = [dict for i in range (5)]
&gt;&gt;&gt; for i in l:   # all elements of the list refer to the same instance
...     print (id (l))
...
139963655730048
139963655730048
139963655730048
139963655730048
139963655730048
</code></pre>
<p>What you would want to do instead is package your desired network<br>
architecture as python class, and then instantiate <code>num</code> instances of<br>
your network class.</p>
<p>This tutorial, <a href="https://pytorch.org/tutorials/beginner/examples_nn/two_layer_net_module.html" rel="noopener nofollow ugc">PyTorch: Custom nn Modules</a>, gives an example of defining<br>
such a class.</p>
<p>Good luck.</p>
<p>K. Frank</p>
GPU runtime error

username: hesham_gabr
cooked: <p>RuntimeError: CUDA out of memory. Tried to allocate 15.94 GiB (GPU 0; 15.78 GiB total capacity; 641.30 MiB already allocated; 13.86 GiB free; 642.00 MiB reserved in total by PyTorch)</p>
<p>i got this error when i  train model</p>

username: ptrblck
cooked: <p>The error indicates that your GPU is running out of memory, so you would have to e.g. reduce the batch size of the input, lower the memory usage by using a smaller model, or e.g. use <code>torch.utils.checkpoint</code> to trade compute for memory.</p>

username: hesham_gabr
cooked: <p>The problem is solved when I reduce the size of data(images) but the information of images are disappeared</p>
<p>the actual size of images is 1024<br>
and i put them as 224</p>
AttributeError: 'tuple' object has no attribute 'to'

username: Najeh_Nafti
cooked: <p>I got this error while trying to train  a GAN model:</p>
<pre><code class="lang-auto">    run(config)
  File "t.py", line 417, in run
    x, y = x.to(device), y.to(device)
AttributeError: 'tuple' object has no attribute 'to'
</code></pre>

username: ptrblck
cooked: <p>Based on the error message it seems that either one of the used objects (<code>x</code> or <code>y</code>) or both are a tuple instead of tensors.<br>
Could you check the type and make sure the <code>to()</code> operation is only called on tensors directly?</p>
Input with missing data

username: Lucas_Gongora
cooked: <p>Hello,</p>
<p>Currently, I have my model trained with dimensions n x 10 x 6.<br>
I want to test how my model performs when there is missing data in the features. I would like to know if it is possible to input some data as 1x10x3 to the previously trained model or if I could use 1x10x6 but with empty values.</p>

username: ptrblck
cooked: <p>Changing the input tensor size might raise errors, especially if you reduce the feature dimension and a particular layer expects its input to have a defined feature size (e.g. <code>nn.Linear</code>).<br>
You could set the “missing” features to some predefined values, but it depends on your use case what these values might be (there are no “empty” numbers and I would avoid using e.g. <code>NaN</code> as a value).</p>
Query on PyTorch thread-model

username: imaginary
cooked: <p>Hello,</p>
<p>Can anyone please give an example of when <a href="https://github.com/pytorch/pytorch/blob/master/docs/source/notes/cpu_threading_torchscript_inference.rst" rel="noopener nofollow ugc">such a situation</a> (discussed in the Threading wiki) can arise? -</p>
<p><em>OpenMP does not guarantee that a single per-process intra-op thread pool is going to be used in the application. On the contrary, two different application or inter-op threads may use different OpenMP thread pools for intra-op work.</em></p>
<p>I want to perform some measurements for such examples.</p>
<p>Thank you!</p>
CUDA out of memory although enough memory is available

username: frawee
cooked: <p>Hey,<br>
I am training a transformer based model on a GPU and get an out of Memory error. This is the error message:</p>
<p>CUDA out of memory. Tried to allocate 578.00 MiB (GPU 2; 10.76 GiB total capacity; 2.79 GiB already allocated; 6.55 GiB free; 3.24 GiB reserved in total by PyTorch)</p>
<p>The model runs fine on a different GPU.<br>
I would truly appreciate any help!<br>
Thank you all</p>

username: cyRi-Le
cooked: <p>you should open your terminal and execute <code>nvidia-smi</code> to see the actual use of your GPU. If you still have free memory include <code>torch.cuda.empty_cache()</code> in your code before loading the model on GPU. Finally reducing batch_size could be a way to solve it.</p>
Using SobolEngine with cuda?

username: squash
cooked: <p>Hi,</p>
<p>I am wondering if I could generate quasirandom number with SobolEngine directly on gpu, i.e. without memory copy between host and device, in PyTorch 1.8.0?</p>
<p>For example, I tried the following code</p>
<pre><code class="lang-auto">a = torch.empty([1],dtype=torch.float,device=torch.device('cuda'))
soboleng = torch.quasirandom.SobolEngine(dimension=1)
soboleng.draw(1,out=a)
</code></pre>
<p>Then I got the following error</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
  File "C:\Users\zz_ro\Anaconda3\envs\pytorch\lib\site-packages\torch\quasirandom.py", line 91, in draw
    out.resize_as_(result).copy_(result)
RuntimeError: Expected object of backend CUDA but got backend CPU for argument #2 'the_template'
</code></pre>
<p>I guess this is because the variable <code>a</code> is allocated on gpu, while in the <code>draw</code> function, the temporary result is generated on CPU. And the error occurs when it tries to copy a CPU tensor (i.e., <code>result</code>) to a cuda tensor (i.e. <code>out</code>). What I am trying here is to call the random number generator directly on GPU, and store the result in a cuda tensor. Is this possible?</p>
Does Modifying an argument of a learning algo (which results in chaging it's concept) make it count as a new veriation or not

username: Kallinteris-Andreas
cooked: <p>I was not sure how to title this.</p>
<p>Let’s start with an example assume we have DDPG algorithm as shown below<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/5/054568156a1657d651fa7f5525eb013e3722b8fc.jpeg" alt="image" data-base62-sha1="KD5AWNNWUdUNb9mULGHBXbRldq" width="553" height="401"><br>
Quick explanation:<br>
After the Init phase we have 2 loops<br>
The inner loop which executes for T time steps (of the simulation)<br>
is responsible for learning policies for each time step<br>
while the outer loop<br>
makes the agent repeat the learning process across multiple simulations (taking the learned policies from the previous states and applying them to new states)</p>
<p>My first Question/case is, assume I set the value of T (the total time steps for the simulation) to 1 (effectively removing the loop part of the inner loop), would it still count as DDPG or should it be considered something else.</p>
<p>My Second Question/case is, assume we treat the entire simulation as a single time step (again effective removing the loop element of the inner loop) would it still count as DDPG.</p>
<p>In Both cases the algorithm still remains online off-policy RL.</p>
<p>Thanks!</p>
Google colab dataset uploading

username: hesham_gabr
cooked: <p>Uploading files directly from the local file system or from google drive in google colab is faster in the training model?</p>

username: SK412
cooked: <p>Hi!  This probably will not answer your question completely. If the dataset is small I would move it to my local file system in colab that is ‘/content/’, this is because in colab there is a cap on number of times you can load the data from drive to bypass it it is recommended to put the data in local file as there is no cap on number times you load the data from  local system.</p>

username: hesham_gabr
cooked: <p>Thanks for your reply<br>
I have another question, please<br>
Run time in google colab during the training process give me a Busy run time<br>
What should I do! (i purchased a pro version of colab)<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/b/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555.png" data-download-href="https://discuss.pytorch.org/uploads/default/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555" title="ccc"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555_2_690x187.png" alt="ccc" data-base62-sha1="xFDBO8hUt3Z5co10JXt7lCCVwC9" width="690" height="187" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555_2_690x187.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555_2_1035x280.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/b/ebfcb2d7ab28a1dc50e0cf21304fb5fde8f9c555_2_1380x374.png 2x" data-dominant-color="31312C"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">ccc</span><span class="informations">1551×421 28.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: SK412
cooked: <p>Well I am not an expert on this if given it was a free version I would have said maybe you exhausted your daily quota for GPU but given the fact you are on a pro version I am not sure of if that would be the reason, from the screenshot it looks the model is still training?<br>
One thing you can do is to check the number of active sessions and kill the sessions which are not needed. In the free version you can use two sessions simultaneously. Hope it helps.</p>
Advice on ways to save which data sample is causing a device-side assert error?

username: seankala
cooked: <p>Hi. I’m trying to solve a device-side assert error. Within my dataset, only a few samples seem to be the culprits of the program crashing, and so I think it’d be convenient if there were a way to target them.</p>
<p>So far I’ve tried adding in try-except blocks as such:</p>
<pre><code class="lang-auto">try:
    some_variable = some_embedding_function(x)
except RuntimeError as e:
    print(e)
    import pdb; pdb.set_trace()
</code></pre>
<p>This returns the PDB interactive shell, but it also doesn’t allow me to access any of the problematic variables (it returns the error message every time I’d try to use, say, <code>x</code>). I’ve also tried printing/saving the value but it seems like the device-side error gets triggered first and shuts down the program.</p>
<p>Would anybody know if there’s a way to pinpoint the problematic samples? Thanks.</p>

username: tom
cooked: <p>Device asserts are asynchronous, so you will typically see the backtrace in a later operation. By <a href="https://lernapparat.de/debug-device-assert/" rel="noopener nofollow ugc">forcing synchronous operations</a>, you can pin down the exact function.<br>
As device asserts invalidate the GPU context, you need to move the data you want to inspect after them to CPU before calling the failing function. That CPU copy can then be accessed.</p>
<p>Best regards</p>
<p>Thomas</p>
Model.apply does not catch the module name

username: yuri
cooked: <p>the main Idea that I want to apply parameter initialization to specific layers. the condition will be tested by the layer name.</p>
<p>experts recommend model.apply(init_fn)</p>
<p>this iterate through modules of the model by calling children() which will call named_chidren() method</p>
<p>named_children output <strong>name</strong> and <strong>module</strong> children however output <strong>only the module</strong>. at this level we lose the name information.</p>
<p>the condition to apply the initialization cannot not be tested</p>
<p>Help!</p>

username: ptrblck
cooked: <p>You could move the condition out of the <code>weight_init</code> method and apply it to a loop iteratint all named children manually. If the condition is met, you could them call into the <code>weight_init</code>, which could check for the <code>type</code> of the method or could even apply the <code>torch.nn.init</code> method directly on the child module.</p>

username: yuri
cooked: <p>thank you very much for your reply!<br>
I suppose this suggest a direct call for the <code>weight_init</code> and not applying the <code>model.apply(weight_init)</code> technique:<br>
something like this:</p>
<pre><code class="lang-auto">init_layers = ['conv12', 'conv2', 'conv31', 'conv4'] # layer that are allowed to be initialized
        for i in model.named_children():
            if i[0] in init_layers: # layer name
               weights_init(i)
</code></pre>
<p>otherwise, I don’t see how to apply your suggestion with the <code>model.apply()</code> technique</p>

username: ptrblck
cooked: <p>Yes, I would probably call the <code>weights_init</code> method on the desired layers directly to have a bit more control over it.</p>