akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
AttributeError: 'float' object has no attribute 'copy'

danielmanu93: <p>Hi everyone, I copied the weights of my trained model using “model.state_dict()” and then I want to update these weights using model.load_state_dict(weights) but I’m getting an error "AttributeError: ‘float’ object has no attribute ‘copy’ " as shown in the images below. Any ideas as to how can I can fix this?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/9/5919ab1848cbe4a42d3103b197e9ac9d1fd21c1b.jpeg" alt="1" data-base62-sha1="cIdujQk8LtcMoGkd8cIZedLDpaP" width="346" height="65"><br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/5/e/5ed83b32d8458f1b14f217c9526680b5108c0d42.jpeg" alt="2" data-base62-sha1="dx29DQt8C6mG0AfY6xQkcbvwxtE" width="379" height="63"><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/1/51d196512dddf51acebc2d4f9aaf9574814e183c.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/51d196512dddf51acebc2d4f9aaf9574814e183c" title="3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/1/51d196512dddf51acebc2d4f9aaf9574814e183c_2_690x51.jpeg" alt="3" data-base62-sha1="bFNIeVqFebm9iiJNVOlji5Ym92I" width="690" height="51" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/1/51d196512dddf51acebc2d4f9aaf9574814e183c_2_690x51.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/5/1/51d196512dddf51acebc2d4f9aaf9574814e183c.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/1/51d196512dddf51acebc2d4f9aaf9574814e183c.jpeg 2x" data-dominant-color="E4E4E4"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">3</span><span class="informations">830×62 26.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Could you post the model definition of <code>x_global_model</code>, as it seems the <code>state_dict()</code> operation returns a <code>float</code>, while an <code>OrderedDict</code> would be expected?</p>

ptrblck: <p>It seems you are manually appending some “weights” to <code>g_global_weights</code> and try to treat it as a <code>state_dict</code>, which is causing the issue.<br>
You would have to use the workflow posted previously, i.e. create the <code>state_dict</code> via:</p>
<pre><code class="lang-python">sd = model.state_dict()
</code></pre>
<p>and load it via:</p>
<pre><code class="lang-python">model.load_state_dict(sd)
</code></pre>
<p>afterwards.<br>
Currently you are replacing the initial <code>state_dict</code> with an empty <code>list</code> and append the (unknown) values.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which would make debugging easier.</p>
Tensorboard fails to plot model weights for all epochs during training

avalon1511: <p>I am trying to plot the progression of my model’s weights during training using <code>add_scalar</code>, however, the tensorboard plot only shows the weights for any one epoch.</p>
<p>What I mean by this is, when I load tensorboard, I only see “epoch 0” in the scalars section even if I have run my model for 10 epochs. However, I dont have this issue while plotting histograms in the same code.</p>
<p>My code is as follows:</p>
<pre><code class="lang-auto">for epoch in total_epochs:
          train model
          calculate error
          optimizer.step()
          calculate average loss
          
          for name,weight in model.named_parameters():
              SummaryWriter.add_histogram(name, weight, epoch)
              SummaryWriter.add_scalar(str(name), weight, epoch)
</code></pre>
<p>Here is an example of what I mean - <a href="https://imgur.com/a/l2ZL5da" rel="noopener nofollow ugc">imgur.com/a/l2ZL5da</a> I trained the model for 10 epochs but tensorboard is only plotting the weights for epoch 0 and 1, whereas the histogram (not pictured) captures the progression for all 10 epochs.</p>
<p>Not sure why this is happening</p>
Implemented Bi Directional Conv LSTM in pytorch

Rituraj_19_Dutta_100: <p>I want to implement Bi Directional Conv LSTM in Pytorch. I did find some solutions for the implementations as mentioned in <a href="https://discuss.pytorch.org/t/passing-hidden-layers-to-convlstm/52814">Passing hidden layers to ConvLSTM</a> but the code throws an error:<br>
<code>TypeError: initialize_weights() missing 1 required positional argument: 'layer'</code>. I have gone through this <a href="https://github.com/ndrplz/ConvLSTM_pytorch" rel="noopener nofollow ugc">Github repo</a> but this doesn’t support Biderctional.</p>
Will the observer be synchronized across GPUs in DistributedDataParallel?

kaizhao: <p>I’m wondering about the behavior of quantization observer, e.g. HistogramObserver, when the model is wrapped with <code>DistributedDataParallel</code> ?</p>
<p>Will the estimated histograms (or min/max values) synchronized across multiple GPUs?</p>

supriyar: <p>The observer min/max values are stored as buffers in the module. So they get broadcast from the rank 0 machine when run with DDP. This ensures the values are synchronized across all machines.</p>

kaizhao: <p>Thx for clarify.</p>
<p>Does this mean only a part of samples (samples on GPU-0) are used to estimate the quantization parameters ?</p>

supriyar: <p>Yes, we use the samples on rank 0 to determine the quantization parameters.</p>
Libtorch 1.9.0 android aar

julien-thierry: <p>Hi,</p>
<p>I see that on <a href="https://oss.sonatype.org/#nexus-search;gav~org.pytorch~pytorch_android~~~~kw,versionexpand" class="inline-onebox" rel="noopener nofollow ugc">Nexus Repository Manager</a> , the 1.9.0 release does not have a downloadable .aar contrary to the previous ones, and has a .module instead.</p>
<p>It seems the .module contains information about the .aar, but I can’t figure out how to get it to be downloaded. How can I obtain the .aar file based on the .module?</p>
<p>Thanks,</p>
<p>Julien</p>

julien-thierry: <p>Any information about this?</p>

julien-thierry: <p>Finaly found it.<br>
If anyone searches for it:<br>
<a href="https://mvnrepository.com/artifact/org.pytorch/pytorch_android/1.9.0" class="onebox" target="_blank" rel="noopener nofollow ugc">https://mvnrepository.com/artifact/org.pytorch/pytorch_android/1.9.0</a></p>
How to store images in a large custom dataset

endrew: <p>Hi,<br>
I have a large custom made dataset of images, larger than my memory, and I don’t now what is the correct approach to store and use for training. Should I save the images as JPG/PNGs? Should I save them in a ZIP file? Or CSV? Any important considerations when implementing the dataloader?</p>
<p>Thanks!</p>

Manuel_Alejandro_Dia: <p>Usually Datasets are much bigger than the memory available in machines. This is why we give the data to the network by batches.\</p>
<p>For a custom dataset, I would advice to save the images in PNG if possible. Are both inputs and labels images?</p>
<p>And to use them you could follow <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html" rel="noopener nofollow ugc">this tutorial</a> that tells you how to use the Pytorch Dataloaders to load your data.</p>
What is a good training loss?

arya47: <p>What classifies as a good training loss for RESNET based classification?</p>

Manuel_Alejandro_Dia: <p>Have a look at <a href="https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learning-neural-networks/" rel="noopener nofollow ugc">this post</a>.</p>
<p>Even though the code is in Keras, the explications are quite good and you can easily find the corresponding losses in the <a href="https://pytorch.org/docs/stable/nn.html#loss-functions" rel="noopener nofollow ugc">Pytorch Docs</a></p>