akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Looking for the troubleshooting docs

mrhobbeys: <p>I’m having trouble to install pytorch. I only have a couple of hours on some nights of the week and I have an error that I think is related to to setup. But I am not finding help docs.</p>
<p>When  I run import torch so I can check if the install worked I get the following error:</p>
<hr>
<p>ImportError                               Traceback (most recent call last)</p>
<p> in <br>
1 from <strong>future</strong> import print_function<br>
----&gt; 2 import torch<br>
3 x = torch.rand(5, 3)<br>
4 print(x)<br>
5</p>
<p>c:\users\mrhob.virtualenvs\torchy-fl7c4wvq\lib\site-packages\torch_<em>init</em>_.py in <br>
79 del _dl_flags<br>
80<br>
—&gt; 81 from torch._C import *<br>
82<br>
83 <strong>all</strong> += [name for name in dir(_C)</p>
<p>ImportError: DLL load failed: The specified module could not be found</p>
<p>Any help would be greatly appreciated!</p>

tom: <p>So assuming your install directory <code>c:\users\mrhob.virtualenvs\torchy-fl7c4wvq\lib\site-packages\torch</code> has a <code>_C.*.pyd</code> on Windows (with * something), which is should be loading, I seem to recall that Windows might have a rather opaque and uninformative error message when one of the dependency DLLs cannot be found.</p>
<p>How did you install? Generally the anaconda method seems to be the most reliable and most tested. (Even though I must admit I invariably use pip3 without virtual env.)</p>
<p>Best regards</p>
<p>Thomas</p>
create_graph=True vs create_graph=False

vainaijr: <p>Are these two equivalent?</p>
<pre><code class="lang-auto">import torch, torch.nn as nn
loss_fn = nn.CrossEntropyLoss()
scores = torch.randn(1, 10,requires_grad=True)
y_a_i = torch.tensor([5])
lin = nn.Linear(10, 10)
set_loss = loss_fn(lin(scores), y_a_i) 
fast_parameters = list(lin.parameters())
grad = torch.autograd.grad(set_loss, fast_parameters, create_graph=True) 
grad = [g.detach() for g in grad]
</code></pre>
<pre><code class="lang-auto">import torch, torch.nn as nn
loss_fn = nn.CrossEntropyLoss()
scores = torch.randn(1, 10,requires_grad=True)
y_a_i = torch.tensor([5])
lin = nn.Linear(10, 10)
set_loss = loss_fn(lin(scores), y_a_i) 
fast_parameters = list(lin.parameters())
grad = torch.autograd.grad(set_loss, fast_parameters, create_graph=False) 
</code></pre>

tom: <p>Well, in practical terms yes, but <code>create_graph=True</code> does a lot of work and allocate memory (that then goes out of scope) and stuff.</p>
Data Parallel multi-gpu training results in a reduction of performance and very noisy learning curves and losses

jalvara1: <p>Hi, I am currently experiencing the problem that when I go from single-gpu to multi-gpu training the performance degrades severely. Also I’ve noticed that the learning curves and losses are much noisier. The only difference between the two setups in the script is the value of the environmental variable CUDA_VISIBLE_DEVICES. I am using torch.nn.DataParallel to parallelize the model as in <a href="https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html</a>.</p>
<p>What could be the reason of this highly puzzling difference between training with 1 gpu or &gt; 1 gpu? Or what can I do to isolate the source of the problem?</p>
<p>Thanks in advance</p>

coyote: <p>Any update on this issue ? Have you solved the problem ? I’m facing with the same issue currently.</p>
Torch.save / TypeError: file must have a 'write' attribute

susht3: <p>i load the old model and train, but when i save the model with the same name, i get this error:</p>
<p>Traceback (most recent call last):<br>
File “main.py”, line 66, in <br>
train_model(config, dataset)<br>
File “main.py”, line 41, in train_model<br>
model_controller.train()<br>
File “/data1//workspace/matchLSTM-pytorch/utils/model_controller.py”, line 97, in train    self.save_model(epoch)<br>
File “/data1/workspace/matchLSTM-pytorch/utils/model_controller.py”, line 210, in save_model<br>
torch.save(model_weight, self.model_weight_path)<br>
File “/data1//local/anaconda3/lib/python3.5/site-packages/torch/serialization.py”, line 161, in save<br>
return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))<br>
File “/data1/local/anaconda3/lib/python3.5/site-packages/torch/serialization.py”, line 118, in _with_file_like<br>
return body(f)<br>
File “/data1/local/anaconda3/lib/python3.5/site-packages/torch/serialization.py”, line 161, in <br>
return _with_file_like(f, “wb”, lambda f: _save(obj, f, pickle_module, pickle_protocol))<br>
File “/data1/local/anaconda3/lib/python3.5/site-packages/torch/serialization.py”, line 227, in _save<br>
pickle_module.dump(MAGIC_NUMBER, f, protocol=pickle_protocol)<br>
TypeError: file must have a ‘write’ attribute</p>

susht3: <p>i solve this problem by remove old model path</p>

StCross: <p>How to remove old path</p>