akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Can't find References

ahsan44411: <p>Where is the References section on this website? I am following the Object Detection tutorial and can not find some files that the tutorial says is in the References.</p>

ptrblck: <p>I think you are looking for the code folder <a href="https://github.com/pytorch/vision/tree/master/references" rel="nofollow noopener"><code>vision/references</code></a>.</p>

ahsan44411: <p>Thanks where can I find pycocotools?</p>

ptrblck: <p>You can install it via</p>
<pre><code class="lang-auto">git clone https://github.com/cocodataset/cocoapi.git
cd cocoapi/PythonAPI
python setup.py build_ext install
</code></pre>

lyh: <p>Thank you for solving my problem</p>

lyh: <p>Sorry,pycocotools fail to install</p>
<p>C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -ID:\Python\lib\site-packages\numpy\core\i<br>
nclude -I…/common -ID:\Python\include -ID:\Python\include “-IC:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE” “-IC:\Program Files (x86)\Wi<br>
ndows Kits\10\include\10.0.10240.0\ucrt” “-IC:\Program Files (x86)\Windows Kits\8.1\include\shared” “-IC:\Program Files (x86)\Windows Kits\8.1\include\um”<br>
“-IC:\Program Files (x86)\Windows Kits\8.1\include\winrt” /Tc…/common/maskApi.c /Fobuild\temp.win-amd64-3.6\Release…/common/maskApi.obj -Wno-cpp -Wno-<br>
unused-function -std=c99<br>
cl : Command line error D8021 : invalid numeric argument ‘/Wno-cpp’<br>
error: command ‘C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\x86_amd64\cl.exe’ failed with exit status 2</p>

ptrblck: <p><a href="https://github.com/cocodataset/cocoapi/issues/51">This cocoapi issue</a> is describing the same problem.<br>
In one of the <a href="https://github.com/cocodataset/cocoapi/issues/9">linked answers</a> you could probably find a Windows build.</p>

lyh: <p>It’s done.Thank you!</p>
Loss and accuracy (Newbie)

Daniel_Joseph: <p>Hello everyone ,<br>
I want to get the test loss and accuracy for every image in every batch and its index, not accuracy and loss for the hole bach. i.e : dictionary that store every single image tested label, loss, index.<br>
Thanks in advance</p>

ptrblck: <p>You could get the sample-wise losses by setting <code>reduction='none</code> in the initialization of your criterion.<br>
I’m not sure, which use case you are working ob, but a accuracy for each image can be calculated regardless of the loss reduction.</p>

Daniel_Joseph: <p>Here is actually where i wanna get the individual class precisions :</p>
<pre><code class="lang-auto">def test(epoch):
    model.eval()
    test_loss = AverageMeter()
    acc = AverageMeter()
    for data, target in test_loader:
        if use_cuda:
            data, target = data.cuda(), target.cuda()
        data, target = Variable(data, volatile=True), Variable(target)
        data = data.float()
        output = model(data)
        entr = F.cross_entropy(output, torch.max(target, 1)[0], size_average=True).data[0]
        test_loss.update(entr,target.data.size(0))
        prec1, = accuracy(output.data[i], target.data[i]) # test precison in one batch
        acc.update(prec1, target.data.size(0))
    print('\nTest set: Average loss: {:.4f}, Accuracy: {:.2f}%\n'.format(test_loss.avg, acc.avg))
    writer.add_scalar('Loss/Test', test_loss.avg, epoch)
    writer.add_scalar('Accuracy/Test', acc.avg, epoch)
    return acc.avg

</code></pre>
<p>Want to have something like this :</p>
<pre><code class="lang-auto">class 1 92% 
class 2 93%
....
class i  xx%
</code></pre>

ptrblck: <p>I would recommend to store all predictions and targets in a list (don’t forget to wrap this code in a <code>with torch.no_grad()</code> block or detach the predicitons) inside the loop.<br>
Once you’ve collected all predictions and the corresponding target, you could use <code>sklearn.metrics.confusion_matrix</code> (or search here in the board for a PyTorch implementation we’ve posted some time ago).<br>
To get the per-class accs, you could use:</p>
<pre><code class="lang-python">conf = confusion_matrix(targets, preds)
conf = conf / conf.sum(axis=1)[:, np:newaxis]
print(conf.diagonal())
</code></pre>
Use gpu from two machines

vainaijr: <p>I have two computer, each one has one gpu, how do I use these two gpu combined to train one model?<br>
Also, how do I combine colab gpu with local machine gpu to train model?</p>

Eta_C: <aside class="quote no-group" data-username="vainaijr" data-post="1" data-topic="66736">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vainaijr/48/49167_2.png" class="avatar"> vainaijr:</div>
<blockquote>
<p>how do I use these two gpu combined to train one model</p>
</blockquote>
</aside>
<p>Use <a href="https://pytorch.org/tutorials/intermediate/ddp_tutorial.html" rel="noopener nofollow ugc">DDP</a> or <a href="https://github.com/ray-project/ray" rel="noopener nofollow ugc">Ray</a>.</p>
Are the outputs of bidirectional GRU concatenated?

NLPpupil: <p>In the document of class torch.nn.GRU(*args, **kwargs):</p>
<p>Outputs: output, h_n<br>
output of shape (seq_len, batch, hidden_size * num_directions): tensor containing the output features h_t from the last layer of the RNN, for each t. If a torch.nn.utils.rnn.PackedSequence has been given as the input, the output will also be a packed sequence.<br>
h_n of shape (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t = seq_len</p>
<p>We know that bidirectional RNN reads the sequence forward then get a sequence of forward hidden states, and read the sequence backward then get a sequence of backward hidden states. When do machine translation , we need to concatenate the forward and backward hidden states. Since the size of output is (seq_len, batch, hidden_size * num_directions), does this mean the hidden states have already be concatenated?</p>

austin: <p>Yes, if you want to sum them instead you can do:</p>
<pre><code class="lang-python"> rnn_out = (rnn_out[:, :, :self.hidden_dim] +
                rnn_out[:, :, self.hidden_dim:])
</code></pre>

NLPpupil: <p>Thank you very much!</p>

ymeng: <p>Just to make sure. Are the outputs of the two directions reversed? I mean, can we sum them as in your code, or we need to reverse one of them before summation?</p>

tarun: <p>And how exactly are the hidden state outputs given? Are they concatenated like {h_layer1_fw , h_layer1_bw , h_layer2_fw, h_layer2_bw} or {h_layer1_fw , h_layer2_fw , h_layer1_bw, h_layer2_bw}. And how do I sum them up?</p>

galactica147: <p>feel the same as OP, the official docs should’ve been better clarified and demonstrated with more code snippet examples.</p>

vdw: <p>Maybe this <a href="https://discuss.pytorch.org/t/rnn-output-vs-hidden-state-dont-match-up-my-misunderstanding/43280">older post of mine</a> helps a bit, I had a similar question.</p>
PyTorch v1.4.0 is live: customizable mobile builds, Distributed Model Parallelism via experimental RPC API, Java Bindings, Chaining LRSchedulers

smth: <h1>v1.4: customizable mobile builds, Distributed Model Parallelism via experimental RPC API, Java Bindings, Chaining LRSchedulers</h1>
<p>Summary: <a href="https://pytorch.org/blog/pytorch-1-dot-4-released-and-domain-libraries-updated/">https://pytorch.org/blog/pytorch-1-dot-4-released-and-domain-libraries-updated/</a><br>
Release Notes: <a href="https://github.com/pytorch/pytorch/releases/tag/v1.4.0">https://github.com/pytorch/pytorch/releases/tag/v1.4.0</a></p>
<p>Last release for Python 2 (bye bye!)</p>
<p>first release for Python 3.8: binaries for the entire matrix of Python 3.8 {pytorch, torchvision, torchaudio} configurations will be live by Jan 23rd. Some configurations are live already</p>

smth:
How to get (print) convolution_algorithm chosen by CUDNN autotune?

Shubhankar: <p>How to get the convolution algorithm chosen by CUDNN autotune in pytorch and how to manually define it later?</p>
<ol>
<li>Is it possible to interact with CUDNN API from inside Pytorch. Following <a href="http://www.goldsborough.me/cuda/ml/cudnn/c++/2017/10/01/14-37-23-convolutions_with_cudnn/" rel="nofollow noopener">CUDNN function</a> returns the type of algorithm to be used defined by <code>CUDNN_CONVOLUTION_FWD_PREFER_FASTEST</code>:</li>
</ol>
<pre><code class="lang-auto">cudnnGetConvolutionForwardAlgorithm(cudnn,
                                        input_descriptor,
                                        kernel_descriptor,
                                        convolution_descriptor,
                                        output_descriptor,
                                        CUDNN_CONVOLUTION_FWD_PREFER_FASTEST,
                                        /*memoryLimitInBytes=*/0,
                                        &amp;convolution_algorithm)
</code></pre>
<ol start="2">
<li>Is it possible to use CUDNN <a href="https://docs.nvidia.com/deeplearning/sdk/cudnn-developer-guide/index.html#api-logging" rel="nofollow noopener">logs</a> somehow?</li>
<li>Is it possible to do it via pytroch C++ API?</li>
</ol>

Shubhankar: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Could you help with this?</p>
C++ Frontend Tutorial and GCC/CPP version

Maziar: <p>Hi all,</p>
<p>I am trying to run the DCGAN tutorial for C++ frontend:<br>
<a href="https://pytorch.org/tutorials/advanced/cpp_frontend.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_frontend.html</a></p>
<p>But when I run the command:<br>
cmake --build . --config Release</p>
<p>There are many errors:</p>
<p>First error states that I am using an old GCC version and I have to use at least GCC 5 (I am using 9).</p>
<p>Second error says that I need C++14 to compile PyTorch. However in CMake in tutorial we add a line:<br>
set_property(TARGET dcgan PROPERTY CXX_STANDARD 11)</p>

Maziar: <p>Any help, pointers, directions is still appreciated…</p>

dhpollack: <p>You can change the cmake file to <code>set_property(TARGET dcgan PROPERTY CXX_STANDARD 14)</code>.  It’s impossible to tell what the problem is without seeing the code and the error logs.  I am using GCC9 as well.  Looking at my code it differs from the tutorial a little bit but perhaps you can look at it <a href="https://github.com/dhpollack/programming_notebooks/blob/master/c%2B%2B/pytorch_dcgan_tutorial/dcgan.cpp" rel="nofollow noopener">here</a> and figure out your problem.</p>

Maziar: <p>Hi David,</p>
<p>Thank you. I had changed it to 14 but I neglected to recreate the Makefile through cmake -DCMAKE_PREFIX_PATH=/path/to/libtorch … before rebuilding the project. That solved the problem.</p>
<p>And thanks whomever updated the tutorial.</p>

Maziar: <p>Just a quick update. The error on my side was also related to the <em>cc</em> version on my machine. It seems that <em>CMake</em> looks at <em>cc</em> when creating the <em>Makefile</em> not <em>gcc</em>. While my <em>gcc</em> was version 9.x my <em>cc</em> was 4.x which lead to the compiler version error.</p>
<p>This could be solved by adding an environment variable for <em>cc</em> (if newer versions already exist):<br>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/17275348/how-to-specify-new-gcc-path-for-cmake" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/988709/tidy" target="_blank" rel="nofollow noopener">
    <img alt="tidy" src="https://www.gravatar.com/avatar/ad8a700cff2fea2fba87c2309f784026?s=128&amp;d=identicon&amp;r=PG" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/17275348/how-to-specify-new-gcc-path-for-cmake" target="_blank" rel="nofollow noopener">How to specify new GCC path for CMake</a>
</h4>

<div class="tags">
  <strong>gcc, cmake</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/988709/tidy" target="_blank" rel="nofollow noopener">
    tidy
  </a>
  on <a href="https://stackoverflow.com/questions/17275348/how-to-specify-new-gcc-path-for-cmake" target="_blank" rel="nofollow noopener">12:21PM - 24 Jun 13 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
Loading pretrained model error

SangHyeon_Na: <p>Hi,</p>
<p>I was downaloding vgg19 pretrained model, but it stopped because of not enough space.</p>
<p>So, after making enough space I tried redownloading the model with</p>
<p><code>torchvision.models.vgg19(pretrained=True)</code></p>
<p>but I got error saying</p>
<blockquote>
<hr>
<p>RuntimeError                              Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 vgg = torchvision.models.vgg19(pretrained=True)</p>
</blockquote>
<blockquote>
<p>~/anaconda3/lib/python3.6/site-packages/torchvision-0.1.9-py3.6.egg/torchvision/models/vgg.py in vgg19(pretrained, &gt; &gt; **kwargs)<br>
164     model = VGG(make_layers(cfg[‘E’]), **kwargs)<br>
165     if pretrained:<br>
 → 166         model.load_state_dict(model_zoo.load_url(model_urls[‘vgg19’]))<br>
167     return model<br>
168</p>
</blockquote>
<blockquote>
<p>~/anaconda3/lib/python3.6/site-packages/torch/utils/model_zoo.py in load_url(url, model_dir, map_location)<br>
56         hash_prefix = HASH_REGEX.search(filename).group(1)<br>
57         _download_url_to_file(url, cached_file, hash_prefix)<br>
—&gt; 58     return torch.load(cached_file, map_location=map_location)<br>
59<br>
60</p>
</blockquote>
<blockquote>
<p>~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module)<br>
229         f = open(f, ‘rb’)<br>
230     try:<br>
 → 231         return _load(f, map_location, pickle_module)<br>
232     finally:<br>
233         if new_fd:</p>
</blockquote>
<blockquote>
<p>~/anaconda3/lib/python3.6/site-packages/torch/serialization.py in _load(f, map_location, pickle_module)<br>
384     for key in deserialized_storage_keys:<br>
385         assert key in deserialized_objects<br>
 → 386         deserialized_objects[key]._set_from_file(f, offset)<br>
387         offset = None<br>
388</p>
</blockquote>
<blockquote>
<p>RuntimeError: unexpected EOF. The file might be corrupted.</p>
</blockquote>
<p>How can I solve it?</p>

jdhao: <p>Try to clear the downloaded model and download it again? On my CentOS, the model is stored in <code>~/.torch/models</code> directory.</p>

SangHyeon_Na: <p>Oh thanks a lot!!</p>
<p>I thought it is ~/torch/models not ~/.torch/models so I couldn’t find downloaded file.</p>

jdhao: <p>If it solves your problem, you can mark my answer as solution.</p>

prasadheeramani: <p>If you are using Window 10 machine, deleting <code>vgg16-something</code> in folder <code>C:\Users\UserName\.cache\torch\checkpoints</code> would solve probelm.</p>
Pick certain indices from a tensor

supervan: <p>I have two tensors A and B. A is a float tensor with shape (batch size, hidden dim). B is a Long tensor with shape (batch size, data len). What I want is somewhat like A[:, B], a float tensor still with shape (batch size, data len), the elements are certain indices from A which depends on B.</p>
<p>An example would be A=[[5, 2, 6], [7, 3, 4]] and B=[[0, 2, 1, 1], [2, 2, 1, 0]]. Then what I want is a tensor [[5, 6, 2, 2], [4, 4, 3, 7]]. Is there any way to achieve this?</p>
<p>I have tried A[:, B], but what I achieve is a tensor with shape (batch size, batch size, data len), which is a large tensor. And I only want the “diagonal value” from this tensor.</p>

Eta_C: <pre><code class="lang-python">import torch
import torch.nn as nn

A=torch.tensor([[5, 2, 6], [7, 3, 4]]).float()  
B=torch.tensor([[0, 2, 1, 1], [2, 2, 1, 0]]).long()
result = torch.gather(A, 1, B)
print(result)
</code></pre>

supervan: <p>Thanks for the reply! How about when A has another dimension, now the shape of A is (batch size, hidden dim, data dim). And B is still the same, a long tensor with shape (batch size, data len). I actually want to have the resulting tensor with shape (batch size, data len, data dim).</p>
<p>For example, A=[[[5], [2], [6]], [[7], [3], [4]]] and B=[[0, 2, 1, 1], [2, 2, 1, 0]]. Then what I want is a tensor [[[5], [6], [2], [2]], [[4], [4], [3], [7]]]. Basically now data dim is 1 as an example.</p>
<p>It seems that torch.gather(A, 1, B) requires A and B to have the same shapes.</p>
<p><strong>In this case, just expand B as the same shape of A in order to solve the problem.</strong></p>