akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Installing Anaconda and Pytorch on Centos 6.9

AhmedM: <p>I have machine with two GPU. But its operating system is Centos 6.9. Can I install Anaconda and Pytorch on this machine? Is the standard installation step of Pytorch using coda possible on Centos 6.9? Any help or directions would be much appreciated.</p>
<p>Thanks!</p>

ptrblck: <p>Based on the <a href="https://docs.anaconda.com/anaconda/install/">anaconda docs</a> CentOS 6+ seems to be supported.</p>

AhmedM: <p>Thanks so much for pointing anaconda’s support for CentOS 6+. This is helpful. I looked at Pytorch’s documentation and website for installation <a href="https://pytorch.org/get-started/locally/" rel="nofollow noopener">https://pytorch.org/get-started/locally/</a> It seems that Pytorch requires CentOS 7.3+, at least for the standard installation method. Is there a way to install Pytorch on CentOS 6? Thanks!</p>

ptrblck: <p>I don’t know, as I’ve only tried to build it with CentOS7.<br>
Where did you find the information about CentOS7.3+? I cannot find it in the linked page, but would assume it’s the minimal version if it’s on an official website (you might still get it to work in older version, but this might require some hacks).</p>

AhmedM: <p>From the linked page, select ‘Linux’ in ‘Your OS’ row, then see Supported Linux Distributions. Any suggestions about how to find the hacks that would be required. Thanks!</p>

ptrblck: <aside class="quote no-group" data-username="AhmedM" data-post="5" data-topic="94116">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ahmedm/48/11828_2.png" class="avatar"> AhmedM:</div>
<blockquote>
<p>Any suggestions about how to find the hacks that would be required. Thanks!</p>
</blockquote>
</aside>
<p>No suggestions, as I haven’t tried it.<br>
I think you would need to try building it and see, if it’s working and if not how much work fixing the issues would be.<br>
I would also suggest to use a docker container for it, as you can just nuke it afterwards, if you’ve messed up the system somehow.</p>

AhmedM: <p>Thanks so much for your recommendation and help!</p>
RuntimeError: sizes is inconsistent with indices. Pytorch 0.4.1

TheRealPieter: <p>When using pytorch 0.4.1 I constantly get RunTimeError when creating sparse FloatTensors:</p>
<p>num_nodes = 200<br>
num_edges = 390<br>
n2n_idxes = torch.LongTensor(2, num_edges<em>2)<br>
n2n_vals = torch.FloatTensor(num_edges</em>2)<br>
n2n_sp = torch.sparse.FloatTensor(n2n_idxes, n2n_vals, torch.Size([num_nodes, num_nodes]))</p>
<p>with the following error:<br>
RuntimError: size is inconsistent with indices: for dim 0, size is 200 but found 49529943793360929.</p>
<p>The value changes naturally, but it seems to happen with large values only or when they appear to be negative within n2n_idxes. Any solutions?</p>

ptrblck: <p>Could you update to the latest stable PyTorch version (<code>1.6</code>), as this might have been a known bug, which might have been fixed already.</p>
Any ImageNet 64x64 dataloader

Deeply: <p>Is there any code to load ImageNet 64x64, or 32x32 in PyTorch?</p>

fmassa: <p>Just use the standard <code>ImageFolder</code> dataset class, and add a transform that resizes the image to 64x64 or 32x32? This will be a bit slower than having converted straight away the images to 32x32 and loading them, but it’s much more flexible (and saves space in disk!)</p>

Deeply: <p><a href="http://www.image-net.org/" rel="nofollow noopener">http://www.image-net.org/</a> provides the 64x64 and the 32x32 versions of ImageNet, as described by Chrabaszcz et al, “<a href="https://patrykchrabaszcz.github.io/Imagenet32/" rel="nofollow noopener">A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets</a>”.</p>
<p>After downloading, the training set has 10 train_data_batches (train_data_batch1,…,10), one batch for the validation data, and the size of all the data is  14.2 GB.<br>
So, I think there is someway to load them, without downloading the original ImageNet data, and then, resizing it. No?</p>

Usama-Zafar: <p>Did you find a way to load the down-sampled dataset? I am trying the same by trouble getting it done.</p>

michaelklachko: <p>I just downloaded the 64x64 version, and I’m thinking about loading each batch into GPU memory (one at a time), to eliminate any potential bottleneck of a data transfer on each forward pass. What you think - is this worth doing? Each batch is 1.3GB so I’d be wasting ~10% of memory.</p>
Passing tuple/int to custom conv forward

Liam_Hiley: <p>Hi,<br>
I’m trying to use the C++ Extension tutorial to write a custom convolution function for use in python.</p>
<pre><code class="lang-auto">torch::Tensor conv2d_forward(
    const torch::Tensor input,
    const torch::Tensor weight,
    const torch::Tensor bias,
    torch::ExpandingArray&lt;2&gt; stride,
    torch::ExpandingArray&lt;2&gt; padding,
    torch::ExpandingArray&lt;2&gt; dilation,
    int64_t groups
){
    return F::conv2d(input,weight,F::Conv2dFuncOptions().bias(bias).stride(stride).padding(padding).dilation(dilation).groups(groups));
}
</code></pre>
<p>I was hoping that I could then call this function using pybind like this:</p>
<pre><code class="lang-auto">torchexplain_cpp.conv2d_forward(input, weight, bias, (1,1),(1,1),(1,1),1)
</code></pre>
<p>But I get an incompatible arguments error regarding the tuples.<br>
How would I appropriately pass python tuples OR ints to the function?<br>
Thank you</p>

ptrblck: <p>Could you try to use <code>c10::ArrayRef&lt;long int&gt;</code> as the type for <code>stride</code>, <code>padding</code>, and <code>dilation</code> and rerun the script?</p>

Liam_Hiley: <p>Thank you for your reply, that works now. I’m still new to C++ and I realise now that <code>ExpandingArray</code> is a template. Is it possible to use it for parameter types? or would I have to either:<br>
Convert the ints to tuples in python<br>
or<br>
Write separate functions in C++ for int options vs. tuple options?</p>

ptrblck: <p>If you would like to pass integers directly, you could use the <code>int64_t</code> type in C++, as you’ve already done with the <code>groups</code> parameter.<br>
Could you explain the use case a bit, if I misunderstood it?</p>

Liam_Hiley: <p>Of course sorry. To clarify I was hoping that by specifying the optional parameters as ExpandingArray, then the function would accept either tuples, or integers that would then be expanded to tuples. So I could either pass (1,1) or simply 1 for my stride and see the same behaviour.<br>
Again I’m still learning c++ and it seems now incorrect.<br>
Thank you again for your help!</p>

ptrblck: <p>Ah OK, thanks for the explanation.<br>
I’m not sure what the proper way would be, but I’ve seen the usage of <code>c10::ArrayRef&lt;long int&gt;</code> in C++ and passing a tuple with different number of elements on the Python side via:</p>
<pre><code class="lang-python">(1,)
(1, 2)
(1, 2, 3)
...
</code></pre>
DataLoader worker thread-local state

vadimkantorov: <p>What’s the proper place for storing per-worker thread-local state, like buffer tensors? Is it dataset object (since it’s being copied over to all different processes and is supposed to not be shared)?</p>
<p>E.g. my <code>__getitem__</code> would like to store some buffers to be reused in the next <code>__getitem__</code> call in order to save some CPU NumPy allocations</p>

albanD: <p>cc <a class="mention" href="/u/vitalyfedyunin">@VitalyFedyunin</a> that has been looking at the DataLoader recently.</p>

VitalyFedyunin: <p>My best suggestion is to wait and use <a href="https://github.com/pytorch/pytorch/pull/35795" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/35795</a> functionality,<br>
Second best is to do on-demand buffers allocation</p>
Order of nodes in scripted graph

ttipri: <p>Suppose I have the following graph and example code.</p>
<pre><code class="lang-auto">import torch

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()

    def forward(self, x):
        z = x.squeeze(0)
        z.add_(2)
        x.mul_(5)
        return x

if __name__ == "__main__":
    script = torch.jit.script(Net().eval())
    print(str(script))
    print("-" * 10)
    for node in script.graph.nodes():
        print(node)
</code></pre>
<p>Is the loop guaranteed to iterate over the nodes in an order that preserves the semantics of the in-place operations in the graph?<br>
In this simple example its just enough that the add is looped over before the mul, but the general case is more important to me.<br>
I do get the correct order when running this script, but I’m interested if this is guaranteed.</p>

tom: <p>Reodering is done with checking dependencies, including side effects:<br>
</p><aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/58a7e73a95865e5c02a581703c39edd671495a6c/torch/csrc/jit/ir/alias_analysis.h#L98-L108" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/58a7e73a95865e5c02a581703c39edd671495a6c/torch/csrc/jit/ir/alias_analysis.h#L98-L108" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/58a7e73a95865e5c02a581703c39edd671495a6c/torch/csrc/jit/ir/alias_analysis.h#L98-L108</a></h4>
<pre class="onebox"><code class="lang-h"><ol class="start lines" start="98" style="counter-reset: li-counter 97 ;">
<li>// Move 'n' (already in the graph) after 'movePoint' in the topological order.</li>
<li>//</li>
<li>// Tries to preserve value dependencies, so other nodes might be moved. We</li>
<li>// make two guarantees about the postcondition of the node list:</li>
<li>//   - `n` is directly after `movePoint`.</li>
<li>//   - only nodes between `n` and `movePoint` have been moved.</li>
<li>//</li>
<li>// Returns `false` if it's impossible to move `n` after `MovePoint` without</li>
<li>// violating dependencies, otherwise executes the move and returns `true`</li>
<li>TORCH_API bool moveAfterTopologicallyValid(Node* n, Node* movePoint);</li>
<li>TORCH_API bool moveBeforeTopologicallyValid(Node* n, Node* movePoint);</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Best regards</p>
<p>Thomas</p>

ttipri: <p>Thanks! Greatly appreciated (:</p>
Error when re-defining Faster R-CNN module

Luppo: <p>Hello everyone,</p>
<p>I’m facing a problem when trying to re-define Faster R-CNN.<br>
<br><br>
I have copied all of the code in <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/detection/faster_rcnn.py" rel="nofollow noopener">the torchvision implementation of Faster R-CNN</a>, properly modifying the path to libraries (ex: prepending ‘torchvision.models.detection’ to libraries requiring it).<br>
<br><br>
The strange thing is: if I do not re-define <code>class FasterRCNN(GeneralizedRCNN)</code> everythin is good and no error arises. But as soon as I write - COPY AND PASTE from the link, no modifications - such class I get an error.</p>
<pre><code class="lang-auto">/usr/local/lib/python3.6/dist-packages/torchvision/ops/poolers.py in setup_scales(self, features, image_shapes)
    109         # get the levels in the feature map by leveraging the fact that the network always
    110         # downsamples by a factor of 2 at each level.
--&gt; 111         lvl_min = -torch.log2(torch.tensor(scales[0], dtype=torch.float32)).item()
    112         lvl_max = -torch.log2(torch.tensor(scales[-1], dtype=torch.float32)).item()
    113         self.scales = scales

IndexError: list index out of range
</code></pre>
<br>
Any suggestion?
<p>My current version of <code>torchvision</code> is 0.4.2, which should be the last stable version.</p>
<br>
<p>Thank you.</p>

calamansi: <p>Hello Luppo,</p>
<p>I had the same problem. It worked as soon as I used a String in <code>featmap_names = ['0']</code>.</p>
<pre><code class="lang-auto">        roi_pooler = torchvision.ops.MultiScaleRoIAlign(featmap_names=['0'],
                                                        output_size=7,
                                                        sampling_ratio=2)
</code></pre>
<p>Maybe it can help you.</p>

Luppo: <p>Thank you for your response!<br>
Actually I’ve solved the problem by inspecting the code in my library and extracting the diff with that on github. I then found that box_roi_pool was defined as</p>
<pre><code class="lang-auto">box_roi_pool = MultiScaleRoIAlign(
                featmap_names=[0, 1, 2, 3],
                output_size=7,
                sampling_ratio=2)
</code></pre>
<p>In Faster R-CNN code instead, <code>featmap_names=['0', '1', '2', '3']</code>. I really don’t know why this change was made, but changing it solved my problem.</p>
<p>Hope it could help someone else.</p>

rmoraes: <p>Thanks for the solution, it worked for me.<br>
I believe <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="nofollow noopener">this</a> PyTorch tutorial (which I was following) should be updated to account for <code>featmap_names</code> being list[str], as defined in its <a href="https://github.com/pytorch/vision/blob/master/torchvision/ops/poolers.py" rel="nofollow noopener">documentation</a>.</p>

mapneto: <p>Thank you! Also helped me a lot</p>
How to do a groupby in torch tensor?

Icia_C: <p><em>Hi,</em><br>
<em>Could you help me with a problem?</em> <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>
<p>I have a tensor like this:</p>
<p>labels = [0,0,0,1,…,1998]<br>
samples = [1,20,3,…,3]</p>
<p>I need to pick for the same class, the maximun value of samples. E.g in the previous tensor the result must be:</p>
<pre><code class="lang-auto">[[0,20],
...
[1998, _ ]]
</code></pre>
<p><em>Thank you in advance for your help.</em></p>