akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Data splitting into train/test sets or random choosing?

username: Chame_call
cooked: <p>What will be more accurate and more right?</p>
<p>I took a notebook from <a href="https://pytorch.org/tutorials/intermediate/char_rnn_classification_tutorial.html" rel="noopener nofollow ugc">pytorch docs</a> to classify surnames by language (RNN).</p>
<p>I <a href="https://colab.research.google.com/drive/1Hve8-Hsej92eXXBTyu0YOYibxBkDaADh" rel="noopener nofollow ugc">modified it</a> a bit to test trained model on a validation set and got about 55% percent of accuracy.</p>
<p>But I’d like to train the model on a train dataset and test on a validation one and not to choose data points randomly.</p>
<p>So I <a href="https://colab.research.google.com/drive/1kqzS4AR55txl8dSHSwFsOOuzS_D17rdL" rel="noopener nofollow ugc">modified it</a> some more.<br>
And I’ve got only about 30 percent of accuracy and the next confusion matrix:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/1/4/14b64a6a7250b3c2d1c19c60847d0bb696b76c6a.png" alt="plot%20(2)" data-base62-sha1="2Xe6xaKw20A0mGNlCKU8kR78qLg" width="432" height="288"></p>
<p>Maybe somebody can explain that.<br>
Is that related to the fact of splitting data into two data sets or is it something else?</p>

username: Chame_call
cooked: <p>That’s the confusion matrix from the original notebook:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/2/b/2baae857b10d932a349c6af5b1b87eb2c2e9f665.png" alt="plot%20(3)" data-base62-sha1="6eiHl8I4XCZJKskGMXm9CyZQBN3" width="432" height="288"></p>

username: ptrblck
cooked: <p>Since the evaluation was done using random training samples, the confusion matrix will give you biased results. In your case, it seems that you are overfitting the training data as explained <a href="https://discuss.pytorch.org/t/only-one-category-classified-on-the-whole-dataset-in-the-confusion-matrix/60552/14">in your other topic</a>.</p>
How to implement gating?

username: isalirezag
cooked: <p>I am reading <a href="https://arxiv.org/pdf/1712.04851.pdf" rel="noopener nofollow ugc">Rethinking Spatiotemporal Feature Learning</a> paper and would like to implement some part of it.<br>
in the paper they have this gating thing which is defined as:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/0/00311d24aa09ad2c0a8a711d541fa22bba17b97a.png" alt="image" data-base62-sha1="1He0qGhcKrRVgRwdbpJAeMUqFA" width="171" height="46"></p>
<p>where X in <img src="https://discuss.pytorch.org/uploads/default/original/3X/9/3/939ed17da77897657d54bc6548dac9f470fdac71.png" alt="image" data-base62-sha1="l3Uscqio8gf4xuRwh0RuBuzQnsZ" width="98" height="23"> .<br>
i know how to use the sigma or elementwise product, but im not sure how to do<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/c/d/cd8da576a46964efef1aa993f97674f7d04b8ee3.png" alt="image" data-base62-sha1="tkpm9zzEe0SNKG9t1TqEzSSjBDl" width="138" height="52"><br>
for 2D or 3D tensor?<br>
can someone please explain to me how to do it in code?<br>
I think if x is 1D i should use <code>nn.linear</code>, but for 2D and 3D i dont know.</p>

username: zimmer550
cooked: <p>I think you are looking for this? <a href="https://github.com/jacobkimmel/pytorch_convgru" rel="nofollow noopener">https://github.com/jacobkimmel/pytorch_convgru</a></p>

username: isalirezag
cooked: <p>Thanks for your reply!<br>
hmmm they are using the convolution version, I am not 100% sure if this is exactly the same  <img src="https://discuss.pytorch.org/images/emoji/apple/thinking.png?v=9" title=":thinking:" class="emoji" alt=":thinking:"></p>

username: isalirezag
cooked: <p>any suggestion regarding this question?</p>
How can I read a image pair at the same time

username: shn
cooked: <p>for the Consumer-to-shop Clothes Retrieval dataset,  it need to read a pair image at the same time, it will be okay as followed code?</p>
<pre><code class="lang-auto">
img1 = read('...')

img2 = read('...')

img = [img1, img2]

</code></pre>
<p>thanks for replying me <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: ptrblck
cooked: <p>I assume you would like to create a <code>Dataset</code> returning both images?<br>
The easiest way would be to create an own <code>Dataset</code> class and use your code snippet to load the data:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, image1_paths, image2_paths, transform=None):
        self.image1_paths = image1_paths
        self.image2_paths = image2_paths
        self.transform = transform
        
    def __getitem__(self, index):
        img1 = Image.open(self.image1_paths[index])
        img2 = Image.open(self.image2_paths[index])
        
        if self.transform:
            img1 = self.transform(img1)
            img2 = self.transform(img2)
            
        return img1, img2
    
    def __len__(self):
        return len(self.image1_paths)


dataset = MyDataset(
    image1_paths, image2_paths, transform=transforms.ToTensor())
</code></pre>
<p>Note that you might want to use <code>torchvision.transforms.functional</code>, if you want to apply random transformations like <code>RandomCrop</code> on both images in the same way.</p>
<p>Also, have a look at the <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">data loading tutorial</a> for more information.</p>

username: shn
cooked: <p>Thanks for your reply. <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=6" title=":smile:" class="emoji" alt=":smile:"></p>

username: Quintia_Zhao
cooked: <p>but how to read a series images pairswise?<br>
the file name is connected very similar<br>
one group is like (“pp_001_cc.gipl”“pp_002_cc.gipl”…)<br>
and the other group is (“pp_001_mm.gipl”,“pp_002_mm.gipl”…)</p>

username: ptrblck
cooked: <p>You could modify the read logic in <code>__getitem__</code> and e.g. use a for loop to read all corresponding images.<br>
In my current approach I’m just reading two files, but you could also read all matching file names.</p>
Trying to port a matlab model to pytorch

username: shinx
cooked: <p>I’m trying to port <a href="https://github.com/anilbas/3DMMasSTN" rel="nofollow noopener">anilbas/3DMMasSTN</a> model to pytorch <a href="https://github.com/suvojit-0x55aa/3DMMasSTN-Pytorch" rel="nofollow noopener">suvojit-0x55aa/3DMMasSTN-Pytorch</a></p>
<p>I tried to implement it as closely to MATLAB code as possible but it seems I can’t train the model.</p>
<ul>
<li>Firstly, original code has a bilinear sampler, I looked into the code and Pytorch’s <code>F.grid_sample</code> looked like had what it was trying to implement, but seems there are differences, cause the pytorch model is generating garbage.</li>
<li>Moreover MATLAB uses <code>[0..255]</code> range for its input image, unlike <code>[0..1]</code> for Pytorch, which range should be used for weights converted from matlab to pytorch.</li>
<li>The original code <a href="https://github.com/anilbas/3DMMasSTN/blob/c6562b5fda5c2f742a27dc1b4a7ff15ec5e83837/dagnn_3dmmasstn_init.m#L9" rel="nofollow noopener">sets some weights manually</a> but when I do the same for pytorch, it’s training behaviour is different than MATLAB’s.</li>
<li>What are part of the code should I look for differences, since I’m not well versed in MATLAB.</li>
</ul>
<p>I appreciate anyone taking the time to look into it and giving some pointer so that can I can implement it correctly. Thanks in advance.</p>

username: edgarriba
cooked: <p>hi,</p>
<p>in <a href="http://www.kornia.org" rel="nofollow noopener">www.kornia.org</a> we a have a reimplementation of OpenCV <a href="https://docs.opencv.org/master/da/d54/group__imgproc__transform.html#gab75ef31ce5cdfb5c44b6da5f3b908ea4" rel="nofollow noopener"><code>remap</code></a> which for what you describe I think might be similar assuming your sampling tensor in pixel space.</p>
Enumerate/iter for multiple dataloaders

username: zelazny
cooked: <p>I am trying to divide MNIST into separate subsets where each subset is a different class.</p>
<pre><code class="lang-auto">train_dataset = datasets.MNIST('/some/dir/', train=True, download=True, transform=self.dataset_transform)

for i in range(10):
    idx = train_dataset.targets==i
    train_dataset.targets[idx]                
    train_subsets.append(torch.utils.data.Subset(train_dataset, idx))
    train_loaders.append(torch.utils.data.DataLoader(train_subsets[i], batch_size=self.batch_size, shuffle=True))
    train_iters.append(iter(train_loaders[i]))
</code></pre>
<p>Beginning of my training loop:</p>
<pre><code class="lang-auto">for label_id in range(10):
    data, target = train_iters[label_id].next()
</code></pre>
<p>When training, I get a <code>ValueError</code>:</p>
<pre><code class="lang-auto">only one element tensors can be converted to Python scalars
</code></pre>
<p>What is going on here?</p>

username: MariosOreo
cooked: <p>Hello,</p>
<p>In my shallow opinion, there might be something wrong with your snippet.</p>
<aside class="quote no-group" data-username="zelazny" data-post="1" data-topic="41771">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/87869e/48.png" class="avatar"> zelazny:</div>
<blockquote>
<p>idx = train_dataset.targets==i</p>
</blockquote>
</aside>
<p>Here, idx is tensor with values 0 and 1. So if you pass it as indices argument to <code>Subset</code>, the subset will only contain the first and the second target (means position 0 and position 1).<br>
And <code>train_dataset.targets[idx]</code> does not make sense to me.</p>
<aside class="quote no-group" data-username="zelazny" data-post="1" data-topic="41771">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/87869e/48.png" class="avatar"> zelazny:</div>
<blockquote>
<p>only one element tensors can be converted to Python scalars</p>
</blockquote>
</aside>
<p>For this error, you could try to print <code>train_iters[lable_id]</code> 's shape and debug it.</p>

username: zelazny
cooked: <p>Finally, I was able to make this work with:</p>
<pre><code class="lang-auto">for i in range(10):
    idx = train_dataset.targets==i
    idx_np = idx.numpy()
    idx_num = idx_np * range(len(idx))
    idx_no_zeros = list(filter(lambda a: a != 0, idx_num))
    train_subsets.append(torch.utils.data.Subset(train_dataset, idx_no_zeros))
    train_loaders.append(torch.utils.data.DataLoader(train_subsets[i], batch_size=self.batch_size, shuffle=True))
</code></pre>
Some confusion about the grad of torch.sign()

username: jxgu1016
cooked: <p>Here’ s my demo and why <code>x_binary</code> 's grad is always <strong>None</strong> ?</p>
<p>I do understand the grad of <strong>torch.sign()</strong> does not exist mathematically, but <strong>F.conv2d</strong> should give <code>x_binary</code> grad even though it won’t be passed to <code>x</code>.</p>
<pre><code class="lang-python">import torch
import torch.nn.functional as F
</code></pre>
<pre><code class="lang-python">x = torch.randn(2,3,4,4, requires_grad=True)
# print x
x_binary = x.sign().requires_grad_()
# print x_binary
</code></pre>
<pre><code class="lang-python">weight = torch.ones(3,3,1,1, requires_grad=True)
# print weight
</code></pre>
<pre><code class="lang-python">out = F.conv2d(x_binary, weight, None, 1, 0, 1, 1)
# out = F.conv2d(x, weight, None, 1, 0, 1, 1)
# print out
</code></pre>
<pre><code class="lang-python">loss = out.mean()
print loss
</code></pre>
<pre><code>tensor(-0.3750)
</code></pre>
<pre><code class="lang-python">print x.grad, x_binary.grad
</code></pre>
<pre><code>None None
</code></pre>
<pre><code class="lang-python">loss.backward()
</code></pre>
<pre><code class="lang-python">print x.grad, x_binary.grad
</code></pre>
<pre><code>tensor([[[[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]]],


        [[[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]],

         [[ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.],
          [ 0.,  0.,  0.,  0.]]]]) None
</code></pre>
<pre><code class="lang-python">print weight.grad
</code></pre>
<pre><code>tensor([[[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]],


        [[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]],


        [[[-6.2500e-02]],

         [[-6.2500e-02]],

         [[ 1.8626e-09]]]])</code></pre>

username: ruotianluo
cooked: <p><code>x_binary = x.sign().detach().requires_grad_()</code> works.</p>
<p>Looks like a bug to me.</p>

username: jxgu1016
cooked: <p>Any plan to report this?</p>

username: ruotianluo
cooked: <p>I think you should report this.</p>

username: jxgu1016
cooked: <p>see the discussion <a href="https://github.com/pytorch/pytorch/issues/7940#issuecomment-393010897" rel="nofollow noopener">here</a></p>

username: gaoliming
cooked: <p>I use your method in the code such as:</p>
<pre><code class="lang-auto">symbol_alpha = relation_alpha.sign().detach().requires_grad_()
</code></pre>
<p>but when the loss backward, the parameters of relation_alpha is none , my pytorch version is 1.0.<br>
why?</p>
Shutdown graphical user interface in order to use more GPU ram

username: farhad-bat
cooked: <p>I wrote a program in PyTorch but unfortunately I need one more GB of GPU ram. My GPU is 1080 Ti and my OS is Ubuntu 18.04.<br>
In other to solve my problem I need to shutdown the Graphical User Interface of Ubuntu 18.04. However I don’t know how do that.</p>

username: JuanFMontesinos
cooked: <aside class="onebox stackexchange">
  <header class="source">
      <a href="https://askubuntu.com/questions/148321/how-do-i-stop-gui" target="_blank" rel="nofollow noopener">askubuntu.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://askubuntu.com/users/69371/anthony" target="_blank" rel="nofollow noopener">
    <img alt="Anthony" src="https://www.gravatar.com/avatar/a0f0b99b3e5fa7febe97340524edfb33?s=128&amp;d=identicon&amp;r=PG" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://askubuntu.com/questions/148321/how-do-i-stop-gui" target="_blank" rel="nofollow noopener">How do I stop GUI?</a>
</h4>

<div class="tags">
  <strong>command-line</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://askubuntu.com/users/69371/anthony" target="_blank" rel="nofollow noopener">
    Anthony
  </a>
  on <a href="https://askubuntu.com/questions/148321/how-do-i-stop-gui" target="_blank" rel="nofollow noopener">10:35PM - 08 Jun 12 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
Is there a sequence data loader in pytorch?

username: tastyminerals
cooked: <p>Back in the days <code>Torch</code> had a <a href="https://github.com/Element-Research/dataload" rel="nofollow noopener">nice collection</a> of data loaders. In sequence modeling for getting the <code>[25 x 32 x 100]</code> tensor for RNN training I used <code>SequenceLoader</code> which accepted <code>(sequence, batchsize, bidirectional)</code> parameters where <code>25</code> is a sequence length, <code>32</code> is the batch size and <code>100</code> is the embedding size. Quickly skimming through current <code>torch.utils.data</code> docs I found nothing for sequence, maybe I am not seeing it but how do you do sequence batching in pytorch now?</p>