akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
diff b/w grad.zero_ or zero_grad

tejaswi_lakshmi: <p>diff b/w grad.zero_ or zero_grad</p>

tom: <p>Convenience and <code>detach_</code> (in case you have used <code>build_graph=True</code>):</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/blob/4460c8b034f8fd544ff9c271c4aa21698644d352/torch/optim/optimizer.py#L159-L166" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/pytorch/blob/4460c8b034f8fd544ff9c271c4aa21698644d352/torch/optim/optimizer.py#L159-L166" target="_blank" rel="nofollow noopener">pytorch/pytorch/blob/4460c8b034f8fd544ff9c271c4aa21698644d352/torch/optim/optimizer.py#L159-L166</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="159" style="counter-reset: li-counter 158 ;">
<li>def zero_grad(self):</li>
<li>    r"""Clears the gradients of all optimized :class:`torch.Tensor` s."""</li>
<li>    for group in self.param_groups:</li>
<li>        for p in group['params']:</li>
<li>            if p.grad is not None:</li>
<li>                p.grad.detach_()</li>
<li>                p.grad.zero_()</li>
<li>
</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Best regards</p>
<p>Thomas</p>
How efficiently slice a multi-dimension tensor

Chan_Simmer: <p>For example, there is a 3D tensor whose dimseion is (10, 5, 100), (TimeStep, BS, HiddenDimension). I wanna slice the tensor along the TimeStep dimension, with a tensor whose shape is (8, 5, 100) as output . The problem is, for each sample in batch the selective index is different, namely a mask matrix with shape (8, 5). One intuitive solution is as follows:</p>
<pre><code class="lang-auto">tmp = torch.zeros(8, 5, 100)
for i in range(5):
     tmp[:,i, :] = input_tensor[mask_matrix[:, i], i, :]
input_tensor = tmp
</code></pre>
<p>The operation will be done in Module’s forward func, which I think may slow down the computation. So, any suggestion about effieient solutions?</p>

ptrblck: <p>It <code>mask_matrix</code> really a mask, i.e. it contains zeros and ones or does it store indices?</p>

Chan_Simmer: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  Sorry for the ambiguity, mask_matrix stores indices.</p>

Oli: <p>Bump. Anyone got an answer?</p>

Vanjoy: <p>Bump. Anyone got an answer?</p>

1112: <p>do you get the answer?</p>
Batched torch.arange

G.M: <p>I want to create a tensor that looks like this:</p>
<pre><code class="lang-auto">[ [0, 1, 2, 3, ...],
  [0, 1, 2, 3, ...],
  ...               ]
</code></pre>
<p>I know I can do this with a loop and <code>torch.arange</code>:</p>
<pre><code class="lang-auto">tc.stack([tc.arange(0, L) for i in range(B)])
</code></pre>
<p>However, this method can be inefficient because of big python loops. I’m wondering is there a way to achieve this without loops?</p>
<p>Thanks</p>

krishnavishalv: <p>This should work,</p>
<pre><code class="lang-auto">temp = torch.ones(B, L)
result = torch.arange(0, L) * temp
</code></pre>

apsod: <p>If you just need a view, then the following should work:</p>
<pre><code class="lang-auto">torch.arange(L).expand(B, -1)
</code></pre>
Saved model have higher loss

laptopmutia: <p>hello<br>
I try to save my model while in training so that I can resume it later, but why my saved model always have higher loss compared to non resumed training?<br>
I’m following this <a href="https://discuss.pytorch.org/t/saving-and-loading-a-model-in-pytorch/2610">thread</a> to save my models, I save my decoder and encoder model and I also save my adam optimizer</p>
<pre><code class="lang-auto">def save_checkpoint(state):
    torch.save(state, os.path.join(model_path, 'checkpoint-{}-{}.pth'.format(epoch+1, i+1)))

for epoch in range(300):
    for i, (images, captions, lengths) in enumerate(data_loader):
        
        # Set mini-batch dataset
        images = images.to(device)
        captions = captions.to(device)
        targets = pack_padded_sequence(captions, lengths, batch_first=True)[0]
        
        # Forward, backward and optimize
        features = encoder(images)
        outputs = decoder(features, captions, lengths)        
        loss = criterion(outputs, targets)
        encoder.zero_grad()
        decoder.zero_grad()
        loss.backward()
        optimizer.step()
        
        if (i+1) % 50 == 0:
            print('Epoch [{}/{}], Step [{}/{}], Loss:{:.4f}, Perplexity: {:5.4f}'
                  .format(epoch, 300, i+1, total_step, loss.item(), np.exp(loss.item())))
                  
        # Save the model checkpoints
        if (epoch+1) % 100 == 0 and (i+1) % total_step == 0:
            save_checkpoint({
              'epoch': epoch + 1,
              'encoder': encoder.state_dict(),
              'decoder': decoder.state_dict(),
              'optimizer' : optimizer.state_dict(),
            })
</code></pre>
<p>you can check the whole code in here, the notebook also have training data for download <a href="https://github.com/laptopmutia/pix2codepytorch/blob/master/Pix2Code_Pytorch_SAVE_LOAD.ipynb" rel="nofollow noopener">https://github.com/laptopmutia/pix2codepytorch/blob/master/Pix2Code_Pytorch_SAVE_LOAD.ipynb</a></p>
<p>and here is the code how I load the model</p>
<p>I run this code right before training code block</p>
<pre><code class="lang-auto">#LOAD model
checkpoint = torch.load('drive/My Drive/model/checkpoint-200-350.pth')
start_epoch = checkpoint['epoch']
encoder.load_state_dict(checkpoint['encoder'])
decoder.load_state_dict(checkpoint['decoder'])
optimizer.load_state_dict(checkpoint['optimizer'])
</code></pre>
<p>edit 1:<br>
hello all after further investigating I think the problem is within my optimizer state_dict</p>
<p>because everytime I reset my environment my saved optimizer always have different state_dict</p>

Qian_Wang: <p>Hi, I have the same problem when saving adam optimizer. As you mentioned, we got different state_dict, but how can prevent it?<br>
How did you fix your code, please?</p>

yingda.yin: <p>Hi, what do you mean by “saved optimizer always have different state_dict”<br>
since you saved state_dict of optimizer and reload it, too.</p>
Computation of gradients (numerical computation or analytical derivation)

navid_mahmoudian: <p>Hello,</p>
<p>I am new to PyTorch and I have a fundamental question. I want to know how the gradients are computed in the backpropagation process? Is it computed from the analytical closed-form derivation of the function or the numerical differentiation? For example, for the simple ReLU function for which the analytical derivation is simple, do they compute the gradients numerically?</p>

krishnavishalv: <p>The gradients are not computed numerically. They are computed by a method called <a href="https://en.wikipedia.org/wiki/Automatic_differentiation" rel="nofollow noopener">“Automatic Differentiation”</a></p>
<p>These lecture slides explain it well: <a href="https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf" rel="nofollow noopener">https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/slides/lec10.pdf</a></p>

navid_mahmoudian: <p>Thanks for the answer!</p>
CUDA Extension: ImportError

dizzy77777: <p>I want to re-write the following CUDA extension, previously written for pytorch 0.4.0 and CUDA 8, to work with pytorch 1.1.0 and CUDA 10.</p>
<p><a href="https://github.com/martkartasev/sepconv/tree/master/libs/sepconv" rel="noopener nofollow ugc">https://github.com/martkartasev/sepconv/tree/master/libs/sepconv</a></p>
<p>At the moment I get the following error…</p>
<blockquote>
<p>ImportError: dynamic module does not define module export function</p>
</blockquote>
<p>I do not really know how much I have to change to get it to work.</p>
<p>thx</p>

MartinBagge: <p>Did you ever find out what the problem was?<br>
I basically have the same problem and it seems to originate from Cython but I haven’t been able to solve it yet.</p>
WeightedRandomSampler slow with replacement=False

Michiel: <pre><code class="lang-auto">import torch,torchvision
list(torch.utils.data.WeightedRandomSampler([0.1, 0.9, 0.4, 0.7, 3.0, 0.6]*10000000, 500, replacement=False))
</code></pre>
<p>takes very long to run, whereas with <code>replacement=True</code> this is more or less instantaneous.<br>
Anyone an idea why the first option takes so long?</p>

mese79: <p>Any fix for this? I have the same problem.</p>

Michiel: <p>No fix yet. A bugreport has been filed at:<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/32221#issuecomment-575270838" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/32221#issuecomment-575270838" target="_blank" rel="nofollow noopener">WeightedRandomSampler with replacement=False incredibly slow</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-01-15" data-time="14:06:30" data-timezone="UTC">02:06PM - 15 Jan 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/michielkallenberg" target="_blank" rel="nofollow noopener">
          <img alt="michielkallenberg" src="https://avatars3.githubusercontent.com/u/40028714?v=4" class="onebox-avatar-inline" width="20" height="20">
          michielkallenberg
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">🐛 Bug
WeightedRandomSampler with replacement=False incredibly slow
To Reproduce
Steps to reproduce the behavior:
import torch,torchvision list(torch.utils.data.WeightedRandomSampler([0.1, 0.9, 0.4, 0.7, 3.0, 0.6]*10000000, 500, replacement=False))
Expected behavior
Above...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">topic: performance</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
Add ReLU activation to last layer of pre-trained VGG16

ysuter: <p>Hi everyone,</p>
<p>I would like to add a ReLU activation to the last linear layer of a pre-trained VGG16 force the outputs to be positive.</p>
<p>I tried the following:</p>
<pre><code class="lang-auto">m = models.vgg16(pretrained=True)
m.classifier[6] = nn.Linear(4096, 4)
m.classifier[6].act = nn.ReLU(inplace=True)
</code></pre>
<p>Printing the model gives for the classifier part:</p>
<pre><code class="lang-auto">(classifier): Sequential(
    (0): Linear(in_features=25088, out_features=4096, bias=True)
    (1): ReLU(inplace=True)
    (2): Dropout(p=0.5, inplace=False)
    (3): Linear(in_features=4096, out_features=4096, bias=True)
    (4): ReLU(inplace=True)
    (5): Dropout(p=0.5, inplace=False)
    (6): Linear(
      in_features=4096, out_features=4, bias=True
      (act): ReLU(inplace=True)
    )
</code></pre>
<p>I get negative outputs with this. Any hints on how to do this highly appreciated.<br>
Thank you!</p>

vainaijr: <p>maybe,</p>
<pre><code class="lang-auto">x.classifier.add_module('7', nn.ReLU())
</code></pre>

ysuter: <p>Works perfectly, thank you!</p>
Using Pytorch model for processing a dataframe

banksiaboy: <p>I want to use my trained model to classify images whose file-names are stored in a column of a pandas dataframe. I would like a new column which contains the classification of the images.<br>
I have not been able to find any examples which do not produce a web-service as the deployment. Most of those are toys which classify a single image.<br>
I have my data, and a trained classifier model. I want to process  tens of thousands of images at once. Eventually I’d like to do some stream processing.<br>
— Cheers…</p>
How and when to use `torch.set_num_threads`?

lcswillems: <p>I have a laptop with 4 cores (4 CPUs I assume?). If I don’t set the number of threads (with <code>torch.set_num_threads</code>), the performance is really bad. I have two questions:</p>
<ul>
<li>which number should I give to <code>torch.set_num_threads</code>? I don’t really understand what “number of threads” means. I have 4 cores, should I set it to 4 or to 1?</li>
<li>when should I set the number of threads? At the beginning of all my Pytorch programs?</li>
</ul>
<p>Thank you in advance for your answers <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Qiong_Yang: <p>I also want to know the answer to this question.</p>

hsluoyz: <p>Also want to know + 1</p>

Muppet: <p>I also want to know the answer to this question.</p>

GoingMyWay: <p>I think if you use GPU, you can set it 1, but if you use CPUs for training, you can set it with 4 since  there are 4 cores. It depends. But you can test the speedup.</p>

Yunxiang_LI: <p>+1. And as Willems said the performance is really bad without <code>set_num_threads</code>, why we need <code>set_num_threads</code>? How it works?</p>
Torch Tensor from `numpy` complex not working

eduardo4jesus: <p>How much of the complex API is actually supported on Torch?</p>
<p><strong>1.4.0 Documentation</strong></p>
<pre><code class="lang-auto">&gt;&gt;&gt; torch.real(torch.tensor([-1 + 1j, -2 + 2j, 3 - 3j]))
tensor([ -1,  -2,  3])
</code></pre>
<p><strong>Running on Colab, Torch version: 1.4.0</strong></p>
<pre><code class="lang-auto">RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-65-6ab8515aa959&gt; in &lt;module&gt;()
----&gt; 1 torch.real(torch.tensor([-1 + 1j, -2 + 2j, 3 - 3j]))

RuntimeError: Could not infer dtype of complex
</code></pre>

anjali411: <p>Hey <a class="mention" href="/u/eduardo4jesus">@eduardo4jesus</a> this should work on the latest master build. I fixed it last week: <a href="https://github.com/pytorch/pytorch/pull/33361" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/33361</a></p>

eduardo4jesus: <p>Oh I see. I actually though about it, but then I checked that the documentation version was right. So, I got confused.</p>
<p>Any chance of having built in complex multiplication coming soon?</p>
<p>Thanks a lot.</p>

anjali411: <p>yeah we are working on adding the matrix multiplication. should be out soon</p>
Dropout rate of a saved model

rzhang63: <p>Hi,</p>
<p>I’m just wondering how I can access the dropout rate used in a saved model.<br>
When I save a model that has dropout layers, does the dropout rate also got saved inside the model.state_dict()? I can’t seem to find it in the dict.</p>

sdv4: <p>You can just load the model and then print it. For instance:</p>
<pre><code class="lang-auto">class Model_test(nn.Module):
    def __init__(self):
        super().__init__()
        self.drop_layer = nn.Dropout(0.5)

    def forward(self, inputs):
        return self.drop_layer(inputs)
    
model = Model_test()
</code></pre>
<p>Now printing the model (<code>print(model) </code>) returns:</p>
<pre><code class="lang-auto">Model_test(
  (drop_layer): Dropout(p=0.5, inplace=False)
)
</code></pre>
<p>which gives you the dropout rate you are looking for.</p>

rzhang63: <p>Yes, but I have the dropout rate as an input argument in my model class: <strong>init</strong>(self,drop_rate).</p>
<p>After training and saving the model, the next time I wish to load the model, first I need to create the model instance which actually requires me inputting the drop_rate argument. But this sets the dropout rate and then loading from the saved file does not change it.</p>

sdv4: <p>Are you the one saving the state_dict? If so, you could do something like this:</p>
<pre><code class="lang-auto">torch.save({
            'model_state_dict': model.state_dict(),
            'drop_rate': drop_rate
            }, PATH)
</code></pre>
<p>Then load it as:</p>
<pre><code class="lang-auto">checkpoint = torch.load(PATH)
drop_rate = checkpoint['drop_rate']

model = theModelClass(drop_rate)
model.load_state_dict(checkpoint['model_state_dict'])
</code></pre>

rzhang63: <p>Thanks! I’ll try this.</p>
Slang translation

Shalom.Broyer: <p>I want to translate slang sentences to formal sentences.<br>
Where most of the words in the source sentence will be part of the target sentence.<br>
Could you tell me what should be the strategy for this issue?</p>

sdv4: <p>Do you have input and target sentence pairs? If so, a logical place to start is to consider a sequence to sequence model like those used for more traditional machine translation tasks. Take a look at this tutorial:</p>
<p><a href="https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html#sphx-glr-intermediate-seq2seq-translation-tutorial-py" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html#sphx-glr-intermediate-seq2seq-translation-tutorial-py</a></p>
<p>This tutorial is well written and should get you going. Btw, your project sounds like an interesting one. Good luck!</p>
[caffe2] Post train quantization

zetyquickly: <p>Hello there,</p>
<p>Since <code>pytorch==1.3</code> has released there is an ability to perform quantization of the model during training time. And as I understand quantization works with QNNPACK.<br>
Before that we had a way to run caffe2 models using QNNPACK too.</p>
<p>The question is following is there a way to quantize caffe2 model (already trained) today?</p>

jerryzh168: <aside class="quote no-group" data-username="zetyquickly" data-post="1" data-topic="69830">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/zetyquickly/48/22489_2.png" class="avatar"> zetyquickly:</div>
<blockquote>
<p>The question is following is there a way to quantize caffe2 model (already trained) today?</p>
</blockquote>
</aside>
<p>We have internal script to quantize these models but it’s not open sourced I think. Our recommendation is to switch to PyTorch</p>
Limitations of Torch Mobile//Torchscript (Object Detection + Segmentation)

HussainHaris: <p>Hi,</p>
<p>I was wondering if its possible to script YOLO, MobileSSD, or FasterRCNN as Object Detection models to run in iOS or android via Libtorch on Pytorch mobile. Are all of these models currently unavailable? Is there a custom approach to running FasterRCNN via Torch/TorchScript on mobile?</p>
<p>Same question but in the case of Semantic Segmentation models such as PSPNet, DeepLabv3, and MaskRCNN.</p>
<p>I believe someone mentioned that TorchVision is not supported but I don’t understand the difference between that and the Resnet and Inception models that are already currently supported/scriptable for mobile. Are resnets/inceptions not considered Torchvision?</p>
<p>Thanks in advance,<br>
Haris</p>

David_Reiss: <p>Basic torchvision models are supported, but specialized ops for detection are currently not included.  We are looking into it.</p>

RicCu: <p>Is there an ETA for obj detection in PyTorch mobile? In the meantime is there a tutorial on how to use Caffe2 mobile for obj detection?</p>
Loss function with small amount of positives

Rachel_Copperman: <p>Hello,<br>
I have a binary classification model. My data is having 1 million samples, and only 100 of the are tagged with 1 (rest tagged with 0).<br>
My model now is a simple fully-connected.<br>
I’m having some problems with finding a loss function that will be able to overcome the small amount of true positives.<br>
I saw BCEWithLogitsLoss, but had some problems with it, so I used the next loss function I run into (which actually makes the same, I think) :</p>
<p>def weighted_binary_cross_entropy(output, target, weights=None):<br>
output = torch.clamp(output,min=1e-8,max=1-1e-8)</p>
<pre><code>if weights is not None:
    assert len(weights) == 2
    loss = weights[1] * (target * torch.log(output)) + \
           weights[0] * ((1 - target) * torch.log(1 - output))
else:
    loss = target * torch.log(output) + (1 - target) * torch.log(1 - output)

return torch.neg(torch.mean(loss))
</code></pre>
<p>My problem is - what values do I put in the weights?<br>
I tried to put 1 to the 0 tag andfor the 1 tag : (len(train)-len(1-tagged))/len(1-tagged)</p>
<p>But the model seems to not learn.</p>
<p>Is this way not good?<br>
Or maybe the weight are incorrect?<br>
Thanks for the helpers!</p>
Evaluate during training with distributed

maralm: <p>I am using the distributed training package to train on multiple gpus. Training works fine but I would like to be able to evaluate during training either on one gpu or multiple gpus. If I directly call evaluate function during training, each model produces different results. How can I get evaluation results every certain steps while using the distributed package for training?</p>

ptrblck: <p>If you are using DDP, the model replica should be initialized in the same manner. Since DDP performs an all-reduce step on gradients and assumes that they will be modified by the optimizer in all processes in the same way, the model output should be the same.<br>
Are you also observing different outputs during training?</p>

maralm: <p>When I evaluate during training, it runs on all gpus and each one produce different results. I am actually running this script:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/huggingface/transformers/blob/master/examples/run_glue.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/huggingface/transformers/blob/master/examples/run_glue.py" target="_blank" rel="nofollow noopener">huggingface/transformers/blob/master/examples/run_glue.py</a></h4>
<pre><code class="lang-py"># coding=utf-8
# Copyright 2018 The Google AI Language Team Authors and The HuggingFace Inc. team.
# Copyright (c) 2018, NVIDIA CORPORATION.  All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
""" Finetuning the library models for sequence classification on GLUE (Bert, XLM, XLNet, RoBERTa, Albert, XLM-RoBERTa)."""


import argparse
import glob
</code></pre>

  This file has been truncated. <a href="https://github.com/huggingface/transformers/blob/master/examples/run_glue.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>On line 248, it is mentioned “Only evaluate when single GPU otherwise metrics may not average well”. I don’t understand why and how to change it to be able to evaluate correctly.</p>
Given groups=1, weight of size 1 64 9 9, expected input[1, 1, 16, 16] to have 64 channels, but got 1 channels instead

jerry0098: <p>Hi,</p>
<p>I’m using PyTorch to implement a CNN model for collaborative filtering but I got an error like this:<br>
<strong>Given groups=1, weight of size 1 64 9 9, expected input[1, 1, 16, 16] to have 64 channels, but got 1 channels instead</strong> .</p>
<p>Input for the model is a tensor having <strong>torch.Size([1, 1, 256, 64])</strong><br>
Code for the CNN:</p>
<pre><code>CNN_modules = []
CNN_modules.append(nn.Conv2d(1, 64, (225, 33)))
CNN_modules.append(nn.ReLU())	
CNN_modules.append(nn.Conv2d(64, 1, (17, 17)))
CNN_modules.append(nn.ReLU())
CNN_modules.append(nn.Conv2d(64, 1, (9, 9)))
CNN_modules.append(nn.ReLU())
CNN_modules.append(nn.Conv2d(64, 1, (5, 5)))
CNN_modules.append(nn.ReLU())
CNN_modules.append(nn.Conv2d(64, 1, (3, 3)))
CNN_modules.append(nn.ReLU())
CNN_modules.append(nn.Conv2d(64, 1, (1, 1)))
CNN_modules.append(nn.ReLU())
</code></pre>
<p>I want to keep 64 features for all hidden conv layers. I think the error may come from the data because this tensor is not a grayscale image. Can anybody help me to solve the problem? Thank you so soo much !!!<br>
This is data of the input tensor:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/0/609e321781043b0fbae2c591b3c674552be34cb9.png" data-download-href="https://discuss.pytorch.org/uploads/default/609e321781043b0fbae2c591b3c674552be34cb9" title="Screen Shot 2020-02-24 at 9.30.56 PM"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/609e321781043b0fbae2c591b3c674552be34cb9_2_690x268.png" alt="Screen Shot 2020-02-24 at 9.30.56 PM" data-base62-sha1="dMILJcWD50cjx4uW7x5FarzifpL" width="690" height="268" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/609e321781043b0fbae2c591b3c674552be34cb9_2_690x268.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/0/609e321781043b0fbae2c591b3c674552be34cb9_2_1035x402.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/6/0/609e321781043b0fbae2c591b3c674552be34cb9.png 2x" data-dominant-color="F1F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screen Shot 2020-02-24 at 9.30.56 PM</span><span class="informations">1356×528 47 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Your second conv layer is returning an activation with a single output channel, while the next conv layer expects 64 input channels (same issue in all following layers).<br>
You should set the number of output channels to 64 and your model should work.</p>

jerry0098: <p>It works, I have misunderstanding of what out_channels is. Thank you so much for your solution.</p>
How can I visualise the filters from the first layer of the network?

Tino: <p>Hi,  I want to compare the mentioned filters before training, halfway during training and after the training is completed. I have read some resources but I am struggling with implementation.</p>
<p>This is my network</p>
<blockquote>
<p>ConvNet(<br>
(conv1): Sequential(<br>
(0): Conv2d(3, 16, kernel_size=(3, 3), stride=(1, 1))<br>
(1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(2): ReLU()<br>
(3): Dropout(p=0.3, inplace=False)<br>
)<br>
(conv2): Sequential(<br>
(0): Conv2d(16, 24, kernel_size=(4, 4), stride=(1, 1))<br>
(1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(2): ReLU()<br>
(3): Dropout(p=0.3, inplace=False)<br>
)<br>
(conv3): Sequential(<br>
(0): Conv2d(24, 32, kernel_size=(4, 4), stride=(1, 1))<br>
(1): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)<br>
(2): ReLU()<br>
(3): Dropout(p=0.3, inplace=False)<br>
)<br>
(fc1): Sequential(<br>
(0): Linear(in_features=26912, out_features=512, bias=True)<br>
)<br>
(fc2): Sequential(<br>
(0): Linear(in_features=512, out_features=10, bias=True)<br>
(1): Softmax(dim=1)<br>
)<br>
)</p>
</blockquote>

Tino: <p>I have tried using this method here:</p>
<blockquote>
<p>weight_tensor = model_gpu.conv1.weight.data.cpu()</p>
<p>a = weight_tensor[0] #</p>
<p>print(a)</p>
<p>print(a.size())</p>
<p>plt.imshow(a[0, …])</p>
</blockquote>
<p>But I keep getting the error:</p>
<blockquote>
<p>(0.0, 351.0, ‘0’, ‘black’, ‘center_baseline’, ‘right’, -7373694105942136849, None, None, 72.0, &lt;weakref at 0x7f6aeb629f98; dead&gt;, 1.2)</p>
</blockquote>

ptrblck: <p>Try to transform the kernel to a numpy array via <code>a = a.numpy()</code> and make sure you are passing this array with the expected shapes for <code>plt.imshow</code>, which is either <code>[height, width, channels]</code> or <code>[height, width]</code>.</p>

Tino: <p>Yes that worked, I can now visualize the filters before training. Do you know anyway to visualize the filters during or after training has completed?</p>

ptrblck: <p>You could just call your function to visualize the filters during training and at the end, no?</p>
Can we infer about data from Model Parameters?,

jmandivarapu1: <p>Hi All,</p>
<p>I have normal question which I thought. Imagine if I have a model(either linear regression or neural network …etc) and trained on certain Data(D) which has 1000 samples(for example) and now after I finished training if I discarded the data. So, now I am left with model (either linear regression or neural network …etc).</p>
<p>My question is Can we infer anything(literally) about the data just from the Model?</p>
<p>Just an open ended question curious to know other views or any links related to these topic.</p>
<p>Thank you all in advance</p>

albanD: <p>Hi,</p>
<p>The subfield you’re looking for is privacy preserving machine learning I think. <a href="https://towardsdatascience.com/perfectly-privacy-preserving-ai-c14698f322f5">this blogpost</a> give a nice intro about it.</p>
How to use a customized dataset for training with PyTorch/few-shot-vid2vid

bingobingolotto: <p>I’d like to use my own dataset created from the <a href="http://niessnerlab.org/projects/roessler2018faceforensics.html" rel="nofollow noopener">FaceForensics footage</a> with <a href="https://github.com/NVlabs/few-shot-vid2vid" rel="nofollow noopener"><code>few-show-vid2vid</code></a>. So I generated image sequences with <code>ffmpeg</code> and keypoints with <code>dlib</code>. When I try to start the training script, I get the following error. What exactly is the problem? The provided small dataset was working for me.</p>
<pre><code class="lang-auto">CustomDatasetDataLoader
485 sequences
dataset [FaceDataset] was created
Resuming from epoch 1 at iteration 0
create web directory ./checkpoints/face/web...
---------- Networks initialized -------------
---------- Optimizers initialized -------------
./checkpoints/face/latest_net_G.pth not exists yet!
./checkpoints/face/latest_net_D.pth not exists yet!
model [Vid2VidModel] was created
Traceback (most recent call last):
  File "train.py", line 73, in &lt;module&gt;
    train()
  File "train.py", line 40, in train
    for idx, data in enumerate(dataset, start=trainer.epoch_iter):
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in __next__
    return self._process_data(data)
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
    data.reraise()
  File "/home/keno/.local/lib/python3.7/site-packages/torch/_utils.py", line 369, in reraise
    raise self.exc_type(msg)
IndexError: Caught IndexError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/keno/.local/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in &lt;listcomp&gt;
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/keno/repos/few-shot-vid2vid/data/fewshot_face_dataset.py", line 103, in __getitem__
    Li = self.get_face_image(keypoints, transform_L, ref_img.size)
  File "/home/keno/repos/few-shot-vid2vid/data/fewshot_face_dataset.py", line 168, in get_face_image
    x = keypoints[sub_edge, 0]
IndexError: index 82 is out of bounds for axis 0 with size 82
</code></pre>
<p><strong>EDIT:</strong> How I set up my dataset. I created image sequences from the video footage with <code>ffmpeg -i _video_ -o %05d.jpg</code>, following the directory structure of the provided sample dataset. Then I generated the keypoints by using landmark detection with <code>dlib</code>, based on the code example <a href="http://dlib.net/face_landmark_detection.py.html" rel="nofollow noopener">provided on the <code>dlib</code> website</a>. I expanded the sample code to 68 points and saved them to .txt files.</p>
Will torchvison.transforms.ToTensor() normalize a PIL image?

crissallan: <p>Hello, I was recently doing some image pre-processing  using Pytorch. I use PIL to load a 1 channel image from disk, then I use PIL.ImageStat(img).mean and  PIL.ImageStat(img).stddev to obtain the mean and standard deviation of this image. After this, i converted the PIL image into tensor by using  torchvison.transforms.ToTensor() . Finally, I use torchvison.transforms.Normalize() to normalize the tensor image.<br>
The problem is, the value of the maximum pixel of the tensor image will above 1.0. And the normalized image looks wierd. Did I make any mistakes？<br>
Many thanks!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/f/7f552ac0fb701858cecfa6df871169b9e1071ffa.png" data-download-href="https://discuss.pytorch.org/uploads/default/7f552ac0fb701858cecfa6df871169b9e1071ffa" title="企业微信截图_15825135728763"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/f/7f552ac0fb701858cecfa6df871169b9e1071ffa_2_428x500.png" alt="企业微信截图_15825135728763" data-base62-sha1="iar6HZsRbIgwHoOBHjBuJp2SQaK" width="428" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/f/7f552ac0fb701858cecfa6df871169b9e1071ffa_2_428x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/f/7f552ac0fb701858cecfa6df871169b9e1071ffa_2_642x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/f/7f552ac0fb701858cecfa6df871169b9e1071ffa.png 2x" data-dominant-color="B9B9B9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">企业微信截图_15825135728763</span><span class="informations">754×879 228 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p><code>transforms.Normalize</code> will use the provided <code>mean</code> and <code>std</code> to create a tensor with zero mean and unit variance, so the values might take values outside of <code>[-1, 1]</code>.<br>
Your output might be clipped by by the <code>display</code> function, which would create these saturated regions.</p>

crissallan: <p>Thank you ! Then, in this case, is it correct to do normalization before I send the tensor image to CNN?</p>

ptrblck: <p>That’s the standard approach, as it often helps in your training.</p>
Visualising Compression of a weight quantized deep learning model

dekopra: <p>Hi</p>
<p>I have performed low bit weight quantization on certain DL models (from 8 bit to 2 bit). As you know the main motivation behind quantization is to compress the size of the model mainly the parameters by representing it in lower bit precisions, I would like to know if there is any way by which I can measure how much the weights have been compressed. I had tried using the summary method in PyTorch but it didn’t help me. If anyone has done something similar please help me out</p>
Can I change the target in criterion for each iteration?

dlwowlb: <p>Can I change the target in the criterion for each iteration?</p>
<blockquote>
<p>outputs = net(inputs)<br>
loss = criterion(outputs,outputs+inputs)</p>
</blockquote>
<p>Is it possible to update the weights and biases?<br>
or Should I bring the reinforcement learning concept?</p>
<p>Thank you!</p>
How to realize the function of looking up table with PyTorch in a most efficient way?

songyuc: <p>Hi guys,<br>
I am studying about semantic segmentation these days.<br>
And I need to transform a grey image into a matrix with the value of trainId, as<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/1/e1c3fa79ba5f30cae90f6e42dd84b63f6bd11c19.png" alt="微信图片_20200223164520" data-base62-sha1="wddjnWdAX9zpnKXZ6YYfbXzLYU1" width="313" height="329"><br>
So, I want to use PyTorch to realize the function of looking up table in a most efficient way.<br>
And your suggestion and idea will be appreciated!</p>

albanD: <p>Hi,</p>
<p>I don’t think any pytorch-specific feature is needed here. You can use regular python dictionaries to do this <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

songyuc: <p>Thanks for your kind response.<br>
And I just wanted to make sure whether there is a PyTorch function can help me realize a looking up table function.</p>
Batch normalization forward pass vs compute by hand

mystic123: <p>Hello.</p>
<p>I try to translate model from PyTorch to Tensorflow. I successfully converted net architecture and weights from PyTorch to TF but I found inconsistency in batch normalization layers. Expected results differ much between models with exactly same weights. After some investigation I found something I can’t explain.</p>
<p>In PyTorch, when I compute output of bn layer directly from equation, the result is much different from result obtained from forward pass.</p>
<p>Can anyone explain this to me?</p>
<p>(Python 3.6, PyTorch 0.4.0)</p>
<pre><code class="lang-auto">import numpy as np
import torch
import torch.nn as nn
from torch.autograd import Variable

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.module_list = nn.ModuleList()
        module = nn.Sequential()

        conv = nn.Conv2d(3, 32, 3, 1, 1, bias=False)
        module.add_module('conv_0', conv)

        bn = nn.BatchNorm2d(32)
        module.add_module('batch_norm_0', bn)

        gamma = np.random.rand(32)
        gamma = torch.from_numpy(gamma)
        bn.weight.data.copy_(gamma)

        beta = np.random.rand(32)
        beta = torch.from_numpy(beta)
        bn.bias.data.copy_(beta)

        mean = np.random.rand(32)
        mean = torch.from_numpy(mean)
        bn.running_mean.data.copy_(mean)

        var = np.random.rand(32)
        var = torch.from_numpy(var)
        bn.running_var.data.copy_(var)

        self.module_list.append(module)

    def forward(self, input):
        conv = self.module_list[0][0](input)
        bn = self.module_list[0][1](conv)
        return conv, bn


if __name__ == '__main__':
    model = Model()
    bn = model.module_list[0][1]
    gamma = bn.weight.data.numpy().reshape(1, 32, 1, 1)
    beta = bn.bias.data.numpy().reshape(1, 32, 1, 1)
    mean = bn.running_mean.numpy().reshape(1, 32, 1, 1)
    var = bn.running_var.numpy().reshape(1, 32, 1, 1)

    x = np.random.rand(1, 3, 64, 64)
    x = Variable(torch.from_numpy(x).float())
    conv_out, bn_out = model.forward(x)

    conv_out = conv_out.data.numpy()
    x = ((conv_out - mean) / np.sqrt(var + 1e-05)) * gamma + beta

    # I expect this to be negligible but is ~0.5
    print(np.sum(np.abs(bn_out.data.numpy() - x))/np.prod(x.shape))

</code></pre>

albanD: <p>Is your model in <code>eval</code> mode?</p>

mystic123: <p>Thank you! Now it works just as I expected <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=5" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

lin_ye: <p>Thank you for the good solution!</p>
How to correctly use a normal distribution?

alexis-jacq: <p>Let’s look at this small code:</p>
<pre><code class="lang-python">import torch
import torch.nn as nn
from torch.distributions import Normal

class Proba(nn.Module):
    def __init__(self, size):
        super(Proba, self).__init__()
        self.mu = nn.Linear(size, size)
        
    def forward(self, obs):
        mu = self.mu(obs)
        dist = Normal(mu, 0.2)
        log_prob = dist.log_prob(obs)
        return log_prob.mean()

proba = Proba(10)
optimizer = torch.optim.Adam(proba.parameters(), lr=1e-3)

# fixed observation to reconstruct:
obs = torch.rand(10)

# want to optimize the probability to reconstruct obs:
if __name__=="__main__":

    for i in range(500):
        loss = -proba(obs)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        if i%100==0:
            # should never be larger than 0:
            print("log_prob of obs =", -loss.item())
</code></pre>
<p>The goal of this code is to reconstruct a signal “obs” by training a linear model that overfits an element-wise normal distribution with a fixed small standard deviation (0.2).</p>
<p>However, after ~500 iterations, the predicted log_prob starts being positive, while log_probabilites should never be positive.</p>
<p>Is something wrong in the code and the utilization of <code>distributions.Normal</code>?</p>

alexis-jacq: <p>Okay, I’ve just been through the source code and understood that it’s actually the log of the density, and not the log of a probability.</p>
<p>It makes sense since the measure of the event is null. I was rather expecting the proba of being in the interval [mu-|obs|, mu+|obs|]</p>

Mauricio_Maroto: <p>Hi <a class="mention" href="/u/alexis-jacq">@alexis-jacq</a> by looking at your question, is this a way to fit probabilistic models in pytorch, say estima the mean and variance for a normal likelyhood regression model?</p>

Mauricio_Maroto: <p><a class="mention" href="/u/alexis-jacq">@alexis-jacq</a> Basically, I am trying to replicate this code from Keras to Pytorch:</p>
<pre><code class="lang-auto">def NLL(y, distr):
  return -distr.log_prob(y) #A

def my_dist(params):
  return tfd.Normal(loc=params, scale=1)

# set the sd to the fixed value 1
inputs = Input(shape=(1,))
params = Dense(1)(inputs)
dist = tfp.layers.DistributionLambda(my_dist)(params)

model_sd_1 = Model(inputs=inputs, outputs=dist)
model_sd_1.compile(Adam(), loss=NLL)
</code></pre>
<p>Here, we set up a (minimal) probabilistic neural network as a regression.</p>
<p>Any ideas how in PyTorch?</p>
No module named 'torchvision'; 'torchvision' is not a package

jingyu_han: <p>I gonna use some submodules(<code>datasets</code>, <code>models</code>,<code>transforms</code>) of <code>torchvision</code>. I have installed <code>torchvision</code> correctly by typing <code>conda install pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=10.0 -c pytorch</code><br>
then I opened a <code>python </code>envs to test my versions<br>
<img src="https://user-images.githubusercontent.com/38068286/75126573-f24bf400-56fd-11ea-9dc2-b3b1c321c22d.png" alt="Screenshot from 2020-02-24 12-05-15" width="584" height="202"><br>
everything is no problem.<br>
However, when I trying to launch my own project, I got a very strange error,</p>
<pre><code class="lang-auto">import torchvision.models as models
ModuleNotFoundError: No module named 'torchvision.models'; 'torchvision' is not a package

</code></pre>
<p>I searched for solutions to this error on google, but don’t find any useful messages. So any cues would be highly appreciated. Thank you in advance.</p>

ptrblck: <p>Are you running the script in the same environment?<br>
You would usually see this kind of error, if your current Python env does not contain the installed libs (often a mismatch of the used Jupyter Python kernel vs. your terminal).</p>
Using pytorch with colab pro 25 GB ram but only has 16 GB all the time

JonathanSum: <p>I have been using colab pro for a month, but it only has 16gb when using it. On the upper right, it says it has 25gb, but it only has 16 GB.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/e/9e28b89e9561f111b7d917a42de7f5087a84bbf3.png" data-download-href="https://discuss.pytorch.org/uploads/default/9e28b89e9561f111b7d917a42de7f5087a84bbf3" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e28b89e9561f111b7d917a42de7f5087a84bbf3_2_690x313.png" alt="image" data-base62-sha1="mz8GtgZiL1tIFzOfFYKriKGxB2b" width="690" height="313" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e28b89e9561f111b7d917a42de7f5087a84bbf3_2_690x313.png, https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e28b89e9561f111b7d917a42de7f5087a84bbf3_2_1035x469.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/9/e/9e28b89e9561f111b7d917a42de7f5087a84bbf3_2_1380x626.png 2x" data-dominant-color="434445"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1441×655 78.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/f/bfe295381525b339d424f814d929d430102dd738.png" data-download-href="https://discuss.pytorch.org/uploads/default/bfe295381525b339d424f814d929d430102dd738" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfe295381525b339d424f814d929d430102dd738_2_690x458.png" alt="image" data-base62-sha1="rnuEcEkY7uswci1WGWl0GCb1Ay4" width="690" height="458" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfe295381525b339d424f814d929d430102dd738_2_690x458.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfe295381525b339d424f814d929d430102dd738_2_1035x687.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/f/bfe295381525b339d424f814d929d430102dd738_2_1380x916.png 2x" data-dominant-color="393839"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1413×939 89.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>I assume the output of Colab mentions the system (CPU) RAM, not the GPU memory.</p>
Selecting specific labels in MNIST

utkarshsharma00: <p>I want to train MNIST first on only 0’s and 1’s, then next on 0,1 and 2 only and then next on 0,1,2, and 3 only and so on until I am training on the entire dataset. How to do this?</p>

ptrblck: <p>You could recreate the dataset using <a href="https://discuss.pytorch.org/t/change-labels-in-data-loader/36823/2">this code snippet</a> as a starter.</p>
<p>PS: Please don’t tag people in general, as this might demotivate others to post a better answer. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

jmandivarapu1: <p>You can calculate all the indexes first to avoid rewriting and reloading the data again.</p>
<pre><code class="lang-auto">train_Indexs=[]
for i in range(10) :train_Indexs.append([] )
test_Indexs = []
for i in range(10) :test_Indexs.append([] )

for num in range(0,len(trainset.labels)):
    train_Indexs[trainset.labels[num]].append(num)
for num in range(0,len(testset.labels)):
    test_Indexs[testset.labels[num]].append(num)

index_train=train_Indexs[0]#the class u want
 modified_train_labels = [1 if (trainset.train_labels[x] in postive_class) else 0 for x in index_train]#if you want to modify the label
trainset.train_labels=modified_train_labels#train set labels
trainset.train_data=trainset.train_data[index_train]#train set data
</code></pre>