akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Change model's attribute during training

username: FruitVinegar
cooked: <p>Hello.<br>
I’d like to change my model’s attribute during training.<br>
In order to that, now I’m calling my model as global variable.</p>
<p>For example:</p>
<pre><code class="lang-auto">trainer = create_supervised_trainer(~)

@trainer.on(Events.EPOCH_STARTED(once=10)
def change_attribute(engine):
    global model
    model.my_attribute = new_attribute
</code></pre>
<p>Is there more clear way using trainer?<br>
I investigated <code>trainer.__dict__</code> and <code>trainer.state.__dict__</code>, but there’s no attributes that keeps model.</p>

username: vfdev-5
cooked: <p><a class="mention" href="/u/fruitvinegar">@FruitVinegar</a></p>
<p>Trainer defined as</p>
<pre><code class="lang-auto">trainer = create_supervised_trainer(~)
</code></pre>
<p>is just a configured Engine. Engine, by default, does not store any info on model, optimizer, loss etc.<br>
This can be done if needed like that:</p>
<pre data-code-wrap="python"><code class="lang-python">trainer = ...
# trainer.state is not None in recent nightly releases: 0.4.0.dev202005XX
trainer.state.model = model
trainer.state.optimizer = optimizer
# otherwise, in stable v0.3.0, you need to set attributes in a handler attached to `Events.STARTED`

@trainer.on(Events.EPOCH_STARTED(once=10)
def change_attribute(engine):
    trainer.state.model.my_attribute = new_attribute
</code></pre>
<p>About using Engine, see <a href="https://pytorch.org/ignite/concepts.html#engine" rel="noopener nofollow ugc">https://pytorch.org/ignite/concepts.html#engine</a> .<br>
About State, please, see the documentation on it : <a href="https://pytorch.org/ignite/engine.html#ignite.engine.State" class="inline-onebox" rel="noopener nofollow ugc">ignite.engine — PyTorch-Ignite v0.4.13 Documentation</a> and <a href="https://pytorch.org/ignite/concepts.html#state" rel="noopener nofollow ugc">https://pytorch.org/ignite/concepts.html#state</a></p>
<blockquote>
<p>A good practice is to use <code>State</code> also as a storage of user data created in update or handler functions. For example, we would like to save new_attribute in the state:</p>
</blockquote>
<pre><code class="lang-auto">def user_handler_function(engine): 
    engine.state.new_attribute = 12345
</code></pre>
<p>HTH</p>
Calculating F1 score over batched data

username: ed-fish
cooked: <p>I have a multi-label problem where I need to calculate the F1 Metric, currently using SKLearn Metrics f1_score with samples as average.</p>
<p>Is it correct that I need to add the f1 score for each batch and then divide by the length of the dataset to get the correct value. Currently I am getting a 40% f1 accuracy which seems too high considering my uneven dataset.</p>
<ul>
<li>
<p>My data is multi-label an example target would be  <code>[0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1]</code></p>
</li>
<li>
<p>I am using BCEWithLogitsLoss and am using a sigmoid on the output with a threshold to get the comparable prediction (<code>predicted</code>) in code below.</p>
</li>
</ul>
<p><strong>Code Example</strong></p>
<pre><code class="lang-auto">for epoch in range(500):
    running_f = 0
    
    for batch in enumerate(custom_train_loader):
        # ...training / loss etc
        running_f += f1_score(labels.cpu().int().numpy(), predicted.cpu().int().numpy(), average='samples') * batch_size

    epoch_f1 = running_f1 / len(val_dataset)
   

</code></pre>

username: ptrblck
cooked: <p>I don’t think you can simply calculate the average of the F1 score, as shown in this small dummy example:</p>
<pre><code class="lang-python">preds = np.random.randint(0, 2, (100,))
targets = np.random.randint(0, 2, (100,))

f1_ref = f1_score(targets, preds)


f1_running = 0
batch_size = 10
for i in range(0, preds.shape[0], batch_size):
    pred = preds[i:i+batch_size]
    target = targets[i:i+batch_size]
    f1_running += f1_score(pred, target)

f1_running /= batch_size

print(f1_ref, f1_running)
&gt; 0.4444444444444445 0.423989898989899
</code></pre>
<p>You could append the current <code>labels</code> and <code>predicted</code> arrays to separate lists, create arrays from them, and calculate the F1 score after the epoch is done.</p>

username: ed-fish
cooked: <p>As suggested, my updated method. Thanks <a class="mention" href="/u/ptrblck">@ptrblck</a>.</p>
<pre><code class="lang-auto">for epoch in range(500):
    targets = []
    outputs = []

    for batch in enumerate(custom_train_loader):
        outputs.append(out)
        targets.append(label)
    
    outputs = np.concatenate(outputs)
    targets = np.concatenate(targets)

    f1 = f1_score(outputs, targets, average='samples')</code></pre>
Is it possible to give a lower learning rate on specific weight of specific layer?

username: hongjunchoi
cooked: <p>Is it possible to give a lower learning rate on specific weight of specific layer?<br>
e.g.)<br>
if the weight index list = [1,2,3,4,5]<br>
and give <strong>convolution layer 5 weight index list to 0.1</strong> learning rate<br>
and give <strong>other convolution layer 5 weight</strong> to  0.01</p>
<p>Is it possible?</p>

username: Isaac_Kargar
cooked: <p>I think you can do it by defining two optimizers and passing the desired params to each one with different learning rates:</p>
<pre><code class="lang-auto">params1 = ...
params1 = ...
opt1 = torch.optim.Adam(params1, lr1)
opt2 = torch.optim.Adam(params2, lr2)
</code></pre>
<p>or you can also do it like this, from pytorch tutorials <a href="https://pytorch.org/docs/stable/optim.html" rel="nofollow noopener">here</a>:</p>
<pre><code class="lang-auto">optim.SGD([
                {'params': model.base.parameters()},
                {'params': model.classifier.parameters(), 'lr': 1e-3}
            ], lr=1e-2, momentum=0.9)
</code></pre>
<p>This means that  <code>model.base</code> ’s parameters will use the default learning rate of  <code>1e-2</code> ,  <code>model.classifier</code> ’s parameters will use a learning rate of  <code>1e-3</code> , and a momentum of  <code>0.9</code>  will be used for all parameters.</p>

username: hongjunchoi
cooked: <p>Thanks! I’ll give it a try <img src="https://discuss.pytorch.org/images/emoji/apple/clap.png?v=9" title=":clap:" class="emoji" alt=":clap:"></p>
PyTorch sparse convlution

username: avithecoat
cooked: <p>Hi, did anyone worked with sparse convolutions in PyTorch? Should I expect a feed forward speed up increase when using a sparse cnn on gpu/cpu?<br>
Thanks!</p>

username: albanD
cooked: <p>Hi,</p>
<p>I don’t think such thing exists.<br>
It would be tricky to implement efficiently and most likely will require a very very high level of sparsity to be faster for gpu.</p>

username: yoookoo
cooked: <p>Nvidia Ampere Architecture has supported the sparisity inference <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>
Model not training properly on GPU

username: Ausizio
cooked: <p>Hi guys,</p>
<p>so I have been trying to get into reinforcement learning and have taken to a Udemy course, which has worked perfectly fine, until it came to an implementation of a deep Q learning algorithm.<br>
The issue I am having is that my model works fine on the CPU (only issue it being significantly slower), with an expected learning curve. The agent seems to learn how to play pong after 220 games and comes to an average score of 16 (over the last 100 games) after 500 games.</p>
<p>If I try to run it on my GPU (GTX980M) though to improve performance, the learning curve seems to fall off a cliff at some point, seemingly reaching a maximum of -18 points on average and then dropping back to -21. The learning curve for the runs on the GPU looks as follows:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/4/0/40a41fe9a6acc587c7af1eba2a5f2d877879ad19.png" alt="PongNoFrameskip-v4_DQN_pyTorch_50000Mem_300Games_cu_results" data-base62-sha1="9dQc04SRPDPwaDIqNKgnfr0sh7z" width="640" height="480"></p>
<p>I have since reversed to simply cloning the repository from the course ( <a href="https://github.com/philtabor/Deep-Q-Learning-Paper-To-Code/tree/master/DQN" class="inline-onebox" rel="noopener nofollow ugc">Deep-Q-Learning-Paper-To-Code/DQN at master · philtabor/Deep-Q-Learning-Paper-To-Code · GitHub</a> ), but the issue persists and I have not yet found out why. I have reinstalled PyTorch and Cuda recently without success (Python 3.6.8, Torch 1.5.0, Cuda 10.1)</p>
<p>Any ideas as to why I am getting such an odd behaviour?</p>
<p>Any input would be greatly appreciated.</p>
<p>Thanks in advance,<br>
Alex</p>

username: ptrblck
cooked: <p>How reproducible is this behavior? I.e. for 10 runs with different seeds, how many times does the CPU model converge and the GPU model diverge?</p>

username: Ausizio
cooked: <p>On the same system I have been able to reproduce the behavior 9 out of 10 times. I have not been able to reproduce it on a different system and hardware though.</p>

username: ptrblck
cooked: <p>Were you using the same setup on the different systems, i.e. same OS, PyTorch version, CUDA, cudnn, etc. or what were the differences?<br>
Did you see any hardware issues in the first system before and could you run a stress test on the GPU?</p>

username: Ausizio
cooked: <p>The second system was running the same version of Python, but other than that I am not sure.<br>
I also had the same issue on the first system with different versions of PyTorch and Cuda earlier, but I do not remember which exact versions I used, as I have since reinstalled and updated them.<br>
I did not find any hardware issues while running the stress test.</p>
What are the mean/std for LSUN

username: kirk86
cooked: <p>Does anyone know what are the mean and std for LSUN dataset.<br>
It would have been nice if the data loaders in pytorch had these values at time of construction of the data loader.</p>
<p>For instance if the dataloader would have the following args <code>mean</code>, <code>std</code>, <code>normalize</code> loaded by default.</p>
<p>MWE:</p>
<pre><code class="lang-auto">data = tv.datasets.LSUN(....)
loader = torch.utils.data.DataLoader(data, normalize=True)
loader.mean &lt;--- print mean for dataset
loader.std &lt;-- print std for dataset
</code></pre>
<p>The arg <code>normalize=True</code> would normalize the dataset by default no need to add it to transforms unless the user requires it by manually setting <code>normalize=False</code>, then s/he could pass whatever normalization they would like via the transforms.</p>
<p>Just a thought?</p>
Loading pretrained model from ".npz" file

username: Mert_Cokelek
cooked: <p>Have a good day everyone.<br>
I am trying to load a pretrained torch model, which is saved in “.npz” file.<br>
Normally, I use torch.load() for “.pt” files, however I couldn’t handle this one.<br>
How can I load it?<br>
Thanks, regards.</p>

username: ptrblck
cooked: <p><code>np.load</code> should work, as <code>npz</code> is a numpy file format.</p>

username: Mert_Cokelek
cooked: <p>There was a mismatch between layer names and I solved it by hard-coding.  Thank you.</p>
DataLoader efficiency with multiple workers

username: phiwei
cooked: <p>Hi,</p>
<p>I have noticed that my dataloader gets slower if I add more workers compared to num_workers=0.</p>
<p>My dataset definition is quite simple:</p>
<pre><code class="lang-auto">class Dataset(torch.utils.data.Dataset):
    def __init__(self, file_paths, labels, transform=None):

        self.file_paths = file_paths
        self.labels = labels
        self.transform = transform

    def __len__(self):
        return len(self.file_paths)

    def __getitem__(self, index):

        path_img = self.file_paths[index]
        image = accimage.Image(path_img)
        target = self.labels[index]

        if self.transform is not None:
            image = self.transform(image)

        return image, target
</code></pre>
<p>file_paths and labels can be quite large (above 1 million entries each). So to my understanding, the dataloader would push the dataset onto a worker every time it is indexed? Could it be slow because this creates an overhead?</p>
<p>If this is indeed the reason for slow performance, how could i work around that? Would it be possible to only parallelize loading and transforming an image where I send a path, the label and the transforms to a worker?</p>
<p>Sorry if this just reveals severe misconceptions about how parallelism is implemented here (or works conceptually in general). My setup is a single CPU with 4 connected GPUs, if that is relevant.</p>
<p>Thanks!</p>

username: ptrblck
cooked: <p>If you are using multiple workers, the <code>Dataset</code> will be copied, if I’m not mistaken.<br>
The first iteration would include these copies as well as the first batch creation in each process, which might be slow.<br>
However, the following iterations should be faster.<br>
Are you consistently seeing a slowdown using <code>num_workers&gt;=1</code> compared to <code>num_workers=0</code>?</p>
<p>Generally, <a href="https://discuss.pytorch.org/t/how-to-prefetch-data-when-processing-with-gpu/548/19">this post</a> is really helpful when it comes to data loading bottlenecks, but your issue seems to be unrelated.<br>
Are you storing the data on a local SSD or somewhere else?</p>

username: phiwei
cooked: <p>Thank you, I will look into that!</p>
Different classification results

username: oasjd7
cooked: <p>1st code is just classification test code as usual.<br>
2nd code is to get accuracy from class-wise accuracy.<br>
Why two codes have different accuracy?</p>
<pre><code class="lang-auto">def test(dataloader, model):
    model.eval()
    correct = 0
    total = 0
    for batch_idx, (inputs, labels) in enumerate(dataloader):
        inputs, labels = inputs.cuda(), labels.cuda()
        with torch.no_grad():
            outputs = model(inputs)
        _, predicted = outputs.max(1)
        total += labels.size(0)
        correct += predicted.eq(labels).sum().item()
    model.train()
    return correct / total
</code></pre>
<pre><code class="lang-auto">confusion_matrix = torch.zeros(nb_classes, nb_classes)
model.eval()
for batch_idx, (inputs, labels) in enumerate(dataloader):
    inputs, labels = inputs.cuda(), labels.cuda()
    outputs = model(inputs)
    _, preds = torch.max(outputs, 1)
    for t, p in zip(labels.view(-1), preds.view(-1)):
        confusion_matrix[t.long(), p.long()] += 1
ele_wise_acc = confusion_matrix.diag() / confusion_matrix.sum(1)
total_acc = ele_wise_acc.mean()
model.train()
return total_acc
</code></pre>

username: ptrblck
cooked: <p>Try to disable shuffling in the <code>DataLoader</code> and compare the outputs of both code snippets.<br>
Also make sure that the indexing in <code>confusion_matrix</code> is right and compare the number of class predictions to the <code>correct</code> value in the first code.</p>
Mismatch in batch size

username: Tehreem
cooked: <p><strong>When i am trying to train the model, i get the following mismatch error given at the end of this question. As i am new to PyTorch, i am having difficulties in understanding how to resolve this issue. Kindly guide me what changes should i made in my code.</strong></p>
<pre><code>import torch
import torch.nn as nn
import torch.nn.functional as F

class  CharCNN(nn.Module):
 def __init__(self):
    super(CharCNN, self).__init__()
    
    self.conv1 = nn.Sequential(
        nn.Conv1d(num_channels, depth_1, kernel_size=kernel_size_1, stride=stride_size),
        nn.ReLU(),
        nn.MaxPool1d(kernel_size=kernel_size_1, stride=stride_size),
        nn.Dropout(0.1),
    )

    self.conv2 = nn.Sequential(
        nn.Conv1d(depth_1, depth_2, kernel_size=kernel_size_2, stride=stride_size),
        nn.ReLU(),
        nn.MaxPool1d(kernel_size=kernel_size_2, stride=stride_size),
        nn.Dropout(0.25)
    )            
        
         
    self.fc1 = nn.Sequential(
        nn.Linear(128*64, num_hidden),
        nn.ReLU(),
        nn.Dropout(0.5)
    )
    
    self.fc2 = nn.Sequential(
        nn.Linear(num_hidden, 11),
        nn.ReLU(),
        nn.Dropout(0.5)
    )
    
    

def forward(self, x):
    out = self.conv1(x)
    out = self.conv2(out)

    # collapse
    out = out.reshape(-1,128*64)
    #out = out.view(out.size(0), -1) 
    # linear layer
    out = self.fc1(out)
    # output layer
    out = self.fc2(out)
    #out = self.log_softmax(x,dim=1)
    
    return out
</code></pre>
<p><strong>This is my model, when i try to train my network like this:</strong></p>
<pre><code> criterion = nn.CrossEntropyLoss()  
 opt = torch.optim.Adam(model.parameters(),lr=learning_rate)

for e in range(training_epochs):
   if(train_on_gpu):
    net.cuda()
   train_losses = []    

 for batch in iterate_minibatches(train_x, train_y, batch_size):
    x, y = batch
    inputs, targets = torch.from_numpy(x), torch.from_numpy(y)
    if(train_on_gpu):
        inputs, targets = inputs.cuda(), targets.cuda()
        #inputs= input.view(batch_size,  128*64)
        #targets = targets.view(batch_size)
    opt.zero_grad()   
    output = model(inputs)
        
    loss = criterion(output, targets.long())
    train_losses.append(loss.item())
    loss.backward()
    opt.step()
val_losses = []
accuracy=0
f1score=0
print("Epoch: {}/{}...".format(e+1, training_epochs),
              "Train Loss: {:.4f}...".format(np.mean(train_losses)))
</code></pre>
<p><strong>I get the following error:</strong></p>
<pre><code> ValueError: Expected input batch_size (1) to match target batch_size (64).
</code></pre>
<p><strong>Please help me to resolve this issue.</strong></p>

username: ptrblck
cooked: <p>Also posted <a href="https://discuss.pytorch.org/t/training-1d-cnn-in-pytorch/83525/10">here</a> with follow-up.</p>
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[5, 64, 70, 70] to have 3 channels, but got 64 channels instead

username: CS.Enthu
cooked: <p>I am implementing cycle gans .<a href="https://github.com/arnab39/cycleGAN-PyTorch" rel="nofollow noopener">https://github.com/arnab39/cycleGAN-PyTorch</a>.</p>
<p>I have changed the code according to my custom dataset which are dicom images. After extracting the image patches and loading into the network .</p>
<p>I get the above mentioned error. I assume this is because input channels for RGB are 3.<br>
The size of my patches are torch.Size([112, 64, 64, 64]).</p>
<p>Can anyone explain how can I modify for my image dataset.</p>
<p>"<br>
RuntimeError: Given groups=1, weight of size [64, 3, 7, 7], expected input[5, 64, 70, 70] to have 3 channels, but got 64 channels instead</p>
<p>Process finished with exit code 1<br>
"</p>

username: ptrblck
cooked: <p>Answered <a href="https://discuss.pytorch.org/t/cycle-gans-error-runtimeerror-dataloader-worker-pid-s-30412-26524-28704-10456-30900-exited-unexpectedly/83104/7">here</a>.<br>
Let’s keep the discussion in this thread.</p>
Default initialization behavior

username: Yashas
cooked: <p>The initialization is layer-dependent. How does pytorch seed the RNGs by default?</p>
<p>If I have to train a model N times to see the average performance of the model, do I have to have special code to ensure that the initializations are different? Can I assume that I have different initialization on each training run?</p>

username: ptrblck
cooked: <p>I don’t think PyTorch seeds the code by default, as this would mean you could get deterministic result for each run, which is not the case.</p>
<p>You can execute this core repeatedly in your terminal and will get different values:</p>
<pre><code class="lang-auto">python -c "import torch; print(torch.randn(10))"
</code></pre>
Time_step of LSTM in classification

username: Array
cooked: <p>Hello everyone,</p>
<p>I am now dealing with string classification, and I want to use LSTM, but I have some questions about setting time_step. My data set looks like:</p>
<pre><code class="lang-auto">Data ----- Label

DKWL----0
FCHN----0
KDQP----0
IHGS----1
....
</code></pre>
<p>then I convert it into binary code:</p>
<pre><code class="lang-auto">
00011101000001000111-----1
.....
shape:(N,20)
</code></pre>
<p>Now I build LSTM:</p>
<pre><code class="lang-auto">class RNN(nn.Module):
    def __init__(self):
        super(RNN, self).__init__()

        self.rnn = nn.LSTM(     
            input_size=20,     # Each sample has 20 features 
            hidden_size=64,     
            num_layers=1,       
            batch_first=True,   
        )

        self.out = nn.Linear(64, 2) # I set output 2, because there are two classes 0 &amp; 1

    def forward(self, x):
        r_out, (h_n, h_c) = self.rnn(x, None)   
        out = self.out(r_out[:, -1, :])
        return out
</code></pre>
<p>Then I try to train it with following codes:</p>
<pre><code class="lang-auto">for epoch in range(EPOCH):
    for step, (x, b_y) in enumerate(train_loader):   # gives batch data
        b_x = x.view(-1, ?, 20)   # at this step I need to reshape x to
                                  # (batch, time_step, input_size), but how I set this 
                                  # time_step? 0 or 1?
        output = rnn(b_x)               
        loss = loss_func(output, b_y)   
        optimizer.zero_grad()          
        loss.backward()                 
        optimizer.step()  
</code></pre>
<p>Thanks in advance.</p>

username: ptrblck
cooked: <p>I’m not sure to fully understand the use case.<br>
How is dim1 defined in your input of shape <code>[N, 20]</code>? Is it the feature or temporal dimension?</p>
<p>In the first case, you should use <code>x.view(-1, 1, 20)</code>, while in the latter <code>x.view(-1, 20, 1)</code>.<br>
Based on the definition of your <code>LSTM</code>, it seems that the first approach would be the valid one.</p>
The type error, make code doesn't work

username: ActonMartin
cooked: <p>when I use a pytorch-template for my dataset, some errors just like monster on the way. When I solve one, another one is coming and say hello to me. But this error make me feel wondering.<br>
I just write my dataset dataloader in dataloader/dataloaders.py;<br>
make a new loss in model/loss.py</p>
<p>THE ERROR HINTS IS</p>
<pre><code class="lang-auto">Trainable parameters: 31042369
Train Epoch: 1 [0/27 (0%)] Loss: 0.669998
Traceback (most recent call last):
  File "train.py", line 68, in &lt;module&gt;
    main(config)
  File "train.py", line 49, in main
    trainer.train()
  File "E:\kaggle\pytorch-template\base\base_trainer.py", line 66, in train
    result = self._train_epoch(epoch)
  File "E:\kaggle\pytorch-template\trainer\trainer.py", line 60, in _train_epoch
    self.writer.add_image('input', make_grid(data.cpu(), nrow=8, normalize=True))
  File "D:\Anaconda3\lib\site-packages\torchvision\utils.py", line 67, in make_grid
    norm_range(tensor, range)
  File "D:\Anaconda3\lib\site-packages\torchvision\utils.py", line 61, in norm_range
    norm_ip(t, float(t.min()), float(t.max()))
  File "D:\Anaconda3\lib\site-packages\torchvision\utils.py", line 55, in norm_ip
    img.add_(-min).div_(max - min + 1e-5)
RuntimeError: result type Float can't be cast to the desired output type Byte
</code></pre>
<p>And I upload code to <a href="http://git@github.com:ActonMartin/Unet_pytorch.git" rel="nofollow noopener">git@github.com:ActonMartin/Unet_pytorch.git</a></p>

username: ptrblck
cooked: <p>It seems that <code>img</code> is a <code>ByteTensor</code>, which you are trying to normalize with floating point values.<br>
This simple code snippet will raise the same issue:</p>
<pre><code class="lang-python">img = torch.randint(0, 255, (3, 24, 24)).byte()
img.add_(torch.tensor(1.))
</code></pre>
<p>Try to transform <code>img</code> to a <code>FloatTensor</code> via <code>img = img.float()</code> before normalizing it.</p>

username: ActonMartin
cooked: <p>THANK YOU. <img src="https://discuss.pytorch.org/images/emoji/apple/yum.png?v=9" title=":yum:" class="emoji" alt=":yum:"> Finally it works.AWESOME.</p>
Why can i import "import_class"

username: Sujin_Jo
cooked: <p>import argparse<br>
import sys<br>
import torchlight<br>
from torchlight import import_class</p>
<p>Traceback (most recent call last):<br>
File “/Users/josujin/Downloads/AS-GCN-master/main.py”, line 4, in <br>
from torchlight import import_class<br>
ImportError: cannot import name ‘import_class’ from ‘torchlight’ (/Users/josujin/anaconda3/lib/python3.7/site-packages/torchlight/<strong>init</strong>.py)</p>
<p>how can i import import_class from torchlight?</p>
<p>my environment :<br>
anaconda3<br>
python3.7.6</p>

username: SANTOSH_S
cooked: <p>Hi <a class="mention" href="/u/sujin_jo">@Sujin_Jo</a><br>
Can you try this steps out: <a href="https://github.com/limaosen0/AS-GCN/issues/4#issuecomment-554174658" rel="nofollow noopener">link</a></p>
<p>Let me know if this worked for you.</p>

username: Sujin_Jo
cooked: <p>First, Thanks for ur reply.</p>
<p>That problem has been solved,<br>
but I have new problem.</p>
<p>Traceback (most recent call last):<br>
File “/Users/josujin/AS-GCN/main.py”, line 14, in <br>
processors[‘demo’] = import_class(‘processor.demo.Demo’)<br>
File “/Users/josujin/AS-GCN/torchlight/io.py”, line 186, in import_class<br>
__ import__(mod_str)<br>
<strong>ModuleNotFoundError: No module named 'processor.demo’</strong></p>
<p>So, I did commented out this line,<br>
<strong>processors[‘demo’] = import_class(‘processor.demo.Demo’)</strong></p>
<p>Can I do this?</p>

username: SANTOSH_S
cooked: <p>Yes. That should solve your issue.<br>
Let me know if it worked as expected.</p>
Why is torch.Tensor.fill_() not named torch.Tensor.fill()?

username: johnestar
cooked: <p>I’m curious why this API is named with a trailing underscore, how do we know whether there should be a trailing underscore or not without referring to the documentation? Curious what is the convention here</p>

username: Nikronic
cooked: <p>Hi,</p>
<p>There is a <code>tensor.function_</code> for every <code>tensor.function</code>. The difference is that first one is in-place operation while the latter is not. Both of them use same operations.<br>
In <a href="https://pytorch.org/docs/stable/tensors.html" rel="nofollow noopener">this page</a> (and some others), you will find similar functions.</p>
<p>For instance,</p>
<pre><code class="lang-auto">x = torch.tensor([1,2,3])
print(x)  # tensor([1, 2, 3])
print(x.add(1)) # tensor([2, 3, 4])
print(x)  # tensor([1, 2, 3])
print(x.add_(1))  # tensor([2, 3, 4])  -&gt; inplace operation, changes the value of x in memory
print(x)  # tensor([2, 3, 4])
</code></pre>
<p>Bests</p>
Validation loss increasing instead of decreasing

username: billal5542
cooked: <p>Hi Guys,<br>
I am very new at pytorch and deep learning. I am trying to use a CNN+LSTM structure for a regression problem. Where input is time series data (1,5120). CNN is for feature extraction purpose. My output is (1,2) vector. Even though my training loss is decreasing, the validation loss does the opposite. I tried several things, couldn’t figure out what is wrong.<br>
Can you give me any suggestion?<br>
here is my network. the MSE loss  plots</p>
<p>class ConvNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(ConvNet, self).<strong>init</strong>()<br>
self.layer1 = torch.nn.Conv1d(in_channels=1, out_channels=5, kernel_size=3, stride=2,bias=0)<br>
self.pool = nn.MaxPool1d(3, stride = 2)<br>
self.layer2 = torch.nn.Conv1d(in_channels=5, out_channels=10, kernel_size=3, stride=2,bias=0)<br>
self.drop_out = nn.Dropout2d()<br>
self.fc1 = nn.Linear(512, 512,bias=True)</p>
<pre><code>def forward(self, x):
    
    out = self.layer1(x)
    out = F.relu(out)
    out = self.pool(out)
    out = self.layer2(out)
    out = F.relu(out)
    out = out.reshape(out.size(0), -1)
    '''
    out = self.drop_out(out)
   
    out = self.fc1(x)
    out = F.relu(out)
    '''
    return out
</code></pre>
<p>class Combine(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(Combine, self).<strong>init</strong>()<br>
self.cnn = ConvNet()<br>
self.lstm1 = nn.LSTM(<br>
input_size=639,<br>
hidden_size=128,<br>
num_layers=1,<br>
bidirectional = True,<br>
batch_first=True)<br>
self.drop_out = nn.Dropout()<br>
self.lstm2 = nn.LSTM(<br>
input_size=256,<br>
hidden_size=100,<br>
num_layers=1,<br>
bidirectional = False,<br>
batch_first=True)<br>
self.batchnorm = nn.BatchNorm1d(10 *100,affine = False)<br>
self.fc = nn.Linear(1000,2, bias=True)<br>
def forward(self, x):<br>
xin = x.view(x.size(0),1,-1)<br>
cn_out = self.cnn(xin)</p>
<pre><code>    c_in = cn_out.view(batch.size(0), 10, 639)
    
    h0 = Variable(torch.zeros(2, batch.size(0), 128)).cuda()
    c0 = Variable(torch.zeros(2, batch.size(0), 128)).cuda()
    
    outputs, (ht, ct) = self.lstm1(c_in,(h0,c0))
    outputs = self.drop_out(outputs)
    
    h0 = Variable(torch.zeros(1, batch.size(0), 100)).cuda()
    c0 = Variable(torch.zeros(1, batch.size(0), 100)).cuda()
    
    outputs = outputs.view(batch.size(0), 10,256)
    out3,(ht, ct)  = self.lstm2(outputs)
    out = self.fc(out3.contiguous().view(batch.size(0), -1))
    return out
</code></pre>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/3/738df30ab2be069933c65947ab2861af0f27358a.png" alt="image" data-base62-sha1="guf27EIK7Wr22OQ5DYtemul1LMC" width="378" height="252"> <img src="https://discuss.pytorch.org/uploads/default/original/3X/7/1/7175defb91d292e11376e28cde65fdfb3f7bf361.png" alt="training" data-base62-sha1="gbIutYCPhu8r9AsIjKY7OBUAw0x" width="378" height="252"></p>
Qr decomposition method

username: Jonathan_Brokman
cooked: <p>tl;dr what is the method used for QR decomposition? Is it Gram Schmidt?</p>
<p>Given a matrix M we perform Q, R = torch.qr(M).</p>
<p>Initially I assumed  Q’s columns are achieved by applying the Gram-Schmidt procedure on M’s columns.</p>
<p>But then I noticed that even fro cases in which M’s columns are linearly dependent - Q’s columns still turn up othonormal (even for cases where Q’s and M’s dimension are equal). This means that span(Q) is not equal to span(M)</p>
<p>Since this indicates the method is not Gram Schmidt, I am curious about which method you use for the torch.qr() function</p>
<p>Thanks, Jonathan</p>

username: ptrblck
cooked: <p>The <code>qr</code> call eventually dispatches to <a href="https://icl.cs.utk.edu/projectsfiles/magma/doxygen/group__magma__geqrf.html#ga6998f468207dbed0239ebcfa5f110f8e">magma</a> or <a href="https://www.netlib.org/lapack/lug/node40.html">lapack</a>, which provide some information about their implementation.</p>
<p>Let me know, if that answers your question. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
MobileNet-V1-SSD Transfer Learning

username: Jimmy_2times
cooked: <p>Hello<br>
I’m totally new to transfer learning.<br>
I’m did transfer learning on a MobileNet-V1-SSD to detect strawberries in a picture.<br>
I used plastic strawberries which are all identical.</p>
<p>My training data size is 424, test is around 50, validation around 15 (small dataset)<br>
epochs 130<br>
learning rate 0.01<br>
t_max 100<br>
basenet learning rate 0.001<br>
scheduler cosine</p>
<p>The lowest loss was around 1.92</p>
<p>The strawberry pictures were taken of different sizes and angles and all on a white background.</p>
<p>The resulting model was very fast (on Jetson Nano) but it was taking pretty much everything for a strawberry (table, hand, head, cell phone… )</p>
<p>Since I’m completely new to this, I wonder if there is something obvious that I’m doing wrong?<br>
Is there anything that looks wrong in what I’m doing? Too small dataset, white background… ? Poor parameters?</p>
<p>Thanks a lot!!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/e/ee47121d9191fa13d6ed158c1f06a49f152abd16.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/ee47121d9191fa13d6ed158c1f06a49f152abd16" title="20200531_235814"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47121d9191fa13d6ed158c1f06a49f152abd16_2_690x335.jpeg" alt="20200531_235814" data-base62-sha1="xZTUc6JtLHcrPsL5jE324unJmyW" width="690" height="335" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47121d9191fa13d6ed158c1f06a49f152abd16_2_690x335.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47121d9191fa13d6ed158c1f06a49f152abd16_2_1035x502.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/e/ee47121d9191fa13d6ed158c1f06a49f152abd16_2_1380x670.jpeg 2x" data-dominant-color="B7B1AF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">20200531_235814</span><span class="informations">2016×980 863 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
Questions about the Stream object destroy

username: Zebulun
cooked: <p>Hello, there is a problem with code reading in CUDAStream +32: LeakyStreamInternals objects need to be destroyed here, and cudaStream_t also needs to be released using cudaStreamDestroy(stream). It seems strange that the code and the comments are there.</p>
<p>Where is cudaStreamDestroy called that causes it to be destroyed repeatedly?</p>
<p><a href="https://github.com/pytorch/pytorch/blob/master/c10/cuda/CUDAStream.cpp" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/c10/cuda/CUDAStream.cpp</a> +32</p>

username: ptrblck
cooked: <p>Most likely the CUDA runtime was already destroyed at some point, so that the deallocation would raise a segmentation fault and is thus removed.</p>
Learning Transfer or Fine tuning Yolo or MobilenetV2

username: Jimmy_2times
cooked: <p>Hi<br>
I was wondering if it is possible to train <strong>YOLOv2</strong> or <strong>MobileNetV2</strong> with PyTorch?<br>
I’m looking into building a real-time webcam object detector and do transfer learning to teach it to recognize specific objects of my own.<br>
These two would be perfect for my application.<br>
Thanks!</p>

username: zarkopafilis
cooked: <p>You can use pretrained versions of these models.</p>
<p>Here is a very good 3-loc example straight from pytorch hub:</p>
<aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://pytorch.org/favicon.ico?" class="site-icon" width="15" height="15">
      <a href="https://pytorch.org/hub/pytorch_vision_mobilenet_v2/" target="_blank" rel="nofollow noopener">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="https://pytorch.org/assets/images/pytorch-logo.png" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://pytorch.org/hub/pytorch_vision_mobilenet_v2/" target="_blank" rel="nofollow noopener">PyTorch</a></h3>

<p>An open source deep learning platform that provides a seamless path from research prototyping to production deployment.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


username: Jimmy_2times
cooked: <p>Thanks a lot, appreciate it!!</p>
Cache Memory Allocation

username: maralm
cooked: <p>Wondering what can cause increase in cache memory?<br>
In my code, torch.cuda.max_memory_allocated(0) shows only 3GB but nvidia-smi shows 10GB. I expected some cache but not 3 times the real memory. How should I investigate and reduce that?<br>
I think empty_cache will slow down the algorithm, right?</p>

username: ptrblck
cooked: <p><code>empty_cache</code> will force PyTorch to reallocate the memory, if necessary, and thus might slow down the code.<br>
The large cache might be created during the training, as the forward pass will create intermediate activations, which could be needed for the gradient calculation during the backward pass.<br>
E.g. a simple <code>nn.Conv2d</code> might create an activation, which uses a multiple of the memory footprint of the input tensor, if the number of <code>out_channels</code> is large.</p>

username: maralm
cooked: <p>I understand.<br>
My problem is that I am deleting a lot of activation and other tensors after using them in my code and when their are not used any more but their memory also goes into the cache. I don’t want to use empty_cache() since as you said it will reallocate the memory needed for the model and slows down the code. But is there a way to avoid deleted tensor’s memory go into the cache?</p>

username: maralm
cooked: <p>another example is cloning a tensor which creates cache!</p>

username: ptrblck
cooked: <p>Why would you like to avoid creating the cache?<br>
This would force PyTorch to reallocate device memory, which will synchronize the code and thus might introduce a potential performance hit.</p>

username: maralm
cooked: <p>I can have 2x to 4x increase in batch size if I don’t have cache for those deleted tensors. So probably I can gain performance there.</p>

username: ptrblck
cooked: <p>How did you increase the batch size by removing the cache?<br>
Note that the memory stored in the cache can be reused.<br>
If you empty the cache, PyTorch would have to reallocate the memory.<br>
Freeing the cache should not save memory, but just slow down your code due to <code>free</code> and <code>alloc</code> calls.</p>
How to compute gradients and do back propagation by a skipping way

username: Cerr
cooked: <p>Hi, I got confused about how to compute gradients and do back propagation by a skipping way.<br>
For example, I have a net as follows</p>
<pre><code class="lang-python">class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        
        self.A = ModuleA()
        self.B = ModuleB()

    def forward(self, x, y):
        a = self.A(x)
        b = self.B(a)
        c = self.B(y)
        return b, c

loss = Loss(b) + Loss(c)
optimizer.zero_grad()
loss.backward()
optimizer.step()
</code></pre>
<p>When execute backward and step, <code>Loss(c)</code> will lead to updating weights of <code>ModuleB</code>, that’s OK. But at the same time, what if I want to update weights of <code>ModuleA</code> without updating weights of <code>ModuleB</code> according to <code>Loss(b)</code>?<br>
I tried to use</p>
<pre><code class="lang-python">    with torch.no_grad():
        b = self.B(a)
</code></pre>
<p>but it doesn’t look like a reasonable solution for my problem.<br>
Are there any other better solutions? Thanks a lot!</p>

username: albanD
cooked: <p>Yes doing with torch.no_grad() will prevent gradient from flowing back all the way to moduleA.</p>
<p>You can set the requires_grad flag on each parameter to False though to avoid that:</p>
<pre><code class="lang-auto">self.B.requires_grad_(False)
b = self.B(a)
self.B.requires_grad_(True)
c = self.B(y)
</code></pre>
<p>That will make sure that what is computed using <code>b</code>, won’t update the gradients of module.B.</p>
Back propagate about leaf node

username: JIE_LIU
cooked: <p>Afer <code>tensor.detach()</code> operation, the tensor will be a leaf node and the grad will not be backpropagated even if the tensor is the intermediate output.<br>
Is this because the all leaf node wouldn’t backpropagate? Or other reasons?<br>
thanks.</p>

username: albanD
cooked: <p>Hi,</p>
<p>No it is because the <code>.detach()</code> breaks the link between the two. And so the backprop cannot go back.</p>
Autoencoders Types

username: Mike2004
cooked: <p>Hi, can someone explain to me the differences between this models of autoencoders: <a href="https://github.com/L1aoXingyu/pytorch-beginner/tree/master/08-AutoEncoder" rel="nofollow noopener">https://github.com/L1aoXingyu/pytorch-beginner/tree/master/08-AutoEncoder</a></p>
<p>And explain me better the network structure of all of them? I want to know more about this models.<br>
Thanks.</p>

username: albanD
cooked: <p>Hi,</p>
<p>The best way to do this is to google each name and read the relate paper. Each of them has quite a different approach and a lot of details into them <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Mike2004
cooked: <p>Thank you, i already did that. For the simple one, how can i classify it?</p>

username: albanD
cooked: <p>You might want to check the code in more details, but the “basic architecture” described in the wikipedia article seems to match the code in spirit.</p>
Softmax activation

username: Arman
cooked: <p>hi im new to pytorch there is something that keeps my mind busy<br>
i see these code for classifying mnist</p>
<p>class Classifier(nn.Module):</p>
<pre><code>def __init__(self, input_featurs, h1, h2,  output_featurs):
    super().__init__()
    self.linear= nn.Linear(input_featurs, h1)
    self.linear1 = nn.Linear(h1, h2)
    self.linear2 = nn.Linear(h2, output_featurs)
      
def forward(self, x):
    x =F.relu(self.linear(x))
    x =F.relu(self.linear1(x))  
    x =self.linear2(x)
    return x 
</code></pre>
<p>why we dont use softmax activation function for last  layer<br>
thank you</p>

username: Caruso
cooked: <p>It could be that <code>nn.CrossEntropyLoss</code> was used as a loss function, which does combine <code>F.log_softmax</code> and <code>nn.NLLLoss</code> (<a href="https://pytorch.org/docs/stable/nn.html#crossentropyloss" rel="nofollow noopener">https://pytorch.org/docs/stable/nn.html#crossentropyloss</a>). So therefor you don’t need a activation function at the end!</p>
<p>Softmax could also be applied later on in the training loop, right before the loss calculation!</p>
Use to use GPU if available on mobile

username: Bernicchi
cooked: <p>Hi</p>
<p>I have neural network that I trained on my PC and then jit compiled and run inferences on my Android phone. I can see from Android Studio debugger it is processing using the mobile’s CPU. How can I check and use GPU if available on android mobile?</p>
<p>Thanks in advance</p>

username: albanD
cooked: <p>Hi,</p>
<p>Unfortunately, mobile GPU is a completely different language and architecture to everything else. So we don’t support it yet.<br>
It is work in progress though and you can see all the PRs working it here: <a href="https://github.com/pytorch/pytorch/pulls?q=is%3Apr+is%3Aopen+vulkan">https://github.com/pytorch/pytorch/pulls?q=is%3Apr+is%3Aopen+vulkan</a></p>
Efficient layer combination needed

username: cruu
cooked: <p>Hi, I’d like to train a NN with a given dataset (all including some kind of object, for example: a dog), after the training the NN should help me classifying my images (downloaded from instagram) as “image includes a dog (with probability:0.XX)” or “image doesn’t include a dog (with probability: 0.XX)”.</p>
<p>Obviously images from instagram-images do not always have the same size (but they all have the same format (.jpg) due to filtering), and the images from my dataset do not have the same size as well.</p>
<p>How should I handle this? Wich NN Architecture is recommended?<br>
Should I just center crop all images to the same size?</p>
<p>While testing I’m using the " Stanford Dogs Dataset", but the final product should be able to handle different training datasets as well.<br>
The NN does not have to be extremely good, I just want to filter the downloaded images in the groups:</p>
<ol>
<li>has the given object</li>
<li>don’t has the given object</li>
</ol>
<p>With, for example, the confidence of 0.6 .</p>
<p>Thanks in advance</p>

username: ngoan.dev
cooked: <p>I think you just need resize all images to have the same size.</p>

username: cruu
cooked: <p>Fixed all bugs now.<br>
Just need help with the neuronal network architecture.<br>
I have images with the type: input[1, 3, 256, 256]. Now my Questions:</p>
<p>What kind of layer combination would work well?<br>
What parameters do I have to use in the first layer?<br>
For my kind of use the output layer should have 2 neurons, right?</p>
<p>My Code: <a href="https://hastebin.com/igekafidos.py" rel="nofollow noopener">https://hastebin.com/igekafidos.py</a></p>
<p>Thanks for every reader.</p>
Masked instance norm

username: Fawaz_Sammani
cooked: <p>Hello. I would like to use instance normalization (1d), however I cannot use  nn.InstanceNorm1d because my objects are masked. For example, I have an input of shape <code>batch_size (N), num_objects (L), features(C)</code>, and each batch has different number of objects, and the number of objects is not fixed. Therefore, I have a boolean mask of shape <code>batch_size (N), num_objects (L)</code> for that. So using nn.InstanceNorm1d would compute the mean and std across the padded objects, which I don’t want. Therefore, I’ve implemented it from scratch. But it dosen’t seem to give me the accuracy I am expecting (i know the accuracy because i am re-implementing a paper). It would be helpful to check my code.</p>
<p><strong>EDIT:</strong> I realized that the problem was that detaching the mean and var calculation is necessary. I placed the detach below in the code.</p>
<pre><code class="lang-auto">def masked_instance_norm(x, mask, eps = 1e-5):
    """
    x of shape: [batch_size (N), num_objects (L), features(C)]
    mask of shape: [batch_size (N), num_objects (L)]
    """
    mask = mask.float().unsqueeze(-1)  # (N,L,1)
    mean = (torch.sum(x * mask, 1) / torch.sum(mask, 1))   # (N,C)
    mean = mean.detach()
    var_term = ((x - mean.unsqueeze(1).expand_as(x)) * mask)**2  # (N,L,C)
    var = (torch.sum(var_term, 1) / torch.sum(mask, 1))  #(N,C)
    var = var.detach()
    mean_reshaped = mean.unsqueeze(1).expand_as(x)  # (N, L, C)
    var_reshaped = var.unsqueeze(1).expand_as(x)    # (N, L, C)
    ins_norm = (x - mean_reshaped) / torch.sqrt(var_reshaped + eps)   # (N, L, C)
    return ins_norm
</code></pre>
<p>Verifying using PyTorch nn.InstanceNorm (without mask):</p>
<pre><code class="lang-auto"># PyTorch nn.InstanceNorm
m = nn.InstanceNorm1d(100)
input = torch.randn(20, 100, 40)
pytorch_output = m(input)

# Mines
input = input.permute(0,2,1)  # shape required by nn.InstanceNorm1d
mask = torch.ones(20,40)   # mask is all ones for comparison purpose
my_output = masked_instance_norm(input, mask)
my_output = my_output.permute(0,2,1) # back to original shape
</code></pre>
<p>They give the same result</p>
Extract features pretrained Resnet50

username: Giuseppe
cooked: <p>Hy guys, I want to extract the in_feature(2048) of FC layer, passing an image to resnet50.</p>
<pre><code class="lang-auto">def get_vector(image):
    
    layer = model._modules.get('avgpool')
    
    my_embedding = torch.zeros(2048) #2048 is the in_features of FC , output of avgpool
   
    def copy_data(m, i, o):
        my_embedding.copy_(o.data)
        
    
    h = layer.register_forward_hook(copy_data)
    
    model(image)#error
  
    h.remove()
    
    # return the vector
    return my_embedding
</code></pre>
<pre><code class="lang-auto">from PIL import Image
from torch.utils.data import DataLoader
from torchvision import transforms
from os import listdir
from os import path
import time
import h5py
import pandas as pd

since = time.time()
print("---- START FEATURES EXTRACTION ----")
print

column = ["FlickrID", "Features"]

path = "./train_dataset/train_imgs/"

pathCSV = "./train_dataset/features/img_info_TRAIN.csv"



f_id=[]
features_extr=[]

df = pd.DataFrame(columns=column)


tr=transforms.Compose([transforms.Resize(256),
                       transforms.CenterCrop(224),
                       transforms.ToTensor(),
                       transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])


test = Dataset(path, pathCSV, transform=tr)

test_loader = DataLoader(test, batch_size=1, num_workers=2, shuffle = False)
for batch in test_loader:
    nome = batch['FlickrID']
    f_id.append(nome)
    image = batch['image']
    
    
    with torch.no_grad():
        pred = get_vector(image)#error
   features_extr.append(pred)
    
df["FlickrID"] = f_id
df["Features"] = features_extr  


df.to_hdf("Places.h5", key='df', mode='w')

time_elapsed = time.time() - since
print('---- FEATURES SAVED in {:.0f}m {:.0f}s ----'.format(time_elapsed // 60, time_elapsed % 60)) 
</code></pre>
<p>I have this error: output with shape [2048] doesn’t match the broadcast shape [1, 2048, 1, 2048]  in the line of code that I have commented with <span class="hashtag">#error</span></p>
<p>Can you help me?</p>

username: Giuseppe
cooked: <p>solved with:</p>
<pre><code class="lang-auto">model.fc = nn.Identity()
features = model(image)
</code></pre>