akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to get a intermediate value from nn.Sequential?

mshmoon: <p>Fox example,if I have a layer:</p>
<p>nn.Sequential(<br>
nn.Linear(1,1),<br>
nn.ReLU(),<br>
nn.Linear(1,1),<br>
nn.ReLU(),<br>
nn.Linear(1,1),<br>
)</p>
<p>My question is how can i got a output   form  the second nn.Linear(1,1)  from this Sequential</p>

mshmoon: <p>oh I got it.I can use .children()</p>

shadeofplato: <p><a class="mention" href="/u/mshmoon">@mshmoon</a>, how do you do that more specifically? I’m dealing with the same problem, so your advice’d be appreciated. Thanks a lot in advance!</p>

InnovArul: <p>What kind of issue are you facing?</p>
<p>You could do something like below:</p>
<pre><code class="lang-auto">seq = nn.Sequential(nn.Linear(5,5), nn.Dropout(0.2), nn.Linear(5,1))
x = torch.rand(5,5)
for layer in seq:
     x = layer(x)
     print(x)

</code></pre>

shadeofplato: <p>Hi <a class="mention" href="/u/innovarul">@InnovArul</a>,<br>
The problem I was concerned with was getting output of the hidden layers of AlexNet. It turned out that <a class="mention" href="/u/fmassa">@fmassa</a> has already provided a simple solution:</p><aside class="quote quote-modified" data-post="3" data-topic="119">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fmassa/48/58_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/how-to-extract-features-of-an-image-from-a-trained-model/119/3">How to extract features of an image from a trained model</a> 
  </div>
  <blockquote>
    To complement <a class="mention" href="/u/apaszke">@apaszke</a> reply, once you have a trained model, if you want to extract the result of an intermediate layer (say fc7 after the relu), you have a couple of possibilities. 
You can either reconstruct the classifier once the model was instantiated, as in the following example: 
import torch
import torch.nn as nn
from torchvision import models

model = models.alexnet(pretrained=True)

# remove last fully-connected layer
new_classifier = nn.Sequential(*list(model.classifier.children())[:-…
  </blockquote>
</aside>


Choung_young_jae: <p>Thanks your question &amp; solution page helped me to find solution</p>
<p>And the code below works for me</p>
<pre><code class="lang-python">&gt; fnn.model

Sequential(
  (0): Linear(in_features=4, out_features=3, bias=True)
  (1): ReLU()
  (2): Linear(in_features=3, out_features=3, bias=True)
  (3): LayerNorm()
  (4): ReLU()
  (5): Softmax()
)


&gt; fnn.model[:-1](data_torch_variable))

i.e. 
fnn.model[:-1](pre.trans_to_variable(data_x))[0:3]

tensor([[0.0000, 0.5921, 0.4040],
        [0.0000, 0.5703, 0.4293],
        [0.0000, 0.5618, 0.4390]], grad_fn=&lt;SliceBackward&gt;)
</code></pre>

RylanSchaeffer: <p>How does <code>.children()</code> work?</p>
RuntimeError: Error(s) in loading state_dict for VAEGen:

kshama: <p>when i was trying to the load the model with pre trained weights I was getting the following error</p>
<p>RuntimeError: Error(s) in loading state_dict for VAEGen:<br>
Missing key(s) in state_dict: “enc.model.0.conv.weight”, “enc.model.0.conv.bias”, “enc.model.1.conv.weight”, “enc.model.1.conv.bias”, “enc.model.2.conv.weight”, “enc.model.2.conv.bias”, “enc.model.3.model.0.model.0.conv.weight”, “enc.model.3.model.0.model.0.conv.bias”, “enc.model.3.model.0.model.1.conv.weight”, “enc.model.3.model.0.model.1.conv.bias”, “enc.model.3.model.1.model.0.conv.weight”, “enc.model.3.model.1.model.0.conv.bias”, “enc.model.3.model.1.model.1.conv.weight”, “enc.model.3.model.1.model.1.conv.bias”, “enc.model.3.model.2.model.0.conv.weight”, “enc.model.3.model.2.model.0.conv.bias”, “enc.model.3.model.2.model.1.conv.weight”, “enc.model.3.model.2.model.1.conv.bias”, “enc.model.3.model.3.model.0.conv.weight”, “enc.model.3.model.3.model.0.conv.bias”, “enc.model.3.model.3.model.1.conv.weight”, “enc.model.3.model.3.model.1.conv.bias”, “dec.model.0.model.0.model.0.conv.weight”, “dec.model.0.model.0.model.0.conv.bias”, “dec.model.0.model.0.model.1.conv.weight”, “dec.model.0.model.0.model.1.conv.bias”, “dec.model.0.model.1.model.0.conv.weight”, “dec.model.0.model.1.model.0.conv.bias”, “dec.model.0.model.1.model.1.conv.weight”, “dec.model.0.model.1.model.1.conv.bias”, “dec.model.0.model.2.model.0.conv.weight”, “dec.model.0.model.2.model.0.conv.bias”, “dec.model.0.model.2.model.1.conv.weight”, “dec.model.0.model.2.model.1.conv.bias”, “dec.model.0.model.3.model.0.conv.weight”, “dec.model.0.model.3.model.0.conv.bias”, “dec.model.0.model.3.model.1.conv.weight”, “dec.model.0.model.3.model.1.conv.bias”, “dec.model.2.norm.gamma”, “dec.model.2.norm.beta”, “dec.model.2.conv.weight”, “dec.model.2.conv.bias”, “dec.model.4.norm.gamma”, “dec.model.4.norm.beta”, “dec.model.4.conv.weight”, “dec.model.4.conv.bias”, “dec.model.5.conv.weight”, “dec.model.5.conv.bias”.<br>
Unexpected key(s) in state_dict: “a”, “b”.</p>
<p>I don’t know what could be the reason</p>

ptrblck: <p>Did you rename the submodules from <code>a</code> and <code>b</code> to <code>enc</code> and <code>dec</code>?<br>
If so, you could iterate the <code>state_dict</code> and replace the old names with the new ones.</p>
How to Calculate Accuracy against MSELoss function

Shariq_Ali: <p>Hello. I am new on machine learning and pytorch. ? I am doing segmentation and using the MSEloss function for calculating loss but i want to find the accuracy as well. I have tried different example but didn’t find it. Please help me following is my code and the shapes.</p>
<p>for data, target in train_loader:</p>
<pre><code>    data = data.to(device) 
    target  = target.to(device=device, dtype=torch.int64)

    input_var = Variable(data.float())
    target_var = Variable(target.float())

    optimizer.zero_grad()
    output = model(input_var)
    loss = criterion(output, target_var)
    dice =  dice_coef(output,target)

    loss.backward()
    optimizer.step()

    train_loss += loss.item()*data.size(0)
    train_metric =  dice*data.size(0)
</code></pre>
<p>Shapes:<br>
target_var  torch.Size([2, 3, 512, 512])<br>
oututput  torch.Size([2, 3, 512, 512])</p>

ptrblck: <p>Would <a href="https://discuss.pytorch.org/t/calculate-train-accuracy-of-the-model-in-segmentation-task/33581/13">this approach</a> from your other post help?</p>
Question about '_rebuild_tensor_v2'

Bowen_Zhang: <p>Hi,</p>
<p>I have a model trained using the newest master branch of pytorch (0.4.0a0+18a76f5).<br>
Now I want to use old version of pytorch (‘0.4.0’)  to load the model.</p>
<p><strong>It reports an error: ‘module’ object has no attribute ‘_rebuild_tensor_v2’</strong></p>
<p>I noticed that ‘_rebuild_tensor_v2’ was added recently.<br>
Is there any method to use ‘0.4.0’ to load a model trained by ‘0.4.0a0+18a76f5’?</p>
<p>bowen</p>

lucasb-eyer: <p>I have the same problem, I develop and train using latest PyTorch from source, save a model, and want to deploy it on a robot which is using the stable PyTorch 0.3.1 from the repo.</p>
<p>It seems that this is <a href="https://github.com/pytorch/pytorch/issues/5729#issuecomment-372828432" rel="nofollow noopener">intended behaviour</a>, though. It is really annoying, actually.</p>

lucasb-eyer: <p>Thanks to Python’s incredible flexibility, we can actually monkey-patch our way to making this work. This is very ugly, but gets the job done, add it at the top of your file, after <code>import torch</code>:</p>
<pre><code class="lang-auto"># Monkey-patch because I trained with a newer version.
# This can be removed once PyTorch 0.4.x is out.
# See https://discuss.pytorch.org/t/question-about-rebuild-tensor-v2/14560
import torch._utils
try:
    torch._utils._rebuild_tensor_v2
except AttributeError:
    def _rebuild_tensor_v2(storage, storage_offset, size, stride, requires_grad, backward_hooks):
        tensor = torch._utils._rebuild_tensor(storage, storage_offset, size, stride)
        tensor.requires_grad = requires_grad
        tensor._backward_hooks = backward_hooks
        return tensor
    torch._utils._rebuild_tensor_v2 = _rebuild_tensor_v2
</code></pre>

Naveen_Benny: <p>Thanks man! This worked for me!</p>

Ying_Cui: <p>It works for me! Thanks!</p>

Happsky: <p>That works for me, thanks!</p>

juunnn: <p>how to use this? do you rebuild it from source? like you edit from environment or put it somewhere in the code for loading?<br>
Sorry for newbie question here</p>

nvs: <p>just put it before loading checkpoint</p>

Candice_Han: <p>Thank you so much!! These few lines saved my afternoon!</p>

llaier: <p>This worked for me! Thanks a lot!</p>

Lin_Qiu: <p>This worked for me! Thank you very much!</p>