akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Error when use inception_v3 model

111186: <p>i want to use the inception_v3 model, and my input is [128,3,44,44], i want to classifier 7.<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/8/388f34c9af69efecf6613583342fec8d5d81252d.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/388f34c9af69efecf6613583342fec8d5d81252d" title="122"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/388f34c9af69efecf6613583342fec8d5d81252d_2_690x61.jpeg" alt="122" data-base62-sha1="84lyu6xzokj9D3hlDl0REmJaniR" width="690" height="61" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/388f34c9af69efecf6613583342fec8d5d81252d_2_690x61.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/388f34c9af69efecf6613583342fec8d5d81252d_2_1035x91.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/3/8/388f34c9af69efecf6613583342fec8d5d81252d_2_1380x122.jpeg 2x" data-dominant-color="2C2D27"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">122</span><span class="informations">1971×177 19 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/9/d971622990305df0bf9088926ba989b66f90cfcd.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/d971622990305df0bf9088926ba989b66f90cfcd" title="QQ截图20200108221039"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/9/d971622990305df0bf9088926ba989b66f90cfcd_2_690x181.jpeg" alt="QQ截图20200108221039" data-base62-sha1="v1Ax6HxXNm7ge8mrU6PCgckLWUd" width="690" height="181" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/9/d971622990305df0bf9088926ba989b66f90cfcd_2_690x181.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/d/9/d971622990305df0bf9088926ba989b66f90cfcd_2_1035x271.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/d/9/d971622990305df0bf9088926ba989b66f90cfcd_2_1380x362.jpeg 2x" data-dominant-color="343330"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20200108221039</span><span class="informations">3337×877 323 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>Your input tensor is too small for this architecture and at some point the activation output will be an empty tensor.<br>
The Inception model expects your output to have a spatial size of <code>299x299</code>.<br>
You could use <code>torchvision.transform.Resize</code> to increase the spatial size of your images.</p>

111186: <p>thanks so much <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>
Share a solution for cudaCheckError() failed : invalid device function

chengjuzhou: <p>I test the code from <a href="https://github.com/McDo/PSROIAlign-Multi-Batch-PyTorch" rel="nofollow noopener">https://github.com/McDo/PSROIAlign-Multi-Batch-PyTorch</a> in example.py,<br>
and find it always end up with <strong>cudaCheckError() failed : invalid device function</strong>. The error info comes from line 124 of PSROIPool_cuda.cu. After several tries, I find the reason why it comes with such error is that I have added a new different GPU before creating new Ananconda env for pytorch. The details are follows:</p>
<ol>
<li>two 1080Ti GPU on my workstation, creating Anaconda env test_1</li>
<li>using a Titan V GPU to replace one 1080Ti GPU. Now, one 1080Ti and one Titan V on my workstation. Create Anaconda env test_2</li>
<li>Encounter  <strong>cudaCheckError() failed : invalid device function</strong>   when using Anaconda env test_2. No error when using Anaconda env test_1</li>
</ol>
<p>Guess: The pytorch would do some specific thing related with GPU when installing, and may have something wrong when there have two different GPU cards on workstation.</p>

albanD: <p>Thanks for sharing.</p>
<p>Yes different GPU require different low level code. This makes install time and binary very large. So if we know in advance what the code is going to run on, we only compile for this GPU. You will need to update if you add more GPU indeed.</p>

0e5ec01b9c14c98dcd37: <p>The code is based on the older version of ROIAlign by facebook. You may need to change</p>
<pre><code class="lang-auto">const int threads = 1024;
</code></pre>
<p>for your GPU spec and recompile it.</p>
About crossentropyloss and nllloss

fupanbo: <p>as i know  nn.CrossEntropyLoss equal to  nn.logsoftmax + nn.NLLLoss<br>
but the nn.CrossEntropyLoss seems like have a function with one hot<br>
so if i want to  use  nn.logsoftmax + nn.NLLLoss   to replace nn.CrossEntropyLoss<br>
should i write a  function def onehot?</p>

Eta_C: <p>Just try it.</p>
<pre><code class="lang-python">import torch
import torch.nn as nn

ce = nn.CrossEntropyLoss()
ls = nn.LogSoftmax(dim=-1)
nll = nn.NLLLoss() 

batch_size = 5
num_classes = 8
x = torch.rand(batch_size, num_classes)
y = torch.randint(num_classes, (batch_size,))

print(ce(x, y))
print(nll(ls(x), y))
</code></pre>
<p>PS: Here is one_hot API <a href="https://pytorch.org/docs/stable/nn.functional.html?highlight=one_hot#torch.nn.functional.one_hot" rel="nofollow noopener">nn.functional.one_hot</a>.</p>

fupanbo: <p>thanku for your help<br>
the print(ce(x, y)) 's answer is equal to<br>
print(nll(ls(x), y)) 's<br>
but i get a new trouble<br>
print(ce(ls(x),y)) is also get a same answer<br>
it is seems like the logsoftmax is invalid?</p>

Eta_C: <pre><code class="lang-auto">import torch
import torch.nn.functional as F

x = torch.rand(1, 2, 3, 4)
ls = F.log_softmax(x)
lsls = F.log_softmax(F.log_softmax(x))
print((lsls - ls).abs().max())
</code></pre>

fupanbo: <p>in my forward network<br>
the last one is  x=nn.linear(xxxx,classnums)<br>
i use the  loss=crossentropyloss(x,target)</p>
<p>if after x=nn.linear(xxxx,classnums)<br>
follow x=F.log_softmax(x)</p>
<p>loss1=nn.crossentropyloss(x,target)<br>
loss2=nn.NLLLoss（x,target）<br>
seems loss1=loss2?<br>
thats strange…</p>

ptrblck: <p>Since <code>LogSoftmax</code> is idempotent, you’ll get the same output as shown by <a class="mention" href="/u/eta_c">@Eta_C</a>’s example.<br>
Internally <code>nn.CrossEntropyLoss</code> will apply another <code>F.log_softmax</code> on the inputs.</p>
<p>However, I would recommend to stick to:</p>
<ul>
<li>
<code>nn.LogSoftmax</code> + <code>nn.NLLLoss</code> or</li>
<li>raw logits + <code>nn.CrossentropyLoss</code>
</li>
</ul>
<p>as you won’t get any benefit from it using these loss functions.</p>

Eta_C: <p>Although <code>LogSoftmax</code> is idempotent, using it twice will still produce floating point precision error. <img src="https://discuss.pytorch.org/images/emoji/apple/rofl.png?v=9" title=":rofl:" class="emoji" alt=":rofl:"></p>
How to add Validation and Test set in Non Linear Regression ANN

rerere_L: <p>I have Non Linear Regression <code>Model ANN( X = [1000,3] , Y = [1000,8] )</code> with One hidden <code>Layer(Nh = 6)</code> .</p>
<p>How add Validation and Test analisys in my Non Linear Regression ANN with Pytorch in my case?</p>
<ul>
<li>
<p>Add code for split</p>
</li>
<li>
<p>Add code for test and validation in “epoch” loop</p>
</li>
</ul>
<p><strong>Model :</strong></p>
<pre><code class="lang-auto">` ```
x = torch.from_numpy(x).float()
y = torch.from_numpy(y).float()

N, D_in, H, D_out = x.shape[0], x.shape[1], 6, y.shape[1]

model = nn.Sequential(OrderedDict([ ('fc1', nn.Linear(D_in, H)), 
                                    #('Sig', nn.Sigmoid()),
                                    ('ISRU', ISRU()), # Add ISRU
                                    ('fc2', nn.Linear(H, D_out))]))

# Error -----
loss_fn = torch.nn.L1Loss(reduction='mean')

# Train -----
optimizer = torch.optim.Adam(model.parameters(), lr=1,eps=2**(-EPS))
epoch = 250
for t in range(epoch):
    # Forward pass: compute predicted y by passing x to the model.
    clear_output(wait=True)
    y_pred = model(X)

    # Compute and print loss.
    loss = loss_fn(y_pred, Y)
    if t % 100 == 99:
        print(t, loss.item())

    optimizer.zero_grad() ;
    loss.backward() ;
    optimizer.step() ;
if loss.item() &lt; diff : lista = np.vstack((lista, [loss.item(),2,EPS])) ; diff = loss.item()
</code></pre>
<pre><code class="lang-auto"></code></pre>

ptrblck: <p>Since you are using the all data samples at the moment, I think the easiest way would be to split your numpy arrays via <code>sklean.model_selection.train_test_split</code> and transform each split into a tensor with <code>torch.from_numpy</code>.</p>
<p>For the evaluation loop, I would refer to the <a href="https://github.com/pytorch/examples/blob/master/mnist/main.py">MNIST example</a>.<br>
Note that a <code>Dataset</code> and <code>DataLoader</code> was used in the linked example, but you could simply use the tensors directly as is done in your current code snippet.</p>

rerere_L: <p>For split Dataset i dont have problem, im doing what you saying.<br>
In the MNIST example i can see test loop, but not Validation set, loop with train set and relative loss calculation !?</p>

ptrblck: <p>You could use the test loop for your validation use case. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
Difference between two method for save tensor to image

hyuntae: <p>I try to save the output of model with two method.</p>
<p>First one is using ‘from torchvision.utils import save_image’.<br>
Second is using ‘torchvision.transforms.ToPILImage()’.</p>
<p>But the results of two method was different.</p>
<p>Is there any big difference between two method?</p>

ptrblck: <p>How large are the differences?<br>
I just tested both methods and they look at least visually the same.<br>
Based on the code from <a href="https://github.com/pytorch/vision/blob/07cbb46aba8569f0fac95667d57421391e6d36e9/torchvision/utils.py#L107">save_image</a> and <a href="https://github.com/pytorch/vision/blob/c226bb958b47b54dde9bdb2e685305b4aff42f5d/torchvision/transforms/functional.py#L139">to_pil_image</a> I would also expect to get the same result (I’m not sure, if the addition of <code>0.5</code> will be visible or not).</p>
Finetuning vgg18

dato_nefaridze: <p>in pytorch transfer learning tutorial there is following code:</p>
<pre><code class="lang-auto">model_ft = models.resnet18(pretrained=True)
num_ftrs = model_ft.fc.in_features
# Here the size of each output sample is set to 2.
# Alternatively, it can be generalized to nn.Linear(num_ftrs, len(class_names)).
model_ft.fc = nn.Linear(num_ftrs, 2)
</code></pre>
<p>they removed last fully connected layer, and replaced new one.i still don’t understand how model_ft.fc.in_features works, there is no docs about that.what does model_ft.fc return? last layer?and if model has 5 fully connected layer, what it will return?</p>

ptrblck: <p>You can see the layers via <code>print(model)</code> or have a look at the <a href="https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py">source code</a>.</p>
<p>You’ll see that, <code>model.fc</code> addresses the last linear layer, so you can just reassign a new layer to this attribute.</p>
<p>Each <code>nn.Linear</code> layer has a weight matrix (and optionally bias), which is defined by the input and output shape of this particular layer. <code>in_features</code> returns the given number of input features for this layer.</p>
Error while importing models

SHASHANK_KUMAR_MISHR: <p>While running the below command I am getting the error</p>
<p>from models import InferSent<br>
ERROR:-<br>
No module named ‘models’</p>
<p>How to resolve it</p>

ptrblck: <p>This seems to be a Python issue rather than PyTorch-related.<br>
I assume <code>models.py</code> is not in your current working directory, which would throw this error.<br>
Have a look at the <a href="https://docs.python.org/3/reference/import.html">Import system in Python</a> for a detailed description.</p>

SHASHANK_KUMAR_MISHR: <p>worked like charm thanks <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
Train test split

SHASHANK_KUMAR_MISHR: <p>I have a csv file on which I want to perform some operation .I hgave read the file using pandas in jupyter notebook.Now how  to use it in Dataloader for pytorch and then perform train test split.<br>
The csv file has two columns.</p>
<p>Please mention the code for it.</p>

ptrblck: <p>I would recommend to write a custom <code>Dataset</code> as explained in <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">this tutorial</a> or alternatively, get the data of your pandas DataFrame as numpy arrays, transform them to tensors via <code>torch.from_numpy()</code> and use a <code>TensorDataset</code>.</p>
<p>Once you’ve created a dataset, you can use <code>torch.utils.data.random_split</code> to create different splits.</p>
Onnx doesn't support preprocess when convert to CoreML

MarcSteven: <p>I have a pytorch model and then I convert to the model .But onnx doesn’t support preprocess ,so does it mean I need to add scaler when convert to coreml . <a class="mention" href="/u/ptrblck">@ptrblck</a></p>

ptrblck: <p>Answered <a href="https://discuss.pytorch.org/t/how-to-add-preprocess-into-the-model/66154/2">here</a>. Please let’s keep the discussion at one place.</p>

MarcSteven: <p>Yeah thank for your info. I will reviewed it</p>
【Pytorch】This is a question about network output as nan

cclover: <p><strong>Unitary linear regression</strong><br>
this is net</p>
<pre><code class="lang-auto">class LinearRegression(nn.Module):
    def __init__(self):
        super(LinearRegression, self).__init__()
        self.linear = nn.Linear(1, 1)

    def forward(self, x):
        x = self.linear(x)
        return x
</code></pre>
<p>This is the entire code</p>
<pre><code class="lang-auto">import torch as t
from matplotlib import pyplot as plt
from IPython import display
import torch.nn as nn
from torch.autograd import Variable
t.manual_seed(1000)
def get_fake_data(batch_size = 800):
    """y=2*x+3,生成x，并加入噪声"""
    x = t.rand(batch_size, 1) * 20
    y = 2 * x + 3 + t.randn(batch_size, 1)
    return x, y


x, y = get_fake_data()
plt.scatter(x.numpy(), y.numpy())
#  随机初始化参数
w = t.randn(1, 1)
b = t.zeros(1, 1)
lr = 0.001  # learning rate


class LinearRegression(nn.Module):
    def __init__(self):
        super(LinearRegression, self).__init__()
        self.linear = nn.Linear(1, 1)

    def forward(self, x):
        x = self.linear(x)
        return x


print(x[1],y[1])
</code></pre>
<p><strong>Thank you for your help</strong></p>

Eta_C: <p>Could you describe your problem more clearly? I can run the following code and get the correct result.</p>
<pre><code class="lang-python">import torch as t
import numpy as np
from matplotlib import pyplot as plt
from IPython import display
import torch.nn as nn
from torch.autograd import Variable
import torch.optim as optim

t.manual_seed(1000)
def get_fake_data(batch_size = 800):
    x = t.rand(batch_size, 1) * 20
    y = 2 * x + 3 + t.randn(batch_size, 1)
    return x, y


x, y = get_fake_data()
w = t.randn(1, 1)
b = t.zeros(1, 1)
lr = 0.001  # learning rate


class LinearRegression(nn.Module):
    def __init__(self):
        super(LinearRegression, self).__init__()
        self.linear = nn.Linear(1, 1)

    def forward(self, x):
        x = self.linear(x)
        return x


net = LinearRegression()
net.train()
loss_function = nn.L1Loss()
opt = optim.Adam(net.parameters())
bs = 20
for ep in range(1000):
    ep_loss = list()
    for i in range(0, 40):
        opt.zero_grad()
        y_pred = net(x[i*20:(i+1)*20, ...])
        loss = loss_function(y[i*20:(i+1)*20, ...], y_pred)
        loss.backward()
        ep_loss.append(loss.item())
        opt.step()
    if ep % 100 == 0:
        print(np.mean(ep_loss))

test_x, test_y = get_fake_data(batch_size = 40)
with t.no_grad():
    net.eval()
    y_pred = net(test_x)
    print((y_pred - test_y).abs().mean())   # 0.6565
</code></pre>
MaxPool2d output size

lucius323: <p>Hi All,</p>
<p>I found out the output size of the MaxPool2d could be not consistent with the formula in the document.</p>
<p>My MaxPool2d and the input are declared as:</p>
<pre><code class="lang-auto">nn.MaxPool2d(kernel_size=4, stride=4, padding=1)
inputs = torch.randn(1, 1, 1, 1)
</code></pre>
<p>According to the output size formula in the document, the output height/width should be:<br>
floor((1 + 2 * 1 - 1 * (4 - 1) - 1) / 4 + 1) = floor(0.75) = 0</p>
<p>However the actual output size is (1x1x1x1)</p>
<p>The PyTorch version is 1.2.0<br>
Does anyone have the same question here? Many thanks!</p>

Eta_C: <aside class="quote no-group" data-username="lucius323" data-post="1" data-topic="66118">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/lucius323/48/13310_2.png" class="avatar"> lucius323:</div>
<blockquote>
<p>floor((1 + 2 * 1 - 1 * (4 - 1) - 1) / 4 + 1) = floor(0.75) = 0</p>
</blockquote>
</aside>
<p>Your <code>square_size</code> is <code>1</code> and it didn’t raise an runtime error?</p>

lucius323: <p>Yes, my square_size is 1. And PyTorch generate an output of size 1x1x1x1.</p>
Use 4 gpu to train model, loss batch_size = batch_size * 4

TripleTry: <p>my code:</p>
<pre><code class="lang-auto">batch_size = 64
model = nn.DataParallel(model, device_ids=[0,1,2,3], dim=0)
model.cuda()
criterion = nn.BCEWithLogitsLoss()
criterion = cuda()

for s, t in loader:
    logits = model(s, t)
    loss = model.module.compute_loss(logits, tgt, criterion)
</code></pre>
<p>when I compute_loss, raise ValueError, say logits.shape is (256, num_classes), but t.shape is (64, num_classes), I want to know why</p>

albanD: <p>Hi,</p>
<p>I think your code is missing some important bits.<br>
Could you give a small code sample that we can run that shows the issue please?</p>

TripleTry: <p>Thank you very much <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=9" title=":grinning:" class="emoji" alt=":grinning:">, I have solve this problem.<br>
There is something wrong with my dataloader function, when I load data, I use padding to process my data, but I forgot to turn list into tensor, as a result nn.Dataparallel to split data wrong in batch dim. <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"></p>
Why is there additional parameter in torch.nn.Model

Ren_yf: <p>I found there seems to be one additional <code>parameter</code> in the <code>torch.nn.Model</code> for convnet as shown in the example below. And the shape the additional <code>parameter</code> seems like [in_channel, out_channl]. Can anyone help explain this?</p>
<p>Thanks in advance!</p>
<pre><code class="lang-python">conv1 = torch.nn.Conv2d(1, 2, 1)
print("conv1:", conv1.weight)

class myModel(torch.nn.Module):
    def __init__(self):
        super(myModel, self).__init__()
        self.conv1 = torch.nn.Conv2d(1, 2, 1)
    def forward(self, x):
        return torch.nn.functional.conv2d(x, self.filter)

model = myModel()# .cuda()
print("model:", [i for i in model.parameters()])
</code></pre>
<p>The output or first <code>print</code> is:</p>
<pre><code class="lang-python">conv1: Parameter containing:
tensor([[[[-0.6750]]],
        [[[ 0.2480]]]], requires_grad=True)
</code></pre>
<p>The output of second <code>print</code> is:</p>
<pre><code class="lang-python">model: [Parameter containing:
tensor([[[[ 0.9591]]],
        [[[-0.9882]]]], requires_grad=True), Parameter containing:
tensor([0.3482, 0.2676], requires_grad=True)]
</code></pre>

albanD: <p>Hi,</p>
<p>The second parameter is the <code>bias</code>. You can access it with <code>conv1.bias</code>.<br>
It has the same size as the number of output channels (it is created <a href="https://github.com/pytorch/pytorch/blob/66647038424666b943dfd41b5b40ee73720a8662/torch/nn/modules/conv.py#L43">here</a>).<br>
You can pass <code>bias=False</code> to the conv2d constructor if you don’t want it.</p>

Ren_yf: <p>I almost forget about that and Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"></p>
How to use torch.utils.tensorboard's SummaryWriter add_graph with dictionary outputs?

RylanSchaeffer: <p>Cross Posting from Stack Overflow for visibility: <a href="https://stackoverflow.com/questions/59656306/how-to-use-torch-utils-tensorboards-summarywriter-add-graph-with-dictionary-out" rel="nofollow noopener">https://stackoverflow.com/questions/59656306/how-to-use-torch-utils-tensorboards-summarywriter-add-graph-with-dictionary-out</a></p>
<p>I have a PyTorch model that inherits from <code>nn.Module</code> and that has a <code>forward</code> method which returns a dictionary containing multiple tensors (stored as values). When I try to log this model to TensorBoard using <code>torch.utils.tensorboard</code>'s <code>SummaryWriter.add_graph()</code> method I receive the following error:</p>
<p><code>RuntimeError("Tracer cannot infer type of &lt;output dictionary&gt;</code></p>
<p>How can I use <code>add_graph()</code> when the model’s <code>forward()</code> method returns a dictionary?</p>
GPU gets out of memory

Rocket: <p>Hi</p>
<pre><code class="lang-auto">batch_size = 100

train = TensorDataset(train_X,train_Y)
trainLoader = DataLoader(train,batch_size, shuffle=True,num_workers=1)

model=model.cuda()

for i,data in enumerate(trainLoader):        
    train_x, train_y = data
    train_x = train_x/255
    #train_x,train_y = Variable(train_x), Variable(train_y)
    train_x = train_x.cuda()
    train_y = train_y.cuda()           
        
    encoded, decoded = model(train_x)
</code></pre>
<p>I trained model using the code above successfully.</p>
<p>After training done, I try to get the decoded(output) using the code above same manner.</p>
<p>But my GPU gets out of memory only when computing decoded(output) using trained model.</p>
<p>I don’t know why GPU memory get full although it is trained successfully during training with same code.</p>

ptrblck: <p>Are you using the same batch size to compute the new output?<br>
If you don’t need to train the model, wrap the code in a <code>with torch.no_grad()</code> block to save memory.</p>

Eta_C: <p>You may be interested in this topic. There are some discussion here.</p><aside class="quote" data-post="1" data-topic="12274">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/wasiahmad/48/248_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/out-of-memory-error-during-evaluation-but-training-works-fine/12274">Out of memory error during evaluation but training works fine!</a> 
  </div>
  <blockquote>
    I have recently upgraded pytorch from 0.2 to 0.3. Surprisingly my old programs are throwing an out of memory error during evaluation (in eval() mode) but training works just fine. I am using the same batch size for training and evaluation. I am totally clueless what is happening? Did anyone face similar issue? Is there any possible solution?
  </blockquote>
</aside>
Testing and Confidence score of Network trained with nn.CrossEntropyLoss()

VikasRajashekar: <p>I have trained a network with the following structure <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<blockquote>
<p>Intent_LSTM(<br>
(attention): Attention()<br>
(embedding): Embedding(34601, 400)<br>
(lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)<br>
(dropout): Dropout(p=0.5, inplace=False)<br>
(fc): Linear(in_features=512, out_features=3, bias=True)<br>
)</p>
</blockquote>
<p>Now here my loss function:</p>
<pre><code class="lang-auto">lr=0.001

criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr=lr)
</code></pre>
<p>Now I want to test this trained network and also get the confidence score of classification.<br>
Here is my current implementation of test function:</p>
<pre><code class="lang-auto">def test_func(loader, model_current, criterion,folder_name):
    train_on_gpu = torch.cuda.is_available()
    labels_CM = []
    predictions_CM = []
    print_and_write(folder_name,"Test result for Test Data")
    test_losses = []  # track loss

    correct = 0
    total = 0

    model_current.eval()
    # iterate over test data
    for inputs, labels in loader:

        if (train_on_gpu):
            inputs, labels = inputs.cuda(), labels.cuda()

        # get predicted outputs
        output = model_current(inputs)

        # calculate loss
        test_loss = criterion(output.squeeze(), labels)
        test_losses.append(test_loss.item())

        # convert probablities to integer
        pred = torch.round(output.squeeze())

        pred = pred.argmax(dim=1, keepdim=True)
        predictions_CM += list(pred.cpu().numpy())

        labels = labels.unsqueeze(dim=1)
        labels_CM += list(labels.cpu().numpy())

        correct += (pred.cpu() == labels.cpu()).sum()

    print(folder_name,"Test loss: {:.3f}".format(np.mean(test_losses)))

    test_acc = correct.item() / len(loader.dataset)
    print(folder_name,"Test accuracy: {:.3f}".format(test_acc))
</code></pre>
<p>Now my question isas follows.</p>
<ol>
<li>
<p>Here pred is just the output from Fully connected layer from my network without softwax (as required by loss function). Is this(pred = pred.argmax(dim=1, keepdim=True)) the right way to get the predictions? Or should i pass the output from the network to a softmax layer and then do argmax?</p>
</li>
<li>
<p>How do I get the confidence score? Should I pass the output from the network to a softmax layer and select the argmax as the confidence of the class?</p>
</li>
</ol>
Output shape of model?

zacharynew: <p>Is there a good way to get the output shape of a nn.Module object without knowing the input shape? Everything I can come up with seems to need a few extra assumptions on the structure of the network.</p>
<p>Thanks!</p>

albanD: <p>Given that the <code>forward</code> function can contain arbitrary python code, I don’t think this is possible. <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"><br>
You will need extra assumption about the structure of the Module and the submodules it can contain.</p>

zacharynew: <p>Thanks for the response. That’s what I was thinking but I hoped I was wrong. Related to the first question, do you know what defines the order of the state dict? If the last layer will be made up of the last tensors in the state dict then I might be able to get a “close enough” solution.</p>

albanD: <p>There is no strict order. It might be the same order as the registration. But someone can register the last layer first and then this will be wrong <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>
<p>Note that for some models, the output size will also depend on the input that is given (fully convolutional neural nets for example).<br>
But if you know the input size, you can create a dummy input Tensor that you forward to see the output size. That will work for most network (except for the weird ones where the output depends on the values in the input Tensor).</p>

zacharynew: <p>Yeah, I think that’s what I will have to do. I was hoping to avoid the requirement of knowing the shape of the input tensor. Thanks for the help!</p>
What do the underscores at the end of functions mean in pytorch?

Brando_Miranda: <p>I’ve been seeing underscores at the end of many functions in pytorch. What does this mean?</p>
<p>I think I read in some chat or question that it means operations are done “in place” but I wanted to make sure that was correct (and provide a way for future people to search for this question) and to clarify what “in place” meant.</p>

fmassa: <p>This has been replied in <a href="https://discuss.pytorch.org/t/how-does-one-make-sure-that-the-parameters-are-update-manually-in-pytorch-using-modules/6076/7?u=fmassa">How does one make sure that the parameters are update manually in pytorch using modules?</a></p>

Brando_Miranda: <p>From fmassa’s answer somewhere else (source: <a href="https://discuss.pytorch.org/t/how-does-one-make-sure-that-the-parameters-are-update-manually-in-pytorch-using-modules/6076" class="inline-onebox">How does one make sure that the parameters are update manually in pytorch using modules?</a>):</p>
<blockquote>
<p>Whenever we have an underscore in the end of a function in pytorch, that means that the function is in-place.</p>
</blockquote>
<p>If you you don’t know what in place means see albanD’s answer (source: <a href="https://discuss.pytorch.org/t/what-is-in-place-operation/16244" class="inline-onebox">What is `in-place operation`?</a>):</p>
<blockquote>
<p>An in-place operation is an operation that changes directly the content of a given Tensor without making a copy. Inplace operations in pytorch are always postfixed with a  <code>_</code> , like  <code>.add_()</code>  or  <code>.scatter_()</code> . Python operations like  <code>+=</code>  or  <code>*=</code>  are also inplace operations.</p>
</blockquote>
<p>as a side comment, these sort of operations don’t get added to the computation graph, so they should be used with caution.</p>
Official Implementation of Graph Attention Networks

akashs: <p>Is there any official implementation of <a href="https://arxiv.org/abs/1710.10903" rel="nofollow noopener">Graph Attention Network</a> in pytroch? or is there any plan to include the GAT in pytorch nn module?</p>
<p>I found one repo in pytorch <a href="https://github.com/Diego999/pyGAT" rel="nofollow noopener">pyGAT</a>? But I am not sure about this repo.<br>
Thanks</p>
Error when building pytorch 1.1.0 from source

Dale_Song: <p>When building pytorch 1.1.0 from source with explicitly setting  <code>-D_GLIBCXX_USE_CXX11_ABI=0</code> , I met such error:</p>
<pre><code class="lang-auto">pytorch/build/lib/libcaffe2_gpu.so: undefined reference to `gloo::getCudaPCIBusID(int)'
pytorch/build/lib/libcaffe2_gpu.so: undefined reference to `gloo::EnforceNotMet::EnforceNotMet(char const*, int, char const*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;)'
</code></pre>
<p>the building steps is:</p>
<pre><code class="lang-auto">git clone https://github.com/pytorch/pytorch.git &amp;&amp; git checkout v1.1.0
...  # using `git reset --hard` do some change to reset the third_party submodules to the correct commit with tags/v1.1.0
export USE_NINJA=OFF
export CFLAGS="-D_GLIBCXX_USE_CXX11_ABI=0 $CFLAGS"
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
python setup.py install
</code></pre>
<p>how to solve that?<br>
Details is here: <a href="https://github.com/pytorch/pytorch/issues/31943" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/31943</a></p>