akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
One of the variables needed for gradient computation has been modified by an inplace operation,customize loss function

Jun_Ren: <p>Concordance Correlation Coefficient (CCC) as loss function. does not decreace, any ideas</p>
<pre><code class="lang-auto">def __init__(self):
    super(ConcordanceCorCoeff, self).__init__()
    self.mean = torch.mean
    self.var = torch.var
    self.sum = torch.sum
    self.sqrt = torch.sqrt
    self.std = torch.std
def forward(self, prediction, ground_truth):
    mean_gt = self.mean (ground_truth, 0)
    mean_pred = self.mean (prediction, 0)
    var_gt = self.var (ground_truth, 0)
    var_pred = self.var (prediction, 0)
    v_pred = prediction - mean_pred
    v_gt = ground_truth - mean_gt
    cor = self.sum (v_pred * v_gt) / (self.sqrt(self.sum(v_pred ** 2)) * self.sqrt(self.sum(v_gt ** 2)))
    sd_gt = self.std(ground_truth)
    sd_pred = self.std(prediction)
    numerator=2*cor*sd_gt*sd_pred
    denominator=var_gt+var_pred+(mean_gt-mean_pred)**2
    ccc = numerator/denominator
    return 1-ccc
</code></pre>

ptrblck: <p>The loss seems to be decreasing:</p>
<pre><code class="lang-python">criterion = ConcordanceCorCoeff()
model = nn.Linear(10, 10)
data = torch.randn(10, 10)
target = torch.randn(10, 10)

optimizer = torch.optim.SGD(model.parameters(), lr=10.)

for epoch in range(10):
    optimizer.zero_grad()
    output = model(data)
    loss = criterion(output ,target)
    loss.mean().backward()
    optimizer.step()
    print('Epoch {}, loss {}'.format(epoch, loss.mean().item()))

&gt; Epoch 0, loss 0.9816429018974304
Epoch 1, loss 0.4328025281429291
Epoch 2, loss 0.43783488869667053
Epoch 3, loss 0.49120745062828064
Epoch 4, loss 0.5858393907546997
Epoch 5, loss 0.4273451864719391
Epoch 6, loss 0.11961951106786728
Epoch 7, loss 0.10366281121969223
Epoch 8, loss 0.6455065011978149
Epoch 9, loss -0.01740434765815735
</code></pre>
<p>I assume the error in the title is solved already, as I cannot find any inplace operations?</p>
<p>Also, I’m not familiar with this loss function, but it will return a NaN value, if you pass a single sample (but that might be an illegal use case).</p>
Fine Tune MobileNet_V2

sunnysharma03: <p>Hi Team,<br>
I am new to pytorch and I am trying to Create a Classifier where I have around 10 kinds of Images Folder Dataset, for this task I am using Pretrained model( MobileNet_v2 ) but the problem is I am not able to chnage the fc layer of it. There is not model.fc attribute.<br>
Can anyone help me to do this.<br>
Thanks</p>

ptrblck: <p><code>MobileNetV2</code> uses a <code>classifier</code> head with an <code>nn.Dropout</code> layer and the final <code>nn.Linear</code> layer:</p>
<pre><code class="lang-python">model = models.mobilenet_v2()
print(model.classifier)
&gt; Sequential(
  (0): Dropout(p=0.2)
  (1): Linear(in_features=1280, out_features=1000, bias=True)
)
</code></pre>
<p>If you want to change the last linear layer, try this code:</p>
<pre><code class="lang-python">model.classifier[1] = nn.Linear(1280, 10)
</code></pre>
<p>e.g. to change the number of classes to 10.<br>
model.classifier[1] = nn.Linear(1280, 10)</p>
How Does the BCEWithLogitsLoss is computed

dito: <p>Hi All,<br>
I’m a newbie in pytorch and datascience, I was looking at the documentation of the nn.BCEWithLogitsLoss in pytroch and I wanted to replicate the formula but I couldn’t get the same result.</p>
<p>I have a multilabel classification problem with 31 classes.<br>
example of two records  :</p>
<p>y_scores = torch.tensor([[-0.0089,  0.0000, -0.0922, -0.9179, -0.0000, -0.0506,  0.6511,  0.7989,<br>
0.1416, -0.0000,  0.4984,  0.2442, -0.8102, -0.5201, -0.1818, -0.0000, -0.1521, -0.5098, -0.0000,  0.0000, -0.0000, -0.2269,  0.1799,  0.9420, 0.4986,  0.3374, -0.8464, -0.0976, -0.0000, -0.0000, -0.6304],<br>
[-0.0089,  0.1645, -0.0922, -0.9179, -0.2571, -0.0000,  0.6511,  0.7989, 0.0000, -0.0000,  0.4984,  0.0000, -0.8102, -0.5201, -0.1818, -0.1467,-0.1521, -0.5098, -0.0112,  0.7157, -0.2090, -0.2269,  0.1799,  0.0000, 0.4986,  0.3374, -0.0000, -0.0000, -0.4324, -0.8032, -0.6304]])</p>
<p>y_true = torch.tensor([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.],<br>
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.]])</p>
<p>So I first calculated the different parts of the BCEWithLogitsLoss  formula:</p>
<p>y_pred            = torch.sigmoid(y_scores )<br>
y_pred_log     = torch.log(y_pred_log )<br>
y_pred_log_1 = torch.log(1-y_pred)</p>
<p>My_LOSS = -((y_true * y_pred_log) + (1 - y_true ) * (y_pred_log_1))<br>
loss_fn_none = nn.BCEWithLogitsLoss(reduction=“none”)</p>
<p>the result is not the same however. What Am I doing wrong.</p>
<p>Thanks all for your help<br>
Cheers,</p>

ptrblck: <p>The maximal difference is <code>~1.7e-7</code>, which is due to limited floating point precision using FP32.<br>
If you need more precision, you could use <code>DoubleTensors</code>, but your manual implementation should work fine.</p>

dito: <p>Okdio Thanks a lot for your response :). Have a good day!</p>
pytorch 0.4.1 | dataloader worker is killed by signal 

Jiahang_WANG: <p>While training the model, for the beginning several epochs, all is ok. But aftering running for several epochs, the thread is killed. The error shows ‘Dataloader workers is killed by a signal: bus error’ .</p>
<p>However, there is definitely enough GPU memory showed by ‘nivida-smi -l’.</p>
<p>Besides, I need to do many data processing, including open several images and resize them or embedding some data. This error is often appear after finishing validation. I don’t know the real reason. Really need your help.</p>
<p>Thanks in advance.</p>

ptrblck: <p>This error might be thrown, if you don’t have enough shared memory using multiple workers.<br>
Are you using a docker container? If so, use <code>--ipc=host</code> while launching it.<br>
Otherwise, try to increase your shared memory in your system.</p>

Jiahang_WANG: <p>Thanks for your reply. I didn’t use docker. I wonder if use torch.cuda.empty_cache after validation would help or not?</p>
<p>Besides, the memory is shown as follows:</p>
<pre><code>          total        used        free      shared  buff/cache   available
</code></pre>
<p>Mem:         257662       41849      166048       44896       49764      169666<br>
Swap:         16383        3492       12891</p>
Why do we need mean of 0 and std of 1 for prediction?

vainaijr: <p>What is the reason behind this?</p>

KFrank: <p>Hi Vainaijr!</p>
<aside class="quote no-group" data-username="vainaijr" data-post="1" data-topic="52385" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/vainaijr/48/49167_2.png" class="avatar"> vainaijr:</div>
<blockquote>
<p>What is the reason behind this?</p>
</blockquote>
</aside>
<p>The short answer is that we don’t need a mean of 0 and a standard<br>
deviation of 1.</p>
<p>I assume that you are talking about “targets” – the output values you<br>
are using to train with and then will be using the trained network to<br>
predict.</p>
<p>There can be some benefits to “normalizing” your targets – rescaling<br>
your targets so that their mean and standard deviation are roughly<br>
0 and 1, respectively, but this is in no way necessary.</p>
<p>(My comments apply equally well to the question of normalizing<br>
inputs, but let me speak in terms of targets for simplicity.)</p>
<p>If your targets are <em>very</em> large, you could run into overflow problems<br>
(NaNs) when training.</p>
<p>If you start out with a sensible random initialization of your network<br>
weights (and biases), your network won’t start out knowing anything<br>
about the mean and standard deviation of your targets, and will<br>
(sort of) start out tending to predict them to be 0 and 1, respectively.<br>
So it will have to learn the mean and standard deviation, but this<br>
isn’t a big deal.</p>
<p>If you change the scale of your targets you will often be changing the<br>
scale of your loss function as a result.  For example, if you double<br>
your targets, you will, in effect, multiply the mean-squared-error<br>
loss function by four.  This will then, in effect, multiply your learning<br>
rate by four.</p>
<p>But (with the exception of overflow for extreme values) these are<br>
minor issues and won’t significantly affect the training or predictive<br>
performance of your network.</p>
<p>You can see this for yourself.  Generate some multivariate regression<br>
data (with a little noise in it).  Your targets don’t have to be a Gaussian<br>
distribution, but don’t make them too wacky.  Rescale the targets<br>
twice – once to a mean and standard deviation of 0 and 1, and<br>
again to something else, say a mean of -5 and a standard deviation<br>
of 10.  Train a simple network, say a single hidden layer and a<br>
mean-squared-error loss function.  Train on both the normalized<br>
and unnormalized datasets.  You should get very similar results,<br>
with (perhaps) slightly slower initial training for the unnormalized<br>
data as the network learns the overall scale of the data.</p>
<p>The biggest difference will be that by scaling up the standard<br>
deviation by a factor of ten you will have, in effect, multiplied<br>
your learning rate by one hundred.  So reducing your learning<br>
rate for the unnormalized data will make the two training runs<br>
even more similar.</p>
<p>Have fun!</p>
<p>K. Frank</p>
OSError: [Errno 12] Cannot allocate memory

xiao: <p>There is such an error in the operation, why? Because the memory is not enough? It was still good when I was running a few days ago.</p>

InnovArul: <p>Can you check your disk space? It seems your disk is full.</p>

kangkang_li: <p>Did you solve it? I meet this error too.</p>

syx0629: <p>Did you solve it? I meet this error too.</p>

ptrblck: <p>Could you check your memory usage (RAM and swap)? This error is usually thrown, if your run out of memory.</p>

syx0629: <p>I know how to avoid this mistake. If the host configuration is not high and the amount of data is large, it is recommended to turn off multithreaded reading, that is, num_workers is set to 0</p>
Adding graph for a model with MaxUnpool layers

eiriksfa: <p>Hello, using torch.utils.tensorboard and tb-nightly I am having some issues adding a graph to the summary. The model I am using is similar to the rgbd ENet implementation (ENetDepth at <a href="https://github.com/montrealrobotics/ENetDepth/blob/master/models/enet.py" rel="nofollow noopener">https://github.com/montrealrobotics/ENetDepth/blob/master/models/enet.py</a>).<br>
I am getting the error:</p>
<pre><code class="lang-auto">UserWarning: ONNX export failed on ATen operator max_unpool2d because torch.onnx.symbolic.max_unpool2d does not exist
  .format(op_name, op_name))
</code></pre>
<p>The model uses maxunpool layers in the UpsamplingBottlenecks:</p>
<pre><code class="lang-auto">self.main_unpool1 = nn.MaxUnpool2d(kernel_size=2)
</code></pre>
<p>and forwarded:</p>
<pre><code class="lang-auto">        # Main branch shortcut
        main = self.main_conv1(x)
        main = self.main_unpool1(main, max_indices)
        # Extension branch
        ext = self.ext_conv1(x)
        ext = self.ext_conv2(ext)
        ext = self.ext_conv3(ext)
        ext = self.ext_regul(ext)

        # Add main and extension branches
        out = main + ext

        return self.out_prelu(out)
</code></pre>
<p>I was wondering if anyone had any tips on a solution to my problem, or a quick workaround?</p>
<p>Thanks</p>
Implementation Customized Convoultion Kernels

Lanselott1: <p>Hi,</p>
<p>I’m going to implement an convolution layer with custom kernels.<br>
For each location in tensors, I want to do convolution with different kernels.<br>
For example:<br>
for input[:,:,4,5], do convolution with kernel1,<br>
for intput[:,:,4,6], do convolutino with kernel2.</p>
<p>How can I implement it?<br>
Thanks!</p>
<p>Regards</p>

imaluengo: <p>You can:</p>
<ol>
<li>Create a convolutional kernel<br>
<code>conv = nn.Conv2(in, out, ksize, ...)</code>
</li>
<li>Copy your kernel weights to it<br>
<code>conv.weights.copy_(myweights)</code>
</li>
</ol>
<p>Just be aware that you will need to use <code>torch.no_grad</code> if you dont want that convoluton’s weights to be updated while training.</p>

Lanselott1: <p>Thanks! <a class="mention" href="/u/imaluengo">@imaluengo</a> I will try it.</p>

imaluengo: <p>Also for avoiding grads, this might help:</p>
<pre><code class="lang-auto">for p in conv.parameters():
    p.requires_grad = False
</code></pre>

Mosquito_ya: <p>Have you solved this?</p>
Caffe2 c++ api doesn't work on Ubuntu

nurettinabaci: <p>Hello, I need to run <a href="https://github.com/leonardvandriel/caffe2_cpp_tutorial/blob/master/src/caffe2/binaries/intro.cc" rel="nofollow noopener">these</a> tutorials. I installed PyTorch on Ubuntu 16 with the instructions in <a href="https://books.google.com.tr/books?id=sTmbDwAAQBAJ&amp;pg=PA14&amp;lpg=PA14&amp;dq=caffe2+c%2B%2B&amp;source=bl&amp;ots=YdWJPGmAt2&amp;sig=ACfU3U24TdQpRvI3ItbMWMmhp85TAA6_mw&amp;hl=tr&amp;sa=X&amp;ved=2ahUKEwi-_IfO_NHjAhXE86YKHafXAU4Q6AEwCXoECAkQAQ#v=onepage&amp;q&amp;f=false" rel="nofollow noopener">this book</a> and it works well with Python, but when I test it in c++  (Caffe2 C++ api), it prints that:</p>
<pre><code class="lang-auto">root@baki:~/caffe2_cpp_tutorial/build# cmake ..
../caffe2: warning: directory does not exist.
CMake Error at CMakeLists.txt:36 (message):
  Caffe2 lib not found


-- Configuring incomplete, errors occurred!

</code></pre>
<p>What can I do?</p>
How to load previously saved model in cpp frontend?

Ronit-j: <p>Hello, I was trying to load a previous model which was saved using a python script in cpp frontend but got some error. Also is there a way to use load_state_dict function which is present in python to load the saved state dict in cpp?<br>
thanks in advance.</p>

mhubii: <p><a href="https://discuss.pytorch.org/t/can-i-creat-model-during-runtime/52205/2">Here</a> is actually a nice explanation by <a class="mention" href="/u/yf225">@yf225</a></p>
<p>Should answer your question as well. Please report back if not.</p>

yf225: <p><a class="mention" href="/u/ronit-j">@Ronit-j</a> Besides trying out the post that Martin linked, could you also share the error you see, and the code to reproduce the error?</p>
<p>We are working on adding <code>load_state_dict</code> to C++ frontend, but one caveat is that the exact same model structure needs to <strong>also</strong> be defined in C++ frontend (i.e. you’ll need to write C++ code that defines the same model structure as the Python model). If this is undesirable, you can also trace the model using JIT and then load it into C++ frontend (<a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html</a> is a good guide on this).</p>

Ronit-j: <p>I tried running  the  example that you referred I get this error <a href="https://paste.ubuntu.com/p/3KMv7xBwKc/" rel="nofollow noopener">https://paste.ubuntu.com/p/3KMv7xBwKc/</a></p>

Ronit-j: <p>Thank you I’ll try it and let you know.</p>

mhubii: <p>I think due to some changes <code>torch::jit::load</code> does not return a shared pointer anymore. Check for example the <a href="https://github.com/pytorch/pytorch/blob/5b87049c66f0c64b405145ff57869de98729dae9/test/custom_operator/test_custom_ops.cpp#L51" rel="nofollow noopener">tests</a>. You would need to replace every shared pointer like so</p>
<pre><code class="lang-auto">// Replace
std::shared_ptr&lt;torch::jit::script::Module&gt; module = torch::jit::load(module_path);
// by this
torch::jit::script::Module module = torch::jit::load(module_path);

// replace every -&gt; by .
// for example
torch::Tensor prediction = module-&gt;forward(inputs).toTensor();
// by 
torch::Tensor prediction = module.forward(inputs).toTensor();
</code></pre>
<p>I am not too familiar with the inner workings here so I dont know why this was done. But it will fix your problem. The <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">tutorial</a> linked earlier, is also currently being updated according to this <a href="https://github.com/pytorch/tutorials/pull/560" rel="nofollow noopener">pull request</a>.</p>
Overflow error with Python 2 but not Python 3?

Kaixhin: <p>When running my <a href="https://github.com/Kaixhin/Rainbow" rel="nofollow noopener">Rainbow agent</a> in Python 2, e.g. to quickly fail via <code>python main.py --learn-start 1000 --evaluation-interval 1100 --disable-cuda</code>, I get the error <code>RuntimeError: invalid argument 2: out of range at /opt/conda/conda-bld/pytorch_1556653194318/work/aten/src/TH/generic/THTensor.cpp:639</code> on <a href="https://github.com/Kaixhin/Rainbow/blob/master/agent.py#L79" rel="nofollow noopener">this line</a>. In Python 2, the two last arguments get incredibly large negative numbers (e.g. <code>-9223372036854775808</code>), and <code>-inf</code>, but the same set of arguments under Python 3 get the expected inputs: <code>0</code>, <code>1</code>, …, <code>1630</code>, and <code>0.0049</code>, <code>0.0053</code>… Not sure why changing Python version should cause this problem.</p>
<p>Using Python 1.1.0 with CUDA 10 for both Python 2 and 3.</p>

ptrblck: <p>Thanks for the reproducible code snippet.<br>
In Python2.x the <code>/</code> operator is an integer division, if the inputs are integers (in Python3 it’s <code>//</code>).<br>
Therefore, <code>self.delta_z</code> will be zero:</p>
<pre><code class="lang-python">#Python2.7
(10 - -10) / (51 - 1)
&gt; 0

#Python3.6
(10 - -10) / (51 - 1)
&gt; 0.4
</code></pre>
<p>Since <code>delta_z</code> is zero, <code>b</code> will be <code>Inf</code>.<br>
The small number (<code>-922...</code>) is apparently the “int64-representation” of an Inf, but I’m not sure. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
<p>Add this line as the first import in <code>agent.py</code> and your script should work:</p>
<pre><code class="lang-python">from __future__ import division
</code></pre>

Kaixhin: <p>Ah thanks for spotting that! Not used to writing completely cross-compatible Python code, but someone requested and I thought I’d try. Probably the only place it is an issue, but sprinkling that import in any file with a division just in case <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=9" title=":sweat_smile:" class="emoji" alt=":sweat_smile:"> With this I’ve hopefully tracked down any other cross-compatibility issues as I’ve now got it running fine on Python 2 <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>
Torch.nn.functional.lstm

livc: <p>Hi, all</p>
<p>I want to use a functional lstm, just as <code>torch.nn.functional.linear</code>, how to reach that?</p>
<p>Thanks.</p>

silverjoda: <p>Just implement the function yourself. It will take you a few mins. You can find the equations of the LSTM<br>
in the docs under nn.LSTM .</p>

anton_schwarz: <p>It is possible, using the _VF.lstm() function found here: <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/rnn.py" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/nn/modules/rnn.py</a></p>
<p>It will run faster than a manual implementation and be compatible with additional PyTorch features such as dropout.</p>
<p>Model parameters can be passed in using [tensor for tensor in ordereddict.values()] provided the OrderedDict’s keys are ordered as those of the standard pytorch lstm.state_dict().</p>
RuntimeError: size mismatch, m1: [192 x 68], m2: [1024 x 68] at /opt/conda/conda-bld/pytorch_/work/aten/src/THC/generic/THCTensorMathBlas.cu:268

aerinykim: <p>I’m getting a size mismatch error that I can’t understand.</p>
<pre><code>(Pdb) self.W_di
Linear(in_features=68, out_features=1024, bias=True)
(Pdb) indices.size()
torch.Size([32, 6, 68])
(Pdb) self.W_di(indices)
*** RuntimeError: size mismatch, m1: [192 x 68], m2: [1024 x 68] at /opt/conda/conda-bld/pytorch_1556653099582/work/aten/src/THC/generic/THCTensorMathBlas.cu:268
</code></pre>
<p>Why is there a mismatch?<br>
Maybe because of the way I defined the weight in <code>forward</code> (instead of <code>_init_</code>)?</p>
<p>This is how I defined <code>self.W_di</code>:</p>
<pre><code>def forward(self):

    if self.W_di is None:
        self.W_di_weight = nn.Parameter(torch.randn(mL_n * 2,1024).to(device))
        self.W_di_bias = nn.Parameter(torch.ones(1024).to(device))                  

    self.W_di = nn.Linear(mL_n * 2, 1024)
    self.W_di.weight = self.W_di_weight
    self.W_di.bias = self.W_di_bias

    result = self.W_di(indices)
</code></pre>
<p>Any pointer would be highly appreciated!</p>

Nikronic: <p>Hi,</p>
<p>I have tested your code and error is about your weight initialization. Actually, you have to transpose the <code>weight</code> you have considered or reverse sizes.<br>
but the conventional way of initializing <code>weight</code> or <code>std</code> is using the <code>torch.nn.init</code> module.</p>
<blockquote>
<p>Here is your code using this convention, and it works:</p>
</blockquote>
<pre data-code-wrap="python"><code class="lang-python">
import torch
import torch.nn as nn

w_di = nn.Linear(68, 1024, bias= True).cuda()
indices = torch.randn(32, 6, 68).cuda()
torch.nn.init.normal_(w_di.weight)
torch.nn.init.constant_(w_di.bias, 1.0)
w_di(indices)
</code></pre>
<blockquote>
<p>And this is your updated code:</p>
</blockquote>
<pre data-code-wrap="python"><code class="lang-python">
import torch
import torch.nn as nn

w_di = nn.Linear(68, 1024, bias= True).cuda()
indices = torch.randn(32, 6, 68).cuda()
w_di.weight = nn.Parameter(torch.randn(34 * 2,1024).t().cuda())
w_di.bias = nn.Parameter(torch.ones(1024).cuda())  
w_di(indices)
</code></pre>
<p>Bests<br>
Nik</p>
Pytorch most efficient Jacobian / Hessian calculation

alireSa_Ranjbar: <p>I am looking for the most efficient way to get the Jacobian of a function through Pytorch and have so far come up with the following solutions:</p>
<pre><code class="lang-auto">def func(X):
    return torch.stack((
                     X.pow(2).sum(1),
                     X.pow(3).sum(1),
                     X.pow(4).sum(1)
                      ),1)  

X = Variable(torch.ones(1,int(1e5))*2.00094, requires_grad=True).cuda()

                                # Solution 1:
t = time()
Y = func(X)
J = torch.zeros(3, int(1e5))
for i in range(3):
    J[i] = grad(Y[0][i], X, create_graph=True, retain_graph=True, allow_unused=True)[0]
print(time()-t)
Output: 0.002 s

                                # Solution 2:
def Jacobian(f,X):
    X_batch = Variable(X.repeat(3,1), requires_grad=True)
    f(X_batch).backward(torch.eye(3).cuda(),  retain_graph=True)
    return X_batch.grad

t = time()
J2 = Jacobian(func,X)
print(time()-t)
Output: 0.001 s

</code></pre>
<p>Since there seem to be not a big difference between using a loop in the first solution than the second one, I wanted to ask if there might still be be a faster way to calculate a Jacobian in pytorch.</p>
<p>My other question is then also about what might be the most efficient way to calculate the Hessian.</p>
<p>Finally, does anyone know if something like this can be done easier or more efficient in TensorFlow?</p>

nima_rafiee: <p>Continuing the discussion from <a href="https://discuss.pytorch.org/t/pytorch-most-efficient-jacobian-hessian-calculation/47417">Pytorch most efficient Jacobian / Hessian calculation</a>:</p>
<p>This is the fastest way I can tell<br>
suppose you have a loss function of x</p>
<pre><code class="lang-auto">        loss = f(x)
        first_drv = torch.zeros(batch_size, x_dim)
        hessian = torch.zeros(batch_size, x_dim, x_dim)
        for n in range(batch_size):
            first_drv[n] = torch.autograd.grad(loss[n], dz,
                                                     create_graph=True, retain_graph=True)[0][n]
            for i in range(x_dim):
                hessian[n][i] = torch.autograd.grad(first_drv[n][i], dz,
                                                        create_graph=True, retain_graph=True)[0][n]
</code></pre>
Building multi label network in pytorch

RegularGuY: <p>Trying to solve a multi label image classification problem,an 8 class multi label problem . My labels look like this</p>
<pre><code>	[0. 1. 0. 0. 1. 0. 0. 0.]
	[0. 1. 0. 0. 0. 0. 0. 0.]
	[0. 1. 0. 1. 0. 0. 0. 0.]
</code></pre>
<p>The code looks like this</p>
<pre><code>model = models.resnet50(pretrained=True)
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 8)
model = model.cuda()
# Decay LR by a factor of 0.1 every 7 epochs
# exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)

optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
loss_fn = torch.nn.BCELoss()



# def train(epoch):
for epoch in range(15):
    model.train()
    for batch_idx, (data, target) in enumerate(train_loader):
        # import pdb;pdb.set_trace()
        data, target = data.cuda(), target.float().cuda()
        optimizer.zero_grad()
        output = model(data)
        output = torch.sigmoid(output)
        loss = loss_fn(output, target)
        loss.backward()
        optimizer.step()
</code></pre>
<p>For now the results are not good, am i doing something wrong?<br>
Any suggestions would be really helpful.Thanks in advance.</p>

wen75741: <p>Try to use adam optimizer, tune learning rate and batch size.</p>

RegularGuY: <p><a class="mention" href="/u/wen75741">@wen75741</a> Thanks let me check</p>
Sharing parameters of multiple traces

trewick: <p>Is it possible to share parameters of multiple JIT traces? I have a module that has two methods that I’d like to call from C++. Both methods touch the same hierarchy of submodules of the module, but in different ways (e.g. different order). I also need to optimize the parameters in C++, so the parameters will change gradually and both methods need to see the latest values.</p>
<p>My understanding is that if I trace each method (in 1.1), I will obtain two ScriptModule objects, each having its own copy of the parameters. It is not clear to me if it is possible to share the parameters somehow.</p>
<p>Looking at the docs of 1.2 (1.2.0a0+84c2c89), it seems one can trace multiple methods of one module. Will the parameters be shared correctly?</p>
<p>Finally, since the submodules are hierarchical, will parameters() on the module return all parameters touched by the traces?</p>

villedepommes: <blockquote>
<p>Will the parameters be shared correctly?</p>
</blockquote>
<p>Yes, parameters should be shared across module’s methods.</p>
<p>In this <a href="https://github.com/pytorch/pytorch/blob/master/test/test_jit.py#L3159" rel="noopener nofollow ugc">test</a>, for example, <code>conv.weight</code> is shared by both <code>forward</code> and <code>weighted_kernel_sum</code>.</p>
<blockquote>
<p>Finally, since the submodules are hierarchical, will parameters() on the module return all parameters touched by the traces</p>
</blockquote>
<p>Yes, that should work as well, unless I’ve misunderstood the question.</p>
Pytorch Whl not supported Windows 7

Caleb_Grohs: <p>Hello,<br>
I am trying to install Pytorch using the Quick Start Locally menu on the website. I chose the Stable 1.1 version for Windows, selected Pip as my package manager, chose my python version (3.7), and no CUDA. I have already installed numpy and updated my Pip, but when I run the commands in my command line, I get “ERROR: torch-1.1.0-cp37-cp37m-win_amd64.whl is not a supported wheel on this platform” and “ERROR: torchvision-0.3.0-cp37-cp37m-win_amd64.whl is not a supported wheel on this platform”. I have already checked to make sure that I am using the correct command for my python version and tried reading other solutions online. Could anyone give me some advice and guidance for fixing this? Thanks!<br>
FYI: I am using windows 7 64 bit with python 3.7.4 installed and pip 19.2.1 without cuda or the Anaconda thing</p>

Caleb_Grohs: <p>Just figured it out, I had 64 bit python installed, but I was using a 32 bit version. Thanks anyway!</p>
Modify output gradients per layer in back-propagation

omar: <p>How can I modify the output gradients (<em>dE/dO</em>) per layer before calculating the input gradients (<em>dE/dI</em>) during back-propagation? I need to mask the output gradients as follows: <em>dE/dI = w * mask1(dE/dO)</em>. I checked <em>register_backward_hook()</em>, but the documentation mentions I can only modify the input gradients <em>dE/dI</em> after it has been calculated. Same question applies for masking output gradients before calculating weight gradients <em>dE/dW = I * mask2(dE/dO)</em>. Please note that <em>mask1</em> and <em>mask2</em> can be different.</p>
Accuracy in Semantic Segmentation

Nishanth_Sasankan: <p>Hi All,</p>
<p>I am performing Semantic segmentation I can print the loss during the iteration using the code below</p>
<pre><code class="lang-auto">for iter in range(num_epochs):
    print(iter)	
    for (i,l) in trainloader:
        i= i.to(device)
        l = l.to(device=device, dtype=torch.int64)
        outt = model(i)
        loss = criterion(outt['out'], l.squeeze(1))
        print(loss)
        loss.backward()
        optimizer.step()
</code></pre>
<p>Is there a way to print out accuracy as well? Any help is appreciated</p>
<p>Thanks<br>
Nishanth</p>

ptrblck: <p>Something like this should work:</p>
<pre><code class="lang-python">N = 5
nb_classes = 10
h, w = 24, 24
output = torch.randn(N, nb_classes, h, w)
target = torch.randint(0, nb_classes, (N, h, w))

pred = torch.argmax(output, 1)
acc = (pred == target).float().mean()
</code></pre>
Compute gradient of computation

LeanderK: <p>I am interested in computing f’(x) where f is an ordinary pytorch computation. The thing is that I don’t really care about f(x), I only need the gradient. I could write the computation by hand but that’s pretty tedious. Is using pytorchs autograd-framework efficient for these sort of things? f’(x) is inside a torch.no_grad(), so I am not interested in the gradient of the gradient-computation.</p>
<p>I have a few use-cases, but one is that I need the gradient for |1-Var(x)| for x where the variance is computed over the batch-dim.</p>
Accessing label during training

DDong: <p>Is there a way to access the real label of a training sample for use during the backprop process? I’m implementing this paper:<br>
<a href="https://arxiv.org/pdf/1901.02413.pdf" rel="nofollow noopener">Interpretable CNNs</a><br>
and, to do the optimizations in the backpropagation section, I need to calculate c-hat, which is computed using the real labels, as described in section 3.2.</p>
Importing torch causes Segfault

TGM: <p>I’ve been trying to get PyTorch to work on my AMD GPU using the ROCm drivers. And I’ve had to compile PyTorch from source in order to accomplish this (the prebuilt PyTorch binaries with ROCm enabled only support Vega GPUs and I have an RX 470). There were no errors during the build process and the installation, but it seems something must’ve gone wrong because when I try to just run “import torch” in the python shell, it appears to start loading for a few seconds before returning this and exiting:</p>
<p>python3: Relink <code>/lib/x86_64-linux-gnu/libsystemd.so.0' with</code>/lib/x86_64-linux-gnu/librt.so.1’ for IFUNC symbol <code>clock_gettime' python3: Relink</code>/lib/x86_64-linux-gnu/libudev.so.1’ with <code>/lib/x86_64-linux-gnu/librt.so.1' for IFUNC symbol</code>clock_gettime’<br>
Segmentation fault (core dumped)</p>
<p>I can’t find any information on this error online, so any help in getting this to work would be greatly appreciated.</p>
Nightly build for Linux missing from Anaconda repos

calincru: <p>As the title says, the latest nightly build for linux-64 is missing. See <a href="https://anaconda.org/pytorch/pytorch-nightly-cpu" rel="nofollow noopener">https://anaconda.org/pytorch/pytorch-nightly-cpu</a>.</p>
<p>This only affects the builds for CPU (i.e., python-nightly-cpu, not python-nightly).  Does anyone know what could cause that?  I was highly expecting a feature that has been pushed yesterday.</p>
Is there an implementation of IndRNN?

pinocchio: <p>I was looking at <a href="https://arxiv.org/abs/1803.04831" rel="nofollow noopener">IndRNN</a>:</p>
<p><code>Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN</code></p>
<p>and was looking for an <strong>official</strong> pytorch implementation. Is there one?</p>
Using a frozen model right before the output?

amirhf: <p>Does it make sense to have a frozen model right before the output and update earlier parameters?</p>
<p>I am training a model like:</p>
<p>input -&gt; FC layers -&gt; frozen RNN model -&gt; binary output -&gt; loss</p>
<p>I just want to update the FC layer weights. If there’s any related research or topic you can refer me to, that would be appreciated.</p>
Data Preprocessing for Tiny ImageNet

deJQK: <p>Is there any good methods of data preprocessing for tiny imagenet? It seems the data augmentation methods for imagenet does not work well for tiny imagenet. Here is my code:</p>
<pre><code class="lang-auto">    normalize = transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
    train_dataset = datasets.ImageFolder(
        train_dir,
        transforms.Compose([
            transforms.RandomResizedCrop(224),
            transforms.RandomHorizontalFlip(),
            transforms.ToTensor(),
            normalize
        ]))
    train_loader = torch.utils.data.DataLoader(
        train_dataset,
        batch_size=args.batch_size, shuffle=True, **kwargs)
    valid_loader = torch.utils.data.DataLoader(
        datasets.ImageFolder(valid_dir, transforms.Compose([
            transforms.Resize(256),
            transforms.CenterCrop(224),
            transforms.ToTensor(),
            normalize
        ])),
        batch_size=args.test_batch_size, shuffle=False, **kwargs)

</code></pre>

ginger_ali: <p>I was also wondering if there is an accepted standard data augmentation procedure for Tiny ImageNet?</p>
<p><a class="mention" href="/u/dejqk">@deJQK</a> tiny ImageNet images are 64x 64 so taking crops of 224 pixels, or resizing to 256 is probably not such a great idea.</p>
Semantic segmentation loss function

Tingyu_Wang: <p>Hi Everyone,<br>
I am pretty new to pytorch and I am trying to implement semantic segmentation using<br>
<code>torchvision.models.segmentation.</code> <code>fcn_resnet50</code> as mentioned on <a href="https://pytorch.org/docs/stable/torchvision/models.html#semantic-segmentation" rel="nofollow noopener">https://pytorch.org/docs/stable/torchvision/models.html#semantic-segmentation</a></p>
<p>While running my code, I receive the following error:<br>
‘collections.OrderedDict’ object has no attribute ‘log_softmax’</p>
<p>This correspond to the line in the code —&gt; 10     loss = criterion(outputs, labels)<br>
I defined my criterion loss function as criterion = nn.CrossEntropyLoss()</p>
<p>Any help is appreciated!!!<br>
Thanks<br>
Troy</p>

ptrblck: <p>Note that (some) torchvision segmentation models will use a <code>dict</code> as the output.<br>
Could you check that?</p>

shyxiaoxiao: <p>Hi, <a class="mention" href="/u/ptrblck">@ptrblck</a><br>
I used the model <code>deeplabv3_resnet101</code> and met the same error. I checked that the output is indeed a dict object.<br>
Could you tell me how to use the output in the criterion?<br>
Here is my code in which <code>inputs</code> is a tesor of NxCxHxW and  <code>labels</code> is Nx1xHxW</p>
<pre><code class="lang-python">criterion = nn.CrossEntropyLoss(ignore_index=0)
outputs = model(inputs)
loss = criterion(outputs, labels)
</code></pre>
<p>I tried to use<br>
<code>loss = criterion(outputs["out"], labels)</code>.<br>
But there is a RuntimeError: invalid argument 3: only batches of spatial targets supported (3D tensors) but got targets of dimension: 4 at …\aten\src\THNN/generic/SpatialClassNLLCriterion.c:59</p>
<p>Another question is that my dataset(images) have 5 classes and some ignoer pixels then should I set the <code>num_classes</code> to 5 or 6 in the model ?<br>
Thank you!</p>

ptrblck: <p><code>outputs['out']</code> should work and the error points to the additional dimension in your <code>labels</code> tensor.<br>
Squeeze dim1 before passing <code>labels</code> to the loss function and it should work:</p>
<pre data-code-wrap="python"><code class="lang-python">labels = labels.squeeze(1)
</code></pre>
<aside class="quote no-group" data-username="shyxiaoxiao" data-post="3" data-topic="48431">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/13edae/48.png" class="avatar"> shyxiaoxiao:</div>
<blockquote>
<p>Another question is that my dataset(images) have 5 classes and some ignoer pixels then should I set the <code>num_classes</code> to 5 or 6 in the model ?</p>
</blockquote>
</aside>
<p>If you have 5 valid classes and an additional class you want to ignore, you could use 6 classes in total and pass <code>ignore_index=class_index_to_ignore</code> to your loss function.</p>

shyxiaoxiao: <p><a class="mention" href="/u/ptrblck">@ptrblck</a>  It works now. Many thanks!</p>
Some questions about torch.unique()

bulubulubu-Yu: <p>Hello, I have some questions about torch.unique().</p>
<p>For a cpu tensor:</p>
<pre><code class="lang-auto">input = torch.tensor([[2,5,7,6],[9,7,4,8],[1,3,2,3],[2,5,7,6]])

input                                                      
tensor([[2, 5, 7, 6],
        [9, 7, 4, 8],
        [1, 3, 2, 3],
        [2, 5, 7, 6]])

torch.unique(input)
tensor([3, 1, 8, 4, 9, 6, 7, 5, 2])

torch.unique(input, sorted=True)
tensor([1, 2, 3, 4, 5, 6, 7, 8, 9])

torch.unique(input, dim=0)
tensor([[1, 3, 2, 3],
        [2, 5, 7, 6],
        [9, 7, 4, 8]])

torch.unique(input, sorted=True, dim=0)
tensor([[1, 3, 2, 3],
        [2, 5, 7, 6],
        [9, 7, 4, 8]])

torch.unique(input, sorted=False, dim=0)
tensor([[1, 3, 2, 3],
        [2, 5, 7, 6],
        [9, 7, 4, 8]])

</code></pre>
<p>I note that for torch.unique the default sorted=True in official documentation (<a href="https://pytorch.org/docs/stable/_modules/torch/functional.html#unique" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torch/functional.html#unique</a>). However, when using torch.unique(input), the output is unsorted. When I use torch.unique(input, sorted=False, dim=0), the output is still sorted.</p>
<p>Furthermore, I also test a gpu tensor, and find all the output results are sorted regardless of whether setting sorted=True or False.</p>
<p>Why? There seems something wrong about it?</p>

ptrblck: <p>Depending on the underlying implementation the tensor is sorted e.g. for performance reasons.<br>
All CUDA tensors are sorted by default, due to limitations in <code>thrust</code>.</p>

bulubulubu-Yu: <p>So the only special case is that when input is a cpu tensor and the dim arg is None, output is unsorted by default. Right?</p>

ptrblck: <p>If I haven’t missed a code path, I think you are right.</p>

bulubulubu-Yu: <p>OK, gotcha. Thanks for your reply!</p>
Writing a small Pytorch extension in C++

prabhuteja12: <p>Hi</p>
<p>I’m trying to write a small extension to PyTorch in C++. I’ve tried to look at the examples provided and see that the syntax is quite similar to what the python syntax is.</p>
<p><strong>Part - 1</strong><br>
Here is my code for the extension</p>
<pre><code class="lang-auto">#include &lt;torch/extension.h&gt;
#include &lt;vector&gt;
#include &lt;iostream&gt;


torch::Tensor compute_grad(torch::Tensor in, torch::Tensor out)
{
    std::cout &lt;&lt; "Input is:" &lt;&lt; in &lt;&lt; "\n" &lt;&lt; "Output is: " &lt;&lt; out &lt;&lt; "\n";
    auto b = out.sum();
    std::cout &lt;&lt; "Out_sum:" &lt;&lt; "\t" &lt;&lt; b &lt;&lt; std::endl;
    b.backward();
    std::cout &lt;&lt; __LINE__ &lt;&lt; std::endl;
    return in.grad();
}

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m)
{
    m.def("compute_grad", &amp;compute_grad, "Compute Gradient");
}

</code></pre>
<p>and I call it with</p>
<pre><code class="lang-auto">import torch
import DenseGrad

if __name__ == "__main__":
    x = (2 * torch.ones(2, 2)).requires_grad_()
    y = x ** 3
    print(DenseGrad.compute_grad(x, y))
</code></pre>
<p>For some reason, the <code>backward()</code> call is getting stuck and the only way to kill it is through <code>top</code>. Can someone tell me what I’m doing wrong?</p>

LeviViana: <p>I tried running your program with gdb, I’m getting a deadlock on the autograd execution engine. There are some posts related to this one, and a currently opened issue on PyTorch’s GitHub.</p>
<p>It seems to me that people have solved this by changing the environment, it could be related to PyTorch’s build and dependencies.</p>
<ul>
<li><a href="https://discuss.pytorch.org/t/solved-archlinux-using-variable-backwards-appears-to-hang-program-indefinitely/1675/21">Post related</a></li>
<li><a href="https://discuss.pytorch.org/t/deadlock-during-backward/18863/10">Another one</a></li>
<li><a href="https://github.com/pytorch/pytorch/issues/20355" rel="nofollow noopener">Issue</a></li>
</ul>

prabhuteja12: <p>Hi!<br>
Thanks for taking the time to put it through gdb. From <a href="https://discuss.pytorch.org/t/solved-archlinux-using-variable-backwards-appears-to-hang-program-indefinitely/1675/23">this</a>, it seems to have been fixed a while ago, and I’m using conda to get the latest version of pytorch (1.1.0).</p>
<p>I’m unsure how to go about it though.</p>

mhubii: <p>Hi there. <a class="mention" href="/u/haydnspass">@Haydnspass</a> and me have created a simple example extension. You can find it <a href="https://github.com/mhubii/simple_pytorch_extension" rel="nofollow noopener">here</a>. Hope it helps</p>
Update weights of a particular layer instead of back propagating

Rakesh_Kota: <p>I am using custom loss function to calculate loss in each layer. I need to update the weights for that layer only. Is this possible</p>

ptrblck: <p>Could you explain your approach a bit more?<br>
Generally, if you only pass the parameters of your special layer to the optimizer, only these parameters will be updated.<br>
Would that work for you?</p>

Rakesh_Kota: <p>consider i have 3 layers and i am calculating loss that is caused by each layer(3 loses) separately by a custom loss function and i need to update the weights</p>

ptrblck: <p>Are you able to call <code>.backward()</code> on your custom loss?<br>
If so, my approach should work.</p>

Rakesh_Kota: <p>thank you first of all and can u explain how to optimize only single layer ,like we pass model.parameters() in optimizer what should be passed for single layer and yes .backward is working</p>

ptrblck: <p>For a single layer, you could pass only these parameters to the optimizer:</p>
<pre><code class="lang-python">optimizer = torch.optim.SGD(model.layer1.parameters(), lr=1e-3)
</code></pre>
<p>If you need to pass some combination of different parameter sets, you can just pass them as a list:</p>
<pre><code class="lang-python">params = list(model.layer1.parameters()) + list(model.layer17.parameters())
torch.optim.SGD(params, lr=1e-3)
</code></pre>

Rakesh_Kota: <p>Really thanks man its working.Thanks for ur help very much.</p>
Vgg and softmax + features

RicLaGra: <p>Hello everyone, can anyone suggest me the right approach to get from data <em>features</em> and <em>output</em> from <strong>softmax</strong> using VGG19?<br>
I watched the class of VGG, but that return only the features. Is it possible to obtain the value from the softmax? I need to this for the cross-entropy and to the testing phase.<br>
Thanks in advance.</p>

ptrblck: <p>Hi Ric,</p>
<p>for multi-class classification use cases, you could use <code>nn.CrossEntropyLoss</code> as the loss function. This criterion expects logits as the model output, which explains the missing softmax (or log_softmax) after the final layer in the <code>torchvision</code> models.</p>
<p>If you really want to apply a softmax on the output, you could use the following code:</p>
<pre><code class="lang-python">import torch.nn.functional as F

output = model(data)
prob = F.softmax(output, 1)
</code></pre>
<p>However, as described, you <strong>should not</strong> pass the softmax to a loss function.</p>
<p>Alternatively to the raw logits, you could instead apply <code>F.log_softmax(output, 1)</code> and pass it to <code>nn.NLLLoss</code>, which is exactly, was <code>nn.CrossEntropyLoss</code> is doing internally.</p>
<p>Let me know, if that clarifies the question or if you need more information.</p>

RicLaGra: <p>Thanks for reply <a class="mention" href="/u/ptrblck">@ptrblck</a>. I applied as you said, but probably i’ve a problem into test phase. For a better understanding a put the code:</p>
<pre><code>with torch.no_grad():
    for data, labels in testloader:
        if use_gpu:
            data, labels = data.cuda(), labels.cuda()

        features = model(data)
        sm = torch.nn.Softmax()
        probabilities = sm(features)

        predictions = probabilities.data.max(1)[1]
        total += labels.size(0)
        correct += (predictions == labels.data).sum()
</code></pre>
<p>Is it correct to pass the layer of Softmax result to get accuracy? This is my first time I use VGG19. Thanks for your help.</p>

ptrblck: <p>Yes, your code should be alright.<br>
However, the <code>.max(1)</code> call will also return the same indices, if you apply it on <code>features</code> directly, so you don’t really need the softmax here.</p>
<p>Also, I would recommend to cast the equality check to <code>float</code> before summing, as it’ll return a <code>torch.uint8</code> tensor, which might overflow, if you are using large patches or e.g. pixel-wise accuracy:</p>
<pre><code class="lang-python">correct += (predictions == labels.data).float().sum()
</code></pre>

RicLaGra: <p>ok, i think i solved insering as follow into forward function of VGG:</p>
<p><strong>x = F.log_softmax(out)</strong><br>
Do you think is a good approach? In this way i return features and the result of softmax.</p>

ptrblck: <p><code>F.log_softmax</code> is fine if you are using <code>nn.NLLLoss</code>.</p>
Confused about STN

joshhu: <p>In STN(Spatial Transformer Networks), we need 6 parameters to do affine transforms.  Why the output of a transformation is the previous layer(l-1) of a network while the input is the latter layer(l)? I am so confused, thank you</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/0/06784bb36b6498a016a2977f8c724f1640ed70e8.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/06784bb36b6498a016a2977f8c724f1640ed70e8" title="stnconfuse"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/0/06784bb36b6498a016a2977f8c724f1640ed70e8_2_690x145.jpeg" alt="stnconfuse" data-base62-sha1="VeB1zWG1TlBi3fxAGzK4U223fG" width="690" height="145" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/0/06784bb36b6498a016a2977f8c724f1640ed70e8_2_690x145.jpeg, https://discuss.pytorch.org/uploads/default/optimized/2X/0/06784bb36b6498a016a2977f8c724f1640ed70e8_2_1035x217.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/0/06784bb36b6498a016a2977f8c724f1640ed70e8.jpeg 2x" data-dominant-color="EDEBEA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">stnconfuse</span><span class="informations">1123×236 47 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>