akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Layer-wise output

Nilakshan_Kunanantha: <p>I’m having a PyTorch densenet model for classification purpose.In order to get layer wise output for an image,I can make a submodel upto the relevant layer and donthe inferencenupto that layer.<br>
PyTorch allow me to create a submodel with the required layer,if it is a child.<br>
In case I have a child of block which having some convolution layers.How can I add the particular layer within the child block?or anyways to get output of a layer inside a child-block.<br>
Eg.There are layers inside a denseblock,if I want to do the inference upto a particular layer within the dense block.</p>

JuanFMontesinos: <aside class="quote no-group" data-username="Nilakshan_Kunanantha" data-post="1" data-topic="89462">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/nilakshan_kunanantha/48/15470_2.png" class="avatar"> Nilakshan_Kunanantha:</div>
<blockquote>
<p>elevant layer and donthe infe</p>
</blockquote>
</aside>
<p>To get intermediate outputs you can use hooks.<br>
Here is a tutorial<br>
<a href="https://towardsdatascience.com/the-one-pytorch-trick-which-you-should-know-2d5e9c1da2ca" class="onebox" target="_blank" rel="noopener nofollow ugc">https://towardsdatascience.com/the-one-pytorch-trick-which-you-should-know-2d5e9c1da2ca</a></p>

Nilakshan_Kunanantha: <p>Yes I did use them,but it dumps the funcyion called in forward function.Is there alternate ways to dunps the output of layers within a child?</p>

JuanFMontesinos: <p>You can modify the sourcecode of the network to return any value that you need. In principle hooks are the way to get intermidiate outputs not modifying the network.<br>
Another way is to save it in self during forward.</p>
Could plz tell me what is the meaning or function about parameter 【kwargs】and the symbol 【××】before kwargs？

111289: <pre><code class="lang-auto">import torch
from torch import nn

class MLP(nn.Module):
    # 声明带有模型参数的层，这里声明了两个全连接层
    def __init__(self, **kwargs):
        # 调用MLP父类Module的构造函数来进行必要的初始化。这样在构造实例时还可以指定其他函数
        # 参数，如“模型参数的访问、初始化和共享”一节将介绍的模型参数params
        super(MLP, self).__init__(**kwargs)
        self.hidden = nn.Linear(784, 256) # 隐藏层
        self.act = nn.ReLU()
        self.output = nn.Linear(256, 10)  # 输出层


    # 定义模型的前向计算，即如何根据输入x计算返回所需要的模型输出
    def forward(self, x):
        a = self.act(self.hidden(x))
        return self.output(a)
</code></pre>

Hmrishav_Bandyopadhy: <p>Hi,<br>
<code>**kwargs</code> enables you to pass additional parameters to a function with specific variable names. To demonstrate, –</p>
<pre><code class="lang-auto">def explaining_kwargs(**kwargs):  
    for key, value in kwargs.items(): 
        print ("Key {} Value {}".format(key, value))
  
explaining_kwargs(Yes="Shi",I="Yishi",teacher="Laoshi")
</code></pre>
<p>This would print:</p>
<pre><code class="lang-auto">Key Yes Value Shi
Key I Value Yishi
Key teacher Value Laoshi
</code></pre>
<p>Does that answer your query?</p>
How much of a performance difference is there between Using ATen and MKL?

khalil: <p>Today I compile the source code without MKL Library but I set USE_OPENMP=1.However when I execute the following code:</p>
<pre><code class="lang-auto">import timeit
import torch
runtimes = []
threads = [1] + [t for t in range(2, 49, 2)]
for t in threads:
    torch.set_num_threads(t)
    r = timeit.timeit(setup = "import torch; x = torch.randn(1024, 1024); y = torch.randn(1024, 1024)", stmt="torch.mm(x, y)", number=100)
    runtimes.append(r)
# ... plotting (threads, runtimes) ...
</code></pre>
<p>I found my machine has almost only one core working.My machine has 24 cores, but when I set the thread number to 12, only one core has a load of 100% and the others are about 2% ~ 10%.<br>
However when I compile the source code with MKL, the situation is different. The thread parallelism is great and many cores have a load of 100%.</p>
<p>Can I use ATen to achieve thread parallelism?</p>
How to install pytorch for Ryzen CPU + Nvidia GPU for best performance?

Kushaj: <ul>
<li>CPU - Ryzen 9 3900x</li>
<li>GPU - RTX 2080ti</li>
<li>OS - Ubuntu 20.04</li>
<li>PyTorch - From source</li>
</ul>
<p>I know this question is asked a lot but I was not able to come to a solution. It is getting confusing for me. So I am asking this on the forums. Can someone comment on this, and point the mistakes I made, or things I missed?</p>
<h2>GPU</h2>
<ol>
<li>Install driver (450.), CUDA (11.0), CUDNN (8.0.1).</li>
<li>Add path to <code>.bashrc</code>
</li>
<li>Environment variables for pytorch install <code>USE_CUDA=1 USE_CUDNN=1 TORCH_CUDA_ARCH_LIST="7.0"</code>
</li>
</ol>
<p>I think it is enough for GPU. For my use case, I will only train models (no inference setup).</p>
<h2>Build options</h2>
<p>I found the following environment variables to be useful:</p>
<ul>
<li><code>USE_FBGEMM=1</code></li>
<li><code>USE_MKLDNN=0</code></li>
<li>
<code>USE_DISTRIBUTED=0</code> (Reasoning I have only one GPU)</li>
<li><code>USE_PROF=1</code></li>
</ul>
<h2>CPU libraries</h2>
<p>This is where it gets confusing for me, which libraries to install, how to install them.</p>
<ul>
<li>BLAS/OpenBLAS -&gt; OpenBLAS is optimized version of BLAS, so I should use this.</li>
<li>LAPACK -&gt; It is built on top of BLAS. So I use OpenBLAS, then it would use OpenBLAS.</li>
</ul>
<p>Should I compile these libraries from source or use available ubuntu packages? Also, I think <code>cmake</code> will automatically find these libraries so I don’t have to provide any specific build options.</p>
<p>Is this enough for my use case of PyTorch or I missed something? My use case is mostly training models (including mixed precision, quantization).</p>
<h2>Non pytorch libraries</h2>
<p>This is a random question. If I have installed OpenBLAS on my system, will doing <code>pip install numpy {or some other library}</code> automatically use my system libraries, or will I have to compile these libraries from source also. <strong>Assuming I am working in a conda environment</strong>.</p>
Discriminator identifies fakes much easier than reals

sibidora: <p>Hi I am working on a GAN that generates videos. I have 4 discriminators checking 4 different things in videos. The problem is the accuracy of the discriminators for fakes are much higher than reals.  So it can easily identify the fakes but not the reals. Will it create a problem? Also how can I solve it? I searched about this problem but couldn’t find anything.</p>
<p>Thank you</p>
Flip input for biLSTM

emad: <p>I want to create a bidirectional LSTM from LSTM without using the option <code>bidirectional=True</code> .<br>
So the way to do that is to send the input to LSTM then flip the input and send it back to another LSTM.<br>
My question is: if data dimension is (batch_size, 1, seq_len), what is the correct flip dimension? 0 or 2? and why?</p>
DataParallel vs increasing # workers in data loader

l_s: <p>I’m trying to understand what the differences are in using DataParallel vs increasing the num_workers in the DataLoader.</p>
<p>It seems that DataParallel divides the batch uniformly across the available GPUs, allowing the forward and backward passes to be done on each split up batch in parallel.</p>
<p>But what does increasing the num_workers in DataLoader do? That is, does each process generated to consume a new batch?</p>

gouthamvgk: <p>If num_worker is &gt; 0 then that much amount of separated processes will be spawned to do the data loading job. Each process will generate a single batch. This prevents bottleneck due to dataloading as multiple processes are working on it compared to num_workers=0 where after the forward pass gpu waits for the next batch of data to be loaded.</p>
Hi，could plz tell if i run these code in PC，when these code execute function 【forward】，actually i dont call function 【forward】】

111289: <p>import torch<br>
from torch import nn</p>
<p>class MLP(nn.Module):<br>
# 声明带有模型参数的层，这里声明了两个全连接层<br>
def <strong>init</strong>(self, **kwargs):<br>
# 调用MLP父类Module的构造函数来进行必要的初始化。这样在构造实例时还可以指定其他函数<br>
# 参数，如“模型参数的访问、初始化和共享”一节将介绍的模型参数params<br>
super(MLP, self).<strong>init</strong>(**kwargs)<br>
self.lzw_hidden = nn.Linear(784, 256) # 隐藏层<br>
self.lzw_act = nn.ReLU()<br>
self.output = nn.Linear(256, 10)  # 输出层</p>
<pre><code># 定义模型的前向计算，即如何根据输入x计算返回所需要的模型输出
def forward(self, x):
    a = self.lzw_act(self.lzw_hidden(x))
    return self.output(a)
</code></pre>
<p>print(“hello world”)</p>
<p>X = torch.rand(2, 784)<br>
net = MLP()<br>
print(net)<br>
<span class="hashtag">#print</span>(net(X))<br>
result = net(X)<br>
print(X)<br>
print(result)</p>

gouthamvgk: <p>Calling the MLP class which is a subclass of nn.Module will automatically call the forward function of it.(as nn.Module internally implements <strong>call</strong> in such a way)</p>
In place rotation and flipping

GObriel: <p>Hello there,</p>
<p>I’m looking to do some flipping and rotation for the Go board.<br>
I would like to now if there exist in place rotation operations and flip. Something like torch.rot90_(…) for instance.</p>
<p>I know this topic (<a href="https://discuss.pytorch.org/t/do-you-have-inplace-flip-operation/36482" class="inline-onebox">Do you have inplace flip operation?</a>) asks a similar question, but it has not been demonstrated.</p>
<p>Thanks !</p>
Indexing a indexed tensor

LeErnst: <p>Hello guys,</p>
<p>i am trying to access an indexed tensor in the following way:</p>
<pre><code class="lang-auto">
x = tr.tensor([True]*10, dtype=tr.bool)
y = tr.tensor([True]*10, dtype=tr.bool)

x[3]    = False
y[~x]   = False # works fine
x[y][3] = False # does not change the tensor x, but why?
</code></pre>
<p>Obviously the second indexing <code>x[y][3]</code> creates a copy of the tensor x[y], which is a little bit weird but ok. My question is how i can access the original tensor x?</p>

LeErnst: <p>The unsatisfactory answer:</p>
<pre><code class="lang-auto">x = tr.tensor([True]*10, dtype=tr.bool)
y = tr.tensor([True]*10, dtype=tr.bool)

x[3]    = False
y[~x]   = False # works fine
temp = x[y]
temp[3] = False
x[y] = temp # works fine
</code></pre>
<p>But c’mon is there any nice solution? Actually the tensors contain more than 1000 bools and i have to do these operations very often in a for loop so i do not want to copy them.</p>

ptrblck: <p>You could try to replace the sequential indexing with</p>
<pre><code class="lang-python">x[y.nonzero()[3]] = False
</code></pre>
<p>which should yield the desired result based on your example.</p>

LeErnst: <p>And again you nailed it. Thanks a lot.</p>
How could I save tensor result to a .csv file?

lk1983823: <p>I get results in the form of tensor from a model and I want to save the result in a .csv file. I use torch::save(input_tensors2.data(), “./tensor_test.csv”); but the .csv file can not be opened. Anyone can give some suggestions? Thanks.</p>

ptrblck: <p>You could get the numpy array, create a <code>pandas.DataFrame</code> and save it to a <code>csv</code> via:</p>
<pre><code class="lang-python">import torch
import pandas as pd
import numpy as np

x = torch.randn(1)
x_np = x.numpy()
x_df = pd.DataFrame(x_np)
x_df.to_csv('tmp.csv')
</code></pre>

Matej_Kompanek: <p>In C++, you will probably have to write your own, assuming your tensor contains results from N batches and you wanted to write it as one sample per row, something like this could work:</p>
<pre><code class="lang-auto">void SaveCsv(torch::Tensor t, const std::string&amp; filepath, const std::string&amp; separator = ",")
{
	t = t.flatten(1).contiguous().cpu();
	float* ptr = t.data_ptr&lt;float&gt;();

	std::ofstream csvHandle(filepath);

	for (size_t i = 0; i &lt; t.sizes()[0]; ++i)
	{
		for (size_t k = 0; k &lt; t.sizes()[1]; ++k)
		{
			csvHandle &lt;&lt; *ptr++;

			if (k &lt; (t.sizes()[1] - 1))
			{
				csvHandle &lt;&lt; separator;
			}
		}

		csvHandle &lt;&lt; "\n";
	}
}

void main()
{
	try
	{
		auto t = torch::randn({ 3, 4, 5 });

		SaveCsv(t, "D:\\test.csv");
	}
	catch (std::runtime_error&amp; e)
	{
		std::cout &lt;&lt; e.what() &lt;&lt; std::endl;
	}
	catch (const c10::Error&amp; e)
	{
		std::cout &lt;&lt; e.msg() &lt;&lt; std::endl;
	}

	system("PAUSE");
}
</code></pre>

lk1983823: <p><a class="mention" href="/u/matej_kompanek">@Matej_Kompanek</a>  Thanks very much!</p>
What is seq_len in documentation?

adm: <p>I don’t understand why the parameter <code>seq_len</code> is called this way. As far as I know this is the number of sequences. <code>seq_len</code> sounds like a sequence length, which is completely different thing.</p>
<p>Honestly the all terminology in documentation of RNN looks quite strange. Let’s take an example:</p>
<p>Description of the input to RNN (<a href="https://pytorch.org/docs/master/generated/torch.nn.RNN.html" class="inline-onebox" rel="noopener nofollow ugc">RNN — PyTorch master documentation</a>):</p>
<blockquote>
<p><strong>input</strong>  of shape (seq_len, batch, input_size): tensor containing the features of the input sequence.</p>
</blockquote>
<p>Do you see that? Tensor containing the FEATURES of the input sequence. So where do I need to provide the FEATURES? Because I only see following: <code>(seq_len, batch, input_size)</code> - nothing about features.</p>
<p>In my opinion it should be <code>(number_of_sequences, number_of_elements, number_of_features)</code>, where usually we have <code>number_of_sequences</code> sequences which are <code>number_of_elements</code> long each having <code>number_of_features</code> features  isn’t it?</p>
<p>Sorry I am very confused, but maybe I don’t understand something so I would appreciate if someone can clarify.</p>

vdw: <p><code>seq_len</code> is indeed the length of the sequence such as the number of words in a sentence or the number of characters in a string.</p>
<p><code>input_size</code> reflects the number of features. Again, in terms of sequences being words in a sentence, this would be the size of the word vectors (e.g, 300). Whatever the number of features is, that will be your <code>input_size</code>.</p>
<p>I full example: Let’s say your batch has 32 sentences of the same length of 50 words (e.g., with padding to ensure equal lengths), and you use word vectors (e.g. word2vec or GloVe) of size 300. then you get</p>
<ul>
<li><code>batch_size = 32</code></li>
<li><code>seq_len = 50</code></li>
<li><code>input_size = 300</code></li>
</ul>
<p>By default, RNNs in PyTorch expect as input shape <code>(seq_len, batch_size, input_size)</code>. But if you define an RNN with <code>batch_first=True</code>, the expected input shape is <code>(batch_size, seq_len, input_size)</code>. It’s really just a matter of preference and convenience.</p>

adm: <p>Is it the same case for output? If I define <code>batch_first=True</code>, do I get the batch first also in output?</p>

Sagnik_Mukherjee: <p>Nope. It works for the input in that way, but not for the output. From what I understand of this practice is – its just for convenience. Otherwise you would have to put unnecessary transpose statements and lengthen your code. The output fashion is dictated in the documentation as you can see in the link you provided.<br>
Hope that helps.</p>
OneCycle LR and Differential LR for groups

Jaideep_Valani: <p>lr=1e-4<br>
If we set up the Differential lr for layer groups  say<br>
<code>[{'params':group1,'lr': lr/10},{'param':group2,'lr':lr]</code></p>
<p>will this lr also varry as per Scheduler LR cycle ?</p>
Save the scheduler before or after the step?

maccn: <p>Hello everybody,<br>
I’m creating a script in order to save a model and, then, load the checkpoint in a second moment to resume the training.</p>
<p>It works very well, but I have a doubt on the point in which I have to save the scheduler. I write here the two codes and could you tell me which one is correct?</p>
<ol>
<li>Save the scheduler before the step function</li>
</ol>
<pre><code class="lang-auto">for e in range(start_epoch, epochs):
    train(...)
    val(...)
    save_checkpoint(...)
    scheduler.step()
</code></pre>
<ol start="2">
<li>Save the scheduler after the step function</li>
</ol>
<pre><code class="lang-auto">for e in range(start_epoch, epochs):
    train(...)
    val(...)
    scheduler.step()
    save_checkpoint(...)
</code></pre>
<p>With the save_checkpoint() function, I save the model, the number of the epoch, the optimizer and the scheduler.</p>
<p>Thank you very much for your help.</p>

ptrblck: <p>I think it might depend on how you are planning to restore the model and continue the training.<br>
If you restore the model, optimizer, and scheduler and rerun the same code, you would skip the <code>scheduler.step()</code> in the first example.<br>
If you look at the complete training, you would thus use the same learning rate in two epochs, which might not be the desired use case.</p>
How to load weights to model using .npy model?

iamharshit: <p>Hi everyone,</p>
<p>I am making a CNN and I need to load weights from a preexisting .npy file.<br>
Up until now I have made this function to do it.</p>
<pre><code class="lang-auto">def load_weights():
    params = net.state_dict()
    pathtoweights = os.path.join(os.path.abspath("."), "saved_models", "numpy", "weights.npy")
    pretrained_weights = np.load(pathtoweights allow_pickle=True).item()
    with torch.no_grad(): 
        for key in params:
            if "weight" == key[-6:]:
                npkey = bytes(key[:-7], "utf-8")
                if npkey in pretrained_weights.keys():
                    print("Weight found for " + npkey + "layer")
                    params[key].copy_(torch.from_numpy(pretrained_weights[npkey]["weights"]).type(Tensor))
            elif "bias" == key[-5:]:
                npkey = bytes(key[:-5], "utf-8")
                if npkey in pretrained_weights.keys():
                    print("Bias found for " + npkey + "layer")
                    params[key].copy_(torch.from_numpy(pretrained_weights[npkey]["biases"]).type(Tensor))
</code></pre>
<p>However, when I run this function, the values inside params does get changed but it remains same when I print net.state_dict(). Why is it happening, tell me if I am doing anything wrong or see if you can find a fix. Greatly appreciate your help.</p>

ptrblck: <p>Your approach should work, as seen in this example:</p>
<pre><code class="lang-python">lin = nn.Linear(1, 1)
sd = lin.state_dict()
print(sd)
&gt; OrderedDict([('weight', tensor([[-0.5414]])), ('bias', tensor([0.8279]))])

sd['weight'].copy_(torch.tensor([[1.]]))
print(lin.state_dict())
&gt; OrderedDict([('weight', tensor([[1.]])), ('bias', tensor([0.8279]))])

print(lin.weight)
&gt; Parameter containing:
tensor([[1.]], requires_grad=True)
</code></pre>

iamharshit: <p>Thanks, I just checked again and it was working this time as you said. I don’t know what happened last time, but it was not showing up. Maybe I made some error while printing state_dict after updating it. Anyways thank you for the response.</p>
Torch.onnx export model IndexError: vector::_M_range_check: __n (which is 18446744073709551615)

1LOVESTaylor: <p>I’m trying to convert my pytorch model into onnx, my code is as below:</p>
<p>dummy_input = Variable(torch.randn(1, 3, 256, 256))<br>
torch.onnx.export(model_G, dummy_input, “test.onnx”)</p>
<p>And terminal says:</p>
<p>Traceback (most recent call last):<br>
File “Pkl2Onnx.py”, line 49, in <br>
torch.onnx.export(model_G, dummy_input, “test.onnx”)<br>
File “/home/liuchang/.local/lib/python2.7/site-packages/torch/onnx/<strong>init</strong>.py”, line 25, in export<br>
return utils.export(*args, **kwargs)<br>
File “/home/liuchang/.local/lib/python2.7/site-packages/torch/onnx/utils.py”, line 84, in export<br>
_export(model, args, f, export_params, verbose, training, input_names, output_names)<br>
File “/home/liuchang/.local/lib/python2.7/site-packages/torch/onnx/utils.py”, line 153, in _export<br>
proto, export_map = trace.export(list(_unique_state_dict(model).values()),_onnx_opset_version, defer_weight_export)<br>
IndexError: vector::_M_range_check: __n (which is 18446744073709551615) &gt;= this-&gt;size() (which is 173)</p>
<p>Has anyone met this problem before?</p>

1LOVESTaylor: <p>Waiting for solutions</p>

lysuhin: <p>Were you able to resolve this? I suppose there’s a need to fix something in network architecture, but I’m not sure exactly what should it be.</p>
<p>I think you should post your <code>model_G</code> code here.</p>

billaram: <p>I am also facing a similar issue during tensorrt inference.</p>
<pre><code class="lang-auto">I0723 05:07:35.746679 1 libtorch_backend.cc:805] vector::_M_range_check: __n (which is 18446744073709551615) &gt;= this-&gt;size() (which is 3)
</code></pre>
<p>Any help is appreciated.</p>

ptrblck: <p>Could you post an executable code snippet so that we could debug it?</p>
How to get gradients of loss with individual sample in min-batch

LeeDoYup: <p>Hello.</p>
<p>I want to get gradients of each sample’s loss w.r.t. each parameters.<br>
Specifically, when</p>
<ul>
<li>the number of total data is N</li>
<li>mini-batch size: B</li>
<li>parameter (filter) shape: <code>[C_in, C_out, w, h]</code>
</li>
</ul>
<p>I want to get the gradients for each sample, that is</p>
<ul>
<li>gradients shape: <code>[N, C_in, C_out, w, h]</code>.</li>
</ul>
<p>To get the gradients for all samples,<br>
I tried to get the gradients for each mini-batch.</p>
<ul>
<li>graidnets shape: <code>[B, C_in, C_out, w, h]</code>
</li>
</ul>
<p>However, after i use</p>
<pre><code class="lang-python">loss = torch.sum(-onehot*pred, dim=1)
# onehot: [B, # of class]
# pred: [B, # of class]
loss.backward(gradient=torch.ones_like(loss))
</code></pre>
<p>I found that the gradients of loss w.r.t. parameters</p>
<pre><code class="lang-auto">print(conv1.weight.grad.shape)
</code></pre>
<p>have the shape of <code>[C_in, C_out, w, h]</code> not <code>[B, C_in, C_out, w, h]</code>.</p>
<p>That is, the gradients of loss for each sample are accumulated.<br>
How can i get the gradient for each sample?</p>
<p>Is the only way to solve using for-loop?</p>
<p>Thank you…</p>
<p>Doyup Lee</p>

ptrblck: <p><a href="https://discuss.pytorch.org/t/per-sample-gradient-should-we-design-each-layer-differently/57293/2">This topic</a> deals with the same question and suggests to use <a class="mention" href="/u/yaroslav_bulatov">@Yaroslav_Bulatov</a>’s repository. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>
Trouble with Layer Architecture

lbrickweg: <p>Hi all,</p>
<p>I am a noob with pytorch, but I have been making neural network models in JMP for a while now.</p>
<p>I am trying to match the structure of JMP models in pytorch which have 10 input parameters, a single<br>
tanh layer with 10 nodes, feeding a single continuous prediction value.</p>
<p>The final equation ends up being:</p>
<p>predicted_value = b + weight_n1 * [ tanh(0.5*(weight_p1<em>parameter1 + weight_p2</em>parameter2…)]<br>
+ weight_n2 * [ tanh(0.5*(weight_p11* parameter1 + weight_p12*parameter2…)]<br>
+ …</p>
<p>…and so on for 10 total nodes. I have tried to use:</p>
<pre><code>def __init__(self, in_features, out_features):
    super().__init__()
    self.in_features = in_features
    self.out_features = out_features
    self.input_layer = torch.nn.Linear(self.in_features,10)
    self.hidden_layer = torch.nn.Linear(10, self.out_features)

def forward(self, x):
    out = self.input_layer(x)
    out = self.hidden_layer(torch.tanh(0.5*out))    
    
    return out
</code></pre>
<p>How do I set up the forward function for this? Any time I add any nonlinear activation function I get a single predicted value.</p>
<p>Thanks in advance!</p>

ptrblck: <p>The nonlinearity should not change the number of outputs. In your current code snippet the number of output values would be determined by <code>self.out_features</code>.<br>
Let me know, if I misunderstood the issue.</p>
DataLoaders folder Structure

Ratan: <p>How should my folder structure be for multilabel multiclass output using <code>torchvision.datasets.ImageFolder</code> ?</p>

ptrblck: <p><code>ImageFolder</code> cannot be used for a multi-label classification, since each subfolder will get a single class index. For this use case you would have to implement a custom <code>Dataset</code> and create or read the targets in it.</p>

Ratan: <p>Yes just as i thought. Thank You.<br>
I have one  ore issue. How to skip if an image is not available and fetch another image<br>
Here is my code<br>
<code>def __getitem__(self, index):</code></p>
<pre><code>    imagePath = self.listImagePaths[index]
    
    imageData = Image.open(imagePath).convert('RGB')
    imageLabel= torch.FloatTensor(self.listImageLabels[index])
    
    if self.transform != None: imageData = self.transform(imageData)
    
    return imageData, imageLabel`
</code></pre>
<p>some image names in csv are not available which rises error.</p>

ptrblck: <p>The “clean” solution would be to iterate the dataset once before the training, store all invalid indices in a <code>list</code> or any other container and remove these invalid samples afterwards.<br>
During training you could sample another index randomly inside <code>__getitem__</code>, but this would yield these randomly samples multiple times during training, which might not be desirable.</p>
Override forward method in GeneralizedRCNN

kl_divergence: <p>I want to override forward method of GeneralizedRCNN. I want to return losses and features from ROI head during training. I have tried doing:</p>
<pre><code class="lang-auto">model = torchvision.models.detection.fasterrcnn_resnet50_fpn(True).cuda()
model.forward = forward_modified(self,images, targets=None)
</code></pre>
<p>It doesn’t work quite well (throws an error from transform). Can someone share a working snippet of efficiently going about it ?</p>

kl_divergence: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Can you suggest something ?</p>

goldfish: <p>Please refer to this:</p>
<pre><code class="lang-auto">   def forward(self, images, targets=None):
        """
        Arguments:
            images (list[Tensor]): images to be processed
            targets (list[Dict[Tensor]]): ground-truth boxes present in the image (optional)

        Returns:
            result (list[BoxList] or dict[Tensor]): the output from the model.
                During training, it returns a dict[Tensor] which contains the losses.
                During testing, it returns list[BoxList] contains additional fields
                like `scores`, `labels` and `mask` (for Mask R-CNN models).

        """
        if self.training and targets is None:
            raise ValueError("In training mode, targets should be passed")
        original_image_sizes = [img.shape[-2:] for img in images]
        images, targets = self.transform(images, targets)
        features = self.backbone(images.tensors)
        if isinstance(features, torch.Tensor):
            features = OrderedDict([(0, features)])
        proposals, proposal_losses = self.rpn(images, features, targets)
        detections, detector_losses = self.roi_heads(features, proposals, images.image_sizes, targets)
        detections = self.transform.postprocess(detections, images.image_sizes, original_image_sizes)

        losses = {}
        losses.update(detector_losses)
        losses.update(proposal_losses)

        if self.training:
            return losses

        return detections
</code></pre>
Convolutions 6 CNN

jasg: <p>If I have a 57 pixel image and I want to do 6 convolutions with max pool of 2, stride of 1, padding of 1, kernel of 3, it is possible to do this</p>

ptrblck: <p>Have you tried it out or where are you stuck? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

jasg: <p>when I do 6 convolutions with the operation: (((w-f + 2p) / s) +1) / max pool; for an image of 57 I have 0 of value</p>

ptrblck: <p>In that case it won’t be working and you would either have to increase the spatial size or reduce the pooling.</p>
Linking issues with torch

yetanadur: <p>I am building a custom C++ shared library that uses pytorch C++ libraries. But I am getting one undefined symbol - “typeinfo for torch::jit::GraphAttr”. I have included torch and c10 into my target_link_libraries. Am I missing additional pytorch libraries to link ?</p>

yetanadur: <p>resolved the issue. It seems when I build pytorch code with<br>
<code>python setup.py install </code> the shared object executables are installed without execution permission on files - as<br>
<code>-rw-r--r-- </code> . After manually correcting file permissions with<br>
<code>chmod +x &lt;file.so&gt;</code> , the link issues are resolved.</p>
Obtaining the forward pass computation graph structure

Hanson_Lu: <p>Is there any way that we can <strong>automatically</strong> extract a computation graph that represents the <strong>forward pass</strong> in a series of pytorch computations? We would like the nodes in the graph to have access to the functions that make up the forward pass, so that we can call them individually. We would also like to have access to the values of intermediate outputs of each function.</p>
<p>This would be something different from the computation graph constructed by <code>autograd</code>, which generally only represents the <em>backward</em> pass. Its nodes only contain a reference to a <code>grad_fn</code>, which is the derivative of the forward function.</p>
<p>For instance, if we provide the following code:</p>
<pre><code class="lang-auto">import torch

x = torch.randn(10)
M = torch.randn(10, 10)
h = torch.matmul(M, x)
h = torch.add(h, x)
h = torch.tanh(h)
y = torch.sum(h)
</code></pre>
<p>We would like to automatically extract a graph structure like</p>
<pre><code class="lang-auto">     y
     |
   [sum]
     |
   [tanh]
     |
   [add] 
  /     \
  | [matmul]     
  | /      \
  x         M
</code></pre>
<p>Where each bracketed node could be an object (call <code>Node</code>, for instance) that stores a reference to its corresponding function as an attribute, and stores the output of the function during the forward pass.</p>
<p>It would be great if anyone could point to any pytorch APIs that enables some kind of functionality like this!</p>
Bayesian Convolutionnal network with Dropout

MERAH_Samia: <p>I’m trying to implement Bayesian Convolutionnal network with Dropout. I have defined two classes: the first class  <code>Net_do(nn.Module) (mcdo=Net_do())</code>  for training data and another class  <code>Net(nn.Module) (net=Net())</code>  for test data. During the training phase, I did  <code>Outputs=mcdo(inputs)</code>  and during the testing phase, I did  <code>Outputs=Net(inputs)</code>  I searched for help, but I can´t find a solution to my problem. I will leave the error here, I’m stuck I’m really gonna appreciate your help, Thank you<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/2/9225a9a72445e59de387fe89eb83246c92383fd1.png" data-download-href="https://discuss.pytorch.org/uploads/default/9225a9a72445e59de387fe89eb83246c92383fd1" title="sa"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/2/9225a9a72445e59de387fe89eb83246c92383fd1.png" alt="sa" data-base62-sha1="kQSoXPIsaxYKpZNUqxirA8sQg3T" width="690" height="349" data-dominant-color="F5F6F6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">sa</span><span class="informations">831×421 23.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ayalaa2: <p>So when we perform a <code>model(...)</code> call, it actually ends up calling <code>model.forward(...)</code>. What you’re seeing here is that it’s calling that method (line 550) but apparently your <code>Net_do</code> model does <em>not</em> have such a method. Therefore your model instead inherits the forward method from its parent (nn.Module) which raises the <code>NotImplementedError</code>.</p>
<p>Does your <code>Net_do</code> have a forward method? If so, can you please post the class code for <code>Net_do</code>?</p>

MERAH_Samia: <p>Thanks for reply,<br>
yes it has a forward method<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/c/dc36217b2f6355e68aabdef2a5d96ec375fd0ef8.png" alt="1" data-base62-sha1="vq517R4hWT3YKuYhTZ2FONgZLlK" width="458" height="446"> <img src="https://discuss.pytorch.org/uploads/default/original/3X/b/3/b3248ae58e3e8624e4745e8f3eabcb4546d7aa39.png" alt="2" data-base62-sha1="pyLL4JkVig5ZIoNhgAiAjYBWq0h" width="526" height="437"></p>

ayalaa2: <p>Is it Net_MCDO? The forward method is indented one too many times</p>

MERAH_Samia: <p>Oh yes ! I just noticed<br>
Thank’s a lot</p>
How to make NLLLoss possible in multitarget loss?

11179: <p>Hello,</p>
<p>I want to make NLLLoss in pytorch to successfully treat multilabel target (e.g [0 0 1 0 1 1] ).</p>
<p>However I’ve got error that this loss is not for multitarget,<br>
so I saw documents and find it considers input shape as (N,C) and target (N,).</p>
<p>I want to see how pytorch calculate NLLLoss since it expects values from log_softmax not softmax,</p>
<p>so thinking there may be difference with function I could build like below.</p>
<p>import torch<br>
def NLLLoss(logs, targets):<br>
ㅡout = torch.zeros_like(targets, dtype=torch.float)<br>
ㅡfor i in range(len(targets)):<br>
ㅡㅡout[i] = logs[i][targets[i]]<br>
ㅡreturn -out.sum()/len(out)</p>
<p>But it seems it is made from <em>C</em> file, I don’t know C language sadly.</p>
<p>Is someone knows how NLLLoss is built in Pytorch (please change my code if possible…) or how to make nn.NLLLoss to successfully calculate for multitargetloss?</p>
<p>(I don’t want to use multimarginloss)</p>

KFrank: <p>Hello Changrok!</p>
<aside class="quote no-group" data-username="11179" data-post="1" data-topic="90432" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/11179/48/23294_2.png" class="avatar"> 11179:</div>
<blockquote>
<p>I want to make NLLLoss in pytorch to successfully treat multilabel target (e.g [0 0 1 0 1 1] ).</p>
</blockquote>
</aside>
<p>I’m not sure what you mean by “multitarget.”</p>
<p>If you are working on a multi-label, multi-class classification problem,<br>
then you don’t want to be using <code>NLLLoss</code> (or <code>CrossEntropyLoss</code>).<br>
You should use <code>BCEWithLogitsLoss</code>.</p>
<p>The key idea is that a multi-label, multi-class problem (say, <code>nClass</code><br>
classes) can be understood as <code>nClass</code> <em>binary</em> classification problems.<br>
That is, each sample has <code>nClass</code> binary predictions / labels:  Class-1<br>
is present or absent; class-2 is (at the same time as class-1) present<br>
or absent, and so on.</p>
<p>If this doesn’t answer your question, please give a brief description of<br>
the conceptual content of your task, and tell use the shapes of the<br>
output of your model and your target (including the <code>nBatch</code> dimension).</p>
<p>Good luck.</p>
<p>K. Frank</p>
Pytorch NCCL Point to Point

Yi_Zhang: <p>Hi,</p>
<p>According to NCCL documentation, since NCCL 2.7 Point-to-point communication can be achieved using ncclSend and ncclRecv. However in Pytorch, the newest stable version still doesn’t support send and receive when using NCCL as backend. I’m wondering is there anyway to achieve point to point communication between GPUs in Pytorch? And is there any way to integrate ncclSend and ncclRecv in Pytorch distributed?</p>
<p>Thanks.</p>

mrshenli: <p>Hey <a class="mention" href="/u/yi_zhang">@Yi_Zhang</a>, we are working on adding P2P to NCCL ProcessGroup backend. We just bumped up the NCCL submodule version in <a href="https://github.com/pytorch/pytorch/pull/41608" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/41608</a>.</p>
<p>For now, to work around it, you can create a sub group of 2 ranks, and  then use <code>dist.broadcast(tensor, src, group=sub_group)</code> to mimic P2P send/recv. PipeDream is already using that.</p>
<p>If you need general P2P support, you could try the <a href="https://pytorch.org/docs/master/rpc.html" rel="nofollow noopener">RPC API</a>. A caveat is that we are still working on improving support for GPU tensors. <a href="https://github.com/pytorch/pytorch/issues/41369" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/41369</a></p>

Yi_Zhang: <p>Thank you for the reply.</p>
Batch Norm and weight initialization

Hmrishav_Bandyopadhy: <p>Hi,</p>
<p>From what I have observed  particularly from <a href="https://course.fast.ai/videos/?lesson=8" rel="nofollow noopener">here</a> and <a href="https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79" rel="nofollow noopener">here</a> is that weight initialization is something that can help us prevent vanishing and exploding gradients in very deep neural nets by properly setting the initial values of the weights. Initializing weights properly ensures that the output of a layer has a mean of zero and a standard deviation of one.</p>
<p>So my question here is , this can be actually very easily attained by batch normalization, as batch normalization normalizes along the batches–to bring about a mean of zero and a standard deviation of one. So to be sure, layer weight initialization is not particularly needed when we use batch norm?</p>
Memory mapped GPU tensor

Markus-Goetz: <p>Hi all,</p>
<p>I would like to have a memory-mapped tensor on the GPU, exceeding the GPU’s memory, backed by a file on disk. I was partially successful in doing that on CPU, like that</p>
<pre><code class="lang-auto">import torch
s = torch.DoubleStorage("a_path", shared=True, size=1000000000000)
t = torch.DoubleTensor(s)
t += 1
</code></pre>
<p>The actual computation would work fine and the data successfully synced to the file, however, at the end of the operation the program would die with a bus error (might as well be a problem with the system, not sure).</p>
<p>Doing the same for a CUDA tensor would die with</p>
<p><code>RuntimeError: not available yet for THCStorage at /pytorch/aten/src/THC/generic/THCStorage.cpp:84</code></p>
<p>My understanding of the C++ API of PyTorch is very limited (yet?). Hence, I would want to avoid digging deep in the C++ implementation. Is there a way that I could provide a, potentially slow, object to a torch.cuda.Tensor where I could implement the memory swapping myself. If so, how? Would it need to implement the buffer interface, the iterator protocol, …?</p>
<p>Cheers and thanks,<br>
Markus</p>
PyTorch equivalent to numpy.divide(where) function

alpha_omega_alpha: <p>Hi all,</p>
<p>I know that PyTorch has a <code>torch.div()</code> function, but it doesn’t have the <code>where</code> parameter that <code>numpy.divide()</code> (<a href="https://numpy.org/doc/stable/reference/generated/numpy.divide.html" rel="nofollow noopener">reference</a>) has.</p>
<p>Does anyone know if PyTorch has a function that can do this?<br>
Or maybe a way of doing this in PyTorch?</p>
<p>I appreciate your time and support.</p>

tom: <p>I don’t know of anything. You need to use torch.where, probably.</p>
Indexing 3D tensor using 2 1D ones

Raialbe: <p>Hi Everyone, i’m facing a problem in indexing a tensor an i hope someone her could help.</p>
<p>i have a 3D tensor T with dimensions [batch_size,h,w] and two other tensors H and W both with dimension [batch_size]</p>
<p>What i need to do is gather (or create a mask) that takes for every element in the position b of the minibatch the value at the position [b,h[b],w[b]]</p>
<p>so for example for the element t[0] i would like to return the value t[0,h[0],w[0]]</p>
<p>right now I’m trying to use something like this:</p>
<pre><code class="lang-auto">        h = h.long()
        w = w.long()

        mask = torch.zeros_like(T).bool()
 
        mask[:,h, w] = 1
        print(torch.sum(mask))  # should be equal to batch_size
        value = torch.masked_select(T, mask)
</code></pre>
<p>but it is not working.</p>
<p>EDIT:<br>
I found the solution to my problem, posting it if anyone needs it!</p>
<pre><code class="lang-auto">t[torch.arange(t.shape[0]),h,w)]
</code></pre>
<p>Thanks anyway, happy coding!</p>
Is it possible to avoid releasing memory after tensor is destroyed?

be7f984b2f2e66ba7969: <p>Hi,</p>
<p>I want to integrate pytorch pybind11 bindings into my project.</p>
<p>My project would initialize a fixed size cuda memory as the network output. And I would pass this to torch::tensor by from_blob.</p>
<p>After that, I can manipulate the torch tensor in python.</p>
<p>However, wheneven the torch tensor is destroyed (not referenced anymore) in python,  it would release the memory I want to keep.</p>
<p>Is is possible to avoid release memory when torch tensor is destroyed?</p>

albanD: <p>Hi,</p>
<p>When the Tensor is created with <code>from_blob</code>, it does not actually release the memory when it is destroyed (it does not know how it was allocated).<br>
It is your responsibility to make sure the blob stays valid at least as long as the Tensor uses it.</p>

be7f984b2f2e66ba7969: <p>it sounds good.</p>
<p>I just found another great solution.</p>
<p>we can define an object having <strong>cuda_array_interface</strong> and pass through torch.as_tensor, and we can make sure both object share the same memory location.</p>
Can Pytorch run on the development board which running Linux kernel?

xifanlover: <p>hi all：<br>
Can Pytorch be cross compiled to the development board which running Linux kernel?It is particularly important to note that：it is ARM+LinuxKernel,not ARM+ubantu.</p>

albanD: <p>Hi,</p>
<p>Can you get python built for your board?<br>
Do you have a full libc?</p>
AttributeError: module 'torch.cuda' has no attribute 'amp'

braindotai: <p>I’m running <code>from torch.cuda.amp import GradScaler, autocast</code> and got the error as in title.</p>
<p>Pytorch version: - 1.4.0</p>
<p>Thanks.</p>

boto: <p>Hi!</p>
<p>Have you installed the CUDA version of pytorch? Does your environment recognize <code>torch.cuda</code>?</p>

ptrblck: <p><code>torch.cuda.amp</code> is available in the nightly binaries, so you would have to update.</p>

JosueCom: <p>I just got the following error when attempting to use amp. Do you know how I can fix it?</p>
<pre><code class="lang-python">     84         if amp_enable:
---&gt; 85             with th.cuda.amp.autocast():
     86                 out1 = model(sub, inp)
     87                 out2 = temp_ly(sub, out1)

AttributeError: module 'torch.cuda.amp' has no attribute 'autocast'
</code></pre>

ptrblck: <p>You might need to install the nightly binary, since Autocasting wasn’t shipped in <code>1.5</code>.</p>
Batched boolean indexing

Tin: <p>Hello, I have the following problem that would be nice if it were possible to solve it efficiently with some vectorized operations.<br>
In my dataset, I have a discrete set of items, each tagged with a subset of 20 possible categories.<br>
During training, I output a distribution over the set of items in my dataset which I sample from.</p>
<p>When training with batches, I am given the tensor <code>items_per_category</code> of shape <code>[num_categories, num_items]</code> (the tensor is only square due to padding - each category entry has a variable number of items that are tagged with this category).<br>
I also have a batch of tensors <code>binary_categories</code> of shape <code>[batch_size, num_categories]</code>.<br>
Let’s say we have a batch element <code>indexx = [1, 0, 1]</code>, then I would like to perform the following operation: <code>categories = items_per_category[index]</code>, which would retrieve rows 0 and 2 from <code>items_per_category</code>.<br>
For clarity, the following works as intended:</p>
<pre><code class="lang-auto">&gt;&gt;&gt; items_per_category = torch.tensor([[1, 2, 3], [1, 4, 7], [2, 3, 5]], dtype=torch.int32)
&gt;&gt;&gt; index = torch.tensor([1, 0, 1]).type(torch.BoolTensor)
&gt;&gt;&gt; items_per_category[x, :]
tensor([[1, 2, 3],
        [2, 3, 5]], dtype=torch.int32)
</code></pre>
<p>However, if I have a batch of indices, it does not.</p>
<pre><code class="lang-auto">&gt;&gt;&gt; batch_index = torch.tensor([[1, 0, 0],
        [1, 0, 1],
        [0, 0, 1]])
&gt;&gt;&gt; items_per_category[batch_index.type(torch.BoolTensor), :]
Traceback (most recent call last):
  File "&lt;stdin&gt;", line 1, in &lt;module&gt;
IndexError: too many indices for tensor of dimension 2
</code></pre>
<p>Now, I would like to perform a batched indexing. I am wondering if it is possible to obtain, for each batch element, a flat tensor with unique discrete elements, and overall obtain a square tensor (perhaps padded). Otherwise, since each batch element has a variable number of 1s, not sure what shape I can expect the output of this batched indexing to be.<br>
Thanks!</p>
<p>EDIT: as an alternative, would I be better off converting the binary indices to the actual integer positions? E.g. <code> x =&gt; [0, 2]</code>? Unsure about the approach I’m taking</p>

ptrblck: <p>If you are using a batched index tensor, you would have to either add a batch dimension to the items tensor:</p>
<pre><code class="lang-python">items_per_category = items_per_category.unsqueeze(0).repeat(3, 1, 1)
items_per_category[batch_index.type(torch.BoolTensor)]
</code></pre>
<p>or you could index both dimensions by removing the <code>:</code> from the indexing operation.</p>
<p>Let me know, if I misunderstood your use case.</p>
Calculating Accuracy in a Single machine, multi-GPU setup

ayrts: <p>Hello,</p>
<p>I read Thomas Woolf’s <a href="https://medium.com/huggingface/training-larger-batches-practical-tips-on-1-gpu-multi-gpu-distributed-setups-ec88c3e51255" rel="nofollow noopener">article on balanced loads when using multiple GPUs</a>, and I would like to adapt it for my training. He mentioned that when using <code>DataParallelModel</code> and compared to <code> torch.nn.DataParallel</code>, the predictions in the forward pass (<code>predictions = parallel_model(inputs)</code>) would be a tuple of <em>n</em> tensors, with each tensor being located on a specific GPU (There are <em>n</em> GPUs used for training).</p>
<p><a href="https://gist.github.com/thomwolf/7e2407fbd5945f07821adae3d9fd1312" rel="nofollow noopener">Here</a> is the code for his implementation. To recap, I simply wrap the model like this:</p>
<pre><code class="lang-auto">parallel_model = DataParallelModel(model) 
predictions =parallel_model(inputs)
</code></pre>
<p>This will affect how I currently compute the accuracy because I use torch.max to get the prediction from the tensors like this:</p>
<pre><code class="lang-auto">_, pred = torch.max(predictions.data, dim=1)
correct += (pred == label).sum().item()
total += label.size(0)
std_acc= (correct/total) * 100
</code></pre>
<p>I am thinking this can be solved by iterating through the tuple <code>predictions</code> like this:</p>
<pre><code class="lang-auto">for i in range(len(gpu_list)):
   _, pred = torch.max(predictions[i].data, dim=1)
  correct += (pred == label).sum().item()
  total += label.size(0)
std_acc= (correct/total) * 100
</code></pre>
<p>However, each tensor is located on a different GPU, so iterating through it doesn’t seem to make sense. How can I access the <code>pred</code> for each tensor in <code>predictions</code> given that they are in different GPUs, and how can I compare them to their original labels?</p>
<p>Or is there a better way to calculate the accuracy? Thank you.</p>

ptrblck: <p>You could use the loop and push all predictions to the default device (or the device, where <code>label</code> is stored).<br>
Note that, the usage of <code>.data</code> is deprecated and might yield unwanted side effects, so you should call <code>torch.max</code> on the tensor directly.</p>