akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
How to check if NCCL is installed correctly and can be used by PyTorch?

laoreja: <p>How to check if NCCL is installed correctly and can be used by PyTorch? I can import torch.cuda.nccl, but I’m not sure how to test if it’s installed correctly.</p>

XinDongol: <p>You can try</p>
<pre><code class="lang-auto">torch.cuda.nccl.version()
</code></pre>

Brando_Miranda: <p>is there a way to check it with a command line without pytorch?</p>
PyTorch version compatibility with GPU devices

fran_bl: <p>Hello, I am having issues with compatibility between PyTorch versions / GPU devices / operating systems. Specifically, I am training and saving a neural network on a GPU device and then loading it to a different device (different GPU) with a different PyTorch version - this results in the neural network not being loaded properly. In addition, I am also training the neural network on Linux, while I load it on Windows (not sure if this can also cause problems).</p>
<p><strong>Could anyone please direct me to any documentation online mentioning which GPU devices are compatible with which PyTorch versions / operating systems?</strong></p>
<p>Below are the detailed information on the GPU device names and PyTorch versions I used, which I know for sure that definitely are not compatible.<br>
Training: GPU device name:  GeForce GTX 1080 Ti; PyTorch version 1.5.1; Linux.<br>
Loading: GPU device name: GeForce RTX 2070 with Max-Q Design; PyTorch version: 1.3.0; Windows.</p>
<p>Thanks a lot in advance!</p>

ptrblck: <p>Could you explain what “not being loaded properly” means?<br>
Are you checking the parameters before saving and after loading and see differences?<br>
Also, note that forward-compatibility is not supported. I.e. saving a model in a newer PyTorch release and loading in an older might fail (the other way should work), so could you update both PyTorch installation to the latest release?</p>
Worker shutdown function

Latope2-150: <p>Hi,</p>
<p>In order to load data, I need to use licensed software that I need to allocate in each process (=worker) init, and that I need to free on process shutdown.<br>
For the init, this is easy using the <code>worker_init_fn</code> argument of the <code>DataLoader</code> class, but for shutdown, I was not able to find a working solution. I tried implementing it in the <code>__del__ </code> method of my dataset, but this does not appear to work. Any suggestion?</p>
<p>This would be much easier if I could use multithreading instead of multiprocessing for the DataLoader workers but this is not available yet (this appears to be in the works <a href="https://github.com/pytorch/pytorch/issues/41292" class="inline-onebox" rel="noopener nofollow ugc">[RFC, Tracker] DataLoader improvements · Issue #41292 · pytorch/pytorch · GitHub</a>).</p>
Get a thread safe copy of torch::nn::Sequential object

afshin67: <p>I need to get a copy of <code>shared</code> neural network of type <code>torch::nn::Sequential</code> in a multi-threaded code. In each thread, I want to change the weights of the neural net and I need to make sure that the weights do not mix up with each other. It seems that I cannot directly use <code>auto l_model = model;</code> since it only provides a new pointer which points to the same address in the memory that the original object lives in. So, whenever I change the weights in one thread, all other threads see that change, which is not desirable. I also declared and instantiated the new model as <code>auto l_model = torch::nn::Sequential(model);</code> which still results in the same issue.</p>
<p>I appreciate any help or comment.</p>
<pre><code>#include &lt;omp.h&gt;
#include "iostream"
#include &lt;ATen/ATen.h&gt;
#include &lt;torch/torch.h&gt;

void set_weights(torch::nn::Sequential &amp;model, float w){

    torch::autograd::GradMode::set_enabled(false);
    for (auto &amp;p : model-&gt;named_parameters()) {
        std::string y = p.key();
        auto z = p.value(); // note that z is a Tensor, same as &amp;p : layers-&gt;parameters

        if (y.compare(2, 6, "weight") == 0)
            z.uniform_(w, w);
        else if (y.compare(2, 4, "bias") == 0)
            z.uniform_(0, 0);
    }
    torch::autograd::GradMode::set_enabled(true);

}

void print_weights(torch::nn::Sequential &amp;model) {
    //Returns an iterator over module parameters, yielding both the name of the parameter as well as the parameter itself.
    auto params = model-&gt;named_parameters(true /*recurse*/);
    for (const auto &amp;pair : params) {
        std::cout &lt;&lt; pair.key() &lt;&lt; ": " &lt;&lt; pair.value() &lt;&lt; std::endl;
    }
}

int main()
{
    int N=3;
    torch::Tensor x = torch::tensor({1,1,1,1}).toType(torch::kFloat32);
    int thread;

    torch::nn::Sequential model(torch::nn::Linear(4,2),
                                torch::nn::Functional(torch::relu),
                                torch::nn::Linear(2,1));
    torch::Device device_ = torch::Device(torch::kCPU);
    model-&gt;to(device_);
    model-&gt;to(torch::kFloat32);
    set_weights(model, 1.);
    omp_set_num_threads(2);
#pragma omp parallel for private(thread) shared(N) //, private(x)
    for (int i=0; i&lt;N; i++) {
        thread = omp_get_thread_num();
        auto l_model = model;

        std::cout &lt;&lt; "g-model" &lt;&lt;std::endl;
        print_weights(model);
        set_weights(l_model, (float)thread);
        std::cout &lt;&lt; "l-model" &lt;&lt;std::endl;
        print_weights(l_model);
        std::cout &lt;&lt; "g-model" &lt;&lt;std::endl;
        print_weights(model);
        auto out = l_model-&gt;forward(x);
        auto tmp = out.item&lt;float&gt;();
        result[i][0] = (float)thread;
        result[i][1] = *out_f;
    }
    for (int i=0; i &lt;N ; i++){
        std::cout &lt;&lt; "i=" &lt;&lt; i &lt;&lt; " thread=" &lt;&lt; result[i][0] &lt;&lt; " out=" &lt;&lt; result[i][1] &lt;&lt; std::endl;
    }

    return 0;
}</code></pre>
Does random seed in pytorch has the effect on the function of drop out?

Yongjie_Shi: <p>I know the random seed affect the initialization of the net.<br>
However, I wonder if the random seed has the effect on the randomness of drop out layer?</p>

ptrblck: <p>Yes, dropout layers will also be affected by the random seed, as they are sampling from the PRNG:</p>
<pre><code class="lang-python">x = torch.randn(1, 10)

torch.manual_seed(2809)
d11 = F.dropout(x, p=0.5, training=True)
d12 = F.dropout(x, p=0.5, training=True)

torch.manual_seed(2809)
d21 = F.dropout(x, p=0.5, training=True)
d22 = F.dropout(x, p=0.5, training=True)

print((d11==d21).all())
print((d12==d22).all())
</code></pre>

Yongjie_Shi: <p>Yeah, I have verified it~Thank you very much!</p>

AsgharR: <p>Does this mean that the exact same nodes will be dropped out in every epoch during model training?</p>

ptrblck: <p>Yes, if you re-seed the code, the same random numbers will be drawn and thus your training would lose the “randomness”, which is why you would usually only set the seed at the beginning of the training, if you want reproducible results (besides the other setup mentioned in the <a href="https://pytorch.org/docs/stable/notes/randomness.html">Reproducibility docs</a>).</p>
Scaling linalg ops

rfeinman: <p>While working on linear inverse and matrix determinant problems, I’ve found that some of pytorch’s matrix factorization algorithms scale well to “batch” mode (i.e. batches of matrices vs. single matrices) and others do not. It would be useful to have this documented somewhere for users, or at least some general information on what to expect.</p>
<p>My findings so far:</p>
<ol>
<li>Cholesky factorization scales well to batch (although only applicable for sym PD matrices)</li>
<li>LU factorization scales well to batch (e.g. <code>torch.lu</code>, <code>torch.solve</code>, <code>torch.linalg.solve</code>)</li>
<li>Symmetric eigendecomposition does not scale to batch (neither <code>torch.symeig(*, eigenvectors=True)</code> nor <code>torch.linalg.eigh</code>)</li>
<li>QR factorization does not scale to batch (neither <code>torch.qr</code> nor <code>torch.linalg.qr</code>)</li>
<li>
<code>torch.lstsq</code> has no batch implementation (I believe this uses QR under the hood?)</li>
</ol>
<p>Other thoughts:</p>
<p>A) LDL factorization<br>
I can’t find an implementation of LDL factorization or LDL-based solvers. This is usually the 2nd-best choice after Cholesky for sym matrices (i.e. when they are not PD). It would be great to see an implementation in the future.</p>
<p>B) Least-squares and least-norm<br>
Currently <code>torch.lstsq</code> has no batch implementation. Given that Cholesky scales so well, it would make sense to offer a Cholesky-based method that solves A^T A x = A^T b (with diagonal Tikhonov regularization for the least-norm case of m &lt; n). This is a classical approach in the literature.</p>
<p>Update:<br>
In general what I mean by “scale well” is that the algorithm scales in constant time, as though all batch entries were run in parallel, vs. linear time, as though they were run sequentially.</p>
Style Transfer using closed form matting

Hugh_T: <p>I’m a newbie to computer vision. I’m trying to perform style transfer of images using Levin’s closed-form solution python implementation from this repository <a href="https://github.com/MarcoForte/closed-form-matting" class="inline-onebox" rel="noopener nofollow ugc">GitHub - MarcoForte/closed-form-matting: Python implementation of A. Levin D. Lischinski and Y. Weiss. A Closed Form Solution to Natural Image Matting. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2006, New York</a>. I understand the step I have to perform using the prior, and prior with confidence but I don’t know how to do it. Whenever I try I get the foreground image exactly same as the input image. Can anyone help me in understanding what the prior and prior with confidence is and how to perform style transfer using it?</p>
Manually change parameter

Joe_Harrison: <p>How do I change/update a registered parameter? I want to manually change the value of a tensor that is registered as a parameter.</p>
<pre><code class="lang-auto">        std::cout &lt;&lt; selected_node-&gt;parameters() &lt;&lt; std::endl;
        selected_node-&gt;value = selected_node-&gt;value + torch::randn({1});

        selected_node-&gt;parameters()[0] = selected_node-&gt;value;
        std::cout &lt;&lt; selected_node-&gt;parameters() &lt;&lt; std::endl;
</code></pre>
<p>This prints the same parameters twice.</p>
<pre><code class="lang-auto">        selected_node-&gt;register_parameter("value", selected_node-&gt;value);
        selected_node-&gt;value = selected_node-&gt;value + torch::randn({1});
        selected_node-&gt;register_parameter("value", selected_node-&gt;value);
</code></pre>
<p>“reregistering” is not allowed.</p>
<p>How do I update/change/alter a parameter that is already registered?</p>

Joe_Harrison: <p>I found it eventually. For anyone with the same question: you can do it with data()</p>
<pre><code class="lang-auto">selected_node-&gt;parameters()[0].data() = selected_node-&gt;parameters()[0].data() + torch::randn({1});
</code></pre>
How torch calculates the grads for the scalar and non-scalar tensors?

AbishekBashyal: <p>I was reading the <strong>Optional Reading: Tensor Gradients and Jacobian Products</strong> section of <a href="https://pytorch.org/tutorials/beginner/basics/autogradqs_tutorial.html#optional-reading-tensor-gradients-and-jacobian-products" rel="noopener nofollow ugc">this blog</a> and it stated:</p>
<blockquote>
<p>In many cases, we have a scalar loss function, and we need to compute the gradient with respect to some parameters. However, there are cases when the output function is an arbitrary tensor. In this case, PyTorch allows you to compute the so-called <strong>Jacobian product</strong> and not the actual gradient.</p>
</blockquote>
<p>So, does it means that <strong>Jacobian product</strong> is calculated only for the arbitrary tensor i.e. non-scalar tensor &amp; scalar tensors gradients are calculated in <strong>different way</strong>?</p>

albanD: <p>Hi,</p>
<p>The idea is that it always does a vector Jacobian product. It just happens that when the output is scalar, it is 1D and so the vector is of size 1 and can be replaced with just the value 1. that will give you the full jacobian (and thus gradients).</p>

AbishekBashyal: <p><a class="mention" href="/u/alband">@albanD</a> thanks it makes sense now</p>
Priority queue in pytorch

nionjo: <p>is there any implementation of priority queues available in pytorch?</p>

Ajinkya_Ambatwar: <p>This can possibly work</p>
<pre><code class="lang-auto">class priorityQ_torch(object):
    """Priority Q implelmentation in PyTorch

    Args:
        object ([torch.Tensor]): [The Queue to work on]
    """

    def __init__(self, val):
        self.q = torch.tensor([[val, 0]])
        # self.top = self.q[0]
        # self.isEmpty = self.q.shape[0] == 0

    def push(self, x):
        """Pushes x to q based on weightvalue in x. Maintains ascending order

        Args:
            q ([torch.Tensor]): [The tensor queue arranged in ascending order of weight value]
            x ([torch.Tensor]): [[index, weight] tensor to be inserted]

        Returns:
            [torch.Tensor]: [The queue tensor after correct insertion]
        """
        if type(x) == np.ndarray:
            x = torch.tensor(x)
        if self.isEmpty():
            self.q = x
            self.q = torch.unsqueeze(self.q, dim=0)
            return
        idx = torch.searchsorted(self.q.T[1], x[1])
        print(idx)
        self.q = torch.vstack([self.q[0:idx], x, self.q[idx:]]).contiguous()

    def top(self):
        """Returns the top element from the queue

        Returns:
            [torch.Tensor]: [top element]
        """
        return self.q[0]

    def pop(self):
        """pops(without return) the highest priority element with the minimum weight

        Args:
            q ([torch.Tensor]): [The tensor queue arranged in ascending order of weight value]

        Returns:
            [torch.Tensor]: [highest priority element]
        """
        if self.isEmpty():
            print("Can Not Pop")
        self.q = self.q[1:]

    def isEmpty(self):
        """Checks is the priority queue is empty

        Args:
            q ([torch.Tensor]): [The tensor queue arranged in ascending order of weight value]

        Returns:
            [Bool] : [Returns True is empty]
        """
        return self.q.shape[0] == 0
</code></pre>
RuntimeError: result type Float can't be cast to the desired output type Long when using log_prob

Niels_PyTorch: <p>Suppose that you have token logits from a Transformer model. The logits are of shape <code>(batch_size, sequence_length)</code>. In the example below, <code>(4,512)</code>.</p>
<p>I would like to turn every token logit into an independent Bernouilli distribution (as I’m implementing a research paper). Next, the loss is defined as the log probability of the ground-truth labels.</p>
<pre><code class="lang-auto">import torch

logits = torch.randn((4,512))

dist_per_token = torch.distributions.Bernoulli(logits=logits) # every token is now a Bernouilli distribution 

labels = torch.randint(0, 1, (4,512))
loss_per_token = dist_per_token.log_prob(labels)
</code></pre>
<p>However, when my labels is a LongTensor, i get the following error:</p>
<pre><code class="lang-auto">---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
&lt;ipython-input-5-196b06f84a85&gt; in &lt;module&gt;()
      1 labels = torch.randint(0, 1, (4,512))
      2 
----&gt; 3 dist_per_token.log_prob(labels)

1 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   2824         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   2825 
-&gt; 2826     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
   2827 
   2828 

RuntimeError: result type Float can't be cast to the desired output type Long
</code></pre>
<p>Shouldn’t this work, as the Bernouilli is a discrete distribution? Or should one always provide FloatTensors to <code>log_prob</code>? It only works when I type <code>dist_per_token.log_prob(labels.float())</code>.</p>
How to do .forward on part of my model?

111491: <p>The background of this question is as same as <a href="https://discuss.pytorch.org/t/how-could-you-manually-stop-backpropagation/115985">this one</a>. I realized that to train that middle layer parameter I can save the input of that layer and do forward from here, so that the training processs could be finished even faster. But I am not sure about how to implement this in pytorch. Do I have to rewrite the .forward function of the entire model， or there is a simpler way to achieve that?</p>

ptrblck: <p>I think creating a new custom model with a new <code>forward</code> method would be the cleanest approach.<br>
Alternatively, you could also use the modules by directly accessing them via:</p>
<pre><code class="lang-python">middle_activation = ...
out = model.middle_layer1(middle_activation)
out = model.middle_layer2(out)
...
</code></pre>
<p>but this would basically be equivalent to writing a new custom <code>forward</code>.</p>

111491: <p>Thanks！I’m going to try it out.</p>
AttributeError: 'dict' object has no attribute 'size' in BCEWithLogitsLoss

Sofiane_Meraouna: <p>I’m kind of new to Pytorch so sorry if the question makes no sense.<br>
I’m trying to build a simple Convnet for object detection on the PascalVOC dataset.</p>
<p>I run into the following error :</p>
<p>Traceback (most recent call last):<br>
loss = criterion(labels, outputs)<br>
File “C:\Users\Sofiane\anaconda3\lib\site-packages\torch\nn\modules\module.py”, line 727, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “C:\Users\Sofiane\anaconda3\lib\site-packages\torch\nn\modules\loss.py”, line 629, in forward<br>
return F.binary_cross_entropy_with_logits(input, target,<br>
File “C:\Users\Sofiane\anaconda3\lib\site-packages\torch\nn\functional.py”, line 2579, in binary_cross_entropy_with_logits<br>
if not (target.size() == input.size()):<br>
AttributeError: ‘dict’ object has no attribute ‘size’</p>
<p>When I try to run this snippet :</p>
<pre><code class="lang-auto">criterion = torch.nn.BCEWithLogitsLoss()
</code></pre>
<p>and later call :</p>
<pre><code class="lang-auto">    for i, data in enumerate(train_loader):

        images, labels = data

        images = images.to(device)

        outputs = model(images)
        loss = criterion(outputs, labels)
</code></pre>
<p>I tried printing out <code>labels</code> and I get the following</p>
<p>{‘annotation’: {‘filename’: [‘2009_000133.jpg’], ‘folder’: [‘VOC2012’], ‘object’: [{‘name’: [‘aeroplane’], ‘bndbox’: {‘xmax’: [‘383’], ‘xmin’: [‘111’], ‘ymax’: [‘234’], ‘ymin’: [‘144’]}, ‘difficult’: [‘0’], ‘occluded’: [‘0’], ‘pose’: [‘Unspecified’], ‘truncated’: [‘0’]}], ‘segmented’: [‘1’], ‘size’: {‘depth’: [‘3’], ‘height’: [‘273’], ‘width’: [‘500’]}, ‘source’: {‘annotation’: [‘PASCAL VOC2009’], ‘database’: [‘The VOC2009 Database’], ‘image’: [‘flickr’]}}}</p>
<p>So I’m wondering what I’m doing wrong.</p>

ptrblck: <p>As the error message and the print suggests, <code>labels</code> is a <code>dict</code>, while a tensor is expected.<br>
I’m not familiar with your use case, but you would have to extract the valid target value out of the current <code>dict</code>, which contains the filename, folder, and more information.</p>
TypeError: relu_(): argument 'input' (position 1) must be Tensor, not int

bc060400164: <p>Dear ,</p>
<p>Creating a NN with relu is giving an error. but its working fine on linear one</p>
<p>class LSTM(nn.Module):<br>
def <strong>init</strong>(self, input_size=1, hidden_layer_size=20, output_size=1):<br>
super().<strong>init</strong>()</p>
<pre><code>self.hidden_layer_size = hidden_layer_size
self.lstm = nn.LSTM(input_size, hidden_layer_size)
self.relu = nn.functional.relu(hidden_layer_size, output_size),   
                    torch.zeros(1, 1, self.hidden_layer_size))
</code></pre>
<p>def forward(self, input_seq):<br>
lstm_out, self.hidden_cell = self.lstm(input_seq.view(len(input_seq), 1, -1), self.hidden_cell)<br>
predictions = self.linear(lstm_out.view(len(input_seq), -1))<br>
return predictions[-1]</p>
<p>model = LSTM()<br>
loss_function = nn.MSELoss()<br>
optimizer = torch.optim.Adam(model.parameters(), lr = 0.0001)</p>
<p>after this getting an error:</p>
<p>TypeError                                 Traceback (most recent call last)<br>
 in <br>
----&gt; 1 model = LSTM()<br>
2 loss_function = nn.MSELoss()<br>
3 optimizer = torch.optim.Adam(model.parameters(), lr = 0.0001)</p>
<p> in <strong>init</strong>(self, input_size, hidden_layer_size, output_size)<br>
5     self.hidden_layer_size = hidden_layer_size<br>
6     self.lstm = nn.LSTM(input_size, hidden_layer_size)<br>
----&gt; 7     self.relu = nn.functional.relu(hidden_layer_size, output_size)     <span class="hashtag">#activation</span> unit # linear # try with relu ###<br>
8<br>
9     self.hidden_cell = (torch.zeros(1, 1, self.hidden_layer_size),    <span class="hashtag">#Basic</span> units of LSTM networks are LSTM layers that have multiple LSTM cells.Cells do have internal cell state, often abbreviated as “c”, and cells output is what is called a “hidden state”, abbreviated as “h”.Regular RNNs do have just the hidden state and no cell state. It turns out that RNNs have difficulty of accessing information from a long time ago.</p>
<p>/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py in relu(input, inplace)<br>
1059             return handle_torch_function(relu, (input,), input, inplace=inplace)<br>
1060     if inplace:<br>
 → 1061         result = torch.relu_(input)<br>
1062     else:<br>
1063         result = torch.relu(input)</p>
<p>TypeError: relu_(): argument ‘input’ (position 1) must be Tensor, not int</p>

ptrblck: <p>The issue seems to be related to <a href="https://discuss.pytorch.org/t/activation-function-typeerror-tensor-object-is-not-callable/116035">your other post</a>.<br>
If you want to create an <code>nn.ReLU</code> module, use:</p>
<pre><code class="lang-python">self.relu = nn.ReLU()
</code></pre>
<p>Currently you are trying to feed integers to the functional API, which expects a tensor and the optional <code>inplace</code> argument.</p>
Why .cuda() or .cpu() behaves differently for torch.tensor and torch.nn.Module instance

Qianfeng: <p>I found .cuda() and .cpu() works differently for torch.tensor and nn.Module instance.It makes me really confused…</p>
<p>for a tensor:</p>
<pre><code class="lang-auto">a_cuda = torch.tensor([1.0,2.0],device='cuda:0')
a_cpu = a_cuda.cpu()
print('----cuda before-------')
print(a_cuda)

print('----cpu before-------')
print(a_cpu)

print('\n')
a_cuda += 100

print('----cuda after-------')
print(a_cuda)

print('----cpu after-------')
print(a_cpu)
</code></pre>
<p>the output is</p>
<pre><code class="lang-auto">----cuda before-------
tensor([1., 2.], device='cuda:0')
----cpu before-------
tensor([1., 2.])


----cuda after-------
tensor([101., 102.], device='cuda:0')
----cpu after-------
tensor([1., 2.])
</code></pre>
<p>I think it’s fine. but for a network:</p>
<pre><code class="lang-auto">net_cuda = Model().cuda()
net_cpu = net_cuda.cpu()
print('----cuda before-------')
for p in net_cuda.parameters():
    print(p)

print('----cpu before-------')
for p in net_cpu.parameters():
    print(p)

print('\n\n')
for p in net_cuda.parameters():
    with torch.no_grad():
        p += 100

print('----cuda after-------')
for p in net_cuda.parameters():
    print(p)

print('----cpu after-------')
for p in net_cpu.parameters():
    print(p)
</code></pre>
<p>the output is:</p>
<pre><code class="lang-auto">----cuda before-------
Parameter containing:
tensor([[ 0.0328, -0.0506]], requires_grad=True)
----cpu before-------
Parameter containing:
tensor([[ 0.0328, -0.0506]], requires_grad=True)



----cuda after-------
Parameter containing:
tensor([[100.0328,  99.9494]], requires_grad=True)
----cpu after-------
Parameter containing:
tensor([[100.0328,  99.9494]], requires_grad=True)
</code></pre>
<p>if  .cpu() makes a copy of the original data, why it changes after I modify the original cuda data?</p>

Qianfeng: <p>and if I execute <code>net_cpu = net_cuda.cpu()</code> ,  the <code>net_cuda</code>'s parameter will be on cpu…</p>

ptrblck: <p>The <code>cuda()</code>, <code>cpu()</code>, and <code>to()</code> operations work recursively (inplace) in <code>nn.Module</code>s, so</p>
<pre><code class="lang-python">net_cpu = net_cuda.cpu()
</code></pre>
<p>will move <code>net_cuda</code> to the CPU as well and create a reference to it as <code>net_cpu</code>.<br>
If you want to keep a model on the GPU and a clone on the CPU, you could <code>copy.deepcopy</code> it.</p>
How to reset GRU?

SungmanHong: <p>I’m using video MOS predicting AI with GRU.</p>
<p>GRU will feed all of the frame’s data and return one single MOS value.</p>
<p>I’ll load pre-trained AI model and predict MOS for one video file.</p>
<p>Also, I want to predict another MOS value from another video file.</p>
<p>In this case, loading AI model to GPU takes about 20~30secs in my old laptop.</p>
<p>So I don’t want to reload the AI model and just want to reset the hidden state of the GRU.</p>
<p>my question is,</p>
<ol>
<li>
<p>Should I reset hidden state of GRU to predict MOS value for another video file?<br>
because hidden state of GRU may affect different behavior?</p>
</li>
<li>
<p>If yes, how to reset the hidden state of the GRU, like end of the train-step?</p>
</li>
</ol>
<p>Thanks in advance.</p>

ptrblck: <p>The <code>nn.GRU</code> module expects the initial hidden state as the second input argument and will default it to zeros, if none is provided.<br>
In case you are not manually passing the hidden state to it, you are already using the zeroed out hidden state.</p>

SungmanHong: <p>Thank you, I recently discovered that my question has a little unfamiliar word.<br>
I’ll add some comment, what I want to mean ‘MOS’ is ‘video quality score’.</p>
<p>Also, After I read your comment, I realized that when the GRU’s second input argument is not defined, GRU is automatically reset as pre-trained state when I loaded first.<br>
So I don’t need to reset GRU when input is changed, because the second input argument is not defined.</p>
<p>Thanks a lot.</p>
Does adaptive_avg_pool2d and mean(multidim) dispatch to the same kernel?

vadimkantorov: <p>Do <code>y</code> and <code>z</code> computation dispatch to the same kernel? Are they the same for efficiency concerns? (also asked in <a href="https://discuss.pytorch.org/t/difference-between-mean-method-and-avgpooling/15954" class="inline-onebox">Difference between mean() method and AvgPooling</a>)</p>
<pre><code class="lang-python">x = torch.rand(16, 512, 64, 64)

y = F.adaptive_avg_pool2d(x, (1, 1)) # should I refactor my code to do this instead of the following line? (it's maybe less flexible)

z = x.movedim(-3, -1).mean(dim = [-3, -2], keepdim = True).movedim(-1, -3) # some convoluted reshapes and keepdim mean
</code></pre>

ptrblck: <p>Both approaches would calculate the <code>mean</code> only as seen <a href="https://github.com/pytorch/pytorch/blob/b7b481bd07b3a90f06f90596accd63818bb22429/aten/src/ATen/native/AdaptiveAveragePooling.cpp#L102-L112">here</a>. The dispatch would be a bit different in case you are checking the stack trace.</p>
How to keep samples in each mini batch unshuffled?

NAsrin791: <p>Hi,<br>
I need to have sequential data samples in each mini batch (frames of a video). Using  torch.utils.data.DataLoader, and  shuffle =true, it shuffles data indices within each mini batch, and shuffle=false return the mini batches in order.<br>
How can I have random mini batches containing unshuffled data samples?</p>
<p>I used new_dataloader = random.sample(list(dataloader) , len(dataloader)), but this occupies a huge memory. Is there any other mechanism in Pytorch?</p>
<p>I hope you can help me:)</p>

ptrblck: <p>You could either load the sequence in the <code>__getitem__</code> method while the <code>DataLoader</code> and sampler are only providing the “start index” (you would need to adapt the <code>__len__</code> of your <code>Dataset</code> for this approach) or you could create a custom sampler, which will yield the desired “shuffled” indices through a <code>BatchSampler</code>.<br>
<a href="https://discuss.pytorch.org/t/sampler-to-sample-contiguous-data/20302/2">Here</a> is a simple example of the first use case.</p>
Changine image mode produce wrong results for resnet 50 model

ravishk1: <p>While quantizing the resnet50-v1 model with image mode 0to1, for the example cat image, it predicts the class 285. For the image mode = neg1to1, it predicts class id: 281, but for image mode = 0to255, it predicts 416 which is wrong.<br>
The command I am using for quantization is,<br>
<em>./image-classifier cat_285.png -image-mode=neg1to1 -model=resnet50-v1-7.onnx -model-input-name=data -dump-profile=profile.yml</em></p>
<p>Why just changing the image mode produces the wrong results?</p>

jfix: <p>Image mode changes how we normalize the input image data. If the model is trained with some specific normalization then we should try to use that same normalization to match to achieve the best accuracy.</p>
Get the gradients from a hook

Intel_Novel: <p>I hoped to read module gradients from <code>register_backward_hook</code></p>
<pre><code> module.weight.grad
</code></pre>
<p>But it is either None or all zeros. What hook should I use for the gradients?<br>
(I can get gradients w/o problem from the training loop).</p>
<p>This is my model and I tested on MNIST.</p>
<pre><code class="lang-auto">class M2(nn.Module):
    def __init__(self):
        super().__init__()        
        self.l1 = nn.Linear(28*28, 10)
        
    def forward(self, x):
        x = x.reshape(-1, 28*28)        
        x = F.relu(self.l1(x))        
        return x
</code></pre>
<p>Here is the full notebook:</p>
<aside class="onebox githubgist">
  <header class="source">
      <a href="https://gist.github.com/dejanbatanjac/86d02413118cd7fb10b479c1d4c8f318" target="_blank" rel="nofollow noopener">gist.github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://gist.github.com/dejanbatanjac/86d02413118cd7fb10b479c1d4c8f318" target="_blank" rel="nofollow noopener">https://gist.github.com/dejanbatanjac/86d02413118cd7fb10b479c1d4c8f318</a></h4>
<h5>Grad catch.ipynb</h5>
<pre><code class="Jupyter Notebook">{
  "cells": [
    {
      "metadata": {
        "trusted": true
      },
      "cell_type": "code",
      "source": "%matplotlib inline\n\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import DataLoader, Dataset, TensorDataset\nfrom torch.optim import *\nimport torchvision\n\n",
      "execution_count": 1,
      "outputs": []</code></pre>
This file has been truncated. <a href="https://gist.github.com/dejanbatanjac/86d02413118cd7fb10b479c1d4c8f318" target="_blank" rel="nofollow noopener">show original</a>

<p>
</p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


saeed_soori: <p>I have the same issue. Could you find a solution?<br>
I want to modify the gradient (module.weight.grad) based on the g_inp and g_out of a module.</p>
Problem using "deepcopy(model)"

WindVChen: <aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/54580" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/pytorch/issues/54580" target="_blank" rel="noopener nofollow ugc">Problem about using "deepcopy(model)"</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2021-03-24" data-time="08:57:40" data-timezone="UTC">08:57AM - 24 Mar 21 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/WindVChen" target="_blank" rel="noopener nofollow ugc">
          <img alt="WindVChen" src="https://avatars.githubusercontent.com/u/56162412?v=4" class="onebox-avatar-inline" width="20" height="20">
          WindVChen
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">I find a weird phenomenon when debugging programs these days !
I want to use "deepcopy(model)", but when there is any assignment...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">needs reproduction</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">triaged</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
Pytorch The confusion of L-BFGS

Sjk_Jae: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/8/081963a70b86bcf9963a12b68715f14cc77d9555.png" data-download-href="https://discuss.pytorch.org/uploads/default/081963a70b86bcf9963a12b68715f14cc77d9555" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/081963a70b86bcf9963a12b68715f14cc77d9555_2_597x499.png" alt="image" data-base62-sha1="19EdonirTMmfcVWkEqaXYmzsuAl" width="597" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/081963a70b86bcf9963a12b68715f14cc77d9555_2_597x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/8/081963a70b86bcf9963a12b68715f14cc77d9555_2_895x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/0/8/081963a70b86bcf9963a12b68715f14cc77d9555.png 2x" data-dominant-color="EBE4E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1106×926 82.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I don’t quite understand the meaning of the parameter max_iter of L-BFGS. Is a step mentioned in the document an epoch? The mini-batch method is not used in the code. Why does L-BFGS iterate many times in an epoch?</p>
Can you recommend me some papers on sequence prediction?

Sjk_Jae: <p>I have always imitated other people’s code to do some nlp tasks, but it is difficult to improve my ability. I want to read some papers to improve my ability, but there are too many ACL papers, and I don’t know which papers to read. For example, for sequence prediction tasks, can you recommend some papers?</p>

AbdulsalamBande: <p>Try attention is all you need paper. <a href="https://arxiv.org/abs/1706.03762" rel="noopener nofollow ugc">Attention Is All You Need</a><br>
If you find it hard to implement, you can check  <a href="https://www.youtube.com/watch?v=U0s0f995w14&amp;ab_channel=AladdinPersson" rel="noopener nofollow ugc">Video on Attention is all you need</a>. It explains the code in detail.</p>

Sjk_Jae: <p>thank you for your recommended paper ！</p>
How could you manually stop backpropagation?

111491: <p>For some reason I need to call loss.backward() to tarin a special parameter in the middle layer of my model, but for the sake of saving calculation expenses I don’t want the backpropagation to continue once I get the gradient for my parameter, otherwise I’ll run out of my kaggle-GPU free usage <img src="https://discuss.pytorch.org/images/emoji/apple/disappointed_relieved.png?v=9" title=":disappointed_relieved:" class="emoji" alt=":disappointed_relieved:">. So is there any way to <strong>manually stop backpropagation</strong> after It was done for a specific parameter or a module?</p>

albanD: <p>Hi,</p>
<p>You can use <code>autograd.grad(loss, input)</code> and it will return gradients for that input and only perform the necessary computations to get that value.<br>
If you prefer for the <code>.grad</code> fields to be populated, you can use <code>loss.backward(inputs=input)</code>.</p>

111491: <aside class="quote no-group" data-username="albanD" data-post="2" data-topic="115985">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/48/215_2.png" class="avatar"> albanD:</div>
<blockquote>
<p><code>autograd.grad(loss, input)</code></p>
</blockquote>
</aside>
<p>Helped me out！Thanks alt</p>
Compute Gaussian Density Map on a 2D Point Set

dominjune: <p>Hi all, I have a question about how to efficiently compute a Gaussian density image on a given 2D point set. My current implementation is as the following with a plot:</p>
<pre><code class="lang-auto">import torch
import numpy as np
from time import time
import matplotlib.pyplot as plt
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

def computeGaussian(p, res=128, kernel_sigma=0.05):
    ksize = round(res * kernel_sigma * 6)
    sigma = kernel_sigma * res
    x = np.linspace(0, 1, int(res))
    y = np.linspace(0, 1, int(res))
    xv, yv = np.meshgrid(x, y)
    txv = torch.from_numpy(xv).unsqueeze(0).float()
    tyv = torch.from_numpy(yv).unsqueeze(0).float()
    mesh = torch.cat((txv, tyv), 0).to(device)   # positions
    img = torch.zeros((3, res, res)).to(device)  # create an empty gaussian density image

    for i in range(p.size()[0]):
        center = p[i, 0:2]  # go through each point
        hw = round(3 * kernel_sigma * res)  # only consider band-limited gaussian with 3sigma
        coord_center = torch.floor(center * res)

        up = torch.max(coord_center[1] - hw, torch.Tensor([0]).to(device))  # take the boundary into account
        down = torch.min(coord_center[1] + hw + 1, torch.Tensor([res]).to(device))
        left = torch.max(coord_center[0] - hw, torch.Tensor([0]).to(device))
        right = torch.min(coord_center[0] + hw + 1, torch.Tensor([res]).to(device))
        up = up.long()
        down = down.long()
        left = left.long()
        right = right.long()

        r, g, b = 1, 1, 1

        # apply gaussian kernel on the pixels based on their distance to the center
        img[0, up:down, left:right] += r * torch.exp(
            -(mesh.permute(1, 2, 0)[up:down, left:right, :] - center).pow(2).sum(2) / (2 * kernel_sigma ** 2))
        img[1, up:down, left:right] += g * torch.exp(
            -(mesh.permute(1, 2, 0)[up:down, left:right, :] - center).pow(2).sum(2) / (2 * kernel_sigma ** 2))
        img[2, up:down, left:right] += b * torch.exp(
            -(mesh.permute(1, 2, 0)[up:down, left:right, :] - center).pow(2).sum(2) / (2 * kernel_sigma ** 2))

    return img


pts = torch.rand(16, 2).to(device)
start = time()
density_map = computeGaussian(pts)
end = time()
print('Time: {:.4f}'.format(end - start))
plt.figure(1)
plt.subplot(121)
plt.imshow(density_map.permute(1, 2, 0).detach().cpu().numpy())
plt.subplot(122)
plt.scatter(pts[:, 0].detach().cpu().numpy(), (1 - pts[:, 1]).detach().cpu().numpy())
plt.axis('equal')
plt.xlim([0, 1])
plt.ylim([0, 1])
plt.show()

</code></pre>
<p>The problem is that I need to loop over all the points since they have different size of grid(i.e. [down:up, left:right]) to consider, because of the boundary. This makes it slow for a large point set and it’s even faster to run on CPU than CUDA. Is there any way to make it without the loop? Thanks in advance.</p>
No module named torch

costa98: <p>I am trying to run a python file that uses ‘import torch’ but I get the error</p>
<pre><code class="lang-auto">ModuleNotFoundError: No module named 'torch'

</code></pre>
<p>I tried uninstalling it and reinstalling it with pip and conda but nothing works, it is correctly installed but I still get the error</p>

ptrblck: <p>It seems that PyTorch wasn’t correctly installed in your current environment.<br>
You can check it via <code>conda env list | grep torch</code> or the <code>pip</code> equivalent.</p>

Jay1: <p>Hello,<br>
Can you help me? I tried almost all that I could find on the web but still get an error message.</p>
<p>Basically, I created a new conda environment and installed pytorch in a computer cluster.<br>
It seems correctly installed but I still get the error message "ModuleNotFoundError: No module named ‘torch.’</p>
<p>(env_pytorch) [asidasi@hpcc ~]$<br>
(env_pytorch) [asidasi@hpcc ~]$ python -c “import torch”<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
ModuleNotFoundError: No module named ‘torch’<br>
(env_pytorch) [asidasi@hpcc ~]$<br>
(env_pytorch) [asidasi@hpcc ~]$<br>
(env_pytorch) [asidasi@hpcc ~]$ conda list | grep torch</p>
<h4>packages in environment at /home/asidasi/.conda/envs/env_pytorch:</h4>
<p>ffmpeg                    4.3                  hf484d3e_0    pytorch<br>
pytorch                   1.8.0           py3.6_cuda10.2_cudnn7.6.5_0    pytorch<br>
torchaudio                0.8.0                      py36    pytorch<br>
torchvision               0.9.0                py36_cu102    pytorch<br>
(env_pytorch) [asidasi@hpcc ~]$</p>

AlphaBetaGamma96: <p>Have you activated the conda environment? You might be running Python which defaults to pip?</p>

Jay1: <p>Thanks for a quick reply!<br>
Yes, I already activated my conda environment (env_pytorch), as you can see in my post.<br>
I also double-checked which python I am using. I am using the correct python in my env.</p>

AlphaBetaGamma96: <p>can you check by typing “which python” in the terminal? That’ll tell what python executable it’s using. Also, if you’re submitting a job via a job scheduler it might be changing the environment post-launch. For example with PBS you need to state the environment at the beginning of your .py script.</p>

Jay1: <p>I tried that, too. See below.</p>
<p>(env_pytorch) [asidasi@hpcc ~]$ which python<br>
~/.conda/envs/env_pytorch/bin/python<br>
(env_pytorch) [asidasi@hpcc ~]$ ~/.conda/envs/env_pytorch/bin/python -c “import torch”<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
ModuleNotFoundError: No module named ‘torch’<br>
(env_pytorch) [asidasi@hpcc ~]$</p>
Can a DLTensor be used to initialize a tensor in C++ Frontend

arif_saeed: <p>I am starting to experiment with Pytorch C++ frontend in my C++ application.</p>
<p>My data is held in data arrays in my application and can easily be stored as DLTensors.</p><aside class="onebox githubfolder">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="16" height="16">
      <a href="https://github.com/dmlc/dlpack" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars.githubusercontent.com/u/11508361?s=400&amp;amp;v=4" class="thumbnail onebox-full-image" width="60" height="60">

<h3><a href="https://github.com/dmlc/dlpack" target="_blank" rel="noopener nofollow ugc">dmlc/dlpack</a></h3>


  <p><span class="label1">RFC for common in-memory tensor structure and operator interface for deep learning system</span></p>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>Is there a no copy way of initializing Pytorch Tensors using DLTensors or data arrays.</p>
<p>The only method I can find is torch::from_blob and for this, I need to clone the data to use it on GPU’s<br>
Ideally, I’m looking for a no copy way to initialize tensors, like move semantics.</p>
Question in pyytorch result

randinoo: <p>how i get the classification result as percentage accuracy% and also as an image with CNN?</p>
<p>help me plzz</p>

Dwight_Foster: <p>To get the accuracy you can first initialize a total and correct amount above your dataloader loop but below your epoch loop like this:</p>
<pre><code class="lang-auto">for e in epochs:
     total = 0.
     correct = 0.
     for data in dataloader:
</code></pre>
<p>then in the dataloader loop you can do this</p>
<pre><code class="lang-auto">    output = model(input)
    pred = output.data.max(1, keepdim=True)[1] 
    compare predictions to true label
    correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
    total += images.size(0)
</code></pre>
<p>and just print it out when you print out your loss by something like this</p>
<pre><code class="lang-auto">print('\nTest Accuracy: %2d%% (%2d/%2d)' % (
        100. * correct / total, correct, total))  
</code></pre>
<p>and I am not sure what you mean by the image with CNN. This is also assuming you are using softmax.</p>

randinoo: <p><a class="mention" href="/u/dwight_foster">@Dwight_Foster</a> hi, I want the classification result with CNN as an image (the result), is not fair I display the value of the accuracy</p>
Please hiw can i solve this problem

muhammad_umar: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/3/e396e289b04406628afdad0e76b0755103fcd7e6.png" data-download-href="https://discuss.pytorch.org/uploads/default/e396e289b04406628afdad0e76b0755103fcd7e6" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e396e289b04406628afdad0e76b0755103fcd7e6_2_690x387.png" alt="image" data-base62-sha1="wtlECHcSg0GsjA2bRfgSaLvfwlU" width="690" height="387" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e396e289b04406628afdad0e76b0755103fcd7e6_2_690x387.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/3/e396e289b04406628afdad0e76b0755103fcd7e6_2_1035x580.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/3/e396e289b04406628afdad0e76b0755103fcd7e6.png 2x" data-dominant-color="EDEEEE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1366×768 79 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

AlphaBetaGamma96: <p>It seems that model.parameters() is empty. Could you print what <code>model</code> and <code>model.parameters()</code> are? (make sure to include the () after model.parameters)</p>
N networks in Parallel each with its own loss

deepfailure: <p>Hello,</p>
<p>I have N networks with the same input size (but with different values) and the same output size.</p>
<p>I wanted to use the approach explained in [<a href="https://stackoverflow.com/questions/58374980/run-multiple-models-of-an-ensemble-in-parallel-with-pytorch" class="inline-onebox" rel="noopener nofollow ugc">python - Run multiple models of an ensemble in parallel with PyTorch - Stack Overflow</a> for parallelization] of N-heads neural network. If I understand correctly, using the layers:</p>
<pre><code class="lang-auto">nn.Conv1d(in_channels=dim_state * nb_heads, out_channels=hidden_size * nb_heads, kernel_size=1, groups=nb_heads)
</code></pre>
<p>I parallelize the execution of N linear layers. My code is slightly different as the state does not have the same value but it has the same shape.</p>
<p>However, I have also an N-dimensional loss vector instead of a single number for all layers. I do not want that the loss of network 0 affect the weights of other networks. Is there any way to backpropagate the loss for each network separately?</p>
<p>Thanks in advance.</p>