akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Worst accuracy in test phase

username: Muhammad_Izaz
cooked: <p>In train phase the model is showing 0.8 accuracy but in test phase the accuracy is reduced to 0.2<br>
Is there any thing wrong in my code.<br>
test_iterator = torch.utils.data.TensorDataset(x_test, y_test.reshape(-1).long())<br>
test_data = torch.utils.data.DataLoader(test_iterator, batch_size = 64, shuffle = True)</p>
<h1>Test the model</h1>
<p>model.eval()<br>
<span class="hashtag">#with</span> torch.no_grad():<br>
accuracy = 0<br>
for features, labels in test_data:<br>
outputs = model(features)<br>
accuracy += (outputs.argmax(1) == labels).float().mean()</p>
<p>print(f’Accuracy: {(accuracy/batches):.8f}’)</p>

username: braindotai
cooked: <p>You have to divide the accuracy summed over test data by number of batches in test data, not train data. So print accuracy as <code>print(f’Accuracy: {(accuracy/len(test_data):.8f}’)</code></p>

username: Muhammad_Izaz
cooked: <p>its done bro, thanks again for you time.</p>
Cudnn.benchmark slowing execution down

username: koelscha
cooked: <p>Hi,<br>
I just played around a bit with a large architecture and found that my network trains faster when I turn off cudnn.benchmark. The network has a fixed input size. I wanted to see if it is because of my architecture and tested speed with some of the provided torchvision models. For all of them, the same holds. cudnn.benchmark slows testing and training down.</p>
<p>Code to reproduce:</p>
<pre><code class="lang-auto">import torch
import torchvision.models as models
import timeit

torch.manual_seed(0)

torch.backends.cudnn.benchmark = False

model = models.densenet121(pretrained=True)
model.eval()
model.cuda()

in_tensor = torch.empty([16, 3, 224, 224]).cuda()

def bench():
    _ = model(in_tensor)

print(timeit.timeit("bench()", setup="from __main__ import bench", number=100))
</code></pre>
<p>If I run it with cudnn.benchmark = True, I measure 4.1 seconds, and with cudnn.benchmark = False, the program finishes after 3.2 seconds. That’s quite a difference. When training, the difference is even bigger.</p>
<p>I am running pytorch installed from conda. pytorch version is 0.4.1 with cuda 9.0 and cudnn 7.1.2 on Ubuntu 18.04. The GPU is a GTX 1080 Ti.</p>
<p>Am I doing something wrong? If not, then why is cudnn.benchmark enabled by default?</p>
<p>Cheers,<br>
Andreas</p>

username: ngimel
cooked: <p>The first iteration should be excluded from timing, because when cudnn.benchmark is enabled, in the first iteration pytorch will try all available algorithm, and first iteration will be very slow. Also, for proper timing you should be calling cuda.device.synchronize() after you loop.</p>

username: sytelus
cooked: <p>I’m seeing this behavior as well (for all epochs) but only randomly about 20-30% of the runs. When <code>cudnn.benchmark=True</code>, each epoch runs about 8X slower for the bad run compared to good run. Both runs have identical code and hardware but get different random seed. For the bad run GPU is driven to 100% while for the good run it stays around 80%. This is happening for 1080Ti and I’m testing if this also happens on other SKUs. My guess is that for whatever reason cudnn decides to try all algos at each epoch and drives GPU red hot. After disabling cudnn.benchmark, we don’t see this issue.</p>

username: ptrblck
cooked: <p><code>cudnn.benchmark</code> will try different algorithms for each input shape.<br>
If your model expects a lot of varying input shapes, you might want to disable benchmark mode.</p>
Context manager for dtype and device

username: sharvil
cooked: <p>Does PyTorch provide a context manager to specify the dtype and device for all new tensors created inside that context? Roughly, I’m looking for something like:</p>
<pre><code class="lang-python"># `input` is an existing `torch.Tensor`
with default_device_and_dtype(input):   # &lt;-- is there a context manager like this somewhere?
  x = torch.empty(1,2,3)
  y = torch.zeros(4,5)

# expected: `x` and `y` are `torch.Tensor`s with the same dtype and device as `input`
</code></pre>

username: ptrblck
cooked: <p>I’m not aware of such a context manager, so would you mind creating a feature request on GitHub to discuss this proposal?</p>
<p>Meanwhile, you could copy <a href="https://github.com/pytorch/pytorch/blob/463f7920bd45b522b4675663656873fa9689cab0/test/test_type_promotion.py#L17-L29">this util function</a>, which is used in tests and use it in combination with <code>with torch.cuda.device(id)</code>.</p>

username: sharvil
cooked: <p>Thanks for the response. I found the <code>new_*</code> functions on <code>Tensor</code> that seem to provide similar functionality (though maybe more verbose). Not sure if a new feature is worth adding given that there’s an alternative. I suppose a context manager would shrink the API surface and make code more consistent…</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="sharvil" data-post="3" data-topic="73827">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sharvil/48/21517_2.png" class="avatar"> sharvil:</div>
<blockquote>
<p>I suppose a context manager would shrink the API surface and make code more consistent…</p>
</blockquote>
</aside>
<p>Might be, but I personally don’t really like too many context managers, as the code becomes quite unreadable after the 5th indentation. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
<p>Have also a look at the <code>torch.*_like</code> methods, e.g. <code>torch.zeros_like(tensor)</code>.</p>
Pytorch not compatible with React-native

username: Arun_Kumar1
cooked: <p>I’ve been using react native for one of my module.Recently we started using Pytorch . Pytorch along with react native is crashing.<br>
Below is the error</p>
<p>java.lang.UnsatisfiedLinkError: couldn’t find DSO to load: libpytorch_jni.so caused by: dlopen failed: cannot locate symbol “_ZN8facebook3jni10JByteOrder11nativeOrderEv” referenced by “/data/app/*****-4ZGDZJcLItKdhIsz3hj2rQ==/lib/arm/libpytorch_jni.so”.</p>
<p>Please help me.I am in great urgency.</p>

username: mrshenli
cooked: <p>Hey <a class="mention" href="/u/arun_kumar1">@Arun_Kumar1</a></p>
<p>Could you please share a snippet of the PyTorch code that causing this error? Why do you think this relates to <code>torch.distributed</code> or <code>torch.distributed.rpc</code>? Are you using these two packages? It seems to me this questions should be posted under mobile build and Java bindings tag?</p>

username: Arun_Kumar1
cooked: <p>True .I didn’t notice .It should be tagged under mobile</p>
How to stabilize a simple network to small changes in input?

username: chilango
cooked: <p>Beginner question here: I have a small simple fully connected network (1-3 layers, 50 hidden nodes) that is training properly, but I’m getting different results with small changes in input data. The network is deterministic (I have seeded everythig properly, different runs in same data yield exact same result) so that’s not it.</p>
<p>How can I train my simple network to be more robust? I have tried the simple obvious things like playing around with batch size and number of hidden nodes. What else can I look into? Thanks.</p>

username: ptrblck
cooked: <p>I would guess the sensitivity to small changes in the input data depends on the signal and noise in the input data.<br>
E.g. if your input signal has a very low magnitude, your model would have to learn these small differences to correctly predict the output.<br>
On the other hand, you might try to add noise to the input to make the model more robust.</p>
Torch.scatter_add seems not to backprop gradients

username: torchMaster
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/a/6/a648697f8b3d56ee93d86e232b3268dd46d0d7e1.png" data-download-href="https://discuss.pytorch.org/uploads/default/a648697f8b3d56ee93d86e232b3268dd46d0d7e1" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/6/a648697f8b3d56ee93d86e232b3268dd46d0d7e1.png" alt="image" data-base62-sha1="nJ0oWtmYF7G3uhQnnOMFnCZZM77" width="690" height="425" data-dominant-color="32332D"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">803×495 32.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Dear pytorch gurus,<br>
I’m building a neural paraphrasing model and<br>
trying to implement copy mechanism, which uses some tokens in the input sequence as they are when producing output tokens.</p>
<p>Easily put, copy mechanism produces a probability distribution over input tokens. After getting the probability, I try to add this to the probability distribution generated by GRU decoder. In the process, I need to scatter copy distribution into a vector of vocab size, where I used torch.scatter_add.</p>
<p>After that I compute mixture of two probability distribution and compute the loss afterward. The problem is, that loss doesn’t decrease after a few epochs. I doubt the gradient doesn’t flow over this operation. Can anyone suggest an alternative process for this implementation?</p>
<p>Thanks deeply in advance.</p>

username: ptrblck
cooked: <p><code>scatter_add_</code> should correctly propagate the gradients as seen in this example:</p>
<pre><code class="lang-python">x = torch.rand(2, 5, requires_grad=True)
out = torch.ones(3, 5).scatter_add_(0, torch.tensor([[0, 1, 2, 0, 0], [2, 0, 0, 1, 2]]), x)
out.sum().backward()
print(x.grad)
&gt; tensor([[1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1.]])
</code></pre>
<p>Could you check the <code>.grad</code> attribute after calling <code>.backward()</code> for some parameters you are concerned about and make sure they have valid gradients?</p>
Can't put torch.tensor in torch.multiprocess.queue?

username: Liangzhi_Tang
cooked: <p>The following code snippet works fine! it gives the right output :<br>
[0]<br>
[1]<br>
[2]<br>
[3]<br>
[4]<br>
[5]<br>
[6]<br>
[7]<br>
[8]<br>
[9]</p>
<pre><code class="lang-auto">import torch.multiprocessing as mp
import torch

def put(t):
    q.put(t)

def print_queue():
    while True:
        print(q.get())

if __name__ == '__main__':
    q = mp.Queue()
    proc_put_list = []
    for i in range(10):
        proc_put = mp.Process(target=put, args=([i], ))
        proc_put_list.append(proc_put)
        proc_put.start()

    for p in proc_put_list:
        p.join()

    while not q.empty():
        print(q.get())
</code></pre>
<p>However, if I change</p>
<blockquote>
<pre><code>    proc_put = mp.Process(target=put, args=([i], ))
</code></pre>
</blockquote>
<p>to</p>
<blockquote>
<pre><code>    proc_put = mp.Process(target=put, args=(torch.tensor([i]), ))
</code></pre>
</blockquote>
<p>It gives the following error:</p>
<pre><code class="lang-auto">Traceback (most recent call last):
  File "python_mp.py", line 27, in &lt;module&gt;
    print(q.get())
  File "/usr/lib64/python3.6/multiprocessing/queues.py", line 113, in get
    return _ForkingPickler.loads(res)
  File "/usr/local/lib64/python3.6/site-packages/torch/multiprocessing/reductions.py", line 276, in rebuild_storage_fd
    fd = df.detach()
  File "/usr/lib64/python3.6/multiprocessing/resource_sharer.py", line 57, in detach
    with _resource_sharer.get_connection(self._id) as conn:
  File "/usr/lib64/python3.6/multiprocessing/resource_sharer.py", line 87, in get_connection
    c = Client(address, authkey=process.current_process().authkey)
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 487, in Client
    c = SocketClient(address)
  File "/usr/lib64/python3.6/multiprocessing/connection.py", line 614, in SocketClient
    s.connect(address)
</code></pre>
<p><strong>So my question is: can’t put torch.tensor in torch.multiprocess.queue?</strong></p>
API-level Stateful RNNs?

username: OverLordGoldDragon
cooked: <p>Does PyTorch support a Keras-like <code>LSTM(stateful=True)</code>, or it needs to be custom-coded? I’m a TensorFlow user, considering PyTorch, and so far PyTorch’s neural network API appears quite lacking relative to Keras (e.g. <a href="https://github.com/pytorch/pytorch/issues/9572" rel="nofollow noopener">this entire list</a>, most of which Keras supports, also weight constraints).</p>
Run time error when using driver with 16 GPUs

username: idohi
cooked: <p>While running training on BERT model with cluster of AWS with 16 GPUs (p2.16xlarge) I get run time error:</p>
<pre><code class="lang-auto">cuda runtime error (60) : peer mapping resources exhausted at /opt/conda/conda-bld/pytorch_1579040055865/work/aten/src/THC/THCGeneral.cpp:141
</code></pre>
<p>If I’m using cluster with 8 GPUs (p2.8xlarge) it works fine.</p>

username: ptrblck
cooked: <p>Each GPU can only have 8 PCIe peers, so you cannot use a p2p mapping with 16 GPUs in PCIe.<br>
Note that this limitation is specific to PCIe and NVLink should work.</p>

username: idohi
cooked: <p>I’m new to these issues, how to make it work with NVLink?</p>

username: ptrblck
cooked: <p>Your server would have to use NVLink to connect the GPUs.<br>
You can check the connectivity matrix via <code>nvidia-smi topo -m</code>.</p>
Multi-class classification vs binary classification

username: Slamskow
cooked: <p>Why does turning a binary classification problem into a 2-class multi-class classification problem lead to better performance?</p>

username: KFrank
cooked: <p>Hi Sofia!</p>
<aside class="quote no-group" data-username="Slamskow" data-post="1" data-topic="73800" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/b487fb/48.png" class="avatar"> Slamskow:</div>
<blockquote>
<p>Why does turning a binary classification problem into a 2-class multi-class classification problem lead to better performance?</p>
</blockquote>
</aside>
<p>I don’t believe that it does.  Do you have contrary experience?</p>
<p>Best.</p>
<p>K. Frank</p>
Efficient complex-real transform

username: mpariente
cooked: <p>I need to solve a systems of linear equations in the complex domain, in a backpropagable way.<br>
As stated in this research report (<a href="https://hal-ens-lyon.archives-ouvertes.fr/ensl-00125369v2/document" rel="nofollow noopener">https://hal-ens-lyon.archives-ouvertes.fr/ensl-00125369v2/document</a>), complex numbers can be represented as real 2x2 matrices and complex systems of linear equations can be solved using real solver on 2 times bigger matrices.</p>
<p>So, I’m converting vectors containing concatenated real and imaginary parts to these type of matrices. I have a working example :</p>
<pre><code class="lang-auto">import torch

def vec_to_cmat(vec, dim=-1):
    """ Expects tensor with shape 2*(N**2) on dim 'dim'"""
    # Get matrix size (before CR tranform)
    vec_shape = list(vec.shape)
    mtxc_size = int((vec_shape[dim]/2)**0.5)
    # Keep all remaining dimensions (matrix goes last)
    vec_shape.pop(dim)
    mat_shape = vec_shape + [2 * mtxc_size, 2 * mtxc_size]
    mat = vec.new_zeros(mat_shape)
    # Create indexes
    line_idx = torch.LongTensor([2 * m for _ in range(mtxc_size) for
                                 m in range(mtxc_size)])
    col_idx = torch.LongTensor([2 * n for n in range(mtxc_size) for
                                _ in range(mtxc_size)])
    # Split real and imaginary parts
    re, im = torch.split(vec, [mtxc_size**2, mtxc_size**2], dim=dim)
    # re, im = torch.chunk(vec, 2, dim=dim)  # works as well

    # Assign the re or im parts.
    mat[..., line_idx, col_idx] = re
    mat[..., line_idx+1, col_idx+1] = re
    mat[..., line_idx+1, col_idx] = im
    mat[..., line_idx, col_idx+1] = -im
    return mat

inp = torch.randn(10, 2, 18)
complex_mat = vec_to_cmat(inp)  # (10, 2, 6, 6)
</code></pre>
<p>I’ve been looking for a nicer way to assign the values without luck until now. Could anybody help please?<br>
Also, this only works for <code>dim=-1</code> for now, any idea (except transposing vec) on how to make it work for any <code>dim</code></p>
Pytorch jit script error

username: shy
cooked: <p>Hi I’m trying to export my code to jit script, but come across the following error:<br>
RuntimeError: inputs_.size() == 1 INTERNAL ASSERT FAILED at /pytorch/torch/csrc/jit/ir.h:364, please report a bug to PyTorch.  (input at /pytorch/torch/csrc/jit/ir.h:364)<br>
frame <span class="hashtag">#0:</span> c10::Error::Error(c10::SourceLocation, std::string const&amp;) + 0x33 (0x7f193351b813 in /job/.local/lib/python3.7/site-packages/torch/lib/libc10.so)<br>
frame <span class="hashtag">#1:</span>  + 0x51af79 (0x7f19364baf79 in /job/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#2:</span>  + 0x516db5 (0x7f19364b6db5 in /job/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#3:</span> torch::jit::PeepholeOptimizeONNX(std::shared_ptr<a>torch::jit::Graph</a>&amp;, int, bool) + 0xcb (0x7f19364baa5b in /job/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#4:</span>  + 0x4cfbd5 (0x7f193646fbd5 in /job/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)<br>
frame <span class="hashtag">#5:</span>  + 0x210ba4 (0x7f19361b0ba4 in /job/.local/lib/python3.7/site-packages/torch/lib/libtorch_python.so)<br>
<br>
frame <span class="hashtag">#32:</span> __libc_start_main + 0xe7 (0x7f193a3dab97 in /lib/x86_64-linux-gnu/libc.so.6)</p>
<p>Is there any clue on what problem it is? Thanks</p>

username: albanD
cooked: <p>Hi,<br>
It looks quite bad yes.<br>
Can you share a small code sample that reproduces this issue?</p>

username: shy
cooked: <p><a href="https://github.com/kmkurn/pytorch-crf/blob/1b7ced20c01352ccf70b7ae7bb99a58366de8e48/torchcrf/__init__.py#L259" rel="nofollow noopener">https://github.com/kmkurn/pytorch-crf/blob/1b7ced20c01352ccf70b7ae7bb99a58366de8e48/torchcrf/<strong>init</strong>.py#L259</a></p>
<p>It’s pretty deal with this function, i move out this function as a non-class function and add <span class="mention">@torch.jit.script</span></p>

username: James_Reed
cooked: <p>Hi <a class="mention" href="/u/shy">@shy</a>,</p>
<p>This looks like an issue with some precondition in ONNX export. Do you mind <a href="https://github.com/pytorch/pytorch/issues/new/choose" rel="nofollow noopener">filing an issue</a> on github so we can route this to the right developers?</p>
<p>Thanks,<br>
James</p>
Caching with Dataset

username: apytorch
cooked: <p>First, my dataset class does not modify the data loaded (from HDF files, in this case). On the computers I run my models on, there is not enough ram to hold all of the dataset items.</p>
<p>To speed up loading, I have been caching up to a specific count. Then, in <strong>get_item</strong> it just tests if the item is cached, and returns that, or loads the item from disk. However, this means that it takes 10+ minutes at the start to load all of the data that it can. I would prefer to lazily load the data as the dataloader calls the dataset.</p>
<p>First attempt was just to have the workers try to update the cache list, but this failed because the object isn’t updated among the workers.</p>
<p>Second attempt, which I got from here, was to use a multiprocessing manager list.</p>
<pre><code>self.manager = Manager()
self.cache = self.manager.list([None]*self.count)
</code></pre>
<p>And this seemed to work on simple tests (artificially limiting the number of items, I saw that it lazily cached everything it could, and read from the cache on the subsequent epochs). However, when I let it run on the full set, it failed before finishing the all of the batches for the first epoch.</p>
<p>I expect this is because sometimes two workers simultaneously try to update the shared list. The error raised is “convert_to_error” on the line where the item loaded from disk is saved into list:</p>
<pre><code>self.cache[index] = sample
</code></pre>
<p>I’ve searched here and the multiprocessing documentation, but I can’t find any way to add some sort of lock to this line, so that only one worker is writing to the cache at a time.</p>
Forward pass memoization

username: dogfooding
cooked: <p>How to memoize and reuse results of nn.Module in the forward pass?</p>
<pre><code class="lang-auto">class MagicModule(torch.nn.Module):
    def __init__(self):
        super(MagicModule, self).__init__()
        
    def forward(self, x):
        x = torch.norm(x)
        return x
    
class ModelA(torch.nn.Module):
    def __init__(self, magic_module):
        super(ModelA, self).__init__()
        self.magic_module = magic_module
        
    def forward(self, x):
        x = self.magic_module(x)
        x = x ** 2
        return x

class ModelB(torch.nn.Module):
    def __init__(self, magic_module):
        super(ModelB, self).__init__()
        self.magic_module = magic_module
        
    def forward(self, x):
        x = self.magic_module(x)
        x = x ** 3
        return x

magic_module = MagicModule()
A = ModelA(magic_module)
B = ModelB(magic_module)

x = torch.rand(3, 4)
A(x)
B(x)
</code></pre>
<p>In the code example, magic_module(x) is called twice with the same result. In general, how can I memoize the result and make sure the same module shared by other modules/models will only be called once?</p>

username: ptrblck
cooked: <p>I’m not sure if there is a better way than just moving <code>magic_module</code> outside of both models and call only once into <code>magic_module</code>.</p>

username: dogfooding
cooked: <p>Hi thanks for your reply. I kind of trapped in a framework requiring me to structure models in a certain way, so I am not sure what you said is possible. But will take a look if it can hack through.</p>
Output of AdaptiveSoftmaxWithLoss

username: Peyash
cooked: <p>I need help regarding <code>torch.nn.AdaptiveSoftmaxWithLoss</code><br>
It returns both <code>output</code> and <code>loss</code>.<br>
<code>output</code> is the computed target log probabilities for each example, but I can’t seem to make sense of this.<br>
If it was a probability distribution over <code>C</code> classes then I could have predicted a particular class.<br>
How do I interpret this single target log probability? And what can I do with this?</p>
How to load pre-trained model and train with new untrained one

username: shirui-japina
cooked: <p>Now what I want to achieve is, to load a pre-trained model <em><strong>A</strong></em> and then combine it with an untrained model <em><strong>B</strong></em> to form a large new model (<em><strong>A</strong></em> + <em><strong>B</strong></em>) and then train the large new model.</p>
<p>How can this be achieved?</p>

username: simaiden
cooked: <p>Have you tried something like</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from torchvision import models

class MyNewModel(nn.Module):
  def __init__(self):
     super(MyNewModel, self).__init__()
     self.resnet = models.resnet18(pretrained=True)
     self.out = nn.Sequential([nn.Conv2d(...) , nn.Sigmoid()])
  def forward(self,x):
     x = self.resnet(x)
     x = self.out(x)
     return  x


def train():
    model = MyNewModel()
  #model training
</code></pre>
<p>In this case using a pretrained resnet18</p>

username: shirui-japina
cooked: <p>Such a good idea. <img src="https://discuss.pytorch.org/images/emoji/apple/grin.png?v=9" title=":grin:" class="emoji" alt=":grin:"><br>
So, we can define <code>class Model()</code> like:</p>
<pre><code class="lang-auto">class MyNewModel(nn.Module):
  def __init__(self):
    super(MyNewModel, self).__init__()
    self.pre_trained_model = pre_trained_model
    self.untrained_part = untrained_part
  def forward(self, x):
    x = self.pre_trained_model(x)
    x = self.untrained_part(x)
</code></pre>
<p>Thank you so much for your timely help. <img src="https://discuss.pytorch.org/images/emoji/apple/rofl.png?v=9" title=":rofl:" class="emoji" alt=":rofl:"> <img src="https://discuss.pytorch.org/images/emoji/apple/rofl.png?v=9" title=":rofl:" class="emoji" alt=":rofl:"></p>
Pytorch with React Native

username: Chame_call
cooked: <p>Is that possible to train model on desktop with python and then export it to mobile device with React Native (for cross-platform deployment)??</p>

username: VitalyFedyunin
cooked: <p>Consider exporting model to ONNX and using JS libraries to execute it.</p>

username: Chame_call
cooked: <blockquote>
<p>and using JS libraries to execute it.</p>
</blockquote>
<p>Under JS libraries you mean react native?</p>

username: VitalyFedyunin
cooked: <p>You can use ONNX.js from the Microsoft <a href="https://heartbeat.fritz.ai/building-an-image-recognition-app-using-onnx-js-c7147f4f291b" rel="nofollow noopener">https://heartbeat.fritz.ai/building-an-image-recognition-app-using-onnx-js-c7147f4f291b</a></p>
Editing a .pth file

username: Chanakya_Kulkarni
cooked: <p>Hello ,<br>
I want to analyse a densenet-121 model code on my spyder 4.0<br>
but the pth file I downloaded from a link from github is not opening properly<br>
I dont want to “load” the model I want to edit it.<br>
how should i go about it</p>

username: albanD
cooked: <p>Hi,</p>
<p>There files are saved with pickle so they are compressed. I don’t think you will be able to modify them without using a python environment that loads the model, do the modifications and save it back.</p>

username: Chanakya_Kulkarni
cooked: <p>any step by step procedure or blog that you know of<br>
i am using conda with spyder 4.0</p>

username: albanD
cooked: <p>What I have in mind is just the following:</p>
<pre><code class="lang-python">old_model = torch.load("model.pth")
# Change the model here
new_model = your_change(old_model)
torch.save(new_model, "new_model.pth")
</code></pre>

username: Chanakya_Kulkarni
cooked: <p>where can I get a noob friendly Densenet-121 pytorch code</p>

username: albanD
cooked: <p>You can find it in the models available in torchvision: <a href="https://pytorch.org/docs/stable/torchvision/models.html">https://pytorch.org/docs/stable/torchvision/models.html</a></p>
Onnx export of DeepLabv3 model

username: folsson
cooked: <p>I wold be grateful to any example/tutorial showing a succesful onnx exportation of a DeepLabv3+ model. I fail to export a transfer trained model and I have so far been unsuccesful to find any good example showing this.</p>
<p>What I have tried is:</p>
<p><span class="hashtag">#Saving</span> my transfer learnt model<br>
torch.save(trained_model.state_dict() ,os.path.join(mypath,‘weights1.pt’))</p>
<p><span class="hashtag">#Attemp</span> to load and export :</p>
<p>device = torch.device(‘cpu’)<br>
model = models.segmentation.deeplabv3_resnet101(<br>
pretrained=True, progress=True)<br>
model.classifier = models.segmentation.deeplabv3.DeepLabHead(2048, 1)<br>
model.load_state_dict(torch.load(‘weights1.pt’, map_location=device))<br>
model.eval()<br>
dummy_input = torch.randn(1, 3, 224, 224)<br>
torch.onnx.export(model, dummy_input, “outmodel.onnx”)</p>
<p>I keep getting this error message:<br>
RuntimeError: Only tuples, lists and Variables supported as JIT inputs, but got collections.OrderedDict</p>
<p>Thanks,</p>
All_gather trigger unnecessary in-place change

username: boy2000_007man
cooked: <p>Hi. For example, for the code snippet</p>
<pre><code class="lang-auto">a = torch.rand(3).requires_grad_()
l = a.sum()
torch.distributed.all_gather([a,b,c], a)
</code></pre>
<p><code>l.backward()</code> will trigger an error say</p>
<blockquote>
<p>one of the variables needed for gradient computation has been modified by an in-place operation</p>
</blockquote>
<p>I assume this is because all_gather do in-place change for all a,b,c.<br>
But why? As a is emitted by the current process, it’s not necessary to change it and cause this issue. Is there any consideration behind this all_gather behavior?</p>
<p>Thx</p>

username: mrshenli
cooked: <p>We probably can add a shortcut to avoid changing <code>a</code> in this case, but I am not sure if that is a good idea, because that will make all_gather have different behavior depending on underlying storage. Consider two cases.</p>
<p>Case 1:</p>
<pre><code class="lang-auto">x = empty_like(a)
torch.distributed.all_gather([x,b,c], a)
</code></pre>
<p>In this case, we would still need to write data from a to x, right?</p>
<p>Case 2:</p>
<pre><code class="lang-auto">x = a.view(...) # say change the stride
torch.distributed.all_gather([x,b,c], a)
</code></pre>
<p>In this case, <code>x</code> will share the storage with <code>a</code>, but using a different element layout, so we would need to write into x.</p>
<p>To address the above problems, we probably can detect and only skip inplace write if <code>x</code> shares the same storage and meta with <code>a</code>. However, the concerns are 1) does the extra overhead worth it? 2) will the disparity in all_gather’s behavior confuse users?</p>
<p>This <a href="https://github.com/pytorch/pytorch/pull/33924" rel="nofollow noopener">PR</a> might be relevant. It’s trying to avoid a flatten.</p>
Building from source with conda build

username: seliad
cooked: <p>We need to build pytorch from source with our own patch and custom libraries. (version v1.4.0) and distribute it as conda package.</p>
<p>There is an excellent Dockerfile in the main repository, we manage to build with it everything with few changes.<br>
However we encounter many bugs when trying to use pytorch/builder/conda (building fails and can’t see where the bug is, its probably some environment issue).<br>
I wonder, as we already have a working built (modified) pytorch on the Docker image,<br>
Can we use it somehow to distribute a conda package?<br>
Any other workaround?</p>

username: albanD
cooked: <p>Hi,</p>
<p>This repo is really only used for our internal CI and tailored to build the binaries on circleci’s machine (not intended for other use).<br>
What is the exact issue you’re seeing?</p>

username: seliad
cooked: <p>The build fails even though I use the same dockerfile from the repo, all I add is openmpi. (but it fails without it too).<br>
In the new Dockerfile from main pytorch repo it is working with only 1 line change.</p>
<p>I don’t fully understand where it fails and what causing the failure in pytorch/builder.<br>
I made it work ~4 months ago for the nightly version, but I can’t make it work again for pytorch 1.4 (or later).</p>

username: albanD
cooked: <p>cc <a class="mention" href="/u/seemethere">@seemethere</a> might know if it would be easy to re-purpose torch/builder to run outside our CI infra?</p>
In the tutorials,why we use "torchvision.utils.make_grid(images)" to show image?

username: SangYC
cooked: <h2>This is a tutorial code:</h2>
<p>def imshow(img):<br>
img = img / 2 + 0.5     # unnormalize<br>
npimg = img.numpy()<br>
plt.imshow(np.transpose(npimg, (1, 2, 0)))<br>
plt.show()</p>
<h1>get some random training images</h1>
<p>dataiter = iter(trainloader)<br>
images, labels = dataiter.next()</p>
<h1>show images</h1>
<p>imshow(torchvision.utils.make_grid(images))</p>
<h1>print labels</h1>
<h2>print(’ ‘.join(’%5s’ % classes[labels[j]] for j in range(4)))</h2>
<p>What is function of  ‘torchvision.utils.make_grid’ ?why don’t show image using functions 'imshow (images) directly?</p>

username: ptrblck
cooked: <p><a href="https://pytorch.org/docs/stable/torchvision/utils.html#torchvision.utils.make_grid"><code>torchvision.utils.make_grid</code></a> creates a single image containing all passed image tensors as a grid, which might be more convenient in case you want to display a few images.</p>

username: SangYC
cooked: <p>Thanks for your help.</p>

username: rafaelpadilla
cooked: <p>Still concerning this topic:</p>
<p>In the tutorial, the <code>img_grid</code> is sent to tensorboard without normalization:<br>
<code>writer.add_image('four_fashion_mnist_images', img_grid)</code></p>
<p>while the function <code>matplotlib_imshow</code> unnormalizes the images.</p>
<p>As the images are sent to tensorboard to be visualized, shouldn’t tensorboard also receive unnormalized images?</p>
Gamma distribution negative log-likelihood minimization error

username: shashank_gupta1
cooked: <p>Hi,</p>
<p>I am trying to fit a gamma distribution over real-values data, in a regression setting by using LSTM over the time-series input.</p>
<p>I am predicting the k and theta parameter <img src="https://discuss.pytorch.org/uploads/default/original/3X/8/4/8446475608af199daed004fc43a9d1f32ec3ad65.png" alt="Screenshot 2020-03-19 at 8.13.28 PM" data-base62-sha1="iS9B2vrWCVuZOjHyEgaUdJxkkND" width="442" height="84"> from the last state of LSTM over input sequence.</p>
<p>I am using the following NLL function:</p>
<p>def gamma_nll(inp, target):</p>
<pre><code>target = torch.squeeze(target, 1)

eps = 1e-8
# inp is a 2 dim param vector for weibull distribution

k = neta[:, 0]
theta = neta[:, 1]

log_gamma = -2.081061466 - k + 0.0833333 / (3.0 + k + eps) - torch.log(k * (1.0 + k) * (2.0 + k) + eps) + (2.5 + k) * torch.log(3.0 + k + eps)

log_lik_weibull = log_gamma + k * torch.log(theta + eps) - (k-1) * torch.log(target + eps) + torch.div(target, theta + eps)

loss = torch.mean(log_lik)

return loss  
</code></pre>
<p>The loss is increasing as training progresses.</p>
<p>I am not sure what is wrong.</p>
What is the multi-label binary classification?

username: Brans
cooked: <p>Reading PyTorch docs <a href="https://pytorch.org/docs/stable/nn.html" rel="noopener nofollow ugc">BCEWithLogitsLoss</a> I have found:</p>
<blockquote>
<p>where c is the class number (c &gt; 1 for multi-label binary classification, c = 1 for single-label binary classification), nn is the number of the sample in the batch and p_cp c is the weight of the positive answer for the class cc .</p>
</blockquote>
<p>What is the  <strong>multi-label</strong>  binary classification? Binary assume only two labels AFIK</p>

username: torchMaster
cooked: <p>Multi label classification allows a data sample to have more than 2 labels. For instance, you can think of a classifier that infers whether an animal is cat or not and whether the color of the animal is red at the same time. In this case, you have 2 labels(multi label) for the sample and the labels are all binary.</p>
Is that Dataset amount suitable for training end-to-end speech recognition

username: qute012
cooked: <p>Hello. I use DeepSpeech2 for korean speech recognition.<br>
I used nearly 20,000 audio for learning. It’s almost 51.6 hours.<br>
I extract 1202 character in transcripts, 1202+1(pad token)+1(space). It means FC Layer output size 1204.<br>
But Loss almost not decreasing. I think preprocessing or not suitable audio file amount.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/5/954e84c88873d11bf84b1b9305103eb851f1db8e.png" data-download-href="https://discuss.pytorch.org/uploads/default/954e84c88873d11bf84b1b9305103eb851f1db8e" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/5/954e84c88873d11bf84b1b9305103eb851f1db8e.png" alt="image" data-base62-sha1="liPmQUwhvSd04Xpj6cUfqAl2oCW" width="690" height="214" data-dominant-color="E8E9E5"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">822×255 6.79 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Help me…</p>
Ny.rint = in Pytorch?

username: Pkh
cooked: <p>Please can someone let me know what command in Pytorch equals ny.rint.<br>
I need to import black and white images and turn their pixels into only black or white pixels without a gray scale. Can anyone help?</p>

username: yash1994
cooked: <p>Hi,</p>
<p>You can call <code>.round()</code> function on your tensor to achieve <code>np.rint</code> kind of result.</p>

username: Pkh
cooked: <p>I read I can also binary the images. Would you know how that works?</p>

username: yash1994
cooked: <p>Hi,</p>
<p>You can divide your gray scale image array by 255.0 (if you have 8-bit image because for 8-bit image pixel intensities ranges from 0 to 255) (if you have n-bit image then this number would be 2**n - 1) and apply <code>.round()</code> function to get binary image.</p>
How to run sub operators in parallel by contrast with Sequential

username: yqian
cooked: <p>Hi, I have a self-defined operator. For every layer, I need to run multiple operators in parallel, just like multiple convolutional kernels in each layer. As is known, <code>nn.Sequential</code> is a good way to run several operators in sequential order. But I can’t find similar class to run operators in parallel manner, may be <code>nn.Parallel</code>.</p>
<p>I have two optional methods to achieve the goal without considering running time or memory. The first one is to use <code>for</code> loop directly, and the second candidate is preparing multiple copies of the input. Both methods are naive and suffer from inefficiency in either running time or memory.</p>
<p>Is there any elegant way to achieve this?</p>

username: AwesomeLemon
cooked: <p>Hi, I’m looking for exactly the operator you describe. If you’ve managed to find something, could you share it here?</p>

username: yqian
cooked: <p>Sorry, I haven’t found a solution too.</p>
Translating PyTorch into Glow IR

username: yetanadur
cooked: <p>Does glow provide a way to translate PyTorch model directly into Glow IR without first converting into ONNX ? I am trying to find out if ONNX op support a dependency ?</p>
<p>Cheers,<br>
Yetanadur</p>

username: jfix
cooked: <p>Hi <a class="mention" href="/u/yetanadur">@yetanadur</a>,</p>
<p>We have a path that goes through TorchScript, but it is still under active development and so does not have as much support or stability yet. Please see this doc for more info: <a href="https://github.com/pytorch/glow/blob/master/docs/pytorch.md" rel="nofollow noopener">https://github.com/pytorch/glow/blob/master/docs/pytorch.md</a></p>

username: yetanadur
cooked: <p>Thanks for the pointer Jordan!</p>
RuntimeError with WeightedRandomSampler

username: vojtooo
cooked: <p>Hi, I have implemented the following piece of code to do oversampling of my training dataset that is highly imbalanced.</p>
<pre><code class="lang-auto">    if OVERSAMPLING:
        class_weights = 1. / torch.Tensor([25810, 2443, 5292, 873, 708]) # 1/number of samples in each class
        weighted_sampler = torch.utils.data.sampler.WeightedRandomSampler(weights = class_weights, num_samples=NUM_SAMPLES_TRN, replacement=True)
        train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, sampler = weighted_sampler)
</code></pre>
<p>However, I am getting a lot of error messages and therefore have a couple of questions regarding things, that I could not clearly understand from the docs or other questions posted here:</p>
<ol>
<li>
<p>what should the argument <strong>num_samples</strong> in the WeightedRandomSampler value be? batchsize or the size of the whole dataset?</p>
</li>
<li>
<p>what exactly means <strong>replacement=True</strong> and why does my code not run with replacement=False - getting:</p>
</li>
</ol>
<pre><code class="lang-auto">RuntimeError: cannot sample n_sample &gt; prob_dist.size(-1) samples without replacement
</code></pre>
<ol start="3">
<li>the above code snippet with replacement=True results in the following batches</li>
</ol>
<pre><code class="lang-auto">tensor([0, 0, 0, 0, 0, 0, 0, 0])
tensor([0, 0, 0, 0, 0, 0, 0, 0])   
tensor([0, 0, 0, 0, 0, 0, 0, 0])     
tensor([0, 0, 0, 0, 0, 0, 0, 0])     
tensor([0, 0, 0, 0, 0, 0, 0, 0])   
tensor([0, 0, 0, 0, 0, 0, 0, 0])
tensor([0, 0, 0, 0, 0, 0, 0, 0]) 
tensor([0, 0, 0, 0, 0, 0, 0, 0]) 
tensor([0, 0, 0, 0, 0, 0, 0, 0])
tensor([0, 0, 0, 0, 0, 0, 0, 0]) 
tensor([0, 0, 0, 0, 0, 0, 0, 0])   
</code></pre>
<p>Its sampling only from the class 0, which is the most populous. However, I wanted to support sampling from classes, that don’t have that many samples, not this.</p>
<p>Could you please give me hand?</p>
<p>Thanks a lot <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=9" title=":blush:" class="emoji" alt=":blush:"><br>
Vojtooo</p>

username: ptrblck
cooked: <p>The <code>weights</code> argument should contain weights for each sample, not just the class weights.<br>
Have a look at <a href="https://discuss.pytorch.org/t/how-to-handle-imbalanced-classes/11264/2">this example</a>.</p>
<ol>
<li>
<p>Usually you would pass <code>num_samples = len(dataset)</code>, but you could also vary the number of samples from your <code>DataLoader</code>, if that’s your use case.</p>
</li>
<li>
<p>With <code>replacement = True</code> each drawn sample can be drawn again. On the other hand, <code>replacement = False</code> still uses the weights associated with each sample, but removes a drawn sample from the set.</p>
</li>
</ol>
GPU Memory Usage Accumulates

username: uertenli
cooked: <p>Hi!</p>
<p>I am using FasterRCNN from torchvision to perform validation. Everything worked fine until I tried to store the predictions of the model to an array. I am getting only 10 predictions per image and I have 120 frames. Plus, I transfer all the variables to the cpu and store them there. However, at each iteration (i.e. after processing each frame) the GPU usage keeps accumulating by 800 MB per frame. Thus, I run out of memory.</p>
<p>Does anyone have any idea as to how can I store the prediction scores in a delicate way or a fix to my way of solving this ?</p>
<p>A sample piece of code is below:</p>
<pre><code class="lang-auto">for frame in video:
    predictions[i] = model(frame)
    for key, value in predictions[i].items():
        predictions[i][key] = predictions[i][key].to('cpu')
</code></pre>
<p>Best.</p>

username: ptrblck
cooked: <p>Wrap your code in a <code>with torch.no_grad()</code> block, if you don’t need to call <code>.backward()</code> in this part of the code or detach the tensor from the computation graph, so that it can be freed via:</p>
<pre><code class="lang-python">predictions[i][key].detach().to('cpu')
</code></pre>
<p>Even though you are pushing the prediction to the CPU, the attached computation graph, which is still on the GPU, will be kept alive.</p>
Can't load parameter of same shape from state_dict

username: maaft
cooked: <p><code>While copying the parameter named "toolbox_modules.backbone.kx", whose dimensions in the model are torch.Size([3, 3, 3, 3]) and whose dimensions in the checkpoint are torch.Size([3, 3, 3, 3]</code></p>
<p>Err… what?</p>
<p>Hier is my module definition:</p>
<pre><code class="lang-python">        self.kx = nn.Parameter(torch.tensor([[1, 0, -1],
                                             [2, 0, -2],
                                             [1, 0, -1]], dtype=torch.float32).view(1, 1, 3, 3).expand(3, 3, 3, 3),
                               requires_grad=False)

        self.ky = nn.Parameter(torch.tensor([[1, 2, 1],
                                             [0, 0, 0],
                                             [-1, -2, -1]], dtype=torch.float32).view(1, 1, 3, 3).expand(3, 3, 3, 3),
                               requires_grad=False)
</code></pre>

username: ptrblck
cooked: <p>That’s not a very helpful message.<br>
The error comes from the expanded parameter inside your model and you could avoid it by calling <code>.contiguous()</code> on it or use <code>repeat</code> instead of <code>expand</code>.</p>
<p>Also, since your parameter does not require gradients, you could register a buffer instead (via <code>self.register_buffer</code>). If you want to update this parameter (and set <code>requires_grad=True</code>), you’ll most likely get an error claiming:</p>
<blockquote>
<p>RuntimeError: unsupported operation: more than one element of the written-to tensor refers to a single memory location. Please clone() the tensor before performing the operation.</p>
</blockquote>
<p>Would you mind creating an issue on GitHub?</p>
LSTM, what exactly defines the batch size?

username: MC95
cooked: <p>Hello everybody,<br>
I’m pretty new to Pytorch. I have two questions, hope you can help me.<br>
Imagine the following sequence:</p>
<pre><code class="lang-auto">input = [[1,2,3,4,5,6,7,8,9,10]]
</code></pre>
<p>Let’s say I want to learn this sequence using 2 look backs. So I would have:</p>
<pre><code class="lang-auto">input = [[1,2], [2,3], [3,4]....] and so on.
</code></pre>
<p>So my first question is, how do I have pass the input to a LSTM? I know I could shape the input like <code>(seq_len, batch_size, features</code>) so here it would be <code>(2, 9, 1)</code>, right?</p>
<p>Does the following has the same meaning? So i loop over the batches and pass them to the LSTM but with <code>batch_size=1</code>. After all batches I do the <code>backward()</code>?</p>
<pre><code class="lang-auto">...
for i in range(input.size(0)):
    x = [input[i].view(2,1,1)
    seq.forward(x)
    ....
....
loss.backward()
</code></pre>
<p>Is there a difference of setting the <code>batch_size</code> parameter or just looping over the batches?</p>
<ol start="2">
<li>Let’s say I want to predict a value out of  4 look backs. And I have 200 samples of these sequences. How do I have to build the input vector now? Does the number of training samples acutally stands for the batch_size?</li>
</ol>
<p>Thank you!</p>

username: chauby_Zou
cooked: <p>I have a same question about the batch_size and sequence length in Pytorch, so is there anybody to answer the question?</p>
LibTorch Autograd Error

username: esaliya
cooked: <p>I am trying to run the <a href="https://pytorch.org/cppdocs/#autograd" rel="nofollow noopener">LibTorch autograd example</a> but runs into the following error.</p>
<pre><code class="lang-auto">class c10::Error: grad can be implicitly created only for scalar outputs (_make_grads at ..\..\torch\csrc\autograd\autograd.cpp:47)
</code></pre>
<p>The code to reproduce this is.</p>
<pre><code class="lang-auto">#include &lt;torch/csrc/autograd/variable.h&gt;
#include &lt;torch/csrc/autograd/function.h&gt;

int main(int argc, char** argv)
{
  torch::Tensor a = torch::ones({2, 2}, torch::requires_grad());
  torch::Tensor b = torch::randn({2, 2});
  auto c = a + b;
  c.backward(); // a.grad() will now hold the gradient of c w.r.t. a.

  return EXIT_SUCCESS;
}
</code></pre>
<p>This works if <code>c</code> is a scalar. Any help on this?</p>