akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Suppression of false detections in one-class object detection

username: Chame_call
cooked: <p>Hi.<br>
I’m performing object detection class with only one class.<br>
Trained network show good results by detection the class objects but sometimes it identifies false areas as objects with a fairly high percentage of confidence.<br>
Is there are any methods by suppressing such detections?<br>
Thanks.</p>

username: tom
cooked: <p>If you see these in training, too, you could make a point of oversampling them in the next epoch (or an intermediate mini-epoch). More elaborate schemes of training instances where the model gives undesired results go under the name of “hard example mining”.</p>
<p>Another option could be to add a post-hoc classification step where you train a classifier to have a second opinion about whether the detected image shows anything.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Chame_call
cooked: <p>Is that true that when we skip some true positives when label an image it affects in a model’s accuracy decreasing (<a href="https://stackoverflow.com/questions/60322897/is-it-important-to-label-all-visible-objects-for-object-detection" rel="noopener nofollow ugc">discussed in here</a>)?<br>
If so doesn’t it mean that for suppression false positives we can just add empty images not containing true positives?</p>

username: tom
cooked: <p>No, you should label all things correctly, but you can show the network images where nothing of interest is to be seen (and where it mistakenly detects something) more often.</p>
Pytorch without MKL with pip

username: itachi96
cooked: <p>I am currently installing PyTorch with pip using a wheel file from <a href="https://download.pytorch.org/whl/torch_stable.html" rel="noopener nofollow ugc">here</a>. That package is compiled with MKLDNN. I want to know if there’s a flag or function call which changes the backend of BLAS at runtime without any recompilation. I don’t want to use MKL_NUM_THREADS=1 as this won’t change the backend.</p>
Activate eval mode in forward

username: Erica_Zheng
cooked: <p>I’m calling <code>self.eval()</code> in the <code>forward()</code> of a <code>nn.Module</code> object, the loss becomes NaN.<br>
Is this normal?</p>

username: tom
cooked: <p>No, quite likely there is something else amiss.<br>
Calling self.eval() in the forward is likely not the ideal way of doing things, either. But to say something about a better way, it would be good if you could share what should happen that led you to do that.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Erica_Zheng
cooked: <p>Thank you, Thomas!</p>
<p>I want to calculate the value in the evaluation mode, so that it could be compared with the regular forward( ) output.</p>
<p>I just figure out a workaround: copying the code of the evaluation mode, and use an boolean variable to control the execution. It works.</p>
What is the correct tensor?

username: Giuseppe
cooked: <p>Hi guys i’ve extracted the features from resnet50 before the layer of Average pooling.<br>
I achieve a tensor of 1x100352, but i have to respect the structure of the image, so what is the exact shape of the tensor? 2048x7x7 or 7x7x2048?</p>

username: Caruso
cooked: <p>In PyTorch the default is channels first, so the Tensor should be of shape [B, C, H, W] -&gt; [1, 2048, 7, 7]</p>
Loss function without expected output

username: DanielC
cooked: <p>hi there,<br>
I have a question about the mechanism of the Pytorch autograd.<br>
For the usual loss functions, we have two inputs which are yhat(real output) and y(expected output), as shown in the following two codes.</p>
<pre><code class="lang-auto">def mse_loss (yhat, y):
    loss = torch.mean((y - yhat) ** 2)
    return loss
</code></pre>
<pre><code class="lang-auto">def cross_entropy_loss(yhat,y):
    L = len(yhat)
    loss = - (1 / L) * (torch.mm(y.T, torch.log(yhat)) 
                         + torch.mm((1 - y).T, torch.log(1 - yhat)))
    return loss
</code></pre>
<p>However, I want to use a neural network to minimize my own loss function, which doesn’t have y (expected output), will the loss.backward() function still return me cerrect result?</p>
<pre><code class="lang-auto">def my_loss(yhat):
     L = len(yhat)
     loss= (1/L) * torch.mm(yhat.T, torch.log(yhat))
     return loss
</code></pre>

username: tom
cooked: <p>Yes. To the autograd machinery, any scalar function will do as a loss and it will just go over the computational graph regardless of how you arrived there (within the restrictions of needing differentiable things and only following nodes that have <code>requires_grad</code>, which e.g. the true labels usually do not have).</p>
<p>Best regards</p>
<p>Thomas</p>

username: DanielC
cooked: <p>Hi Tom,<br>
Thanks for your answer, I already try this function in my model and it works.</p>
Loading a conv's pretrained weights onto its Decoupled version?

username: Scott_Hoang
cooked: <p>In an attempt to reduce the power usage of a given model, I’m replacing some of its conv2d layers with its Depth-wise Separable version which is composed of a Depth Wise conv2d following a  PointWise conv2d. It is then still possible to use the pre-trained weights of the original conv2d? If so, how can I go about it?</p>
Resnet200 caffe model to pytorch

username: Federico_Vasile
cooked: <p>Hi everyone,</p>
<p>at <a href="https://github.com/yjxiong/anet2016-cuhk/blob/master/models/get_reference_models.sh" rel="noopener nofollow ugc">this link</a> I have two files: .caffemodel and .prototxt, I would like to import these weights in a resnet200 pytorch model.<br>
Any suggestions on how to do it? <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>PS: I found this very good <a href="https://github.com/vadimkantorov/caffemodel2pytorch#run-caffe-models-using-pytorch-as-backend" rel="noopener nofollow ugc">repository</a> that seems to do the job(check under the header “Run Caffe models using PyTorch as backend” in readme) but it requires also a .proto file. Any suggestions?</p>
Passing variable number of frames to my LSTM layer

username: Muhammad_Ahmed
cooked: <p>I’m trying to make a video classification model. I pass 1-N frames from a single video. The number of frames from a single video, depends on the size of video. I want to be able to send variable number of frames for a single video. The problem is, how to pass variable number of frames to my LSTM layer. Is padding the only solution for it (I really don’t want to go for padding since the length of videos is ranging from 15 seconds to 6 mins or even more). I also don’t want to keep my batch size constant (i.e:batch size=1).<br>
Please help<br>
Thanks</p>

username: tom
cooked: <p>The typical solution is to group the videos by length to get buy with as little padding as possible. There are various methods of implementing this “bucketized” sampling, TorchText someting like it, I sometimes implement something like it by grouping them in the dataset and then returning batches of data from the dataset (and using batch size 1 in the dataloader and squeezing the extra dimension).</p>
<p>Best regards</p>
<p>Thomas</p>

username: Muhammad_Ahmed
cooked: <p><a class="mention" href="/u/tom">@tom</a> can  you please specify how to apply the padding operation on videos. I do not understand how padding is supposed to help resolve my problem.</p>
Is there a bug with is_pinned. It throws an exception in debug mode

username: Anders_G
cooked: <p>Im programming with torchlib and I create a small tensor on cpu, not pinned. When I call is_pinned I get an exception if Im in Debug mode, but not in Release. I use torchlib (windows) version 1.6.</p>
<p>Has anyone else seen this problem?</p>
<p>// The code:<br>
auto v = torch::zeros({2, 2})<br>
v.is_pinned();</p>

username: glaringlee
cooked: <p><a class="mention" href="/u/anders_g">@Anders_G</a><br>
I think this is something with the Debug mode build flags.<br>
Please create an issue at ‘<a href="https://github.com/pytorch/pytorch/issues" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues</a>’<br>
We will have windows people looking into this, thanks.</p>
Statically linking to libtorch but also needing the shared library "libcaffe2_nvrtc.so"?

username: davidw82
cooked: <p>Hi,<br>
I’m building a plugin for a commercial software application that is internally using libtorch. This plugin, which is in the form of a shared library (.so file), is statically linking libtorch (i.e. it’s baking all of it into it’s own so-file). The only libraries that are dynamically linked against are the CUDA libs and some standard system libraries of course. This plugin needs to be portable to other systems without the need to install lots of dependencies (CUDA is fine to be needed as an additional install).</p>
<p>Everything works, except I can’t get my shared library to not also need the torch library “libcaffe2_nvrtc.so”. Since it doesn’t exist a static version of this library “libcaffe2_nvrtc.a”, I can’t seem to get around it. Is there a way to not have libtorch dynamically depend on this library and still have GPU support? I can build libtorch with CPU support only, and then I don’t have the problem, but I need the GPU acceleration.</p>
<p>My guess here is that the libcaffe2_nvrtc library can’t be made static since it directly depends on CUDA which doesn’t exist statically, but that is only my guess, I and would like somebody more knowledgeable to confirm that this is the case. If so, I guess I just have to live with being forced to also distribute the file “libcaffe2_nvrtc.so” together with my plugin, right? Any other suggestions or solutions?</p>
<p>Cheers, David</p>

username: glaringlee
cooked: <p><a class="mention" href="/u/davidw82">@davidw82</a><br>
I guess you are right. Can you please open an issue in pytorch github here?<br>
‘<a href="https://github.com/pytorch/pytorch/issues" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues</a>’<br>
we will have build expert helping you.</p>

username: davidw82
cooked: <p>Thanks <a class="mention" href="/u/glaringlee">@glaringlee</a>,<br>
I’ve posted an issue here now: <a href="https://github.com/pytorch/pytorch/issues/45603" rel="noopener nofollow ugc">https://github.com/pytorch/pytorch/issues/45603</a></p>
<p>Cheers,<br>
David</p>

username: glaringlee
cooked: <p><a class="mention" href="/u/davidw82">@davidw82</a><br>
We will triage this soon, please track the update in the issue</p>
Save a tensor to file

username: smu226
cooked: <p>Hello! I want to save a tensor to a file but when I do it using file.write(str(tensor)), what it writes is " tensor(-0.0947, device=‘cuda:0’, grad_fn=MeanBackward0". How can I save just the numerical value (-0.0947). thank you!</p>

username: RicCu
cooked: <p>Use <code>torch.save(tensor, 'file.pt')</code> and <code>torch.load('file.pt')</code></p>

username: JuanFMontesinos
cooked: <p>If you have a single number in the tensor, you can get it by using tensor.item()<br>
If you have a multidimensional tensor you can get just the data by doing tensor.data</p>

username: Brando_Miranda
cooked: <p>what is the way to save it such that we get something like this when we load:</p>
<pre><code class="lang-auto">db = load(file_name)
tensor1 = db["tensor1"]
tensor2 = db["tensor2"]
... etc
</code></pre>
<p>I noticed that pickling might not be supported in the future (so <code>dill</code> might not work later).</p>
<p>I am not saving model parameters. Just tensors.</p>

username: RicCu
cooked: <p>You can save a python map:</p>
<pre><code class="lang-auto">m = {'a': tensor_a, 'b': tensor_b}
torch.save(m, file_name)
loaded = torch.load(file_name)
loaded['a'] == tensor_a
loaded['b'] == tensor_b
</code></pre>
<p>This is actually the same thing (with an OrderedDict) that happens when you store a model’s parameters using <code>torch.save(model.state_dict(), file)</code>.</p>

username: Brando_Miranda
cooked: <p>Do you know if it’s better to save the tensors as numpy data or torch tensors data?</p>
<p>Anyone aware of the pros &amp; cons of using <code>numpy.save</code> vs <code>torch.save</code>?</p>
<p>I wonder if that will cause bugs when using the <code>ToTensor()</code> transform if the data is already saved as torch tensors.</p>
<hr>
<p>related: <a href="https://discuss.pytorch.org/t/what-is-the-recommended-format-to-save-data-in-pytorch/89047" class="inline-onebox">What is the recommended format to save data in pytorch?</a></p>
<hr>
<p>fully functionable example:</p>
<pre><code class="lang-auto">#%%

import torch

from pathlib import Path

path = Path('~/data/tmp/').expanduser()

tensor_a = torch.rand(2,3)
tensor_b = torch.rand(1,3)

db = {'a': tensor_a, 'b': tensor_b}

torch.save(db, path/'torch_db')
loaded = torch.load(path/'torch_db')
print( loaded['a'] == tensor_a )
print( loaded['b'] == tensor_b )
</code></pre>

username: Aldebaran
cooked: <p>Is there an approach to load the tensor file into a Dataloader? Because my tensor file is large and I need to iterate over each row.</p>
Triplet mining batch all

username: PCPJ
cooked: <p>Hello everyone. Sorry if it is a stupid question.<br>
I am trying to train a siamese network for speaker identification.<br>
For this task I am trying to train a small CNN with triplet margin loss to generate embeddings to distinguish each speaker.</p>
<p>Before I try to use some hard triplet mining, I want to test a simple <em>batch all</em> triplet approach.<br>
For this I must pass all triplets in my minibatch through the network, and average their losses by the number of non zero losses.</p>
<p>For this I have a doubt. There is any difference between this two segments of code?</p>
<pre><code class="lang-auto">loss = criterion(anchor_feats, pos_feats, neg_feats)
loss = loss[loss!=0].mean()            
loss.backward()
optimizer.step()
</code></pre>
<pre><code class="lang-auto">loss = criterion(anchor_feats, pos_feats, neg_feats)
loss = loss.sum()/((loss!=0).sum())
loss.backward()
optimizer.step()
</code></pre>
<p>Thank you very much.</p>
Very Strange output with Conv2D and ReLU

username: Justin
cooked: <p>Hi, I’m trying to build a model for monocular depth prediction using a DenseNet inspired architecture. DenseNet inspired meaning that there are sections of the network that are comprised of dense blocks of 2d convolutions, specifically 1x1 conv-&gt;ReLU-&gt;3x3 conv -&gt;ReLU.</p>
<pre><code class="lang-auto">def BNBlock(inputSize,stride):
			return nn.Sequential(
					nn.Conv2d(inputSize,inputSize*stride,kernel_size = 1,stride = stride, padding = 0, bias = False),
					nn.ReLU(inplace=False),
					nn.Conv2d(inputSize,inputSize,kernel_size = 3,stride = 1, padding = 1, bias = False),
					nn.ReLU(inplace=False)
					)

def DenseBlock(numLayers, inputSize, stride):
			layers = []
			for i in range(numLayers):
				layers.append(BNBlock(inputSize,stride))
			return(nn.Sequential(*layers))
</code></pre>
<p>The problem I’m having is that when I run data through these dense blocks, no matter how dense it is, the output comes out either nearly entirely zeros or nearly entirely a singular value. Does anybody know why this could be?</p>
<p>Important things to note:</p>
<ol>
<li>The input to the network is normalized between 0-1</li>
<li>I’ve tried replacing the ReLU layers with LeakyReLU, thinking that it could have fixed the “dying ReLU” problem, but that didn’t work either.</li>
</ol>
<p>Any insight as to what’s happening here would be greatly appreciated!</p>

username: ptrblck
cooked: <p>I get an output containing approx. 50% zeros using <code>randn</code> inputs, which seems to be expected:</p>
<pre><code class="lang-python">model = DenseBlock(5, 3, 1)
x = torch.randn(8, 3, 24, 24)
out = model(x)
print((out == 0.).float().sum() / out.nelement())
&gt; tensor(0.4957)
</code></pre>
TypeError: 'generator' object is not subscriptable

username: 111387
cooked: <p>I run the BERT+BILSM+CRF  code with pytorch,when I run the code and it came cross a error as follows:<br>
File “/home/nlp/anaconda2/envs/Bert-BiLSTM-CRF-pytorch/lib/python3.5/site-packages/torch/nn/modules/rnn.py”, line 146, in check_forward_args check_hidden_size(hidden[0], expected_hidden_size,<br>
TypeError: ‘generator’ object is not subscriptable<br>
the error pytorch code  is：</p>
<blockquote>
<pre><code>          if self.mode == 'LSTM':
        check_hidden_size(hidden[0], expected_hidden_size,
                          'Expected hidden[0] size {}, got {}')
        check_hidden_size(hidden[1], expected_hidden_size,
                          'Expected hidden[1] size {}, got {}')
</code></pre>
</blockquote>
<p>my pytorch version is 0.4.1,how can I solve this problem?</p>

username: tom
cooked: <p>it looks like <code>hidden</code> is a generator rather than a tuple of Tensors (probably from the initial state hx in the call to LSTM).<br>
Feeding it a tuple of Tensors might work better.</p>
Why is looping through pytorch tensors so slow (compared to Numpy)?

username: lakshjaisinghani
cooked: <p>I’ve been working with image transformations recently and came to a situation where I have a large array (shape of 100,000 x 3) where each row represents a point in 3D space like:</p>
<pre><code class="lang-python">pnt = [x y z]
</code></pre>
<p>All I’m trying to do is iterating through each point and matrix multiplying each point with a matrix called T (shape = 3 X 3).</p>
<h2>Test with Numpy:</h2>
<pre><code class="lang-python">def transform(pnt_cloud, T):
    
    depth_array = np.zeros(pnt_cloud.shape[0])
    
    i = 0
    for pnt in pnt_cloud:
        xyz_pnt = np.dot(T, pnt)
        
        if xyz_pnt[0] &gt; 0:
            depth_array[i] = xyz_pnt[0]
            
        i += 1
            
        
    return depth_array
</code></pre>
<p>Calling the following function and calculating runtime (using %time) gives the output:</p>
<pre><code class="lang-auto">Out[190]: CPU times: user 670 ms, sys: 7.91 ms, total: 678 ms
Wall time: 674 ms
</code></pre>
<h2>Test with Pytorch Tensor:</h2>
<pre><code class="lang-python">import torch

tensor_cld = torch.tensor(pnt_cloud)
tensor_T   = torch.tensor(T)

def transform(pnt_cloud, T):

    depth_array = torch.tensor(np.zeros(pnt_cloud.shape[0]))

    i = 0
    for pnt in pnt_cloud:
        xyz_pnt = torch.matmul(T, pnt)
        
        if xyz_pnt[0] &gt; 0:
            depth_array[i] = xyz_pnt[0]
            
        i += 1
            
        
    return depth_array
</code></pre>
<p>Calling the following function and calculating runtime (using %time) gives the output:</p>
<pre><code class="lang-auto">Out[199]: CPU times: user 6.15 s, sys: 28.1 ms, total: 6.18 s
Wall time: 6.09 s
</code></pre>
<p>I would have thought that PyTorch tensor computations would be much faster due to the way PyTorch breaks its code down in the compiling stage. What am I missing here?</p>
<p>Other things I’ve tried:</p>
<ol>
<li>Doing the same with torch.jit only reduces 2s</li>
<li>tried torch.no_grad() as I thought I was accumulating gradients (I realized that’s not how it works)</li>
<li>Numba + Numpy jit works the fastest (120ms)</li>
</ol>

username: tom
cooked: <p>Yeah, so two things</p>
<ul>
<li>operations of 1-3 elements are generally rather expensive in PyTorch as the overhead of Tensor creation becomes significant (this includes setting single elements), I think this is the main thing here. This is also the reason why the JIT doesn’t help a whole lot (it only takes away the Python overhead) and Numby shines (where e.g. the assignment to depth_array[i] is just a memory write).</li>
<li>the matmul itself might differ in speed if you have different BLAS backends for it in PyTorch vs. NumPy.</li>
</ul>
<p>In this specific case, you could likely just do <code>depth_array = torch.matmul(pnt_cloud, T.t()).clamp_(min=0)</code> or so. Similarly in numpy.</p>
<p>Best regards</p>
<p>Thomas</p>
<p>(PS: it would help if you just added dummy data to your code examples so people could just copypaste to look into the benchmarking.)</p>
Difficulties calculating mean square error between 2 tensors

username: univ
cooked: <p>I’m trying to build a loss function which will calculate the mean squared error of 2 tenors of the same size. In other words I need a function that calcualtes the difference of every 2 cells (with the same row and column) cell on matrix A and matrix B, squares it and calculate the mean of the differences.</p>
<p>As far as I understand nn.MSELoss should do exactly that.</p>
<p>When I pass the 2 tensors to nn.MSELoss I’m getting the following error message:</p>
<p><code>RuntimeError: Boolean value of Tensor with more than one value is ambiguous</code></p>
<p>Here’s my code</p>
<p><code>nn.MSELoss(stack_3[0,:],stack_7[0,:])</code></p>
<p>The tensors are floats of same shape.</p>
<pre><code class="lang-auto">stack_3.shape, stack_7.shape
(torch.Size([6131, 784]), torch.Size([6131, 784]))
</code></pre>

username: ptrblck
cooked: <p>You would have to create an object for the loss function and use it afterwards:</p>
<pre><code class="lang-python">criterion = nn.MSELoss()
loss = criterion(stack_3[0, :], stack_7[0, :])
</code></pre>
How to Parallelize Convolution Operation?

username: soulslicer
cooked: <p>I have the following Kernel:</p>
<pre><code class="lang-auto">{'padding': 2, 'weight': tensor([[[[0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.],
          [1., 1., 1., 1., 1.],
          [0., 0., 0., 0., 0.],
          [0., 0., 0., 0., 0.]]]], device='cuda:0')}
</code></pre>
<p>and I have an image of size [B, 320, 256, 128]</p>
<p>I want to apply this kernel to each Channel (treat each channel as a 2D image and apply a standard 2D convolution)</p>
<p>Currently I do it like this, but it is very slow:</p>
<pre><code class="lang-auto">    for b in range(0, dpv_permuted.shape[0]):
        for c in range(0, dpv_permuted.shape[1]):
            dpv_permuted[b,c,:,:] = F.conv2d(dpv_permuted[b,c].unsqueeze(0).unsqueeze(0), **spread_kernel).squeeze(0).squeeze(0)
</code></pre>
<p>How do i make this faster?</p>

username: soulslicer
cooked: <pre><code class="lang-auto">    test = torch.nn.Conv2d(in_channels=320, out_channels=320, groups=320, kernel_size= (5,5), padding=5//2, bias=False).cuda()
    test.weight.requires_grad = False
    test.weight[:,:,:,:] = kernel
    out = test(dpv_permuted)
</code></pre>
Pick the n best weights

username: slavavs
cooked: <p>Can I store the weights for each model, then pick the n best, average them and apply to the model?</p>

username: ptrblck
cooked: <p>You could store multiple <code>state_dict</code>s and afterwards load them and create the average <code>state_dict</code>.<br>
A small example for two different <code>state_dict</code>s is given <a href="https://discuss.pytorch.org/t/average-each-weight-of-two-models/77008/2">here</a>.</p>
Increase 40% data by data augmentation

username: Cuong_Nguyen
cooked: <p>I want to use data loader to increase the number of images in my dataset by 40% using data augmentation. How can I do it ?</p>

username: Chame_call
cooked: <p>Related links:</p>
<ul>
<li><a href="https://pytorch.org/docs/stable/torchvision/transforms.html" rel="noopener nofollow ugc">transforms documentation</a></li>
<li><a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html" rel="noopener nofollow ugc">official tutorial on this topic</a></li>
</ul>
How to print at::tensor in C++

username: alanzhai219
cooked: <p>How to print at::tensor in C++ during debugging a custom op?<br>
1.code line?<br>
2.gdb?</p>

username: albanD
cooked: <p>You can use <code>std::cout &lt;&lt; your_tensor &lt;&lt; std::endl;</code> to print the content of a Tensor.</p>

username: dhpollack
cooked: <p>Is there a way of printing the size of a tensor without iterating through all elements of <code>size()</code>?</p>
<p>So doing something like <code>cout &lt;&lt; t.size() &lt;&lt; end</code>.</p>

username: albanD
cooked: <p>I believe <code>std::cout &lt;&lt; self.sizes() &lt;&lt; std::endl;</code> will work no?</p>

username: alanzhai219
cooked: <p>Thanks for your reply. Indeed, tensor impl has some attributes of tensor.</p>

username: solarflarefx
cooked: <p>How about if you would only like to print a part of a tensor?  For example, if you have a tensor of size {1,1,224,224} what if you wanted to index specific elements?</p>

username: DuaneNielsen
cooked: <p>indexing works!</p>
<pre><code class="lang-auto">std::cout &lt;&lt; system -&gt; m_q[0] &lt;&lt; std::endl;
</code></pre>
<p>0<br>
[ CPUFloatType{1,1} ]</p>
Permutation not working in Quantized Model

username: matteo-ronchetti
cooked: <p>I have a model that contains a custom permutation that I need to apply to the second dimension of a Tensor, this is implemented as</p>
<pre><code class="lang-auto">def forward(self, x: torch.Tensor):
    return x[:, self.permutation]
</code></pre>
<p>where <code>self.permutation</code> is a LongTensor.<br>
When the model is not quantized (x is a FloatTensor) everything works correctly, when I quantize the model I get the following error:</p>
<pre><code class="lang-auto">RuntimeError: Could not run 'aten::empty.memory_format' with arguments from the 'QuantizedCPU' backend. 'aten::empty.memory_format' is only available for these backends: [CPU, CUDA, MkldnnCPU, SparseCPU, SparseCUDA, BackendSelect, Autograd, Profiler, Tracer]
</code></pre>
<p>It seems that the operation is not implemented, I’m using PyTorch 1.6.0.<br>
Is there any alternative permutation operation that I can use?</p>
<p>Thanks,<br>
Matteo</p>

username: hx89
cooked: <p>Maybe you can work around it by adding dequantize before this module and quantize after so that it’s not quantized?</p>

username: matteo-ronchetti
cooked: <p>Yes it is a possible workaround but, because I need the permutation in many layers (I’m using a variant of ShuffleNet), I think the performance will suffer.</p>

username: hx89
cooked: <p><a class="mention" href="/u/jerryzh168">@jerryzh168</a> <a class="mention" href="/u/raghuramank100">@raghuramank100</a> do you know if this is supported in PyTorch quantization?</p>

username: jerryzh168
cooked: <p>I’m not sure what is the operator used here, is it slice?</p>
<p>Please open an issue in <a href="https://github.com/issues" rel="noopener nofollow ugc">https://github.com/issues</a> and provide a repro</p>
Best practice to index one tensor of look up tables by another tensor

username: deJQK
cooked: <p>I have one tensor of look up tables, say <code>lut</code> with shape [15, 6], where the values are all in the range of [0, 5], and can map each index to a value in each table (say the first table lut[0] will map 3 to lut[0][3]). I have one tensor of shape [7, 2, 3, 14, 14, 5] (with values in the range of [0, 5]), and I want each one of the third and last dims (corresponding to 3 and 5) to use one table for mapping. I wonder if there is some elegant and efficient way to implement this, rather than permute and list(map(*)). Thanks.</p>
<p>I noticed <a href="https://discuss.pytorch.org/t/mapping-one-tensor-of-values-and-another-tensor-of-indices-to-a-third-tensor-by-index/65854/2">this topic</a>, but it does not seem to be the same problem.</p>

username: deJQK
cooked: <p>torch.index_select is faster than what I implemented.</p>
Lstm with different input sizes for each batch

username: dan_as
cooked: <p>Hello everyone ,</p>
<p>I have made a Collator that pads each tensor(originally sentence) in each batch into the maxlen of each batch.</p>
<p>so I have different input sizes per each batch.</p>
<p>my question is, how can I specify this to the LSTM?</p>
<p>Thank you in advance</p>
Multi-label classification, model learns negative classes

username: serdarrader
cooked: <p>I am trying to build multi-label classification model using pascal-2007 dataset. There are 20 classes. The accuracy of my model first seemed good, but later I found out that model learned to predict values close to 0 for all the classes since binary cross entropy (with logits) awarding predicting absence of classes too, and I suspect that since, for a given image, there is more absent classes than ones that are in the picture, loss reduces when model predict none of them in the picture.</p>
<p>I tried to give more weight to the present classes but it didn’t work well. Everybody seems to be happy with just binary cross entropy losses, is it just me having this problem ? And if it is, can someone explain to me how bce losss overcome this problem, because for me it is logical for it to pick those in this case. Also how can I overcome this ?</p>

username: tom
cooked: <p>I think giving more weight to positive classes can work in general, but there might be reasons it does not work for you. If some classes are very rare, you might look into oversampling them. In the context of detection, people have used focal loss to get around.</p>
<p>Best regards</p>
<p>Thomas</p>

username: serdarrader
cooked: <p>I am not using transfer learning and there is 2k image in training set while there are 20 classes. Can it also result from not enough data ? Maybe if there are more data, model will see more present ones etc…</p>

username: tom
cooked: <p>Well, if your training method required more data than your dataset had and everything else worked, you would see the network reach very good accuracies on the training set and bad on the validation set.</p>

username: serdarrader
cooked: <p>Yes but I suspect in this case I don’t see it because negative class predictions are increasing accuracy anyway</p>
Is it possible to build a minimal libpytorch.so for c++ inference on mobile device?

username: WangYong
cooked: <p>Hi,</p>
<p>For now we plan to follow the NativeApp demo in <a href="https://github.com/pytorch/android-demo-app" rel="noopener nofollow ugc">https://github.com/pytorch/android-demo-app</a>, to support inference on mobile devices. After OPs reduction we still get  libpytorch_jni.so with size about 11M for arm64-v8a. So,</p>
<ol>
<li>Is it possible to reduce it further? I have tried turned off some cmake options including USE_MKLDNN, but it seems no effect.</li>
<li>Is there any other clean way to build a c++ dedicated library like libtorch_cpu.so getting rid of jni stuff, to support c++ native linkage.</li>
</ol>
<p>Thanks!</p>

username: tom
cooked: <p>Yes, the <a href="https://pytorch.org/mobile/android/#custom-build" rel="noopener nofollow ugc">custom build functionality</a> allows you to build a libtorch with just the PyTorch ops you actually need.</p>
<p>Best regards</p>
<p>Thomas</p>

username: WangYong
cooked: <p>Thanks for reply!</p>
<p>Except operator customization, is it possible to allow reducing the binary size further, e.g., via the build options?</p>
<p>Thanks.</p>

username: tom
cooked: <p>Most of these should already be turned off for PyTorch mobile.</p>

username: WangYong
cooked: <p>Get it, thanks a lot!</p>
<p>BTW, do we have existing option to support building a native library for dedicated c++ linkage, getting rid of jni stuff.</p>
<p>Thanks!</p>
Autograd for relu over a part of tensor

username: Sharpy
cooked: <p>Please, advise how to make this work.</p>
<pre><code class="lang-auto">class HalfReLU(nn.Module):
	def __init__(self):
		super(HalfReLU, self).__init__()

	def forward(self, x):
		half = x.shape[1]//2
		x[:, :half] = nn.functional.relu(x[:, :half], inplace=False)
		return x
</code></pre>
<p>I’m getting</p>
<blockquote>
<p>RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [128, 128, 8, 8]], which is output 0 of SliceBackward, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!</p>
</blockquote>
<p>Whether relu is inplace or not doesn’t matter. Works (or just doesn’t give errors, don’t know for sure) in case of things like</p>
<pre><code class="lang-auto">		x[:, :half] *= 2
</code></pre>
<p>and</p>
<pre><code class="lang-auto">		x[:, :half] = nn.functional.sigmoid(x[:, :half])
</code></pre>
<p>but not in case of relu. I’m guessing it’s sort of a bug/edge case? For leaky_relu also doesn’t work.</p>

username: tom
cooked: <p>I’d just use torch.cat of the pristine and relued parts.</p>

username: Sharpy
cooked: <p>Works. Very slow vs plain ReLU, though. But even simply</p>
<pre><code class="lang-auto">x = nn.functional.relu(x[:, :], inplace=True)
</code></pre>
<p>is itself halfway slower vs</p>
<pre><code class="lang-auto">x = nn.functional.relu(x, inplace=True)
</code></pre>

username: tom
cooked: <p>Numbers? (input sizes, cpu vs. gpu, …)</p>
<p>For me and a decently sized array, it doesn’t matter:</p>
<pre><code class="lang-python">x = torch.randn(1000,1000)
%timeit torch.nn.functional.relu(x[:, :], inplace=True)
71.8 µs ± 59.1 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)

%timeit torch.nn.functional.relu(x, inplace=True)
69 µs ± 141 ns per loop (mean ± std. dev. of 7 runs, 10000 loops each)
</code></pre>
<p>If relu is slowing down your network, you’re doing something wrong.</p>
<p>You can be a tiny bit faster than relu + cat if you clone + use inplace relu (which works with autograd). But TBH I’d stay in the functional world. Optimizing ReLU likely isn’t a huge win.</p>

username: Sharpy
cooked: <p>Interesting. I’ve tried changing that for a large network, so I can’t say what specific size it was slow for. Regarding slowdowns and autograd, what can you say about TensorFlow’s XLA performance? From benchmarks I’ve seen it seems to deliver quite fantastic optimizations. And that’s with partly due to optimizing things like relu (to cut on memory movements in this case), right? Relus and stuff seem to matter much more in the Tensor Core days than they did before Volta/Turing, especially on unlocked GPUs (not 20xx, 30xx series).</p>
Calculating the accuracy metric

username: bing
cooked: <p>Hi guys,<br>
I am trying to calculate the accuracy of my trained model(for multi-class image classification).<br>
My labels are <code>1, 2, 3, 4</code>.<br>
My <code>__getitem__</code> in data loader class looks like below -</p>
<pre><code> def __getitem__(self, index):
    """Generate one sample of data."""
    ID = self.list_IDs[index]
    image = Image.open(os.path.join(self.dir, ID))
    y = self.labels[ID]
    if self.train_transform:
        image = self.train_transforms(image)
    else:
        image = self.valid_transforms(image)
    img = np.array(image)
    return img, y
</code></pre>
<p>My test accuracy and loss function looks like below -</p>
<pre><code>for batch_idx,(data,target) in enumerate(test_loader):
    output = model_transfer(data).squeeze()
    output = torch.unsqueeze(output,0)
    loss = criterion(output, target)
    avg_valid_loss += (1/(batch_idx+1)) * ((loss.data) - avg_valid_loss)
    pred = output.data.max(1, keepdim=True)[1]
    correct += np.sum(np.squeeze(pred.eq(target.data.view_as(pred))).cpu().numpy())
    total += data.size(0)

 accuracy = 100.*(correct/total)
</code></pre>
<p>Am I calculating the accuracy correctly?<br>
Also, regarding the labels are <code>1, 2, 3, 4</code>, should I convert them into some form of one-hot encoding or Is network able to predict it generally?<br>
Any other suggestion regarding it is also welcome?</p>
Different outputs on the same input to the pytorch torchscript model model

username: ZeeshanH
cooked: <p>I am trying to deploy bert with elastic inference on sagemaker.<br>
It requires converting BERT torch model to torchscript. When I converted pytorch model to torchscript and predicted on it got different output for each run of same input to the model.<br>
Pytorch version=1.3.1<br>
Following are the outputs from the model for same input.</p>
<pre><code class="lang-auto">1: [[-4.0112, -4.7550, -4.6516, -3.7064, -3.3309, -5.2377, -7.5709, -7.8646,
-9.1199, -8.8557, -6.8730, -2.2870, -2.9781, -4.6967, -5.6849, -5.1054,
-8.1800, -7.7518, -3.2441]]

2: [[-4.2761, -4.5443, -4.6977, -2.6534, -2.8268, -5.3280, -7.1985, -7.1800,
-9.5470, -8.7595, -6.3672, -1.5636, -3.0138, -4.7585, -6.3858, -5.0327,
-8.6203, -8.0351, -3.9619]]

3: [[-4.5412, -4.4034, -4.4401, -2.7413, -3.2421, -6.0487, -7.3131, -7.4065,
-9.4242, -8.2876, -6.2156, -1.1610, -3.7632, -5.3920, -6.9708, -4.7110,
-8.0398, -8.1571, -4.3893]]
</code></pre>
<p>Prediction function is as follows:</p>
<pre><code class="lang-auto">def predict_fn(input_data, model):
    model.eval()
    print('Generating prediction based on input parameters.')
    device = 'cuda' if cuda.is_available() else 'cpu'
    try:
        with torch.no_grad():
            with torch.jit.optimized_execution(True, {"target_device": "eia:0"}):
                outputs = model(input_data["ids"],input_data["mask"], input_data["token_type_ids"])
    except Exception as ex:
        exc_type, exc_obj, exc_tb = sys.exc_info()
        fname = os.path.split(exc_tb.tb_frame.f_code.co_filename)[1]
        raise Exception(f'3- Error in predict_fn {exc_type, fname, exc_tb.tb_lineno, input_data["ids"].shape, " with exception message: ", ex}')    
    outputs = torch.sigmoid(outputs).cpu().detach().numpy()
    return outputs
</code></pre>
<p>I don’t understand why I am getting this odd behavior from torchscript converted model. Whereas simple pytorch model which is not converted into torchscript gives same output on each run.</p>
Higher-order derivatives of torch::nn::Module

username: mathmerizing
cooked: <p>Hello! I was trying to translate my PyTorch code into LibTorch.<br>
Using the following PyTorch code I was computing second-order derivatives of nn.modules.</p>
<pre><code class="lang-auto">import torch
from torch.autograd import Variable

def derivative(u,x,order = 1):
    ones  = torch.ones_like(u)
    deriv = torch.autograd.grad(u, x, create_graph=True, grad_outputs=ones)[0]
    for i in range(1,order):
        ones  = torch.ones_like(deriv)
        deriv = torch.autograd.grad(deriv, x, create_graph=True, grad_outputs=ones)[0]
    return deriv
</code></pre>
<p>Now I am trying to write the same function in LibTorch:</p>
<pre><code class="lang-auto">torch::Tensor derivative(torch::Tensor u, torch::Tensor x, int order = 1)
{
  torch::Tensor ones = torch::ones_like(u);
  torch::Tensor deriv = torch::autograd::grad({u}, {x}, /*grad_outputs=*/{ones}, /*create_graph=*/true, /*allow_unused=*/false)[0].requires_grad_(true);
  for (int i = 0;i &lt; order; i++)
  {
    torch::Tensor ones = torch::ones_like(deriv);
    deriv = torch::autograd::grad({deriv}, {x}, /*grad_outputs=*/{ones}, /*create_graph=*/true, /*allow_unused=*/false)[0];
  }
  return deriv;
}
</code></pre>
<p>If I omit, the <code>requires_grad_(true)</code> I get an error that the Tensor <code>deriv</code> does not require a gradient, although this was the case in PyTorch… Thus I added <code>requires_grad_(true)</code>.<br>
Now I get the error:</p>
<pre><code class="lang-auto">One of the differentiated Tensors appears to not have been used in the graph. Set allow_unused=True if this is the desired behavior.
</code></pre>
<p>I don’t want to set <code>allow_unused=true</code> in my C++ code, since then I don’t think that my second derivatives will be correct anymore. In Python it all worked with <code>allow_unused=False</code>.</p>
<p>How can I fix my errors and compute the second derivative of a torch::nn::Module in LibTorch correctly?</p>

username: tom
cooked: <aside class="quote no-group" data-username="mathmerizing" data-post="1" data-topic="97751">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/87869e/48.png" class="avatar"> mathmerizing:</div>
<blockquote>
<p>I don’t want to set <code>allow_unused=true</code> in my C++ code, since then I don’t think that my second derivatives will be correct anymore. In Python it all worked with <code>allow_unused=False</code> .</p>
</blockquote>
</aside>
<p>This sounds very fishy and I would double check.</p>

username: mathmerizing
cooked: <p>Yes, you are right. It was an error on  my side, I was computing the third derivative instead of the second derivative… Now everything works properly</p>

username: tom
cooked: <p>Sounds like an error I’d put in my code, too. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> I’m glad you solved it.</p>
Write Custom Loss Function

username: lakehanne
cooked: <p>I want to define a new loss function similar to the one defined in the action recognition using visual attention <a href="https://arxiv.org/pdf/1511.04119.pdf" rel="nofollow noopener">paper</a> i.e. eq. 7.</p>
<p>I thought it would be useful to use the NLL Loss defined in PyTorch as a guide but I can’t seem to find the source code. What I found on github was <a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/functional.py#L500" rel="nofollow noopener">this</a> but I do not see its implementation [here] (<a href="https://github.com/pytorch/pytorch/blob/master/torch/nn/_functions/thnn/loss.py" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/master/torch/nn/_functions/thnn/loss.py</a>).</p>
<p>Would anyone be kind enough to post the link to the source code?</p>

username: fmassa
cooked: <p>Most of the functions are implemented in C / CUDA. For reference, here is the <a href="https://github.com/pytorch/pytorch/blob/master/torch/lib/THNN/generic/ClassNLLCriterion.c">C implementation of NLLLoss</a></p>

username: jekbradbury
cooked: <p>But you don’t have to use that in your custom loss function; you should just implement an <code>nn.Module</code> with ordinary torch math ops in the <code>forward</code> method.</p>

username: RJaikanth
cooked: <p>Sorry I am a bit late to the conversation,  but will there be any performance issues if we just implement it fully in torch ops?</p>
Conda installed cudatoolkit==10.1 but torch.version.cuda returns as 10.2

username: cskarthik7
cooked: <p>I have CUDA=10.1 installed<br>
And I was trying to install maskrcnn-benchmark of pytorch so I created a conda environment and installed ```<br>
conda install pytorch==1.5.1 torchvision==0.6.1 cudatoolkit=10.1 -c pytorch</p>
<pre><code class="lang-auto">And now when I print torch.version.cuda it returns me 10.2 therefore compiling nvidia apex gives me a runtime error!

Please help @ptrblck !</code></pre>

username: ptrblck
cooked: <p>Could you check, if you have multiple PyTorch installations on your system and if so delete them and only install the desired one?</p>
<p>What’s your use case that you want to install an older PyTorch version and want to use apex?<br>
Note that PyTorch <code>1.6</code> ships with native mixed-precision training via <code>torch.cuda.amp</code> and is the recommended approach.</p>
How to handle imbalance dataset on text classification

username: smi222
cooked: <p>Hello Everyone,</p>
<p>I am working on binary text classification problem. How do i apply smote or WeightedRandomSample for the imbalance in my dataset.My code currenlty looks like this:</p>
<pre><code class="lang-auto">class GDataset(Dataset):

  def __init__(self, passage, targets, tokenizer, max_len):
    self.passage = passage
    self.targets = targets
    self.tokenizer = tokenizer
    self.max_len = max_len
  
  def __len__(self):
    return len(self.passage)
  
  def __getitem__(self, item):
    passage = str(self.passage[item])
    target = self.targets[item]
    
    if (target == 1) and self.transform: # minority class
            x = self.transform(x)

    encoding = self.tokenizer.encode_plus(
      passage,
      add_special_tokens=True,
      max_length=self.max_len,
      return_token_type_ids=False,
      pad_to_max_length=True,
      return_attention_mask=True,
      return_tensors='pt',
    )
    return
      'passage_text': passage,
      'input_ids': encoding['input_ids'].flatten(),
      'attention_mask': encoding['attention_mask'].flatten(),
      'targets': torch.tensor(target, dtype=torch.long
</code></pre>
<p>How can i apply other balancing techniques</p>

username: ptrblck
cooked: <p>If each sample contains a single target value, you should be able to directly use the <code>WeightedRandomSampler</code> as described in <a href="https://discuss.pytorch.org/t/how-to-handle-imbalanced-classes/11264/2">this example</a>.</p>
Pointer Arithmetic w/ STDVector/Array of Torch Tensors of Different Dimensions

username: Hiperdyne19012
cooked: <p>Can I create a vector or array of Torch tensors with differing dimensions as long as they have the same data type? I want to use pointer arithmetic to access the tensors.</p>

username: tom
cooked: <p>Yes, <code>std::vector&lt;Tensor&gt;</code> is a standard thing and also used in PyTorch itself for representing lists of tensors. When passing stuff around, the TensorList, an ArrayRef to Tensor often comes in handy.</p>
<p>Best regards</p>
<p>Thomas</p>

username: Hiperdyne19012
cooked: <p>So if I push a tensor of dim 2x20x200 into the std::vector vector then later I push a tensor of dim 999x999x999 into it, there will be no issues?</p>

username: tom
cooked: <p>After a few 4GB tensors (assuming 32 bit values), you might run out of memory, but other than that, yes, it works.</p>
Create a Bidirectional Stacked LSTM with Deep Hidden to Hidden Transition

username: simon1
cooked: <p>Hello,<br>
I’d like to build with Pytorch a Bidirectional Stacked LSTM ( Stacked DT-RNN)with fully connected layers between the hidden states as suggested in  Pascanu, Razvan, Gulcehre, Caglar, Cho, Kyunghyun, and Bengio, Yoshua. How to Construct Deep Recurrent Neural Networks. arXiv<br>
preprint arXiv:1312.6026, pp. 1–10, 2013.<br>
Pythorch  <strong>nn.LSTM</strong> don’t support this features: you don’t have the control on each time step. So you can’t  add a fully connected network which takes in input the hidden state  at time t and  gives an output for the hidden state at time t+1  . So i suppose that the usage of <strong>nn.LSTMCel</strong>l is the only option.  At this point many issues born:</p>
<ol>
<li>LSTM cell  implementation is not bidirectional and it works on single timesteps. So if i’d like to build a 2 layer LSTM i need 2  LSTM cell and 4 for loops in the forward method to iterate over the sequences ( one for the original sequence and one for the reversed one for each layer). Operations which i am constrained to do sequentially, but that are strongly parallelizable. It would be very slow.</li>
<li>The input sequences, which i use, have different lenghts; So i have padded them. LSTM cell don’t support pad_packed_sequences, because it works on  a single time step. I’d like to work with batch samples to have reasonable time performances. I have tried to get the the resulting  tensor from torch.nn.utils.rnn.pack_padded_sequence(myBatch)  and feed it into a single LSTM cell, grouping them in smaller batches (let us call  them “mini_batches”), whose length vary with the length of sequences. In this case the issue is that the mini_batches have different sizes and consequently the hidden state size to feed in the hidden-to-hidden transition too.<br>
Do you have any suggestions to solve these issues?<br>
Thank you in advance</li>
</ol>

username: Avi_Chawla
cooked: <p>Hey</p>
<p>I am looking for an implementation of the DT-RNN but facing the same issues are you are. Just wanted to ask if you have found any solution to this?</p>
Some gpus take 100% gpu utilization; while some 0

username: amsword
cooked: <p>The training task is image classification on imagenet 22k dataset. The images are organized as one file which contains 1.4TB data. Initially, the process needs to download the whole data after init_process_group and before model forward/backward. The download normally takes 3 hours+. Actually, it is using microsoft azure blobfuse. In this case, i find some GPU in some nodes takes 100% utilization; while some nodes takes 0% GPU utilization. The training also never proceeds. Pytorch version is 1.6. Are there any potential issues in pytorch under this special situation?</p>
How to save computation graph in pytorch?

username: freezek
cooked: <p>Is there any method to save computation graph in pytorch? Just like tensorflow could save graph…</p>

username: tom
cooked: <p>The typical thing is to use the JIT to trace or script your model. This is - at a very high level - somewhat similar as the computational graph in TensorFlow.</p>

username: freezek
cooked: <p>Thanks for reply, I think it’s really helpful for me!</p>