akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Tensor.long() precision problem

kuangliu: <pre><code class="lang-auto">a=torch.tensor([2.5])
b=(a-2.4)/0.1
print(b)
print(b.long())
</code></pre>
<p>outputs:</p>
<pre><code class="lang-auto">tensor([1.0000])
tensor([0])
</code></pre>
<p>Seemed .long() has a numeric precision problem.<br>
Any ideas?</p>

Tony-Y: <p><a href="https://pytorch.org/docs/stable/torch.html#torch.round" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/torch.html#torch.round</a></p>

colesbury: <aside class="quote no-group" data-username="kuangliu" data-post="1" data-topic="44501">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kuangliu/48/1637_2.png" class="avatar"> kuangliu:</div>
<blockquote>
<pre><code class="lang-auto">Seemed .long() has a numeric precision problem 
</code></pre>
</blockquote>
</aside>
<p>No – that’s just how floating-point arithmetic works. <code>print(b)</code> rounds to 4 decimal points. If you print out <code>b.item()</code> you’ll see: <code>0.999999046325683</code> which rounds down to zero.</p>
<p>See <a href="https://www.exploringbinary.com/why-0-point-1-does-not-exist-in-floating-point/" class="inline-onebox">Why 0.1 Does Not Exist In Floating-Point - Exploring Binary</a> for more explanation.</p>
How I can use a tensor in torch.nn.threshold

isalirezag: <p>I am wondering if I can use a tensor in the <a href="https://pytorch.org/docs/stable/nn.html#torch.nn.functional.threshold" rel="nofollow noopener">threshold</a> function.<br>
I read this which kinda post the same question in the past but it is confusing.<br>
The <code>torch.nn.functional.threshold(input, threshold, value, inplace=False)</code> take input and a threshold.<br>
if i use threshold = torch.tensor(1.) it shoots an error. but i wanna use tensor so i can keep the gradient for back-propagation.<br>
Does anyone have any suggestion?</p>
Backward with two networks

wentao: <p>I have two net works A and B, the output is out = A(x1) + B(x2), it is a matrix. Now I want use out.backward(matrixj). I know there should be a maxtrxj, which has the same size with out, but I’m not sure if it is ok I just use a matrix with all element is 1.</p>

Oli: <p>It’s possible to do the backward with two networks. You have to create a loss function before you can do that though. Once you’ve calculated the loss you can do <code>my_loss.backward()</code></p>
<p>See the pytorch <a href="https://pytorch.org/tutorials/beginner/pytorch_with_examples.html" rel="nofollow noopener">examples</a> for how it’s done with one network. It’s pretty much the same with two networks <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

wentao: <p>Hi, thank you for your reply, but I think you maybe misunderstand me, I know we can use loss function for the back propagation, but what I’m doing now is use the matrix for the back propagation, I want use some thing like Jacobi matrix, I want to know if I can set all element is 1 in matrixj</p>

Oli: <p>Okay I see. Never done that myself. Good luck <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
Different losses for different images in the same batch

smani: <p>I would like to use different loss functions for different images which are in the same batch. For example, lets say my batch size is 8, and for the images 1,2 and 8 I am going to use Loss 1, and for the rest of the images I am going to use Loss 2. Assume that both are cross-entropy losses. Is there any way to easily implement it using PyTorch? In other words I have to select which loss should be used for each of the images in a particular batch.</p>

ptrblck: <p>You could slice the output and target in order to pass each chunk to the desired loss function.<br>
Afterwards you could sum or average these losses and call <code>backward</code> on the reduced loss.</p>
Torchvision.transforms RandomHorizontalFlip - only parts of the image are flipped

jphreid: <p>Hello,</p>
<p>I am trying to implement transformation to a set of images.</p>
<p>x = Normalized [0,1] FloatTensor with dimensions of [40479, 3, 40, 40] where 3 corresponds to the channel RGB.</p>
<p>I created a class (which I found on the web):</p>
<pre><code class="lang-auto">class CustomTensorDataset(Dataset):
    """TensorDataset with support of transforms.
    """
    def __init__(self, tensors, transform=None):
        assert all(tensors[0].size(0) == tensor.size(0) for tensor in tensors)
        self.tensors = tensors
        self.transform = transform

    def __getitem__(self, index):
        x = self.tensors[0][index]      
        if self.transform:
            x = self.transform(x)
        y = self.tensors[1][index]

        return x, y

    def __len__(self):
        return self.tensors[0].size(0)
</code></pre>
<p>I then define the transformations</p>
<pre><code class="lang-auto">transformations = transforms.Compose([
    transforms.ToPILImage(), 
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor()
])
</code></pre>
<p>Finally, I create the dataset with:</p>
<p><code>data_transform = CustomTensorDataset(tensors=(xx, yy), transform=transformations)</code></p>
<p>See the figure below. The top image is the original and the bottom image is the transformed image. As you can see, only the left and right panels are horizontally flipped. Is that normal?</p>
<p>Help would be appreciated. Merci!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/1/146a554031b599c9f2a09eeff0c4dd05bee94334.png" data-download-href="https://discuss.pytorch.org/uploads/default/146a554031b599c9f2a09eeff0c4dd05bee94334" title="41%20PM"><img src="https://discuss.pytorch.org/uploads/default/original/2X/1/146a554031b599c9f2a09eeff0c4dd05bee94334.png" alt="41%20PM" data-base62-sha1="2UBmK6MJEhR0NVfrLenepCMQe8I" width="372" height="500" data-dominant-color="BCB6AB"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">41%20PM</span><span class="informations">389×522 16.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

ptrblck: <p>That doesn’t seem to be alright, since your image was somehow cut into these “panels”.<br>
Are you using another transformation or do you process the image somehow, to create these panels?<br>
Also the color color channels seem to be divided.</p>
<p>Is your posted code snippet the whole code you are using to create these transformed images?</p>
Which set of CNN Model parameters are selected after training for evaluating/testing?

UsamaMaq: <p>Hello</p>
<p>I was following this <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" rel="nofollow noopener">tutorial</a>. I have a question that after training for multiple epochs and over all mini batches of complete training set. Which set of weights are selected for the network when we are testing it.</p>
<p>Is it the weights of the epoch with lowest running loss or the last epoch?</p>
<p>If it is the weight for last epoch which function should i use to save the model parameters for lowest running loss or epoch loss and use that for testing?</p>

ptrblck: <p>Often you would want to save the <code>state_dict</code> of your model for the lowest validation loss (or highest validation accuracy).<br>
Depending on your use case, you could split the training data into a training and validation set (e.g. using <code>SubsetRandomSampler</code> or <code>Subset</code>), observe the validation loss, and save the best model.<br>
After saving you could calculate the test accuracy to get a signal of your model’s performance on unseen data.</p>

UsamaMaq: <p>Ok. Just to confirm. After training through each epoch i should run the validation set through the model and see validation loss. And should use the model for least validation loss after training through all epochs is complete. Right?</p>
<p>Also if my above lines are not coded in algo, model will use parameters that are set from the last training epoch for testing. As i referred to the tutorial same is being done in that. Weights for last epoch are being used for testing. Am i missing something in this?</p>
Trouble with ConcatDataset()

ougni: <p>Hi Everyone,</p>
<p>I’m trying to understand the ConcatDataset()  function. I’m running this example code from here:<br>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/pytorch/examples/blob/master/mnist/main.py" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/examples/blob/master/mnist/main.py" target="_blank" rel="nofollow noopener">pytorch/examples/blob/master/mnist/main.py</a></h4>
<pre><code class="lang-py">from __future__ import print_function
import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 20, 5, 1)
        self.conv2 = nn.Conv2d(20, 50, 5, 1)
        self.fc1 = nn.Linear(4*4*50, 500)
        self.fc2 = nn.Linear(500, 10)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2, 2)
</code></pre>

  This file has been truncated. <a href="https://github.com/pytorch/examples/blob/master/mnist/main.py" target="_blank" rel="nofollow noopener">show original</a>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>The only change I’m making here is this:</p>
<pre><code class="lang-auto">train_mnist = datasets.MNIST('data', download=True, train=True, transform=transforms.Compose([transforms.ToTensor(),
                              transforms.Normalize((0.1307,), (0.3081,)),
                              ]))
new_mnist = torch.utils.data.TensorDataset(train_mnist.data.unsqueeze(1), train_mnist.targets)
trainset = torch.utils.data.ConcatDataset([new_mnist]) 
train_loader = torch.utils.data.DataLoader(trainset, batch_size=args.batch_size, shuffle=True)
</code></pre>
<p>Therefore, instead of passing the <code>train_mnist</code> to the <code>DataLoader</code> - I’m creating a <code>tensorDatset</code> and then using <code>ConcatDataset</code> to see if I get the same train/test behavior. Surprisingly, the training/testing behavior is very different - instead of getting high 90% accuracy in testing, testing accuracy drops to 10%. Something is definitely wrong!  Anyone knows whats going on?</p>

ptrblck: <p>The transformation will be lost, if you pass the <code>train_mnist.data</code> directly to <code>TensorDataset</code> as it is applied in the <code>__getitem__</code> method for each <code>.data</code> sample. You could try to normalize the data inplace before passing it to the <code>TensorDataset</code>.</p>
What is the right way to train a network for two tasks simultaneously?

Ozan: <p>Hi,</p>
<p>I am training a network for two separate tasks. My feature extractor F (e.g., resnet) features are fed two small networks with fully connected layers (and some nonlinearities),  T1 and T2. So,</p>
<p>T1(F(Input1)) = Output 1<br>
T2(F(Input2)) = Output 2</p>
<p>I’ve set up the model such that its input is input1 and input2:</p>
<p>pred1, pred2 = model(input1, input2)</p>
<p>loss = lossfn(pred1, label1) + lossfn(pred1, gt2)<br>
optimizer.zero_grad()<br>
loss.backward()<br>
optimizer.step()</p>
<p>Here I have a problem. Since inputs 1 &amp; 2 share the same feature extractor, I am afraid that the gradients are corrupted. In code, I essentially have a feature extractor + task1 network + task2 network. Although task1 only receives input from input1 (ie first term in model(., .). the backward path is shared with task 2.</p>
<p>Alternatively, I did this:</p>
<p>pred1, _ = model(input1, None)<br>
optimizer.zero_grad()<br>
loss = lossfn(pred1, label1)<br>
loss.backward()<br>
optimizer.step()</p>
<p>_, pred2 = model(input2, None)<br>
optimizer.zero_grad()<br>
loss = lossfn(pred2, label2)<br>
loss.backward()<br>
optimizer.step()</p>
<p>This way, I only use the backward path once per task.</p>
<p>Is this the right way to do it?<br>
I essentially want to train for separate tasks simultaneously, not letting one task get better while losing accuracy on the other.</p>
<p>The inputs are not the same, but they are similar, so feature extractor actually can extract reasonable representations.</p>
<p>Thank you!</p>

MariosOreo: <p>Hello <a class="mention" href="/u/ozan">@Ozan</a>,</p>
<p>These two approaches did the same thing and they will get the same results, the first way is neater. Because the backward perform accumulation of gradient w.r.t. loss of each task, you don’t need worry the gradients will be corrupted.</p>
<p>Thus, both way is correct in your use case.<br>
(If I misunderstand it, correct me please.)</p>
TFRecords for Pytorch

shangeth: <p>Hi, is there a direct wat to get TFRecords dataset as Pytorch Dataset?</p>
<p>Now i am using Tensorflow to get the dataset to numpy and to Torch Tensor.</p>
<p>But , this is slow.</p>

smth: <p>there isn’t a direct path. via numpy is the way to go</p>

alex.veuthey: <p>If you are using the dataset often, I would suggest extracting it once and saving it in another format, this way you can have fast data reads when training.</p>
<p>I personnaly used LMDB for a larger-than-memory sized dataset, and data loading is fast enough to saturate the GPU(s) at all times (as advised by <a class="mention" href="/u/smth">@smth</a> in <a href="https://discuss.pytorch.org/t/whats-the-best-way-to-load-large-data/2977">this post</a> BTW).</p>
How to achieve subtract and division in subtensor efficiently?

trick: <p>a = torch.Tensor([[1,3], [2,6], [4,5]])<br>
I want to get the result:<br>
temp = tensor([2, 4, 1]) where subtraction is performed along subtensor.<br>
and final res = tensor([0.5, 0.5, 0.25, 0.25, 0.25, 0.25, 1.0]) where divided by 1 is performed on tensor temp.<br>
How to do this in PyTorch efficiently?</p>
How to do this in PyTorch?

trick: <p>a = torch.tensor([1,2,3,4,5,6,7,8,9])<br>
b = [3,2,4]<br>
I want to get a result == tensor([6, 9, 30]) where 6 == 1+2+3(three elements), 9 == 4+5(two elements), 30 == 6+7+8+9(four elements).<br>
How to do this in Pytorch efficiently?</p>

vdw: <p>Not sure if best solution, but it yields the required result:</p>
<pre><code>import torch
a = torch.tensor([1,2,3,4,5,6,7,8,9])
b = [3,2,4]
chunks = torch.split(a, b)
results = torch.tensor([ c.sum() for c in chunks ])</code></pre>
Image captioning with LSTM

lanka: <p>hi,<br>
can anyone explain me to LSTM image captioning training, suppose as an example single image has 5 image captions(all sentence are equal length). how do we train LSTM? do we need to train 5 times or only ones with a random sentence?<br>
Thank you</p>

vdw: <aside class="quote no-group" data-username="lanka" data-post="1" data-topic="44425">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/l/e480ec/48.png" class="avatar"> lanka:</div>
<blockquote>
<p>do we need to train 5 times or only ones with a random sentence?</p>
</blockquote>
</aside>
<p>I don’t think there’s any real difference. Once you do multiple epochs, the network basically will see every image-caption pair anyway.</p>
Image normalization data range

Xiaoyu_Song: <p>Hi, I would like to normalize the image data to the range [-1,1], Here is the code I used to do the transform.</p>
<pre><code class="lang-auto">compose_T1 = transforms.Compose([transforms.ToPILImage(),
                                         transforms.Resize((128,128),interpolation=Image.NEAREST),
                                         transforms.ToTensor(),
                                         transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
</code></pre>
<p>But after the transform, I find my data is in the range [-1,0.3412], checking the original data, I find out my orignal data range is [0,172], in this case, should I do the preprocess using data/172*255 in order to get the final data range [-1,1]?</p>

InnovArul: <p>It depends on how you read the image. Typically,  torch dataset classes read the image and store in the range 0-1, hence using mean 0.5, std 0.5 makes the data in the range -1, 1. If RGB image from your dataset is with range 0-255, you can either divide it by 255 to keep it in consensus with typical torch approach or use mean 128, std 127.</p>
<p>About using a specific value 172, you may need to consider a larger use case rather than just the dataset that you have. If it’s RGB, ofcourse there could be test images with range more than 172.</p>
<p>Hope it helps.</p>
Is there any implmentation of vgg+unet on pytorch?

lavender99: <p>i am looking for the  source of unet and vgg as an encoder on pytorch</p>

Tony-Y: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="16" height="16">
      <a href="https://github.com/ternaus/robot-surgery-segmentation" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars0.githubusercontent.com/u/5481618?s=400&amp;amp;v=4" class="thumbnail onebox-avatar" width="60" height="60">

<h3><a href="https://github.com/ternaus/robot-surgery-segmentation" target="_blank" rel="nofollow noopener">ternaus/robot-surgery-segmentation</a></h3>

<p>Wining solution and its improvement for MICCAI 2017 Robotic Instrument Segmentation Sub-Challenge - ternaus/robot-surgery-segmentation</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


lavender99: <p>thank you <a class="mention" href="/u/tony-y">@Tony-Y</a> , the probelm when i try to follow the implemntation , i am gitting this error</p>
<pre><code class="lang-auto">usage: ipykernel_launcher.py [-h] [--mode {base,adv,semi}] [--lam_adv LAM_ADV]
                             [--lam_semi LAM_SEMI] [--t_semi T_SEMI] [--nogpu]
                             [--max_epoch MAX_EPOCH]
                             [--start_epoch START_EPOCH] [--snapshot SNAPSHOT]
                             [--snapshot_dir SNAPSHOT_DIR]
                             [--batch_size BATCH_SIZE] [--val_orig]
                             [--d_label_smooth D_LABEL_SMOOTH]
                             [--d_optim {sgd,adam}] [--no_norm]
                             [--init_net {imagenet,mscoco}] [--d_lr D_LR]
                             [--g_lr G_LR] [--seed SEED]
                             [--wait_semi WAIT_SEMI] [--split SPLIT]
                             prefix dataset_dir
ipykernel_launcher.py: error: the following arguments are required: dataset_dir
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2
</code></pre>

Tony-Y: <p>You should post an issue on the repository if you have any problems.</p>
Missing ImageNet in torchvision v0.2.2

audball: <p>I’m trying to use torchvision’s ImageNet dataset, but it looks like it doesn’t exist in v0.2.2. Is there a reason for this, or am I just missing something?</p>
<p><strong>UPDATE:</strong> <a class="mention" href="/u/thecodez">@TheCodez</a> answered this for me.</p>

TheCodez: <p><a class="mention" href="/u/audball">@audball</a> that’s because it was added after the 0.2.2 release. It will be in the next release, until then you can install torchvision from source:<br>
<code>python setup.py install</code></p>
Torch.as_tensor not available in TorchScript

TheCodez: <p>I am trying to convert a list of tuples into a tensor using <code> torch.as_tensor</code> which throws the following error:</p>
<blockquote>
<p>unknown builtin op: aten::as_tensor<br>
Could not find any similar ops to aten::as_tensor. This op may not exist or may not be currently supported in TorchScript</p>
</blockquote>
<p>Is there any other way to accomplish this? Using <code>torch.tensor</code> also doesn’t work.<br>
I’m using PyTorch version 1.1.0</p>
Upsampling trilinear for 3D data

Naina_Dhingra: <p>I am trying to upsample 5D tensor using nn.Upsample:<br>
size2=(16, 56,56)<br>
self.interpolation1 = nn.Upsample(size=size2,mode=‘trilinear’)</p>
<p>and then using it in forward as:</p>
<pre><code>         out_interp = self.interpolation1(out_middle_2r_blocks) + out_down_residual_blocks1
</code></pre>
<p>I get an error as:</p>
<pre><code>                  return torch._C._nn.upsample_trilinear3d(input, _output_size(3), align_corners)
                   RuntimeError: Expected a list of 3 ints but got 2 for argument #2 'output_size'
</code></pre>
<p>Please help!</p>

ptrblck: <p>Your code seems to work for a 5-dimensional input:</p>
<pre><code class="lang-python">x = torch.randn(1, 3, 4, 5, 6)
size2=(16, 56,56)
interpolation1 = nn.Upsample(size=size2,mode='trilinear', align_corners=False)
output = interpolation1(x)
</code></pre>
Pytorch Error Linear object

Fazeee_Td: <p>I got an inexplicable error:<br>
Traceback (most recent call last):<br>
File “temp.py”, line 39, in <br>
loss = criterion(out, target)<br>
File “/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 493, in <strong>call</strong><br>
result = self.forward(*input, **kwargs)<br>
File “/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py”, line 443, in forward<br>
return F.mse_loss(input, target, reduction=self.reduction)<br>
File “/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py”, line 2244, in mse_loss<br>
if not (target.size() == input.size()):<br>
File “/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py”, line 539, in <strong>getattr</strong><br>
type(self).<strong>name</strong>, name))<br>
AttributeError: ‘Linear’ object has no attribute ‘size’</p>
<p>Here is my code:<br>
import torch<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
import torch.optim as optim</p>
<p>class NeuralNetwork(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(NeuralNetwork, self).<strong>init</strong>()<br>
self.lin1 = nn.Linear(10, 10)<br>
self.lin2 = nn.Linear(10, 10)</p>
<pre><code>def forward(self, x):
    x = F.relu(self.lin1(x))
    x = self.lin2
    return x

def num_flat_features(self, x):
    size = x.size()[1:]
    num = 1
    for i in size:
        num *= i
    return num
</code></pre>
<p>netz = NeuralNetwork()<br>
<span class="hashtag">#----------INPUT</span>/Output----------------------<br>
x = [1,0,0,0,1,0,0,0,1,1]<br>
input = torch.Tensor([x for _ in range(10)])<br>
out = netz(input)</p>
<p>x = [0,1,1,1,0,1,1,1,0,0]<br>
target = torch.Tensor([x for _ in range(10)])</p>
<p><span class="hashtag">#-------------------------------------</span></p>
<p><span class="hashtag">#-------LERN-ALG0---------------------</span><br>
criterion = nn.MSELoss()<br>
loss = criterion(out, target)<br>
print(loss)<br>
<span class="hashtag">#-------------------------------------</span></p>
<p>out = netz(input)<br>
print(out)</p>
<p>As you can imagine, I’m a real beginner so it would be very kind if someone could help me…<br>
Thanks</p>

ptrblck: <p>It looks like you forgot to call <code>self.lin2</code>:</p>
<pre><code class="lang-python">x = self.lin2(x)
return x
</code></pre>

Fazeee_Td: <p>I will try it soon. It‘s really amazing how fast you answered. Thanks a lot!</p>
Runtime Error: $ Torch: not enough memory

TharanginiS: <p>Hey guys, I am replicating the Edge-Connect [1] in an experiment and I am facing the following error:</p>
<p>$ RuntimeError: $ Torch: not enough memory: you tried to allocate 0GB. Buy new RAM! at …\aten\src\TH\THGeneral.cpp:201</p>
<p>I have looked at all possible resources on the web but couldn’t figure out why this might be happening. Any help will be appreciated.</p>
<p>[1. <a href="https://github.com/knazeri/edge-connect" rel="nofollow noopener">https://github.com/knazeri/edge-connect</a>]</p>
How to get a full Q matrix in QR factorization?

jiujing23333: <pre><code class="lang-auto">A = torch.randn((1152, 128))
Q, R = torch.qr(A)
</code></pre>
<p>Then I get a thin Q with shape 1152x128. However, I want to obtain the full Q with shape 1152x1152. How to achieve it?</p>

Tony-Y: <p><a href="https://www.netlib.org/lapack/lug/node40.html" class="onebox" target="_blank" rel="nofollow noopener">https://www.netlib.org/lapack/lug/node40.html</a></p>
<p>Please check the QR factorization algorithm.</p>
How to threshold each channel in a tensor selectively

isalirezag: <p>I have a tensor <code>A</code><br>
I want to threshold each of its channel selectively.<br>
meaning that if the value in that channel is more than the mean of the channel set that to one and if it is less than that set that to 0<br>
Here is what i did.</p>
<pre><code class="lang-auto">A = torch.rand(1,10,8,8)
Means = torch.mean(torch.mean(A,dim=3),dim=2)
Ones = torch.ones(A.size())
Zeros = torch.zeros(A.size())
Thresholded = torch.where(A &gt; Means, Ones, Zeros)
</code></pre>
<p>but it does not work</p>
<p>any suggestions?</p>

ngimel: <p>Means size should be broadcastable to A</p>
<pre><code class="lang-auto">A = torch.rand(1,10,8,8)
Means = torch.mean(torch.mean(A,dim=3),dim=2).unsqueeze(-1).unsqueeze(-1)
Ones = torch.ones(A.size())
Zeros = torch.zeros(A.size())
Thresholded = torch.where(A &gt; Means, Ones, Zeros)
</code></pre>

isalirezag: <p>do you know what should i do if I want to use <code>torch.clamp(A,min=Means)</code> instead of <code>torch.where</code>?</p>

ngimel: <aside class="quote" data-post="3" data-topic="42185">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ngimel/48/14766_2.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/torch-clamp-where-min-and-max-are-tensors-of-the-same-shape-is-input-tensor/42185/3">Torch.clamp() where min and max are tensors of the same shape is input tensor?</a> 
  </div>
  <blockquote>
    You can use 2 torch.where operations, similarly to an answer here <a href="https://discuss.pytorch.org/t/clipping-a-tensor-between-two-corresponding-tensor-values/42961" class="inline-onebox">Clipping a tensor between two corresponding tensor values</a>. It’s not ideal in terms of performance, but it should work.
  </blockquote>
</aside>
C extension function with learnable parameters

AlbertZhang: <p>Hi,</p>
<p>I want to implement a custom function/layer in C/CUDA extension. This function/layer contains <code>learnable parameters</code>. Is there any tutorial for this task? I have browsed some source code of pytorch and an  C extension example <a href="https://github.com/facebookresearch/maskrcnn-benchmark/tree/master/maskrcnn_benchmark" rel="nofollow noopener">maskrcnn</a> but I did not get the point to implement my custom function.</p>
<p>Any help will be appreciated. Thank you.</p>
About Efficient code

Lukas_Petravicius: <p>Hello , I’m partialy new here and wanted to ask about efficiency of c++ frontend . Maybe some one has few tips about implementing models and then sending them to gpu . What are good practice in libtorch and what things to keep in mind when building models to save extra miliseconds .</p>
<p>I’m building GA network for my environment which will be placend in gpu memory too. So basically idea is. 1 environment and thousands of agents .</p>
<p>Maybe there is some tutorials of managing libtorch to fill full potential , because it’s main reason I switched to c++ frontend.</p>
<p>Thanks</p>
AttributeError: Can't pickle local object 'get.<locals>.target_transform'

srikanthram: <p>I am currently using torchvision to utilize the SVHN dataset but however, I am getting the following error:</p>
<hr>
<p>AttributeError                            Traceback (most recent call last)<br>
 in <br>
58 optimizer=torch.optim.Adam(model.parameters(),lr=lr,weight_decay=weight_decay)<br>
59 for epoch in range(num_epochs):<br>
—&gt; 60     for i,(images,labels) in zip(train_loader):<br>
61         images=images.to(device)<br>
62         labels=labels.to(device)</p>
<p>~\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in <strong>iter</strong>(self)<br>
817<br>
818     def <strong>iter</strong>(self):<br>
–&gt; 819         return _DataLoaderIter(self)<br>
820<br>
821     def <strong>len</strong>(self):</p>
<p>~\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py in <strong>init</strong>(self, loader)<br>
558                 #     before it starts, and <strong>del</strong> tries to join but will get:<br>
559                 #     AssertionError: can only join a started process.<br>
–&gt; 560                 w.start()<br>
561                 self.index_queues.append(index_queue)<br>
562                 self.workers.append(w)</p>
<p>~\Anaconda3\lib\multiprocessing\process.py in start(self)<br>
103                ‘daemonic processes are not allowed to have children’<br>
104         _cleanup()<br>
–&gt; 105         self._popen = self._Popen(self)<br>
106         self._sentinel = self._popen.sentinel<br>
107         # Avoid a refcycle if the target function holds an indirect</p>
<p>~\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)<br>
221     <span class="mention">@staticmethod</span><br>
222     def _Popen(process_obj):<br>
–&gt; 223         return _default_context.get_context().Process._Popen(process_obj)<br>
224<br>
225 class DefaultContext(BaseContext):</p>
<p>~\Anaconda3\lib\multiprocessing\context.py in _Popen(process_obj)<br>
320         def _Popen(process_obj):<br>
321             from .popen_spawn_win32 import Popen<br>
–&gt; 322             return Popen(process_obj)<br>
323<br>
324     class SpawnContext(BaseContext):</p>
<p>~\Anaconda3\lib\multiprocessing\popen_spawn_win32.py in <strong>init</strong>(self, process_obj)<br>
63             try:<br>
64                 reduction.dump(prep_data, to_child)<br>
—&gt; 65                 reduction.dump(process_obj, to_child)<br>
66             finally:<br>
67                 set_spawning_popen(None)</p>
<p>~\Anaconda3\lib\multiprocessing\reduction.py in dump(obj, file, protocol)<br>
58 def dump(obj, file, protocol=None):<br>
59     ‘’‘Replacement for pickle.dump() using ForkingPickler.’’’<br>
—&gt; 60     ForkingPickler(file, protocol).dump(obj)<br>
61<br>
62 #</p>
<p>AttributeError: Can’t pickle local object ‘get..target_transform’</p>
What is the default initialization for torch.Tensor?

AlbertZhang: <p>Hi,<br>
Can anybody tell me how the tensors that are created by ‘torch.Tensor’ are initialized ? Since I witnessed some huge numbers occured in the tensor that generated by <code>torch.Tensor</code>.<br>
For example, using <code>torch.Tensor(3, 64)</code> I got<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/4/4ec67b5cd8baf46c5fbb7233135d741b574bf5b4.png" data-download-href="https://discuss.pytorch.org/uploads/default/4ec67b5cd8baf46c5fbb7233135d741b574bf5b4" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/2X/4/4ec67b5cd8baf46c5fbb7233135d741b574bf5b4.png" alt="image" data-base62-sha1="beSugxoUfSHdWbjW9e8HicDR15O" width="633" height="500" data-dominant-color="141414"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">704×556 17.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>The version of PyTorch is 1.0.1.</p>

ptrblck: <p><code>torch.Tensor</code> won’t initialize the data using a specific scheme, but rather returns an uninitialized tensor.<br>
Whatever was stored in the RAM before allocating the tensor will be interpreted as the <code>dtype</code>.<br>
I would recommend to initialize your tensors using <code>torch.tensor</code>, <code>torch.ones</code>, etc.</p>

AlbertZhang: <p>Wow… Thank you so much <a class="mention" href="/u/ptrblck">@ptrblck</a>.</p>
How to tie embeddings?

Rajesh_Karmakar: <p>Hi,</p>
<p>I have seen people using tied embeddings in machine translation,</p>
<ol>
<li>
<p>but since the source and target languages will have different vocabularies, how can we use target embeddings?</p>
</li>
<li>
<p>How to implement tied embeddings in Pytorch? Just using the same embedding would word, but what if I have defined my encoder and decoder as separate classes?</p>
</li>
</ol>
<p>Thanks.</p>

vdw: <p>Hm, I use the same embedding for autoencoder networks, but here the vocabularies are obviously the same. No idea how this would work for different languages/vocabularies.</p>
<p>If your setup allows you to use the same embedding, implementing it is easy: you just define the embedding layer first, and then give it to the encoder and decoder class (e.g., in the constructor). <a href="https://github.com/chrisvdweth/ml-toolkit/tree/master/pytorch/models/text/autoencoder" rel="nofollow noopener">You can have I look how I do it for a RNN and CNN autoencoder</a>. The important snippet is:</p>
<pre><code>class TextCnnAE:

    def __init__(self, device, params, criterion):
        self.params = params
        self.device = device
        self.vocab_size = params.vocab_size
        self.embed_dim = params.embed_dim

        # Embedding layer, shared by encoder and decoder
        self.embedding = nn.Embedding(self.vocab_size, self.embed_dim, max_norm=1, norm_type=2)

        # Calculate the 2-tuples for the kernel sizes (the last one depends on the max_seq_len)
        max_seq_len, kernel_sizes = self.calc_last_seq_len()
        self.params.kernel_sizes[-1] = max_seq_len

        # Create encoder rnn and decoder rnn module
        self.encoder = Encoder(device, self.embedding, params, max_seq_len, kernel_sizes)
        self.decoder = Decoder(device, self.embedding, params, max_seq_len, kernel_sizes)

        ...</code></pre>

vdw: <p>Sorry, I’ve made an error. The tying of weights does not refer to simply sharing the embedding layer among encoder and encoder, although this makes sense as well, of course. I’ve looked into that and extended my <a href="https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/models/text/autoencoder/textrnnae.py" rel="nofollow noopener">RNN-based</a> and <a href="https://github.com/chrisvdweth/ml-toolkit/blob/master/pytorch/models/text/autoencoder/textcnnae.py" rel="nofollow noopener">CNN-based</a> autoencoders to allow for tied weights (just search for <code>self.params.tie_weights</code> in the code)</p>
<p>I don’t think that it’s the perfectly correct implementation, but it works and actually seems to improve the training - I only tested it so far with some smaller dataset.</p>
RuntimeError: size mismatch

mbkinaci: <p>Hello friends, I have a problem on my code . It gives an error below:<br>
<strong>RuntimeError: size mismatch, m1: [1 x 1682], m2: [20 x 1682] at d:\downloads\pytorch-master-1\torch\lib\th\generic/THTensorMath.c:1238</strong></p>
<p>I am putting the problematic code below . Could anyone help me ? I got crazy due to this error .</p>
<p>for epoch in range(1,nb_epoch +1):<br>
train_loss = 0<br>
s = 0.<br>
for id_user in range(nb_users):<br>
input = Variable(training_set[id_user]).unsqueeze(0)<br>
target = input.clone()<br>
if torch.sum(target.data &gt; 0) &gt; 0:<br>
output = sae(input)<br>
target.require_grad = False<br>
output[target == 0] = 0<br>
loss = criterion(output,target)<br>
mean_corrector = nb_movies / float(torch.sum(target.data &gt; 0 ) +1e-10)<br>
loss.backward()<br>
train_loss += np.sqrt(loss.data[0]*mean_corrector)<br>
s += 1.<br>
optimizer.step()<br>
print('epoch: ’ + str(epoch) + ’ loss: ’ + str(train_loss/s))</p>

alexis-jacq: <p>This code is from tutorials of a friend on Udemy. I know it because I did all the codes presented by Hadelin (we did the Bachelor together, and he buys me the codes for a few euros).</p>
<p>I want to apologise, I am doing a PhD in Portugal, with a Portuguese salary. Selling codes for tutorials doubled my salary so I can regularly take the flight to see my family and friends in France and Switzerland.</p>
<p>But, as I did the code, I can tell you: <strong>Don’t trust it!</strong> I did it quickly (very quickly) on my free time. This one is from the autoencoder for movie rate prediction, and the results of prediction are very far from the state-of-the-art results presented in the paper <a href="http://users.cecs.anu.edu.au/~u5098633/papers/www15.pdf">http://users.cecs.anu.edu.au/~u5098633/papers/www15.pdf</a>.</p>
<p>Also, I would not recommend non-free tutorials: most of the time, there is a ‘bad-spirit’ business hidden behind, they are not made for sharing knowledge but to make huge profits. This one, on Udemy, is a good example. My friend who is presenting the code is not understanding it at all. You have free tutorials with much better quality on the official websites and everywhere on Github.</p>

alexis-jacq: <p>However, your code is just the same with some changed names. So it should work. I am pretty sure your error is coming from another part (can you show your whole script? or at least the part with your modifications?)</p>

herleeyandi: <p>As far as I know, the size mismatch always comes from the network architecture like the number of convolution which not have the required size. My suggestion you can go to your network code. From your code I see <code>output = sae(input)</code> . You can see in class <code>sae</code> and check the size for every layer.</p>

lanka: <p>check your network and forward function.</p>
Sample each image from dataset N times in single batch (with DistributedSampler)

starphoenix: <p>I’m currently working on task of learning representation (deep embeddings). The dataset I use have only one example image per object. I also use augmentation.</p>
<p>During training, each batch must contain N different augmented versions of single image in dataset (dataset[index] always returns new random transformation).</p>
<p>Is there some standard solution or library for this purpose, that will work with torch.utils.data.distributed.DistributedSampler?</p>
Bug with reduction='sum' and reduction='mean'?

Oliver_Limoyo: <p>I’m trying to understand the difference between reduction=‘sum’ and reduction=‘mean’.</p>
<p>From what I understand, with ‘sum’ the loss is summed for every example across every element. For ‘mean’ the loss is summed for every example across every element and then divided by the total amount of examples*elements.</p>
<p>I’m confused why the code below produces different losses:</p>
<pre><code class="lang-auto">loss_REC = nn.MSELoss(reduction='sum')
loss_REC_mean = nn.MSELoss(reduction='mean')

L = loss_REC(x_t_hat, x_t)/float(100*64*64) # The shape of x_t is [100, 1, 64, 64]
L_mean = loss_REC_mean(x_t_hat, x_t)

print("L: ", L) 
print("L_mean: ", L_mean)
</code></pre>
<p>Results:<br>
L: 0.5480<br>
L_mean: 0.0815</p>

liangbright: <p>Pytorch 1.1.0</p>
<p>import torch<br>
import torch.nn as nn<br>
x_t=torch.rand((100,1,64,64))<br>
x_t_hat=torch.rand((100,1,64,64))</p>
<p>loss_REC = nn.MSELoss(reduction=‘sum’)<br>
loss_REC_mean = nn.MSELoss(reduction=‘mean’)</p>
<p>L = loss_REC(x_t_hat, x_t)/float(100<em>64</em>64) # The shape of x_t is [100, 1, 64, 64]<br>
L_mean = loss_REC_mean(x_t_hat, x_t)</p>
<p>print("L: ", L)<br>
print("L_mean: ", L_mean)</p>
<p>L is the same as L_mean</p>

Oliver_Limoyo: <p>I also did this test with random tensors and got the same result as you (L being the same as L_mean).</p>
<p>I looked into my actual code more, and I realized it was just my own misunderstanding of python and the loss being mutable:</p>
<pre><code class="lang-auto">L1 = loss_REC(x_t_hat, x_t)/float(100*64*64)
L1_mean = loss_REC_mean(x_t_hat, x_t)

# Losses match here
print('reconstruction loss:{:.4f}'.format(L1.item()))
print('reconstruction loss mean:{:.4f}'.format(L1_mean.item()))

# Problem in my other code          
loss = L1
loss += 1

# Losses no longer match
print('reconstruction loss:{:.4f}'.format(L1.item()))
print('reconstruction loss mean:{:.4f}'.format(L1_mean.item()))
</code></pre>
<p>Thanks for double checking!</p>
Trilinear and Bilinear interpolation

SreenivasVRao: <p>Hi,</p>
<p>I’m wondering if trilinear and bilinear interpolation would be the same if I retained one of the dimensions. For example:</p>
<pre><code class="lang-auto">x = torch.rand([5, 6, 3, 30, 50]) # B C T H W
F.upsample(x, size=(3, 60, 100), mode='trilinear') # 5 6 3 60 100
</code></pre>
<p>Is the second tensor equivalent to bilinear upsampling of each tensor along the temporal axis?</p>

SreenivasVRao: <p>Looks like it is.</p>
<pre><code class="lang-auto">import torch, torch.nn.functional as F
x = torch.rand([5, 6, 3, 30, 50]) # B C T H W
z = F.upsample(x, size=(3, 60, 100), mode='trilinear') # 5 6 3 60 100

z2 =  []
for t in range(3):
   x_t  = x[:, :, t, :, :]
   z2.append(F.upsample(x_t, size=(60, 100), mode='bilinear')) # 5 6 60 100
z2 = torch.stack(z2, dim=2) # 5 6 3 60 100

print(bool((z==z2).all())) # True

</code></pre>
Ho to make a scalar/tensor learnable?

isalirezag: <p>Imagine I have a scalar <code>T</code> , this T is gonna be used as a threshold in my network. i.e.<br>
<code>TensorA = torch.where(TensorB &gt; T*Means, Ones, Zeros)</code>.<br>
Right now I have <code>T = torch.tensor(1.0)</code>, but I want to give it the ability to change and be learnable.<br>
Is the way to do that?<br>
In other words, how can I wrap it in a way to be learnable?</p>

pramod.srinivasan: <p>By learnable, are you referring to <code>requires_grad=True</code> ? <a href="https://pytorch.org/docs/stable/notes/autograd.html#requires-grad" rel="nofollow noopener">More details here</a></p>

isalirezag: <p>Hmmm i dont think that is enough…<br>
I think i need to do something like this:<br>
<code>LearnableParameter = nn.Parameter(Parameter, requires_grad=True)</code><br>
but im not %100 sure<br>
it would be helpful if someone can confirm it</p>

jmaronas: <p>If you use  nn.Parameter it will Be learnable</p>
ImportError on mac with pytorch 1.1

Liang_Zhang: <p>Hi, I just updated the pytorch version from 1.0 to 1.1. When importing pytorch, I was given the following error:</p>
<pre><code class="lang-auto">Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 26 2018, 19:50:54)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.4.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import torch
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
&lt;ipython-input-1-eb42ca6e4af3&gt; in &lt;module&gt;()
----&gt; 1 import torch

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/__init__.py in &lt;module&gt;()
     77 del _dl_flags
     78
---&gt; 79 from torch._C import *
     80
     81 __all__ += [name for name in dir(_C)

ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/_C.cpython-36m-darwin.so, 9): Library not loaded: /usr/local/opt/libomp/lib/libomp.dylib
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/torch/lib/libshm.dylib
  Reason: image not found
</code></pre>
<p>Any suggestion to fix it?</p>
<p>Thanks,<br>
Liang</p>

BSZM: <p>You may need to install libomp. There was a similar issue discussed on GitHub.<br>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/pytorch/issues/20030" target="_blank" rel="nofollow noopener">github.com/pytorch/pytorch</a>
  </header>
  <article class="onebox-body">
    <a href="https://github.com/kaszperro" rel="nofollow noopener">
<img src="https://avatars1.githubusercontent.com/u/2826889?v=2&amp;s=96" class="thumbnail onebox-avatar" width="60" height="60">
</a>

<h4><a href="https://github.com/pytorch/pytorch/issues/20030" target="_blank" rel="nofollow noopener">Issue: I can't import PyTorch, libomp.dylib can't be loaded.</a></h4>

<div class="date" style="margin-top:10px;">
	<div class="user" style="margin-top:10px;">
	opened by <a href="https://github.com/kaszperro" target="_blank" rel="nofollow noopener">kaszperro</a>
	on <a href="https://github.com/pytorch/pytorch/issues/20030" target="_blank" rel="nofollow noopener">2019-05-01</a>
	</div>
	<div class="user">
	closed by <a href="https://github.com/kaszperro" target="_blank" rel="nofollow noopener">kaszperro</a>
	on <a href="https://github.com/pytorch/pytorch/issues/20030" target="_blank" rel="nofollow noopener"></a>
	</div>
</div>

<pre class="content" style="white-space: pre-wrap;">🐛 Bug
I tried to install PyTorch, but I can't use it. It can't link dynamic library libomp.dylib
To Reproduce
Steps to reproduce the...</pre>

<div class="labels">
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>Cheers,<br>
Yicheng</p>

Liang_Zhang: <p><code>brew install libomp</code> fixed the issues, thanks!</p>
Separate Optimizer

1414b35e42c77e0a57dd: <p>What is the criteria that separate optimizers?<br>
What I want to build is :</p>
<pre><code class="lang-auto">Encoder A -&gt; Classifier A
Encoder A -&gt; Classifier B
</code></pre>
<p>I want to make Classifier B backpropagate only to Encoder A (not to Classifier A.)<br>
then this optimizer can work? or should I separate optimizers?</p>
<pre><code class="lang-auto">   optimizer = optim.RMSprop(
        list(encoderA.parameters()) +
        list(classifierA.parameters()) +
        list(classifierB.parameters()),
        lr=1e-3)
</code></pre>

ptrblck: <p>The optimizer will update all passed parameters.<br>
However, the optimizer does not define how the gradients are being calculated.</p>
<p>The computation graph is defined during your forward pass. If you call <code>.backward()</code> on the loss (or any other tensor), the Autograd will backpropagate using the computation graph to calculate the gradients.</p>
<p>E.g. if you calculate the loss using the output of Classifier B, <code>loss.backward()</code> will calculate the gradients of the parameters in Classifier B as well as Encoder A (, since Classifier A was never touched to calculate this particular loss).</p>
Object detection with maskrcnn-benchmark on custom dataset

pramod.srinivasan: <p>I have a dataset of 13k classes with 6 labels (none of which are in COCO) and I am trying to use the Fast RCNN model to train an object detector. Has anyone had success using this <a href="https://github.com/facebookresearch/maskrcnn-benchmark/issues/521" rel="nofollow noopener">https://github.com/facebookresearch/maskrcnn-benchmark/issues/521</a>?</p>
<p>Currently, the blockers are:</p>
<ul>
<li>Formatting the dataset in COCO-style.</li>
<li>Implementing the <code>Dataset</code> class as shown <a href="https://github.com/facebookresearch/maskrcnn-benchmark#adding-your-own-dataset" rel="nofollow noopener">here</a>.</li>
</ul>
Backpropagation with numpy arrays

AntoineMrj: <p>Hello !</p>
<p>I am currently implementing my own loss function :<br>
The loss function takes a tensor containing N predicted masks and another tensor containing N ground truth masks.<br>
Then I compute an IoU matrix (Intersection over Union) to get the accuracy of all predicted mask over the groundtruth masks. So I got NxN IoU matrix (values between 0 and 1). At this point everything’s good.</p>
<p>Then I want to associate to every predicted mask a unique ground truth so I use an Hungarian algorithm to do this and I finally have a function to compute the loss as soon as I know wich prediction corresponds to which truth. Problem is that I use an existing Hungarian implementation which uses numpy arrays instead of Tensors.</p>
<p>So this is my question : Will it be a problem for the backpropagation when the loss will be computed ? (I am almost certain that it is but just want to be sure)</p>
<p>If it is a problem, is it easy to “adapt” a code using numpy arrays to Tensors ? By easy to adapt I mean is there an equivalent for every numpy basic operation in pytorch ?</p>
<p>Thank you for your time <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>