akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
torch.Tensor.scatter

I3aer: <p>Hi,</p>
<p>Could someone please explain me the following</p>
<p><code>The backward pass is implemented only for </code>src.shape == index.shape<code>.</code></p>
<p>Does this condition for torch.Tensor.scatter requires that src and index must have same number of dimension or their size must be be same? According to the condition, their shape,i.e., size must be same but this is not meaningful. The src stores features with N-D, but index stores 1-D/2-D integer indices.</p>
<p>Thank you</p>
Need help debugging issue on fasterrcnn_mobilenet

SarthakJain: <p>Hi,</p>
<p>For the last couple of weeks, I have been really struggling to debug this new problem when training my detection model. Couple of weeks before the training worked great, but now all of a sudden I get this new unheard error.</p>
<p>I make my model and train overall like this but still get an error.</p>
<pre><code class="lang-auto">#Model 
torch.set_default_dtype(torch.float)
backbone = torchvision.models.detection.fasterrcnn_mobilenet_v3_large_320_fpn(pretrained=True)
backbone.roi_heads.box_predictor.cls_score.out_features = len(classes) 
backbone.roi_heads.box_predictor.bbox_pred.out_features = 4 * (len(classes))

#Training 
for epoch in range(epochs):
      train_one_epoch(net, optimizer, train_loader, device, epoch, print_freq=10)
      evaluate(net, test_loader, device=device)
    
    print("Time for Total Training {:0.2f}".format(time.time() - start_time))

    return net
</code></pre>
<p>I get some weird error that look like this:</p>
<pre><code> 26     for epoch in range(epochs):
</code></pre>
<p>—&gt; 27       train_one_epoch(net, optimizer, train_loader, device, epoch, print_freq=10)<br>
28       evaluate(net, test_loader, device=device)<br>
29</p>
<p>/content/engine.py in train_one_epoch(model, optimizer, data_loader, device, epoch, print_freq)<br>
44         if not math.isfinite(loss_value):<br>
45             print(“Loss is {}, stopping training”.format(loss_value))<br>
—&gt; 46             print(loss_dict_reduced)<br>
47             sys.exit(1)<br>
48</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/_tensor.py in backward(self, gradient, retain_graph, create_graph, inputs)<br>
253                 create_graph=create_graph,<br>
254                 inputs=inputs)<br>
 → 255         torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)<br>
256<br>
257     def register_hook(self, hook):</p>
<p>/usr/local/lib/python3.7/dist-packages/torch/autograd/<strong>init</strong>.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)<br>
147     Variable.<em>execution_engine.run_backward(<br>
148         tensors, grad_tensors</em>, retain_graph, create_graph, inputs,<br>
 → 149         allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag<br>
150<br>
151</p>
<p>RuntimeError: Found dtype Double but expected Float.</p>
<p>I also have boxes and labels encoded like this <code>boxes = torch.as_tensor(boxes, dtype = torch.float32)       labels = torch.as_tensor(labels, dtype = torch.int64)</code> and my images are float tensors.</p>
<p>How do I get rid of this runtime error?<br>
For all of my code with my data class, imported libs, and train check over <a href="https://github.com/SarthakJaingit/Fruits-and-Spot-Detection/blob/main/FruitObjectDetection%20(60).ipynb" rel="noopener nofollow ugc">here</a></p>
<p>Thanks for the help,<br>
Sarthak</p>

ptrblck: <p>Could you remove the <code>torch.set_default_dtype</code> line of code and explicitly cast the tensors to the desired dtype?</p>

SarthakJain: <p>Thanks for the help. Explicitly casting tensors to desired dtype helped.</p>
How to introduce the manual weight and bias?

pramesh: <p>I want to use the same weight and bias  for the two model respectively at 1st epoch and need to be updated accordingly.</p>
<pre><code class="lang-auto">def init_normal(m):
    if type(m) == nn.Linear:
        nn.init.kaiming_uniform_(m.weight)
        if m.bias is not None:
            nn.init.constant_(m.bias.data, 0.001)
model = MLP(INPUT_DIM, OUTPUT_DIM)
model=model.apply(init_normal)

</code></pre>
<p>As i am doing the iteration ,the weight and bias is updated .And affecting the overall accuracy of the next model.<br>
I tried the following peice of code</p>
<pre><code class="lang-auto">model1=copy.deepcopy(model)
</code></pre>
<p>this is working fine for first epoch but at the second epoch it is using the same weight and bias.</p>
<p>I want to use the updated one at the second epoch.</p>
<p>Any reference code will be highly appriciated…<br>
I am new to pytorch…please support…</p>
<p>thanks with best regards,<br>
pramesh</p>
Functions That Only Return NumPy Arrays

cheeky_meek: <p>I’m in a situation where I want to use a function that accepts and returns a NumPy array in a PyTorch Net, and I’m not sure the best way to do this without disrupting the computational graph.  The function doesn’t have any learnable parameters, so it doesn’t need to be part of the graph.  In my example, I’m trying to use the norm.ppf function from scipy.stats.  My solution right now is to make a copy of my tensor, do the operation that returns a NumPy array on the copy, and then override the data in the original tensor, but this feels kinda hacky.  Here’s the code I have now:</p>
<pre><code class="lang-auto"># x is a tensor of mini-batch inputs
x_copy = x.clone().detach().cpu()
x_copy = (torch.argsort(x_copy, dim=0)+0.99) / x_copy.shape[0]
x_copy = norm.ppf(x_copy)
x.data = torch.Tensor(x_copy).to(get_device())
</code></pre>
<p>I’m doing it this way because if I don’t make a copy then the computational graph gets wiped out after the tensor gets converted to a NumPy array in line 3.</p>
Training sub-modules with different losses

Maria_Fernanda_De_La: <p>I want to better understand PyTorch computational graphs. When declaring multiple optimizers sending in only a layer or a module, and then backpropagating through multiple losses in the following way, is each layer/module updated only based on its corresponding backward loss call?</p>
<pre><code class="lang-auto">self.optimizer_critic = torch.optim.Adam(self.acmodel.critic.parameters(), lr, eps=adam_eps)
self.optimizer_vae = torch.optim.Adam(self.acmodel.vae.parameters(), lr, eps=adam_eps)
self.optimizer_mdnrnn = torch.optim.Adam(self.acmodel.mdnrnn.parameters(), lr, eps=adam_eps)

self.optimizer_critic.zero_grad()
batch_loss.backward(retain_graph=True)
self.optimizer_critic.step()
                

self.optimizer_mdnrnn.zero_grad()
batch_mdnrnn_loss.backward(retain_graph=True)
self.optimizer_mdnrnn.step()

            

self.optimizer_vae.zero_grad()
batch_vae_loss.backward(retain_graph=True)
self.optimizer_vae.step()
</code></pre>
How to use inbuilt `int` in torch.fx

Alexander_Soare: <p>I’m trying to symbolically trace these lines:</p>
<pre><code class="lang-python">B, T, _, C = x.shape
grid_size = int(math.sqrt(T))
</code></pre>
<p>The problem is that <code>int</code> can’t handle a <code>Proxy</code> as an input. Is there any way I can customize the tracer to handle this particular use case?</p>
Question about Purpose of RNNs & Partitioning of Sets

drbeethoven: <p>Hello,<br>
I understand Recurrence Neural Networks are used frequently in natural language processing or similar processes. For example, predicting the next word in the sentence: “I took my dog for a ___” (walk). As an analogy, I’m trying to instead use RNNs for a version of generating what can be seen as a “coherent” sentence, given multiple other sentences.</p>
<p>What I’m really trying to do is, given multiple sets of points, and how each set of points is partitioned, I’m trying to find how I would partition another set of points in order to match the pattern of the previous sets.</p>
<p>RNNs seemed like a good method to do this, purely because of their high data interdependence (nearby data decisions affecting other data), which I want so that the decision of what cluster a specific point belongs to affects future decisions within the same set. Does an RNN seem like the correct method for this, or does something else come to mind?</p>
<p>One difficulty I have already come across is that the “sequence size” parameter kind of changes with what point you are currently deciding. So for example, if you are making a decision on the third point in the set, your “sequence size” should be 2, since you are deciding based on current weights &amp; what you classified the other two points as. On the other hand, if you are deciding the 10th point, your sequence size should be 9. This is different from RNNs ive seen, where the sequence size is constant.</p>
<p>Thank you so much for reading or if you decide to respond!</p>
Small object localization

hadichko: <p>Hi<br>
I have a small dataset (around 50) of 2D CT images which I need to use to build a (CNN) network in order to detect the x and y dimensions of a specific landmark in every image. I know that every image contains the landmark. However, I need to pinpoint the coordinates of the landmark. The images are 512 x 512 pixels. I have already built a vanilla FCN model where I use Adam optimizer and MSE criterion for this “regression” problem. The issue is that to increase the accuracy of the predictions, I would need a substantially larger dataset, which I do not have currently. Instead, I decided to enhance the complexity of the model by dividing each image into patches (e.g. 4x4 grid cells) and use a multi output approach by first classifying if the patch may contain the landmark and then try to find the coordinate of the landmark using regression in the positive patches. I, however, am not sure if this approach would make sense and increase the accuracy of the predictions. Moreover, I am not sure if an example of such or a similar approach exists in PyTorch. See the illustration of the problem: the red dot is the landmark whose coordinates are given in y_regression and the grid containing the dot is labeled as 1 in the y_classification.<br>
I appreciate any hints and/or suggestions.<br>
Hadi<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ae3a21470827c6872debe6e82f6de3589493aae.png" data-download-href="https://discuss.pytorch.org/uploads/default/7ae3a21470827c6872debe6e82f6de3589493aae" title="dataset_illustration"><img src="https://discuss.pytorch.org/uploads/default/original/3X/7/a/7ae3a21470827c6872debe6e82f6de3589493aae.png" alt="dataset_illustration" data-base62-sha1="hx7X5LSlaidr5VKavILhwR1vgqG" width="690" height="355" data-dominant-color="294272"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">dataset_illustration</span><span class="informations">1257×647 12.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
How to implement counting loss

marcpaga: <p>Let’s assume that we want to develop a model to count the number of words in an audio sample.<br>
How one would approach the last layer and the loss? I have thought of the following ideas, but perhaps there is a better approach to this.</p>
<ul>
<li>
<p>Use a linear layer with N dimensions and a softmax activation and cross-entropy loss. This makes it so that each amount is a class. The problem is that when the truth is 4 words, predicting 3 words or predicting 20 words is equally wrong.</p>
</li>
<li>
<p>Use a linear layer with 1 dimension and relu activation and MSE loss. This seems like the most direct approach and perhaps is just this simple.</p>
</li>
<li>
<p>Use linear layer with N dimensions and softmax activation and KL loss. In this case, we can label our data so that instead of being one count, you distribute the probabilities to neighbor counts. For example, for a max of 10 counts, the count 5 can be labelled as <code>[0, 0, 0, 0.2, 0.6, 0.2, 0, 0, 0, 0, 0]</code> instead of <code>[0, 0, 0, 0, 1, 0, 0, 0, 0, 0]</code>. In this case, predicting a 4 or a 6 is less wrong than predicting other values.</p>
</li>
</ul>
<p>One has to try out to see what works best, but was wondering if anyone else has found any of this or another approach best.</p>
Solving an ode using pytorch

Anurag_Ranjak: <p>I am trying to solve an ode using pytorch. The ode has the form</p>
<p><code>du/dt = cos(2*3.14*t)</code></p>
<p>I parameterise my neural network as a two layer linear network<br>
with tanh as an activation function in between. The layer takes in 1 dimensional input and returns 1 dimensional output with hidden layer size being 32.</p>
<pre><code class="lang-auto">    
def f(x):
    
    """
    function that computes the output of the neural net
    
    """
    
    l1 = torch.matmul(W1.T, x).reshape(-1, 1)
    l1_act = leakyrelu(l1)
    l2 = torch.matmul(W2.T, l1_act)
    return l2

def g(t):
    """
    the form of the solution chosen such that boundary condition
    u(0) = 1. is satisfied 
    """
    return t*f(t) + torch.tensor([1.])

def loss(t, eps):
    """
    the loss function which is simply the loss 
    on the gradient of the net being equal to the analytical gradient 
    value
    """
    return torch.mean(((g(t+eps) - g(t)) /(eps) - torch.cos(torch.tensor([2*3.14*t])))**2)

</code></pre>
<p>These are the model definition</p>
<pre><code class="lang-auto">x = torch.tensor([.01])
eps = torch.tensor([0.000345])
T0 = 0
T1 = 1
nsamples = 100
t = torch.linspace(T0, T1, nsamples)

W1 = Variable(torch.ones(1, 32), requires_grad=True)
W2 = Variable(torch.ones(32, 1), requires_grad = True)
</code></pre>
<p>I generate a 100 datapoints between 0 and 1 and train the network on these datapoints for 5000 epochs. My training loop looks something like this</p>
<pre><code class="lang-auto">
learning_rate = 1e-3
for it in tqdm(range(5000)):
    err = 0
    for ti in t:
    
        ti = torch.tensor([ti])
        err += loss(ti, eps)
    
    err = err / nsamples
    err.backward()
    
    W1.data -= learning_rate * W1.grad.data
    W2.data -= learning_rate * W2.grad.data
    
    
    if it%100==0:
        print(err.item())
        grad_w1 =  W1.grad.data.detach().numpy().flatten()
        grad_w2 =  W2.grad.data.detach().numpy().flatten()
        fig, a= plt.subplots(1, 2, figsize =(6, 3))
        a[0].plot(grad_w1)
        a[1].plot(grad_w2)
        plt.show()
    
    W1.grad.data.zero_()
    W2.grad.data.zero_()
</code></pre>
<p>What I notice is that the gradient updates go to 0 and the network doesnt manage to learn the values. I had been following along in an example in Julia and there the code seems to work. I was wondering if there is something wrong in my specification. If someone can point me in the right direction it would be great. This is my second attempt to solve. I tried using the nn.Module along with nn.Linear layer  along with Adam optimizer and there too the error seems to be the same</p>
Multi30k torchtext datatset splits Error

Harsh_Sharma: <p>Actually, I am trying to code a Seq2Seq Machine Translation using Pytorch and TorchText Libraray using Multi30k Datasets.</p>
<p>I was trying to perform <code>train,valid,test=Multi30k.splits()</code> , but got this error -</p>
<pre><code class="lang-auto">AttributeError                            Traceback (most recent call last)
&lt;ipython-input-17-cc515cd9fb6e&gt; in &lt;module&gt;()
----&gt; 1 train_data, valid_data, test_data = Multi30k.splits(
      2     exts=(".de", ".en"), fields=(german, english)
      3 )

AttributeError: 'function' object has no attribute 'splits'
</code></pre>
<p>Can someone please help me in solving that.<br>
I am also attaching the Google Colab Notebook for further reference.</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://colab.research.google.com/drive/1ZjT3rRzAx8hf6d0_s4iI_habl94HwRqa?usp=sharing">
  <header class="source">
      <img src="https://colab.research.google.com/img/favicon.ico?vrz=colab-20210716-060119-RC00_385112924" class="site-icon" width="128" height="128">

      <a href="https://colab.research.google.com/drive/1ZjT3rRzAx8hf6d0_s4iI_habl94HwRqa?usp=sharing" target="_blank" rel="noopener nofollow ugc">colab.research.google.com</a>
  </header>

  <article class="onebox-body">
    <img src="https://colab.research.google.com/img/colab_favicon_256px.png" class="thumbnail onebox-avatar" width="260" height="260">

<h3><a href="https://colab.research.google.com/drive/1ZjT3rRzAx8hf6d0_s4iI_habl94HwRqa?usp=sharing" target="_blank" rel="noopener nofollow ugc">Google Colaboratory</a></h3>



  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


tom: <p>The TorchText module has undergone a major overhaul over the last year, changing how you instantiate things. The suggested way to use Multi30k from <a href="https://github.com/pytorch/text/releases/tag/v0.10.0" rel="noopener nofollow ugc">the release notes</a> is</p>
<pre><code class="lang-python">from torchtext.datasets import Multi30k
train_data, valid_data, test_data = Multi30k()
</code></pre>
<p>More parameters are given in the <a href="https://pytorch.org/text/stable/datasets.html#multi30k" rel="noopener nofollow ugc">documentation</a>.</p>
<p>Best regards</p>
<p>Thomas</p>

Harsh_Sharma: <p>Oh yes, it worked successfully.<br>
Thanks for your help…</p>
Upgrade from 1.8.1 to 1.9.0: missing function in dll

Steve_Miller: <p>Hi,</p>
<p>I have been using libtorch 1.8.1 with cuda 11.1 in visual studio 2019 with no problems. I tried converting to use libtorch 1.9.0 (cuda 11.1 as well); no changes to my code, all I am altering is include and link paths.</p>
<p>Everything builds and links, but there is a runtime error:</p>
<hr>
<h2>
<a name="the-procedure-entry-point-1autodispatchbelowadinplaceorviewatqeaaxz-could-not-be-located-in-the-dynamic-link-library-my-program-name-1" class="anchor" href="#the-procedure-entry-point-1autodispatchbelowadinplaceorviewatqeaaxz-could-not-be-located-in-the-dynamic-link-library-my-program-name-1"></a>The procedure entry point<br>
??1AutoDispatchBelowADInplaceOrView@at@<span class="mention">@QEAA</span>@XZ<br>
could not be located in the dynamic link library<br>
</h2>
<p>and a quick google shows this is a libtorch function.</p>
<p>I have tried downloading libtorch 1.9.0 from</p>
<p><a href="https://download.pytorch.org/libtorch/cu111/libtorch-win-shared-with-deps-1.9.0%2Bcu111.zip" class="onebox" target="_blank" rel="noopener nofollow ugc">https://download.pytorch.org/libtorch/cu111/libtorch-win-shared-with-deps-1.9.0%2Bcu111.zip</a></p>
<p>several times and get the same error.</p>
<p>Can anyone offer a diagnosis, or any advice please?</p>
<p>Steve.</p>

Steve_Miller: <p>Hi,</p>
<p>I noticed this is still a problem, can anyone help?</p>
<p>Steve.</p>
Converting Conv2d to Conv3d

Aarya: <p>Hello.</p>
<p>I want to convert a 2D Conv into 3D Conv for video RGB input.<br>
Since, I am new to pytorch, it would be grateful if anyone can help me.</p>
<p>Thank you in advance.</p>
<pre><code class="lang-auto">class EmbeddingNet(nn.Module):
    def __init__(self):
        super(EmbeddingNet, self).__init__()
        self.convnet = nn.Sequential(nn.Conv2d(3, 32, 5), nn.PReLU(),
                                     nn.MaxPool2d(2, stride=2),
                                     nn.Conv2d(32, 64, 5), nn.PReLU(),
                                     nn.MaxPool2d(2, stride=2))

        self.fc = nn.Sequential(nn.Linear(64 * 61 * 61, 256),
                                nn.PReLU(),
                                nn.Linear(256, 256),
                                nn.PReLU(),
                                nn.Linear(256, 2)
                                )

    def forward(self, x):
        output = self.convnet(x)
        output = output.view(output.size()[0], -1)
        output = self.fc(output)
        return output

    def get_embedding(self, x):
        return self.forward(x)
</code></pre>

Aarya: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Any suggestions on this? I really appreciate if you help</p>

ptrblck: <p>From the code perspective you could replace all <code>...2d</code> layers with their <code>...3d</code> equivalents first.<br>
If you want to reuse the <code>...2d</code> parameters, you would have to come up with a way to copy these parameters into the <code>...3d</code> modules and expand/repeat them.<br>
I don’t have any information unfortunately, which approach would work best, as I assume it’s use case dependent.<br>
However, if you want to train the new model from scratch, replacing the layers might work without any further changes.</p>

Aarya: <p>Thank you for your suggestion.</p>
Manual_seed vs use_deterministic_algorithms

malioboro: <p>Do we still need to use <code>manual_seed</code> if we have set <code>use_deterministic_algorithms(True)</code> to create a reproducible code?</p>
<p>Because as far as I understand, the two functions are interchangeable. Is this true?</p>

ptrblck: <p>You would need to use both, as setting the seed would make sure the pseudorandom number generator would return a deterministic output, while using deterministic algorithms would force e.g. cudnn to select deterministic kernels and could also use a different code path for native functions.</p>

malioboro: <p>ah…I see the difference now… thank you</p>
Totally different results when using TorchScript on Android

minhduc0711: <p>Here’s <a href="https://github.com/minhduc0711/neural-style-transfer/blob/35e7cd82bafc82e18cc7e35491082935834580d9/style_transfer/models.py#L27" rel="nofollow noopener">my model code</a>, which is basically the generative model from the fast neural style PyTorch example.<br>
I converted it to TorchScript:</p>
<pre><code class="lang-auto">net = TransformerNet(alpha=0.3)
net.load_state_dict(torch.load("trained_models/starry_night_small.pth",
                               map_location='cpu'))
net.eval()

# Convert to TorchScript
ts_module = torch.jit.script(net, torch.ones(1, 3, 256, 256))  # I also tried jit.trace() but no luck
ts_module.save("trained_models/starry_night.zip")
</code></pre>
<p>I tested right in Python to see if eager mode and tracing give the same results, and strangely the results are the <strong>exact</strong> same (I thought there has to be a small error due to conversion):</p>
<pre><code class="lang-auto">example_input = torch.randn(1,3,256,256)
with torch.no_grad():
    orig_out = net(example_input)
    ts_out = ts_module(example_input)
diff = F.l1_loss(orig_out.flatten(), ts_out.flatten()).item()
print("Error: ", diff)
# output: 0.0
</code></pre>
<p>And when I brought the model onto an Android app, the model gave way different results.<br>
Here’s the Android code:</p>
<pre><code class="lang-java">float[] dummyInput = new float[3 * 5 * 5];
long[] shape = new long[]{1, 3, 5, 5};
Arrays.fill(dummyInput, 1.0f);
final Tensor inputTensor = Tensor.fromBlob(dummyInput, shape);
Tensor outputTensor = net.forward(IValue.from(inputTensor)).toTensor();
float[] rawPixels = outputTensor.getDataAsFloatArray();
// rawPixels = {0.38102132, 0.15379308, 0.21999769, 0.33937144, 0.24030314, 0.15130955, 0.3354013, 0.19232287,...}
</code></pre>
<p>while the results on Python for <code>torch.ones()</code> is just:</p>
<pre><code class="lang-python">ts_module(torch.ones(1,3,5,5)).flatten().unique()
# output: tensor([0.2949, 0.3554, 0.4184], grad_fn=&lt;NotImplemented&gt;)
</code></pre>
<p>Sorry for the lengthy post, but I would really appreciate it if someone points me in the right direction to debug this.<br>
Thank you in advance!</p>

Silbernitrat: <p>Have you solved this problem？I meet this problem now, the result in android and PC is different.</p>

minhduc0711: <p>Not yet sadly. However I’ve kinda encountered this problem <a href="https://discuss.pytorch.org/t/module-output-slightly-off-on-android/62838/10">before</a>, but the cause due to was jpg files. You can check it out to see if it’s related.</p>

Silbernitrat: <p>I modified the architecture of the network and loaded the old weights, it work well. But l don’t know why the old net work differently and the new net work well.</p>

Lakshya1: <p>Hello, can you please provide code or point me to how to perform this copying business you have mentioned in Android or is the copying done in python only.</p>
Recurrent Batch Normalization Implementation in Pytorch

Dark: <p>I want to use the implementation of RBN by jihunchoi which I found on github : <a href="https://github.com/jihunchoi/recurrent-batch-normalization-pytorch/blob/master/bnlstm.py" class="inline-onebox" rel="noopener nofollow ugc">recurrent-batch-normalization-pytorch/bnlstm.py at master · jihunchoi/recurrent-batch-normalization-pytorch · GitHub</a><br>
I had a question regarding the code snippet in <strong>forward</strong> method of LSTM class:<br>
if hx is None:<br>
hx = (Variable(nn.init.xavier_uniform(weight.new(self.num_layers, batch_size, self.hidden_size))),<br>
Variable(nn.init.xavier_uniform(weight.new(self.num_layers, batch_size, self.hidden_size))))</p>
<p>I get my compiler complaining about the fact that weight is not defined (indeed, it is not defined…). I’m not sure about how to fix the problem  - since I don’t see what was intended to do by this code snippet(and especially by using weight.new() in it). Any help on explaining this code snippet and how to fix it would be very much appreciated!</p>

ptrblck: <p>Since <code>weight</code> seems to be undefined, replace it with either <code>torch.empty</code> or <code>torch.randn</code> with the desired shapes and make sure the <code>dtype</code> (<code>float32</code> by default) and the device is as expected.<br>
Also, note that <code>Variable</code>s are deprecated since PyTorch <code>0.4</code>, so you can use tensors now.</p>
Why does faster-rcnn have a long delay(~10s) at the beginning?

KouWeiBin: <p>Hello,<br>
I used pytorch to train a faster-rcnn model and then convert it into TorchScript model in order to deploy in c++ platform. After I deployed it in c++ platfrom, there is about 10s delay in predicting the second frame. so can anyone help me to solve it or give some valualbe comments? Thanks</p>

tom: <p>This is when the JIT does its optimizations.<br>
Typically, the first pass is an unoptimized profiling pass where the JIT collects tensor shape and layout information (and profiling means that, not something with timing). Then before the second pass, the JIT specializes the graph for the observed shapes and compiles fusion kernels etc. This is likely the delay you are seeing (though 10s seems a lot). After the second run (unless shapes change), you’ll be hitting the cached optimized path.</p>
<p>(That it is one profiling pass before the optimization kicks in is a configurable parameter.)</p>
<p>Some more details can be found e.g. on my blog posts <a href="https://lernapparat.de/jit-optimization-intro/" rel="noopener nofollow ugc">on JIT optimization</a> and <a href="https://lernapparat.de/jit-runtime-overview/" rel="noopener nofollow ugc">the JIT runtime</a>.</p>
<p>Best regards</p>
<p>Thomas</p>

KouWeiBin: <p>Thanks your reply!<br>
what you said is actually what I need.</p>
Select columns from a batch of matrices by index

s.sni: <p>Let’s consider</p>
<pre><code class="lang-auto">a = torch.arange(8).view(2,2,2)
</code></pre>
<p>which should look like</p>
<pre><code class="lang-auto">[[[0, 1],
[2, 3]],

[[4, 5],
[6, 7]]]
</code></pre>
<p>and also</p>
<pre><code class="lang-auto">index = torch.tensor([1, 0])
</code></pre>
<p>“index” shows column indexes which should be extracted from “a”.<br>
so the output should be look like this:</p>
<pre><code class="lang-auto">[[1, 4],
 [3, 6]]
</code></pre>
<p>although it seems not difficult, using torch.gather and other indexing methods couldn’t help me. Any suggestions?</p>

ptrblck: <p>You could directly index the tensor and permute the output:</p>
<pre><code class="lang-python">res = a[torch.arange(2), :, index]
print(res)
&gt; tensor([[1, 3],
          [4, 6]])
print(res.t())
&gt; tensor([[1, 4],
          [3, 6]])
</code></pre>

s.sni: <p>Oh. Nice. I tried multiplying ‘a’ by one hot vectors:</p>
<pre><code class="lang-auto">index_tensor = index.view(*index.size(), -1)
one_hot = torch.zeros(*index.size(), 2, dtype=index.dtype)
one_hot = one_hot.scatter(1, index_tensor, 1).unsqueeze(-1)
result = a.matmul(one_hot).squeeze(-1).transpose(-2, -1)
</code></pre>
<p>Thinking too complicated always makes trouble!!<br>
Thank you</p>

adithya1: <p>Hi,</p>
<p>I am a bit confused with understanding the autograd implications of this operation. I am essentially trying to do this on the outputs of an LSTM. So, was wondering if this doesn’t break the computational graph. Thank you.</p>

ptrblck: <p>Assuming you are concerned about the indexing and transpose operation: no, these operation won’t break the computation graph as seen here:</p>
<pre><code class="lang-python">a = torch.arange(8).view(2,2,2).float().requires_grad_()
index = torch.tensor([0, 1])
res = a[torch.arange(2), :, index]
res = res.t()
print(res)
res.mean().backward()
print(a.grad)
&gt; tensor([[[0.2500, 0.0000],
           [0.2500, 0.0000]],

          [[0.0000, 0.2500],
           [0.0000, 0.2500]]])
</code></pre>
<p>The gradient will be calculated for the selected elements properly.</p>

adithya1: <p>Thanks for the clarification, I was concerned about the indexing part but now I am clear.</p>
Simpler way to get predicted results in batches after training

kyy: <p>Hi, I am trying to get predicted results after model training.  Since the input data is too large to fit into memory, I would need to divide the data into batches for evaluation.  I know it is possible to do the following in pytorch:</p>
<pre><code class="lang-auto">with torch.no_grad():
    result_list = []
    for batch in dataloader:
        result = net(batch)
        result_list.append(result)
# then concatenate list to get predicted results
</code></pre>
<p>I am just wondering if there are simpler ways to do this, something like <code>mode.predict(input, batch_size=batch_size)</code> in Keras where <code>batch_size</code> option automatically takes care of the splitting and recombination process?</p>
<p>Thank you!</p>

ptrblck: <p>For a higher-level API you could take a look at e.g. <code>Ignite</code>, <code>Catalyst</code>, and <code>Lightning</code>, which I guess are providing these kind of APIs.</p>
Where to use softmax?

arya47: <p>I was working on implementing a RESNET50 model and was wondering if i should use nn.Softmax() inside the model or outside (in training loop).<br>
I can’t see why it would make a difference, tho with a few implementation i’ve seen people use it outside</p>
<p>Thanks</p>

KFrank: <p>Hi Aryaman!</p>
<aside class="quote no-group" data-username="arya47" data-post="1" data-topic="127187" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/arya47/48/41192_2.png" class="avatar"> arya47:</div>
<blockquote>
<p>I was working on implementing a RESNET50 model and was wondering if i should use nn.Softmax() inside the model or outside (in training loop).</p>
</blockquote>
</aside>
<p>I won’t speak directly to ResNet-50, but I assume that your use case<br>
is image classification.  It’s not the only reasonable choice, but one<br>
would typically train such a classifier with <code>CrossEntropyLoss</code>.</p>
<p>In pytorch (for good reason), <code>CrossEntropyLoss</code> expects predictions<br>
that are raw-score <em>logits</em> (rather than, say, probabilities).  For this<br>
reason, in your training loop, you want <code>Softmax</code> (which converts logits<br>
to probabilities) neither inside nor outside of your model.  You would<br>
typically feed the output of the final <code>Linear</code> layer of your classifier<br>
(understood to be logits) directly to <code>CrossEntropyLoss</code>.</p>
<p>If you happen to need the probabilities (for some other reason than<br>
computing the loss), you would pass the logits through <code>Softmax</code>,<br>
but it’s often the case that you <em>don’t</em> need explicit probabilities and<br>
can perform any additional desired processing with the logits directly.</p>
<p>So:  No <code>Softmax</code> for training (neither inside nor outside of your model),<br>
and probably no <code>Softmax</code> at all.</p>
<p>(As a side note, internal to pytorch’s <code>CrossEntropyLoss</code> is a<br>
<code>log_softmax()</code> computation, so a conversion to probabilities is<br>
being carried out under the hood, but in log-probability space.)</p>
<p>Best.</p>
<p>K. Frank</p>
How to build a hierarchical LSTM model with two hierarchical variable length sequences?

Wwwwei: <p>I am trying to build a hierarchical sequence model for time series classification (refer to the paper: hierarchical attention networks for document classification). But I’m very confused about how to mask the hierarchical sequences.</p>
<p>My data is a two hierarchical time series. Specifically, each sample is composed of multiple sub-sequences and each sub-sequence is a multivariate time series (just like word–&gt; sentence --&gt;document in NLP). So I need to pad and mask it twice. This is critical as a document will often not have the same number of sentences (or all sentences the same number of words). Finally, I get data as follows:</p>
<pre><code class="lang-auto">array([[[[0.21799476, 0.26063576],
         [0.2170655 , 0.53772384],
         [0.18505535, 0.30702454],
         [0.        , 0.        ]
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]],

        [[0.2160176 , 0.23789616],
         [0.2675753 , 0.21807681],
         [0.26932836, 0.21914595],
         [0.26932836, 0.21914595],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]]],

       [[[0.03941338, 0.33808291],
         [0.04766269, 0.30310882],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]],

        [[0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]]],

       [[[0.63451399, 0.38098294],
         [0.40768279, 0.35421815],
         [0.22714901, 0.17020352],
         [0.71249301, 0.72023917],
         [0.43149343, 0.42213124],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]],

        [[0.54531993, 0.62312294],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ],
         [0.        , 0.        ]]]], dtype=float32)
</code></pre>
<p>I wonder how to build a hierarchical LSTM model with two hierarchical variable length sequences?<br>
Anyone can show me an example?<br>
Thanks very much!</p>
How to get val loss

Programmer-RD-AI: <p>how to get val loss CNN Pytorch Output 1 (Sigmoid) Output Layer Criterion = BCELoss()</p>

ptrblck: <p>I don’t quite understand the question, so could you provide more information, please?<br>
The loss would be generally given as the output of a criterion (<code>nn.BCELoss</code> in this case).</p>

Programmer-RD-AI: <p>def loss(self, model, criterion, X_batch, y_batch):<br>
preds = model(X_batch.view(-1, 3, 84, 84).float().to(device))<br>
loss = criterion(<br>
preds.view(-1, 1).float().to(“cpu”).float(),<br>
y_batch.view(-1, 1).float().to(“cpu”).float(),<br>
)<br>
loss = loss.item()<br>
return loss</p>

Programmer-RD-AI: <p>I am getting my loss like above is this way correct if not what is the correct way thank you. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

ptrblck: <p><code>loss.item()</code> will return a Python scalar value, which is detached from the computation graph.<br>
If you want to use this loss for training, you should remove this line of code.<br>
Also, I’m unsure why you need to push the <code>preds</code> and <code>y_batch</code>, to the CPU, but I’m also not familiar with your use case, so you should double check if this is needed.</p>

Programmer-RD-AI: <p>OK thank you so much</p>
Torch.matmul does not work properly

Kana-alt: <p>I calculated the rank4xrank4 matrix using torch.mamtmul.<br>
As a result, all the values became zero.</p>
<p>Can you tell me the cause of this?</p>
<p>The code is below. In the second line of the calculation, the values are all set to zero.</p>
<p>dis_norm = (self.ctl_ts.view(opts.n_hypo,-1,1,3) - pred_v.view(2*local_batch_size,opts.n_hypo,-1,3)[0,:,None].detach()) # p-v, H,J,1,3 - H,1,N,3<br>
dis_norm = dis_norm.matmul(kornia.quaternion_to_rotation_matrix(self.ctl_rs).view(opts.n_hypo,-1,3,3)) # h,j,n,3<br>
dis_norm = self.log_ctl.exp().view(opts.n_hypo,-1,1,3) * dis_norm.pow(2) # (p-v)^TS(p-v)</p>

ptrblck: <p>Could you post an executable code snippet (with random inputs, assuming they recreate the issue) so that we could try to reproduce it, please?</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

ptrblck: <aside class="quote no-group quote-post-not-found" data-username="Kana-alt" data-post="3" data-topic="127123">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kana-alt/48/40422_2.png" class="avatar"> Kana-alt:</div>
<blockquote>
<p>If I reproduce it so that it works, it looks like this.</p>
</blockquote>
</aside>
<p>I’m not sure I understand the sentence correctly. Are you able to reproduce the all-zero output using:</p>
<pre data-code-wrap="python"><code class="lang-python">import kornia
import torch
opts=8
dis_norm=torch.rand([opts, 20, 642, 3])

a1=torch.zeros(160,1)
a2=torch.zeros(160,1)
a3=torch.zeros(160,1)
b=torch.ones(160,1)
ctl=torch.cat((a1,a2,a3,b),1).reshape(160,4)

dis_norm = dis_norm.matmul(kornia.quaternion_to_rotation_matrix(ctl).view(opts,-1,3,3)) # h,j,n,3
</code></pre>
<p>If so, could you post the output of <code>python -m torch.utils.collect_env</code> here, please?</p>
Usage of shared_cache in reductions.py

Abhishek_Singh1: <p>Hey Everyone</p>
<p>i have a question on the usage of shared_cache storage in “torch/multiprocessing/reductions.py”.<br>
specifically in the routine def rebuild_storage_fd(cls, df, size):</p>
<p>if i disable cache lookup “storage = storage_from_cache(cls, fd_id(fd))” in rebuild_storage_fd(cls, df, size).<br>
it will end up allocating new storage “cls._new_shared_fd(fd, size)” instead of reading from the cache.</p>
<p>will disabling the shared_cache lookup have any other side effect on training job &amp; its outcome?<br>
are there any other usage of the shared_cache lookup in rebuild_storage_fd(cls, df, size) apart from data loader scenario?</p>
Compute scores for only a subset of inputs

Adamits: <p>Hi!</p>
<p>I am working on an SRL implementation, wherein the goal is to compute a tag for every possible predicate x span combination. This means that we can end up with an intractable scoring space of</p>
<p>T1 = batch_size x |tokens| x |tokens^2| x |num_tags|</p>
<p>where |tokens| is for predicates and |tokens^2| is for spans</p>
<p>In practice, papers report pruning the predicates and spans to sets of top-k candidates, and computing scores for only these. I can get these top-k no problem, but the issue is that I then need to compute a loss for a gold tensor that does not match the indices of the top-k predicates and top-k spans.</p>
<p>I <em>think</em> the solution is to simply expand the batch_size x (|top-k predicates| * |top-k spans|) x |tags| tensor back into its original size matching the shape tensor T1 above, where everything that did not receive a score in the top-k combinations is masked with 0’s.</p>
<p>The approach I was thinking I need is to instantiate the tensor T1 to all 0’s:</p>
<pre><code class="lang-auto">T1 = torch.zeros(batch_size, |tokens| * |tokens^2|, |num_tags|)
</code></pre>
<p>,  and then use torch.scatter to insert the scores I computed:</p>
<pre><code class="lang-auto">T1.scatter(dim=?, index=i, src=top-k-combos_tensor)
</code></pre>
<p>But I am not sure how to easily index in such a way. Is there an easier way to achieve what I described here without running the entire space of computations? Or otherwise, can someone describe how I can use torch.scatter correctly here? I may be having some trouble understanding the <code>scatter</code> method</p>

Adamits: <p>Update: I <em>think</em> <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.masked_scatter_.html#torch.Tensor.masked_scatter_" rel="noopener nofollow ugc">masked_scatter</a> will solve my problem.</p>
Image not displaying properly

Abdul-Abdul: <p>I have the following code portion:</p>
<pre><code>dataset = trainDataset()
train_loader = DataLoader(dataset,batch_size=1,shuffle=True)

device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')

images = []
image_labels = []

for i, data in enumerate(train_loader,0):
    inputs, labels = data
    inputs, labels = inputs.to(device), labels.to(device)
    inputs, labels = inputs.float(), labels.float()
    images.append(inputs)
    image_labels.append(labels)

image = images[7]
image = image.numpy()
image = image.reshape(416,416,3)
img = Image.fromarray(image,'RGB')
img.show()
</code></pre>
<p>The issue is that the image doesn’t display properly. For instance, the dataset I have contains images of cats and dogs. But, the image displayed looks as shown below. Why is that?</p>
<p><a href="https://i.stack.imgur.com/tBDB4.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/tBDB4.png" alt="enter image description here" width="496" height="500"></a></p>

ptrblck: <p>Assuming the original image is in the channels-first memory layout, this line of code:</p>
<pre><code class="lang-python">image = image.reshape(416,416,3)
</code></pre>
<p>would interleave the data and would thus result in the posted image.<br>
To permute the axes, use <code>image = image.permute(1, 2, 0)</code>.</p>
Convert fp32 tensor to fp16 in cuda kernel

ItalianSC: <p>Hi:<br>
I had a torchscript model with fp16 precision, so I must feed fp16 data to the model to do inference; I convert a fp32 image to fp16 in a cuda kernel,I use the “__float2half()” function to do the conversion,but “__float2half()” has more than one version,such as “__float2half”,"__float2half_rd","__float2half_rz",…,so which version should I use?</p>

ptrblck: <p>Assuming you don’t want or cannot transform the tensors to <code>torch.float16</code> and want to use the CUDA ops, it would depend on the rounding mode you want to apply.</p>
Matrix Reduction Help (Solved)

kjans123: <p>Hello, I have set of 30 xyz Cartesian coordinate points (30x3). There are 759 of these sets stacked on top of each other (759x30x3). I need to dot multiply them by another set of points so that the output is 759x30. I’ve tried torch.bmm, torch.matmul and torch.tensordot. So far, I’ve had no luck getting the correct output. Any push in the right direction would be hugely appreciated. Thanks! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

tom: <p>Either use einsum with bij,bij-&gt;bi or use unsqueeze(-2) and unsqueeze(-1) on the inputs of matmul and squeeze the result. (matmul reduces the second to last and last axes of the first and second input, respectively) or you could multiply with * and then sum at the expense of some time and memory.</p>
Complicated transform makes Pytorch dataloader extremely slow

rkd1137: <p>I’m performing a slightly complicated form of compression on the MNIST dataset, that involves sequentially taking QR decompositions of the input vector (i.e., start with an input vector of length L, split it into a vector with shape (2, L/2), take a QR decomposition of that, then repeat on the R matrix). Because it is sequential, it is hard to parallelize, and generally takes some nontrivial time to perform (about 1 second on my machine). This would be fine, but Pytorch does not appear to be doing any kind of caching when I do this – successive epochs are just as slow as the first one. I’m fairly new to Pytorch, so I wanted to ask if this is the expected behavior? If so, should I perform the transformation first, then save my own local copy of the transformed dataset, and make a new dataloader from that?</p>

ptrblck: <p>The <code>Dataset</code> doesn’t use any caching mechanism, as you would often lazily load each sample and transform it on-the-fly. Caching could thus disable the random transformations and could additionally blow up your memory (in case you want to store each sample).<br>
Assuming your transformation is not random, then your suggestion sounds valid and you could indeed process all samples once, store them, and load these samples in the <code>Dataset.__getitem__</code>.</p>

rkd1137: <p>Thanks. Is there a preferred method to do the processing? Right now, I thought to do something like:</p>
<pre><code class="lang-python">dataset = MNIST(f'/tmp/mnist/',
                download=True,
                transform=Compose([# List of transforms
                                   ]), train=True)

dl = DataLoader(dataset, batch_size=60000)
x, y = next(iter(dl))
torch.save(x, "train_data.pt")
torch.save(y, "train_targets.pt")

</code></pre>
<p>Is there a more efficient/preferred method?</p>

ptrblck: <p>If your entire dataset contains 60000 samples and they also fit into your memory (MNIST would, but I won’t know if you’ve used it as a placeholder), then your approach looks correct and you could later <code>torch.load</code> the data.<br>
However, in case your dataset is larger (and doesn’t fit into your RAM), you might want to store each sample or multiple samples in each file (the latter case might make the data loading a bit more complicated).</p>