akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
RNN - Number of Parameters

mmg: <p>Hi,</p>
<p>I am trying to find out the number of parameters for a PyTorch model</p>
<pre><code class="lang-auto">class SimpleRNN(nn.Module):
    def __init__(self):
        super(SimpleRNN, self).__init__()
        self.rnn = nn.RNN(input_size=1,
                         hidden_size=20,
                         num_layers=1,
                         nonlinearity='tanh',
                         batch_first=True
                         )
        self.fc = nn.Linear(20, 1)
    def forward(self,X):
        out, _ = self.rnn(X)
        out = self.fc(out[:,-1, :])
        return out
        
model_sr = SimpleRNN()

from torchsummary import summary
summary(model_sr, (50,1))
</code></pre>
<p>But it says 0 params for the RNN layer! Is there a way for me to find out? (e.g. in Keras’ model.summary(), it does give a number)</p>
<p>Thanks</p>

shivammehta007: <pre><code class="lang-auto">def count_parameters(model):
    return sum(param.numel() for param in model.parameters())
</code></pre>
<p>This function will help you to count the number of parameters in your model</p>
How to get centroid in prototypes?

oasjd7: <p>I have prototypes and each class-label.<br>
I want to get centroid of each class. How to compute centroid ?<br>
Here is the shape of prototypes.</p>
<pre><code class="lang-auto">print(prototypes.shape): [64,31] # batch_size : 64, number of class: 30
</code></pre>

ksz: <p>Please better describe your problem, maybe by showing some examples. I can understand your problem in several different ways.</p>
<p>I guess that you need: <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor.permute" class="inline-onebox" rel="noopener nofollow ugc">torch.Tensor — PyTorch 1.7.1 documentation</a></p>

oasjd7: <p>Actually, that’s all I explained…<br>
I already have prototypes so I just need to compute centroids of these prototypes according to the labels.</p>
<pre><code class="lang-auto">for inputs, labels in train_loader:
   prototypes = network(inputs) # network: Resnet + FC layer.
   # I want to compute centroids of each class.
</code></pre>
<p>I think computing centroid is something like this…</p>
<pre><code class="lang-auto">for i, prototypes, labels enumerate(all_samples):
   if label == i:
      centroid[i] = prototypes / number_of_class_i
</code></pre>
<p>But I’m not sure about this.</p>

ksz: <pre><code class="lang-auto">import torch.nn

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.conv1 = nn.Conv2d(3, 10, 3)
        self.conv2 = nn.Conv2d(10, 20, 3)
        self.conv3 = nn.Conv2d(20, 30, 3)
        self.maxpool = nn.MaxPool2d((2,2))
        self.linear = nn.Linear(120, 10)


    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = self.maxpool(x)
        
        x = F.relu(self.conv2(x))
        x = self.maxpool(x)
        
        x = F.relu(self.conv3(x))
        x = self.maxpool(x)
        
        x = x.view(x.shape[0], -1)
        x = self.linear(x)
        return x


network = Model() #example

res = []
for i in range(8): #should be: for inputs, labels in train_loader:
    inputs = torch.rand(32, 3, 32, 32) #example
    prototypes = network(inputs) # network: Resnet + FC layer.
    prototypes = prototypes.permute(1, 0)
    res.append(prototypes)
    
res = torch.cat(res, dim=1)
print(res.shape)
print(res.mean(dim=1).shape)
</code></pre>
<p>Here the mean is calculated over all outputs from the network - not the highest one. In the ideal situation - where there are no biases and the dataset is well balanced - all values should be similar.</p>
Convolutional layers weights are not saved after training

armando: <p>I wrote a model using convolutional filters of different sizes, from 1 to max_size, and I implemented it the following way:</p>
<pre><code class="lang-auto">def __init__(
    self,
    embedding_dim,
    hidden_dim=200,
    dropout_rate=0.3,
    max_conv_size=3,
    criterion=nn.L1Loss(),
    optimizer = optim.Adadelta,
):
    """ This architecture performs 2D convolutions using 1D kernels with size 1,...,max_conv_size """
    super(AuthorConvNet, self).__init__()
    self.criterion = criterion
    self.optimizer = optimizer
    self.dropout = nn.Dropout(dropout_rate)
    self.max_conv_size = max_conv_size
    self.max_pool = lambda x: torch.max(x, dim=len(x.size())-2)[0]    # max_pool here is the max over the input raws

    self.embedding_dim = embedding_dim
    self.hidden_dim = hidden_dim
    # max_conv_size 1D convolutional filters of size 1,...,max_conv_size
    self.convs = [nn.Conv2d(1,1,(i,1)) for i in range(1, self.max_conv_size+1)]
    self.relu = nn.ReLU()
    self.linear1 = nn.Linear(embedding_dim*max_conv_size, hidden_dim)
    self.linear2 = nn.Linear(hidden_dim, hidden_dim)
    self.linear3 = nn.Linear(hidden_dim, 1)

def forward(self, x):
    z = x.unsqueeze(1)   # second dimension corresponds to in_channels, 
                         # so it should be 1 (1st is batch  size)
    C = []
    for convolution in self.convs:          # apply convolution for each kernel size between 1 and q
        conv = convolution(z).squeeze()
        C.append(conv)
    H = [self.relu(conv) for conv in C]
    p = [self.max_pool(h) for h in H]
    p = torch.cat(p, dim=len(p[0].size())-1)
    # multi-layer net
    out = self.linear1(p)
    out = self.dropout(out)
    out = self.relu(out)

    out = self.linear2(out)
    out = self.dropout(out)
    out = self.relu(out)
    
    out = self.linear3(out)
    return out
</code></pre>
<p>However, after training, it appears that using the model for predictions yield different results for same input. So I looked at the state_dict saved from training and only linear layers weights are saved (which explains why I obtain different results each time, the network convolutions are reinitialized randomly at each time). Does someone may have an explaination for this ?</p>

Eta_C: <p>I do not know your input shape, but you can try to rewrite</p>
<pre><code class="lang-python">self.convs = [nn.Conv2d(1,1,(i,1)) for i in range(1, self.max_conv_size+1)]
</code></pre>
<p>such as</p>
<pre><code class="lang-python">self.convs = nn.ModuleList([nn.Conv2d(1,1,(i,1)) for i in range(1, self.max_conv_size+1)])
</code></pre>

armando: <p>Yes, that is exactly what I was looking for ! Thanks !</p>
How to select and identify objects through the computer camera

miller_eric: <p>I built a simple neural network myself，It can identify the type of fruit.<br>
I want to import this model and use it to call the computer’s camera to recognize something.<br>
Achieve the effect of the following pictures,Make it possible to box out objects and label them.<br>
Where should I get the relevant study materials.</p>

miller_eric: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/b/6/b686f29a2b4fb09ccf46b1a395546258727959b0.png" data-download-href="https://discuss.pytorch.org/uploads/default/b686f29a2b4fb09ccf46b1a395546258727959b0" title="QQ截图20210227193959"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b686f29a2b4fb09ccf46b1a395546258727959b0_2_345x142.png" alt="QQ截图20210227193959" data-base62-sha1="q2I1sWWoofXBhpNJJ1TzUD6h7BC" width="345" height="142" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b686f29a2b4fb09ccf46b1a395546258727959b0_2_345x142.png, https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b686f29a2b4fb09ccf46b1a395546258727959b0_2_517x213.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/b/6/b686f29a2b4fb09ccf46b1a395546258727959b0_2_690x284.png 2x" data-dominant-color="795241"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">QQ截图20210227193959</span><span class="informations">1217×503 604 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> <a>Uploading: QQ截图20210227193959.png…</a></p>

Eta_C: <p><a href="https://docs.opencv.org/3.4/d8/dfe/classcv_1_1VideoCapture.html" rel="noopener nofollow ugc">cv2.VideoCapture</a></p>
How to find the equal element in more than 3 tensors

B.W_Zhang: <p>for example,</p>
<pre><code class="lang-auto">a = torch.tensor([[0,1],[2,3]])
b = torch.tensor([[4,1],[2,3]])
c = torch.tensor([[5,1],[8,8]])
</code></pre>
<p>since the element with value 1 is equal in all 3 tensors, I want to get the result like</p>
<pre><code class="lang-auto">result = torch.tensor([[0,1],[0,0]])
</code></pre>
<p>or</p>
<pre><code class="lang-auto">result = torch.tensor([[False,True],[False,False]])
</code></pre>

GOBINATH_A: <p>Try this<br>
compare = (a==b)==c<br>
print(compare)<br>
tensor([[False,  True],<br>
[False, False]])</p>

B.W_Zhang: <p>Thank you for the suggestion.<br>
I think this happens to be correct because c has value 1 at c[0,1], and a==b returns True at position [0,1].<br>
If they all have, say value 3 at the position [0,1] then this will not work.</p>
<p>What’s more important is I may also want to compare among more than 3 tensors, like choosing equal elements in 5 tensors. In that case, judging one by one may seem too wordy and inefficient.</p>

Eta_C: <pre><code class="lang-python">mask = (torch.stack([a, b, c], dim=0) == a).sum(dim=0) == 3
</code></pre>
<p>If <code>a, b, c</code> comes from a larger tensor <code>d = torch.rand(3, 2, 2)</code>, do</p>
<pre><code class="lang-python">mask = (d == d[0]).sum(dim=0) == d.shape[0]
</code></pre>

B.W_Zhang: <p>Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

GOBINATH_A: <p>Thanks your solution is good and works in all case.<br>
I think below while be better<br>
mask = (torch.stack([b, c], dim=0) == a).sum(dim=0) == 2</p>
Inplace operation error about indexing or tensor

blackbeanman: <p>When I am trying to run my code on a new machine(on another machine it works well), there occurs an error about auto_grad and in place operation. The torch.autograd.set_detect_anomaly find that the error occurs at: F[b][ind[b][i]] += similarity[b][i][j] * Fij.permute(1, 0), where F,  Fij are tensors and ind[b][i], similarity[b][i][j] are numpy ndarrays. The gradient will pass through the F, because my code is actually backprop on some region Fij of the tensor F. The regions are controlled by the index ndarray ind[b][i]</p>
<p>The exception message:<br>
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1024, 512]], which is output 0 of SelectBackward, is at version 31; expected version 30 instead.</p>
<p>my code:</p>
<pre><code class="lang-auto">class net(...):
    def forward(similarity, ...other params):
        F = torch.zeros(another_feature.shape).to(DEVICE)
        for b in range(...):
            for i in range(...):
                for j in range(...):
                    Fij = ... 
                    F[b][ind[b][i]] += similarity[b][i][j] * Fij.permute(1, 0)
      return F
</code></pre>
<p>The error is triggered by F[b][ind[b][i]] += similarity[b][i][j] * Fij.permute(1, 0)</p>
<p>Does anyone know how to solve this problem?<br>
Maybe this could be computed by mask select in pytorch</p>

blackbeanman: <p>The aim of my code is to compute the gradient with regard to some tensor regions. So I firstly created the zero tensor F and then update some regions(F[b][ind[b][i]) of it.</p>
Indexing 3D tensor using 2D tensor

nasa203: <p>New with Pytorch here,</p>
<p>I have a 3D tensor <code>(Batch X features X attributes)</code>, and an indexing tensor of 2D <code>(Batch X features)</code>.<br>
In the indexing tensor each row in the batch is the index of the desired attribute of that feature (for example:<code> indexing = torch.tensor([[1, 2],[2, 5],[5, 2]])</code> a tensor of 3X2 size. For the first row I want attribute 1 for the first feature and 2 attribute for the second feature, for the second row I want attribute 2 for the first feature and 5 attribute for the second feature etc…).</p>
<p>The output tensor should be of size <code>(Batch X features)</code>.</p>
<p>I tried to use <code>.gather</code> but I am not sure it is right for this, or which <code>dim</code> to use it on.</p>
<p>How can I achieve that?</p>

Eta_C: <pre><code class="lang-python">torch.gather(x, 2, index[..., None])
</code></pre>

nasa203: <p>Thanks a lot!<br>
This worked very well.</p>
Column-wise matrix multiplication

mattief: <p>I have an M x N matrix and would like to multiply each M x 1 column by an L x M matrix with learnable elements, yielding N L-dimensional vectors. Is there a simple way to do this, something like * but for matrix multiplication? Bonus: I’d ultimately like to return an (N.L) stacked vector where each column multiplication is stacked on top of each other. Any help to do this simply (and such that gradients can pass through these operators) would be amazing <img src="https://discuss.pytorch.org/images/emoji/apple/ok_hand/2.png?v=9" title=":ok_hand:t2:" class="emoji" alt=":ok_hand:t2:"></p>

Eta_C: <p>It sounds like a normal matrix multiplication?</p>
<pre><code class="lang-python">M, N, L = 3, 4, 5
input = torch.rand(M, N).float()
weight = nn.Parameter(torch.rand(L, M).float(), True)
result = torch.mm(weight, input).t()
print(result.shape)
</code></pre>
Conversion to LongTensor

nrr1509: <p>What happens to the tensor when I change its type from <strong>Float</strong> to <strong>Long</strong><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/4/3484e668fc69ff9a4ac06cef5ee83a3314221517.png" data-download-href="https://discuss.pytorch.org/uploads/default/3484e668fc69ff9a4ac06cef5ee83a3314221517" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3484e668fc69ff9a4ac06cef5ee83a3314221517_2_690x498.png" alt="image" data-base62-sha1="7uBz3IPUBIrjTmikrRcnmdDgVAH" width="690" height="498" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3484e668fc69ff9a4ac06cef5ee83a3314221517_2_690x498.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/3484e668fc69ff9a4ac06cef5ee83a3314221517_2_1035x747.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/4/3484e668fc69ff9a4ac06cef5ee83a3314221517.png 2x" data-dominant-color="B3B3B3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1104×797 91.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>This is supposed to be a segmentation mask of an image, and it gets completely wrecked when I change it to LongTensor.</p>

Unity05: <p>Hi, If you convert your float tensor to long tensor (I assume you’re storing values between 0.0 and 1.0 in your float tensor) all your values get rounded off (in your example to 0, what’s the reason for the plot being black only). To solve this, you could multiply your original float tensor with a appropriate value before converting it to long.<br>
<code>imshow()</code> also has the <code>vmin</code> and <code>vmax</code> parameters to specify the range, however by default it takes the range of values of the given data, so that should work anyways.</p>
<p>Regrads.</p>

Eta_C: <pre><code class="lang-python">ax[0].imshow(tensor, cmap="gray")
ax[1].imshow((tensor * 255).long(), cmap="gray")
</code></pre>
Error in iterating over dataloader

Nathaniel_Tan: <p>I am trying to train a BERT-base model for text classification and sentiment analysis. I managed to create the dataloader objects but when I iterate through them (either through the iter() function or for loops), I get this error, followed by my IPython console hanging (running on Spyder)</p>
<pre><code class="lang-auto">2021-01-26 14:10:36.370089: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations:  AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-01-26 14:10:36.528957: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
2021-01-26 14:10:36.780247: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
2021-01-26 14:10:36.805938: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
2021-01-26 14:10:40.337708: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
2021-01-26 14:10:40.456556: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 93763584 exceeds 10% of free system memory.
Traceback (most recent call last):
  File "&lt;string&gt;", line 1, in &lt;module&gt;
  File "C:\Users\mcnat\anaconda3\envs\ds\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\mcnat\anaconda3\envs\ds\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
AttributeError: Can't get attribute 'CommentsDataset' on &lt;module '__main__' (built-in)&gt;
</code></pre>
<p>Any ideas on how to deal with this error?</p>
<p>(I have been following this tutorial (<a href="https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/" rel="noopener nofollow ugc">https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/</a>), with some modifications as I am using it with my own dataset for survey comments classification)</p>

MJimitater: <p>Hi <a class="mention" href="/u/nathaniel_tan">@Nathaniel_Tan</a> , interesting question, I also ran into a similar problem. Could you find a solution yet?</p>
The fastest way to allocate temp memory on GPU in cpp extension

huangdi: <p>Hi, all.</p>
<p>I’ve meet a speed problem when allocating temp memory in cpp/cuda extension.</p>
<p>I’m writing an cuda operator, which needs <strong>temp memory</strong> during gpu computing. However, after measuring the speed of the operator, I found that the slowest part in the operator is the allocation of the temp memory. Moreover, allocating it in python is much slower than allocating it in cuda. The examples are shown below, and the time is measured by <code>time.perf_counter()</code>.</p>
<p>Code 1: Allocating inside the python warper with zero initialization.</p>
<pre><code class="lang-python">tmp_buffer1 = torch.zeros(4, 1, 20, 20, 20, 32).cuda()
tmp_buffer2 = torch.zeros(4, 32, 32).cuda()
tmp_buffer3 = torch.zeros(4, 20, 20, 20, 20, 32).cuda()
#######  my operator... ########
</code></pre>
<p>result:</p>
<pre><code class="lang-auto">time: 0.0035348859382793307
</code></pre>
<p>Code 2: Allocating inside the python warper without initialization.</p>
<pre><code class="lang-python">tmp_buffer1 = torch.Tensor(4, 1, 20, 20, 20, 32).cuda()
tmp_buffer2 = torch.Tensor(4, 32, 32).cuda()
tmp_buffer3 = torch.Tensor(4, 20, 20, 20, 20, 32).cuda()
#######  my operator... ########
</code></pre>
<p>result:</p>
<pre><code class="lang-auto">time: 0.0029048159390687943
</code></pre>
<p>Code 3: Allocating inside the cuda code without initialization.</p>
<pre><code class="lang-auto">float *tmp_buffer1 = nullptr;
float *tmp_buffer2 = nullptr;
float *tmp_buffer3 = nullptr;
cudaMalloc((void**)&amp;tmp_buffer1, 4*1*20*20*20*32*sizeof(float));
cudaMalloc((void**)&amp;tmp_buffer2, 4*32*32*sizeof(float));
cudaMalloc((void**)&amp;tmp_buffer3, 4*20*20*20*20*32*sizeof(float));
/////// some kernels for computing... ///////
cudaFree(tmp_buffer1);
cudaFree(tmp_buffer2);
cudaFree(tmp_buffer3);
</code></pre>
<p>result:</p>
<pre><code class="lang-auto">time: 0.000754036940634244
</code></pre>
<p>Is there anything like <code>context-&gt;allocate_temp()</code> in Tensorflow cpp extension?</p>
<p>What’s the fastest way of doing this?</p>

albanD: <p>Hi,</p>
<p>Which <code>time</code> library are you using? Is it cuda aware and doing the proper synchronization (via torch.cuda.synchronize() for example)?<br>
Also your code creates the Tensors on cpu then move them to the gpu, you can do the following to avoid this: <code>torch.zeros(4, 1, 20, 20, 20, 32, device="cuda")</code></p>
<p>The problem with cudaMalloc/Free is that they need to access the GPU and so can be fairly slow. Allocating a Tensor should be faster because we use a custom allocator.</p>

huangdi: <p>Thanks. The problem has been found to be that I did not use device=“cuda” but use .cuda(), which costs a lot of time.</p>
I don't understand something that I encounter when I read the official documentation

111466: <p>When I read the official documentation of maxpool2d,it looked like this</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/9/394db0525d8002ddaa17d8fd46b9500666ad7d65.png" data-download-href="https://discuss.pytorch.org/uploads/default/394db0525d8002ddaa17d8fd46b9500666ad7d65" title="{I5F~XRC2MW@DARN6ER91B"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/9/394db0525d8002ddaa17d8fd46b9500666ad7d65_2_690x285.png" alt="{I5F~XRC2MW@DARN6ER91B" data-base62-sha1="8aVF7nmD0NjailA90MfrK18ehKd" width="690" height="285" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/9/394db0525d8002ddaa17d8fd46b9500666ad7d65_2_690x285.png, https://discuss.pytorch.org/uploads/default/optimized/3X/3/9/394db0525d8002ddaa17d8fd46b9500666ad7d65_2_1035x427.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/9/394db0525d8002ddaa17d8fd46b9500666ad7d65.png 2x" data-dominant-color="F2F1F1"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">{I5F~XRC2MW@DARN6ER91B</span><span class="informations">1190×493 49.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
I don’t understand the meaning of union and[T,] and tuple[T,]</p>

nitaifingerhut: <p>you can either use this argument as a single variable of type T or as a tuple containing variables of type T.<br>
for eg: you can use as kernel_size=2 or as kernel_size=(2,3)</p>

Eta_C: <p>Check <a href="https://docs.python.org/3/library/typing.html" rel="noopener nofollow ugc">python docs</a> for more infomations.</p>

111466: <p>Thank you  very much!</p>
ResNet34 to estimate occlusion Probability

Samiha: <p>In the MOT-20 dataset’s ground truth, a visibility score is given which denotes how much of the object is visible. I want to crop objects from this dataset and input it to ResNet-34 with a visibility score. The goal is to learn to predict the visibility score when an object is given. To perform this task, what features could be added to ResNet so that it can learn if an object is occluded and its occlusion score? How should I design the loss function?</p>
Concatenating Dataloader

John_Grabner: <p>What is the best way to combine Dataloaders that have different batch sizes and different data shapes while still preserving the multi-process capabilities of the Dataloader?</p>
<p>dl1 = Dataloader(dataset1, batch_size=3)<br>
dl2 = Dataloader(dataset2, batch_size=5)</p>
<p>where data from dl1<br>
data_a.shape = (3, 7) , where 3 is the batch size from dl1<br>
data_b.shape = (3, 11)</p>
<p>where data from dl2<br>
data_c.shape = (5, 13), where 5 is the batch size from dl2</p>
<p>I would like the combined data to be a simple joining of the data<br>
data_a.shape = (3, 7), where 3 is the batch size from dl1<br>
data_b.shape = (3, 11), where 3 is the batch size from dl1<br>
data_c.shape = (5, 13), where 5 is the batch size from dl2</p>

ptrblck: <p>You should be able to iterate these <code>DataLoader</code>s together e.g. via <code>zip</code>:</p>
<pre><code class="lang-python">datasetA = TensorDataset(torch.randn(12, 7), torch.randn(12, 11))
loaderA = DataLoader(datasetA, batch_size=3)

datasetB = TensorDataset(torch.randn(10, 13))
loaderB = DataLoader(datasetB, batch_size=5)

for (dataA, dataB), (dataC,) in zip(loaderA, loaderB):
    print(dataA.shape, dataB.shape, dataC.shape)
</code></pre>
<p>Also, <code>itertools</code> might provide you some methods, which e.g. could iterate the longest <code>DataLoader</code> by cycling the shorter one or by returning <code>None</code> instead (the standard <code>zip</code> would use the shortest loader).</p>

John_Grabner: <p>I was making things too complicated in my mind for nothing mostly because I though<br>
Dataloader returned something more than an iterator.</p>
<p>Thank you</p>
How to recycle gradient tensors allocated in C++ during backprop

peiyunh: <p>Hi folks,</p>
<p>I wrote a C++/CUDA extension to produce a gradient tensor, e.g. <code>dLdX</code>, where <code>L</code> is the loss and <code>X</code> is a tensor. When I call <code>X.backward(dLdX)</code>, the memory dLdX occupies (allocated in C++) will not get recycled. My question is: how can I tell the computational graph to recycle <code>dLdX</code> once it is no longer needed during the execution of <code>X.backward(dLdX)</code>.</p>
<p>Thanks!</p>

ptrblck: <p>How did you check, that the memory is not recycled? Was <code>torch.cuda.memory_allocated()</code> after the operation higher and stayed this way?<br>
If so, are you storing any references or in your extension?</p>

peiyunh: <p>Thank you so much for the reply!</p>
<p>If I call <code>torch.cuda.memory_allocated()</code> after <code>X.backward(dLdX)</code>, it does release <code>dLdX</code>'s GPU memory. But I am wondering if there is a way to do that in the middle of the backward pass. The analogy would be as if X is an intermediate variable whose gradient gets released as soon as it is no longer needed for the rest of the backprop. Am I making sense?</p>
<p>I made <a href="https://discuss.pytorch.org/t/how-to-recycle-the-gradient-tensor-when-calling-backward/113290">another post</a> after I realized I was not asking the right question – this behavior is not specific to tensors allocated in C++.</p>
Torch.tensordot - Dot product in Tensor along 3rd dim

quornmd: <p>I am two tensors <code>A</code>, <code>B</code>, each of shape <code>n x n x m</code> and I would like to get a tensor with dimensions <code>n x n</code> where each entry is the dot product of long the third dimension, i.e. $$C_{ij}=\sum_k A_{ijk}B_{ijk}$$.</p>
<p>Example:</p>
<pre><code class="lang-auto">A=B=[[[ 0.,  1.,  2.,  3.,  4.], [ 5.,  6.,  7.,  8.,  9.]],
        [[10., 11., 12., 13., 14.], [15., 16., 17., 18., 19.]]]
</code></pre>
<p>I would like to get</p>
<pre><code class="lang-auto">[[  30.,  255.],
[ 730., 1455.]]
</code></pre>
<p>Code:</p>
<pre><code class="lang-auto">a = torch.arange(20.).reshape(2, 2, 5)
b = torch.arange(20.).reshape(2, 2, 5)
</code></pre>
<p>to</p>
<pre><code class="lang-auto">for i in range(2):
    for j in range(2):
        d[i,j] = a[i,j].dot(b[i,j])
</code></pre>
<p>using a single operation</p>

quornmd: <p>Overlooked the most obvious way to solve this: <code>c = (a*b).sum(2)</code></p>
How to recycle the gradient tensor when calling backward()

peiyunh: <p>Hi folks,</p>
<p>I am wondering if there is a good way to recycle the gradient tensor when calling backward(). Here is an example.</p>
<pre><code class="lang-auto">import torch
x = torch.rand(4,4)
w = torch.nn.Linear(4,4)
with torch.set_grad_enabled(True):
    y = w(x)
    dy = torch.rand(4,4)
    y.backward(dy)
</code></pre>
<p>I am hoping to recycle <code>dy</code> in the middle of <code>y.backward()</code> as soon as it is no longer needed for the chain rule. How can I do that?</p>
<p>Thanks!</p>
Adding class label to Autoencoder

mattt: <p>Hey PyTorch community,</p>
<p>I am using an Autoenoder for reconstruction purposes. My current problem is that I want to take the class label into account. So, during the training Autoenoder knows that which type of data (class) is using by inputting a class label (or probability of multiple classes). During the testing, by having that label class (or probability of multiple classes) the Autoenoder can reconstruct but based on the class pre-knowledge.  I do not know how to implement the network or loss function in this case. Would you please help me?</p>
Learning to Sample from Complex Distribution (High-Dimensional)

Kamer_Ali_Yuksel: <p>Hello everyone,</p>
<p>I would like a NN to learn how to sample from a complex distribution in high-dimensions (e.g. 10-100K dimensions).</p>
<p>I guess that would be possible via learning parameters of a multivariate normal distribution e.g. MultivariateNormal</p>
<p>However, in high-dimensional problems, the parameter-size becomes too much and this becomes memory inefficient.</p>
<p>Can you please suggest to me an alternative if it exists in the framework or another method in order for achieving this?</p>
<p>Note: The scalability of the method is the most crucial aspect for me, the resulting expressiveness is not that important.</p>
<p>Thank you very much.</p>
<p>Sincerely,<br>
Kamer</p>

KFrank: <p>Hi Kamer!</p>
<aside class="quote no-group" data-username="Kamer_Ali_Yuksel" data-post="1" data-topic="113224" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kamer_ali_yuksel/48/7641_2.png" class="avatar"> Kamer_Ali_Yuksel:</div>
<blockquote>
<p>I would like a NN to learn how to sample from a complex distribution in high-dimensions (e.g. 10-100K dimensions).</p>
</blockquote>
</aside>
<p>I don’t think that neural networks are a natural approach for sampling<br>
from high-dimensional distributions – not that I know what your<br>
specific use case is.</p>
<p>Your distribution is described by a probability-destiny function that is<br>
a function of the 10-100K coordinates of your high-dimensional space.<br>
Can you compute reasonably efficiently the <em>changes</em> in the PDF as<br>
you take “simple” steps that move you about in that space?</p>
<p>If so, the “standard” approach for sampling from such a distribution<br>
is a <a href="https://en.wikipedia.org/wiki/Markov_chain_Monte_Carlo" rel="noopener nofollow ugc">Markov-chain Monte Carlo</a>.</p>
<p>Best.</p>
<p>K. Frank</p>

Kamer_Ali_Yuksel: <p>Hello Frank,</p>
<p>Thank you for your response. I was also recently looking into MCMC sampling methods and will look deeper after your advise. Normalizing Flows seems to be the most popular approach but I am unsure if they’re scalable enough for my use-case or would need too much additional parameters to be trained.</p>
<p>Sincerely,<br>
Kamer</p>
How to use pretrained weights to train a extended network architecture

szahan: <p>Hi I am designing a architecture that contains two modules (GCN+RNN) in the class definition. I will train it for around 100 epochs. I know I can save the model using torch.save and use a pretrained model using torch.load. But how can I use the weights from the pretrained model to initialize the same of modules of an extended network. For example, say my new network contains three modules (GCN+RNN+CNN) with the final fc layer being the last layer.</p>

szahan: <p>I have used the following codes to update part of the models states and it works</p>
<p>data = torch.load(PATH)<br>
pretrained_states = data[‘model_state_dict’]<br>
model_dict = model.state_dict()<br>
pretrained_states = {k: v for k, v in pretrained_states.items() if k in model_dict}</p>
<p>model_dict.update(pretrained_states)<br>
model.load_state_dict(model_dict)</p>
<p>but when I am trying to load the optmizer state_dict() it gives an error<br>
“loaded state dict contains a parameter group that doesn’t match the size of optimizer’s group”</p>
<p>I am trying to update optimizer state the same way but its not working.</p>
How to train a torchscripted model in pytorch?

fermat97: <p>I would like to train a model in Pytorch which has been already torchscripted. I am curious if it is possible at all to do so. I think it is possible with the <code>torch.jit.trace</code> but I am not sure if it is possible with the <code>torch.jit.script</code>. Any idea?</p>

ptrblck: <p>Yes, you should be able to train a scripted model (as long as you are able to script it).</p>
2D dilated convolution on 1x1xC tensor

jmontoyaz: <p>I’m trying to understand the lines below:</p>
<pre><code class="lang-auto">class MyModule(nn.Module):
    def __init__(self, in_channels, out channels):
        super(MyModule, self).__init__()
        self.avg_pool = nn.AdaptiveAvgPool2d(1)
        self.conv3_3 = nn.Conv2d(in_channels, out_channels, kernel_size=3, stride=1, padding=3, dilation=3)
    def forward(self, x):
        x_y = self.avg_pool(x)
        x_c = self.conv3_3(x_y)
        return x_c
</code></pre>
<p>which translates into: <code>CONV3_DIL3(AVG_POOL1(X))</code>. Now:</p>
<ul>
<li>Imagine that <code>X.shape = [1, 64, 16, 16]</code>, i.e. <code>BxCxHxW</code>.</li>
<li>After applying the  <code>X_Y=AVG_POOL1(X)</code>, we obtain <code>X_Y.shape = [1, 64, 1, 1]</code> =&gt; <code>1xCx1x1</code>
</li>
<li>Finally, after applying <code>X_C= CONV3_DIL3(X_Y)</code> we obtain <code>X_Y.shape = [1, 64, 1, 1]</code> =&gt; <code>1xCx1x1</code> because of the padding.</li>
</ul>
<p>Now, my questions are: (i) what does it exactly mean to apply a 2D Conv with a <code>3x3</code> filter with <code>dilation=3</code> into a <code>1xCx1x1</code> tensor, especially because there’s no spatial information in a flat <code>1xCx1x1</code> tensor (ii) why does this make sense at all?</p>

ptrblck: <p>I don’t think the larger kernel size, padding, and dilation would make any difference than applying a <code>1x1</code> kernel using the center kernel pixel.<br>
Here is a small code snippet to show the equivalence:</p>
<pre><code class="lang-python"># setup
x = torch.randn(1, 3, 24, 24)
avg_pool = nn.AdaptiveAvgPool2d(1)
conv3_3 = nn.Conv2d(3, 1, kernel_size=3, stride=1, padding=3, dilation=3)

# reference calculation
out_ref = conv3_3(avg_pool(x))

# using a kernel size of 1
conv_new = nn.Conv2d(3, 1, kernel_size=1)
with torch.no_grad():
    conv_new.weight.copy_(conv3_3.weight[:, :, 1:2, 1:2])
    conv_new.bias.copy_(conv3_3.bias)
out_new = conv_new(avg_pool(x))

# compare
print(torch.allclose(out_ref, out_new))
&gt; True
print((out_ref - out_new).abs().max())
&gt; tensor(0., grad_fn=&lt;MaxBackward1&gt;)
</code></pre>
Includes only some features of the dataset in a dataloader

Stefano_Barone: <p>I created a dataset and to creat a dataloader with Pytorch. I have the following problem, I would like to create a dataloader with only the features <strong>data</strong> and <strong>y_vector</strong>. As I apply torch.utils.data.random_split if I want the nth element of the test_dataset if I don’t include z_vector I can’t understand which element of z_vector I have to take. I do not have to give in input z_vector to the network and it seems useless to include it in the dataloader. How to do?</p>
<pre><code class="lang-auto">class Dataset():

    def __init__(self, X_np_array, y_np_vector,z_np_vector, transform=None): 

        self.transform = transform
        self.data = torch.Tensor(X_np_array[:,None, :, :])
        self.y_vector = torch.Tensor(y_np_vector)
        self.z_np_vector= torch.Tensor(z_np_vector)


    def __getitem__(self, index): 
        return self.data[index], self.y_vector[index],self.z_np_vector[index]

    def __len__(self): 
        return len(list(self.data))
    
    def train_test_split(self,dataset,splitting_percentage):

        torch.manual_seed(int(1996))
        train_dim = int(len(dataset)*splitting_percentage)
        test_dim = len(dataset)-train_dim
        train_dataset, test_dataset = torch.utils.data.random_split(dataset, [train_dim, test_dim])
        return train_dataset, test_dataset
</code></pre>

ptrblck: <p>I’m not sure I understand the issue correctly, so please add more information in case I misunderstand it indeed.</p>
<aside class="quote no-group" data-username="Stefano_Barone" data-post="1" data-topic="113123">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stefano_barone/48/31769_2.png" class="avatar"> Stefano_Barone:</div>
<blockquote>
<p>I would like to create a dataloader with only the features <strong>data</strong> and <strong>y_vector</strong>.</p>
</blockquote>
</aside>
<p>Wouldn’t removing <code>self.z_np_vector</code> work in this case? It seems, this particular tensor is also not needed in the model, so I’m unsure why it’s still there.</p>
<p>However, this description seems to point towards a usage of this tensor:</p>
<aside class="quote no-group" data-username="Stefano_Barone" data-post="1" data-topic="113123">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/stefano_barone/48/31769_2.png" class="avatar"> Stefano_Barone:</div>
<blockquote>
<p>I want the nth element of the test_dataset if I don’t include z_vector I can’t understand which element of z_vector I have to take.</p>
</blockquote>
</aside>
<p>Where is this tensor used, if not in the model?</p>
Floating point results in Android, TensorFlow and Pytorch are different

mustansarsaeed: <p>Hi, I am trying to compare the floating point numbers on Android, Tensorflow and Pytorch. What I have observed is I am getting same result for Tensorflow and Android but different on Pytorch as Android and Tensorflow are performing round-down operation. Please see the following result:</p>
<p><strong>TensorFlow</strong></p>
<pre><code class="lang-auto">import tensorflow as tf
a=tf.convert_to_tensor(np.array([0.9764764, 0.79078835, 0.93181187]), dtype=tf.float32)

session = tf.Session()
result = session.run(a*a*a*a)

print(result) 
</code></pre>
<p><strong>PyTorch</strong></p>
<pre><code class="lang-auto">import torch as th

th.set_printoptions(precision=8)

a=th.from_numpy(np.array([0.9764764, 0.79078835, 0.93181187])).type(th.FloatTensor)

result = a*a*a*a


print(result)
</code></pre>
<p><strong>Android:</strong></p>
<pre><code class="lang-auto">for (index in 0 until a.size) {
   var res = a[index] * a[index] * a[index] * a[index]
   result.add(res)
}

print("r=$result")
</code></pre>
<p><strong>The result is as follows:</strong></p>
<p><strong>Android</strong>:           [0.9091739,  0.3910579,  0.7538986]<br>
<strong>TensorFlow</strong>:     [0.9091739,  0.3910579,  0.7538986]<br>
<strong>PyTorch</strong>:          [0.90917391, 0.39105791, 0.75389862]</p>
<p>You can see that PyTorch value is different. I know that this effect is minimal in this example but when we are performing training, and we are running for 1000 rounds with different batches and epochs, this different can be accumulated and can show un-desirable results. Can anyone point out how can we fix to have same number on three platforms.</p>
<p>Thanks.</p>

ptrblck: <p>The difference seems to come from the different precision in the printing options.<br>
If you use <code>precision=7</code>, you’ll get the same values. Alternatively, increase the precision printing options in TF and your Android application to compare more decimals.</p>
VGG networks configurations

Sahil_malik: <p>Hi,</p>
<p>Sorry for a very basic question. I am little bit confused while reading the VGG paper. The paper shows its configurations from 11-19 . Were they released in the same year? Especially, VGG16 and VGG19 which came first and was applied to ImageNet dataset?</p>
<p>Thank you!</p>
How should I select random part of tensor?

hadaev8: <p>I want to feed part of the model’s output to auxiliary loss.</p>
<p>With it Im getting the error</p>
<pre><code class="lang-auto">x = torch.rand(32, 128, 1000)
lens = torch.randint(0, 1000, (32,))
mask_start = (torch.rand(32) * (lens - 64)).long()
x[:,:,mask_start:mask_start+64]

TypeError: only integer tensors of a single element can be converted to an index</code></pre>

Dwight_Foster: <p>can you print the shape of mask_start</p>

hadaev8: <p>Shape is torch.Size([32])</p>

Dwight_Foster: <p>Ok why are you making your mask start variable have a size of 32. You cannot do that for indexes. Is there a reason for 32.</p>

hadaev8: <p>Forum formatting eats first line, fixed it.<br>
32 is batch size.</p>

Dwight_Foster: <p>Ok you could just use a for loop like this:</p>
<pre><code class="lang-auto">for i in range(x.shape[0]):
   x[i, :, mask_start[i]: mask_start[i] + 64]</code></pre>

hadaev8: <p>It should damage performance</p>

Dwight_Foster: <p>Ya it would but that is the easiest option.</p>

hadaev8: <p>Well, i would not use loops in tensors.<br>
Seems like torch.gather is thing i need.<br>
But not sure how to generate tensor with all indeces i need.<br>
Like if mask start at 10 and ends at 74, i should have tensor with 10,11,12… 73, 73.</p>

Dwight_Foster: <p>Ya that should work.</p>
Question on a toy example, when .to(device) is necessary?

huangh12: <p>The following code is runnable and bug-free. I am puzzled why the <code>self.relu</code> dosen’t need to be cast to GPU device while the <code>self.net1</code> and <code>self.net1</code> is cast to GPU.  In my considering,  the default device of <code>self.relu</code> should be CPU if there is no explict casting.  If this is correct, the output of <code>self.net1</code> is on gpu, which mismatch the <code>self.relu</code> device type.  Why this is bug-free?</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.optim as optim


class ToyModel(nn.Module):
    def __init__(self):
        super(ToyModel, self).__init__()
        self.net1 = torch.nn.Linear(10, 10).to('cuda:0')
        self.relu = torch.nn.ReLU()
        self.net2 = torch.nn.Linear(10, 5).to('cuda:0')

    def forward(self, x):
        x = self.relu(self.net1(x.to('cuda:0')))
        return self.net2(x.to('cuda:0'))


model = ToyModel()
loss_fn = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.001)

optimizer.zero_grad()
outputs = model(torch.randn(20, 10))
labels = torch.randn(20, 5).to('cuda:0')
loss_fn(outputs, labels).backward()
optimizer.step()
</code></pre>

ptrblck: <p><code>nn.ReLU</code> doesn’t contain any parameters, so nothing has to be moved to the device.<br>
Internally the functional API will be used on the input (which is already on the GPU) via <code>F.relu</code>.</p>
<p>You can check for parameters in a modules via: <code>print(dict(module.named_parameters()))</code>.</p>
<p>Also, while this code works fine (and I guess it’s just used as an example for this topic), I would generally recommend to remove the <code>to</code> operations from the model internals, and move the complete model to the device via <code>model.to('cuda')</code>.<br>
If you want to use model sharding using different devices, your workflow would be correct.</p>
Paper behind Keypoint R-CNN implementation in torchvision

Francesco_Pochetti: <p>Hi team,<br>
I can’t seem to find the paper behind the <a href="https://pytorch.org/vision/0.8/models.html#keypoint-r-cnn" rel="noopener nofollow ugc">Keypoint R-CNN</a> implementation in <code>torchvision</code>.<br>
I’d like to better understand and study the architecture and I was looking for some support in the literature, please.<br>
Is it, actually, just the Mask R-CNN paper <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"> ?<br>
They mention Keypoints Detection in their work.</p>
<p>Thanks</p>

ptrblck: <p>The initial PR to create the Keypoint R-CNN was <a href="https://github.com/pytorch/vision/pull/921">this one</a>, where <code>mask_rcnn.py</code> was split into different files by <a class="mention" href="/u/fmassa">@fmassa</a>.<br>
Based on this I assume the Mask R-CNN paper was the base of these implementations, but let’s see what Francisco says.</p>
Curious behavior while getting partial derivatives

arijitthegame: <p>Hi all,</p>
<p>I am having a weird issue. Suppose I have some data say x = torch.rand(5,1) and some function f(x), g(x), say $f(x) =x^2$ and $g(x) = x^3$. I know how to take derivative (via autograd) of both the functions separately w.r.t. to x. But now suppose the data is given in the form (x,f(x)) and (x,g(x)) as described below.</p>
<pre><code class="lang-auto">x = torch.rand(5,1).requires_grad_(True)
def f(x):
  return x**2
def g(x): 
 return x**3
data_withf = torch.cat([x, f(x)], dim=1)
data_withg = torch.cat([x, g(x)], dim=1)
full_data = torch.cat([data_withf, data_withg], dim=0)
# Now I want to take the derivative of the 2nd coordinate of full_data w.r.t. x (i.e. first coordinate)
y = full_data[:,1].view(-1,1).requires_grad_(True)
input_x =  full_data[:,0].view(-1,1).requires_grad_(True)
grad(y, input_x, grad_outputs = torch.ones_like(input_x), 
                    create_graph=True, retain_graph=True, 
                   only_inputs=True,
                   allow_unused=True
                  )[0]
</code></pre>
<p>But this returns nothing. The reason to do this convoluted complicated way of doing is the following: I have some input x’s and a bunch of given functions f and I want to train a neural network using x and the outputs of the functions f and their derivatives. Is there some easy to fix my issue? If it helps, I am happy to write down how I want to train a Neural Network to use the x’s, f(x)‘s and f’(x)s.</p>
<p>Thank you.</p>
Create Tensor on CUDA device in C++

JoshVarty: <p>I have an array that I would like to place inside a Tensor that should live on the GPU.</p>
<p>I have previously done this with:</p>
<pre><code class="lang-auto">double array[] = { 1, 2, 3, 4, 5};
auto options = torch::TensorOptions().dtype(torch::kFloat64).device(torch::kCUDA, 1);
torch::Tensor tharray = torch::from_blob(array, {5}, options);
</code></pre>
<p>It has been awhile (2 years) since I used this approach but I seem to remember it working and it’s reasonably upvoted <a href="https://stackoverflow.com/a/58631791/300908" rel="noopener nofollow ugc">on StackOverflow</a>.</p>
<p>Today I tried running this code and I get the error:</p>
<pre><code class="lang-auto">terminate called after throwing an instance of 'c10::Error'
  what():  CUDA error: invalid argument
Exception raised from getDeviceFromPtr at ../aten/src/ATen/cuda/CUDADevice.h:13 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;) + 0x69 (0x7ffff7ba5eb9 in /home/josh/git/AlphaZeroCpp/libtorch/lib/libc10.so)
frame #1: &lt;unknown function&gt; + 0x31d398f (0x7fffb3cc998f in /home/josh/git/AlphaZeroCpp/libtorch/lib/libtorch_cuda.so)
frame #2: at::Context::getDeviceFromPtr(void*, c10::DeviceType) + 0xba (0x55555559f52c in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #3: at::from_blob(void*, c10::ArrayRef&lt;long&gt;, c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;) + 0xb7 (0x55555559fbac in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #4: at::from_blob(void*, c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;) + 0x86 (0x5555555a00c9 in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #5: torch::from_blob(void*, c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;)::{lambda()#1}::operator()() const + 0x70 (0x5555555a0fd2 in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #6: torch::from_blob(void*, c10::ArrayRef&lt;long&gt;, c10::TensorOptions const&amp;) + 0x57 (0x5555555a108a in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #7: main + 0x101 (0x5555555ab9ee in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
frame #8: __libc_start_main + 0xe7 (0x7fffaf760bf7 in /lib/x86_64-linux-gnu/libc.so.6)
frame #9: _start + 0x2a (0x55555558bf2a in /home/josh/git/AlphaZeroCpp/build/AlphaZeroCpp)
</code></pre>
<p>Has something changed regarding creating tensors from blobs? Am I doing anything obviously incorrect?</p>
<p>I can work around this by creating my tensor on the CPU and then moving it to the GPU, but I would like to understand why I can’t create the Tensor on the GPU directly?</p>