akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Is model(batch) FIFO or LIFO?

username: formidiable
cooked: <p>Hi,</p>
<p>I have trained reinforcement learning model with windowed time-series data and now i am getting inference. But i have a doubt that model(batch) works in order to Last IN First OUT. Cause, otherwise it seems like the last action is the first which always leave the action_list.</p>
<p>(An action is being decided in respect to the current one line and past 19 line of data.)</p>
<p>Is there anybody who knows which order the model function works in Pytorch, First in First out or Last in First Out?</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/formidiable">@formidiable</a> model generally works on the indices of the data. The lower index data is considered to be the first data point. Can you share your model and point out the section which gave you this doubt</p>
Detectron2 on a Raspberry pi

username: crytoy
cooked: <p>Hi there,<br>
Can detectron2 be installed and used on a Raspberry pi device?</p>

username: Cagri_Kaplan
cooked: <p>Hello. Did you find the answer to this question?</p>

username: harbe
cooked: <p>Hi Çağrı,Did you find the answer?</p>

username: Cagri_Kaplan
cooked: <p>Hey. Yes. We made it on a 64 bit OS raspberry pi.</p>
How to configure input dataset in RNN/LSTM Many to One problem

username: Seungmin_Lee
cooked: <p>Hello everyone</p>
<p>I’m currently working on RNN/LSTM many to one classification problem.</p>
<p>There are a 4 labels in total for classfication, and each label consists of a dataset with sizes of [30(time step), 6,500~11,000(batch size), 4(input(feature) size)].</p>
<p>In order to designate one label per matrix of [30 x 4] to organize the dataset required for training (x, y(label)), how should I configure the data set in the pytorch environment?</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/3/536185d01157f81aa1624e436e3fa9980ee2ea64.png" alt="image" data-base62-sha1="bTCzqM2uxw79Zc4TLPy7AMNUSYQ" width="271" height="237"></p>
Get intermediate layer outputs

username: pymon
cooked: <p>I have Alexnet neural network:</p>
<pre><code class="lang-auto">class AlexNet(nn.Module):
def __init__(self, num_classes=100):
    super(AlexNet, self).__init__()
    self.features = nn.Sequential(
        nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=3, stride=2),
        nn.Conv2d(64, 192, kernel_size=5, padding=2),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=3, stride=2),
        nn.Conv2d(192, 384, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.Conv2d(384, 256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.Conv2d(256, 256, kernel_size=3, padding=1),
        nn.ReLU(inplace=True),
        nn.MaxPool2d(kernel_size=3, stride=2),
    )
    self.classifier = nn.Sequential(
        nn.Dropout(),
        nn.Linear(256 * 6 * 6, 4096),
        nn.ReLU(inplace=True),
        nn.Dropout(),
        nn.Linear(4096, 4096),
        nn.ReLU(inplace=True),
        nn.Linear(4096, num_classes),
    )

def forward(self, x):
    x = self.features(x)
    x = x.view(x.size(0), 256 * 6 * 6)
    x = self.classifier(x)
    return x
</code></pre>
<p>I am trying to get the information of the intermediate layers (for example the penultimate layer ).<br>
Have I changed the forward function?</p>

username: paganpasta
cooked: <p>Maybe an existing <a href="https://discuss.pytorch.org/t/how-can-i-extract-intermediate-layer-output-from-loaded-cnn-model/77301/27">thread</a> and this <a href="https://pytorch.org/blog/FX-feature-extraction-torchvision/" rel="noopener nofollow ugc">blog</a> post help you.</p>
<p>I don’t see how you have changed the forward call ? At the moment your network’s forward return the output logits. If you wish you can return x just before classifier as well.</p>
Is this code for 3d dice loss correct?

username: Sai_Kishore
cooked: <pre><code class="lang-auto">def dice_loss(preds, labels, classes):
    """
    Masks are of the Size : (N,C,D,H,W)
    Labels are of the Size: (N,1,D,H,W)
    """
    softmax = nn.Softmax(dim=1)
    preds_prob = softmax(preds)
    preds_flat = preds_prob.view(-1, classes)

    labels_onehot = F.one_hot(labels, classes)
    labels_oh_flat = labels_onehot.view(-1, classes)

    smooth = 1e-7

    intersection = 2 * torch.sum(labels_oh_flat * preds_flat, axis=0) + smooth
    denominator = torch.sum(preds_flat * preds_flat, axis=0) + torch.sum(labels_oh_flat * labels_oh_flat,
                                                                         axis=0) + smooth
    dice_loss = -1 * torch.mean(intersection / denominator)

    return dice_loss
</code></pre>
Detect CUDA during cmake and use the python library in cpu mode

username: vitalite
cooked: <p>Hey there,</p>
<p>I wonder if it is possible to detect if CUDA is installed and turn pytorch into cpu mode if not (or not at the right version). At this time cmake run just fine with CUDA recent versions.</p>
<p>I use this lines in the CMakeLists.txt:<br>
execute_process(<br>
COMMAND bash -c “python3 -c ‘import torch;print(torch.utils.cmake_prefix_path, end = “”)’”<br>
OUTPUT_VARIABLE libtorch_local_path<br>
)</p>
<p>list(APPEND CMAKE_PREFIX_PATH “${libtorch_local_path}”)</p>
<p>What do you think?<br>
Thanks in advance!</p>
M1 pytorch wheels configuration

username: Lukasz_Wojciak
cooked: <p>Hi!</p>
<p>I was recently building the latest pytorch version from source and after running it I’ve noticed that the performance is worse than the 1.10.2 version I install from pypi.</p>
<p>I’ve decided to build 1.10.2 from source and test that - the results are the same.</p>
<p>I suspect this is an issue with my build configuration, is there a place where I can see exactly with what settings the pypi builds were created?</p>
<p>Cheers,<br>
Lukasz</p>
Load YoloV5 Model offline in PyTorch

username: Duc_Nguyen
cooked: <p>Hi PyTorch,</p>
<p>I’m learning about how to use pre-trained models in PyTorch.</p>
<p>I saw I can load a SSD model offline by: torchvision.models.detection.ssd_vgg16(), …</p>
<p>But I can not see a yoloV5 model. Why?</p>
<p>And how can I load a yoloV5 model offline like the SSD model?</p>
<p>Thank you so much</p>

username: ptrblck
cooked: <p>YoloV5 is not part of the <code>torchvision</code> repository, so you might want to check other repositories. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:"></p>

username: Duc_Nguyen
cooked: <p>Thank you.</p>
<p>But I see it’s really great when I can load SSD model by Torchvision.models.detection.ssd_vgg300().<br>
It’s good when I can not use the internet at my workplace.</p>
<p>So, Can I load YoloV5 model same the above way?<br>
Maybe not Torchvision, Which Torch can I load yoloV5 model in PyTorch?</p>

username: Duc_Nguyen
cooked: <p>Can anyone help me? Please</p>

username: thecho7
cooked: <p>Which torch is hard to understand but,<br>
You cannot load YOLOv5 model in the same manner.<br>
It means <strong>PyTorch does not support YOLOv5 through torchvision and other PyTorch libraries yet</strong></p>
<p>Please follow the instruction provided by YOLOv5 github. <a href="https://github.com/ultralytics/yolov5" rel="noopener nofollow ugc">https://github.com/ultralytics/yolov5</a></p>

username: Duc_Nguyen
cooked: <p>Thank you so much.</p>
<p>Another question for my curious,</p>
<p>PyTorch has torchvision and they create a lot of model inside this torchvision.models.</p>
<p>And we can create a model from SSD from Torchvision.models.detection.ssd()…</p>
<p>Why they don’t import yolov5 into Torchvision?</p>
<p>So we can use Torchvision.models.detection.yolov5().</p>
<p>Anyway, your answer helps me a lot.</p>
Ecosystem without reply

username: XIXIXIX
cooked: <p>I want to submit this project: <a href="https://github.com/benchenas/BenchENAS" rel="noopener nofollow ugc">benchenas/BenchENAS: BenchENAS is a platform to help researchers conduct fair comparisons upon Evolutionary algorithm based Neural Architecture Search (ENAS) algorithms. (github.com)</a><br>
I filled out the form before two weeks but dose not receive any reply.<br>
If did not come through,  would you send me the reason which is easy to improve.</p>
Does a sequence length of 1 for GRU makes sense?

username: fcvm
cooked: <p>I am planning to use a GRU in the following real-time scenario but having doubts since I have not really understood the effect of the sequence length of the input data.</p>
<p><strong>The scenario:</strong><br>
At training, the GRU would learn offline on recorded data from a sensor.<br>
At test time, the data samples from the sensor are coming in one after another at a high frequency.<br>
Because I would like to make a prediction directly after a sample has arrived,<br>
I would set the sequence length during test and training time to 1.<br>
But I am not sure if by doing so the GRU would lose all or part of its “memory” abilities.<br>
Actually, I assume that the “memory” of the GRU does not depend on the sequence length but on the hidden state? The hidden state in turn depends on the learned weights and therefore it would make sense to use a sequence length of 1? If that is true, what is actually the benefit of having a sequence length &gt;1 ?</p>
<p>I would really appreciate it if a more experienced PyTorch user could give me clarification on this.<br>
Thanks in advance.</p>

username: googlebot
cooked: <p>hidden state contains a fixed size summary of a variable length sequence. it is supposed to gradually change as information is accumulated with each timestep. the use mode that you’re describing is valid, if you carry a changing hidden state between evaluations, the outputs will be the same as with bulk multi-step evaluation, just available earlier.</p>

username: fcvm
cooked: <p>Thanks for your answer, I just do not understand your explanation.<br>
Could you please explain what do you mean with bulk multi-step evaluation?</p>

username: googlebot
cooked: <p>if you have N unprocessed sequence elements, you can feed them to a RNN at once, and get N intermediate states - that’s the usual way in training mode (though for some tasks you discard non-final states)</p>
<p>but under the hood elements are processed one by one (timesteps), so you can receive the same states in “realtime” mode.</p>
<p>I.e. single step mode looks like</p>
<p>h1=RNN(x0,h0)<br>
h2=RNN(x1,h1)</p>
<p>and multi-step computes the same:</p>
<p>[h1,h2,…] =RNN([x0,x1,…], h0)</p>
Calculating the top k accuracy using Sklearn

username: OBouldjedri
cooked: <p>I want to calculate the top k accuracy using the sklearn implementation:I was wondring if using this is correct</p>
<p>output = model(data)</p>
<p>target_top_numpy=target.cpu().detach().numpy()</p>
<p>predicted_top_numpy=output.cpu().detach().numpy()</p>
<p>top2=top2+top_k_accuracy_score(target_top_numpy, predicted_top_numpy, k=2,normalize=False,labels=range(11))</p>
<p>my question much more precisly should I apply softmax function on the output before fiding it to the top k score function or it is working fine this way?</p>

username: ptrblck
cooked: <p>From the <a href="https://scikit-learn.org/stable/modules/generated/sklearn.metrics.top_k_accuracy_score.html">docs</a>:</p>
<blockquote>
<p><strong>y_score</strong> array-like of shape (n_samples,) or (n_samples, n_classes)<br>
Target scores. These can be either probability estimates or non-thresholded decision values (as returned by <a href="https://scikit-learn.org/stable/glossary.html#term-decision_function">decision_function</a> on some classifiers). …</p>
</blockquote>
<p>If you copy/paste their example and e.g. scale or shift <code>y_score</code>, the results will still be the same so you wouldn’t need to use <code>softmax</code> on your model output.</p>
Custom conv op in conv2d

username: Dizzy_Hint
cooked: <p>Hi, Is there any way to re-write the convolution operation in nn.conv2d? My goal is to create a module myconv2d. It basically inherent everything(including backward) in conv2d but with a rewrite convolution in C++/CUDA.<br>
I found <a href="https://pytorch.org/tutorials/advanced/cpp_extension.html" rel="noopener nofollow ugc">this</a> to be useful but I dont know how to only replace the conv op with my version, instead of rewrite the whole conv2d.</p>

username: ptrblck
cooked: <p>Using a custom extension sounds like the right approach and you could try to reuse the native conv implementation from <a href="https://discuss.pytorch.org/t/source-code-f-conv2d-definition-location/81569/7">here</a>.</p>
Hi, I have a model in the format hdf5 which is taught using keras pyhton. Now I want to load this model in C ++ and use it. I do not know if libtorch loads the model. How should this be done?

username: reza1
cooked: <p>Hi, I have a model in the format  hdf5 which is taught using keras  pyhton. Now I want to load this model in C ++ and use it. I do not know if libtorch loads the model. How should this be done?</p>

username: ptrblck
cooked: <p>No, <code>libtorch</code> won’t be able to load Keras models, so you would need to export the PyTorch model first e.g. as described in <a href="https://pytorch.org/tutorials/advanced/cpp_export.html">this tutorial</a> or write the model directly in <code>libtorch</code>.</p>
Has anybody observed that interpolate is slow in certain cases?

username: coincheung
cooked: <p>I find that in the mode of <code>amp.autocast(enabled=True)</code>, and <code>scale_factor</code> is large(for example <code>scale_factor=16</code> and <code>align_corners=False</code> and <code>mode=bilinear</code>, the training process is much longer. It seems that the following operator makes training slow greatly. Has anyone observed this and tried to fix it?</p>
<pre><code class="lang-auto">model = nn.Upsample(scale_factor=16, mode='bilinear', align_corners=False)
with amp.autocast(enabled=True):
    out = model(inp)
    ....
  
</code></pre>

username: ptrblck
cooked: <p>Which PyTorch release are you using and could you post the profiling code as well as the results, please?</p>
Rotating tensor inside model

username: Ambrozy
cooked: <p>I need in my model layer that rotates a tensor depending on input data and it has to rotate each sample in batch differently, torch.rot90 doesn’t work on batches, so how this should be done?</p>

username: ptrblck
cooked: <p>If you want to rotate only some slices, you might want to index these particular slices of the input tensor, rotate them, and recreate the tensor afterwards using e.g. <code>torch.cat</code>.</p>
Strange bug about CUDA illegal memory access error

username: Vp_SoLo
cooked: <p>When I use cropped data(256x512) to train my module, it caused the error like this<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/2247f8073fc42f92856ad11f9e74c25a0c879bb4.png" data-download-href="https://discuss.pytorch.org/uploads/default/2247f8073fc42f92856ad11f9e74c25a0c879bb4" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/2/2/2247f8073fc42f92856ad11f9e74c25a0c879bb4.png" alt="image" data-base62-sha1="4TgqrJ66UpYoQfbFrzNVtPxM9Q8" width="690" height="223" data-dominant-color="2E2E2E"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">848×275 9.16 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
But when I ues a larger cropped data(372x512), this error disappeared.<br>
It seems that smaller data will cause this error. Could this be a bug？</p>

username: ptrblck
cooked: <p>Could you update PyTorch to the latest nightly release and check if you are still hitting this issue (in case you are using an older release).<br>
If so, could you post a minimal, executable code snippet to reproduce the issue as well as the output of <code>python -m torch.utils.collect_env</code>, please?</p>
How at::mm works? Where is the source code?

username: Ziyu_Huang
cooked: <p>Hi! I am interested in how at::mm works, and I can not find its source code….especially when using GPU, does mm uses cublas directly? Or how to… parallelly computes?</p>
<p>Thank you!!!</p>

username: ptrblck
cooked: <p>Yes, cublas will be used on the GPU as seen <a href="https://github.com/pytorch/pytorch/blob/26f88eb0e65daf9d3ee7ad399a0e56e709094c53/aten/src/ATen/cuda/CUDABlas.cpp">here</a>.</p>

username: Ziyu_Huang
cooked: <p>Thank you!! But I did not find at::mm function in this link, I guess here exist many other CUDA blas implementation… Could you specify where is the at::mm function is? Thank you!!</p>

username: ptrblck
cooked: <p>You can fine the <code>mm</code> definition in <code>native_functions.yaml</code> <a href="https://github.com/pytorch/pytorch/blob/1776caf361fc2c652f0c00b2fd42714447cefc29/aten/src/ATen/native/native_functions.yaml#L3128-L3141">here</a> and follow their GPU dispatching to <code>mm_out_cuda</code> and eventually to <code>addmm_out_cuda</code> <a href="https://github.com/pytorch/pytorch/blob/d8c3ab11aebf0ec7a439bc9c703fba2169e59514/aten/src/ATen/native/cuda/Blas.cpp#L93">here</a>.</p>
Very Low CTCLoss for incorrect prediction

username: Hari_Krishnan
cooked: <p>After completing training with a very low CTC Loss, the OCR model was performing very poorly. The following code shows on how I calculated the loss and how I structured the input</p>
<pre><code class="lang-auto">for data in test_dataloader:
    loss_criterion = nn.CTCLoss()
    target_lengths = data['lengths']
    N = data['pixel_values'].size()[0]
    data.pop("lengths")
    pred = model(**data)
    pred = pred['logits'].permute(1,0,2)
    pred = nn.functional.log_softmax(pred, dim = 2)
    input_lengths = torch.full(size=(N,), fill_value=128, dtype=torch.long)
    custom_loss = loss_criterion(pred, data['labels'].long(), input_lengths, target_lengths)
    pred = pred.max(1)
    print("Loss",custom_loss.item())
    print(input_lengths, target_lengths)
    print(pred[1][0][:10])
    print(data['labels'][0][:10])
    break
</code></pre>
<p>The output for the above code was</p>
<pre><code class="lang-auto">Loss 2.682149897736963e-05
tensor([128]) tensor([4])
tensor([0, 0, 0, 0, 0, 0, 0, 0, 0, 0], device='cuda:0')
tensor([    2, 16869, 11603,     3,     0,     0,     0,     0,     0,     0],
       device='cuda:0')
</code></pre>
<p>You can see that the expected output and predicted output are completely different, but the loss is extremely low. The lengths of the output are padded to length 128 as you can see it. The loss calculation is done the same way as it is in the <a href="https://pytorch.org/docs/stable/generated/torch.nn.CTCLoss.html" rel="noopener nofollow ugc"> documentation </a>. What could be the reason for this?</p>
Dimensional transformation

username: Gorgen
cooked: <p>How do you know a series of dimension transformations such as dimension exchange in decoder_inputs = decoder_inputs.transpose(1, 2)? why, on what basis</p>
<p>here is link. <a href="https://github.com/NVIDIA/tacotron2/blob/master/model.py" class="inline-onebox" rel="noopener nofollow ugc">tacotron2/model.py at master · NVIDIA/tacotron2 · GitHub</a></p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d821cecf71170e58f562eb1e3c158db0e498792.png" data-download-href="https://discuss.pytorch.org/uploads/default/1d821cecf71170e58f562eb1e3c158db0e498792" title="d5e79233078d7b7ed50e2363399bad9"><img src="https://discuss.pytorch.org/uploads/default/original/3X/1/d/1d821cecf71170e58f562eb1e3c158db0e498792.png" alt="d5e79233078d7b7ed50e2363399bad9" data-base62-sha1="4d2BYdDz7TJ0jKDEPrxnF7mhv5E" width="690" height="366" data-dominant-color="FBF9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">d5e79233078d7b7ed50e2363399bad9</span><span class="informations">1187×630 18.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/gorgen">@Gorgen</a> In order to understand this, you can think about this logically</p>
<ul>
<li>
<p>B denotes batch_size. This cannot change</p>
</li>
<li>
<p>n_mel_channels represent features. This can change</p>
</li>
<li>
<p>T_out also represent features and can change</p>
</li>
</ul>
<p>The above operation breaks out n_channels into further features based on the n_frames_per_step. If the n_frames_per_step &gt; 1 then n_channels will increase. vice versa will happen to the T_out</p>
Modifying parameters after back propagation

username: Saadmaghani
cooked: <p>Hi everyone!</p>
<p>I want to know if it’s possible to modify the parameters of an nn.Module after backpropagation has been done.</p>
<p>I want to clamp the parameters of a module such that they are increasing ie. C[0] &lt; C[1] &lt; … &lt; C[n-1]. But only <em>after</em> they have been updated via gradient descent.</p>
<p>Thanks!<br>
PS. not sure what the category would be so I left it under “Uncategorized”</p>

username: ptrblck
cooked: <p>Yes, similar to how the <code>optimizer.step()</code> function manipulates the parameters, you could apply a similar logic by e.g. manipulating the parameters inplace in a <code>no_grad()</code> context.</p>

username: Saadmaghani
cooked: <p>Thanks ptrblck for your reply!</p>
<p>I should have been more clear - I want to do it automatically whenever the backwards call is done. I’m developing an nn.module that can be imported and used by just adding it to another module. I don’t want end-users to modify their code so they have to manually clamp the values of my module after the optimizer.step() is called.</p>
<p>I looked into backward hooks but they modify the gradients before the parameters are updated. I want to make something like a “backward_post” hook that executes after the parameter gets updated.</p>

username: grudloff
cooked: <p>Maybe using a parametrization could be useful in your case. Although a parametrization would modify the parameter before it is used, I think this would achieve the same result. Here is an example of how you could do that:</p>
<pre><code class="lang-auto">from torch import nn
import torch.nn.utils.parametrize as parametrize
class Clamp(nn.Module):
    """Reparametrize a variable to be bounded in [0, 1] with a clamp"""
    def forward(self, X):
        return X.clamp(0, 1)
    def right_inverse(self, Z):
        return Z

linear = nn.Linear(1,2) 
parametrize.register_parametrization(linear, 'weight', Clamp())
</code></pre>

username: Saadmaghani
cooked: <p>Thanks grudloff! Parametrization solves my problem</p>
<p>I was thinking about using pre_forward_hooks but, if I am not mistaken, those are supposed to be used for modifying the inputs to the module and not the module itself. Parametrization is basically that but for parameters.</p>
<p>I also didn’t know about parameterization and read the tutorial on it after seeing your reply. So thanks for introducing it to me too.</p>
Num_gpu <= torch.cuda.device_count() Assertion error

username: Bohan_Zhang
cooked: <p>Hello there,<br>
I meet an issue which shows above<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/6/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/6/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56_2_690x444.jpeg" alt="image" data-base62-sha1="gS1e1JJLjU2zJBlzj0hIN41JoY6" width="690" height="444" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/6/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56_2_690x444.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/7/6/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/6/763e060463fb0f6cd7aaa05d1f8cf6af4c6c9f56.jpeg 2x" data-dominant-color="313336"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">884×570 186 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
However as I search through the internet I couldn’t find any answer for that. Does anyone have any idea what’s the issues for that and give me any solution? Thanks!!</p>

username: thecho7
cooked: <p>How many GPUs do you have?<br>
num gpus in your code should be less than the number of GPUs you have</p>

username: Bohan_Zhang
cooked: <p>From terminal it says only one gpu available, as I’m implementing git from</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/ucbdrive/few-shot-object-detection">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/ucbdrive/few-shot-object-detection" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/87f1f15b086aaa78d4676462253f1ce5da2939befdadc6b0439b9f6545ad141f/ucbdrive/few-shot-object-detection" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/ucbdrive/few-shot-object-detection" target="_blank" rel="noopener nofollow ugc">GitHub - ucbdrive/few-shot-object-detection: Implementations of few-shot...</a></h3>

  <p>Implementations of few-shot object detection benchmarks - GitHub - ucbdrive/few-shot-object-detection: Implementations of few-shot object detection benchmarks</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>From github it doesn’t specify gpu num</p>

username: thecho7
cooked: <p>In argument, num gpus is set to 8.</p>
<pre><code class="lang-auto">$ python xxx.py --num-gpus 1
</code></pre>
<p>Let’s see how this change</p>
LSTM - Did I correctly change my forward function and my architecture to work bidirectional?

username: Patrick1
cooked: <p>As I feel like there is very little help to this in the documentation or somewhere else I am just trying to figuere out if I did everything correctly. At first I only changed bidirectional to True and to my suprise my model did still run without any error message. Now I also changed the dimensions of the next linear layer and my forward method.</p>
<p>Here is my thought process:</p>
<ol>
<li>
<p>I am thinking, since bidirection doubles my lstm layer I need to double the Input size of the next layer. Which is weird because, like I said, after changing bidirectional to True without changing the input size of the next layer my model still ran.</p>
</li>
<li>
<p>I need to take both outputs of my lstm hn[0] and hn[-1] and bot feed them to the next layer. So I then thought, maybe my model did not give me an error since I didnt have this implemented yet.</p>
</li>
</ol>
<p>Would be just nice to know for sure how to correctly change the lstm structure. Any help appreciated.</p>
<p>EDIT: I just realised that ptorch documantation states<br>
My LSTM Class now looks like this.</p>
<pre><code class="lang-auto">class Model_LSTM(nn.Module):

    def __init__(self, n_features, n_classes, n_hidden, n_layers):            # bidirectional möglich
        super().__init__()
        self.lstm = nn.LSTM(
            input_size=n_features,
            hidden_size=n_hidden,
            num_layers=n_layers,
            batch_first=True,
            dropout=0,
            bidirectional=True
        )
        
        self.dense1 = nn.Linear(n_hidden*2, n_hidden)

        self.classifier = nn.Linear(n_hidden, n_classes)

        torch.nn.init.xavier_uniform_(self.lstm.weight_ih_l0)
        torch.nn.init.xavier_uniform_(self.lstm.weight_hh_l0)

        torch.nn.init.xavier_uniform_(self.classifier.weight)
        torch.nn.init.xavier_uniform_(self.dense1.weight)
        

    def forward(self, x):
        _, (hn, _) = self.lstm(x)                  
        out=torch.concat([hn[0], hn[-1]], dim=1)       
        out = F.relu(self.dense1(out))
        return self.classifier(out)
</code></pre>
Initializing weights and adding trainable variables

username: amin_p
cooked: <p>Hello,</p>
<p>I’m new to the PyTorch, and I’m trying to build a network with some additional trainable variables.</p>
<pre><code class="lang-auto">#===============================================================================
# Nnet
#===============================================================================
class Nnet(nn.Module):

    def __init__(self, layers):
        super(Nnet, self).__init__()
        self.weights = []
        self.biases = []
        num_layers = len(layers)
        
        for l in range(num_layers-1):
            W = nn.Parameter(torch.nn.init.xavier_uniform_([layers[l], layers[l+1]], gain=1.0))
            b = nn.Parameter(torch.zeros([1,layers[l+1]], dtype=np.float32))
            self.weights.append(W)
            self.biases.append(b)

                
        self.p = nn.Parameter([0.0])
        self.K = nn.Parameter([10.0])


    def forward(self, t, y):
        H = torch.cat([y,t],axis=-1)

        for l in range(len(self.weights)-1):
            W = self.weights[l]
            b = self.biases[l]
            H = torch.tanh(torch.add(torch.matmul(H, W), b))
            
        W = self.weights[-1]
        b = self.biases[-1]
        Y = torch.add(torch.matmul(H, W), b)
        
        return Y
</code></pre>
<pre><code class="lang-auto">layers = [2, 20, 20, 20, 20, 1]

nnet = Nnet(layers)

nnet = nnet.to(device)
</code></pre>
<p>I’m receiving he following error:</p>
<p>AttributeError: ‘list’ object has no attribute ‘dim’</p>
<p>I would appreciate it if anybody could help me.</p>

username: ptrblck
cooked: <p>The error is raised since <code>torch.nn.init.xavier_uniform_</code> expects a tensor as its input, not a <code>list</code> of sizes.<br>
Use:</p>
<pre><code class="lang-python">W = nn.Parameter(torch.nn.init.xavier_uniform_(torch.empty([layers[l], layers[l+1]]), gain=1.0))
</code></pre>
<p>and the error should be solved.<br>
However, since you are using plain Python <code>list</code>s to store the parameters, they won’t be registered properly, so use <code>nn.ParameterList</code> for <code>self.weights</code> and <code>self.biases</code> instead.</p>
<p>Once this error is solved, you will run into an argument error in <code>b = ...</code>, since you are passing <code>np.float32</code> to a PyTorch tensor. Use:</p>
<pre><code class="lang-python">b = nn.Parameter(torch.zeros([1,layers[l+1]], dtype=torch.float32))
</code></pre>
<p>and it should work.</p>
<p>Afterwards, <code>self.p</code> and <code>self.K</code> will fail, since <code>nn.Parameter</code> expects a tensor, not a <code>list</code> of floats.</p>

username: amin_p
cooked: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> Thank you very much for your helpful answer.<br>
It solved my problem.</p>
How to create multiple MultiVariateNormal at one time

username: wwd
cooked: <p>If I call</p>
<pre><code class="lang-auto">MultivariateNormal(torch.randn(2), torch.eye(2))
</code></pre>
<p>I got one MultivariateNormal distribution,</p>
<p>If I want to create 100 of them; the means are in tensor of shape (100, 2), the covariance matrices are in a tensor of shape (100, 2, 2), essentially the batch shape is 100 and event shape is 2, how could I create these 100 MultivariateNormal distribution without a loop? Thanks!</p>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/wwd">@wwd</a> some kind of looping will have to be done if you want to use this function as each distribution is separate from the other. However you can parallelize without using direct loops</p>
<p>e.g.</p>
<pre><code class="lang-auto">loc = torch.randn(100, 2)
cov = torch.zeros(100, 2, 2)
cov[:, 0, 0] = 1
cov[:, 1, 1] = 1
def get_distribution(x):
    return torch.distributions.MultivariateNormal(loc[x], cov[x])
list(map(get_distribution, range(100)))
</code></pre>
"Illegal instruction" using AutoModelWithLMHead on 64bit Raspberry Pi 4B

username: MarsRon
cooked: <pre><code class="lang-auto">from transformers import AutoModelWithLMHead
model = AutoModelWithLMHead.from_pretrained('fine-tuned-DialoGPT')
model.generate(...)
</code></pre>
<p>This my code, and I’m running this on a 64-bit Raspberry Pi 4B. It crashes when <code>model.generate(...)</code> is executed, and <em>only</em> prints out <code>Illegal instruction</code>. It works fine on my computer though.<br>
What may be causing this issue?</p>

username: MarsRon
cooked: <p>Update: I’ve tried using <a href="https://github.com/KumaTea/pytorch-aarch64" rel="noopener nofollow ugc">https://github.com/KumaTea/pytorch-aarch64</a> and I got a response with <code>eatures mathemat mathemat mathemat mathemat mathemat (repeat this 1000 times).......</code>. It took around 5 minutes to generate that response.</p>
<p>I still don’t know how to use pytorch on a 64-bit RasPi 4B. Help is appreciated.</p>

username: marksaroufim
cooked: <p>Not an expert but maybe these instructions will help you get started <a href="https://github.com/pytorch/tutorials/pull/1821" rel="noopener nofollow ugc">https://github.com/pytorch/tutorials/pull/1821</a></p>
<p><a class="mention" href="/u/d4l3k">@d4l3k</a></p>

username: MarsRon
cooked: <p>Turns out it was an issue with 64-bit Raspberry Pi OS beta. I’ve updated it to the official release and it miraculously worked.</p>
RuntimeError: Function 'LogBackward0' returned nan values in its 0th output

username: Ruochen
cooked: <p>I just slightly changed SoftMax to make sure that if input is 0, then output will be 0 as well.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/0/b/0bd5292e57e8a47ab9937011432adbff52b69a76.png" alt="1644022867(1)" data-base62-sha1="1GFWJ6GUOWNrYiZZe3WLN7MdIyi" width="605" height="262"><br>
but I got the error image from the loss function:<br>
RuntimeError: Function ‘LogBackward0’ returned nan values in its 0th output.</p>
<p>Is that something wrong with the epsilon settings?</p>

username: my3bikaht
cooked: <p>What’s your input tensor? With large enough value you can easily reach +inf for some element due to ~e**2x, which will lead to nan after division</p>

username: AlphaBetaGamma96
cooked: <p>Can you share a minimal reproducible example? I can see your function could failing in the case of <code>eps=0</code> and your <code>x</code> Tensor containing a zero, as you’d be dividing by zero.</p>

username: Ruochen
cooked: <p>Sorry guys, check my new topics I have put the main error codes there</p>

username: AlphaBetaGamma96
cooked: <p>If you found a solution to this problem, please do share (and mark) the solution so others may learn from it! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: Ruochen
cooked: <p>I am sure I will. Thanks for suggestions</p>
Function 'MulBackward0' returned nan values in its 1th output

username: Ruochen
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5cd9dc5600a7bb7c30e3b0bf177d6343d6e359d2.png" data-download-href="https://discuss.pytorch.org/uploads/default/5cd9dc5600a7bb7c30e3b0bf177d6343d6e359d2" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5cd9dc5600a7bb7c30e3b0bf177d6343d6e359d2.png" alt="image" data-base62-sha1="dfoGSHPL0J1VK5rRqFd4jWu12yS" width="439" height="500" data-dominant-color="31313B"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">672×764 31.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90a9ea91870b168d287840eb7018111481fd5413.png" data-download-href="https://discuss.pytorch.org/uploads/default/90a9ea91870b168d287840eb7018111481fd5413" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/0/90a9ea91870b168d287840eb7018111481fd5413.png" alt="image" data-base62-sha1="kDKNA3qlHueKz14DVObOYdI9bp1" width="690" height="228" data-dominant-color="F0F0F0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">698×231 6.63 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Hey guys, I have a customized loss function here. When I train the model, following errors occurs,<br>
to avoid such issues, I always add torch.clamp() to scale the vector especially for division and log,<br>
I dont know why “MulBackward0” returned nan here, any ideas?</p>
<p>The error detected within the read box I draw.</p>

username: AlphaBetaGamma96
cooked: <p>You can debug your code by using <a href="https://pytorch.org/docs/stable/autograd.html#torch.autograd.graph.saved_tensors_hooks" rel="noopener nofollow ugc">set_detect_analomy</a>.</p>
<p>Given the errors within <code>MulBackward0</code> you’re most likely dividng by zero when calculating the gradient. So, using <code>set_detect_analomy</code> will help find it.</p>
<p>Finally, when sharing code make sure to copy and paste in the source rather than uploading a picture, of it. You can wrap code in 3 backticks ``` to get it formatted correctly. For example,</p>
<pre><code class="lang-auto">import torch
x = torch.randn(5)
mean = torch.mean(x)
</code></pre>

username: Ruochen
cooked: <p>Many thanks! I am new using forums, next time I will use this format to make my codes and questions clear, thanks</p>
Tensor.le and gradient flowing

username: marchinidavide
cooked: <p>Dumb question: is there any hack around gradient not flowing in the following computation?</p>
<pre><code class="lang-auto">value = 1.0 # or a 0D tensor with requires_grad=True
tensor = torch.linspace(0, 10, requires_grad=True)  # or any 1D tensor with requires_grad=True
rank_value_in_tensor = tensor.le(value).float().mean() # this has no gradient since .le breaks gradient flowing
</code></pre>
<p>Unsuccessful things I tried:</p>
<ol>
<li>
<code>rank_value_in_tensor = torch.relu(tensor - value).clamp(0, 1e-6).mean() / 1e-6</code> but it’s definitely not a robust solution (plus gradient likely to have weird behaviour)</li>
</ol>

username: anantguptadbl
cooked: <p><a class="mention" href="/u/marchinidavide">@marchinidavide</a><br>
This is the closes that i got to</p>
<pre><code class="lang-auto">value = 1.0 # or a 0D tensor with requires_grad=True
tensor = torch.linspace(0, 10, requires_grad=True)  # or any 1D tensor with requires_grad=True
rank_value_in_tensor = F.relu(tensor-value)
rank_value_in_tensor[rank_value_in_tensor&gt;0]=1
rank_value_in_tensor = rank_value_in_tensor.mean()
print(rank_value_in_tensor.requires_grad)
</code></pre>
How do i fold patches back to form a complete image

username: binbbaz
cooked: <p>I have do I fold patches back to get original image</p>
<pre><code class="lang-auto">def perturb_patch(img, patch_idx, patch_size, stride):
    img = img.unsqueeze(0)
    patches = img.unfold(2, patch_size, stride).unfold(3, patch_size, stride)
    patches = patches.reshape(1, 3, -1, patch_size, patch_size)
    patches = patches.squeeze(0).permute(1, 0, 2, 3)
    patches[patch_idx][0, :, :] = 0.09803922
    patches[patch_idx][1,:, :] = 0.21333333
    patches[patch_idx][2,:, :] = 0.61176471

    merged_patches = nnf.fold(patches.reshape(196, 768, 1), (224, 224), kernel_size=16, stride=16)
    return merged_patches
</code></pre>
<p>I am getting the <code>merged_patches</code> wrong. For the patches, I get a shape of <code>(196, 3, 16, 16)</code> how do I merge it to get a complete image? My image is <code>(3, 224, 224)</code> .</p>

username: binbbaz
cooked: <p>This helped me</p>
<aside class="quote quote-modified" data-post="3" data-topic="61395">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/a/f04885/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/fold-patches-of-images-back-to-single-image/61395/3">Fold patches of images back to single image</a> 
  </div>
  <blockquote>
    Thank you! It works: 
def jigsaw_to_image(x, grid_size=(2, 2)):
    # x shape is batch_size x num_patches x c x jigsaw_h x jigsaw_w
    batch_size, num_patches, c, jigsaw_h, jigsaw_w = x.size()
    assert num_patches == grid_size[0] * grid_size[1]
    x_image = x.view(batch_size, grid_size[0], grid_size[1], c, jigsaw_h, jigsaw_w)
    output_h = grid_size[0] * jigsaw_h
    output_w = grid_size[1] * jigsaw_w
    x_image = x_image.permute(0, 3, 1, 4, 2, 5).contiguous()
    x_image = x_image.view(ba…
  </blockquote>
</aside>
RuntimeError: Could not export Python function call 'MishCudaFunction'. Remove calls to Python functions before export. Did you forget to add @script or @script_method annotation? If this is a nn.ModuleList, add it to __constants__:

username: fardinadii
cooked: <p>hi<br>
I’m trying to convert yolov4 pytorch model to Torchscript but i faced with this error.<br>
my model convert and saved but it does not work.</p>
<p>Fusing layers… Model Summary: 235 layers, 5.25029e+07 parameters, 5.04494e+07 gradients<br>
/pqc_yolov4/models/yolo.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can’t record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!<br>
if self.grid[i].shape[2:4] != x[i].shape[2:4]:<br>
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py:950: TracerWarning: Encountering a list at the output of the tracer might cause the trace to be incorrect, this is only valid if the container structure does not change based on the module’s inputs. Consider using a constant container instead (e.g. for <code>list</code>, use a <code>tuple</code> instead. for <code>dict</code>, use a <code>NamedTuple</code> instead). If you absolutely need this and know the side effects, pass strict=False to trace() to allow this behavior.<br>
module._c._create_method_from_trace(<br>
Traceback (most recent call last):<br>
File “pqc_Torchscript.py”, line 29, in <br>
torch.jit.save(traced_gpu, “./pqc_result/final_model__new_TorchScript.pt”)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_serialization.py”, line 81, in save<br>
m.save(f, _extra_files=_extra_files)<br>
File “/opt/conda/lib/python3.8/site-packages/torch/jit/_script.py”, line 495, in save<br>
return self._c.save(str(f), **kwargs)<br>
RuntimeError:<br>
Could not export Python function call ‘MishCudaFunction’. Remove calls to Python functions before export. Did you forget to add <span class="mention">@script</span> or <span class="mention">@script_method</span> annotation? If this is a nn.ModuleList, add it to <strong>constants</strong>:<br>
/opt/conda/lib/python3.8/site-packages/mish_cuda/<strong>init</strong>.py(20): forward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl<br>
/pqc_yolov4/models/common.py(34): fuseforward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl<br>
/pqc_yolov4/models/yolo.py(129): forward_once<br>
/pqc_yolov4/models/yolo.py(109): forward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(860): _slow_forward<br>
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py(887): _call_impl<br>
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py(950): trace_module<br>
/opt/conda/lib/python3.8/site-packages/torch/jit/_trace.py(733): trace<br>
pqc_Torchscript.py(28): </p>
Determinism for MaxPool3d and AvgPool3d

username: skim2257
cooked: <p>Hi everyone, first post here and not sure if this is the right category <img src="https://discuss.pytorch.org/images/emoji/apple/sweat_smile.png?v=12" title=":sweat_smile:" class="emoji" alt=":sweat_smile:">. Mods please move my post to the right category if this is the wrong one.</p>
<p>I mainly train 3D CNN models and I recently updated from PyTorch 1.5 to 1.10. While trying to train my model on the updated environment, I encountered this error:</p>
<pre><code class="lang-auto">RuntimeError: avg_pool3d_backward_cuda does not have a deterministic 
implementation, but you set 'torch.use_deterministic_algorithms(True)'. You 
can turn off determinism just for this operation if thats acceptable for your 
application. You can also file an issue at https://github.com/pytorch/pytorch/issues
to help us prioritize adding deterministic support for this operation
</code></pre>
<p>I’ve trained non-deterministic models in the past but the main limitation of 3D medical imaging datasets is that they’re very small, and are thus prone to large fluctuations in performance based on the random seed. I found that there are large fluctuations even with a fixed torch.manual-seed().</p>
<p>I’d really appreciate if support for MaxPool3d and AvgPool3d could be prioritized as (I hope) refactoring MaxPool2d/AvgPool2d determinism for these operations shouldn’t be very difficult.</p>
<p>Thanks dev team for the great work as always <img src="https://discuss.pytorch.org/images/emoji/apple/grinning.png?v=12" title=":grinning:" class="emoji" alt=":grinning:"></p>
What does .view do in this basic model?

username: final
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/e/fe0f8c834181044ead58fc7b25aae85cb2707c5b.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/fe0f8c834181044ead58fc7b25aae85cb2707c5b" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fe0f8c834181044ead58fc7b25aae85cb2707c5b_2_690x240.jpeg" alt="image" data-base62-sha1="AfwAD4PBqJwssGr142bzu7VJwHF" width="690" height="240" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/e/fe0f8c834181044ead58fc7b25aae85cb2707c5b_2_690x240.jpeg, https://discuss.pytorch.org/uploads/default/original/3X/f/e/fe0f8c834181044ead58fc7b25aae85cb2707c5b.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/e/fe0f8c834181044ead58fc7b25aae85cb2707c5b.jpeg 2x" data-dominant-color="0D1118"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1017×355 97.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
Specifically, what do the parameters mean?</p>

username: ptrblck
cooked: <p>From the <a href="https://pytorch.org/docs/stable/generated/torch.Tensor.view.html?highlight=view#torch.Tensor.view">docs</a>:</p>
<blockquote>
<p>Returns a new tensor with the same data as the <code>self</code> tensor but of a different <code>shape</code>.<br>
The returned tensor shares the same data and must have the same number of elements, but may have a different size. For a tensor to be viewed, the new view size must be compatible with its original size and stride, i.e., each new view dimension must either be a subspace of an original dimension, or only span across original dimensions …</p>
</blockquote>
<p>In other words: <code>view</code> is similar to <code>reshape</code>, but doesn’t trigger a copy of the tensor.<br>
The values passed to <code>view</code> are used as the shape of the view.</p>