akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Training Mask R-CNN yields 'RuntimeError: stack expects each tensor to be equal size'

David1: <p>I am trying to train the <code>maskrcnn_resnet50_fpn</code> model on a custom dataset with only two classes (including background), by copying what they’re doing in the ‘TorchVision Instance Segmentation Finetuning’ Tutorial (<a href="https://colab.research.google.com/github/pytorch/tutorials/blob/gh-pages/_downloads/torchvision_finetuning_instance_segmentation.ipynb" class="inline-onebox" rel="noopener nofollow ugc">Google Colab</a>). I get the error <code>RuntimeError: stack expects each tensor to be equal size, but got [11, 1024, 1024] at entry 0 and [47, 1024, 1024] at entry 1</code>. It seems to me that this is saying that the mask tensors for two different images have to be of the same size, i.e. that all images must contain the same number of instances, which makes no sense.</p>
<p>I’ve written the following minimal example, where all the 50 images and corresponding masks just contain zeros (i.e. are totally black). Image number <em>i</em> has <em>i</em> masks. The bounding boxes cover the entire images.<br>
The engine module is found in <a href="https://github.com/pytorch/vision/tree/main/references/detection" class="inline-onebox" rel="noopener nofollow ugc">vision/references/detection at main · pytorch/vision · GitHub</a>.</p>
<pre><code class="lang-auto">import torch
from torchvision.models.detection import maskrcnn_resnet50_fpn
from torch.utils.data import DataLoader
from engine import train_one_epoch

class WeirdDataset(torch.utils.data.Dataset):
    def __len__(self):
        return 50

    def __getitem__(self, idx):
        img = torch.zeros(1, 1024, 1024)

        target = {}
        bbox = torch.tensor([1, 1, 1024, 1024], dtype=torch.float)
        target['masks'] = torch.zeros(idx, 1024, 1024, dtype=torch.uint8)
        target['boxes'] = bbox.repeat(idx, 1)
        target['labels'] = torch.ones(idx, dtype=torch.int64)

        return img, target

model = maskrcnn_resnet50_fpn(num_classes = 2)
model.train()
train_dataset = WeirdDataset()

train_data_loader = DataLoader(train_dataset, batch_size=2, shuffle=True)
learning_rate = 0.01
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)

for epoch in range (5):
    train_one_epoch(model, optimizer, train_data_loader, device='cpu', epoch=epoch, print_freq=1)
</code></pre>

AlphaBetaGamma96: <aside class="quote no-group" data-username="David1" data-post="1" data-topic="158473">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/david1/48/52066_2.png" class="avatar"> David1:</div>
<blockquote>
<p>[11, 1024, 1024] at entry 0 and [47, 1024, 1024]</p>
</blockquote>
</aside>
<p>If you have two tensors of size <code>[11, 1024, 1024]</code> and <code>[47, 1024, 1024]</code>, you’ll want to use <code>torch.cat</code> instead,</p>
<pre><code class="lang-auto">a=torch.randn(11,1024,1024)
b=torch.randn(47,1024,1024)
c=torch.cat((a,b), dim=0)
print(c.shape) #returns [58,1024,1024]`
</code></pre>
LSTM documentation

python_Dev: <p>Can someone please explain to me how is the documentation interpreter?<br>
Why it has this ’ (L, N, H_{in})’ when in fact it takes an integer and not a tuple?</p>

thecho7: <p>Could you explain in detail?</p>

python_Dev: <p>I am a bit new in LSTM, I read online how they work but I am bit confused how to implement them. Can I have as input shape a tuple? Also, how can I define the number of LSTMs stack on top of each other and how can I define different designs such as many-to-many</p>

thecho7: <p>Sorry, it seems good to you to read other materials before asking something here.<br>
Try googling with keywords ‘lstm pytorch example’ or something.</p>
<p>Thanks</p>

python_Dev: <p>I had a fresh look again today over the documentation and some examples. Just to confirm what I read.</p>
<p>lstm = nn.LSTM(input_size = 3,hidden_size = 4,num_layers = 2, batch_first = True, bidirectional=True)</p>
<p>This means, that the input tensor has the structure of (batch_size, seq_len, input_size) and the output tensor has the shape of (batch_size, seq_len, 2*hidden_size) since bach_first = True. <br>
Also, the 2<em>hidden_size arises because bidirectional=True. <br>
Lastly, the num_layers determine the pairs of (hidden_state, cell_state) of the network. <br>
inputs.shape = (100, 5, 3), hidden.shape = (100, 3, 2</em>4) <br>
out, hidden = lstm(inputs, hidden) <br>
where:<br>
‘out’ has all the new hidden state values of the network<br>
‘hidden’ is a tuple of (hidden_state, cell_state) of the last layer only.</p>

AbdulsalamBande: <p>You could watch this video <a href="https://www.youtube.com/watch?v=4p9SXDSwxqA&amp;ab_channel=PytorchModulesExplained" rel="noopener nofollow ugc">torch.nn.RNN Module explained</a>. It is very similar to <code>torch.nn.LSTM</code></p>
Loading of duplicated data in distributed train

Innisfree: <p>Hi, I would like to pretrain BERT by using DDP.<br>
I saved pretrain dataset(350GB of large corpus) as torch.tensor.</p>
<p>When I run the code below, dataset is loaded in memory 8 times.<br>
python -m torch.distributed.launch --nproc_per_node=8 train.py</p>
<p>How can I prevent it?<br>
Thanks.</p>

ptrblck: <p>Did you store the complete dataset in a single tensor?<br>
If so, I think you might need to load it once and store smaller chunks of the data (and load only certain chunks in each process) or load the data lazily from the beginning.</p>

Innisfree: <p>Yes, I did<br>
As you said, I stored smaller chunks of the data.</p>
<p>Thanks for your reply.</p>

mrshenli: <p>I thought it is expected to have dedicated data loader in each process? So that 8 processes will have 8 dataloaders and 8 DDP instances?</p>
<p>cc <a class="mention" href="/u/vincentqb">@vincentqb</a> please correct me if I am wrong.</p>

vincentqb: <p>Right, depending on the details in the code is organized, I would expect 8 processes/gpus getting different chunk of data as you said.</p>

Messiah: <p>Hi, I am facing a similar issue as you describe.</p>
<p>May you elaborate on how you went about storing smaller chunks of data and loading these in each process, as <a class="mention" href="/u/ptrblck">@ptrblck</a> mentions?</p>
How to subtract tensors like matmul?

suryak: <p>I want to apply another function instead of multiplication operation in matmul.</p>
<p>Is that possible?</p>

AbdulsalamBande: <p>I do not think you can, <a href="https://pytorch.org/docs/stable/generated/torch.matmul.html" rel="noopener nofollow ugc">torch.matmul</a> only does “Matrix product of two tensors.” as the site says. You can implement your custom function though.</p>
Bert model in pytroch

samuel_fipps: <p>Is there a simple bert model in pytorch? Because i know the base model is almost based on a normal transformer so the source code could be to long. I just can’t seem to find it.</p>

AbdulsalamBande: <p>You can implement a simple bert model, but you would I have to reduce of some parameters (like number of heads) and also the dimensions of some layers like the embedding layer. You can use <a href="https://pytorch.org/docs/stable/generated/torch.nn.TransformerEncoderLayer.html" rel="noopener nofollow ugc">torch.nn.TransformerEncoderLayer</a> for implementation. To undersdand the mechanics behind the module, you can watch <a href="https://www.youtube.com/watch?v=aSx2uAytLFY&amp;ab_channel=PytorchModulesExplained" rel="noopener nofollow ugc">torch.nn.TransformerEncoderLayer - Part 1</a> and the subsequent videos.</p>
I can't output the object detection test. No labels or bounding boxes, just images

NagaYu: <pre><code class="lang-auto">pread_list = []
model.eval()
with torch.no_grad():

    for i in test_data:
        image = Image.open(i)
        image = transform(image)
        image = image.unsqueeze(0)
        image = image.to(device)
        output = model(image)  
        preds = torch.max(output, 600, 600)#, dim=1)[:, 1].tolist() 
        #id_list.append(_id)
        #pred_list.append(preds[0])
</code></pre>
<p>TypeError                                 Traceback (most recent call last)<br>
Input In [12], in &lt;cell line: 3&gt;()<br>
9 image = image.to(device)<br>
10 output = model(image)<br>
—&gt; 11 preds = torch.max(output, 600, 600)</p>
<p>TypeError: max() received an invalid combination of arguments - got (list, int, int), but expected one of:</p>
<ul>
<li>(Tensor input)</li>
<li>(Tensor input, Tensor other, *, Tensor out)</li>
<li>(Tensor input, int dim, bool keepdim, *, tuple of Tensors out)</li>
<li>(Tensor input, name dim, bool keepdim, *, tuple of Tensors out)</li>
</ul>

ptrblck: <p><code>torch.max</code> expects the <code>input</code>, <code>dim</code>, and <code>keepdim</code> input arguments as explained in the <a href="https://pytorch.org/docs/stable/generated/torch.max.html#torch.max">docs</a> while you are passing two integers to this call which seem to be a spatial size of the tensor.</p>
Different transformations for train and test dataset

Ali_Aghababaei: <p>Hi everyone,<br>
I have created a custom Dataset class for a number of images, and want to apply K-fold cross-validation on them to split them into train and test datasets (I don’t want to specify a classic validation dataset). The torchvision transforms for train and test datasets are different since I want to apply data augmentation on just the train dataset, not the test dataset. My question is that what is the best way to manage different transformations for each dataset in each fold. should I define a different class for each train or test dataset in each fold?</p>

sudomaze: <p>Hi Ali,</p>
<p>You can create a function that returns a <code>Compose</code> and the parameter passed to it determines if this is a train or test operation. Here is an example:</p>
<pre><code class="lang-python">
import torchvision.transforms as T

def get_transforms(mode):
    transforms = []
    if mode == 'train':
        transforms.extend([
            # ...
        ])
    transforms.extend([
        T.Rescale((256,256)),
        T.ToTensor(),
        T.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
    ])
    return T.Compose(transforms)

train_set = Dataset(get_transforms('train'))
test_set = Dataset(get_transforms('test'))
# ...
</code></pre>
<p>You can refer to <a href="https://pytorch.org/vision/stable/transforms.html" rel="noopener nofollow ugc">this tutorial for more examples</a>.</p>
Concatenate the tensors within the tensors - Pytorch

Vishak_Raj: <p>hello,</p>
<p>I have a task to complete -<br>
<code>a</code> is a tensor of shape <code>torch.Size([2, 1, 25, 25])</code></p>
<p>From this tensor, I convert this shape to <code>torch.Size([2, N, 25, 25])</code>, where N is the variable,</p>
<p>If N&gt;1, then the third and fourth dimension of tensor <code>a</code> should be concatenate <code>N</code> number of times, but the third and fourth dimension tensor will be different for every tensors in the first dimension, in this example - <code>a</code> tensor has 2 data/values in the first dimension</p>
<p>For example, if N=3, then the output tensor shape should be <code>torch.Size([2, 3, 25, 25])</code></p>
<p>For achieving this, I tried <code>list</code> but it takes so much time, how to done this using pure pytorch tensor ops.</p>
<p>thanks</p>

sudomaze: <p>Hi Vishak,</p>
<p>To understand the problem, you have a Tensor with this dim <code>[B, N, X, Y]</code>, where <code>B=Batch size</code>, <code>N=Length of the data</code>, and <code>X, Y=dim of your input</code>.</p>
<p>The issue that you are trying to solve is that every <code>N</code>-element has a different <code>X, Y</code> size.</p>
<p>If that is the problem and the issue that you are having, I would suggest padding using <a href="https://pytorch.org/docs/stable/generated/torch.nn.utils.rnn.pad_sequence.html" rel="noopener nofollow ugc"><code>pad_sequence</code></a> module, so every <code>N</code>-element has a constant size which is the max size in your dataset. <a href="https://discuss.pytorch.org/t/using-variable-sized-input-is-padding-required/18131/7">Look into this post for more information</a>.</p>
<p>Then, you will be using the <a href="https://pytorch.org/docs/stable/generated/torch.stack.html" rel="noopener nofollow ugc"><code>stack</code></a> module to stack those tensors on the second dimension.</p>
Vectorization in torch.autograd

GN2225: <p>Hi. I have an input vector input in of dimension (Nxd) of N samples which get mapped to an output vector output of dimensions (Nxm), i.e. each sample gets mapped from R^d ro R^m.<br>
Now, I want to compute the gradients of each entry of the result with respect to the corresponding input sample, i.e, I want to obtain a N x m x d (or N x d x m or similar) tensor A with the respective gradients. Until now, I used a for loop over the second dimension of the output vector (i.e. let i run from 1 to m), selected o[:,i]  and used the command<br>
torch.autograd.grad(output[:,i], input, torch.ones_like(o[:,i]))[0]<br>
to obtain a Nxd vector on each of the m iterations of the for loop. However, to speed up my code, I now want to vectorize this procedure. Sadly, I could not see how to do this. Is there a simple way by just using torch.autograd.grad or if not, how can I proceed?</p>
Online Triplet Mining - TripletMarginLoss

Rane90: <p>Hi,</p>
<p>From what I understand using <a href="https://pytorch.org/docs/stable/generated/torch.nn.TripletMarginLoss.html" rel="noopener nofollow ugc">this loss function</a> without modifying the data loader is considered an “offline” implementation - i.e. the triplets are chosen randomly.</p>
<p>Are there any recommendations or even other implementations for an “online” triplet loss? I’m looking for ways that while training, the model chooses the anchor, positive and negative samples such that they are considered semi-hard-triplets.</p>
<p>A good explanation for what I mean can be found in this post -  <a href="https://omoindrot.github.io/triplet-loss#offline-and-online-triplet-mining" class="inline-onebox" rel="noopener nofollow ugc">Triplet Loss and Online Triplet Mining in TensorFlow | Olivier Moindrot blog</a></p>
Learning rate in adam

robinho: <p>How to choose an optimum initial learning rate in adam?<br>
When it’s too big, why does the net converge to a wrong solution? Normally too big a learning rate should make the loss oscillate too much but not get stuck in a wrong solution?</p>

ptrblck: <p>You could either run a few experiments using different learning rates or use some utils. which could find the “optimal” learning rate e.g. such as the <code>learn.lr_find()</code> operation from <a href="https://docs.fast.ai/examples/app_examples.html#keypoints">fastai</a>.</p>

robinho: <p>thanks so much. Is this learn.lr_find() restricted to any particular network? I still don’t know how to use it as I don’t use the net in the link you posted.</p>

ptrblck: <p>No, I don’t think this operation depends on the model architecture. You would need to install and use the <code>fastai</code> package or reimplement their learning rate finder in pure PyTorch.</p>
Version 1.11 fitting ability worse than 1.4

yeungkong: <p>Hi developers of pytorch. I am switching my model from version 1.4 to version 1.11 and I am pretty sure that the neural network models in 1.4 and 1.11 are the same. The model trained in 1.4 fits well with the data and the model trained in 1.11 does not fit as well as its counterpart does in 1.4. So to me, it seems that some regularization technique is added to version 1.11?! How can I solve this problem? Thank you for any guidance.</p>
Any way to compute the Eigenvalues of the Hessian during pptimizer.step()?

Adi_Dew: <p>Basically the title. I’m currently coding a custom optimizer, and I’m trying to create a hessian matrix and compute eigenvalues based on all of the parameters during the .step() method.</p>
<p>The main problem is that for optimizer.step one must iterate over all params in param_group and check if they have gradients, which makes it difficult to piece together a matrix or compute eigenvalues of a param_group. Is there a way to compute the hessian and eigenvalues of a param_group without iterating over each of the parameters?</p>
<p>I was thinking of doing this by iterating and then storing all of the gradients for a group, and then finding the Hessian - but I think there might be a more PyTorch-oriented way of doing the same thing.</p>
<p>What I currently have in optimizer.step():</p>
<pre><code class="lang-auto">        #Iterate over parameter groups
        for param_group in self.param_groups:
            for param in param_group:
                if not param.requires_grad(): continue
                else:
                    gradients = param.grad_data
                    #This doesn't really make sense, as the hessian should be made from param_group and not from parameters
                    param["HESSIAN"] = torch.autograd.grad(gradients, param)
</code></pre>
Permute vs Transpose

picklerick: <p>I understand that there are a couple of posts that explain the difference between permute and transpose verbally. But is anyone aware of a visual explanation that shows the difference between the two, perhaps with an example tensor? (I would also be super grateful if someone could also make a visual explanation <img src="https://discuss.pytorch.org/images/emoji/apple/hugs.png?v=12" title=":hugs:" class="emoji" alt=":hugs:" loading="lazy" width="20" height="20"> - it would help me really internalise the concept). Thanks in advance!!</p>

ptrblck: <p>I would claim that the difference is small and while <code>transpose</code> expects two dimensions, <code>permute</code> expects an input corresponding to all dims. However, you could achieve the same results if you are only permuting two dimensions as seen here:</p>
<pre><code class="lang-python">x = torch.arange(2*3*4*5).view(2, 3, 4, 5)
print(x)

y1 = x.transpose(1, 2)
print(y1)
y2 = x.permute(0, 2, 1, 3)
print(y2)

print((y1 == y2).all())
# tensor(True)
</code></pre>
Pytorch geometric Data Object

planet: <p>Is it possible to store two graphs in the PyTorch geometrics Data object?<br>
e.g.<br>
<code>Data(x=[B, 1100], edge_index=[B, 2, 4400], edge_attr=[B, 4400, 35], y=[B, 2])</code><br>
where <code>B=2</code> or <code>B=1</code></p>
<p>In the official documentation, it has mentioned that the object should be as follows<br>
<code>Data(x=[1100], edge_index=[2, 4400], edge_attr=[4400, 35], y=[2])</code></p>
<p>Can someone clarify, I was trying to create a custom Dataloader.</p>
Patching subimages into an image using vectorization

Johnny_Cash: <p>Hey, I have this following piece of code:</p>
<pre><code class="lang-auto"># image is a torch tensors being on the gpu representing an image
# bounding boxes is a list of lists describing 4 coordinates

for bb in bounding_boxes:

    x1 = bb[0]
    x2 = bb[1]
    y1 = bb[2]
    y2 = bb[3]

    # cutout image 
    image_cutout = image[y1:y2, x1:x2, ...]

    # here do some operation on image_cutout on the gpu

    # paste it back
    image[y1:y2, x1:x2, ...] = image_cutout
</code></pre>
<p>Is there some way to vectorize this? I would like to, having all the image cutouts, put them back into original image. Thanks for any help!</p>
Masking image on GPU

Johnny_Cash: <p>Is there a straightforward way given a mask and an image to blur the part of the image the mask corresponds to on torch.cuda (meaning doing everything on the GPU). The GaussianBlur is implemented, but how would I do the masking (would it be the same as I would do it in numpy or is there some optimized method for it)?</p>
<p>Thanks!</p>

sudomaze: <p>Yes. You can move your tensors into your device in your transformation before applying the GaussianBlur. Here is an example:</p>
<pre><code class="lang-python">import torch
import torchvision
import torchvision.transforms as T
# ...
device = torch.device('cuda') 
transform = T.Compose([
    # ...
    T.ToTensor(),
    T.Lambda(lambda x: x.to(device))
    T.GaussianBlur((7, 13))
    # ...
])
imgs = torch.stack([copy.deepcopy(transform(img)) for i in range(10)])

for i in range(imgs.shape[0]):
  print(imgs[i].device)
# &gt;&gt;&gt; cuda:0
# ...
# &gt;&gt;&gt; cuda:0
</code></pre>
<p>You might consider looking into <a href="https://discuss.pytorch.org/t/to-device-gives-an-error-when-used-inside-transforms-compose/51387">this post</a> for troubleshooting when having your data loader to have multiple workers.</p>

Johnny_Cash: <p>I was talking rather about the masking, but I figured it out. Thanks anyway!</p>
Pytorch geometric dataset loading is slow

Edan_Patt: <p>Hello everyone,<br>
I created my own large dataset. Every time I want to run or train anything the dataset has to be processed. I tried my best to optimize this process but after profiling my run I found out that the function<br>
“file_exists”  (<a href="https://pytorch-geometric.readthedocs.io/en/1.4.3/_modules/torch_geometric/data/dataset.html" class="inline-onebox" rel="noopener nofollow ugc">torch_geometric.data.dataset — pytorch_geometric 1.4.3 documentation</a>)<br>
is taking up 87% of my runtime with its calls to posix.stat. Is there any clear cut way to optimize this part of the function call? Or anything I can do with the way my files are written or ordered to help out with this?</p>
<p>Thanks</p>

ejguan: <p>Based on the implementation, <code>files_exist</code> only loops through files and figure our if any file is missing from your FS.</p>
<p>Do you want to try with <a href="https://github.com/pytorch/data" rel="noopener nofollow ugc">torchdata</a> since we are advocating streaming way to load data. So, you can download your data on the fly rather than downloading them beforehand. Here is the <a href="https://pytorch.org/data/beta/index.html" rel="noopener nofollow ugc">doc</a>.</p>
How to build pytorch without AVX?

pi-null-mezon: <p>I want to run torch 1.11.0 on old machine equipped by AMD Phenom II N850 (no AVX / AVX2 / AVX 512). So, i decide to build torch from sources:</p>
<pre><code class="lang-auto">wget https://github.com/pytorch/pytorch/releases/download/v1.11.0/pytorch-v1.11.0.tar.gz &amp;&amp; \
tar -xvzf pytorch-v1.11.0.tar.gz &amp;&amp; rm pytorch-v1.11.0.tar.gz &amp;&amp; \
cd pytorch-v1.11.0 &amp;&amp; mkdir build &amp;&amp; cd build &amp;&amp; \
cmake -D CMAKE_BUILD_TYPE=Release \
      -D C_HAS_AVX_2=OFF \
      -D C_HAS_AVX2_2=OFF \
      -D CXX_HAS_AVX_2=OFF \
      -D CXX_HAS_AVX2_2=OFF \
      -D CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS=OFF \
      .. &amp;&amp; \
make -j4 &amp;&amp; \
sudo make install &amp;&amp; \
sudo ldconfig
</code></pre>
<p>Building process successfully finishes. But in my test application, however, when inferences should be done, application crashes with OS-message:</p>
<pre><code class="lang-auto">Illegal instruction (core dumped)
</code></pre>
<p>So, what i am doing wrong? It seems fro me, that I do not properly disable all AVX-specfic flags at build time. Please help.</p>

pi-null-mezon: <p>AMD Phenom II N850 specs, as reported by OS:</p>
<pre><code class="lang-auto">Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              3
On-line CPU(s) list: 0-2
Thread(s) per core:  1
Core(s) per socket:  3
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          16
Model:               5
Model name:          AMD Phenom(tm) II N850 Triple-Core Processor
Stepping:            3
CPU MHz:             800.000
CPU max MHz:         2200,0000
CPU min MHz:         800,0000
BogoMIPS:            4389.27
Virtualization:      AMD-V
L1d cache:           64K
L1i cache:           64K
L2 cache:            512K
NUMA node0 CPU(s):   0-2
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate vmmcall npt lbrv svm_lock nrip_save
</code></pre>

pi-null-mezon: <p>Finally I have found set of flags, that allows me to properly build torch for my hardware. So, I want to share with the community:</p>
<pre><code class="lang-auto">cmake -D CMAKE_BUILD_TYPE=Release \
      -D USE_AVX=OFF \
      -D USE_NNPACK=OFF \
      -D USE_MKLDNN=OFF \
      -D C_HAS_AVX_2=OFF \
      -D C_HAS_AVX2_2=OFF \
      -D CXX_HAS_AVX_2=OFF \
      -D CXX_HAS_AVX2_2=OFF \
      -D CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS=OFF \
      ..
</code></pre>
What is the purpose of "torch.Tensor.name" attribute?

khshim20: <p>Hi.</p>
<p>While trying named tensors,<br>
I found torch.Tensor have attribute ‘name’.<br>
It prints None, and also is not writable.<br>
I think there is no documentation for this…</p>
<p>Can you help me understand what ‘name’ does?</p>

yash1994: <p>Hi,<br>
Documentation is available for named tensor:</p>
<ol>
<li><a href="https://pytorch.org/docs/stable/named_tensor.html" rel="nofollow noopener">Named Tensor</a></li>
<li><a href="https://pytorch.org/tutorials/intermediate/named_tensor_tutorial.html" rel="nofollow noopener">Named Tensor Tutorial</a></li>
</ol>

khshim20: <p>Thanks. But I’m pretty sure ‘name’ is not related to named tensors…</p>
<p>I can access tensor names by ‘torch.Tensor.names’ for named tensors. [‘N’, ‘C’, …]<br>
But ‘torch.Tensor.name’ is still None for named tensors.</p>

yash1994: <p>Yes, you are right <code>tensor.name</code> returns always <code>None</code> and there doesn’t seem to be a documentation for it. In tensorflow <code>tensor.name</code> gives name of variable/tensor in graph execution. Sorry for mis-direction.</p>

ptrblck: <p>Based on <a href="https://github.com/pytorch/pytorch/pull/25424#issuecomment-526620112">this comment</a> it seems <code>tensor.name</code> is currently used in ONNX (but I’m not sure how and where).</p>

TriKri: <p>At the same time, the <a href="https://pytorch.org/docs/stable/named_tensor.html#torch.Tensor.rename" rel="noopener nofollow ugc"><code>rename</code></a> method seems to update the <code>names</code> attribute, not the <code>name</code> attribute. So it seems like the most reasonable name for a method that renames a tensor is already taken by a method that changes its dimension names.</p>
Detectron2 interference issue

manju_kannan: <p>Hello,</p>
<p>I am using the detectron2 model. Using Google Colab, I have trained and tested the custom dataset and downloaded the config and pth file to local. I am facing an issue with inference. trained weight file not returning bounding box value it gives empty result.</p>
<p>Can you guide me to find out issue.</p>
<p>Thanks in advance.</p>
<pre><code>@app.post('/api/ocr/table')
async def predict_api(file: UploadFile = File(...)):
fp = Path((r'{}data/input/img.png').format(uri))
imgjson = ''
async with aiofiles.open(fp, 'wb') as fh:
    while True:
        chunk = await file.read(CHUNK_SIZE)
        if not chunk:
            break
        await fh.write(chunk)
img_path = fp

image1 = cv2.imread((f'{img_path}').format(uri))
cfg = get_cfg()
cfg.merge_from_file((r"{}Detectron2_model/config.yml").format(uri))"""
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.8
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 2 
cfg.MODEL.WEIGHTS = (r"{}Detectron2_model/model_final.pth").format(uri)

predictor = DefaultPredictor(cfg)

outputs = predictor(image1)

return {"details":"saved", "result": outputs }
</code></pre>
<p>Response:</p>
<pre><code class="lang-auto">{
  "details": "saved",
  "result": {
    "instances": {
      "_image_size": [
        3301,
        2550
      ],
      "_fields": {
        "pred_boxes": {
          "tensor": {}
        },
        "scores": {},
        "pred_classes": {}
      }
    }
  }
}
</code></pre>
Handle big dataset with limited memory resources

Torcione: <p>Hi everyone,<br>
I’m trying to run my experiments on INaturalist dataset for the first time.<br>
The latter is one of the built-in dataset available on pytorch (<a href="https://pytorch.org/vision/main/generated/torchvision.datasets.INaturalist.html" rel="noopener nofollow ugc">INaturalist</a>).<br>
When I arrive at the initialization of the dataset the code get stuck for a while; finally the execution stops and i get the message:</p>
<blockquote>
<p>/usr/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown</p>
</blockquote>
<p>I read that a possible problem could be the fact that (<a href="https://pytorch.org/vision/0.12/_modules/torchvision/datasets/inaturalist.html" rel="noopener nofollow ugc">torchvision.datasets.INaturalist</a>) try to load the whole dataset in the memory during the init call (early loading), instead of performing the <a href="https://www.analyticsvidhya.com/blog/2021/09/torch-dataset-and-dataloader-early-loading-of-data/" rel="noopener nofollow ugc">lazy loading</a>.<br>
My questions are:<br>
Does my memory get saturated because (<a href="https://pytorch.org/vision/0.12/_modules/torchvision/datasets/inaturalist.html" rel="noopener nofollow ugc">torchvision.datasets.INaturalist</a>) performing early loading?<br>
If yes, is it possible to easly modify the built-in class in order to pass to a lazy loading approach or I should define a new class to import the dataset? How should I proceed?</p>
How to select specific vector in 3D tensor beautifully?

jamestang0219: <p>There is a 3D tensor(batch_size * seq_len * hidden_dim), and I wanna get the different hidden vector for each sequence, do you have a beautiful method to do it except use for loop?</p>
<p>Tensor:<br>
[[[1, 2, 3],<br>
[4, 5, 6],<br>
[7, 8, 9]],</p>
<p>[[11, 12, 13],<br>
[14, 15, 16],<br>
[17, 18, 19]]]</p>
<p>Target matrix:<br>
[[4, 5, 6],<br>
[17, 18, 19]]</p>
<p>and I have the index[1, 2] for select the specific vector, now I’m using for loop to get every vector for each sequence. And I found torch.gather and torch.index_select may be not suitable for my question.</p>
<p>Thank you for help!</p>

ptrblck: <p>This should work:</p>
<pre><code class="lang-python">x = torch.tensor([[[1, 2, 3],
                   [4, 5, 6],
                   [7, 8, 9]],
                  [[11, 12, 13],
                   [14, 15, 16],
                   [17, 18, 19]]])

idx = torch.tensor([1, 2])
x[torch.arange(x.size(0)), idx]
</code></pre>

jamestang0219: <p>Thanks, I’ve tried slicing: x[:, idx], but it’s not correct. So using indexing is correct, thank you again.</p>

SeunghyunSEO: <p>thanks from 2022 XD, it works pretty well !</p>

yuekaizhang: <p>Hi ptrblck, I have the similar situation. However, I have boolean index mats</p>
<p>[[False, True, False],<br>
[False, True, True]]<br>
I would like select corresponding columns:</p>
<p>[4,5,6]</p>
<p>[14,15,16]<br>
[17,18,19]</p>
<p>Then padding them with -1:</p>
<p>The expected results:<br>
[[[4,5,6],<br>
[-1,-1,-1]<br>
[-1, -1, -1]],</p>
<p>[[14, 15, 16],<br>
[17, 18, 19],<br>
[-1, -1, -1]]]</p>
<p>Do you know if it is possible to do it except use for loop?</p>

ptrblck: <p>Something like this should work:</p>
<pre><code class="lang-python">x = torch.tensor([[[1, 2, 3],
                   [4, 5, 6],
                   [7, 8, 9]],
                  [[11, 12, 13],
                   [14, 15, 16],
                   [17, 18, 19]]])

mask = torch.tensor([[False, True, False],
                     [False, True, True]])

res = torch.ones_like(x) * -1
idx1 = mask.nonzero()[:, 0]
idx2 = torch.cat([torch.arange(a) for a in mask.cumsum(1).max(1).values])
res[idx1, idx2] = x[mask]

print(res)
# tensor([[[ 4,  5,  6],
#          [-1, -1, -1],
#          [-1, -1, -1]],

#         [[14, 15, 16],
#          [17, 18, 19],
#          [-1, -1, -1]]])
</code></pre>
How can i find the function "as_strided" definition or someone else in libtorch in pytorch git repo？

liguangops: <p>I have cloned the pytorch repo on github， and Used <code>grep</code> to check the keyword <code>as_strided</code>. But the displayed results did not find any useful information. I also gone to the website for libtorch documention to find some information. But the website only gives the declaration of the function.</p>
Pin_memory() cuda runtime error

heilaw: <p>I am using pinned memory to speed up the training process. In my program, I use a thread to move the training data to the pinned memory. Occasionally, I encountered the following error in the middle of the training:</p>
<pre><code>THCudaCheck FAIL file=/py/conda-bld/pytorch_1493677666423/work/torch/lib/THC/THCCachingHostAllocator.cpp line=258 error=11 : invalid argument
Exception in thread Thread-4:
Traceback (most recent call last):
  File "/home/heilaw/.conda/envs/pytorch/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/home/heilaw/.conda/envs/pytorch/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "./train.py", line 43, in pin_memory
    data["xs"] = [x.pin_memory() for x in data["xs"]]
  File "./train.py", line 43, in &lt;listcomp&gt;
    data["xs"] = [x.pin_memory() for x in data["xs"]]
  File "/home/heilaw/.conda/envs/pytorch/lib/python3.5/site-packages/torch/tensor.py", line 78, in pin_memory
    return type(self)().set_(storage.pin_memory()).view_as(self)
  File "/home/heilaw/.conda/envs/pytorch/lib/python3.5/site-packages/torch/storage.py", line 84, in pin_memory
    return type(self)(self.size(), allocator=allocator).copy_(self)
RuntimeError: cuda runtime error (11) : invalid argument at /py/conda-bld/pytorch_1493677666423/work/torch/lib/THC/THCCachingHostAllocator.cpp:258
</code></pre>
<p>Any idea why this would happen? How can I debug it?</p>

smth: <p>if you can give me a script to reproduce this, what I would do is put a printf in <a href="https://github.com/pytorch/pytorch/blob/master/torch/lib/THC/THCCachingHostAllocator.cpp#L258">https://github.com/pytorch/pytorch/blob/master/torch/lib/THC/THCCachingHostAllocator.cpp#L258</a> and see what the size or ptr values are when the failure occurs. From there, I would backtrack to see why these failure values are being generated.</p>

heilaw: <p>It’s an ongoing project so I cannot share that with you. But I will see if I can write a script for you to reproduce that. Thanks!</p>

skaufman: <p>I haven’t had a second to write a self-contained, reproducing example script, but wanted to chime in to say that I resolved this issue by making the Tensor contiguous before pinning it. (I had taken a slice earlier.)</p>

jud: <p>I ran into the same problem. Because I had a variable batch-size and limited memory on the GPU I had to split the batch into junks having a maximum allowed batch-size. Because ‘split’ only creates a view of the tensor and I still wanted to pin the memory, I had to make each junk contiguous.</p>
<pre><code class="lang-auto">z = torch.cat([
    self.feature_extractor(
        batch.contiguous().to(device=device)  # push batch to device
    )['flatten']  # extract the output of the flatten layer
    for batch in torch.split(x, max_batch_size)  # loop over junks
])
</code></pre>