akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Wrong outputs from torch.sub()?

JCouture: <p>I’m currently using torch.sub alongside torch.div to obtain the MAPE between my predicted and true labels for my neural network although I’m not getting the answers I’m expecting. According to the example in the documentation, I should be getting a 4x1 tensor, not 4x4.<br>
Could anyone clear this up for me?</p>
<blockquote>
<p>print('y_true ', y_true)<br>
y_true  tensor([[ 46],<br>
[262],<br>
[ 33],<br>
[ 35]], device=‘cuda:0’, dtype=torch.int16)</p>
</blockquote>
<blockquote>
<p>print('y_pred ', y_pred)<br>
y_pred  tensor([[[308.5075]],<br>
[[375.8983]],<br>
[[389.4587]],<br>
[[406.4957]]], device=‘cuda:0’, grad_fn=)</p>
</blockquote>
<blockquote>
<p>print('torch.sub ', torch.sub(y_true, y_pred))<br>
torch.sub  tensor([[[-262.5075],<br>
[ -46.5075],<br>
[-275.5075],<br>
[-273.5075]],</p>
<pre><code>    [[-329.8983],
     [-113.8983],
     [-342.8983],
     [-340.8983]],

    [[-343.4587],
     [-127.4587],
     [-356.4587],
     [-354.4587]],

    [[-360.4957],
     [-144.4957],
     [-373.4957],
     [-371.4957]]], device='cuda:0', grad_fn=&lt;SubBackward0&gt;)
</code></pre>
</blockquote>

JCouture: <p>I figured out the issue which was that y_pred actually had a shape of [4,1,1] whilst y_true had only [4,1]. I got rid of the third dimension by using torch.squeeze(y_pred, 2) and this has fixed the issue.</p>
PlaneRCNN nms and roialign

d4riush: <p>Hi, I want to use NVIDIA’s <a href="https://github.com/NVlabs/planercnn" rel="noopener nofollow ugc">PlaneRCNN</a>. The code is old, and I guess it belongs to the pre torchvision era. Some parts of code need <code>nms</code> and <code>roialign</code> functionality but to use them, some <a href="https://github.com/multimodallearning/pytorch-mask-rcnn#installation" rel="noopener nofollow ugc">compilation</a> should be done. I tried that, and there is a lot of error in windows, and it turns out this is a <a href="https://github.com/multimodallearning/pytorch-mask-rcnn/issues/106" rel="noopener nofollow ugc">known issue</a>.<br>
As a workaround, I want to use torchvision <code>nms</code> and <code>roialign</code> functionality. I somehow implemented expected <code>nms</code> for the code as follow:</p>
<pre><code class="lang-python">import torchvision
import torch
def nms(dets, thresh):
    iou_threshold = thresh
    x1 = dets[:, 1][..., None]
    y1 = dets[:, 0][..., None]
    x2 = dets[:, 3][..., None]
    y2 = dets[:, 2][..., None]
    boxes = torch.concat([x1, y1, x2, y2], axis=1)
    scores = dets[:, 4]
    return torchvision.ops.nms(boxes, scores, iou_threshold)
</code></pre>
<p>I don’t know if it is working, but I think it is ok. For the <code>roialign</code> part, there is an import line as follow:</p>
<pre><code class="lang-python">from roialign.roi_align.crop_and_resize import CropAndResizeFunction
</code></pre>
<p>I want to know whether there is any viable solution in torchvision to implement this. Any further help and suggestion about the approach are highly appreciated.</p>
Roberta from transformer library giving same output during evaluation

chetan06: <p>I am using roberta from transformers library. I have trained the model for the classification task and taken the <code>model.pooler_output</code> and passed it to a classifier. While predicting I am getting same prediction for all the inputs. What could be the possible reason</p>
<p>This is my model</p>
<pre><code class="lang-python">class Roberta_Arch(nn.Module):
    def __init__(self, roberta): 
        super(Roberta_Arch, self).__init__()
        self.roberta = roberta
        self.classifier = nn.Sequential(
            nn.Linear(768, 256), 
            nn.ReLU(),
            nn.Dropout(0.15),
            nn.Linear(256, 2),
            nn.LogSoftmax(dim = 1)
        )

    def forward(self, sent_id, mask):
        output = self.roberta(sent_id, attention_mask=mask)
        return self.classifier(output.pooler_output)
</code></pre>
<p>And this is my evaluation function</p>
<pre><code class="lang-python">def evaluate():
    print('\n\n Predicting \n')
    model.eval()
    total_preds = []

    for step, batch in tqdm(enumerate(test_dataloader)):
        batch = [t.to(device) for t in batch]
        sent_id, mask = batch

        with torch.no_grad():
            preds = model(sent_id, mask)
            preds = preds.detach().cpu().numpy()
            print(preds)
            total_preds.append(preds)

    return total_preds
</code></pre>
Training Mask R-CNN on MS COCO (single class): urllib error

sigma_x: <p>The dataset interface runs problem-free for a couple of epochs, then outputs this error:</p>
<blockquote>
<p>urllib.error.HTTPError: HTTP Error 503: Service Unavailable</p>
</blockquote>
<p>when running the line with <code>scikit-image</code> method for opening a URL:</p>
<blockquote>
<p>im = np.array(io.imread(self.img_data[idx][‘coco_url’]))</p>
</blockquote>
<p>I’m training a single class (‘toothbrush’), and the model has 2 labels, background+class. Toothbrush has label 80, so I change 80 to 1 when training. None of the errors I could relate to are reported, just this one.</p>
<p>EDIT: another error that pops up (same method):</p>
<blockquote>
<p>ConnectionResetError: [Errno 104] Connection reset by peer</p>
</blockquote>

nivek: <p>I think it is possible that the server that the URLs are pointing have a quota or is overloaded, such that your request gets an HTTP Error 503.</p>
<p>Is it possible for you to download the data in advance and load them from disk?</p>

sigma_x: <p>Yeah, that’s right. After a bit of investigation I discovered that I used this solution a year ago!</p>
Calculate kernel by blocks

leo2k: <p>Hi all, I’m currently have a tensor with side BxNxD (batch size x num sample each block x dimension)<br>
Can anybody come across give me a help, please? I want to speed up the following computation:</p>
<pre><code class="lang-auto">K = Kernel() # similar with https://github.com/activatedgeek/svgd/blob/master/rbf.py
sum = 0
for i in range(B):
    sum += K(tensor[i], tensor[i]) 
</code></pre>
<p>It calculate kernel of a NxD tensor (which returns a NxN tensor) and then sum over the minibatch</p>

anantguptadbl: <p><a class="mention" href="/u/leo2k">@leo2k</a> This sounds more like a process bottleneck. Recommend the usage of multiprocessing pool to speed up</p>
<p><a href="https://docs.python.org/3/library/multiprocessing.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://docs.python.org/3/library/multiprocessing.html</a></p>
<p>if you are up for adventure, you can use the cuda cores to do this calculation. This will be blazing fast</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.geeksforgeeks.org/running-python-script-on-gpu/">
  <header class="source">
      <img src="https://media.geeksforgeeks.org/wp-content/cdn-uploads/gfg_favicon.png" class="site-icon" width="16" height="16">

      <a href="https://www.geeksforgeeks.org/running-python-script-on-gpu/" target="_blank" rel="noopener nofollow ugc" title="10:47AM - 21 August 2019">GeeksforGeeks – 21 Aug 19</a>
  </header>

  <article class="onebox-body">
    <img src="https://media.geeksforgeeks.org/wp-content/cdn-uploads/gfg_200x200-min.png" class="thumbnail onebox-avatar" width="200" height="200">

<h3><a href="https://www.geeksforgeeks.org/running-python-script-on-gpu/" target="_blank" rel="noopener nofollow ugc">Running Python script on GPU. - GeeksforGeeks</a></h3>

  <p>A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>


leo2k: <p>can anyone give me a help in implementing this, please?</p>

leo2k: <p>[Bump up]<br>
can anyone give me a help in implementing this, please?</p>
Is_leaf behavior for weight_norm'ed modules and implications for deepcopy

nrxsvzo: <p>Using Pytorch 0.4.1</p>
<p>I noticed that immediately after initializing a module with weight_norm, the value of module.weight.is_leaf is False.  After executing the module within a torch.no_grad() statement, the value changes to True.  Is this expected?</p>
<p>When the value is False, attempts to copy the module with copy.deepcopy fail with the error message: “Only tensors created explicitly by the user (graph leaves) support deepcopy at the moment”.</p>
<p>I noticed because I happen to want to be able to load a model and copy it before running any inference calls.  It’s not a big deal as I can work around it, but it seemed like a potentially unintended behavior.</p>

pozhijisi: <p>The same error happens in Pytorch version 1.6.0.</p>