akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Best Practice: Segmentation evaluation metrics

username: jankunze
cooked: <p>Hi <span class="mention">@all</span>,</p>
<p>I have a more general question concerning the evaluation metrics for segmentation. Which ones are commonly used or do you see as ‘best practice’ for researchers? Which one would you include in publications?</p>
<p>I am talking more about comparability than efficiency. I am aware that the use of metrics can be task dependent. Some applications will best work with very specialized or custom methods. I am also aware that the selection of metrics can be predefined by cirumstances (e.g.Kaggle competition). But if I am new to (semantic) segmentation, which metrics should I implement in my code to evaluate the outcome of my segmentation model?</p>
<p>Here is a non-comprehensive list of metrics that I’ve found (and partially used) so far. I did not always include ‘statistical variations’ like mean/average, median, standard deviation, etc… For a deeper look into the topic see: <em>Taha, Abdel Aziz, and Allan Hanbury. “Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool.” BMC medical imaging 15.1 (2015): 29.</em></p>
<p><strong>Metrics that seem to be commonly used</strong> (e.g. <a href="https://towardsdatascience.com/metrics-to-evaluate-your-semantic-segmentation-model-6bcb99639aa2" rel="nofollow noopener">[1]</a> <a href="https://www.jeremyjordan.me/evaluating-image-segmentation-models/" rel="nofollow noopener">[2]</a>)</p>
<p>Based on confusion matrix</p>
<ul>
<li>Global/Per_Class Accuracy</li>
<li>Precision</li>
<li>Recall</li>
<li>Intersection over Unit(IOU) = Jaccard Index</li>
<li>F1 score = Sørensen–Dice coefficient (Dice)</li>
</ul>
<p><strong>Metrics that I rarely encounter</strong></p>
<p>Based on confusion matrix</p>
<ul>
<li>Cohen’s kappa / Observed Accuracy</li>
<li>Global Consistency Error (GCE) / Local Consistency Error(LCE)</li>
</ul>
<p>Base on pair counting</p>
<ul>
<li>Rand Index / Adjusted Rand Index</li>
</ul>
<p>Based on spatial distance</p>
<ul>
<li>Hausdorff Distance and weighted Hausdorff Distance</li>
</ul>
<p>There are lots of more metrics out there.  But did I miss some major ones? Which of these do you suggest to use for starters? And which ones do you use?</p>

username: Neda
cooked: <p>As you said, it’s very task-dependent. In medical application, I would say most papers are using dice coefficient, Jaccard index (iOU), Hausdorff Distance. You need to check what are the most common metrics in your field.</p>
RuntimeError converting mask rcnn to c++

username: nclk
cooked: <p>Hi everyone,</p>
<p>I have built mask r cnn code in <a href="https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html" rel="nofollow noopener">https://pytorch.org/tutorials/intermediate/torchvision_tutorial.html</a>. I can train model, and test well. But when I run code:</p>
<p>“example = torch.rand(1, 3, 559, 536)<br>
model.to(‘cpu’)<br>
traced_script_module = torch.jit.trace(model, example)”</p>
<p>I got an error:</p>
<p>"File “C:\Users\ncali\anaconda3\envs\pt_37\lib\site-packages\torchvision\models\detection_utils.py”, line 218, in decode_single pred_ctr_x = dx * widths[:, None] + ctr_x[:, None]</p>
<p>RuntimeError: expected device cpu but got device cuda:0"</p>
<p>I think it is related with “RuntimeError after moving fasterrcnn_resnet50_fpn to CPU from GPU <span class="hashtag">#1738</span>”</p>

username: nclk
cooked: <p>I have just added some trace line in _utils.py:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/b/1/b16178341315da8e5468c2f7954a8df6b987167f.png" alt="image" data-base62-sha1="pjbl4dcOuncmQDU7YPjJzuTcGlN" width="531" height="185"></p>
<p>in training phase, it returns :<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/a/8/a879542f391e31b8ff9d0c3331608f4b58fc81c5.png" alt="image" data-base62-sha1="o2o9Ysahj9O3Ry1x6RUVD3gLrAV" width="145" height="124"></p>
<p>but after  “model.to(‘cpu’)”, I have get it:<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/8/1/816f22ed85601be4c7ea5d6d4d5ac4aab3ed3127.png" alt="image" data-base62-sha1="it1Hy2UZMLfU86T6464lKBN9hfp" width="335" height="87"></p>

username: nclk
cooked: <p>Here is my new question:<br>
I have tried by training over cpu, and overcome it. But now, I got this error:</p>
<p>“RuntimeError: Only tensors or tuples of tensors can be output from traced functions (getOutput at …\torch\csrc\jit\tracer.cpp:212)<br>
(no backtrace available)” <img src="https://discuss.pytorch.org/images/emoji/apple/confused.png?v=9" title=":confused:" class="emoji" alt=":confused:"></p>
Exponential decay learning rate

username: _joker
cooked: <p>I am trying to implement  – “The learning rate is decayed exponentially from 0.01 to 0.0001 for 30 epochs.”</p>
<p>I found <code>lr_scheduler = optim.lr_scheduler.ExponentialLR(optimizer, gamma)</code></p>
<pre><code class="lang-auto">optimizer = optim.Adam()
lr_scheduler = optim.lr_scheduler.ExponentialLR(optimizer, gamma)

for epoch in range(nb_epoch):
        for data in train_loader:
                ....
                ....
                optimizer.step()
lr_scheduler.step()
</code></pre>
<p>So how do I use the epoch and learning rate 0.01 to 0.0001 in this case?</p>

username: charan_Vjy
cooked: <p>If you want to decay the learning rate every 30 epochs you could use<br>
<code>torch.optim.lr_scheduler.StepLR (optimizer , 30 , gamma=0.01)</code></p>

username: _joker
cooked: <p>Thanks for the response, but the decay has to be exponentially. So can I do the same in ExponentialLR ?</p>

username: AladdinPerzon
cooked: <p>If I’ve understood it correctly you want to use exponential lr rate scheduler and starting at 0.01 you want it to decrease to 0.0001 after 30 epochs. We can do some calculations where we should have:</p>
<p>0.01 * gamma^30 = 0.0001, from this we can imply that gamma = 0.01^(1/30) = 0.85769…</p>
<p>So set gamma approximately to 0.86 and it should work as you wanted (approximately).</p>
<p>I think your <code>lr_scheduler.step()</code> should be inside the epoch for-loop.</p>
Problem with LibTorch gcc version

username: celia
cooked: <p>I’m following this guide <a href="https://pytorch.org/tutorials/advanced/cpp_export.html" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/cpp_export.html</a>  to load pytorch bert model in C++, the problem is that libtorch requires gcc 5+, but I can only use gcc 4.8.5.<br>
I’m not allowed to update gcc, so is there any solutions?<br>
Or any other ways to load a bert model in C++?</p>

username: albanD
cooked: <p>I’m afraid we use features from gcc 5+ so you won’t be able to compile for older versions. You will need to update your gcc if you want to use the cpp API.</p>

username: celia
cooked: <p>Thank you for replying~ but if I can only use gcc 4.8.5, is there any other methods to load or build a Bert model in C++ environment?  don’t have to use pytorch to build model…</p>

username: albanD
cooked: <p>I’m afraid you won’t be able to compile cpp code if you don’t have the right version.<br>
You might be able to compile using it as a shared library but I’m not sure it will work. You might needs to get the the pre-cxx11 ABI version though.</p>
Libtorch operator/syntax errors in Visual Studio

username: theycallmefm
cooked: <p>Hi recently I’ve installed Libtorch and I was able to use it in my new Visual Studio project without a problem. Currently I am trying to use Libtorch in an existing CUDA project. But I am having these strange errors when I include torch header and I couldn’t find any solution in the internet regarding to my problem. Does anyone have any idea what might be the cause of these errors?</p>
<pre><code class="lang-auto">Severity	Code	Description	Project	File	Line	Suppression State
Error	C2833	'operator {' is not a recognized operator or type	DepthSensing	e:\research\libtorch\include\c10\util\flat_hash_map.h	1433	
Error	C2059	syntax error: 'newline'	DepthSensing	e:\research\libtorch\include\c10\util\flat_hash_map.h	1433	
Error	C2238	unexpected token(s) preceding ';'	DepthSensing	e:\research\libtorch\include\c10\util\flat_hash_map.h	1433	
Error	C2143	syntax error: missing ';' before 'const'	DepthSensing	e:\research\libtorch\include\c10\util\flat_hash_map.h	1433	
Error	C2833	'operator {' is not a recognized operator or type	DepthSensing	e:\research\libtorch\include\c10\util\order_preserving_flat_hash_map.h	1552	
Error	C2059	syntax error: 'newline'	DepthSensing	e:\research\libtorch\include\c10\util\order_preserving_flat_hash_map.h	1552	
Error	C2238	unexpected token(s) preceding ';'	DepthSensing	e:\research\libtorch\include\c10\util\order_preserving_flat_hash_map.h	1552	
Error	C2143	syntax error: missing ';' before 'const'	DepthSensing	e:\research\libtorch\include\c10\util\order_preserving_flat_hash_map.h	1552
</code></pre>
<p>My environment:<br>
Windows 10<br>
CUDA 10.1<br>
Visual Studio 2017<br>
C++ 14</p>

username: theycallmefm
cooked: <p>There was a macro in another library which has same name as a typename in Libtorch library(which was a macro called <strong>V</strong> in my case), that’s why it was confused in compilation.</p>
Fixing attention of CNNs for image classification

username: TinfoilHat0
cooked: <p>I’d like to fix attention of a CNN to a particular region of a given image.</p>
<p>For example, given 28x28 image, I’d like my model to only consider only the top-left 5x5 region.<br>
What’s the proper way of doing this, if possible?</p>
<p>A naive work-around I can think of is that, I could overwrite the unwanted regions with black-pixels. Not sure if this is a nice solution though.</p>

username: ptrblck
cooked: <p>Would it work, if you just crop the input image to the desired shape and pass this small patch to the model?</p>

username: TinfoilHat0
cooked: <p>Worth trying. Thanks.</p>
Learning Rate Scheduler Not Working as Expected

username: Remote_Senser
cooked: <p>I tried to implement a learning rate scheduler using StepLR on Pytorch using the instructions provided.  This is my code:</p>
<pre><code class="lang-auto">        optimizer = optim.SGD(model.parameters(), lr=LR, weight_decay=decay, momentum=momentum, dampening=dampening)
        scheduler = StepLR(optimizer, step_size=2, gamma=0.1)
        trainset = TrainDataset(train, trainlabels)
        train_loader = torch.utils.data.DataLoader(
                trainset,
                batch_size=batch_size,
                shuffle=True,
                )
        for epoch in range(0, 6):
            print('Epoch: ', epoch, ', LR: ', scheduler.get_lr())
            running_loss = 0
            batches = 0
            for inputs, labels in train_loader:
                batches = batches+1
                inputs, labels = inputs.to(device), labels.to(device)
                optimizer.zero_grad()
                output = model(inputs)
                output = output.squeeze()
                _, dimx, dimy = output.shape
                loss = criterion(output)
                loss.backward()
                optimizer.step()
                running_loss += loss.item()
            scheduler.step()
            print('Loss :{:.4f} Epoch[{}/{}]'.format(running_loss/batches, epoch, epochs))
</code></pre>
<p>The learning rate scheduler does not work as expected. This is how the output:</p>
<p>Epoch:  0 , LR:  [0.1]<br>
Loss :0.1981 Epoch[0/6]</p>
<p>Epoch:  1 , LR:  [0.1]<br>
Loss :0.1957 Epoch[1/6]</p>
<p>Epoch:  2 , LR:  [0.0010000000000000002]<br>
Loss :0.1360 Epoch[2/6]</p>
<p>Epoch:  3 , LR:  [0.010000000000000002]<br>
Loss :0.1332 Epoch[3/6]</p>
<p>Epoch:  4 , LR:  [0.00010000000000000003]<br>
Loss :0.1293 Epoch[4/6]</p>
<p>Epoch:  5 , LR:  [0.0010000000000000002]<br>
Loss :0.1289 Epoch[5/6]</p>
<p>As you can see in Epoch 2 and Epoch 4 (the transition epochs to the new learning rate), the learning rates are actually 10x less than what it should be. Why is this happening? Have I implemented StepLR correctly?</p>

username: AladdinPerzon
cooked: <p>I think the issue is with the <code>scheduler.get_lr()</code> when reading the <a href="https://github.com/pytorch/pytorch/blob/fde94e75568b527b424b108c272793e096e8e471/torch/optim/lr_scheduler.py#L344-L350" rel="nofollow noopener">implementation</a>, it seems that you should instead use <code>scheduler.get_last_lr()</code>.</p>
Efficiently Creating 4D Mask?

username: RylanSchaeffer
cooked: <p>I’d like to create a 4D mask where the elements are determined by $M[i, j, k, l] = \delta_{i+j, k+l}$. Does anyone have a recommendation for how to do this?</p>

username: Geeks_Sid
cooked: <p>Can you please rewrite the equation? It’s hard to understand.</p>

username: RylanSchaeffer
cooked: <p>Sure. <code>M[i, j, k, l] = 1</code> if <code>i + j == k + l</code> and 0 otherwise.</p>
Discriminator in GAN determines real and fake with prob 0.5( BCE loss log(0.5)= 0.69), what can I do to imrpove discriminator?

username: anindyasdas
cooked: <p>[1/600][182]<br>
Loss_D: 1.42 Loss_G: 0.74 Time: 319.68s</p>
<p>[2/600][182]<br>
Loss_D: 1.40 Loss_G: 0.70 Time: 317.61s</p>
<p>[3/600][182]<br>
Loss_D: 1.40 Loss_G: 0.69 Time: 318.28s</p>
<p>[4/600][182]<br>
Loss_D: 1.39 Loss_G: 0.69 Time: 319.15s</p>
<p>[5/600][182]<br>
Loss_D: 1.39 Loss_G: 0.69 Time: 319.27s</p>
<p>[6/600][182]<br>
Loss_D: 1.39 Loss_G: 0.69 Time: 319.53s</p>
<p>[7/600][182]<br>
Loss_D: 1.39 Loss_G: 0.69 Time: 319.34s</p>
<p>[8/600][182]<br>
Loss_D: 1.39 Loss_G: 0.69 Time: 318.61s</p>
<p>[9/600][182]<br>
Loss_D: 1.38 Loss_G: 0.70 Time: 319.03s</p>
<p>[10/600][182]<br>
Loss_D: 1.38 Loss_G: 0.69 Time: 319.61s</p>
<p>I am mapping 1024 dimentional vector into a 100-d vector</p>
<p>class MAP_NET_IT(nn.Module):<br>
<span class="hashtag">#Generator</span> from Image Embedding to text embedding####<br>
def <strong>init</strong>(self):<br>
super(MAP_NET_IT, self).<strong>init</strong>()<br>
<span class="hashtag">#self</span>.df_dim = cfg.TEXT.DIMENSION<br>
self.ef_dim = cfg.TEXT.EMBEDDING_DIM<br>
self.in_dim = cfg.GAN.Z_DIM + cfg.TEXT.DIMENSION<br>
self.define_module()</p>
<pre><code>def define_module(self):
    ndf = self.in_dim
    efg = self.ef_dim
    

    self.fc1 = nn.Sequential(
        nn.Linear(ndf, ndf//2),
        #nn.BatchNorm1d(ndf//2),
        #Swish(),
        nn.LeakyReLU(0.2, inplace=True),
        nn.Linear(ndf//2, ndf//4),
        #nn.BatchNorm1d(ndf//4),
        #Swish(),
        #nn.Linear(ndf//4, ndf//8),
        #nn.BatchNorm1d(ndf//8),
        #Swish())
        nn.LeakyReLU(0.2, inplace=True))
    self.fc2 = nn.Sequential(
            nn.Linear(ndf//4, efg),
            #nn.Linear(ndf//8, efg),
            nn.Tanh())
            

def forward(self, z_code, c_code):
    in_code = torch.cat((c_code, z_code), 1)
    x_code = self.fc1(in_code)
    output = self.fc2(x_code)
    
    return output
</code></pre>
<p>class D_NET_TEXT(nn.Module):<br>
<span class="hashtag">#To</span> discriminate for text embedding######################<span class="hashtag">#3</span><br>
def <strong>init</strong>(self):<br>
super(D_NET_TEXT, self).<strong>init</strong>()<br>
self.df_dim = cfg.TEXT.EMBEDDING_DIM<br>
self.define_module()</p>
<pre><code>def define_module(self):
    ndf = self.df_dim

    self.encodings = nn.Sequential(
        nn.Linear(ndf, ndf//2),
        nn.BatchNorm1d(ndf//2),
        Swish(),
        nn.Linear(ndf//2, ndf//4),
        nn.BatchNorm1d(ndf//4),
        Swish(),
        nn.Linear(ndf//4, ndf//8),
        nn.BatchNorm1d(ndf//8),
        Swish())
    self.logits = nn.Sequential(
            nn.Linear(ndf//8, 1),
            nn.Sigmoid())
            

def forward(self, x_var):
    x_code = self.encodings(x_var)
    output = self.logits(x_code)
    
    return [output.view(-1)]</code></pre>
How to fix in-place operation in recurrent cycle Pytorch

username: 111273
cooked: <p><strong>So, when I’m calling</strong></p>
<pre><code class="lang-auto">for data, label in train_loader:  
      a, cutoff = model(torch.t(data),R,1)
      loss = (cutoff-label)**2
      loss.backward()   
</code></pre>
<p><strong>I have this</strong></p>
<pre><code class="lang-auto">RuntimeError: one of the variables needed for gradient computation 
has been modified by an inplace operation: [torch.FloatTensor [1, 1]], 
which is output 0 of SelectBackward, is at version 51; 
expected version 50 instead. Hint: the backtrace further above shows 
the operation that failed to compute its gradient. The variable 
in question was changed in there or anywhere later. Good luck!   
</code></pre>
<p><strong>There is my Net forward function</strong></p>
<pre><code class="lang-auto">def forward(self, x,R,L):
          a = torch.zeros(R,L,1)
          for cycle in range (R):
           if cycle ==0:  
               a[0]=(self.w*torch.mm(self.wih,x))

           a[cycle]= relu(torch.mm(self.whh,a[cycle-1])+ (self.w*torch.mm(self.wih,x)))            
          cutoff =(sqr(torch.norm(torch.mm(torch.t(self.wih),a[R-1])))/sqr(torch.norm(x)))
          return a, cutoff
</code></pre>
<p><strong>And problem is in</strong></p>
<pre><code class="lang-auto">a[cycle] = .....a[cycle-1]....
</code></pre>
<p>And that <code>cycle</code> is changing<br>
<strong>How can I fix this in-place operation or cycle?</strong></p>
<p>There is Traceback:</p>
<pre><code class="lang-auto">Traceback (most recent call last):

  File "C:\Users\A877\untitled2.py", line 69, in &lt;module&gt;
    loss.backward()

  File "C:\Users\A877\.conda\envs\test\lib\site-packages\torch\tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)

  File "C:\Users\A877\.conda\envs\test\lib\site-packages\torch\autograd\__init__.py", line 97, in backward
    Variable._execution_engine.run_backward(
</code></pre>

username: googlebot
cooked: <p>You can change a to a list, append Lx1 tensors to it, and use torch.stack after the loop. Also, if self.w is scalar, your loop can be replaced with nn.RNNCell applied to rescaled x. (if it is a trainable vector, it is too tied to W_ih to be useful, I think).</p>