akankshakshurai/cuda-error-resolution-analysis huggingface dataset:
Merging 2 Neural Networks

username: sharjeel
cooked: <p>I have 2 trained models one is for Image Segmentation and one for Image localization.<br>
The problem here is that i want to Segment a few classes and localize the others ( on the same Input Image).<br>
I have tried to run both of those Networks on the same image and it does the job, but it is extremely slow.<br>
So i was wondering if there is a way to merge the 2 networks, so that it would then give me 2 outputs the segmentation mask and the localization coordinates.</p>
<p>I was planning to use the Encoder Decoder network and freezing the weights of the encoder while training the decoder to give to 2 outputs at 3 scales.</p>
<p>But i am not sure if it will work, i need suggestions and help…</p>

username: ptrblck
cooked: <p>What kind of CNNs are you using at the moment and how do these architectures relate to your encoder-decoder model?</p>
<p>Do your current models share some layers (e.g. is the base of the model “similar”)?<br>
If so, you could try to retrain a new model by keeping the common architecture and use two different heads for each task.</p>

username: sharjeel
cooked: <p>I am using a U-Net for image segmentaion and Yolo-V3 for localization, they don’t share any layer yet.</p>
<p>I was planning to edit the U-Net by adding just the yolo layer at the end along with the segmentation layers and then train the decoder in the U-Net, but not sure if it’ll work</p>
Grammatical problems with nn.Sequential

username: ky_Pa
cooked: <p>Hi,<br>
What is the syntax error here? How should I modify it?<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/e/8/e858f86242c675f024e6bbbd013d2096c7640230.png" alt="微信图片_20200624141203" data-base62-sha1="x9rof623AxUWH85x3JoDbyxIw00" width="642" height="195"></p>
<pre><code class="lang-auto">class Classifier(nn.Module):
    def __init__(self, input_nc):
        super(Classifier, self).__init__()
       
        self.classifier = nn.Sequential(
            # batch_size x input_nc x 256 x 256
            nn.Conv2d(input_nc, 32, kernel_size=5, stride=2, padding=2),
            nn.ReLU(True),
            # batch_size x 32 x 128 x 128
            nn.MaxPool2d(kernel_size=2, stride=2, padding=0)
            # batch_size x 32 x 64 x 64
            nn.Conv2d(32, 64, kernel_size=5, stride=2, padding=2),
            nn.ReLU(True),
            # batch_size x 64 x 32 x 32
            nn.MaxPool2d(kernel_size=2, stride=2, padding=0),
            # batch_size x 64 x 16 x 16
            nn.Conv2d(64, 128, kernel_size=5, stride=2, padding=2),
            nn.ReLU(True),
            # batch_size x 128 x 8 x 8
            nn.MaxPool2d(kernel_size=2, stride=2, padding=0),
            # batch_size x 128 x 4 x 4
            nn.Linear(2048, 512),
            nn.ReLU(True),
            # batch_size x 512
            nn.Linear(512, 128),
            nn.ReLU(True),
            # batch_size x 128
            nn.Linear(128, 3)
        )</code></pre>

username: harsha_g
cooked: <aside class="quote no-group" data-username="ky_Pa" data-post="1" data-topic="86669">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/k/eada6e/48.png" class="avatar"> ky_Pa:</div>
<blockquote>
<p><code> nn.MaxPool2d(kernel_size=2, stride=2, padding=0)</code></p>
</blockquote>
</aside>
<p>missing a comma <code>,</code> after <code>nn.MaxPool2d(kernel_size=2, stride=2, padding=0)</code></p>

username: ky_Pa
cooked: <p>Thank! I neglected it.</p>
How to get KL-Divergence?

username: oasjd7
cooked: <pre><code class="lang-auto">import torch.nn.functional as F
import torch.nn as nn

class Network(nn.Module):
    def __init__(self, channel, classes=10):
        super(Network, self).__init__()
        self.fc = nn.Linear(channel, classes)
    def forward(self, x):
        x = self.fc(x)
        return x

net = Network()
out = net(input)
# out.shape == [16,10] 

Q = out
P = out
print((P * (P / Q).log()).sum()) # return 0
print(F.kl_div(Q, P, None, None, 'sum')) # return negative value.

</code></pre>
<p>I heard<br>
<code>(P * (P / Q).log()).sum()</code><br>
is equal to<br>
<code>F.kl_div(Q.log(), P, None, None, 'sum')</code>.</p>
<p>With above my test code, <code>(P * (P / Q).log()).sum()</code>  returns 0 when Q and P are the same, but<code> F.kl_div</code> returns negative values.</p>
<p>What`s the correct way to get KL-Divergence ?</p>

username: ptrblck
cooked: <p>Based on the <a href="https://pytorch.org/docs/stable/nn.html?highlight=kldivloss#torch.nn.KLDivLoss">docs</a>, the first argument should contain the log probabilities, while the second one the probabilities:</p>
<blockquote>
<p>As with <a href="https://pytorch.org/docs/stable/nn.html?highlight=kldivloss#torch.nn.NLLLoss"> <code>NLLLoss</code> </a>, the input given is expected to contain  <em>log-probabilities</em>  and is not restricted to a 2D Tensor. The targets are given as  <em>probabilities</em>  (i.e. without taking the logarithm).</p>
</blockquote>
<p>while <code>Q</code> and <code>P</code> seem to contain logits in your use case.</p>
Tensorboard stops showing embeddings

username: Vitalik
cooked: <p>Hi, just yesterday everything was all right, but today launching tensorboard do my computer too slow, so I reboot it and after it, tensorboard stops showing the embeddings, writing:</p>
<pre><code class="lang-auto">No checkpoint was found.
Probable causes:

No checkpoint has been saved yet. Please refresh the page periodically.
You are not saving any checkpoint. To save your model, create a tf.train.Saver and save your model periodically by calling saver.save(session, LOG_DIR/model.ckpt, step).
</code></pre>
<p>So, what can be the reason for it, and how to fix it?</p>
<p>(the logdir folder isn’t empty)</p>

username: Vitalik
cooked: <p>So, it was fixed by itself and I don’t know exactly the reason for this bug. What I did - simply add a comment while trying to write vectors one more time and as a result when I refreshed the tab with tensorboard it found new embeddings and later written embeddings.</p>
Merge different channels into one

username: pilmus
cooked: <p>Hi all! I am following <a href="https://colab.research.google.com/github/pytorch/vision/blob/temp-tutorial/tutorials/torchvision_finetuning_instance_segmentation.ipynb" rel="nofollow noopener">this</a> tutorial on finetuning an object detector. As one of the steps, I split my mask image into separate masks and filter out masks corresponding to unwanted labels. Afterwards, I would like to merge the masks back into one channel and display it as a PIL image. The problem is: I cannot for the life of me figure out how to do this!</p>
<p>I’ve tried <code>torch.cat</code>, but this increases the size of the image in one dimension. E.g.: <code>torch.cat(t, dim=1)</code>, where <code>t.shape == (2,640,230)</code> creates a tensor of size <code>(640,460)</code>, while what I want is a tensor of size <code>(640,230)</code> with the values of the respective channels in the correct positions. The masks do not overlap at all, so I feel like this should be possible.</p>
<p>Any help is much appreciated, and if there are questions please let me know!</p>

username: ptrblck
cooked: <p>If each mask has no channel dimension and contains the current class index, you could simply add the masks.<br>
Assuming the background class has the class index 0 and no other classes overlap.</p>

username: pilmus
cooked: <p>Thanks, this is what I initially wanted! However, it turns out the classes overlap after all <img src="https://discuss.pytorch.org/images/emoji/apple/sweat.png?v=9" title=":sweat:" class="emoji" alt=":sweat:"> Is there another (standard?) way of merging the masks back into one image, I guess giving preference to masks with a higher confidence score?</p>

username: ptrblck
cooked: <p>How would be the confidence of the mask be defined?<br>
If you just want to keep the largest class index, you could use <code>torch.max</code>:</p>
<pre><code class="lang-python">a = torch.randint(0, 4, (2, 4, 4))
b = torch.randint(0, 4, (2, 4, 4))
c = torch.max(a, b)
</code></pre>
About nn.KLDivLoss() input

username: oasjd7
cooked: <p>Hi, all.<br>
As far as I know, 1st parameter in <code>nn.KLDivLoss() </code> need to be <code>F.log_softmax()</code>.<br>
What about the 2nd parameter?<br>
(In my case, there is no softmax in classifier.)</p>
<pre><code class="lang-auto">class Classifier(nn.Module):
    def __init__(self, channel, classes=10):
        super(Classifier, self).__init__()
        self.fc = nn.Linear(channel, classes)

    def forward(self, x):
        x = self.fc(x)
        return x
</code></pre>
<pre><code class="lang-auto">pred1 = classifier(input)
pred2 = classifier(input)
1) kld_loss = nn.KLDivLoss()(F.log_softmax(pred1, dim=1), pred2)
2) kld_loss = nn.KLDivLoss()(F.log_softmax(pred1, dim=1), F.softmax(pred2, dim=1))
</code></pre>

username: ptrblck
cooked: <p>Based on the <a href="https://pytorch.org/docs/stable/nn.html?highlight=kldivloss#torch.nn.KLDivLoss">docs</a>, the second argument should be passed as probabilities:</p>
<blockquote>
<p>As with <a href="https://pytorch.org/docs/stable/nn.html?highlight=kldivloss#torch.nn.NLLLoss"> <code>NLLLoss</code> </a>, the input given is expected to contain  <em>log-probabilities</em>  and is not restricted to a 2D Tensor. The targets are given as  <em>probabilities</em>  (i.e. without taking the logarithm).</p>
</blockquote>
Please take a look at these two pictures. Is my model normal?

username: Foolish
cooked: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/9/c/9c94a353ae5e8038973b1eb8b184c0cdfce5e6a1.png" alt="图片2" data-base62-sha1="mlaWkxptm4I2XzEhHGpS9hFnZ73" width="392" height="262"> <img src="https://discuss.pytorch.org/uploads/default/original/3X/5/e/5e8afc752595c1410c676ada35d1934acd5bf248.png" alt="图片1" data-base62-sha1="dumEQ39cSktZ6hv7qtIOoMoRKvS" width="389" height="266"></p>

username: Foolish
cooked: <p>Does this model have overfitting？</p>

username: BTNug
cooked: <p>Hi <a class="mention" href="/u/foolish">@Foolish</a>,<br>
It is totally normal. The only thing that might concern is “why is the loss suddenly drop so much?” , isn’t it?</p>
<p><strong>Some suggestion if you want to get a better training loss curve</strong><br>
The reason is because you set the learning rate too high. If you want a better loss curve, you might want to take a lower learning rate.</p>
<p>i.e.: your current learning rate: 0.001, try to reduce it by a factor of 10 or 100 -&gt; 0.0001 or 0.00001. Then you will see a better loss curve.</p>
<p>Hope it helps, cheers~</p>

username: Foolish
cooked: <p>Thank you for your suggestions, I will optimize again.</p>

username: Rahul_pillai
cooked: <p>is it bad if the loss suddenly drop?. The final loss is near zero ,so why does it matter?.</p>
The probem with pydocs

username: km_proton
cooked: <p>the docs in this site are not userfriendly . you can better first give syntax and then give the example .<br>
but in docs you have given only the example . moreover please give like tensor() is a class and torch is module (i mean detailed explanation of what is what and what are the directories and classes present in pytorch) . the pytorch community is very helpfull wasting their precious time for solving someone else’s problem . if you want new users to learn pytorch insted of tensorflow please provide better docs .</p>

username: mathematics
cooked: <p>Yeah, you are right, Pytorch documentation as compared to tensorflow , it is not wisely and not maintained.<br>
We want it better.</p>
<p>I know some html/css, i would do little help for making ui better.</p>

username: km_proton
cooked: <p>i know some pytorch stuff i can spend my time for helping document pytorch</p>

username: mathematics
cooked: <p>we need to go deeper.</p>

username: ptrblck
cooked: <p>If you have certain feature requests or proposals, please create an <a href="https://github.com/pytorch/pytorch/issues">issue</a> and discuss it there.</p>
<p>CC <a class="mention" href="/u/km_proton">@km_proton</a></p>

username: km_proton
cooked: <p>no <a class="mention" href="/u/ptrblck">@ptrblck</a> thats not what i meant its hard for new beginners to grasp</p>

username: ptrblck
cooked: <p>I understood that you have certain ideas and proposals on how to improve the documentation.<br>
If that’s the case, we would like to hear them (best in a feature request) and would be more than happy to discuss them there and accept PRs for improvement. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
How to load 2d array

username: XiaoChen1992
cooked: <p>Hi, I am new here.</p>
<p>I have two folders. A and B<br>
In A flole, the file structure like following:</p>
<ol>
<li>txt<br>
2.txt<br>
3.txt<br>
…<br>
each txt file is a 2d array.</li>
</ol>
<p>In B folder,  the file structure like:</p>
<ol>
<li>txt<br>
2.txt<br>
3.txt<br>
…<br>
each file is a score (single number) for A folder’ array.</li>
</ol>
<p>A folder is too large to load in memory.<br>
Do you know how to load this dataset?</p>
<p>Thanks</p>

username: Scott_Hoang
cooked: <p>Are A and B paired data?</p>

username: XiaoChen1992
cooked: <p>They are paired.  I solved this problem.</p>
<p>Thanks</p>
Neural Net for text retrieval

username: GabrielGhe
cooked: <p>I’m wondering how one would go about creating a neural net to retrieve text from a document.</p>
<p>A practical example would be for web crawling. Let’s say we want to track real estate listings across multiple websites. We don’t want to have 10+ different crawlers for each site as that’s not scalable so we would want to train a neural net that can adapt to different sites.</p>
<p>I was thinking that the same way CNNs use a sliding grid to cycle through a 2d image, perhaps it could also go through an extremely long 1d embedded array of HTML and we can put bounding boxes around certain HTML elements.</p>
<p>Full disclosure: I’m a newbie</p>

username: vdw
cooked: <p>I’m not really sure what your trying to do, but for your example I would argue that writing 10+ crawlers/scrapers is a perfectly good solutions.</p>
<p>Training deep networks such as CNNs require large amount of data to get good results. Where do you get that training data from? In principle what you’re do is sequence labeling, with each token/word/tag/etc. in your HTML string is associated with a label (similar to named entity recognition). But again, this is arguably not an easy task to learn.</p>
Bespoke Cross Entropy Loss

username: Tank
cooked: <p>Is it possible to use cross entropy loss where some mistakes are penalized less. E.g. lets us assume we have four categories horse, zebra, whale, shark. If the network chooses a horse instead of zebra the error should be smaller than if it chooses a whale instead of a zebra and so forth.</p>

username: KFrank
cooked: <p>Hello Tank!</p>
<aside class="quote no-group" data-username="Tank" data-post="1" data-topic="86622" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/tank/48/12334_2.png" class="avatar"> Tank:</div>
<blockquote>
<p>Is it possible to use cross entropy loss where some mistakes are penalized less. E.g. lets us assume we have four categories horse, zebra, whale, shark. If the network chooses a horse instead of zebra the error should be smaller than if it chooses a whale instead of a zebra and so forth.</p>
</blockquote>
</aside>
<p>Yes, you can certainly do something like this, and it is a reasonable<br>
thing to consider.</p>
<p><em>One</em> way to do it is as follows:</p>
<p>First, remember that <code>CrossEntropyLoss</code>, as implemented in pytorch,<br>
is a special case of cross entropy.  Cross entropy is a measure of the<br>
mismatch between two probability distributions – your predicted<br>
distribution and your target (known, “ground truth”) distribution.</p>
<p>Your predicted distribution is a set of class probabilities that sum to 1.<br>
But for pytorch’s <code>CrossEntropyLoss</code>, your <em>target</em> is not really a full<br>
probability distribution – it is a single integer class label.  That is, it<br>
represents one class having known probability of 1, and the other<br>
classes having probability 0.</p>
<p>What you could do is write your own cross-entropy loss that takes a<br>
full set of class probabilities as its targets.  You could, then, for example,<br>
assign your target probabilities as follows:</p>
<pre><code class="lang-plaintext">P = [horse, zebra, whale, shark]
class label = horse:
   P = [0.8, 0.2, 0.0, 0.0]
class label = zebra:
   P = [0.3, 0.7, 0.0, 0.0]
class label = whale:
   P = [0.0, 0.0, 0.9, 0.1]
class label = shark:
   P = [0.0, 0.0, 0.1, 0.9]
</code></pre>
<p>This would be instead of the “pure” single-class probabilities that<br>
<code>CrossEntropyLoss</code> uses implicitly:</p>
<pre><code class="lang-plaintext">   P = [1.0, 0.0, 0.0, 0.0]
   P = [0.0, 1.0, 0.0, 0.0]
   P = [0.0, 0.0, 1.0, 0.0]
   P = [0.0, 0.0, 0.0, 1.0]
</code></pre>
<p>respectively.</p>
<p>Or you could do something like:</p>
<p>For each of the three wrong answers, you could calculate their pairwise<br>
cross entropies with the right answer, and sum them together with<br>
weights.  Thus, you could weight the horse-zebra misprediction cross<br>
entropy less, for example, than the whale-zebra misprediction cross<br>
entropy.</p>
<p>(As a matter of nomenclature, I probably wouldn’t call this second<br>
approach cross entropy, but it’s related, and might do what you want.)</p>
<p>Good luck.</p>
<p>K. Frank</p>
Train LSTM with no input and give only initial hidden state inititialization

username: Pratha
cooked: <p>Hi, I am trying to model a state model evolution  (beta_t = F(beta_{t-1}) + noise) using LSTM and then use its output and some other input (X)  in the linear layer above this LSTM.</p>
<p>Is there a way for me to not give LSTM any input and give it the initial estimate of the hidden state beta_0 so that my LSTM learns the beta evolution, using the MSE loss between my observation y and output of linear layer y_hat?</p>
<p>A bit confused here! Appreciate any help.</p>
Maxpool2d output size inconsistent with formula

username: MaazJamal
cooked: <p>According to the equation <a href="https://pytorch.org/docs/stable/nn.html#maxpool2d" rel="nofollow noopener">here</a> . The output from maxpool2d should be 24 in my case, but i am not getting that result. Am i missing something here?</p>
<pre><code class="lang-auto">class mnist_conv2d(nn.Module):
  
  def __init__(self,classes):
    super(mnist_conv2d,self).__init__()
    self.conv1 = nn.Conv2d(in_channels=1,out_channels=16,kernel_size=3) # 1 in channel as the are greyscale
    self.conv2 = nn.Conv2d(in_channels=16,out_channels=32,kernel_size=3)
    self.drop = nn.Dropout2d()
    self.fc1 = nn.Linear(in_features=128,out_features=50) 
    self.fc2 = nn.Linear(in_features=50,out_features=classes)
  
  def forward(self, x):
    print("Original: ", x.size())
    x = self.conv1(x)
    print("Conv2d: ", x.size())
    #x size = (28+2*0-1*(3-1)-1)/1 + 1 = 26 
    x = F.max_pool2d(x,kernel_size=3)
    print("Maxpool: ", x.size())
    #x size = 26-(3-1)-1/1 + 1 = 24
    x = F.relu(x)
    print("Relu: ", x.size())
    x = self.conv2(x)
    print("Conv2d: ",x.size())
    #x size = 24-(3-1)-1/1 + 1 = 22
    x = self.drop(x)
    print("Dropout: ",x.size())
    x = F.max_pool2d(x,kernel_size=3)
    print("Maxpool: ",x.size())
    #x size = 24-(3-1)-1/1 + 1 = 20
    x = F.relu(x)
    print("Relu: ",x.size())
    x = x.view(x.shape[0],-1)
    print("Flattened: ",x.size()) 
    x = self.fc1(x)
    x = F.relu(x)
    x = self.fc2(x)
    x = F.log_softmax(x,dim=1)
    return x
</code></pre>
<p>Passing a 1,1,28,28 image or tensor to this model results in this output:</p>
<pre><code class="lang-auto">Original:  torch.Size([1, 1, 28, 28])
Conv2d:  torch.Size([1, 16, 26, 26])
Maxpool:  torch.Size([1, 16, 8, 8])
Relu:  torch.Size([1, 16, 8, 8])
Conv2d:  torch.Size([1, 32, 6, 6])
Dropout:  torch.Size([1, 32, 6, 6])
Maxpool:  torch.Size([1, 32, 2, 2])
Relu:  torch.Size([1, 32, 2, 2])
Flattened:  torch.Size([1, 128])
</code></pre>
<p>Shouldnt the output from the maxpool layers be of size 24x24 and not 8x8?</p>

username: MaazJamal
cooked: <p>Nevermind I missed this part from the docs:</p>
<blockquote>
<p><strong>stride</strong>  – the stride of the window. Default value is  <code>kernel_size</code></p>
</blockquote>
<p>So the equation is (26+2<em>0-1(3-1)-1)/3 + 1 = 8<br>
and not (26+2</em>0-1(3-1)-1)/1 + 1 = 24<br>
The stride in denominator is 3 not 1. As is the case for conv2d layers.</p>
Binary Image classification using Pytorch

username: shravan
cooked: <p>Hi All,</p>
<p>I am doing a binary image classification using pytorch and I am using CNN architecture to do that.</p>
<p>In the final layer of architecture - do I need to give number of outputs parameter as “2” or “1”?  I placed as 1 and My final target columns have values [0,1,0,0,0,…]</p>
<p>Here is my code for the final layer:</p>
<pre><code>        nn.Linear(128, 64),
        nn.ReLU(),
        nn.Linear(64, 1),
        nn.Sigmoid()
    )
</code></pre>
<p>Please clarify if it is correct. I am having some follow up questions on this.</p>
<p>–Shravan</p>

username: maudzung
cooked: <p>For binary classification, the number of units in the last fully-connected layer can be 1 or 2.</p>
<ul>
<li>If you use Sigmoid activation, you must apply <strong>1</strong> unit for the las FC layer (Your above code is correct). In this case, the loss function should be BCE.</li>
<li>If you use Softmax activation, you should apply <strong>2</strong> units. In this case, the loss function should be CE.</li>
</ul>

username: shravan
cooked: <p>I am getting -ve value predictions for the binary classification problem. Any idea where is the mistake? my train and validation is working fine and showing a decent 75% accuracy.</p>
<p><img src="https://discuss.pytorch.org/uploads/default/original/3X/a/3/a314b671e317fcfc7990d4f83455d417cf0391aa.png" alt="image" data-base62-sha1="ngGcFEK7WQc3EWJdymFBFnpgKx4" width="602" height="302"></p>
<p>Here is the prediction code:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/f/efb6cab293d23a5f06087021c55c0fba1e6b0485.png" data-download-href="https://discuss.pytorch.org/uploads/default/efb6cab293d23a5f06087021c55c0fba1e6b0485" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/efb6cab293d23a5f06087021c55c0fba1e6b0485_2_677x500.png" alt="image" data-base62-sha1="ycBKbUQl7p4fEk2KX1pMZNobGeN" width="677" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/efb6cab293d23a5f06087021c55c0fba1e6b0485_2_677x500.png, https://discuss.pytorch.org/uploads/default/original/3X/e/f/efb6cab293d23a5f06087021c55c0fba1e6b0485.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/f/efb6cab293d23a5f06087021c55c0fba1e6b0485.png 2x" data-dominant-color="F1F4F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">848×626 46.2 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>how to get the output in the format of 1/0?</p>
Pytorch install error

username: Sundar_Krishna
cooked: <blockquote>
<blockquote>
<blockquote>
<p>import torch<br>
Traceback (most recent call last):<br>
File “”, line 1, in <br>
File “D:\Python36\lib\site-packages\torch_<em>init</em>_.py”, line 81, in <br>
ctypes.CDLL(dll)<br>
File “D:\Python36\lib\ctypes_<em>init</em>_.py”, line 348, in <strong>init</strong><br>
self._handle = _dlopen(self._name, mode)<br>
OSError: [WinError 126] The specified module could not be found</p>
</blockquote>
</blockquote>
</blockquote>
<p>After installing pytorch I get these error please help!</p>

username: Sundar_Krishna
cooked: <p>I installed microsoft visual studio redistributable and problem solved</p>
Transforms: Resize vs CenterCrop

username: Wesley_Neill
cooked: <p>What are your preferences and the reasoning behind them?</p>
<p>I saw this transform in my Udacity class, and it seemed redundant in the use of both:</p>
<pre><code class="lang-auto">transform = transforms.Compose([transforms.Resize(255),
                                 transforms.CenterCrop(224),
                                 transforms.ToTensor()])
</code></pre>
<p>I was thinking <code>Resize</code> keeps the amount of information the same, but distorts it. It seems like <code>CenterCrop</code> risks cutting out important bits, but what it does keep isn’t overly distorted.</p>
<p>Just a newb question! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: Scott_Hoang
cooked: <p>it seems to me that whomever created this transform function is expecting non-uniform input image. the Resize functions is there only to ensure uniformity, and center crop reduce background noise.</p>

username: Wesley_Neill
cooked: <p>What do you mean non-uniform? As in the size is non-uniform (which is true), or something else?</p>
<p>What sort of background would one expect? This is the Cat and Dog dataset, by the way.</p>

username: Scott_Hoang
cooked: <p>non uniform as in the size of the image.<br>
And most likely the subject of interest in at the center of the image, hence the center crop.</p>
Types of layers for Quantization

username: Shani_Gamrian
cooked: <p>What types of layers are supported in PyTorch’s quantization framework? (especially ones that are more related to convnets).<br>
I found Conv2D, Conv3D, Relu, but I couldn’t find any types of BatchNorm)</p>

username: ptrblck
cooked: <p>I’ll move the category to Quantization to get a bit more visibility for this topic. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

username: jerryzh168
cooked: <p>you can checkout <a href="https://pytorch.org/docs/stable/quantization.html#operation-coverage" rel="nofollow noopener">https://pytorch.org/docs/stable/quantization.html#operation-coverage</a>, this might be a little bit out of date, to find the most up to date supported ops, you can take a look at: <a href="https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native/quantized/cpu" rel="nofollow noopener">https://github.com/pytorch/pytorch/tree/master/aten/src/ATen/native/quantized/cpu</a></p>
Why? use_gpu is False

username: milad66
cooked: <p>Hi, Every Body<br>
My Plat Form Is Jupyter Notebook And When I Use<br>
use_gpu It Return False.<br>
I Don’t Know How Can I Fix It (COnvert To True).<br>
Thanks.</p>

username: albanD
cooked: <p>Hi,</p>
<p>Could you share more context? <code>use_gpu</code> is not a pytorch construct but specific to the script you’re using so you will need to give more info.</p>

username: milad66
cooked: <p>Thanks<br>
I have jupyter + python I m working on deep learning so I want use GPU(for speed up processing) Nop CPU<br>
needs to gpu ………know how can I use gpu?</p>

username: harsha_g
cooked: <aside class="quote no-group" data-username="milad66" data-post="3" data-topic="86559">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/91b2a8/48.png" class="avatar"> milad66:</div>
<blockquote>
<p>needs to gpu ………know how can I use gpu?</p>
</blockquote>
</aside>
<p>Did you try <a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#training-on-gpu" rel="noopener nofollow ugc">this</a> part of the tutorial?</p>
What is the need of fusing layers in MobileNetv2?

username: Midhilesh
cooked: <p>I have seen the static quantization tutorial, where the layers are fused before itself. I got good result with fused layers but if I don’t fuse the layers, My accuracy is very poor.</p>
<p>What is the effect of layer fusion?</p>
<p>Please do help me with this.</p>

username: jerryzh168
cooked: <p>layer fusion is going to fuse Conv+BN into a Conv module or Conv + BN + ReLU into a ConvRelu module. this does not change numerics itself. Without fusion conv, bn and relu will be quantized independently, that might be the reason why the accuracy drops.</p>

username: Midhilesh
cooked: <p>But, what is the drawback of quantizing convolution, batchnorm, relu operations independently?</p>

username: jerryzh168
cooked: <p>quantizing them independently will have worse performance, and also may suffer from bigger accuracy loss.</p>
@torch.no_grad() vs with torch.no_grad()

username: Rahul_pillai
cooked: <p>What is the difference between <span class="mention">@torch.no_grad</span>() and the usual with torch.no_grad().<br>
Are both the same?,if it is when to use which?</p>

username: SANTOSH_S
cooked: <p>Hi <a class="mention" href="/u/rahul_pillai">@Rahul_pillai</a></p>
<p><code>with torch.no_grad():</code><br>
is used to locally turn of the gradient tracking.</p>
<p><code>@torch.no_grad()</code> is another option to turn off gradients and is used with a function. So anytime this function is called the its gradient tracking will offed locally within the execution of this function. Few examples can be found at <a href="https://pytorch.org/docs/stable/autograd.html" rel="nofollow noopener">link</a></p>

username: Rahul_pillai
cooked: <p>Thanks for answering <a class="mention" href="/u/santosh_s">@SANTOSH_S</a></p>
<pre><code class="lang-auto">@torch.no_grad()
def fn1():
    &lt;statements&gt;

def fn2():
    &lt;statements&gt;
</code></pre>
<p>Here will it turn off the gradients for fn1 alone or both fn1 and fn2.</p>

username: harsha_g
cooked: <p>Just function <code>fn1</code> alone.</p>

username: Rahul_pillai
cooked: <p>Thank you <a class="mention" href="/u/harsha_g">@harsha_g</a>  <img src="https://discuss.pytorch.org/images/emoji/apple/slightly_smiling_face.png?v=9" title=":slightly_smiling_face:" class="emoji" alt=":slightly_smiling_face:"></p>
What does next(self.parameters()).data mean?

username: wasiahmad
cooked: <p>I found the following piece of code in one of the example of pytorch. Its from an <code>EncoderRNN</code> class.</p>
<pre><code>def init_weights(self, bsz):
    """Initialize weight parameters for the encoder."""
    weight = next(self.parameters()).data
    num_directions = 2 if self.bidirectional else 1
    if self.rnn_type == 'LSTM':
        return (Variable(weight.new(self.n_layers * num_directions, bsz, self.hidden_size).zero_()),
                Variable(weight.new(self.n_layers * num_directions, bsz, self.hidden_size).zero_()))
    else:
        return Variable(weight.new(self.n_layers * num_directions, bsz, self.hidden_size).zero_())
</code></pre>
<p>I have two question.</p>
<ol>
<li>What is happening in this statement - <code>weight = next(self.parameters()).data</code> ?</li>
<li>What does <code>weight.new()</code> mean? Why we can’t create a Variable without using <code>weight.new()</code>? Are we somehow attaching the hidden variables to the model (encoder) parameters?</li>
</ol>

username: Response777
cooked: <ol>
<li>
<code>next</code> retrieve the next item from the iterator by calling its <strong>next</strong>() method.
<ul>
<li>here, it returns the first parameter from the class.</li>
</ul>
</li>
<li>Well, you <strong>can</strong> create a Variable as you wish, <strong>but</strong> you need to specify the data type under such circumstance. By <code>new</code>, we constructs a new tensor of the same data type(as the first parameter).</li>
</ol>

username: manal_elaidouni
cooked: <p><code>self.parameters()</code>  is a generator method that iterates over the parameters of the model.<br>
So  <code>weight</code> variable simply holds a parameter of the model.<br>
Then <code>weight.new()</code> creates a tensor that has the same data type, same device as the produced parameter.</p>
<pre><code class="lang-auto">class Model(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(10,1)
        
model = Model()
weight = next(model.parameters()).data

print(weight)
# Output :  tensor([[ 0.0398,  0.0729, -0.2676,  0.2354, -0.0853,  0.1141,  0.0297,  0.0257, \ 
         -0.1303,  0.2208]])

print(type(weight), weight.device, weight.dtype, weight.requires_grad)
# Output : (torch.Tensor, device(type='cpu'), torch.float32, False)

var = weight.new(4,5)

print(type(var), var.device, var.dtype,  var.requires_grad)
# Output : (torch.Tensor, device(type='cpu'), torch.float32, False)

</code></pre>
From torch.utils.tensorboard import SummaryWriter

username: lixin_zju
cooked: <p>Hello,<img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
from torch.utils.tensorboard import SummaryWriter<br>
warning: Embedding dir exists, did you set global_step for add_embedding()?</p>
<p>Spyder(Python3.7)<br>
torch.<strong>version</strong>= ‘1.5.0+cpu’</p>
<p>Thank you very much!</p>
PyTorch equivalent of TF MultivariateNormalDiag?

username: Sumit_Kumar
cooked: <p>What is the PyTorch equivalent of TensorFlow’s MultivariateNormalDiag distribution? Specifically, I have a B x N x D mean tensor and B x N x D variance tensor where B is batch size, N is number of data points, D is the dimension of each data point. I want to create a multi-variate normal distribution with diagonal covariance from these tensors. How can this be implemented ?</p>

username: Nikronic
cooked: <p>Hi,</p>
<p>Please see this issue <a href="https://github.com/pytorch/pytorch/pull/11178" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/11178</a></p>
<p>Bests</p>

username: kirk86
cooked: <p>Does that mean that</p>
<pre><code class="lang-auto">import tensorflow as tf
import torch
D = torch.distributions

tf.MultivariateNormalDiag(...) == D.Independent(D.Normal(...))
</code></pre>
<p>Also from computational efficiency perspective?</p>
K means implementation with Pytoch

username: alex_gilabert
cooked: <p>Hello.</p>
<p>I am trying to implement a k-means algorithm for a CNN that first of all calculate the centroids of the k-means.</p>
<p>I have a tensor of dims [80,1000] with the features of one layer of the CNN.<br>
Then i randomly create a tensor of the same dims.</p>
<p>I calculate the euclidean dist. and take the minimum of this tensor.  Then, i assign the new centroids corresponding to the index of the tensor with the features.</p>
<p>My problem is that i want to loop this until the centroids dont change their position but i am not very sure how i could achieve so and how to go updating the values received for the centroids.</p>
<p>the code is the following:</p>
<pre><code class="lang-auto">def k_means_torch(dictionary, model):
    centroids = torch.randn(len(dictionary), 1000).cuda()
    dist_centroids = torch.cdist(dictionary,centroids, p=2.0)
    (values, indices) = torch.min(dist_centroids, dim=1)
    centroids_new = dictionary[indices]
    while 1:
      dist_centroids_loop = torch.cdist(dictionary,centroids_new, p=2.0)
      (values_, indices_) = torch.min(dist_centroids_loop, dim=1)

      if (torch.equal(centroids_new, dictionary[indices_])):
        break
      else:
        centroids_new = ((centroids_new + dictionary[indices_])/2)
</code></pre>
<p>Any ideas are more than welcome <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: alex_gilabert
cooked: <p>This code would resolve the precission problems.</p>
<pre><code class="lang-auto">a = torch.all(torch.lt(torch.abs(torch.add(centroids_new, -new_centers)), 1e-5))
</code></pre>
<p>This gets rid of the last digit of 1.0000000001 as we are taking into account Float values.</p>
Custom dataloader to load 2 tensordatasets simultaneously

username: chaslie
cooked: <p>Hi,</p>
<p>I would like to convert:</p>
<blockquote>
<p>dataloader_data1=torch.load(‘data1.pt’)<br>
dataloader_1 = torch.utils.data.DataLoader(dataloader_data1, batch_size=BATCHSIZE_G,<br>
shuffle=True, num_workers=workers)</p>
<p>dataloader_data2=torch.load(‘data2.pt’)<br>
dataloader_2 = torch.utils.data.DataLoader(dataloader_data2, batch_size=BATCHSIZE,<br>
shuffle=True, num_workers=workers)</p>
</blockquote>
<p>to allow me to load both tensordatasets into a single dataloader:</p>
<blockquote>
<p>class <strong>LoadDataset</strong>(Dataset):</p>
<pre><code>def __init__(self):
    self.data_1 = torch.load('data1.pt')
    self.data_2 = torch.load('data2.pt')


def __getitem__(self, index):
    return self.data_1, self.data_2

def __len__(self):
    return min(len(self.data_1), len(self.data_2))
</code></pre>
<p>temp_data=LoadDataset()<br>
dataloader = torch.utils.data.DataLoader(temp_data, batch_size=BATCHSIZE,<br>
shuffle=True, num_workers=workers)<br>
This goes some way as it creates the tuple, however when i try and unpack it using</p>
</blockquote>
<blockquote>
<p>for i, (data,quant) in enumerate(dataloader):</p>
</blockquote>
<p>I get:</p>
<p>TypeError: default_collate: batch must contain tensors, numpy arrays, numbers, dicts or lists; found &lt;class ‘torch.utils.data.dataset.TensorDataset’&gt;</p>
<p>I should add that data1 and data2 are of different lengths and also contain 3 fields each.</p>
<p>I would like to load the model with fields 1 &amp; 2 from the data 1 &amp; data 2 in the same loop.</p>
<p>can anyone help please?</p>
<p>chaslie</p>

username: chaslie
cooked: <p>SOLVED - FORGOT TO ADD index BEHIND SELF.DATA1 &amp; SELF.DATA2</p>
How to input the tensor from CNN to RNN

username: renzilin
cooked: <p>Hi,<br>
When I was dealing a model that is 1-d CNN + RNN, I found that the size of output from CNN is (batch_size, out_channels, features) where the features are time series.<br>
However, the RNN requires the size of input is (batch_size, time_step, “features”). So I can’t put the CNN output into RNN neural network directly. Because the “features” in RNN is out_channels in CNN and I need to reshape it.</p>
<p>So, is there any function can solve this format problem? With GPU, can I use a for loop to deal with it?</p>
<p>Thank you!</p>

username: ptrblck
cooked: <p>You could <code>permute</code> the activation and pass the right shape via:</p>
<pre><code class="lang-python">print(out.shape) # [batch_size, out_channels, seq_len]
out = out.permute(0, 2, 1) # [batch_size, seq_len, out_channels]
</code></pre>
<p>This would shift the temporal dimension to dim1 and the output channels to dim2.</p>

username: renzilin
cooked: <p>It works! Thank you very much, bro!</p>
Unable to view optimized graph while using torch.jit.script

username: Midhilesh
cooked: <p>I have currently started learning torchscript and tried to visualize the optimized graph but I am unsuccessful. The following is my code</p>
<pre><code class="lang-auto">@torch.jit.script
def cell_end(ingate, forgetgate, cellgate, outgate, cx):
    ingate = torch.sigmoid(ingate)
    forgetgate = torch.sigmoid(forgetgate)
    cellgate = torch.tanh(cellgate)
    outgate = torch.sigmoid(outgate)

    cy = (forgetgate * cx) + (ingate * cellgate)
    hy = outgate * torch.tanh(cy)

    return hy, cy
</code></pre>
<p>For the above function i got the following graph</p>
<pre><code class="lang-auto">graph(%ingate.1 : Tensor,
      %forgetgate.1 : Tensor,
      %cellgate.1 : Tensor,
      %outgate.1 : Tensor,
      %cx.1 : Tensor):
  %19 : int = prim::Constant[value=1]()
  %ingate.3 : Tensor = aten::sigmoid(%ingate.1) # &lt;ipython-input-2-8da29633480c&gt;:3:13
  %forgetgate.3 : Tensor = aten::sigmoid(%forgetgate.1) # &lt;ipython-input-2-8da29633480c&gt;:4:17
  %cellgate.3 : Tensor = aten::tanh(%cellgate.1) # &lt;ipython-input-2-8da29633480c&gt;:5:15
  %outgate.3 : Tensor = aten::sigmoid(%outgate.1) # &lt;ipython-input-2-8da29633480c&gt;:6:14
  %15 : Tensor = aten::mul(%forgetgate.3, %cx.1) # &lt;ipython-input-2-8da29633480c&gt;:8:10
  %18 : Tensor = aten::mul(%ingate.3, %cellgate.3) # &lt;ipython-input-2-8da29633480c&gt;:8:30
  %cy.1 : Tensor = aten::add(%15, %18, %19) # &lt;ipython-input-2-8da29633480c&gt;:8:10
  %23 : Tensor = aten::tanh(%cy.1) # &lt;ipython-input-2-8da29633480c&gt;:9:19
  %hy.1 : Tensor = aten::mul(%outgate.3, %23) # &lt;ipython-input-2-8da29633480c&gt;:9:9
  %27 : (Tensor, Tensor) = prim::TupleConstruct(%hy.1, %cy.1)
  return (%27)
</code></pre>
<p>Running with an input</p>
<pre><code class="lang-auto">inp = torch.randn(5, 10, 4)
cell_end(*inp)
</code></pre>
<p>Even after running the input over the graph I got the same graph</p>
<pre><code class="lang-auto">graph(%ingate.1 : Tensor,
      %forgetgate.1 : Tensor,
      %cellgate.1 : Tensor,
      %outgate.1 : Tensor,
      %cx.1 : Tensor):
  %5 : int = prim::Constant[value=1]()
  %ingate.3 : Tensor = aten::sigmoid(%ingate.1) # &lt;ipython-input-2-8da29633480c&gt;:3:13
  %forgetgate.3 : Tensor = aten::sigmoid(%forgetgate.1) # &lt;ipython-input-2-8da29633480c&gt;:4:17
  %cellgate.3 : Tensor = aten::tanh(%cellgate.1) # &lt;ipython-input-2-8da29633480c&gt;:5:15
  %outgate.3 : Tensor = aten::sigmoid(%outgate.1) # &lt;ipython-input-2-8da29633480c&gt;:6:14
  %10 : Tensor = aten::mul(%forgetgate.3, %cx.1) # &lt;ipython-input-2-8da29633480c&gt;:8:10
  %11 : Tensor = aten::mul(%ingate.3, %cellgate.3) # &lt;ipython-input-2-8da29633480c&gt;:8:30
  %cy.1 : Tensor = aten::add(%10, %11, %5) # &lt;ipython-input-2-8da29633480c&gt;:8:10
  %13 : Tensor = aten::tanh(%cy.1) # &lt;ipython-input-2-8da29633480c&gt;:9:19
  %hy.1 : Tensor = aten::mul(%outgate.3, %13) # &lt;ipython-input-2-8da29633480c&gt;:9:9
  %15 : (Tensor, Tensor) = prim::TupleConstruct(%hy.1, %cy.1)
  return (%15)
</code></pre>
<p>I don’t know the reason behind it, Anyone please help me with this</p>
DQN gives same output for any inputs

username: spadel
cooked: <p>So I have recently already created a topic, but the problem has been reduced to a more simple one.<br>
I have the problem, that my DQN after training always returns the same outputs for whatever input I give. The DQN looks as follows:</p>
<pre><code class="lang-auto">class DQN(nn.Module):
    def __init__(self, lr, input_dims, fc1_dims, fc2_dims, n_actions):
        super(DQN, self).__init__()
        self.input_dims = input_dims
        self.fc1_dims = fc1_dims
        self.fc2_dims = fc2_dims
        self.n_actions = n_actions
        self.fc1 = nn.Linear(*self.input_dims, self.fc1_dims)    # in Tutorial steht *self.input_dims
        self.fc2 = nn.Linear(self.fc1_dims, self.fc2_dims)
        self.out = nn.Linear(self.fc2_dims, self.n_actions)
        self.optimizer = optim.Adam(self.parameters(), lr=lr)
        self.loss = nn.MSELoss()
        self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
        self.to(self.device)

    def forward(self, observation):
        state = torch.Tensor(observation).to(self.device)
        x = F.relu(self.fc1(state))
        x = F.relu(self.fc2(x))
        actions = self.out(x)

        return actions
</code></pre>
<p>After training, if I type something like:</p>
<pre><code class="lang-auto">print(trained_brain.Q_eval.forward([1, 0, 0, -1, 0, 0, 0, 0, 0]))
print(trained_brain.Q_eval.forward([1, 1, 0, -1, 0, -1, 0, 0, 0]))
</code></pre>
<p>I get the results:</p>
<pre><code class="lang-auto">tensor([0.2661, 0.1787, 0.1648, 0.1722, 0.2262, 0.1747, 0.2077, 0.1326, 0.2635],
       grad_fn=&lt;AddBackward0&gt;)
tensor([0.2661, 0.1787, 0.1648, 0.1722, 0.2262, 0.1747, 0.2077, 0.1326, 0.2635],
       grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>despite the fact, the the inputs are different. I assume, that there might be a problem with the datatypes, but I cannot figure where.</p>
Torch.distributions.kl.kl_divergence( ) raises NotImplementedError

username: oasjd7
cooked: <p>I want to get KL-divergence from two softmax outputs.(in pytorch version is 1.1.0.)<br>
but  <code>torch.distributions.kl.kl_divergence( )</code> raises <code>NotImplementedError</code>.<br>
How to use this divergence?</p>

username: googlebot
cooked: <p>this function expects Distribution objects, i.e. you’d need to wrap tensors with Categorical(probs=p)</p>
Normalising a tensor

username: Chiara
cooked: <p>I have a tensor with shape [10000,1,1024]. 10000 is the number of samples I have. I have to normalise each sample and i have no problem finding the maximum value by which I should divide each sample. i have problems understanding how to divide the 1024 values in each sample by that value and obtain a new tensor with shape  [10000,1,1024] and normalised samples. Can anyone help?</p>

username: alex_gilabert
cooked: <p>I think this doc would help.</p>
<p>Tensor are treated in a element-wise way, so if you code</p>
<pre><code class="lang-auto">A = torch.torch.randn(10000, 1, 1024)

B = A / 255

print(B)
print(A)
</code></pre>
<p>Even though, there is some specific documentation about lib torchvision.transforms that include normalize<br>
<a href="https://pytorch.org/docs/stable/_modules/torchvision/transforms/transforms.html#Normalize" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/docs/stable/_modules/torchvision/transforms/transforms.html#Normalize</a></p>
<p>I hope this helps <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
How to transform a pytorch JIT pth model to caffe prototxt/caffemodel

username: arsen
cooked: <p>For example:<br>
I use the static quantization tutorial and generate a scripted quantized model.<br>
<a href="https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html" class="onebox" target="_blank" rel="nofollow noopener">https://pytorch.org/tutorials/advanced/static_quantization_tutorial.html</a></p>
<p>Since the quantized model is different from the float model owing to quantizing (combining conv+bn into conv and so on). So I can’t obtain the .py file descripting the quantized model after quantization. Therefore I would like to know how to generate caffe prototxt which draws the whole network?</p>

username: driazati
cooked: <p>Can you elaborate on your higher-level goals a little bit? We don’t have any direct way to produce a caffe2 model from a PyTorch model, but you can see a description of the compiled model like so</p>
<pre><code class="lang-python">model = torch.jit.load(model_file)
print(model.graph)
</code></pre>

username: Shani_Gamrian
cooked: <p>What about exporting to caffe2 in an indirect way? Is it possible to somehow use the scale/zero_point and get the same outputs as in PyTorch?</p>
Our custom loss does not converge with PyTorch

username: Filippo_Portera
cooked: <p>Here is the loss code:</p>
<pre><code>    def our_loss(output, target):
         global S
         global batchSize
         global _EPSILON

         y_pred = torch.clamp(output, _EPSILON, 1.0-_EPSILON)
         out = -(target * torch.log(y_pred) + (1.0 - target) * torch.log(1.0 - y_pred))
         out = torch.flatten(out)
         tmp = torch.mv(S.float() , out.float())
         loss = torch.dot(out.t(), tmp)
         return loss
</code></pre>
<p>where S is a RBF generated Gram matrix of the input patterns.</p>