Implementing custom loss function in pytorch with weights

ailneni_rao_19210068: <p>This is the loss function i want to implement:</p>
<blockquote>
<p>def corr_basic_calc(weights, y_pred):</p>
<pre><code>num_classes = len(weights[:,0])

m_c = []

sum_m_c = []

for i in range(num_classes):

    m_c.append(y_pred[:,i])

    sum_m_c.append(torch.sum(m_c[i]))

return num_classes, torch.tensor(m_c,dtype=torch.long), torch.tensor(sum_m_c,dtype=torch.long)
</code></pre>
</blockquote>
<blockquote>
<p>Blockquote</p>
</blockquote>
<p>def loss_fn(outputs,targets,weights,uncorrelated_c_pairs):</p>
<blockquote>
<p>num_classes, m_c, sum_m_c = corr_basic_calc(weights,outputs)</p>
<p>corr_loss=0</p>
<p>for uncorr_c_pair in uncorrelated_c_pairs:</p>
<pre><code>c1, c2, v = uncorr_c_pair

corr_loss += torch.abs(torch.sum(m_c[c1]*m_c[c2])/sum_m_c[c1]-v)
</code></pre>
<p>return torch.mean(((weights[:,0]*(1-targets))+(weights[:,1]*targets))<em>torch.nn.BCEWithLogitsLoss()(outputs, targets),axis=1)+0.05</em>corr_loss/len(uncorrelated_c_pairs)</p>
</blockquote>
<p>But m getting this error:<br>
 in corr_basic_calc(weights, y_pred)<br>
6         m_c.append(y_pred[:,i])<br>
7         sum_m_c.append(torch.sum(m_c[i]))<br>
----&gt; 8     return num_classes, torch.tensor(m_c,dtype=torch.long), torch.tensor(sum_m_c,dtype=torch.long)</p>
<p>ValueError: only one element tensors can be converted to Python scalars</p>
<p>Can you help me solve it?</p>

mMagmer: <pre><code class="lang-auto">def corr_basic_calc(weights, y_pred):
    num_classes = len(weights[:,0])
    m_c = []
    sum_m_c = []
    for i in range(num_classes):
        m_c.append(y_pred[:,i])
        sum_m_c.append(torch.sum(m_c[i]))

    return num_classes, torch.tensor(m_c,dtype=torch.long), torch.tensor(sum_m_c,dtype=torch.long)
</code></pre>
<p>check out <a href="https://pytorch.org/docs/stable/generated/torch.tensor.html#torch.tensor" rel="noopener nofollow ugc">this</a>.</p>
<blockquote>
<p><code>torch.</code> <code>tensor</code> (<em>data</em>, *, <em>dtype=None</em>, <em>device=None</em>, <em>requires_grad=False</em>, <em>pin_memory=False</em> ) → <a href="https://pytorch.org/docs/stable/tensors.html#torch.Tensor" rel="noopener nofollow ugc">Tensor</a></p>
<p>Constructs a tensor with <code>data</code>.</p>
<p>Parameters</p>
<p><strong>data</strong> (<em>array_like</em>) – Initial data for the tensor. Can be a list, tuple, NumPy <code>ndarray</code>, scalar, and other types.</p>
</blockquote>
<p>you’re using list of tensors as data. it’s <strong>incompatible</strong><br>
maybe you can use <code>torch.stack</code> or <code>torch.cat</code> instead.</p>

ailneni_rao_19210068: <p>I tried using torch.stack but instead of getting a single element loss, i got this:</p>
<blockquote>
<p>tensor([0.9472, 0.6654, 0.6768, 0.4873, 0.4726, 0.4873, 0.4873, 0.6212, 0.6212,</p>
<pre><code>    0.5999, 0.6359, 0.5999, 0.7993, 0.4873, 0.6212, 0.6768],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9482, 0.6664, 0.6778, 0.4883, 0.4736, 0.4883, 0.4883, 0.6223, 0.6223,</p>
<pre><code>    0.6010, 0.6370, 0.6010, 0.8003, 0.4883, 0.6223, 0.6778],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9484, 0.6666, 0.6780, 0.4886, 0.4738, 0.4886, 0.4886, 0.6225, 0.6225,</p>
<pre><code>    0.6012, 0.6372, 0.6012, 0.8006, 0.4886, 0.6225, 0.6780],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9498, 0.6680, 0.6794, 0.4899, 0.4752, 0.4899, 0.4899, 0.6239, 0.6239,</p>
<pre><code>    0.6026, 0.6386, 0.6026, 0.8020, 0.4899, 0.6239, 0.6794],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9501, 0.6683, 0.6797, 0.4902, 0.4754, 0.4902, 0.4902, 0.6241, 0.6241,</p>
<pre><code>    0.6028, 0.6388, 0.6028, 0.8022, 0.4902, 0.6241, 0.6797],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9511, 0.6693, 0.6807, 0.4912, 0.4765, 0.4912, 0.4912, 0.6251, 0.6251,</p>
<pre><code>    0.6038, 0.6399, 0.6038, 0.8032, 0.4912, 0.6251, 0.6807],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>tensor([0.9524, 0.6706, 0.6820, 0.4925, 0.4778, 0.4925, 0.4925, 0.6265, 0.6265,</p>
<pre><code>    0.6052, 0.6412, 0.6052, 0.8045, 0.4925, 0.6265, 0.6820],

   device='cuda:0', dtype=torch.float64, grad_fn=&lt;AddBackward0&gt;
</code></pre>
</blockquote>
<p>can you help me figure out what I’m doing wrong? aI probably need to correct the last line in my loss function.</p>

mMagmer: <p>i really don’t get what you want to do.<br>
what is <code>uncorrelated_c_pairs</code>?</p>
<p>also , i think the below  part will riase an error OR it’s really hard to read</p>
<aside class="quote no-group" data-username="ailneni_rao_19210068" data-post="1" data-topic="140615">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ailneni_rao_19210068/48/45390_2.png" class="avatar"> ailneni_rao_19210068:</div>
<blockquote>
<p>return torch.mean(((weights[:,0]*(1-targets))+(weights[:,1]*targets))<em>torch.nn.BCEWithLogitsLoss()(outputs, targets),axis=1)+0.05</em>corr_loss/len(uncorrelated_c_pairs)</p>
</blockquote>
</aside>
<p>what is mathematical expression for the loss?</p>

mMagmer: <p>BTW, if you’re using <code>torch.max</code> for <code>y_pred</code> , you can’t backward through it.</p>

ailneni_rao_19210068: <p>It is a weighted binary cross entropy loss + label non-co-occurrence loss. weights and uncorrelated pairs are calculated beforehand and passed to the loss function.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/d/6/d6bf6384cd362201516ef928010515a8c38a0732.jpeg" alt="loss" data-base62-sha1="uDKe5fjwJEpqymlyXL6u9bZnSGS" width="612" height="78"></p>
<p>This is the loss function.<br>
First compute the set of uncorrelated pairs (as per the training data); Su = {i, j | M(i, j) = 0, i &lt; j, 1 ≤ i, j ≤ q}. For each (i, j) ∈ Su, we sum the model-based conditional co-occurrence probabilities Pˆ(j|i)<br>
and Pˆ(i|j) here</p>

mMagmer: <p>i not sure, but maybe this work.</p>
<pre><code class="lang-auto">def corr_loss(weights, y_pred,uncorrelated_c_pairs):
    num_classes = len(weights[:,0])
   
    loss=0
    for uncorr_c_pair in uncorrelated_c_pairs:
          c1, c2, v = uncorr_c_pair
          loss += torch.sum(y_pred[:,c1]*y_pred[:,c2])*((y_pred[:,c1].sum()+y_pred[:,c2].sum()))/(y_pred[:,c1].sum()*y_pred[:,c2].sum())
 
    return loss
</code></pre>