Why does my loss value of autoencoder in PyTorch is negative?

Ryanz: <p>Hi, I am trying to implement <a href="https://www.kdd.org/kdd2016/papers/files/rfp0191-wangAemb.pdf" rel="noopener nofollow ugc">SDNE</a>, a algorithm uses deep auto encoder to map a graph to latent representation <code>d</code> dimension.</p>
<p>The idea is kind of simple, SDNE uses the adjacency matrix as input and output is embedding layer (also node embedding of graph). The loss function tries to minimize the reconstruction layer and input layer which defined $\hat{X}$ and $X$. To keeping node embedding <code>Y</code> (embedding layer) is close together like the original node in graph input, loss function continue minimizes $||y_i - y_j||*s_{ij}$ with $s_{ij}$ is weight of node $v_i$ and $v_j$, if not link $s_{ij} = 0$.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/9/e961c3d5e6164e85b535fb9f9b5476b7927f28c5.png" data-download-href="https://discuss.pytorch.org/uploads/default/e961c3d5e6164e85b535fb9f9b5476b7927f28c5" title="Screenshot from 2020-08-16 02-36-01"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e961c3d5e6164e85b535fb9f9b5476b7927f28c5_2_690x189.png" alt="Screenshot from 2020-08-16 02-36-01" data-base62-sha1="xiAI5yIbUpREUg9QfRqI1SdRPSZ" width="690" height="189" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e961c3d5e6164e85b535fb9f9b5476b7927f28c5_2_690x189.png, https://discuss.pytorch.org/uploads/default/original/3X/e/9/e961c3d5e6164e85b535fb9f9b5476b7927f28c5.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/9/e961c3d5e6164e85b535fb9f9b5476b7927f28c5.png 2x" data-dominant-color="F8F8F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2020-08-16 02-36-01</span><span class="informations">810×222 20.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>When I implement code in PyTorch, I got <strong>negative</strong> loss value although the loss function does not negative value.</p>
<p>This is my code:</p>
<pre data-code-wrap="python"><code class="lang-python">def _compute_loss(self, x, x_hat, y, L):
        def loss_1st(Y, L):
            Y_ = torch.matmul(torch.transpose(Y, 0, 1), L)
            return 2 * torch.trace(torch.matmul(Y_, Y))

        def loss_2nd(X_hat, X, beta):
            B = torch.ones_like(X).to(device)
            B[X != 0] = beta
            return torch.sum(torch.square((X_hat - X) * B))

        batch_size = x.shape[0]
        # TODO: check if divide batch_size
        loss_1 = loss_1st(y, L)
        loss_2 = loss_2nd(x_hat, x, self.beta)
        loss = loss_2 + self.alpha * loss_1
        return loss
</code></pre>
<p>I trained on all batch dataset which means <code>batch_size = len(Adjancency_matrix) = len(A)</code>.<br>
Actually, I’m not sure about this algorithm can train on <code>batch_size</code> = 64, 128,… because I can not calcualte <code>L</code> on mini batch and mini batch also can not represent for the whole graph.</p>
<p>So, can you see in my code what reason why my loss function is <strong>negative</strong>?</p>
<p>Thanks!</p>

Ryanz: <p>Sorry this post can not support latex.<br>
This my post on stack overflow: <a href="https://ai.stackexchange.com/questions/23061/why-does-my-loss-value-of-autoencoder-in-pytorch-is-negative" rel="nofollow noopener">https://ai.stackexchange.com/questions/23061/why-does-my-loss-value-of-autoencoder-in-pytorch-is-negative</a></p>