F.binary_cross_entropy() input size should be between 0 and 1

Hannes_Kickhop: <p>Hi everyone,<br>
I’ve tried to set up an vae for audio files, but I get the error input size should be between 0 and 1 when using the binary_cross_entropy() function, even though I didn’t have any of these issues when testing with MNIST Image files.<br>
I suspect the error to result from the following function:</p>
<pre><code>def reparameterize(self, mu, logvar):
    if self.training:
        std = logvar.mul(0.5).exp_()  
        eps = Variable(std.data.new(std.size()).normal_())
        return eps.mul(std).add_(mu)
</code></pre>
<p>because that’s where any of my data starts to be out of the 0 to 1 range in the whole code. the processing continues as follows:</p>
<pre><code>*def forward(self, x, sampleCount):*
</code></pre>
<ul>
<li>
<pre><code>   x = x.view(1,1,sampleCount).float()                                         #Reshape the audio data to fit Conv*
</code></pre>
</li>
<li>
<pre><code>   mu, logvar = self.encode(x)*
  z = self.reparameterize(mu, logvar)
  return z, mu, logvar
</code></pre>
</li>
</ul>
<p>next is the loss function, which causes the error:</p>
<p>BCE = F.binary_cross_entropy(z.view(-1, sampleCount), x.view(-1, sampleCount).float())</p>
<p>can anyone please help me?<br>
I’ve already tried using binary_cross_entropy_with_logits which didn’t crash my code, but yield any useable results.</p>
<p>Thanks</p>

chetan_patil: <p>I think the error is pointing out that the span of <code>x</code> is not between 0 and 1.<br>
Because, target needs to be either 0 or 1, in your case, the variable <code>x</code> in <code>F.binary_crossentropy()</code>.<br>
Could you check what’s the min and max value of  <code>x</code> before you pass it to the loss function ?<br>
And can you paste the error ?</p>

Hannes_Kickhop: <p>Thanks for the reply! I’m sorry I didn’t paste the error, I forgot it… my bad.</p>
<p>The torch x before passing it through the loss function: [0.5729, 0.5729, 0.5729,  …, 0.5160, 0.5266, 0.5555] (min 0, max 1)</p>
<p>The error:<br>
all elements of input should be between 0 and 1</p>
<p>However, z (input for binary_cross_entropy) looks like that: [-0.9888,  0.1598,  1.3676,  …,  1.4632, -2.4632,  2.5672]  (min -4.68, max 5.9)<br>
which is causing the error.</p>
<p>If I understood you correctly, both target and input should be either 0 and 1, which means that my data is not correctly transformed for that function? If so, is there a better loss function to use, since audio data is pretty hard to transform to only 0 and 1 while keeping the data loss to a minimum? That would also explain why it worked well on the mnist data, because that’s easy to transform to only 0 and 1 I guess.</p>

KFrank: <p>Hello Hannes!</p>
<aside class="quote no-group" data-username="Hannes_Kickhop" data-post="3" data-topic="86850" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/hannes_kickhop/48/26016_2.png" class="avatar"> Hannes_Kickhop:</div>
<blockquote>
<p>If I understood you correctly, both target and input should be either 0 and 1,</p>
</blockquote>
</aside>
<p>Just to clarify, <code>binary_cross_entropy()</code> does <em>not</em> require the <code>input</code><br>
and <code>target</code> values to be either <code>0</code> or <code>1</code>.  Rather, it requires these<br>
values to be probabilities, that is, values in the range <code>[0.0, 1.0]</code><br>
(inclusive).</p>
<p>In practice, it is often the case that the <code>target</code> values are <code>0</code> or <code>1</code><br>
(and can be understood as binary class labels), but forcing your<br>
input values to be <code>0</code> or <code>1</code> will break differentiablity, and hence<br>
break back-propagation.</p>
<p>Two more general questions:  Why are you reparameterizing your<br>
data?  Conceptually, why are you using <code>binary_cross_entropy()</code>,<br>
and how is this supposed to interact with your reparameterization?</p>
<p>Best.</p>
<p>K. Frank</p>

Hannes_Kickhop: <p>I fixed my issue, I had a function which imported the files and scaled the data to the range 0-1 which was not behaving properly. Everything is working now. Thanks to everybody helping, even if it didn’t solve my issue (because I didn’t upload enough of the code) it still helped me a understanding how pytorch works.</p>