Crazy behavior of masked_fill

skerlet_flandorle: <p>I was writing a speech recognition code, but he got -inf for some values in the mel spectrum<br>
I identified that it was caused by the extension part filled with 0 to align with the batch</p>
<p>So if you reset -inf to 0 with masked_fill as shown below, it should be restored.But it doesn’t work.Why is this? ?</p>
<pre><code class="lang-auto">model=VQVAE(hp).to("cuda")

inp=torch.randn(32,96000)
melns=torch.tensor([376,370,376,376,
                     280,270,300,260]).to("cuda")

inp[7][90000:]*=0

inp=inp.masked_fill(float('-inf') == inp,0.)

inpspec = spectrogram(inp).log2().to("cuda")
print(inpspec.size(),inpspec[7])
</code></pre>
<p>result</p>
<pre><code class="lang-auto">tensor([[ 9.2075,  7.7891,  5.3404,  ...,    -inf,    -inf,    -inf],
        [ 9.1560,  7.3915,  5.9799,  ...,    -inf,    -inf,    -inf],
        [ 9.0153,  5.9043,  6.7280,  ...,    -inf,    -inf,    -inf],
        ...,
        [ 9.5787,  9.9251, 10.2523,  ...,    -inf,    -inf,    -inf],
        [ 8.1497, 10.1253, 10.8566,  ...,    -inf,    -inf,    -inf],
        [ 9.8009, 10.7202, 11.2168,  ...,    -inf,    -inf,    -inf]],
       device='cuda:0')
</code></pre>
<p>Also, if you try float(‘-inf’) != inp,0., everything will be -inf</p>

ptrblck: <p>I don’t quite understand your code snippet since <code>masked_fill</code> won’t change anything.<br>
You are initializing <code>inp</code> with values from a normal distribution in:</p>
<pre><code class="lang-python">inp=torch.randn(32,96000)
</code></pre>
<p>which will not create <code>-Inf</code> values in it. Afterwards, you are zeroing out specific values:</p>
<pre><code class="lang-python">inp[7][90000:]*=0
</code></pre>
<p>and are then trying to mask all <code>-Inf</code> in <code>inp</code> via:</p>
<pre><code class="lang-python">inp=inp.masked_fill(float('-inf') == inp,0.)
</code></pre>
<p>while none were created at all.</p>
<p><code>spectrogram(inp)</code> will create zeros where you’ve previously zeroed out the values:</p>
<pre><code class="lang-python">out = spectrogram(inp)
out[7]

tensor([[376.9002, 215.6078,   3.0867,  ...,   0.0000,   0.0000,   0.0000],
        [293.0263, 160.2865, 128.0155,  ...,   0.0000,   0.0000,   0.0000],
        [273.2487, 144.9103, 331.6011,  ...,   0.0000,   0.0000,   0.0000],
        ...,
        [182.5855, 218.7867, 359.0259,  ...,   0.0000,   0.0000,   0.0000],
        [168.2010,  65.0044, 162.7816,  ...,   0.0000,   0.0000,   0.0000],
        [ 22.9736, 337.6405,   7.2782,  ...,   0.0000,   0.0000,   0.0000]])
</code></pre>
<p>and <code>.log2()</code> will then of course create <code>-Inf</code>s which is also expected.</p>

skerlet_flandorle: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="162798">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>and <code>.log2()</code> will then of course create <code>-Inf</code>s which is also expected.</p>
</blockquote>
</aside>
<p>That’s it! ! appreciate<br>
And I was making a very stupid mistake<br>
sorry</p>