What wrong with mine masking?

hadaev8: <p>Pytorch 1.2, tacotron gst model.<br>
I wonna try to mask out gst outputs.<br>
I have two 3d array with shape [32, 177, 512]<br>
                    <a href="https://i.imgur.com/0rTAjED.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/0rTAjED.png" width="670" height="164">
          </a>

<br>
                    <a href="https://i.imgur.com/tNnTDAT.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/tNnTDAT.png" width="638" height="120">
          </a>

<br>
So i just wonna have zeros in first tensor like in second</p>
<p>Taking mask and expand shape<br>
gst_mask = ~get_mask_from_lengths(text_lengths)<br>
gst_mask = gst_mask.unsqueeze(-1).expand_as(gst_outputs)<br>
Seems ok<br>
                    <a href="https://i.imgur.com/sjZcH6Q.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/sjZcH6Q.png" width="650" height="156">
          </a>

</p>
<p>Then i do<br>
gst_outputs.data.masked_fill_(gst_mask, 0.0)</p>
<p>But get unexpected output:<br>
                    <a href="https://i.imgur.com/p4Cfq72.png" target="_blank" class="onebox" rel="nofollow noopener">
            <img src="https://i.imgur.com/p4Cfq72.png" width="690" height="172">
          </a>

</p>

albanD: <p>Hi,</p>
<p>How is gst_outputs created? Can your add a <code>gst_outputs = gst_outputs.clone()</code> before doing the masking?</p>

hadaev8: <p>Basically it is like this</p>
<aside class="onebox githubblob">
  <header class="source">
      <a href="https://github.com/mozilla/TTS/blob/master/models/tacotrongst.py#L70" target="_blank" rel="nofollow noopener">github.com</a>
  </header>
  <article class="onebox-body">
    <h4><a href="https://github.com/mozilla/TTS/blob/master/models/tacotrongst.py#L70" target="_blank" rel="nofollow noopener">mozilla/TTS/blob/master/models/tacotrongst.py#L70</a></h4>
<pre class="onebox"><code class="lang-py"><ol class="start lines" start="60" style="counter-reset: li-counter 59 ;">
<li>
</li>
<li>def inference(self, characters, speaker_ids=None, style_mel=None):</li>
<li>    B = characters.size(0)</li>
<li>    inputs = self.embedding(characters)</li>
<li>    encoder_outputs = self.encoder(inputs)</li>
<li>    encoder_outputs = self._add_speaker_embedding(encoder_outputs,</li>
<li>                                                  speaker_ids)</li>
<li>    if style_mel is not None:</li>
<li>        gst_outputs = self.gst(style_mel)</li>
<li>        gst_outputs = gst_outputs.expand(-1, encoder_outputs.size(1), -1)</li>
<li class="selected">        encoder_outputs = encoder_outputs + gst_outputs</li>
<li>    mel_outputs, alignments, stop_tokens = self.decoder.inference(</li>
<li>        encoder_outputs)</li>
<li>    mel_outputs = mel_outputs.view(B, -1, self.mel_dim)</li>
<li>    linear_outputs = self.postnet(mel_outputs)</li>
<li>    linear_outputs = self.last_linear(linear_outputs)</li>
<li>    return mel_outputs, linear_outputs, alignments, stop_tokens</li>
<li>
</li>
<li>def _add_speaker_embedding(self, encoder_outputs, speaker_ids):</li>
<li>    if hasattr(self, "speaker_embedding") and speaker_ids is not None:</li>
<li>        speaker_embeddings = self.speaker_embedding(speaker_ids)</li>
</ol></code></pre>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>

<p>will the gradient propagate if i copy tensor?</p>

albanD: <p>Yes it will <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
As I was suspecting, <code>gst_outputs</code> is created with an <code>expand</code>. This means that this is not backed by actual memory and writing to one place of it will have effect on other places.<br>
Using clone will force it to be backed by real memory and remove this problem !</p>