No embbeding.grad.data

username: Rohan_Kumar
cooked: <p>as mentioned in the title, no grad.data is present in nn.embedding, so how do we train the embeddings?</p>

username: ptrblck
cooked: <p>The <code>weight</code> parameter of <code>nn.Embedding</code> does have this attribute:</p>
<pre><code class="lang-python">emb = nn.Embedding(10, 10)
x = torch.randint(0, 10, (10,))
out = emb(x)
out.mean().backward()
print(emb.weight.grad)
print(emb.weight.grad.data)
</code></pre>
<p>However, note that the usage of <code>.data</code> is not recommended.</p>

username: Rohan_Kumar
cooked: <p>I Have used the embeddings in a similar fashion, but the weights of embedding layer, before and after have not changed at all.</p>

username: Rohan_Kumar
cooked: <pre><code class="lang-auto">for epoch in tqdm(range(epochs)):
        model.train()
        epoch_loss = 0
        for batch in tqdm(train_iterator):
            src_sentence = Variable(batch['src_sentence']).cuda()
            src_speaker = Variable(batch['src_speaker']).cuda()
            trg_sentence = Variable(batch['trg_sentence']).cuda()
            trg_speaker = Variable(batch['trg_speaker']).cuda()

            optimizer.zero_grad()
            output, predicted_speaker = model(src_sentence,src_speaker, trg_sentence, trg_speaker)
            
            output = output[1:].view(-1, output.shape[-1])
            trg = trg_sentence.permute(1,0)
            trg = trg[1:].reshape(-1)

            predicted_speaker = log_softmax(predicted_speaker)
            
            loss1 = criterion1(output, trg)
            loss2 = criterion2(predicted_speaker, trg_speaker)
            loss = loss1 + loss2 
            loss.backward()

            torch.nn.utils.clip_grad_norm_(model.parameters(), 1)

            optimizer.step()
            epoch_loss += loss.item()
</code></pre>
<p>This is my code for training and computing loss</p>

username: Rohan_Kumar
cooked: <pre><code class="lang-auto">enc = Encoder(WEIGHTS, HID_DIM, EMB_DIM, PERSONAS, PERSONA_DIM, DROPOUT).cuda()
dec = Decoder(EMB_DIM, PERSONA_DIM, HID_DIM, OUTPUT_DIM, DROPOUT).cuda()

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model = Seq2Seq(enc, dec, len(characters), HID_DIM, device).to(device)
</code></pre>
<p>initialization of the model, as you can see i have 3 classes, one for encoder, one for decoder and one for seq2seq.</p>
<p>and inside encoder here is how i have used embedding,</p>
<pre><code class="lang-auto">class Encoder(nn.Module):
    def __init__(self, weights_matrix, hid_dim, emb_dim, personas, persona_dim, dropout):
        super().__init__()
        self.word_embedding = nn.Embedding.from_pretrained(weights_matrix, freeze = True)
        self.rnn = nn.GRU(emb_dim + persona_dim, hid_dim)
        self.dropout = nn.Dropout(dropout)
        self.persona_embedding = nn.Embedding(personas+10, persona_dim)

    def forward(self, src_sentence, src_speaker):
        """
        ------------------------------------------------------
        Encoder for Seq2seq persona embedding creation 
        Variable    ----    Size
        src_sentence        [batch size, sequence length]
        src_speaker         [batch size]  
        
        embedded_sentence   [batch size, sequence length, emb dim]
        embedded_speaker    [batch size, persona dim]

        emb_con             [batch size, sequence length, person dim + emb dim]
        emb_con(permute)    [sequence length, batch size, person dim + emb dim]

        hidden              [1, batch size, hidden dim]
        ------------------------------------------------------
        """
        embedded_sentence = self.word_embedding(src_sentence)
        embedded_speaker = self.dropout(self.persona_embedding(src_speaker))
        
        embedded_speaker = embedded_speaker.unsqueeze(1).repeat(1,embedded_sentence.shape[1], 1)
        
        emb_con = torch.cat((embedded_sentence, embedded_speaker), dim = 2)
        emb_con = emb_con.permute(1,0,2)
        output, hidden = self.rnn(emb_con)

        return hidden

</code></pre>

username: Rohan_Kumar
cooked: <p>is it possible that torch.cat is breaking the graph and hence embedding is not being trained?</p>

username: ptrblck
cooked: <p>This shouldn’t be the case.<br>
Skimming through your code, I cannot find anything which breaks the graph or detaches the tensors.<br>
Could you try to isolate the issue and post a code to reproduce this behavior?</p>

username: Rohan_Kumar
cooked: <p>Isolating the issue is one of the key problems here, as the model being a sequence to sequence network is pretty big, and hence difficult to debug.</p>
<pre><code class="lang-auto">class Decoder(nn.Module):
    def __init__(self, emb_dim, persona_dim, hid_dim, trg_vocab_size, dropout):
        super().__init__() 
        self.output_dim = trg_vocab_size 
        self.word_embedding = nn.Embedding.from_pretrained(weights_matrix, freeze = True)
        self.rnn = nn.GRU(emb_dim + hid_dim, hid_dim)
        self.out = nn.Linear(emb_dim + hid_dim*2, trg_vocab_size)
        self.dropout = nn.Dropout(dropout)
    
    def forward(self, x, hidden, context):
        """
        ------------------------------------------------------
        Decoder for Seq2seq persona embedding creation 
        Variable    ----    Size
        x                   [1, batch size]
        embedded            [1, batch size, emb dim]
        
        context             [1, batch size, hid dim]
        emb_con             [1, batch size, hid dim + emb dim]
        
        hidden              [1, batch size, hid dim]
        output              [1, batch size, hid dim]

        output              [batch size, hid dim*2 + emb dim]
        prediction          [batch size, trg vocab size]
        ------------------------------------------------------
        """
        x = x.unsqueeze(0)

        embedded = self.dropout(self.word_embedding(x))
        emb_con = torch.cat((embedded, context), dim = 2)

        output, hidden = self.rnn(emb_con, hidden)

        output = torch.cat((embedded.squeeze(0), output.squeeze(0), context.squeeze(0)), dim = 1)
        prediction = self.out(output)

        return prediction, hidden
</code></pre>
<p>^the decoder part.</p>
<pre><code class="lang-auto">class Seq2Seq(nn.Module):
    def __init__(self,encoder, decoder, personas, hid_dim ,  device):
        super().__init__()
        self.encoder = encoder 
        self.decoder = decoder 
        self.device = device 
        self.fc = nn.Linear(hid_dim, personas+1)
    def forward(self, src,src_speaker,trg, trg_speaker, teacher_forcing_ratio  = 0.5):
        """
        ------------------------------------------------------
        End-to-end network for Seq2seq persona embedding creation 
        Variable    ----    Size    
        trg                 [batch size, sequence length]   
        src                 [batch size, sequence length]
        outputs             [sequence length, batch size, trg vocab size]
        hidden              [1, batch size, hid dim]
        input               [batch size]
        ------------------------------------------------------
        """
        batch_size = trg.shape[0]
        trg = trg.permute(1,0)
        max_len = trg.shape[0] 
        trg_vocab_size = self.decoder.output_dim 
        
        outputs = torch.zeros(max_len, batch_size, trg_vocab_size).to(self.device)
        
        context = self.encoder(src, src_speaker)
        
        hidden = context
        input = torch.LongTensor(np.ones(batch_size))
        
        for t in range(1,max_len):
            input = Variable(input).cuda()
            hidden = Variable(hidden).cuda()
            context = Variable(context).cuda()
            output, hidden = self.decoder(input, hidden, context)
            outputs[t] = output
            
            teacher_force = random.random() &lt; teacher_forcing_ratio
            
            top1 = output.argmax(1) 
            
            input = trg[t] if teacher_force else top1
        
        persona_out = self.fc(context).squeeze(0)
        
        return outputs, persona_out
</code></pre>
<p>^ The main sequence to sequence network</p>
<pre><code class="lang-auto">model.encoder.persona_embedding.weight.detach().cpu()
</code></pre>
<p>^ I print this before and after training, and it is exactly the same in both the cases.</p>

username: ptrblck
cooked: <p>You are freezing the embedding layer in your initialization:</p>
<pre><code class="lang-python">nn.Embedding.from_pretrained(weights_matrix, freeze = True)
</code></pre>
<p>Could you set <code>freeze=False</code> and rerun your code?</p>

username: Rohan_Kumar
cooked: <p>If you see in encoder class, there is one nn.embedding without frozen weights, that is where the problem lies, which is not updating.</p>

username: ptrblck
cooked: <p>I don’t know your setup and shapes, but based on this minimal code snippet, the mentioned embedding gets valid gradients:</p>
<pre><code class="lang-python">enc = Encoder(torch.randn(1, 1), 1, 1, 1, 1, 0.0)
dec = Decoder(torch.randn(1, 1), 1, 1, 1, 1, 0.0)

model = Seq2Seq(enc, dec, 1, 1, 'cpu')
src = torch.tensor([[0]]).expand(1, 2)
src_speaker = torch.tensor([0])
trg = torch.tensor([[0]]).expand(1, 2)
trg_speaker = torch.tensor([[0]]).expand(1, 2)
enc(src, src_speaker)

output = model(src, src_speaker, trg, trg_speaker)
output[0].mean().backward(retain_graph=True)
output[1].mean().backward()

print(model.encoder.persona_embedding.weight.grad)
&gt; tensor([[-0.0550],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000],
        [ 0.0000]])
</code></pre>

username: Rohan_Kumar
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/c/2/c2aec13689d42ae19a8346c18a46d0d5426fbbce.png" data-download-href="https://discuss.pytorch.org/uploads/default/c2aec13689d42ae19a8346c18a46d0d5426fbbce" title="Screenshot_2020-01-07 Google Colaboratory"><img src="https://discuss.pytorch.org/uploads/default/original/3X/c/2/c2aec13689d42ae19a8346c18a46d0d5426fbbce.png" alt="Screenshot_2020-01-07 Google Colaboratory" data-base62-sha1="rMf2s626SrOA8K1NGl6WrTIXGyy" width="690" height="241" data-dominant-color="EDEEED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-01-07 Google Colaboratory</span><span class="informations">941×329 27.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I tried running the code that you wrote and I am still getting no grad.</p>

username: Rohan_Kumar
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/7/27d03b10eb6192162be4683d69a0a6df927ee895.png" data-download-href="https://discuss.pytorch.org/uploads/default/27d03b10eb6192162be4683d69a0a6df927ee895" title="Screenshot_2020-01-07 Google Colaboratory(1)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27d03b10eb6192162be4683d69a0a6df927ee895_2_690x290.png" alt="Screenshot_2020-01-07 Google Colaboratory(1)" data-base62-sha1="5GcKIVjNGe9HeHDeslsHkDSnsfX" width="690" height="290" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27d03b10eb6192162be4683d69a0a6df927ee895_2_690x290.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/7/27d03b10eb6192162be4683d69a0a6df927ee895_2_1035x435.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/7/27d03b10eb6192162be4683d69a0a6df927ee895.png 2x" data-dominant-color="EFF1EF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-01-07 Google Colaboratory(1)</span><span class="informations">1073×451 36.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
But when i only use encoder as a single class, then there is grad present. Its only when i initialize the seq2seq class contiaining the encoder the class, that grad is absent.</p>

username: Rohan_Kumar
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/7/1/719697404aca7b6c73d81d0efe39bdb50b027dc4.png" data-download-href="https://discuss.pytorch.org/uploads/default/719697404aca7b6c73d81d0efe39bdb50b027dc4" title="Screenshot_2020-01-07 Google Colaboratory(2)"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/719697404aca7b6c73d81d0efe39bdb50b027dc4_2_690x226.png" alt="Screenshot_2020-01-07 Google Colaboratory(2)" data-base62-sha1="gcQANtclWEwwNz5KGFeKMv3nHHm" width="690" height="226" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/719697404aca7b6c73d81d0efe39bdb50b027dc4_2_690x226.png, https://discuss.pytorch.org/uploads/default/optimized/3X/7/1/719697404aca7b6c73d81d0efe39bdb50b027dc4_2_1035x339.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/7/1/719697404aca7b6c73d81d0efe39bdb50b027dc4.png 2x" data-dominant-color="EEEFED"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-01-07 Google Colaboratory(2)</span><span class="informations">1073×352 31.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I changed the model and used encoder outside the seq2seq class, and still the grad disappeared</p>

username: Rohan_Kumar
cooked: <pre><code class="lang-auto">class Seq2Seq(nn.Module):
    def __init__(self,encoder, decoder, personas, hid_dim ,  device):
        super().__init__()
        self.encoder = encoder 
        self.decoder = decoder 
        self.device = device 
        self.fc = nn.Linear(hid_dim, personas+1)
    def forward(self, src,src_speaker,trg, trg_speaker, teacher_forcing_ratio  = 0.5):
        """
        ------------------------------------------------------
        End-to-end network for Seq2seq persona embedding creation 
        Variable    ----    Size    
        trg                 [batch size, sequence length]   
        src                 [batch size, sequence length]
        outputs             [sequence length, batch size, trg vocab size]
        hidden              [1, batch size, hid dim]
        input               [batch size]
        ------------------------------------------------------
        """
        batch_size = trg.shape[0]
        trg = trg.permute(1,0)
        max_len = trg.shape[0] 
        trg_vocab_size = self.decoder.output_dim 
        
        outputs = torch.zeros(max_len, batch_size, trg_vocab_size).to(self.device)
        
        context = self.encoder(src, src_speaker)
        
        # hidden = context
        # input = torch.LongTensor(np.ones(batch_size))
        
        # for t in range(1,max_len):
        #     input = Variable(input).cuda()
        #     hidden = Variable(hidden).cuda()
        #     context = Variable(context).cuda()
        #     output, hidden = self.decoder(input, hidden, context)
        #     outputs[t] = output
            
        #     teacher_force = random.random() &lt; teacher_forcing_ratio
            
        #     top1 = output.argmax(1) 
            
        #     input = trg[t] if teacher_force else top1
        
        persona_out = self.fc(context).squeeze(0)
        
        return persona_out
</code></pre>
<p>I changed the seq2seq class so that it calls encoder but doesn’t call decoder, then the grad is present.</p>
<p>So i think the problem might be in decoder, or the way i am calling the decoder in seq2seq class.</p>

username: Rohan_Kumar
cooked: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d5afba2458d90f4466709e074c7639360021f7a9.png" data-download-href="https://discuss.pytorch.org/uploads/default/d5afba2458d90f4466709e074c7639360021f7a9" title="Screenshot_2020-01-07 Google Colaboratory(3)"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d5afba2458d90f4466709e074c7639360021f7a9.png" alt="Screenshot_2020-01-07 Google Colaboratory(3)" data-base62-sha1="uumc4SvF6QoELLIq8ejsKDdHFNf" width="690" height="405" data-dominant-color="EFEFEF"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot_2020-01-07 Google Colaboratory(3)</span><span class="informations">715×420 23.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I think I might have solved this, if i keep everything as is and only comment out contex = Variable(context).cuda(), the grad for embedding is present.</p>
<p>I think declaring the output of encoder class as Variable somehow breaks the class, am i correct?</p>

username: ptrblck
cooked: <p>That would be the first time I’m seeing deprecated <code>Variables</code> break the functionality of some code.<br>
And sorry for not mentioning the fact, that I’ve removed them, but I’ve missed it apparently while trying to make the code run.</p>