How to define/convert a LSTM in PyTorch, that I have in Keras for Music Generation?

Deb_Prakash_Chatterj: <p>Hey everyone, I have a Keras LSTM code, that I want to transfer in LSTM in PyTorch for Music Generation. This is my code for Keras LSTM -</p>
<pre><code class="lang-auto">def built_model(batch_size, seq_length, unique_chars):
    model = Sequential()
    
    model.add(Embedding(input_dim = unique_chars, output_dim = 512, batch_input_shape = (batch_size, seq_length), name = "embd_1")) 
    
    model.add(LSTM(512, return_sequences = True, stateful = True, name = "lstm_first"))
    model.add(Dropout(0.4, name = "drp_1"))
    
    model.add(LSTM(512, return_sequences = True, stateful = True))
    model.add(Dropout(0.4))
    
    model.add(LSTM(512, return_sequences = True, stateful = True))
    model.add(Dropout(0.2))
    
    model.add(TimeDistributed(Dense(unique_chars)))
    model.add(Activation("softmax"))
    
    return model
</code></pre>
<p>And is the one that I am trying to do in PyTorch from my Sentiment Analysis model-</p>
<pre><code class="lang-auto">import torch.nn as nn

class SentimentRNN(nn.Module):
    """
    The RNN model that will be used to perform Sentiment analysis.
    """

    def __init__(self, batch_size, seq_length, unique_chars, embedding_dim, output_size, hidden_dim, n_layers, drop_prob=0.5):
        """
        Initialize the model by setting up the layers.
        """
        super(SentimentRNN, self).__init__()

        self.output_size = output_size
        self.n_layers = n_layers
        self.hidden_dim = hidden_dim
        
        # embedding and LSTM layers
        self.embedding = nn.Embedding(unique_chars, embedding_dim)
        self.lstm = nn.LSTM(embedding_dim, hidden_dim, n_layers,
                            dropout=drop_prob, batch_first=True)
        
        # dropout layer
        self.dropout = nn.Dropout(0.3)
        
        # linear and sigmoid layer
        self.fc = nn.Linear(hidden_dim, unique_chars)
        self.sig = nn.Softmax()

    def forward(self, x, hidden):
        """
        Perform a forward pass of our model on some input and hidden state.
        """
        batch_size = x.size(0)
        
        # embeddings and lstm_out
        embeds = self.embedding(x)
        lstm_out, hidden = self.lstm(embeds, hidden)
        
        # stack up lstm outputs
        lstm_out = lstm_out.contiguous().view(-1, self.hidden_dim)
        
        # dropout and fully connected layer
        out = self.dropout(lstm_out)
        out = self.fc(out)
        
        # sigmoid function
        sig_out = self.sig(out)
        
        # reshape to be batch_size first
        sig_out = sig_out.view(batch_size, -1)
        sig_out = sig_out[:, -1] # get last batch of labels
        
        # return last sigmoid output and hidden state
        return sig_out, hidden
    
    
    def init_hidden(self, batch_size):
        ''' Initializes hidden state '''
        # Create two new tensors with sizes n_layers x batch_size x hidden_dim,
        # initialized to zero, for hidden state and cell state of LSTM
        weight = next(self.parameters()).data
        
        if(train_on_gpu):
          hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().cuda(),
                   weight.new(self.n_layers, batch_size, self.hidden_dim).zero_().cuda())
        else:
          hidden = (weight.new(self.n_layers, batch_size, self.hidden_dim).zero_(),
                   weight.new(self.n_layers, batch_size, self.hidden_dim).zero_())
        
        return hidden
        
</code></pre>
<p>I am trying to do this on my own. But I am clueless, where to start, can anyone please help to find one? Thanks.</p>

urw7rs: <p>start by trying tutorials in <a href="http://pytorch.org" rel="nofollow noopener">pytorch.org</a> and read docs for further explanations</p>

ptrblck: <p>I would try to narrow down the problem a bit and e.g. start with a single layer.<br>
Once you get the same outputs (up to floating point precision) for the embedding layer, I would try to match the output of the next LSTM layer and so on.</p>