Summary of LSTM Model

astra1234567: <p>Hi !<br>
I have problem with summary method. I am working with basic Lstm model and I don’t know how fix the problem.<br>
Model:</p>
<pre><code class="lang-auto">class LSTMModel(nn.Module):
    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim):
        super(LSTMModel, self).__init__()
        self.hidden_dim = hidden_dim
        self.layer_dim = layer_dim
        self.lstm = nn.LSTM(input_dim, hidden_dim, layer_dim, batch_first=True)
        self.fc = nn.Linear(64 * hidden_dim, output_dim)

    def forward(self, x):
        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
        c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
        out, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach()))
        out = torch.flatten(out,start_dim= 1, end_dim= 2)
        out = self.fc(out) 
        return out
model = LSTMModel(8, 18, 7, 9)
summary(model,(64, 8))
</code></pre>
<p>I receive AttributeError: ‘tuple’ object has no attribute ‘size’. So how can I make summary ?<br>
Can you help me with this ? Thanks !</p>

ptrblck: <p>Your input seems to be missing the batch dimension and after fixing it <code>torchinfo.summary</code> works:</p>
<pre><code class="lang-python">class LSTMModel(nn.Module):
    def __init__(self, input_dim, hidden_dim, layer_dim, output_dim):
        super(LSTMModel, self).__init__()
        self.hidden_dim = hidden_dim
        self.layer_dim = layer_dim
        self.lstm = nn.LSTM(input_dim, hidden_dim, layer_dim, batch_first=True)
        self.fc = nn.Linear(64 * hidden_dim, output_dim)

    def forward(self, x):
        h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
        c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_()
        out, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach()))
        out = torch.flatten(out,start_dim= 1, end_dim= 2)
        out = self.fc(out) 
        return out

model = LSTMModel(8, 18, 7, 9)
x = torch.randn(64, 8)
out = model(x)
# RuntimeError: For unbatched 2-D input, hx and cx should also be 2-D but got (3-D, 3-D) tensors

x = torch.randn(1, 64, 8)
out = model(x) # works

torchinfo.summary(model, (1, 64, 8), device="cpu")
# ==========================================================================================
# Layer (type:depth-idx)                   Output Shape              Param #
# ==========================================================================================
# LSTMModel                                [1, 9]                    --
# ├─LSTM: 1-1                              [1, 64, 18]               18,432
# ├─Linear: 1-2                            [1, 9]                    10,377
# ==========================================================================================
# Total params: 28,809
# Trainable params: 28,809
# Non-trainable params: 0
# Total mult-adds (M): 1.19
# ==========================================================================================
# Input size (MB): 0.00
# Forward/backward pass size (MB): 0.01
# Params size (MB): 0.12
# Estimated Total Size (MB): 0.13
# ==========================================================================================
</code></pre>

astra1234567: <p>Thanks! It works <img src="https://discuss.pytorch.org/images/emoji/apple/blush.png?v=12" title=":blush:" class="emoji" alt=":blush:" loading="lazy" width="20" height="20">. Are there any differences between torchinfo.summary and torchsummary.summary? For torchsummary it does not work.</p>

ptrblck: <p>In case you are referring to <a href="https://github.com/sksq96/pytorch-summary">this repository</a> then be aware of the note in the README:</p>
<blockquote>
<p>Use the new and updated <a href="https://github.com/TylerYep/torchinfo">torchinfo</a>.</p>
</blockquote>
<p>as the last change to <code>torchsummary</code> directly was done ~3 years ago.</p>

Bibek_Chalise: <pre><code class="lang-python">class LSTMModel(nn.Module):
  def __init__(self,hidden_dim=80, input_dim=99,  n_layers=1):
    super(LSTMModel, self).__init__()
    self.input_dim = input_dim
    self.n_layers = n_layers
    self.hidden_dim = hidden_dim
    self.rnn = nn.LSTM(input_dim,hidden_dim, n_layers, batch_first=True)
    self.fc = nn.Linear(hidden_dim, 64)
    self.relu1  = nn.ReLU()
    self.fc1 = nn.Linear(64,20)
    self.relu2  = nn.ReLU()
    self.fc2 = nn.Linear(20,6) 
  def forward(self, input):
    h0 = torch.zeros(self.n_layers,  self.hidden_dim)
    c0 = torch.zeros(self.n_layers,  self.hidden_dim)
    out,( _,_) = self.rnn(input,(h0, c0))
    pred = self.fc(out)
    pred = self.relu1(pred)
    pred = self.fc1(pred)
    pred = self.relu2(pred)
    pred = self.fc2(pred)
    output = nn.Softmax( dim=1)(pred)
    return output
</code></pre>
<p>Can anyone help me find out the summary of this model?</p>

ptrblck: <p>Calling <code>torchinfo.summary</code> seems to work:</p>
<pre><code class="lang-python">model = LSTMModel()
torchinfo.summary(model, input_size=(80, 99), device="cpu")

# ==========================================================================================
# Layer (type:depth-idx)                   Output Shape              Param #
# ==========================================================================================
# LSTMModel                                [80, 6]                   --
# ├─LSTM: 1-1                              [80, 80]                  57,920
# ├─Linear: 1-2                            [80, 64]                  5,184
# ├─ReLU: 1-3                              [80, 64]                  --
# ├─Linear: 1-4                            [80, 20]                  1,300
# ├─ReLU: 1-5                              [80, 20]                  --
# ├─Linear: 1-6                            [80, 6]                   126
# ==========================================================================================
# Total params: 64,530
# Trainable params: 64,530
# Non-trainable params: 0
# Total mult-adds (M): 371.22
# ==========================================================================================
# Input size (MB): 0.03
# Forward/backward pass size (MB): 0.11
# Params size (MB): 0.26
# Estimated Total Size (MB): 0.40
# ==========================================================================================
</code></pre>