TypeError: 'bool' object is not callable

username: vin
cooked: <p>Hi, I am trying to compile CNN + LSTM for the imaging that first uses the CNN to train the network, then feeds to the LSTM while defining in sequence each time series of each predicted image.<br>
Below is the code that generates the structure of my network. Indeed, the CNN network works well. Error is identified when I export the CNN module to LSTM. How to solve the error? I face two problems. The first one is located in this part of my script " images = images.view(-1, sequence_dim, INPUT_DIM).requires_grad()<br>
TypeError: ‘bool’ object is not callable". The second displays this error “TypeError: ‘CNNLetNet’ object is not iterable”, when I remove the "view(-1, sequence_dim, INPUT_DIM).requires_grad()“. Need help, thanks.</p>
<pre><code class="lang-auto">class CNNLetNet(nn.Module):
    def __init__(self):
        super(CNNLetNet, self).__init__()
        """
         @ defining four cnn network to perform our data 
         in a set of sequential layers
        """
        self.conv1 = nn.Sequential(
            nn.Conv2d(1, 6, 5, stride=1, padding=1)
        )
        self.conv2 = nn.Sequential(
            nn.Conv2d(6, 16, 5, stride=1, padding=1)
        )
        self.conv3 = nn.Sequential(
            nn.Conv2d(16, 32, 2, stride=1, padding=1)
        )
        self.fc1 = nn.Sequential(
            nn.Linear(32 * 3 * 3, 120)
        )
        self.fc2 = nn.Sequential(
            nn.Linear(120, 60)
        )
        self.fc3 = nn.Sequential(
            nn.Linear(60, 10)
        )
        self.pool = nn.Sequential(
            nn.MaxPool2d(2, 2)
        )
</code></pre>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from cnn_net import CNNLetNet, trainset, testset
from torch.utils.data import DataLoader


class LSTMNet(nn.Module):
    """
     @ the primary goal is to design a sequential lstm network
     with two layers to sequentially train the data
    """
    def __init__(self, input_dim, hidden_dim1, hidden_dim2, layers_dim1, layers_dim2, output_dim):
        super(LSTMNet, self).__init__()
        # Hidden dimensions
        self.hidden1 = hidden_dim1
        self.hidden2 = hidden_dim2
        # Number of hidden layers
        self.layers_dim1 = layers_dim1
        self.layers_dim2 = layers_dim2
        self.cnns = CNNLetNet().to(device='cuda' if torch.cuda.is_available() else 'cpu')
        # print(self.cnns)

        self.lstm1 = nn.Sequential(
            nn.LSTM(input_dim, hidden_dim1, layers_dim1, batch_first=True)
        )
        self.lstm2 = nn.Sequential(
            nn.LSTM(hidden_dim1, hidden_dim2, layers_dim2, batch_first=True)
        )

        self.fc1 = nn.Sequential(
            nn.Linear(hidden_dim2, output_dim)
        )

    def forward(self, x):
        # Initialize hidden state with zeros
        hidden_state1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()
        # Initialize cell state
        hidden_cell_1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()

        hidden_state2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()
        hidden_cell_2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()

        cnn_outs = []   # creating an empty liste to affect CNN trained model
        for conv in self.cnns:
            cnn_outs.append(conv)
        outs = torch.cat(cnn_outs)
        outs = torch.flatten(outs, 1)
        _, outputs1 = self.lstm1(outs, (hidden_state1, hidden_cell_1))
        _, outputs2 = self.lstm2(outputs1, (hidden_state2, hidden_cell_2))

        outputs = self.fc(outputs2[:, -1, :])
        return outputs
</code></pre>

username: ptrblck
cooked: <p><code>tensor.requires_grad</code> is an attribute returning a <code>bool</code> so you can’t call it.<br>
Maybe you want to use <code>tensor.requires_grad_()</code> instead which will change the internal attribute?</p>

username: vin
cooked: <p>I exactly use this <code>tensor.requires_grad_() </code> and have an error. Or my problem is based on the concatenate part (where all CNN layers are concatenated)? where:</p>
<pre><code class="lang-auto">cnn_outs = []   # creating an empty liste to affect CNN trained model
        for conv in self.cnns:
            cnn_outs.append(conv)
        outs = torch.cat(cnn_outs)
        outs = torch.flatten(outs, 1)
        _, outputs1 = self.lstm1(outs, (hidden_state1, hidden_cell_1))
        _, outputs2 = self.lstm2(outputs1, (hidden_state2, hidden_cell_2))

        outputs = self.fc(outputs2[:, -1, :])
</code></pre>
<p>Cause it shows this message:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/2/22c49e4e82b039aa8503551af6faa8b24f90dd30.png" data-download-href="https://discuss.pytorch.org/uploads/default/22c49e4e82b039aa8503551af6faa8b24f90dd30" title="1111"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c49e4e82b039aa8503551af6faa8b24f90dd30_2_690x131.png" alt="1111" data-base62-sha1="4XzubOeyPDFxy5A39azhWY4ljwc" width="690" height="131" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c49e4e82b039aa8503551af6faa8b24f90dd30_2_690x131.png, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c49e4e82b039aa8503551af6faa8b24f90dd30_2_1035x196.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/2/2/22c49e4e82b039aa8503551af6faa8b24f90dd30_2_1380x262.png 2x" data-dominant-color="2F2F32"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">1111</span><span class="informations">1616×309 40 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="vin" data-post="3" data-topic="149155">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/v/a87d85/48.png" class="avatar"> vin:</div>
<blockquote>
<p>I exactly use this <code>tensor.requires_grad_() </code> and have an error.</p>
</blockquote>
</aside>
<p>In your failed code snippet you showed:</p>
<pre data-code-wrap="python"><code class="lang-python">images = images.view(-1, sequence_dim, INPUT_DIM).requires_grad()
</code></pre>
<p>which is not the same, as the trailing underscore is missing and will thus fail as previously described.</p>

username: vin
cooked: <p>after a change in the code, it shows this message:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/5/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d.png" data-download-href="https://discuss.pytorch.org/uploads/default/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d" title="22"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/d/5/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d_2_690x139.png" alt="22" data-base62-sha1="uu4HCB488pH4WLpbLqvAOe4M0iN" width="690" height="139" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/d/5/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d_2_690x139.png, https://discuss.pytorch.org/uploads/default/optimized/3X/d/5/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d_2_1035x208.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/d/5/d5a7902a4fceb2aebbc8f6169bfb1de96f1ae30d_2_1380x278.png 2x" data-dominant-color="2E2E31"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">22</span><span class="informations">1665×336 40.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: vin
cooked: <p>U can check out my full code:</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
from cnn_net import CNNLetNet, trainset, testset
from torch.utils.data import DataLoader


class LSTMNet(nn.Module):
    """
     @ 
    """
    def __init__(self, input_dim, hidden_dim1, hidden_dim2, layers_dim1, layers_dim2, output_dim):
        super(LSTMNet, self).__init__()
        # Hidden dimensions
        self.hidden1 = hidden_dim1
        self.hidden2 = hidden_dim2
        # Number of hidden layers
        self.layers_dim1 = layers_dim1
        self.layers_dim2 = layers_dim2
        self.cnns = CNNLetNet().to(device='cuda' if torch.cuda.is_available() else 'cpu')
        # print(self.cnns)

        self.lstm1 = nn.Sequential(
            nn.LSTM(input_dim, hidden_dim1, layers_dim1, batch_first=True)
        )
        self.lstm2 = nn.Sequential(
            nn.LSTM(hidden_dim1, hidden_dim2, layers_dim2, batch_first=True)
        )

        self.fc1 = nn.Sequential(
            nn.Linear(hidden_dim2, output_dim)
        )

    def forward(self, x):
        # Initialize hidden state with zeros
        hidden_state1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()
        # Initialize cell state
        hidden_cell_1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()

        hidden_state2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()
        hidden_cell_2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()

        cnn_outs = []   # creating an empty liste to affect CNN trained model
        for conv in self.cnns:
            cnn_outs.append(conv)
        outs = torch.cat(cnn_outs)
        outs = torch.flatten(outs, 1)
        _, outputs1 = self.lstm1(outs, (hidden_state1, hidden_cell_1))
        _, outputs2 = self.lstm2(outputs1, (hidden_state2, hidden_cell_2))

        outputs = self.fc(outputs2[:, -1, :])
        return outputs


if __name__ == '__main__':
    INPUT_DIM = 10
    HIDDEN_DIM1 = 8
    HIDDEN_DIM2 = 5
    LAYERS_DIM1 = 1
    LAYERS_DIM2 = 2
    OUTPUT_DIM = 2

    model_lstm = LSTMNet(INPUT_DIM, HIDDEN_DIM1, HIDDEN_DIM2, LAYERS_DIM1, LAYERS_DIM2, OUTPUT_DIM)
    print(model_lstm)

    epochs = 20
    learning_rate = 0.001
    batch_size = 100

    trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)
    testloader = DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=0)

    criterion = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model_lstm.parameters(), lr=learning_rate)
    # Number of steps to unroll
    sequence_dim = 28
    loss_sequence = 0.0
    iteration_over = 0
    for epoch in range(epochs):
        model_lstm.train()
        for i, (images, labels) in enumerate(trainloader):
            model_lstm.train()
            images = images.view(-1, sequence_dim, INPUT_DIM).requires_grad_()

            outputs = model_lstm(images)
            loss = criterion(outputs, labels)
            optimizer.zero_grad()

            # Getting gradients
            loss.backward()

            # Updating parameters
            optimizer.step()

            iteration_over += 1
            loss_sequence += loss.item()

            if iteration_over % 100 == 0:

                # print(f"nIteration : {iteration_over}")
                # with torch.no_grad():
                model_lstm.eval()
                correct = 0
                total = 0
                for images in testloader:
                    model_lstm.eval()
                    images = images.view(-1, sequence_dim, INPUT_DIM)

                    outputs = model_lstm(images)
                    _, predictions = torch.max(outputs.data, 1)
                    total += labels.size(0)
                    correct += (predictions == labels).sum()

                total_accuracy = 100 * correct / total
                print(f"nIteration : {iteration_over} | nLoss : {loss_sequence / 100} | "
                      f"Total accuracy over the data : {total_accuracy}")
</code></pre>

username: ptrblck
cooked: <p>As the error explains: <code>CNNLetNet</code> is not iterable as it’s a custom <code>nn.Module</code>.<br>
If you want to iterate modules, use e.g. <code>nn.ModuleList</code>.</p>

username: vin
cooked: <p>It seems working well with <code>nn.ModuleList</code>. But now another issue while running the entire code. I’ll paste all code in sequence:</p>
<p>Model:</p>
<pre><code class="lang-auto">class LSTMNet(nn.Module):
    """
     @ 
    """
    def __init__(self, model, input_dim, hidden_dim1, hidden_dim2, layers_dim1, layers_dim2, output_dim):
        super(LSTMNet, self).__init__()
        # Hidden dimensions
        self.hidden1 = hidden_dim1
        self.hidden2 = hidden_dim2
        # Number of hidden layers
        self.layers_dim1 = layers_dim1
        self.layers_dim2 = layers_dim2
        self.cnns = model # import CNN network model from CNNLetNet
        # print(self.cnns)

        self.lstm1 = nn.Sequential(
            nn.LSTM(input_dim, hidden_dim1, layers_dim1, batch_first=True)
        )
        self.lstm2 = nn.Sequential(
            nn.LSTM(hidden_dim1, hidden_dim2, layers_dim2, batch_first=True)
        )

        self.fc1 = nn.Sequential(
            nn.Linear(hidden_dim2, output_dim)
        )

    def forward(self, x):
        # Initialize hidden state with zeros
        hidden_state1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()
        # Initialize cell state
        hidden_cell_1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()

        hidden_state2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()
        hidden_cell_2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()

        _, outputs1 = self.lstm1(x, (hidden_state1.detach(), hidden_cell_1.detach()))
        _, outputs2 = self.lstm2(outputs1, (hidden_state2.detach(), hidden_cell_2.detach()))

        outputs = F.relu(self.fc1(outputs2[:, -1, :]))
        return outputs

</code></pre>
<p>Training loop part:</p>
<pre><code class="lang-auto">if __name__ == '__main__':
    INPUT_DIM = 10
    HIDDEN_DIM1 = 8
    HIDDEN_DIM2 = 5
    LAYERS_DIM1 = 1
    LAYERS_DIM2 = 2
    OUTPUT_DIM = 2

    model = CNNLetNet()

    model_lstm = LSTMNet(model, INPUT_DIM, HIDDEN_DIM1, HIDDEN_DIM2, LAYERS_DIM1, LAYERS_DIM2, OUTPUT_DIM)
    print(model_lstm)

    epochs = 20
    learning_rate = 0.001
    batch_size = 100

    trainloader = DataLoader(trainset, batch_size=batch_size, shuffle=True, num_workers=0)
    testloader = DataLoader(testset, batch_size=batch_size, shuffle=False, num_workers=0)

    criterion = nn.CrossEntropyLoss()
    optimizer = torch.optim.Adam(model_lstm.parameters(), lr=learning_rate)
    # Number of steps to unroll
    sequence_dim = 28
    loss_sequence = 0.0
    iteration_over = 0
    for epoch in range(epochs):
        model_lstm.train()
        for i, (images, labels) in enumerate(trainloader):
            model_lstm.train()
            images = images.view(-1, sequence_dim, INPUT_DIM).requires_grad_()

            outputs = model_lstm(images)
            loss = criterion(outputs, labels)
            optimizer.zero_grad()

            # Getting gradients
            loss.backward()

            # Updating parameters
            optimizer.step()

            iteration_over += 1
            loss_sequence += loss.item()

            if iteration_over % 100 == 0:

                # print(f"nIteration : {iteration_over}")
                # with torch.no_grad():
                model_lstm.eval()
                correct = 0
                total = 0
                for images in testloader:
                    model_lstm.eval()
                    images = images.view(-1, sequence_dim, INPUT_DIM)

                    outputs = model_lstm(images)
                    _, predictions = torch.max(outputs.data, 1)
                    total += labels.size(0)
                    correct += (predictions == labels).sum()

                total_accuracy = 100 * correct / total
                print(f"nIteration : {iteration_over} | nLoss : {loss_sequence / 100} | "
                      f"Total accuracy over the data : {total_accuracy}")

</code></pre>
<p>Error :</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/2/e268c2172ac4e69aa6d19bbe995ebaf171333261.png" data-download-href="https://discuss.pytorch.org/uploads/default/e268c2172ac4e69aa6d19bbe995ebaf171333261" title="3"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e268c2172ac4e69aa6d19bbe995ebaf171333261_2_690x174.png" alt="3" data-base62-sha1="wiUlNLIjN7UtptnRoviPOvJJ4Z3" width="690" height="174" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e268c2172ac4e69aa6d19bbe995ebaf171333261_2_690x174.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e268c2172ac4e69aa6d19bbe995ebaf171333261_2_1035x261.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/2/e268c2172ac4e69aa6d19bbe995ebaf171333261_2_1380x348.png 2x" data-dominant-color="302F33"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">3</span><span class="informations">1619×409 56.6 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: ptrblck
cooked: <p>You are trying to pass multiple input arguments to an <code>nn.Sequential</code> module:</p>
<pre><code class="lang-python">_, outputs1 = self.lstm1(x, (hidden_state1.detach(), hidden_cell_1.detach()))
</code></pre>
<p>which won’t work.<br>
Remove the <code>nn.Sequential</code> container and assign <code>self.lstm1</code> directly to the <code>nn.LSTM</code> module.</p>

username: vin
cooked: <p>I have corrected the issue, thanks. But, after correction I have this error raising :</p>
<pre><code class="lang-auto">    def forward(self, x):
        # Initialize hidden state with zeros
        hidden_state1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()
        # Initialize cell state
        hidden_cell_1 = torch.zeros(self.layers_dim1, x.size(0), self.hidden1).requires_grad_()

        hidden_state2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()
        hidden_cell_2 = torch.zeros(self.layers_dim2, x.size(0), self.hidden2).requires_grad_()

        _, outputs1 = self.lstm1(x, (hidden_state1.detach(), hidden_cell_1.detach()))

        _, outputs2 = self.lstm2(outputs1, (hidden_state2.detach(), hidden_cell_2.detach()))

        outputs = self.fc1(outputs2[:, -1, :])
        return outputs
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/0/2/02af89ed9a095703f5df988e987107a6ddb1d990.png" data-download-href="https://discuss.pytorch.org/uploads/default/02af89ed9a095703f5df988e987107a6ddb1d990" title="44"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02af89ed9a095703f5df988e987107a6ddb1d990_2_690x193.png" alt="44" data-base62-sha1="nL2N9NxWzVCTE6tlBpnSwCkCxW" width="690" height="193" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02af89ed9a095703f5df988e987107a6ddb1d990_2_690x193.png, https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02af89ed9a095703f5df988e987107a6ddb1d990_2_1035x289.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/0/2/02af89ed9a095703f5df988e987107a6ddb1d990_2_1380x386.png 2x" data-dominant-color="2F2F33"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">44</span><span class="informations">1671×468 65.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>