Activation function " TypeError: 'Tensor' object is not callable"

bc060400164: <p>I am trying to use the relu activation function in PyTorch LSTM but getting the error on " Tensor object is not callable. any guideline or help? Can i use the different activation function in forward propogation? but i am using the same activation function in hidden layers and forward propagation. your kind review will be more help full.</p>
<p>class LSTM(nn.Module):<br>
def <strong>init</strong>(self, input_size=1, hidden_layer_size=20, output_size=1):<br>
super().<strong>init</strong>()</p>
<pre><code>self.hidden_layer_size = hidden_layer_size
self.lstm = nn.LSTM(input_size, hidden_layer_size)
self.relu = nn.functional.relu(torch.FloatTensor(hidden_layer_size), torch.FloatTensor(output_size))     
self.hidden_cell = (torch.zeros(1, 1, self.hidden_layer_size),    
                    torch.zeros(1, 1, self.hidden_layer_size))
 def forward(self, input_seq):
lstm_out, self.hidden_cell = self.lstm(input_seq.view(len(input_seq), 1, -1), self.hidden_cell)
predictions = self.relu(lstm_out.view(len(input_seq), -1))
return predictions[-1]



model = LSTM()
loss_function = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr = 0.0001)





epochs = 150

 for i in range(epochs):
   for seq, labels in train_inout_seq:
    optimizer.zero_grad()
    model.hidden_cell = (torch.zeros(1, 1, model.hidden_layer_size),
                        torch.zeros(1, 1, model.hidden_layer_size))    
   y_pred = model(seq)

single_loss = loss_function(y_pred, labels)
single_loss.backward()
optimizer.step()

 if i%25 == 1:
print(f'epoch: {i:3} loss: {single_loss.item():10.8f}')

print(f'epoch: {i:3} loss: {single_loss.item():10.10f}')
</code></pre>
<p><code>After that i am getting an error:</code></p>
<hr>
<p>TypeError                                 Traceback (most recent call last)<br>
 in <br>
6     model.hidden_cell = (torch.zeros(1, 1, model.hidden_layer_size),<br>
7                          torch.zeros(1, 1, model.hidden_layer_size))<br>
----&gt; 8     y_pred = model(seq)<br>
9<br>
10     single_loss = loss_function(y_pred, labels)</p>
<p>/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in <strong>call</strong>(self, *input, **kwargs)<br>
548             result = self._slow_forward(*input, **kwargs)<br>
549         else:<br>
 → 550             result = self.forward(*input, **kwargs)<br>
551         for hook in self._forward_hooks.values():<br>
552             hook_result = hook(self, input, result)</p>
<p> in forward(self, input_seq)<br>
12   def forward(self, input_seq):<br>
13     lstm_out, self.hidden_cell = self.lstm(input_seq.view(len(input_seq), 1, -1), self.hidden_cell)<br>
—&gt; 14     predictions = self.relu(lstm_out.view(len(input_seq), -1))<br>
15     return predictions[-1]</p>
<p>TypeError: ‘Tensor’ object is not callable```</p>
<pre><code class="lang-auto"></code></pre>

ptrblck: <p>You are trying to define <code>self.relu</code> as a tensor in:</p>
<pre><code class="lang-python">self.relu = nn.functional.relu(torch.FloatTensor(hidden_layer_size), torch.FloatTensor(output_size))     
</code></pre>
<p>Note that this call is most likely not, what you are trying to achieve, as the second argument would be used as the <code>inplace</code> argument.<br>
Also, torch.FloatTensor<code>will create a new tensor in the provided shape with uninitialized memory, so I would recommend to use the factory methods, such as</code>torch.zeros<code>, </code>torch.randn` etc.</p>
<p>The original error is later raised, since you are trying to call the tensor in:</p>
<pre><code class="lang-python">self.relu(lstm_out.view(len(input_seq), -1))
</code></pre>
<p>Assuming you would like to use <code>self.relu</code> as a module, use <code>self.relu = nn.ReLU()</code>.</p>

bc060400164: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="116035">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p>self.relu = nn.ReLU()</p>
</blockquote>
</aside>
<p>As i used the self.relu = nn.ReLU() but how i will give the input size and hidden layer size to that relu function?</p>

bc060400164: <p>As in linear activation function we use the hidden layer size and output size so how we will give the hiiden layer size and output size in relu function?<br>
self.linear = nn.Linear(hidden_layer_size, output_size)</p>
<p>As u guided to use the <code>self.relu = nn.ReLU()</code>so i tried and i cannot see my loss is changing its getting same on every epoch:</p>
<p>epoch:   1 loss: 0.88755035<br>
epoch:  26 loss: 0.88755035<br>
epoch:  51 loss: 0.88755035<br>
epoch:  76 loss: 0.88755035<br>
epoch: 101 loss: 0.88755035<br>
epoch: 126 loss: 0.88755035<br>
epoch: 149 loss: 0.8875503540</p>

ptrblck: <p>The <code>nn.ReLU()</code> module doesn’t have any parameters and will work on any input, so you don’t need to provide a shape for it.</p>