Model(data)-using googlenet in a class

python_Dev: <pre><code class="lang-auto">class gN_changed():
  def __init__(self,  latent_dim = 512):
    super(gN_changed, self).__init__()
    self.model = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True) 

    #freeze paramters (trains faster and keeps weight values of ImageNet)
    for params in self.model.parameters():
      params.requires_grad = False

    #change last fully completerd layer
    self.model.fc = nn.Linear(self.model.fc.in_features, latent_dim)
    
  def forward(self, x):
        return self.model(x)
</code></pre>
<p>I use this code. When I create the model everything runs ok but when I pass the image data this get this error</p>
<blockquote>
<p>TypeError: ‘gN_changed’ object is not callable</p>
</blockquote>
<p>However, when I use the same commands not in a class everything works<br>
ie</p>
<pre><code class="lang-auto">c = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True) 
d = models.gN_changed()
for params in c.parameters():
  params.requires_grad = False

c.fc = nn.Linear(c.fc.in_features, 512)
</code></pre>

fpozzi: <p>Hi <a class="mention" href="/u/python_dev">@python_Dev</a>,</p>
<p>the class <code>gN_changed</code> must inherit from <a href="https://pytorch.org/docs/stable/generated/torch.nn.Module.html" rel="noopener nofollow ugc">nn.Module</a> to be itself a valid PyTorch <code>Module</code>.</p>
<pre><code class="lang-python">class gN_changed(nn.Module):
  def __init__(self,  latent_dim = 512):
    super(gN_changed, self).__init__()
    self.model = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True)
</code></pre>

python_Dev: <p>Thank you, I forgot that haha.<br>
I have another issue now, I get this:</p>
<blockquote>
<p>RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the same</p>
</blockquote>
<p>This is my complete code:</p>
<pre><code class="lang-auto">#initiated GoogleNet with last layer modified (LSTM follows)
class gN_changed(nn.Module):
  def __init__(self,  latent_dim = 512):
    super(gN_changed, self).__init__()
    self.model = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True)

    #freeze paramters (trains faster and keeps weight values of ImageNet)
    for params in self.model.parameters():
      params.requires_grad = False

    #change last fully completerd layer
    self.model.fc = nn.Linear(self.model.fc.in_features, latent_dim)
    
  def forward(self, x):
        return self.model(x)
    
class Lstm(nn.Module):
    def __init__(self, latent_dim = 512, hidden_size = 256, lstm_layers = 2, bidirectional = True):
        super(Lstm, self).__init__()
        self.Lstm = nn.LSTM(latent_dim, hidden_size=hidden_size, num_layers=lstm_layers, batch_first=True, bidirectional=bidirectional)
        self.hidden_state = None

    def reset_hidden_state(self):
        self.hidden_state = None

    def forward(self,x):
        output, self.hidden_state = self.Lstm(x, self.hidden_state)
        return output
    
class ConvLstm(nn.Module):
    def __init__(self, latent_dim = 512, hidden_size = 256, lstm_layers = 2, bidirectional = True, n_class = 10):
        super(ConvLstm, self).__init__()
        self.model = gN_changed(latent_dim)
        self.Lstm = Lstm(latent_dim, hidden_size, lstm_layers, bidirectional)
        self.output_layer = nn.Sequential(
            nn.Linear(2 * hidden_size if bidirectional==True else hidden_size, n_class),
            nn.Softmax(dim=-1)
        )

    def forward(self, x):
        batch_size, timesteps, channel_x, h_x, w_x = x.shape
        conv_input = x.view(batch_size * timesteps, channel_x, h_x, w_x)
        conv_output = self.model.forward(conv_input)
        lstm_input = conv_output.view(batch_size, timesteps, -1)
        lstm_output = self.Lstm(lstm_input)
        lstm_output = lstm_output[:, -1, :]
        output = self.output_layer(lstm_output)
        return output
</code></pre>
<p>And when I create the model, I make sure that both the model and the data are in the same device (cuda). I am not sure what goes wrong</p>

fpozzi: <aside class="quote no-group" data-username="python_Dev" data-post="3" data-topic="155009">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/python_dev/48/50776_2.png" class="avatar"> python_Dev:</div>
<blockquote>
<pre><code class="lang-auto">class gN_changed(nn.Module):
  def __init__(self,  latent_dim = 512):
    super(gN_changed, self).__init__()
    self.model = torch.hub.load('pytorch/vision:v0.10.0', 'googlenet', pretrained=True)

    #freeze paramters (trains faster and keeps weight values of ImageNet)
    for params in self.model.parameters():
      params.requires_grad = False

    #change last fully completerd layer
    self.model.fc = nn.Linear(self.model.fc.in_features, latent_dim)
    
  def forward(self, x):
        return self.model(x)
    
class Lstm(nn.Module):
    def __init__(self, latent_dim = 512, hidden_size = 256, lstm_layers = 2, bidirectional = True):
        super(Lstm, self).__init__()
        self.Lstm = nn.LSTM(latent_dim, hidden_size=hidden_size, num_layers=lstm_layers, batch_first=True, bidirectional=bidirectional)
        self.hidden_state = None

    def reset_hidden_state(self):
        self.hidden_state = None

    def forward(self,x):
        output, self.hidden_state = self.Lstm(x, self.hidden_state)
        return output
    
class ConvLstm(nn.Module):
    def __init__(self, latent_dim = 512, hidden_size = 256, lstm_layers = 2, bidirectional = True, n_class = 10):
        super(ConvLstm, self).__init__()
        self.model = gN_changed(latent_dim)
        self.Lstm = Lstm(latent_dim, hidden_size, lstm_layers, bidirectional)
        self.output_layer = nn.Sequential(
            nn.Linear(2 * hidden_size if bidirectional==True else hidden_size, n_class),
            nn.Softmax(dim=-1)
        )

    def forward(self, x):
        batch_size, timesteps, channel_x, h_x, w_x = x.shape
        conv_input = x.view(batch_size * timesteps, channel_x, h_x, w_x)
        conv_output = self.model.forward(conv_input)
        lstm_input = conv_output.view(batch_size, timesteps, -1)
        lstm_output = self.Lstm(lstm_input)
        lstm_output = lstm_output[:, -1, :]
        output = self.output_layer(lstm_output)
        return output
</code></pre>
</blockquote>
</aside>
<p>The following bunch of code works fine.</p>
<pre data-code-wrap="python"><code class="lang-python">model = ConvLstm()
x = torch.rand(1, 2, 3, 224, 224)
device = torch.device("cuda")

model = model.to(device)
x = x.to(device)

_ = model(x)
</code></pre>
<p>The error occurs if you switch device without re-instanciating the network. That’s because the <code>to</code> operator does not move <code>model.Lstm.hidden_state</code> to the new device (since it’s a tensor).</p>
<pre data-code-wrap="python"><code class="lang-python">model = ConvLstm()
x = torch.rand(1, 2, 3, 224, 224)

_ = model(x) # here, model.Lstm.hidden_state is on CPU

device = torch.device("cuda")

model = model.to(device)
x = x.to(device)

_ = model(x) # here, all the modules are on GPU but model.Lstm.hidden_state.
</code></pre>

python_Dev: <p>Thank you, now it works <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>