Forward method in PyTorch

hiepnguyen034: <p>I am following this <a href="https://pytorch.org/tutorials/beginner/nn_tutorial.html" rel="nofollow noopener">tutorial</a> and I am confused by the part where the forward method is defined. Specifically, from the tutorial, we have this class</p>
<pre><code class="lang-auto">class Mnist_Logistic(nn.Module):
    def __init__(self):
        super().__init__()
        self.lin = nn.Linear(784, 10)

    def forward(self, xb):
        return self.lin(xb)
</code></pre>
<p>. Then, when we do</p>
<pre><code class="lang-auto">model = Mnist_Logistic()
print(loss_func(model(xb), yb))
</code></pre>
<p>, <code>model(xb)</code> uses the <code>forward</code> method automatically. Why can we do that instead of doing something like <code>print(loss_func(model.forward(xb),yb))</code>?</p>
<p>Thank you!</p>

ptrblck: <p>The recommended way is to call the model directly, which will execute the <code>__call__</code> method as seen in <a href="https://github.com/pytorch/pytorch/blob/472be69a736c0b2aece4883be9f8b18e2f3dfbbd/torch/nn/modules/module.py#L487" rel="nofollow noopener">this line of code</a>.<br>
This makes sure that all hooks are properly registered and calls <code>forward</code> afterwards.</p>

Asgard: <p>My PyTorch method isn’t automatically calling the forward method.</p>
<p>I’m trying to embed my graph adjacency matrix by aggregating neighbours and combining them (similar to GraphSAGE)</p>
<p>An adjacency matrix is of size nXn and the embedding will be of size nXd where d&lt;n.</p>
<p>So, basically in my code, an adjacency matrix of a graph is fed as input for the purpose of embedding and the forward method should return the embedding.</p>
<pre><code class="lang-python">
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import numpy as np

class Setting(nn.Module):
    
    def __init__(self, A):
        
        super(Setting, self).__init__()
        
        self.A = A
        self.X = np.array(np.sum(A, axis=1))
        self.feature_len = 10

        self.L = 3
        self.n = 10
        self.z_dim = 8
        
        self.h = torch.empty(self.L-1,self.n, self.z_dim)
        
        self.h0 = torch.from_numpy(self.X)
        
        W0 = nn.init.xavier_uniform_(torch.empty(self.feature_len, self.z_dim))
        
        self.h[0] = torch.empty(self.n, self.z_dim)

        for v in range(self.n):
            self.h[0][v] = F.relu(self.h0[v]*W0)
        
    def forward(self, A):
        
        d_u = np.empty([self.n, self.n])
        
        for v in range(self.n):
            d_u[v] = self.X[v]*self.A[v]

        d_u = torch.from_numpy(d_u)        
                
        h_n = torch.empty(self.L-1, self.n, self.z_dim)

        rnn = nn.GRUCell(self.z_dim, self.z_dim)

        H = F.normalize(self.h[0], p=2, dim=1)
        
        for l in range(0,self.L-1):
            
            h_n[l] = torch.mm(d_u, self.h[l-1].double())
            self.h[l] = rnn(self.h[l-1], h_n[l]).detach().float()
            self.h[l] = F.normalize(self.h[l], p=2, dim=1)
                        
            H = torch.max(H, self.h[l])
   
        return H

</code></pre>
<p>Now based on the value of H, I want to train the weight W0, by taking MSELoss of H and H1 (another already known embedding).</p>
<pre><code class="lang-python">net = Setting(A).double()
loss = nn.MSELoss()
optimizer = torch.optim.Adam(Setting.parameters(), lr=0.1)

for epoch in range(10):
    H = net(A)
    loss_calc = loss(H, H1)
    loss_calc.backward()
    optimizer.zero_grad()

print(H)
</code></pre>
<p>How to train the weight W0 based on this architecture? Any modifications in the code are highly appreciated.</p>
<p>Thanks in advance.</p>

ptrblck: <p>Double post from <a href="https://discuss.pytorch.org/t/forward-method-call-for-weight-training/84314">here</a>. Let’s continue the discussion in the created topic please.</p>