Training routine inside model definition class

alpapado: <p>Hello to all,</p>
<p>Suppose I have defined my model as a standard class that inherits torch.nn.Module.<br>
I was wondering if it is possible to encapsulate the training routine of my model as a function of this class and to<br>
perform training by creating a model object and then calling the relevant function</p>
<p>In other words, is the following possible (or good practice)?</p>
<pre><code class="lang-auto">class MyModel(torch.nn.Module):
    def __init__():
        blah

    def __forward__():
        blah

    def fit(num_epochs):
    """ Perform training for a number of epochs """
        blah

my_model = MyModel()
my_model.fit()
</code></pre>
<p>Thanks a lot</p>

justusschock: <p>Yes it is possible. In general: nearly everything that can be done with classes can be done by inheriting <code>torch.nn.Module</code> and defining the missing things yourself.</p>
<p>Whether it is good practice is hard to tell. If it helps, I can ensure you, that I have done something similar quite often as it is one of the easiest ways to define a API. Another approach would be to only define a closure for each network. I’ve done this to write a generic trainer class.</p>

ioannisa92: <p>Hello, I am new to the community. Excited to be here.</p>
<p>As i am a new user of pytorch facing a similar problem, could someone provide an example on how the train method would look like inside the class?</p>
<p>For example in my code below i call self.forward() in the fit method, but is that correct? Calling self.model throws the following error: TypeError: forward() takes 1 positional argument but 2 were given</p>
<pre><code class="lang-auto">class AE(nn.Module):

    def __init__(self,h=[500,300,100,300,500], x_dim=None, dropout=0.1, lr=0.1):
        super(AE,self).__init__()

        self.x_dim = x_dim
        self.h = h
        self.n_layers = len(h)
        self.dropout = dropout
        self.lr = lr

        x=self.x_dim
        self.model = nn.ModuleDict()
        for i in np.arange(self.n_layers):
            n_nodes = self.h[i]
            if i == int(self.n_layers/2):

                self.model['latent_layer'] = nn.Linear(x, n_nodes, bias=True)
            else:
                self.model['hidden%d'%i] = nn.Linear(x, n_nodes, bias=True)
            x = n_nodes

        self.output = nn.Linear(x, self.x_dim,bias=True)

    def forward(self,x):

        for name,layer in self.model.items():
            x = F.relu(layer(x))

        output = self.output(x) # linear output
        return output


    def optimizer(self):

        return optim.Adam(self.model.parameters(),lr=self.lr)

   def fit(self,loader, epochs=10, batch_size=128):

        loss = RMSELoss()
        opt = self.optimizer()
        n_minibatches = len(loader)

        print('training on %d samples'%(n_minibatches*batch_size))
        for epoch in range(epochs):
            epoch_loss = 0

            for batch_x, batch_y in loader:
                #print(batch_x.shape, batch_y.shape)
                prediction = self.forward(batch_x.float()) 
                batch_loss = loss(prediction,batch_y.float())

                opt.zero_grad()   # clear gradients for next train
                batch_loss.backward()         # backpropagation, compute gradients
                opt.step()        # apply gradients

                epoch_loss += batch_loss.item()

            print('[Epoch %d] loss: %.3f'%(epoch+1, epoch_loss/n_minibatches))
</code></pre>
<p>Any help would be appreciated!<br>
Thank you for your time</p>

ptrblck: <p>You could call <code>self(batch_x.float())</code> directly, which would be similar to the <code>model(data)</code> call, as it will call into <code>.__call__</code> directly.</p>

jrepo: <p>Hi everyone,<br>
I’m quite new to pytorch and I have a question, regarding this topic.</p>
<p>If, as I read from <a class="mention" href="/u/justusschock">@justusschock</a> and <a class="mention" href="/u/ptrblck">@ptrblck</a>, one could implement a training routine inside model definition class, is there a reason why generally you see only the <code>forward</code> function defined and not the <code>train</code>, <code>evalutate</code> etc? Is it related somehow to performance?</p>

ptrblck: <p>I think it depends on your coding style and which abstractions you would like to use.<br>
E.g. I personally prefer to isolate the model, optimizer, loss function, and write a training routine using these objects.<br>
You could pass the optimizer and loss function to the model and use a class function for training, if you think making these objects part of the model is clearer.</p>

Jack_Lovell: <p><a class="mention" href="/u/ptrblck">@ptrblck</a> I am trying to do this now, do you have any examples? I’m having some serious trouble calling an instance of my model in the training script and giving it the attributes I’d like</p>

Jack_Lovell: <p>I found this guy: <a href="https://github.com/ptrblck/DiscoGAN/blob/master/discogan/image_translation.py" class="inline-onebox" rel="noopener nofollow ugc">DiscoGAN/image_translation.py at master · ptrblck/DiscoGAN · GitHub</a></p>
<p>Instead could you write this as a class <code>Train(model)</code> where in the constructor <code>self.args = args</code> just gets the arguments from <code>argparse</code> and you can then write a method that uses a similar training loop to the one you have? <code>model</code> can be any of the objects in <code>model.py</code>.</p>
<p>I am trying to do this, and keep getting an attributerror after calling an instance of my model using <code>super().__init__(args)</code> within the constructor of my <code>Train</code> object…</p>

ptrblck: <p>I don’t have a code snippet ready, as I’m not using the approach of putting everything in a “global” class.<br>
What kind of errors are you seeing using your approach and could you post a code snippet, which would reproduce this error?</p>