Multiply sub-model by known input parameters

Jacky_Chen: <p>Hi I’m a newbie in Pytorch.<br>
I try to create a trainable model having a fixed multiplication and addition operation to the sub-models, which shown as below.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5cc3a3ef9d64870b716ded14477ebf74cd991833.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/5cc3a3ef9d64870b716ded14477ebf74cd991833" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cc3a3ef9d64870b716ded14477ebf74cd991833_2_626x500.jpeg" alt="image" data-base62-sha1="deD5gavq8OlYm7mXhuQzv84nZqH" width="626" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cc3a3ef9d64870b716ded14477ebf74cd991833_2_626x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cc3a3ef9d64870b716ded14477ebf74cd991833_2_939x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cc3a3ef9d64870b716ded14477ebf74cd991833_2_1252x1000.jpeg 2x" data-dominant-color="F6F5F3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2510×2004 346 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>Here tensor X is training data, and var_in is also an input tensor varying by each set of training data used to multiple the output of the sub-model.</p>
<p>Below is my attempt. But I don’t get a reasonable result.<br>
Is there anything wrong here? Or is it possible to achieve a model like this?<br>
Thanks for any suggestion.</p>
<pre><code class="lang-auto">
class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.l1_1 = nn.Linear(3, 600)
        self.l2_1 = nn.Linear(600, 200)
        self.l3_1 = nn.Linear(200, 1)

        self.l1_2 = nn.Linear(3, 600)
        self.l2_2 = nn.Linear(600, 200)
        self.l3_2 = nn.Linear(200, 1)

        self.l1_3 = nn.Linear(3, 600)
        self.l2_3 = nn.Linear(600, 200)
        self.l3_3 = nn.Linear(200, 1)


    def forward(self, x, var_in):
        out_1 = F.relu(self.l1_1(x))
        out_1 = F.relu(self.l2_1(out_1))
        out_1 = self.l3_1(out_1)

        out_2 = F.relu(self.l1_2(x))
        out_2 = F.relu(self.l2_2(out_2))
        out_2 = self.l3_2(out_2)

        out_3 = F.relu(self.l1_3(x))
        out_3 = F.relu(self.l2_3(out_3))
        out_3 = self.l3_3(out_3)

        Finalout = var_in[:,0].view(-1,1) * out_1 + var_in[:,1].view(-1,1) * out_2 + var_in[:,2].view(-1,1) * out_3

        return Finalout


y_pred = model.forward(TrainData_input,Var_in)
l = loss(y_pred, TrainData_output)
l.backward()
</code></pre>

ptrblck: <p>Your code looks correct besides the explicit <code>forward</code> call, which is unrelated to your issue. Call the model instead via <code>y_pred = model(TrainData_input, Var_in)</code> to make sure all hooks are used. Note that this is a general recommendation and won’t fix your code as no hooks are used.</p>

Jacky_Chen: <p>Hey ptrblck,<br>
Thanks for your reply.<br>
I’m doubting this code has some problems because that I use the same training data to verify the model’s prediction, the model output is off (it yield to a similar shape but with a large offset) although the loss become small (like 0.001) after 10000 epoch of learning.</p>