Find global minimun with GD

username: Tomer_Boyarski
cooked: <p>I have trained a NN with a vector input and scalar output (regression).<br>
Now I want to find the global minimun of the NN using GD with pytorch.</p>
<p>I’m new to programming in general, python specifically, and pytorch even more specifically.</p>
<p>I believe what I’m trying to do must have been done a thousand times before, if not ten thousand times. I’ll be super happy and grateful if anyone could point me to some code somewhere (maybe in github) where there’s an example of what I’m trying to do that I could adjust to my needs.</p>

username: Tomer_Boyarski
cooked: <p>meaning:<br>
I want to know what inputs will give the minimum (maximum) output</p>

username: rasbt
cooked: <blockquote>
<p>Now I want to find the global minimun of the NN using GD with pytorch.<br>
I believe what I’m trying to do must have been done a thousand times before, if not ten thousand times.</p>
</blockquote>
<p>Have a look at this example here:</p>
<p><a href="https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-tensors-and-autograd" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/beginner/pytorch_with_examples.html#pytorch-tensors-and-autograd</a></p>
<p>It is updating the parameters manually in the step</p>
<pre><code>with torch.no_grad():
    w1 -= learning_rate * w1.grad
    w2 -= learning_rate * w2.grad
</code></pre>
<p>to minimize the loss function, but you can also use the SGD optimizer, which is explained here: <a href="https://pytorch.org/docs/stable/optim.html#module-torch.optim" class="inline-onebox" rel="noopener nofollow ugc">torch.optim — PyTorch 2.1 documentation</a></p>
<p>If you want to do GD specifically, not SGD, then each batch would basically be your entire training set (if you can fit it into memory). Otherwise, just accumulate the error before you do the manual update</p>

username: Tomer_Boyarski
cooked: <p>thank you very much !<br>
I have changed the code you sent me so as to fit my problem.<br>
my problem is not to train the net parameters, as in the code you sent, but rather to “train” the input.<br>
in other words, after having trained my net, I want to find the input that will give maximum output.</p>
<p>the chages I did are as follows:</p>
<ol>
<li>I added ‘requires grad’ to the input</li>
<li>I changed the loss to be ‘-output’ (the output with minus sign) because I’m looking the maximum output</li>
<li>Instead of updating the parameters (w1, w2 in the example) I’m updating the input (x in the example)<br>
here is the code with my changes:</li>
</ol>
<p>I workes well.</p>
<p>however, what I want to do now is to load my pre-trained net and then instead of:<br>
y_pred = x.mm(w1).clamp(min=0).mm(w2)</p>
<p>I will have:<br>
y_pred = net(x)</p>
<p>however, this does not work for some reason. meaning, even after the ‘.backward()’ command the ‘.grad’ of ‘x’ remains ‘None’. note this does not happen if I do:<br>
y_pred = x.mm(w1).clamp(min=0).mm(w2)<br>
!!!</p>
<p>very strange…</p>
<p>I don’t understand why with these ‘.mm’ commands the ‘.grad’ of inputs is being calculated and saved and with ‘net()’ it isn’t happening</p>
<p>here is what my net looks like (it has 3 hidden layers, input layer size is 11, output layer size is 1, all 3 middle layers have a size of 100):<br>
class non_linear_Net_3_hidden(nn.Module):<br>
    def <strong>init</strong>(self):<br>
        super(non_linear_Net_3_hidden, self).<strong>init</strong>()<br>
        self.fc1 = nn.Linear(input_layer_size, middle_layer_size)<br>
        self.fc2 = nn.Linear(middle_layer_size, middle_layer_size)<br>
        self.fc4 = nn.Linear(middle_layer_size, middle_layer_size)<br>
        self.fc3 = nn.Linear(middle_layer_size, final_layer_size)<br>
    def forward(self, x):<br>
        x = F.sigmoid(self.fc1(x))<br>
        x = F.sigmoid(self.fc2(x))<br>
        x = F.sigmoid(self.fc4(x))<br>
        x = self.fc3(x)<br>
        return x</p>

username: rasbt
cooked: <p>I see, so in this case you can take the same approach but go into the opp direction of the gradient of the input features rather than the weights. I think that’s what you are doing now.</p>
<blockquote>
<p>I will have:<br>
y_pred = net(x)</p>
</blockquote>
<blockquote>
<p>however, this does not work for some reason. meaning, even after the ‘.backward()’ command the ‘.grad’ of ‘x’</p>
</blockquote>
<p>Don’t know why it wouldn’t work, but you sure do have a lot of x’s there <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>
<pre data-code-wrap="def"><code class="lang-def">x = F.sigmoid(self.fc1(x))
x = F.sigmoid(self.fc2(x))
x = F.sigmoid(self.fc4(x))
x = self.fc3(x)
return x
</code></pre>
<p>To disentangle that a bit, I would make sure that you keep your input “x” distinct from the rest and see if that solves the problem</p>

username: Tomer_Boyarski
cooked: <p>thanks!<br>
by ‘disentangle’, do you mean something like this:</p>
<p>def forward(self, x):<br>
    y = F.sigmoid(self.fc1(x))<br>
    y = F.sigmoid(self.fc2(y))<br>
    y = F.sigmoid(self.fc4(y))<br>
    y = self.fc3(y)<br>
    return y</p>
<p>have tried it. didn’t help.</p>
<p>here is something I’ve discovered which may shed some light on this problem:<br>
the output also remains with ‘grad=None’, even though it also has ‘requires grad = true’.</p>
<p>…</p>
<p>edit:<br>
I was given some more good advice and decided to do it like this:<br>
grad = torch.autograd.grad(loss, input)<br>
grad = torch.tensor(grad[0])<br>
input -= learning_rate * grad</p>
<p>which works, only now I get another problem:<br>
my input need to go through a softmax function (needs to be all positive and with a sum of 1)<br>
and for some reason it always becomes a one-hot-vector. (one element equals 1 and all the others 0)<br>
which makes the softmax function output ‘Nan’.<br>
every time it’s a different number which becomes 1 with all the others being zero.</p>

username: Tomer_Boyarski
cooked: <p>Ok. <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=6" title=":smile:" class="emoji" alt=":smile:"> thank you very much for your help. Everything works now. Have a great autumn.</p>

username: Ruiguo_Zhu
cooked: <p>It seems you had solved the problem. Could you show the solution in a brief way?</p>