How to do projected gradient descent?

sakuraiiiii: <p>Hi, I want to do a constrained optimization with <code>PyTorch</code>. I want to find the minimum of a function $f(x_1, x_2, \dots, x_n)$, with \sum_{i=1}^n x_i=5 and x_i \geq 0. I think this could be done via <code>Softmax</code>. So I follow the <a href="https://discuss.pytorch.org/t/how-to-do-constrained-optimization-in-pytorch/60122/2" class="inline-onebox">How to do constrained optimization in PyTorch</a></p>
<pre><code class="lang-python">import torch
from torch import nn

x = torch.rand(2)
x.requires_grad = True
lin = nn.Linear(2, 1)
optimizer = torch.optim.Adam([x], lr=0.1)
for i in range(100):
    optimizer.zero_grad()
    y = lin(x)
    y.backward()
    optimizer.step()

    with torch.no_grad():
        x = nn.Softmax(dim=-1)(x) * 5
</code></pre>
<p>If <code>print(y)</code> in each step,the output is:</p>
<pre><code class="lang-python">tensor([-0.2826], grad_fn=&lt;AddBackward0&gt;)
tensor([-0.9759], grad_fn=&lt;AddBackward0&gt;)
tensor([-0.9794], grad_fn=&lt;AddBackward0&gt;)
tensor([-0.9880], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0062], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0284], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0360], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
...
</code></pre>
<p>The following value does not change.</p>
<pre><code class="lang-python">for i in lin.parameters():
    print(i)
</code></pre>
<p>The weight of the linear layer is</p>
<pre><code class="lang-python">Parameter containing:
tensor([[-0.2240, -0.1983]], requires_grad=True)
Parameter containing:
tensor([0.0823], requires_grad=True)
</code></pre>
<p>So the formula is $y=-0.224<em>x_1-0.1983</em>x_2+0.0823=-0.0257*x_1-0.9092$,whose minimum should be -1.0377. Is the above code wrong? How to modify? Thank you very much.</p>

albanD: <p>Hi,</p>
<p>Is there a reason you use softmax instead of clamp? That is not doing the projection of pgd right?</p>

KFrank: <p>Hiiiii Sakuraiiiii!</p>
<aside class="quote no-group quote-modified" data-username="sakuraiiiii" data-post="1" data-topic="85909" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sakuraiiiii/48/19481_2.png" class="avatar"> sakuraiiiii:</div>
<blockquote>
<p>I want to find the minimum of a function $f(x_1, x_2, \dots, x_n)$, with \sum_{i=1}^n x_i=5 and x_i \geq 0. I think this could be done via <code>Softmax</code>.</p>
<pre data-code-wrap="python"><code class="lang-python">    with torch.no_grad():
        x = nn.Softmax(dim=-1)(x) * 5
</code></pre>
<p>If <code>print(y)</code> in each step,the output is:</p>
<pre data-code-wrap="python"><code class="lang-python">...
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
tensor([-1.0368], grad_fn=&lt;AddBackward0&gt;)
</code></pre>
<p>So the formula is $y=-0.224<em>x_1-0.1983</em>x_2+0.0823=-0.0257*x_1-0.9092$,whose minimum should be -1.0377.</p>
</blockquote>
</aside>
<p>The short answer is that <code>Softmax</code> isn’t the right way to enforce your<br>
constraint.</p>
<p>The medium-short answer is that you’re getting almost the right<br>
answer, but for the wrong reason.</p>
<p>The somewhat longer short answer is that <code>Softmax</code> isn’t the right<br>
way to enforce your constraint.</p>
<p>Some details:</p>
<p>Because you’re minimizing a linear function with respect to linear<br>
constraints, the minimum will occur when the constraints are<br>
saturated.</p>
<p>Specifically, as you have worked out, the minimum occurs when<br>
<code>x_1 = 5.0</code> and <code>x_2 = 0.0</code>, and the minimum is <code>-1.0377</code>.</p>
<p>The short-answer details are that your constraint enforcement<br>
doesn’t map your solution back to itself:</p>
<pre data-code-wrap="python"><code class="lang-python">5.0 * softmax ([5.0, 0.0]) = [4.9665, 0.0335]
</code></pre>
<p>and if you iterate <code>5.0 * softmax()</code> several times, you get<br>
<code>[4.9641, 0.0359]</code>.  The value of your function for this value<br>
of <code>(x_1, x_2)</code> is <code>-1.03678</code>, as returned by your training.</p>
<p>Each step of your training is, in effect, iterating your constraint<br>
enforcement.  (For reasons I won’t go into, your constraint<br>
enforcement pretty much washes away the changes in <code>x_1</code><br>
and <code>x_2</code> made by your optimizer step.)</p>
<p>Why are you getting almost the right answer?  <code>Softmax</code> pushes<br>
one value of <code>x_i</code> (the largest) towards <code>1</code>, and the others towards<br>
<code>0</code>.  This is where your constraints are saturated, and therefore where<br>
the minimum lies.  That is, in your two-variable case, <code>5 * Softmax</code><br>
is pushing <code>x_1</code> close to <code>5</code> (specifically to 4.9641), and <code>x_2</code> close<br>
to <code>0</code>.</p>
<p>The longer answer:</p>
<p>Even though <code>5 * Softmax</code> does enforce your constraint (variables<br>
are greater than <code>0</code> and sum to <code>5</code>), it is, in general, not the right way<br>
to enforce such constraints.</p>
<p>The problem (among other things) is that applying <code>Softmax</code> a second<br>
time changes the <code>x_i</code> a second time.  If your constraint is already<br>
satisfied, reapplying the constraint shouldn’t changes things.</p>
<p>Related to this, if you’re at the correct minimum (<code>[5, 0]</code>) applying<br>
the constraint moves you <em>away</em> from the minimum (even though<br>
the constraint was properly satisfied).  This clearly isn’t what you<br>
want.  (It’s only by luck that because of the specific function you<br>
were minimizing you got almost the right answer using <code>Softmax</code>.)</p>
<p>To enforce your constraint, I would recommend something like:</p>
<pre data-code-wrap="python"><code class="lang-python">x_i = (5.0 - sum (x_i)) / n
x_i = max (0.0, x_i)
x_i = (5.0 / sum (x_i)) * x_i
</code></pre>
<p>(As an aside, you can also add terms to your loss function that push<br>
the <code>x_i</code> in the direction of satisfying your constraints, and in the<br>
case of your equality constraint, <code>sum (x_i) = 5</code>, you could add<br>
a <em>Langrange-multiplier</em> term to your loss function to enforce the<br>
equality constraint.)</p>
<p>Good luck.</p>
<p>K. Frank</p>

sakuraiiiii: <p>Thank you very much. That’s very clear.</p>