How to program y = x ^ 2 using MLP?

migu3l705: <p>Hello, I am new in pytorch, I need help, how can I program a multilayer perceptron whose output is the function y = x ^ 2, starting from x = […- 2, -1,0,1,2 …]<br>
I have tried, but I have only been able to get linear functions, like y = a * x + b</p>

ptrblck: <p>I’m not sure I understand the use case correctly, but your operation doesn’t seem to contain trainable parameters.</p>
<p>You could create the input tensor and just call <code>x**2</code> to get the output:</p>
<pre><code class="lang-python">x = torch.linspace(-10, 10, 21)
y = x**2
</code></pre>
<p>However, since no parameter is used, this won’t be trainable.</p>

migu3l705: <p>In the case of y = 3x, for example, it could be y = w1 * x, where w1 must be trained until reaching the value of 3, but in y = x ^ 2, I can’t find which parameter to train, I need help.</p>

ptrblck: <p>That’s exactly what I mean. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
If your target function is <code>x**2</code>, then there is nothing to train or would you like to make the exponent trainable?</p>

migu3l705: <p>the objective is to enter any value into the NN and that it returns, as output, the square of that number.</p>

ptrblck: <p>So should the exponent be trained?<br>
If so, you could define it as a parameter and try to optimize it.<br>
However, you would have to take care of negative input values (root of then yields NaN) and probably using the log will be more stable:</p>
<pre><code class="lang-python">e = nn.Parameter(torch.empty(1).uniform_(0, 1))

data = torch.linspace(1, 10, 9)
target = torch.log(data**2)

optimizer = torch.optim.SGD([e], lr=1e-3)
criterion = nn.L1Loss()

for epoch in range(1000):
    optimizer.zero_grad()
    out = e * torch.log(data)
    loss = criterion(out, target)
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}, e {}'.format(epoch, loss.item(), e.item()))
</code></pre>

migu3l705: <p>I thought about training the exponent, or better said, optimizing it, but I didn’t get anywhere, besides that I didn’t use MPL. I will test the code you uploaded, thanks.</p>

KFrank: <p>Hello Miguel!</p>
<aside class="quote no-group" data-username="migu3l705" data-post="1" data-topic="73850" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/migu3l705/48/21898_2.png" class="avatar"> migu3l705:</div>
<blockquote>
<p>how can I program a multilayer perceptron whose output is the function y = x ^ 2, starting from x = […- 2, -1,0,1,2 …]</p>
</blockquote>
</aside>
<p>Let me speculate a little about what you are asking.</p>
<p>As a learning exercise you might be asking how to train a neural<br>
network to reproduce the function <code>x^2</code> <em>without</em> building into it by<br>
hand any knowledge of that specific function.</p>
<p>One of the interesting and important features of neural networks<br>
is that their linear layers plus non-linear activations can be used<br>
to reproduce / approximate  many interesting functions.  See, for<br>
example, the <a href="https://en.wikipedia.org/wiki/Universal_approximation_theorem" rel="noopener nofollow ugc">“Universal approximation theorem”</a> wikipedia article.</p>
<p>I haven’t experimented with this in particular, but you might try<br>
training a network like (just making some something up):</p>
<pre data-code-wrap="python"><code class="lang-python">model = torch.nn.Sequential (
    torch.nn.Linear (1, 50),
    torch.nn.Tanh(),
    torch.nn.Linear (50, 50),
    torch.nn.Tanh(),
    torch.nn.Linear (50, 1)
)
</code></pre>
<p><code>torch.nn.MSELoss</code> would be the appropriate loss function.</p>
<p>(Your inputs would be your various values of <code>x</code>, e.g., your<br>
[…- 2, -1,0,1,2 …].  Your targets would be the corresponding <code>x^2</code>.)</p>
<p>You could also try other activations, e.g., <code>torch.nn.ReLU</code>, and<br>
wider / narrower or more / fewer hidden layers.</p>
<p>If you get such a network working for <code>x^2</code> it would be informative<br>
to retrain it (from scratch) on something like <code>abs (x)^3</code>.</p>
<p>Good luck.</p>
<p>K. Frank</p>

jsswoosh: <p>I am facing a similar issue. I am not able to train a MLP model for y=x**2 data<br>
Here is my Code. Please help.</p>
<blockquote>
<p>def train(epoch=1000):<br>
t = np.arange(0,1,0.001)   # 1000 points<br>
sq = np.square(t)<br>
X_t, X_v, Y_t, Y_v = train_test_split(t, sq, test_size=0.2, random_state=1)<br>
print(X_t.shape)<br>
model = nn.Sequential(<br>
nn.Linear(1, 8), nn.ReLU(),<br>
nn.Linear(8, 32), nn.ReLU(),<br>
nn.Linear(32, 64), nn.ReLU(),<br>
nn.Linear(64, 32), nn.ReLU(),<br>
nn.Linear(32, 4), nn.ReLU(),<br>
nn.Linear(4, 1),<br>
).type(torch.DoubleTensor)<br>
loss = nn.MSELoss().type(torch.DoubleTensor)<br>
optim = torch.optim.Adam(model.parameters(), lr=0.01)<br>
#<br>
for i in range(epoch):<br>
model.train()<br>
pred = model(torch.tensor(X_t).unsqueeze(1))<br>
l = loss(pred.squeeze(), torch.tensor(Y_t) )<br>
l.backward()<br>
optim.step()<br>
optim.zero_grad()<br>
if i%100 ==0:<br>
print(f’loss-{l} at epoch:{i}')</p>
</blockquote>
<p>train()</p>