My very simpe net don't learn

piodd: <p>I made a simple net and very simple data. (just want learn pytorch) .</p>
<p>data<br>
def make_data(nr_of_data):<br>
x_list = []<br>
y_list = []<br>
for i in range(nr_of_data):<br>
x = [random.randint(0, 1), random.randint(0, 1)]<br>
y = x[0]<br>
x_list.append(x)<br>
y_list.append(y)<br>
return x_list, y_list</p>
<p>model<br>
class LinearClassifier(nn.Module):<br>
def <strong>init</strong>(self, lr, n_classes, input_dims):<br>
super(LinearClassifier, self).<strong>init</strong>()<br>
self.linear_1 = nn.Linear(input_dims, 128)<br>
self.linear_2 = nn.Linear(128, 256)<br>
self.linear_3 = nn.Linear(256, n_classes)<br>
self.optimizer = optim.Adam(self.parameters(), lr=lr)<br>
self.loss = nn.CrossEntropyLoss()<br>
self.device = T.device(“cuda:0” if T.cuda.is_available() else “cpu”)<br>
self.to(self.device)</p>
<pre><code>def forward(self, data):
    result = F.relu(self.linear_1(data))
    result = F.relu(self.linear_2(result))
    result = self.linear_3(result)
    return result

def learn(self, data, labels):
    self.optimizer.zero_grad()
    data = T.tensor(data).to(self.device)
    labels = T.tensor(labels).to(self.device)
    predictions = self(data)
    cost = self.loss(predictions, labels)
    cost.backward()
    self.optimizer.step()
</code></pre>
<p>and model just can’t learn that simple data .<br>
why ?</p>

AbdulsalamBande: <p>I will highly recommend you go with this tutorial instead.</p>
<p>(<a href="https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Neural Networks — PyTorch Tutorials 1.11.0+cu102 documentation</a>)</p>

eqy: <p>How are you observing that your net isn’t learning? Are the parameters not changing, loss isn’t decreasing, or via some other metric?</p>
<p>I’d consider checking if the model can overfit a small dataset (e.g., a single batch) with a very small learning rate as a sanity check.</p>

piodd: <p>i made very simple dada (you can see code to create it )<br>
and give 100 epoch for learn and get 0 accurace increase<br>
(loss decrese )</p>

eqy: <p>How are you measuring the accuracy and is the test data the same as the training data? (I wouldn’t expect a model to be able to generalize from one random dataset to another).</p>

piodd: <blockquote>
<p>X, y = make_data(50)<br>
X = torch.FloatTensor(X)<br>
y = torch.tensor(y)</p>
</blockquote>
<blockquote>
<p>for i in range(100):<br>
linear_classifier.learn(X, y)<br>
prediction = linear_classifier(X)<br>
c = 0<br>
good = 0<br>
for a in zip(prediction, y):<br>
c += 1<br>
if np.argmax([0]) == a[1]:<br>
good += 1<br>
print(“acc==”,good/c)</p>
</blockquote>
<p>And i use only train data (i just want learny torch “API” so i want just overfit very simple data but i can’t XD )</p>

piodd: <p><img src="https://discuss.pytorch.org/uploads/default/original/3X/3/0/30c30cbfe576f7ec095c3a4a9bc73c2f63059f18.png" alt="image" data-base62-sha1="6XmOgW1xIAPcU0DCsgL7XSPgYgo" width="289" height="365"></p>
<p>sorry i can’t edit text . My first post (:</p>

piodd: <p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/0/50fdb4098dc9761c214fadc0ba4572ee9270ac59.png" data-download-href="https://discuss.pytorch.org/uploads/default/50fdb4098dc9761c214fadc0ba4572ee9270ac59" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50fdb4098dc9761c214fadc0ba4572ee9270ac59_2_493x500.png" alt="image" data-base62-sha1="bytKLRtyOUhn8gksGLD79izuUIh" width="493" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50fdb4098dc9761c214fadc0ba4572ee9270ac59_2_493x500.png, https://discuss.pytorch.org/uploads/default/optimized/3X/5/0/50fdb4098dc9761c214fadc0ba4572ee9270ac59_2_739x750.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/0/50fdb4098dc9761c214fadc0ba4572ee9270ac59.png 2x" data-dominant-color="2F2F2F"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">785×796 61.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

piodd: <p>Ach and data is not random . The target is just a first argument of data y=x[0]</p>

eqy: <p>I believe there is just a small typo in the code, please try this loop instead:</p>
<pre><code class="lang-auto">for i in range(100):
    linear_classifier.learn(X, y)
    with torch.no_grad():
        prediction = linear_classifier(X)
    c = 0
    good = 0
    for a in zip(prediction, y):
        c += 1
        if np.argmax(a[0]) == a[1]:
            good += 1
    print("acc==",good/c)
</code></pre>
<p>(argmax of just 0 was called in the original)<br>
no_grad is added to make the numpy conversion happy</p>
<p>Note that you can also use <code>torch.argmax</code> with <code>axis=1</code> instead of numpy as well.</p>

piodd: <p>Och Seriussly i just make that typo (argmax(a[0])  . I spend few hours.  THX</p>