Problem with maxium likelihood implementation in PyTorch

Anthony_197: <p>Recently I am learning to use PyTorch to solve a maximum likelihood problem as described below, and I got a problem with the updates of the parameters.</p>
<ol>
<li>Data Description<br>
The data comes from the function<code>sample_data</code>. It contains 10000 observed value of a random variable, each value denotes the interval it lies.</li>
</ol>
<pre><code class="lang-auto">import numpy as np 

def sample_data():
    count = 10000
    rand = np.random.RandomState(0)
    a = 0.3 + 0.1 * rand.randn(count)
    b = 0.8 + 0.05 * rand.randn(count)
    mask = rand.rand(count) &lt; 0.5
    samples = np.clip(a * mask + b * (1 - mask), 0.0, 1.0)
    return np.digitize(samples, np.linspace(0.0, 1.0, 100))
</code></pre>
<ol start="2">
<li>Problem Description<br>
for these observed data, I used the maximum likelihood method with the softmax function to fit the distribution of each interval from 0 - 100.<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/f/d/fd2e39a273c0fdb8eddc42840f57bffc37504dd1.png" alt="image75x50" data-base62-sha1="A7JPT7aG6xKv7tYp1j4DWSufQOZ" width="277" height="164"></li>
</ol>
<p>Here x_i is an One-hot encoding vector of the same size with θ, and my reasoning processing for the maximum likelihood is in the below pic.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/6/2654af1efc8582c473af3594d9a9febd598ee264.png" data-download-href="https://discuss.pytorch.org/uploads/default/2654af1efc8582c473af3594d9a9febd598ee264" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2654af1efc8582c473af3594d9a9febd598ee264_2_500x500.png" alt="image" data-base62-sha1="5t5zSlbQDBCN3kYyi2GadoKk77m" width="500" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/6/2654af1efc8582c473af3594d9a9febd598ee264_2_500x500.png, https://discuss.pytorch.org/uploads/default/original/3X/2/6/2654af1efc8582c473af3594d9a9febd598ee264.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/6/2654af1efc8582c473af3594d9a9febd598ee264.png 2x" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">625×540 27.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<ol start="3">
<li>Problem with PyTorch implementation</li>
</ol>
<p>Here is my implementation for this problem</p>
<pre><code class="lang-auto">import torch 
import torch.nn as nn 
from collections import Counter

def sum_x(x):
    dict_item = Counter(x)
    keys_item = dict_item.keys()
    input_of_x = np.zeros((100, 1))
    for key in keys_item:
        input_of_x[key, 0] = dict_item[key]
    return input_of_x

def loss_function(theta, x_fre):
    x = torch.from_numpy(x_fre).float()
    loss = -1/8000 * torch.mm(theta, x) + torch.log(torch.mm(torch.exp(theta), x))
    return loss

X = sample_data()
X_train, X_test = X[:8000], X[8000:]
x_fre = sum_x(X_train)
loss_list = []
theta = torch.zeros((1, 100), requires_grad = True).float()
optimizer = torch.optim.Adam([theta], lr=0.001)

for index in range(1000):
    loss = loss_function(theta, x_fre)
    loss.backward()    
    optimizer.step()
    optimizer.zero_grad()
    loss_list.append(loss)



    
plt.plot(range(len(loss_list)), loss_list)
plt.show()
</code></pre>
<p>and the result is show as below<br>
<img src="https://discuss.pytorch.org/uploads/default/original/3X/9/2/92b7529270c5a9236644877b7f86b670a967c8c7.png" alt="image" data-base62-sha1="kVUtBfckPwRj25ECjRwVXpnzyfB" width="300" height="300"></p>
<p>but the prob distribution  should have the shape like mixed of two gaussian for<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/3/4/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f.png" data-download-href="https://discuss.pytorch.org/uploads/default/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f_2_690x426.png" alt="image" data-base62-sha1="7x7K5MxyLt3m8E88OhupnXZ7c3Z" width="690" height="426" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/3/4/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f_2_690x426.png, https://discuss.pytorch.org/uploads/default/original/3X/3/4/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/3/4/34cdcbaa83d38c2f9a76d9f048ffd01e5495379f.png 2x" data-dominant-color="F5F8FA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">756×467 43.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div> ,</p>
<p>It makes me confusing for days. Is anywhere I made a mistake?</p>
<p>Thanks for anyone who can help me with this.</p>

111364: <p>Hi Anthony, do you solve this problem? I have similar problen and as I think that weights didnt updated</p>

tom: <p>I think that, unfortunately, the program as described has both mathematical and PyTorch errors to make it quite a riddle what is meant.</p>
<ul>
<li>The derivation of the second term of the loss function is not broken. As a result, I would expect to see <code>logsumexp(theta + x, -1)</code> for some appropriately shaped matrix in the loss function.</li>
<li>You would want to clamp the reference probabilities away from 0 to avoid -inf negative log likelihood.</li>
<li>
<code>theta = torch.zeros((1, 100), requires_grad = True).float()</code> makes theta a non-leaf, which is broken. You really, absolutely need to avoid calling any function between constructor and assignment, so here (if it were not the default) one could use use dtype=torch.float in the constructor.</li>
<li>There are several other oddities / <code>don't dos</code>. e.g. you should only collect <code>loss.item()</code>, the order of optimizer steps is OK, but the usual way is <code>zero_grad</code>, <code>backward</code>,  <code>step</code> etc.</li>
</ul>
<p>In summary, I would recommend to re-do the derivation unless Anthony has an update that makes the intention and code clearer.</p>
<p>Best regards</p>
<p>Thomas</p>