Help me understand the pytorch translation of this mathemetical equation

username: Ashutosh_Dwivedi
cooked: <p>I am looking at the second problem of the homework solution from the course unsupervised learning <a href="https://github.com/rll/deepul/blob/master/homeworks/solutions/hw1_solutions.ipynb" rel="nofollow noopener">here</a>.<br>
Can someone explain how does the author convert this equation:</p>
<p>$p_\theta(x) = \sum_{i=1}^4 \pi_i[\sigma((x+0.5 - \mu_i)/s_i) - \sigma((x-0.5-\mu_i)/s_i)]$</p>
<p><a href="https://i.stack.imgur.com/oCJR8.png" rel="nofollow noopener"><img src="https://i.stack.imgur.com/oCJR8.png" alt="equation" width="690" height="73"></a></p>
<p>Into the following piece of code in Pytorch ( I am having trouble understanding the forward function).<br>
Specific questions that. I have are :</p>
<ul>
<li>Why is <code>log_space</code> which represents $s_i$ from the equations, a log value who’s <code>exp</code> we have to calculate instead of a simple vector of 4</li>
<li>Why do we need to calculate <code>log_cdf_delta</code> and why do we need to use <code>torch.clamp</code>
</li>
<li>Why do we need to calculate <code>x_log_probs</code> and why are we selecting different values based on <code>x &lt; 0.001</code>
</li>
</ul>
<pre><code class="lang-auto">
class MixtureOfLogistics(nn.Module):
  def __init__(self, d, n_mix=4):
    super().__init__()
    self.d = d
    self.n_mix = n_mix

    self.logits = nn.Parameter(torch.zeros(n_mix), requires_grad=True)
    self.means = nn.Parameter(torch.arange(n_mix).float() / (n_mix - 1) * d, 
                              requires_grad=True)
    self.log_scales = nn.Parameter(torch.randn(n_mix), requires_grad=True)

  def forward(self, x):
    d = self.d
    x = x.float()
    x = x.unsqueeze(1).repeat(1, self.n_mix) # b x n_mix
    means, log_scales = self.means.unsqueeze(0), self.log_scales.unsqueeze(0) # 1 x n_mix
    inv_scales = torch.exp(-log_scales)

    plus_in = inv_scales * (x + 0.5 - means)
    min_in = inv_scales * (x - 0.5 - means)

    cdf_plus = torch.sigmoid(plus_in) # CDF of logistics at x + 0.5
    cdf_min = torch.sigmoid(min_in) # CDF of logistics at x - 0.5

    cdf_delta = cdf_plus - cdf_min # probability of x in bin [x - 0.5, x + 0.5]
    log_cdf_delta = torch.log(torch.clamp(cdf_delta, min=1e-12))
    log_cdf_plus = torch.log(torch.clamp(torch.sigmoid(inv_scales * (0.5 - means)),
                                                       min=1e-12))
    log_cdf_min = torch.log(torch.clamp(1 - torch.sigmoid(inv_scales * (d - 1.5 - means)),
                                        min=1e-12))

    x_log_probs = torch.where(x &lt; 0.001, log_cdf_plus,
                              torch.where(x &gt; d - 1 - 1e-3,
                                          log_cdf_min, log_cdf_delta))
    pi_log_probs = F.log_softmax(self.logits, dim=0).unsqueeze(0)
    log_probs = x_log_probs + pi_log_probs
    return torch.logsumexp(log_probs, dim=1)

  def loss(self, x):
    return -torch.mean(self(x))

  def get_distribution(self):
    with torch.no_grad():
      x = torch.FloatTensor(np.arange(self.d)).cuda()
      distribution = self(x).exp()
    return distribution.detach().cpu().numpy()

</code></pre>