They are scaled by the learning rate inplace beforehand in the previous
lines. This makes the computation of the gradient for components faster.

On Sun, Jan 20, 2013 at 6:39 PM, Vlad Niculae <notifications@github.com>wrote:

> In sklearn/neural_networks/rbm.py:
>
> > +        ----------
> > +        [1] Tieleman, T. Training Restricted Boltzmann Machines using
> > +            Approximations to the Likelihood Gradient. International Conference
> > +            on Machine Learning (ICML) 2008
> > +        """
> > +        h_pos = self.mean_hiddens(v_pos)
> > +        v_neg = self.sample_visibles(self.h_samples_)
> > +        h_neg = self.mean_hiddens(v_neg)
> > +
> > +        lr = self.learning_rate / self.n_particles
> > +        v_pos *= lr
> > +        v_neg *= lr
> > +        self.components_ += safe_sparse_dot(v_pos.T, h_pos).T
> > +        self.components_ -= np.dot(v_neg.T, h_neg).T
> > +        self.intercept_hidden_ += lr * (h_pos.sum(0) - h_neg.sum(0))
> > +        self.intercept_visible_ += (v_pos.sum(0) - v_neg.sum(0))
>
> Is it intended for the visible intercept to not be scaled by the learning
> rate? If so, could you explain why?
> I'm experimenting with momentum and this jumped at me.
>
> —
> Reply to this email directly or view it on GitHub<https://github.com/scikit-learn/scikit-learn/pull/1200/files#r2708196>.
>
>