I will change it to the _ce version.

The situation with the Hessian is a bit peculiar here.  There are two
parameterizations of the covariance matrix.  One is used for optimization
and has the advantage of avoiding boundary issues.  The other
parameterization is the traditional one that the user expects to see in the
output.

I don't think the code that calculates the Hessian numerically will ever be
invoked.  I already have a function (hessian_full) that calculates the
Hessian
with respect to the user-facing parameterization.  It gets invoked one time
at the end to get standard errors.

If one wanted to do optimization using a Hessian (e.g. Newton Raphson),
then you need a Hessian with respect to the parameterization used in the
optimization.  Converting the Hessian for the user-facing parameterization
to the alternative parameterization is not trivial (I did it for the
gradient, and that was already tricky).

In any case, I don't think we really want to use the Hessian for
optimization in this case.  The log-likelihood is not very quadratic.
 Conjugate gradient methods seem to work better here.  The bfgs method is
the only method that consistently works for me.


On Fri, Apr 4, 2014 at 2:24 AM, Skipper Seabold <notifications@github.com>wrote:

> In statsmodels/regression/lme.py:
>
> > +        results.k_fe = self.k_fe
> > +        results.k_re = self.k_re
> > +        results.k_re2 = self.k_re2
> > +
> > +        return results
> > +
> > +
> > +    def hessian(self, params):
> > +        """
> > +        Hessian of log-likelihood evaluated at `params`.  Note that
> > +        this uses either `cov_re` or its square root (L) depending on
> > +        the value of `use_L`.  `hessian_full` is an analytic
> > +        implementation of the Hessian with respect to cov_re.
> > +        """
> > +        from statsmodels.tools.numdiff import approx_hess
> > +        return approx_hess(params, self.loglike)
>
> Generally, we've found that approx_hess_cs is the more accurate. Might
> see if it compares more favorable to test numbers than 1-2 significant
> digits.
>
> --
> Reply to this email directly or view it on GitHub<https://github.com/statsmodels/statsmodels/pull/1398/files#r11265165>
> .
>