Repository: statsmodels/statsmodels

Issue:
BUG: self.history in GLM doesn't seem like it does anything...
Hello,

The `history` attribute in GLM doesn't seem like it really does anything. It seems like what's happening is that the history is created by each of the district `fit` methods. 

See this line of code:
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/genmod/generalized_linear_model.py#L291

You can uncomment it all and GLMs seem to work just as before. A few tests fail when they try to reference that attribute, however, so there's some hunting that will be necessary to clean this up. 



Discussion:
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/genmod/generalized_linear_model.py#L989

`history` is used in the convergence check in fit_irls. it checks the last element in the list,
At least, that is what it is supposed to be doing.

It's also available as a results attribute, AFAIR, so we can check the history of irls optimization.
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/genmod/generalized_linear_model.py#L1002

Sorry I wasn't clear.

The `history` that is defined in `GLM` in the `initialize` section appears to be ignored in the `fit_irls` section. There, `history` is created as a new dict, totally independent of the one in the `GLM` instance. That new `history` is later saved as part of the model result. The original history, in the model, is totally disconnected.

Ok, my guess is that the part in initialize with `self.history` is a an orphan because of the refactoring
https://github.com/statsmodels/statsmodels/commit/9cb2a022040e96472c68e136027ab09c5b702ab1

We try to avoid having fit specific information attached to the model. So using history and attaching it to the results instance is cleaner.
It looks to me we can remove it from `initialize`, but we need to check what the let over usage of it is that you say raise an error.  Maybe those tests should check the results.history, but they might not test much if `self.history`, where self is the model, never gets updated.

Sounds good.  I think this is just an inconvenience... Not major by any stretch. Just I decided to put some time back into the convergence issue I had before and this confused me for a bit.

> On May 31, 2016, at 12:49 PM, Josef Perktold notifications@github.com wrote:
> 
> Ok, my guess is that the part in initialize with self.history is a an orphan because of the refactoring
> 9cb2a02
> 
> We try to avoid having fit specific information attached to the model. So using history and attaching it to the results instance is cleaner.
> It looks to me we can remove it from initialize, but we need to check what the let over usage of it is that you say raise an error. Maybe those tests should check the results.history, but they might not test much if self.history, where self is the model, never gets updated.
> 
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub, or mute the thread.


Resolving patch:
diff --git a/statsmodels/genmod/generalized_linear_model.py b/statsmodels/genmod/generalized_linear_model.py
--- a/statsmodels/genmod/generalized_linear_model.py
+++ b/statsmodels/genmod/generalized_linear_model.py
@@ -355,11 +355,6 @@ def initialize(self):
         """
         Initialize a generalized linear model.
         """
-        # TODO: intended for public use?
-        self.history = {'fittedvalues': [],
-                        'params': [np.inf],
-                        'deviance': [np.inf]}
-
         self.df_model = np.linalg.matrix_rank(self.exog) - 1
 
         if (self.freq_weights is not None) and \
@@ -1409,8 +1404,6 @@ def fit_constrained(self, constraints, start_params=None, **fit_kwds):
         res._results.constraints = lc
         res._results.k_constr = k_constr
         res._results.results_constrained = res_constr
-        # TODO: the next is not the best. history should bin in results
-        res._results.model.history = res_constr.model.history
         return res
 
 
diff --git a/statsmodels/sandbox/stats/multicomp.py b/statsmodels/sandbox/stats/multicomp.py
--- a/statsmodels/sandbox/stats/multicomp.py
+++ b/statsmodels/sandbox/stats/multicomp.py
@@ -1,4 +1,4 @@
-'''
+r'''
 
 from pystatsmodels mailinglist 20100524
 


Test patch: