Repository: statsmodels/statsmodels

Issue:
slim ols results
Add slim parameter to summary function. slim=True generates a minimal result table, default or slim=False generates the original output. 

- [ ] closes #xxxx
- [ ] tests added / passed. 
- [x] code/documentation is well formatted.  
- [ ] properly formatted commit message. See 
      [NumPy's guide](https://docs.scipy.org/doc/numpy-1.15.1/dev/gitwash/development_workflow.html#writing-the-commit-message). 

<details>


**Notes**:

* It is essential that you add a test when making code changes. Tests are not 
  needed for doc changes.
* When adding a new function, test values should usually be verified in another package (e.g., R/SAS/Stata).
* When fixing a bug, you must add a test that would produce the bug in master and
  then show that it is fixed with the new code.
* New code additions must be well formatted. Changes should pass flake8. If on Linux or OSX, you can
  verify you changes are well formatted by running 
  ```
  git diff upstream/master -u -- "*.py" | flake8 --diff --isolated
  ```
  assuming `flake8` is installed. This command is also available on Windows 
  using the Windows System for Linux once `flake8` is installed in the 
  local Linux environment. While passing this test is not required, it is good practice and it help 
  improve code quality in `statsmodels`.
* Docstring additions must render correctly, including escapes and LaTeX.

</details>



Discussion:

[![Coverage Status](https://coveralls.io/builds/32080879/badge)](https://coveralls.io/builds/32080879)

Coverage increased (+0.003%) to 88.029% when pulling **69460e0c4dd386b15f9f361326866e5376c2c77a on janosbiro:slim-results** into **5c6e3f1a3b417b58c932c52ef885fa29c7d00b15 on statsmodels:master**.

This pull request **introduces 2 alerts** when merging 7e0ec3c9e01c11d89da8f11857621c846fd294b7 into 79c51ffbc63d62b2c9c7ca4d3f45d89b9c745fd8 - [view on LGTM.com](https://lgtm.com/projects/g/statsmodels/statsmodels/rev/pr-236f81b66a3e49f904b3a15132442fb15c312cef)

**new alerts:**

* 2 for Unused local variable
looks like too much code duplication to me.
The overlapping parts should be only once in the code.

What's removed when "slim"? AIC, BIC, diagnostic table, others ?

hey,

the slim results table is the following:
[image: image.png]

We are economists who are recoding their codes from stata and R. In the
slim results we keep the most necessary results in table.

Slim parameter removes diagnostics (bottom panel, method, date,time,
aic,bic, etc...).

br,
J

Josef Perktold <notifications@github.com> ezt írta (időpont: 2020. júl.
10., P, 22:30):

> looks like too much code duplication to me.
> The overlapping parts should be only once in the code.
>
> What's removed when "slim"? AIC, BIC, diagnostic table, others ?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/statsmodels/statsmodels/pull/6880#issuecomment-656876122>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AHYWCDU5QXQ63JG4DAEKMBLR2525VANCNFSM4OW3OJAQ>
> .
>


-- 
Biró János

The image doesn't show up.

It's easier if you just copy-paste the text version from `print(res.summary(slim=True))`
```
OLS Regression Results
==============================================================================
Dep. Variable:                lnprice   R-squared:                       0.244
Model:                            OLS   Adj. R-squared:                  0.234
No. Observations:                  77   F-statistic:                     24.20
Covariance Type:            nonrobust   Prob (F-statistic):           4.99e-06
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept      3.1375      0.285     11.025      0.000       2.571       3.704
rating         0.3540      0.072      4.920      0.000       0.211       0.497
==============================================================================
```


The implementation has too much copy-pasta for my tastes. I think this isn't a good way forward since it is still incredibly specific. A better way would be to have a function or class that would be configured to produce tables using standard and common property names.  
Hello @janosbiro! Thanks for updating this PR. We checked the lines you've touched for [PEP 8](https://www.python.org/dev/peps/pep-0008) issues, and found:

* In the file [`statsmodels/regression/linear_model.py`](https://github.com/statsmodels/statsmodels/blob/69460e0c4dd386b15f9f361326866e5376c2c77a/statsmodels/regression/linear_model.py):

> [Line 2668:1](https://github.com/statsmodels/statsmodels/blob/69460e0c4dd386b15f9f361326866e5376c2c77a/statsmodels/regression/linear_model.py#L2668): [W293](https://duckduckgo.com/?q=pep8%20W293) blank line contains whitespace


This pull request **introduces 2 alerts** when merging 69460e0c4dd386b15f9f361326866e5376c2c77a into 5c6e3f1a3b417b58c932c52ef885fa29c7d00b15 - [view on LGTM.com](https://lgtm.com/projects/g/statsmodels/statsmodels/rev/pr-a5ad10767e8d43a77feb429b3ceb2ddc3d836326)

**new alerts:**

* 2 for Unused local variable
@bashtage Can you check it now please? :) I've rewrote without code duplication
@josef-pkt can you check the last update please?

We would like to use this python-package to teach econometrics for economists. But the original summary table is confusing for student's and most of the table elements not used, it's better to use a slim table. e.g.: Stata or R regression outputs is lightweight but they present the most important diagnostics and you can make the extras
looks good to me, but I want to check it out because we don't have proper unit tests for summary

A smoke test for the slim option should be added to the unit test, or one that assert the partial content. AFAIR we have just one or a few of those.

Resolving patch:
diff --git a/statsmodels/regression/linear_model.py b/statsmodels/regression/linear_model.py
--- a/statsmodels/regression/linear_model.py
+++ b/statsmodels/regression/linear_model.py
@@ -2618,7 +2618,7 @@ def get_prediction(self, exog=None, transform=True, weights=None,
             self, exog=exog, transform=transform, weights=weights,
             row_labels=row_labels, **kwargs)
 
-    def summary(self, yname=None, xname=None, title=None, alpha=.05):
+    def summary(self, yname=None, xname=None, title=None, alpha=.05, slim=False):
         """
         Summarize the Regression Results.
 
@@ -2650,7 +2650,7 @@ def summary(self, yname=None, xname=None, title=None, alpha=.05):
             durbin_watson,
             jarque_bera,
             omni_normtest,
-        )
+            )
 
         jb, jbpv, skew, kurtosis = jarque_bera(self.wresid)
         omni, omnipv = omni_normtest(self.wresid)
@@ -2696,19 +2696,28 @@ def summary(self, yname=None, xname=None, title=None, alpha=.05):
                      ('BIC:', ["%#8.4g" % self.bic])
                      ]
 
-        diagn_left = [('Omnibus:', ["%#6.3f" % omni]),
-                      ('Prob(Omnibus):', ["%#6.3f" % omnipv]),
-                      ('Skew:', ["%#6.3f" % skew]),
-                      ('Kurtosis:', ["%#6.3f" % kurtosis])
-                      ]
-
-        diagn_right = [('Durbin-Watson:',
-                        ["%#8.3f" % durbin_watson(self.wresid)]
-                        ),
-                       ('Jarque-Bera (JB):', ["%#8.3f" % jb]),
-                       ('Prob(JB):', ["%#8.3g" % jbpv]),
-                       ('Cond. No.', ["%#8.3g" % condno])
-                       ]
+        if slim:
+            slimlist = ['Dep. Variable:', 'Model:', 'No. Observations:',
+                        'Covariance Type:', 'R-squared:', 'Adj. R-squared:',
+                        'F-statistic:', 'Prob (F-statistic):']
+            diagn_left = []
+            diagn_right = []
+            top_left = [elem for elem in top_left if elem[0] in slimlist]
+            top_right = [elem for elem in top_right if elem[0] in slimlist]
+        else:
+            diagn_left = [('Omnibus:', ["%#6.3f" % omni]),
+                          ('Prob(Omnibus):', ["%#6.3f" % omnipv]),
+                          ('Skew:', ["%#6.3f" % skew]),
+                          ('Kurtosis:', ["%#6.3f" % kurtosis])
+                          ]
+
+            diagn_right = [('Durbin-Watson:',
+                            ["%#8.3f" % durbin_watson(self.wresid)]
+                            ),
+                           ('Jarque-Bera (JB):', ["%#8.3f" % jb]),
+                           ('Prob(JB):', ["%#8.3g" % jbpv]),
+                           ('Cond. No.', ["%#8.3g" % condno])
+                           ]
 
         if title is None:
             title = self.model.__class__.__name__ + ' ' + "Regression Results"
@@ -2720,10 +2729,10 @@ def summary(self, yname=None, xname=None, title=None, alpha=.05):
                              yname=yname, xname=xname, title=title)
         smry.add_table_params(self, yname=yname, xname=xname, alpha=alpha,
                               use_t=self.use_t)
-
-        smry.add_table_2cols(self, gleft=diagn_left, gright=diagn_right,
-                             yname=yname, xname=xname,
-                             title="")
+        if not slim:
+            smry.add_table_2cols(self, gleft=diagn_left, gright=diagn_right,
+                                 yname=yname, xname=xname,
+                                 title="")
 
         # add warnings/notes, added to text format only
         etext = []


Test patch:
diff --git a/statsmodels/regression/tests/test_regression.py b/statsmodels/regression/tests/test_regression.py
--- a/statsmodels/regression/tests/test_regression.py
+++ b/statsmodels/regression/tests/test_regression.py
@@ -331,6 +331,12 @@ def test_norm_resid(self):
         model_norm_resid = self.res1.resid_pearson
         assert_almost_equal(model_norm_resid, norm_resid, DECIMAL_7)
 
+    def test_summary_slim(self):
+        # check that slim summary is smaller, does not verify content
+        summ = self.res1.summary(slim=True)
+        assert len(summ.tables) == 2
+        assert len(str(summ)) < 6700
+
     def test_norm_resid_zero_variance(self):
         with warnings.catch_warnings(record=True):
             y = self.res1.model.endog