Repository: pandas-dev/pandas

Issue:
Can't pd.options.display.max_columns = 0 by default?
`pd.options.display.max_columns` is 20 by default, but when running in a terminal this value could be auto-detected by setting it to 0. I know that this setting can be customized, but for the average user it would be much more convenient if there was no extra step involved. Is there a way to detect if pandas is running in a terminal (where autodetection works)? If so, the value should be set to 0. Otherwise, for example in Jupyter notebooks, it could remain to be 20.


Discussion:
Say you auto-detect (via get_terminal_size) and its 80, then what, how many columns should it be?

This is a pretty reasonable default.
Maybe I wasn't clear. If the width gets autodetected, a much more reasonable choice for `max_columns` would be `0`. That way, the printed object would fit within the terminal width.
@jreback could you reconsider this? Here's an example to show what I mean:

```
import numpy as np
import pandas as pd
pd.DataFrame(np.random.rand(5, 10))
```

In a terminal (mine has 100 columns), the default output of this dataframe looks like this:
```
          0         1         2         3         4         5         6  \
0  0.173421  0.463432  0.829303  0.557628  0.257765  0.712062  0.129705   
1  0.113790  0.310267  0.076708  0.576933  0.562503  0.623521  0.002787   
2  0.628581  0.740680  0.327314  0.129068  0.816575  0.470844  0.771247   
3  0.947981  0.984844  0.113195  0.449247  0.494137  0.349844  0.207327   
4  0.087635  0.678417  0.255774  0.924561  0.597169  0.089508  0.650698   

          7         8         9  
0  0.817547  0.337570  0.348707  
1  0.051971  0.983914  0.972759  
2  0.566701  0.627593  0.126499  
3  0.619493  0.292824  0.653050  
4  0.530746  0.773546  0.496640  
```

Now if I set `pd.options.display.max_columns = 0`, the output looks much nicer:

```
         0         1         2         3     ...           16        17        18        19
0  0.238365  0.438112  0.307173  0.226128    ...     0.547763  0.319377  0.688708  0.840808
1  0.153844  0.472930  0.128777  0.197224    ...     0.533801  0.530466  0.417650  0.174715
2  0.964627  0.626036  0.433339  0.834303    ...     0.036854  0.248769  0.932532  0.669207
3  0.375633  0.581266  0.026618  0.394931    ...     0.049323  0.195138  0.502164  0.454383
4  0.799988  0.137955  0.067402  0.012556    ...     0.587728  0.226844  0.447049  0.117987
```

So I was wondering why `pd.options.display.max_columns` can't be `0` by default?
@TomAugspurger ?
I usually set the max columns to 0 in my IPython init. I think it would be ok to change the default.

Do we still run into issues where we can't reliably detect the terminal width, or have those all been fixed?
I'll create a PR so that this stays on the radar. I'd also like to propose to change `pd.options.display.max_rows` to 20 instead of 60, because ideally I'd like to see a summary of the data frame including the column names. This is also how tibbles in R are displayed BTW.
Regarding the detection of the terminal width, I think it doesn't work in PyCharm's integrated terminal. However, the default of 80 is a reasonable value in this case.

Resolving patch:
diff --git a/pandas/core/config_init.py b/pandas/core/config_init.py
--- a/pandas/core/config_init.py
+++ b/pandas/core/config_init.py
@@ -13,6 +13,7 @@
 from pandas.core.config import (is_int, is_bool, is_text, is_instance_factory,
                                 is_one_of_factory, is_callable)
 from pandas.io.formats.console import detect_console_encoding
+from pandas.io.formats.terminal import is_terminal
 
 # compute
 
@@ -314,7 +315,11 @@ def table_schema_cb(key):
     cf.register_option('max_categories', 8, pc_max_categories_doc,
                        validator=is_int)
     cf.register_option('max_colwidth', 50, max_colwidth_doc, validator=is_int)
-    cf.register_option('max_columns', 20, pc_max_cols_doc,
+    if is_terminal():
+        max_cols = 0  # automatically determine optimal number of columns
+    else:
+        max_cols = 20  # cannot determine optimal number of columns
+    cf.register_option('max_columns', max_cols, pc_max_cols_doc,
                        validator=is_instance_factory([type(None), int]))
     cf.register_option('large_repr', 'truncate', pc_large_repr_doc,
                        validator=is_one_of_factory(['truncate', 'info']))
diff --git a/pandas/io/formats/format.py b/pandas/io/formats/format.py
--- a/pandas/io/formats/format.py
+++ b/pandas/io/formats/format.py
@@ -625,7 +625,8 @@ def to_string(self):
                 max_len += size_tr_col  # Need to make space for largest row
                 # plus truncate dot col
                 dif = max_len - self.w
-                adj_dif = dif
+                # '+ 1' to avoid too wide repr (GH PR #17023)
+                adj_dif = dif + 1
                 col_lens = Series([Series(ele).apply(len).max()
                                    for ele in strcols])
                 n_cols = len(col_lens)
diff --git a/pandas/io/formats/terminal.py b/pandas/io/formats/terminal.py
--- a/pandas/io/formats/terminal.py
+++ b/pandas/io/formats/terminal.py
@@ -17,7 +17,7 @@
 import sys
 import shutil
 
-__all__ = ['get_terminal_size']
+__all__ = ['get_terminal_size', 'is_terminal']
 
 
 def get_terminal_size():
@@ -48,6 +48,23 @@ def get_terminal_size():
     return tuple_xy
 
 
+def is_terminal():
+    """
+    Detect if Python is running in a terminal.
+
+    Returns True if Python is running in a terminal or False if not.
+    """
+    try:
+        ip = get_ipython()
+    except NameError:  # assume standard Python interpreter in a terminal
+        return True
+    else:
+        if hasattr(ip, 'kernel'):  # IPython as a Jupyter kernel
+            return False
+        else:  # IPython in a terminal
+            return True
+
+
 def _get_terminal_size_windows():
     res = None
     try:


Test patch:
diff --git a/pandas/tests/frame/test_dtypes.py b/pandas/tests/frame/test_dtypes.py
--- a/pandas/tests/frame/test_dtypes.py
+++ b/pandas/tests/frame/test_dtypes.py
@@ -875,10 +875,11 @@ def test_astype_str(self):
                              columns=self.tzframe.columns)
         tm.assert_frame_equal(result, expected)
 
-        result = str(self.tzframe)
-        assert ('0 2013-01-01 2013-01-01 00:00:00-05:00 '
-                '2013-01-01 00:00:00+01:00') in result
-        assert ('1 2013-01-02                       '
-                'NaT                       NaT') in result
-        assert ('2 2013-01-03 2013-01-03 00:00:00-05:00 '
-                '2013-01-03 00:00:00+01:00') in result
+        with option_context('display.max_columns', 20):
+            result = str(self.tzframe)
+            assert ('0 2013-01-01 2013-01-01 00:00:00-05:00 '
+                    '2013-01-01 00:00:00+01:00') in result
+            assert ('1 2013-01-02                       '
+                    'NaT                       NaT') in result
+            assert ('2 2013-01-03 2013-01-03 00:00:00-05:00 '
+                    '2013-01-03 00:00:00+01:00') in result
diff --git a/pandas/tests/frame/test_repr_info.py b/pandas/tests/frame/test_repr_info.py
--- a/pandas/tests/frame/test_repr_info.py
+++ b/pandas/tests/frame/test_repr_info.py
@@ -172,8 +172,8 @@ def test_repr_column_name_unicode_truncation_bug(self):
                                       'the CSV file externally. I want to Call'
                                       ' the File through the code..')})
 
-        result = repr(df)
-        assert 'StringCol' in result
+        with option_context('display.max_columns', 20):
+            assert 'StringCol' in repr(df)
 
     def test_latex_repr(self):
         result = r"""\begin{tabular}{llll}
diff --git a/pandas/tests/io/formats/test_format.py b/pandas/tests/io/formats/test_format.py
--- a/pandas/tests/io/formats/test_format.py
+++ b/pandas/tests/io/formats/test_format.py
@@ -961,7 +961,8 @@ def test_pprint_thing(self):
 
     def test_wide_repr(self):
         with option_context('mode.sim_interactive', True,
-                            'display.show_dimensions', True):
+                            'display.show_dimensions', True,
+                            'display.max_columns', 20):
             max_cols = get_option('display.max_columns')
             df = DataFrame(tm.rands_array(25, size=(10, max_cols - 1)))
             set_option('display.expand_frame_repr', False)
@@ -979,7 +980,8 @@ def test_wide_repr(self):
         reset_option('display.expand_frame_repr')
 
     def test_wide_repr_wide_columns(self):
-        with option_context('mode.sim_interactive', True):
+        with option_context('mode.sim_interactive', True,
+                            'display.max_columns', 20):
             df = DataFrame(np.random.randn(5, 3),
                            columns=['a' * 90, 'b' * 90, 'c' * 90])
             rep_str = repr(df)
@@ -987,7 +989,8 @@ def test_wide_repr_wide_columns(self):
             assert len(rep_str.splitlines()) == 20
 
     def test_wide_repr_named(self):
-        with option_context('mode.sim_interactive', True):
+        with option_context('mode.sim_interactive', True,
+                            'display.max_columns', 20):
             max_cols = get_option('display.max_columns')
             df = DataFrame(tm.rands_array(25, size=(10, max_cols - 1)))
             df.index.name = 'DataFrame Index'
@@ -1008,7 +1011,8 @@ def test_wide_repr_named(self):
         reset_option('display.expand_frame_repr')
 
     def test_wide_repr_multiindex(self):
-        with option_context('mode.sim_interactive', True):
+        with option_context('mode.sim_interactive', True,
+                            'display.max_columns', 20):
             midx = MultiIndex.from_arrays(tm.rands_array(5, size=(2, 10)))
             max_cols = get_option('display.max_columns')
             df = DataFrame(tm.rands_array(25, size=(10, max_cols - 1)),
@@ -1030,7 +1034,8 @@ def test_wide_repr_multiindex(self):
         reset_option('display.expand_frame_repr')
 
     def test_wide_repr_multiindex_cols(self):
-        with option_context('mode.sim_interactive', True):
+        with option_context('mode.sim_interactive', True,
+                            'display.max_columns', 20):
             max_cols = get_option('display.max_columns')
             midx = MultiIndex.from_arrays(tm.rands_array(5, size=(2, 10)))
             mcols = MultiIndex.from_arrays(
@@ -1044,15 +1049,16 @@ def test_wide_repr_multiindex_cols(self):
             wide_repr = repr(df)
             assert rep_str != wide_repr
 
-        with option_context('display.width', 150):
+        with option_context('display.width', 150, 'display.max_columns', 20):
             wider_repr = repr(df)
             assert len(wider_repr) < len(wide_repr)
 
         reset_option('display.expand_frame_repr')
 
     def test_wide_repr_unicode(self):
-        with option_context('mode.sim_interactive', True):
-            max_cols = get_option('display.max_columns')
+        with option_context('mode.sim_interactive', True,
+                            'display.max_columns', 20):
+            max_cols = 20
             df = DataFrame(tm.rands_array(25, size=(10, max_cols - 1)))
             set_option('display.expand_frame_repr', False)
             rep_str = repr(df)
@@ -1442,17 +1448,17 @@ def test_repr_html_mathjax(self):
             assert 'tex2jax_ignore' in df._repr_html_()
 
     def test_repr_html_wide(self):
-        max_cols = get_option('display.max_columns')
+        max_cols = 20
         df = DataFrame(tm.rands_array(25, size=(10, max_cols - 1)))
-        reg_repr = df._repr_html_()
-        assert "..." not in reg_repr
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            assert "..." not in df._repr_html_()
 
         wide_df = DataFrame(tm.rands_array(25, size=(10, max_cols + 1)))
-        wide_repr = wide_df._repr_html_()
-        assert "..." in wide_repr
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            assert "..." in wide_df._repr_html_()
 
     def test_repr_html_wide_multiindex_cols(self):
-        max_cols = get_option('display.max_columns')
+        max_cols = 20
 
         mcols = MultiIndex.from_product([np.arange(max_cols // 2),
                                          ['foo', 'bar']],
@@ -1467,8 +1473,8 @@ def test_repr_html_wide_multiindex_cols(self):
                                         names=['first', 'second'])
         df = DataFrame(tm.rands_array(25, size=(10, len(mcols))),
                        columns=mcols)
-        wide_repr = df._repr_html_()
-        assert '...' in wide_repr
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            assert '...' in df._repr_html_()
 
     def test_repr_html_long(self):
         with option_context('display.max_rows', 60):
@@ -1512,14 +1518,15 @@ def test_repr_html_float(self):
             assert u('2 columns') in long_repr
 
     def test_repr_html_long_multiindex(self):
-        max_rows = get_option('display.max_rows')
+        max_rows = 60
         max_L1 = max_rows // 2
 
         tuples = list(itertools.product(np.arange(max_L1), ['foo', 'bar']))
         idx = MultiIndex.from_tuples(tuples, names=['first', 'second'])
         df = DataFrame(np.random.randn(max_L1 * 2, 2), index=idx,
                        columns=['A', 'B'])
-        reg_repr = df._repr_html_()
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            reg_repr = df._repr_html_()
         assert '...' not in reg_repr
 
         tuples = list(itertools.product(np.arange(max_L1 + 1), ['foo', 'bar']))
@@ -1530,20 +1537,22 @@ def test_repr_html_long_multiindex(self):
         assert '...' in long_repr
 
     def test_repr_html_long_and_wide(self):
-        max_cols = get_option('display.max_columns')
-        max_rows = get_option('display.max_rows')
+        max_cols = 20
+        max_rows = 60
 
         h, w = max_rows - 1, max_cols - 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
-        assert '...' not in df._repr_html_()
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            assert '...' not in df._repr_html_()
 
         h, w = max_rows + 1, max_cols + 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
-        assert '...' in df._repr_html_()
+        with option_context('display.max_rows', 60, 'display.max_columns', 20):
+            assert '...' in df._repr_html_()
 
     def test_info_repr(self):
-        max_rows = get_option('display.max_rows')
-        max_cols = get_option('display.max_columns')
+        max_rows = 60
+        max_cols = 20
         # Long
         h, w = max_rows + 1, max_cols - 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
@@ -1555,7 +1564,8 @@ def test_info_repr(self):
         h, w = max_rows - 1, max_cols + 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
         assert has_horizontally_truncated_repr(df)
-        with option_context('display.large_repr', 'info'):
+        with option_context('display.large_repr', 'info',
+                            'display.max_columns', max_cols):
             assert has_info_repr(df)
 
     def test_info_repr_max_cols(self):
@@ -1575,8 +1585,8 @@ def test_info_repr_max_cols(self):
         # fmt.set_option('display.max_info_columns', 4)  # exceeded
 
     def test_info_repr_html(self):
-        max_rows = get_option('display.max_rows')
-        max_cols = get_option('display.max_columns')
+        max_rows = 60
+        max_cols = 20
         # Long
         h, w = max_rows + 1, max_cols - 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
@@ -1588,7 +1598,8 @@ def test_info_repr_html(self):
         h, w = max_rows - 1, max_cols + 1
         df = DataFrame({k: np.arange(1, 1 + h) for k in np.arange(w)})
         assert '<class' not in df._repr_html_()
-        with option_context('display.large_repr', 'info'):
+        with option_context('display.large_repr', 'info',
+                            'display.max_columns', max_cols):
             assert '&lt;class' in df._repr_html_()
 
     def test_fake_qtconsole_repr_html(self):