Repository: pandas-dev/pandas

Issue:
BUG: mod_wsgi restricting access to stdout throws unhandled IOError.
#### Problem description
We're using pandas on the backend of our internal web server for a few utilities. Any attempt to import the library causes an IOError to be thrown due to our setup of mod_wsgi restricting stdout access by default (WSGIRestrictStdout On).

The cause of this error is an attempt to get the client encoding on init when registering options. A call is made to detect_client_encoding() in console.py under pandas/io/formats/ which is throwing the IOError. The call to sys.stdout.encoding in this function is contained inside a try/except block, however, the except statement is limited to an AttributeError.

I propose we expand this except statement to allow for both AttributeErrors and IOErrors.

#### Details
<details>
INSTALLED VERSIONS

------------------

commit: e2d5958bb111cb9a87b307a963f57ea8cc775646
python: 2.7.15.final.0
python-bits: 64
OS: Linux
OS-release: 4.16.14-300.fc28.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: None.None

pandas: 0.24.0.dev0+118.ge2d5958bb
pytest: 3.6.1
pip: 9.0.3
setuptools: 39.2.0
Cython: 0.28.3
numpy: 1.14.5
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
</details>


Discussion:

Resolving patch:
diff --git a/pandas/io/formats/console.py b/pandas/io/formats/console.py
--- a/pandas/io/formats/console.py
+++ b/pandas/io/formats/console.py
@@ -21,7 +21,7 @@ def detect_console_encoding():
     encoding = None
     try:
         encoding = sys.stdout.encoding or sys.stdin.encoding
-    except AttributeError:
+    except (AttributeError, IOError):
         pass
 
     # try again for something better


Test patch:
diff --git a/pandas/tests/io/formats/test_console.py b/pandas/tests/io/formats/test_console.py
new file mode 100644
--- /dev/null
+++ b/pandas/tests/io/formats/test_console.py
@@ -0,0 +1,74 @@
+import pytest
+
+from pandas.io.formats.console import detect_console_encoding
+
+
+class MockEncoding(object):  # TODO(py27): replace with mock
+    """
+    Used to add a side effect when accessing the 'encoding' property. If the
+    side effect is a str in nature, the value will be returned. Otherwise, the
+    side effect should be an exception that will be raised.
+    """
+    def __init__(self, encoding):
+        super(MockEncoding, self).__init__()
+        self.val = encoding
+
+    @property
+    def encoding(self):
+        return self.raise_or_return(self.val)
+
+    @staticmethod
+    def raise_or_return(val):
+        if isinstance(val, str):
+            return val
+        else:
+            raise val
+
+
+@pytest.mark.parametrize('empty,filled', [
+    ['stdin', 'stdout'],
+    ['stdout', 'stdin']
+])
+def test_detect_console_encoding_from_stdout_stdin(monkeypatch, empty, filled):
+    # Ensures that when sys.stdout.encoding or sys.stdin.encoding is used when
+    # they have values filled.
+    # GH 21552
+    with monkeypatch.context() as context:
+        context.setattr('sys.{}'.format(empty), MockEncoding(''))
+        context.setattr('sys.{}'.format(filled), MockEncoding(filled))
+        assert detect_console_encoding() == filled
+
+
+@pytest.mark.parametrize('encoding', [
+    AttributeError,
+    IOError,
+    'ascii'
+])
+def test_detect_console_encoding_fallback_to_locale(monkeypatch, encoding):
+    # GH 21552
+    with monkeypatch.context() as context:
+        context.setattr('locale.getpreferredencoding', lambda: 'foo')
+        context.setattr('sys.stdout', MockEncoding(encoding))
+        assert detect_console_encoding() == 'foo'
+
+
+@pytest.mark.parametrize('std,locale', [
+    ['ascii', 'ascii'],
+    ['ascii', Exception],
+    [AttributeError, 'ascii'],
+    [AttributeError, Exception],
+    [IOError, 'ascii'],
+    [IOError, Exception]
+])
+def test_detect_console_encoding_fallback_to_default(monkeypatch, std, locale):
+    # When both the stdout/stdin encoding and locale preferred encoding checks
+    # fail (or return 'ascii', we should default to the sys default encoding.
+    # GH 21552
+    with monkeypatch.context() as context:
+        context.setattr(
+            'locale.getpreferredencoding',
+            lambda: MockEncoding.raise_or_return(locale)
+        )
+        context.setattr('sys.stdout', MockEncoding(std))
+        context.setattr('sys.getdefaultencoding', lambda: 'sysDefaultEncoding')
+        assert detect_console_encoding() == 'sysDefaultEncoding'