Repository: spyder-ide/spyder

Issue:
Kernel dies and can't report errors because of failure to read stderr file
## Description of your problem
Hi, I am a very inexperienced user, I launched spyder to learn write one of my first, very simple programs, but i repeatedly, once each several seconds, get message:

WARNING:traitlets:kernel died: 3.0 
Traceback (most recent call last):
  File "C:\Users\Wojtek\Anaconda2\lib\site-packages\spyder\widgets\ipythonconsole\client.py", line 357, in kernel_restarted_message
    stderr = codecs.open(self.stderr_file, 'r', encoding='utf-8').read()
  File "C:\Users\Wojtek\Anaconda2\lib\codecs.py", line 686, in read
    return self.reader.read(size)
  File "C:\Users\Wojtek\Anaconda2\lib\codecs.py", line 492, in read
    newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbf in position 2136: invalid start byte


**What steps will reproduce the problem?**

1. It happens just after opening spyder
2. It keeps turning to internal console to show this message, so i'm in fact unable to use spyder
3. 

**What is the expected output? What do you see instead?**


**Please provide any additional information below**


## Versions and main components

* Spyder Version: 3.1.3
* Python Version: 2.7
* Qt Version: 5.6.2
* PyQt Version: 5.6.0
* Operating system: windows 7


## Dependencies

Please go to the menu entry `Help > Optional Dependencies` (or
`Help > Dependencies`), press the button `Copy to clipboard`
and paste the contents below:

jedi =0.9.0      :  0.9.0 (OK)
matplotlib >=1.0 :  2.0.0 (OK)
nbconvert >=4.0  :  4.2.0 (OK)
numpy >=1.7      :  1.11.3 (OK)
pandas >=0.13.1  :  0.19.2 (OK)
pep8 >=0.6       :  1.7.0 (OK)
pyflakes >=0.5.0 :  1.5.0 (OK)
pygments >=2.0   :  2.1.3 (OK)
pylint >=0.25    :  1.6.4 (OK)
qtconsole >=4.2.0:  4.2.1 (OK)
rope >=0.9.4     :  0.9.4 (OK)
sphinx >=0.6.6   :  1.5.1 (OK)
sympy >=0.7.3    :  1.0 (OK)



Discussion:
It seems the console is failing with an error message that contains characters that are not Unicode symbols.

Unicode is the standard for international characters, but I don't know how's that handled on non-English versions of Windows.

I'm very sorry, but I really don't know how to help you for now.
Hi, thx very much for your response. My teacher managed to figure out how to overcome problem with encoding and make spyder show the actual error. 
It was enough to subsitute "utf-8" with "cp437" in this line "stderr = codecs.open(self.stderr_file, 'r', encoding='utf-8').read()".
I'm using windows in polish.

Cheers!
I was also thinking that was the problem. Thanks for letting us know about it :-)
@dalthviz, this is the bug I talked you about.

Resolving patch:
diff --git a/spyder/widgets/ipythonconsole/client.py b/spyder/widgets/ipythonconsole/client.py
--- a/spyder/widgets/ipythonconsole/client.py
+++ b/spyder/widgets/ipythonconsole/client.py
@@ -32,9 +32,11 @@
 from spyder.config.gui import get_font, get_shortcut
 from spyder.utils import icon_manager as ima
 from spyder.utils import sourcecode
+from spyder.utils.encoding import get_coding
 from spyder.utils.programs import TEMPDIR
 from spyder.utils.qthelpers import (add_actions, create_action,
                                     create_toolbutton)
+from spyder.py3compat import to_text_string
 from spyder.widgets.browser import WebView
 from spyder.widgets.mixins import SaveHistoryMixin
 from spyder.widgets.ipythonconsole import ShellWidget
@@ -378,7 +380,17 @@ def restart_kernel(self):
     @Slot(str)
     def kernel_restarted_message(self, msg):
         """Show kernel restarted/died messages."""
-        stderr = codecs.open(self.stderr_file, 'r', encoding='utf-8').read()
+        try:
+            stderr = codecs.open(self.stderr_file, 'r',
+                                 encoding='utf-8').read()
+        except UnicodeDecodeError:
+            # This is needed since the stderr file could be encoded
+            # in something different to utf-8.
+            # See issue 4191
+            try:
+                stderr = self._read_stderr()
+            except:
+                stderr = None
 
         if stderr:
             self.show_kernel_error('<tt>%s</tt>' % stderr)
@@ -438,3 +450,10 @@ def _hide_loading_page(self):
 
         document = self.get_control().document()
         document.contentsChange.disconnect(self._hide_loading_page)
+
+    def _read_stderr(self):
+        """Read the stderr file of the kernel."""
+        stderr_text = open(self.stderr_file, 'rb').read()
+        encoding = get_coding(stderr_text)
+        stderr = to_text_string(stderr_text, encoding)
+        return stderr


Test patch:
diff --git a/spyder/plugins/tests/test_ipythonconsole.py b/spyder/plugins/tests/test_ipythonconsole.py
--- a/spyder/plugins/tests/test_ipythonconsole.py
+++ b/spyder/plugins/tests/test_ipythonconsole.py
@@ -4,6 +4,7 @@
 # Licensed under the terms of the MIT License
 #
 
+import codecs
 import os
 import os.path as osp
 import shutil
@@ -55,6 +56,23 @@ def close_widget():
 #==============================================================================
 # Tests
 #==============================================================================
+def test_read_stderr(ipyconsole, qtbot):
+    """
+    Test the read operation of the stderr file of the kernel
+    """
+
+    shell = ipyconsole.get_current_shellwidget()
+    client = ipyconsole.get_current_client()
+    qtbot.waitUntil(lambda: shell._prompt_html is not None, timeout=SHELL_TIMEOUT)
+
+    # Set contents of the stderr file of the kernel
+    content = 'Test text'
+    stderr_file = client.stderr_file
+    codecs.open(stderr_file, 'w', 'cp437').write(content)
+    # Assert that content is correct
+    assert content == client._read_stderr()
+
+
 @flaky(max_runs=10)
 @pytest.mark.skipif(os.name == 'nt', reason="It times out on Windows")
 def test_run_doctest(ipyconsole, qtbot):