Repository: spyder-ide/spyder

Issue:
Spyder crashes with Unicode symbols in editor
Spyder 3.0.0dev, Windows x64

I cut-and-pasted a degree symbol (°) into the Spyder editor.

I get:

Spyder Internal Console

This console is used to report application
internal errors and to inspect Spyder
internals with the following commands:
  spy.app, spy.window, dir(spy)

Please don't use it to run your code

> > > Traceback (most recent call last):
> > >   File "C:\WinPython-64bit-3.4.4.2\python-3.4.4.amd64\lib\site-packages\spyderlib\widgets\editor.py", line 1325, in analyze_script
> > >     finfo.run_code_analysis(run_pyflakes, run_pep8)
> > >   File "C:\WinPython-64bit-3.4.4.2\python-3.4.4.amd64\lib\site-packages\spyderlib\widgets\editor.py", line 206, in run_code_analysis
> > >     source_code = self.get_source_code().encode(enc)
> > > UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 8248: ordinal not in range(128)
> > > Traceback (most recent call last):
> > >   File "C:\WinPython-64bit-3.4.4.2\python-3.4.4.amd64\lib\site-packages\spyderlib\widgets\editor.py", line 1325, in analyze_script
> > >     finfo.run_code_analysis(run_pyflakes, run_pep8)
> > >   File "C:\WinPython-64bit-3.4.4.2\python-3.4.4.amd64\lib\site-packages\spyderlib\widgets\editor.py", line 206, in run_code_analysis
> > >     source_code = self.get_source_code().encode(enc)
> > > UnicodeEncodeError: 'ascii' codec can't encode character '\xb0' in position 8248: ordinal not in range(128)



Discussion:
I'm not sure of your exact test case. Maybe it works in  WinPython-3.4.4.3.
It may depend of your Windows version/locals, so you may check if it's indeed better.

![spyder_unicode_test](https://cloud.githubusercontent.com/assets/4312421/17085984/7e59477a-51e6-11e6-9ffa-5eeec9c79a06.PNG)

Will check and post here. 

FYI, I copied the symbol from https://en.wikipedia.org/wiki/Degree_symbol (just a handy place to find one).

I'm using the US version of Windows; I'll upgrade to WinPython -3.4.4.3 and try it.  While I've got your attention, what's the difference between the builds with/without "Qt5" in the name?

the one with "Qt5" are with  PyQt5, and so Qt5.

same problem with russian language in comments
python 3.5.2 32b
windows 10 64b
pip freeze - http://pastebin.com/fU4zS4mx
I just had the same issue on Linux with Spyder 3.0.1, but I upgraded to 3.0.2 and it went away.

So, it seems that the issue was fixed in this release.

In both cases this is with Python 3.5.2 from Anaconda 4.2.0 (x86_64). The system is Red Hat EL 6.8.
I typed in a degree symbol ° (with Compose-o-o). With Spyder 3.0.1, I got the same traceback in the internal console as in the first message, and it kept popping up whenever I edited the file with that character in it. With 3.0.2 there are no problems.
We didn't do anything special to fix this bug. Maybe it's something related to Qt4, so please update to Qt5 to see if that fixes it.
@ccordoba12 I'm able to reproduce this bug, It is caused with a file with encoding different that UTF-8. (e.g. when in an ASCII file you write non-ascii characters.)

A quick fix could be to prefer UTF-8 over ASCII (if chardet detect ascii use utf-8 instead). but It won't solve this problem with other encodings.

Somehow this is related with #3753, I could fix this after I finish code folding :smile: 
Thanks for being able to reproduce this bug, I failed at it :-)

> Somehow this is related with #3753, I could fix this after I finish code folding

Ok, please finish code folding first and then come back to this. 

Resolving patch:
diff --git a/spyder/widgets/editor.py b/spyder/widgets/editor.py
--- a/spyder/widgets/editor.py
+++ b/spyder/widgets/editor.py
@@ -204,7 +204,7 @@ def run_code_analysis(self, run_pyflakes, run_pep8):
         self.pep8_results = []
         if self.editor.is_python():
             enc = self.encoding.replace('-guessed', '').replace('-bom', '')
-            source_code = self.get_source_code().encode(enc)
+            source_code, enc = encoding.encode(self.get_source_code(), enc)
             if run_pyflakes:
                 self.pyflakes_results = None
             if run_pep8:


Test patch: