Repository: xonsh/xonsh

Issue:
Installation on Windows with Anaconda
I'm trying to install xonsh on Windows x64 with Anaconda and python 3.4 I get this error:

```
>pip install xonsh
Collecting xonsh
  Using cached xonsh-0.2.3.tar.gz
Requirement already satisfied (use --upgrade to upgrade): ply in c:\users\laser2002j\miniconda3\lib\site-packages (from xonsh)
Building wheels for collected packages: xonsh
  Running setup.py bdist_wheel for xonsh
  Complete output from command C:\Users\laser2002j\Miniconda3\python.exe -c "import setuptools;__file__='C:\\Users\\LASER2~1\\AppData\\Local\\Temp\\pip-build-egozq_by\\xonsh\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d C:\Users\LASER2~1\AppData\Local\Temp\tmpd0h8znkypip-wheel-:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\Users\laser2002j\Miniconda3\lib\encodings\cp1252.py", line 23, in decode
      return codecs.charmap_decode(input,self.errors,decoding_table)[0]
  UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 4434: character maps to <undefined>

  ----------------------------------------
  Failed building wheel for xonsh
Failed to build xonsh
Installing collected packages: xonsh
  Running setup.py install for xonsh
Successfully installed xonsh-0.2.3
```



Discussion:
```
conda install xonsh -c scopatz
```

works without errors instead.

Thanks for reportung @tritemio. Can you give us more info about your system? Are you sure that this pip you have was coming from conda? What is your default encoding?

This is a miniconda3 installation (user folder) using conda's pip in the root environment. The installation is a few weeks old and is using python 3.4 as default python. The latest Miniconda3 installer may have switched to python 3.5 in the meanwhile however.

For the default encoding, I don't know, I never changed it. I guess is windows 7 default.

I can confirm this issue on windows as well. I think the problem is that beautiful Conch logo in the setup.py is angering windows when trying to use it's default encoding  CP-1252.

I am running miniconda 3.4 as well and can replicate the issue just trying to read setup.py from a python repl opened from a windows cmd prompt.

``` python
>>> open('setup.py').read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Miniconda3\envs\sdmtools\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 4584: character maps to <undefined>
```

However if you pass the encoding then it opens just fine

``` python
open('setup.py', encoding='utf-8').read()
```

Unfortunately that's as far as I can get, I'm not sure where you would tell wheels to use that encoding when opening the setup file.

Oh, I think we need to have a UTF-8 encoding comment in the header then.

On Mon, Nov 16, 2015 at 10:36 AM, Anthony Scopatz notifications@github.com
wrote:

> Oh, I think we need to have a UTF-8 encoding comment in the header then.

It has a coding comment but it's incorrectly formatted. And most of the
other python modules don't have any coding comment. I'll whip up a lint PR
for this.

## 

Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

Thanks Kurtis

Argh! The setup.py module does have a valid coding comment on line 2; it's just formatted differently than I'm used to seeing. Looking more closely at the original problem report we can see that pip is directly opening and reading setup.py rather than importing it. And that's why the encoding comment is ignored:

```
Complete output from command C:\Users\laser2002j\Miniconda3\python.exe -c "import setuptools;__file__='C:\\Users\\LASER2~1\\AppData\\Local\\Temp\\pip-build-egozq_by\\xonsh\\setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d C:\Users\LASER2~1\AppData\Local\Temp\tmpd0h8znkypip-wheel-:
```

This is not a xonsh problem -- it's a pip bug. A pip github issue was opened exactly two years ago about this problem:  https://github.com/pypa/pip/issues/1291. What is particularly sad is that PEP 263, which defined the way to specify the character encoding of a python module, was written almost fourteen years ago and the people commenting on that pip issue seem to be unaware of it.

This document has the answer: http://stackoverflow.com/questions/25465993/python-yet-another-encoding-issue-windows-8-1-both-latest-pythons-pip-throw:

```
set PYTHONIOENCODING=utf-8
```

before running `pip install`.

I'm still going to do an encoding comment lint PR just because too many of the modules are missing an explicit encoding and the inconsistencies in comment style is silly. I'll also update the xonsh docs to include the solution above.

Grumble. Setting the PYTHONIOENCODING env var will not fix this as that only affects stdin/stdout/stderr. The person who proposed that on stackoverflow obviously didn't test it. I think we're going to have to limit setup.py to ASCII for the time being.

Uggh that sucks.  Maybe we could move the conch shell logo to a separate file called `logo.txt` or something and then read it in and then print it.  That should leave the setup.py file ASCII-only.

Yeah the issue definitely appears to be pip related, there was another bug submitted where pip had an issue reading encoding from setup.py during installations. Apparently in python 3.2 they added tokenize.open which does detect specified encodings in headers so pip fixed their issue by using. tokenize.open() instead of open to open setup.py, 

However it doesn't look like they used tokenize when using the wheel command which is where this is failing. Might attempt to submit a pr to pip later today if I can get it working. 

@tritemio and @jsarver does changing the default code page by first running `chcp 65001` fix the problem?

Having pondered this some more while I took my dogs for a walk I am disinclined to change setup.py to be pure ASCII just to accommodate the fact MS-Windows refuses to enter the 21st century and embrace Unicode. So either pip will need more changes or a workaround such as using the chcp command will hopefully work.

@krader1961 I will give that command a try and report back. Yeah it's really frustrating to have such a limited default encoding.

Running chcp 65001 did not seem to work. Interestingly enough, if you use setup.py instead of pip, both install and wheel operations run perfectly.
Both

``` python
python setup.py install
```

and 

``` python
python setup.py bdist_wheel 
```

Run just fine without even listing a utf-8 encoding in the setup.py so definitely appears to be pip specific behavior.

Sadly, as awful as this is, I think we need to accommodate this bug. Thanks for testing things out @jsarver and @krader1961 

Sure thing. I'm gonna submit an issue with pip and will put the # in the comments once I get one. Hopefully this will be a non-issue soon.

> ...run just fine without even listing a utf-8 encoding in the setup.py...

Actually, the utf-8 encoding is specified at the top of the module. It's simply formatted in a manner I'm unaccustomed to hence my initial reaction that it didn't have a valid encoding. Because it does have a valid encoding the python run-time does the right thing. The problem is that pip is reading it in a manner that relies on the system default encoding. Which on MS Windows is unlikely to be UTF-8.

You are correct, sorry for the confusion. I had removed it on my local repo of xonsh just to see if it would work.

> Sadly, as awful as this is, I think we need to accommodate this bug.

Ugh. That will mean moving the logo into a separate file, reading it with an explicit utf-8 encoding, and writing it to stdout using the default encoding. Sigh. I'll take care of it.


Resolving patch:
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -1,6 +1,8 @@
 #!/usr/bin/env python
-# -*- coding: utf-8 -*-
+# -*- coding: ascii -*-
 """The xonsh installer."""
+# Note: Do not embed any non-ASCII characters in this file until pip has been
+# fixed. See https://github.com/scopatz/xonsh/issues/487.
 from __future__ import print_function, unicode_literals
 import os
 import sys
@@ -35,6 +37,7 @@
 
 
 def clean_tables():
+    """Remove the lexer/parser modules that are dynamically created."""
     for f in TABLES:
         if os.path.isfile(f):
             os.remove(f)
@@ -42,6 +45,7 @@ def clean_tables():
 
 
 def build_tables():
+    """Build the lexer/parser modules."""
     print('Building lexer and parser tables.')
     sys.path.insert(0, os.path.dirname(__file__))
     from xonsh.parser import Parser
@@ -51,16 +55,17 @@ def build_tables():
 
 
 def install_jupyter_hook(root=None):
+    """Make xonsh available as a Jupyter kernel."""
     if not HAVE_JUPYTER:
         print('Could not install Jupyter kernel spec, please install '
               'Jupyter/IPython.')
         return
     spec = {"argv": [sys.executable, "-m", "xonsh.jupyter_kernel",
-                                     "-f", "{connection_file}"],
+                     "-f", "{connection_file}"],
             "display_name": "Xonsh",
             "language": "xonsh",
             "codemirror_mode": "shell",
-            }
+           }
     if CONDA:
         d = os.path.join(sys.prefix + '/share/jupyter/kernels/xonsh/')
         os.makedirs(d, exist_ok=True)
@@ -81,6 +86,7 @@ def install_jupyter_hook(root=None):
 
 
 class xinstall(install):
+    """Xonsh specialization of setuptools install class."""
     def run(self):
         clean_tables()
         build_tables()
@@ -89,6 +95,7 @@ def run(self):
 
 
 class xsdist(sdist):
+    """Xonsh specialization of setuptools sdist class."""
     def make_release_tree(self, basedir, files):
         clean_tables()
         build_tables()
@@ -97,6 +104,7 @@ def make_release_tree(self, basedir, files):
 
 if HAVE_SETUPTOOLS:
     class xdevelop(develop):
+        """Xonsh specialization of setuptools develop class."""
         def run(self):
             clean_tables()
             build_tables()
@@ -104,10 +112,14 @@ def run(self):
 
 
 def main():
+    """The main entry point."""
     if sys.version_info[0] < 3:
         sys.exit('xonsh currently requires Python 3.4+')
     try:
         if '--name' not in sys.argv:
+            logo_fname = os.path.join(os.path.dirname(__file__), 'logo.txt')
+            with open(logo_fname, 'rb') as f:
+                logo = f.read().decode('utf-8')
             print(logo)
     except UnicodeEncodeError:
         pass
@@ -134,7 +146,7 @@ def main():
         skw['entry_points'] = {
             'pygments.lexers': ['xonsh = xonsh.pyghooks:XonshLexer',
                                 'xonshcon = xonsh.pyghooks:XonshConsoleLexer',
-                                ],
+                               ],
             'console_scripts': ['xonsh = xonsh.main:main'],
             }
         skw['cmdclass']['develop'] = xdevelop
@@ -142,45 +154,6 @@ def main():
         skw['scripts'] = ['scripts/xonsh', 'scripts/xonsh.bat']
     setup(**skw)
 
-logo = """
-                           ╓██▄
-                          ╙██▀██╕
-                         ▐██4Φ█▀█▌
-                       ²██▄███▀██^██
-                     -███╩▀ " ╒▄█████▀█
-                      ║██▀▀W╤▄▀ ▐║█╘ ╝█
-                 ▄m▀%Φ▀▀  ╝*"    ,α█████▓▄,▄▀Γ"▀╕
-                 "▀██¼"     ▄═╦█╟║█▀ ╓ `^`   ,▄ ╢╕
-                  ,▀╫M█▐j╓╟▀ ╔▓▄█▀  '║ ╔    ╣║▌  ▀▄
-               ▄m▀▀███╬█╝▀  █▀^      "ÜM  j▐╟╫╨▒   ╙▀≡═╤═m▀╗
-               █æsæ╓  ╕, ,▄Ä   ▐'╕H   LU  ║║╠╫Å^2=⌐         █
-            ▄æ%Å███╠█ª╙▄█▀      $1╙       ║║╟╫╩*T▄           ▌
-           ╙╗%▄,╦██▌█▌█╢M         ╕      M║║║║█═⌐ⁿ"^         ╫
-             ╙╣▀████@█░█    ▌╕╕   `      ▌║▐▐║█D═≈⌐¬ⁿ      s ║⌐
-               ╙╬███▓║█`     ▌╚     ╕   ╕▌║▐▐╣▌⌐*▒▒Dù`       ▐▌
-                ╙╬██╨U█      ╟      $ ▌ ▌▌▐▐▐M█▄═≤⌐%       ╓⌐ ▌
-                 ║║█▄▌║             ╟ ▌ ▌M▐▐▐M█▀▒▒▒22,       ▐▌
-                  ███╙^▌            ║ ▌ ⌐M▐▐▐M█≤⌐⌐¬──        ▐M
-                  ║██ ▌╙   ╓       H║ ▌╒ M║▐▐M█"^^^^^"ⁿ      ║
-                   ██╕╙@▓   ╕      ▌║ H'  ║▐▐▐█══=.,,,       █
-                   ╙█▓╔╚╚█     ╠   ▌└╒ ▌▐ ╚║║║▀****ⁿ -      ╓▌
-                    ╙█▌¼V╚▌   ▌  ╕ ▌ ║╒ ║ ▌▒╠█▀≤≤≤≤≤⌐       █
-                     ╙█▌╔█╚▌     ┘ M ▌║ ╫ UUM██J^^"        ▐▌
-                      ╙██╙█╙▌  ╕$j  ▐⌐▌ ▌║╝╟█Å%%%≈═        █
-                       ╙╣█╣█^▌ ╠║▐  ║ ▌▐.DU██^[""ⁿ       -╒▌
-                         ▀█▄█`▌ ░M▀ ▌▐ Å£╝╝█╜%≈═╓""w   ⁿ⌐ █
-                          `▀▄▀`▌ ▌█▐⌐║▐UW╖██%≤═░*─    =z ▄Γ
-                            ╙██╙▄▌█  ▌Å╛╣██╨%╤ƒⁿ=    -` ▄┘
-                              █▌╢▓▌▌ W £6█╤,"ⁿ `   ▄≡▀▀▀
-                               █"█▌▌╟Å╓█╓█▀%`    ▄▀
-                               ╙▌██`▒U▓U█%╗*     █
-                                ▌╫║ ▌ÅÅ║▀╛¬`      `"█
-                                ▌╫  ╫╟ █▄     ~╦%▒╥4^
-                                ▌▌  "M█ `▀╕ X╕"╗▄▀^
-                                █▌   ╓M   ╙▀e▀▀^
-                                ╙██▄▄▀
-                                  ^^
-"""
 
 if __name__ == '__main__':
     main()


Test patch: