Repository: pypa/pip

Issue:
UnicodeDecodeError when using pip
* Pip version: 10.0.0.dev0
* Python version: 3.5.4
* Operating system: windows 10, 1709, 64-bit

### Description:
When i used `pip list` command in CMD, i got this error: `UnicodeDecodeError: 'gbk' codec can't decode byte 0x90 in position 101: illegal multibyte sequence`. Other pip command like `pip install xxx`  gave me same error.

I have tried following methods, but nothing changed.
+ try pip 9.0.1(after this failed, i tried pip 10.0.0 installed it from sorce code)
+ try `chcp 65001`
+ try python 3.6.4
+ try robinxb's method from issue #4251, though the error occured were not quite the same.

### What I've run:

```
C:\Users\cyl>pip list
Traceback (most recent call last):
  File "C:\Users\cyl\AppData\Local\Programs\Python\Python35\Scripts\pip-script.py", line 11, in <module>
    load_entry_point('pip==10.0.0.dev0', 'console_scripts', 'pip')()
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\__init__.py", line 229, in main
    cmd_name, cmd_args = parseopts(args)
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\__init__.py", line 178, in parseopts
    general_options, args_else = parser.parse_args(args)
  File "C:\Users\cyl\AppData\Local\Programs\Python\Python35\lib\optparse.py", line 1370, in parse_args
    values = self.get_default_values()
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\baseparser.py", line 227, in get_default_values
    defaults = self._update_defaults(self.defaults.copy())  # ours
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\baseparser.py", line 181, in _update_defaults
    self.config.load()
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\configuration.py", line 111, in load
    self._load_config_files()
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\configuration.py", line 262, in _load_config_files
    parser = self._load_file(variant, fname)
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\configuration.py", line 270, in _load_file
    parser = self._construct_parser(fname)
  File "c:\users\cyl\appdata\local\programs\python\python35\lib\site-packages\pip-10.0.0.dev0-py3.5.egg\pip\_internal\configuration.py", line 286, in _construct_parser
    parser.read(fname)
  File "C:\Users\cyl\AppData\Local\Programs\Python\Python35\lib\configparser.py", line 696, in read
    self._read(fp, filename)
  File "C:\Users\cyl\AppData\Local\Programs\Python\Python35\lib\configparser.py", line 1012, in _read
    for lineno, line in enumerate(fp, start=1):
UnicodeDecodeError: 'gbk' codec can't decode byte 0x90 in position 101: illegal multibyte sequence
```



Discussion:
Hi @TDilcy!

This is because of an encoding releated error while reading pip's configuration file. Could you look in the pip configuration file to check and see if there's a stray character there that might be causing this crash? It should be `%APPDATA%\pip\pip.ini`.

None the less, pip should probably print a better error message in this case. :)

Specifically, your `pip.ini` file should contain text in your PC's default (ANSI) encoding. Using `chcp` won't affect that, as that only alters the *console* codepage, not the ANSI one.

As you noticed, #4251 is a different issue, and the fix for that (which is in pip 10.0.0dev0) doesn't affect this problem. (Prior to Python 3.2, the encoding support in `ConfigParser` was very limited, so we'll have problems improving the current behaviour until we can drop Python 2.x support :cry: )
Just to confirm...

@pfmoore just printing a message saying "hey, I couldn't read this configuration file because of this error" which prints the traceback on verbose, would be an acceptable improvement for now; correct?

Yeah, that's a fair point :-) You could even note the expected encoding by checking `locale.getpreferredencoding(False)`.
> You could even note the expected encoding by checking locale.getpreferredencoding(False).

Sounds good. :)
Uhm... So I forgot that configuration is loaded _before_ command line arguments are parsed. So, we can't really have it conditional on verbosity.

Printing the entire traceback every single time is not a really the best thing we can do but that's the only idea I have right now; thoughts?

If we're catching precisely that error in precisely that line of code, i don't see that we need the traceback. Just give a helpful error message, terminate the program, and you're done.

```
> pip list
ERROR: Configuration file contains invalid cp1252 characters.
Please fix your config (located at C:\Users\XXXX\AppData\Roaming\Pip\pip.ini)
```
@pfmoore thanks, but i have reinstalled my windows 10 to solve the problem...... :joy: 

Resolving patch:
diff --git a/src/pip/_internal/baseparser.py b/src/pip/_internal/baseparser.py
--- a/src/pip/_internal/baseparser.py
+++ b/src/pip/_internal/baseparser.py
@@ -9,7 +9,7 @@
 
 from pip._vendor.six import string_types
 
-from pip._internal.configuration import Configuration
+from pip._internal.configuration import Configuration, ConfigurationError
 from pip._internal.utils.misc import get_terminal_size
 
 logger = logging.getLogger(__name__)
@@ -177,9 +177,6 @@ def _update_defaults(self, defaults):
         the environ. Does a little special handling for certain types of
         options (lists)."""
 
-        # Load the configuration
-        self.config.load()
-
         # Accumulate complex default state.
         self.values = optparse.Values(self.defaults)
         late_eval = set()
@@ -224,6 +221,12 @@ def get_default_values(self):
             # Old, pre-Optik 1.5 behaviour.
             return optparse.Values(self.defaults)
 
+        # Load the configuration, or error out in case of an error
+        try:
+            self.config.load()
+        except ConfigurationError as err:
+            self.exit(2, err.args[0])
+
         defaults = self._update_defaults(self.defaults.copy())  # ours
         for option in self._get_all_options():
             default = defaults.get(option.dest)
diff --git a/src/pip/_internal/configuration.py b/src/pip/_internal/configuration.py
--- a/src/pip/_internal/configuration.py
+++ b/src/pip/_internal/configuration.py
@@ -11,6 +11,7 @@
   A single word describing where the configuration key-value pair came from
 """
 
+import locale
 import logging
 import os
 
@@ -283,8 +284,14 @@ def _construct_parser(self, fname):
         # Doing this is useful when modifying and saving files, where we don't
         # need to construct a parser.
         if os.path.exists(fname):
-            parser.read(fname)
-
+            try:
+                parser.read(fname)
+            except UnicodeDecodeError:
+                raise ConfigurationError((
+                    "ERROR: "
+                    "Configuration file contains invalid %s characters.\n"
+                    "Please fix your configuration, located at %s\n"
+                ) % (locale.getpreferredencoding(False), fname))
         return parser
 
     def _load_environment_vars(self):


Test patch: