Display hashdigest instead of hash object upon failure.
Vendor html5lib because regex are a terrible way to parse html
Use html5lib to parse HTML instead of using regexs

HTML is not a regular language and thus cannot be accurately parsed
with a regex. The regexs that pip had were gnarly, and required
multiple levels of them in order to "work".

Using html5lib instead of regexs makes the parsing step simpler,
better tested, and much more robust.

There is still some use of regex to parse html. Namely the hack
that looks for a <th> tag with a "rel" after it. This hack is
required to work inside of HTML comments so it cannot be parsed
as HTML.
Document the reasons why we are monkeypatching six into sys.modules
Rewrite six dependency to import from pip.vendor.six instead of six

* It was determined that simply rewriting the imports was less
  invasive than patching sys.modules.
Merge remote-tracking branch 'skwash/develop' into develop

Conflicts:
	pip/download.py
Merge pull request #973 from pypa/parse-html

Use html5lib to parse HTML instead of using regexs
update git/ssh vcs forms;  clarify which forms support '-e'
4 more wheel tests
Do a check prior to returning HTMLPage that it is indeed html

Previously pip would check with a HEAD request if the url looked
like it contained an archive and skip the file. However this didn't
work if the url didn't look like an archive and instead just
redirected to an archive.

This will therefore, after the url has been fetched, inspect the
headers and look to see if the Content-Type is text/html.
Merge pull request #984 from pypa/skip-non-html

Do a check prior to returning HTMLPage that it is indeed html
Differentiate between internal and external links where possible

* By default ignore external links
* Add the ``--allow-external`` flag that enables external links
  globally
* Fallback to allowing all links if we cannot determine the
  API version of the parsed page
* Inform the user of ``--allow-external`` if nothing was found
  to install
In accordance with PEP438 default to allowing external urls

* After one release has been made allowing external urls, future
  releases will disallow by default
Differentiate between safe and unsafe urls where possible

* Links and HTMLPages know if they are "trusted"
* File Links know if they are safe or not
* A "Safe" file Link comes from a trusted Link/HTMLPage and has
  a hash allowing verification of the download
* Adds a --allow-unsafe PACKAGE argument to allow unsafe files on
  a per package basis
* Optimizes scraping external sites by short circuiting if
  the current trust rules won't allow using it's files anyways
Default to allowing unsafe urls and allow disabling them via option
Change wording from "unsafe" to "insecure"
Test the different variations of allow_insecure and allow_external
Change --allow-external to work per project and add --allow-all-external
Document the logic behind Externally Hosted Files
Enable --no-allow-insecure inside of a requirements.txt