scrapy/w3lib — pull request #28 by kmike (2014-08-13 22:04:39+00:00)
Title: FIX w3lib.http.basic_auth_header should return bytes

This is probably backwards-incompatible and can break some code!

Scrapy expects header values to be bytes, but it auto-encodes unicode values to bytes in Headers class, so this change doesn't change anything when the result of basic_auth_header is used as a Headers value.

It also fixes some Python 3.x compatibility issues for Scrapy, e.g. this check: https://github.com/scrapy/scrapy/blob/8fece4b0b8eb8772a07673b4166cdcdb5c017eb8/scrapy/utils/request.py#L67 was incorrect because `request.headers['Authorization']` returns bytes and so it always returned False in Python 3.x.

This change should fix unicode errors on Windows in https://github.com/scrapinghub/shub project.