Repository: kevin1024/vcrpy

Issue:
TypeError: a bytes-like object is required, not 'str' when processing header object
```
headers = {'X-Amz-Target': b'<target>', 'Content-Type': b'<content-type>', 'User-Agent': b'<user-agent>', Signature=b'<sig>', 'Content-Length': '<len>'}

    def checker(headers):
>       return value in headers.get(header, '').lower()
E       TypeError: a bytes-like object is required, not 'str'

.venv/lib/python3.6/site-packages/vcr/matchers.py:56: TypeError
```

The code on line 56 in matchers fails unexpectedly. The content type return as a bytes like object   but the code still fails. When we convert the bytes object to a string it works. Replacing the current function at https://github.com/kevin1024/vcrpy/blob/master/vcr/matchers.py#L54-L57:
```
def _header_checker(value, header='Content-Type'):
    def checker(headers):
        return value in headers.get(header, '').lower()
    return checker
```
with  
```
def _header_checker(value, header='Content-Type'):
    def checker(headers):
        return value in headers.get(header, '').decode('utf-8').lower()
    return checker
```
fixes this issue especially with boto3 requests. 

Would there be any risk with making this change?

Discussion:
simahawk: travis red on some combinations as `master`. Apparently not related to this PR.
neozenith: Changes look good. Thanks for the PR!

You’re right, there is something on master CI we haven’t got around to addressing that’s throwing some red. 

When I get some spare time I’ll have a look to clear that so we can push this through. 

If you could add the associated test that’d be appreciated. 

And.... add in `docs/Changelog.rst` a line to claim credit. 
neozenith: Turns out the CI red is due to a Yarl version issue raised in #485 
neozenith: Fixed in #489, please rebase with master and let's have another crack at getting this pushed through. Due to release to PyPI the accumulated patches too.
simahawk: @neozenith rebased, tests updated
simahawk: forgot to run black. Any reason not to use https://pre-commit.com/ here?
codecov-io: # [Codecov](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@d843d2a`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/kevin1024/vcrpy/pull/488/graphs/tree.svg?width=650&token=YOLqUk37fb&height=150&src=pr)](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master     #488   +/-   ##
=========================================
  Coverage          ?   89.41%           
=========================================
  Files             ?       24           
  Lines             ?     1549           
  Branches          ?      216           
=========================================
  Hits              ?     1385           
  Misses            ?      135           
  Partials          ?       29
```


| [Impacted Files](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [vcr/matchers.py](https://codecov.io/gh/kevin1024/vcrpy/pull/488/diff?src=pr&el=tree#diff-dmNyL21hdGNoZXJzLnB5) | `96% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=footer). Last update [d843d2a...f7457d9](https://codecov.io/gh/kevin1024/vcrpy/pull/488?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/docs/changelog.rst b/docs/changelog.rst
index e728107..bda16ba 100644
--- a/docs/changelog.rst
+++ b/docs/changelog.rst
@@ -7,6 +7,7 @@ Changelog
   - Fix exception when body is empty (@keithprickett)
   - Add `pytest-recording` to the documentation as an alternative Pytest plugin (@Stranger6667)
   - Fix yarl and python3.5 version issue (@neozenith)
+  - Fix header matcher for boto3 - fixes #474 (@simahawk)
 -  2.1.0 - Add a `rewind` method to reset a cassette (thanks @khamidou)
    New error message with more details on why the cassette failed to play a request (thanks @arthurHamon2, @neozenith)
    Handle connect tunnel URI (thanks @jeking3)
diff --git a/vcr/matchers.py b/vcr/matchers.py
index 7a969f4..eabd61f 100644
--- a/vcr/matchers.py
+++ b/vcr/matchers.py
@@ -53,7 +53,10 @@ def headers(r1, r2):
 
 def _header_checker(value, header="Content-Type"):
     def checker(headers):
-        return value in headers.get(header, "").lower()
+        _header = headers.get(header, "")
+        if isinstance(_header, bytes):
+            _header = _header.decode("utf-8")
+        return value in _header.lower()
 
     return checker
 


Test patch:
diff --git a/tests/unit/test_matchers.py b/tests/unit/test_matchers.py
index 4d161ca..34eadd3 100644
--- a/tests/unit/test_matchers.py
+++ b/tests/unit/test_matchers.py
@@ -54,6 +54,16 @@ req2_body = (
     b"<member><name>a</name><value><string>1</string></value></member>"
     b"</struct></value></data></array></value></param></params></methodCall>"
 )
+boto3_bytes_headers = {
+    "X-Amz-Content-SHA256": b"UNSIGNED-PAYLOAD",
+    "Cache-Control": b"max-age=31536000, public",
+    "X-Amz-Date": b"20191102T143910Z",
+    "User-Agent": b"Boto3/1.9.102 Python/3.5.3 Linux/4.15.0-54-generic Botocore/1.12.253 Resource",
+    "Content-MD5": b"GQqjEXsRqrPyxfTl99nkAg==",
+    "Content-Type": b"text/plain",
+    "Expect": b"100-continue",
+    "Content-Length": "21",
+}
 
 
 @pytest.mark.parametrize(
@@ -110,6 +120,11 @@ req2_body = (
                 "POST", "http://host.com/", '{"b": 2, "a": 1}', {"content-type": "application/json"}
             ),
         ),
+        (
+            # special case for boto3 bytes headers
+            request.Request("POST", "http://aws.custom.com/", b"123", boto3_bytes_headers),
+            request.Request("POST", "http://aws.custom.com/", b"123", boto3_bytes_headers),
+        ),
     ],
 )
 def test_body_matcher_does_match(r1, r2):