Repository: encode/httpx

Issue:
Too much data for declared Content-Length when passing string with non-ascii characters via files parameter
### Checklist

<!-- Please make sure you check all these items before submitting your bug report. -->

- [x ] The bug is reproducible against the latest release and/or `master`.
- [ x] There are no similar issues or pull requests to fix it yet.

### Describe the bug

`h11._util.LocalProtocolError: Too much data for declared Content-Length` is raised when passing string with non-ascii characters via files parameter.

### To reproduce

```
import httpx

response = httpx.post(
    "https://httpbin.org/post",
    files={
        'upload-file': ('example.txt', '\u00E9', 'text/plain; charset=utf-8')
    }
)
response.raise_for_status()
print(response.read())
```
-->

### Expected behavior

An exception not be raised and the content length properly computed

### Actual behavior

An exception is raised.

### Debugging material

```
Traceback (most recent call last):
  File "/***redacted***/httpx_bug.py", line 5, in <module>
    response = httpx.post(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_api.py", line 296, in post
    return request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_api.py", line 93, in request
    return client.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 733, in request
    return self.send(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 767, in send
    response = self._send_handling_auth(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 805, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 837, in _send_handling_redirects
    response = self._send_single_request(request, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpx/_client.py", line 861, in _send_single_request
    (status_code, headers, stream, ext) = transport.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/connection_pool.py", line 218, in request
    response = connection.request(
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/connection.py", line 106, in request
    return self.connection.request(method, url, headers, stream, ext)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 66, in request
    self._send_request_body(stream, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 112, in _send_request_body
    self._send_event(event, timeout)
  File "/***redacted***/.venv/lib/python3.9/site-packages/httpcore/_sync/http11.py", line 123, in _send_event
    bytes_to_send = self.h11_state.send(event)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_connection.py", line 468, in send
    data_list = self.send_with_data_passthrough(event)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_connection.py", line 501, in send_with_data_passthrough
    writer(event, data_list.append)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_writers.py", line 58, in __call__
    self.send_data(event.data, write)
  File "/***redacted***/.venv/lib/python3.9/site-packages/h11/_writers.py", line 78, in send_data
    raise LocalProtocolError("Too much data for declared Content-Length")
h11._util.LocalProtocolError: Too much data for declared Content-Length
```
-->

### Environment

- OS: macOS
- Python version: `Python 3.9.1`
- HTTPX version: `0.16.1`
- Async environment: n/a
- HTTP proxy: no
- Custom certificates: no

### Additional context
n/a



Discussion:
Looks like we calculate `Content-Length` using Unicode length:
https://github.com/encode/httpx/blob/0f280af8b170ed5cc48c12a894f71a8b5762f748/httpx/_multipart.py#L90-L91

but sends with bytes:
https://github.com/encode/httpx/blob/0f280af8b170ed5cc48c12a894f71a8b5762f748/httpx/_multipart.py#L122-L123


Resolving patch:
diff --git a/httpx/_multipart.py b/httpx/_multipart.py
--- a/httpx/_multipart.py
+++ b/httpx/_multipart.py
@@ -40,11 +40,7 @@ def render_headers(self) -> bytes:
 
     def render_data(self) -> bytes:
         if not hasattr(self, "_data"):
-            self._data = (
-                self.value
-                if isinstance(self.value, bytes)
-                else self.value.encode("utf-8")
-            )
+            self._data = to_bytes(self.value)
 
         return self._data
 
@@ -88,7 +84,7 @@ def get_length(self) -> int:
         headers = self.render_headers()
 
         if isinstance(self.file, (str, bytes)):
-            return len(headers) + len(self.file)
+            return len(headers) + len(to_bytes(self.file))
 
         # Let's do our best not to read `file` into memory.
         try:


Test patch:
diff --git a/tests/test_multipart.py b/tests/test_multipart.py
--- a/tests/test_multipart.py
+++ b/tests/test_multipart.py
@@ -133,6 +133,29 @@ def test_multipart_encode(tmp_path: typing.Any) -> None:
         assert content == b"".join(stream)
 
 
+def test_multipart_encode_unicode_file_contents() -> None:
+    files = {"file": ("name.txt", "<únicode string>")}
+
+    with mock.patch("os.urandom", return_value=os.urandom(16)):
+        boundary = os.urandom(16).hex()
+
+        headers, stream = encode_request(files=files)
+        assert isinstance(stream, typing.Iterable)
+
+        content = (
+            '--{0}\r\nContent-Disposition: form-data; name="file";'
+            ' filename="name.txt"\r\n'
+            "Content-Type: text/plain\r\n\r\n<únicode string>\r\n"
+            "--{0}--\r\n"
+            "".format(boundary).encode("utf-8")
+        )
+        assert headers == {
+            "Content-Type": f"multipart/form-data; boundary={boundary}",
+            "Content-Length": str(len(content)),
+        }
+        assert content == b"".join(stream)
+
+
 def test_multipart_encode_files_allows_filenames_as_none() -> None:
     files = {"file": (None, io.BytesIO(b"<file content>"))}
     with mock.patch("os.urandom", return_value=os.urandom(16)):