Repository: seanmonstar/reqwest (language: rust)

Issue:
400 Bad Request after upgrading to 0.12.1
Some requests have started failing after upgrading from 0.11.27 to 0.12.1.

In this case, api.figma.com works in 0.11.27 and fails in 0.12.1. A test of www.figma.com shows that both versions of reqwest are fine.

Results on 0.11.27:
```text
Success!
```

Results on 0.12.1:
```text
Failed to retrieve data: 400 Bad Request
[src/main.rs:12:9] res = "<html>\r\n<head><title>400 Bad Request</title></head>\r\n<body>\r\n<center><h1>400 Bad Request</h1></center>\r\n</body>\r\n</html>\r\n"
```

main.toml:
```toml
[package]
name = "fetch-fail"
version = "0.1.0"
edition = "2021"
publish = false

[dependencies]
anyhow = "1.0.81"
reqwest = "0.11.27"
#reqwest = "0.12.1"
tokio = { version = "1.36.0", features = ["full"] }
```

main.rs:
```rust
#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // let res = reqwest::get("https://www.figma.com/").await?;
    // let res = reqwest::get("https://api.figma.com/v1/activity_logs").await?;
    let res = reqwest::get("https://api.figma.com/").await?;
    let status = res.status();
    let res = res.text().await?;

    if status == 200 {
        println!("Success!");
    } else {
        eprintln!("Failed to retrieve data: {status}");
        dbg!(res);
    }

    Ok(())
}
```


Discussion / hints:
Cannot reproduce. You may check if you request were risk controlled for missing headers, invalid UA, etc.
Thanks for taking a look. I don't think the headers are the issue in this case, but it's definitely possible that the problem lies elsewhere, probably a weird interaction between Figma's tech stack and Reqwest's dependencies. I'll try to do some additional testing, varying by egress location, platform, architecture, and TLS provider.

With regards to bad headers, I'm intentionally not submitting any headers to keep the test case brief. With the code as written I get a 200 Success in 0.11.27, and a 400 Error on 0.12.x. The code, hardware, environment, and IP address are all identical. If _you_ aren't experiencing the same change of behavior between versions, then something subtle and strange is happening somewhere. :)

Just to add some data points, `https://api.figma.com/v1/me`,  0.11.27 correctly returns a 403 Forbidden with body `{"status":403, "err": "Invalid token"}` whereas 0.12.1 still comes back with a 400 Bad Request error identical to the above. When supplying a proper bearer token, I'm able to query the API without issue in 0.11.27, while I only get 400's in 0.12.x.

Is it the HTTP version? I wonder if the recent fix for ALPN is what was missing?
I tried adding the `native-tls-alpn` feature, and all of my 0.12 tests are failing, rather than just the ones against the API.
`thread 'figma_root_12' panicked at src/main.rs:23:59:
called `Result::unwrap()` on an `Err` value: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("figma.com")), port: None, path: "/", query: None, fragment: None }, source: Error { kind: SendRequest, source: Some(hyper::Error(Parse(Version))) } }`

Updated main.rs for easier testing:
```rust
#[tokio::main]
async fn main() {
    println!("It Works! (on my machine)");
}

#[tokio::test]
async fn figma_root_11() {
    let res = reqwest_11::get("https://figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_root_12() {
    let res = reqwest_12::get("https://figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_www_root_11() {
    let res = reqwest_11::get("https://www.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_www_root_12() {
    let res = reqwest_12::get("https://www.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_11() {
    let res = reqwest_11::get("https://api.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_12() {
    let res = reqwest_12::get("https://api.figma.com/").await.unwrap();
    assert_eq!(res.status(), 200, "Expected 200 Success: {res:#?}");
}

#[tokio::test]
async fn figma_api_me_11() {
    let res = reqwest_11::get("https://api.figma.com/v1/me")
        .await
        .unwrap();
    assert_eq!(res.status(), 403, "Expected 403 Forbidden: {res:#?}");
}

#[tokio::test]
async fn figma_api_api_me_12() {
    let res = reqwest_12::get("https://api.figma.com/v1/me")
        .await
        .unwrap();
    assert_eq!(res.status(), 403, "Expected 403 Forbidden: {res:#?}");
}
```

cargo.toml:
```toml
[package]
name = "reqwest-narrowing"
version = "0.1.0"
edition = "2021"
publish = false

[dependencies]
anyhow = "1.0.81"
reqwest_11 = { package = "reqwest", version = "=0.11.27",  default-features = false, features = ["default-tls", "native-tls-alpn"]  }
reqwest_12 = { package = "reqwest", version = "=0.12.1", default-features = false, features = ["default-tls", "native-tls-alpn"]  }
tokio = { version = "1.36.0", features = ["full"] }
```

Using `native-tls` without `native-tls-alpn`, I get the original errors. Looking a bit more closely at it, CloudFront is definitely unhappy with something.
```text
thread 'figma_api_root_12' panicked at src/main.rs:39:5:
assertion `left == right` failed: Expected 200 Success: Response {
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "api.figma.com",
            ),
        ),
        port: None,
        path: "/",
        query: None,
        fragment: None,
    },
    status: 400,
    headers: {
        "content-type": "text/html",
        "content-length": "122",
        "connection": "keep-alive",
        "server": "awselb/2.0",
        "date": "Sat, 23 Mar 2024 17:15:28 GMT",
        "x-cache": "Error from cloudfront",
        "via": "1.1 9857ab0da41c7a88865f55b9cdc654ac.cloudfront.net (CloudFront)",
        "x-amz-cf-pop": "BRU50-P1",
        "alt-svc": "h3=\":443\"; ma=86400",
        "x-amz-cf-id": "KjDkHp4V54YzJ07HnVqvrRR84p5spOSlo_ZERc4pJreT3iB2Z-BPjQ==",
        "strict-transport-security": "max-age=31536000; includeSubDomains; preload",
    },
}
  left: 400
 right: 200
```

I successfully reproduce it and native-tls alpn should take the blame. #2165 partially fix this issue(through feat `native-tls-alpn` enabled). I'm investigating details.
With regards to HTTP version, I've tried forcing HTTP 1:
```rust
#[tokio::test]
async fn figma_api_root_http1_12() {
    let client = reqwest_12::Client::builder().http1_only().build().unwrap();
    let res = client.get("https://api.figma.com/")
        .send()
        .await
        .unwrap();
    assert_eq!(res.status(), 200, "Expected 403 Forbidden: {res:#?}");
}

#[tokio::test]
async fn figma_api_root_http1_11() {
    let client = reqwest_11::Client::builder().http1_only().build().unwrap();
    let res = client.get("https://api.figma.com/")
        .send()
        .await
        .unwrap();
    assert_eq!(res.status(), 200, "Expected 403 Forbidden: {res:#?}");
}
```

Still no luck in 0.12:
```text
thread 'figma_api_root_http1_12' panicked at src/main.rs:66:5:
assertion `left == right` failed: Expected 403 Forbidden: Response {
    url: Url {
        scheme: "https",
        cannot_be_a_base: false,
        username: "",
        password: None,
        host: Some(
            Domain(
                "api.figma.com",
            ),
        ),
        port: None,
        path: "/",
        query: None,
        fragment: None,
    },
    status: 400,
    headers: {
        "content-type": "text/html",
        "content-length": "122",
        "connection": "keep-alive",
        "server": "awselb/2.0",
        "date": "Sat, 23 Mar 2024 17:34:07 GMT",
        "x-cache": "Error from cloudfront",
        "via": "1.1 00f66bc6263192200d1a0cdb83e969f8.cloudfront.net (CloudFront)",
        "x-amz-cf-pop": "AMS1-P1",
        "alt-svc": "h3=\":443\"; ma=86400",
        "x-amz-cf-id": "EteG0kIXI6Rp5Nea2spoG8GO1apYn8leUqtCg0umO4hJAtNGC00__A==",
        "strict-transport-security": "max-age=31536000; includeSubDomains; preload",
    },
}
  left: 400
 right: 200
```
Through investgation, I think `reqwest` may do nothing wrong but `hyper`.

0.11.27
```
GET /v1/me HTTP/1.1
accept: */*
host: api.figma.com
```

0.12.1
```
GET /v1/me HTTP/1.1
accept: */*
host: api.figma.com
content-length: 0
```

We can see what's different is just `content-length`.

It's added here in hyper:
![image](https://github.com/seanmonstar/reqwest/assets/70561268/240f9a59-24ba-496e-aec6-587af99a2ab7)

Tested manually with `openssl s_client -connect api.figma.com:443`:

![image](https://github.com/seanmonstar/reqwest/assets/70561268/4513747f-27c9-44e3-9123-dce62a02674f)
![image](https://github.com/seanmonstar/reqwest/assets/70561268/b5d69a86-c3e1-4852-937a-b1e66010ec40)
![image](https://github.com/seanmonstar/reqwest/assets/70561268/882d7efb-71f7-42a2-aba3-890f1aa6634b)
![image](https://github.com/seanmonstar/reqwest/assets/70561268/ff63ab51-2742-4a74-a846-674fa065fced)

So funny test result...

When enable alpn, h2 will be used and `content-length` will not be added, so everything works well.

Not familiar with `hyper` and more details should be investigated.


Hmmm. Or it could be that as part of the upgrade, since I changed around the body implementation, it might be indicating there is a body to send (but ends up being empty).
I am willing to try to fix this issue.


When making a request to an HTTPS website, a Content-Length: 0 request header is added. 

However, for HTTP websites, this request header is not added.


---
request to `https://httpbin.org/anything` 
```
❯ cargo run --example simple  https://httpbin.org/anything
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/chen/.rust-target/debug/examples/simple 'https://httpbin.org/anything'`
Fetching "https://httpbin.org/anything"...
Response: HTTP/1.1 200 OK
Headers: {
    "date": "Tue, 26 Mar 2024 07:03:33 GMT",
    "content-type": "application/json",
    "content-length": "339",
    "connection": "keep-alive",
    "server": "gunicorn/19.9.0",
    "access-control-allow-origin": "*",
    "access-control-allow-credentials": "true",
}

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Content-Length": "0", 
    "Host": "httpbin.org", 
    "X-Amzn-Trace-Id": "Root=1-66027345-4fa3fe350e01878628a9e502"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "103.167.134.150", 
  "url": "https://httpbin.org/anything"
}
```
request to `http://httpbin.org/anything` 
```
❯ cargo run --example simple  http://httpbin.org/anything
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `/Users/chen/.rust-target/debug/examples/simple 'http://httpbin.org/anything'`
Fetching "http://httpbin.org/anything"...
Response: HTTP/1.1 200 OK
Headers: {
    "connection": "close",
    "content-length": "383",
    "access-control-allow-credentials": "true",
    "access-control-allow-origin": "*",
    "content-type": "application/json",
    "date": "Tue, 26 Mar 2024 07:03:48 GMT",
    "server": "gunicorn/19.9.0",
}

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip", 
    "Host": "httpbin.org", 
    "User-Agent": "Go-http-client/1.1", 
    "X-Amzn-Trace-Id": "Root=1-66027354-206384396fcdddbc0e226b85"
  }, 
  "json": null, 
  "method": "GET", 
  "origin": "103.167.134.150", 
  "url": "http://httpbin.org/anything"
}
```





Fix patch:
diff --git a/src/async_impl/body.rs b/src/async_impl/body.rs
index ff5446e53..3ba9350de 100644
--- a/src/async_impl/body.rs
+++ b/src/async_impl/body.rs
@@ -267,14 +267,17 @@ impl HttpBody for Body {
 
     fn size_hint(&self) -> http_body::SizeHint {
         match self.inner {
-            Inner::Reusable(ref bytes) => {
-                let mut hint = http_body::SizeHint::default();
-                hint.set_exact(bytes.len() as u64);
-                hint
-            }
+            Inner::Reusable(ref bytes) => http_body::SizeHint::with_exact(bytes.len() as u64),
             Inner::Streaming(ref body) => body.size_hint(),
         }
     }
+
+    fn is_end_stream(&self) -> bool {
+        match self.inner {
+            Inner::Reusable(ref bytes) => bytes.is_empty(),
+            Inner::Streaming(ref body) => body.is_end_stream(),
+        }
+    }
 }
 
 // ===== impl TotalTimeoutBody =====
@@ -352,6 +355,8 @@ where
 
 #[cfg(test)]
 mod tests {
+    use http_body::Body as _;
+
     use super::Body;
 
     #[test]
@@ -360,4 +365,19 @@ mod tests {
         let body = Body::from(&test_data[..]);
         assert_eq!(body.as_bytes(), Some(&test_data[..]));
     }
+
+    #[test]
+    fn body_exact_length() {
+        let empty_body = Body::empty();
+        assert!(empty_body.is_end_stream());
+        assert_eq!(empty_body.size_hint().exact(), Some(0));
+
+        let bytes_body = Body::reusable("abc".into());
+        assert!(!bytes_body.is_end_stream());
+        assert_eq!(bytes_body.size_hint().exact(), Some(3));
+
+        let stream_body = Body::streaming(bytes_body);
+        assert!(!stream_body.is_end_stream());
+        assert_eq!(stream_body.size_hint().exact(), None);
+    }
 }


Test patch:
diff --git a/tests/blocking.rs b/tests/blocking.rs
index c4ea04bef..7d57db7b4 100644
--- a/tests/blocking.rs
+++ b/tests/blocking.rs
@@ -1,7 +1,6 @@
 mod support;
 
-#[cfg(feature = "json")]
-use http::header::CONTENT_TYPE;
+use http::header::{CONTENT_LENGTH, CONTENT_TYPE, TRANSFER_ENCODING};
 use http_body_util::BodyExt;
 #[cfg(feature = "json")]
 use std::collections::HashMap;
@@ -21,6 +20,29 @@ fn test_response_text() {
     assert_eq!(b"Hello", body.as_bytes());
 }
 
+#[test]
+fn donot_set_conent_length_0_if_have_no_body() {
+    let server = server::http(move |req| async move {
+        let headers = req.headers();
+        assert_eq!(headers.get(CONTENT_LENGTH), None);
+        assert!(headers.get(CONTENT_TYPE).is_none());
+        assert!(headers.get(TRANSFER_ENCODING).is_none());
+        dbg!(&headers);
+        http::Response::default()
+    });
+
+    let url = format!("http://{}/conent-length", server.addr());
+    let res = reqwest::blocking::Client::builder()
+        .no_proxy()
+        .build()
+        .expect("client builder")
+        .post(&url)
+        .send()
+        .expect("request");
+
+    assert_eq!(res.status(), reqwest::StatusCode::OK);
+}
+
 #[test]
 #[cfg(feature = "charset")]
 fn test_response_non_utf_8_text() {
diff --git a/tests/client.rs b/tests/client.rs
index 02f32fc85..bc6002fc7 100644
--- a/tests/client.rs
+++ b/tests/client.rs
@@ -3,8 +3,7 @@ mod support;
 
 use support::server;
 
-#[cfg(feature = "json")]
-use http::header::CONTENT_TYPE;
+use http::header::{CONTENT_LENGTH, CONTENT_TYPE, TRANSFER_ENCODING};
 #[cfg(feature = "json")]
 use std::collections::HashMap;
 
@@ -54,6 +53,30 @@ async fn auto_headers() {
     assert_eq!(res.remote_addr(), Some(server.addr()));
 }
 
+#[tokio::test]
+async fn donot_set_conent_length_0_if_have_no_body() {
+    let server = server::http(move |req| async move {
+        let headers = req.headers();
+        assert_eq!(headers.get(CONTENT_LENGTH), None);
+        assert!(headers.get(CONTENT_TYPE).is_none());
+        assert!(headers.get(TRANSFER_ENCODING).is_none());
+        dbg!(&headers);
+        http::Response::default()
+    });
+
+    let url = format!("http://{}/conent-length", server.addr());
+    let res = reqwest::Client::builder()
+        .no_proxy()
+        .build()
+        .expect("client builder")
+        .get(&url)
+        .send()
+        .await
+        .expect("request");
+
+    assert_eq!(res.status(), reqwest::StatusCode::OK);
+}
+
 #[tokio::test]
 async fn user_agent() {
     let server = server::http(move |req| async move {