Repository: caddyserver/caddy (language: go)

Issue:
HTTP_PROXY env not working
## 1. Environment
### 1a. Operating system and version
Using the docker `caddy:2-alpine` image
```
Linux 4d5a25798135 5.10.104-linuxkit #1 SMP PREEMPT Thu Mar 17 17:05:54 UTC 2022 aarch64 Linux
```


### 1b. Caddy version (run `caddy version` or paste commit SHA)

```
v2.6.2 h1:wKoFIxpmOJLGl3QXoo6PNbYvGW4xLEgo32GPBEjWL8o=
```


## 2. Description

### 2a. What happens (briefly explain what is wrong)
For a whole week now I'm trying to setup caddy as reverse proxy to proxy requests over tailscale into my private home network.
Over time my minimal reproducable case has evolved into using tinyproxy except of the tailscale proxy from the [userspace networking mode](https://tailscale.com/kb/1112/userspace-networking/) and also currently I'm testing everything from within the docker container itself. Thats why currently the caddy file binds to `http://172.17.0.2` except for an external domain.

I have gotten multiple successful curl requests by specifying the proxy
```sh
// using the tailscale proxy
ALL_PROXY=localhost:3128 curl http://100.81.184.38:3000 //or www.google.com if you don't have a tailscale node
```

But no matter if I use `ALL_PROXY` `http_proxy` or `HTTP_PROXY` I always fail to use the proxy through caddy. Help!

### 2c. Log output

```
http.log.error	dial tcp 100.81.184.38:3000: connect: connection refused	request": {"remote_ip": "172.17.0.2", "remote_port": "38404", "proto": "HTTP/1.1", "method": "GET", "host": "172.17.0.2", "uri": "/api", "headers": {"User-Agent": ["curl/7.83.1"], "Accept": ["*/*"]}}, "duration": 0.041261001, "status": 502, "err_id": "wv1ugcp6m", "err_trace": "reverseproxy.statusError (reverseproxy.go:1272)"}
```

### 2e. Related issues
I know that this seems similar to #1433 and I know that this issue ended because of no minimal reproduction case. Thats why I already put some work in, to put everything into a Dockerfile and Repo so it should be reproducible across machines

### 2d. Related Links
For the tailscale case, [this doc](https://tailscale.com/kb/1107/heroku/) is helpful because I have to use tailscale in [userspace networking mode](https://tailscale.com/kb/1112/userspace-networking/) because of my deployment server

## 3. Tutorial (minimal steps to reproduce the bug)
Reproduction Repo: https://github.com/8BitJonny/caddy-proxy-minimal-repro/tree/main with Docker.

This reproduction repo is setup to show my struggle to use caddy with a proxy, with two different proxies. You can run the docker container and reproduce it with the [tailscale](https://tailscale.com/) proxy or with the most minimal proxy using [tinyproxy](https://github.com/tinyproxy/tinyproxy)

Use the repo and run 
```sh
docker build -t caddy_proxy:latest .
docker run -ti -p 43890:43890 --env TAILSCALE_AUTHKEY=TailScale-ApiKey --name caddy caddy_proxy
```
to start the docker container. You'll only need the `TAILSCALE_AUTHKEY` env if you want to reproduce it with tailscale. Currently its setup for reproduction with tinyproxy and then the env won't be used.

And in the second terminal the following fails
```sh
docker exec -it caddy /bin/sh
curl http://172.17.0.2/api //This errors with either proxy successfully forwarding the request
```




Discussion / hints:
Thanks for opening an issue.

To make sure I understand, you are expecting Caddy to honor the `HTTP_PROXY` environment variable when using Caddy's `reverse_proxy`? i.e. when the reverse proxy dials the backend, it should be honoring the `HTTP_PROXY` value? Is this documented in our docs somewhere? :thinking: 
Yes, that was my expectation of it. I'm not that experienced in networking but through the tailscale documentation:
> Many widely used networking packages support these proxies already, generally by setting the ALL_PROXY and HTTP_PROXY environment variables

I thought maybe caddie doesn't support it directly but indirectly by using a network library under the hood that does support and react to these env vars.

And then I saw https://github.com/caddyserver/caddy/issues/1433 where everybody was talking about it as if Caddy was honoring these env vars. This plus the [caddy v1 -> v2 migration guide](https://caddyserver.com/docs/v2-upgrade#proxy) mentioning that 
> The v2 equivalent [of proxy] is reverse_proxy

I then assumed that it was in fact a valid way to pass a proxy value to caddy because you yourself, said in https://github.com/caddyserver/caddy/issues/1433 that you couldn't reproduce the issue and hence that Caddy honors the value of HTTP_PROXY, despite the docs never mentioning it anywhere.
But I mean, that is also a valid answer in and of itself. If you say, this is not a feature of Caddy anymore, then I can quite trying to get it to work like this and try a different approach 😄 
I'm a bit confused, what in the v1->v2 upgrade guide says that the proxy uses HTTP_PROXY env var? :monocle_face: 

Anyway, it's not a feature I ever used, so if it worked, it was by accident. I'm not saying it shouldn't work, but in general, with v2, we don't use env vars because they're not flexible/powerful enough for what v2 requires.

I suppose this issue is more of a feature request than a bug report, then?

Anyone is welcome to contribute to this!
Hello, jumping on this discussion as we have a similar problem. We have a requirement that traffic must go through a forward proxy (Squid proxy) before reaching the internet. We set up the HTTPS_PROXY and HTTP_PROXY env variables on the Caddy VM but it does not seems to work: Caddy doesn't send the Letsencrypt connections and others through the proxy, it just get stuck (no internet connection). Any workarounds? Thanks 

Fix patch:
diff --git a/modules/caddyhttp/reverseproxy/httptransport.go b/modules/caddyhttp/reverseproxy/httptransport.go
index ec5d2f23aed..71d06d5a6ce 100644
--- a/modules/caddyhttp/reverseproxy/httptransport.go
+++ b/modules/caddyhttp/reverseproxy/httptransport.go
@@ -172,12 +172,19 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e
 		}
 	}
 
-	// Set up the dialer to pull the correct information from the context
 	dialContext := func(ctx context.Context, network, address string) (net.Conn, error) {
-		// the proper dialing information should be embedded into the request's context
+		// For unix socket upstreams, we need to recover the dial info from
+		// the request's context, because the Host on the request's URL
+		// will have been modified by directing the request, overwriting
+		// the unix socket filename.
+		// Also, we need to avoid overwriting the address at this point
+		// when not necessary, because http.ProxyFromEnvironment may have
+		// modified the address according to the user's env proxy config.
 		if dialInfo, ok := GetDialInfo(ctx); ok {
-			network = dialInfo.Network
-			address = dialInfo.Address
+			if strings.HasPrefix(dialInfo.Network, "unix") {
+				network = dialInfo.Network
+				address = dialInfo.Address
+			}
 		}
 
 		conn, err := dialer.DialContext(ctx, network, address)
@@ -188,8 +195,8 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e
 			return nil, DialError{err}
 		}
 
-		// if read/write timeouts are configured and this is a TCP connection, enforce the timeouts
-		// by wrapping the connection with our own type
+		// if read/write timeouts are configured and this is a TCP connection,
+		// enforce the timeouts by wrapping the connection with our own type
 		if tcpConn, ok := conn.(*net.TCPConn); ok && (h.ReadTimeout > 0 || h.WriteTimeout > 0) {
 			conn = &tcpRWTimeoutConn{
 				TCPConn:      tcpConn,
@@ -203,6 +210,7 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e
 	}
 
 	rt := &http.Transport{
+		Proxy:                  http.ProxyFromEnvironment,
 		DialContext:            dialContext,
 		MaxConnsPerHost:        h.MaxConnsPerHost,
 		ResponseHeaderTimeout:  time.Duration(h.ResponseHeaderTimeout),


Test patch:
diff --git a/caddytest/integration/reverseproxy_test.go b/caddytest/integration/reverseproxy_test.go
index f7b1967232f..4333d129ffa 100644
--- a/caddytest/integration/reverseproxy_test.go
+++ b/caddytest/integration/reverseproxy_test.go
@@ -22,39 +22,39 @@ func TestSRVReverseProxy(t *testing.T) {
 		},
 		"apps": {
 			"pki": {
-				"certificate_authorities" : {
-					"local" : {
-					"install_trust": false
+				"certificate_authorities": {
+					"local": {
+						"install_trust": false
 					}
 				}
 			},
-		  "http": {
-			"grace_period": 1,
-			"servers": {
-			  "srv0": {
-				"listen": [
-				  ":8080"
-				],
-				"routes": [
-				  {
-					"handle": [
-					  {
-						"handler": "reverse_proxy",
-						"upstreams": [
-						  {
-							"lookup_srv": "srv.host.service.consul"
-						  }
+			"http": {
+				"grace_period": 1,
+				"servers": {
+					"srv0": {
+						"listen": [
+							":18080"
+						],
+						"routes": [
+							{
+								"handle": [
+									{
+										"handler": "reverse_proxy",
+										"upstreams": [
+											{
+												"lookup_srv": "srv.host.service.consul"
+											}
+										]
+									}
+								]
+							}
 						]
-					  }
-					]
-				  }
-				]
-			  }
+					}
+				}
 			}
-		  }
 		}
-	  }
-  `, "json")
+	}
+	`, "json")
 }
 
 func TestSRVWithDial(t *testing.T) {
@@ -62,39 +62,39 @@ func TestSRVWithDial(t *testing.T) {
 	{
 		"apps": {
 			"pki": {
-				"certificate_authorities" : {
-				  "local" : {
-					"install_trust": false
-				  }
+				"certificate_authorities": {
+					"local": {
+						"install_trust": false
+					}
 				}
-			  },
-		  "http": {
-			"grace_period": 1,
-			"servers": {
-			  "srv0": {
-				"listen": [
-				  ":8080"
-				],
-				"routes": [
-				  {
-					"handle": [
-					  {
-						"handler": "reverse_proxy",
-						"upstreams": [
-						  {
-							"dial": "tcp/address.to.upstream:80",
-							"lookup_srv": "srv.host.service.consul"
-						  }
+			},
+			"http": {
+				"grace_period": 1,
+				"servers": {
+					"srv0": {
+						"listen": [
+							":18080"
+						],
+						"routes": [
+							{
+								"handle": [
+									{
+										"handler": "reverse_proxy",
+										"upstreams": [
+											{
+												"dial": "tcp/address.to.upstream:80",
+												"lookup_srv": "srv.host.service.consul"
+											}
+										]
+									}
+								]
+							}
 						]
-					  }
-					]
-				  }
-				]
-			  }
+					}
+				}
 			}
-		  }
 		}
-	  }
+	}
 	`, "json", `upstream: specifying dial address is incompatible with lookup_srv: 0: {\"dial\": \"tcp/address.to.upstream:80\", \"lookup_srv\": \"srv.host.service.consul\"}`)
 }
 
@@ -138,41 +138,41 @@ func TestDialWithPlaceholderUnix(t *testing.T) {
 		},
 		"apps": {
 			"pki": {
-				"certificate_authorities" : {
-				  "local" : {
-					"install_trust": false
-				  }
+				"certificate_authorities": {
+					"local": {
+						"install_trust": false
+					}
 				}
-			  },
-		  "http": {
-			"grace_period": 1,
-			"servers": {
-			  "srv0": {
-				"listen": [
-				  ":8080"
-				],
-				"routes": [
-				  {
-					"handle": [
-					  {
-						"handler": "reverse_proxy",
-						"upstreams": [
-						  {
-							"dial": "unix/{http.request.header.X-Caddy-Upstream-Dial}"
-						  }
+			},
+			"http": {
+				"grace_period": 1,
+				"servers": {
+					"srv0": {
+						"listen": [
+							":18080"
+						],
+						"routes": [
+							{
+								"handle": [
+									{
+										"handler": "reverse_proxy",
+										"upstreams": [
+											{
+												"dial": "unix/{http.request.header.X-Caddy-Upstream-Dial}"
+											}
+										]
+									}
+								]
+							}
 						]
-					  }
-					]
-				  }
-				]
-			  }
-			}
-		  }
+					}
+				}
+		  	}
 		}
-	  }
+	}
 	`, "json")
 
-	req, err := http.NewRequest(http.MethodGet, "http://localhost:8080", nil)
+	req, err := http.NewRequest(http.MethodGet, "http://localhost:18080", nil)
 	if err != nil {
 		t.Fail()
 		return
@@ -190,18 +190,18 @@ func TestReverseProxyWithPlaceholderDialAddress(t *testing.T) {
 		},
 		"apps": {
 			"pki": {
-				"certificate_authorities" : {
-				  "local" : {
-					"install_trust": false
-				  }
+				"certificate_authorities": {
+					"local": {
+						"install_trust": false
+					}
 				}
-			  },
+			},
 			"http": {
 				"grace_period": 1,
 				"servers": {
 					"srv0": {
 						"listen": [
-							":8080"
+							":18080"
 						],
 						"routes": [
 							{
@@ -264,14 +264,14 @@ func TestReverseProxyWithPlaceholderDialAddress(t *testing.T) {
 			}
 		}
 	}
-  	`, "json")
+	`, "json")
 
 	req, err := http.NewRequest(http.MethodGet, "http://localhost:9080", nil)
 	if err != nil {
 		t.Fail()
 		return
 	}
-	req.Header.Set("X-Caddy-Upstream-Dial", "localhost:8080")
+	req.Header.Set("X-Caddy-Upstream-Dial", "localhost:18080")
 	tester.AssertResponse(req, 200, "Hello, World!")
 }
 
@@ -284,18 +284,18 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) {
 		},
 		"apps": {
 			"pki": {
-				"certificate_authorities" : {
-				  "local" : {
-					"install_trust": false
-				  }
+				"certificate_authorities": {
+					"local": {
+						"install_trust": false
+					}
 				}
-			  },
+			},
 			"http": {
 				"grace_period": 1,
 				"servers": {
 					"srv0": {
 						"listen": [
-							":8080"
+							":18080"
 						],
 						"routes": [
 							{
@@ -340,7 +340,7 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) {
 										"handler": "reverse_proxy",
 										"upstreams": [
 											{
-												"dial": "tcp/{http.request.header.X-Caddy-Upstream-Dial}:8080"
+												"dial": "tcp/{http.request.header.X-Caddy-Upstream-Dial}:18080"
 											}
 										]
 									}
@@ -358,7 +358,7 @@ func TestReverseProxyWithPlaceholderTCPDialAddress(t *testing.T) {
 			}
 		}
 	}
-  	`, "json")
+	`, "json")
 
 	req, err := http.NewRequest(http.MethodGet, "http://localhost:9080", nil)
 	if err != nil {
@@ -375,42 +375,42 @@ func TestSRVWithActiveHealthcheck(t *testing.T) {
 		"apps": {
 			"pki": {
 				"certificate_authorities" : {
-				  "local" : {
-					"install_trust": false
-				  }
+					"local" : {
+						"install_trust": false
+					}
 				}
-			  },
-		  "http": {
-			"grace_period": 1,
-			"servers": {
-			  "srv0": {
-				"listen": [
-				  ":8080"
-				],
-				"routes": [
-				  {
-					"handle": [
-					  {
-						"handler": "reverse_proxy",
-						"health_checks": {
-							"active": {
-								"path": "/ok"
+			},
+			"http": {
+				"grace_period": 1,
+				"servers": {
+					"srv0": {
+						"listen": [
+							":18080"
+						],
+						"routes": [
+							{
+								"handle": [
+									{
+										"handler": "reverse_proxy",
+										"health_checks": {
+											"active": {
+												"path": "/ok"
+											}
+										},
+										"upstreams": [
+											{
+												"lookup_srv": "srv.host.service.consul"
+											}
+										]
+									}
+								]
 							}
-						},
-						"upstreams": [
-						  {
-							"lookup_srv": "srv.host.service.consul"
-						  }
 						]
-					  }
-					]
-				  }
-				]
-			  }
+					}
+				}
 			}
-		  }
 		}
-	  }
+	}
 	`, "json", `upstream: lookup_srv is incompatible with active health checks: 0: {\"dial\": \"\", \"lookup_srv\": \"srv.host.service.consul\"}`)
 }
 
@@ -440,7 +440,7 @@ func TestReverseProxyHealthCheck(t *testing.T) {
 			health_timeout 100ms
 		}
 	}
-  `, "caddyfile")
+	`, "caddyfile")
 
 	time.Sleep(100 * time.Millisecond) // TODO: for some reason this test seems particularly flaky, getting 503 when it should be 200, unless we wait
 	tester.AssertGetResponse("http://localhost:9080/", 200, "Hello, World!")