Repository: prometheus/prometheus (language: go)

Issue:
Add proxy support for AWS EC2 Service Discovery
<!--

    Please do *NOT* ask support questions in Github issues.

    If your issue is not a feature request or bug report use our
    community support.

    https://prometheus.io/community/

    There is also commercial support available.

    https://prometheus.io/support-training/

-->
## Proposal
**Use case. Why is this important?**

The AWS EC2 Service Discovery Configuration does not allow users to specify an http(s) proxy for the underlying AWS EC2 API calls. In environments where VPC egress is controlled, either VPC-Endpoints have to be used to allow egress to AWS Services, or, an explicit proxy (like squid or envoy) can be configured to allow egress to the AWS API's.

Similar features are enabled for[ azure_sd_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#azure_sd_config)

I see https://github.com/prometheus/prometheus/issues/2026 is closed, as is https://github.com/prometheus/prometheus/pull/2027.

I don't think setting `HTTP_PROXY` or `HTTPS_PROXY` environment variables is desirable, as we only want the AWS SDK instance used for EC2 Service Discovery to use the proxy. Additionally, the proxy shouldn't affect the Credential Chain (using the Task or Instance Metadata to use the IAM Role).

[AWS has documentation regarding configuring a proxy with the golang sdk](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/custom-http/#configuring-a-proxy).


Discussion / hints:
We ended up adding the entire standard HTTP client to the Azure Discovery in https://github.com/prometheus/prometheus/pull/9267, and I think that's probably what we'll want to do here as well. This should work: https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/custom-http/#overriding-during-configuration-loading.

cc @roidelapluie 

Fix patch:
diff --git a/discovery/aws/ec2.go b/discovery/aws/ec2.go
index 7519f58da15..ca9921159df 100644
--- a/discovery/aws/ec2.go
+++ b/discovery/aws/ec2.go
@@ -66,8 +66,9 @@ const (
 
 // DefaultEC2SDConfig is the default EC2 SD configuration.
 var DefaultEC2SDConfig = EC2SDConfig{
-	Port:            80,
-	RefreshInterval: model.Duration(60 * time.Second),
+	Port:             80,
+	RefreshInterval:  model.Duration(60 * time.Second),
+	HTTPClientConfig: config.DefaultHTTPClientConfig,
 }
 
 func init() {
@@ -91,6 +92,8 @@ type EC2SDConfig struct {
 	RefreshInterval model.Duration `yaml:"refresh_interval,omitempty"`
 	Port            int            `yaml:"port"`
 	Filters         []*EC2Filter   `yaml:"filters"`
+
+	HTTPClientConfig config.HTTPClientConfig `yaml:",inline"`
 }
 
 // Name returns the name of the EC2 Config.
@@ -171,11 +174,17 @@ func (d *EC2Discovery) ec2Client(ctx context.Context) (*ec2.EC2, error) {
 		creds = nil
 	}
 
+	client, err := config.NewClientFromConfig(d.cfg.HTTPClientConfig, "ec2_sd")
+	if err != nil {
+		return nil, err
+	}
+
 	sess, err := session.NewSessionWithOptions(session.Options{
 		Config: aws.Config{
 			Endpoint:    &d.cfg.Endpoint,
 			Region:      &d.cfg.Region,
 			Credentials: creds,
+			HTTPClient:  client,
 		},
 		Profile: d.cfg.Profile,
 	})
diff --git a/discovery/aws/lightsail.go b/discovery/aws/lightsail.go
index 016d78a67f7..e671769ca37 100644
--- a/discovery/aws/lightsail.go
+++ b/discovery/aws/lightsail.go
@@ -56,8 +56,9 @@ const (
 
 // DefaultLightsailSDConfig is the default Lightsail SD configuration.
 var DefaultLightsailSDConfig = LightsailSDConfig{
-	Port:            80,
-	RefreshInterval: model.Duration(60 * time.Second),
+	Port:             80,
+	RefreshInterval:  model.Duration(60 * time.Second),
+	HTTPClientConfig: config.DefaultHTTPClientConfig,
 }
 
 func init() {
@@ -74,6 +75,8 @@ type LightsailSDConfig struct {
 	RoleARN         string         `yaml:"role_arn,omitempty"`
 	RefreshInterval model.Duration `yaml:"refresh_interval,omitempty"`
 	Port            int            `yaml:"port"`
+
+	HTTPClientConfig config.HTTPClientConfig `yaml:",inline"`
 }
 
 // Name returns the name of the Lightsail Config.
@@ -144,11 +147,17 @@ func (d *LightsailDiscovery) lightsailClient() (*lightsail.Lightsail, error) {
 		creds = nil
 	}
 
+	client, err := config.NewClientFromConfig(d.cfg.HTTPClientConfig, "lightsail_sd")
+	if err != nil {
+		return nil, err
+	}
+
 	sess, err := session.NewSessionWithOptions(session.Options{
 		Config: aws.Config{
 			Endpoint:    &d.cfg.Endpoint,
 			Region:      &d.cfg.Region,
 			Credentials: creds,
+			HTTPClient:  client,
 		},
 		Profile: d.cfg.Profile,
 	})
diff --git a/docs/configuration/configuration.md b/docs/configuration/configuration.md
index 7345aca4a3c..c6edcf23393 100644
--- a/docs/configuration/configuration.md
+++ b/docs/configuration/configuration.md
@@ -1064,6 +1064,45 @@ See below for the configuration options for EC2 discovery:
 filters:
   [ - name: <string>
       values: <string>, [...] ]
+
+# Authentication information used to authenticate to the EC2 API.
+# Note that `basic_auth`, `authorization` and `oauth2` options are
+# mutually exclusive.
+# `password` and `password_file` are mutually exclusive.
+
+# Optional HTTP basic authentication information, currently not supported by AWS.
+basic_auth:
+  [ username: <string> ]
+  [ password: <secret> ]
+  [ password_file: <string> ]
+
+# Optional `Authorization` header configuration, currently not supported by AWS.
+authorization:
+  # Sets the authentication type.
+  [ type: <string> | default: Bearer ]
+  # Sets the credentials. It is mutually exclusive with
+  # `credentials_file`.
+  [ credentials: <secret> ]
+  # Sets the credentials to the credentials read from the configured file.
+  # It is mutuall exclusive with `credentials`.
+  [ credentials_file: <filename> ]
+
+# Optional OAuth 2.0 configuration, currently not supported by AWS.
+oauth2:
+  [ <oauth2> ]
+
+# Optional proxy URL.
+[ proxy_url: <string> ]
+
+# Configure whether HTTP requests follow HTTP 3xx redirects.
+[ follow_redirects: <boolean> | default = true ]
+
+# Whether to enable HTTP2.
+[ enable_http2: <bool> | default: true ]
+
+# TLS configuration.
+tls_config:
+  [ <tls_config> ]
 ```
 
 The [relabeling phase](#relabel_config) is the preferred and more powerful
@@ -2066,6 +2105,45 @@ See below for the configuration options for Lightsail discovery:
 # The port to scrape metrics from. If using the public IP address, this must
 # instead be specified in the relabeling rule.
 [ port: <int> | default = 80 ]
+
+# Authentication information used to authenticate to the Lightsail API.
+# Note that `basic_auth`, `authorization` and `oauth2` options are
+# mutually exclusive.
+# `password` and `password_file` are mutually exclusive.
+
+# Optional HTTP basic authentication information, currently not supported by AWS.
+basic_auth:
+  [ username: <string> ]
+  [ password: <secret> ]
+  [ password_file: <string> ]
+
+# Optional `Authorization` header configuration, currently not supported by AWS.
+authorization:
+  # Sets the authentication type.
+  [ type: <string> | default: Bearer ]
+  # Sets the credentials. It is mutually exclusive with
+  # `credentials_file`.
+  [ credentials: <secret> ]
+  # Sets the credentials to the credentials read from the configured file.
+  # It is mutuall exclusive with `credentials`.
+  [ credentials_file: <filename> ]
+
+# Optional OAuth 2.0 configuration, currently not supported by AWS.
+oauth2:
+  [ <oauth2> ]
+
+# Optional proxy URL.
+[ proxy_url: <string> ]
+
+# Configure whether HTTP requests follow HTTP 3xx redirects.
+[ follow_redirects: <boolean> | default = true ]
+
+# Whether to enable HTTP2.
+[ enable_http2: <bool> | default: true ]
+
+# TLS configuration.
+tls_config:
+  [ <tls_config> ]
 ```
 
 ### `<linode_sd_config>`


Test patch:
diff --git a/config/config_test.go b/config/config_test.go
index ddf65a06824..9ee8fe1a02d 100644
--- a/config/config_test.go
+++ b/config/config_test.go
@@ -586,6 +586,7 @@ var expectedConf = &Config{
 							Values: []string{"web", "db"},
 						},
 					},
+					HTTPClientConfig: config.DefaultHTTPClientConfig,
 				},
 			},
 		},
@@ -602,12 +603,13 @@ var expectedConf = &Config{
 
 			ServiceDiscoveryConfigs: discovery.Configs{
 				&aws.LightsailSDConfig{
-					Region:          "us-east-1",
-					AccessKey:       "access",
-					SecretKey:       "mysecret",
-					Profile:         "profile",
-					RefreshInterval: model.Duration(60 * time.Second),
-					Port:            80,
+					Region:           "us-east-1",
+					AccessKey:        "access",
+					SecretKey:        "mysecret",
+					Profile:          "profile",
+					RefreshInterval:  model.Duration(60 * time.Second),
+					Port:             80,
+					HTTPClientConfig: config.DefaultHTTPClientConfig,
 				},
 			},
 		},