Repository: kubernetes-sigs/aws-ebs-csi-driver
Pull request #522 (2020-06-08 11:39:05)

Allow volume attach limit overwrite via command line parameter
**Is this a bug fix or adding new feature?**
/kind feature

**What is this PR about? / Why do we need it?**
This PR is a part of https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/347 and proceeds with the way suggested in https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/347#issuecomment-638177835: In order to allow existing clusters that are leveraging/relying on the `KUBE_MAX_PD_VOLS` feature for in-tree provisioners in Kubernetes (https://kubernetes.io/docs/concepts/storage/storage-limits/#custom-limits) to migrate to CSI, the EBS CSI driver is supporting the `--volume-attach-limit-overwrite` flag now until a more sophisticated solution can be implemented.

PS: I was adding a few missing doc strings along the way and reduced package/function name stuttering according to the [official Go suggestions](https://blog.golang.org/package-names). Hope that's fine.


k8s-ci-robot: Hi @rfranzke. Thanks for your PR.

I'm waiting for a [kubernetes-sigs](https://github.com/orgs/kubernetes-sigs/people) member to verify that this patch is reasonable to test. If it is, they should reply with `/ok-to-test` on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should [join the org](https://git.k8s.io/community/community-membership.md#member) to skip this step.

Once the patch is verified, the new status will be reflected by the `ok-to-test` label.

I understand the commands that are listed [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Faws-ebs-csi-driver).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#" title="Author self-approved">rfranzke</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **d-nishi**
You can assign the PR to them by writing `/assign @d-nishi` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Faws-ebs-csi-driver).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["d-nishi"]} -->
rfranzke: /assign @leakingtapan 
rfranzke: cc @gnufied
coveralls: ## Pull Request Test Coverage Report for [Build 1113](https://coveralls.io/builds/31308503)

* **7** of **29**   **(24.14%)**  changed or added relevant lines in **5** files are covered.
* **2** unchanged lines in **2** files lost coverage.
* Overall coverage decreased (**-0.2%**) to **78.687%**

---

|  Changes Missing Coverage | Covered Lines | Changed/Added Lines | % |
| :-----|--------------|--------|---: |
| [pkg/driver/node.go](https://coveralls.io/builds/31308503/source?filename=pkg%2Fdriver%2Fnode.go#L81) | 3 | 8 | 37.5%
| [pkg/driver/driver.go](https://coveralls.io/builds/31308503/source?filename=pkg%2Fdriver%2Fdriver.go#L67) | 1 | 18 | 5.56%
<!-- | **Total:** | **7** | **29** | **24.14%** | -->

|  Files with Coverage Reduction | New Missed Lines | % |
| :-----|--------------|--: |
| [pkg/driver/driver.go](https://coveralls.io/builds/31308503/source?filename=pkg%2Fdriver%2Fdriver.go#L80) | 1 | 29.27% |
| [pkg/driver/node.go](https://coveralls.io/builds/31308503/source?filename=pkg%2Fdriver%2Fnode.go#L92) | 1 | 69.44% |
<!-- | **Total:** | **2** |  | -->

|  Totals | [![Coverage Status](https://coveralls.io/builds/31308503/badge)](https://coveralls.io/builds/31308503) |
| :-- | --: |
| Change from base [Build 1111](https://coveralls.io/builds/31005643): |  -0.2% |
| Covered Lines: | 1414 |
| Relevant Lines: | 1797 |

---
##### 💛  - [Coveralls](https://coveralls.io)

k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#" title="Author self-approved">rfranzke</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **leakingtapan**
You can assign the PR to them by writing `/assign @leakingtapan` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Faws-ebs-csi-driver).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["leakingtapan"]} -->
gnufied: /ok-to-test
k8s-ci-robot: @gnufied: Cannot trigger testing until a trusted user reviews the PR and leaves an `/ok-to-test` message.

<details>

In response to [this](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#issuecomment-642609560):

>/ok-to-test


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
hardikdr: /ok-to-test
hardikdr: @leakingtapan @wongma7 It would be helpful if you could also take a quick look.

It seems like a reasonable change, to not break the user experience with CSI migration on AWS/EKS. Basically the `KUBE_MAX_PD_VOLS` on kube-scheduler  is ineffective with CSI-drivers and leaves no way for end-user to overwrite  max-attachable-volumes. 
- This PR at least helps in keeping existing experience intact.
wongma7: Giving @leakingtapan a chance to chime in as he had some objections to this but I agree it's necessary to make the limit configurable somehow and a flag seems the best way.
rfranzke: @leakingtapan Do you have some time to check the PR? Any thoughts on it?
rfranzke: Hm, what's the general review process now? As @leakingtapan seems to be busy/inactive, can another owner/approver help to progress with this PR? @bertinatto @jsafrane @gnufied
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#" title="Author self-approved">rfranzke</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **leakingtapan**
You can assign the PR to them by writing `/assign @leakingtapan` in a comment when ready.

The full list of commands accepted by this bot can be found [here](?repo=kubernetes-sigs%2Faws-ebs-csi-driver).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["leakingtapan"]} -->
k8s-ci-robot: @rfranzke: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-aws-ebs-csi-driver-e2e-single-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-single-az/1275300214399307777) | `/test pull-aws-ebs-csi-driver-e2e-single-az`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=aws-ebs-csi-driver&pr=522). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3rfranzke). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @rfranzke: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-aws-ebs-csi-driver-e2e-single-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-single-az/1275300214399307777) | `/test pull-aws-ebs-csi-driver-e2e-single-az`
pull-aws-ebs-csi-driver-e2e-multi-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-multi-az/1275300214399307779) | `/test pull-aws-ebs-csi-driver-e2e-multi-az`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=aws-ebs-csi-driver&pr=522). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3rfranzke). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
rfranzke: /retest
k8s-ci-robot: @rfranzke: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-aws-ebs-csi-driver-e2e-multi-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-multi-az/1275300214399307779) | `/test pull-aws-ebs-csi-driver-e2e-multi-az`
pull-aws-ebs-csi-driver-e2e-single-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-single-az/1275303611722633216) | `/test pull-aws-ebs-csi-driver-e2e-single-az`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=aws-ebs-csi-driver&pr=522). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3rfranzke). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @rfranzke: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-aws-ebs-csi-driver-e2e-single-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-single-az/1275303611722633216) | `/test pull-aws-ebs-csi-driver-e2e-single-az`
pull-aws-ebs-csi-driver-e2e-multi-az | 0b554b6975e3be904f953252056d8047248070c1 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-multi-az/1275303611722633217) | `/test pull-aws-ebs-csi-driver-e2e-multi-az`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=aws-ebs-csi-driver&pr=522). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3rfranzke). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
rfranzke: @gnufied I've dropped all unrelated changes.
rfranzke: Btw, any pointers on how to get the e2e tests fixed? They fail unrelatedly, see https://storage.googleapis.com/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-e2e-multi-az/1275303611722633217/build-log.txt.

```
$ curl https://sum.golang.org/lookup/github.com/aws/aws-k8s-tester@v1.3.10-0.20200623051819-1bafc137b062 -I
HTTP/2 410
```
gnufied: Looks good to me but we might want to wait for @leakingtapan feedback too. 

/lgtm
/hold

k8s-ci-robot: @gnufied: changing LGTM is restricted to collaborators

<details>

In response to [this](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#issuecomment-648821492):

>Looks good to me but we might want to wait for @leakingtapan feedback too. 
>
>/lgtm
>/hold
>


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
gnufied: /approve

k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#issuecomment-648906367" title="Approved">gnufied</a>*, *<a href="https://github.com/kubernetes-sigs/aws-ebs-csi-driver/pull/522#" title="Author self-approved">rfranzke</a>*

The full list of commands accepted by this bot can be found [here](?repo=kubernetes-sigs%2Faws-ebs-csi-driver).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/OWNERS)~~ [gnufied]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: New changes are detected. LGTM label has been removed.
k8s-ci-robot: @rfranzke: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-aws-ebs-csi-driver-unit | 56e2559aa6d94ba483f006279e3ebc394dbd4db7 | [link](https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_aws-ebs-csi-driver/522/pull-aws-ebs-csi-driver-unit/1277482869895204864) | `/test pull-aws-ebs-csi-driver-unit`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=aws-ebs-csi-driver&pr=522). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3rfranzke). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/cmd/main.go b/cmd/main.go
index 6bf98fba..855bbb63 100644
--- a/cmd/main.go
+++ b/cmd/main.go
@@ -32,6 +32,7 @@ func main() {
 		driver.WithEndpoint(options.ServerOptions.Endpoint),
 		driver.WithExtraVolumeTags(options.ControllerOptions.ExtraVolumeTags),
 		driver.WithMode(options.DriverMode),
+		driver.WithVolumeAttachLimit(options.NodeOptions.VolumeAttachLimit),
 	)
 	if err != nil {
 		klog.Fatalln(err)
diff --git a/cmd/options/node_options.go b/cmd/options/node_options.go
index 0f696e03..91190f16 100644
--- a/cmd/options/node_options.go
+++ b/cmd/options/node_options.go
@@ -21,6 +21,18 @@ import (
 )
 
 // NodeOptions contains options and configuration settings for the node service.
-type NodeOptions struct{}
+type NodeOptions struct {
+	// VolumeAttachLimit specifies the value that shall be reported as "maximum number of attachable volumes"
+	// in CSINode objects. It is similar to https://kubernetes.io/docs/concepts/storage/storage-limits/#custom-limits
+	// which allowed administrators to specify custom volume limits by configuring the kube-scheduler. Also, each AWS
+	// machine type has different volume limits. By default, the EBS CSI driver parses the machine type name and then
+	// decides the volume limit. However, this is only a rough approximation and not good enough in most cases.
+	// Specifying the volume attach limit via command line is the alternative until a more sophisticated solution presents
+	// itself (dynamically discovering the maximum number of attachable volume per EC2 machine type, see also
+	// https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/347).
+	VolumeAttachLimit int64
+}
 
-func (s *NodeOptions) AddFlags(fs *flag.FlagSet) {}
+func (o *NodeOptions) AddFlags(fs *flag.FlagSet) {
+	fs.Int64Var(&o.VolumeAttachLimit, "volume-attach-limit", -1, "Value value for the maximum number of volumes attachable for all nodes. If the flag is not specified then the value is approximated from the instance type.")
+}
diff --git a/deploy/kubernetes/base/node.yaml b/deploy/kubernetes/base/node.yaml
index 7a84b8b7..6c28a04b 100644
--- a/deploy/kubernetes/base/node.yaml
+++ b/deploy/kubernetes/base/node.yaml
@@ -43,6 +43,7 @@ spec:
           args:
             - node
             - --endpoint=$(CSI_ENDPOINT)
+          # - --volume-attach-limit=42
             - --logtostderr
             - --v=5
           env:
diff --git a/docs/design.md b/docs/design.md
index d3490942..96c3ffae 100644
--- a/docs/design.md
+++ b/docs/design.md
@@ -235,3 +235,15 @@ Example 2: `AWS_REGION=us-west-1 /bin/aws-ebs-csi-driver controller --extra-volu
 
 - `node`: This will only start the node service of the CSI driver.\
 Example: `/bin/aws-ebs-csi-driver node --endpoint=unix://...`
+
+## Custom volume limits
+
+For the Kubernetes in-tree volume provisioners (including the `kubernetes.io/aws-ebs` provisioner) it was possible for administrators to provide a custom volume limit overwrite (see https://kubernetes.io/docs/concepts/storage/storage-limits/#custom-limits).
+This solution is not working with CSI any longer.
+As part of [#347](https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/347) we discuss how we can implement a sophisticated computation of the volume attach limit per node (e.g., based on the used machine types and already attached network interfaces).
+However, it turns out that such optimal implementation is not easily achievable.
+Each AWS machine type has different volume limits.
+Today, the EBS CSI driver parses the machine type name and then decides the volume limit.
+Unfortunately, this is only a rough approximation and not good enough in most cases.
+In order to allow existing clusters that are leveraging/relying on this feature to migrate to CSI, the EBS CSI driver is supporting the `--volume-attach-limit` flag.
+Specifying the volume attach limit via command line is the alternative until a more sophisticated solution presents itself (dynamically discovering the maximum number of attachable volume per EC2 machine type).
diff --git a/pkg/driver/driver.go b/pkg/driver/driver.go
index e54df9b9..60305d5b 100644
--- a/pkg/driver/driver.go
+++ b/pkg/driver/driver.go
@@ -53,9 +53,10 @@ type Driver struct {
 }
 
 type DriverOptions struct {
-	endpoint        string
-	extraVolumeTags map[string]string
-	mode            Mode
+	endpoint          string
+	extraVolumeTags   map[string]string
+	mode              Mode
+	volumeAttachLimit int64
 }
 
 func NewDriver(options ...func(*DriverOptions)) (*Driver, error) {
@@ -81,10 +82,10 @@ func NewDriver(options ...func(*DriverOptions)) (*Driver, error) {
 	case ControllerMode:
 		driver.controllerService = newControllerService(&driverOptions)
 	case NodeMode:
-		driver.nodeService = newNodeService()
+		driver.nodeService = newNodeService(&driverOptions)
 	case AllMode:
 		driver.controllerService = newControllerService(&driverOptions)
-		driver.nodeService = newNodeService()
+		driver.nodeService = newNodeService(&driverOptions)
 	default:
 		return nil, fmt.Errorf("unknown mode: %s", driverOptions.mode)
 	}
@@ -155,3 +156,9 @@ func WithMode(mode Mode) func(*DriverOptions) {
 		o.mode = mode
 	}
 }
+
+func WithVolumeAttachLimit(volumeAttachLimit int64) func(*DriverOptions) {
+	return func(o *DriverOptions) {
+		o.volumeAttachLimit = volumeAttachLimit
+	}
+}
diff --git a/pkg/driver/node.go b/pkg/driver/node.go
index 649e7c94..cea55ad7 100644
--- a/pkg/driver/node.go
+++ b/pkg/driver/node.go
@@ -70,23 +70,25 @@ var (
 
 // nodeService represents the node service of CSI driver
 type nodeService struct {
-	metadata cloud.MetadataService
-	mounter  Mounter
-	inFlight *internal.InFlight
+	metadata      cloud.MetadataService
+	mounter       Mounter
+	inFlight      *internal.InFlight
+	driverOptions *DriverOptions
 }
 
 // newNodeService creates a new node service
 // it panics if failed to create the service
-func newNodeService() nodeService {
+func newNodeService(driverOptions *DriverOptions) nodeService {
 	metadata, err := cloud.NewMetadata()
 	if err != nil {
 		panic(err)
 	}
 
 	return nodeService{
-		metadata: metadata,
-		mounter:  newNodeMounter(),
-		inFlight: internal.NewInFlight(),
+		metadata:      metadata,
+		mounter:       newNodeMounter(),
+		inFlight:      internal.NewInFlight(),
+		driverOptions: driverOptions,
 	}
 }
 
@@ -514,6 +516,9 @@ func findNvmeVolume(findName string) (device string, err error) {
 
 // getVolumesLimit returns the limit of volumes that the node supports
 func (d *nodeService) getVolumesLimit() int64 {
+	if d.driverOptions.volumeAttachLimit >= 0 {
+		return d.driverOptions.volumeAttachLimit
+	}
 	ebsNitroInstanceTypeRegex := "^[cmr]5.*|t3|z1d"
 	instanceType := d.metadata.GetInstanceType()
 	if ok, _ := regexp.MatchString(ebsNitroInstanceTypeRegex, instanceType); ok {


Test patch:
diff --git a/cmd/options/node_options_test.go b/cmd/options/node_options_test.go
index 483d7cac..dc4191a5 100644
--- a/cmd/options/node_options_test.go
+++ b/cmd/options/node_options_test.go
@@ -27,6 +27,11 @@ func TestNodeOptions(t *testing.T) {
 		flag  string
 		found bool
 	}{
+		{
+			name:  "lookup desired flag",
+			flag:  "volume-attach-limit",
+			found: true,
+		},
 		{
 			name:  "fail for non-desired flag",
 			flag:  "some-flag",
diff --git a/cmd/options_test.go b/cmd/options_test.go
index cd58e19d..ef4bffe1 100644
--- a/cmd/options_test.go
+++ b/cmd/options_test.go
@@ -20,6 +20,7 @@ import (
 	"flag"
 	"os"
 	"reflect"
+	"strconv"
 	"testing"
 
 	"github.com/kubernetes-sigs/aws-ebs-csi-driver/pkg/driver"
@@ -45,6 +46,9 @@ func TestGetOptions(t *testing.T) {
 			extraVolumeTagKey: extraVolumeTagValue,
 		}
 
+		VolumeAttachLimitFlagName := "volume-attach-limit"
+		var VolumeAttachLimit int64 = 42
+
 		args := append([]string{
 			"aws-ebs-csi-driver",
 		}, additionalArgs...)
@@ -55,6 +59,9 @@ func TestGetOptions(t *testing.T) {
 		if withControllerOptions {
 			args = append(args, "-"+extraVolumeTagsFlagName+"="+extraVolumeTagKey+"="+extraVolumeTagValue)
 		}
+		if withNodeOptions {
+			args = append(args, "-"+VolumeAttachLimitFlagName+"="+strconv.FormatInt(VolumeAttachLimit, 10))
+		}
 
 		oldArgs := os.Args
 		defer func() { os.Args = oldArgs }()
@@ -82,6 +89,16 @@ func TestGetOptions(t *testing.T) {
 			}
 		}
 
+		if withNodeOptions {
+			VolumeAttachLimitFlag := flagSet.Lookup(VolumeAttachLimitFlagName)
+			if VolumeAttachLimitFlag == nil {
+				t.Fatalf("expected %q flag to be added but it is not", VolumeAttachLimitFlagName)
+			}
+			if options.NodeOptions.VolumeAttachLimit != VolumeAttachLimit {
+				t.Fatalf("expected VolumeAttachLimit to be %d but it is %d", VolumeAttachLimit, options.NodeOptions.VolumeAttachLimit)
+			}
+		}
+
 		return options
 	}
 
diff --git a/pkg/driver/driver_test.go b/pkg/driver/driver_test.go
new file mode 100644
index 00000000..70f86002
--- /dev/null
+++ b/pkg/driver/driver_test.go
@@ -0,0 +1,58 @@
+/*
+Copyright 2020 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package driver
+
+import (
+	"reflect"
+	"testing"
+)
+
+func TestWithEndpoint(t *testing.T) {
+	value := "endpoint"
+	options := &DriverOptions{}
+	WithEndpoint(value)(options)
+	if options.endpoint != value {
+		t.Fatalf("expected endpoint option got set to %q but is set to %q", value, options.endpoint)
+	}
+}
+
+func TestWithExtraVolumeTags(t *testing.T) {
+	value := map[string]string{"foo": "bar"}
+	options := &DriverOptions{}
+	WithExtraVolumeTags(value)(options)
+	if !reflect.DeepEqual(options.extraVolumeTags, value) {
+		t.Fatalf("expected extraVolumeTags option got set to %+v but is set to %+v", value, options.extraVolumeTags)
+	}
+}
+
+func TestWithMode(t *testing.T) {
+	value := Mode("mode")
+	options := &DriverOptions{}
+	WithMode(value)(options)
+	if options.mode != value {
+		t.Fatalf("expected mode option got set to %q but is set to %q", value, options.mode)
+	}
+}
+
+func TestWithVolumeAttachLimit(t *testing.T) {
+	var value int64 = 42
+	options := &DriverOptions{}
+	WithVolumeAttachLimit(value)(options)
+	if options.volumeAttachLimit != value {
+		t.Fatalf("expected volumeAttachLimit option got set to %d but is set to %d", value, options.volumeAttachLimit)
+	}
+}
diff --git a/pkg/driver/node_test.go b/pkg/driver/node_test.go
index 36714e4e..8592b2db 100644
--- a/pkg/driver/node_test.go
+++ b/pkg/driver/node_test.go
@@ -1243,25 +1243,44 @@ func TestNodeGetCapabilities(t *testing.T) {
 
 func TestNodeGetInfo(t *testing.T) {
 	testCases := []struct {
-		name             string
-		instanceID       string
-		instanceType     string
-		availabilityZone string
-		expMaxVolumes    int64
+		name              string
+		instanceID        string
+		instanceType      string
+		availabilityZone  string
+		volumeAttachLimit int64
+		expMaxVolumes     int64
 	}{
 		{
-			name:             "success normal",
-			instanceID:       "i-123456789abcdef01",
-			instanceType:     "t2.medium",
-			availabilityZone: "us-west-2b",
-			expMaxVolumes:    39,
+			name:              "success normal",
+			instanceID:        "i-123456789abcdef01",
+			instanceType:      "t2.medium",
+			availabilityZone:  "us-west-2b",
+			volumeAttachLimit: -1,
+			expMaxVolumes:     39,
 		},
 		{
-			name:             "success normal with NVMe",
-			instanceID:       "i-123456789abcdef01",
-			instanceType:     "m5d.large",
-			availabilityZone: "us-west-2b",
-			expMaxVolumes:    25,
+			name:              "success normal with overwrite",
+			instanceID:        "i-123456789abcdef01",
+			instanceType:      "t2.medium",
+			availabilityZone:  "us-west-2b",
+			volumeAttachLimit: 42,
+			expMaxVolumes:     42,
+		},
+		{
+			name:              "success normal with NVMe",
+			instanceID:        "i-123456789abcdef01",
+			instanceType:      "m5d.large",
+			availabilityZone:  "us-west-2b",
+			volumeAttachLimit: -1,
+			expMaxVolumes:     25,
+		},
+		{
+			name:              "success normal with NVMe and overwrite",
+			instanceID:        "i-123456789abcdef01",
+			instanceType:      "m5d.large",
+			availabilityZone:  "us-west-2b",
+			volumeAttachLimit: 30,
+			expMaxVolumes:     30,
 		},
 	}
 	for _, tc := range testCases {
@@ -1269,17 +1288,25 @@ func TestNodeGetInfo(t *testing.T) {
 			mockCtl := gomock.NewController(t)
 			defer mockCtl.Finish()
 
+			driverOptions := &DriverOptions{
+				volumeAttachLimit: tc.volumeAttachLimit,
+			}
+
+			mockMounter := mocks.NewMockMounter(mockCtl)
+
 			mockMetadata := mocks.NewMockMetadataService(mockCtl)
 			mockMetadata.EXPECT().GetInstanceID().Return(tc.instanceID)
-			mockMetadata.EXPECT().GetInstanceType().Return(tc.instanceType)
 			mockMetadata.EXPECT().GetAvailabilityZone().Return(tc.availabilityZone)
 
-			mockMounter := mocks.NewMockMounter(mockCtl)
+			if tc.volumeAttachLimit < 0 {
+				mockMetadata.EXPECT().GetInstanceType().Return(tc.instanceType)
+			}
 
 			awsDriver := &nodeService{
-				metadata: mockMetadata,
-				mounter:  mockMounter,
-				inFlight: internal.NewInFlight(),
+				metadata:      mockMetadata,
+				mounter:       mockMounter,
+				inFlight:      internal.NewInFlight(),
+				driverOptions: driverOptions,
 			}
 
 			resp, err := awsDriver.NodeGetInfo(context.TODO(), &csi.NodeGetInfoRequest{})
diff --git a/pkg/driver/sanity_test.go b/pkg/driver/sanity_test.go
index 799eafdb..8afb81dc 100644
--- a/pkg/driver/sanity_test.go
+++ b/pkg/driver/sanity_test.go
@@ -53,8 +53,9 @@ func TestSanity(t *testing.T) {
 				Region:           "region",
 				AvailabilityZone: "az",
 			},
-			mounter:  newFakeMounter(),
-			inFlight: internal.NewInFlight(),
+			mounter:       newFakeMounter(),
+			inFlight:      internal.NewInFlight(),
+			driverOptions: &DriverOptions{},
 		},
 	}
 	defer func() {