Repository: kubernetes-sigs/aws-ebs-csi-driver
Pull request #2130 (2024-08-26 15:07:40)

Add local snapshots on outposts
**Is this a bug fix or adding new feature?**

fixes https://github.com/kubernetes-sigs/aws-ebs-csi-driver/issues/2087

**What is this PR about? / Why do we need it?**

This PR will allow customers to pass the outpost ARN as a parameter to the VSC.
This will enable customers to have local snapshots saved onto S3 on their outpost and not just in S3 on the region their outpost is in.

**What testing is done?** 

This is currently being tested in outpost. 



Patch:
diff --git a/.github/workflows/trivy.yaml b/.github/workflows/trivy.yaml
index cab314b4..e5a40300 100644
--- a/.github/workflows/trivy.yaml
+++ b/.github/workflows/trivy.yaml
@@ -28,7 +28,7 @@ jobs:
         uses: actions/checkout@v3
 
       - name: Run Trivy vulnerability scanner in repo mode
-        uses: aquasecurity/trivy-action@0.17.0
+        uses: aquasecurity/trivy-action@0.24.0
         with:
           scan-type: 'fs'
           ignore-unfixed: true
diff --git a/docs/faq.md b/docs/faq.md
index 6d046714..88c6059f 100644
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -32,9 +32,9 @@ Warning  FailedAttachVolume      6m51s              attachdetach-controller  Mul
   systemctl restart kubelet
 ```
 
-3. **Spot Instances and Karpenter Best Practices:** When using Spot Instances with Karpenter, enable [interruption handling](https://aws.github.io/aws-eks-best-practices/karpenter/#enable-interruption-handling-when-using-spot) to manage involuntary interruptions gracefully. Karpenter supports native interruption handling, which cordons, drains, and terminates nodes ahead of interruption events, maximizing workload cleanup time.
-
-4. **Set `.node.tolerateAllTaints=false` when deploying the EBS CSI Driver:** This allows Karpenter to safely drain the `ebs-csi-node` pod before terminating the instance. If you are relying on the `ebs-csi-node` pod to tolerate certain taints, please manually add those tolerations to the driver with `.node.tolerations`.
+3. **Karpenter Best Practices:**
+  - Upgrade to Karpenter version ≥ v1.0.0, where Karpenter will now wait to terminate nodes until all volumes have been detached from them. 
+  - When using Spot Instances with Karpenter, enable [interruption handling](https://aws.github.io/aws-eks-best-practices/karpenter/#enable-interruption-handling-when-using-spot) to manage involuntary interruptions gracefully. Karpenter supports native interruption handling, which cordons, drains, and terminates nodes ahead of interruption events, maximizing workload cleanup time.
 
 ### What is the PreStop lifecycle hook?
 
diff --git a/docs/fast-snapshot-restores.md b/docs/snapshot.md
similarity index 61%
rename from docs/fast-snapshot-restores.md
rename to docs/snapshot.md
index fe45a04f..5643cdf3 100644
--- a/docs/fast-snapshot-restores.md
+++ b/docs/snapshot.md
@@ -1,11 +1,23 @@
+## Supported Parameters
+| Parameter                          | Description of value                                      |
+|------------------------------------|-----------------------------------------------------------|
+| fastSnapshotRestoreAvailabilityZones | Comma separated list of availability zones                |
+| outpostArn                         | Arn of the outpost you wish to have the snapshot saved to | 
+
+The AWS EBS CSI Driver supports [tagging](tagging.md) through `VolumeSnapshotClass.parameters` (in v1.6.0 and later). 
+## Prerequisites
+
+- Install the [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd) and external-snapshotter sidecar. For installation instructions, see [CSI Snapshotter Usage](https://github.com/kubernetes-csi/external-snapshotter#usage).
+
 # Fast Snapshot Restores
 
 The EBS CSI Driver provides support for [Fast Snapshot Restores(FSR)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-fast-snapshot-restore.html) via `VolumeSnapshotClass.parameters.fastSnapshotRestoreAvailabilityZones`.
 
-Amazon EBS fast snapshot restore (FSR) enables you to create a volume from a snapshot that is fully initialized at creation. This eliminates the latency of I/O operations on a block when it is accessed for the first time. Volumes that are created using fast snapshot restore instantly deliver all of their provisioned performance.
 
 Availability zones are specified as a comma separated list.
 
+- The EBS CSI Driver must be given permission to access the [`EnableFastSnapshotRestores` EC2 API](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_EnableFastSnapshotRestores.html). This example snippet can be used in an IAM policy to grant access to `EnableFastSnapshotRestores`:
+
 **Example**
 ```
 apiVersion: snapshot.storage.k8s.io/v1
@@ -18,12 +30,6 @@ parameters:
   fastSnapshotRestoreAvailabilityZones: "us-east-1a, us-east-1b"
 ```
 
-## Prerequisites
-
-- Install the [Kubernetes Volume Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd) and external-snapshotter sidecar. For installation instructions, see [CSI Snapshotter Usage](https://github.com/kubernetes-csi/external-snapshotter#usage).
-
-- The EBS CSI Driver must be given permission to access the [`EnableFastSnapshotRestores` EC2 API](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_EnableFastSnapshotRestores.html). This example snippet can be used in an IAM policy to grant access to `EnableFastSnapshotRestores`:
-
 ```json
 {
   "Effect": "Allow",
@@ -34,6 +40,25 @@ parameters:
 }
 ```
 
-## Failure Mode
+## Failure Mode 
 
 The driver will attempt to check if the availability zones provided are supported for fast snapshot restore before attempting to create the snapshot. If the `EnableFastSnapshotRestores` API call fails, the driver will hard-fail the request and delete the snapshot. This is to ensure that the snapshot is not left in an inconsistent state.
+
+
+# Amazon EBS Local Snapshots on Outposts
+
+The EBS CSI Driver provides support for [Amazon EBS local snapshots on Outposts](https://docs.aws.amazon.com/ebs/latest/userguide/snapshots-outposts.html) via `VolumeSnapshotClass.parameters.outpostArn`.
+
+
+
+**Example**
+```
+apiVersion: snapshot.storage.k8s.io/v1
+kind: VolumeSnapshotClass
+metadata:
+  name: csi-aws-vsc
+driver: ebs.csi.aws.com
+deletionPolicy: Delete
+parameters:
+  outpostarn: {arn of your outpost}
+```
diff --git a/pkg/cloud/cloud.go b/pkg/cloud/cloud.go
index 4e946008..da71ef13 100644
--- a/pkg/cloud/cloud.go
+++ b/pkg/cloud/cloud.go
@@ -252,7 +252,8 @@ type ListSnapshotsResponse struct {
 
 // SnapshotOptions represents parameters to create an EBS volume
 type SnapshotOptions struct {
-	Tags map[string]string
+	Tags       map[string]string
+	OutpostArn string
 }
 
 // ec2ListSnapshotsResponse is a helper struct returned from the AWS API calling function to the main ListSnapshots function
@@ -1275,6 +1276,7 @@ func (c *cloud) CreateSnapshot(ctx context.Context, volumeID string, snapshotOpt
 	descriptions := "Created by AWS EBS CSI driver for volume " + volumeID
 
 	var tags []types.Tag
+	var request *ec2.CreateSnapshotInput
 	for key, value := range snapshotOptions.Tags {
 		tags = append(tags, types.Tag{Key: aws.String(key), Value: aws.String(value)})
 	}
@@ -1282,12 +1284,14 @@ func (c *cloud) CreateSnapshot(ctx context.Context, volumeID string, snapshotOpt
 		ResourceType: types.ResourceTypeSnapshot,
 		Tags:         tags,
 	}
-	request := &ec2.CreateSnapshotInput{
+	request = &ec2.CreateSnapshotInput{
 		VolumeId:          aws.String(volumeID),
 		TagSpecifications: []types.TagSpecification{tagSpec},
 		Description:       aws.String(descriptions),
 	}
-
+	if snapshotOptions.OutpostArn != "" {
+		request.OutpostArn = aws.String(snapshotOptions.OutpostArn)
+	}
 	res, err := c.ec2.CreateSnapshot(ctx, request, func(o *ec2.Options) {
 		o.Retryer = c.rm.createSnapshotRetryer
 	})
diff --git a/pkg/driver/constants.go b/pkg/driver/constants.go
index 7775bbbd..17d6ab69 100644
--- a/pkg/driver/constants.go
+++ b/pkg/driver/constants.go
@@ -102,6 +102,9 @@ const (
 	// TagKeyPrefix contains the prefix of a volume parameter that designates it as
 	// a tag to be attached to the resource
 	TagKeyPrefix = "tagSpecification"
+
+	// OutpostArn represents key for outpost's arn
+	OutpostArnKey = "outpostarn"
 )
 
 // constants of keys in snapshot parameters
diff --git a/pkg/driver/controller.go b/pkg/driver/controller.go
index 7b460fd6..5af42fba 100644
--- a/pkg/driver/controller.go
+++ b/pkg/driver/controller.go
@@ -669,6 +669,7 @@ func (d *ControllerService) CreateSnapshot(ctx context.Context, req *csi.CreateS
 
 	snapshotName := req.GetName()
 	volumeID := req.GetSourceVolumeId()
+	var outpostArn string
 
 	// check if a request is already in-flight
 	if ok := d.inFlight.Insert(snapshotName); !ok {
@@ -709,6 +710,12 @@ func (d *ControllerService) CreateSnapshot(ctx context.Context, req *csi.CreateS
 		case FastSnapshotRestoreAvailabilityZones:
 			f := strings.ReplaceAll(value, " ", "")
 			fsrAvailabilityZones = strings.Split(f, ",")
+		case OutpostArnKey:
+			if arn.IsARN(value) {
+				outpostArn = value
+			} else {
+				return nil, status.Errorf(codes.InvalidArgument, "Invalid parameter value %s is not a valid arn", value)
+			}
 		default:
 			if strings.HasPrefix(key, TagKeyPrefix) {
 				vscTags = append(vscTags, value)
@@ -741,7 +748,8 @@ func (d *ControllerService) CreateSnapshot(ctx context.Context, req *csi.CreateS
 	}
 
 	opts := &cloud.SnapshotOptions{
-		Tags: snapshotTags,
+		Tags:       snapshotTags,
+		OutpostArn: outpostArn,
 	}
 
 	// Check if the availability zone is supported for fast snapshot restore


Test patch:
diff --git a/pkg/cloud/cloud_test.go b/pkg/cloud/cloud_test.go
index f7acef7b..5d3c0611 100644
--- a/pkg/cloud/cloud_test.go
+++ b/pkg/cloud/cloud_test.go
@@ -2059,6 +2059,25 @@ func TestCreateSnapshot(t *testing.T) {
 			},
 			expErr: nil,
 		},
+		{
+			name:         "success: outpost",
+			snapshotName: "snap-test-name",
+			snapshotOptions: &SnapshotOptions{
+				Tags: map[string]string{
+					SnapshotNameTagKey: "snap-test-name",
+					AwsEbsDriverTagKey: "true",
+					"extra-tag-key":    "extra-tag-value",
+				},
+				OutpostArn: "arn:aws:outposts:us-east-1:222222222222:outpost/aa-aaaaaaaaaaaaaaaaa",
+			},
+			expSnapshot: &Snapshot{
+				SnapshotID:     "snap-test-name",
+				SourceVolumeID: "snap-test-volume",
+				Size:           10,
+				ReadyToUse:     true,
+			},
+			expErr: nil,
+		},
 	}
 
 	for _, tc := range testCases {
diff --git a/pkg/driver/controller_test.go b/pkg/driver/controller_test.go
index 819c17d5..63fc630c 100644
--- a/pkg/driver/controller_test.go
+++ b/pkg/driver/controller_test.go
@@ -2271,6 +2271,49 @@ func TestCreateSnapshot(t *testing.T) {
 				}
 			},
 		},
+		{
+			name: "success outpost",
+			testFunc: func(t *testing.T) {
+				req := &csi.CreateSnapshotRequest{
+					Name: "test-snapshot",
+					Parameters: map[string]string{
+						"outpostArn": "arn:aws:outposts:us-east-1:222222222222:outpost/aa-aaaaaaaaaaaaaaaaa",
+					},
+					SourceVolumeId: "vol-test",
+				}
+				expSnapshot := &csi.Snapshot{
+					ReadyToUse: true,
+				}
+
+				ctx := context.Background()
+				mockSnapshot := &cloud.Snapshot{
+					SnapshotID:     fmt.Sprintf("snapshot-%d", rand.New(rand.NewSource(time.Now().UnixNano())).Uint64()),
+					SourceVolumeID: req.GetSourceVolumeId(),
+					Size:           1,
+					CreationTime:   time.Now(),
+				}
+				mockCtl := gomock.NewController(t)
+				defer mockCtl.Finish()
+
+				mockCloud := cloud.NewMockCloud(mockCtl)
+				mockCloud.EXPECT().CreateSnapshot(gomock.Eq(ctx), gomock.Eq(req.GetSourceVolumeId()), gomock.Any()).Return(mockSnapshot, nil)
+				mockCloud.EXPECT().GetSnapshotByName(gomock.Eq(ctx), gomock.Eq(req.GetName())).Return(nil, cloud.ErrNotFound)
+
+				awsDriver := ControllerService{
+					cloud:    mockCloud,
+					inFlight: internal.NewInFlight(),
+					options:  &Options{},
+				}
+				resp, err := awsDriver.CreateSnapshot(context.Background(), req)
+				if err != nil {
+					t.Fatalf("Unexpected error: %v", err)
+				}
+
+				if snap := resp.GetSnapshot(); snap == nil {
+					t.Fatalf("Expected snapshot %v, got nil", expSnapshot)
+				}
+			},
+		},
 		{
 			name: "success with cluster-id",
 			testFunc: func(t *testing.T) {
@@ -2418,6 +2461,35 @@ func TestCreateSnapshot(t *testing.T) {
 				}
 			},
 		},
+		{
+			name: "fail outpost arn not valid",
+			testFunc: func(t *testing.T) {
+				req := &csi.CreateSnapshotRequest{
+					Name: "test-snapshot",
+					Parameters: map[string]string{
+						"outpostArn": "notAnArn",
+					},
+					SourceVolumeId: "vol-test",
+				}
+
+				ctx := context.Background()
+
+				mockCtl := gomock.NewController(t)
+				defer mockCtl.Finish()
+
+				mockCloud := cloud.NewMockCloud(mockCtl)
+				mockCloud.EXPECT().GetSnapshotByName(gomock.Eq(ctx), gomock.Eq(req.GetName())).Return(nil, cloud.ErrNotFound)
+
+				awsDriver := ControllerService{
+					cloud:    mockCloud,
+					inFlight: internal.NewInFlight(),
+					options:  &Options{},
+				}
+				_, err := awsDriver.CreateSnapshot(context.Background(), req)
+				checkExpectedErrorCode(t, err, codes.InvalidArgument)
+
+			},
+		},
 		{
 			name: "fail same name different volume ID",
 			testFunc: func(t *testing.T) {