Repository: kubernetes-sigs/gcp-filestore-csi-driver
Pull request #266 (2022-04-29 17:04:38)

Implement NodeGetVolumeStats for NodeServer
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Follow the instructions for writing a release note: https://git.k8s.io/community/contributors/guide/release-notes.md
3. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

**What type of PR is this?**
> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
>
> /kind api-change
> /kind bug
> /kind cleanup
> /kind design
> /kind documentation
> /kind failing-test
> kind feature
> /kind flake

**What this PR does / why we need it**:

**Which issue(s) this PR fixes**:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

**Special notes for your reviewer**:

**Does this PR introduce a user-facing change?**:
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".
-->
```release-note
Expose volume metrics by implementing NodeGetVolumeStats for NodeServer.
```



Patch:
diff --git a/CHANGELOG/CHANGELOG-1.2.md b/CHANGELOG/CHANGELOG-1.2.md
deleted file mode 100644
index fda002ea..00000000
--- a/CHANGELOG/CHANGELOG-1.2.md
+++ /dev/null
@@ -1,36 +0,0 @@
-# v1.2.0 - Changelog since v1.1.4
-
-## Changes by Kind
-
-### Feature
-
-- Metric emission for multishare ([#239](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/239), [@leiyiz](https://github.com/leiyiz))
-- Cap the max retry interval time for provisioner sidecar to 1 min instead of 5mins. ([#258](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/258), [@saikat-royc](https://github.com/saikat-royc))
-- Multishare create/delete volume based on APIs ([#254](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/254), [@saikat-royc](https://github.com/saikat-royc))
-- New 'multishare' overlay ([#255](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/255), [@leiyiz](https://github.com/leiyiz))
-- Add storageclass webhook into cloudbuild ([#233](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/233), [@leiyiz](https://github.com/leiyiz))
-- Implement cloudprovider code for resize multishare instance ([#263](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/263), [@saikat-royc](https://github.com/saikat-royc))
-### Documentation
-
-### Bug or Regression
-
-- Handle uninitialized multishare controller object ([#252](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/252), [@saikat-royc](https://github.com/saikat-royc))
-
-### Other (Cleanup or Flake)
-
-- Add share cloud provider implementation ([#253](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/253), [@saikat-royc](https://github.com/saikat-royc))
-- Multishare create/delete volume based on APIs ([#254](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/254), [@saikat-royc](https://github.com/saikat-royc))
-- Fix error code mappings ([#249](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/249), [@amacaskill](https://github.com/amacaskill))
-- Dont consider multishare instances in error state as non-ready instances ([#257](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/257), [@saikat-royc](https://github.com/saikat-royc))
-- Update logging for multishares volume provisioning ([#259](https://github.com/kubernetes-sigs/gcp-filestore-csi-driver/pull/259), [@saikat-royc](https://github.com/saikat-royc))
-
-## Dependencies
-
-### Added
-_Nothing has changed._
-
-### Changed
-_Nothing has changed._
-
-### Removed
-_Nothing has changed._
diff --git a/README.md b/README.md
index ce779351..2b24c6ad 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,7 @@ dynamically created and mounted by workloads.
 ## Project Status
 Status: GA
 
-Latest image: `k8s.gcr.io/cloud-provider-gcp/gcp-filestore-csi-driver:v1.2.0`
+Latest image: `k8s.gcr.io/cloud-provider-gcp/gcp-filestore-csi-driver:v1.1.4`
 
 Also see [known issues](KNOWN_ISSUES.md) and [CHANGELOG](CHANGELOG.md).
 
@@ -25,21 +25,18 @@ This plugin is compatible with CSI version 1.3.0.
 The following table captures the compatibility matrix of the core filestore driver binary
 `k8s.gcr.io/cloud-provider-gcp/gcp-filestore-csi-driver`
 
-| Filestore CSI Driver\Kubernetes Version | 1.16 | 1.17+ |
-| --------------------------------------- | ---- | ----- |
-| v0.2.0 (alpha)                          |  no  |  no   |
-| v0.3.1 (beta)                           |  yes |  yes  |
-| v0.4.0 (beta)                           |  yes |  yes  |
-| v0.5.0 (beta)                           |  yes |  yes  |
-| v0.6.0 (beta)                           |  yes |  yes  |
-| v0.6.1 (beta)                           |  yes |  yes  |
-| v1.0.0 (GA)                             |  yes |  yes  |
-| v1.1.1 (GA)                             |  yes |  yes  |
-| v1.1.2 (GA)                             |  yes |  yes  |
-| v1.1.3 (GA)                             |  yes |  yes  |
-| v1.1.4 (GA)                             |  yes |  yes  |
-| v1.2.0 (GA)                             |  yes |  yes  |
-| master                                  |  yes |  yes  |
+| Filestore CSI Driver\Kubernetes Version | 1.14 | 1.15 | 1.16 | 1.17+ |
+| --------------------------------------- | ---- | ---- | ---- | ----- |
+| v0.2.0 (alpha)                          | yes  |  no  |  no  |  no   |
+| v0.3.1 (beta)                           | no   |  yes |  yes |  yes  |
+| v0.4.0 (beta)                           | no   |  yes |  yes |  yes  |
+| v0.5.0 (beta)                           | no   |  yes |  yes |  yes  |
+| v0.6.0 (beta)                           | no   |  yes |  yes |  yes  |
+| v0.6.1 (beta)                           | no   |  yes |  yes |  yes  |
+| v1.0.0 (GA)                             | no   |  yes |  yes |  yes  |
+| v1.1.3 (GA)                             | no   |  yes |  yes |  yes  |
+| v1.1.4 (GA)                             | no   |  yes |  yes |  yes  |
+| master                                  | no   |  yes |  yes |  yes  |
 
 The manifest bundle which captures all the driver components (driver pod which includes the containers csi-external-provisioner, csi-external-resizer, csi-external-snapshotter, gcp-filestore-driver, csi-driver-registrar, csi driver object, rbacs, pod security policies etc) can be picked up from the master branch [overlays](deploy/kubernetes/overlays) directory. We structure the overlays directory per minor version of kubernetes because not all driver components can be used with all kubernetes versions. For example volume snapshots are supported 1.17+ kubernetes versions thus [stable-1-16](deploy/kubernetes/overlays/stable-1-16) driver manifests does not contain the snapshotter sidecar. Read more about overlays [here](docs/release/overlays.md).
 
diff --git a/deploy/kubernetes/images/prow-gke-release-staging-rc-master/image.yaml b/deploy/kubernetes/images/prow-gke-release-staging-rc-master/image.yaml
index 1b7bb7c7..8b51f6e8 100644
--- a/deploy/kubernetes/images/prow-gke-release-staging-rc-master/image.yaml
+++ b/deploy/kubernetes/images/prow-gke-release-staging-rc-master/image.yaml
@@ -41,5 +41,5 @@ metadata:
 imageTag:
   name: k8s.gcr.io/cloud-provider-gcp/gcp-filestore-csi-driver
   newName: gcr.io/k8s-staging-cloud-provider-gcp/gcp-filestore-csi-driver
-  newTag: "v1.2.0-rc2"
+  newTag: "v1.1.4-rc1"
 ---
diff --git a/deploy/kubernetes/images/stable-master/image.yaml b/deploy/kubernetes/images/stable-master/image.yaml
index 97a6e02e..f62ef2d7 100644
--- a/deploy/kubernetes/images/stable-master/image.yaml
+++ b/deploy/kubernetes/images/stable-master/image.yaml
@@ -40,5 +40,5 @@ metadata:
   name: imagetag-gce-fs-driver
 imageTag:
   name: k8s.gcr.io/cloud-provider-gcp/gcp-filestore-csi-driver
-  newTag: "v1.2.0"
+  newTag: "v1.1.4"
 ---
diff --git a/pkg/cloud_provider/file/file.go b/pkg/cloud_provider/file/file.go
index c33bdb6a..b208a640 100644
--- a/pkg/cloud_provider/file/file.go
+++ b/pkg/cloud_provider/file/file.go
@@ -153,8 +153,7 @@ const (
 	backupURIFmt    = locationURIFmt + "/backups/%s"
 	shareURIFmt     = instanceURIFmt + "/shares/%s"
 	// Patch update masks
-	fileShareUpdateMask                  = "file_shares"
-	multishareInstanceCapacityUpdateMask = "capacity_gb"
+	fileShareUpdateMask = "file_shares"
 )
 
 var _ Service = &gcfsServiceManager{}
@@ -760,30 +759,8 @@ func (manager *gcfsServiceManager) StartDeleteMultishareInstanceOp(ctx context.C
 }
 
 func (manager *gcfsServiceManager) StartResizeMultishareInstanceOp(ctx context.Context, obj *MultishareInstance) (*filev1beta1multishare.Operation, error) {
-	instanceuri := instanceURI(obj.Project, obj.Location, obj.Name)
-	targetinstance := &filev1beta1multishare.Instance{
-		MultiShareEnabled: true,
-		Tier:              obj.Tier,
-		Networks: []*filev1beta1multishare.NetworkConfig{
-			{
-				Network:         obj.Network.Name,
-				Modes:           []string{"MODE_IPV4"},
-				ReservedIpRange: obj.Network.ReservedIpRange,
-				ConnectMode:     obj.Network.ConnectMode,
-			},
-		},
-		CapacityGb:  util.BytesToGb(obj.CapacityBytes),
-		KmsKeyName:  obj.KmsKeyName,
-		Labels:      obj.Labels,
-		Description: obj.Description,
-	}
-	op, err := manager.multishareInstancesService.Patch(instanceuri, targetinstance).UpdateMask(multishareInstanceCapacityUpdateMask).Context(ctx).Do()
-	if err != nil {
-		return nil, fmt.Errorf("Patch operation failed: %v for instance %+v", err, targetinstance)
-	}
-
-	klog.Infof("Started instance update operation %s for instance %+v", op.Name, targetinstance)
-	return op, nil
+	// TODO
+	return nil, nil
 }
 
 func (manager *gcfsServiceManager) StartCreateShareOp(ctx context.Context, share *Share) (*filev1beta1multishare.Operation, error) {
diff --git a/pkg/csi_driver/gcfs_driver.go b/pkg/csi_driver/gcfs_driver.go
index 70707253..039d24c1 100644
--- a/pkg/csi_driver/gcfs_driver.go
+++ b/pkg/csi_driver/gcfs_driver.go
@@ -88,6 +88,7 @@ func NewGCFSDriver(config *GCFSDriverConfig) (*GCFSDriver, error) {
 	if config.RunNode {
 		nscap := []csi.NodeServiceCapability_RPC_Type{
 			csi.NodeServiceCapability_RPC_STAGE_UNSTAGE_VOLUME,
+			csi.NodeServiceCapability_RPC_GET_VOLUME_STATS,
 		}
 		driver.ns = newNodeServer(driver, config.Mounter, config.MetadataService)
 		driver.addNodeServiceCapabilities(nscap)
diff --git a/pkg/csi_driver/node.go b/pkg/csi_driver/node.go
index 9be1a73e..e1fcd19d 100644
--- a/pkg/csi_driver/node.go
+++ b/pkg/csi_driver/node.go
@@ -25,6 +25,7 @@ import (
 	csi "github.com/container-storage-interface/spec/lib/go/csi"
 	"github.com/golang/glog"
 	"golang.org/x/net/context"
+	"golang.org/x/sys/unix"
 	"google.golang.org/grpc/codes"
 	"google.golang.org/grpc/status"
 	mount "k8s.io/mount-utils"
@@ -179,6 +180,46 @@ func (s *nodeServer) NodeGetCapabilities(ctx context.Context, req *csi.NodeGetCa
 	}, nil
 }
 
+func (s *nodeServer) NodeGetVolumeStats(ctx context.Context, req *csi.NodeGetVolumeStatsRequest) (*csi.NodeGetVolumeStatsResponse, error) {
+	if len(req.VolumeId) == 0 {
+		return nil, status.Error(codes.InvalidArgument, "NodeGetVolumeStats volume ID was empty")
+	}
+	if len(req.VolumePath) == 0 {
+		return nil, status.Error(codes.InvalidArgument, "NodeGetVolumeStats volume path was empty")
+	}
+
+	_, err := os.Lstat(req.VolumePath)
+	if err != nil {
+		if os.IsNotExist(err) {
+			return nil, status.Errorf(codes.NotFound, "path %s does not exist", req.VolumePath)
+		}
+		return nil, status.Errorf(codes.Internal, "unknown error when stat on %s: %v", req.VolumePath, err)
+	}
+
+	available, capacity, used, inodesFree, inodes, inodesUsed, err := getFSStat(req.VolumePath)
+	if err != nil {
+		return nil, status.Errorf(codes.Internal, "failed to get fs info on path %s: %v", req.VolumePath, err)
+	}
+
+	return &csi.NodeGetVolumeStatsResponse{
+		Usage: []*csi.VolumeUsage{
+			{
+				Unit:      csi.VolumeUsage_BYTES,
+				Available: available,
+				Total:     capacity,
+				Used:      used,
+			},
+			{
+				Unit:      csi.VolumeUsage_INODES,
+				Available: inodesFree,
+				Total:     inodes,
+				Used:      inodesUsed,
+			},
+		},
+	}, nil
+
+}
+
 func (s *nodeServer) NodeStageVolume(ctx context.Context, req *csi.NodeStageVolumeRequest) (*csi.NodeStageVolumeResponse, error) {
 	// Validate Arguments
 	volumeID := req.GetVolumeId()
@@ -389,3 +430,23 @@ func validateMultishareVolumeAttributes(attr map[string]string) error {
 	}
 	return nil
 }
+
+func getFSStat(path string) (available, capacity, used, inodesFree, inodes, inodesUsed int64, err error) {
+	statfs := &unix.Statfs_t{}
+	err = unix.Statfs(path, statfs)
+	if err != nil {
+		err = fmt.Errorf("failed to get fs info on path %s: %v", path, err)
+		return
+	}
+
+	// Available is blocks available * fragment size
+	available = int64(statfs.Bavail) * int64(statfs.Bsize)
+	// Capacity is total block count * fragment size
+	capacity = int64(statfs.Blocks) * int64(statfs.Bsize)
+	// Usage is block being used * fragment size (aka block size).
+	used = (int64(statfs.Blocks) - int64(statfs.Bfree)) * int64(statfs.Bsize)
+	inodes = int64(statfs.Files)
+	inodesFree = int64(statfs.Ffree)
+	inodesUsed = inodes - inodesFree
+	return
+}
diff --git a/pkg/csi_driver/node_unimpl.go b/pkg/csi_driver/node_unimpl.go
index 4fa56c7e..4fcf2240 100644
--- a/pkg/csi_driver/node_unimpl.go
+++ b/pkg/csi_driver/node_unimpl.go
@@ -26,7 +26,3 @@ import (
 func (s *nodeServer) NodeExpandVolume(ctx context.Context, req *csi.NodeExpandVolumeRequest) (*csi.NodeExpandVolumeResponse, error) {
 	return nil, status.Error(codes.Unimplemented, "NodeUnStageVolume unsupported")
 }
-
-func (s *nodeServer) NodeGetVolumeStats(ctx context.Context, req *csi.NodeGetVolumeStatsRequest) (*csi.NodeGetVolumeStatsResponse, error) {
-	return nil, status.Error(codes.Unimplemented, "NodeUnStageVolume unsupported")
-}


Test patch:
diff --git a/pkg/csi_driver/node_test.go b/pkg/csi_driver/node_test.go
index e390905e..581842aa 100644
--- a/pkg/csi_driver/node_test.go
+++ b/pkg/csi_driver/node_test.go
@@ -557,6 +557,79 @@ func TestNodeGetInfo(t *testing.T) {
 func TestNodeGetCapabilities(t *testing.T) {
 }
 
+func TestNodeGetVolumeStats(t *testing.T) {
+	testEnv := initTestNodeServer(t)
+	ns := testEnv.ns
+
+	// Setup mount target path
+	tempDir, err := ioutil.TempDir("", "ngvs")
+	if err != nil {
+		t.Fatalf("failed to setup testdir: %v", err)
+	}
+	defer os.RemoveAll(tempDir)
+	targetPath := filepath.Join(tempDir, "mount")
+	stagingPath := filepath.Join(tempDir, "staging")
+
+	req := &csi.NodePublishVolumeRequest{
+		VolumeId:          testVolumeID,
+		TargetPath:        targetPath,
+		StagingTargetPath: stagingPath,
+		Readonly:          false,
+		VolumeCapability:  testVolumeCapability,
+		VolumeContext:     testVolumeAttributes,
+	}
+	_, err = ns.NodePublishVolume(context.Background(), req)
+	if err != nil {
+		t.Fatalf("Failed to set up test by publishing default vol: %v", err)
+	}
+
+	testCases := []struct {
+		name       string
+		volumeID   string
+		volumePath string
+		expectErr  bool
+	}{
+		{
+			name:       "normal",
+			volumeID:   testVolumeID,
+			volumePath: targetPath,
+		},
+		{
+			name:       "no vol id",
+			volumePath: targetPath,
+			expectErr:  true,
+		},
+		{
+			name:      "no vol path",
+			volumeID:  testVolumeID,
+			expectErr: true,
+		},
+		{
+			name:       "bad vol path",
+			volumeID:   testVolumeID,
+			volumePath: "/mnt/fake",
+			expectErr:  true,
+		},
+	}
+
+	for _, tc := range testCases {
+		t.Run(tc.name, func(t *testing.T) {
+			req := &csi.NodeGetVolumeStatsRequest{
+				VolumeId:   tc.volumeID,
+				VolumePath: tc.volumePath,
+			}
+			_, err := ns.NodeGetVolumeStats(context.Background(), req)
+			if err != nil && !tc.expectErr {
+				t.Fatalf("Got unexpected err: %v", err)
+			}
+			if err == nil && tc.expectErr {
+				t.Fatal("Did not get error but expected one")
+			}
+		})
+	}
+
+}
+
 func validateMountPoint(t *testing.T, name string, fm *mount.FakeMounter, e *mount.MountPoint) {
 	if e == nil {
 		if len(fm.MountPoints) != 0 {