Change:
What happened: kubelet crashed due to nil pointer dereference inside calcAndStoreStats function in Jul 03 06:50:42 kubelet[4440]: E0703 06:50:42. 4440 ] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference) Jul 03 06:50:42 kubelet[4440]: goroutine 3224 [running]: Jul 03 06:50:42 kubelet[4440]: (0x3992120, 0x773bdd0) Jul 03 06:50:42 kubelet[4440]: +0x7b Jul 03 06:50:42 kubelet[4440]: (0x0, 0x0, 0x0) Jul 03 06:50:42 kubelet[4440]: +0x82 Jul 03 06:50:42 kubelet[4440]: panic(0x3992120, 0x773bdd0) Jul 03 06:50:42 kubelet[4440]: +0x1b5 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).parsePodVolumeStats(0xc00178ef00, 0xc000c96cc0, 0xe, 0xc0014b8880, 0x0, 0xc000c96cc0, 0xe, 0x0, 0x0, 0x0, ...) Jul 03 06:50:42 kubelet[4440]: +0x73 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).calcAndStoreStats(0xc00178ef00) Jul 03 06:50:42 kubelet[4440]: +0x753 Jul 03 06:50:42 kubelet[4440]: (volumeStatCalculator).StartOnce.func1.1() Jul 03 06:50:42 kubelet[4440]: +0x2a Jul 03 06:50:42 kubelet[4440]: (0xc001c02d10) Jul 03 06:50:42 kubelet[4440]: +0x54 Jul 03 06:50:42 kubelet[4440]: (0xc001c02d10, 0xdf8475800, 0x3ff0000000000000, 0xc000f1dd01, 0xc0012c37a0) Jul 03 06:50:42 kubelet[4440]: +0xf8 Jul 03 06:50:42 kubelet[4440]: created by (volumeStatCalculator).StartOnce.func1 Jul 03 06:50:42 kubelet[4440]: +0x9d I think this happens because GetMetrics function can return nil for both metric(), and error, but calcAndStoreStats expects error to be not nil when metric is nil() What you expected to happen: Skip calculating stats for volumes that do not support volume stats. How to reproduce it (as minimally and precisely as possible): Create a volume using a csi driver that does not support volume stats. Anything else we need to know?: Environment: Kubernetes version (use ): Server version 15.0 Cloud provider or hardware configuration: OS (e.g: ): Ubuntu 16.04 LTS Kernel (e.g. ): 4.4.0-141-generic Install tools:kubeadm Network plugin and version (if this is a network-related bug): Others: Tested using synology-csi driver()
Sounds like you have a PR ready. Mind letting us know the PR number ? Thanks
I do not have a PR yet, but happy to make one. I'll add the PR number once I created it.
I created a PR and linked to this issue.
Didnt this solve this issue ?
No, it does not fix the problem that metric itself can be nil. Also, my pull request fixes the problem that csi_metrics returns nil value for error and the caller expects non nill value when csi driver does not support metrics.

Discussion:
if err != nil { return nil, err } // if plugin doesnot support volume status, return. if !volumeStatsSet { <del> return nil, nil </del> <ins> return nil, volume.NewNotSupportedErrorWithDriverName( string(mc.csiClientGetter.driverName)) </ins> } // Get Volumestatus metrics, err := csiClient.NodeGetVolumeStats(ctx, mc.volumeID, mc.targetPath)

csipbv1 "github.com/container-storage-interface/spec/lib/go/csi" "k8s.io/apimachinery/pkg/api/resource" <ins> "k8s.io/kubernetes/pkg/volume" </ins> "k8s.io/kubernetes/pkg/volume/csi/fake" )

} } <ins> // test GetMetrics with a volume that does not support stats func TestGetMetricsDriverNotSupportStats(t *testing.T) { tests := []struct { name          string volumeID      string targetPath    string expectSuccess bool }{ { name:          "volume created by simple driver", expectSuccess: true, volumeID:      "foobar", targetPath:    "/mnt/foo", }, } for _, tc := range tests { metricsGetter := &metricsCsi{volumeID: tc.volumeID, targetPath: tc.targetPath} metricsGetter.csiClient = &csiDriverClient{ driverName: "com.simple.SimpleDriver", nodeV1ClientCreator: func(addr csiAddr) (csipbv1.NodeClient, io.Closer, error) { nodeClient := fake.NewNodeClientWithVolumeStats(false /* VolumeStatsCapable */) fakeCloser := fake.NewCloser(t) nodeClient.SetNodeVolumeStatsResp(getRawVolumeInfo()) return nodeClient, fakeCloser, nil }, } metrics, err := metricsGetter.GetMetrics() if err == nil { t.Fatalf("for %s: expected error, but got nil error", tc.name) } if !volume.IsNotSupported(err) { t.Fatalf("for %s, expected not supported error but got: %v", tc.name, err) } if metrics != nil { t.Fatalf("for %s, expected nil metrics, but got: %v", tc.name, metrics) } } } </ins> func getRawVolumeInfo() *csipbv1.NodeGetVolumeStatsResponse { return &csipbv1.NodeGetVolumeStatsResponse{ Usage: []*csipbv1.VolumeUsage{

} } <del> // NewNoPathDefined creates a new MetricsError with code NoPathDefined. </del> <ins> // NewNotSupportedErrorWithDriverName creates a new MetricsError with code NotSupported. // driver name is added to the error message. func NewNotSupportedErrorWithDriverName(name string) *MetricsError { return &MetricsError{ Code: ErrCodeNotSupported, Msg:  fmt.Sprintf("metrics are not supported for %s volumes", name), } } // NewNoPathDefinedError creates a new MetricsError with code NoPathDefined. </ins> func NewNoPathDefinedError() *MetricsError { return &MetricsError{ Code: ErrCodeNoPathDefined,