Repository: ceph/ceph-csi
Pull request #2478 (2021-09-01 19:18:55)

rbd-nbd: provide alternatives to preserve the ceph log files
<!-- Please take a look at our [Contributing](https://github.com/ceph/ceph-csi/blob/devel/docs/development-guide.md#Code-contribution-workflow)
documentation before submitting a Pull Request!
Thank you for contributing to ceph-csi! -->

# Describe what this PR does #

* **Fix cephLogDir passing to storageclass via helm**

   `cephLogDir`: is a storage class option that is passed to rbd-nbd daemon.
   `cephLogDirHostPath`: is a nodeplugin daemonset level option that helps in using the right host-path while bind-mounting

* **provide alternatives to preserve the ceph log files**
  Currently, we delete the ceph client log file on unmap/detach.

  This patch provides additional alternatives for users who would like to persist the log files.

  **Strategies:**
  
  `remove`: delete log file on unmap/detach
  `compress`: compress the log file to gzip on unmap/detach, in case there exists a `.gz` file from previous map/unmap of the same volume, then override the previous log with new log.
  `preserve`: preserve the log file in text format

  Note that the default strategy will be remove on unmap, and these options can be tweaked from the storage class

  Compression size details example:

  **On Map:** (with debug-rbd=20)
  $ ls -lh
  -rw-r--r-- 1 root root 526K Sep  1 18:15 rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.log

  **On unmap:**
  $ ls -lh
  -rw-r--r-- 1 root root  33K Sep  1 18:15 rbd-nbd-0001-0024-fed5480a-f00f-417a-a51d-31d8a8144c03-0000000000000003-d2e89c87-0b4d-11ec-8ea6-160f128e682d.gz

Signed-off-by: Prasanna Kumar Kalever \<prasanna.kalever@redhat.com\>

---

<details>
<summary>Show available bot commands</summary>

These commands are normally not required, but in case of issues, leave any of
the following bot commands in an otherwise empty comment in this PR:

- `/retest ci/centos/<job-name>`: retest the `<job-name>` after unrelated
  failure (please report the failure too!)
- `/retest all`: run this in case the CentOS CI failed to start/report any test
  progress or results

</details>


pkalever: > looks like a useful functionality, I hope the e2e tests can be extended to consume this too

yes, that is the intention behind this PR.

pkalever: > Please fix CI failures.

@Madhu-1 Done now
mergify[bot]: This pull request now has conflicts with the target branch. Could you please resolve conflicts and force push the corrected changes? 🙏
pkalever: @Madhu-1 @humblec please reveiw.
pkalever: @Mergifyio rebase
mergify[bot]: **Command `rebase`: success**
> **Branch has been successfully rebased**
> 

pkalever: @Yuggupta27 @Rakshith-R @yati1998 please review.
Madhu-1: @mergifyio rebase
mergify[bot]: **Command `rebase`: success**
> **Branch already up to date**
> 


Patch:
diff --git a/charts/ceph-csi-rbd/README.md b/charts/ceph-csi-rbd/README.md
index ef2cc6d51..46223c54d 100644
--- a/charts/ceph-csi-rbd/README.md
+++ b/charts/ceph-csi-rbd/README.md
@@ -138,7 +138,7 @@ charts and their default values.
 | `provisionerSocketFile`                        | The filename of the provisioner socket                                                                                                               | `csi-provisioner.sock`                             |
 | `pluginSocketFile`                             | The filename of the plugin socket                                                                                                                    | `csi.sock`                                         |
 | `kubeletDir`                                   | kubelet working directory                                                                                                                            | `/var/lib/kubelet`                                 |
-| `cephLogDir`                                   | Host path location for ceph client processes logging, ex: rbd-nbd                                                                                    | `/var/log/ceph`                                    |
+| `cephLogDirHostPath`                           | Host path location for ceph client processes logging, ex: rbd-nbd                                                                                    | `/var/log/ceph`                                    |
 | `driverName`                                   | Name of the csi-driver                                                                                                                               | `rbd.csi.ceph.com`                                 |
 | `configMapName`                                | Name of the configmap which contains cluster configuration                                                                                           | `ceph-csi-config`                                  |
 | `externallyManagedConfigmap`                   | Specifies the use of an externally provided configmap                                                                                                | `false`                                            |
@@ -151,6 +151,8 @@ charts and their default values.
 | `storageClass.thickProvision`                  | Specifies whether thick provision should be enabled                                                                                                  | `false`                                            |
 | `storageclass.imageFeatures`                   | Specifies RBD image features                                                                                                                         | `layering`                                         |
 | `storageClass.mounter`                         | Specifies RBD mounter                                                                                                                                | `""`                                               |
+| `storageClass.cephLogDir`                      | ceph client log location, it is the target bindmount path used inside container                                                                      | `"/var/log/ceph"`                                  |
+| `storageClass.cephLogStrategy`                 | ceph client log strategy, available options `remove` or `compress` or `preserve`                                                                     | `"remove"`                                         |
 | `storageClass.volumeNamePrefix`                | Prefix to use for naming RBD images                                                                                                                  | `""`                                               |
 | `storageClass.encrypted`                       | Specifies whether volume should be encrypted. Set it to true if you want to enable encryption                                                        | `""`                                               |
 | `storageClass.encryptionKMSID`                 | Specifies the encryption kms id                                                                                                                      | `""`                                               |
diff --git a/charts/ceph-csi-rbd/templates/nodeplugin-daemonset.yaml b/charts/ceph-csi-rbd/templates/nodeplugin-daemonset.yaml
index 76b45bf38..1ccc49d08 100644
--- a/charts/ceph-csi-rbd/templates/nodeplugin-daemonset.yaml
+++ b/charts/ceph-csi-rbd/templates/nodeplugin-daemonset.yaml
@@ -175,7 +175,7 @@ spec:
             type: DirectoryOrCreate
         - name: ceph-logdir
           hostPath:
-            path: {{ .Values.cephLogDir }}
+            path: {{ .Values.cephLogDirHostPath }}
             type: DirectoryOrCreate
         - name: host-dev
           hostPath:
diff --git a/charts/ceph-csi-rbd/templates/nodeplugin-psp.yaml b/charts/ceph-csi-rbd/templates/nodeplugin-psp.yaml
index fc10f134d..9fb310063 100644
--- a/charts/ceph-csi-rbd/templates/nodeplugin-psp.yaml
+++ b/charts/ceph-csi-rbd/templates/nodeplugin-psp.yaml
@@ -42,7 +42,7 @@ spec:
       readOnly: true
     - pathPrefix: '/lib/modules'
       readOnly: true
-    - pathPrefix: '{{ .Values.cephLogDir }}'
+    - pathPrefix: '{{ .Values.cephLogDirHostPath }}'
       readOnly: false
     - pathPrefix: '{{ .Values.kubeletDir }}'
       readOnly: false
diff --git a/charts/ceph-csi-rbd/templates/storageclass.yaml b/charts/ceph-csi-rbd/templates/storageclass.yaml
index e8cb6fc60..88d12c1a4 100644
--- a/charts/ceph-csi-rbd/templates/storageclass.yaml
+++ b/charts/ceph-csi-rbd/templates/storageclass.yaml
@@ -18,6 +18,12 @@ parameters:
 {{- if .Values.storageClass.mounter }}
   mounter: {{ .Values.storageClass.mounter }}
 {{- end }}
+{{- if .Values.storageClass.cephLogDir }}
+  cephLogDir: {{ .Values.storageClass.cephLogDir }}
+{{- end }}
+{{- if .Values.storageClass.cephLogStrategy }}
+  cephLogStrategy: {{ .Values.storageClass.cephLogStrategy }}
+{{- end }}
 {{- if .Values.storageClass.dataPool }}
   dataPool: {{ .Values.storageClass.dataPool }}
 {{- end }}
diff --git a/charts/ceph-csi-rbd/values.yaml b/charts/ceph-csi-rbd/values.yaml
index 1e967cc20..655fce1aa 100644
--- a/charts/ceph-csi-rbd/values.yaml
+++ b/charts/ceph-csi-rbd/values.yaml
@@ -287,6 +287,22 @@ storageClass:
   # mounter: rbd-nbd
   mounter: ""
 
+  # (optional) ceph client log location, eg: rbd-nbd
+  # By default host-path /var/log/ceph of node is bind-mounted into
+  # csi-rbdplugin pod at /var/log/ceph mount path. This is to configure
+  # target bindmount path used inside container for ceph clients logging.
+  # See docs/rbd-nbd.md for available configuration options.
+  # cephLogDir: /var/log/ceph
+  cephLogDir: ""
+
+  # (optional) ceph client log strategy
+  # By default, log file belonging to a particular volume will be deleted
+  # on unmap, but you can choose to just compress instead of deleting it
+  # or even preserve the log file in text format as it is.
+  # Available options `remove` or `compress` or `preserve`
+  # cephLogStrategy: remove
+  cephLogStrategy: ""
+
   # (optional) Prefix to use for naming RBD images.
   # If omitted, defaults to "csi-vol-".
   # volumeNamePrefix: "foo-bar-"
@@ -403,7 +419,7 @@ pluginSocketFile: csi.sock
 # kubelet working directory,can be set using `--root-dir` when starting kubelet.
 kubeletDir: /var/lib/kubelet
 # Host path location for ceph client processes logging, ex: rbd-nbd
-cephLogDir: /var/log/ceph
+cephLogDirHostPath: /var/log/ceph
 # Name of the csi-driver
 driverName: rbd.csi.ceph.com
 # Name of the configmap used for state
diff --git a/docs/rbd-nbd.md b/docs/rbd-nbd.md
index cadf0c400..fc409c47c 100644
--- a/docs/rbd-nbd.md
+++ b/docs/rbd-nbd.md
@@ -31,38 +31,48 @@ in the StorageClass.
 
 ### Configuring logging path
 
-If you are using the default rbd nodeplugin daemonset and StorageClass
+If you are using the default rbd nodePlugin DaemonSet and StorageClass
 templates then `cephLogDir` will be `/var/log/ceph`, this directory will be
 a host-path and the default log file path will be
 `/var/log/ceph/rbd-nbd-<volID>.log`. rbd-nbd creates a log file per volume
 under the `cephLogDir` path on NodeStage(map) and removed the same on
 the respective NodeUnstage(unmap).
 
-In case if you need a customized log path, you should do below:
+- There are different strategies to maintain the logs
+  - `remove`: delete log file on unmap/detach (default behaviour)
+  - `compress`: compress the log file to gzip on unmap/detach, in case there
+    exists a `.gz` file from previous map/unmap of the same volume, then
+    override the previous log with new log.
+  - `preserve`: preserve the log file in text format
 
-- Edit the daemonset templates to change the `cephLogDir`
-  - If you are using helm charts, then you can use key `cephLogDir`
+  You can tweak the log strategies through `cephLogStrategy` option from the
+storageclass yaml
 
-    ```
-    helm install --set cephLogDir=/var/log/ceph-csi/my-dir
-    ```
+- In case if you need a customized log path, you should do below:
 
-  - For standard templates edit [csi-rbdplugin.yaml](../deploy/rbd/kubernetes/csi-rbdplugin.yaml)
-    to update `hostPath` for `ceph-logdir`, also edit psp [csi-nodeplugin-psp.yaml](../deploy/rbd/kubernetes/csi-nodeplugin-psp.yaml)
-    to update `pathPrefix` spec entries.
-- Update the StorageClass with the customized log directory path
-  - Now update rbd StorageClass for `cephLogDir`, for example
+  - Edit the DaemonSet templates to change the ceph log directory host-path
+    - If you are using helm charts, then you can use key `cephLogDirHostPath`
 
-    ```
-    cephLogDir: "/var/log/prod-A-logs"
-    ```
+      ```
+      helm install --set cephLogDirHostPath=/var/log/ceph-csi/my-dir
+      ```
+
+    - For standard templates edit [csi-rbdplugin.yaml](../deploy/rbd/kubernetes/csi-rbdplugin.yaml)
+      to update `hostPath` for `ceph-logdir`, also edit psp [csi-nodeplugin-psp.yaml](../deploy/rbd/kubernetes/csi-nodeplugin-psp.yaml)
+      to update `pathPrefix` spec entries.
+  - Update the StorageClass with the customized log directory path
+    - Now update rbd StorageClass for `cephLogDir`, for example
+
+      ```
+      cephLogDir: "/var/log/prod-A-logs"
+      ```
 
 `NOTE`:
 
 - On uninstall make sure to delete `cephLogDir` on host manually to freeup
   some space just in case if there are any uncleaned log files.
-- In case if you do not need the rbd-nbd logging to persistent, then just
-  update the StorageClass for `cephLogDir` to use a non-persistent path.
+- In case if you do not need the rbd-nbd logging to persistent at all, then
+  simply update the StorageClass for `cephLogDir` to use a non-persistent path.
 
 ## Status
 
diff --git a/examples/rbd/storageclass.yaml b/examples/rbd/storageclass.yaml
index 80b421412..61182eb10 100644
--- a/examples/rbd/storageclass.yaml
+++ b/examples/rbd/storageclass.yaml
@@ -71,10 +71,18 @@ parameters:
 
    # (optional) ceph client log location, eg: rbd-nbd
    # By default host-path /var/log/ceph of node is bind-mounted into
-   # csi-rbdplugin pod at /var/log/ceph mount path. See docs/rbd-nbd.md
-   # for available configuration options.
+   # csi-rbdplugin pod at /var/log/ceph mount path. This is to configure
+   # target bindmount path used inside container for ceph clients logging.
+   # See docs/rbd-nbd.md for available configuration options.
    # cephLogDir: /var/log/ceph
 
+   # (optional) ceph client log strategy
+   # By default, log file belonging to a particular volume will be deleted
+   # on unmap, but you can choose to just compress instead of deleting it
+   # or even preserve the log file in text format as it is.
+   # Available options `remove` or `compress` or `preserve`
+   # cephLogStrategy: remove
+
    # (optional) Prefix to use for naming RBD images.
    # If omitted, defaults to "csi-vol-".
    # volumeNamePrefix: "foo-bar-"
diff --git a/internal/rbd/nodeserver.go b/internal/rbd/nodeserver.go
index 47d1a8008..c8a0e114e 100644
--- a/internal/rbd/nodeserver.go
+++ b/internal/rbd/nodeserver.go
@@ -225,6 +225,10 @@ func populateRbdVol(
 	if rv.LogDir == "" {
 		rv.LogDir = defaultLogDir
 	}
+	rv.LogStrategy = req.GetVolumeContext()["cephLogStrategy"]
+	if rv.LogStrategy == "" {
+		rv.LogStrategy = defaultLogStrategy
+	}
 
 	return rv, err
 }
@@ -846,8 +850,9 @@ func (ns *NodeServer) NodeUnstageVolume(
 		volumeID:          req.GetVolumeId(),
 		unmapOptions:      imgInfo.UnmapOptions,
 		logDir:            imgInfo.LogDir,
+		logStrategy:       imgInfo.LogStrategy,
 	}
-	if err = detachRBDImageOrDeviceSpec(ctx, dArgs); err != nil {
+	if err = detachRBDImageOrDeviceSpec(ctx, &dArgs); err != nil {
 		log.ErrorLog(
 			ctx,
 			"error unmapping volume (%s) from staging path (%s): (%v)",
diff --git a/internal/rbd/rbd_attach.go b/internal/rbd/rbd_attach.go
index 350e57393..789bf097e 100644
--- a/internal/rbd/rbd_attach.go
+++ b/internal/rbd/rbd_attach.go
@@ -102,6 +102,7 @@ type detachRBDImageArgs struct {
 	volumeID          string
 	unmapOptions      string
 	logDir            string
+	logStrategy       string
 }
 
 // rbdGetDeviceList queries rbd about mapped devices and returns a list of rbdDeviceInfo
@@ -383,8 +384,9 @@ func createPath(ctx context.Context, volOpt *rbdVolume, device string, cr *util.
 				volumeID:          volOpt.VolID,
 				unmapOptions:      volOpt.UnmapOptions,
 				logDir:            volOpt.LogDir,
+				logStrategy:       volOpt.LogStrategy,
 			}
-			detErr := detachRBDImageOrDeviceSpec(ctx, dArgs)
+			detErr := detachRBDImageOrDeviceSpec(ctx, &dArgs)
 			if detErr != nil {
 				log.WarningLog(ctx, "rbd: %s unmap error %v", imagePath, detErr)
 			}
@@ -436,14 +438,14 @@ func detachRBDDevice(ctx context.Context, devicePath, volumeID, unmapOptions str
 		unmapOptions:      unmapOptions,
 	}
 
-	return detachRBDImageOrDeviceSpec(ctx, dArgs)
+	return detachRBDImageOrDeviceSpec(ctx, &dArgs)
 }
 
 // detachRBDImageOrDeviceSpec detaches an rbd imageSpec or devicePath, with additional checking
 // when imageSpec is used to decide if image is already unmapped.
 func detachRBDImageOrDeviceSpec(
 	ctx context.Context,
-	dArgs detachRBDImageArgs) error {
+	dArgs *detachRBDImageArgs) error {
 	if dArgs.encrypted {
 		mapperFile, mapperPath := util.VolumeMapper(dArgs.volumeID)
 		mappedDevice, mapper, err := util.DeviceEncryptionStatus(ctx, mapperPath)
@@ -490,10 +492,7 @@ func detachRBDImageOrDeviceSpec(
 	}
 	if dArgs.isNbd && dArgs.logDir != "" {
 		logFile := getCephClientLogFileName(dArgs.volumeID, dArgs.logDir, "rbd-nbd")
-		if err = os.Remove(logFile); err != nil {
-			log.WarningLog(ctx, "failed to remove logfile: %s, error: %v",
-				logFile, err)
-		}
+		go strategicActionOnLogFile(ctx, dArgs.logStrategy, logFile)
 	}
 
 	return nil
diff --git a/internal/rbd/rbd_util.go b/internal/rbd/rbd_util.go
index df5323da4..aeb63ab6a 100644
--- a/internal/rbd/rbd_util.go
+++ b/internal/rbd/rbd_util.go
@@ -53,6 +53,7 @@ const (
 	rbdDefaultMounter        = "rbd"
 	rbdNbdMounter            = "rbd-nbd"
 	defaultLogDir            = "/var/log/ceph"
+	defaultLogStrategy       = "remove" // supports remove, compress and preserve
 
 	// Output strings returned during invocation of "ceph rbd task add remove <imagespec>" when
 	// command is not supported by ceph manager. Used to check errors and recover when the command
@@ -140,6 +141,7 @@ type rbdVolume struct {
 	MapOptions         string
 	UnmapOptions       string
 	LogDir             string
+	LogStrategy        string
 	VolName            string `json:"volName"`
 	MonValueFromSecret string `json:"monValueFromSecret"`
 	VolSize            int64  `json:"volSize"`
@@ -1515,8 +1517,9 @@ type rbdImageMetadataStash struct {
 	UnmapOptions   string `json:"unmapOptions"`
 	NbdAccess      bool   `json:"accessType"`
 	Encrypted      bool   `json:"encrypted"`
-	DevicePath     string `json:"device"` // holds NBD device path for now
-	LogDir         string `json:"logDir"` // holds the client log path
+	DevicePath     string `json:"device"`          // holds NBD device path for now
+	LogDir         string `json:"logDir"`          // holds the client log path
+	LogStrategy    string `json:"logFileStrategy"` // ceph client log strategy
 }
 
 // file name in which image metadata is stashed.
@@ -1548,6 +1551,7 @@ func stashRBDImageMetadata(volOptions *rbdVolume, metaDataPath string) error {
 	if volOptions.Mounter == rbdTonbd && hasNBD {
 		imgMeta.NbdAccess = true
 		imgMeta.LogDir = volOptions.LogDir
+		imgMeta.LogStrategy = volOptions.LogStrategy
 	}
 
 	encodedBytes, err := json.Marshal(imgMeta)
@@ -2021,3 +2025,23 @@ func CheckSliceContains(options []string, opt string) bool {
 
 	return false
 }
+
+// strategicActionOnLogFile act on log file based on cephLogStrategy.
+func strategicActionOnLogFile(ctx context.Context, logStrategy, logFile string) {
+	var err error
+
+	switch strings.ToLower(logStrategy) {
+	case "compress":
+		if err = log.GzipLogFile(logFile); err != nil {
+			log.ErrorLog(ctx, "failed to compress logfile %q: %v", logFile, err)
+		}
+	case "remove":
+		if err = os.Remove(logFile); err != nil {
+			log.ErrorLog(ctx, "failed to remove logfile %q: %v", logFile, err)
+		}
+	case "preserve":
+		// do nothing
+	default:
+		log.ErrorLog(ctx, "unknown cephLogStrategy option %q: hint: 'remove'|'compress'|'preserve'", logStrategy)
+	}
+}
diff --git a/internal/util/log/log_utils.go b/internal/util/log/log_utils.go
new file mode 100644
index 000000000..18a778996
--- /dev/null
+++ b/internal/util/log/log_utils.go
@@ -0,0 +1,52 @@
+/*
+Copyright 2021 The Ceph-CSI Authors.
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+    http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package log
+
+import (
+	"compress/gzip"
+	"io/ioutil"
+	"os"
+	"strings"
+)
+
+// GzipLogFile convert and replace log file from text format to gzip
+// compressed format.
+func GzipLogFile(pathToFile string) error {
+	// Get all the bytes from the file.
+	content, err := ioutil.ReadFile(pathToFile) // #nosec:G304, file inclusion via variable.
+	if err != nil {
+		return err
+	}
+
+	// Replace .log extension with .gz extension.
+	newExt := strings.Replace(pathToFile, ".log", ".gz", -1)
+
+	// Open file for writing.
+	gf, err := os.OpenFile(newExt, os.O_CREATE|os.O_RDWR|os.O_TRUNC, 0o644) // #nosec:G304,G302, file inclusion & perms
+	if err != nil {
+		return err
+	}
+	defer gf.Close() // #nosec:G307, error on close is not critical here
+
+	// Write compressed data.
+	w := gzip.NewWriter(gf)
+	defer w.Close()
+	if _, err = w.Write(content); err != nil {
+		os.Remove(newExt) // #nosec:G104, not important error to handle
+
+		return err
+	}
+
+	return os.Remove(pathToFile)
+}


Test patch:
diff --git a/internal/rbd/rbd_util_test.go b/internal/rbd/rbd_util_test.go
index a121daaf8..eeb94c69e 100644
--- a/internal/rbd/rbd_util_test.go
+++ b/internal/rbd/rbd_util_test.go
@@ -17,6 +17,9 @@ limitations under the License.
 package rbd
 
 import (
+	"context"
+	"io/ioutil"
+	"os"
 	"strings"
 	"testing"
 
@@ -208,3 +211,75 @@ func TestGetCephClientLogFileName(t *testing.T) {
 		})
 	}
 }
+
+func TestStrategicActionOnLogFile(t *testing.T) {
+	t.Parallel()
+	ctx := context.TODO()
+	tmpDir := t.TempDir()
+
+	var logFile [3]string
+	for i := 0; i < 3; i++ {
+		f, err := ioutil.TempFile(tmpDir, "rbd-*.log")
+		if err != nil {
+			t.Errorf("creating tempfile failed: %v", err)
+		}
+		logFile[i] = f.Name()
+	}
+
+	type args struct {
+		logStrategy string
+		logFile     string
+	}
+	tests := []struct {
+		name string
+		args args
+	}{
+		{
+			name: "test for compress",
+			args: args{
+				logStrategy: "compress",
+				logFile:     logFile[0],
+			},
+		},
+		{
+			name: "test for remove",
+			args: args{
+				logStrategy: "remove",
+				logFile:     logFile[1],
+			},
+		},
+		{
+			name: "test for preserve",
+			args: args{
+				logStrategy: "preserve",
+				logFile:     logFile[2],
+			},
+		},
+	}
+	for _, tt := range tests {
+		tt := tt
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+			strategicActionOnLogFile(ctx, tt.args.logStrategy, tt.args.logFile)
+
+			var err error
+			switch tt.args.logStrategy {
+			case "compress":
+				newExt := strings.Replace(tt.args.logFile, ".log", ".gz", -1)
+				if _, err = os.Stat(newExt); os.IsNotExist(err) {
+					t.Errorf("compressed logFile (%s) not found: %v", newExt, err)
+				}
+				os.Remove(newExt)
+			case "remove":
+				if _, err = os.Stat(tt.args.logFile); !os.IsNotExist(err) {
+					t.Errorf("logFile (%s) not removed: %v", tt.args.logFile, err)
+				}
+			case "preserve":
+				if _, err = os.Stat(tt.args.logFile); os.IsNotExist(err) {
+					t.Errorf("logFile (%s) not preserved: %v", tt.args.logFile, err)
+				}
+				os.Remove(tt.args.logFile)
+			}
+		})
+	}
+}
diff --git a/internal/util/log/log_utils_test.go b/internal/util/log/log_utils_test.go
new file mode 100644
index 000000000..bd05fdfc1
--- /dev/null
+++ b/internal/util/log/log_utils_test.go
@@ -0,0 +1,47 @@
+/*
+Copyright 2021 ceph-csi authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package log
+
+import (
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"os"
+	"strings"
+	"testing"
+)
+
+func TestGzipLogFile(t *testing.T) {
+	t.Parallel()
+	tmpDir := t.TempDir()
+	logFile, err := ioutil.TempFile(tmpDir, "rbd-*.log")
+	if err != nil {
+		fmt.Println(err)
+	}
+	defer os.Remove(logFile.Name())
+
+	if err = GzipLogFile(logFile.Name()); err != nil {
+		t.Errorf("GzipLogFile failed: %v", err)
+	}
+
+	newExt := strings.Replace(logFile.Name(), ".log", ".gz", -1)
+	if _, err = os.Stat(newExt); errors.Is(err, os.ErrNotExist) {
+		t.Errorf("compressed logFile (%s) not found: %v", newExt, err)
+	}
+
+	os.Remove(newExt)
+}