Repository: percona/percona-postgresql-operator
Pull request #1113: K8SPG-737: expose `/pgdata` mountpoint for node_exporter
Author: login: pooknull
Created: 2025-03-31T11:07:17Z | Merged: 2025-04-08T17:30:49Z

[![K8SPG-737](https://badgen.net/badge/JIRA/K8SPG-737/green)](https://jira.percona.com/browse/K8SPG-737) [<img width="16" alt="Powered by Pull Request Badge" src="https://user-images.githubusercontent.com/1393946/111216524-d2bb8e00-85d4-11eb-821b-ed4c00989c02.png">](https://pullrequestbadge.com/?utm_medium=github&utm_source=percona&utm_campaign=badge_info)<!-- PR-BADGE: PLEASE DO NOT REMOVE THIS COMMENT -->

https://perconadev.atlassian.net/browse/K8SPG-737

**DESCRIPTION**
---
**Problem:**
*`pmm-client`'s `node_exporter` does not send metrics for the `/pgdata` mount point*

**Cause:**
*The `/pgdata` volume is not mounted in the instance pod. In addition, even if the volume is mounted, `node_exporter` will not collect usage data for `/pgdata` unless `.spec.shareProcessNamespace` is set to `false`.*

**Solution:**
*Add `/pgdata` volume mount to the instance pods and replace  `.spec.shareProcessNamespace` to `false`*

**CHECKLIST**
---
**Jira**
- [ ] Is the Jira ticket created and referenced properly?
- [ ] Does the Jira ticket have the proper statuses for documentation (`Needs Doc`) and QA (`Needs QA`)?
- [ ] Does the Jira ticket link to the proper milestone (Fix Version field)?

**Tests**
- [ ] Is an E2E test/test case added for the new feature/change?
- [ ] Are unit tests added where appropriate?

**Config/Logging/Testability**
- [ ] Are all needed new/changed options added to default YAML files?
- [ ] Are all needed new/changed options added to the [Helm Chart](https://github.com/percona/percona-helm-charts)?
- [ ] Did we add proper logging messages for operator actions?
- [ ] Did we ensure compatibility with the previous version or cluster upgrade process?
- [ ] Does the change support oldest and newest supported PG version?
- [ ] Does the change support oldest and newest supported Kubernetes version?

[K8SPG-737]: https://perconadev.atlassian.net/browse/K8SPG-737?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ


Diff:
diff --git a/e2e-tests/tests/monitoring-pmm3/08-check-mountpoint-expose.yaml b/e2e-tests/tests/monitoring-pmm3/08-check-mountpoint-expose.yaml
new file mode 100644
index 0000000000..2ebf03a182
--- /dev/null
+++ b/e2e-tests/tests/monitoring-pmm3/08-check-mountpoint-expose.yaml
@@ -0,0 +1,20 @@
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+  - script: |-
+      set -o errexit
+      set -o xtrace
+
+      source ../../functions
+
+      series_fetched=$(curl --insecure -G "https://admin:admin@$(get_service_ip monitoring-service)/prometheus/api/v1/query_range" \
+           --data-urlencode "query=node_filesystem_free_bytes{mountpoint=\"/pgdata\"}" \
+           --data-urlencode "start=$(($(date +%s) - 300))" \
+           --data-urlencode "end=$(date +%s)" \
+           --data-urlencode "step=5s" | jq -r '.stats.seriesFetched')
+
+      if [[ $series_fetched == 0 ]]; then 
+        echo "seriesFetched is 0"
+        exit 1
+      fi
+    timeout: 360
diff --git a/e2e-tests/tests/monitoring/08-check-mountpoint-expose.yaml b/e2e-tests/tests/monitoring/08-check-mountpoint-expose.yaml
new file mode 100644
index 0000000000..2ebf03a182
--- /dev/null
+++ b/e2e-tests/tests/monitoring/08-check-mountpoint-expose.yaml
@@ -0,0 +1,20 @@
+apiVersion: kuttl.dev/v1beta1
+kind: TestStep
+commands:
+  - script: |-
+      set -o errexit
+      set -o xtrace
+
+      source ../../functions
+
+      series_fetched=$(curl --insecure -G "https://admin:admin@$(get_service_ip monitoring-service)/prometheus/api/v1/query_range" \
+           --data-urlencode "query=node_filesystem_free_bytes{mountpoint=\"/pgdata\"}" \
+           --data-urlencode "start=$(($(date +%s) - 300))" \
+           --data-urlencode "end=$(date +%s)" \
+           --data-urlencode "step=5s" | jq -r '.stats.seriesFetched')
+
+      if [[ $series_fetched == 0 ]]; then 
+        echo "seriesFetched is 0"
+        exit 1
+      fi
+    timeout: 360
diff --git a/e2e-tests/tests/monitoring/08-deleted-from-server-inventory.yaml b/e2e-tests/tests/monitoring/09-deleted-from-server-inventory.yaml
similarity index 100%
rename from e2e-tests/tests/monitoring/08-deleted-from-server-inventory.yaml
rename to e2e-tests/tests/monitoring/09-deleted-from-server-inventory.yaml
diff --git a/internal/controller/postgrescluster/instance.go b/internal/controller/postgrescluster/instance.go
index ee616a0395..ec8c8b8adc 100644
--- a/internal/controller/postgrescluster/instance.go
+++ b/internal/controller/postgrescluster/instance.go
@@ -1352,6 +1352,9 @@ func generateInstanceStatefulSetIntent(_ context.Context,
 	// containers see each other's processes.
 	// - https://docs.k8s.io/tasks/configure-pod-container/share-process-namespace/
 	sts.Spec.Template.Spec.ShareProcessNamespace = initialize.Bool(true)
+	if cluster.CompareVersion("2.7.0") >= 0 {
+		sts.Spec.Template.Spec.ShareProcessNamespace = initialize.Bool(false) // K8SPG-737: should be false
+	}
 
 	// Patroni calls the Kubernetes API and pgBackRest may interact with a cloud
 	// storage provider. Use the instance ServiceAccount and automatically mount
diff --git a/percona/pmm/pmm.go b/percona/pmm/pmm.go
index 08ae2a31fe..27a7a10ecc 100644
--- a/percona/pmm/pmm.go
+++ b/percona/pmm/pmm.go
@@ -7,6 +7,7 @@ import (
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/util/intstr"
 
+	"github.com/percona/percona-postgresql-operator/internal/postgres"
 	v2 "github.com/percona/percona-postgresql-operator/pkg/apis/pgv2.percona.com/v2"
 )
 
@@ -42,6 +43,21 @@ func sidecarContainerV2(pgc *v2.PerconaPGCluster) corev1.Container {
 
 	pmmSpec := pgc.Spec.PMM
 
+	volumeMounts := []corev1.VolumeMount{
+		{
+			Name:      "cert-volume",
+			MountPath: "/pgconf/tls",
+			ReadOnly:  true,
+		},
+	}
+	if pgc.CompareVersion("2.7.0") >= 0 {
+		volumeMounts = append(volumeMounts, corev1.VolumeMount{
+			Name:      postgres.DataVolumeMount().Name,
+			MountPath: postgres.DataVolumeMount().MountPath,
+			ReadOnly:  true,
+		})
+	}
+
 	container := corev1.Container{
 		Name:            "pmm-client",
 		Image:           pmmSpec.Image,
@@ -74,13 +90,7 @@ func sidecarContainerV2(pgc *v2.PerconaPGCluster) corev1.Container {
 				},
 			},
 		},
-		VolumeMounts: []corev1.VolumeMount{
-			{
-				Name:      "cert-volume",
-				MountPath: "/pgconf/tls",
-				ReadOnly:  true,
-			},
-		},
+		VolumeMounts: volumeMounts,
 		Env: []corev1.EnvVar{
 			{
 				Name: "POD_NAME",
@@ -306,7 +316,13 @@ func sidecarContainerV3(pgc *v2.PerconaPGCluster) corev1.Container {
 				MountPath: "/pgconf/tls",
 				ReadOnly:  true,
 			},
+			{
+				Name:      postgres.DataVolumeMount().Name,
+				MountPath: postgres.DataVolumeMount().MountPath,
+				ReadOnly:  true,
+			},
 		},
+
 		Env: []corev1.EnvVar{
 			{
 				Name: "POD_NAME",
diff --git a/percona/pmm/pmm_test.go b/percona/pmm/pmm_test.go
index 03c2e4c926..4502616aad 100644
--- a/percona/pmm/pmm_test.go
+++ b/percona/pmm/pmm_test.go
@@ -6,7 +6,6 @@ import (
 	"github.com/pkg/errors"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
-
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
@@ -136,7 +135,6 @@ func TestContainer(t *testing.T) {
 			assert.Contains(t, container.Env, tt.verificationEnvVar())
 		})
 	}
-
 }
 
 func TestSidecarContainerV2(t *testing.T) {
@@ -248,7 +246,7 @@ func TestSidecarContainerV2(t *testing.T) {
 		}
 	}
 
-	assert.Len(t, container.VolumeMounts, 1)
+	assert.Len(t, container.VolumeMounts, 2)
 	assert.Equal(t, "/pgconf/tls", container.VolumeMounts[0].MountPath)
 	assert.True(t, container.VolumeMounts[0].ReadOnly)
 }
@@ -357,7 +355,7 @@ func TestSidecarContainerV3(t *testing.T) {
 		}
 	}
 
-	assert.Len(t, container.VolumeMounts, 1)
+	assert.Len(t, container.VolumeMounts, 2)
 	assert.Equal(t, "/pgconf/tls", container.VolumeMounts[0].MountPath)
 	assert.True(t, container.VolumeMounts[0].ReadOnly)
 }


Reviews:
user: login: egegunes
state: DISMISSED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T07:40:32Z
user: login: gkech
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T07:59:53Z
user: login: pooknull
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T13:25:25Z
user: login: hors
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-08T14:47:44Z
user: login: gkech
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T15:11:03Z
user: login: nmarukovich
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-08T16:06:31Z

Review comments:
diff_hunk: @@ -267,6 +277,20 @@ func sidecarContainerV3(pgc *v2.PerconaPGCluster) corev1.Container {
 	if pgc.Spec.PMM.CustomClusterName != "" {
 		clusterName = pgc.Spec.PMM.CustomClusterName
 	}
+	volumeMounts := []corev1.VolumeMount{
+		{
+			Name:      "cert-volume",
+			MountPath: "/pgconf/tls",
+			ReadOnly:  true,
+		},
+	}
+	if pgc.CompareVersion("2.7.0") >= 0 {
path: percona/pmm/pmm.go
user: login: gkech
body: We can skip this part with the compare version for PMM3 given we are introducing it in the 2.7 as a whole.
created_at: 2025-04-08T07:59:43Z
author_association: CONTRIBUTOR
diff_hunk: @@ -267,6 +277,20 @@ func sidecarContainerV3(pgc *v2.PerconaPGCluster) corev1.Container {
 	if pgc.Spec.PMM.CustomClusterName != "" {
 		clusterName = pgc.Spec.PMM.CustomClusterName
 	}
+	volumeMounts := []corev1.VolumeMount{
+		{
+			Name:      "cert-volume",
+			MountPath: "/pgconf/tls",
+			ReadOnly:  true,
+		},
+	}
+	if pgc.CompareVersion("2.7.0") >= 0 {
path: percona/pmm/pmm.go
user: login: pooknull
body: https://github.com/percona/percona-postgresql-operator/pull/1113/commits/9816674fffaf0bf2fc62a26f77ab73c1301770b3
created_at: 2025-04-08T13:25:25Z
author_association: CONTRIBUTOR

PR comments:
user: login: JNKPercona
created_at: 2025-04-08T16:10:48Z
body: | Test name | Status |
| ------------- | ------------- |
| custom-extensions | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/custom-extensions.log) |
| custom-tls | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/custom-tls.log) |
| demand-backup | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/demand-backup.log) |
| finalizers | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/finalizers.log) |
| init-deploy | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/init-deploy.log) |
| monitoring | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/monitoring.log) |
| monitoring-pmm3 | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/monitoring-pmm3.log) |
| one-pod | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/one-pod.log) |
| operator-self-healing | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/operator-self-healing.log) |
| pitr | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/pitr.log) |
| scaling | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/scaling.log) |
| scheduled-backup | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/scheduled-backup.log) |
| self-healing | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/self-healing.log) |
| sidecars | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/sidecars.log) |
| start-from-backup | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/start-from-backup.log) |
| tablespaces | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/tablespaces.log) |
| telemetry-transfer | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/telemetry-transfer.log) |
| upgrade-consistency | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/upgrade-consistency.log) |
| upgrade-minor | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/upgrade-minor.log) |
| users | [passed](https://percona-jenkins-artifactory-public.s3.amazonaws.com/cloud-pg-operator/PR-1113/496af0db8/users.log) |
| We run 20 out of 20|

commit: https://github.com/percona/percona-postgresql-operator/pull/1113/commits/496af0db8d7ef6b0fb49105a0e4f33d87cc57975
image: `perconalab/percona-postgresql-operator:PR-1113-496af0db8`

author_association: COLLABORATOR