Repository: cloudnative-pg/cloudnative-pg

Issue:
[Bug]: rollout gets stuck when adding extensions and postgresql parameters
### Is there an existing issue already for this bug?

- [x] I have searched for an existing issue, and could not find anything. I believe this is a new bug.

### I have read the troubleshooting guide

- [x] I have read the troubleshooting guide and I think this is a new bug.

### I am running a supported version of CloudNativePG

- [x] I have read the troubleshooting guide and I think this is a new bug.

### Contact Details

_No response_

### Version

trunk (main)

### What version of Kubernetes are you using?

1.33

### What is your Kubernetes environment?

Self-managed: kind (evaluation)

### How did you install the operator?

YAML manifest

### What happened?

E2e fail when changing parameters at the same time they're adding extensions. See https://github.com/cloudnative-pg/cloudnative-pg/actions/runs/16500232246/job/46686624475.

Pod rollout waits forever for the pods to have the same configuration:

```
{"level":"debug","ts":"2025-07-28T11:18:37.234290216Z","msg":"Waiting for all Pods to have the same PostgreSQL configuration","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"postgresql-with-extensions","namespace":"cluster-imagevolume-extensions-1-4534"},"namespace":"cluster-imagevolume-extensions-1-4534","name":"postgresql-with-extensions","reconcileID":"09d79cc5-ee2a-47f3-b282-834c92eacea2","caller":"internal/controller/cluster_controller.go:1024","configurationReport":[{"podName":"postgresql-with-extensions-1","configHash":"c53e52a204c767c3b366e36cad88bba5335cafcd0eb24b39a0b2a813e0b765f2"},{"podName":"postgresql-with-extensions-2","configHash":"c53e52a204c767c3b366e36cad88bba5335cafcd0eb24b39a0b2a813e0b765f2"},{"podName":"postgresql-with-extensions-3","configHash":"c12307a05882e159fc4472edd0a523981caaeff3a7dfed42ac6df98886620aff"}]}

{"level":"debug","ts":"2025-07-28T11:18:37.234333476Z","msg":"Reconciliation loop end","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"postgresql-with-extensions","namespace":"cluster-imagevolume-extensions-1-4534"},"namespace":"cluster-imagevolume-extensions-1-4534","name":"postgresql-with-extensions","reconcileID":"09d79cc5-ee2a-47f3-b282-834c92eacea2","caller":"internal/controller/cluster_controller.go:155"}

{"level":"debug","ts":"2025-07-28T11:18:38.234701446Z","msg":"Reconciliation loop start","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"postgresql-with-extensions","namespace":"cluster-imagevolume-extensions-1-4534"},"namespace":"cluster-imagevolume-extensions-1-4534","name":"postgresql-with-extensions","reconcileID":"fb38f913-537d-4f8b-90a9-da52ccb18238","caller":"internal/controller/cluster_controller.go:153"}

{"level":"debug","ts":"2025-07-28T11:18:38.329256323Z","msg":"Waiting for all Pods to have the same PostgreSQL configuration","controller":"cluster","controllerGroup":"postgresql.cnpg.io","controllerKind":"Cluster","Cluster":{"name":"postgresql-with-extensions","namespace":"cluster-imagevolume-extensions-1-4534"},"namespace":"cluster-imagevolume-extensions-1-4534","name":"postgresql-with-extensions","reconcileID":"fb38f913-537d-4f8b-90a9-da52ccb18238","caller":"internal/controller/cluster_controller.go:1024","configurationReport":[{"podName":"postgresql-with-extensions-1","configHash":"c53e52a204c767c3b366e36cad88bba5335cafcd0eb24b39a0b2a813e0b765f2"},{"podName":"postgresql-with-extensions-2","configHash":"c53e52a204c767c3b366e36cad88bba5335cafcd0eb24b39a0b2a813e0b765f2"},{"podName":"postgresql-with-extensions-3","configHash":"c12307a05882e159fc4472edd0a523981caaeff3a7dfed42ac6df98886620aff"}]}
```

### Cluster resource

```shell

```

### Relevant log output

```shell

```

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Hints from issue comments:
@dosubot: <!-- Answer -->
There are no existing issues tracking this exact problem, but a recent change on main (PR #8115) updated the rollout logic: now, any change to container images—including adding extensions—takes precedence over parameter changes. When both are changed at once, the operator will first roll out the image/extension change, then apply parameter changes in a second step. If you try to do both in a single update, the rollout can get stuck waiting for all pods to converge on the same configuration hash, as you observed. The updated docs now warn about this behavior and recommend applying changes in separate steps if you hit this issue. See the [PR and doc update](https://github.com/cloudnative-pg/cloudnative-pg/pull/8115) for details.

If you're still seeing stuck rollouts after separating changes, double-check that your operator is running the latest main with this fix, and that your workflow aligns with the new sequencing.


<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*

---
<sup>How did I do? [Good](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=hallucination) | [Report 🐛](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/2a5b1b9c-101b-4dbf-8ccb-292be1f7f381?feedback_type=other)</sup>&nbsp;&nbsp;[![Chat with Dosu](https://dosu.dev/dosu-chat-badge.svg)](https://app.dosu.dev/89b818f3-8b5e-449b-bfb3-64a9360d9665/ask?utm_source=github)&nbsp;[![Join Discord](https://img.shields.io/badge/join-5865F2?logo=discord&logoColor=white&label=)](https://go.dosu.dev/discord-bot)&nbsp;[![Share on X](https://img.shields.io/badge/X-share-black)](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/cloudnative-pg/cloudnative-pg/issues/8162)

Solution patch:
diff --git a/docs/src/imagevolume_extensions.md b/docs/src/imagevolume_extensions.md
index bc89d3b6..4b9fba63 100644
--- a/docs/src/imagevolume_extensions.md
+++ b/docs/src/imagevolume_extensions.md
@@ -93,6 +93,13 @@ Adding an extension to a database in CloudNativePG involves a few steps:
 3. Declare the extension in the `Database` resource where you want it
    installed, if the extension supports `CREATE EXTENSION`.
 
+!!! Warning
+    Avoid making changes to extension images and PostgreSQL configuration
+    settings (such as `shared_preload_libraries`) simultaneously.
+    First, allow the pod to roll out with the new extension image, then update
+    the PostgreSQL configuration.
+    This limitation will be addressed in a future release of CloudNativePG.
+
 For illustration purposes, this guide uses a simple, fictitious extension named
 `foo` that supports `CREATE EXTENSION`.
 
diff --git a/docs/src/rolling_update.md b/docs/src/rolling_update.md
index 6dda17e9..1d04a1e2 100644
--- a/docs/src/rolling_update.md
+++ b/docs/src/rolling_update.md
@@ -1,39 +1,32 @@
 # Rolling Updates
 <!-- SPDX-License-Identifier: CC-BY-4.0 -->
 
-The operator allows you to change the PostgreSQL version used in a cluster
-while applications continue running against it.
+The operator allows changing the PostgreSQL version used in a cluster while
+applications are running against it.
 
-Rolling upgrades are triggered when:
+!!! Important
+    Only upgrades for PostgreSQL minor releases are supported.
 
-- you change the `imageName` attribute in the cluster specification;
+Rolling upgrades are started when:
 
-- you change the list of extension images in the `.spec.postgresql.extensions`
-  stanza of the cluster specification;
+- the user changes the `imageName` attribute of the cluster specification;
 
-- the [image catalog](image_catalog.md) is updated with a new image for the
-  major version used by the cluster;
+- the [image catalog](image_catalog.md) is updated with a new image for the major used by the cluster;
 
-- a change in the PostgreSQL configuration requires a restart to apply;
+- a change in the PostgreSQL configuration requires a restart to be
+  applied;
 
-- you change the `Cluster` `.spec.resources` values;
+- a change on the `Cluster` `.spec.resources` values
 
-- you resize the persistent volume claim on AKS;
+- a change in size of the persistent volume claim on AKS
 
-- the operator is updated, ensuring Pods run the latest instance manager
-  (unless [in-place updates are enabled](installation_upgrade.md#in-place-updates-of-the-instance-manager)).
+- after the operator is updated, to ensure the Pods run the latest instance
+  manager (unless [in-place updates are enabled](installation_upgrade.md#in-place-updates-of-the-instance-manager)).
 
-!!! Warning
-    Any change to container images (including extensions) takes precedence over
-    all other changes and will trigger a rollout first. For example, if you update
-    both the PostgreSQL configuration and the PostgreSQL version at the same time,
-    the container image change will take priority, and the configuration change
-    will be applied in a subsequent rollout.
+The operator starts upgrading all the replicas, one Pod at a time, and begins
+from the one with the highest serial.
 
-During a rolling upgrade, the operator upgrades all replicas one Pod at a time,
-starting from the one with the highest serial.
-
-The primary is always the last node to be upgraded.
+The primary is the last node to be upgraded.
 
 Rolling updates are configurable and can be either entirely automated
 (`unsupervised`) or requiring human intervention (`supervised`).
diff --git a/internal/management/controller/instance_controller.go b/internal/management/controller/instance_controller.go
index ee04eac8..454b9247 100644
--- a/internal/management/controller/instance_controller.go
+++ b/internal/management/controller/instance_controller.go
@@ -34,7 +34,6 @@ import (
 	"github.com/cloudnative-pg/machinery/pkg/fileutils"
 	"github.com/cloudnative-pg/machinery/pkg/log"
 	pgTime "github.com/cloudnative-pg/machinery/pkg/postgres/time"
-	"github.com/cloudnative-pg/machinery/pkg/stringset"
 	corev1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
@@ -382,11 +381,6 @@ func (r *InstanceReconciler) refreshConfigurationFiles(
 	}
 	reloadNeeded = reloadNeeded || reloadIdent
 
-	// We give priority to images changes before applying the configuration ones.
-	if r.requiresImagesRollout(ctx, cluster) {
-		return reloadNeeded, nil
-	}
-
 	// Reconcile PostgreSQL configuration
 	// This doesn't need the PG connection, but it needs to reload it in case of changes
 	reloadConfig, err := r.instance.RefreshConfigurationFilesFromCluster(
@@ -408,40 +402,6 @@ func (r *InstanceReconciler) refreshConfigurationFiles(
 	return reloadNeeded, nil
 }
 
-func (r *InstanceReconciler) requiresImagesRollout(ctx context.Context, cluster *apiv1.Cluster) bool {
-	contextLogger := log.FromContext(ctx)
-
-	latestImages := stringset.New()
-	latestImages.Put(cluster.Spec.ImageName)
-	for _, extension := range cluster.Spec.PostgresConfiguration.Extensions {
-		latestImages.Put(extension.ImageVolumeSource.Reference)
-	}
-
-	if r.runningImages == nil {
-		r.runningImages = latestImages
-		contextLogger.Info("Detected running images", "runningImages", r.runningImages.ToSortedList())
-
-		return false
-	}
-
-	contextLogger.Trace(
-		"Calculated image requirements",
-		"latestImages", latestImages.ToSortedList(),
-		"runningImages", r.runningImages.ToSortedList())
-
-	if latestImages.Eq(r.runningImages) {
-		return false
-	}
-
-	contextLogger.Info(
-		"Detected drift between the bootstrap images and the configuration. Skipping configuration reload",
-		"runningImages", r.runningImages.ToSortedList(),
-		"latestImages", latestImages.ToSortedList(),
-	)
-
-	return true
-}
-
 func (r *InstanceReconciler) reconcileFencing(ctx context.Context, cluster *apiv1.Cluster) *reconcile.Result {
 	contextLogger := log.FromContext(ctx)
 
diff --git a/internal/management/controller/manager.go b/internal/management/controller/manager.go
index e756afa7..76929fef 100644
--- a/internal/management/controller/manager.go
+++ b/internal/management/controller/manager.go
@@ -25,7 +25,6 @@ import (
 	"context"
 	"fmt"
 
-	"github.com/cloudnative-pg/machinery/pkg/stringset"
 	"go.uber.org/atomic"
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/apimachinery/pkg/types"
@@ -43,9 +42,8 @@ import (
 // the one of this PostgreSQL instance. Also, the configuration in the
 // ConfigMap is applied when needed
 type InstanceReconciler struct {
-	client        ctrl.Client
-	instance      *postgres.Instance
-	runningImages *stringset.Data
+	client   ctrl.Client
+	instance *postgres.Instance
 
 	secretVersions  map[string]string
 	extensionStatus map[string]bool
diff --git a/internal/webhook/v1/cluster_webhook.go b/internal/webhook/v1/cluster_webhook.go
index 776ff309..29ab7a5a 100644
--- a/internal/webhook/v1/cluster_webhook.go
+++ b/internal/webhook/v1/cluster_webhook.go
@@ -21,6 +21,7 @@ package v1
 
 import (
 	"context"
+	"encoding/json"
 	"fmt"
 	"slices"
 	"strconv"
@@ -240,6 +241,7 @@ func (v *ClusterCustomValidator) validateClusterChanges(r, old *apiv1.Cluster) (
 	type validationFunc func(*apiv1.Cluster, *apiv1.Cluster) field.ErrorList
 	validations := []validationFunc{
 		v.validateImageChange,
+		v.validateConfigurationChange,
 		v.validateStorageChange,
 		v.validateWalStorageChange,
 		v.validateTablespacesChange,
@@ -1259,6 +1261,30 @@ func parsePostgresQuantityValue(value string) (resource.Quantity, error) {
 	return resource.ParseQuantity(value)
 }
 
+// validateConfigurationChange determines whether a PostgreSQL configuration
+// change can be applied
+func (v *ClusterCustomValidator) validateConfigurationChange(r, old *apiv1.Cluster) field.ErrorList {
+	var result field.ErrorList
+
+	if old.Spec.ImageName != r.Spec.ImageName {
+		diff := utils.CollectDifferencesFromMaps(old.Spec.PostgresConfiguration.Parameters,
+			r.Spec.PostgresConfiguration.Parameters)
+		if len(diff) > 0 {
+			jsonDiff, _ := json.Marshal(diff)
+			result = append(
+				result,
+				field.Invalid(
+					field.NewPath("spec", "imageName"),
+					r.Spec.ImageName,
+					fmt.Sprintf("Can't change image name and configuration at the same time. "+
+						"There are differences in PostgreSQL configuration parameters: %s", jsonDiff)))
+			return result
+		}
+	}
+
+	return result
+}
+
 func validateSyncReplicaElectionConstraint(constraints apiv1.SyncReplicaElectionConstraints) *field.Error {
 	if !constraints.Enabled {
 		return nil



Test patch:
diff --git a/internal/webhook/v1/cluster_webhook_test.go b/internal/webhook/v1/cluster_webhook_test.go
index 9de2d3d6..f06a2075 100644
--- a/internal/webhook/v1/cluster_webhook_test.go
+++ b/internal/webhook/v1/cluster_webhook_test.go
@@ -514,6 +514,54 @@ var _ = Describe("configuration change validation", func() {
 		v = &ClusterCustomValidator{}
 	})
 
+	It("doesn't complain when the configuration is exactly the same", func() {
+		clusterOld := &apiv1.Cluster{
+			Spec: apiv1.ClusterSpec{
+				ImageName: "postgres:10.4",
+			},
+		}
+		clusterNew := clusterOld.DeepCopy()
+		Expect(v.validateConfigurationChange(clusterNew, clusterOld)).To(BeEmpty())
+	})
+
+	It("doesn't complain when we change a setting which is not fixed", func() {
+		clusterOld := &apiv1.Cluster{
+			Spec: apiv1.ClusterSpec{
+				ImageName: "postgres:10.4",
+			},
+		}
+		clusterNew := &apiv1.Cluster{
+			Spec: apiv1.ClusterSpec{
+				ImageName: "postgres:10.4",
+				PostgresConfiguration: apiv1.PostgresConfiguration{
+					Parameters: map[string]string{
+						"shared_buffers": "4G",
+					},
+				},
+			},
+		}
+		Expect(v.validateConfigurationChange(clusterNew, clusterOld)).To(BeEmpty())
+	})
+
+	It("complains when changing postgres major version and settings", func() {
+		clusterOld := &apiv1.Cluster{
+			Spec: apiv1.ClusterSpec{
+				ImageName: "postgres:10.4",
+			},
+		}
+		clusterNew := &apiv1.Cluster{
+			Spec: apiv1.ClusterSpec{
+				ImageName: "postgres:10.5",
+				PostgresConfiguration: apiv1.PostgresConfiguration{
+					Parameters: map[string]string{
+						"shared_buffers": "4G",
+					},
+				},
+			},
+		}
+		Expect(v.validateConfigurationChange(clusterNew, clusterOld)).To(HaveLen(1))
+	})
+
 	It("produces no error when WAL size settings are correct", func() {
 		clusterNew := &apiv1.Cluster{
 			Spec: apiv1.ClusterSpec{
diff --git a/tests/e2e/fixtures/imagevolume_extensions/cluster-with-extensions.yaml.template b/tests/e2e/fixtures/imagevolume_extensions/cluster-with-extensions.yaml.template
index 332cc3ef..5db523de 100644
--- a/tests/e2e/fixtures/imagevolume_extensions/cluster-with-extensions.yaml.template
+++ b/tests/e2e/fixtures/imagevolume_extensions/cluster-with-extensions.yaml.template
@@ -18,9 +18,9 @@ spec:
     extensions:
       - name: postgis
         ld_library_path:
-          - system
+          - syslib
         image:
-          reference: ghcr.io/niccolofei/postgis:18beta1-master-bullseye # wokeignore:rule=master
+          reference: ghcr.io/niccolofei/postgis:18beta2-master-bullseye # wokeignore:rule=master
 
   bootstrap:
     initdb:
diff --git a/tests/e2e/imagevolume_extensions_test.go b/tests/e2e/imagevolume_extensions_test.go
index 51c4c32f..c3f94302 100644
--- a/tests/e2e/imagevolume_extensions_test.go
+++ b/tests/e2e/imagevolume_extensions_test.go
@@ -198,7 +198,7 @@ var _ = Describe("ImageVolume Extensions", Label(tests.LabelPostgresConfiguratio
 					apiv1.ExtensionConfiguration{
 						Name: "pgvector",
 						ImageVolumeSource: corev1.ImageVolumeSource{
-							Reference: "ghcr.io/niccolofei/pgvector:18beta1-master-bullseye", // wokeignore:rule=master
+							Reference: "ghcr.io/niccolofei/pgvector:18beta2-master-bullseye", // wokeignore:rule=master
 						},
 					})
 				g.Expect(env.Client.Update(env.Ctx, cluster)).To(Succeed())