Repository: redpanda-data/redpanda
Pull request #26054: Decommission status improvements
Author: login: mmaslankaprv
Labels: name: area/build
name: area/redpanda
Created: 2025-05-06T14:24:12Z | Merged: 2025-05-14T15:57:13Z

When partition replicas area moved in the cluster from whichever reason Redpanda should expose all information required to identify issues experienced during that operation. Previously partition balancer planner included only the basic information about the failures in the status. That was the list of `ntps` that replicas failed to be reallocated and the total number of reallocations failures. In order to obtain more detailed information looking into the partition balancer log was required. Moreover the reallocations that were not even attempted during preparing an allocation plan were not reported at all. This lead to confusion as f.e. partitions with only one replica on the node that is offline that can not be moved was not reported ans the failed one.  In order to address this problem we introduced a new `allocation_failure_details` map in the partition balancer planner result. The map provides a detailed information about the failed reallocation. Additionally reallocation failure is reported when partition is marked as `immutable` in the process of planning balancer action. This makes it easy for the operator to identify any partition balancer issues without the need to look for the specific log entries.  The change is fully backward compatible as all the changes are incremental i.e. when no detailed information about failures is available Redpanda will fallback to the old basic `ntp` list. 

### REST API Changes

New field `reallocation_failure_details` was added to `decommission_status` returned from `GET /v1/brokers/{id}/decommission` endpoint.

#### Example output

```json
{
    "finished": false,
    "replicas_left": 4,
    "allocation_failures": [
        "kafka/topic-qtlpeczast/1",
        "kafka/topic-qtlpeczast/4",
    ],
    "partitions": [
        {
            "ns": "kafka",
            "topic": "__consumer_offsets",
            "partition": 0,
            "moving_to": {
                "node_id": 2,
                "core": 12312312
            },
            "bytes_left_to_move": 470,
            "bytes_moved": 0,
            "partition_size": 470,
            "reconfiguration_policy": "target_initial_retention"
        }
    ],
    "reallocation_failure_details": [
        {
            "ns": "kafka",
            "topic": "topic-qtlpeczast",
            "partition": 1,
            "error": "Missing partition size information, all replicas may be offline"
        },
        {
            "ns": "kafka",
            "topic": "topic-qtlpeczast",
            "partition": 4,
            "error": "Missing partition size information, all replicas may be offline"
        }
    ]
}

```
Fixes: INC-257
<!--
See https://github.com/redpanda-data/redpanda/blob/dev/CONTRIBUTING.md#pull-request-body
for more details and examples of what is expected in a PR body.

Content in this top section is REQUIRED. Describe, in plain language, the motivation
behind the change (bug fix, feature, improvement) in this PR and how the included
commits address it.

Add the GitHub keyword `Fixes` to link to bug(s) this PR will fix, e.g.
  Fixes #ISSUE-NUMBER, Fixes #ISSUE-NUMBER, ...

If this PR is a backport, link to the original with `Backport of PR`, e.g.
  Backport of PR #PR-NUMBER
-->

## Backports Required

<!-- Checking at least one of the checkboxes is REQUIRED if this PR is not a backport. -->

- [ ] none - not a bug fix
- [ ] none - this is a backport
- [ ] none - issue does not exist in previous branches
- [ ] none - papercut/not impactful enough to backport
- [ ] v25.1.x
- [ ] v24.3.x
- [ ] v24.2.x

## Release Notes

<!--
If the changes in this PR do not need to be mentioned in the release
notes, then don't add a sub-section and simply list `none`, e.g.

* none

Otherwise, adding a sub-section or `none` is REQUIRED if the PR is not a backport PR.
If this is a backport PR, adding contents to this section will override
the release notes section inherited from the original PR to dev.

Add one or more of the sub-sections with a short description bullet
point of the change, e.g.

### Bug Fixes

* Short description of the bug fix if this is a PR to `dev` branch.

### Features

* Short description of the feature. Explain how to configure.

### Improvements

* Short description of how this PR improves existing behavior.

-->

### Improvements
- Made it easier to detect and diagnose node operation issues



Diff:
diff --git a/src/v/cluster/BUILD b/src/v/cluster/BUILD
index 68b9bf0ad943d..afa223f2ff750 100644
--- a/src/v/cluster/BUILD
+++ b/src/v/cluster/BUILD
@@ -368,6 +368,7 @@ redpanda_cc_library(
         "partition_balancer_planner.cc",
         "partition_balancer_rpc_handler.cc",
         "partition_balancer_state.cc",
+        "partition_balancer_types.cc",
         "partition_leaders_table.cc",
         "partition_manager.cc",
         "partition_probe.cc",
diff --git a/src/v/cluster/CMakeLists.txt b/src/v/cluster/CMakeLists.txt
index 8528b125f2ecf..4a372db3efccb 100644
--- a/src/v/cluster/CMakeLists.txt
+++ b/src/v/cluster/CMakeLists.txt
@@ -185,6 +185,7 @@ v_cc_library(
     partition_balancer_backend.cc
     partition_balancer_state.cc
     partition_balancer_rpc_handler.cc
+    partition_balancer_types.cc
     producer_state.cc
     producer_state_manager.cc
     node_status_backend.cc
diff --git a/src/v/cluster/controller_api.cc b/src/v/cluster/controller_api.cc
index 90700de0fbe08..275499ed1a7a5 100644
--- a/src/v/cluster/controller_api.cc
+++ b/src/v/cluster/controller_api.cc
@@ -389,7 +389,7 @@ controller_api::get_partitions_reconfiguration_state(
     co_return ret;
 }
 
-ss::future<result<ss::chunked_fifo<model::ntp>>>
+ss::future<result<chunked_hash_map<model::ntp, reallocation_failure_details>>>
 controller_api::get_decommission_allocation_failures(model::node_id node) {
     using result_t = std::variant<
       cluster::partition_balancer_overview_reply,
@@ -439,15 +439,38 @@ controller_api::get_decommission_allocation_failures(model::node_id node) {
     } else {
         co_return std::get<cluster::errc>(result);
     }
-
-    ss::chunked_fifo<model::ntp> ret;
-    auto it = overview.decommission_realloc_failures.find(node);
-    if (it == overview.decommission_realloc_failures.end()) {
+    /**
+     * If allocation failures are empty there are two possibilities, either a
+     * previous version of the reply was received or there are no failures,
+     * either way we are going to use decommission_realloc_failures map to
+     * determine the failures.
+     */
+    chunked_hash_map<model::ntp, reallocation_failure_details> ret;
+    if (overview.reallocation_failures.empty()) {
+        auto it = overview.decommission_realloc_failures.find(node);
+        if (it == overview.decommission_realloc_failures.end()) {
+            co_return ret;
+        }
+        ret.reserve(it->second.size());
+        for (const auto& ntp : it->second) {
+            ret.emplace(
+              ntp,
+              reallocation_failure_details{
+                .replica_to_move = node,
+                .reason = change_reason::node_decommissioning,
+                .error = reallocation_error::unknown_error,
+              });
+        }
         co_return ret;
     }
-    for (const auto& ntp : it->second) {
-        ret.push_back(ntp);
+    for (auto& [ntp, details] : overview.reallocation_failures) {
+        if (
+          details.replica_to_move == node
+          && details.reason == change_reason::node_decommissioning) {
+            ret.emplace(ntp, details);
+        }
     }
+
     co_return ret;
 }
 
diff --git a/src/v/cluster/controller_api.h b/src/v/cluster/controller_api.h
index 7e495dbc34f96..4a329849a646e 100644
--- a/src/v/cluster/controller_api.h
+++ b/src/v/cluster/controller_api.h
@@ -94,7 +94,8 @@ class controller_api {
     ss::future<std::optional<backend_operation>>
       get_current_op(model::ntp, ss::shard_id);
 
-    ss::future<result<ss::chunked_fifo<model::ntp>>>
+    ss::future<
+      result<chunked_hash_map<model::ntp, reallocation_failure_details>>>
       get_decommission_allocation_failures(model::node_id);
 
     model::node_id _self;
diff --git a/src/v/cluster/partition_balancer_backend.cc b/src/v/cluster/partition_balancer_backend.cc
index 1b8a3cd580002..ed3b885b79794 100644
--- a/src/v/cluster/partition_balancer_backend.cc
+++ b/src/v/cluster/partition_balancer_backend.cc
@@ -414,8 +414,8 @@ ss::future<> partition_balancer_backend::do_tick() {
 
     _cur_term->last_tick_time = clock_t::now();
     _cur_term->last_violations = std::move(plan_data.violations);
-    _cur_term->last_tick_decommission_realloc_failures = std::move(
-      plan_data.decommission_realloc_failures);
+    _cur_term->last_tick_reallocation_failures = std::move(
+      plan_data.reallocation_failures);
     if (
       _state.topics().has_updates_in_progress()
       || plan_data.status == planner_status::actions_planned) {
@@ -571,8 +571,7 @@ partition_balancer_overview_reply partition_balancer_backend::overview() const {
 
     ret.status = _cur_term->last_status;
     ret.violations = _cur_term->last_violations;
-    ret.decommission_realloc_failures
-      = _cur_term->last_tick_decommission_realloc_failures;
+    ret.set_reallocation_failures(_cur_term->last_tick_reallocation_failures);
     ret.partitions_pending_force_recovery_count
       = _state.topics().partitions_to_force_recover().size();
     if (ret.partitions_pending_force_recovery_count > 0) {
diff --git a/src/v/cluster/partition_balancer_backend.h b/src/v/cluster/partition_balancer_backend.h
index 00e6b0642c638..8094ec932126a 100644
--- a/src/v/cluster/partition_balancer_backend.h
+++ b/src/v/cluster/partition_balancer_backend.h
@@ -139,8 +139,8 @@ class partition_balancer_backend {
           = partition_balancer_status::starting;
         size_t last_tick_in_progress_updates = 0;
 
-        absl::flat_hash_map<model::node_id, absl::btree_set<model::ntp>>
-          last_tick_decommission_realloc_failures;
+        chunked_hash_map<model::ntp, reallocation_failure_details>
+          last_tick_reallocation_failures;
 
         bool _ondemand_rebalance_requested = false;
         bool _force_health_report_refresh = false;
diff --git a/src/v/cluster/partition_balancer_planner.cc b/src/v/cluster/partition_balancer_planner.cc
index e9e6ac9796a1a..15bee270a25ab 100644
--- a/src/v/cluster/partition_balancer_planner.cc
+++ b/src/v/cluster/partition_balancer_planner.cc
@@ -66,6 +66,33 @@ distinct_from(const absl::flat_hash_set<model::node_id>& nodes) {
     return hard_constraint(std::make_unique<impl>(nodes));
 }
 
+reallocation_failure_details map_result_to_failure_details(
+  change_reason reason, std::error_code ec, model::node_id replica_to_move) {
+    reallocation_failure_details details{
+      .replica_to_move = replica_to_move,
+      .reason = reason,
+      .error = details.error = reallocation_error::unknown_error};
+    if (ec.category() == cluster::error_category()) {
+        switch (static_cast<cluster::errc>(ec.value())) {
+        case cluster::errc::no_eligible_allocation_nodes:
+            details.error = reallocation_error::no_eligible_node_found;
+            break;
+        case cluster::errc::topic_invalid_partitions_fd_limit:
+            details.error = reallocation_error::over_partition_fd_limit;
+            break;
+        case cluster::errc::topic_invalid_partitions_core_limit:
+            details.error = reallocation_error::over_partition_core_limit;
+            break;
+        case cluster::errc::topic_invalid_partitions_memory_limit:
+            details.error = reallocation_error::over_partition_memory_limit;
+            break;
+        default:
+            break;
+        }
+    }
+
+    return details;
+}
 } // namespace
 
 partition_balancer_planner::partition_balancer_planner(
@@ -187,8 +214,8 @@ class partition_balancer_planner::request_context {
 
     void increment_missing_size_count() { _partitions_with_missing_size++; }
 
-    void
-    report_decommission_reallocation_failure(model::node_id, const model::ntp&);
+    void report_reallocation_failure(
+      const model::ntp&, reallocation_failure_details);
 
     node2count_t& get_topic_to_node_count(model::topic_namespace_view tp_ns) {
         auto it = _topic2node_counts.find(tp_ns);
@@ -240,10 +267,9 @@ class partition_balancer_planner::request_context {
     // we track missing partition size info separately as it requires force
     // refresh of health report
     size_t _partitions_with_missing_size = 0;
-    // Tracks ntps with allocation failures grouped by decommissioning node.
-    static constexpr size_t max_ntps_with_reallocation_falures = 25;
-    absl::flat_hash_map<model::node_id, absl::btree_set<model::ntp>>
-      _decommission_realloc_failures;
+    static constexpr size_t max_reallocation_failures_reported = 25;
+    chunked_hash_map<model::ntp, reallocation_failure_details>
+      _reallocation_failures;
     absl::node_hash_set<model::ntp> _cancellations;
     bool _counts_rebalancing_finished = false;
     ss::abort_source& _as;
@@ -544,13 +570,13 @@ bool partition_balancer_planner::request_context::increment_failure_count() {
     }
 }
 
-void partition_balancer_planner::request_context::
-  report_decommission_reallocation_failure(
-    model::node_id node, const model::ntp& ntp) {
-    auto& ntps = _decommission_realloc_failures.try_emplace(node).first->second;
-    if (ntps.size() < max_ntps_with_reallocation_falures) {
-        ntps.emplace(ntp);
+void partition_balancer_planner::request_context::report_reallocation_failure(
+  const model::ntp& ntp, reallocation_failure_details details) {
+    if (_reallocation_failures.size() >= max_reallocation_failures_reported) {
+        return;
     }
+
+    _reallocation_failures.try_emplace(ntp, details);
 }
 
 static bool has_quorum(
@@ -590,7 +616,7 @@ class partition_balancer_planner::reassignable_partition {
     result<reallocation_step> move_replica(
       model::node_id replica,
       double max_disk_usage_ratio,
-      partition_balancer_planner::change_reason reason);
+      change_reason reason);
 
     void revert(const reallocation_step&);
 
@@ -672,7 +698,7 @@ class partition_balancer_planner::moving_partition {
 
     bool cancel_requested() const { return _cancel_requested; }
 
-    void request_cancel(partition_balancer_planner::change_reason reason) {
+    void request_cancel(change_reason reason) {
         if (!_cancel_requested) {
             vlog(
               clusterlog.info,
@@ -725,9 +751,7 @@ class partition_balancer_planner::moving_partition {
         }
     }
 
-    void report_failure(
-      std::string_view reason,
-      partition_balancer_planner::change_reason change_reason) {
+    void report_failure(std::string_view reason, change_reason change_reason) {
         if (_ctx.increment_failure_count()) {
             vlog(
               clusterlog.info,
@@ -785,8 +809,7 @@ class partition_balancer_planner::immutable_partition {
 
     immutability_reason reason() const { return _reason; }
 
-    void
-    report_failure(partition_balancer_planner::change_reason change_reason) {
+    void report_failure(change_reason change_reason) {
         ss::sstring reason;
         switch (_reason) {
         case immutability_reason::batch_full:
@@ -824,6 +847,54 @@ class partition_balancer_planner::immutable_partition {
           change_reason);
     }
 
+    bool contains_replica(model::node_id replica) const {
+        return contains_node(_replicas, replica);
+    }
+
+    reallocation_failure_details map_immutability_reason_to_failure_details(
+      change_reason reason, model::node_id replica_to_move) {
+        reallocation_failure_details details{
+          .replica_to_move = replica_to_move,
+          .reason = reason,
+          .error = details.error = reallocation_error::unknown_error};
+
+        switch (_reason) {
+        case immutability_reason::no_quorum:
+            details.error = reallocation_error::no_quorum;
+            break;
+        case immutability_reason::no_size_info:
+            details.error = reallocation_error::missing_partition_size_info;
+            break;
+        case immutability_reason::reconfiguration_state:
+            details.error = reallocation_error::reconfiguration_in_progress;
+            break;
+        case immutability_reason::disabled:
+            details.error = reallocation_error::partition_disabled;
+            break;
+        case immutability_reason::batch_full:
+            break;
+        }
+
+        return details;
+    }
+
+    void report_immutable_partition_as_reallocation_failure(
+      request_context& ctx,
+      const std::vector<model::node_id>& replicas_to_move,
+      change_reason reason) {
+        if (_reason == immutability_reason::batch_full) {
+            return;
+        }
+        // report per replica reallocation failures
+        for (const auto replica : replicas_to_move) {
+            if (contains_replica(replica)) {
+                ctx.report_reallocation_failure(
+                  ntp(),
+                  map_immutability_reason_to_failure_details(reason, replica));
+            }
+        }
+    }
+
 private:
     friend class request_context;
 
@@ -1178,9 +1249,7 @@ partition_balancer_planner::reassignable_partition::get_allocation_constraints(
 
 result<reallocation_step>
 partition_balancer_planner::reassignable_partition::move_replica(
-  model::node_id replica,
-  double max_disk_usage_ratio,
-  partition_balancer_planner::change_reason reason) {
+  model::node_id replica, double max_disk_usage_ratio, change_reason reason) {
     if (!_reallocated) {
         _reallocated = request_context::reassignment_info{
           .partition = _ctx._parent._partition_allocator
@@ -1467,7 +1536,7 @@ void partition_balancer_planner::force_reassignable_partition::
 ss::future<> partition_balancer_planner::get_node_drain_actions(
   request_context& ctx,
   const absl::flat_hash_set<model::node_id>& nodes,
-  partition_balancer_planner::change_reason reason) {
+  change_reason reason) {
     if (nodes.empty()) {
         co_return;
     }
@@ -1502,8 +1571,10 @@ ss::future<> partition_balancer_planner::get_node_drain_actions(
                         ctx.config().hard_max_disk_usage_ratio,
                         reason);
                       if (!result) {
-                          ctx.report_decommission_reallocation_failure(
-                            replica, part.ntp());
+                          ctx.report_reallocation_failure(
+                            part.ntp(),
+                            map_result_to_failure_details(
+                              reason, result.error(), replica));
                       }
                   }
               }
@@ -1527,7 +1598,11 @@ ss::future<> partition_balancer_planner::get_node_drain_actions(
               }
           },
           [&](force_reassignable_partition&) {},
-          [&](immutable_partition& part) { part.report_failure(reason); });
+          [&](immutable_partition& part) {
+              part.report_failure(reason);
+              part.report_immutable_partition_as_reallocation_failure(
+                ctx, to_move, reason);
+          });
 
         return ss::stop_iteration::no;
     });
@@ -1594,15 +1669,25 @@ ss::future<> partition_balancer_planner::get_rack_constraint_repair_actions(
                       if (part.is_original(replica)) {
                           // only move replicas that haven't been moved for
                           // other reasons
-                          (void)part.move_replica(
+                          auto r = part.move_replica(
                             replica,
                             ctx.config().soft_max_disk_usage_ratio,
                             change_reason::rack_constraint_repair);
+                          if (r.has_error()) {
+                              ctx.report_reallocation_failure(
+                                part.ntp(),
+                                map_result_to_failure_details(
+                                  change_reason::rack_constraint_repair,
+                                  r.error(),
+                                  replica));
+                          }
                       }
                   }
               },
-              [](immutable_partition& part) {
+              [&](immutable_partition& part) {
                   part.report_failure(change_reason::rack_constraint_repair);
+                  part.report_immutable_partition_as_reallocation_failure(
+                    ctx, to_move, change_reason::rack_constraint_repair);
               },
               [](moving_partition&) {},
               [](force_reassignable_partition&) {});
@@ -1819,12 +1904,23 @@ partition_balancer_planner::get_full_node_actions(request_context& ctx) {
                         });
 
                       for (const auto& replica : full_node_replicas) {
-                          (void)part.move_replica(
+                          auto r = part.move_replica(
                             replica.node_id,
                             ctx.config().soft_max_disk_usage_ratio,
                             change_reason::disk_full);
+                          if (r.has_error()) {
+                              ctx.report_reallocation_failure(
+                                part.ntp(),
+                                map_result_to_failure_details(
+                                  change_reason::disk_full,
+                                  r.error(),
+                                  replica.node_id));
+                          }
                       }
                   },
+                  [&](immutable_partition& part) {
+                      part.report_failure(change_reason::disk_full);
+                  },
                   [](auto&) {});
             });
         }
@@ -1945,6 +2041,12 @@ ss::future<> partition_balancer_planner::get_counts_rebalancing_actions(
                   ctx.config().soft_max_disk_usage_ratio,
                   change_reason::partition_count_rebalancing);
                 if (!res) {
+                    ctx.report_reallocation_failure(
+                      part.ntp(),
+                      map_result_to_failure_details(
+                        change_reason::partition_count_rebalancing,
+                        res.error(),
+                        node));
                     return;
                 }
 
@@ -1964,6 +2066,8 @@ ss::future<> partition_balancer_planner::get_counts_rebalancing_actions(
             },
             [&](immutable_partition& p) {
                 p.report_failure(change_reason::partition_count_rebalancing);
+                p.report_immutable_partition_as_reallocation_failure(
+                  ctx, {node}, change_reason::partition_count_rebalancing);
                 should_stop = false;
             },
             [](auto&) {});
@@ -2053,8 +2157,7 @@ void partition_balancer_planner::request_context::collect_actions(
 
     result.counts_rebalancing_finished = _counts_rebalancing_finished;
 
-    result.decommission_realloc_failures = std::move(
-      _decommission_realloc_failures);
+    result.reallocation_failures = std::move(_reallocation_failures);
 
     if (
       !result.cancellations.empty() || !result.reassignments.empty()
@@ -2110,26 +2213,4 @@ partition_balancer_planner::plan_actions(
     co_return result;
 }
 
-std::ostream&
-operator<<(std::ostream& o, partition_balancer_planner::change_reason r) {
-    switch (r) {
-    case partition_balancer_planner::change_reason::rack_constraint_repair:
-        fmt::print(o, "rack_constraint_repair");
-        break;
-    case partition_balancer_planner::change_reason::partition_count_rebalancing:
-        fmt::print(o, "partition_count_rebalancing");
-        break;
-    case partition_balancer_planner::change_reason::node_unavailable:
-        fmt::print(o, "node_unavailable");
-        break;
-    case partition_balancer_planner::change_reason::node_decommissioning:
-        fmt::print(o, "node_decommissioning");
-        break;
-    case partition_balancer_planner::change_reason::disk_full:
-        fmt::print(o, "disk_full");
-        break;
-    }
-    return o;
-}
-
 } // namespace cluster
diff --git a/src/v/cluster/partition_balancer_planner.h b/src/v/cluster/partition_balancer_planner.h
index e3eb90d474f99..d81778ff6a991 100644
--- a/src/v/cluster/partition_balancer_planner.h
+++ b/src/v/cluster/partition_balancer_planner.h
@@ -79,26 +79,18 @@ class partition_balancer_planner {
         waiting_for_reports,
         missing_sizes,
     };
-    /**
-     * class describing a reason underlying partition replica set change
-     */
-    enum class change_reason {
-        rack_constraint_repair,
-        partition_count_rebalancing,
-        node_decommissioning,
-        node_unavailable,
-        disk_full,
-    };
 
     struct plan_data {
         partition_balancer_violations violations;
         std::vector<ntp_reassignment> reassignments;
         std::vector<model::ntp> cancellations;
-        absl::flat_hash_map<model::node_id, absl::btree_set<model::ntp>>
-          decommission_realloc_failures;
+        chunked_hash_map<model::ntp, reallocation_failure_details>
+          reallocation_failures;
         bool counts_rebalancing_finished = false;
         size_t failed_actions_count = 0;
         status status = status::empty;
+
+        void maybe_add_reallocation_failure();
     };
 
     ss::future<plan_data>
@@ -138,8 +130,6 @@ class partition_balancer_planner {
     planner_config _config;
     partition_balancer_state& _state;
     partition_allocator& _partition_allocator;
-
-    friend std::ostream& operator<<(std::ostream&, change_reason);
 };
 
 } // namespace cluster
diff --git a/src/v/cluster/partition_balancer_rpc_handler.h b/src/v/cluster/partition_balancer_rpc_handler.h
index 0f7fd0ae0daf5..9649422b74fa6 100644
--- a/src/v/cluster/partition_balancer_rpc_handler.h
+++ b/src/v/cluster/partition_balancer_rpc_handler.h
@@ -26,8 +26,8 @@ class partition_balancer_rpc_handler final
       ss::smp_service_group,
       ss::sharded<partition_balancer_backend>&);
 
-    virtual ss::future<partition_balancer_overview_reply> overview(
-      partition_balancer_overview_request, rpc::streaming_context&) override;
+    ss::future<partition_balancer_overview_reply> overview(
+      partition_balancer_overview_request, rpc::streaming_context&) final;
 
 private:
     ss::sharded<partition_balancer_backend>& _backend;
diff --git a/src/v/cluster/partition_balancer_types.cc b/src/v/cluster/partition_balancer_types.cc
new file mode 100644
index 0000000000000..cb53cffd05c6e
--- /dev/null
+++ b/src/v/cluster/partition_balancer_types.cc
@@ -0,0 +1,220 @@
+/*
+ * Copyright 2025 Redpanda Data, Inc.
+ *
+ * Licensed as a Redpanda Enterprise file under the Redpanda Community
+ * License (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * https://github.com/redpanda-data/redpanda/blob/master/licenses/rcl.md
+ */
+
+#include "cluster/partition_balancer_types.h"
+
+#include "utils/human.h"
+#include "utils/to_string.h"
+
+namespace cluster {
+
+node_disk_space::node_disk_space(
+  model::node_id node_id, uint64_t total, uint64_t used)
+  : node_id(node_id)
+  , total(total)
+  , used(used) {}
+
+double node_disk_space::final_used_ratio() const {
+    // it sometimes may happen  that the partition replica size on one node
+    // is out of date with the total used size reported by a node space
+    // manager. This may lead to an overflow of final used ratio.
+    if (released >= used + assigned) {
+        return 0.0;
+    }
+
+    return double(used + assigned - released) / total;
+}
+
+std::ostream& operator<<(std::ostream& o, const node_disk_space& d) {
+    fmt::print(
+      o,
+      "{{total: {}, used: {}, assigned: {}, released: {}; "
+      "used ratios: orig: {:.4}, peak: {:.4}, final: {:.4}}}",
+      human::bytes(d.total),
+      human::bytes(d.used),
+      human::bytes(d.assigned),
+      human::bytes(d.released),
+      d.original_used_ratio(),
+      d.peak_used_ratio(),
+      d.final_used_ratio());
+    return o;
+}
+
+partition_balancer_violations::unavailable_node::unavailable_node(
+  model::node_id id, model::timestamp unavailable_since)
+  : id(id)
+  , unavailable_since(unavailable_since) {}
+
+std::ostream& operator<<(
+  std::ostream& o, const partition_balancer_violations::unavailable_node& u) {
+    fmt::print(o, "{{ id: {} since: {} }}", u.id, u.unavailable_since);
+    return o;
+}
+
+partition_balancer_violations::full_node::full_node(
+  model::node_id id, uint32_t disk_used_percent)
+  : id(id)
+  , disk_used_percent(disk_used_percent) {}
+
+std::ostream&
+operator<<(std::ostream& o, const partition_balancer_violations::full_node& f) {
+    fmt::print(
+      o, "{{ id: {} disk_used_percent: {} }}", f.id, f.disk_used_percent);
+    return o;
+}
+
+partition_balancer_violations::partition_balancer_violations(
+  std::vector<unavailable_node> un, std::vector<full_node> fn)
+  : unavailable_nodes(std::move(un))
+  , full_nodes(std::move(fn)) {}
+
+void partition_balancer_overview_reply::set_reallocation_failures(
+  const chunked_hash_map<model::ntp, reallocation_failure_details>&
+    reallocations) {
+    reallocation_failures.reserve(reallocations.size());
+    for (const auto& [ntp, details] : reallocations) {
+        reallocation_failures.emplace(ntp, details);
+        /**
+         * Fill in the decommission_realloc_failures map with the reallocation
+         * failures for backward compatibility.
+         */
+        if (details.reason == change_reason::node_decommissioning) {
+            auto& failed_ntps
+              = decommission_realloc_failures[details.replica_to_move];
+            failed_ntps.insert(ntp);
+        }
+    }
+}
+
+partition_balancer_overview_reply
+partition_balancer_overview_reply::copy() const {
+    partition_balancer_overview_reply copy;
+    copy.error = error;
+    copy.last_tick_time = last_tick_time;
+    copy.status = status;
+    copy.violations = violations;
+    copy.partitions_pending_force_recovery_count
+      = partitions_pending_force_recovery_count;
+    copy.partitions_pending_force_recovery_sample
+      = partitions_pending_force_recovery_sample;
+    copy.decommission_realloc_failures = decommission_realloc_failures;
+    copy.reallocation_failures.reserve(reallocation_failures.size());
+    for (const auto& [ntp, details] : reallocation_failures) {
+        copy.reallocation_failures.emplace(ntp, details);
+    }
+    return copy;
+}
+
+std::ostream&
+operator<<(std::ostream& o, const partition_balancer_violations& v) {
+    fmt::print(
+      o,
+      "{{ unavailable_nodes: {} full_nodes: {} }}",
+      v.unavailable_nodes,
+      v.full_nodes);
+    return o;
+}
+
+std::ostream& operator<<(std::ostream& os, partition_balancer_status status) {
+    switch (status) {
+    case partition_balancer_status::off:
+        os << "off";
+        break;
+    case partition_balancer_status::starting:
+        os << "starting";
+        break;
+    case partition_balancer_status::ready:
+        os << "ready";
+        break;
+    case partition_balancer_status::in_progress:
+        os << "in_progress";
+        break;
+    case partition_balancer_status::stalled:
+        os << "stalled";
+        break;
+    }
+    return os;
+}
+
+std::ostream&
+operator<<(std::ostream& o, const partition_balancer_overview_request&) {
+    fmt::print(o, "{{}}");
+    return o;
+}
+
+std::ostream&
+operator<<(std::ostream& o, const partition_balancer_overview_reply& rep) {
+    fmt::print(
+      o,
+      "{{ error: {} last_tick_time: {} status: {} violations: {}, "
+      "partitions_pending_force_recovery: {}, "
+      "decommission_reallocation_failures_count: {}, reallocation_failures: "
+      "{}}}",
+      rep.error,
+      rep.last_tick_time,
+      rep.status,
+      rep.violations,
+      rep.partitions_pending_force_recovery_count,
+      rep.decommission_realloc_failures.size(),
+      fmt::join(rep.reallocation_failures | std::views::values, ", "));
+    return o;
+}
+
+std::ostream& operator<<(std::ostream& o, change_reason reason) {
+    switch (reason) {
+    case change_reason::rack_constraint_repair:
+        return o << "rack_constraint_repair";
+    case change_reason::partition_count_rebalancing:
+        return o << "partition_count_rebalancing";
+    case change_reason::node_decommissioning:
+        return o << "node_decommissioning";
+    case change_reason::node_unavailable:
+        return o << "node_unavailable";
+    case change_reason::disk_full:
+        return o << "disk_full";
+    }
+}
+
+std::ostream& operator<<(std::ostream& o, reallocation_error err) {
+    switch (err) {
+    case reallocation_error::missing_partition_size_info:
+        return o << "Missing partition size information, all replicas may be "
+                    "offline";
+    case reallocation_error::no_eligible_node_found:
+        return o << "No eligible node found to move replica";
+    case reallocation_error::over_partition_fd_limit:
+        return o << "Over the total partition file descriptor limit";
+    case reallocation_error::over_partition_memory_limit:
+        return o << "Over the total partition memory limit";
+    case reallocation_error::over_partition_core_limit:
+        return o << "Over the partition per core limit";
+    case reallocation_error::no_quorum:
+        return o << "No quorum, majority of replicas are offline";
+    case reallocation_error::reconfiguration_in_progress:
+        return o << "Non cancellable reconfiguration in progress";
+    case reallocation_error::partition_disabled:
+        return o << "Partition is disabled";
+    case reallocation_error::unknown_error:
+        return o << "Unknown error or error not reported";
+    }
+}
+
+std::ostream&
+operator<<(std::ostream& o, const reallocation_failure_details& details) {
+    fmt::print(
+      o,
+      "{{replica_to_move: {}, reason: {}, error: {}}}",
+      details.replica_to_move,
+      details.reason,
+      details.error);
+    return o;
+}
+
+} // namespace cluster
diff --git a/src/v/cluster/partition_balancer_types.h b/src/v/cluster/partition_balancer_types.h
index d629145cac446..75e1b61c75c76 100644
--- a/src/v/cluster/partition_balancer_types.h
+++ b/src/v/cluster/partition_balancer_types.h
@@ -11,23 +11,19 @@
 #pragma once
 
 #include "cluster/errc.h"
+#include "container/chunked_hash_map.h"
 #include "model/fundamental.h"
-#include "model/metadata.h"
 #include "model/timestamp.h"
 #include "serde/envelope.h"
 #include "serde/rw/enum.h"
 #include "serde/rw/envelope.h"
 #include "serde/rw/map.h"
 #include "serde/rw/optional.h"
-#include "serde/rw/rw.h"
 #include "serde/rw/set.h"
 #include "serde/rw/vector.h"
-#include "utils/human.h"
-#include "utils/to_string.h"
 
 #include <absl/container/btree_set.h>
 #include <absl/container/flat_hash_map.h>
-#include <absl/container/flat_hash_set.h>
 
 namespace cluster {
 
@@ -40,41 +36,14 @@ struct node_disk_space {
     // total size of partitions moved from this node
     uint64_t released = 0;
 
-    inline node_disk_space(
-      model::node_id node_id, uint64_t total, uint64_t used)
-      : node_id(node_id)
-      , total(total)
-      , used(used) {}
-
+    node_disk_space(model::node_id node_id, uint64_t total, uint64_t used);
     double original_used_ratio() const { return double(used) / total; }
 
     double peak_used_ratio() const { return double(used + assigned) / total; }
 
-    double final_used_ratio() const {
-        // it sometimes may happen  that the partition replica size on one node
-        // is out of date with the total used size reported by a node space
-        // manager. This may lead to an overflow of final used ratio.
-        if (released >= used + assigned) {
-            return 0.0;
-        }
-
-        return double(used + assigned - released) / total;
-    }
+    double final_used_ratio() const;
 
-    friend std::ostream& operator<<(std::ostream& o, const node_disk_space& d) {
-        fmt::print(
-          o,
-          "{{total: {}, used: {}, assigned: {}, released: {}; "
-          "used ratios: orig: {:.4}, peak: {:.4}, final: {:.4}}}",
-          human::bytes(d.total),
-          human::bytes(d.used),
-          human::bytes(d.assigned),
-          human::bytes(d.released),
-          d.original_used_ratio(),
-          d.peak_used_ratio(),
-          d.final_used_ratio());
-        return o;
-    }
+    friend std::ostream& operator<<(std::ostream& o, const node_disk_space& d);
 };
 
 struct partition_balancer_violations
@@ -91,22 +60,15 @@ struct partition_balancer_violations
         model::timestamp unavailable_since;
 
         unavailable_node() noexcept = default;
-        unavailable_node(model::node_id id, model::timestamp unavailable_since)
-          : id(id)
-          , unavailable_since(unavailable_since) {}
+        unavailable_node(model::node_id id, model::timestamp unavailable_since);
 
         friend std::ostream&
-        operator<<(std::ostream& o, const unavailable_node& u) {
-            fmt::print(o, "{{ id: {} since: {} }}", u.id, u.unavailable_since);
-            return o;
-        }
+        operator<<(std::ostream& o, const unavailable_node& u);
 
         auto serde_fields() { return std::tie(id, unavailable_since); }
 
-        bool operator==(const unavailable_node& other) const {
-            return id == other.id
-                   && unavailable_since == other.unavailable_since;
-        }
+        friend bool operator==(const unavailable_node&, const unavailable_node&)
+          = default;
     };
 
     struct full_node
@@ -116,25 +78,13 @@ struct partition_balancer_violations
         uint32_t disk_used_percent;
 
         full_node() noexcept = default;
-        full_node(model::node_id id, uint32_t disk_used_percent)
-          : id(id)
-          , disk_used_percent(disk_used_percent) {}
-
-        friend std::ostream& operator<<(std::ostream& o, const full_node& f) {
-            fmt::print(
-              o,
-              "{{ id: {} disk_used_percent: {} }}",
-              f.id,
-              f.disk_used_percent);
-            return o;
-        }
+        full_node(model::node_id id, uint32_t disk_used_percent);
+
+        friend std::ostream& operator<<(std::ostream& o, const full_node& f);
 
         auto serde_fields() { return std::tie(id, disk_used_percent); }
 
-        bool operator==(const full_node& other) const {
-            return id == other.id
-                   && disk_used_percent == other.disk_used_percent;
-        }
+        friend bool operator==(const full_node&, const full_node&) = default;
     };
 
     std::vector<unavailable_node> unavailable_nodes;
@@ -143,26 +93,17 @@ struct partition_balancer_violations
     partition_balancer_violations() noexcept = default;
 
     partition_balancer_violations(
-      std::vector<unavailable_node> un, std::vector<full_node> fn)
-      : unavailable_nodes(std::move(un))
-      , full_nodes(std::move(fn)) {}
+      std::vector<unavailable_node> un, std::vector<full_node> fn);
 
     friend std::ostream&
-    operator<<(std::ostream& o, const partition_balancer_violations& v) {
-        fmt::print(
-          o,
-          "{{ unavailable_nodes: {} full_nodes: {} }}",
-          v.unavailable_nodes,
-          v.full_nodes);
-        return o;
-    }
+    operator<<(std::ostream& o, const partition_balancer_violations& v);
 
     auto serde_fields() { return std::tie(unavailable_nodes, full_nodes); }
 
-    bool operator==(const partition_balancer_violations& other) const {
-        return unavailable_nodes == other.unavailable_nodes
-               && full_nodes == other.full_nodes;
-    }
+    friend bool operator==(
+      const partition_balancer_violations&,
+      const partition_balancer_violations&)
+      = default;
 
     bool is_empty() const {
         return unavailable_nodes.empty() && full_nodes.empty();
@@ -177,27 +118,7 @@ enum class partition_balancer_status {
     stalled,
 };
 
-inline std::ostream&
-operator<<(std::ostream& os, partition_balancer_status status) {
-    switch (status) {
-    case partition_balancer_status::off:
-        os << "off";
-        break;
-    case partition_balancer_status::starting:
-        os << "starting";
-        break;
-    case partition_balancer_status::ready:
-        os << "ready";
-        break;
-    case partition_balancer_status::in_progress:
-        os << "in_progress";
-        break;
-    case partition_balancer_status::stalled:
-        os << "stalled";
-        break;
-    }
-    return os;
-}
+std::ostream& operator<<(std::ostream& os, partition_balancer_status status);
 
 struct partition_balancer_overview_request
   : serde::envelope<
@@ -207,21 +128,81 @@ struct partition_balancer_overview_request
     using rpc_adl_exempt = std::true_type;
 
     friend std::ostream&
-    operator<<(std::ostream& o, const partition_balancer_overview_request&) {
-        fmt::print(o, "{{}}");
-        return o;
-    }
-
+    operator<<(std::ostream& o, const partition_balancer_overview_request&);
     auto serde_fields() { return std::tie(); }
 };
 
+/**
+ * class describing a reason underlying partition replica set change
+ */
+enum class change_reason {
+    rack_constraint_repair,
+    partition_count_rebalancing,
+    node_decommissioning,
+    node_unavailable,
+    disk_full,
+};
+
+std::ostream& operator<<(std::ostream& o, change_reason rep);
+/**
+ * Enum providing a details about partition replica reallocation failure.
+ */
+enum class reallocation_error : int8_t {
+    missing_partition_size_info,
+    no_eligible_node_found,
+    over_partition_fd_limit,
+    over_partition_memory_limit,
+    over_partition_core_limit,
+    no_quorum,
+    reconfiguration_in_progress,
+    partition_disabled,
+    unknown_error,
+};
+
+std::ostream& operator<<(std::ostream& o, reallocation_error rep);
+
+/**
+ * Struct providing details about partition replica reallocation failure.
+ * The details provided include the change reason, the replica that was
+ * intended to be moved and the error.
+ */
+struct reallocation_failure_details
+  : serde::envelope<
+      reallocation_failure_details,
+      serde::version<0>,
+      serde::compat_version<0>> {
+    model::node_id replica_to_move;
+    change_reason reason;
+    reallocation_error error;
+
+    auto serde_fields() { return std::tie(replica_to_move, reason, error); }
+    friend bool operator==(
+      const reallocation_failure_details&, const reallocation_failure_details&)
+      = default;
+
+    friend std::ostream&
+    operator<<(std::ostream& o, const reallocation_failure_details& rep);
+};
+
 struct partition_balancer_overview_reply
   : serde::envelope<
       partition_balancer_overview_reply,
-      serde::version<2>,
+      serde::version<3>,
       serde::compat_version<0>> {
     using rpc_adl_exempt = std::true_type;
 
+    partition_balancer_overview_reply() noexcept = default;
+    partition_balancer_overview_reply(const partition_balancer_overview_reply&)
+      = delete;
+    partition_balancer_overview_reply(partition_balancer_overview_reply&&)
+      = default;
+    partition_balancer_overview_reply&
+    operator=(const partition_balancer_overview_reply&)
+      = delete;
+    partition_balancer_overview_reply&
+    operator=(partition_balancer_overview_reply&&)
+      = default;
+
     errc error;
     model::timestamp last_tick_time;
     partition_balancer_status status;
@@ -230,6 +211,12 @@ struct partition_balancer_overview_reply
       decommission_realloc_failures;
     size_t partitions_pending_force_recovery_count;
     std::vector<model::ntp> partitions_pending_force_recovery_sample;
+    chunked_hash_map<model::ntp, reallocation_failure_details>
+      reallocation_failures;
+
+    void set_reallocation_failures(
+      const chunked_hash_map<model::ntp, reallocation_failure_details>&
+        reallocations);
 
     auto serde_fields() {
         return std::tie(
@@ -239,33 +226,19 @@ struct partition_balancer_overview_reply
           violations,
           decommission_realloc_failures,
           partitions_pending_force_recovery_count,
-          partitions_pending_force_recovery_sample);
+          partitions_pending_force_recovery_sample,
+          reallocation_failures);
     }
 
-    bool operator==(const partition_balancer_overview_reply& other) const {
-        return error == other.error && last_tick_time == other.last_tick_time
-               && status == other.status && violations == other.violations
-               && decommission_realloc_failures
-                    == other.decommission_realloc_failures
-               && partitions_pending_force_recovery_count
-                    == other.partitions_pending_force_recovery_count
-               && partitions_pending_force_recovery_sample
-                    == other.partitions_pending_force_recovery_sample;
-    }
+    friend bool operator==(
+      const partition_balancer_overview_reply&,
+      const partition_balancer_overview_reply&)
+      = default;
 
     friend std::ostream&
-    operator<<(std::ostream& o, const partition_balancer_overview_reply& rep) {
-        fmt::print(
-          o,
-          "{{ error: {} last_tick_time: {} status: {} violations: {}, "
-          "partitions_pending_force_recovery: {}}}",
-          rep.error,
-          rep.last_tick_time,
-          rep.status,
-          rep.violations,
-          rep.partitions_pending_force_recovery_count);
-        return o;
-    }
+    operator<<(std::ostream& o, const partition_balancer_overview_reply& rep);
+
+    partition_balancer_overview_reply copy() const;
 };
 
 class balancer_tick_aborted_exception final : public std::runtime_error {
diff --git a/src/v/cluster/types.h b/src/v/cluster/types.h
index e6d3e3b8aed93..d8b16f30f7bac 100644
--- a/src/v/cluster/types.h
+++ b/src/v/cluster/types.h
@@ -16,6 +16,7 @@
 #include "cluster/feature_update_action.h"
 #include "cluster/fwd.h"
 #include "cluster/nt_revision.h"
+#include "cluster/partition_balancer_types.h"
 #include "cluster/remote_topic_properties.h"
 #include "cluster/snapshot.h"
 #include "cluster/topic_configuration.h"
@@ -3228,7 +3229,8 @@ struct node_decommission_progress {
     // number of replicas left on decommissioned node
     size_t replicas_left{0};
     // Replicas on the node with failures during reallocation.
-    ss::chunked_fifo<model::ntp> allocation_failures;
+    chunked_hash_map<model::ntp, reallocation_failure_details>
+      allocation_failures;
     // list of currently ongoing partition reconfigurations
     chunked_vector<partition_reconfiguration_state> current_reconfigurations;
 };
diff --git a/src/v/compat/cluster_json.h b/src/v/compat/cluster_json.h
index bcbb970893696..ee5fb9784f956 100644
--- a/src/v/compat/cluster_json.h
+++ b/src/v/compat/cluster_json.h
@@ -884,4 +884,24 @@ read_value(const json::Value& rd, cluster::delete_acls_result& data) {
     read_member(rd, "bindings", data.bindings);
 }
 
+inline void rjson_serialize(
+  json::Writer<json::StringBuffer>& w,
+  const cluster::reallocation_failure_details& data) {
+    w.StartObject();
+    w.Key("replica_to_move");
+    rjson_serialize(w, data.replica_to_move);
+    w.Key("reason");
+    rjson_serialize(w, data.reason);
+    w.Key("error");
+    rjson_serialize(w, data.error);
+    w.EndObject();
+}
+
+inline void
+read_value(const json::Value& rd, cluster::reallocation_failure_details& data) {
+    read_member(rd, "error", data.error);
+    read_member(rd, "replica_to_move", data.replica_to_move);
+    read_member(rd, "reason", data.reason);
+}
+
 } // namespace json
diff --git a/src/v/compat/json.h b/src/v/compat/json.h
index 90367935c26bb..a8489a6dc3a73 100644
--- a/src/v/compat/json.h
+++ b/src/v/compat/json.h
@@ -327,6 +327,21 @@ inline void read_value(const json::Value& rd, absl::flat_hash_map<T, V>& obj) {
     }
 }
 
+template<typename T, typename V>
+inline void rjson_serialize(
+  json::Writer<json::StringBuffer>& w, const chunked_hash_map<T, V>& m) {
+    w.StartArray();
+    for (const auto& e : m) {
+        w.StartObject();
+        w.Key("key");
+        rjson_serialize(w, e.first);
+        w.Key("value");
+        rjson_serialize(w, e.second);
+        w.EndObject();
+    }
+    w.EndArray();
+}
+
 template<typename V>
 inline void read_value(const json::Value& rd, absl::node_hash_set<V>& obj) {
     for (const auto& e : rd.GetArray()) {
@@ -345,6 +360,17 @@ inline void read_value(const json::Value& rd, absl::btree_set<V>& obj) {
     }
 }
 
+template<typename T, typename V>
+inline void read_value(const json::Value& rd, chunked_hash_map<T, V>& obj) {
+    for (const auto& e : rd.GetArray()) {
+        T key;
+        read_member(e, "key", key);
+        V value;
+        read_member(e, "value", value);
+        obj.emplace(std::move(key), std::move(value));
+    }
+}
+
 template<typename T>
 inline void rjson_serialize(
   json::Writer<json::StringBuffer>& w, const ss::bool_class<T>& b) {
diff --git a/src/v/compat/partition_balancer_compat.h b/src/v/compat/partition_balancer_compat.h
index ec2aac5540e5e..f244cde9da81c 100644
--- a/src/v/compat/partition_balancer_compat.h
+++ b/src/v/compat/partition_balancer_compat.h
@@ -31,6 +31,7 @@ GEN_COMPAT_CHECK_SERDE_ONLY(
       json_write(decommission_realloc_failures);
       json_write(partitions_pending_force_recovery_count);
       json_write(partitions_pending_force_recovery_sample);
+      json_write(reallocation_failures);
   },
   {
       json_read(error);
@@ -40,6 +41,7 @@ GEN_COMPAT_CHECK_SERDE_ONLY(
       json_read(decommission_realloc_failures);
       json_read(partitions_pending_force_recovery_count);
       json_read(partitions_pending_force_recovery_sample);
+      json_read(reallocation_failures);
   })
 
 } // namespace compat
diff --git a/src/v/compat/partition_balancer_generator.h b/src/v/compat/partition_balancer_generator.h
index 7e8a2009777eb..a81adfe60f3df 100644
--- a/src/v/compat/partition_balancer_generator.h
+++ b/src/v/compat/partition_balancer_generator.h
@@ -23,6 +23,39 @@ namespace compat {
 
 EMPTY_COMPAT_GENERATOR(cluster::partition_balancer_overview_request);
 
+template<>
+struct instance_generator<cluster::change_reason> {
+    static cluster::change_reason random() {
+        return random_generators::random_choice({
+          cluster::change_reason::rack_constraint_repair,
+          cluster::change_reason::partition_count_rebalancing,
+          cluster::change_reason::node_decommissioning,
+          cluster::change_reason::node_unavailable,
+          cluster::change_reason::disk_full,
+        });
+    }
+
+    static std::vector<cluster::change_reason> limits() { return {}; }
+};
+template<>
+struct instance_generator<cluster::reallocation_error> {
+    static cluster::reallocation_error random() {
+        return random_generators::random_choice({
+          cluster::reallocation_error::missing_partition_size_info,
+          cluster::reallocation_error::no_eligible_node_found,
+          cluster::reallocation_error::over_partition_fd_limit,
+          cluster::reallocation_error::over_partition_memory_limit,
+          cluster::reallocation_error::over_partition_core_limit,
+          cluster::reallocation_error::no_quorum,
+          cluster::reallocation_error::reconfiguration_in_progress,
+          cluster::reallocation_error::partition_disabled,
+          cluster::reallocation_error::unknown_error,
+        });
+    }
+
+    static std::vector<cluster::reallocation_error> limits() { return {}; }
+};
+
 template<>
 struct instance_generator<cluster::partition_balancer_overview_reply> {
     static cluster::partition_balancer_overview_reply random() {
@@ -31,23 +64,40 @@ struct instance_generator<cluster::partition_balancer_overview_reply> {
               tests::random_named_int<model::node_id>(),
               tests::random_btree_set<model::ntp>(model::random_ntp));
         };
-        return {
-          .error = instance_generator<cluster::errc>::random(),
-          .last_tick_time = model::timestamp(
-            random_generators::get_int<int64_t>()),
-          .status = tests::random_balancer_status(),
-          .violations = tests::random_partition_balancer_violations(),
-          .decommission_realloc_failures = tests::
-            random_flat_hash_map<model::node_id, absl::btree_set<model::ntp>>(
-              std::move(generator)),
-          .partitions_pending_force_recovery_count
+
+        auto failure_details_generator = [] {
+            return std::make_pair(
+              model::random_ntp(),
+              cluster::reallocation_failure_details{
+                .replica_to_move = tests::random_named_int<model::node_id>(),
+                .reason = instance_generator<cluster::change_reason>::random(),
+                .error
+                = instance_generator<cluster::reallocation_error>::random(),
+              });
+        };
+        cluster::partition_balancer_overview_reply reply;
+
+        reply.error = instance_generator<cluster::errc>::random(),
+        reply.last_tick_time = model::timestamp(
+          random_generators::get_int<int64_t>()),
+        reply.status = tests::random_balancer_status(),
+        reply.violations = tests::random_partition_balancer_violations(),
+        reply.decommission_realloc_failures = tests::
+          random_flat_hash_map<model::node_id, absl::btree_set<model::ntp>>(
+            std::move(generator)),
+        reply.partitions_pending_force_recovery_count
           = random_generators::get_int<size_t>(),
-          .partitions_pending_force_recovery_sample = tests::random_vector(
-            model::random_ntp)};
+        reply.partitions_pending_force_recovery_sample = tests::random_vector(
+          model::random_ntp);
+        reply.reallocation_failures = tests::random_chunked_hash_map<
+          model::ntp,
+          cluster::reallocation_failure_details>(
+          std::move(failure_details_generator));
+        return reply;
     }
 
     static std::vector<cluster::partition_balancer_overview_reply> limits() {
-        return {{}};
+        return {};
     }
 };
 
diff --git a/src/v/redpanda/admin/api-doc/broker.json b/src/v/redpanda/admin/api-doc/broker.json
index 2f6c73a809a50..2c501ea7f39cd 100644
--- a/src/v/redpanda/admin/api-doc/broker.json
+++ b/src/v/redpanda/admin/api-doc/broker.json
@@ -262,13 +262,15 @@
                         "application/json"
                     ],
                     "parameters": [
-                      {
-                        "in": "query",
-                        "name": "limit",
-                        "schema": {"type": "integer"},
-                        "required": false,
-                        "description": "Limit the number of partitions listed for each risk type (default is 128)"
-                      }
+                        {
+                            "in": "query",
+                            "name": "limit",
+                            "schema": {
+                                "type": "integer"
+                            },
+                            "required": false,
+                            "description": "Limit the number of partitions listed for each risk type. Default is 128."
+                        }
                     ]
                 }
             ]
@@ -414,6 +416,26 @@
                 }
             }
         },
+        "reallocation_failure_details": {
+            "id": "reallocation_failure_details",
+            "description": "Detailed information about reallocation failure",
+            "properties": {
+                "ns": {
+                    "type": "string",
+                    "description": "namespace"
+                },
+                "topic": {
+                    "type": "string"
+                },
+                "partition": {
+                    "type": "int"
+                },
+                "error": {
+                    "type": "string",
+                    "description": "Reallocation failure description. The description contains the cause of failure"
+                }
+            }
+        },
         "decommission_status": {
             "id": "decommission_status",
             "description": "Node decommissioning status",
@@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
+                    "type": "array",
+                    "items": {
+                        "type": "reallocation_failure_details"
+                    },
+                    "description": "Array containing allocation failure details. This is helpful for diagnosing node decommissioning problems."
                 }
             }
         },
diff --git a/src/v/redpanda/admin/server.cc b/src/v/redpanda/admin/server.cc
index a4c4c35cb8028..15fed3b9545e6 100644
--- a/src/v/redpanda/admin/server.cc
+++ b/src/v/redpanda/admin/server.cc
@@ -2572,10 +2572,21 @@ admin_server::get_decommission_progress_handler(
 
     ret.replicas_left = decommission_progress.replicas_left;
     ret.finished = decommission_progress.finished;
-
-    for (const auto& ntp : decommission_progress.allocation_failures) {
+    ret.reallocation_failure_details._elements.reserve(
+      decommission_progress.allocation_failures.size());
+    for (const auto& [ntp, details] :
+         decommission_progress.allocation_failures) {
         ret.allocation_failures.push(
           fmt::format("{}/{}/{}", ntp.ns(), ntp.tp.topic(), ntp.tp.partition));
+        using failure_details_t
+          = ss::httpd::broker_json::reallocation_failure_details;
+        failure_details_t f_details;
+        f_details.ns = ntp.ns;
+        f_details.topic = ntp.tp.topic;
+        f_details.partition = ntp.tp.partition;
+        f_details.error = fmt::to_string(details.error);
+
+        ret.reallocation_failure_details.push(f_details);
     }
 
     for (auto& p : decommission_progress.current_reconfigurations) {
diff --git a/src/v/test_utils/BUILD b/src/v/test_utils/BUILD
index 819031be4b8c8..3e12b8fcbfe3e 100644
--- a/src/v/test_utils/BUILD
+++ b/src/v/test_utils/BUILD
@@ -91,6 +91,7 @@ redpanda_test_cc_library(
     visibility = ["//visibility:public"],
     deps = [
         "//src/v/base",
+        "//src/v/container:chunked_hash_map",
         "//src/v/container:fragmented_vector",
         "//src/v/random:generators",
         "//src/v/utils:tristate",
diff --git a/src/v/test_utils/randoms.h b/src/v/test_utils/randoms.h
index b006b9ea984e8..4559f46202385 100644
--- a/src/v/test_utils/randoms.h
+++ b/src/v/test_utils/randoms.h
@@ -10,6 +10,7 @@
  */
 #pragma once
 
+#include "container/chunked_hash_map.h"
 #include "container/fragmented_vector.h"
 #include "random/generators.h"
 #include "utils/tristate.h"
@@ -155,6 +156,13 @@ inline MapType<Key, Value> random_map(Fn&& gen, size_t size = 20) {
     return hm;
 }
 
+template<typename Key, typename Value, typename Fn>
+inline chunked_hash_map<Key, Value>
+random_chunked_hash_map(Fn&& gen, size_t size = 20) {
+    return random_map<chunked_hash_map, Key, Value, Fn>(
+      std::forward<Fn>(gen), size);
+}
+
 template<typename Key, typename Value, typename Fn>
 inline absl::node_hash_map<Key, Value>
 random_node_hash_map(Fn&& gen, size_t size = 20) {
diff --git a/tests/rptest/tests/nodes_decommissioning_test.py b/tests/rptest/tests/nodes_decommissioning_test.py
index 336a22f9b1160..5ac1d304028ec 100644
--- a/tests/rptest/tests/nodes_decommissioning_test.py
+++ b/tests/rptest/tests/nodes_decommissioning_test.py
@@ -194,6 +194,23 @@ def _wait_for_node_removed(self, decommissioned_id):
 
         self._check_state_consistent(decommissioned_id)
 
+    def _wait_for_allocation_failures_to_be_reported(self, decommissioned_id):
+        def _failures_are_present():
+            status = self.admin.get_decommission_status(decommissioned_id)
+            failed_to_reallocate_topics = {
+                f['topic']
+                for f in status['reallocation_failure_details']
+                if f['topic'] != "__consumer_offsets"
+            }
+            return len(failed_to_reallocate_topics) > 0
+
+        return wait_until(
+            _failures_are_present,
+            timeout_sec=60,
+            backoff_sec=2,
+            retry_on_exc=True,
+            err_msg="Timeout waiting for allocation failures to be reported")
+
     def _decommission(self, node_id, node=None):
         def decommissioned():
             try:
@@ -349,6 +366,29 @@ def test_decommissioning_working_node(self, delete_topic, tick_interval):
         if not delete_topic:
             self.verify()
 
+    @cluster(num_nodes=6, log_allow_list=CHAOS_LOG_ALLOW_LIST)
+    def test_decommissioning_node_rf_1_replica(self):
+
+        self.start_redpanda()
+        self._create_topics(replication_factors=[1])
+
+        self.start_producer()
+        self.start_consumer()
+
+        to_decommission = self.redpanda.nodes[1]
+        node_id = self.redpanda.node_id(to_decommission)
+        self.redpanda.stop_node(node=to_decommission)
+        self.logger.info(f"decommissioning node: {node_id}", )
+        self._decommission(node_id)
+        self._wait_for_allocation_failures_to_be_reported(node_id)
+        # start the node back to finish decommissioning
+        self.redpanda.start_node(node=to_decommission,
+                                 auto_assign_node_id=True,
+                                 omit_seeds_on_idx_one=False)
+        self._wait_for_node_removed(node_id)
+
+        self.verify()
+
     @cluster(num_nodes=6, log_allow_list=CHAOS_LOG_ALLOW_LIST)
     def test_decommissioning_crashed_node(self):
 


Reviews:
user: login: micheleRP
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-07T16:38:41Z
user: login: micheleRP
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-07T16:39:06Z
user: login: micheleRP
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-07T16:39:40Z
user: login: bharathv
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-08T18:55:38Z
user: login: daisukebe
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T01:13:55Z
user: login: daisukebe
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-09T05:42:36Z
user: login: mmaslankaprv
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-09T09:28:58Z
user: login: mmaslankaprv
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-09T09:29:30Z
user: login: mmaslankaprv
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-09T09:29:37Z
user: login: micheleRP
state: COMMENTED
author_association: NONE
submitted_at: 2025-05-09T13:46:08Z
user: login: daisukebe
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-12T00:45:04Z
user: login: micheleRP
body: thank you!
state: APPROVED
author_association: NONE
submitted_at: 2025-05-12T13:45:15Z
user: login: bharathv
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-05-14T14:49:37Z
user: login: dotnwat
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-05-14T19:12:24Z

Review comments:
diff_hunk: @@ -262,13 +262,15 @@
                         "application/json"
                     ],
                     "parameters": [
-                      {
-                        "in": "query",
-                        "name": "limit",
-                        "schema": {"type": "integer"},
-                        "required": false,
-                        "description": "Limit the number of partitions listed for each risk type (default is 128)"
-                      }
+                        {
+                            "in": "query",
+                            "name": "limit",
+                            "schema": {
+                                "type": "integer"
+                            },
+                            "required": false,
+                            "description": "Limit the number of partitions listed for each risk type (default is 128)"
path: src/v/redpanda/admin/api-doc/broker.json
user: login: micheleRP
body: ```suggestion
                            "description": "Limit the number of partitions listed for each risk type. Default is 128."
```
created_at: 2025-05-07T16:38:40Z
author_association: NONE
diff_hunk: @@ -414,6 +416,26 @@
                 }
             }
         },
+        "reallocation_failure_details": {
+            "id": "reallocation_failure_details",
+            "description": "Detailed information about reallocation failure",
+            "properties": {
+                "ns": {
+                    "type": "string",
+                    "description": "namespace"
+                },
+                "topic": {
+                    "type": "string"
+                },
+                "partition": {
+                    "type": "int"
+                },
+                "error": {
+                    "type": "string",
+                    "description": "Reallocation failure description. The description contain a cause of failure"
path: src/v/redpanda/admin/api-doc/broker.json
user: login: micheleRP
body: ```suggestion
                    "description": "Reallocation failure description. The description contains the cause of failure."
```
created_at: 2025-05-07T16:39:05Z
author_association: NONE
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
+                    "type": "array",
+                    "items": {
+                        "type": "reallocation_failure_details"
+                    },
+                    "description": "Array containing allocation failure details, this is helpful to diagnose node decommissioning problems."
path: src/v/redpanda/admin/api-doc/broker.json
user: login: micheleRP
body: ```suggestion
                    "description": "Array containing allocation failure details. This is helpful for diagnosing node decommissioning problems."
```
created_at: 2025-05-07T16:39:40Z
author_association: NONE
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
+                    "type": "array",
+                    "items": {
+                        "type": "reallocation_failure_details"
+                    },
+                    "description": "Array containing allocation failure details, this is helpful to diagnose node decommissioning problems."
                 }
path: src/v/redpanda/admin/api-doc/broker.json
user: login: bharathv
body: I think this requires an rpk change that reports "ALLOCATION FAILURES" section: cc: @daisukebe 
created_at: 2025-05-08T18:46:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -439,15 +439,38 @@ controller_api::get_decommission_allocation_failures(model::node_id node) {
     } else {
         co_return std::get<cluster::errc>(result);
     }
-
-    ss::chunked_fifo<model::ntp> ret;
-    auto it = overview.decommission_realloc_failures.find(node);
-    if (it == overview.decommission_realloc_failures.end()) {
+    /**
+     * If allocation failures are empty there are two possibilities, either a
+     * previous version of the reply was received or there are no failures,
+     * either way we are going to use decommission_realloc_failures map to
+     * determine the failures.
+     */
+    chunked_hash_map<model::ntp, reallocation_failure_details> ret;
+    if (overview.reallocation_failures.empty()) {
+        auto it = overview.decommission_realloc_failures.find(node);
+        if (it == overview.decommission_realloc_failures.end()) {
+            co_return ret;
+        }
+        ret.reserve(it->second.size());
+        for (const auto& ntp : it->second) {
+            ret.emplace(
+              ntp,
+              reallocation_failure_details{
+                .replica_to_move = node,
+                .reason = change_reason::node_decommissioning,
+                .error = reallocation_error::unknown_error,
+              });
+        }
         co_return ret;
     }
-    for (const auto& ntp : it->second) {
-        ret.push_back(ntp);
+    for (auto& [ntp, details] : overview.reallocation_failures) {
+        if (
+          details.replica_to_move == node
+          && details.reason == change_reason::node_decommissioning) {
path: src/v/cluster/controller_api.cc
user: login: bharathv
body: I was wondering if other failures (non decommissioning) should be reported in the balancer status end point, can be a follow up but that'd be useful too I think. 
created_at: 2025-05-08T18:55:20Z
author_association: CONTRIBUTOR
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
+                    "type": "array",
+                    "items": {
+                        "type": "reallocation_failure_details"
+                    },
+                    "description": "Array containing allocation failure details, this is helpful to diagnose node decommissioning problems."
                 }
path: src/v/redpanda/admin/api-doc/broker.json
user: login: daisukebe
body: Thanks for the heads up and for the improvement in this area. Curious: can this be backported to released versions like 25.1.x, 24.3.x?
created_at: 2025-05-09T01:13:55Z
author_association: CONTRIBUTOR
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
path: src/v/redpanda/admin/api-doc/broker.json
user: login: daisukebe
body: Given we have this new field, I have an impression we no longer need the `allocation_failures` field. As long as a partition is listed in `reallocation_failure_details`, it will be listed in `allocation_failures` as well, right?
created_at: 2025-05-09T05:42:36Z
author_association: CONTRIBUTOR
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
path: src/v/redpanda/admin/api-doc/broker.json
user: login: mmaslankaprv
body: Yes, exactly. I left it for the backward compatibility.
created_at: 2025-05-09T09:28:57Z
author_association: MEMBER
diff_hunk: @@ -439,15 +439,38 @@ controller_api::get_decommission_allocation_failures(model::node_id node) {
     } else {
         co_return std::get<cluster::errc>(result);
     }
-
-    ss::chunked_fifo<model::ntp> ret;
-    auto it = overview.decommission_realloc_failures.find(node);
-    if (it == overview.decommission_realloc_failures.end()) {
+    /**
+     * If allocation failures are empty there are two possibilities, either a
+     * previous version of the reply was received or there are no failures,
+     * either way we are going to use decommission_realloc_failures map to
+     * determine the failures.
+     */
+    chunked_hash_map<model::ntp, reallocation_failure_details> ret;
+    if (overview.reallocation_failures.empty()) {
+        auto it = overview.decommission_realloc_failures.find(node);
+        if (it == overview.decommission_realloc_failures.end()) {
+            co_return ret;
+        }
+        ret.reserve(it->second.size());
+        for (const auto& ntp : it->second) {
+            ret.emplace(
+              ntp,
+              reallocation_failure_details{
+                .replica_to_move = node,
+                .reason = change_reason::node_decommissioning,
+                .error = reallocation_error::unknown_error,
+              });
+        }
         co_return ret;
     }
-    for (const auto& ntp : it->second) {
-        ret.push_back(ntp);
+    for (auto& [ntp, details] : overview.reallocation_failures) {
+        if (
+          details.replica_to_move == node
+          && details.reason == change_reason::node_decommissioning) {
path: src/v/cluster/controller_api.cc
user: login: mmaslankaprv
body: exactly, that was my plan, will do it in follow up
created_at: 2025-05-09T09:29:30Z
author_association: MEMBER
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
path: src/v/redpanda/admin/api-doc/broker.json
user: login: mmaslankaprv
body: Yes, exactly. I left it for the backward compatibility.
created_at: 2025-05-09T09:29:37Z
author_association: MEMBER
diff_hunk: @@ -414,6 +416,26 @@
                 }
             }
         },
+        "reallocation_failure_details": {
+            "id": "reallocation_failure_details",
+            "description": "Detailed information about reallocation failure",
+            "properties": {
+                "ns": {
+                    "type": "string",
+                    "description": "namespace"
+                },
+                "topic": {
+                    "type": "string"
+                },
+                "partition": {
+                    "type": "int"
+                },
+                "error": {
+                    "type": "string",
+                    "description": "Reallocation failure description. The description contain the cause of failure"
path: src/v/redpanda/admin/api-doc/broker.json
user: login: micheleRP
body: ```suggestion
                    "description": "Reallocation failure description. The description contains the cause of failure."
```
created_at: 2025-05-09T13:46:08Z
author_association: NONE
diff_hunk: @@ -440,6 +462,13 @@
                         "type": "partition_reconfiguration_status"
                     },
                     "description": "Array of partition reconfiguration statues"
+                },
+                "reallocation_failure_details": {
path: src/v/redpanda/admin/api-doc/broker.json
user: login: daisukebe
body: That's fair. rpk-wise, we'll do this way, which must be backward compatible. 

```
if reallocation_failure_details is not null, list partitions with failure details
else if allocation_failures is not null, list partitions
else there's no failed partitions
```
created_at: 2025-05-12T00:45:03Z
author_association: CONTRIBUTOR
diff_hunk: @@ -66,6 +66,33 @@ distinct_from(const absl::flat_hash_set<model::node_id>& nodes) {
     return hard_constraint(std::make_unique<impl>(nodes));
 }
 
+reallocation_failure_details map_result_to_failure_details(
+  change_reason reason, std::error_code ec, model::node_id replica_to_move) {
+    reallocation_failure_details details{
+      .replica_to_move = replica_to_move,
+      .reason = reason,
+      .error = details.error = reallocation_error::unknown_error};
path: src/v/cluster/partition_balancer_planner.cc
user: login: dotnwat
body: @mmaslankaprv what's going on with this line? seems like a typo that happened to compile?
created_at: 2025-05-14T19:08:10Z
author_association: MEMBER
diff_hunk: @@ -824,6 +847,54 @@ class partition_balancer_planner::immutable_partition {
           change_reason);
     }
 
+    bool contains_replica(model::node_id replica) const {
+        return contains_node(_replicas, replica);
+    }
+
+    reallocation_failure_details map_immutability_reason_to_failure_details(
+      change_reason reason, model::node_id replica_to_move) {
+        reallocation_failure_details details{
+          .replica_to_move = replica_to_move,
+          .reason = reason,
+          .error = details.error = reallocation_error::unknown_error};
path: src/v/cluster/partition_balancer_planner.cc
user: login: dotnwat
body: same here. is it setting the same field twice?
created_at: 2025-05-14T19:09:08Z
author_association: MEMBER
diff_hunk: @@ -155,6 +156,13 @@ inline MapType<Key, Value> random_map(Fn&& gen, size_t size = 20) {
     return hm;
 }
 
+template<typename Key, typename Value, typename Fn>
+inline chunked_hash_map<Key, Value>
path: src/v/test_utils/randoms.h
user: login: dotnwat
body: template implies inline
created_at: 2025-05-14T19:12:15Z
author_association: MEMBER

PR comments:
user: login: mmaslankaprv
created_at: 2025-05-06T14:33:08Z
body: /ci-repeat 1
author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-06T17:01:07Z
body: /ci-repeat 1

author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-06T17:15:52Z
body: 
/ci-repeat 1


author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-06T19:29:45Z
body: /ci-repeat 1
author_association: MEMBER
user: login: vbotbuildovich
created_at: 2025-05-07T01:24:55Z
body: <!--TEST RESULTS-->
<!--adac38f3-efb2-49da-93f3-e0f1ce9d81bb-->
<!--TEST RESULTS-->
<!--fe446caa-c0e5-4816-ad7c-2769da326ab7-->
<!--TEST RESULTS-->
<!--e53945da-647d-4dd2-bfe8-0187098f65ea-->
<!--TEST RESULTS-->
<!--d8e9b912-8437-4e9e-be59-b26e1f6d066f-->
<!--TEST RESULTS-->
<!--1d495f47-4f65-4e7a-8cd8-00fb6b26d3c9-->
<!--title-->
## CI test results
<!--/title-->
<!--body-->
<details>
<summary>test results on <a href="https://buildkite.com/redpanda/redpanda/builds/65585">build#65585</a></summary>


| test_id                                                                                                                                                                                                                                           | test_kind   | job_url                                                                                   | test_status   | passed   |
|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------|:------------------------------------------------------------------------------------------|:--------------|:---------|
| rptest.tests.datalake.rest_catalog_connection_test.RestCatalogConnectionTest.test_redpanda_connection_to_rest_catalog.cloud_storage_type=CloudStorageType.S3                                                                                      | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65585#0196a778-c253-4b55-9603-98a70451e7d9 | FLAKY         | 20/21    |
| rptest.tests.random_node_operations_test.RandomNodeOperationsTest.test_node_operations.enable_failures=True.mixed_versions=False.with_tiered_storage=True.with_iceberg=False.with_chunked_compaction=False.cloud_storage_type=CloudStorageType.S3 | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65585#0196a778-c252-419f-aefd-0137a9728040 | FLAKY         | 18/19    |
| rptest.tests.scaling_up_test.ScalingUpTest.test_scaling_up_with_recovered_topic                                                                                                                                                                   | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65585#0196a778-c252-419f-aefd-0137a9728040 | FLAKY         | 7/21     |
| rptest.tests.scram_test.SaslPlainTest.test_plain_authn.client_type=ClientType.KCLI.scram_type=SCRAM-SHA-256.sasl_plain_enabled=True                                                                                                               | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65585#0196a778-c252-419f-aefd-0137a9728040 | FLAKY         | 20/21    |
</details>
<details>
<summary>test results on <a href="https://buildkite.com/redpanda/redpanda/builds/65645">build#65645</a></summary>


| test_id                                                                                              | test_kind   | job_url                                                                                   | test_status   | passed   |
|:-----------------------------------------------------------------------------------------------------|:------------|:------------------------------------------------------------------------------------------|:--------------|:---------|
| rptest.tests.upgrade_test.UpgradeBackToBackTest.test_upgrade_with_all_workloads.single_upgrade=False | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65645#0196abdf-cdd6-43b6-a91e-a736ab6c4ed1 | FLAKY         | 15/21    |
</details>
<details>
<summary>test results on <a href="https://buildkite.com/redpanda/redpanda/builds/65733">build#65733</a></summary>


| test_class                       | test_method              | test_arguments   | test_kind   | job_url                                                                                   | test_status   | passed   | reason                                                                                                                                                                 |
|:---------------------------------|:-------------------------|:-----------------|:------------|:------------------------------------------------------------------------------------------|:--------------|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| AutomaticLeadershipBalancingTest | test_automatic_rebalance |                  | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65733#0196b4ba-de7c-4385-97d8-05926a697ca3 | FLAKY         | 20/21    | upstream reliability is '99.55357142857143'. current run reliability is '95.23809523809523'. drift is 4.31548 and the allowed drift is set to 50. The test should PASS |
</details>
<details>
<summary>test results on <a href="https://buildkite.com/redpanda/redpanda/builds/65808">build#65808</a></summary>


| test_class                   | test_method                                       | test_arguments                                                                                                                                                           | test_kind   | job_url                                                                                   | test_status   | passed   | reason                                                                                                                                                                 |
|:-----------------------------|:--------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------|:------------------------------------------------------------------------------------------|:--------------|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| DataMigrationsApiTest        | test_higher_level_migration_api                   |                                                                                                                                                                          | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65808#0196c37e-7b20-4791-ad1a-9b8bd8f46617 | FLAKY         | 20/21    | upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 50. The test should PASS              |
| EndToEndHydrationTimeoutTest | test_hydration_completes_when_consumer_killed     |                                                                                                                                                                          | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65808#0196c37e-7b1f-4af8-b2c7-108a34ba6aa5 | FLAKY         | 20/21    | upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 50. The test should PASS              |
| PartitionReassignmentsTest   | test_add_partitions_with_inprogress_reassignments |                                                                                                                                                                          | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65808#0196c37e-7b1e-4432-92c8-710e91c0819e | FLAKY         | 17/21    | upstream reliability is '89.88326848249028'. current run reliability is '80.95238095238095'. drift is 8.93089 and the allowed drift is set to 50. The test should PASS |
| RandomNodeOperationsTest     | test_node_operations                              | {"cloud_storage_type": 1, "enable_failures": false, "mixed_versions": false, "with_chunked_compaction": true, "with_iceberg": true, "with_tiered_storage": false}        | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65808#0196c381-a6af-4a12-be76-716210814d58 | FLAKY         | 20/21    | upstream reliability is '99.55156950672645'. current run reliability is '95.23809523809523'. drift is 4.31347 and the allowed drift is set to 50. The test should PASS |
| TieredStorageTest            | test_tiered_storage                               | {"cloud_storage_type_and_url_style": [1, "virtual_host"], "test_case": {"name": "(TS_Read == True, TS_TxRangeMaterialized == True, SpilloverManifestUploaded == True)"}} | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65808#0196c37e-7b20-4791-ad1a-9b8bd8f46617 | FAIL          | 0/1      | The test has failed across all retries                                                                                                                                 |
</details>
<details>
<summary>test results on <a href="https://buildkite.com/redpanda/redpanda/builds/65885">build#65885</a></summary>


| test_class                | test_method                              | test_arguments                                                                                                                                                    | test_kind   | job_url                                                                                   | test_status   | passed   | reason                                                                                                                                                                 |
|:--------------------------|:-----------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------|:------------------------------------------------------------------------------------------|:--------------|:---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| AvailabilityTests         | test_recovery_after_catastrophic_failure |                                                                                                                                                                   | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65885#0196c8b7-8729-4fc9-af59-4154aa992f6e | FLAKY         | 20/21    | upstream reliability is '99.10714285714286'. current run reliability is '95.23809523809523'. drift is 3.86905 and the allowed drift is set to 50. The test should PASS |
| RestCatalogConnectionTest | test_redpanda_connection_to_rest_catalog | {"cloud_storage_type": 1}                                                                                                                                         | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65885#0196c8b7-872b-44f9-b2cf-14a8cda39fb5 | FLAKY         | 20/21    | upstream reliability is '97.15302491103202'. current run reliability is '95.23809523809523'. drift is 1.91493 and the allowed drift is set to 50. The test should PASS |
| PartitionBalancerTest     | test_unavailable_nodes                   |                                                                                                                                                                   | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65885#0196c8ab-a688-4184-b939-dcca9476776b | FLAKY         | 14/21    | upstream reliability is '71.24183006535948'. current run reliability is '66.66666666666666'. drift is 4.57516 and the allowed drift is set to 50. The test should PASS |
| RandomNodeOperationsTest  | test_node_operations                     | {"cloud_storage_type": 2, "enable_failures": false, "mixed_versions": false, "with_chunked_compaction": true, "with_iceberg": false, "with_tiered_storage": true} | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65885#0196c8ab-a686-44f4-b939-72d3265a0e8d | FLAKY         | 20/21    | upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 50. The test should PASS              |
| RandomNodeOperationsTest  | test_node_operations                     | {"cloud_storage_type": 2, "enable_failures": false, "mixed_versions": true, "with_chunked_compaction": true, "with_iceberg": false, "with_tiered_storage": false} | ducktape    | https://buildkite.com/redpanda/redpanda/builds/65885#0196c8ab-a686-44f4-b939-72d3265a0e8d | FLAKY         | 20/21    | upstream reliability is '100.0'. current run reliability is '95.23809523809523'. drift is 4.7619 and the allowed drift is set to 50. The test should PASS              |
</details>
<!--/body-->
author_association: COLLABORATOR
user: login: mmaslankaprv
created_at: 2025-05-07T08:25:47Z
body: /ci-repeat 1
author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-07T10:47:39Z
body: /ci-repeat 1
author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-07T14:23:14Z
body: /ci-repeat 1
author_association: MEMBER
user: login: vbotbuildovich
created_at: 2025-05-12T09:47:44Z
body: <!--FAILURES SLASH COMMAND https://buildkite.com/redpanda/redpanda/builds/65808-->
<!--c1652060-0e79-49df-9622-0e2efdf6cea8-->
<!--title-->
## Retry command for [Build#65808](https://buildkite.com/redpanda/redpanda/builds/65808)
please wait until all jobs are finished before running the slash command
<!--/title-->
<!--body-->
```
/ci-repeat 1
tests/rptest/tests/tiered_storage_model_test.py::TieredStorageTest.test_tiered_storage@{"cloud_storage_type_and_url_style":[1,"virtual_host"],"test_case":{"name":"(TS_Read == True, TS_TxRangeMaterialized == True, SpilloverManifestUploaded == True)"}}
```
<!--/body-->
author_association: COLLABORATOR
user: login: mmaslankaprv
created_at: 2025-05-14T19:20:55Z
body: /backport v25.1.x
author_association: MEMBER
user: login: mmaslankaprv
created_at: 2025-05-14T19:21:01Z
body: /backport v24.3.x
author_association: MEMBER
user: login: vbotbuildovich
created_at: 2025-05-14T19:22:17Z
body: Failed to create a backport PR to v24.3.x branch. I tried:

```
git remote add upstream https://github.com/redpanda-data/redpanda.git
git fetch --all
git checkout -b backport-pr-26054-v24.3.x-42 remotes/upstream/v24.3.x
git cherry-pick -x 3b20ef27f7 f1f97ccd05 936098d525 48834cf3ed ba820b0e8b 17ff1d96c7 8f4b6a4ecd
```

[Workflow run logs](https://github.com/redpanda-data/redpanda/actions/runs/15029242108).
author_association: COLLABORATOR