Repository: ginkgo-project/ginkgo
Pull request #1431: Add a two-level Schwarz preconditioner
Author: login: pratikvn
Labels: name: reg:testing
description: This is related to testing.
name: mod:core
description: This is related to the core module.
name: reg:example
description: This is related to the examples.
name: type:preconditioner
description: This is related to the preconditioners
name: 1:ST:ready-to-merge
description: This PR is ready to merge.
name: type:distributed-functionality
Created: 2023-10-15T13:25:16Z | Merged: 2025-04-11T07:38:02Z

This PR uses the distributed coarse level generation from PGM to use as a coarse level for the additive Schwarz preconditioner. The only requirement is that the galerkin product generator ($RAP$) generate a `multigrid::MultigridLevel` which is a triplet of `restrict`, `prolong` and `coarse` operators. The user additionally also needs to set the solver for the coarse level.

### TODO

~~+ [ ] Add options to switch between multiplicative and additive~~
~~+ [ ] Options for arbitrary number of levels.~~

### Possible issues
1. The coarse level solve significantly reduces the number of iterations, but can be extremely expensive. Maybe makes sense to apply coarse solve, only a few times instead of every preconditioner apply (need to be careful of consistency of preconditioner apply) ? 
2. Weighting between the local solver solution and the coarse solution is unclear. 
3. Though the coarse solve and the local solve are independent, we have no way of performing them in parallel. This will probably require a rewrite of the apply interface to enable asynchronicity, in addition to support for multiple streams/queues.


Diff:
diff --git a/core/distributed/preconditioner/schwarz.cpp b/core/distributed/preconditioner/schwarz.cpp
index 1ba3a09861b..da357784a67 100644
--- a/core/distributed/preconditioner/schwarz.cpp
+++ b/core/distributed/preconditioner/schwarz.cpp
@@ -18,6 +18,7 @@
 #include <ginkgo/core/matrix/csr.hpp>
 #include <ginkgo/core/matrix/dense.hpp>
 #include <ginkgo/core/matrix/identity.hpp>
+#include <ginkgo/core/multigrid/multigrid_level.hpp>
 
 #include "core/base/utils.hpp"
 #include "core/config/config_helper.hpp"
@@ -59,6 +60,20 @@ Schwarz<ValueType, LocalIndexType, GlobalIndexType>::parse(
     if (auto& obj = config.get("l1_smoother")) {
         params.with_l1_smoother(obj.get_boolean());
     }
+    if (auto& obj = config.get("coarse_level")) {
+        params.with_coarse_level(
+            gko::config::parse_or_get_factory<const LinOpFactory>(
+                obj, context, td_for_child));
+    }
+    if (auto& obj = config.get("coarse_solver")) {
+        params.with_coarse_solver(
+            gko::config::parse_or_get_factory<const LinOpFactory>(
+                obj, context, td_for_child));
+    }
+    if (auto& obj = config.get("coarse_weight")) {
+        params.with_coarse_weight(gko::config::get_value<ValueType>(obj));
+    }
+
     return params;
 }
 
@@ -87,8 +102,48 @@ template <typename VectorType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
+        if (this->local_solver_) {
+            this->local_solver_->apply(gko::detail::get_local(dense_b),
+                                       gko::detail::get_local(dense_x));
+        }
+        auto coarse_level =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_);
+        auto restrict_op = coarse_level->get_restrict_op();
+        auto prolong_op = coarse_level->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                coarse_level->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call if the number of rhs is
+        // unchanged.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse_op->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse_op->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse_op->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+        crhs_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        // Additive apply of coarse correction
+        restrict_op->apply(dense_b, crhs_cache_.get());
+        // TODO: Does it make sense to restrict dense_x (to csol_cache) to
+        // provide a good initial guess for the coarse solver ?
+        if (this->coarse_solver_->apply_uses_initial_guess()) {
+            csol_cache_->copy_from(crhs_cache_.get());
+        }
+        this->coarse_solver_->apply(crhs_cache_.get(), csol_cache_.get());
+        prolong_op->apply(this->coarse_weight_, csol_cache_.get(),
+                          this->local_weight_, dense_x);
+    } else if (this->local_solver_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
     }
@@ -130,24 +185,29 @@ template <typename ValueType, typename LocalIndexType, typename GlobalIndexType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
     std::shared_ptr<const LinOp> system_matrix)
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     if (parameters_.local_solver && parameters_.generated_local_solver) {
         GKO_INVALID_STATE(
             "Provided both a generated solver and a solver factory");
     }
-
     if (!parameters_.local_solver && !parameters_.generated_local_solver) {
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
-
     if (parameters_.generated_local_solver) {
         this->set_solver(parameters_.generated_local_solver);
         return;
     }
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
 
-    auto local_matrix =
-        as<Matrix<ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-            ->get_local_matrix();
+    auto dist_mat =
+        as<Matrix<ValueType, LocalIndexType, GlobalIndexType>>(system_matrix);
+    auto local_matrix = dist_mat->get_local_matrix();
 
     if (parameters_.l1_smoother) {
         auto exec = this->get_executor();
@@ -183,6 +243,40 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight >= 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>() -
+             static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (this->coarse_level_ == nullptr) {
+            GKO_NOT_SUPPORTED(this->coarse_level_);
+        }
+        if (auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op()) {
+            this->coarse_solver_ = share(parameters_.coarse_solver->generate(
+                as<Matrix<ValueType, LocalIndexType, GlobalIndexType>>(
+                    coarse)));
+            if (this->coarse_solver_ == nullptr) {
+                GKO_NOT_SUPPORTED(this->coarse_solver_);
+            }
+        }
+    }
 }
 
 
diff --git a/core/test/config/preconditioner.cpp b/core/test/config/preconditioner.cpp
index 18e771296e9..447ec5aefc9 100644
--- a/core/test/config/preconditioner.cpp
+++ b/core/test/config/preconditioner.cpp
@@ -11,6 +11,7 @@
 #include <ginkgo/core/config/config.hpp>
 #include <ginkgo/core/distributed/preconditioner/schwarz.hpp>
 #include <ginkgo/core/matrix/dense.hpp>
+#include <ginkgo/core/multigrid/pgm.hpp>
 #include <ginkgo/core/preconditioner/gauss_seidel.hpp>
 #include <ginkgo/core/preconditioner/ic.hpp>
 #include <ginkgo/core/preconditioner/ilu.hpp>
@@ -30,6 +31,8 @@ using namespace gko::config;
 
 using DummyIr = gko::solver::Ir<float>;
 
+using DummyMgLevel = gko::multigrid::Pgm<float, int>;
+
 
 template <typename ChangedType, typename DefaultType>
 struct PreconditionerConfigTest {
@@ -402,7 +405,6 @@ struct GaussSeidel
 
 #if GINKGO_BUILD_MPI
 
-
 struct Schwarz
     : PreconditionerConfigTest<
           ::gko::experimental::distributed::preconditioner::Schwarz<float, int,
@@ -428,17 +430,34 @@ struct Schwarz
             param.with_local_solver(
                 detail::registry_accessor::get_data<gko::LinOpFactory>(
                     reg, "solver"));
+            config_map["coarse_solver"] = pnode{"solver"};
+            param.with_coarse_solver(
+                detail::registry_accessor::get_data<gko::LinOpFactory>(
+                    reg, "solver"));
+            config_map["coarse_level"] = pnode{"c_level"};
+            param.with_coarse_level(
+                detail::registry_accessor::get_data<gko::LinOpFactory>(
+                    reg, "c_level"));
         } else {
             config_map["local_solver"] =
                 pnode{{{"type", pnode{"solver::Ir"}},
                        {"value_type", pnode{"float32"}}}};
             param.with_local_solver(DummyIr::build().on(exec));
+            config_map["coarse_solver"] =
+                pnode{{{"type", pnode{"solver::Ir"}},
+                       {"value_type", pnode{"float32"}}}};
+            param.with_coarse_solver(DummyIr::build().on(exec));
+            config_map["coarse_level"] =
+                pnode{{{"type", pnode{"multigrid::Pgm"}}}};
+            param.with_coarse_level(DummyMgLevel::build().on(exec));
         }
         config_map["generated_local_solver"] = pnode{"linop"};
         param.with_generated_local_solver(
             detail::registry_accessor::get_data<gko::LinOp>(reg, "linop"));
         config_map["l1_smoother"] = pnode{true};
         param.with_l1_smoother(true);
+        config_map["coarse_weight"] = pnode{0.1};
+        param.with_coarse_weight(decltype(param.coarse_weight){0.1});
     }
 
     template <bool from_reg, typename AnswerType>
@@ -449,14 +468,25 @@ struct Schwarz
 
         if (from_reg) {
             ASSERT_EQ(res_param.local_solver, ans_param.local_solver);
+            ASSERT_EQ(res_param.coarse_solver, ans_param.coarse_solver);
+            ASSERT_EQ(res_param.coarse_level, ans_param.coarse_level);
         } else {
             ASSERT_NE(
                 std::dynamic_pointer_cast<const typename DummyIr::Factory>(
                     res_param.local_solver),
                 nullptr);
+            ASSERT_NE(
+                std::dynamic_pointer_cast<const typename DummyIr::Factory>(
+                    res_param.coarse_solver),
+                nullptr);
+            ASSERT_NE(
+                std::dynamic_pointer_cast<const typename DummyMgLevel::Factory>(
+                    res_param.coarse_level),
+                nullptr);
         }
         ASSERT_EQ(res_param.generated_local_solver,
                   ans_param.generated_local_solver);
+        ASSERT_EQ(res_param.coarse_weight, ans_param.coarse_weight);
     }
 };
 
@@ -477,9 +507,11 @@ class Preconditioner : public ::testing::Test {
           u_solver(DummyIr::build().on(exec)),
           factorization(DummyIr::build().on(exec)),
           linop(gko::matrix::Dense<>::create(exec)),
+          coarse_level(DummyMgLevel::build().on(exec)),
           reg()
     {
         reg.emplace("solver", solver_factory);
+        reg.emplace("c_level", coarse_level);
         reg.emplace("l_solver", l_solver);
         reg.emplace("u_solver", u_solver);
         reg.emplace("factorization", factorization);
@@ -492,6 +524,7 @@ class Preconditioner : public ::testing::Test {
     std::shared_ptr<typename DummyIr::Factory> l_solver;
     std::shared_ptr<typename DummyIr::Factory> u_solver;
     std::shared_ptr<typename DummyIr::Factory> factorization;
+    std::shared_ptr<typename DummyMgLevel::Factory> coarse_level;
     std::shared_ptr<gko::LinOp> linop;
     registry reg;
 };
diff --git a/core/test/mpi/distributed/preconditioner/schwarz.cpp b/core/test/mpi/distributed/preconditioner/schwarz.cpp
index 1cf3f04d311..f06ec811576 100644
--- a/core/test/mpi/distributed/preconditioner/schwarz.cpp
+++ b/core/test/mpi/distributed/preconditioner/schwarz.cpp
@@ -7,6 +7,7 @@
 #include <ginkgo/config.hpp>
 #include <ginkgo/core/distributed/preconditioner/schwarz.hpp>
 #include <ginkgo/core/matrix/csr.hpp>
+#include <ginkgo/core/multigrid/pgm.hpp>
 #include <ginkgo/core/preconditioner/jacobi.hpp>
 #include <ginkgo/core/solver/cg.hpp>
 #include <ginkgo/core/stop/iteration.hpp>
@@ -14,9 +15,6 @@
 #include "core/test/utils.hpp"
 
 
-namespace {
-
-
 template <typename ValueLocalGlobalIndexType>
 class SchwarzFactory : public ::testing::Test {
 protected:
@@ -29,6 +27,8 @@ class SchwarzFactory : public ::testing::Test {
     using Schwarz = gko::experimental::distributed::preconditioner::Schwarz<
         value_type, local_index_type, global_index_type>;
     using Jacobi = gko::preconditioner::Jacobi<value_type, local_index_type>;
+    using Pgm = gko::multigrid::Pgm<value_type, local_index_type>;
+    using Cg = gko::solver::Cg<value_type>;
     using Mtx =
         gko::experimental::distributed::Matrix<value_type, local_index_type,
                                                global_index_type>;
@@ -36,10 +36,15 @@ class SchwarzFactory : public ::testing::Test {
     SchwarzFactory()
         : exec(gko::ReferenceExecutor::create()),
           jacobi_factory(Jacobi::build().on(exec)),
+          pgm_factory(Pgm::build().on(exec)),
+          cg_factory(Cg::build().on(exec)),
           mtx(Mtx::create(exec, MPI_COMM_WORLD))
     {
         schwarz = Schwarz::build()
                       .with_local_solver(jacobi_factory)
+                      .with_coarse_level(pgm_factory)
+                      .with_coarse_solver(cg_factory)
+                      .with_coarse_weight(value_type{0.4})
                       .on(exec)
                       ->generate(mtx);
     }
@@ -57,11 +62,19 @@ class SchwarzFactory : public ::testing::Test {
         ASSERT_EQ(a->get_size(), b->get_size());
         ASSERT_EQ(a->get_parameters().local_solver,
                   b->get_parameters().local_solver);
+        ASSERT_EQ(a->get_parameters().coarse_level,
+                  b->get_parameters().coarse_level);
+        ASSERT_EQ(a->get_parameters().coarse_solver,
+                  b->get_parameters().coarse_solver);
+        ASSERT_EQ(a->get_parameters().coarse_weight,
+                  b->get_parameters().coarse_weight);
     }
 
     std::shared_ptr<const gko::Executor> exec;
     std::unique_ptr<Schwarz> schwarz;
     std::shared_ptr<typename Jacobi::Factory> jacobi_factory;
+    std::shared_ptr<typename Pgm::Factory> pgm_factory;
+    std::shared_ptr<typename Cg::Factory> cg_factory;
     std::shared_ptr<Mtx> mtx;
 };
 
@@ -82,6 +95,26 @@ TYPED_TEST(SchwarzFactory, CanSetLocalFactory)
 }
 
 
+TYPED_TEST(SchwarzFactory, CanSetCoarseLevelFactory)
+{
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_level, this->pgm_factory);
+}
+
+
+TYPED_TEST(SchwarzFactory, CanSetCoarseSolverFactory)
+{
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_solver, this->cg_factory);
+}
+
+
+TYPED_TEST(SchwarzFactory, CanSetCoarseWeight)
+{
+    using value_type = typename TestFixture::value_type;
+
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_weight, value_type{0.4});
+}
+
+
 TYPED_TEST(SchwarzFactory, CanBeCloned)
 {
     auto schwarz_clone = clone(this->schwarz);
@@ -92,11 +125,20 @@ TYPED_TEST(SchwarzFactory, CanBeCloned)
 
 TYPED_TEST(SchwarzFactory, CanBeCopied)
 {
+    using value_type = typename TestFixture::value_type;
     using Jacobi = typename TestFixture::Jacobi;
+    using Pgm = typename TestFixture::Pgm;
+    using Cg = typename TestFixture::Cg;
     using Schwarz = typename TestFixture::Schwarz;
     using Mtx = typename TestFixture::Mtx;
+    auto bj = gko::share(Jacobi::build().on(this->exec));
+    auto pgm = gko::share(Pgm::build().on(this->exec));
+    auto cg = gko::share(Cg::build().on(this->exec));
     auto copy = Schwarz::build()
-                    .with_local_solver(Jacobi::build())
+                    .with_local_solver(bj)
+                    .with_coarse_level(pgm)
+                    .with_coarse_solver(cg)
+                    .with_coarse_weight(value_type{0.4})
                     .on(this->exec)
                     ->generate(Mtx::create(this->exec, MPI_COMM_WORLD));
 
@@ -108,12 +150,21 @@ TYPED_TEST(SchwarzFactory, CanBeCopied)
 
 TYPED_TEST(SchwarzFactory, CanBeMoved)
 {
+    using value_type = typename TestFixture::value_type;
     using Jacobi = typename TestFixture::Jacobi;
+    using Pgm = typename TestFixture::Pgm;
+    using Cg = typename TestFixture::Cg;
     using Schwarz = typename TestFixture::Schwarz;
     using Mtx = typename TestFixture::Mtx;
     auto tmp = clone(this->schwarz);
+    auto bj = gko::share(Jacobi::build().on(this->exec));
+    auto pgm = gko::share(Pgm::build().on(this->exec));
+    auto cg = gko::share(Cg::build().on(this->exec));
     auto copy = Schwarz::build()
-                    .with_local_solver(Jacobi::build())
+                    .with_local_solver(bj)
+                    .with_coarse_level(pgm)
+                    .with_coarse_solver(cg)
+                    .with_coarse_weight(value_type{0.4})
                     .on(this->exec)
                     ->generate(Mtx::create(this->exec, MPI_COMM_WORLD));
 
@@ -125,10 +176,14 @@ TYPED_TEST(SchwarzFactory, CanBeMoved)
 
 TYPED_TEST(SchwarzFactory, CanBeCleared)
 {
+    using value_type = typename TestFixture::value_type;
     this->schwarz->clear();
 
     ASSERT_EQ(this->schwarz->get_size(), gko::dim<2>(0, 0));
     ASSERT_EQ(this->schwarz->get_parameters().local_solver, nullptr);
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_level, nullptr);
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_solver, nullptr);
+    ASSERT_EQ(this->schwarz->get_parameters().coarse_weight, value_type{-1});
 }
 
 
@@ -166,6 +221,3 @@ TYPED_TEST(SchwarzFactory, ApplyUsesInitialGuessAsLocalSolver)
     ASSERT_EQ(schwarz_with_jacobi->apply_uses_initial_guess(), false);
     ASSERT_EQ(schwarz_with_cg->apply_uses_initial_guess(), true);
 }
-
-
-}  // namespace
diff --git a/examples/distributed-solver/distributed-solver.cpp b/examples/distributed-solver/distributed-solver.cpp
index 82eaef91ffc..61f23391703 100644
--- a/examples/distributed-solver/distributed-solver.cpp
+++ b/examples/distributed-solver/distributed-solver.cpp
@@ -1,4 +1,4 @@
-// SPDX-FileCopyrightText: 2017 - 2024 The Ginkgo authors
+// SPDX-FileCopyrightText: 2017 - 2025 The Ginkgo authors
 //
 // SPDX-License-Identifier: BSD-3-Clause
 
@@ -53,6 +53,7 @@ int main(int argc, char* argv[])
     using schwarz = gko::experimental::distributed::preconditioner::Schwarz<
         ValueType, LocalIndexType, GlobalIndexType>;
     using bj = gko::preconditioner::Jacobi<ValueType, LocalIndexType>;
+    using pgm = gko::multigrid::Pgm<ValueType, LocalIndexType>;
 
     // Create an MPI communicator get the rank of the calling process.
     const auto comm = gko::experimental::mpi::communicator(MPI_COMM_WORLD);
@@ -63,10 +64,13 @@ int main(int argc, char* argv[])
     // - The executor, defaults to reference.
     // - The number of grid points, defaults to 100.
     // - The number of iterations, defaults to 1000.
+    // - One-level, two-level preconditioner, and no preconditioner, defaults to
+    // one-level.
     if (argc == 2 && (std::string(argv[1]) == "--help")) {
         if (rank == 0) {
             std::cerr << "Usage: " << argv[0]
                       << " [executor] [num_grid_points] [num_iterations] "
+                         "[schwarz_prec_type] "
                       << std::endl;
         }
         std::exit(-1);
@@ -79,6 +83,7 @@ int main(int argc, char* argv[])
         static_cast<gko::size_type>(argc >= 3 ? std::atoi(argv[2]) : 100);
     const auto num_iters =
         static_cast<gko::size_type>(argc >= 4 ? std::atoi(argv[3]) : 1000);
+    std::string schw_type = argc >= 5 ? argv[4] : "one-level";
 
     const std::map<std::string,
                    std::function<std::shared_ptr<gko::Executor>(MPI_Comm)>>
@@ -188,20 +193,73 @@ int main(int argc, char* argv[])
 
     // Setup the local block diagonal solver factory.
     auto local_solver = gko::share(bj::build().on(exec));
+    // Setup the coarse solver. If it is more accurate, then the outer
+    // iterations will reduce, but the cost of the coarse solve increases.
+    // The coarse solver can in turn have another Schwarz preconditioner if
+    // needed.
+    auto coarse_solver = gko::share(
+        solver::build()
+            .with_preconditioner(
+                schwarz::build().with_local_solver(local_solver).on(exec))
+            .with_criteria(
+                gko::stop::Iteration::build().with_max_iters(1000u).on(exec),
+                gko::stop::ResidualNorm<ValueType>::build()
+                    .with_reduction_factor(1e-7)
+                    .on(exec))
+            .on(exec));
+
+    auto pgm_fac = gko::share(pgm::build().on(exec));
 
     // Setup the stopping criterion and logger
     const gko::remove_complex<ValueType> reduction_factor{1e-8};
     std::shared_ptr<const gko::log::Convergence<ValueType>> logger =
         gko::log::Convergence<ValueType>::create();
-    auto Ainv = solver::build()
-                    .with_preconditioner(
-                        schwarz::build().with_local_solver(local_solver))
-                    .with_criteria(
-                        gko::stop::Iteration::build().with_max_iters(num_iters),
-                        gko::stop::ResidualNorm<ValueType>::build()
-                            .with_reduction_factor(reduction_factor))
-                    .on(exec)
-                    ->generate(A);
+    std::shared_ptr<gko::LinOp> Ainv{};
+    // The benefit of the two-level Schwarz preconditioner is generally
+    // observable (in runtime and in number of iterations) usually for larger
+    // problem sizes and for larger number of ranks.
+    if (schw_type == "two-level") {
+        Ainv =
+            solver::build()
+                .with_preconditioner(schwarz::build()
+                                         .with_local_solver(local_solver)
+                                         .with_coarse_level(pgm_fac)
+                                         .with_coarse_solver(coarse_solver)
+                                         .on(exec))
+                .with_criteria(
+                    gko::stop::Iteration::build().with_max_iters(num_iters).on(
+                        exec),
+                    gko::stop::ResidualNorm<ValueType>::build()
+                        .with_reduction_factor(reduction_factor)
+                        .on(exec))
+                .on(exec)
+                ->generate(A);
+    } else if (schw_type == "one-level") {
+        Ainv =
+            solver::build()
+                .with_preconditioner(
+                    schwarz::build().with_local_solver(local_solver).on(exec))
+                .with_criteria(
+                    gko::stop::Iteration::build().with_max_iters(num_iters).on(
+                        exec),
+                    gko::stop::ResidualNorm<ValueType>::build()
+                        .with_reduction_factor(reduction_factor)
+                        .on(exec))
+                .on(exec)
+                ->generate(A);
+    } else {
+        schw_type = "no-precond";
+        Ainv =
+            solver::build()
+                .with_criteria(
+                    gko::stop::Iteration::build().with_max_iters(num_iters).on(
+                        exec),
+                    gko::stop::ResidualNorm<ValueType>::build()
+                        .with_reduction_factor(reduction_factor)
+                        .on(exec))
+                .on(exec)
+                ->generate(A);
+    }
     // Add logger to the generated solver to log the iteration count and
     // residual norm
     Ainv->add_logger(logger);
@@ -219,8 +277,8 @@ int main(int argc, char* argv[])
     ValueType t_end = gko::experimental::mpi::get_walltime();
 
     // Get the residual.
-    auto res_norm = gko::clone(exec->get_master(),
-                               gko::as<vec>(logger->get_residual_norm()));
+    auto res_norm = gko::as<vec>(logger->get_residual_norm());
+    auto host_res = gko::make_temporary_clone(exec->get_master(), res_norm);
 
     // @sect3{Printing Results}
     // Print the achieved residual norm and timings on rank 0.
@@ -228,7 +286,8 @@ int main(int argc, char* argv[])
         // clang-format off
         std::cout << "\nNum rows in matrix: " << num_rows
                   << "\nNum ranks: " << comm.size()
-                  << "\nFinal Res norm: " << res_norm->at(0, 0)
+                  << "\nPrecond type: " << schw_type
+                  << "\nFinal Res norm: " << *host_res->get_const_values()
                   << "\nIteration count: " << logger->get_num_iterations()
                   << "\nInit time: " << t_init_end - t_init
                   << "\nRead time: " << t_read_setup_end - t_init
diff --git a/include/ginkgo/core/distributed/preconditioner/schwarz.hpp b/include/ginkgo/core/distributed/preconditioner/schwarz.hpp
index 9578aa27260..66ae800103c 100644
--- a/include/ginkgo/core/distributed/preconditioner/schwarz.hpp
+++ b/include/ginkgo/core/distributed/preconditioner/schwarz.hpp
@@ -20,6 +20,7 @@
 #include <ginkgo/core/distributed/matrix.hpp>
 #include <ginkgo/core/distributed/vector.hpp>
 #include <ginkgo/core/distributed/vector_cache.hpp>
+#include <ginkgo/core/solver/solver_base.hpp>
 
 
 namespace gko {
@@ -38,10 +39,22 @@ namespace preconditioner {
  * generalizes the Block Jacobi preconditioner, incorporating options for
  * different local subdomain solvers and overlaps between the subdomains.
  *
+ * A L1 smoother variant is also available, which updates the local matrix with
+ * the sums of the non-local matrix row sums.
+ *
  * See Iterative Methods for Sparse Linear Systems (Y. Saad) for a general
  * treatment and variations of the method.
  *
- * @note Currently overlap and coarse grid correction are not supported (TODO).
+ * A Two-level variant is also available. To enable two-level preconditioning,
+ * you need to specify a LinOpFactory that can generate a
+ * multigrid::MultigridLevel and a solver for the coarse level solution.
+ * Currently, only additive coarse correction is supported with an optional
+ * weighting between the local and the coarse solutions, for cases when the
+ * coarse solutions might tend to overcorrect.
+ * - See Smith, Bjorstad, Gropp, Domain Decomposition, 1996, Cambridge
+ * University Press.
+ *
+ * @note Currently overlap is not supported (TODO).
  *
  * @tparam ValueType  precision of matrix element
  * @tparam LocalIndexType  local integer type of the matrix
@@ -97,8 +110,37 @@ class Schwarz
          * sum of the non-local matrix entries. The diagonal matrix
          * is then added to the system matrix when generating the
          * local solver.
+         *
+         * Note: The L1 smoother will not be used for the coarse level matrix
+         * generation, and the original system matrix will still be used.
          */
         bool GKO_FACTORY_PARAMETER_SCALAR(l1_smoother, false);
+
+        /**
+         * Coarse weighting.
+         *
+         * By default the coarse and the local solutions are added together
+         * (when the coarse weight is < 0 or > 1). A weighting can instead be
+         * provided if the coarse solution tends to over-correct.
+         */
+        ValueType GKO_FACTORY_PARAMETER_SCALAR(coarse_weight, ValueType{-1.0});
+
+        /**
+         * Operator factory list to generate the triplet (prolong_op, coarse_op,
+         * restrict_op), `A_c = R * A * P`
+         *
+         * Note: The linop factory must generate the triplet (R, A_c, P). For
+         * example, any coarse level generator from multigrid::MultigridLevel
+         * can be used.
+         */
+        std::shared_ptr<const LinOpFactory> GKO_DEFERRED_FACTORY_PARAMETER(
+            coarse_level);
+
+        /**
+         * Coarse solver factory.
+         */
+        std::shared_ptr<const LinOpFactory> GKO_DEFERRED_FACTORY_PARAMETER(
+            coarse_solver);
     };
     GKO_ENABLE_LIN_OP_FACTORY(Schwarz, parameters, Factory);
     GKO_ENABLE_BUILD_METHOD(Factory);
@@ -144,7 +186,8 @@ class Schwarz
                      std::shared_ptr<const LinOp> system_matrix)
         : EnableLinOp<Schwarz>(factory->get_executor(),
                                gko::transpose(system_matrix->get_size())),
-          parameters_{factory->get_parameters()}
+          parameters_{factory->get_parameters()},
+          system_matrix_{system_matrix}
     {
         this->generate(system_matrix);
     }
@@ -171,8 +214,18 @@ class Schwarz
     void set_solver(std::shared_ptr<const LinOp> new_solver);
 
     std::shared_ptr<const LinOp> local_solver_;
+    std::shared_ptr<const LinOp> system_matrix_;
 
+    // Used for advanced apply
     detail::VectorCache<ValueType> cache_;
+    // Used in apply for two-level method
+    detail::VectorCache<ValueType> csol_cache_;
+    detail::VectorCache<ValueType> crhs_cache_;
+
+    std::shared_ptr<const LinOp> coarse_level_;
+    std::shared_ptr<const LinOp> coarse_solver_;
+    std::shared_ptr<const matrix::Dense<ValueType>> coarse_weight_;
+    std::shared_ptr<const matrix::Dense<ValueType>> local_weight_;
 };
 
 
diff --git a/include/ginkgo/core/distributed/vector.hpp b/include/ginkgo/core/distributed/vector.hpp
index dc278459fd4..eefdebe39b9 100644
--- a/include/ginkgo/core/distributed/vector.hpp
+++ b/include/ginkgo/core/distributed/vector.hpp
@@ -538,7 +538,7 @@ class Vector
      * @note  The data form the local_vector will be moved into the new
      *        distributed vector. You could either move in a std::unique_ptr
      *        directly, copy a local vector with gko::clone, or create a
-     *        unique non-owining view of a given local vector with
+     *        unique non-owning view of a given local vector with
      *        gko::make_dense_view.
      *
      * @param exec  Executor associated with this vector
@@ -561,7 +561,7 @@ class Vector
      * @note  The data form the local_vector will be moved into the new
      *        distributed vector. You could either move in a std::unique_ptr
      *        directly, copy a local vector with gko::clone, or create a
-     *        unique non-owining view of a given local vector with
+     *        unique non-owning view of a given local vector with
      *        gko::make_dense_view.
      *
      * @param exec  Executor associated with this vector
diff --git a/test/mpi/preconditioner/schwarz.cpp b/test/mpi/preconditioner/schwarz.cpp
index 912c9e9a652..7fdc333f22c 100644
--- a/test/mpi/preconditioner/schwarz.cpp
+++ b/test/mpi/preconditioner/schwarz.cpp
@@ -20,9 +20,11 @@
 #include <ginkgo/core/log/logger.hpp>
 #include <ginkgo/core/matrix/csr.hpp>
 #include <ginkgo/core/matrix/dense.hpp>
+#include <ginkgo/core/multigrid/pgm.hpp>
 #include <ginkgo/core/preconditioner/jacobi.hpp>
 #include <ginkgo/core/solver/bicgstab.hpp>
 #include <ginkgo/core/solver/cg.hpp>
+#include <ginkgo/core/solver/multigrid.hpp>
 #include <ginkgo/core/stop/iteration.hpp>
 #include <ginkgo/core/stop/residual_norm.hpp>
 
@@ -50,8 +52,12 @@ class SchwarzPreconditioner : public CommonMpiTestFixture {
         gko::experimental::distributed::preconditioner::Schwarz<
             value_type, local_index_type, global_index_type>;
     using solver_type = gko::solver::Bicgstab<value_type>;
+    using mg_prec_type = gko::solver::Multigrid;
     using local_prec_type =
         gko::preconditioner::Jacobi<value_type, local_index_type>;
+    using coarse_solver_type =
+        gko::preconditioner::Jacobi<value_type, local_index_type>;
+    using coarse_level_type = gko::multigrid::Pgm<value_type, local_index_type>;
     using local_matrix_type = gko::matrix::Csr<value_type, local_index_type>;
     using non_dist_matrix_type =
         gko::matrix::Csr<value_type, global_index_type>;
@@ -92,6 +98,8 @@ class SchwarzPreconditioner : public CommonMpiTestFixture {
         dist_b->fill(-gko::one<value_type>());
         dist_x = gko::share(gko::clone(exec, dist_result));
         dist_x->fill(gko::zero<value_type>());
+        dist_x2 = gko::share(gko::clone(exec, dist_result));
+        dist_x2->fill(gko::zero<value_type>());
         auto non_dist_result = local_vec_type::create(ref, global_size, nrhs);
         non_dist_result->fill(-gko::one<value_type>());
         non_dist_b = gko::share(gko::clone(exec, non_dist_result));
@@ -100,6 +108,13 @@ class SchwarzPreconditioner : public CommonMpiTestFixture {
 
         local_solver_factory =
             local_prec_type::build().with_max_block_size(1u).on(exec);
+        coarse_solver_factory =
+            solver_type::build()
+                .with_criteria(gko::stop::Iteration::build().with_max_iters(5u))
+                .on(exec);
+        pgm_factory = coarse_level_type::build().on(exec);
+        one_ = gko::initialize<local_vec_type>({gko::one<value_type>()},
+                                               this->exec);
     }
 
     void SetUp() override { ASSERT_EQ(comm.size(), 3); }
@@ -112,12 +127,16 @@ class SchwarzPreconditioner : public CommonMpiTestFixture {
     std::shared_ptr<dist_mtx_type> dist_mat;
     std::shared_ptr<dist_vec_type> dist_b;
     std::shared_ptr<dist_vec_type> dist_x;
+    std::shared_ptr<dist_vec_type> dist_x2;
     std::shared_ptr<non_dist_matrix_type> non_dist_mat;
     std::shared_ptr<local_vec_type> non_dist_b;
     std::shared_ptr<local_vec_type> non_dist_x;
+    std::shared_ptr<local_vec_type> one_;
     std::shared_ptr<gko::LinOpFactory> non_dist_solver_factory;
     std::shared_ptr<gko::LinOpFactory> dist_solver_factory;
     std::shared_ptr<gko::LinOpFactory> local_solver_factory;
+    std::shared_ptr<gko::LinOpFactory> pgm_factory;
+    std::shared_ptr<gko::LinOpFactory> coarse_solver_factory;
 
     void assert_equal_to_non_distributed_vector(
         std::shared_ptr<dist_vec_type> dist_vec,
@@ -264,6 +283,48 @@ TYPED_TEST(SchwarzPreconditioner, CanApplyPreconditioner)
 }
 
 
+TYPED_TEST(SchwarzPreconditioner, CanApplyTwolevelPreconditioner)
+{
+    using value_type = typename TestFixture::value_type;
+    using prec = typename TestFixture::dist_prec_type;
+    using mg_prec = typename TestFixture::mg_prec_type;
+    using dist_vec = typename TestFixture::dist_vec_type;
+
+    auto precond_factory = prec::build()
+                               .with_local_solver(this->local_solver_factory)
+                               .with_coarse_solver(this->coarse_solver_factory)
+                               .with_coarse_level(this->pgm_factory)
+                               .on(this->exec);
+    auto local_precond_factory =
+        prec::build()
+            .with_local_solver(this->local_solver_factory)
+            .on(this->exec);
+    auto mg_precond_factory =
+        mg_prec::build()
+            .with_coarsest_solver(this->coarse_solver_factory)
+            .with_mg_level(this->pgm_factory)
+            .with_post_smoother(nullptr)
+            .with_pre_smoother(nullptr)
+            .with_max_levels(1u)
+            .with_min_coarse_rows(1u)
+            .with_criteria(gko::stop::Iteration::build().with_max_iters(1u))
+            .on(this->exec);
+    auto precond = precond_factory->generate(this->dist_mat);
+    auto mg_precond = mg_precond_factory->generate(this->dist_mat);
+    auto local_precond = local_precond_factory->generate(this->dist_mat);
+    auto local_dist_x = gko::clone(this->dist_x);
+    local_precond->apply(this->dist_b.get(), local_dist_x.get());
+    mg_precond->apply(this->dist_b.get(), this->dist_x2.get());
+    this->dist_x2->add_scaled(this->one_, local_dist_x);
+
+    precond->apply(this->dist_b.get(), this->dist_x.get());
+
+    GKO_ASSERT_MTX_NEAR(this->dist_x->get_local_vector(),
+                        this->dist_x2->get_local_vector(),
+                        r<value_type>::value);
+}
+
+
 TYPED_TEST(SchwarzPreconditioner, CanAdvancedApplyPreconditioner)
 {
     using value_type = typename TestFixture::value_type;


Reviews:
user: login: MarcelKoch
body: Generally looks good, I left some minor nits.
Did you have a case, where the coarse grid solve was beneficial? Right now the provided example doesn't seem to benefit from the coarse grid solve.
Also, since the coarse and local solves are not computed in parallel, as you mentioned, which will probably be the case for a while, maybe implement it as a multiplicative version directly.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-02-03T09:57:41Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-14T09:34:07Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-14T09:41:10Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-14T09:52:47Z
user: login: MarcelKoch
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-14T10:02:14Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-14T10:56:00Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-02-17T09:02:57Z
user: login: MarcelKoch
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-17T14:11:49Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T10:05:01Z
user: login: MarcelKoch
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-18T10:10:30Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T12:48:33Z
user: login: yhmtsai
body: #1806 seems to be merged first. We will need to discuss the two-level should take the matrix after or before $\ell_1$ modification.
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-18T13:11:29Z
user: login: yhmtsai
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:18:16Z
user: login: yhmtsai
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-18T13:20:12Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T13:45:52Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T13:46:51Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-24T14:18:35Z
user: login: MarcelKoch
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-26T14:21:11Z
user: login: yhmtsai
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-03-27T09:25:28Z
user: login: MarcelKoch
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-27T09:27:35Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T19:56:11Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T20:16:59Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T20:19:48Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T08:42:08Z
user: login: yhmtsai
state: CHANGES_REQUESTED
author_association: MEMBER
submitted_at: 2025-04-10T09:04:14Z
user: login: pratikvn
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-10T12:12:12Z
user: login: yhmtsai
state: APPROVED
author_association: MEMBER
submitted_at: 2025-04-10T12:43:18Z

Review comments:
diff_hunk: @@ -89,6 +91,19 @@ class Schwarz
          */
         std::shared_ptr<const LinOp> GKO_FACTORY_PARAMETER_SCALAR(
             generated_local_solver, nullptr);
+
+        /**
+         * Operator factory to generate the triplet (prolong_op, coarse_op,
+         * restrict_op).
+         */
+        std::shared_ptr<const LinOpFactory> GKO_DEFERRED_FACTORY_PARAMETER(
+            galerkin_ops);
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: MarcelKoch
body: I'm not too convinced of the name and the doc. At least to me, `galerkin_ops` is too unspecific, maybe something like `coarse_level_factory` would be better suited. Also the doc should contain that this will be used to create a coarse level system, maybe 
```
Operator factory to generate the coarse system `A_c = R * A * P`. 
```
created_at: 2025-02-03T09:10:38Z
author_association: MEMBER
diff_hunk: @@ -77,11 +87,40 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
+
     if (this->local_solver_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
     }
+
+    if (this->coarse_solver_ != nullptr && this->galerkin_ops_ != nullptr) {
+        auto restrict = as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                            ->get_restrict_op();
+        auto prolong = as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                           ->get_prolong_op();
+        auto coarse =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                    ->get_coarse_op());
+        GKO_ASSERT(this->half_ != nullptr);
+
+        // Coarse solve vector cache init
+        // Should allocare only in the first apply call.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        restrict->apply(dense_b, csol_cache_.get());
+        this->coarse_solver_->apply(csol_cache_.get(), csol_cache_.get());
+        prolong->apply(this->half_, csol_cache_.get(), this->half_, dense_x);
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: I don't think it's typical to weight these solutions. For example the book Domain Decomposition by Smith, Bjorstad, and Gropp, just sums all contributions up (p. 47) regardless of if they are from the subdomains, or the coarse grid.
created_at: 2025-02-03T09:13:15Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: maybe there should be an exception, if only one of those is set. I think that would be a configuration error, and the user should be notified.
created_at: 2025-02-03T09:14:36Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
+        this->galerkin_ops_ =
+            share(parameters_.galerkin_ops->generate(system_matrix));
+        if (as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                ->get_coarse_op()) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: Can this actually be null? If so, maybe that should be considered an error.
created_at: 2025-02-03T09:15:51Z
author_association: MEMBER
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: I think adjustments to the copy/move are necessary. However, they seem to be already incomplete as of now, so that might also be done in a separate PR.
created_at: 2025-02-03T09:24:04Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
+        this->galerkin_ops_ =
+            share(parameters_.galerkin_ops->generate(system_matrix));
+        if (as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                ->get_coarse_op()) {
+            auto coarse =
+                as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                     GlobalIndexType>>(
+                    as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                        ->get_coarse_op());
+            auto exec = coarse->get_executor();
+            auto comm = coarse->get_communicator();
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: unused
```suggestion
```
created_at: 2025-02-03T09:24:51Z
author_association: MEMBER
diff_hunk: @@ -79,6 +80,7 @@ int main(int argc, char* argv[])
         static_cast<gko::size_type>(argc >= 3 ? std::atoi(argv[2]) : 100);
     const auto num_iters =
         static_cast<gko::size_type>(argc >= 4 ? std::atoi(argv[3]) : 1000);
+    std::string schw_type = argc >= 5 ? argv[4] : "multi-level";
path: examples/distributed-solver/distributed-solver.cpp
user: login: MarcelKoch
body: Just as a note, I was running this example with
```
mpirun -n 4 ./distributed-solver reference 10000 1000
```
and the `one-level` version was faster and used less iteration. So maybe it should be the default.
created_at: 2025-02-03T09:54:03Z
author_association: MEMBER
diff_hunk: @@ -77,11 +87,40 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
+
     if (this->local_solver_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
     }
+
+    if (this->coarse_solver_ != nullptr && this->galerkin_ops_ != nullptr) {
+        auto restrict = as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                            ->get_restrict_op();
+        auto prolong = as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                           ->get_prolong_op();
+        auto coarse =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->galerkin_ops_)
+                    ->get_coarse_op());
+        GKO_ASSERT(this->half_ != nullptr);
+
+        // Coarse solve vector cache init
+        // Should allocare only in the first apply call.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        restrict->apply(dense_b, csol_cache_.get());
+        this->coarse_solver_->apply(csol_cache_.get(), csol_cache_.get());
+        prolong->apply(this->half_, csol_cache_.get(), this->half_, dense_x);
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: For multiplicative versions that is valid, but I think in this case it generally results in an over-correction and leads to solver divergence.
created_at: 2025-02-14T09:34:07Z
author_association: MEMBER
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: Tracking it here: https://github.com/ginkgo-project/ginkgo/issues/1779
created_at: 2025-02-14T09:41:10Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: I think we need in general a better way to validate our configuration. If multiple factory parameters are dependent, then I think there should be a way to communicate that directly to the user rather than having the logic in the core file. Maybe we can think about adding some dependency support checks in the factory parameter setup agnostic to the concrete class.
created_at: 2025-02-14T09:52:47Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: We could do that at a later point, but for now we should still throw if only one is set. I don't think it would be much of a burden here.
created_at: 2025-02-14T10:02:14Z
author_association: MEMBER
diff_hunk: @@ -79,6 +80,7 @@ int main(int argc, char* argv[])
         static_cast<gko::size_type>(argc >= 3 ? std::atoi(argv[2]) : 100);
     const auto num_iters =
         static_cast<gko::size_type>(argc >= 4 ? std::atoi(argv[3]) : 1000);
+    std::string schw_type = argc >= 5 ? argv[4] : "multi-level";
path: examples/distributed-solver/distributed-solver.cpp
user: login: pratikvn
body: Yes, it seems to depend on the problem size. I see that 100000 multi-level reduces the number of iterations, but it is slower. But I will change the default to one-level.
created_at: 2025-02-14T10:56:00Z
author_association: MEMBER
diff_hunk: @@ -129,16 +170,35 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
 
     if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
-
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.galerkin_ops && parameters_.coarse_solver) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: Yes, sure.
created_at: 2025-02-17T09:02:57Z
author_association: MEMBER
diff_hunk: @@ -89,6 +98,32 @@ class Schwarz
          */
         std::shared_ptr<const LinOp> GKO_FACTORY_PARAMETER_SCALAR(
             generated_local_solver, nullptr);
+
+        /**
+         * Coarse weighting.
+         *
+         * By default the coarse and the local solutions are added together. A
+         * weighting can instead be provided if the coarse solution tends to
+         * over-correct.
+         */
+        double GKO_FACTORY_PARAMETER_SCALAR(coarse_weight, double{-1.0});
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: MarcelKoch
body: Maybe mention what a negative value means?
created_at: 2025-03-17T13:57:27Z
author_association: MEMBER
diff_hunk: @@ -121,6 +156,7 @@ class Schwarz
      */
     explicit Schwarz(std::shared_ptr<const Executor> exec)
         : EnableLinOp<Schwarz>(std::move(exec))
+
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: MarcelKoch
body: ```suggestion
```
created_at: 2025-03-17T13:58:21Z
author_association: MEMBER
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: MarcelKoch
body: I think the config parser is not adapted.
created_at: 2025-03-17T13:59:11Z
author_association: MEMBER
diff_hunk: @@ -20,6 +20,8 @@
 #include <ginkgo/core/distributed/matrix.hpp>
 #include <ginkgo/core/distributed/vector.hpp>
 #include <ginkgo/core/distributed/vector_cache.hpp>
+#include <ginkgo/core/multigrid/multigrid_level.hpp>
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: MarcelKoch
body: This is probably not necessary here.
created_at: 2025-03-17T13:59:35Z
author_association: MEMBER
diff_hunk: @@ -130,15 +178,53 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: I think a check for `coarse_weight > 1` wouldn't hurt.
created_at: 2025-03-17T14:00:27Z
author_association: MEMBER
diff_hunk: @@ -130,15 +178,53 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: you could use `if(auto coarse_level = std::dynamic_pointer_cast<const multigrid::MultigridLevel>(this->coarse_level_))` to remove redundancy.
created_at: 2025-03-17T14:02:53Z
author_association: MEMBER
diff_hunk: @@ -130,15 +178,53 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: IMO, the current one is easier to understand.
created_at: 2025-03-18T10:05:01Z
author_association: MEMBER
diff_hunk: @@ -47,6 +48,19 @@ Schwarz<ValueType, LocalIndexType, GlobalIndexType>::parse(
             gko::config::parse_or_get_factory<const LinOpFactory>(
                 obj, context, td_for_child));
     }
+    if (auto& obj = config.get("coarse_level")) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: all config parameters are missing tests.
created_at: 2025-03-18T10:09:32Z
author_association: MEMBER
diff_hunk: @@ -47,6 +48,19 @@ Schwarz<ValueType, LocalIndexType, GlobalIndexType>::parse(
             gko::config::parse_or_get_factory<const LinOpFactory>(
                 obj, context, td_for_child));
     }
+    if (auto& obj = config.get("coarse_level")) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: Should be added now
created_at: 2025-03-18T12:48:33Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: throw an exception when only one available?
created_at: 2025-03-18T12:51:53Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto restrict_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_restrict_op();
+        auto prolong_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                    ->get_coarse_op());
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
        auto mg_level = as<gko::multigrid::MultigridLevel>(this->coarse_level_);
        auto restrict_op = mg_level->get_restrict_op();
        auto prolong_op = mg_level->get_prolong_op();
        auto coarse_op =
            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
                                                 GlobalIndexType>>(
                mg_level->get_coarse_op());
```
created_at: 2025-03-18T12:54:00Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto restrict_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_restrict_op();
+        auto prolong_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                    ->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call.
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
        // Should allocate only in the first apply call if the number of right hand side are unchanged.
```
created_at: 2025-03-18T12:54:46Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto restrict_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_restrict_op();
+        auto prolong_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                    ->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse_op->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse_op->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse_op->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        // Additive apply of coarse correction
+        restrict_op->apply(dense_b, csol_cache_.get());
+        this->coarse_solver_->apply(csol_cache_.get(), csol_cache_.get());
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: It is wrong. 
When solver updates the solution it will also update the right hand side.
some kernel will also face issues when the input and output are on the same memory
created_at: 2025-03-18T12:57:22Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: I see that you handle it in the generation phase
created_at: 2025-03-18T12:58:38Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto restrict_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_restrict_op();
+        auto prolong_op =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                ->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                as<gko::multigrid::MultigridLevel>(this->coarse_level_)
+                    ->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse_op->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse_op->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse_op->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        // Additive apply of coarse correction
+        restrict_op->apply(dense_b, csol_cache_.get());
+        this->coarse_solver_->apply(csol_cache_.get(), csol_cache_.get());
+        prolong_op->apply(this->coarse_weight_, csol_cache_.get(),
+                          this->local_weight_, dense_x);
+    } else {
+        // One-level
+        if (this->local_solver_ != nullptr) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
    } else if (this->local_solver_ != nullptr) {
        // One-level
```
created_at: 2025-03-18T12:59:06Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: because we only use it in ValueType, the type of coarse_weight should directly use ValueType not double
created_at: 2025-03-18T13:02:12Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op();
+            // Cannot guarantee that coarse op is not nullptr
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: should it throw the error rather than make the operation silent?
created_at: 2025-03-18T13:04:05Z
author_association: MEMBER
diff_hunk: @@ -63,10 +64,12 @@ int main(int argc, char* argv[])
     // - The executor, defaults to reference.
     // - The number of grid points, defaults to 100.
     // - The number of iterations, defaults to 1000.
+    // - One-level or two-level preconditioner, defaults to one-level.
path: examples/distributed-solver/distributed-solver.cpp
user: login: yhmtsai
body: it also has another option for no-preconditioned in the code
created_at: 2025-03-18T13:07:39Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: I think you can move the local_solver part to the beginning without another condition.
```
if (local_solver_ != nullptr) {...}
if (this->coarse_solver_ ...) {...}
```
created_at: 2025-03-18T13:09:09Z
author_association: MEMBER
diff_hunk: @@ -89,6 +97,32 @@ class Schwarz
          */
         std::shared_ptr<const LinOp> GKO_FACTORY_PARAMETER_SCALAR(
             generated_local_solver, nullptr);
+
+        /**
+         * Coarse weighting.
+         *
+         * By default the coarse and the local solutions are added together
+         * (when the coarse weight is < 0). A weighting can instead be provided
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: yhmtsai
body: ```suggestion
         * (when the coarse weight is <= 0 or > 1). A weighting can instead be provided
```
created_at: 2025-03-18T13:18:16Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: putting coarse_weight = 1 or 0 to the above formula will not get the same weight here, right?
created_at: 2025-03-18T13:20:11Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: Yes, coarse_weight 1 should mean that only the coarse level solve is used, and coarse_weight 0 means that only the local solves are used. If no coarse_weight is used (implying to weighting between two solutions), then the solutions are just added up.
created_at: 2025-03-24T13:45:52Z
author_association: MEMBER
diff_hunk: @@ -77,10 +91,46 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
     using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: I prefer to keep them in separate code paths for better code readability.
created_at: 2025-03-24T13:46:51Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op();
+            // Cannot guarantee that coarse op is not nullptr
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: I guess that depends on whether we consider generation of a coarse level valid with empty triplet (R, A, P = nullptr)
created_at: 2025-03-24T14:18:35Z
author_association: MEMBER
diff_hunk: @@ -87,10 +102,48 @@ template <typename VectorType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto coarse_level =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_);
+        auto restrict_op = coarse_level->get_restrict_op();
+        auto prolong_op = coarse_level->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                coarse_level->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call if the number of rhs is
+        // unchanged.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse_op->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse_op->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse_op->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+        crhs_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        // Additive apply of coarse correction
+        restrict_op->apply(dense_b, crhs_cache_.get());
+        // TODO: Does it make sense to restrict dense_x (to csol_cache) to
+        // provide a good initial guess for the coarse solver ?
+        this->coarse_solver_->apply(crhs_cache_.get(), csol_cache_.get());
+        prolong_op->apply(this->coarse_weight_, csol_cache_.get(),
+                          this->local_weight_, dense_x);
+    } else {
+        // One-level
+        if (this->local_solver_ != nullptr) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: nit: else if (...)
created_at: 2025-03-26T14:03:45Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,39 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: This can only return false if `coarse_level_` has dynamic type `MultigridLevel` and is the nullptr. Can this actually happen? If it doesn't have the correct dynamic type, then `as` will already throw an exception.
created_at: 2025-03-26T14:20:01Z
author_association: MEMBER
diff_hunk: @@ -87,10 +102,48 @@ template <typename VectorType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
         this->local_solver_->apply(gko::detail::get_local(dense_b),
                                    gko::detail::get_local(dense_x));
+        auto coarse_level =
+            as<gko::multigrid::MultigridLevel>(this->coarse_level_);
+        auto restrict_op = coarse_level->get_restrict_op();
+        auto prolong_op = coarse_level->get_prolong_op();
+        auto coarse_op =
+            as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                                 GlobalIndexType>>(
+                coarse_level->get_coarse_op());
+
+        // Coarse solve vector cache init
+        // Should allocate only in the first apply call if the number of rhs is
+        // unchanged.
+        auto cs_ncols = dense_x->get_size()[1];
+        auto cs_local_nrows = coarse_op->get_local_matrix()->get_size()[0];
+        auto cs_global_nrows = coarse_op->get_size()[0];
+        auto cs_local_size = dim<2>(cs_local_nrows, cs_ncols);
+        auto cs_global_size = dim<2>(cs_global_nrows, cs_ncols);
+        auto comm = coarse_op->get_communicator();
+        csol_cache_.init(exec, comm, cs_global_size, cs_local_size);
+        crhs_cache_.init(exec, comm, cs_global_size, cs_local_size);
+
+        // Additive apply of coarse correction
+        restrict_op->apply(dense_b, crhs_cache_.get());
+        // TODO: Does it make sense to restrict dense_x (to csol_cache) to
+        // provide a good initial guess for the coarse solver ?
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: you still need to set it to zero or right hand side here, or the initial guess can be invalid value, which might destroy the iterative method
created_at: 2025-03-27T08:53:47Z
author_association: MEMBER
diff_hunk: @@ -130,24 +183,29 @@ template <typename ValueType, typename LocalIndexType, typename GlobalIndexType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
     std::shared_ptr<const LinOp> system_matrix)
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     if (parameters_.local_solver && parameters_.generated_local_solver) {
         GKO_INVALID_STATE(
             "Provided both a generated solver and a solver factory");
     }
-
     if (!parameters_.local_solver && !parameters_.generated_local_solver) {
         GKO_INVALID_STATE(
             "Requires either a generated solver or an solver factory");
     }
-
     if (parameters_.generated_local_solver) {
         this->set_solver(parameters_.generated_local_solver);
         return;
     }
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: It does not handle the l1 two level schwarz properly. add another check `(coarse_solver && coarse_level && l1_smoother)`.
created_at: 2025-03-27T08:59:43Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,39 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: I assume you mean only return true?
created_at: 2025-03-27T09:02:00Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op();
+            // Cannot guarantee that coarse op is not nullptr
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: I think we should throw an error now. The previous issue here is caused by that the test does not initialize the coarse solver and coarse level properly. The slight operation on this will give user wrong performance thought. If someone comes to use for this behavior later, we can add another parameter for that but not the default behavior.
created_at: 2025-03-27T09:04:59Z
author_association: MEMBER
diff_hunk: @@ -99,6 +107,32 @@ class Schwarz
          * local solver.
          */
         bool GKO_FACTORY_PARAMETER_SCALAR(l1_smoother, false);
+
+        /**
+         * Coarse weighting.
+         *
+         * By default the coarse and the local solutions are added together
+         * (when the coarse weight is < 0 or > 1). A weighting can instead be
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: yhmtsai
body: ```suggestion
         * (when the coarse weight is <= 0 or > 1). A weighting can instead be
```
created_at: 2025-03-27T09:06:56Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,39 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
            {static_cast<ValueType>(parameters_.coarse_weight)}, this->get_executor());
```
there is narrow-down warning here
created_at: 2025-03-27T09:11:26Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,39 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
            {one<ValueType>()}, this->get_executor());
```
same for next one to avoid the narrow-down warning.
created_at: 2025-03-27T09:12:07Z
author_association: MEMBER
diff_hunk: @@ -99,6 +107,32 @@ class Schwarz
          * local solver.
          */
         bool GKO_FACTORY_PARAMETER_SCALAR(l1_smoother, false);
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: yhmtsai
body: add a comment about it is unsupported with two level
created_at: 2025-03-27T09:13:12Z
author_association: MEMBER
diff_hunk: @@ -99,6 +107,32 @@ class Schwarz
          * local solver.
          */
         bool GKO_FACTORY_PARAMETER_SCALAR(l1_smoother, false);
+
+        /**
+         * Coarse weighting.
+         *
+         * By default the coarse and the local solutions are added together
+         * (when the coarse weight is < 0 or > 1). A weighting can instead be
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: yhmtsai
body: changing the condition in the core might be better for symmetric 
created_at: 2025-03-27T09:14:46Z
author_association: MEMBER
diff_hunk: @@ -264,6 +276,22 @@ TYPED_TEST(SchwarzPreconditioner, CanApplyPreconditioner)
 }
 
 
+TYPED_TEST(SchwarzPreconditioner, CanApplyMultilevelPreconditioner)
path: test/mpi/preconditioner/schwarz.cpp
user: login: yhmtsai
body: The test is not enough because it does not verify the result based on something we tested before.
You can do the workflow again like you did in core. also need two coarse weight testcase: one is in the [0, 1] and the other is default.

I was thinking whether extracting the same thing from core to here is considered valid or not.
It seems to be duplication but the routines here can be considered verified by the other tests.
created_at: 2025-03-27T09:25:23Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,39 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: MarcelKoch
body: No, but the first comma is confusing, so I remove it.
created_at: 2025-03-27T09:27:34Z
author_association: MEMBER
diff_hunk: @@ -130,15 +182,52 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
             "Requires either a generated solver or an solver factory");
     }
 
-    if (parameters_.local_solver) {
-        this->set_solver(gko::share(parameters_.local_solver->generate(
-            as<experimental::distributed::Matrix<
-                ValueType, LocalIndexType, GlobalIndexType>>(system_matrix)
-                ->get_local_matrix())));
+    if ((parameters_.coarse_level && !parameters_.coarse_solver) ||
+        (!parameters_.coarse_level && parameters_.coarse_solver)) {
+        GKO_INVALID_STATE(
+            "Requires both coarse solver and coarse level to be set.");
+    }
+    auto dist_mat =
+        as<experimental::distributed::Matrix<ValueType, LocalIndexType,
+                                             GlobalIndexType>>(system_matrix);
+    if (parameters_.coarse_weight > 0 && parameters_.coarse_weight <= 1) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1} - ValueType{parameters_.coarse_weight}},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{parameters_.coarse_weight}}, this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {ValueType{1.0}}, this->get_executor());
+    }
 
+    if (parameters_.local_solver) {
+        this->set_solver(gko::share(
+            parameters_.local_solver->generate(dist_mat->get_local_matrix())));
     } else {
         this->set_solver(parameters_.generated_local_solver);
     }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (as<multigrid::MultigridLevel>(this->coarse_level_)) {
+            auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op();
+            // Cannot guarantee that coarse op is not nullptr
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: No, the issue is that when pgm (or MultigridLevel) is generated with an empty matrix, then the coarse op is null. We should probably ensure that the triplet generated by MultigridLevel cannot be null. If we throw here then it means that the Schwarz preconditioner will not be in a valid state. 
created_at: 2025-04-07T19:56:11Z
author_association: MEMBER
diff_hunk: @@ -99,6 +107,32 @@ class Schwarz
          * local solver.
          */
         bool GKO_FACTORY_PARAMETER_SCALAR(l1_smoother, false);
path: include/ginkgo/core/distributed/preconditioner/schwarz.hpp
user: login: pratikvn
body: It is supported with two-level. L1 being a local smoother is separate from the coarse solve. 
created_at: 2025-04-07T20:16:59Z
author_association: MEMBER
diff_hunk: @@ -264,6 +276,22 @@ TYPED_TEST(SchwarzPreconditioner, CanApplyPreconditioner)
 }
 
 
+TYPED_TEST(SchwarzPreconditioner, CanApplyMultilevelPreconditioner)
path: test/mpi/preconditioner/schwarz.cpp
user: login: pratikvn
body: Which core tests do you mean ? I dont think there are any tests in core which do a proper apply. 
created_at: 2025-04-07T20:19:48Z
author_association: MEMBER
diff_hunk: @@ -264,6 +276,22 @@ TYPED_TEST(SchwarzPreconditioner, CanApplyPreconditioner)
 }
 
 
+TYPED_TEST(SchwarzPreconditioner, CanApplyMultilevelPreconditioner)
path: test/mpi/preconditioner/schwarz.cpp
user: login: pratikvn
body: Should be updated now. I just use Multigrid as a preconditioner to emulate the coarse apply and a local precond separately and then add the solutions together. 
created_at: 2025-04-10T08:42:08Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,34 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: ```suggestion
    if (cweight >= 0.0 && cweight <= 1.0) {
```
created_at: 2025-04-10T08:57:23Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,34 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>() -
+             static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op()) {
+            this->coarse_solver_ = share(parameters_.coarse_solver->generate(
+                as<Matrix<ValueType, LocalIndexType, GlobalIndexType>>(
+                    coarse)));
+        }
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: throw an error or assert if coarse_level_ or coarse_solver are nullptr.
Do you need the fallback option? 
For example, the problem is small enough or different reason such that multigrid level does not generate proper one.
User seems to easily have a wrong expectation. we do not perform the two-level schwarz.
created_at: 2025-04-10T09:03:28Z
author_association: MEMBER
diff_hunk: @@ -87,8 +102,46 @@ template <typename VectorType>
 void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::apply_dense_impl(
     const VectorType* dense_b, VectorType* dense_x) const
 {
+    using Vector = matrix::Dense<ValueType>;
+    using dist_vec = experimental::distributed::Vector<ValueType>;
     auto exec = this->get_executor();
-    if (this->local_solver_ != nullptr) {
+
+    // Two-level
+    if (this->coarse_solver_ != nullptr && this->coarse_level_ != nullptr) {
+        this->local_solver_->apply(gko::detail::get_local(dense_b),
+                                   gko::detail::get_local(dense_x));
path: core/distributed/preconditioner/schwarz.cpp
user: login: yhmtsai
body: you need to wrap if (this->local_solver) to ensure it is not nullptr
created_at: 2025-04-10T09:04:10Z
author_association: MEMBER
diff_hunk: @@ -183,6 +241,34 @@ void Schwarz<ValueType, LocalIndexType, GlobalIndexType>::generate(
         this->set_solver(
             gko::share(parameters_.local_solver->generate(local_matrix)));
     }
+
+    gko::remove_complex<ValueType> cweight =
+        gko::detail::real_impl(parameters_.coarse_weight);
+    if (cweight > 0.0 && cweight <= 1.0) {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>() -
+             static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {static_cast<ValueType>(parameters_.coarse_weight)},
+            this->get_executor());
+    } else {
+        this->local_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+        this->coarse_weight_ = gko::initialize<matrix::Dense<ValueType>>(
+            {one<ValueType>()}, this->get_executor());
+    }
+
+    if (parameters_.coarse_level && parameters_.coarse_solver) {
+        this->coarse_level_ =
+            share(parameters_.coarse_level->generate(system_matrix));
+        if (auto coarse = as<multigrid::MultigridLevel>(this->coarse_level_)
+                              ->get_coarse_op()) {
+            this->coarse_solver_ = share(parameters_.coarse_solver->generate(
+                as<Matrix<ValueType, LocalIndexType, GlobalIndexType>>(
+                    coarse)));
+        }
path: core/distributed/preconditioner/schwarz.cpp
user: login: pratikvn
body: I would say that is out of scope of what Schwarz should do. I think the user has to ensure that the coarse level has valid inputs. The fallback option will be that there is no coarse level solve.
created_at: 2025-04-10T12:12:12Z
author_association: MEMBER

PR comments:
user: login: pratikvn
created_at: 2025-01-31T16:05:31Z
body: As a first step, I just added a two-level preconditioner with equal weighting for the local solution and the coarse solution. I think it maybe makes sense to try this out in some applications and then think about arbitrary number of levels, additive/multiplicative etc. 
author_association: MEMBER
user: login: pratikvn
created_at: 2025-01-31T16:06:19Z
body: format!
author_association: MEMBER
user: login: yhmtsai
created_at: 2025-03-24T13:52:20Z
body: `pgm_factory` and `coarse_solver_factory` are not initialized
author_association: MEMBER
user: login: pratikvn
created_at: 2025-03-24T19:55:44Z
body: Regarding the L1 smoother, I think it should not affect the coarse solve. The levels (fine-local and coarse) should still be generated from the input matrix rather than an implicitly modified matrix. 

IMO, the L1 smoother should have been separate from Schwarz class and not included in the Schwarz class at all. It should be equivalent to the user calling `diag_mat->apply(one, I, one, dist_mat)`, right (After summing the non-local matrix rows) ? I am not entirely sure if it should be a part of the Schwarz preconditioner class directly. I think that makes sense for the Jacobi preconditioner, but I am not sure it makes sense for the other preconditioners ? 
author_association: MEMBER
user: login: yhmtsai
created_at: 2025-03-26T14:19:59Z
body: for L1, yes it is equivalent. However, user will not be able to handle it in adding the additional computation by themselves because it should only be used in the preconditioner. the solver will not adapt the process.
author_association: MEMBER
user: login: sonarqubecloud[bot]
created_at: 2025-04-11T07:24:28Z
body: ## [![Quality Gate Passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/qg-passed-20px.png 'Quality Gate Passed')](https://sonarcloud.io/dashboard?id=ginkgo-project_ginkgo&pullRequest=1431) **Quality Gate passed**  
Issues  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [7 New issues](https://sonarcloud.io/project/issues?id=ginkgo-project_ginkgo&pullRequest=1431&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/accepted-16px.png '') [0 Accepted issues](https://sonarcloud.io/project/issues?id=ginkgo-project_ginkgo&pullRequest=1431&issueStatuses=ACCEPTED)

Measures  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=ginkgo-project_ginkgo&pullRequest=1431&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [85.3% Coverage on New Code](https://sonarcloud.io/component_measures?id=ginkgo-project_ginkgo&pullRequest=1431&metric=new_coverage&view=list)  
![](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/passed-16px.png '') [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=ginkgo-project_ginkgo&pullRequest=1431&metric=new_duplicated_lines_density&view=list)  
  
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=ginkgo-project_ginkgo&pullRequest=1431)


author_association: NONE
user: login: codecov[bot]
created_at: 2025-04-11T07:30:49Z
body: ## [Codecov](https://app.codecov.io/gh/ginkgo-project/ginkgo/pull/1431?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project) Report
Attention: Patch coverage is `97.61905%` with `3 lines` in your changes missing coverage. Please review.
> Project coverage is 89.36%. Comparing base [(`fc280ab`)](https://app.codecov.io/gh/ginkgo-project/ginkgo/commit/fc280abd951ed1b605e21119b431eaa46344bcaa?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project) to head [(`d6c3ebc`)](https://app.codecov.io/gh/ginkgo-project/ginkgo/commit/d6c3ebc1156df5f1b72cb4eb666c0985893e8a6b?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project).
> Report is 23 commits behind head on develop.

| [Files with missing lines](https://app.codecov.io/gh/ginkgo-project/ginkgo/pull/1431?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project) | Patch % | Lines |
|---|---|---|
| [core/distributed/preconditioner/schwarz.cpp](https://app.codecov.io/gh/ginkgo-project/ginkgo/pull/1431?src=pr&el=tree&filepath=core%2Fdistributed%2Fpreconditioner%2Fschwarz.cpp&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project#diff-Y29yZS9kaXN0cmlidXRlZC9wcmVjb25kaXRpb25lci9zY2h3YXJ6LmNwcA==) | 94.00% | [3 Missing :warning: ](https://app.codecov.io/gh/ginkgo-project/ginkgo/pull/1431?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project) |

<details><summary>Additional details and impacted files</summary>


```diff
@@             Coverage Diff             @@
##           develop    #1431      +/-   ##
===========================================
- Coverage    89.57%   89.36%   -0.21%     
===========================================
  Files          824      824              
  Lines        68815    68983     +168     
===========================================
+ Hits         61643    61649       +6     
- Misses        7172     7334     +162     
```

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/ginkgo-project/ginkgo/pull/1431?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ginkgo-project).

<details><summary> :rocket: New features to boost your workflow: </summary>

- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
</details>
author_association: NONE