Repository: stan-dev/math
Pull request #3171: Replace simplex transform with one based on the ILR transform
Author: login: WardBrian
Created: 2025-04-02T20:17:05Z | Merged: 2025-04-28T18:49:18Z

## Summary

Closes #3170.

This updates the simplex transform and the stochastic matrices to use the ILR transform rather than stick breaking. 
The stochastic matrices delegate more to the simplex functions for now, they could be rewritten for efficiency later.


Q: This completely replaces the existing functions, but we may want to preserve them under a different name. If we're fine replacing them, this also closes #3168 

## Tests

Existing simplex tests pass, and I added one more to test some edge behavior

## Side Effects



## Release notes

The simplex transform is now defined in terms of the isometric log ratio transform instead of stick breaking.

## Checklist

- [x] Copyright holder: Simons Foundation

    The copyright holder is typically you or your assignee, such as a university or company. By submitting this pull request, the copyright holder is agreeing to the license the submitted work under the following licenses:
      - Code: BSD 3-clause (https://opensource.org/licenses/BSD-3-Clause)
      - Documentation: CC-BY 4.0 (https://creativecommons.org/licenses/by/4.0/)

- [x] the basic tests are passing

    - unit tests pass (to run, use: `./runTests.py test/unit`)
    - header checks pass, (`make test-headers`)
    - dependencies checks pass, (`make test-math-dependencies`)
    - docs build, (`make doxygen`)
    - code passes the built in [C++ standards](https://github.com/stan-dev/stan/wiki/Coding-Style-and-Idioms) checks (`make cpplint`)

- [x] the code is written in idiomatic C++ and changes are documented in the doxygen

- [x] the new changes are tested


Linked issue #3170: Add inverse ILR simplex transform
Add inverse ILR simplex. 

@WardBrian we can do this in 2 loops using the online softmax https://arxiv.org/abs/1805.02867. The first loop constructs the sum to zero, get the max value of that sum to zero vec, and returns the sum of exponentials with the max subtracted. The second loop does the safe exponential of the sum to zero vector with the max subtracted and divides by the sum of exponentials output from the first loop. The jacobian is output after.

```cpp
inline plain_type_t<Vec> simplex_ilr_constrain(const Vec& y, Lp& lp) {
  const auto N = y.size();

  plain_type_t<Vec> z = Eigen::VectorXd::Zero(N + 1);
  if (unlikely(N == 0)) {
    return z;
  }

  auto&& y_ref = to_ref(y);
  value_type_t<Vec> sum_w(0);

  // new   
  double d = 0;  // sum of exponentials
  double max_val = 0;
  double max_val_old = 0;

  for (int i = N; i > 0; --i) {
    double n = static_cast<double>(i);
    auto w = y_ref(i - 1) * inv_sqrt(n * (n + 1));
    sum_w += w;

    z.coeffRef(i - 1) += sum_w;
    z.coeffRef(i) -= w * n;
    
    // new
    max_val = max(max_val_old, z.coeff(i));
    d = d * exp(max_val_old - max_val) + exp(z.coeff(i) - max_val);
    max_val_old = max_val;
  }

  // new loop
  for (int i = 0; i < N; ++i) {
   z.coeffRef(i) = exp(z.coeff(i) - max_val) / d;
  }

  lp += -N * log(d) + 0.5 * log(N);

  return z;
 }
```
Linked issue comments: user: login: WardBrian
created_at: 2025-04-01T16:15:58Z
body: I think this would be the first time we ever change a constraint parametrization, so there are some details to work out. I would propose:

1. Renaming the existing `simplex_[constrain, free]` to something like `simplex_stickbreaking_...`
2. Adding the ILR version under the name `simplex_ilr_...`
3. Defining new `simplex_[constrain, free]` functions that just call the `simplex_ilr_...` friends


My reasoning:
Doing (1) lets us keep the existing simplex transform exposed as functions for the user.
Doing (3) actually changes the meaning of `simplex[N]` in the language.
At first, (2) might seem odd, but it lets the user be explicit if they're deciding to manually call our functions. They can either call `simplex_stickbreak_jacobian`, `simplex_ilr_jacobian`, or `simplex_jacobian`, with the idea being that `simplex_jacobian` is always going to be the same as whatever `simplex[N]` does. So if someone wants to keep using the ILR even if we changed the default again in the future, they could use `..._ilr_...`, but if they want the default they can just use `simplex_jacobian`.

How does this sound @spinkney @bob-carpenter?
author_association: MEMBER
user: login: bob-carpenter
created_at: 2025-04-01T19:14:30Z
body: I'm curious where you think people will want the current simplex transform.  @spinkney---do you know of anywhere it outperforms the ILR version?  If it's just backwards compatibility of sampling, that's not something we guarantee release over release.   We will be changing the meaning of `simplex[K]` declarations in the language anyway.

author_association: MEMBER
user: login: spinkney
created_at: 2025-04-01T22:48:54Z
body: > I'm curious where you think people will want the current simplex transform. @spinkney---do you know of anywhere it outperforms the ILR version? If it's just backwards compatibility of sampling, that's not something we guarantee release over release. We will be changing the meaning of `simplex[K]` declarations in the language anyway.

I don't know where the stickbreaking outperforms though it might be more competitive after this change that @WardBrian adds. Regardless, my push to have multiple constraint transform options for the same transformation is that there are cases where one transform is better than another for different models. I'd like users to have the freedom to swap them out. 

I'm thinking that we can tell people that our dev priority is with the default transforms that have been shown to work well in many cases. Transforms which work well in specific cases will be low priority to include. Transforms shown to outperform the default in most cases will be considered to replace the current default. 
author_association: COLLABORATOR
user: login: WardBrian
created_at: 2025-04-02T13:42:51Z
body: My reasoning for having the existing transform still around was motivated by 

1. It’s not particularly hard to keep around, this part of code is relatively slow moving
2. It sets a precedent for cases where there may not be as clear an advantage but we still want to give users a different option than the current default


It’s also possible there are users really relying on specific implementation details who would want to keep updating while not changing too much else of their pipeline, though I don’t imagine that’s a huge group 
author_association: MEMBER
user: login: WardBrian
created_at: 2025-04-02T13:43:55Z
body: On the original topic: I assume we would also want to move the existing row/column stochastic types to the ILR technique?
author_association: MEMBER
user: login: spinkney
created_at: 2025-04-02T13:54:43Z
body: > On the original topic: I assume we would also want to move the existing row/column stochastic types to the ILR technique?

yep!
author_association: COLLABORATOR
user: login: WardBrian
created_at: 2025-04-02T16:14:24Z
body: There were a couple issues with the online softmax in the code in the OP. I believe this fixes them:


```C++
template <typename Vec, typename Lp, require_eigen_vector_t<Vec>* = nullptr,
          require_not_st_var<Vec>* = nullptr,
          require_convertible_t<value_type_t<Vec>, Lp>* = nullptr>
inline plain_type_t<Vec> simplex_constrain(const Vec& y, Lp& lp) {
  using std::log;
  using T = value_type_t<Vec>;
  const auto N = y.size();

  plain_type_t<Vec> z = Eigen::VectorXd::Zero(N + 1);
  if (unlikely(N == 0)) {
    // FIX: size zero input should yield a vector of {1}, not {0}
    z.coeffRef(0) = 1;
    return z;
  }

  auto&& y_ref = to_ref(y);
  T sum_w(0);

  T d(0);  // sum of exponentials
  T max_val(0);
  // FIX: initialize at -inf
  T max_val_old(negative_infinity());

  // this is equivalent to softmax(sum_to_zero_constrain(y))
  // but is more efficient and stable if computed this way

  for (int i = N; i > 0; --i) {
    double n = static_cast<double>(i);
    auto w = y_ref(i - 1) * inv_sqrt(n * (n + 1));
    sum_w += w;

    z.coeffRef(i - 1) += sum_w;
    z.coeffRef(i) -= w * n;


    max_val = max(max_val_old, z.coeff(i));
    d = d * exp(max_val_old - max_val) + exp(z.coeff(i) - max_val);
    max_val_old = max_val;
  }

  // NEW:
  // above loop doesn't reach i==0
  max_val = max(max_val_old, z.coeff(0));
  d = d * exp(max_val_old - max_val) + exp(z.coeff(0) - max_val);

  // FIX: off by one error in original loop end
  for (int i = 0; i <= N; ++i) {
    z.coeffRef(i) = exp(z.coeff(i) - max_val) / d;
  }

  lp += -(N + 1) * (max_val + log(d)) + 0.5 * log(N + 1);

  return z;
}
```
author_association: MEMBER
user: login: spinkney
created_at: 2025-04-02T19:04:38Z
body: > There were a couple issues with the online softmax in the code in the OP. I believe this fixes them:

Awesome! That's what I get for not testing it
author_association: COLLABORATOR

Diff:
diff --git a/stan/math/prim/constraint/simplex_constrain.hpp b/stan/math/prim/constraint/simplex_constrain.hpp
index 852123ddaee..cdae665c1e8 100644
--- a/stan/math/prim/constraint/simplex_constrain.hpp
+++ b/stan/math/prim/constraint/simplex_constrain.hpp
@@ -3,10 +3,11 @@
 
 #include <stan/math/prim/meta.hpp>
 #include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
+#include <stan/math/prim/fun/constants.hpp>
 #include <stan/math/prim/fun/log.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
-#include <stan/math/prim/fun/logit.hpp>
+#include <stan/math/prim/fun/fmax.hpp>
+#include <stan/math/prim/fun/exp.hpp>
+#include <stan/math/prim/fun/inv_sqrt.hpp>
 #include <cmath>
 
 namespace stan {
@@ -18,7 +19,11 @@ namespace math {
  * to 0 that sum to 1.  A vector with (K-1) unconstrained values
  * will produce a simplex of size K.
  *
- * The transform is based on a centered stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This code is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`, but is more efficient and
+ * stable if computed this way thanks to the use of the online
+ * softmax algorithm courtesy of https://arxiv.org/abs/1805.02867.
  *
  * @tparam Vec type of the vector
  * @param y Free vector input of dimensionality K - 1.
@@ -27,20 +32,42 @@ namespace math {
 template <typename Vec, require_eigen_vector_t<Vec>* = nullptr,
           require_not_st_var<Vec>* = nullptr>
 inline plain_type_t<Vec> simplex_constrain(const Vec& y) {
-  // cut & paste simplex_constrain(Eigen::Matrix, T) w/o Jacobian
-  using std::log;
   using T = value_type_t<Vec>;
+  const auto N = y.size();
+
+  plain_type_t<Vec> z = Eigen::VectorXd::Zero(N + 1);
+  if (unlikely(N == 0)) {
+    z.coeffRef(0) = 1;
+    return z;
+  }
+
+  auto&& y_ref = to_ref(y);
+  T sum_w(0);
+
+  T d(0);  // sum of exponentials
+  T max_val(0);
+  T max_val_old(negative_infinity());
 
-  int Km1 = y.size();
-  plain_type_t<Vec> x(Km1 + 1);
-  T stick_len(1.0);
-  for (Eigen::Index k = 0; k < Km1; ++k) {
-    T z_k = inv_logit(y.coeff(k) - log(Km1 - k));
-    x.coeffRef(k) = stick_len * z_k;
-    stick_len -= x.coeff(k);
+  for (int i = N; i > 0; --i) {
+    double n = static_cast<double>(i);
+    auto w = y_ref(i - 1) * inv_sqrt(n * (n + 1));
+    sum_w += w;
+
+    z.coeffRef(i - 1) += sum_w;
+    z.coeffRef(i) -= w * n;
+
+    max_val = fmax(max_val_old, z.coeff(i));
+    d = d * exp(max_val_old - max_val) + exp(z.coeff(i) - max_val);
+    max_val_old = max_val;
   }
-  x.coeffRef(Km1) = stick_len;
-  return x;
+
+  // above loop doesn't reach i==0
+  max_val = fmax(max_val_old, z.coeff(0));
+  d = d * exp(max_val_old - max_val) + exp(z.coeff(0) - max_val);
+
+  z.array() = (z.array() - max_val).exp() / d;
+
+  return z;
 }
 
 /**
@@ -48,8 +75,11 @@ inline plain_type_t<Vec> simplex_constrain(const Vec& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This code is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`, but is more efficient and
+ * stable if computed this way thanks to the use of the online
+ * softmax algorithm courtesy of https://arxiv.org/abs/1805.02867.
  *
  * @tparam Vec type of the vector
  * @tparam Lp A scalar type for the lp argument. The scalar type of Vec should
@@ -62,26 +92,46 @@ template <typename Vec, typename Lp, require_eigen_vector_t<Vec>* = nullptr,
           require_not_st_var<Vec>* = nullptr,
           require_convertible_t<value_type_t<Vec>, Lp>* = nullptr>
 inline plain_type_t<Vec> simplex_constrain(const Vec& y, Lp& lp) {
-  using Eigen::Dynamic;
-  using Eigen::Matrix;
   using std::log;
   using T = value_type_t<Vec>;
+  const auto N = y.size();
 
-  int Km1 = y.size();  // K = Km1 + 1
-  plain_type_t<Vec> x(Km1 + 1);
-  T stick_len(1.0);
-  for (Eigen::Index k = 0; k < Km1; ++k) {
-    double eq_share = -log(Km1 - k);  // = logit(1.0/(Km1 + 1 - k));
-    T adj_y_k = y.coeff(k) + eq_share;
-    T z_k = inv_logit(adj_y_k);
-    x.coeffRef(k) = stick_len * z_k;
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x.coeff(k);  // equivalently *= (1 - z_k);
+  plain_type_t<Vec> z = Eigen::VectorXd::Zero(N + 1);
+  if (unlikely(N == 0)) {
+    z.coeffRef(0) = 1;
+    return z;
   }
-  x.coeffRef(Km1) = stick_len;  // no Jacobian contrib for last dim
-  return x;
+
+  auto&& y_ref = to_ref(y);
+  T sum_w(0);
+
+  T d(0);  // sum of exponentials
+  T max_val(0);
+  T max_val_old(negative_infinity());
+
+  for (int i = N; i > 0; --i) {
+    double n = static_cast<double>(i);
+    auto w = y_ref(i - 1) * inv_sqrt(n * (n + 1));
+    sum_w += w;
+
+    z.coeffRef(i - 1) += sum_w;
+    z.coeffRef(i) -= w * n;
+
+    max_val = fmax(max_val_old, z.coeff(i));
+    d = d * exp(max_val_old - max_val) + exp(z.coeff(i) - max_val);
+    max_val_old = max_val;
+  }
+
+  // above loop doesn't reach i==0
+  max_val = fmax(max_val_old, z.coeff(0));
+  d = d * exp(max_val_old - max_val) + exp(z.coeff(0) - max_val);
+
+  z.array() = (z.array() - max_val).exp() / d;
+
+  // equivalent to z.log().sum() + 0.5 * log(N + 1)
+  lp += -(N + 1) * (max_val + log(d)) + 0.5 * log(N + 1);
+
+  return z;
 }
 
 /**
diff --git a/stan/math/prim/constraint/simplex_free.hpp b/stan/math/prim/constraint/simplex_free.hpp
index fef16fd9112..4c4a27c33bd 100644
--- a/stan/math/prim/constraint/simplex_free.hpp
+++ b/stan/math/prim/constraint/simplex_free.hpp
@@ -5,7 +5,7 @@
 #include <stan/math/prim/err.hpp>
 #include <stan/math/prim/fun/Eigen.hpp>
 #include <stan/math/prim/fun/log.hpp>
-#include <stan/math/prim/fun/logit.hpp>
+#include <stan/math/prim/fun/sqrt.hpp>
 #include <stan/math/prim/fun/to_ref.hpp>
 #include <cmath>
 
@@ -17,8 +17,8 @@ namespace math {
  * the specified simplex.  It applies to a simplex of dimensionality
  * K and produces an unconstrained vector of dimensionality (K-1).
  *
- * <p>The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using isometric log ratio (ILR)
+ * transform
  *
  * @tparam ColVec type of the simplex (must be a column vector)
  * @param x Simplex of dimensionality K.
@@ -28,20 +28,20 @@ namespace math {
  */
 template <typename Vec, require_eigen_vector_t<Vec>* = nullptr>
 inline plain_type_t<Vec> simplex_free(const Vec& x) {
-  using std::log;
   using T = value_type_t<Vec>;
 
   const auto& x_ref = to_ref(x);
   check_simplex("stan::math::simplex_free", "Simplex variable", x_ref);
   Eigen::Index Km1 = x_ref.size() - 1;
   plain_type_t<Vec> y(Km1);
-  T stick_len = x_ref.coeff(Km1);
-  for (Eigen::Index k = Km1; --k >= 0;) {
-    stick_len += x_ref.coeff(k);
-    T z_k = x_ref.coeff(k) / stick_len;
-    y.coeffRef(k) = logit(z_k) + log(Km1 - k);
-    // note: log(Km1 - k) = logit(1.0 / (Km1 + 1 - k));
+
+  T cumsum = 0.0;
+  for (int i = 0; i < Km1; ++i) {
+    cumsum += log(x_ref.coeff(i));
+    double n = static_cast<double>(i + 1);
+    y.coeffRef(i) = (cumsum - n * log(x_ref.coeff(i + 1))) / sqrt(n * (n + 1));
   }
+
   return y;
 }
 
diff --git a/stan/math/prim/constraint/stochastic_column_constrain.hpp b/stan/math/prim/constraint/stochastic_column_constrain.hpp
index 22904978244..58ef35ea0f6 100644
--- a/stan/math/prim/constraint/stochastic_column_constrain.hpp
+++ b/stan/math/prim/constraint/stochastic_column_constrain.hpp
@@ -3,10 +3,6 @@
 
 #include <stan/math/prim/meta.hpp>
 #include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
-#include <stan/math/prim/fun/log.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
-#include <stan/math/prim/fun/logit.hpp>
 #include <stan/math/prim/constraint/simplex_constrain.hpp>
 #include <cmath>
 
@@ -16,7 +12,8 @@ namespace math {
 /**
  * Return a column stochastic matrix.
  *
- * The transform is based on a centered stick-breaking process.
+ * The transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam Mat type of the Matrix
  * @param y Free Matrix input of dimensionality (K - 1, M)
@@ -39,8 +36,8 @@ inline plain_type_t<Mat> stochastic_column_constrain(const Mat& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam Mat type of the Matrix
  * @tparam Lp A scalar type for the lp argument. The scalar type of Mat should
diff --git a/stan/math/prim/constraint/stochastic_row_constrain.hpp b/stan/math/prim/constraint/stochastic_row_constrain.hpp
index 8c7357c9ab9..290f3d3b72f 100644
--- a/stan/math/prim/constraint/stochastic_row_constrain.hpp
+++ b/stan/math/prim/constraint/stochastic_row_constrain.hpp
@@ -3,10 +3,6 @@
 
 #include <stan/math/prim/meta.hpp>
 #include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
-#include <stan/math/prim/fun/log.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
-#include <stan/math/prim/fun/logit.hpp>
 #include <stan/math/prim/constraint/simplex_constrain.hpp>
 #include <cmath>
 
@@ -16,7 +12,8 @@ namespace math {
 /**
  * Return a row stochastic matrix.
  *
- * The transform is based on a centered stick-breaking process.
+ * The transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam Mat type of the Matrix
  * @param y Free Matrix input of dimensionality (N, K - 1).
@@ -27,23 +24,17 @@ template <typename Mat, require_eigen_matrix_dynamic_t<Mat>* = nullptr,
 inline plain_type_t<Mat> stochastic_row_constrain(const Mat& y) {
   auto&& y_ref = to_ref(y);
   const Eigen::Index N = y_ref.rows();
-  int Km1 = y_ref.cols();
-  plain_type_t<Mat> x(N, Km1 + 1);
-  using eigen_arr = Eigen::Array<scalar_type_t<Mat>, -1, 1>;
-  eigen_arr stick_len = eigen_arr::Constant(N, 1.0);
-  for (Eigen::Index k = 0; k < Km1; ++k) {
-    auto z_k = inv_logit(y_ref.array().col(k) - log(Km1 - k));
-    x.array().col(k) = stick_len * z_k;
-    stick_len -= x.array().col(k);
+  plain_type_t<Mat> ret(N, y_ref.cols() + 1);
+  for (Eigen::Index i = 0; i < N; ++i) {
+    ret.row(i) = simplex_constrain(y_ref.row(i));
   }
-  x.array().col(Km1) = stick_len;
-  return x;
+  return ret;
 }
 
 /**
  * Return a row stochastic matrix.
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam Mat type of the matrix
  * @tparam Lp A scalar type for the lp argument. The scalar type of Mat should
@@ -59,21 +50,11 @@ template <typename Mat, typename Lp,
 inline plain_type_t<Mat> stochastic_row_constrain(const Mat& y, Lp& lp) {
   auto&& y_ref = to_ref(y);
   const Eigen::Index N = y_ref.rows();
-  Eigen::Index Km1 = y_ref.cols();
-  plain_type_t<Mat> x(N, Km1 + 1);
-  Eigen::Array<scalar_type_t<Mat>, -1, 1> stick_len
-      = Eigen::Array<scalar_type_t<Mat>, -1, 1>::Constant(N, 1.0);
-  for (Eigen::Index k = 0; k < Km1; ++k) {
-    const auto eq_share = -log(Km1 - k);  // = logit(1.0/(Km1 + 1 - k));
-    auto adj_y_k = (y_ref.array().col(k) + eq_share).eval();
-    auto z_k = inv_logit(adj_y_k);
-    x.array().col(k) = stick_len * z_k;
-    lp += -sum(log1p_exp(adj_y_k)) - sum(log1p_exp(-adj_y_k))
-          + sum(log(stick_len));
-    stick_len -= x.array().col(k);  // equivalently *= (1 - z_k);
+  plain_type_t<Mat> ret(N, y_ref.cols() + 1);
+  for (Eigen::Index i = 0; i < N; ++i) {
+    ret.row(i) = simplex_constrain(y_ref.row(i), lp);
   }
-  x.col(Km1).array() = stick_len;
-  return x;
+  return ret;
 }
 
 /**
diff --git a/stan/math/rev/constraint/simplex_constrain.hpp b/stan/math/rev/constraint/simplex_constrain.hpp
index e81cc53557d..f7c4534b352 100644
--- a/stan/math/rev/constraint/simplex_constrain.hpp
+++ b/stan/math/rev/constraint/simplex_constrain.hpp
@@ -1,16 +1,13 @@
 #ifndef STAN_MATH_REV_CONSTRAINT_SIMPLEX_CONSTRAIN_HPP
 #define STAN_MATH_REV_CONSTRAINT_SIMPLEX_CONSTRAIN_HPP
 
+#include <stan/math/prim/fun/Eigen.hpp>
 #include <stan/math/rev/meta.hpp>
 #include <stan/math/rev/core/reverse_pass_callback.hpp>
 #include <stan/math/rev/core/arena_matrix.hpp>
-#include <stan/math/rev/fun/value_of.hpp>
-#include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
+#include <stan/math/rev/constraint/sum_to_zero_constrain.hpp>
+#include <stan/math/prim/constraint/simplex_constrain.hpp>
 #include <cmath>
-#include <tuple>
-#include <vector>
 
 namespace stan {
 namespace math {
@@ -21,7 +18,9 @@ namespace math {
  * to 0 that sum to 1.  A vector with (K-1) unconstrained values
  * will produce a simplex of size K.
  *
- * The transform is based on a centered stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T Type of vector to constrain
  * @param y Free vector input of dimensionality K - 1
@@ -31,38 +30,24 @@ template <typename T, require_rev_col_vector_t<T>* = nullptr>
 inline auto simplex_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    arena_z.coeffRef(k) = inv_logit(arena_y.val().coeff(k) - log_N_minus_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val());
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    auto&& res_val = arena_x.val();
+
+    // backprop for softmax
+    Eigen::VectorXd x_pre_softmax_adj = -res_val * arena_x.adj().dot(res_val)
+                                        + res_val.cwiseProduct(arena_x.adj());
+
+    // backprop for sum_to_zero_constrain
+    internal::sum_to_zero_vector_backprop(arena_y.adj(), x_pre_softmax_adj);
   });
 
   return ret_type(arena_x);
@@ -73,8 +58,9 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
@@ -82,50 +68,35 @@ inline auto simplex_constrain(const T& y) {
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    auto&& res_val = arena_x.val();
+
+    // backprop for log jacobian contribution to log density is equivalent to
+    // arena_x.adj().array() += lp.adj() / res_val.array();
+    // but is folded into the following to avoid needing to modify the adjoints
+    // in-place
+
+    // backprop for softmax
+    Eigen::VectorXd x_pre_softmax_adj
+        = -res_val * (arena_x.adj().dot(res_val) + res_val.size() * lp.adj())
+          + (res_val.cwiseProduct(arena_x.adj()).array() + lp.adj()).matrix();
+
+    // backprop for sum_to_zero_constrain
+    internal::sum_to_zero_vector_backprop(arena_y.adj(), x_pre_softmax_adj);
   });
 
   return ret_type(arena_x);
diff --git a/stan/math/rev/constraint/stochastic_column_constrain.hpp b/stan/math/rev/constraint/stochastic_column_constrain.hpp
index 02970513ff2..725b5483541 100644
--- a/stan/math/rev/constraint/stochastic_column_constrain.hpp
+++ b/stan/math/rev/constraint/stochastic_column_constrain.hpp
@@ -1,23 +1,22 @@
 #ifndef STAN_MATH_REV_CONSTRAINT_STOCHASTIC_COLUMN_CONSTRAIN_HPP
 #define STAN_MATH_REV_CONSTRAINT_STOCHASTIC_COLUMN_CONSTRAIN_HPP
 
+#include <stan/math/prim/fun/Eigen.hpp>
 #include <stan/math/rev/meta.hpp>
 #include <stan/math/rev/core/reverse_pass_callback.hpp>
 #include <stan/math/rev/core/arena_matrix.hpp>
 #include <stan/math/rev/fun/value_of.hpp>
-#include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
+#include <stan/math/prim/constraint/stochastic_column_constrain.hpp>
+#include <stan/math/rev/constraint/sum_to_zero_constrain.hpp>
 #include <cmath>
-#include <tuple>
-#include <vector>
 
 namespace stan {
 namespace math {
 
 /**
  * Return a column stochastic matrix.
- * The transform is based on a centered stick-breaking process.
+ * The transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam T Type of matrix to constrain
  * @param y Free matrix input of dimensionality (K - 1, M)
@@ -26,44 +25,37 @@ namespace math {
 template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_column_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  using eigen_mat_rowmajor
-      = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
-  arena_t<eigen_mat_rowmajor> x_val(N + 1, M);
+
+  const auto N = y.rows();
+  const auto M = y.cols();
+  arena_t<T> arena_y = y;
+
+  arena_t<ret_type> arena_x = stochastic_column_constrain(arena_y.val_op());
+
   if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
-  arena_t<change_eigen_options_t<T, Eigen::RowMajor>> arena_y = y;
-  arena_t<eigen_mat_rowmajor> arena_z(N, M);
-  using arr_vec = Eigen::Array<double, 1, -1>;
-  arr_vec stick_len = arr_vec::Constant(M, 1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    const double log_N_minus_k = std::log(N - k);
-    arena_z.row(k)
-        = inv_logit(arena_y.array().row(k).val_op() - log_N_minus_k).matrix();
-    x_val.row(k) = stick_len.array() * arena_z.array().row(k);
-    stick_len -= x_val.array().row(k);
+    return arena_x;
   }
-  x_val.row(N) = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    const Eigen::Index N = arena_y.rows();
-    auto arena_x_arr = arena_x.array();
-    auto arena_y_arr = arena_y.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x.array().row(N).val().eval();
-    auto stick_len_adj = arena_x.array().row(N).adj().eval();
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x_arr.row(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.row(k).val();
-      stick_len_adj += arena_x_arr.row(k).adj() * arena_z_arr.row(k);
-      auto arena_z_adj = arena_x_arr.row(k).adj() * stick_len_val;
-      arena_y_arr.row(k).adj()
-          += arena_z_adj * arena_z_arr.row(k) * (1.0 - arena_z_arr.row(k));
+
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto M = arena_y.cols();
+
+    auto&& x_val = arena_x.val_op();
+    auto&& x_adj = arena_x.adj_op();
+
+    Eigen::VectorXd x_pre_softmax_adj(x_val.rows());
+    for (Eigen::Index i = 0; i < M; ++i) {
+      // backprop for softmax
+      x_pre_softmax_adj.noalias()
+          = -x_val.col(i) * x_adj.col(i).dot(x_val.col(i))
+            + x_val.col(i).cwiseProduct(x_adj.col(i));
+
+      // backprop for sum_to_zero_constrain
+      internal::sum_to_zero_vector_backprop(arena_y.col(i).adj(),
+                                            x_pre_softmax_adj);
     }
   });
-  return ret_type(arena_x);
+
+  return arena_x;
 }
 
 /**
@@ -71,8 +63,8 @@ inline plain_type_t<T> stochastic_column_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam T type of the matrix to constrain
  * @param y Free matrix input of dimensionality N, K.
@@ -83,51 +75,44 @@ template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_column_constrain(const T& y,
                                                    scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  using eigen_mat_rowmajor
-      = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
-  arena_t<eigen_mat_rowmajor> x_val(N + 1, M);
+
+  const auto N = y.rows();
+  const auto M = y.cols();
+  arena_t<T> arena_y = y;
+
+  double lp_val = 0;
+  arena_t<ret_type> arena_x
+      = stochastic_column_constrain(arena_y.val_op(), lp_val);
+  lp += lp_val;
+
   if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
+    return arena_x;
   }
-  arena_t<change_eigen_options_t<T, Eigen::RowMajor>> arena_y = y;
-  arena_t<eigen_mat_rowmajor> arena_z(N, M);
-  using arr_vec = Eigen::Array<double, 1, -1>;
-  arr_vec stick_len = arr_vec::Constant(M, 1.0);
-  arr_vec adj_y_k(N);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    adj_y_k = arena_y.array().row(k).val() - log_N_minus_k;
-    arena_z.array().row(k) = inv_logit(adj_y_k);
-    x_val.array().row(k) = stick_len * arena_z.array().row(k);
-    lp += sum(log(stick_len)) - sum(log1p_exp(-adj_y_k))
-          - sum(log1p_exp(adj_y_k));
-    stick_len -= x_val.array().row(k);
-  }
-  x_val.array().row(N) = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    const Eigen::Index N = arena_y.rows();
-    auto arena_x_arr = arena_x.array();
-    auto arena_y_arr = arena_y.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x.array().row(N).val().eval();
-    auto stick_len_adj = arena_x.array().row(N).adj().eval();
-    for (Eigen::Index k = N; k-- > 0;) {
-      const double log_N_minus_k = std::log(N - k);
-      arena_x_arr.row(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.row(k).val();
-      stick_len_adj += lp.adj() / stick_len_val
-                       + arena_x_arr.row(k).adj() * arena_z_arr.row(k);
-      auto adj_y_k = arena_y_arr.row(k).val() - log_N_minus_k;
-      auto arena_z_adj = arena_x_arr.row(k).adj() * stick_len_val;
-      arena_y_arr.row(k).adj()
-          += -(lp.adj() * inv_logit(adj_y_k)) + lp.adj() * inv_logit(-adj_y_k)
-             + arena_z_adj * arena_z_arr.row(k) * (1.0 - arena_z_arr.row(k));
+
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto M = arena_y.cols();
+
+    auto&& x_val = arena_x.val_op();
+    auto&& x_adj = arena_x.adj_op();
+
+    const auto x_val_rows = x_val.rows();
+
+    Eigen::VectorXd x_pre_softmax_adj(x_val.rows());
+    for (Eigen::Index i = 0; i < M; ++i) {
+      // backprop for softmax
+      x_pre_softmax_adj.noalias()
+          = -x_val.col(i)
+                * (x_adj.col(i).dot(x_val.col(i)) + lp.adj() * x_val_rows)
+            + (x_val.col(i).cwiseProduct(x_adj.col(i)).array() + lp.adj())
+                  .matrix();
+
+      // backprop for sum_to_zero_constrain
+      internal::sum_to_zero_vector_backprop(arena_y.col(i).adj(),
+                                            x_pre_softmax_adj);
     }
   });
-  return ret_type(arena_x);
+
+  return arena_x;
 }
 
 }  // namespace math
diff --git a/stan/math/rev/constraint/stochastic_row_constrain.hpp b/stan/math/rev/constraint/stochastic_row_constrain.hpp
index 654681107a7..bf9acd724d5 100644
--- a/stan/math/rev/constraint/stochastic_row_constrain.hpp
+++ b/stan/math/rev/constraint/stochastic_row_constrain.hpp
@@ -1,16 +1,14 @@
 #ifndef STAN_MATH_REV_CONSTRAINT_STOCHASTIC_ROW_CONSTRAIN_HPP
 #define STAN_MATH_REV_CONSTRAINT_STOCHASTIC_ROW_CONSTRAIN_HPP
 
+#include <stan/math/prim/fun/Eigen.hpp>
 #include <stan/math/rev/meta.hpp>
 #include <stan/math/rev/core/reverse_pass_callback.hpp>
 #include <stan/math/rev/core/arena_matrix.hpp>
 #include <stan/math/rev/fun/value_of.hpp>
-#include <stan/math/prim/fun/Eigen.hpp>
-#include <stan/math/prim/fun/inv_logit.hpp>
-#include <stan/math/prim/fun/log1p_exp.hpp>
+#include <stan/math/prim/constraint/stochastic_row_constrain.hpp>
+#include <stan/math/rev/constraint/sum_to_zero_constrain.hpp>
 #include <cmath>
-#include <tuple>
-#include <vector>
 
 namespace stan {
 namespace math {
@@ -23,43 +21,39 @@ namespace math {
  * @return Matrix with Simplexes along the rows of dimensionality (N, K)
  */
 template <typename T, require_rev_matrix_t<T>* = nullptr>
-inline plain_type_t<T> stochastic_row_constrain(const T& y) {
+inline auto stochastic_row_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  arena_t<Eigen::MatrixXd> x_val(N, M + 1);
-  if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
+
+  const auto N = y.rows();
+  const auto M = y.cols();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::MatrixXd> arena_z(N, M);
-  Eigen::Array<double, -1, 1> stick_len = Eigen::Array<double, -1, 1>::Ones(N);
-  for (Eigen::Index j = 0; j < M; ++j) {
-    double log_N_minus_k = std::log(M - j);
-    arena_z.col(j).array()
-        = inv_logit((arena_y.col(j).val_op().array() - log_N_minus_k).matrix());
-    x_val.col(j).array() = stick_len * arena_z.col(j).array();
-    stick_len -= x_val.col(j).array();
+
+  arena_t<ret_type> arena_x = stochastic_row_constrain(arena_y.val_op());
+
+  if (unlikely(N == 0 || M == 0)) {
+    return arena_x;
   }
-  x_val.col(M).array() = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    const Eigen::Index M = arena_y.cols();
-    auto arena_y_arr = arena_y.array();
-    auto arena_x_arr = arena_x.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val_arr = arena_x_arr.col(M).val_op().eval();
-    auto stick_len_adj_arr = arena_x_arr.col(M).adj_op().eval();
-    for (Eigen::Index k = M; k-- > 0;) {
-      arena_x_arr.col(k).adj() -= stick_len_adj_arr;
-      stick_len_val_arr += arena_x_arr.col(k).val_op();
-      stick_len_adj_arr += arena_x_arr.col(k).adj_op() * arena_z_arr.col(k);
-      arena_y_arr.col(k).adj() += arena_x_arr.adj_op().col(k)
-                                  * stick_len_val_arr * arena_z_arr.col(k)
-                                  * (1.0 - arena_z_arr.col(k));
+
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto N = arena_y.rows();
+
+    auto&& x_val = arena_x.val_op();
+    auto&& x_adj = arena_x.adj_op();
+
+    Eigen::VectorXd x_pre_softmax_adj(x_val.cols());
+    for (Eigen::Index i = 0; i < N; ++i) {
+      // backprop for softmax
+      x_pre_softmax_adj.noalias()
+          = -x_val.row(i) * x_adj.row(i).dot(x_val.row(i))
+            + x_val.row(i).cwiseProduct(x_adj.row(i));
+
+      // backprop for sum_to_zero_constrain
+      internal::sum_to_zero_vector_backprop(arena_y.row(i).adj(),
+                                            x_pre_softmax_adj);
     }
   });
-  return ret_type(arena_x);
+
+  return arena_x;
 }
 
 /**
@@ -67,8 +61,8 @@ inline plain_type_t<T> stochastic_row_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform
  *
  * @tparam T type of the matrix to constrain
  * @param y Free matrix input of dimensionality (N, K).
@@ -79,47 +73,44 @@ template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_row_constrain(const T& y,
                                                 scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  arena_t<Eigen::MatrixXd> x_val(N, M + 1);
-  if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
+
+  const auto N = y.rows();
+  const auto M = y.cols();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::MatrixXd> arena_z(N, M);
-  Eigen::Array<double, -1, 1> stick_len = Eigen::Array<double, -1, 1>::Ones(N);
-  for (Eigen::Index j = 0; j < M; ++j) {
-    double log_N_minus_k = std::log(M - j);
-    auto adj_y_k = arena_y.col(j).val_op().array() - log_N_minus_k;
-    arena_z.col(j).array() = inv_logit(adj_y_k);
-    x_val.col(j).array() = stick_len * arena_z.col(j).array();
-    lp += sum(log(stick_len)) - sum(log1p_exp(-adj_y_k))
-          - sum(log1p_exp(adj_y_k));
-    stick_len -= x_val.col(j).array();
+
+  double lp_val = 0;
+  arena_t<ret_type> arena_x
+      = stochastic_row_constrain(arena_y.val_op(), lp_val);
+  lp += lp_val;
+
+  if (unlikely(N == 0 || M == 0)) {
+    return arena_x;
   }
-  x_val.col(M).array() = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    const Eigen::Index M = arena_y.cols();
-    auto arena_y_arr = arena_y.array();
-    auto arena_x_arr = arena_x.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x_arr.col(M).val_op().eval();
-    auto stick_len_adj = arena_x_arr.col(M).adj_op().eval();
-    for (Eigen::Index k = M; k-- > 0;) {
-      const double log_N_minus_k = std::log(M - k);
-      arena_x_arr.col(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.col(k).val_op();
-      stick_len_adj += lp.adj() / stick_len_val
-                       + arena_x_arr.adj_op().col(k) * arena_z_arr.col(k);
-      auto adj_y_k = arena_y_arr.col(k).val_op() - log_N_minus_k;
-      arena_y_arr.col(k).adj()
-          += -(lp.adj() * inv_logit(adj_y_k)) + lp.adj() * inv_logit(-adj_y_k)
-             + arena_x_arr.col(k).adj_op() * stick_len_val * arena_z_arr.col(k)
-                   * (1.0 - arena_z_arr.col(k));
+
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto N = arena_y.rows();
+
+    auto&& x_val = arena_x.val_op();
+    auto&& x_adj = arena_x.adj_op();
+
+    const auto x_val_cols = x_val.cols();
+
+    Eigen::VectorXd x_pre_softmax_adj(x_val.cols());
+    for (Eigen::Index i = 0; i < N; ++i) {
+      // backprop for softmax
+      x_pre_softmax_adj.noalias()
+          = -x_val.row(i)
+                * (x_adj.row(i).dot(x_val.row(i)) + lp.adj() * x_val_cols)
+            + (x_val.row(i).cwiseProduct(x_adj.row(i)).array() + lp.adj())
+                  .matrix();
+
+      // backprop for sum_to_zero_constrain
+      internal::sum_to_zero_vector_backprop(arena_y.row(i).adj(),
+                                            x_pre_softmax_adj);
     }
   });
-  return ret_type(arena_x);
+
+  return arena_x;
 }
 
 }  // namespace math
diff --git a/stan/math/rev/constraint/sum_to_zero_constrain.hpp b/stan/math/rev/constraint/sum_to_zero_constrain.hpp
index feed839228e..5e62b560894 100644
--- a/stan/math/rev/constraint/sum_to_zero_constrain.hpp
+++ b/stan/math/rev/constraint/sum_to_zero_constrain.hpp
@@ -9,12 +9,43 @@
 #include <stan/math/prim/fun/sqrt.hpp>
 #include <stan/math/prim/constraint/sum_to_zero_constrain.hpp>
 #include <cmath>
-#include <tuple>
-#include <vector>
 
 namespace stan {
 namespace math {
 
+namespace internal {
+
+/**
+ * The reverse pass backprop for the sum_to_zero_constrain on
+ * vectors. This is separated out so it can also be called by
+ * simplex_constrain.
+ *
+ * @tparam T type of the adjoint vector
+ * @param y_adj The adjoint of the free vector (size N)
+ * @param z_adj The adjoint of the zero-sum vector (size N + 1)
+ */
+template <typename T>
+void sum_to_zero_vector_backprop(T&& y_adj, const Eigen::VectorXd& z_adj) {
+  const auto N = y_adj.size();
+
+  double sum_u_adj = 0;
+  for (int i = 0; i < N; ++i) {
+    double n = static_cast<double>(i + 1);
+
+    // adjoint of the reverse cumulative sum computed in the forward mode
+    sum_u_adj += z_adj.coeff(i);
+
+    // adjoint of the offset subtraction
+    double v_adj = -z_adj.coeff(i + 1) * n;
+
+    double w_adj = v_adj + sum_u_adj;
+
+    y_adj.coeffRef(i) += w_adj / sqrt(n * (n + 1));
+  }
+}
+
+}  // namespace internal
+
 /**
  * Return a vector with sum zero corresponding to the specified
  * free vector.
@@ -49,22 +80,7 @@ inline auto sum_to_zero_constrain(T&& y) {
   arena_t<ret_type> arena_z = sum_to_zero_constrain(arena_y.val());
 
   reverse_pass_callback([arena_y, arena_z]() mutable {
-    const auto N = arena_y.size();
-
-    double sum_u_adj = 0;
-    for (int i = 0; i < N; ++i) {
-      double n = static_cast<double>(i + 1);
-
-      // adjoint of the reverse cumulative sum computed in the forward mode
-      sum_u_adj += arena_z.adj().coeff(i);
-
-      // adjoint of the offset subtraction
-      double v_adj = -arena_z.adj().coeff(i + 1) * n;
-
-      double w_adj = v_adj + sum_u_adj;
-
-      arena_y.adj().coeffRef(i) += w_adj / sqrt(n * (n + 1));
-    }
+    internal::sum_to_zero_vector_backprop(arena_y.adj(), arena_z.adj());
   });
 
   return arena_z;
diff --git a/test/unit/math/prim/constraint/simplex_transform_test.cpp b/test/unit/math/prim/constraint/simplex_transform_test.cpp
index afbd6009160..72535426f1c 100644
--- a/test/unit/math/prim/constraint/simplex_transform_test.cpp
+++ b/test/unit/math/prim/constraint/simplex_transform_test.cpp
@@ -2,6 +2,18 @@
 #include <test/unit/util.hpp>
 #include <gtest/gtest.h>
 
+TEST(prob_transform, simplex_0) {
+  using Eigen::Dynamic;
+  using Eigen::Matrix;
+  double lp = 0;
+  Matrix<double, Dynamic, 1> x(0);
+  auto y = stan::math::simplex_constrain<false>(x, lp);
+  EXPECT_EQ(1, y.size());
+  EXPECT_FLOAT_EQ(1.0, y[0]);
+  auto xrt = stan::math::simplex_free(y);
+  EXPECT_EQ(0, xrt.size());
+}
+
 TEST(prob_transform, simplex_rt0) {
   using Eigen::Dynamic;
   using Eigen::Matrix;


Reviews:
user: login: SteveBronder
body: Good with a few small suggestions
state: DISMISSED
author_association: COLLABORATOR
submitted_at: 2025-04-07T17:31:12Z
user: login: WardBrian
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T18:28:41Z
user: login: SteveBronder
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-07T19:53:06Z
user: login: SteveBronder
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-07T19:55:37Z
user: login: WardBrian
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T19:59:05Z
user: login: SteveBronder
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-07T20:01:40Z
user: login: WardBrian
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-07T20:27:41Z
user: login: SteveBronder
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T17:08:23Z
user: login: WardBrian
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-11T18:26:18Z
user: login: WardBrian
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-04-24T18:47:53Z
user: login: SteveBronder
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-28T18:07:25Z

Review comments:
diff_hunk: @@ -27,29 +32,56 @@ namespace math {
 template <typename Vec, require_eigen_vector_t<Vec>* = nullptr,
           require_not_st_var<Vec>* = nullptr>
 inline plain_type_t<Vec> simplex_constrain(const Vec& y) {
-  // cut & paste simplex_constrain(Eigen::Matrix, T) w/o Jacobian
-  using std::log;
   using T = value_type_t<Vec>;
+  const auto N = y.size();
 
-  int Km1 = y.size();
-  plain_type_t<Vec> x(Km1 + 1);
-  T stick_len(1.0);
-  for (Eigen::Index k = 0; k < Km1; ++k) {
-    T z_k = inv_logit(y.coeff(k) - log(Km1 - k));
-    x.coeffRef(k) = stick_len * z_k;
-    stick_len -= x.coeff(k);
+  plain_type_t<Vec> z = Eigen::VectorXd::Zero(N + 1);
+  if (unlikely(N == 0)) {
+    z.coeffRef(0) = 1;
+    return z;
   }
-  x.coeffRef(Km1) = stick_len;
-  return x;
+
+  auto&& y_ref = to_ref(y);
+  T sum_w(0);
+
+  T d(0);  // sum of exponentials
+  T max_val(0);
+  T max_val_old(negative_infinity());
+
+  for (int i = N; i > 0; --i) {
+    double n = static_cast<double>(i);
+    auto w = y_ref(i - 1) * inv_sqrt(n * (n + 1));
+    sum_w += w;
+
+    z.coeffRef(i - 1) += sum_w;
+    z.coeffRef(i) -= w * n;
+
+    max_val = fmax(max_val_old, z.coeff(i));
+    d = d * exp(max_val_old - max_val) + exp(z.coeff(i) - max_val);
+    max_val_old = max_val;
+  }
+
+  // above loop doesn't reach i==0
+  max_val = fmax(max_val_old, z.coeff(0));
+  d = d * exp(max_val_old - max_val) + exp(z.coeff(0) - max_val);
+
+  for (int i = 0; i <= N; ++i) {
+    z.coeffRef(i) = exp(z.coeff(i) - max_val) / d;
+  }
path: stan/math/prim/constraint/simplex_constrain.hpp
user: login: SteveBronder
body: ```suggestion
   z.array() = (z.array() - max_val).exp() / d;
```
created_at: 2025-04-07T17:04:56Z
author_association: COLLABORATOR
diff_hunk: @@ -26,53 +27,45 @@ namespace math {
 template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_column_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  using eigen_mat_rowmajor
-      = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
-  arena_t<eigen_mat_rowmajor> x_val(N + 1, M);
+
+  const auto N = y.rows();
+  const auto M = y.cols();
+  arena_t<T> arena_y = y;
+
+  arena_t<ret_type> arena_x = stochastic_column_constrain(arena_y.val_op());
+
   if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
-  arena_t<change_eigen_options_t<T, Eigen::RowMajor>> arena_y = y;
-  arena_t<eigen_mat_rowmajor> arena_z(N, M);
-  using arr_vec = Eigen::Array<double, 1, -1>;
-  arr_vec stick_len = arr_vec::Constant(M, 1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    const double log_N_minus_k = std::log(N - k);
-    arena_z.row(k)
-        = inv_logit(arena_y.array().row(k).val_op() - log_N_minus_k).matrix();
-    x_val.row(k) = stick_len.array() * arena_z.array().row(k);
-    stick_len -= x_val.array().row(k);
+    return arena_x;
   }
-  x_val.row(N) = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    const Eigen::Index N = arena_y.rows();
-    auto arena_x_arr = arena_x.array();
-    auto arena_y_arr = arena_y.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x.array().row(N).val().eval();
-    auto stick_len_adj = arena_x.array().row(N).adj().eval();
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x_arr.row(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.row(k).val();
-      stick_len_adj += arena_x_arr.row(k).adj() * arena_z_arr.row(k);
-      auto arena_z_adj = arena_x_arr.row(k).adj() * stick_len_val;
-      arena_y_arr.row(k).adj()
-          += arena_z_adj * arena_z_arr.row(k) * (1.0 - arena_z_arr.row(k));
+
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto M = arena_y.cols();
+
+    const auto& x_val = to_ref(arena_x.val_op());
+    const auto& x_adj = to_ref(arena_x.adj_op());
path: stan/math/rev/constraint/stochastic_column_constrain.hpp
user: login: SteveBronder
body: `arena_x` should not be an expression so you don't need `to_ref` here
```suggestion
    auto&& x_val = arena_x.val_op();
    auto&& x_adj = arena_x.adj_op();
```
created_at: 2025-04-07T17:14:13Z
author_association: COLLABORATOR
diff_hunk: @@ -26,53 +27,45 @@ namespace math {
 template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_column_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  using eigen_mat_rowmajor
-      = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
-  arena_t<eigen_mat_rowmajor> x_val(N + 1, M);
+
+  const auto N = y.rows();
+  const auto M = y.cols();
+  arena_t<T> arena_y = y;
+
+  arena_t<ret_type> arena_x = stochastic_column_constrain(arena_y.val_op());
+
   if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
-  arena_t<change_eigen_options_t<T, Eigen::RowMajor>> arena_y = y;
-  arena_t<eigen_mat_rowmajor> arena_z(N, M);
-  using arr_vec = Eigen::Array<double, 1, -1>;
-  arr_vec stick_len = arr_vec::Constant(M, 1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    const double log_N_minus_k = std::log(N - k);
-    arena_z.row(k)
-        = inv_logit(arena_y.array().row(k).val_op() - log_N_minus_k).matrix();
-    x_val.row(k) = stick_len.array() * arena_z.array().row(k);
-    stick_len -= x_val.array().row(k);
+    return arena_x;
   }
-  x_val.row(N) = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    const Eigen::Index N = arena_y.rows();
-    auto arena_x_arr = arena_x.array();
-    auto arena_y_arr = arena_y.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x.array().row(N).val().eval();
-    auto stick_len_adj = arena_x.array().row(N).adj().eval();
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x_arr.row(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.row(k).val();
-      stick_len_adj += arena_x_arr.row(k).adj() * arena_z_arr.row(k);
-      auto arena_z_adj = arena_x_arr.row(k).adj() * stick_len_val;
-      arena_y_arr.row(k).adj()
-          += arena_z_adj * arena_z_arr.row(k) * (1.0 - arena_z_arr.row(k));
+
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto M = arena_y.cols();
+
+    const auto& x_val = to_ref(arena_x.val_op());
+    const auto& x_adj = to_ref(arena_x.adj_op());
+
+    for (Eigen::Index i = 0; i < M; ++i) {
+      // backprop for softmax
+      Eigen::VectorXd x_pre_softmax_adj
+          = -x_val.col(i) * x_adj.col(i).dot(x_val.col(i))
+            + x_val.col(i).cwiseProduct(x_adj.col(i));
path: stan/math/rev/constraint/stochastic_column_constrain.hpp
user: login: SteveBronder
body: You can just make the memory `x_pre_softmax_adj` once
```suggestion
    Eigen::VectorXd x_pre_softmax_adj(x_val.rows());
    for (Eigen::Index i = 0; i < M; ++i) {
      // backprop for softmax
      x_pre_softmax_adj.noalias()
          = -x_val.col(i) * x_adj.col(i).dot(x_val.col(i))
            + x_val.col(i).cwiseProduct(x_adj.col(i));
```
created_at: 2025-04-07T17:16:10Z
author_association: COLLABORATOR
diff_hunk: @@ -26,53 +27,45 @@ namespace math {
 template <typename T, require_rev_matrix_t<T>* = nullptr>
 inline plain_type_t<T> stochastic_column_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
-  const Eigen::Index N = y.rows();
-  const Eigen::Index M = y.cols();
-  using eigen_mat_rowmajor
-      = Eigen::Matrix<double, Eigen::Dynamic, Eigen::Dynamic, Eigen::RowMajor>;
-  arena_t<eigen_mat_rowmajor> x_val(N + 1, M);
+
+  const auto N = y.rows();
+  const auto M = y.cols();
+  arena_t<T> arena_y = y;
+
+  arena_t<ret_type> arena_x = stochastic_column_constrain(arena_y.val_op());
+
   if (unlikely(N == 0 || M == 0)) {
-    return ret_type(x_val);
-  }
-  arena_t<change_eigen_options_t<T, Eigen::RowMajor>> arena_y = y;
-  arena_t<eigen_mat_rowmajor> arena_z(N, M);
-  using arr_vec = Eigen::Array<double, 1, -1>;
-  arr_vec stick_len = arr_vec::Constant(M, 1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    const double log_N_minus_k = std::log(N - k);
-    arena_z.row(k)
-        = inv_logit(arena_y.array().row(k).val_op() - log_N_minus_k).matrix();
-    x_val.row(k) = stick_len.array() * arena_z.array().row(k);
-    stick_len -= x_val.array().row(k);
+    return arena_x;
   }
-  x_val.row(N) = stick_len;
-  arena_t<ret_type> arena_x = x_val;
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    const Eigen::Index N = arena_y.rows();
-    auto arena_x_arr = arena_x.array();
-    auto arena_y_arr = arena_y.array();
-    auto arena_z_arr = arena_z.array();
-    auto stick_len_val = arena_x.array().row(N).val().eval();
-    auto stick_len_adj = arena_x.array().row(N).adj().eval();
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x_arr.row(k).adj() -= stick_len_adj;
-      stick_len_val += arena_x_arr.row(k).val();
-      stick_len_adj += arena_x_arr.row(k).adj() * arena_z_arr.row(k);
-      auto arena_z_adj = arena_x_arr.row(k).adj() * stick_len_val;
-      arena_y_arr.row(k).adj()
-          += arena_z_adj * arena_z_arr.row(k) * (1.0 - arena_z_arr.row(k));
+
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto M = arena_y.cols();
+
+    const auto& x_val = to_ref(arena_x.val_op());
+    const auto& x_adj = to_ref(arena_x.adj_op());
+
+    for (Eigen::Index i = 0; i < M; ++i) {
+      // backprop for softmax
+      Eigen::VectorXd x_pre_softmax_adj
+          = -x_val.col(i) * x_adj.col(i).dot(x_val.col(i))
+            + x_val.col(i).cwiseProduct(x_adj.col(i));
path: stan/math/rev/constraint/stochastic_column_constrain.hpp
user: login: SteveBronder
body: You could also change `sum_to_zero_vector_backprop` to just take in a matrix 
created_at: 2025-04-07T17:17:55Z
author_association: COLLABORATOR
diff_hunk: @@ -15,6 +15,35 @@
 namespace stan {
 namespace math {
 
+namespace internal {
+
+/**
+ * The reverse pass backprop for the sum_to_zero_constrain on
+ * vectors. This is separated out so it can also be called by
+ * simplex_constrain.
+ */
+template <typename T>
+void sum_to_zero_vector_backprop(T&& y_adj, const Eigen::VectorXd& z_adj) {
+  const auto N = y_adj.size();
path: stan/math/rev/constraint/sum_to_zero_constrain.hpp
user: login: SteveBronder
body: I'd leave a small comment so people know `z_adj` is `y_adj.size() + 1`
created_at: 2025-04-07T17:20:45Z
author_association: COLLABORATOR
diff_hunk: @@ -15,6 +15,35 @@
 namespace stan {
 namespace math {
 
+namespace internal {
+
+/**
+ * The reverse pass backprop for the sum_to_zero_constrain on
+ * vectors. This is separated out so it can also be called by
+ * simplex_constrain.
+ */
+template <typename T>
+void sum_to_zero_vector_backprop(T&& y_adj, const Eigen::VectorXd& z_adj) {
+  const auto N = y_adj.size();
path: stan/math/rev/constraint/sum_to_zero_constrain.hpp
user: login: WardBrian
body: I added `@param`s to the docstring and put the sizes there
created_at: 2025-04-07T18:28:41Z
author_association: MEMBER
diff_hunk: @@ -31,38 +32,24 @@ template <typename T, require_rev_col_vector_t<T>* = nullptr>
 inline auto simplex_constrain(const T& y) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    arena_z.coeffRef(k) = inv_logit(arena_y.val().coeff(k) - log_N_minus_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val());
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: SteveBronder
body: You should not need `to_ref` on any of the `arena_x` vals
created_at: 2025-04-07T19:53:06Z
author_association: COLLABORATOR
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: SteveBronder
body: ~This is kind of confusing, why are we accumulating into the return values adjoints here?~
EDIT: nvm I see this is the rhs of the jacobian adjustment. Though I can't think of anywhere else we change the outputs adjoints in the reverse pass of it's function. I need to think about this
created_at: 2025-04-07T19:55:37Z
author_association: COLLABORATOR
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: WardBrian
body: We can avoid it by making a copy of the adjoints if we want, though I'm pretty sure that nothing else in the AD graph can depend on the adjoints of this by this point
created_at: 2025-04-07T19:59:05Z
author_association: MEMBER
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: SteveBronder
body: Can you just use `auto lp_adj =` so then it's just an expression and you don't need any memory
created_at: 2025-04-07T20:01:40Z
author_association: COLLABORATOR
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: WardBrian
body: Can you elaborate?
created_at: 2025-04-07T20:27:41Z
author_association: MEMBER
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: SteveBronder
body: I mean 
```
 auto lp_adj = lp.adj() / res_val.array();
```

then in the below you can do `res_val * (arena_x.adj() + lp_adj).dot(rev_val)`
created_at: 2025-04-11T17:08:23Z
author_association: COLLABORATOR
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: WardBrian
body: `arena_x.adj()` is used two places, so you'd be duplicating the work, no?
created_at: 2025-04-11T18:26:18Z
author_association: MEMBER
diff_hunk: @@ -73,59 +60,42 @@ inline auto simplex_constrain(const T& y) {
  * and increment the specified log probability reference with
  * the log absolute Jacobian determinant of the transform.
  *
- * The simplex transform is defined through a centered
- * stick-breaking process.
+ * The simplex transform is defined using the inverse of the
+ * isometric log ratio (ILR) transform. This is equivalent to
+ * `softmax(sum_to_zero_constrain(y))`.
  *
  * @tparam T type of the vector to constrain
  * @param y Free vector input of dimensionality N.
  * @param lp Log probability reference to increment.
  * @return Simplex of dimensionality N + 1.
  */
 template <typename T, require_rev_col_vector_t<T>* = nullptr>
-auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
+inline auto simplex_constrain(const T& y, scalar_type_t<T>& lp) {
   using ret_type = plain_type_t<T>;
 
-  size_t N = y.size();
+  const auto N = y.size();
   arena_t<T> arena_y = y;
-  arena_t<Eigen::VectorXd> arena_z(N);
-  Eigen::VectorXd x_val(N + 1);
-
-  double stick_len(1.0);
-  for (Eigen::Index k = 0; k < N; ++k) {
-    double log_N_minus_k = std::log(N - k);
-    double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-    arena_z.coeffRef(k) = inv_logit(adj_y_k);
-    x_val.coeffRef(k) = stick_len * arena_z.coeff(k);
-    lp += log(stick_len);
-    lp -= log1p_exp(-adj_y_k);
-    lp -= log1p_exp(adj_y_k);
-    stick_len -= x_val(k);
-  }
-  x_val.coeffRef(N) = stick_len;
 
-  arena_t<ret_type> arena_x = x_val;
+  double lp_val = 0.0;
+  arena_t<ret_type> arena_x = simplex_constrain(arena_y.val(), lp_val);
+  lp += lp_val;
 
   if (unlikely(N == 0)) {
     return ret_type(arena_x);
   }
 
-  reverse_pass_callback([arena_y, arena_x, arena_z, lp]() mutable {
-    int N = arena_y.size();
-    double stick_len_val = arena_x.val().coeff(N);
-    double stick_len_adj = arena_x.adj().coeff(N);
-    for (Eigen::Index k = N; k-- > 0;) {
-      arena_x.adj().coeffRef(k) -= stick_len_adj;
-      stick_len_val += arena_x.val().coeff(k);
-      double log_N_minus_k = std::log(N - k);
-      double adj_y_k = arena_y.val().coeff(k) - log_N_minus_k;
-      arena_y.adj().coeffRef(k) -= lp.adj() * inv_logit(adj_y_k);
-      arena_y.adj().coeffRef(k) += lp.adj() * inv_logit(-adj_y_k);
-      stick_len_adj += lp.adj() / stick_len_val;
-      stick_len_adj += arena_x.adj().coeff(k) * arena_z.coeff(k);
-      double arena_z_adj = arena_x.adj().coeff(k) * stick_len_val;
-      arena_y.adj().coeffRef(k)
-          += arena_z_adj * arena_z.coeff(k) * (1.0 - arena_z.coeff(k));
-    }
+  reverse_pass_callback([arena_y, arena_x, lp]() mutable {
+    const auto& res_val = to_ref(arena_x.val());
+
+    // backprop for log jacobian contribution to log density
+    arena_x.adj().array() += lp.adj() / res_val.array();
path: stan/math/rev/constraint/simplex_constrain.hpp
user: login: WardBrian
body: I believe this is resolved, the code no longer needs to modify the adjoint of the return
created_at: 2025-04-24T18:47:53Z
author_association: MEMBER

PR comments:
user: login: stan-buildbot
created_at: 2025-04-04T20:02:11Z
body: - - - - - - - - - - - - - - - - - - - - -
| Name | Old Result | New Result | Ratio | Performance change( 1 - new / old ) |
| ------------- |------------- | ------------- | ------------- | ------------- |
| arma/arma.stan | 0.31 | 0.31 | 1.02 | 1.9% faster |
| low_dim_corr_gauss/low_dim_corr_gauss.stan | 0.01 | 0.01 | 1.0 | 0.28% faster |
| gp_regr/gen_gp_data.stan | 0.02 | 0.02 | 1.04 | 4.15% faster |
| gp_regr/gp_regr.stan | 0.09 | 0.09 | 1.02 | 1.5% faster |
| sir/sir.stan | 67.75 | 67.16 | 1.01 | 0.87% faster |
| irt_2pl/irt_2pl.stan | 3.96 | 4.0 | 0.99 | -0.92% slower |
| eight_schools/eight_schools.stan | 0.05 | 0.05 | 1.0 | -0.09% slower |
| pkpd/sim_one_comp_mm_elim_abs.stan | 0.25 | 0.24 | 1.04 | 3.72% faster |
| pkpd/one_comp_mm_elim_abs.stan | 19.49 | 18.8 | 1.04 | 3.53% faster |
| garch/garch.stan | 0.42 | 0.41 | 1.02 | 2.41% faster |
| low_dim_gauss_mix/low_dim_gauss_mix.stan | 2.66 | 2.61 | 1.02 | 1.78% faster |
| arK/arK.stan | 1.73 | 1.75 | 0.99 | -0.63% slower |
| gp_pois_regr/gp_pois_regr.stan | 2.71 | 2.72 | 1.0 | -0.23% slower |
| low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan | 8.7 | 8.53 | 1.02 | 1.97% faster |
| performance.compilation | 184.79 | 177.97 | 1.04 | 3.69% faster |
Mean result: 1.0164970578185035
- - - - - - - - - - - - - - - - - - - - -
[Jenkins Console Log](https://jenkins.flatironinstitute.org/job/Stan/job/Math/view/change-requests/job/PR-3171/6/consoleFull)
[Blue Ocean](https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-3171/6/pipeline)
Commit hash: e082c1dc2ef8131fcbaa13f6637d9e4392971770
- - - - - - - - - - - - - - - - - - - - -
<details><summary>Machine information</summary>
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

CPU: 
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        46 bits physical, 48 bits virtual
CPU(s):                               80
On-line CPU(s) list:                  0-79
Thread(s) per core:                   2
Core(s) per socket:                   20
Socket(s):                            2
NUMA node(s):                         2
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                85
Model name:                           Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:                             4
CPU MHz:                              2400.000
CPU max MHz:                          3700.0000
CPU min MHz:                          1000.0000
BogoMIPS:                             4800.00
Virtualization:                       VT-x
L1d cache:                            1.3 MiB
L1i cache:                            1.3 MiB
L2 cache:                             40 MiB
L3 cache:                             55 MiB
NUMA node0 CPU(s):                    0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
NUMA node1 CPU(s):                    1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          KVM: Mitigation: VMX disabled
Vulnerability L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT vulnerable
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities

G++: 
g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang: 
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

</details>
author_association: CONTRIBUTOR
user: login: WardBrian
created_at: 2025-04-07T14:29:15Z
body: A quick validation:

Using the `multi-logit-normal` from https://github.com/bob-carpenter/transforms

<details><summary>Code and data</summary>
<p>

```stan
functions {
  real multi_logit_normal_cholesky_lpdf(vector theta, vector mu,
                                        matrix L_Sigma) {
    int N = rows(theta);
    vector[N] log_theta = log(theta);
    return multi_normal_cholesky_lpdf(log_theta[1 : N - 1] - log_theta[N] | mu, L_Sigma)
           - sum(log_theta);
  }
}
data {
  int<lower=1> N;
  vector[N - 1] mu;
  matrix[N - 1, N - 1] L_Sigma;
}
parameters {
  simplex[N] x;
}
model {
  target += multi_logit_normal_cholesky_lpdf(x | mu, L_Sigma);
}
```

```json
{
  "N": 5,
  "mu": [-0.0104, -0.006, -0.0062, -0.0081],
  "L_Sigma": [
    [20.8371, 0.0, 0.0, 0.0],
    [65.0306, 58.6011, 0.0, 0.0],
    [-63.5698, 25.1499, 30.1392, 0.0],
    [32.0142, 22.2009, 69.8059, 21.1121]
  ]
}

```

</p>
</details> 

Here are the stansummarys of the current transform (`sb`):

```
$ ./bin/stansummary sb_1.csv sb_2.csv sb_3.csv sb_4.csv

Inference for Stan model: multi_normal_logit_model
4 chains: each with iter=1000; warmup=1000; thin=1; 1000 iterations saved.

Warmup took (0.18, 0.13, 0.14, 0.12) seconds, 0.57 seconds total
Sampling took (0.086, 0.12, 0.080, 0.10) seconds, 0.39 seconds total

                 Mean   MCSE  StdDev      MAD       5%      50%    95%  ESS_bulk  ESS_tail  ESS_bulk/s  R_hat

lp__              -19  0.069     1.4  1.2e+00 -2.2e+01 -1.9e+01    -17       317       717         818    1.0
accept_stat__    0.65  0.060    0.34     0.30  1.1e-03     0.77   1.00        43       279         110    1.1
stepsize__      0.060    nan   0.011    0.010  4.3e-02    0.063  0.071       nan       nan         nan    nan
treedepth__       3.9   0.23     1.5      1.5  1.0e+00      4.0    6.0        48        57         123    1.1
n_leapfrog__       30    5.4      26       28  2.0e+00       26     67        43       346         111    1.1
divergent__      0.36    nan    0.48     0.00  0.0e+00     0.00    1.0       214       214         550    1.0
energy__           21  0.079     1.9      1.8  1.8e+01       21     25       590      1548        1520    1.0

x[1]             0.27  0.025    0.42  1.5e-05  1.7e-23  9.9e-06   1.00       131       528         338    1.0
x[2]             0.12  0.026    0.32  1.8e-21  4.3e-72  1.2e-21   1.00       164       328         423    1.0
x[3]             0.33  0.045    0.46  5.6e-11  1.6e-63  3.8e-11    1.0       135        89         348    1.0
x[4]            0.068  0.031    0.24  5.2e-31  1.2e-79  3.5e-31   0.99       112       346         288    1.0
x[5]             0.21  0.025    0.38  3.2e-06  2.2e-16  2.2e-06   1.00        71       970         182    1.1

$ ./bin/diagnose sb_1.csv sb_2.csv sb_3.csv sb_4.csv

Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
1548 of 1000 (154.80%) transitions ended with a divergence.
These divergent transitions indicate that HMC is not fully able to explore the posterior distribution.
Try increasing adapt delta closer to 1.
If this doesn't remove all divergences, try to reparameterize the model.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

The following parameters had rank-normalized split R-hat greater than 1.01:
  x[1], x[2], x[3], x[5]
Such high values indicate incomplete mixing and biased estimation.
You should consider regularizating your model with additional prior information or a more effective parameterization.
```

And this PR (`ilr`):
```
$ ./bin/stansummary ilr_1.csv ilr_2.csv ilr_3.csv ilr_4.csv

Inference for Stan model: multi_normal_logit_model
4 chains: each with iter=1000; warmup=1000; thin=1; 1000 iterations saved.

Warmup took (0.20, 0.24, 0.20, 0.20) seconds, 0.83 seconds total
Sampling took (0.22, 0.19, 0.20, 0.22) seconds, 0.84 seconds total

                 Mean     MCSE   StdDev      MAD       5%      50%    95%  ESS_bulk  ESS_tail  ESS_bulk/s  R_hat

lp__              -18  3.4e-02      1.4  1.2e+00 -2.1e+01 -1.8e+01    -17      1734      2095        2071    1.0
accept_stat__    0.93    0.010  1.1e-01  4.3e-02     0.70     0.97   1.00       768      2447         917    1.0
stepsize__      0.033      nan  3.3e-03  4.8e-03    0.029    0.033  0.036       nan       nan         nan    nan
treedepth__       5.9    0.023  1.3e+00  1.5e+00      2.0      6.0    7.0      2952       695        3527    1.0
n_leapfrog__       95      1.3  4.9e+01  4.2e+01      7.0      127    127      1972      2421        2356    1.0
divergent__      0.00      nan  0.0e+00  0.0e+00     0.00     0.00   0.00       nan       nan         nan    nan
energy__           20    0.049  2.0e+00  1.8e+00       18       20     24      1696      2159        2026    1.0

x[1]            0.087  5.6e-03     0.27  1.2e-27  6.0e-75  8.1e-28   1.00      2005      2372        2396   1.00
x[2]             0.30  9.9e-03     0.46  4.4e-20  5.9e-97  3.0e-20    1.0      1806      1998        2157    1.0
x[3]             0.32  9.9e-03     0.46  8.9e-20 6.4e-103  6.0e-20    1.0      1587      2230        1896    1.0
x[4]             0.21  8.7e-03     0.40  1.3e-26  9.9e-84  8.8e-27    1.0      1798      2068        2148    1.0
x[5]            0.080  5.5e-03     0.26  2.1e-27  3.6e-75  1.4e-27   0.98      1599      2009        1910    1.0

$ ./bin/diagnose ilr_1.csv ilr_2.csv ilr_3.csv ilr_4.csv

Checking sampler transitions treedepth.
Treedepth satisfactory for all transitions.

Checking sampler transitions for divergences.
No divergent transitions found.

Checking E-BFMI - sampler transitions HMC potential energy.
E-BFMI satisfactory.

Rank-normalized split effective sample size satisfactory for all parameters.

Rank-normalized split R-hat values satisfactory for all parameters.
```
author_association: MEMBER
user: login: spinkney
created_at: 2025-04-07T17:10:46Z
body: @WardBrian there's a bug in the logic for the sampler divergence output calc. It says 154% above

```
Checking sampler transitions for divergences.
1548 of 1000 (154.80%) transitions ended with a divergence.
```
author_association: COLLABORATOR
user: login: WardBrian
created_at: 2025-04-07T17:15:56Z
body: Yeah, it looks like the denominator never changes as you add chains. I’ll take a look up in the cmdstan repo 
author_association: MEMBER
user: login: stan-buildbot
created_at: 2025-04-08T00:21:14Z
body: - - - - - - - - - - - - - - - - - - - - -
| Name | Old Result | New Result | Ratio | Performance change( 1 - new / old ) |
| ------------- |------------- | ------------- | ------------- | ------------- |
| arma/arma.stan | 0.33 | 0.34 | 0.99 | -0.91% slower |
| low_dim_corr_gauss/low_dim_corr_gauss.stan | 0.01 | 0.01 | 1.02 | 2.34% faster |
| gp_regr/gen_gp_data.stan | 0.02 | 0.02 | 1.06 | 6.1% faster |
| gp_regr/gp_regr.stan | 0.09 | 0.09 | 1.05 | 5.21% faster |
| sir/sir.stan | 67.99 | 65.9 | 1.03 | 3.07% faster |
| irt_2pl/irt_2pl.stan | 3.89 | 3.92 | 0.99 | -0.81% slower |
| eight_schools/eight_schools.stan | 0.06 | 0.05 | 1.06 | 5.95% faster |
| pkpd/sim_one_comp_mm_elim_abs.stan | 0.24 | 0.24 | 1.0 | 0.13% faster |
| pkpd/one_comp_mm_elim_abs.stan | 18.82 | 18.84 | 1.0 | -0.1% slower |
| garch/garch.stan | 0.4 | 0.4 | 1.01 | 1.23% faster |
| low_dim_gauss_mix/low_dim_gauss_mix.stan | 2.58 | 2.58 | 1.0 | 0.16% faster |
| arK/arK.stan | 1.72 | 1.7 | 1.01 | 1.16% faster |
| gp_pois_regr/gp_pois_regr.stan | 2.68 | 2.69 | 1.0 | -0.15% slower |
| low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan | 8.34 | 8.41 | 0.99 | -0.88% slower |
| performance.compilation | 190.29 | 177.33 | 1.07 | 6.81% faster |
Mean result: 1.020708780696168
- - - - - - - - - - - - - - - - - - - - -
[Jenkins Console Log](https://jenkins.flatironinstitute.org/job/Stan/job/Math/view/change-requests/job/PR-3171/7/consoleFull)
[Blue Ocean](https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-3171/7/pipeline)
Commit hash: 69dc9570487240d79cce097c90aa37c55f213a30
- - - - - - - - - - - - - - - - - - - - -
<details><summary>Machine information</summary>
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

CPU: 
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        46 bits physical, 48 bits virtual
CPU(s):                               80
On-line CPU(s) list:                  0-79
Thread(s) per core:                   2
Core(s) per socket:                   20
Socket(s):                            2
NUMA node(s):                         2
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                85
Model name:                           Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:                             4
CPU MHz:                              3478.441
CPU max MHz:                          3700.0000
CPU min MHz:                          1000.0000
BogoMIPS:                             4800.00
Virtualization:                       VT-x
L1d cache:                            1.3 MiB
L1i cache:                            1.3 MiB
L2 cache:                             40 MiB
L3 cache:                             55 MiB
NUMA node0 CPU(s):                    0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
NUMA node1 CPU(s):                    1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          KVM: Mitigation: VMX disabled
Vulnerability L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT vulnerable
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities

G++: 
g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang: 
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

</details>
author_association: CONTRIBUTOR
user: login: stan-buildbot
created_at: 2025-04-10T23:01:40Z
body: - - - - - - - - - - - - - - - - - - - - -
| Name | Old Result | New Result | Ratio | Performance change( 1 - new / old ) |
| ------------- |------------- | ------------- | ------------- | ------------- |
| arma/arma.stan | 0.37 | 0.31 | 1.2 | 16.51% faster |
| low_dim_corr_gauss/low_dim_corr_gauss.stan | 0.01 | 0.01 | 1.06 | 5.46% faster |
| gp_regr/gen_gp_data.stan | 0.03 | 0.02 | 1.1 | 9.32% faster |
| gp_regr/gp_regr.stan | 0.09 | 0.09 | 1.04 | 3.96% faster |
| sir/sir.stan | 70.13 | 66.23 | 1.06 | 5.56% faster |
| irt_2pl/irt_2pl.stan | 4.07 | 3.97 | 1.03 | 2.51% faster |
| eight_schools/eight_schools.stan | 0.05 | 0.05 | 1.02 | 1.7% faster |
| pkpd/sim_one_comp_mm_elim_abs.stan | 0.25 | 0.25 | 1.02 | 1.69% faster |
| pkpd/one_comp_mm_elim_abs.stan | 19.23 | 18.86 | 1.02 | 1.94% faster |
| garch/garch.stan | 0.42 | 0.41 | 1.02 | 2.43% faster |
| low_dim_gauss_mix/low_dim_gauss_mix.stan | 2.65 | 2.6 | 1.02 | 1.89% faster |
| arK/arK.stan | 1.76 | 1.73 | 1.02 | 1.81% faster |
| gp_pois_regr/gp_pois_regr.stan | 2.79 | 2.73 | 1.02 | 2.26% faster |
| low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan | 8.76 | 8.42 | 1.04 | 3.86% faster |
| performance.compilation | 184.79 | 177.89 | 1.04 | 3.73% faster |
Mean result: 1.0468621449125708
- - - - - - - - - - - - - - - - - - - - -
[Jenkins Console Log](https://jenkins.flatironinstitute.org/job/Stan/job/Math/view/change-requests/job/PR-3171/8/consoleFull)
[Blue Ocean](https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-3171/8/pipeline)
Commit hash: 69dc9570487240d79cce097c90aa37c55f213a30
- - - - - - - - - - - - - - - - - - - - -
<details><summary>Machine information</summary>
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

CPU: 
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        46 bits physical, 48 bits virtual
CPU(s):                               80
On-line CPU(s) list:                  0-79
Thread(s) per core:                   2
Core(s) per socket:                   20
Socket(s):                            2
NUMA node(s):                         2
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                85
Model name:                           Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:                             4
CPU MHz:                              2400.000
CPU max MHz:                          3700.0000
CPU min MHz:                          1000.0000
BogoMIPS:                             4800.00
Virtualization:                       VT-x
L1d cache:                            1.3 MiB
L1i cache:                            1.3 MiB
L2 cache:                             40 MiB
L3 cache:                             55 MiB
NUMA node0 CPU(s):                    0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
NUMA node1 CPU(s):                    1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          KVM: Mitigation: VMX disabled
Vulnerability L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT vulnerable
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities

G++: 
g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang: 
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

</details>
author_association: CONTRIBUTOR
user: login: spinkney
created_at: 2025-04-24T12:11:24Z
body: Is this waiting for my review? I can do it next week, otherwise what else is needed?
author_association: COLLABORATOR
user: login: WardBrian
created_at: 2025-04-24T12:33:21Z
body: @spinkney this is waiting on final review from @SteveBronder and then us deciding if we’re okay with not preserving the existing code, otherwise it should be good
author_association: MEMBER
user: login: spinkney
created_at: 2025-04-24T12:50:21Z
body: Ok, I still think it's worth another issue on saving different transforms and creating how a user would specify that. For this PR I vote that we can just remove the existing code.
author_association: COLLABORATOR
user: login: stan-buildbot
created_at: 2025-04-25T00:12:42Z
body: - - - - - - - - - - - - - - - - - - - - -
| Name | Old Result | New Result | Ratio | Performance change( 1 - new / old ) |
| ------------- |------------- | ------------- | ------------- | ------------- |
| arma/arma.stan | 0.34 | 0.31 | 1.08 | 7.32% faster |
| low_dim_corr_gauss/low_dim_corr_gauss.stan | 0.01 | 0.01 | 1.02 | 1.49% faster |
| gp_regr/gen_gp_data.stan | 0.03 | 0.02 | 1.07 | 6.5% faster |
| gp_regr/gp_regr.stan | 0.09 | 0.09 | 1.08 | 7.0% faster |
| sir/sir.stan | 70.06 | 69.39 | 1.01 | 0.96% faster |
| irt_2pl/irt_2pl.stan | 4.31 | 4.02 | 1.07 | 6.81% faster |
| eight_schools/eight_schools.stan | 0.06 | 0.05 | 1.03 | 2.64% faster |
| pkpd/sim_one_comp_mm_elim_abs.stan | 0.25 | 0.24 | 1.02 | 2.27% faster |
| pkpd/one_comp_mm_elim_abs.stan | 20.04 | 18.84 | 1.06 | 5.97% faster |
| garch/garch.stan | 0.43 | 0.4 | 1.07 | 6.8% faster |
| low_dim_gauss_mix/low_dim_gauss_mix.stan | 2.76 | 2.62 | 1.05 | 4.97% faster |
| arK/arK.stan | 1.76 | 1.75 | 1.01 | 0.76% faster |
| gp_pois_regr/gp_pois_regr.stan | 3.03 | 2.73 | 1.11 | 9.71% faster |
| low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan | 8.81 | 8.46 | 1.04 | 3.98% faster |
| performance.compilation | 182.82 | 178.55 | 1.02 | 2.33% faster |
Mean result: 1.049411230989343
- - - - - - - - - - - - - - - - - - - - -
[Jenkins Console Log](https://jenkins.flatironinstitute.org/job/Stan/job/Math/view/change-requests/job/PR-3171/9/consoleFull)
[Blue Ocean](https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-3171/9/pipeline)
Commit hash: 10da93427a21be9ac1f5efc603226a592221bc40
- - - - - - - - - - - - - - - - - - - - -
<details><summary>Machine information</summary>
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

CPU: 
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        46 bits physical, 48 bits virtual
CPU(s):                               80
On-line CPU(s) list:                  0-79
Thread(s) per core:                   2
Core(s) per socket:                   20
Socket(s):                            2
NUMA node(s):                         2
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                85
Model name:                           Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:                             4
CPU MHz:                              2400.000
CPU max MHz:                          3700.0000
CPU min MHz:                          1000.0000
BogoMIPS:                             4800.00
Virtualization:                       VT-x
L1d cache:                            1.3 MiB
L1i cache:                            1.3 MiB
L2 cache:                             40 MiB
L3 cache:                             55 MiB
NUMA node0 CPU(s):                    0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
NUMA node1 CPU(s):                    1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          KVM: Mitigation: VMX disabled
Vulnerability L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT vulnerable
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities

G++: 
g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang: 
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

</details>
author_association: CONTRIBUTOR
user: login: stan-buildbot
created_at: 2025-04-26T01:11:48Z
body: - - - - - - - - - - - - - - - - - - - - -
| Name | Old Result | New Result | Ratio | Performance change( 1 - new / old ) |
| ------------- |------------- | ------------- | ------------- | ------------- |
| arma/arma.stan | 0.39 | 0.34 | 1.12 | 10.9% faster |
| low_dim_corr_gauss/low_dim_corr_gauss.stan | 0.01 | 0.01 | 1.01 | 0.73% faster |
| gp_regr/gen_gp_data.stan | 0.02 | 0.02 | 1.04 | 3.59% faster |
| gp_regr/gp_regr.stan | 0.09 | 0.09 | 1.05 | 4.59% faster |
| sir/sir.stan | 67.8 | 66.38 | 1.02 | 2.1% faster |
| irt_2pl/irt_2pl.stan | 3.99 | 4.0 | 1.0 | -0.14% slower |
| eight_schools/eight_schools.stan | 0.06 | 0.05 | 1.03 | 3.11% faster |
| pkpd/sim_one_comp_mm_elim_abs.stan | 0.25 | 0.24 | 1.01 | 1.24% faster |
| pkpd/one_comp_mm_elim_abs.stan | 19.48 | 18.88 | 1.03 | 3.08% faster |
| garch/garch.stan | 0.45 | 0.4 | 1.12 | 10.94% faster |
| low_dim_gauss_mix/low_dim_gauss_mix.stan | 2.77 | 2.62 | 1.06 | 5.56% faster |
| arK/arK.stan | 1.76 | 1.72 | 1.02 | 1.99% faster |
| gp_pois_regr/gp_pois_regr.stan | 2.71 | 2.71 | 1.0 | 0.21% faster |
| low_dim_gauss_mix_collapse/low_dim_gauss_mix_collapse.stan | 8.74 | 8.43 | 1.04 | 3.58% faster |
| performance.compilation | 184.89 | 181.67 | 1.02 | 1.74% faster |
Mean result: 1.0380373007382375
- - - - - - - - - - - - - - - - - - - - -
[Jenkins Console Log](https://jenkins.flatironinstitute.org/job/Stan/job/Math/view/change-requests/job/PR-3171/12/consoleFull)
[Blue Ocean](https://jenkins.flatironinstitute.org/blue/organizations/jenkins/Stan%2FMath/detail/PR-3171/12/pipeline)
Commit hash: 3dcd12419d102d4023893031ac04c0b52dd79879
- - - - - - - - - - - - - - - - - - - - -
<details><summary>Machine information</summary>
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.3 LTS
Release: 20.04
Codename: focal

CPU: 
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Byte Order:                           Little Endian
Address sizes:                        46 bits physical, 48 bits virtual
CPU(s):                               80
On-line CPU(s) list:                  0-79
Thread(s) per core:                   2
Core(s) per socket:                   20
Socket(s):                            2
NUMA node(s):                         2
Vendor ID:                            GenuineIntel
CPU family:                           6
Model:                                85
Model name:                           Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz
Stepping:                             4
CPU MHz:                              2400.000
CPU max MHz:                          3700.0000
CPU min MHz:                          1000.0000
BogoMIPS:                             4800.00
Virtualization:                       VT-x
L1d cache:                            1.3 MiB
L1i cache:                            1.3 MiB
L2 cache:                             40 MiB
L3 cache:                             55 MiB
NUMA node0 CPU(s):                    0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78
NUMA node1 CPU(s):                    1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          KVM: Mitigation: VMX disabled
Vulnerability L1tf:                   Mitigation; PTE Inversion; VMX conditional cache flushes, SMT vulnerable
Vulnerability Mds:                    Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Meltdown:               Mitigation; PTI
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; IBRS
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; IBRS; IBPB conditional; STIBP conditional; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Mitigation; Clear CPU buffers; SMT vulnerable
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts hwp hwp_act_window hwp_epp hwp_pkg_req pku ospke md_clear flush_l1d arch_capabilities

G++: 
g++ (Ubuntu 9.4.0-1ubuntu1~20.04) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Clang: 
clang version 10.0.0-4ubuntu1 
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

</details>
author_association: CONTRIBUTOR