Injective flows for star-like manifolds

## Abstract

Abstract Normalizing Flows (NFs) are powerful and efficient models for density estimation.
When modeling densities on manifolds, NFs can be generalized to injective flows but the Jacobian determinant becomes computationally prohibitive.
Current approaches either consider bounds on the log-likelihood or rely on some approximations of the Jacobian determinant.
In contrast, we propose injective flows for star-like manifolds and show that for such manifolds we can compute the Jacobian determinant exactly and efficiently, with the same cost as NFs.
This aspect is particularly relevant for variational inference settings, where no samples are available and only some unnormalized target is known.
Among many, we showcase the relevance of modeling densities on star-like manifolds in two settings.
Firstly, we introduce a novel Objective Bayesian approach for penalized likelihood models by interpreting level-sets of the penalty as star-like manifolds.
Secondly, we consider probabilistic mixing models and introduce a general method for variational inference by defining the posterior of mixture weights on the probability simplex.

## 1 Introduction

Normalizing Flows (NFs) are flexible and efficient models that allow us to accurately estimate arbitrary probability distributions.
The key idea is to transform a simple base distribution into a complicated one through a series of bijections.
However, in many applications we know that the target density lies on a certain lower-dimensional manifold.
A common approach is to have the base distribution defined on the lower-dimensional space and use an injective transformation to embed it into higher dimensions.
Unfortunately, the computation of the transformed density involves an expensive Jacobian determinant term, which makes the model computationally prohibitive.
Currently, exact and efficient Jacobian determinant is possible only for trivial manifolds like spheres and tori (Gemici et al., ; Rezende et al., ) or for very restrictive transformations (Ross and Cresswell, ).
In practice, most work renounce exact density estimation and approximate the Jacobian determinant term (Kumar et al., ; Caterini et al., ; Sorrenson et al., ), often with high variance estimators.
Exact density estimation might not be critical when training with maximum likelihood, but is crucial in many applications like variational inference, where samples are not available.

In this paper, we introduce injective flows for star-like manifolds and show that we can exactly and efficiently compute the associated Jacobian determinant term, with the same computational cost as NFs.
We consider star-like manifolds with intrinsic dimensionality $d-1$ and embedded in $\mathbb{R}^{d}$, which is a general class of manifolds that are particularly relevant in many statistical applications.
Note that learning distributions, including the uniform distribution, on arbitrary manifolds is non-trivial and often requires complicated sampling schemes (Pennec, ; Diaconis et al., ).
Distributions on the hypersphere are crucial in directional statistics, astrophysics, medicine, biology, meteorology and many other fields (Chikuse, ).
Particularly relevant for geoscience applications is the fact that the Earth is an oblate spheroid, which is a star-like manifold.
The probabilistic simplex, another star-like manifold, is very useful to model vectors that represent true probabilities by construction.
Possible applications include probabilistic mixing models and other variations like Bayesian tracer mixing models, which are common in ecology, geoscience, zoology and many others (Phillips, ; Stock et al., ), and probabilistic treatment of archetype models (Seth and Eugster, ; Keller et al., ).
In some Bayesian settings, level sets of posteriors with one-parameter priors (e.g. sparsity parameter) define star-like manifolds.
As we will argue, defining the posterior on such manifolds allows for an objective Bayesian treatment of the problem.

We showcase the generality of our approach with two very distinct applications in Bayesian modeling.
First, we introduce a novel Objective Bayesian approach to penalized likelihood methods.
In this case the star-like manifold defines a level-set of the penalty constraint.
Second, we consider probabilistic mixing models and introduce a general framework for variational inference on the mixing weights.
We constrain the posterior on the simplex, such that mixing weights always sum to one, and showcase an application in portfolio analysis.

We summarize the contributions of the present work as follows:

- •
We propose injective flows for star-like manifolds.
- •
In contrast to most existing work on injective flows, we show that we can exactly and efficiently compute the associated Jacobian determinant.
- •
We showcase the relevance and broad applicability of the proposed method in a novel Objective Bayesian approach and for posterior inference in probabilistic mixing models.

## 2 Background

##### Density and Jacobian determinant for bijective functions

Let $\bm{x}$ be a $d$-dimensional random variable with unknown distribution $p_{x}(\bm{x})$ and let $\bm{z}$ be a $d$-dimensional random variable with known base distribution $p_{z}(\bm{z})$.
The key idea of NFs is to model the unknown distribution $p_{x}(\bm{x})$ through a transformation $\operatorname{\mathcal{T}}:\operatorname{\mathbb{R}}^{d}\mapsto\operatorname{
\mathbb{R}}^{d}$ such that $\bm{x}=\operatorname{\mathcal{T}}(\bm{z})$.
If $\operatorname{\mathcal{T}}$ is a diffeomorphism, i.e. differentiable bijection with differentiable inverse $\operatorname{\mathcal{T}}^{-1}$, the change of variable formula (Rudin, ) allows us to express $p_{x}(\bm{x})$ solely in terms of the base distribution $p_{z}(\bm{z})$ and $\operatorname{\mathcal{T}}$: $p_{x}(\bm{x})=p_{z}(\bm{z})\left|\det J_{\operatorname{\mathcal{T}}}(\bm{z})
\right|^{-1}$,
where $J_{\operatorname{\mathcal{T}}}$ is the Jacobian of the transformation $\operatorname{\mathcal{T}}$.
Therefore, the trade-off consists of implementing bijections with tractable $\det J_{\operatorname{\mathcal{T}}}$ which are still flexible enough to approximate any well-behaved distribution.
One key idea is to exploit the property that, given a set of bijections $\{\operatorname{\mathcal{T}}^{(i)}\}_{i=1}^{k}$, their composition $\operatorname{\mathcal{T}}=\operatorname{\mathcal{T}}^{(k)}\circ\cdots\circ
\operatorname{\mathcal{T}}^{(1)}$ is still a bijection.
Since for bijections the Jacobian is a square matrix, the determinant of a composition of bijections factorizes as the product of the determinants of the individual bijections.
Overall, NFs are built as

$$ $p_{x}(\bm{x})=p_{z}(\bm{z})\left|\det J_{\operatorname{\mathcal{T}}}(\bm{z}) \right|^{-1}\quad\textrm{with}\quad\det J_{\operatorname{\mathcal{T}}}(\bm{z}) =\prod_{i=1}^{k}\det J_{\operatorname{\mathcal{T}}^{(i)}}(\bm{u}_{i-1})$ (1) $$

where $\bm{u}_{i-1}=\mathcal{T}^{(i-1)}(\bm{u}_{i-2})$ and $\bm{u}_{0}=\bm{z}$.
In general, computing the $\det J_{\operatorname{\mathcal{T}}}$ has a cost of $O(d^{3})$ because it requires some form of matrix decomposition (typically LU or QR).
Instead, what most implementation exploit is the property in Eq. (1): we can efficiently model a bijection $\operatorname{\mathcal{T}}$ by stacking simpler bijective layers $\operatorname{\mathcal{T}}^{(i)}$ with tractable (analytical) Jacobian determinant.
Typically, each $\det J_{\operatorname{\mathcal{T}}^{(i)}}$ is made tractable by designing bijections $\operatorname{\mathcal{T}}^{(i)}$ with triangular Jacobian by construction, such that its determinant is given by the diagonal entries, which has a $O(d)$ complexity.

##### Density and Jacobian determinant for injective functions

NFs are limited by the use of bijections, which prevents modeling densities on lower dimensional manifolds.
In such cases the target distribution lives on a $m$-dimensional manifold $\operatorname{\mathcal{M}}$ embedded in a $d$-dimensional Euclidean space $\operatorname{\mathcal{M}}\subset\operatorname{\mathbb{R}}^{d}$, where $m<d$.
In order to constrain $p_{x}(\bm{x})$ to live on the manifold $\operatorname{\mathcal{M}}$, we need an injective transformation that inflates the dimensionality $\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}:\operatorname{
\mathbb{R}}^{m}\mapsto\operatorname{\mathbb{R}}^{d}$.
The transformed probability distribution $p_{x}(\bm{x})$ can still be computed by accounting for the volume change (Ben-Israel, ; Krantz and Parks, ):

$$ $p_{x}(\bm{x})=p_{z}(\bm{z})\left|\operatorname{\mathrm{vol}}J_{\operatorname{ \operatorname{\mathcal{T}}_{m\rightarrow d}}}(\bm{z})\right|^{-1}\quad\textrm{ with}\quad\operatorname{\mathrm{vol}}J_{\operatorname{\operatorname{\mathcal{T }}_{m\rightarrow d}}}=\sqrt{\det\Big{(}\big{(}J_{\operatorname{\operatorname{ \mathcal{T}}_{m\rightarrow d}}}\big{)}^{T}J_{\operatorname{\operatorname{ \mathcal{T}}_{m\rightarrow d}}}\Big{)}}\;,$ (2) $$

where $J_{\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}(\bm{z})}\in
\operatorname{\mathbb{R}}^{d\times m}$ is a rectangular matrix.
Note that if $m=d$, $J_{\operatorname{\mathcal{T}}}$ is a square matrix so $\operatorname{\mathrm{vol}}J_{\operatorname{\mathcal{T}}}=\sqrt{\det J_{
\operatorname{\mathcal{T}}}^{T}J_{\operatorname{\mathcal{T}}}}=\sqrt{\det J_{
\operatorname{\mathcal{T}}}^{T}\det J_{\operatorname{\mathcal{T}}}}=\det J_{
\operatorname{\mathcal{T}}}$ and Eq. (2) reduces to Eq. (1).
Crucially, since $J_{\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}}$ is now rectangular, the Jacobian determinant cannot be decomposed as the product of stacked transformations anymore, which is the crucial property that makes bijective flows tractable (see Eq. (1)).
Instead, we need to explicitly compute the Jacobian determinant, which is $O(m^{3})$ in general.
This makes injective flows computationally prohibitive for high dimensional manifolds.

Even if we were to stack bijective layers after the injective transformation, we would still be unable to write the overall Jacobian determinants as the product of individual bijections and would remain $O(m^{3})$.
To see this, consider the transformation $\operatorname{\mathcal{T}}=\operatorname{\mathcal{T}}_{d}\circ\operatorname{
\operatorname{\mathcal{T}}_{m\rightarrow d}}\circ\operatorname{\mathcal{T}}_{m}$, where $\operatorname{\mathcal{T}}_{m}:\operatorname{\mathbb{R}}^{m}\mapsto
\operatorname{\mathbb{R}}^{m}$ and $\operatorname{\mathcal{T}}_{d}:\operatorname{\mathbb{R}}^{d}\mapsto
\operatorname{\mathbb{R}}^{d}$ are arbitrary bijections.
Then, $\det J_{\operatorname{\mathcal{T}}}$ factorizes:

$$ $\det J_{\operatorname{\mathcal{T}}}=\det J_{\operatorname{\mathcal{T}}_{m}} \sqrt{\det\Big{(}\big{(}J_{\operatorname{\mathcal{T}}_{d}}J_{\operatorname{ \operatorname{\mathcal{T}}_{m\rightarrow d}}}\big{)}^{T}J_{\operatorname{ \mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}} }\Big{)}}\;.$ (3) $$

We refer to Appendix A.2 for a full derivation.
Note that we can factorize only $\det J_{\operatorname{\mathcal{T}}_{m}}$ (the bijection that precedes the inflating step), while the Jacobian determinant of the bijections $\operatorname{\mathcal{T}}_{d}$ after the inflating step cannot be disentangled.
Naturally, the same would hold true if the bijections $\operatorname{\mathcal{T}}_{d}$ were replaced by arbitrary injective transformations.
In both cases we would need to compute the Jacobian product $J_{\operatorname{\mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_
{m\rightarrow d}}}$ and its determinant, which has cubic complexity.

## 3 Injective flows for star-like manifolds

We now present the proposed injective flow to model densities on arbitrary star-like manifolds.
In particular, in Section 3.1 we introduce the definition of star-like manifolds and show that they can always be parametrized with generalized spherical coordinates.
Then, in Section 3.2 we define injective flows on star-like manifolds and show that the Jacobian determinant can be computed exactly and efficiently.
Lastly, in Section 3.3 we discuss the limitations of the proposed approach.

### 3.1 Star-like manifolds

###### Definition 1 .

We call a domain a *star domain* $\mathcal{S}$ if there exists one point $s_{0}\in\mathcal{S}$ such that, given any other point $s\in\mathcal{S}$ in the domain, the line segment connecting $s_{0}$ to $s$ lies entirely in $\mathcal{S}$.
Furthermore, we define *star-like manifold* $\mathcal{M}_{\mathcal{S}}$ as the manifold defined by the boundary of a star domain.
In this work we consider $d-1$ dimensional star-like manifolds embedded in $\mathbb{R}^{d}$.

Figure: Figure 1: 1D star-like manifold parametrized in spherical coordinates.
Refer to caption: x1.png

In other words, a star-like manifold is such there exists a point from which the entire manifold can be “viewed”.
Intuitively, this suggests that we can always parametrize it with generalized spherical coordinates, which consist of $d-1$ angles $\bm{\theta}\in U^{d-1}_{\theta}\coloneqq[0,\pi]^{d-2}\times[0,2\pi]$ and one radius $r\in\operatorname{\mathbb{R}}_{>0}$.
Let $\mathcal{M}_{\mathcal{S}}$ be a $d-1$ dimensional star-like manifold embedded in $\operatorname{\mathbb{R}}^{d}$.
Then, we need $d-1$ variables to identify any point $\bm{x}\in\mathcal{M}_{\mathcal{S}}$.
In particular, we can parametrize $\bm{x}=[\bm{\theta},r(\bm{\theta})]^{T}$ with $d-1$ spherical angles $\bm{\theta}\in U^{d-1}_{\theta}$ and a suitable radius function $r(\bm{\theta})$.
If we choose $s_{0}$ as the origin of the spherical coordinate system, we can define the radius as the line segment connecting $\bm{x}$ and $s_{0}$.
Crucially, by definition of star-like manifolds, the segment intersects the manifold only once, so the radius is uniquely defined.
See Figure 1 for a graphical representation.
Star-like manifolds are the most general class of manifolds that always allow such parametrization.

###### Definition 1 .

### 3.2 Proposed injective flows for star-like manifolds

We now show how to define injective flows on star-like manifolds such that the Jacobian determinant, and hence the log density, can be evaluated exactly and efficiently.
Relevantly, the modeled distribution is expressed in Cartesian coordinates, which is crucial for most applications.
In order to do so we compose three transformations $\operatorname{\mathcal{T}}\coloneqq\operatorname{\mathcal{T}_{s\rightarrow c}}
\circ\operatorname{\mathcal{T}}_{r}\circ\operatorname{\mathcal{T}}_{\theta}$ (see Figure 2): (i) an arbitrary diffeomorphism that maps to $d-1$ spherical angles $\operatorname{\mathcal{T}}_{\theta}$, (ii) the injective transformation $\operatorname{\mathcal{T}}_{r}$ that parametrizes the radius $r(\bm{\theta})$ as a function of the angles and (iii) the coordinate transformation $\operatorname{\mathcal{T}_{s\rightarrow c}}$ from spherical to Cartesian coordinates.
We refer to Appendix A.1 and  A.4 for the explicit expression of the latter and a discussion of its special structure, respectively.
This way the resulting densities will be defined by construction on the manifold parametrized by $\operatorname{\mathcal{T}}_{r}$, while the density will be conveniently expressed in Cartesian coordinates.
In practice, we can increase the flexibility of $\operatorname{\mathcal{T}}_{\theta}$ by stacking any bijective layers of choice, as shown in
Figure 2.

Naively computing the Jacobian determinant as in Eq. (3) requires cubic complexity.
Instead, in Theorem 1 we show for star-like manifolds how to compute it analytically and efficiently in $O(d^{2})$.
We provide a proof sketch of the theorem below and the full proof in the Appendix A.4.

Figure: Figure 2: Architecture of the proposed injective flows for star-like manifolds, see Theorem 1.
Refer to caption: x2.png

###### Theorem 1 .

Let $\operatorname{\mathcal{T}}\coloneqq\operatorname{\mathcal{T}_{s\rightarrow c}}
\circ\operatorname{\mathcal{T}}_{r}\circ\operatorname{\mathcal{T}}_{\theta}$ as in Figure 2, where
$\operatorname{\mathcal{T}}_{\theta}:\bm{z}\in\operatorname{\mathbb{R}}^{d-1}
\mapsto\bm{\theta}\in U_{\theta}^{d-1}$ is any diffeomorphism to $d$-spherical angles,
$\operatorname{\mathcal{T}}_{r}:\bm{\theta}\in U_{\theta}^{d-1}\mapsto[\bm{
\theta},r(\bm{\theta})]\in U_{\theta}^{d-1}\times\operatorname{\mathbb{R}}_{>0}$ with $r(\cdot):\bm{\theta}\in U_{\theta}^{d-1}\mapsto r\in\operatorname{\mathbb{R}}_
{>0}$ being differentiable, and
$\operatorname{\mathcal{T}_{s\rightarrow c}}:[\bm{\theta},r(\bm{\theta})]^{T}
\in U_{\theta}^{d-1}\times\operatorname{\mathbb{R}}_{>0}\mapsto\bm{x}\in
\operatorname{\mathbb{R}}^{d}$ the $d$-spherical to Cartesian transformation (see Definition 6 in Appendix A.1).

Then, the Jacobian determinant of the full transformation $\operatorname{\mathcal{T}}$ is equal to

$$ $\det\big{(}J_{\operatorname{\mathcal{T}}}\big{)}=\det\big{(}J_{\operatorname{ \mathcal{T}}_{\theta}}\big{)}\>\det\big{(}J_{\operatorname{\mathcal{T}_{s \rightarrow c}}}\big{)}\>\|\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c }}}^{T}\big{)}^{-1}\;y\|_{2}\>,$ (4) $$

where $y\coloneqq\big{[}-\nabla_{\bm{\theta}}r(\bm{\theta}),1\big{]}^{T}$.
Relevantly, $\det J_{\operatorname{\mathcal{T}}}$ can be computed exactly and efficiently in $O(d^{2})$.

###### Proof sketch.

(i) Injectivity
It is easy to see that $\operatorname{\mathcal{T}}\coloneqq\operatorname{\mathcal{T}_{s\rightarrow c}}
\circ\operatorname{\mathcal{T}}_{r}\circ\operatorname{\mathcal{T}}_{\theta}$ is injective because $\operatorname{\mathcal{T}}_{\theta}$ is bijective, $\operatorname{\mathcal{T}}_{r}$ is injective and $\operatorname{\mathcal{T}_{s\rightarrow c}}$ is also bijective.
The injectivity of $\operatorname{\mathcal{T}}_{r}$ can be easily seen by noting that $\bm{\theta}\neq\bm{\theta}^{\prime}\implies[\bm{\theta},r(\bm{\theta})]\neq[
\bm{\theta}^{\prime},r(\bm{\theta}^{\prime})]$, independently of $r(\cdot)$.

(ii) Analytical Jacobian determinant
Since $J_{\operatorname{\mathcal{T}}}\in\operatorname{\mathbb{R}}^{d\times d-1}$ is not square, we cannot use the usual property of bijections in Eq. (1).
Instead, we use the result for injective functions in Eq. (3):

$$ $\det\big{(}J_{\operatorname{\mathcal{T}}}\big{)}=\det\big{(}J_{\operatorname{ \mathcal{T}}_{\theta}})\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c }}\circ\operatorname{\mathcal{T}}_{r}}\big{)}=\det\big{(}J_{\operatorname{ \mathcal{T}}_{\theta}}\big{)}\sqrt{\det\Big{(}\big{(}J_{\operatorname{\mathcal {T}_{s\rightarrow c}}}J_{\operatorname{\mathcal{T}}_{r}}\big{)}^{T}\big{(}J_{ \operatorname{\mathcal{T}_{s\rightarrow c}}}J_{\operatorname{\mathcal{T}}_{r}} \big{)}\Big{)}}\;.$ $$

where $J_{\operatorname{\mathcal{T}}_{r}}\in\mathbb{R}^{d\times d-1}$ and $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}\in\mathbb{R}^{d\times d}$.
Note that, as we show in Appendix A.4 Remark 7, we can factor out only the Jacobian determinant of $\operatorname{\mathcal{T}}_{\theta}$, i.e. the bijection that precedes the dimensional inflation step with $\operatorname{\mathcal{T}}_{r}$.
The term $\det J_{\operatorname{\mathcal{T}}_{\theta}}$ is the standard Jacobian determinant for bijective layers and can be computed efficiently.
We are then left to compute $\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}\circ\operatorname{
\mathcal{T}}_{r}}\big{)}$.
To do so we consider the matrix $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\coloneqq[J_{\operatorname{\mathcal{
T}}_{r}}\;\bm{0}_{d\times 1}]\in\mathbb{R}^{d\times d}$ and substitute the determinant with the pseudo-determinant:

$$ $\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}\circ\operatorname{ \mathcal{T}}_{r}}\big{)}=\sqrt{\det\Big{(}J_{\operatorname{\mathcal{T}}_{r}}^{ T}J^{*}J_{\operatorname{\mathcal{T}}_{r}}\Big{)}}=\sqrt{\operatorname{\mathrm{ det}^{+}}\Big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{ \operatorname{\mathcal{T}}_{r}}\Big{)}}\;,$ $$

where $J^{*}\coloneqq J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}J_{
\operatorname{\mathcal{T}_{s\rightarrow c}}}\in\mathbb{R}^{d\times d}$.
With $\operatorname{\mathrm{det}^{+}}$ we denote the pseudo-determinant, which is defined as the product of all non-zero eigenvalues.
The rest of the proof is based on the key observation that $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$ has rank $d-1$ or, equivalently, that its null space is one-dimensional.
As a result, we can use Lemma 12 (see Appendix A.4) and re-write the pseudo-determinant as trace of the adjugate matrix :

$$ $\operatorname{\mathrm{det}^{+}}\Big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r }}^{T}J^{*}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\Big{)}=\operatorname{ \mathrm{Tr}}\Big{(}\operatorname{\mathrm{adj}}\big{(}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\big{)} \Big{)}=\det\big{(}J^{*})\operatorname{\mathrm{Tr}}\Big{(}\operatorname{ \mathrm{adj}}\big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\big{)}\big{(}J^ {*}\big{)}^{-1}\operatorname{\mathrm{adj}}\big{(}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}^{T}\big{)}\Big{)}\;.$ $$

where $\operatorname{\mathrm{adj}}(\cdot)$ is defined as the transpose of the cofactor matrix.
If $A$ is invertible (as for $J^{*}=J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}J_{\operatorname{
\mathcal{T}_{s\rightarrow c}}}$), $\operatorname{\mathrm{adj}}(A)=\det(A)A^{-1}$.
Since $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}$ has rank $d-1$, Lemma 1 holds (see Appendix A.4):

$$ $\operatorname{\mathrm{adj}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})=\frac{ \operatorname{\mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})}{y ^{T}x}xy^{T}\>,$ $$

where $x=[\bm{0},1]^{T}$ and $y\coloneqq\big{[}-\nabla_{\bm{\theta}}r(\bm{\theta}),1\big{]}^{T}$.
Finally, by substitution we get:

$$ $\operatorname{\mathrm{det}^{+}}\Big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r }}^{T}J^{*}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\Big{)}=\det\big{(}J^{*} \big{)}\frac{\operatorname{\mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal {T}}_{r}})^{2}}{(y^{T}x)^{2}}\operatorname{\mathrm{Tr}}\Big{(}xy^{T}\big{(}J^{ *}\big{)}^{-1}yx^{T}\Big{)}=\det\big{(}J_{\operatorname{\mathcal{T}_{s \rightarrow c}}}\big{)}^{2}\|\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c }}}^{T}\big{)}^{-1}\;y\|_{2}^{2}$ $$

(iii) Complexity We can now analyze the time complexity required to evaluate Eq. (4).
The term $\det J_{\operatorname{\mathcal{T}}_{\theta}}$ is the standard Jacobian determinant for bijective layers, which can be computed in $O(d)$.
The Jacobian determinant for spherical to Cartesian coordinates $\det J_{s\rightarrow c}$ is known in closed form (Muleshkov and Nguyen, ) and can be evaluated in $O(d)$.
Therefore, we only need to show that also $w=\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}\big{)}^{-1}\;y$ can be computed efficiently.
Solving the full linear system would require a complexity of $O(d^{3})$.
However, $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}$ is almost-triangular (see Appendix A.4, Eq. (22)) and we can make it triangular with one step of Gaussian elimination, which requires $O(d^{2})$.
The resulting triangular system can be solved in $O(d^{2})$.
Finally, note that we can compute $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}$ very efficiently and analytically (see Appendix A.4, Eq. (24)), without requiring autograd computations.
∎

In variational inference settings we assume the manifold to be given.
This means that we should have access to the parametrization $r(\bm{\theta})$, which is often trivial (see e.g. Appendix A.4 Eq. (25) for the l-p norm ball and Eq. (26) for the probabilistic simplex).
Even if outside the scope of our work, one could train the proposed injective flows on data via maximum likelihood and learn the manifold alongside the density.
Learning $r(\bm{\theta})$ remains only possible if the data is on a $d-1$ dimensional manifold, while manifold learning often aims for more dimensional reduction  (Cayton, ).
By making $r(\bm{\theta})$ learnable, we would also obtain an explicit parametrization of the learnt manifold.

###### Theorem 1 .

###### Proof sketch.

### 3.3 Limitations

The main limitation of the proposed method is that it cannot be easily generalized to manifolds with intrinsic dimensionality lower than $d-1$.
In particular, the proof of Theorem (1) relies on the fact that the zero-padded Jacobian matrix has rank $d-1$. Secondly, the expressivity of the proposed method also depends on the flexibility of the bijective layers.
Despite state-of-the-art bijective layers being extremely expressive (Perugachi-Diaz et al., ), Liao and He () showed that the number of modes that can be modeled remains limited.

## 4 Related work

##### Normalizing Flows

Normalizing Flows (NFs) consist of a simple base distribution that is transformed into a more complicated one through bijective transformations.
One can show that such a construction allows us to approximate any well-behaved distribution (Papamakarios et al., ).
In practice, the bijective transformations are implemented with neural networks that show a trade-off between expressiveness and computational complexity.
However, recently developed bijective layers provide very efficient transformations that satisfy the universality property (Huang et al., ; Durkan et al., ; Jaini et al., ).
For a comprehensive review of different bijective layers and for a discussion about applications we refer to Papamakarios et al. () and Kobyzev et al. ().

##### Variational Inference with NFs

NFs are popular in two distinct applications: (i) as generative models trained on data or (ii) as powerful density estimators in variational inference settings.
In the first case, we are given some samples and NFs are trained by maximum likelihood (forward KL divergence) to approximate the data generating distribution and to later generate new samples (Dinh et al., ; Papamakarios et al., ).
In the variational inference setting, no samples are available and the target distribution is typically known only up to a normalization constant.
NFs are thus trained by reverse KL divergence and, once trained, they allow for both sampling and evaluation of the (approximate) target distribution (Rezende and Mohamed, ; Kingma et al., ).
There is good evidence that a more faithful posterior approximation leads to an improved performance for variational inference tasks (Rezende and Mohamed, ), which is one reason that makes normalizing flows attractive.
This setting is the focus of our work and is particularly relevant for Bayesian inference (Louizos and Welling, ), where the goal is to learn and sample from the posterior distribution given the (unnormalized) product of the prior and likelihood.
In such settings NFs have also proven to be an attractive alternative to MCMC samplers (Negri et al., ).

##### Injective flows on manifolds

The computational challenge of injective flows is the evaluation of the Jacobian determinant in Eq. (2).
In the literature, exact and efficient computation of the Jacobian determinant has been shown only for trivial manifolds like spheres and tori (Gemici et al., ; Rezende et al., ) or for very restrictive transformations (Ross and Cresswell, ).
In particular, the latter focuses only on maps for which $\mathcal{J}^{T}\mathcal{J}\propto\mathbb{I}$, making the determinant trivial.
These are however the only cases where the modeled density can be computed exactly and efficiently during training.
This has a significant impact on variational inference settings where the quality of the approximation influences the exploration of the distribution domain.
It is, in contrast, less relevant in data driven settings trained by maximum likelihood.
Some early work ignored the determinant term altogether (Brehmer and Cranmer, ), which was shown to have detrimental effects already in simple low-dimensional settings (Caterini et al., ).
Current work is focused instead on finding some tractable approximation to the Jacobian determinant.
The most common one is to employ the Hutchinson’s trace estimator (Mathieu and Nickel, ; Caterini et al., ; Flouris and Konukoglu, ), which is characterized by high variance and is biased if used to estimate the log-determinant of the Jacobian (Kumar et al., ).
State-of-the-art works employ surrogate log-likelihood losses and still approximate the Jacobian determinant (Sorrenson et al., ).

In contrast, we are the first to propose exact and computationally efficient injective flows for a wide class of manifolds, namely star-like manifolds.

## 5 Applications

In this section we showcase some interesting applications of the proposed injective flows.
We consider variational inference tasks (see Eq. (27)) where exact density estimation is known to be beneficial.
In particular, we overcome some limitations of current work with exact density estimation, which are applicable only to trivial manifolds like spheres or for very restrictive transformations.
In Section 5.1 we first illustrate how the proposed approach learns distributions on a simple 3D manifold.
In Section 5.2 we use injective flows to define a novel Objective Bayes approach to penalized likelihood problems.
Lastly, in Section 5.3 we introduce a general framework for variational inference in probabilistic mixing models, where we constrain the posterior on the probabilistic simplex.

### 5.1 Illustrative distributions in 3D

We first illustrate the proposed model on a simple 3D setting.
In particular, we train the injective flow in a variational inference setting where the target distribution is known only up to a constant and no samples are given.
Note that this is much more challenging than the usual maximum likelihood settings where flows are trained on data.
In particular, it is not trivial to learn densities with many modes.
To showcase the effectiveness of the proposed method we show the modeled density for increasingly difficult targets: (i) von Mises-Fisher distribution ($\kappa=5$), (ii) mixture of 50 von Mises-Fisher distributions arranged on a spiral ($\kappa=50$) and (iii) sinusoidal checkerboard ($\log\rho(\theta,\phi)=2\sin(6\theta)\sin(6\phi)$), where $\kappa$ is the concentration parameter and $\theta$ and $\phi$ are the polar and azimuthal angles, respectively.
In Figure 3 we show that the modeled densities perfectly match the ground truth in all cases.
Furthermore, we compare the modeled density with the ground truth over 10’000 samples and obtained an accurate reconstruction: $\textrm{MSE}=0.013$ (a), $0.011$ (b), and $0.024$ (c).

Figure: Figure 3: Reconstructed (“ours”) and ground truth (“gt”) density for the proposed injective flow trained via reverse KL divergence without samples. Only the (unnormalized) target is given: (a) von Mises-Fisher ($\kappa=5$), (b) mixture of 50 von Mises-Fisher ($\kappa=50$) and (c) sinusoidal checkerboard.
Refer to caption: x3.png

### 5.2 Objective Bayesian approach to penalized likelihood

##### Objective and subjective Bayes

Bayesian inference is a powerful statistical method that requires a likelihood term, which explains the observed data, and a prior, which quantifies our initial belief.
However, in many cases we might not have enough problem-specific knowledge to specify an informative subjective prior.
This has led to the development of *objective priors*, which are designed to be minimally informative.
Some objective priors include Jeffreys rule (Jeffreys, ), reference priors (Bernardo, ), and maximum entropy priors (Jaynes, ).
Given the vast literature on objective priors (Berger, ), in this work we do not intend to discuss whether objective priors are preferable or not.
Instead, we provide a new framework to define objective priors in settings where only subjective ones have been explored so far.
Specifically, we consider Bayesian penalized likelihood problems and show that level sets of the penalty define star-like manifolds.
Then, we implicitly define the objective prior as the uniform distribution on such manifolds.
This choice aligns with the literature on objective priors for distributions on surfaces since a uniform distribution assigns equal mass to equal volume (Kass, ; Kass and Wasserman, ).

##### Objective Bayes for penalized likelihood models

Let $\bm{y}\sim\bm{X}\bm{\beta}+\bm{\epsilon}$ with $\bm{\epsilon}\sim\mathcal{N}(\bm{0},\sigma^{2}\mathbb{I}_{n})$, where $\bm{X}\in\operatorname{\mathbb{R}}^{n\times d}$ is the data matrix, $\bm{y}\in\operatorname{\mathbb{R}}^{n}$ the targets and $\bm{\beta}\in\operatorname{\mathbb{R}}^{d}$ the regression coefficients.
We then optimize the mean-squared error $\|\bm{y}-\bm{X}\bm{\beta}\|_{2}^{2}$ subject to the (pseudo-) norm penalties $\|\bm{\beta}\|_{p}^{p}$ with $p>0$, which encourages sparsity for $p\leq 1$.
Note that for $p=1$ we recover the LASSO penalty (Tibshirani, ) and for $p=2$ the Ridge penalty.
Tibshirani () noted that we can interpret such penalized likelihood in a Bayesian way with a Gaussian likelihood and a suitable prior.
Park and Casella () showed that with an independent Laplace prior the Maximum a Posteriori (MAP) of the posterior coincides with the frequentist solution.
The above reasoning can be extended to any $l_{p}$ (pseudo-) norm $\|\cdot\|_{p}$ by using a generalized Gaussian prior on $\bm{\beta}$:

$$ $\operatorname*{arg\,min}\limits_{\bm{\beta}\in\operatorname{\mathbb{R}}^{d}} \tfrac{1}{2\sigma^{2}}\|\bm{y}-\bm{X}\bm{\beta}\|_{2}^{2}+\lambda\|\bm{\beta} \|_{p}^{p}=\operatorname*{arg\,max}\limits_{\bm{\beta}\in\operatorname{\mathbb {R}}^{d}}\underbrace{\mathcal{N}(\bm{X}\bm{\beta},\sigma^{2}\mathbb{I}_{n})}_{ p(\bm{y}|\bm{X},\bm{\beta})}\underbrace{\textstyle\prod_{i}\exp\{-\lambda| \beta_{i}|^{p})}_{p(\bm{\beta}|\lambda)}=\bm{\beta}^{*}\;.$ (5) $$

Figure: Figure 4: $95\%$ posterior C.I. for 3 subjective priors with the same MAP. The choice of prior affects the posterior.
Refer to caption: x4.png

However, the generalized Gaussian is not the only prior for which Eq. (5) holds.
Any monotonic transformation $h$ of $p(\bm{\beta}|\lambda))$ results
in the same MAP solution $\bm{\beta}^{*}$ (see Figure 7 in the Appendix). Therefore, the choice of $h(p(\bm{\beta}|\lambda))$, and hence of the prior, is subjective and, crucially, it influences the posterior.
We show this empirically on toy data with the Laplace prior and two monotonic transformations: the square (“square laplace”) and square root (“root laplace”).
In Figure 4 we can clearly see that the posterior is influenced by the choice of the subjective prior, which is undesirable in the absence of specific assumptions.
In contrast, we circumvent the choice of a subjective prior and propose a general framework for objective priors in penalized likelihood methods.

##### Objective Bayesian penalized likelihood with injective flows

With subjective priors the penalty $\|\bm{\beta}\|_{p}^{p}$ in Eq. (5) is enforced as a soft constraint controlled by $\lambda$ such that $\|\bm{\beta}\|_{p}\leq k(\lambda)$, for some $k(\lambda)$.
Our idea is to enforce the norm penalty as a hard constraint by defining the posterior on the manifold $\|\bm{\beta}\|_{p}=k$ by construction.
This way we do not require to *explicitly* specify a subjective prior and we are *implicitly* assuming a uniform prior on the manifold $\|\bm{\beta}\|_{p}=k$.
In summary:

$$ $\begin{aligned} &\textit{Objective Bayes}\\ &p(\bm{y}|\bm{X},\bm{\beta})=\mathcal{N}(\bm{X}\bm{\beta},\sigma^{2}\mathbb{I} _{n})\\ &\textrm{posterior on manifold:}\quad\|\bm{\beta}\|_{p}=k\end{aligned}\qquad \longleftrightarrow\qquad\begin{aligned} &\textit{Subjective Bayes}\\ &p(\bm{y}|\bm{X},\bm{\beta})=\mathcal{N}(\bm{X}\bm{\beta},\sigma^{2}\mathbb{I} _{n})\\ &\textrm{prior:}\quad p(\bm{\beta}|\lambda)\propto\textstyle\prod_{i}\exp\{- \lambda|\beta_{i}|^{p})\end{aligned}$ $$

The equality $\|\bm{\beta}\|_{p}=k$ induces a star-like manifold which we can parametrize with a suitable radius function; see Appendix A.5 Eq. (25) for the explicit parametrization.
Therefore, with our framework we can define the (approximate) posterior $q_{\theta}(\bm{\beta})$ to be constrained on $\|\bm{\beta}\|_{p}=k$ by construction.

Figure: Figure 5: Above: with the objective prior posterior samples lie on the manifold. Below: the norm of posterior samples depends on the subjective prior.
Refer to caption: x5.png

We now illustrate the differences between the subjective and objective approaches with synthetic data.
We use a NF to approximate the posterior $\mathcal{N}(\bm{X}\bm{\beta}\sigma^{2}\mathbb{I}_{n})p(\bm{\beta}|\lambda)$ with the “square laplace” subjective prior $p(\bm{\beta}|\lambda)$. We choose $\lambda$ such that the MAP has a specific norm $\|\bm{\beta}^{*}\|_{1}=k$.
Furthermore, we use an injective flow defined on $\|\bm{\beta}\|_{1}=k$ to approximate the target $\mathcal{N}(\bm{X}\bm{\beta},\sigma^{2}\mathbb{I}_{n})$ (implicitly uniform prior).
In both cases training is performed by minimizing the reverse KL divergence (see Appendix A.6, Eq.(27)).
Figure 5 shows a crucial difference: samples from the objective posterior lie exactly on the manifold while the subjective posterior is scattered.
The bottom panel of Figure 5 shows the distribution of the sample norms varying significantly with the choice of the subjective prior, which agrees with Figure 4.
We include more implementation details in Appendix A.6.
Lastly, note that we do not claim that the proposed prior is better in general, but rather preferable in cases where a less informative prior is desired.

### 5.3 Variational Inference for probabilistic mixing models

##### Probabilistic mixing models

Mixing models are used to study the relative contribution of sources to an observed mixture and are particularly relevant in ecology, geoscience, meteorology and many other fields  (Phillips, ; Stock et al., ; Jiskra et al., ).
Formally, the aim is to reconstruct the mixture $\bm{\pi}$ of sources from which the observations $\mathcal{D}$ were generated, with $\bm{\pi}$ being defined on the probabilistic simplex $\mathcal{C}^{d}\coloneqq\{\bm{\pi}\in\operatorname{\mathbb{R}}^{d}:\pi_{i}\geq
0
\;,\;\sum_{i=1}^{k}\pi_{i}=1\}$.
In the most general Bayesian formulation, we require a prior $p(\bm{\pi})$ and some likelihood $p(\mathcal{D}|\bm{\pi})$ and the challenge is then to define the posterior $p(\bm{\pi}|\mathcal{D})\propto p(\mathcal{D}|\bm{\pi})p(\bm{\pi})$ on the probabilistic simplex $\mathcal{C}^{d}$.
Most approaches rely on the Dirichlet distribution, which is defined on $\mathcal{C}^{d}$ by construction: $\mathrm{Dir}(\bm{\pi})\propto\prod_{i}\pi_{i}^{\alpha_{i}-1}$ with $\alpha_{i}>0$.
Since the posterior can be obtained in closed form only with a multinomial likelihood, most approaches rely on sophisticated MCMC samplers (Stock et al., ).
As a more flexible alternative to MCMC methods, we present a general variational inference framework where $p(\bm{\pi}|\mathcal{D})$ is always defined on $\mathcal{C}^{d}$, leaving complete freedom in the choice of prior and likelihood.

##### Injective flows vs MCMC in Bayesian mixing models

It is easy to see that the probabilistic simplex $\mathcal{C}^{d}$ is a star-like manifold, see Appendix A.5 Eq. (26) for the explicit parametrization.
Therefore, with the our framework we can define an injective flow $q_{\bm{\theta}}(\bm{\pi})$ on $\mathcal{C}^{d}$ by construction and train it to approximate the posterior $p(\bm{\pi}|\mathcal{D})\propto p(\mathcal{D}|\bm{\pi})p(\bm{\pi})$.
In the simplest case when no prior is specified, we are implicitly assuming a uniform distribution on the simplex, i.e. a Dirichlet prior with $\alpha_{i}=1\;\forall i$.
In the more general case, we can always plug in any combination of likelihood $p(\mathcal{D}|\bm{\pi})$ and prior $p(\bm{\pi})$, and the (approximate) posterior $q_{\bm{\theta}}(\bm{\pi})$ will always be defined on $\mathcal{C}^{d}$ by design.
In contrast, with MCMC methods it is not trivial to guarantee posterior samples to be on $\mathcal{C}^{d}$, already for very simple likelihoods (Altmann et al., ; Baker et al., ).
We compare our method with an MCMC sampler in the conjugate case (Dirichlet prior and multinomial likelihood) such that we can compare with the true posterior, which is available in closed form.
We report the results in Appendix A.6 in Figure 9, together with a description of the MCMC sampler.
Results show that (i) the proposed injective flow correctly estimates the posterior distribution across all tested dimensions and (ii) it outperforms the MCMC sampler already in low dimensions. Interestingly, this experiment shows that our approach is able to learn very sparse solutions with many modes.

##### Application: Bayesian portfolio optimization

We select a minimal Bayesian mixing model that already shows the advantages of our proposed method in terms of flexibly choosing likelihood and prior.
One such setting is index replication in the context of portfolio optimization (Markowitz and Todd, ).
A portfolio is defined as a set of $n$ stocks which are held proportionally to the mixture components $\bm{\pi}\in\operatorname{\mathbb{R}}^{n}_{>0}$, such that $\sum_{i}\pi_{i}=1$.
Let $\bm{R}\in\operatorname{\mathbb{R}}^{T\times n}$ be the returns over the time-steps $t=\{1,\ldots,T\}$ of the $n$ stocks.
We are interested in optimizing the portfolio weights $\bm{\pi}$ such that we replicate the reference index returns $\bm{\rho}\in\operatorname{\mathbb{R}}^{T}$, while also incorporating investors personal preferences.
For instance, a sparse portfolio allows to reduce transaction costs arising from trading (Sokolov and Polson, ).
We formulate the problem in Bayesian fashion by specifying a Gaussian likelihood $p(\bm{\rho}|\bm{R},\bm{\pi})=\mathcal{N}(\bm{R}\bm{\pi},\sigma^{2}\mathbb{I}_{
n})$ and some sparsity-inducing prior $p(\bm{\pi})$.

With the proposed framework we can approximate the posterior $p(\bm{\pi}|\bm{R},\bm{\rho})\propto p(\bm{\rho}|\bm{R},\bm{\pi})p(\bm{\pi})$ with an injective flow $q_{\bm{\theta}}(\bm{\pi})$ defined on $\mathcal{C}^{n}$ by design.
The flow $q_{\bm{\theta}}(\bm{\pi})$ is trained by minimizing the reverse KL divergence with the unnormalized target $p(\bm{\rho}|\bm{R},\bm{\pi})p(\bm{\pi})$.
For the sake of illustration, we select a portfolio with 10 stocks over a period of 200 time steps from the dataset in Tu and Li ().
We define $q_{\bm{\theta}}(\bm{\pi})$ on the manifold and consider two priors: the uniform prior on the simplex and the Dirichlet distribution.
In Figure 6 we show the distribution of non-zero entries of the posterior samples for the uniform and Dirichlet distribution.
In particular, we consider the distribution and the sparsity patterns at 4 fixed values of the likelihood (one per plot).
Despite the likelihood being the same, the Dirichlet prior leads to a sparser solution with fewer non-zero entries.
This is also noticeable in the sparsity patterns of the posterior samples in the bottom panel.
In Appendix A.6 in Figure 10 we also show the cumulative return and how it is affected by sparsity.
Overall, we showed how easily we can specify any likelihood and priors while constraining the posterior on the simplex.

Figure: Figure 6: Trained injective flow on the simplex with target $p(\bm{\pi})|\mathcal{D})\propto p(\mathcal{D}|\bm{\pi})$ (“uniform” prior, implicitly) and $p(\bm{\pi})|\mathcal{D})\propto p(\mathcal{D}|\bm{\pi})\textrm{Dir}(\bm{\pi})$ (“Dirichlet” prior). Above: we compare the distribution of non-zero entries in posterior samples for 4 fixed likelihood values (so at fixed reconstruction). Below: we compare sparsity patterns of samples $\bm{\pi}$ with uniform (blue) and Dirichlet (orange) prior.
Refer to caption: x6.png

## 6 Conclusions

Previous work on injective flows on manifolds relies on approximations or lower bounds to circumvent the computation of the Jacobian determinant term.
In this work we showed how to exactly and efficiently compute the Jacobian determinant term for the general class of star-like manifolds.
We then showed that the proposed flow allows for interesting applications that were not possible before.
First, with the proposed framework we introduced a novel Objective Bayes approach to penalized likelihood methods.
The idea is to circumvent the choice of a subjective prior by constraining the posterior on the manifold defined by level-sets of the prior.
Second, we introduced a general variational inference framework for modeling the posterior in probabilistic mixing models.
Overall, the proposed framework allows us to efficiently model distributions on arbitrary star-like manifolds and to flexibly specify any choice of prior and likelihood.

## Appendix A Appendix

The Appendix is organized in six parts.
In Subsection A.1 we define the generalized spherical coordinate system and define the transformation from spherical to Cartesian coordinates.
In Subsection A.2 and A.3 we provide some auxiliary theorems and Lemmas that are used in the main proof.
In Subsection A.4 we provide the full proof of Theorem 1.
In Subsection A.5 we provide some details about the implementation of the proposed injective flows and we make some further comments about the associated computational cost.
Finally, in Subsection A.6 we include further plots and implementation details about the experiments.

### A.1 Generalized spherical coordinates

###### Definition 2 .

We define the *$d$-spherical coordinate system* as a generalization of the spherical coordinate system for $d$-dimensional Euclidean spaces.
Such coordinate system is defined with $d-1$ angles $\theta_{1},\ldots,\theta_{d-1}$ and one radius $r\in\operatorname{\mathbb{R}}_{>0}$, where $\theta_{i}\in[0,\pi]$ for $i<d-1$ and $\theta_{d-1}\in[0,2\pi]$.
We further define a transformation $\operatorname{\mathcal{T}_{s\rightarrow c}}:\bm{x}_{s}\mapsto\bm{x}_{c}$ that maps spherical coordinates $\bm{x}_{s}=[\theta_{1},\ldots,\theta_{d-1},r]^{T}$ to Cartesian coordinates $\bm{x}_{c}=[x_{1},\ldots,x_{d}]^{T}$ as

$$ $\begin{cases}x_{1}=r\cos\theta_{1}\\ x_{2}=r\sin\theta_{1}\cos\theta_{2}\\ \hskip 5.69054pt\vdots\\ x_{d-1}=r\sin\theta_{1}\sin\theta_{2}\cdots\sin\theta_{d-2}\cos\theta_{d-1}\\ x_{d}=r\sin\theta_{1}\sin\theta_{2}\cdots\sin\theta_{d-2}\sin\theta_{d-1}\end{cases}$ (6) $$

We denote with $U^{d-1}_{\theta}\times\operatorname{\mathbb{R}}_{>0}$ the domain of definition for $d$-spherical coordinate system, where $U^{d-1}_{\theta}\coloneqq[0,\pi]^{d-2}\times[0,2\pi]$.

###### Definition 2 .

### A.2 Jacobian determinant for arbitrary injective flows

###### Remark 1 .

Let $\operatorname{\mathcal{T}}_{m}:\operatorname{\mathbb{R}}^{m}\mapsto
\operatorname{\mathbb{R}}^{m}$ and $\operatorname{\mathcal{T}}_{d}:\operatorname{\mathbb{R}}^{d}\mapsto
\operatorname{\mathbb{R}}^{d}$ be arbitrary bijective transformation and let $\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}:\operatorname{
\mathbb{R}}^{m}\rightarrow\operatorname{\mathbb{R}}^{d}$ be an injective transformation with $m<d$.
The transformation $\operatorname{\mathcal{T}}=\operatorname{\mathcal{T}}_{d}\circ\operatorname{
\operatorname{\mathcal{T}}_{m\rightarrow d}}\circ\operatorname{\mathcal{T}}_{m}$ is also injective and its Jacobian determinant factorizes as

$$ $\det J_{\operatorname{\mathcal{T}}}=\det J_{\operatorname{\mathcal{T}}_{m}} \sqrt{\det\Big{(}\big{(}J_{\operatorname{\mathcal{T}}_{d}}J_{\operatorname{ \operatorname{\mathcal{T}}_{m\rightarrow d}}}\big{)}^{T}J_{\operatorname{ \mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}} }\Big{)}}\;.$ (7) $$

###### Proof.

The injectivity of $\operatorname{\mathcal{T}}$ is trivial since it is by definition a composition of injective functions.
Since $\operatorname{\mathcal{T}}$ is injective, its Jacobian matrix $J_{\operatorname{\mathcal{T}}}\in\operatorname{\mathbb{R}}^{d\times m}$ is not squared and we cannot use the usual property of bijections in Eq.(1).
Instead, we use the definition of Jacobian determinant for injective functions in Eq. (2):

$$ $\det J_{\operatorname{\mathcal{T}}}=\sqrt{\det\Big{(}J_{\operatorname{\mathcal {T}}}^{T}J_{\operatorname{\mathcal{T}}}\Big{)}}=\sqrt{\det\Big{(}\big{(}J_{ \operatorname{\mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{m \rightarrow d}}}J_{\operatorname{\mathcal{T}}_{m}}\big{)}^{T}\big{(}J_{ \operatorname{\mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{m \rightarrow d}}}J_{\operatorname{\mathcal{T}}_{d}}\big{)}\Big{)}}\;,$ (8) $$

where $J_{m}\in\operatorname{\mathbb{R}}^{m\times m}$, $J_{\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}}\in\mathbb{R}^{d
\times m}$ and $J_{\operatorname{\mathcal{T}}_{d}}\in\mathbb{R}^{d\times d}$.
We now show that we can factor out the Jacobian determinant of $\operatorname{\mathcal{T}}_{m}$, i.e. the bijection that precedes the dimensional inflation step with $\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}$.
To do so we use the property that for square matrices $A,B$ $\det(AB)=\det A\det B$ and that $\det A=\det A^{T}$:

$$ $\displaystyle\det J_{\operatorname{\mathcal{T}}}$ $\displaystyle=\sqrt{\det\Big{(}J_{\operatorname{\mathcal{T}}_{m}}^{T}J_{ \operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}}^{T}J_{ \operatorname{\mathcal{T}}_{d}}^{T}J_{\operatorname{\mathcal{T}}_{d}}J_{ \operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}}J_{\operatorname{ \mathcal{T}}_{m}}\Big{)}}$ (9) $\displaystyle=\sqrt{\det J_{\operatorname{\mathcal{T}}_{m}}^{T}\det\Big{(}J_{ \operatorname{\mathcal{T}}_{m}}^{T}\>J_{\operatorname{\operatorname{\mathcal{T }}_{m\rightarrow d}}}^{T}J_{\operatorname{\mathcal{T}}_{d}}^{T}\>J_{ \operatorname{\mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{m \rightarrow d}}}\Big{)}\det J_{\operatorname{\mathcal{T}}_{m}}}$ $\displaystyle=\det J_{\operatorname{\mathcal{T}}_{m}}\sqrt{\det\Big{(}\big{(}J _{\operatorname{\mathcal{T}}_{d}}J_{\operatorname{\operatorname{\mathcal{T}}_{ m\rightarrow d}}}\big{)}^{T}\big{(}J_{\operatorname{\mathcal{T}}_{d}}J_{ \operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d}}}\big{)}\Big{)}}$ $\displaystyle=\det J_{\operatorname{\mathcal{T}}_{m}}\det J_{\operatorname{ \mathcal{T}}_{d}\circ\operatorname{\operatorname{\mathcal{T}}_{m\rightarrow d} }}\;.$ $$

∎

###### Remark 1 .

###### Proof.

### A.3 Auxiliary theorems: adjugate matrix and pseudo-determinant

###### Theorem 2 .

[Castillo and Zaballa, ]
Let $A\in\operatorname{\mathbb{R}}^{d\times d}$ and let $\lambda\in\operatorname{\mathbb{R}}$ be an eigenvalue of A. Let $v,w\in\operatorname{\mathbb{R}}^{d}$ be a right and a left eigenvector, respectively, of $A$ for $\lambda$. Then

$$ $w^{T}v\operatorname{\mathrm{adj}}(\lambda\mathbb{I}_{d}-A)=p^{\prime}_{A}( \lambda)vw^{T}\;.$ (10) $$

where $p^{\prime}_{A}(\lambda)$ is the derivative of the characteristic polynomial $p_{A}(\lambda)=\det(\lambda\mathbb{I}_{d}-A)$.

###### Lemma 1 .

Consider the special case where $A\in\operatorname{\mathbb{R}}^{d\times d}$ and $\operatorname{\mathrm{rank}}A=d-1$ or, in other words, the nullspace of $A$ is one dimensional. Then

$$ $\operatorname{\mathrm{adj}}(A)=\frac{\operatorname{\mathrm{det}^{+}}(A)}{w^{T} v}vw^{T}\;,$ (11) $$

where $\operatorname{\mathrm{det}^{+}}$ is the pseudo-determinant.

###### Proof.

Since $\operatorname{\mathrm{rank}}A=d-1$, then there exists one zero eigenvalue.
For $\lambda=0$ Theorem 2 reduces to $w^{T}v\operatorname{\mathrm{adj}}(-A)=p^{\prime}_{A}(0)vw^{T}$.
We can now use the following property of the adjugate matrix: $\operatorname{\mathrm{adj}}(cA)=c^{d}\operatorname{\mathrm{adj}}(A)$ for any scalar $c$.
As a particular case, for $c=-1$ we have $\operatorname{\mathrm{adj}}(-A)=(-)^{d}\operatorname{\mathrm{adj}}(A)$.
Therefore, we obtain that $w^{T}v\operatorname{\mathrm{adj}}(A)=(-)^{d}p^{\prime}_{A}(0)vw^{T}$.
Now, the pseudo-determinant is equal to the smallest non-zero coefficient of the characteristic polynomial $p(\lambda)=\det(\lambda\mathbb{I}_{d}-A)$[Knill, ].
If we expand the definition we obtain $p(\lambda)=(-)^{d}p(A-\lambda\mathbb{I})=p_{0}\lambda^{d}+(-)p_{1}\lambda^{d-1
}+(-)^{k}p_{k}\lambda^{d-k}+(-)^{d}p_{d}$ (see Proposition 2, 8. in Knill []).
Since $A$ has rank $d-1$, $p_{d}=0$ and the smallest non-zero coefficient is $p_{d-1}$.
Finally, note that $p^{\prime}_{A}(0)=(-)^{d}p_{d-1}=(-)^{d}\operatorname{\mathrm{det}^{+}}(A)$.
∎

###### Lemma 2 .

Consider the special case where $A\in\operatorname{\mathbb{R}}^{d\times d}$ and $\operatorname{\mathrm{rank}}A=d-1$. Then,

$$ $\operatorname{\mathrm{Tr}}(\operatorname{\mathrm{adj}}(A))=\operatorname{ \mathrm{det}^{+}}(A)\;.$ (12) $$

###### Proof.

We take the trace of the left and right-hand side of Eq. (11).
We get $\operatorname{\mathrm{Tr}}(\operatorname{\mathrm{adj}}(A))=\frac{\operatorname
{\mathrm{det}^{+}}(A)}{w^{T}v}\operatorname{\mathrm{Tr}}(vw^{T})=\frac{
\operatorname{\mathrm{det}^{+}}(A)}{w^{T}v}\operatorname{\mathrm{Tr}}(w^{T}v)=
\operatorname{\mathrm{det}^{+}}(A)$.
In the first equality we used the linearity of the trace and factored out the constants $\operatorname{\mathrm{det}^{+}}(A)$ and $w^{T}v$.
Lastly, we used the cyclic property of the trace $\operatorname{\mathrm{Tr}}(w^{T}v)=\operatorname{\mathrm{Tr}}(vw^{T})$.
∎

###### Theorem 2 .

###### Lemma 1 .

###### Proof.

###### Lemma 2 .

###### Proof.

### A.4 Proof of Theorem 1

See 1

###### Proof.

We start the proof by noting that the transformation $\operatorname{\mathcal{T}}\coloneqq\operatorname{\mathcal{T}_{s\rightarrow c}}
\circ\operatorname{\mathcal{T}}_{r}\circ\operatorname{\mathcal{T}}_{\theta}$ is injective because $\operatorname{\mathcal{T}}_{\theta}$ and $\operatorname{\mathcal{T}_{s\rightarrow c}}$ are bijective and $\operatorname{\mathcal{T}}_{r}$ is injective.
The injectivity of $\operatorname{\mathcal{T}}_{r}$ can be easily seen by noting that $\bm{\theta}\neq\bm{\theta}^{\prime}\implies[\bm{\theta},r(\bm{\theta})]\neq[
\bm{\theta}^{\prime},r(\bm{\theta}^{\prime})]$, independently of $r(\cdot)$.
Since the Jacobian matrix $J_{\operatorname{\mathcal{T}}}\in\operatorname{\mathbb{R}}^{d\times d-1}$ is not squared, we cannot use the usual property of bijections in Eq. (1).
Instead, we use the definition of Jacobian determinant for injective functions in Eq. (2):

$$ $\det J_{\operatorname{\mathcal{T}}}=\sqrt{\det\Big{(}J_{\operatorname{\mathcal {T}}}^{T}J_{\operatorname{\mathcal{T}}}\Big{)}}=\sqrt{\det\Big{(}\big{(}J_{ \operatorname{\mathcal{T}_{s\rightarrow c}}}J_{\operatorname{\mathcal{T}}_{r}} J_{\operatorname{\mathcal{T}}_{\theta}}\big{)}^{T}\big{(}J_{\operatorname{ \mathcal{T}_{s\rightarrow c}}}J_{\operatorname{\mathcal{T}}_{r}}J_{ \operatorname{\mathcal{T}}_{\theta}}\big{)}\Big{)}}\;,$ (13) $$

where $J_{\operatorname{\mathcal{T}}_{r}}\in\mathbb{R}^{d\times d-1}$ and $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}\in\mathbb{R}^{d\times d}$.
According to Remark 7 we can factor out the Jacobian determinant of $\operatorname{\mathcal{T}}_{\theta}$, i.e. the bijection that precedes the dimensional inflation step with $\operatorname{\mathcal{T}}_{r}$:

$$ $\det J_{\operatorname{\mathcal{T}}}=\det J_{\operatorname{\mathcal{T}}_{\theta }}\det J_{\operatorname{\mathcal{T}_{s\rightarrow c}}\circ\operatorname{ \mathcal{T}}_{r}}=\det J_{\operatorname{\mathcal{T}}_{\theta}}\sqrt{\det\Big{( }\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}J_{\operatorname{ \mathcal{T}}_{r}}\big{)}^{T}\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c }}}J_{\operatorname{\mathcal{T}}_{r}}\big{)}\Big{)}}\;.$ (14) $$

The term $\det J_{\operatorname{\mathcal{T}}_{\theta}}$ is the standard Jacobian determinant for bijective layers and can be computed efficiently.
We are then left to compute $\det J_{\operatorname{\mathcal{T}_{s\rightarrow c}}\circ\operatorname{\mathcal
{T}}_{r}}$.
We now consider the matrix $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\coloneqq[J_{\operatorname{\mathcal{
T}}_{r}}\;\bm{0}_{d\times 1}]\in\mathbb{R}^{d\times d}$ and substitute the determinant with the pseudo-determinant:

$$ $\det J_{\operatorname{\mathcal{T}_{s\rightarrow c}}\circ\operatorname{\mathcal {T}}_{r}}=\sqrt{\det\Big{(}J_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}J_{ \operatorname{\mathcal{T}}_{r}}\Big{)}}=\sqrt{\operatorname{\mathrm{det}^{+}} \Big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{ \operatorname{\mathcal{T}}_{r}}\Big{)}}\;,$ (15) $$

where $J^{*}\coloneqq J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}J_{
\operatorname{\mathcal{T}_{s\rightarrow c}}}\in\mathbb{R}^{d\times d}$.
With $\operatorname{\mathrm{det}^{+}}$ we denote the pseudo-determinant, which is defined as the product of all non-zero eigenvalues.
The second equality follows from the fact that $J_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}J_{\operatorname{\mathcal{T}}_{r}}$ and $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname{
\mathcal{T}}_{r}}$ have the same spectrum up to zero eigenvalues, so the determinant of the former coincides with the pseudo-determinant of the latter (by definition).
To see that they share the same spectrum up to one zero eigenvalue, consider the explicit structure of the matrix product:

$$ $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}=\begin{bmatrix}J_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}J_ {\operatorname{\mathcal{T}}_{r}}&\bm{0}_{d-1\times 1}\\ \bm{0}_{1\times d-1}&0\end{bmatrix}\;.$ (16) $$

The rest of the proof is based on the key observation that $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$ has rank $d-1$ or, equivalently, that its null space is one-dimensional.
As a consequence, we can use Lemma 12 and re-write the pseudo-determinant in terms of the trace of the adjugate matrix:

$$ $\displaystyle\operatorname{\mathrm{det}^{+}}\Big{(}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\Big{)}$ $\displaystyle=\operatorname{\mathrm{Tr}}\Big{(}\operatorname{\mathrm{adj}}\big {(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname {\mathcal{T}}_{r}}\big{)}\Big{)}$ (17) $\displaystyle=\operatorname{\mathrm{Tr}}\Big{(}\operatorname{\mathrm{adj}}\big {(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}\big{)}\operatorname{\mathrm{ adj}}\big{(}J^{*}\big{)}\operatorname{\mathrm{adj}}\big{(}\tilde{J}_{ \operatorname{\mathcal{T}}_{r}}\big{)}\Big{)}$ $\displaystyle=\det\big{(}J^{*})\operatorname{\mathrm{Tr}}\Big{(}\operatorname{ \mathrm{adj}}\big{(}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\big{)}\big{(}J^ {*}\big{)}^{-1}\operatorname{\mathrm{adj}}\big{(}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}^{T}\big{)}\Big{)}\;.$ $$

In the second equality we used the property that $\operatorname{\mathrm{adj}}(AB)=\operatorname{\mathrm{adj}}(B)\operatorname{
\mathrm{adj}}(A)$ for any $A,B\in\mathbb{R}^{d\times d}$, which easily generalizes to $\operatorname{\mathrm{adj}}(ABC)=\operatorname{\mathrm{adj}}(C)\operatorname{
\mathrm{adj}}(B)\operatorname{\mathrm{adj}}(A)$.
Lastly, if $A$ is invertible, $\operatorname{\mathrm{adj}}(A)=\det(A)A^{-1}$.
In this case $J^{*}=J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}J_{\operatorname{
\mathcal{T}_{s\rightarrow c}}}$ has full rank and is thus invertible.
Since the trace is a linear operator we can take out $\det(J^{*})$, which is a constant.

Since $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$ has rank $d-1$, its nullspace is one dimensional and we can pick $x\in\mathbb{R}^{d}\;|\;\tilde{J}_{\operatorname{\mathcal{T}}_{r}}x=0$ to span the entire nullspace.
The same holds for $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$, or equivalently for the left nullspace of $J_{\operatorname{\mathcal{T}}_{r}}$, and we can pick $y\in\mathbb{R}^{d}\;|\;\tilde{J}_{\operatorname{\mathcal{T}}_{r}}^{T}y=0$.
We can easily compute $x$ and $y$ by looking at the structure of $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$:

$$ $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}=\begin{bmatrix}1&0&\cdots&0&0\\ 0&1&&&0\\ \vdots&&\ddots&&\vdots\\ 0&&&1&0\\ \frac{\partial r}{\partial\theta_{1}}&\frac{\partial r}{\partial\theta_{2}}& \cdots&\frac{\partial r}{\partial\theta_{d-1}}&0\end{bmatrix}\quad x\coloneqq \begin{bmatrix}0\\ 0\\ \vdots\\ 0\\ 1\end{bmatrix}\quad y\coloneqq\begin{bmatrix}-\frac{\partial r}{\partial\theta _{1}}\\ -\frac{\partial r}{\partial\theta_{2}}\\ \cdots\\ -\frac{\partial r}{\partial\theta_{d-1}}\\ 1\end{bmatrix}\;.$ (18) $$

We now make use of Lemma 1 for $\tilde{J}_{\operatorname{\mathcal{T}}_{r}}$, which gives us

$$ $\operatorname{\mathrm{adj}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})=\frac{ \operatorname{\mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})}{y ^{T}x}xy^{T}\>.$ (19) $$

We can now substitute Eq. (19) in Eq. (17):

$$ $\displaystyle\operatorname{\mathrm{det}^{+}}\Big{(}\tilde{J}_{\operatorname{ \mathcal{T}}_{r}}^{T}J^{*}\tilde{J}_{\operatorname{\mathcal{T}}_{r}}\Big{)}$ $\displaystyle=\det\big{(}J^{*}\big{)}\frac{\operatorname{\mathrm{det}^{+}}( \tilde{J}_{\operatorname{\mathcal{T}}_{r}})^{2}}{(y^{T}x)^{2}}\operatorname{ \mathrm{Tr}}\Big{(}xy^{T}\big{(}J^{*}\big{)}^{-1}yx^{T}\Big{)}$ (20) $\displaystyle=\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}J_ {\operatorname{\mathcal{T}_{s\rightarrow c}}}\big{)}\frac{\operatorname{ \mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})^{2}x^{T}x}{(y^{T }x)^{2}}\operatorname{\mathrm{Tr}}\Big{(}y^{T}\big{(}J^{*}\big{)}^{-1}y\Big{)}$ $\displaystyle=\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}\big{) }^{2}\|\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}\big{)}^{-1} \;y\|_{2}^{2}\;.$ $$

In the first equality we used the fact that $\operatorname{\mathrm{adj}}(A^{T})=\operatorname{\mathrm{adj}}(A)^{T}$ and we factored out $\operatorname{\mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})^{2}$ and $(y^{T}x)^{2}$, which are constants.
In the second equality we used the cyclic property of the trace and factored out $x^{T}x$.
Lastly, we substituted the numerical values $\operatorname{\mathrm{det}^{+}}(\tilde{J}_{\operatorname{\mathcal{T}}_{r}})=1$, $y^{t}x=1$ and $x^{T}x=1$.

We can now analyze the time complexity required to evaluate Eq. (4).
The Jacobian determinant for spherical to Cartesian coordinates is known [Muleshkov and Nguyen, ]

$$ $\det J_{s\rightarrow c}=(-)^{d-1}r^{d-1}\prod_{k=1}^{d-2}\sin^{d-k-1}\theta_{k}\;$ (21) $$

and can be computed efficiently in $O(d)$ time.
Therefore, we only need to show that also $w=\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}\big{)}^{-1}\;y$ can be computed efficiently.
Solving the full linear system would require a complexity of $O(d^{3})$.
However, we can exploit the almost-triangular structure of

$$ $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}=\begin{bmatrix}\frac{ \partial x_{1}}{\partial\theta_{1}}&\frac{\partial x_{2}}{\partial\theta_{1}}& \cdots&\frac{\partial x_{d-1}}{\partial\theta_{1}}&\frac{\partial x_{d}}{ \partial\theta_{1}}\\ 0&\frac{\partial x_{2}}{\partial\theta_{2}}&\cdots&\frac{\partial x_{d-1}}{ \partial\theta_{2}}&\frac{\partial x_{d}}{\partial\theta_{2}}\\ \vdots&&\ddots&&\vdots\\ 0&0&&\frac{\partial x_{d-1}}{\partial\theta_{d-1}}&\frac{\partial x_{d}}{ \partial\theta_{d-1}}\\ \frac{\partial x_{1}}{\partial r}&\frac{\partial x_{2}}{\partial r}&\cdots& \frac{\partial x_{d-1}}{\partial r}&\frac{\partial x_{d}}{\partial r}\end{bmatrix}$ (22) $$

to solve the linear system in $O(d^{2})$.
One possibility is to perform one step of Gaussian elimination, which requires $O(d^{2})$, and make the linear system triangular.
The resulting triangular system can be solved in $O(d^{2})$.
Note that we can compute $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}$ very efficiently and analytically (see Eq. (24)), without requiring autograd computations.
Overall, the determinant of the full transformation $\operatorname{\mathcal{T}}$ can be obtained as

$$ $\det J_{\operatorname{\mathcal{T}}}=\det J_{\operatorname{\mathcal{T}}_{\theta }}\det\big{(}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}\big{)}^{2}\|\big{ (}J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}\big{)}^{-1}\;y\|_{F}^{2}$ (23) $$

and can be computed efficiently in $O(d^{2})$.
∎

###### Proof.

### A.5 Implementation details

##### Implementation of injective flows for star-like manifolds

We provide some details about the implementation of the proposed injective flows and particularly for star-like manifolds in Cartesian coordinates as in Figure 2.
We implement the layers in three steps:

- •
bijective layers $\operatorname{\mathcal{T}}_{z}$ and $\operatorname{\mathcal{T}}_{\theta}$.
The first bijection $\operatorname{\mathcal{T}}_{z}:\bm{z}\mapsto\bm{z^{\prime}}$ consists of arbitrary (conditional) bijective layers conditioned on the parameter $\lambda$.
The conditioning is realized with an expressive Residual network.
Then, $\operatorname{\mathcal{T}}_{\theta}:\bm{z^{\prime}}\mapsto\bm{\theta}$ maps the transformed $\bm{z^{\prime}}$ into spherical angles $\bm{\theta}\in U_{\theta}^{d-1}$.
This last transformation is also a bijection that can be implemented with an element-wise non-linear activation like Sigmoid (hence diagonal Jacobian).
Otherwise, one could use a base distribution which is already defined on the $d-1$ spherical angles and use a bijective transformation that transforms $\bm{\theta}$ within their domain $U_{\theta}^{d-1}$ as $\operatorname{\mathcal{T}}_{\textrm{circ}}:\bm{\theta}\in U_{\theta}^{d-1}
\mapsto\bm{\theta^{\prime}}\in U_{\theta}^{d-1}$.
We use the circular bijective layers proposed by Rezende et al. [] because they allow to nicely integrate the boundary conditions arising from the use of spherical coordinates.
In particular, circular layers automatically enforce continuity of the density at the boundary of the domain.
Circular layers require the base distribution to be defined on $U_{\theta}^{d-1}$.
In practice, we use the distribution of spherical angles, which results in uniform points on the $d-1$ dimensional sphere, and can be implemented efficiently.
We use the implementation of circular layers provided in Stimper et al. [].
- •
injective layer $\operatorname{\mathcal{T}}_{r}$.
The injective step $\operatorname{\mathcal{T}}_{r}:\bm{\theta}\mapsto[\theta,r(\bm{\theta})]^{T}$ only consists in padding the spherical angles with some specified radius function $r(\bm{\theta})$.
The specific expression for the radius function depends on the manifold considered and is detailed in Eq. (25) and Eq. (26) for the $l_{p}$ (pseudo-) norm ball and for the probabilistic simplex $\mathcal{C}^{d}$, respectively.
In variational inference settings $\operatorname{\mathcal{T}}_{r}$ is not a learnable transformation.
In maximum likelihood settings trained on samples, if we assume the samples were generated from a $d-1$ star-like manifold, $r(\bm{\theta})$ can be implemented with a neural network and made learnable.
This would allow to learn the manifold and would provide with a very practical global parametrization.
- •
bijective layer $\operatorname{\mathcal{T}_{s\rightarrow c}}$: the bijective layer $\operatorname{\mathcal{T}_{s\rightarrow c}}:[\bm{\theta},r(\bm{\theta})]
\mapsto\bm{x}$ simply implements the spherical to Cartesian transformation in Eq. (6), which is a bijection and can be implemented efficiently. $\operatorname{\mathcal{T}_{s\rightarrow c}}$ is not a trainable transformation.

For the implementation we rely on the (conditional) normalizing flow library *FlowConductor*(^1^11[https://github.com/FabricioArendTorres/FlowConductor](https://github.com/FabricioArendTorres/FlowConductor)), which was introduced in Negri et al. [] and Arend Torres et al. [].

##### Efficient implementation of the Jacobian of spherical to Cartesian transformation

In order to compute the determinant in Eq. (4) we need to compute the Jacobian determinant of the transformation from spherical to Cartesian coordinates $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}$.
By looking at the definition of the coordinate transformation in Eq. (6), we can easily derive the following expression:

$$ $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}=\begin{bmatrix}-rs_{1}&rc_ {1}c_{2}&\cdots&rc_{1}s_{2}\ldots s_{d-2}c_{d-1}&rc_{1}s_{2}\ldots s_{d-2}s_{d -1}\\ 0&-rs_{1}s_{2}&\cdots&rs_{1}c_{2}\ldots s_{d-2}c_{d-1}&rs_{1}c_{2}\ldots s_{d- 2}s_{d-1}\\ 0&0&\ddots&\vdots&\vdots\\ 0&0&\ldots&-rs_{1}s_{2}\cdots s_{d-2}s_{d-1}&s_{1}s_{2}\ldots s_{d-2}c_{d-1}\\ c_{1}&s_{1}c_{2}&\cdots&s_{1}s_{2}\ldots s_{d-2}c_{d-1}&s_{1}s_{2}\ldots s_{d- 2}s_{d-1}\end{bmatrix}$ (24) $$

where we used the shorthand $s_{i}=\sin\theta_{i}$ and $c_{i}=\cos\theta_{i}$.
This allows to compute $J_{\operatorname{\mathcal{T}_{s\rightarrow c}}}^{T}$ extremely efficiently without requiring to use autograd computations and results in a significant speed up.

##### Parametrization of 𝒍 𝒑 subscript 𝒍 𝒑 \bm{l_{p}} bold_italic_l start_POSTSUBSCRIPT bold_italic_p end_POSTSUBSCRIPT (pseudo-) norm balls

Here we show how to parametrize the $l_{p}$ (pseudo-) norm balls in spherical coordinates.
Let the $l_{p}$ (pseudo-) norm of $\bm{x}\in\operatorname{\mathbb{R}}^{d}$ be defined as $\|\bm{x}\|_{p}=(|x_{1}|^{p}+\ldots+|x_{d}|^{p})^{1/p}$ with $p>0$.
We consider now the manifold defined by $\|\bm{x}\|_{p}=t$ for some $k\in\operatorname{\mathbb{R}}_{>0}$.
If we write $\bm{x}$ in spherical coordinates according to Eq. (6), we can take the radius $r$ outside of the norm and express it as a function of the $d-1$ spherical angles as:

$$ $r(\theta_{1},\ldots,\theta_{d-1})=\frac{t}{\bigg{(}|\cos\theta_{1}|^{p}+\sum \limits_{i=2}^{d-1}\bigg{|}\cos\theta_{i}\prod\limits_{k=1}^{i-1}\sin\limits \theta_{k}\bigg{|}^{p}+\bigg{|}\prod\limits_{k=1}^{d-1}\sin\theta_{k}\bigg{|}^ {p}\bigg{)}^{1/p}}\;.$ (25) $$

We can use this expression to parametrize the $l_{p}$ norm balls with the proposed injective flows.
Similarly, we can also parametrize the probabilistic simplex $\mathcal{C}^{d}$.
To see this consider the $l_{1}$ norm ball $\|\bm{x}\|_{1}=|x_{1}|+\ldots+|x_{d}|$.
If we restrict the domain to the positive quadrant $\bm{x}\in\operatorname{\mathbb{R}}^{d}_{\geq 0}$ and set the norm to 1, the resulting manifold is defined as $\|\bm{x}\|_{1}=x_{1}+\ldots+x_{d}=1$ and coincides with $\mathcal{C}^{d}$.
The radius is then parametrized by

$$ $r(\theta_{1},\ldots,\theta_{d-1})=\frac{1}{\cos\theta_{1}+\sum\limits_{i=2}^{d -1}\cos\theta_{i}\prod\limits_{k=1}^{i-1}\sin\theta_{k}+\prod\limits_{k=1}^{d- 1}\sin\theta_{k}}\quad\text{with}\quad\theta_{i}\in[0,\pi/2]\;\forall i\;,$ (26) $$

where the constraint on the angles enforces $\bm{x}\in\operatorname{\mathbb{R}}^{d}_{\geq 0}$.
Note that it is straightforward to analytically derive the expression for the partial derivatives $\frac{\partial r}{\partial\theta_{i}}$ in Eq. (26).
This makes the computation of $y$ in Eq.(4) more efficient than computing the gradients with autograd and results in a speed up.

### A.6 Applications: further details

#### A.6.1 Architecture

We use two different architectures. One is for standard NFs that we use for the subjective penalized likelihood regression problem. The other architecture is the injective flow that is used for the objective Bayes version of the regression problem and the portfolio diversification application.

##### Standard NF

It consists of a normal distribution as base distribution. Then we use 5 blocks of permutation transformation, a sum of Sigmoids layer [Negri et al., ] and an activation norm. The sum of Sigmoid layer consists each of 30 individual Sigmoid functions in three blocks.

##### Injective flows

The base distribution is either the probabilistic simplex or the complete $\|\bm{\beta}\|_{1}=1$ depending on the application. We follow this with again 5 layers of the circular bijective layers [Rezende et al., ], each consisting of three blocks with 8 bins. At the end these values are mapped to Cartesian coordinates with the proposed dimensionality inflation step.

#### A.6.2 Training

Both the standard NFs and the injective flows are trained by minimizing the reverse KL divergence with respect to the (unnormalized) target density $p(\bm{x})$:

$$ $q_{\theta^{*}}(\bm{x})=\operatorname*{arg\,min}_{\theta\in\Theta}\mathrm{KL} \big{(}q_{\theta}(\bm{x})||p(\bm{x})\big{)}=\operatorname*{arg\,min}_{\theta \in\Theta}\operatorname{\text{$\mathbb{E}$}}_{\bm{x}\sim q_{\theta}}\bigg{[} \log\frac{q_{\theta}(\bm{x})}{p(\bm{x})}\bigg{]}\>.$ (27) $$

We optimize the reverse KL divergence using Adam [Kingma and Ba, ] as optimizer with default parameters.
Notably, all trained flows converged in a matter of minutes on a standard consumer-grade GPU (RTX2080Ti in our specific case).

#### A.6.3 Penalized likelihood regression

In the next paragraph we provide further details on the experiment introduced in Section 5.2, which involves the penalized likelihood model defined in Eq. (5).

##### Synthetic dataset creation

The synthetic regression dataset is created by sampling $X^{*}$ from a 5 dimensional Wishart distribution $W_{5}(7,I)$.
The response variable $y$ is then created by $X^{*}\bm{\beta}^{*}+\epsilon$ where $\bm{\beta}^{*}$ is standard normal distributed and $\epsilon$ is normal distributed with zero mean and a standard deviation of 4.0.

##### Subjective Bayes

Figure: Figure 7: MAP solution for Laplace prior, root laplace and square laplace, which are all monotonic transformation of the Laplace distribution. The MAP solution paths coincide.
Refer to caption: x7.png

Figure: Figure 8: Solution paths for subjective prior as a function of $\lambda$, and objective priors, as a function of the norm $\|\bm{\beta}\|_{1}$. Below we report the solution paths in the MAP limit.
Refer to caption: x8.png

The subjective Bayes relies on a prior $p(\bm{\beta}|\lambda)$. The Laplace prior is given by

$$ $p_{lap}(\bm{\beta}|\lambda)\propto\textstyle\prod_{i}\exp\{-\lambda|\beta_{i}| ^{p}\}.$ (28) $$

The two other test priors are $p_{sq}(\bm{\beta}|\lambda)\propto p_{lap}(\bm{\beta}|\lambda)^{2}$ and $p_{rt}(\bm{\beta}|\lambda)\propto p_{lap}(\bm{\beta}|\lambda)^{1/2}$. Any monotonic transformation may change the $\lambda$-axis but leave the MAP solution path unchanged. This can be seen in Figure 7 where we show the MAP solution path for different subjective priors. For this visualization we reparametrize the axis such that the $\lambda$-axis is transformed into a $\|\bm{\beta}\|_{1}$-axis. This makes clear that the solution paths are equivalent.

##### Objective Bayes

The objective Bayes approach circumvents the definition of $p(\bm{\beta}|\lambda)$. The flow is directly defined on the manifolds coinciding with the contour lines of $p(\bm{\beta}|\lambda)$. As such, samples from the posterior all share a chosen norm value $\|\bm{\beta}\|_{1}=k$. Figure 8 highlights the different parametrizations of the subjective and objective approach.

#### A.6.4 Bayesian mixing model

In this section we provide details about the experiments concerning Bayesian mixing models.

##### Comparison with MCMC sampler

We compare the proposed injective flows with an ad-hoc MCMC sampler in the conjugate case (Dirichlet prior and multinomial likelihood).
In this setting we know the posterior analytically and we can compare it to both the MCMC sampler and the proposed injective flow.
In this experiment the injective flow is trained to minimize the reverse KL divergence with the unnormalized target posterior (prior times likelihood).
We compare the methods for increasing dimensionality ($d=15,30,50$) and in Figure 9 we show 95$\%$ credibility intervals of the posterior.
Results show that (i) the proposed injective flow correctly estimates the posterior distribution for all dimensions and (ii) it outperforms the MCMC sampler already in low dimensions ($d=30,50$).
Interestingly, this experiment also shows that our approach is able to learn very sparse solutions characterized by multiple modes.

Figure: Figure 9: $95\%$ posterior credibility intervals for a Bayesian model with multinomial likelihood and Dirichlet prior. We compare the posterior obtained with samples from an ad-hoc MCMC sampler (“mcmc”), samples from the approximate posterior modeled by the proposed injective flow (“flow”) and with the analytical posterior (“gt”). We evaluate the samplers for increasing dimensionality of the problem $d=15,30,50$.
Refer to caption: x9.png

##### Details about the MCMC sampler

In Bayesian mixing models the posterior lives on the simplex, which is known to be challenging for MCMC samplers [Altmann et al., , Baker et al., ] because of the constraint that the sampled vector must be positive and sum up to one.
Designing proposals close to the boundary is particularly difficult [Baker et al., ].
For sparse distributions, where mass is concentrated at the boundaries, this is especially noticeable and explains why the MCMC sampler struggles in high dimensions for sparse solutions.
What makes a fair comparison difficult is that various existing samplers [Altmann et al., , Baker et al., ] are suited for specific families of target distributions.
For ideal performance we would need to change the entire sampler depending on the target, while we can use our flow without any fine-tuning.

We compare to a Metropolis Hastings sampler that can handle mixtures of Gaussians targets well in a non-sparse setting.
It uses proposals that are by construction on the manifold by sampling from a downscaled Dirichlet distribution centered on the current state.
We tried different proposals but did not observe any relevant improvement.
As we showed, this sampler however struggles when significant sparseness is present.
Our sampler uses 1000 chains each with 100’000 samples.
In contrast, the proposed injective flow did not require any fine-tuning and learnt sparse solutions.
For a fair comparison, we set the runtime of the flow to match that of the MCMC samplers and roughly similar memory footprint ($\pm 20\%$) so both methods are compared at similar computational costs.
Lastly, we checked that running the sampler for a significantly longer time ($5\times$longer) did not achieve the level of performance of the flow.

##### Portfolio optimization

In portfolio optimization the cumulative return is often of interest. Figure 10 shows the effect of the different priors on the cumulative return.
The sparser priors lead to a slightly wider distribution of the return.
In this example, this leads to the target index being a closely matched by some of the posterior samples, where the samples of the uniform prior seem to be further away from the target index in some parts of the time interval.
The bottom row of the Figure 10 further shows the sampled sparsity patterns.
These show that the sparse priors can lead to significantly different mixtures with similar data fitting quality.

Figure: Figure 10: Cumulative return as a function of time. We plot $95\%$ posterior C.I. with the uniform prior, Dirichlet prior and $l_{p}$-norm prior. We also plot the 5 samples that are the closest to the ground truth cumulative return. In the bottom panel we visualize the weight samples as a heatmap to highlight sparsity patterns.
Refer to caption: x10.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: Q1. I believe Fig. 2 does not significantly aid in understanding the proposed approach, and a more intuitive figure could better illustrate the concept of the injective flow graphically. Additionally, moving the proof sketch to the appendix could free up space for a more intuitive figure(s), details on the mappings considered (e.g., as in Appendix A.5), and discussions on variational inference, maximum likelihood, and additional experiments.

Q2. The proposed approach appears to rely on the circular bijective layers of Rezende et al. (2020); a brief description in the appendix would be helpful. How does this approach scale in higher dimensions, and is this critical to the proposed approach?

Q3. Regarding experiment 5.1, is it possible that the density effectively modeled through the spherical flows (circular bijective layers) used for the transformation $\mathcal{T}_{\theta}$?

Q4. It would be interesting to include some (synthetic 2D or 3D) experiments where the manifold is more complicated, such as variational inference problems with a posterior concentrated on a star-shaped manifold, where $r(\theta)$ is potentially a learnable function.

Q5. Minor: What is meant by the density will be expressed in Cartesian coordinates (line 160)? Does this simply mean that the push-forward samples $x$ are just points in the ambient space?
soundness: 3
strengths: - The general idea of focusing on this class of manifolds, which has desired properties, is both simple and interesting, and the technical aspects appear solid.
- Based on the experiments conducted, the proposed model supports the claims.
- The paper is generally well-written and well-structured (see Questions for some suggestions).
summary: The authors propose a normalizing flow technique for a specific class of manifolds in which the computation of the Jacobian determinant is both exact and efficient, while related works rely on approximations or predefined manifolds. This approach is particularly relevant for variational inference problems where the manifold is known, and the goal is to approximate a density on it. The effectiveness of the method is demonstrated through variational inference on well-designed problems that meet the manifold constraints.
weaknesses: - Even if this class of manifolds is useful for density modeling in the considered scenarios, it becomes somewhat restrictive if the $r(\theta)$ is not trainable.
- Further experiments and/or comparisons with related works, such as in a maximum likelihood setting, could have been beneficial for demonstrating both the effectiveness and efficiency of the approach.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 4
questions: 1. The asymptotic efficiency of the method is shown to be O(d^2), it could be interested to look at the rate in more detail. Specifically, I am curious about the constant factors in the rate as compared to the normal normalizing flow methods. This can be done either analytically or empirically with comparisons of runtimes.
soundness: 4
strengths: 1. Paper is well written and readable.
2. The method is clearly explained and understandable, and it improves over sampling in the ambient dimension.
3. Claims are supported with proof and numerical evidence.
summary: A method is presented for efficiently computing normalizing flows on star-shaped d-manifolds embedded in (d+1)-dimensional Euclidean space. Using a decomposition based on spherical coordinates, the Jacobian determinant can be analytically computed in O(d^2) time. Numerical examples are given to show that the method outperforms the (d+1)-dimensional default sampler.
weaknesses: 1. The method only applies to star-shaped manifolds embedded in one dimension higher. The authors acknowledge the limitation that their proof cannot be extended to manifolds with higher co-dimension. This is the main concern with the method - since this method only applies to a very specific class of co-dimension 1 submanifolds, I wonder about its applicability to problems of general interest. Since the method applies to a very specific class of manifolds instead of a more general class, the benefits for this specific class need to be greater to warrant acceptance into ICLR, in my opinion
2. The numerical examples seem limited (as a result of 1). To justify this approach, more examples of star-shaped manifolds and real-world applications should be presented and/or the benefit of the method over normal normalizing flows should be significant.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: - In Theorem 1, it is stated that $\mathcal T_\theta$ can be an arbitrary diffeomorphism in $\mathbb{R}^{d-1}$. However, to the best of my understanding of normalizing flows, the evaluation of its Jacobian could be as computationally expensive as $O(d^2)$ if not specifically designed. Do you require any constraints on $\mathcal{T}_\theta$?
- How do the properties of the manifold influence the training process of the flow model?
soundness: 3
strengths: - The proposed injective flow method for star-like manifolds significantly reduces the standard complexity of $O(d^3)$ to $O(d^2)$.
- The injective flow approach appears to be both intriguing and novel, and the proof presented is sound.
- The paper outlines several potential applications, including the objective Bayesian approach and posterior inference in probabilistic mixing models.
summary: This paper addresses the substantial computational cost associated with normalizing flows when applied to densities on manifolds. The proposed method is specifically tailored for star-like manifolds and efficiently computes the Jacobian determinant, outperforming standard normalizing flows. Furthermore, the paper introduces Bayesian methods for penalized likelihood models, where penalty level-sets are regarded as start-like manifolds, and variational inference methods for probabilistic mixture models.
weaknesses: - Experiments are relatively limited, and no baselines are employed for comparison apart from the MCMC sampler in FIgure 9. I would suggest state-of-the-art normalizing flow methods and possibly other generative models for the baselines.
- The purported efficiency enhancement from $O(d^3)$ to $O(d^2)$ is not substantiated by experimental results. The authors could include runtime comparisons between their method and standard normalizing flows for increasing dimensionality $d$. This would provide empirical evidence for the claimed computational improvement.
- A minor issue: the caption of Figure 6 should include the correct probability distribution: $p(\boldsymbol\pi)|\mathcal{D}$.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: I outlined my questions and suggestions in the "weaknesses" part.
soundness: 2
strengths: The paper is easily readable and written in a clear way. As far as I know the approach is new, implementable and computationally tractable. In contrast to prior papers it covers a whole class of manifolds.
summary: The authors propose an embedding of arbitrary d-1 dimensional star-like manifolds (i.e., deformations of the unit sphere) into the $\mathbb{R}^d$ which allows the computationally efficient evaluation of its determinant. They apply this embedding for constructing normalizing flows on such manifolds, by using an injective mapping from a d-1 dimensional latent space to the manifold. The proposed model is applied to several examples.
weaknesses: My main critic on the paper is the missing comparison to existing injective flows. The authors emphasize at several points of the paper that it is important to have density evaluations of the model which are i) exact and ii) have improved complexity. As outlined in the literature part of the authors, there exist plenty of methods meeting one of the properties i) or ii). To validate the importance of meeting both at the same time, I would suggest two experiments:

i) Run an experiments to fit a not too simple distribution on a star-like manifold (maybe something like Fig 3b) and train an injective flow once with naive O(d^3) evaluation of the change-of-variables formula and once with the formula proposed in Thm 1. Then, create a plot of execution time vs dimension.

ii) Run an experiment in a similar setup for variational inference. Train an injective flow with exact determinant (as proposed) and some approximate determinant (e.g., one of the models which are cited in the related-work part) and compare the quality of generated samples (e.g. in Wasserstein/MMD or a similar two-sample comparison of distributions). Following the authors claims, the injective flow with exact determinant should lead to more accurate results than the approximate.

Even though the existing numerical experiments provide some proof-of-concept, I think that the paper could benefit a lot by evaluating more the proposed architecture itself in terms of speed and accuracy instead of using it for specific applications. The comparison with the completely different MCMC samplers looks a bit out of place since it is a completely different model which is known not to scale well along the dimensions (which is also observed in Fig 9).

Additionally, I would like to point the authors to the papers [1,2]. [1] considers a specific architectures which makes the evaluation of the necessary determinant cheap and easy, [2] which trains the normalizing flows with a VAE loss.

From a technical side, I am bit confused since the change from spherical to Cartesian coordinates is not a bijection and definitely not a diffeomorphism. The vectors $(0,\theta_2,...,\theta_{d-1},1)$ map all to the same point in the $\mathbb{R}^d$ regardless of the choice of $\theta_2,...,\theta_{d-1}$. Certainly, one can exclude sets of measure zero from domain and image of this transformation such that it becomes bijective, but then still the determinant of this transform becomes arbitrary close to zero when $\theta_1$ approaches zero such that the inverse is no longer a $C^1$-function (and hence the embedding not a $C^1$ diffeomorphism). I am not completely sure if this can cause limitations in actual applications. But the issue should be treated more thoroughly in the paper. In fact, it is easy to see that there exists no diffeomporphism from $\mathbb{R}^{d-1}$ to a sphere (or any other star-like manifold) since those have different topologies (e.g., $\mathbb{R}^{d-1}$ is contractible in contrast to star-like manifolds). A possible way to overcome such issue is the use of a mixture of several embeddings (for star-like manifolds two should be enough), see [3]. As the authors do not need too many geometrical properties of their embedding an "we exclude zero sets"-solution might be enough, but again, this issue should be discussed in the paper.

In summary, I think some numerical comparisons to existing works are necessary such that I think that the paper is below the acceptance threshhold, but I am willing to raise my score if the critics from above is addressed.


[1] Kothari et al. "Trumpets: Injective Flows for Inference and Inverse Problems", UAI 2021

[2] Chen et al. Learning flat latent manifolds with VAEs, ICML 2020

[3] Alberti et al. "Manifold Learning by Mixture Models of VAEs for Inverse Problems", JMLR 2024

Meta-review: The paper proposes injective flows for star-like manifolds of dimension $d-1$ in Euclidean space $\mathbb{R}^d$ and show that for such manifolds the Jacobian determinant can be computed exactly and efficiently, with the same computational cost as Normalizing Flows (NFs).
The proposed model is supported by several numerical experiments.

Reviewers generally agree that the paper is technically solid, the idea presented interesting, and the presentation is clear and well-structured. I agree that the proposed framework, where Jacobian can be computed exactly, is of sufficient interest to recommend acceptance.

Decision: Accept (Poster)