Multilayer Matrix Factorization via Dimension-Reducing Diffusion Variational Inference
Authors: Junbin Liu
Farzan Farnia
Wing-Kin Ma
Venue: ICML 2025 poster (2025)
Keywords: Multilayer matrix factorization
Variational inference
Variational diffusion models
Dimension reduction
Primary area: probabilistic_methods->variational_inference

Abstract: Multilayer matrix factorization (MMF) has recently emerged as a generalized model of, and potentially a more expressive approach than, the classic matrix factorization.
This paper considers MMF under a probabilistic formulation, and our focus is on inference methods under variational inference.
The challenge in this context lies in determining a variational process that leads to a computationally efficient and accurate approximation of the maximum likelihood inference. 
One well-known example is the variational autoencoder (VAE), which uses neural networks for the variational process.
In this work, we take insight from variational diffusion models in the context of generative models to develop variational inference for MMF.
We propose a dimension-reducing diffusion process that results in a new way to interact with the layered structures of the MMF model.
Experimental results demonstrate that the proposed diffusion variational inference method leads to improved performance scores compared to several existing methods, including the VAE.

Multilayer Matrix Factorization via Dimension-Reduccin Diffusion Variational Inference
Junbin Liu
Farzan Farnia Wing-King Kin Ma
Abstract
researchers in this subject has led to both diverse and sub- stantial developments. In particular, we have seen different ways to exploit the hidden structures of the underlying ma- trix factors, such as statistical independence, sparsity, and non-nee non-negativinn non-negativity. Some of such methods are equipped with desirable results such as identifiability guarantees- is, the guarantees of identifying the underlying ground-truth cestoosswwiie are essential in applications such blind source separation; sec, e.g., (Gillis, 2020; Khemakhem et al., 2020; Wu et al., 2021) and the references therein. MF is in- timately linked with the notion of learning structures from higher-dimensine to vatent-variable component analysis such as independent component analysis (ICA).
Multilayer matrix factorization (MMF) has re- cently emerged as a generalized model of, and potentially a more expressive approach than, the classic matrix factorization. This paper considers MMF under probabilistic formulation, and our focus is inference methods under variational inference. The challenge in this context lies in determining a variational process that leads to a computationally efficient and accurate approxima- tion of the maximum likelihood inference. One well-known example is the variational autoen- coder (VAB), which uses neural networks for the variational process. In this work, we take insight from variational diffusion models in the context of generative models to develop variational infer- ence for MMF We propose a dimension-reducing diffusion process that results in a new way to interact with the layered structures of the MMF model. Experimental results demonstrate that the proposed diffusion variational inference method leads to improved performance scores compared to several existing methods, including the VAE.
data. It closely related
More recently, there has been interest in multilayer, and possibly nonlinear, MF (Trigeorgis et al., 2016; Zhao et al., 2017; Xue et al., 2017; Fan, 2021; De Handschutter & Gillis, 2023). Multilayer MF (MMF) is more general model than the (two-factor) MF model, and is anticipated that MMF should provide more powerful results. It was empiri- cally shown that MMF can extract meaningful hierarchical features from data, which offers new insights in applica- tions such clustering; see, e.g., (Trigeorgis al., 2016; De Handschutter et al., 2021) and the references therein. MMF is also related to nonlinear latent-variable component analysis (Khemakhem et al., 2020). In particular, the nonlinear system is modeled by a neural network, we can see it as multilayer system.
1. Introduction
Over decades, matrix factorization (MF) methods have played a crucial role in wide variety of problems such as dimensionality reduction, low-dimension representation learning, blind source separation (Hyvärinen et al., 2023), hyperspectral unmixing (Ma et al., 2013), topic model- ing (Arora et al., 2012). community detection (Yang & Leskovee, 2013), to name few. The broad interest of
One powerful approach MF MMF is to formulate the factorization model as a latent-variabl model and treat the factorization problem as a probabilistic inference problem. In this direction, variational inference (VI) has been found to be promising in providing a practical way to handle com- plex models (Rezende et al., 2014; Ranganath et al., 2015). In particular, for MMF, variational autoencoders (VAEs) appear to be the only available solution in the literature so far; see (Khemakhem et al., 2020) and also (Li et al., 2024).
'Department of Electronic Engincering, the Chinese Univer- sity of Hong Kong, Hong Kong SAR of China DDepartment of Computer Science and Engineering, the Chinese Univer- sity of Hong Kong, Hong Kong SAR of China. Cor- Junbin Liu <liujunbin @ Wing-Ki Kin Ma
respondence
Lately, diffusion models (DMs) (Sohl-Dickstenn et al., 2015; Ho et al., 2020; Song et al., 2021; Luo, 2022) have caught tremendous attention in the context of generative models. They have been found to provide competitive performance in various generation tasks. There are several ways to derive
Farzan Farnia <farnia@ess www.eeessscccccc..... <wkma www.ee.cc...cu......
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

MMF via Dimension- Reduuing Diffusion VI
and understand DMs: We can consider stochastic differen- tial equations, seeking to reverse a diffusion process (Song et al., 2021); DMs can also be seen an outcome of denois- ing score matching and Langevin dynamics for learning the data distribution and generating data (Song & Ermon, 2019). DMs can also be derived by formulating a latent-variable model and then by performing a specific (diffusion) type of VI (Ho et al., 2020; Kingma et al., 2021)-we we are most attracted by this interpretation. Given the success of DMs, we consider this question: Can we take the VI in DMs and apply it to MMF?
training easier. This is an advantage that has been noted in the context of generative models; see (Luo, 2022).
2. Background
2.1. Multilayer Matrix Factorization
Consider the following problem. Let y E RRM denote a data point. It is modeled to follow a generative model
y = [=(= +
(1)
where So R + RM is function parameterized by O, * E RV is the latent variable associated with y. whose dimension N is assumed to be less than the data dimension M; v is noise and modeled as v ~ _N(0, o I). Let (y y/...... ..) be a given set of data points that follow the model in (1) and are independently distributed. Our goal is to estimate 0 from {y ........B Vr.) and then to estimate the latent variable nn of each yn-
So far, and to our best knowledge, DM-based VI has not been considered for MMF. And DM-based VI cannot be directly applied to MMF. This is because the current DMs assume equal dimensions with the latent variables, while MMF has unequal latent- variable dimensions. In this paper, we explore the application of DM-based VI to MMF We will propose a dimension-reduciin (DR) variational diffu- sion model. The distinct characteristic is that we associate each layer of the DM with a layer of the MMF model, and we seek to use light-weight methods to deal with each layer. This is different from DMs and hierarchical VAEs (Ran- ganath et al., 2016; Sønderby et al., 2016; Vahdat & Kautz, 2020) for generative models, which would employ deep neu- ral networks at each layer. From the proposed DR diffusion model, we will derive a VI scheme. Numerical results will be provided to demonstrate the performance of the proposed DR diffusion VI (DRD-VI).
If fe takes a linear form fe(z) = -Az, with 0 = {A), then the problem can be seen as a matrix factorization problem. In particular, by letting Y = [. yz. and z = [=1,... , =1]. the problem is essentially to recover A and Z from Y such that Y AZ. In recent years, we have seen interest in multilayer matrix factorization (MMF), which considers
A2. fo(z) = A142 ................................ ATZ, 0 (A Ar
and leads to multilayer factorization Y = A ... A. Z. In MMF, we would impose structures at each layer. Let Dt A141...ATr In non-negative MMF, we constrain 2 0 (as well as z 2 0) (Trigeorgis et al., 2016). Such MMF was numerically demonstrated to provide meaningful results in learning attribute representations of images. We have also seen interest in the following model:
It is worth noting that, in the context of generative models, there are studies that consider dimensionality reduction for diffusion models. In (Rombach et al., 2022; Wang et al., 2023), the authors apply dimensionality reduction before the diffusion model. In (Jing et al., 2022: Zhang et al., 2023), the authors concatenate multiple diffusion models, and at each stage dimensionality reduction is applied. In the aforementioned studies, dimensionality reduction is done outside of the diffusion process. Our study differs in that we embed dimensionality reduction inside the diffusion model.
fo(z) = p(Aip(Asp(-..p(
(2)
where is component-wise nonlinear activation function. The function is used to impose structures at each layer; e.g., if p is a ReLU function, we enforce non-negane non-negativity with each layer's output. The model in (2) can be viewed as a neural network, modeling a nonlinear relationship be- tween y and 2. In this sense, we are also dealing with a nonlinear latent-variable component analysis problem; see, e.g., (Khemakhem et al., 2020). Additionally, it was argued that nonlinear factorization fe(z) = p(Az) provides an effective model for low-dimensional embedding of high- dimensional data (Saul, 2022).
In addition it is interesting to have comparison with the hierarchical VAE (HVAE) approach (Sønderby et al., 2016), which, similar to the DM, is also capable of interacting with the layered structures of the model. The HVAE was not considered in the context of MMF, the best of our knowl- edge, although in principle it is possible to do so. As noted earlier, the HVAE employs deep network for each layer of the variation process. While this makes the variational process more powerful, it also makes the HVAE more dif- ficult to train. Moreover, it was argued that the stochastic approximation in the HVAE may have larger variance as the number of layers is larger (Luo, 2022). In comparison, our proposed DRD-VI adopts weight operations at each layer of the variational process, which in turn makes the
2.2. Variational Inference for MMF
We consider a probabilistic framework for MMF Consider the generative model in (1) and (2). Assume that the la- tent variable z follows a known distribution p(z), called
2

---

MMF via CiatenssionRRddciing Diffusion VI
the latent prior. For example, in independent component analysis (ICA) the latent prior is chosen as component- wise independent and non-Gaussian distribution. In simplex component analysis (SCA) (Wu et al., 2021), an important type of non-negative matrix factorization, the latent prior may be chosen as a simplex uniform distribution
hope that the choice of ds would lead to small gap be- tween & and 2, and thereby a good approximation of the log-likelihood function.
Take the famous variational autoencoder (VAE) as an ex- ample. The latent prior is Gaussian, specifically, p(z) = Nzz O. The variational distribution is chosen as (0(z)y) N(z; (o((), Diag( Diag(o)())) (v))), where Hs and d are neural networks with parameter Φ. By the parame- terization trick (see (Kingma & Welling, 2013) for details), it was found that E can be efficiently handled by stochastic approximation. This VAE approach has been employed in ICA and SCA (Khemakhem et al., 2020; Li et al., 2024).
p(z) = 12(2/Z
(3)
where A = R = =20,1 0, 1} is a unit simplex: 1x is indicator function (1x(x)) = 1 if x E X and 1x (zx 0 if æ & X)) Z is a normalizing constant. We can also consider a non-negative bounded uniform distribution
Let write down the VI problem:
(=z) 10 (z)
(4)
for non-negatin non-negative matrix factorization. The distribution of the data point y can be expressed as
max 0.6
Σ 2(0, oz T.
1
Note that the variational model parameter & is also opti- mized for best ELBO approximation given the structure of 9s. Additionally, we should mention the estimation of the latent variables once (0, x)) is obtained from the VI problem. Consider the minimum mean square error (MMSE) estimate 2n = Eo |z] There is no known tractable equation for pe The variational distribution (x(y)) can be seen as an approximation of po(zly) because the ELBO attains equality if and only if (0(2(y) = po zyy. This leads us to employ
p3())= / pa poly ())()d) (p(x) (p)y)))]
(5)
where po =Ng; of))
We want to estimate 0 from given dataset (y.....,, 3
We pursue maximum-likelihodd (ML) estimation. Let
[(:: y) = log (y))
denote the logllikelihood function for y. ML estimation determines 0 by solving
2n = F (o( [z]
(6)
3. CemeensionRReuchiie Diffusion VI for MMF
max
((0:y,):
Our endeavor is to take insight from variational diffusion models (Ho et al., 2020; Kingma et al., 2021) to develop an alternative VI scheme for MMF.
The challenge with the MI. estimatiin problem above is that the log-likelihood ((0;y) y) has no known tractable expres- dilli--imensional
sion in general; this is because (5) is integral that has no closed-form or explicit equation in gen- eral. One can approximate (5) by a stochastic (Monte Carlo sampling) approximation method, but such methods were often found be computationally inefficient in practice.
3.1. Generative Model
The generative model we consider is modification of that in Section 2.1. Denote
To y,
TT 2.
Recent research has considered variational inference (VI), together with stochastic approximation, as a more practical way to approximate the likeliool function. Let (z y be some distribution function with parameter Φ, which will be called the variational distribution in the sequel. Consider the Jensen inequality
The generation of the data point mo from the latent variable ør follows a Markov process
1t-1 = (x.) + ve, T= 1,
where He RR44 is the latent variable at layer t;
C(O: 9) 2 2(0.0: (0 = Egg
log
po(y)z)p(2) q (z(g))
((Aix)),
t=1
(7a)
fx.) = B + Cip(A
2 t < T - 1; (7b)
The function 7 is called the evidence lower bound (ELBO). The idea is to choose a Us such that C would be compu- tationally efficient to compute or approximate. We also
Ayær,
t=T
(7c)
Vt ~ NoO, Σx) represents the modeling error at layer t - for 1 2 2 and noise for & 1. The base latent variable ar
3

---

MMF via Simeension-Rducingg Diffusion VI
is distributed according to given latent prior p(ær), such as (3) and (4). The model parameter 0 contains all the A, ss, By's, C,'s, s, and s.
3.3. Dimension-Reduuing Diffusion VI
Let us examine the ELBO under the above model. We have:
Some justification should be provided for the model of fie- Eq. (7b) has a more general structure than its counterpart
po (o( = po (11) (0
(9a)
in (2), which basically considers fi.e = pl A. The merit will be clear we proceed to diffusion VI. Eq. (7c) has no activation function p. This is to facilitate our VI development which will be described later. Taking out p does not pose an issue: If AT I and Σt = 0, then -T-1 ~ 2r and 4T-2 ~ fr-1.e + vr- As a key assumption, we assume that the dimension of æ, is gradually decreasing:
q (2(21.((20) = ( ... qel (x) (20):
(9b)
where
po(04-1122) = (1 ((0 Ee))
(10)
96 (x)) -N x x (1 -aa)) I),
for L S T 1. When applying (9a) and (9b) to the ELBO
log po
[ Φ mo) = (a mmo)
(11)
g ( (
M do di N
... dr N.
it is natural to match po (0 and lø (mx (x:-1) and then derive a multilayer ELBO expression; this exactly what hierarchical VAEs do; see, e-g., Section 2.3 in (Luo,
3.2. Dimension-Reducee Diffusion Model
Now we describe our proposed diffusion variational process. Consider the following process as our chosen variational process:
2022). But this is not what diffusion VI does. It considers this alternative expression of go (Z1rr
(0( = 20 ((rT II T (x. 1 2o), (12)
x = ValU, x1-1 + VI - aie, t I...., T - 1, (8a) FT
(8b)
which is obtained by applying (0(2)/(((1) l Iz xt- 2o) and (0( zo ta) (2o) =
where 0 < a < e, (O, I) and the e, e are inddpen- dent; U, E Rd4-1xd, xde is semi-orthogonal; 4yy (ærlær-) is latem--prior-deeendent. In SCA (cf. (3)), we may choose ạ, (xT((TT))) as a Dirichlet distribution
Ua (at-1 (rt, vo) (a)))((a)laa (ar|(rr)) to (9b). With (12), we can express L as
T
2(0,0:00 = Σ (00,w:00))
(13)
2 (TTTT/T--1 Dir(arr; (ITT))))
where
where Dir(æ;; ax) denotes a Dirichlet distribution with pa- rameter a; aty is neural network with parameter y. For the non-negative latent prior in (4), we may choose 9y as Beta distribution. For ICA, we may choose Gaussian distribution in the same way as that of the VAE (ef. Section 2.2). The selection criteria of gr (xT) -1) are that (i) the support of 9-y(2 (x7-1) is the same as that of the latent prior p(ar); ); and that (ii) it has analytical expressions with its mean, covariance, and entropy. Table 3 in Appendix A.5 shows some examples. The variational model parameter Φb contains all the a,'s ss U, 's, and 7.
L (x (20) [log po (ara [21)]]
(14)
log 10(#:-1(a0z)
C. = 1,21 (20)
(15)
( ( 200)
for 2 S S1ST- -1,,- and
log p(aT)po( (0, xT x00)
E = Egs 1 mm())
(16)
In the following, we will deal with each E,
is important to note that if dg = ..... = dr and U, = I, (8a) exactly the diffusion model the context of genera- tive models. To the best of our knowledge, the dimension- reducing diffusion model in (8a) has not been considered before. The objective is not only to gradually add noise to the data point 2gg a key part with the previous diffusion models- also to gradually reduce the dimension. The dimension reduction feature is particularly relevant to MMF or the low-dimensional representation of high-diig digh-diimnsional data.
3. LAYER- TERM E
First, consider (14). Denote || x TTT It can be shown that
E x 2 (Fao((r) (2o)
[1:000 - pl ((A
+ +og |Σ||
(1 (..)))))))))))))))))))))
(17)
and that go (0 = ~M(a1; Va wy. (1-a1) This term can readily handled by stochastic approximation.

---

MMF via Dnirension-Reucccin Diffusion VI
3.3.2. LAYER-I TERM 2, 2 -
which is due log(2) S x for x > 0; this approxima- tion is good if i. is small.
Second, consider (15), As key result in diffusion models, q0( (z))) has analytical expression. It can be shown that, for 2 <
The above derivations show that VI intends to match f (xx) and fr (2, This motivates us to fix B, and C₁ in (7b) such that the structure of ft, matches that of pr (2x, 20) in (19). Specifically,
(18) x0 vo) = 11 or ( 20), veoo,,
where
Va (1 δc-1) U at C, U4 (ā. - and U.U. + vaL-11
B.
Var(1 d--1)
U 13 + vit-10, 10
(22)
11,4
dt
(19)
a [ - -00
ae With this choice, Fe is simplified
U.U.U. 100
1 at
1. = (1 - -0-1) I -1-
de U.u!! "----U" 1-
(20)
Rr = 1 C. (U-1100 - -
(23)
U.=UUUU2..U4 0e = a a4. This result is shown by using the fact that g( Be (2)) is Gaussian (for S t S
In fact, nx can be further simplified to
ac-1
U - p(A, |
T - 1). The derivations of (18) are relegated to Appendix A. Eq. (18) leads to a simplified result for L., From (15)
P. = 2(1 a
(24)
we can write
+ 2( 1)
at
|UT (U,_120 - p(Arz)) ||²
C. = E ( (200) [DLL ( (2)
(z:-1 [+),
We relegate the derivation to Appendix A.3. This gives the following result
(21) where (qlpp) ((x) log (g(z)/ (())))((())m)))))))) dæ is the Kull- back-Leible (KI.) divergence of two distributions and Let 54 and 5 denote the sets of all symmetric pos- itive semidefinite and positive definite matrices in Rdxd, respectively. Consider the following lemma.
max B.C.
E - -2 (wo) ((rr
(25)
100,(0.0 mo)
Note that go (x) -a Va, 2o. (1 (1-a,)
a)) this
can be derived from (,( The function r, can be readily handled by stochastic approximation.
Lemma 3.1. Let hii f2 E Rd, E1, Eg E S4+ & E $4. Let p(x))= = Nax En) and ((x)) = N(x: 222 Ez). Consider
3.3.3. SOME INSIGHT
( - DRL tr (2T
Let us pause a moment and try to get some intuitive insight. Eq. (25) suggests that VI intends to approximate
It holds that
U U__T 100 Ro p(A.
min f(E)) log I+ 1 1/2 (W + 1 1/2
In particular, the left-hand side is dimension-reduece æo (U,-1 is semi-orthogonal) while the right-hand side is a nonlinear low-dimension representation. Take the case of o being a ReLU function as an example. We may want the Limension-reduced data point to be non-negative and pos- sess latent happens from layer 1 to layer T.
EiES ++
where W= = ( 11 (12 11) T, and the solution to the above problem is Σ = En + W + v
The proof is provided in Appendix A.2. By applying Lemma 3.1 to Di, and noting that pe (z) in (10) and 20 (x (1 [a,, 2o) in (18) are Gaussian, we obtain
structure. And this gradually
3.3.4. LAYER -T TERM Σx
min D, = log (1 + [[f M1. (xt, 2o) ....): E,
2
Third, consider (16). If (xTT((TT)) takes a Gaussian form, then (16) may be handled in a similar way as in Sec- tion 3.3.2. If not, more work needs to be done; particularly, the key result in (18) no longer applies. Our derivations as follows. We can decompose (16)
have used II + AB I + BAI get the above equation. To facilitate the VI, we apply approximation
min
D. 5 [1,0(1))) (2 12 Pz,
E, E,
-Er x ir (0, Φb: mo) + rT Φ; mo);
(26)
5

---

MMF via Dimension- Reducing Diffusion VI
where
where have used log( S tr(A) for A E $ As described previously, ( (2r-1) (2r is chosen such that MT (x:-( (TT-(=- and ((r((arl)))) have analytical expressions. Table 3 in Appendix A.5 gives some examples. The terms rr and Fr+1 can hence be handled by stochastic approximation.
FT = (ur mn) [T (0, 2T
(27)
log 90 (xT- <0)];
Pr = (0rlorr))) 1) log Po (2r (
TT (ar 1mm)) H (-y
(28)
H (p())) = I p(a:) log p(a:)da: denotes the negative en- tropy of p(=). Also note that
3.3.5, REMAINING ASPECTS
Let assemble the components together. The VI problem
is
( ab) N(zz-ii Lu. (1 (1- -1)
T+1
min ΣΣ
+, + Σ U.U, 1|
can be shown that
0.0 T.
iT= log ATHLT (T (1); 1), Err
(
(34)
(29) where the ry 's are given in (17), (25), (33), and (28); a regu- larization term is added to enforce the semi-orthogonality
tr ETT
BT AT T uT ())))
of Up's: 2 0 is given. Also, is modified as
MT, MT (wT E ( jor 1) [TT),
(30)
E A Ar). The latent variable estimate in (6)
(MT-1) = ( ø7 1)
(31)
given by
see Appendix A.4. Consider the following lemma.
2 F go(er 2o Eao 10, (0 ( 1)}
(35)
Lemma 3.2. Consider the same settings in Lemma 3.1, except that μ4 is changed to pt (,))), which is a function of
where (r is given in (30) and is assumed to have an analytical expression; Monte Carlo sampling may be used to compute 2n.. Alternatively, we can consider
x. Let R = Eq(a)) [[x - (,(xx) (x)))x a p,(x)) Suppose
R + V is positive definite. Then
2n R MT. (Ega(ær-1 Vo,) ( (1)
f(E,)= = log Σ 1/2(12 (R+ E 1/2, min
Prrr (VāT-1U--18) r-1
ELESd
and the solution to the above problem is Σj R + V.
which does not require Monte Carlo sampling.
The proof is provided in Appendix A.2. By applying Lemma 3.2 to (27) and (29), we obtain
4. Numerical Results
1
min Pr-
log
RR + G)
(32)
Table 1. Hyperspectral images for experiments.
Erest
2
1 ύT-1
DATASET
L
dr
do
where
SAMSON
95 x 95
156
R R= 11 MT,
JASPER
100 x 100
4
198
APEX
111 x 122
4
285
(ET-11 - UT, (xTT
URBAN
307 x 307
6
162
G
Ar T. (I A
In this section, we test the proposed DRD-VI VI for MMF with the latent priors in (3) and (4). For the uniform simplex prior in (3), the variational distribution 9, (xr (x--) is chosen as a Dirichlet distribution
We assume that R+ G is positive definite, which is a fairly mild assumption. (32) looks complicated. To facilitate VI, we consider
(x<l/er-1) = Dir( (xT-1))
dr 2
min ET
iT S tr
(R+ (R+G) G)
where
(xT--1) exp(( ( (xT-)) xT-11
(Lapeer-110))
[| 11 (x
α 2(1 -WT-1)
is a one-layer network, and with y = W. For the non- negative bounded uniform prior in (4), the variational distri- bution is chosen as a Beta distribution
+ tr
= (0, Φb; zv),
(33)
( 1) B(xr; ary (xT--))) 1), (x 1));
6

---

MMF via Siinennsion-Redccing Diffusion VI
Table 2. MSF averaged over all EMs (the best MSF among 10 independent trails/standard deviation).
DATASET
SISAL
PRISM
CNNAEU
MISICNET
VASCA
DRD-VI
SAMSON
0.555/0.00
0.646/0. 13
0,453/0.08
0.461/0.00 000/00.0000
0,401/0.14 14
0.328/0.00
JASPER
0.516/0.00
0,452/0.15
0.667/0.08
0.518/0.00
0.634/0.05
0.305/0.09
APEX
0.743/0.00
0.645/0.15
0.812/0.00
0.413/0.00
0.633/0.05
0.609/0.02
URBAN
0.796/0.02
0.824/0.12
0.700/0. 12
0.955/0.00
0.785/0.03
0.677/0.04
where
analysis method (Wu et al., 2021); and deep structures, CN- NAEU (Palsson et al., 2020) and MiSiCNet (Rasti et al., 2022). We also consider the VAE method with the log-norm variational distribution proposed in (Li et al., 2024), termed VASCA. VASCA employs a linear decoder. To make the comparison fair, we extend the linear decoder to the nonlin- ear generative model (2). The dimensions of the nonlinear decoder are the same as those of DRD-VI.
a, exp W TT)- B,- exp W sTr-1),
(36)
are one-layer networks, with 7 = ( g). The ac- tivation function p is set as the ReLU function. In the experiments, we constrain E o² I. We adopt the Adam algorithm (Kingma & Ba, 2015) for optimization.
4.1. Abundance Estimation in Hyperspectral Images
The experimental settings of DRD- VI, detailed in Appendix B.I, are consistent for all the tested hyperspectral images. Each algorithm is executed with 10 random initializations. Table 2 reports the overall MSE results, while the MSE contributions from each EM are provided in Appendix B.
We first apply the proposed DRD-VI, with the uniform simplex prior (3), to the problem of estimating material abundance in hyperspectral images. This is a representative blind inverse problem in geoscience and remote sensing.
Fig. presents the estimated abundance map corresponding to the hyperspectral image Jasper. The abundance map results for other images are provided in Appendix B.
We briefly provide the background. In hyperspectral imag- ing, each image pixel is a do do-dimensional vector capturing the electromagnetic reflectances of materials across dg spec- tral bands, known as spectral signatures or endmembers (EMs). Due to limited spatial resolution, single pixel may contain mixed reflectances from multiple materials. The proportions of these EMs are modeled by a unit simplex variable ar E Rd with dr < dp- Without the precise knowledge of the mixing process, abundance estimation aims to recover the abundance map Xy E R where T is the number of pixels. Here, each column of Xr rep- resents the EMs' abundance in a single pixel, while each row shows the spatial distribution of an EM across the im- age. The task is to retrieve the low-dimensional simplex structures from the high-dimensional hyperspectral image.
The results demonstrate that DRD-VI performs compet- itively, surpassing the state-of-the-an deep structures on some datasets and consistently outperforming VASCA.
4.2. Low- Now-Dimensional Representation Learning
In this subsection, we consider DRD-VI with the non- negative bounded uniform prior in (4). We compare DRD- VI with other state-of-the-art MMF methods following prior work on MMF (e.g.. (Trigeorgis al., 2016)) that evaluates MMF methods by analyzing the learned low-dimensional representations. Specifically, given a data matrix X, with columns as i.i.d. samples, we apply clustering algorithms such as K-means to the low-dimensional representation ma- trix Xr produced by MF and MMF methods. The clus- tering results are evaluated using three standard metries: adjusted rand index (ARI) (Hubert & Arabie, 1985), accu- racy (Acc), and normalized mutual information (NMI) (Cai et al., 2005). Higher values of the three metrics indicate better performance, with maximum of 1. It is believed that higher clustering performance indicates better-learned low-dimensional representations.
We conduct experiments on four hyperspectral image datasets as listed in Table 1. We evaluate the mean squared error (MSE) defined as
1 dr dr
MSE(X X²) =
37 12/
where X is the reference ground truth provided by each dataset; ă; and ä are the i-th row of XT and X*, respec- tively. We use the Hungarian algorithm (Kuhn, 1955) to align the rows of X T returned by algorithms with the rows of X*
The benchmark algorithms are as follows: the one-layer comi-nonnegative matrix factorization (SNMF)&(Ding et al.,
The benchmark algorithms are as follows: Simplex identifi- cation via split augmented Lagrangian (SISAL) (Bioucas-
Codes for CNNAEU.
Codes for MiSiCNet.
Cooes for SNMF.
Dias, 2009); PRobabllistic SiMplex (PRISM) component
7

---

MMF via Dimension-Reduuing Diffusion VI
Ground Truth
SISAL
PRISM
CNNAEU
MiSiCNet
VASCA
DRDVI
Tree
Water
Soil
Road
Figure Estimated abundances the hyperspectral image Jasper.
Acc
NMI
ARI
FachieMNNIST
FashienMNIST
FashieMMNNST
GTSRR
SNMP
U--
0u6
0-4
DANME
DMF
CMUPPE
Oelise-/l Te
CMUPE
Oefiee-TIT Tet
CMUTTE
Semi-NMP
DRD-VI
Silbouates
Silhouctess
Sillmacties
Figure 2. Performance comparison of the MF and MMF methods across the six datasets. image datasets and x color ones.
The latent space dimensions equal 16 for gray
2008), layer-centric deep matrix factorization (LC-DMF) (De Handschutter & Gillis, 2023), deep matrix factoriza- tion (DMF)* (Fan & Cheng, 2018), deep semi-nonnegative matrix factorization (Deep Semi-NMF) Semi-NMT''TTigorrgi 2016), and deep autoencoder-lin nonnegative matrix factor- ization (DANMF)) (Ye al., 2018). In the experiments, the model dimensions for the MMF methods and the dimension of the base latent variable for all methods are identical. We test the methods on six datasets: a freely available version
of CMU PIE (Sim et al., 2002), Caltech 101 Silhouettes?, Fashion MNIST(Xiao, 2017), GTSRB(Houben et al., 2013). DTDpCiippoi et al., 2014), and Oxford-IIIT Pet (Parkhi et al., 2012). Descriptions the datasets and details experimental setups are provided Appendix B.2.
(Trigeorgis et al.,
The MF and MMF methods applied each dataset with 10 independent random initializations. For each trial, K- means clustering is performed on the learned representations with 50 independent random initializations, and the best clustering recorded. The best results among the 10 trials are shown Fig. DRD-VI generally performs well and
*Codes for DMF. 'Codes for Deep Semi-NME.
'Codes for DANME.
'Source of Caltech 101 Silhouettes.
8

---

MMF via Miatenssoo-RRddciing Diffusion VI
comparable to, in some cases outperforms, other state-of- the-art MMF methods. Due to space limitations, additional experimental results, including specific metric values and the effects of varying latent space dimensions, are provided in Appendix B.2.
De Handschutter, P. and Gillis, N. A consistent and flex- ible framework for deep matrix factorizations. Pattern Recognit., 134: 334:109102, 2023.
De Handschutter, P., Gillis, N., and Siebert, X. A survey on deep matrix factorizations. Comput, Sci. Rev. 42:100425 2021.
5. Conclusion
Ding, C. H., Li, T., and Jordan, M. I. Convex and semi- nonnegative matrix factorizations. IEEE Trans. Pattern Anal. Mach. Intell., 32(1):25--5555 2008.
This paper considered the application of diffusion model (DM)-based VI for MMF. We expanded on the idea of the existing variational DM, which assumes equal dimension with the latent variables, to propose dimension-reducing variational DM for MMF. Each layer of the DM associ- ated with layer of the MMF model, the latter of which can be seen as a shallow one-layer network (rather than deep network in DMs for generative models). DMs are known to have the benefit of simple VI, and turned that benefit to build per-layer light-weight scheme for the VI of MMF. Experimental results showed that our proposed dimension-reducing DM-based VI scheme yields promising performance, suggesting the potential variational DMs for MMF
Fan, J. Multi-mode deep matrix and tensor factorization. In Proc. Int. Conf. Learn. Represent, 2021.
Fan, and Cheng, J. Matrix completion by deep matrix factorization. Neural Netw., 98:34-41, 2018.
Gillis, N. Nonnegative matrix factorizaionn SIAM, 2020.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion proba- bilistic models. Proc. Adv. Neural Inf. Process. Syst., 33: 6840-6851, 2020.
Houben, S., Stallkamp. J., Salmen, J., Schlipsing, M., and Igel, C. Detection of traffic signs in real-world images:
Acknowledgements
The German Traffic Sign Detection Benchmark. In Proc. Int. Joint Conf. Neural Netw., pp. 1-8. IEEE, 2013.
This work was supported by a General Research Fund (GRF) of Hong Kong Research Grant Council (RGC) under Project ID CUHK 14203721.
Hubert, L. and Arabie, P. Comparing partitions. J. Classif. 2:193-218, 1985.
Hyvärinen, A., Khemakhem, I., and Morioka, H. Nonlinear independent component analysis for principled disentan- glement in unsupervised deep learning. Patterns, 4(10), 2023.
Impact Statement
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Jing, B., Corso, G., Berlinghieri, R., and Jaakkola, T. Sub- space diffusion generative models. In Eur. Conf. Comput. Vis., pp. 274-289, Springer, 2022.
References
Khemakhem, I., Kingma, D., Monti, R., and Hyvarinen, A. Variational autoencoders and nonlinear ICA: unifying framework. In Proc. Int. Conf. Artif. Intell. Stat., volume 108, pp. 2207-2217. PMLR. 26-28 Aug 2020.
Arora, S., Ge, R., and Moitra, A. Learning topic models- going beyond SVD. In Proc. Annu. IEEE Symp. Found. Comput. Sci., pp. 1-10. IEFE, 2012.
Bioucas-Dias, J. M. A variable splitting augmented La-
Kingma, D., Salimans, T., Poole, B., and Ho, J. Variational diffusion models. Proc. Adv. Neural Inf. Process. Syst., 34:21696-21777, 2021.
grangian approach to linear spectral unmixing. In Proc. Workshop Hyperspectral Image Signal Process. Evol. Re- mote Sens., pp. 1-4. IEEE, 2009.
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Proc. Int. Conf. Learn. Represent. San Diego, CA, USA, 2015.
Boyd, S. Convex optimization. Cambridge UP, 2004.
Cai, D., He, X., and Han, J. Document clustering using locality preserving indexing. JEEE Trans. Knowl. Data Eng., 17(12):1624-1637 2005.
Kingma, D. P. and Welling, M. Auto-encoding variational Bayes. In Proc. Int. Conf. Learn. Represent. 2013.
Cimpoi, M., Maji, S., Kokkinos, I., Mohamed, S., and Vedaldi, A. Describing textures in the wild. In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014.
Kuhn, H. W. The Hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83- 97, 1955.

---

MMF via Mnntension-Reduciing Diffusion VI
Li, Y., Fu, X., and Ma, W.-K. Probabilistic simplex com- ponent analysis via variational auto-encoding. In Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pp. 9671 9675. IEEE, 2024.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilib- rium thermodynamics. In Proc. Int. Conf. Mach. Learn., pp. 2256-22662 2265. PMLR, 2015.
Luo, C. Understanding diffusion models: A unified perspec- tive. arXiv preprint arXiv:2208. 1970, 2022.
Sønderby, C. K.,, Raiko, T., Maaløe, L., Sønderby, S. K., and Winther, O. Ladder variational autoencoders. Proc. Adv. Neural Inf. Process. Syst., 29, 2016.
Ma, W.-K., Bioucas- Dias, J. M., Chan, T.-H., Gillis, N., Gader, P., Plaza, A. J., Ambikapathi, A., and Chi, C.-Y. A signal processing perspective on hyperspectral unmixing: Insights from remote sensing. IEEE Signal Process. Mag., 31(1 31(1):67-81) 2013.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. In Proc. Adv. Neural Inf. Process. Syst. volume 32, 2019.
Song, Y., Sohl- J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In Proc. Inf. Conf. Learn. Represent. 2021.
Meyer, G. P. An alternative probabilistic interpretation of the Huber loss. In Proc. Conf. Comput. Vis. Pattern Recognit. 2021.
pp.
Trigeorgis, G., Bousmalis, K., Zafeiriou, S., and Schuller, B. W. A deep matrix factorization method for learning attribute representations. IEEE Truns. Pattern Anal. Mach. Intell., 39(3):417-429, 2016.
Palsson, B., Ulfarsson, M. O., and Sveinsson, J. R. Con- volutional autoencoder for spectral-spatial hyperspectral unmixing. IEEE Trans. Geosci. Remote Sens., 59(1): 535-549, 2020.
Vahdat, A. and Kautz, J. Nvae: A deep hierarchical varia- tional autoencoder. Proc. Adv. Neural Inf. Process. Syst., 33:19667-19679, 2020.
Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C. Cats and dogs. In Proc. Conf. Comput. Vis. Pattern Recognit., pp. 3498-3505. IEEE, 2012.
Wang. W., Xu, Y., Feng, F., Lin, X., He, X., and Chua, T.-S. Diffusion recommender model. In Proc. Int. ACM SIGIR Conf. Res. Dev. Inf. Retr. pp. $32-841, 2023.
Ranganath, R., Tang, L.,, Charlin, L., and Blei, D. Deep exponential families. Artif. Intell. Stat., pp. 762-771 PMLR, 2015.
Wu, R., Ma, W.-.., Li, Y., So, A. M.-.,, and Sidiropoulos, N. D. Probabilistic simplex component analysis. IEEE Trans. Signal Process., 70:582-599 2021.
Ranganath, R., Tran, D.. and Blei, D. Hierarchical vari- ational models. In Proc. Int. Conf. Mach. Learn., pp. 324-333, PMLR, 2016.
Xiao, H. Mashion-MNIST: a novel image dataset for bench- marking machine learning algorithms. arXiv preprint arXiv: net::0780777777 2017.
Rasti, B., Koirala, B., Scheunders, P., and Chanussot, J. MiSiCNet: Minimum simplex convolutional network for deep hyperspectral unmixing. IEEE Trans. Geosci. Re- mote Sens., 60:1-15, 2022.
Xue, H.-.., Dai, X., Zhang, J., Huang, S., and Chen, J. Deep matrix factorization models for recommender systems. In Proc. Int. Joint Conf. Artif. Intell., volume 17, pp. 3203- 3209. Melbourne, Australia, 2017.
Rezende, D. J., Mohamed, S., and Wierstra, D. Stochas- tic backpropagation and approximate inference in deep generative models. In Proc. Int. Conf. Mach. Learn., volume 32, pp. 1286-1286. PMLR, 2014.
Yang. J. and Leskovec, Overlapping community detection scale: nonnegative matrix factorization approach. In Proc. ACM Int. Conf. Web Search Data Min., pp. 587-596. 2013.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffuuion models. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 10684-10695, 2022.
Ye, F, Chen, C., and Zheng, Z. Deep autoencoder-lit nonnegative matrix factorization for community detection. In Proc. ACM Int. Conf. Inf. Knowl, Manag., pp. 1393- 1402, 2018
Saul, L. K. A nonlinear matrix decomposition for mining the zeros of sparse data. J. Math. Data Sci., 4(2):431-46) 2022.
Zhang. H., Feng, R., Yang, Z., Huang, L., Liu, Y., Zhang, Y., Shen, Y., Zhao, D., Zhou, J., and Cheng, F. Dimensionality-varyinn diffusion process. In Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. pp. 14307-14316, 2023.
Sim, T., Baker, S., and Bsat, M. The CMU pose, illumina- tion, and expression (PIE) database. In Proc. Int Conf. Autom. Face Gesture Recognit,, pp. 53-58. IEEE, 2002.
10

---

MMF via Siitennsion-Redciing Diffusion VI
Zhao, H., Ding, Z., and Fu, Y. Multi-view clustering via deep matrix factorization. In Proc. AAAI Conf. Artif. Intell., volume 31, 2017.

---

MMF via Dimension Dimension-Reduuingg Diffusion VI
A. Derivations and Proofs
A.1. Derivation of (18)
Using Bayes' rule, we have
log ((2; Fe, Fo) log g( xt-1, (0-1,,009900 (0) 20
g( x0)
Na U, xt-1, (1 az) 1t-1: 10-1iva0-11 10-11U L0, (1 a2-1)
= log
N/ D1i Vo,U, To. (1-0a)
(37)
T 11-1 1- a,a. T U,U, xt-1 2aax Ut
U, 1-1
constant
at
04-1
(1 T at
U.U. 1 1
( Voe U,æ,
v U,
T Li-
11-1 B1-1+ 27-1
constant.
0t
1- at
1 at-1
We see that the above takes quadratic form and can be shown that g(xx--1/22)) (2,, wo is still Gaussian distribution. The covariance matrix is given by
a0 U
111)
2ro =
1
1- az-1
0, (1(1 āx-1)
U₂ I+ a:(1 a:(1- --
-
= (1 ac-1)
I
U/
1 -
(38)
0r (1 - :z-1)
1 ar ((1--0-11)
U.U?)
= (1 اz-1) 1)
1 0t
Ot - āt U,U.
= (1 - Ūz-1) 1)
1
1 at
where we have used the matrix inverse formula
(I+ -I-
(39)
for matrices X and Y with proper sizes in the second line, and the semi-orthogonaliiy of U, in the third line. The mean is given by
Vai U + Di Va+-1 U U To
Ve.
-44
1-0
Var(1
az-1)
aryaa (1 0;-1)2
U,
Ot
(1 (1 (1 a+)
(40)
+ va--10 10,1 T
val-ia1()) - U.U.U.1100
1-1
1- 0z
va (1 0z-1)
/ 0t) UUU. 1 Co.
U + Vo 00
1-
A.2. Proof of Lemma 3.1 and Lemma 3.2 First, we consider Lemma 3.1. It can be verified that
2 log Σn 1 (x() Ey(an) (x 11)( 14) p1)
1
d log(2 (2T)
log
(41)
-R
1
Eq(m) [logg(2)) g(x)] = log Ez
(1 + log(2ir)).
(42)
2
2
12

---

MMF via Dimension- Reduucng Diffusion VI
The function can be written
f(2)) (log Σ + tr (Ei V))) +
(43)
where
1 (log] Σ2| + d). It is known that the solution to min E.
f(E1) is uniquely given by Σ = R + V if R +
V
2
is positive definite (PD). To put this into context, consider the change of variable Y = ET The corresponding objective
function
1 f(Y) //4 (-log Y Y +
V)))
is convex, and its gradient equals
VS(Y) = (-Y + R + v)
see, e.g., (Boyd, 2004). It is easy to verify that
R= E=+ - M2) - 14)
(44)
Also, since Σg is PD, is also PD. Putting the optimal solution Σ into f gives
1 (log R Tp/ log Στ)) log ΣS 1/2 1/2(22 V 1/1
1 2
1(2)) =
(45)
and applying (44) to (45) gives the desired result.
Next, we consider Lemma 3.2, The proof is identical to the above, with the previous R being replaced by
R = (()()) [[0 (xxx))(x== 1=(=))
A.3. Derivation of (24)
Recall that the covariance matrices of (3 [2,, 2o) and po (2x)) are the same. The KL divergence can be written as
DKL (x:- zo) ppo -
@1.. 2 (fre(x))) - ( ( 20)) T
2 [1,)(a))))) - 14 (3r, |1
+ constant
1
-1
at
-U.U
+ 1- _z-1 1) _ (fre(x)) - Me. (Zi, zo)) + constant
1
(46)
at
1 ( at |U (Fro(a))) - He (xr, (o))
([5))((0)) - (2, 2
2o)1 112)
+ constant.
2 1- at
1 ū,-
Based on (22), we can further write
UT. an(1 -u(--a--)v U. (U - pl A1x))) 12
Gt |0 (fro(())) - (1,0
a,a=-1
1 Ot
- at
1 1 ãR |u. (U - p(A 11:
(47)
a1(1 ae)
(I - az)²
and
1
(1,0(1))) - (2,.
1 ā=-1
0-11
(I- (e(1 - ā4-1)
1- U.U..)
(0 p(A,zz)
1 01-1
(48)
(a. - a2) - 2e) U.U!) (UI, p(A,z))
Ü - p(Arz))
I
(1-a))² 2z)²
ā:-1
2 - - p(Ax2))
0z - σe)
UU. (U_,00 p(Arx))
(1 0z)²
13

---

MMF via Dimension- Diffusion VI
Adding (47) and (48) up gives
DKL (2 ( To (x)
0--1
U 101 p(A
1/
ū - ar)
0x(2 or - a4))
(U_ o(Arz))) (49)
2(1 )
2 (1 x,)²
(1 x,)
0--1
U 1011200 120 - p(A
|
at
U (U.- xo ,(Arz)) 11
2(1 a
2( - 1)
which leads to the Ft in (24).
A.4. Derivation of (29)
We write Pr
TT (ZrleT
[lg Pa (2T-1
(
[log.1 N Arxr, Er)
-I ((r(err)) 1)
| Arærll
-2 log (2th) (2T|Er)) Erl)
2
1
(1 + ||A 113;, -207 En ATεT)
log (2 Er()
(ar 1)
2
||rr-1
12
+ tr A EET Ar
(ra((rr-)))
-1 ATHLT (xT-1)
1
log E)
= 2
(1æT-11 1 + | p4,, (2T (2T- -207 En ATÆT)
log (2 Erl)
2
tr ET Ay M xz
(xr- AT
= log..o (1 ( 1). ET) tr ET (
(50)
which gives the result in (29).
A.5. Examples of applicable prior distributions
Table 3 presents examples of applicable distribution pairs. We clarify some notation. Given two vectors and y of the same dimension, @ and z/y denote the element-wise product and division, respectively. The symbols I(.) and vh()) denote the Gamma and Digamma functions, respectively. Given vector æ, |az|, [(x), ((x), log(æ), exp(x) and x² denote the element-wise operations of their scalar counterparts. The multivariate Beta distribution B(æ; or, 0, is defined as II, B(z,; ag, 3.) where [xx; a. 3) is the probability density function of the one-dimensional Beta distribution; the same applies the Laplace distribution. The symbol e collects irrelevant constants.
B. Experimental Setups and Additional Results
B.1. Abundance Estimation in Hyperspectral Images
The settings of DI are listed in Table 4. For VASCA, the model dimensions, learning rate, batch size, and the number of epochs are set the same. Table 5 provides a more comprehensive MSE results for each hyperspectral image including contributions from each EM. Figs. 3-5 show the estimated abundance maps. The results show that the deep structure MiSiCNet performs well on the Apex dataset. DRD-VI consistently outperforms VASCA and is very competitive in general.
B.2. oow-Dimensional Representation Learning
The datasets used are summarized in Table 6. Table presents the experiment settings of the DRD-V VI methods which are the same for all the datasets. The model dimensions of all other MMF methods are set the same as those of DRD-VI. Tables
14

---

MMF via SiinennsionRRducinn Diffusion VI
Table 3. Examples of distrihution pairs.
DISTRIBUTION
SETTING
EXPRESSION
p(ar)
L(rr. 1,1) 2 ( (ar-)) (xr--1))
(aty (1)
LAPLACE (MEYER, 2021)
Mr (2T-1)
(3rr)
DIAG(2b2))
Maa (ar mr--1))
exp ( (1,1/bb) + |,,1 log +c
p(ay)
DIR(Zy:
9y (2)-)
D (2r--))
DIRICHLET
Mr,,(2 (20--)
04/(1 (4)
(Ir(
(DIAG(HA) Mm.,) (1
H(g, or 1))
(a, 1)" (v((a)) - (1 (1)a.) a.))
log Fat an)
T( +
p(ær)
B(ær; 1,1)
Mr (#r(er--)
B(æri (, (,,(111))
BBTA
0,, (a + Ba)
(1n(
DIAG (aay By)/ (a, 3.) /(a By + +1)
(0r or 1)) (ay - V(a,) + (8, 1) v(8,)) (ay, + B, 21) (ay + 34,) 1 log
Table 4. Experimental settings DRD-VI abundance estimation.
[d1, d2, ... dr)
A
BATCH SIZE
EPOCH LEARNING RATE
664, 32, 16, 8, dr)
10
ROUND(L/100)
500
0.001
Ground Truth
SISAL
PRISM
CNNAEU
MiSiCNet
VASCA
DRD-VI
Soil
Tree
Water
Figure 3. Estimated abundances for the hyperspectral image Samson.
9, and 10 present the results of applying the MF and MMF methods to the datasets, with the latent space dimension being 16 for gray image datasets and 16 3 for color ones. Note that the Deep Semi-NMF method uses a deterministic initialization.
We also present the results varying the latent space dimension, with the other settings kept the same as before. Fig.
15

---

MMF via SiaiennsionRRduccing Diffusion VI
Table 5. MSE results of ahundance estimation (the best MSE among independent triall/standard deviation).
DATASET
ENDMEMBER
SISAL
PRISM
CNNAEU
MISICNET
VASCA
DRD-VI
SOIL
0.387/0.00 0.000
0.426/0.21
0.410/0.11
0.351/0.00
0.408/0.22
0.269/0.01
TREE 0.494/0.00
0.630/0.22
0.469/0.07
0.335/0.00
0.380/0.09
0.314/0.01
SAMSON
WATER
0.785/0.00
0.882/0.12
0.480/0.14
0.697/0.00
0.416/0.10
0.401/0.01
AVG. MSE
0.555/0.00
0.646/0.13
0.453/0.08
0.461/0.00
0.401/0.14
0.328/0.00
TREE
0.432/0.00 1032/0.00
0.498/0.14
0.450/0.
0.190/0.00
0.469/0.08
0.275/0.00
WATER
0.457/0.00
0.142/0.28
0.266/0.07
0.214/0.00
0.575/0.02
0.220/0.08
JASPER
SOIL
0.605/0.01
0.597/0.25
0.872/0.
0.578/0.00
0.696/0.10
0.297/0.05
ROAD
0.569/0.00
0.570/0.15
1.079/0.16
1.093/0.00
0.794/0.22
0.428/0.23
AVG. MSE
0.516/0.00
0.452/0.15
0.667/0.08
0.518/0.00
0.634/0.05
0.305/0.09
ROAD
0.911/0.00
1.173/0.29
1,596/0.29
0.648/0.00
1.022/0.08
0.978/0.06
TREE
0.543/0.00
0,488/0.13
0.567/0.09
0.255/0.00
0,387/0.08
0.369/0.08
APEX
ROOF
0.664/0.00
0.427/0.15
0.615/0.09
0.299/0.00
0.619/0.05
0.389/0.02
WATER
0.853/0.01
0.491/0.22
0.471/0.15
0.452/0.00
0.503/0.09
0.698/0.01
AVG. MSE
0.743/0.00
0.645/0.15
0.812/0.06
0.413/0.00
0.633/0.05
0.609/0.02
ASPHALT
0.816/0.02
0.923/0.17
0.626/0. 10
0.539/0.00
0.720/0.07
0.617/0.05
GRASS
0.659/0.00 04
0.549/0.17
0.590/0.07
0.515/0.00
0.538/0.06
0.480/0.04
TREE
0.668/0.03
0,468/0.21
0.496/0.07
0.587/0.00
0.695/0.08
0.542/0.01
URBAN
ROOF
1.004/0.
1.024/0.45
0.581/0.09
0.638/0.00
0.895/0.09
0.566/0.01
METAL
0.907/0.05
1.016/0.21
0.977/0.71
2.823/0.00
0.979/0.10
1.337/0. 14
DIRT
0.722/0.06
0.963/0.11
0.927/0.24
0.626/0.00
0.883/0.07
0.517/0.04
AVG. MSE
0.796/0.02
0.824/0.12
0.700/0.12
0.955/0.00
0.785/0.03
0.677/0.04
Ground Truth
SISAL
PRISM
CNNAEU
MiSiCNet
VASCA
DRD-VI
Road
Tree
Roof
Water
Figure 4. Estimated abundances for the hyperspoctral image Apex.
7, and 8 present the results. DRD-VI is generally highly competitive compared to the other state-of-the-arr methods and demonstrates a clearly better performance on the CMUPIE dataset.
16

---

MMF via Dimension-Reducing Diffusion VI
Ground Truth
SISAL
PRISM
CNNAEU
MiSiCNet
VASCA
DRD-VI
Asphalt
Grass
Tree
Roof
Metal
Dirt
Figure 5. Estimated abundances for the hyperspectral image Urban
Table 6. Image datasets used for low- dineninnnio representation learning.
DATASET
DIMENSION
SAMPLES
CLUSTERS
DATA TYPE
CMU PIE (FREE VERSION)
32 x 32
2856
68
FACES
SCHHON-MNIST (TESTING SET)
28 x 28
10000
10
CLOTHES
CALTECH 101 SILHOUETTES
28
6407
101
OBJECT SILHOUETTES
GTSRB (TESTING SET)
32
12630
TRAFFIC SIGNS
OXFORD-IIIT PET (TESTING SET; RESIZED)
40 x
40 x 3
3680
37
PETS
DTD (TESTING SET; RESIZED)
50
50 x 3
1880
47
DESCRIBABLE TEXTURES
Table 7. Low-dimensional representation learning experiment settings of DRD-VI.
Data Type
d, d2, ... dr)
Batch Size
Epoch
Learning Rate
Gray Image
256, 128, 64, 32, 16
10
ROUND(L/100)
500
0.001
Color Image
2556, 128, 64, 32, 16] x 3
108
Table 8, Accuracy (the best result among 10 independent trials/standard deviation).
METHODS
CMU PIE
GTSRB
FASHIoN-MNIST
OXFORD-IIIT PET
SI..OUETTESS
DTD
SNMF
0.237 0.237/0.01 /0.01
0.199/0.01
0.513/0.01
0.081/0.00
0.115/0.00
0.094/0.00
DANME
0.223/0.01
0.186/0.00
0.497/0.00
0.079/0.00
0.118/0.00
0.098/0.00
LC-DME
0.192/0.02
0.179/0.02
0.501/0.08
0.090/0.01
0.217/0.01
0.114/0.00
DMF
0.179/0.01
0.211/0.00
0.620/0.04
0.092/0.00
0.299/0.01
0.109/0.00
DEEP SEMI-NME
0.395
0.279
0.452
0.081
0.304
0.085
DRD-VI
0.615/0.01
0.290/0.01
0.588/0.00
0.092/0.00
0.300/0.01
0.100/0.00
17

---

MMF via Minienssoo-RRduccin Diffusion VI
Table 9. Normalized mutual information (the best result among 10 independent trials/standard deviation).
METHODS
CMU PIE
GTSRB
FASHION-MNIST
OXFORD-III IIIT PET
SILHOUETTES
DTD
SNMF
0.428/0.01
0228/0.011
0.457/0.01
0.076/0.
0.320/0.00
0.137/0.00
DANMF
0.407/0.01
0.286/0.00
0.449/0.00
0.076/0.
0.318/0.00
0.162/0.01
LC-DMF
0.421/0.03
0.281/0.03
0,414/0.06
0.109/0.01
0.441/0.02
0.201/0.00
DMF
0.417/0.02
0.332/0.01
0.549/0.00
0.108/0.00
0.535/0.00
0.163/0.01
DEEP SEMI-NME
0.677
0.416
0.467
0.100
0.530
0.132
DRD-VI
0.792/0.01
0.412/0.01
0.601/0.00
0.103/0.00
0.532/0.00
0.160/0.00
Table 10. Adjusted rand index (the best result among 10 independent trials/standard deviation).
METHODS
CMU PIE
GTSRB
FASHION-MNIST
OXFORD-IIIT
SILHOUETTES
DTD
SNMF
0.103/0.01
0.061/0.00
0.299/0.01
0.007/0.00
0.057/0.00
0.012/0.00
DANMF
0.092/0.00
0.058/0.00
0.272/0.00
0.007/0.00
0.059/0.00
0.014/0.00
LC-DMF
0.080/0.01
0.062/0.01
0.287/0.00
0.013/0.00
0.213/0.03
0.015/0.00
DMF
0.075/0.01
0.069/0.01
0.414/0.03
0.013/0.00
0.280/0.01
0.020/0.00
DEEP SEMI-NME
0.304
0.151
0.296
0.011
0.263
0.008
DRD-VI
0.479/0.01
0.108/0.01
0.445/0.00
0.013/0.00
0.261/0.01
0.016/0.00
Acc performance across datasets
CMUPIE
GTSRB
FashionMNIST
0.3
0.6
0.6
0.4
Acc
Acc
Acc
0.5
0.2
0.2
0.4
0.1
10
20
30
40
60
80
10
20
30
SNMF
DANMF
LC-DMF
DMF
Deep Semi-NMF
-O
DRD-VI
Oxford-IIIT Pet
Silhouettes
DTD
0.3
0.12
0.09
Acc
Acc
0.2
Acc
0.1
0.08
0.08
0.1
0.07
40
60
80
10
20
30
40
60
80
Latent space dimension
Latent space dimension
Latent space dimension
Figure 6. Accuracy vs. latent space dimensions.
18

---

MMF via Minienssoo-RRduccing Diffusion VI
NMI performance across datasets
CMUPIE
GTSRB
FashionMNIST
0.8
0.6
0.4
0.5
NMI
0.6
NMI
NMI
0.4
0.4
0.2
10
20
30
40
60
80
10
20
30
SNMF
DANMF
LC-DMF -X DMF
Deep Semi-NMF
DRD-VI
Oxford-IIIT Pet
Silhouettes
DTD
0.12
0.6
0.2
0.1
NMI
NMI
NMI
0.4
0.15
0.08
0.06
0.2
0.1
40
60
80
10
20
30
40
80
Latent space dimension
Latent space dimension
Latent space dimension
Figure Normalized mutual information vs. latent space dimensions.
ARI performance across datasets
CMUPIE
GTSRB
FashionMNIST
0.6
0.15
0.5
0.4
0.4
ARI
ARI
0.14
ARI
0.2
0.3
0.05
0.2
0
10
20
30
40
60
80
10
20
30
SNMF
DANMF
LC-DMF
DMF
Deep Semi-NMF
0
DRD-VI
x (Ifford-IIIT Pet
Silhouettes
DTD
15
0.3
0.02
0.2
0.015
ARI
10
ARI
ARI
0.1
0.01
5
0.005
40
60
10
30
40
80
Latent space dimension
Latent space dimension
Latent space dimension
Figure Adjusted rand index latent space dimensions
19


Reviews:
[{"review_id": "0VxMPstSfD", "reviewer": "Reviewer_TTJj", "summary": "The paper presents presents a diffusion model (DM) based variational inference (VI) method for multilayer matrix factorization (MMF). They derive a variational process which is computationally efficient and lighter weight than other methods such as VAEs. Their method DRD-VI also reduces latent dimensionality at each step of the diffusion process, satisfying the requirement of matrix factorization to learn a lower dimensional representation of the data. The paper compares the performance of DRD-VI with multiple MF and MMF baselines on the problems of abundance estimation and low-dimensional representation learning with multiple datasets. They find DRD-VI outperforms the baselines on most datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very thankful for your thoughtful comments. We will do our best to revise and improve the paper. Here we would like to give a reflection on the main points you raised.\n\n**Regarding “Essential References not Cited”**\n\nThank you for pointing out some references related to dimension-varying diffusion. We agree that we should cover that, and we plan to add a paragraph in the Introduction to describe the related works and how the current work differs.\n\nLet us take this opportunity to give an explanation. The related papers consider the context of generative models and generative recommendation models. They, from a high-level perspective, can be seen as concatenations of dimensionality-reduction processes and diffusion models. Dimensionality reduction is done outside of the diffusion process. Our proposed method considers the context of multilayer matrix factorization (MMF). We embed dimensionality reduction inside the diffusion process. Each layer of the factorization model (or each layer of the neural network) is associated with one layer of the diffusion process. These two features are not seen in the prior literature in generative models, to the best of our best knowledge, and are the unique parts of our development. In the context of MMF, the two features give us the opportunity to develop a per-layer light-weight scheme for MMF inference. \n\nWhile there are differences between the prior works and our current work, both fundamentally and in the topic of interest, we look back and agree that we should cover the prior papers in dimension-varying diffusion. \n\n**Regarding “Questions to Authors”**\n\nQ1: Absolutely, we can provide the codes. You can find the codes at the [anonymous GitHub repository](https://github.com/AnonymousPaper-Submission/ICML25-submission-codes-share).\n\nQ2: It could be difficult to give a reason when different methods use different models. MiSiCNet considers spatial correlations in its model, and this may give an edge to MiSiCNet in some instances.", "claims_and_evidence": "The claims are well supported and derivation of the proposed VI process is clear.", "methods_and_evaluation_criteria": "The proposed evaluation criteria are appropriate for the problem of MMF. The paper evaluates DRD-VI on the blind inverse problem of abundance estimation of hyperspectral images as well as low dimensional representation learning of six different datasets.", "theoretical_claims": "I did not thoroughly check each proof in the appendix, but the main equations in the paper look correct.", "experimental_designs_or_analyses": "The experimental designs are valid as the authors evaluate on the established MMF blind inverse problem as well as the problem of low-dimensional representation learning using standard datasets such as CMU-PIE, Fashion-MNIST, etc.", "relation_to_broader_scientific_literature": "The key contribution of the paper extends existing variational inference methods for diffusion models to the problem setting of MMF. By designing the VI process for diffusion to reduce dimensionality at each step with light weight layers, this paper connects to the broader areas of multilayer matrix factorization and diffusion model variational inference.", "essential_references_not_discussed": "How do latent diffusion models fit in the context of this paper? The paper notes that diffusion models assume the latent space has the same dimensionality as the data, however there is existing work on latent diffusion models in a lower dimensional latent space including Rombach, Robin, et al. \"High-resolution image synthesis with latent diffusion models.\" (Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022).\n\nAnother point in the paper is that DM-based VI has not been considered for MMF which may be the case, however there has been work done on related dimensionality reduction problems including diffusion based recommendation models such as Wang, Wenjie, et al. \"Diffusion recommender model.\" (Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2023.) This paper in particular proposes a method L-DiffRec which compresses the data to a lower latent dimensionality and proceeds with the diffusion process in this latent space.\n\nI believe both of these works as well as the literature on lower dimensional latent diffusion models as well as recommendation systems should be contextualized as related work in this paper.", "other_strengths_and_weaknesses": "Strengths:\n- The paper provides a thorough description and derivation of a VI method for diffusion model based MMF.\n- The experimental results show DRD-VR is promising for multiple datasets.\n- The method design choices are well justified.\n\nWeaknesses:\n- While the experimental results look good, the authors do not provide code which would help answer the questions regarding reproducibility and transparency.\n- The authors should contextualize latent diffusion models and diffusion based recommendation models in their related work.", "other_comments_or_suggestions": "- Spelling error for section 2.2 title: \"2.2. Varational Inference for MMF\" should be \"2.2. Variational Inference for MMF\".\n- In section 4.1, you misspell \"MiSiCNet\" as \"MiSiNet\" when introducing baselines (line 367).\n- Table 2 in the paper has VAE listed as a method, should this be changed to VASCA as Figure 1 has it listed as VASCA?", "questions_for_authors": "- For the sake of reproducibility and future research in this area, can the authors release their code?\n- The MiSiCNet results look better than DRD-VI for some settings (abundance estimation with the APEX dataset). Is there a reason or hypothesis for why this is the case?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tH57fGoDj6", "reviewer": "Reviewer_VFiD", "summary": "The paper introduces a novel diffusion-model based variational inference method for multilayer matrix factorization (MMF), using a dimension-reducing Markov chain as the noise. The method is evaluated on hyperspectral image unmixing, where it outperforms state-of-the-art MMF and deep learning methods in abundance estimation, and on low-dimensional representation learning, where it achieves competitive clustering performance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your time and effort in reviewing our work, and we are grateful for your generally positive feedback. We will take your advice to improve the paper. We also want to discuss some of the main points you raised.\n\n**Regarding “Essential References not Cited”**\n\nWe agree, and thank you for providing us with the references. We will add a paragraph in the Introduction to describe existing works related to dimension-varying diffusion.\n\nWe take this opportunity to give a reflection. The existing papers for dimension-varying diffusion are in the topic of generative models. They can be seen as concatenations of dimensionality-reduction processes and diffusion models. For the case of the references you provided, they concatenate multiple dimensionality-reduction processes and multiple diffusion models. On the other hand, we consider the topic of multilayer matrix factorization (MMF). We consider a single diffusion model and employ dimensionality reduction at each layer of the diffusion process, with the aim to develop a per-layer light-weight inference scheme. This makes the proposed method quite different from the dimension-varying diffusion methods we see in the prior generative modeling literature. However we agree that covering the prior literature would enhance the quality of this work, providing a broader coverage of related studies.\n\n**Regarding Comparison with Hierarchical VAEs**\n\nYou mentioned that we didn’t give a lot of insights and justification on how diffusion-model based MMF differs from hierarchical VAEs (HVAEs). That is a good question, and we will try to improve the writing on this part in the revision. Initially, when we wrote this paper, we planned to cover the HVAEs in the main development (Section 3.3). However, we ended up abandoning it due to space limitation. It is in fact possible to apply HVAEs to the MMF model, associating each layer of the factorization model with one deep neural network for variational inference. We are not aware of any work on HVAEs for MMF, although it is possible. As described in [[1]](https://arxiv.org/pdf/2208.11970), the stochastic approximation in HVAEs may have larger variance when the number of layers is larger. This, together with the need to use one deep neural network for each layer of the factorization model, may not fit the purpose of per-layer light-weight operations well. One advantage of the diffusion model is that it uses a simple variational (diffusion) model to greatly simplify the variational process and sidestep the aforementioned high-variance issue [[1]](https://arxiv.org/pdf/2208.11970)—and this is our motivation for using diffusion models.\n\nIt is also worthwhile to note the following: In principle, HVAEs can be directly applied to MMF. But the previous diffusion model cannot, because it assumes equal dimension with the latent variables. Our endeavor, simply speaking, is to make the MMF application possible. \n\n[1] Luo, Calvin. \"Understanding diffusion models: A unified perspective.\" arXiv preprint arXiv:2208.11970 (2022).\n\n**Regarding “Other Comments or Suggestions”**\n\nWe see your point and will try our best to streamline and make the writing more concise. The work consists of heavy details at some points, which can be better balanced, though we want to say that our writing style is to make the underlying assumptions and tricks clear to the reader; once again, we will try our best to balance. On “section 3.3.3 can be expanded,” you made a very good point and we agree. This, however, will be left as future work. We believe that from there we can develop more results. \n\n**Regarding “Questions to Authors”**\n\nQ1: We have run some simulations during this rebuttal period. Please see Table [1](https://github.com/AnonymousPaper-Submission/AnonymousPaper-Submission-ICML2025-rebuttal) in the anonymous GitHub repository. So far, our empirical finding is that, in general, the HVAE does not lead to performance improvement compared to the VAE. A possible reason is that the HVAE may require more careful training skills and more computations to achieve good performance.\n\nQ2: One future direction is to perform further analysis to expand or better understand the diffusion-based MMF interpretation in Section 3.3.3. We think that this part is quite unique to MMF in terms of providing interpretable explanations.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I checked the formulation of the model and objective and they seem to be sound (apart from typos, see below). The details of the derivations are not thoroughly checked.", "experimental_designs_or_analyses": "Yes. The experimental designs and analyses are sound.", "relation_to_broader_scientific_literature": "As far as I know, this paper is the first paper to leverage diffusion models for MMF. The proposed method achieves competitive or better performance than the previous state of the art methods on abundance estimation and representation learning.", "essential_references_not_discussed": "While the authors do not claim their paper is the first to propose dimension-reducing diffusion or diffusion for representation learning, it would be good to mention existing work on dimension-reducing diffusion such as Jing et al. 2022 and Zhang et al 2023.\n\nJing, Bowen, et al. \"Subspace diffusion generative models.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\nZhang, Han, et al. \"Dimensionality-varying diffusion process.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.", "other_strengths_and_weaknesses": "- The paper presents the experiments very clearly. The setup on the abundance estimation problem is clear and easy to follow.\n- The paper doesn't give a lot of insights and justifications for why the diffusion-based MMF is better or how it is different from hierarchical VAEs.", "other_comments_or_suggestions": "- In the introduction section \"in particular, for MMF, variational autoencoders (VAEs) appear to be the only viable solution\" needs more clarification.\n- The separation of the $\\gamma$ and $\\phi$ parameters are confusing. Since $\\gamma$ is contained in $\\phi$, maybe $q_\\phi(x_T\\mid x_{T-1})$ can be used instead of $q_\\gamma(x_T\\mid x_{T-1})$?\n- There seems to be typos in eqations (15) and (16). It should be $q_\\phi(x_{t-1}\\mid x_t, x_0)$ and $q_\\phi(x_{T-1}\\mid x_{T}, x_0)$ instead of $q_\\phi(x_{t-1}, x_t \\mid x_0)$ and $q_\\phi(x_{T-1}, x_{T}\\mid x_0)$.\n- The main paper contains too much derivation that does not directly contribute to the narrative of the paper. Many of the technical details can be moved to the appendix to make room for more discussions on the motivations and comparisons with existing methods. For example, section 3.3.3 can be expanded.", "questions_for_authors": "1. In practice, what is the difference between the diffusion VI approach and the hierarchical VAE approach? What's the reason for preferring the diffusion approach over the VAE one?\n2. What are some of the future directions of this work?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9ZWKD8DYq7", "reviewer": "Reviewer_cESg", "summary": "This work presents a diffusion variational inference algorithm for multilayer matrix factorization. The authors treat each layer as a diffusion step. Another difference is that the dimension of the latent variable reduces with the layer depth, termed as dimension reduction diffusion VI. This is the nature of latent presentation learning in matrix factorization, and is different with diffusion VI with equal latent variable length. The dimension reduction is achieved by imposing an orthogonal transform matrix. Based on this, the authors derive the VI objective. Experiments are conducted on two tasks, abundance estimation of hyperspectral images and representation learning on several image datasets, in which the performance is further evaluated by clustering.\n\n## Update after rebuttal\n\nSince the authors address most concerns, I will keep the score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are much obliged to you for your careful review and constructive comments. We will carefully take into account your comments in our revision, and we would like to discuss various aspects as follows.\n\n**Regarding “Other Strengths and Weaknesses”**\n\nPoint 1: Thank you for expressing your view in the beginning that “the use of diffusion VI is promising and interesting to me.” In Point 1, you mentioned that the improvement from the inference viewpoint may not be so significant. \n\nIndeed, once you understand the principle of the equal-dimension diffusion models very well, it is perhaps not difficult to anticipate that dimensional-reducing diffusion models (more accurately, embedding dimensionality reduction directly into the diffusion model) could be constructed—at least intuitively. Still, the latter was not tried before. There are technical details to overcome, and some of them are not trivial. We would say that we start with a simple idea, but there are non-trivial details to work out, both in the development and in experiments.\n\nPoint 2: As with some fundamental research, at this stage we focus on developing a concept (specifically, a new inference technique for MMF) and providing proof of concepts. As future work, the application of the concept to more complex datasets could be considered. Also, the hyperspectral unmixing problem we demonstrated in this paper is an important representative application in the context of hyperspectral remote sensing.\n\nPoint 3: In addition to MSE, researchers in hyperspectral remote sensing also consider the abundance angle distance (AAD), which uses the angle of two vectors as the measure of similarity. We have provided AAD in the anonymous GitHub repository；please see Table [2](https://github.com/AnonymousPaper-Submission/AnonymousPaper-Submission-ICML2025-rebuttal) there.\n\n\n**Regarding “Other Comments Or Suggestions”**\n\nWe agree and will try our best to better streamline the writing.\n\n\n**Regarding “Questions for Authors”**\n\nPoint 1: Yes, optimization with semi-orthogonal matrix constraints is more difficult in principle. But that does not stop researchers from trying. In machine learning, for example, \n\n[1] Moustapha Cisse et al. \"Parseval Networks: Improving Robustness to Adversarial Examples\", ICML 2017\\\n[2] Nitin Bansal et al. \"Can We Gain More from Orthogonality\nRegularizations in Training Deep CNNs?\", NeurIPS 2018,\n\nresearchers have found success with their numerical results. The regularization method we use is essentially the same as that in the above references. Our empirical experience with the MMF application is that the regularization method works reasonably.\n\nPoint 2: To answer your question, we provide some numerical results in Table [3 and 4](https://github.com/AnonymousPaper-Submission/AnonymousPaper-Submission-ICML2025-rebuttal). We observe that the $\\boldsymbol{U}_t$'s are quite close to semi-orthogonality.\n\nConcerning the question why semi-orthogonal $\\boldsymbol{U}_t$'s are used, one important reason is to simplify the variational process. It can be shown that if $\\boldsymbol{U}_t$'s are not semi-orthogonal, the variational operations would be more complicated. In fact, if we do not control the rank of  $\\boldsymbol{U}_t$ (which we do so via semi-orthogonality), there will be a lot of problems.\n\nPoint 3: In the context of MMF, our experience is that we do not need as many layer as in generative models, which can be thousands. We would say tens for MMF. The concept introduced in this work does not pose a constraint on the number of layers one can use, however, and as future work it would be interesting to see other applications that require more layers.", "claims_and_evidence": "The claims are supported by experiments.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "There is no theoretical claim. There are many derivations about the final objective. I did not carefully check them.", "experimental_designs_or_analyses": "The experimental design seems sound, but may be restricted to some simple datasets.", "relation_to_broader_scientific_literature": "The work mainly contributes to deep matrix factorization. This is a classical tool and can be related to many classical methods in signal processing.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "**Strengths**\n\nIt introduces new inference techniques for deep matrix factorization. The model can be extended in many ways, for example changing the priors. The use of diffusion VI is promising and interesting to me. Therefore, I am learning to accept at this stage.\n\n**Weakness**\n\n1. From the perspective of inference techniques, the improvement may be not so significant. The main difference seems to be introducing the transform matrix $U$, in order to adjust the dimensions.\n\n2. The experiments are conducted on simple datasets. I am not sure about the usefulness of the proposed models. Especially for the low dimensional representation learning, there are many simple yet powerful models. It is not sure whether the proposed model can be scaled to more complex and larger problems.\n\n3. In the abundance estimation experiment, is the MSE evaluated on reconstruction error? Are there other metrics to evaluate the abundance estimation accuracy?", "other_comments_or_suggestions": "The overall organization is clear and easy to read. However, it contains numerous unnecessary and colloquial words. It could benefit from revising for conciseness. And there could be more space to present more results.", "questions_for_authors": "1. To facilitate DRD-VI, the authors impose an orthogonal transform $U$ in the diffusion process. I am wondering if learning this $U$ along with VI parameters could make the optimization harder.\n\n2. I am wondering what is the reason for adding the orthogonal constraint for $U$. And in practice, the orthogonality is introduced by regularization, which means that $U$ is not strictly orthogonal. Would this affect the results?\n\n3. Does it require many layers for the diffusion VI?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper develops variational inference for multilayer matrix factorization using a dimension reducing diffusion model. The majority of the paper derives the variational lower bound using the diffusion approach of factorizing the posterior approximation in generative order rather than the markov reverse order.  The reviewers were all positive and the approach looks fresh.

Some other interesting references for this paper:

**Deep Exponential Families**: A multilayer matrix factorization with exponential families (https://arxiv.org/abs/1411.2581)
**Hierarchical Variational Models**: Improves variational bounds with auxiliary latent variables (https://arxiv.org/pdf/1511.02386)
Decision: Accept (poster)