ICML 2025

Multilayer Matrix Factorization via Dimension-Reducing
Diffusion Variational Inference
Junbin Liu 1 Farzan Farnia 2 Wing-Kin Ma 1
Abstract
Multilayer matrix factorization (MMF) has re-
cently emerged as a generalized model of, and
potentially a more expressive approach than, the
classic matrix factorization. This paper considers
MMF under a probabilistic formulation, and our
focus is on inference methods under variational
inference. The challenge in this context lies in
determining a variational process that leads to a
computationally efficient and accurate approxima-
tion of the maximum likelihood inference. One
well-known example is the variational autoen-
coder (VAE), which uses neural networks for the
variational process. In this work, we take insight
from variational diffusion models in the context
of generative models to develop variational infer-
ence for MMF. We propose a dimension-reducing
diffusion process that results in a new way to
interact with the layered structures of the MMF
model. Experimental results demonstrate that the
proposed diffusion variational inference method
leads to improved performance scores compared
to several existing methods, including the VAE.
1. Introduction
Over decades, matrix factorization (MF) methods have
played a crucial role in a wide variety of problems such
as dimensionality reduction, low-dimension representation
learning, blind source separation (Hyv¨arinen et al., 2023),
hyperspectral unmixing (Ma et al., 2013), topic model-
ing (Arora et al., 2012), community detection (Yang &
Leskovec, 2013), to name a few. The broad interest of
1Department of Electronic Engineering, the Chinese Univer-
sity of Hong Kong, Hong Kong SAR of China 2Department
of Computer Science and Engineering, the Chinese Univer-
sity of Hong Kong, Hong Kong SAR of China.
Cor-
respondence to:
Junbin Liu <liujunbin@link.cuhk.edu.hk>,
Farzan
Farnia
<farnia@cse.cuhk.edu.hk>,
Wing-Kin
Ma
<wkma@ee.cuhk.edu.hk>.
Proceedings of the 42 nd International Confe...