Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions
Authors: Sagar Shrestha
Xiao Fu
Venue: ICLR 2025 Poster (2025)
Keywords: unsupervised learning
identifiability
unknown latent dimension
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Understanding identifiability of latent content and style variables from unaligned multi-domain data is essential for tasks such as
domain translation and data generation. Existing works on content-style identification were often developed under somewhat stringent conditions, e.g., that all latent components are mutually independent and that the dimensions of the content and style variables are known. We introduce a new analytical framework via cross-domain *latent distribution matching* (LDM), which establishes content-style identifiability under substantially more relaxed conditions. Specifically, we show that restrictive assumptions such as component-wise independence of the latent variables can be removed. Most notably, we prove that prior knowledge of the content and style dimensions is not necessary for ensuring identifiability, if sparsity constraints are properly imposed onto the learned latent representations. Bypassing the knowledge of the exact latent dimension has been a longstanding aspiration in unsupervised representation learning---our analysis is the first to underpin its theoretical and practical viability. On the implementation side, we recast the LDM formulation into a regularized multi-domain GAN loss with coupled latent variables. We show that the reformulation is equivalent to LDM under mild conditions---yet requiring considerably less computational resource. Experiments corroborate with our theoretical claims.

Published as a conference paper at ICLR 2025
CONTENT-STYLE LEARNING FROM UNALIGNED Do- MAINS: IDENTIFIABILITY UNDER UNKNOWN LATENT DIMENSIONS
Sagar Shrestha and Xiao Fu School of Electrical Engineering and Computer Science Oregon State University Corvallis, OR 97331, USA
(shressag, x xiao. fu) @oregonstate. edu
ABSTRACT
Understanding identifiability of latent content and style variables from unaligned multi-domain data is essential for tasks such as domain translation and data gener- ation. Existing works on content-style identification were often developed under somewhat stringent conditions, e.g., that all latent components are mutually in- dependent and that the dimensions of the content and style variables are known. We introduce new analytical framework via cross-domain latent distribution matching (LDM), which establishes content-style identifiability under substantially relaxed conditions. Specifically, we show that restrictive assumptions such as component wise independence of the latent variables can be removed. Most notably, we prove that prior knowledge of the content and style dimensions is not necessary for ensuring identifiability, if sparsity constraints are properly imposed onto the learned latent representations. Bypassing the knowledge of the exact latent dimension has been longstanding aspiration in unsupervised representation learning-ove analysis is the first to underpin its theoretical and practical viability. On the implementation side, we recast the LDM formulation into regularized multi-domain GAN loss with coupled latent variables. We show that the reformula- tion is equivalent to LDM under mild conditions- yet requiring considerably less computational resource. Experiments corroborate with our theoretical claims.
INTRODUCTION
In multi-domain learning, "domains" are typically characterized by distinet "style" that sets their data apart from others (Choi et al. (020). Take handwritten digits example: writing styles of different persons can define different domains. Shared information across all domains. such as the identities of the digits in this case, is termed "content". Learning content and style representations from multi-domain data facilitates many important applications, e.g., domain translation (Huang et al., 2018). image synthesis (Choi et al. 2020), and self-supervised representation learning (Von Kügelgen
ct al. 2021 Lyu et al. 2022 Daunhawer et al. (223);; see more in Huang et al. (0118) Lee et al. (2020):
Choi et al. (2020); Wang al. (2016): Yang et al. (2020) Wu et al. (2019)
Recent advances showed that understanding the identfiability of the latent content and style compo- nents from multi-domain data allows design more reliable, predicable, and trustworthy learning
systems (Hyvarinen et al. 2019, et al. |022]p20 Xie et al. (2023) Kong et al.) 2022) Shrestha & Fu number of works studied content/style identifiability when the multi-domann data have sample-tossappl cross-domain alignment according to shared contents. Specifically, identifiability was established for sample-alignes multi-domain settings under the assumption that multi-domain data are linear and nonlinear mixtures of latent content and style components, in the context of canonical correlation analysis (CCA), multiview analysis and self-supervised learning (SSL); see Ibrahim et al. (2021): 1y: [Sorensen et al. (2021) Wang
2024 Gresele et al., 2020); Gulrajani & Hashimoto| 2022))
Isola (2020): Von Kügelgen et al. (202T) :: Lyu et al. (2022); Karakasis & Sidiropoulos (023):
Daunhawer al. (2023)

---

Published as a conference paper at ICLR 2025
When cross-domain samples are unaligned, it becomes significantly more challenging to establish identifiability of the content and style components. The recent works in Xie et al. (2023); Sturma et al.)
(2023); ; Kong et al. (2022); Timilsina et al. (2024) made meaningful progresses towards this goal. These works considered mixture models of content and style for each domain, similar to those in Lyu et al.] (2022); Von Kügelgen et al. (202T): Ibrahim et al. (202T); [Sorensen et al. (202T): Karakasis
& Sidiropoulos (2023); Daunhawer ct al. (2023), but without cross-domain alignment. The new results in (Xie et al. [023] Sturma et al. 2023] Kong et al., 2022 Timilsina et al. 2024) provide
theory-backed solutions to suite of timely and important applications, e.g., cross- Tross-language retrieval, multimodal single cell data alignment, causal representation learning, and image data translation and
generation.
Challenges. The content-style identifiability results in existing unaligned multi-domain learning works are intriguing and insightful, but some challenges remain. First, the conditions used in their proofs have a number of restrictions, which limits the proof"s applicability
many cases. For example, Sturma et al. (2023); Timilsina et al. (2024) assume that the all data reside in a linear subspace, which is over-simplitication of reality; Xie et al. (2023); Kong et al. (2022) assume that the content and style variables are component wise independent and that large number of domains exist-both both can be hard to fulfil. Second, the existing identifiability analyses in unaligned multi-
domain learning (Xie et al.] 2023 Kong et al. 2022 Sturma al., 2023 Timilsina et al. 2024) (as well as those in aligned multi-domain learning) all need to know the dimensions of the content and style variables, which are not available in practice. Selecting these dimensions often involves extensive trial and error.
Contributions. In this work, we advance the analytical and computational aspects of content-style learning from unaligned multi-domain data. Our detailed contributions are as follows:
(i) Enhanced Identifiability of Content and Style: We propose a content-style identification criterion via constrained latent distribution matching (LDM). We show that the identifiability conditions under LDM are much more relaxed relative those in existing works. Specifically, our results hold for nonlinear mixture models, as opposed to the linear ones used in Stturm et al. (2023); Timilsina jt al. (2024). Unlike [Xie et al. (2023): Kong et al.] (2022): [Sturma et al. (2023). no elementwise mutual independence assumption is needed in our proof. More importantly, our result holds for as few as two domains (whereas Xie et al. (2023); Kong et al. (2022) needs the existence of a large number of domains). The new results widens the applicability of content-style identifiable models in a substantial way.
(ii) Content-Style Identifiability under Unknown Latent Dimensions: We consider the scenario where the latent content and style dimensions are unknown-whied is the case in practical settings. Note that existing works determine the content and style dimensions often by heuristics, e.g., trial-and-error. However, wrongly selected latent dimensions can largely degrade the performance of some tasks; e.g., an over-estimated style dimension hinders the diversity of data in generation tasks (see Sec. (6). We show that, by imposing proper sparsity constraints onto the LDM formulation, the content-style identifiability is retained even without knowing the exact latent dimensions. To our knowledge, this result is the first of the kind in the context of nonlinear mixture identification.
(iil) Efficient Implementation: We prove that the LDM formulation is equivalent to a sparsity- constrained, latent variable-couple muti-domain GAN loss, under reasonable conditions. Directly realizing the LDM formulation would impose multiple complex modules, including the DM and content-style separation modules, in the learned latent domain. Simultaneously learning the latent space and optimizing these modules can be computationally involved. The GAN-based formulation circumvents such complicated operations and thus substantially simplifies the implementation.
For theory validation, we perform experiments over a series of image translation and generation tasks.
Notation. Please see Appendix A.I for detailed notation designation. A particular remark is that P. and (-) represent the probability measure of x and the probability density function (PDF) of æ, respectively. The "push forward" notation [f] means the distribution of fæ))
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
Content-Style Modeling in Multi-Domain Analysis. Consider the case where the data are acquired over N domains X((m) C R4, where n I,..., .N. We assume that any sample from domain n can be represented as a function (or, a nonlinear mixture) of content and style components, i.e.,
C~ (( P p(nn), (x)) = g((c,(())
(I)
where P axt) and P.. are distributions of the style components in rith domain and the content compo- nents, respectively. Let C. C Rdc and Six) C RRds be the open set supports of IP and IP Pa() Then, we define x²(m) {g(c, Jc, g(m) s(, E C x )tm)} S Rd as the support of x!"))
IP ( Let
X' = = 1 C Rd and S , S(n) C Rds represent the whole data space and the whole style space, respectively. We assume that the nonlinear function C x - X is a differentiable bijective function. This is common assumption in latent component identification works, e.g., Von Kügelgen et al.] (202T): Hyvarinen et al.] (2019): Khemakhem et al. (2020), which basically says that every data sample has associated unique representation in latent domain. A remark is that although X' S ,t and d might be greater than ds + dc. the bijective property can hold as X' resides within a low dimensional manifold (Von Kügelgen al. 202T).
The model in T is widely adopted (explicitly or implic- itly) in multi-domain analysis; see examples from Huang
3
et al. ( Lee et al. (2020); Choi et al. (2020): Wang
eet al. 20166; Yang et al. (2020) : Wu et al. (2019). This
content
2
3
model makes sense when the "domains" are participated
style
using distinguishable semantic meaning; e.g., in Fig.
Learned g
3
"style" includes the writing manners (handwritten/priteed) and display background colors (black/gray). Under the model in I learning g (and its inverse f) as well as
(e)
the latent components e and ((n) is the key to facilitate a number of important applications.
Figure 1: Cross-domain translation from source domain 8 to target domain t.
Application: Cross-Domain Translation. Learning content and style components from sample in the source domain (c,, 8 ( f (x((") and sample from the target domain (c,, sf() f )x)
can assist translate x its corresponding representation in the target domain. This can be realized by generating a new sample (e-+)) g(c,, 88. a/");; Huang et al. 2018): Wang et al. (2016).
); see Fig. 1 for illustration and Lyu et al. 2022);
Application: Data Generation. If c and ((x)) can be learned from the samples, then one can also learn the distributions Pe and Pg(.)) using off- the-s distribution learning tools, e.g., GAN (Goodfellow
al. 2014). This way, one can draw samples from the distributions, i.e., Cnew Pe:
Snew
(e)
a(()
and generate new samples xnew = [CCnew, snew) with intended styles.
Other Applications. We should mention that the content-style modeling is also critical perspective for understanding representation learning paradigms, e.g., the SSL frameworks (Von Küigelgen et al.)
202 Lyu al. 2022 Daunhawer et al. 2023 Wang Isola 2020).
Content-Style Identifiability. In recent years, the identifiability of f. and s(") started drawing attention, due to its usefulness in building more reliable/predictable systems.
Aligned Domains: Results from Superviud Learning (SSL). The works (Von Kügelgen et al.
2021 Daunhawer et al.) 2023: Lyu et al., 2022 Karakasis & Sidiropoulos) 2023) studied content identinability in the context of representation learning, in particular, SSL and multiview learning. It was shown that when N = 2, if content-shared pairs (x=(1), xx(2)) are available, then enforcing (() fx) V content-shared pairs (x(1), x2)) can provably learn c, under reasonable conditions. The learning criterion can be realized by various loss functions, e.g., Euclidean fitting- based (Lyu al. 2022 Karakasis & Sidiropoulos (026) and contrastive loss-based (Von Kügelgen et al. 2021: Daunhawer et al.] (203)) criteria. The identifiabiiit of the style components was also considered under similar aligned domain settings; (Lyu et al. 2022 Eastwood et al. 2023).
Unaligned Domains: Progresses and Challenges. Aligned samples are readily available in applica-
tions such as data-augmented SSL ( (Von Kügelgen al. 2021 Daunhawer al. 2023 Lyu et al.
3

---

Published as a conference paper at ICLR 2025
However, in other applications such image style translation and image generation, aligned samples are hard to acquire (Zhu et al. (017). For unaligned multi-domain data, the identfiability issue of content and style has also been recently addressed. For example, the work of Sturma et al. (2023) extended the linear ICA model to unaligned multi-domain settings, in the context of causal learning. The work of ofmimmllina et al. (2024) took similar línear mixture model but showed content- style identifiability under more relaxed conditions. The work of Xie et al. (2023); Kong et al. (2022) proved content-style identifiability under more realistic nonlinear mixture model similar to that in I However, the main result there relies on number of somewhar hat stringent conditions. That is, two notable assumptions in Xie al. (2023); Kong et al. (2022) boil down (i) that all components in (c, s ( elementwise statistically independent given the domain index nı; and (ii) that there exist at least 2ds + domains. These conditions can be hard to fulfil. See more detailed discussions
2022).
on existing results in Appendix B
The Dimension Knowledge Challenge. Notably, all the existing works in this domain (under both
aligned and unaligned settings) assume that the dimensions of and g(x) are known. However, in mixture model learning, such knowledge is hard acquire (especially in the nonlinear mixture case).
As we will show, using wrongly selected de and ds can be rather detrimental to content-style learning an over-estimated style dimension could lead serious lack of diversity in generated new samples. Consequently, the dimensions are often selected by extensive trial and error in practice.
3 MAIN RESULT
In this work, we revisit content-style learning from a latent distribution matching (LDM) viewpoint. Recall that e and g(x) represent the content and the style of the rath domain, respectively. We assume: Assumption 3.1 (Block Independence). The block variables e E Rdc and (s(m) Rd In=1 are statistically independent, i.c., p(c, s(1), . (N)) = Pe(c) IT Pa(o) (ato) (s(m)).
The assumption used in various
models (Lyu et al. 2022 Eastwood al. 2023
Wang et al., 2016 Choi et al.] 2020; Timilsina et al., (024). It makes sense when the styles can be combined with contents in "arbitrary" way without affecting the contents (e.g., the writing style of digits can change freely without affecting the identity of the digits). Next, we will use this assumption build our learning criterion. We propose the following learning criterion:
find f : X' + RRo+ds injective
s.t. ffcl#p_00 = [f 20) i, [N], (distribution matching)
(2a)
_ [ >(0)>> Vn E [N],
(block-indep. enforcing)
(2b)
where (x)) E Rele represents the first de outputs of f that are designated to represent the content components, f (x)) E Id represents the learned style from domain n, Eq. (2a) matches the distri-
butions of f (x(((() and f fc(a()))iie the learned contents from domains and j. respectively and Eq. (26) imposes a block independence constraint on the learned content f fo (x(()) and style f (x) from each domain following Assumption (3.1)
3.1 WARM UP: ENHANCED IDENTIFIABILITY WITH KNOWN LATENT DIMENSIONS
We first show that the content-style identifiability under I and known de and ds can be substantially enhanced relative existing works. We will remove the need for the dimension knowledge the next subsection. To establish identifiability via solving Problem (2). we make the following assumption: Assumption 3.2 (Domain Variability). Let C :=
be any measurable set that satisfies (i) P P2(.) [A] > 0 for any n E [N] and (ii) A cannot be expressed as B x S for any set B C C. Then, there exists pair of i.A. E [N] such that the following holds:
P [A] A 1 [A],
Note that for any A. we only need one pair of (LA. A) to satisfy the condition, and the pair can change over different A's. Essentially, Eq. (3) requires that the styles have sufficiently diverse distributions. This assumption is a standard characterization for the distributional diversity of the domains in the literature; see Xie al. (2023): Kong al. (2022) and its variant Timilsina et al. (202)).
4

---

Published as a conference paper at ICLR 2025
Under Assumptions 3.1 and (32), denote solution Problem 2 Then, we have:
Theorem 3.3 (Identifiability under Known Latent Dimensions). Under Eq. Q). suppose that As- sumptions 3. and 3.2 hold, and that the f is differentiable. Then, have f((a)))) = 7(c) and ). Vn E [N], where y : IRde and 6 - Rds are injective functions.
(()
8 8(8(m) (m))
The proof of Theorem 3.3 is in Appendix C Theorem 3.3 purports that the solution of Problem identifies the model
including the content/style components and the inverse mapping of the generative function g (up to Y and 6). Theorem 3.3 uses conditions that are sigmificantly more relaxed relative to those in existing works Xie et al. (2023); Sturma et al. (2023): Kong ct al. (2022); Timilsinal et al. (2024). First, instead of assuming the elements of z x(n) (c, g!")) are statistically independent
as in Xie al. 2023); Sturma al. (2023); ; Kong et al. (2022). our proof based on the assumption that the content and styles are block independent (cf. Assumption 3.1). This block boock-indpendencce
assumption, which the key for style identifiability, is similar those in [Ly al.] (2022) and Timilsina al. (2024) -but the former assumes aligned domains and the latter can only work under linear mixture models (see Theorem B.2 Appendix [.2)). Second, Theorem 3.3 does not need the existence of N 2dg + domains as in Xie et al. (2023) ; KKnng et al. (2022) (see Theorem B.3 in Appendix B.3) our result can hold over as few as N = 2 domains. As a result, our Theorem 3.3
our
applies to considerably wider range of cases relative to those in existing works.
3.2 IDENTIFIABILTY WITHOUT DIMENSION KNOWLEDGE
Theorem 3.3 still uses the knowledge of de and ds. In this subsection, we propose modifed learning criterion that does not the exact dimension information. To proceed, let de and dg denote the user-specified latent dimensions for f. i..., f X - Rdo+tss fc X' + Rde and fs : + Rds, Note that these dimensions need not to be exact. We consider the following learning criterion:
minimize f: injective
Σ LE is
[l.]
(4a)
subject to [fcl alt) [Scl#p_ . Vi,j [N], [fsh# (())
(4b)
a(n) Vn E [N],
(4c)
Problem 4 minimizes the "effective dimension" of the extracted style component, while satisfying the distribution matching and independence constraints. The idea is to excessive de and ds that one has enough dimensions to represent the content and style information. Note that trivial solutions could occur when using over-estimated de and ds. For instance, when fC is constant function, fs can still be an injective function of (m) given large enough ds. This pathological solution satisfies both constraints (4b) and (4c). We use the sparsity objective in (4a) to "squeeze out" the redundant dimensions in f This prevents the content information from "leaking" into the learned fs- We
formalize this intuition in the following theorem:
Theorem 3.4 (Identifiability without Dimension Knowledge). Assume that the conditions in Theo- rem 3.3 3.3 hold. Let f represent a solution of Problem 4 and f is differentiable. Assume the following conditions hold: (a) de 2 de and ds 2 ds. (b) 0 < Patw) (z) < co, Vz E Z = C x S, Sn E [N]. Then, there exists injective functions 7 -> Rdo and 8 - Rds Vn [N] such that = (=)
and (z(((()) = ((s(m)), Vn E [N].
The proof of Theorem i.4 in Appendix Theorem 3.4 means that using Problem 4 there is no need know dg or de advance. Also, note that no extra assumptions on and ((n) are needed on top of those in Theorem 3.3 Hence, the identifiability result has significant practical implications for content-style identification, where the latent dimension in practice always hard to acquire
4 IMPLEMENTATION SPARSITY-RGGULARZIA MULTI-DOMAIN GAN
At first glance, a conceptually straightforward realization of the learning criterion in Problem could take the following form:
5

---

Published as a conference paper at ICLR 2025
N
N
N
minimize f: injective
LDM(f So(ao)
sc((o))) + + E Lindep (f c( Js (s(
(5)
where the first term and the second term promotes the distribution matching (DM) constraint (2a)
and the independence constraint (2b). respectively. Similarly, Problem 4 can be implemented in straightforward manner by adding a sparsity regularization term to Problem (5).
Remark 4.1. Problem (5) is potentially viable but can be costly: Both the LDM modules and the block
independence regularization
the learned components often needs rather nontrivial optimization
(see (Lyu et al. 2022)). Enforcing f to be injective also needs extra regularization, e.g., autoencoder type regularization (Lyu et al.
2022: Zhu et al. 2017) and entropy-type regularization (Von Kügelgen
ct al. 2021 Daunhawer et al. 2023
In light of Remark 4.1 instead of using Problem (5) we reformulate Problems 2 and 44
as follows:
log (n)))
(z)(1)) + log (1 (1-d') dd()) (c((rr))) (r's (r("))
(r))))
min max 9,50.es
(6a)
din)
subject to eg (r(")) has minimal ||e55) (r(") (r(")))o. Vrrg
(6b)
The above approximates Problems 2 and 4 when the constraint (6b) is absent and active, respec- tively. In practice, the sparsity constraint can be approximated using sparsity regularization terms (e.g., f norm) easily. Denote de and ds are the estimates of dc and ds, respectively. The idea is to learn invertible nonlinear mappings ec and e(") that transform independent Gaussian variables (i.e., re and (n) represent content e and style respectivly. Generate re ~ No Iag) and construct an invertible eg such that ec(rc) Rdãc Similarly, construct invertible es (n) such that (m) (rs E Rdl with (n)) ~ ~N lag) Then, the content and style are mixed by to match the (m))
distribution of (n)) using logistic loss (i.e.. GAN-type DM). In other words, the formulation looks g(m): g()) = q(ec(rc), eg eg
(T("))) Vn E [N]. This way, instead of directly learning f. we learn the generative process g using q. Our next theorem shows that q is indeed the inverse of f (up to some ambiguities).
for ec. eg" and q such that p_ = P(("))
c(n) To proceed, denote Č and {(n) as the sets representing the range of ēc and ég respectively. Then, the effective domain of q is C S where S U,)) We show that:
Theorem 4.2. Let (q. eo. e((") a) be any differentiable optimal solution of Problem (6) Let C and be simply connected open sets. Let 0 < Pz(m) (z) < oo, Vz 2 = S. Under the assumptions
in Theorem 3.3 we have the following:
(a) If dc de and dg = dg and (bb) absent, then ê S + X' bijective and f q is also solution of Problem (2)
(b) If de > de, dg > ds and q Ō S X' is bijective, f = q- is also a solution of Problem
Problem (6) has a number of practical advantages over the direct implementation in Problem ( Particularly, it avoids complex operations the latent domain. In LDM, performing DM on fc((
and ()) poses quite a nontrivial optimization process. This is because both of the inputs to the DM modules (i.e., ()) for all E [N) change from iteration to iteration- yet the DM module (e.g., GAN and Wasserstein distance-based DM (Goodfellow al. 2014; Arjovsky et al. 2017)) itself often involves complex optimization with its own parameters updated on the fly. The new formulation performs GAN-base DM in the data domain and keeps one input (the real data) to every GAN module fixed. This reduces lot of agony in optimization parameter tuning, Problem (6) also does not need any explicit constrain/reeuularizttion
enforce the block independence of f c and
(due requires
Problem 4
injective. Here, although the learned by Problem d seems to bijective
instead of only injective, the bijectivity is w.r.t. the domains X' C S. The function indeed only injective when considered w.r.t X tis; see Sec. A.2 "Injection, bijection, and surjection".

---

Published as a conference paper at ICLR 2025
Table 1: Evaluation of the data generation task. Standard deviation reported using for style diversity
Method
FID
Style Diversity (f)
Training time, (J)
AFHQ
Ce
CelehA-7
AFHQ
CalebA-HQ
CelebA-77
APHQ
CelehA-HQ
CelehA-7
TrenaitionsI-eCal
38.00
8.12
70.45
29.65
32.56
12.53
SLyIeGAN-ADA
8.17
5.89
72.10
28.46
32.26
11.55
Proposed 6.28
I-GA2
5.91
5,18
0.16 = 0.02
0.07 1 0.03
0.07 10.03
29.53
28.76
12.51
5.70
5.27
0.03
0.36 0.04
0.26
27.36
12:28
fs (which could be resource consuming (Lyu et al. constructed to be block independent.
2022 Gretton et al. 2007 as EC and es are
(n)
Another quite interesting observation is that, the proof of Theorem 4.2 (a) shows that the bijectivity constraint q is automatically fulfilled when additional condition (i.e., that C and S are simply connected) is met. This means that the LDM formulation would need extra modules, c.g. E||r ((x) - 2 [, to impose injectivity constraints, even when ds and de known. When de and ds unknown, solving Problem (6) per se does not ensure q to be bijective. Nonetheless, we observed that not explicitly enforcing bijectivity in implementations does not affect the performance in practice. Similar phenomenon was observed in nICA implementations; see, e.g., (Hyvarinen & Morioka 2017 Hyvarinen et al. 2019).
RELATED WORKS
Nonlinear ICA. Learning content and style components from a nonlinear mixture model is reminis- cent of nonlinear independent analysis (nICA) (Hyvärinen & Pajunen| 1999 Hyvarinen & Morioka 2017; Hyvarinen et al.] 2019). Most nICA works were developed under single domain settings, with some recent generalizations multiple views/domains (Gresele et al., 2020 [Hvarinen et al. 2019). Nonetheless, nICA requires that all the latent variables are (conditionally independent. This considered a somewhat restrictive assumption in content-style learning.
Content-Style Models in Aligned Multi-Domail Learning. Aligned multi-domain content-style learning is key technique in data-augmented SSL and representation learning. There, it was shown that elementwise (conditional) independence is not needed, if the goal is to isolate content from style (Von Kügelgen et al.) |02T] Lyu al.) 2022] Karakasis & Sidiropoulos 2023). It was further shown that block independence (similar to Assumption 3.1) is the key to identity the style (Lyu et al. 2022 Daunhawer et al. 2023). However, all these works require cross-domain data alignment.
Content-Style Identification in Unaligned Multi-Domain Learning. Identifiability of unaligned multi-domain learning was studied in the context of various applications, e.g., image translation
(Shrestha & Fu 2024), data synthesis (Xie et al.) (202)), cross-domain information retrieval Timilsina
ct al. (2024). and domain adaptation (Kong et al. [0022] Gulrajani & Hashimoto 2022 Timilsina
al. 2024). In applications, content-style disentanglement has been applied in various tasks, such as
(Hong et al. 2024| Huang et al. [2022] Dai et al. 2023). However, only a handful of works (Kong
let al. 2022 Xie et al. 20033 "0023|Timm Timilsina et al. 2024) have investigated the identifiability aspects. The work (Kong et al. 2022) postulated a similar content-style model as in (Xie et al.] (2023) and came up with identifiability conditions similar those (Xie al. 2023). The mostly related work to ours is (Xie et al. 2023), as both works are interested in content-style identification under (I). Our implementation Problem (6) partially recovers the marginal distribution matching criterion in (Xie) et al. 2023), despite the fact that our learning criteria started with an LDM perspective. Nonetheless, our method enjoys much less restrictive model assumptions for content-style identifiability. Our multi-domain GAN also admits more relaxed neural architecture (see Appendix G
Content-Style Learning without Knowing Latent Dimensions. The SSL work (Von Kügelgen et al. 2021) presented proof that essentially established that the content can be learned without knowing the exact dimension de. However, their result was under the assumption that the domains are aligned. In addition, the proof could not hold when style learning is also involved. Our proof solved these challenges. The work in (Xie et al. 2023) used a mask- requirement of knowing de and ds. The mask- based formulation has the flavor of sparsity promoting as in our proposed method. However, they still need to know de dg. which is unlikely available in practice. In addition, the mask-based method in (Xie et al. 2023 did not have theoretical supports.
formulation to remove the
7

---

Published as a conference paper at ICLR 2025
Style
Style
Ststc
Contert
Coastook
(a) 1-GAN (Xie ct al. 2023 (b) Proposed (w/o sparsity)
(c) Proposed
Figure 2: Samples generated by learning content (pose of cat) and style (type of cat) from AFHQ.
NUMERICAL EXAMPLES
Multi-Domain Data Generation. For the data generation task, validate our theoretical claims
using three real world datasets: animal faces (AFHQ) (Choi et al.) 2020), CelebA-HQ (Karras et al. 2018 and CelebA (Liu et al. 2015) with 3, 2, and domains, respectively (see Appendix G.6
The baselines here are I GAN (Xie et al. 2023), StyleGAN-ADI ADA Karras et al. 2020 and Transitional- (Shahbazi et al. 2021).
Following Xie et al. (2023). we use StyleGAN2-ADA (Karras et al. 2020) to represent our
generative function in (6a We set de
384 and ds = 128 in all the experiments. We
use an ( regularization term Alles Note that other regularization (such as the lp function with p 1) can also be easily used under our framework, which shows similar effecttivnness (see Appendix H.4. We find that the algorithm is not very sensitive to the choice À as any positive À encourages sparsity of (m) (r(")). We use À 0.3 for all the experiments. More detailed exper- imental settings in Appendix
(n) (TS (")" to approximate the sparsity constraint in (6b).
different style samples
different style samples
(6)
== 2
different domains
different domains
es
==
Fig. 2 shows the qualitative results for content-style identification using various methods for the cat domain (n 1) of AFHQ. For each row, fix the content part с = (c(rc) (i.e., pose of the cat) and randomly sam- ple different styles (11)
2
2
(a) I- I-GAN
Xie et al. 2023
(b) Proposed
eg (1) (rg))
(1) (V. (2.) to generate the images (1) q(c, 8 (1)), This
where Ts
Figure 3: Samples generated by combining the same content
with ((n) for various n's in AFHQ and CelebA-HQ.
way, the samples s for i 1,2, correspond to various types of cats. Fig 2 shows that the GAN appears to generate the same type of cat even when repeatedly sampled from their learned distribution of s This suggests that the style components are not extracted properly. Fig. (b) shows the result of using the proposed method without any sparsity regularization. As explained earlier, this can lead to learning constant content part with all information captured by the style part. Fig. (bb) corroborates with the intuition since we little to pose variation in the sampled contents (i.c., the three rows). Fig. (c) shows the result
of proposed method, i.c., Problem (6). One can see that both content and style parts demonstrate
sufficient diversity, indicating well learned content and style distributions. Appendix H shows similar results for other domains and datasets.
Fig. shows the generated samples of (n) for different n''s n's using models learned from the AFHQ and CelebA-HQ datasets, which correspond to different species of animals (cat, dog, and tiger) and different genders of people, respectively. The top three rows in each figure correspond to the three
8

---

Published as a conference paper at ICLR 2025
n's (i.e., three domains) of the AFHQ dataset, whereas the bottom two correspond to two n 'ss of the CelebA-HQ dataset.
For the jth row associated with each dataset, sample three different styles 8 (m)),, i E {1, 21,2,3} and combine it with fixed content e to generate the image x; (nz) q(c, s (m/)) in the jth row and ith column.
Tranlation Thansiatinn 2
Seu ca
Tramiidden Traraaaion
Both the baseline I-GAN and our method can combine fixed è with (")) for different i to create content consisecn new data (see all the rows). However, one can see that the baseline I-GAN was not able to sample differ- styles in each domain. It seems that every domain n always repeatedly samples the same style components g(") as the same images al- ways appear in the same row. The proposed method can generate quite diverse style samples in all the domains. Additional results are in the Appendix H
(a) Proposed
(b) I I-GAN (Gen:
Figure 4: Translation by combining content (pose)
randomly sampled styles from the dog domain.
mags
peppeeed
CAN (7)
ARCOC ERTTNNEE Aiting 12 tei tad
Table shows the FID (Heusel al., 2017).
style diversity scores, and training time of the different methods. We use LPIPS distance (Zhang et al. (2018) between pairs of images with the same è and different style samples from g(x) to measure the style diversity. The diversity scores are averaged over 6,000 im- ages across all domains, where every 10 images contain the same content with different styles. Note that the baselines Styl 1 eGAN-ADA ADA and T Transf t ona laral-cca c-GGAN do not learn content- style models, and thus the style diversity scores of theirs are not reported. One can see that the FID scores of the methods are similar, mean- ing that all methods generate realistic looking images. However, the style diversity of the pro- posed method is 3 5 times higher than the baseline over all datasets. The conditional gen- erative models (Transitional-on -CAN and StyleGAN -AA) sometimes encountered convergence issues on specific datasets as reflected by their FID scores. Finally, the training time of all the methods are in the similar range, the proposed method being slightly faster for AFHQ and CelebA-HQ datasets.
Figure 5: Guided translation by combining content (first column) with style (second column) of the images.
Translation.
Existing methods use a dedicated system for multi- translation (Choi et al. 2018:: 2020: Yang al. 2023). However, since a multi-domaii generative model can already disentangle content and style (cf. Theorems 4.2 of this work), one can simply use the generative model for domain translation.
Given an image (() in the source domain i, in order to extract the corresponding content e or style g( one can simply solve (2, 8) = arg mine, div(g(c, s), ()),, where div is some distance metric/divergence measure. There exists many approaches to solving the problem, often referred to as GAN inversion (Xia et al.) (022). In our case, we simply use the Adam optimizer for this inversion step. For div, details are in Appendix can be combined with a randomly sampled style (1)) eg (r's ) ~ ~O, T from the target domain t. Additionally, one can also extract style from an image in target domain and combine it
pre-trained VGG16 (Simonyan & Zisserman 2014) neural network. More G To generate the desired translation, the GAN inversion-extracte content
(t) (t) (t)
with extracted content from the source domain for guided translation.

---

Published as a conference paper at ICLR 2025
Table Quantitative evaluation of methods for the translated images.
(L)
Style Diversity (1)
Training time ( hhurs))
Method
AFIQ
CelebA-IIQ
AFIQ
CelebA- IIQ
Generation
Trunslation
Total
StazGARv2
16.83
13.67
0.45 + 0.03
0.45 + 0.03
50.83
50.83
BaCoLhGAN
53.68
29.69
0.14 0.04
0.09 =
30.90
I-GAN (Tt)
19.57
13.26
0.46 + 0.03
0.29 + 0.05
29.53
07.46
96.99
Proposed
16.61
0.53 0.03
27.36
27.36
The baselines used are the method in (Xie al.
2023). StarGANv2 (Choi et al., 2020), and
SmoothGAN (Liu et al.] [02T]. Note that (Xie et al. 2023) proposed a separate system for the domain translation that uses its pre-trained multi-domain generative model to train a separate translation model (see Appendix [P. However, since the aforementioned GAN inversion procedure is also applicable to their generative model as it extracts content and style, we use two versions of their system, namely, GAN (Gen) for the method based on GAN inversion and I I-GAN (Tr) for the separate translation system proposed in (Xie et al. 2023).
Fig. (a) and (b) show the result of translation from wild domain (n 3) to dog domain (n 2) using randomly sampled style components. The content ( , [3] extracted for samples in the wild domain is combined with randomly sampled styles 8) j E [2] in the dog domain to synthesize the translated images. Our translations in each row contain the same content (i.e., pose of wild) as the input source image, but different styles (i.e., dog species). However, I-GAN (Gen) seems to produce unrealistic samples in some cases (first row). Their style diversity also appears to be limited.
(2)
Fig. 5 shows results of quided-translatte for all methods for all pairs of domains in the AFHQ domain. Content extracted from the images in the first column is combined with the style from the second column. One see that the proposed method preserves the style information better than the baselines.
Further experiments on multi-domain translation are presented in Appendix H.2
Table 2 shows that the image quality (see FID) and diversity (see style diversity) of the translated images are competitive or better than the baselines (see qualitative results in Fig. p and [0 of Appendix H). One can also see that the training time (on a single Tesla V100 GPU) of proposed method is at least 22 and 69 hours shorter than the competitive baselines StarGANv2 and I-GAN (Tr). respectively.
7 CONCLUSION
We revisited the problem of content-style identification from unaligned multi-domain data, which a key step for provable domain translation and data generation. We offered LDM perspective. This new viewpoint enabled us to prove identifiability results that enjoy considerably more relaxed conditions compared to those in previous research. Most importantly, we proved that content and style can be identified without knowing the exact dimension of the latent components. To our snowledge, this stands as the first fimension-agnostee identifiability result for content-style learning, We showed that the LDM formulation is equivalent to latent comainnccoulle multi-domain GAN loss, and the latter features a simpler implementation in practice. We validated our theorems using image translation and generation tasks.
Limitations. Our work focused on sufficient conditions for content-style identifiability, yet the necessary conditions were not fully understood which is also of great interest. Additionally, our model considers that the domains are in the range of the same generating function. The applicability limited homogeneous multi-domain data, e.g., images with the same resolution. An interesting extension is to consider heterogeneous multi-domain models that can deal with very different types of data (c.g., text and audio). Additionally, our work also limited to continuous data modalities like images, audio, etc. Discrete data modalities like text will require extension of both theory and implementation. This challenge presents another important future work. Finally, another limitation of our work is that the proposed method is based on the GAN framework which is known to be unstable during training. Therefore, novel implementation methods based on more stable distribution matching modules such as flow matching are also of interest as a future work.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENT
This work was supported in part by the National Science Foundation (NSF) CAREER Award ECCS- 2144889, and in part by Army Research Office (ARO) under Project ARO W911NT-21-1-0277.
REFERENCES
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proc. International Conference on Machine Learning (ICML), pp. 214-223. PMLR, 2017.
Neal L Carothers. Real analysis. Cambridge University Press, 2000.
Yunjey Choi, Minje Choi, Munyoung Kim, Jung- WWo Ha. Sunghun Kim, and Jaegul Choo. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. In Proc. IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp- 8789-8797, 2018.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung- Woo Ha. StarGAN v2: Diverse image synthesis for multiple domains. In Proc. IEEE Computer Vision and Puttern Recognition (CVPR), pp. 8188-8197, 2020.
Pierre Comon. Independent component analysis, new concept? Signal processing, 36(3):287-314, 1994.
Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
Gang Dai, Yifan Zhang, Qingfeng Wang, Qing Du, Zhuliang Yu, Zhuoman Liu, and Shuangping Huang. Disentangling writer and character styles for handwriting generation. In Proceedings of the IEEF/CVF conference computer vision and pattern recognition, pp. 5977-5986, 2023.
George Darmois. Analyse des liaisons de probabilité. In Proc. International Statistic Conferences, pp. 231, 1951.
Imant Daunhawer, Alice Bizeul, Emanuele Palumbo, Alexander Marx, and Julia E Vogt. Identifiability results for multimodal contrastive learning. arXiv preprint 16.1 2023.
Cian Eastwood, Julius von Kügelgen, Linus Ericsson, Diane Bouchacourt, Pascal Vincent, Bernhard Schölkopf, and Mark Ibrahim. Self-supervised disentanglement by leveraging structure in data augmentations. arXiv preprint arXiv:2311.088/5. 2023.
Ryszard Engelking. Dimension theory, volume 19. North-Holland Publishing Company Amsterdam, 1978.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. In Advances in Neural Information Processing Systems (NeurIPS), 2014.
Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf. The incomplete Rosetta stone problem: Identifiability results for multi-view nonlinear ICA. In Proc. of Uncertainty in Artificial Intelligence (UAI), pp. 217-227, 2020.
Arthur Gretton, Kenji Fukumizu, Choon Teo, Le Song, Bernhard Schölkopf, and Alex Smola. A kernel statistical test of independence. Advances in neural information processing systems (NeulPS), 20, 2007.
Ishaan Gulrajani and Tatsunori Hashimoto. Identifiability conditions for domain adaptation. In Proc. International Conference on Machine Learning (ICML), pp. 7982-7997, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.

---

Published as a conference paper at ICLR 2025
Ziming Hong. Zhenyi Wang, Li Shen, Yu Yao, Zhuo Huang. Shiming Chen, Chuanwu Yang, Mingming Gong, and Tongliang Liu. Improving non con-transferable representation learning by harnessing content and style. In The Twelfth International Conference on Learning Representations. 2024.
Xun Huang, Ming- Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-imas translation. In Proc. of European Conference on Computer Vision (ECCV), pp. 172-189, 2018.
Zhuo Huang, Xiaobo Xia, Li Shen, Bo Han, Mingming Gong, Chen Gong, and Tongliang Liu. Harnessing distriibiitiittoon examples via augmenting content and style. arXiv preprint arXiv:2207.03162 162, 2022.
Aapo Hyvarinen and Hiroshi Morioka. Nonlinear ICA of temporally dependent stationary sources. In Proc. Artificial Intelligence and Statisties (AISTATS), pp. 460-469, 2017.
Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Existence and uniqueness results. Neural networks, 12(3):429-439 1999.
Aapo Hyvarinen, Hiroaki Sasaki, and Richard Turner. Nonlinear ica using auxiliary variables and generalized contrastive learning. In Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 859-868. PMLR, 2019.
Mohamed Salah Ibrahim, Ahmed S Zamzam, Aritra Konar, and Nicholas D Sidiropoulos. Cell-edge detection via selective cooperation and generalized canonical correlation. IEEE Transactions on Wireless Communications, 20(11):7431-7444 2021.
Paris A. Karakasis and Nicholas D. Sidiropoulos. Revisiting deep generalized canonical correlation analysis. IEEE Transactions on Signal Processing. 71:4392-4406,, 2023. doi: 10.1109/ 3333212.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In Proc. International Conference on Learning Repre- sentations (ICLR), 2018
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances Neural Information Processing Systems (NeurIPS), 33:12104-12114, 2020.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoen- coders and nonlinear ica: A unifying framework. In Proc. International Conference Artificial Intelligence and Statistics (AISTATS), pp. 2207-2217, PMLR, 2020.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proc. Interna- lional Conference on Learning Representations (ICLR), 2015.
Lingjing Kong, Shaoan Xie, Weiran Yao, Yujia Zheng, Guangyi Chen, Petar Stojanov, Victor Akinwande, and Kun Zhang. Partial disentanglement for domain adaptation. In Proc. International Conference Machine Learning (ICML), pp. 1 1455-11472, 2022.
Hsin-Ying Lee, Hung- Yu Tseng, Qi Mao, Jia-Bin Huang, Yu-Ding Lu, Maneesh Singh, and Ming- Hsuan Yang. DRIT++: Diverse image-to maage-to-image translation via disentangled representations. International Journal of Computer Vision, 188:2402-2417 2020.
John M Lee and John M Lee. Smooth manifolds. Springer, 2012.
Yahui Liu, Enver Sangineto, Yajing Chen, Linchao Bao, Haoxian Zhang, Nicu Sebe, Bruno Lepri, Wei Wang, and Marco De Nadai. Smoothing the disentangled latent style space for unsupervised image- to-image translation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10785-10794, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proc. International Conference Computer Vision (ICCV), December 2015.
12

---

Published as a conference paper at ICLR 2025
Qi Lyu, Xiao Fu, Weiran Wang. and Songtao Lu. Understanding latent correlation- coreelation-based multiview learning and self superviipri An identifiability perspective. In Proc. International Conference on Learning Representations (ICLR), 2022.
Walter Rudin. Principles of mathematical analysis, volume 3. McGraw-hill New York, 1976.
Mohamad Shahbazi, Martin Danelljan, Danda Pani Paudel, and Luc Van Gool. Collapse by condi- tioning: Training class- condiioondii gans with limited data. In Proc. International Conference Learning Representations (ICLR), 2021.
Sagar Shrestha and Xiao Fu. Towards identifiable unsupervised domain translation: A diversified distribution matching approach. Proc. International Conference on Learning Representations (ICLR), 2024.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint
2014.
Mikacl Sørensen, Charilaos I Kanatsoulis, and Nicholas D Sidiropoulos. Generalized canonical correlation analysis: A subspace intersection approach. IEEE Transactions on Signal Processing, 69:2452-2467, 2021.
Nils Sturma, Chandler Squires, Mathias Drton, and Caroline Uhler. Unpaired multi-domain causal rep- resentation learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, 2023.
Subash Timilsina, Sagar Shrestha, and Xiao Fu. Identifiable shared component analysis from unaligned multimodal mixtures. https://aaai.orr/ab22440.... 194222 2024.
Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. In Advances in Neural Information Processing Systems (NeurIPS), volume 34, pp. 16451-16466 2021.
Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through align- ment and uniformity on the hypersphere. In Proc. International Conference Machine Learning (ICML), pp. 9929 99399.999..01 PMLR, 2020.
Weiran Wang, Xinchen Yan, Honglak Lee, and Karen Livescu. Deep variational canonical correlation analysis. arXiv preprint arXiv: arXiv:/610.0344 1610.03454 2016.
Wayne Wu, Kaidi Cao, Cheng Li, Chen Qian, and Chen Change Loy. TransGaGa: Geometry-aware unsupervised mmage-to-image translation. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pp. 8012-8021, 2019.
Weihao Xia, Yulun Zhang, Yujiu Yang, Jing- Hao Xue, Bolei Zhou, and Ming-Hsuar Yang. GAN inversion: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 45(3): 3121-3138, 2022.
Shaoan Xie, Lingjing Kong, Mingming Gong, and Kun Zhang, Multi-domain image generation and translation with identifiability guarantees. In Proc. International Conference on Learning Representations (ICIR), 2023.
Heran Yang, Jian Sun, Aaron Carass, Can Zhao, Junghoon Lee, Jerry L Prince, and Zongben Xu. Unsupervised MR-to-CT synthesis using structure-constrained CycleGAN. IEEE Transactions on Medical Imaging, 5912):4249-4426 2020.
Shuai Yang, Liming Jiang, Ziwei Liu, and Chen Change Loy. GP-UNIT: Generative prior for versatile unsupervised image-to-image translation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (CVPR), pp. 586-595, 2018
13

---

Published as a conference paper at ICLR 2025
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-imade transla- tion using cycle-conse cycle-consistent adversarial networks. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pp. 2223-2232, 2017.
Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In Proc. International Conference on Machine Learning (ICML). pp.
2021.
14

---

Published as a conference paper at ICLR 2025
Supplementary Material of "Content-Style Learning from Unaligned Domains: Identifiability under Unknown Latent Dimensions"
A PRELIMINARIES
A.1 NOTATION
A set A is said to have strictly positive measure under Pa if and only [m(( > 0.
For a (random) vector 2; (x) and [az]; denote the ith element of æ, and a(i: f) and [a]ad denotes [æ<i), ai .),
The notation [M] (I,
ID E RDxD denotes identity matrix of size D.
For a differentiable function f RM - RN, the Jacobian of with respect its input æ, denoted by J f()), is the matrix of partial derivatives as follows:
@[f())
@ff()]
deM
J (x
ff=)y des
(
de
de) denotes the determinant of matrix X.
y denotes that is statistically independent of y.
æ denotes the absolute value of
a lo denotes the number of non-zero elements in
c-ball: Given metric space X C RD for some D E N,N((z) denotes the e-neighborhood of E X defined as
(z(2) = ( X || - 2/12-0111 < c).
Image set: For function m W - Z, and a set C W, m(.A) = (m(w) w
A}
For an injective function f X y, we denote f-1 f(x) + y as the left inverse of f. i..., f f(x) = x, Væ X.
Pre-Image set: For a general function m : - Z, we overload the notation m -1 to represent the pre-image: i.e., for set A Z, m (AA) {w W m(w) E A). If m is injective, then m simply the left inverse of m; i.e., m o m is identity.
For a function f X' + y and a probability measure P..... defined on X' [f] denotes the push forward measure defined by [f] [A] = P (=A))(())
(AA) for any measurable A C y.
dim(A') denotes the covering dimensio of the set X (see Appendix A.2
A.2 DEFINITIONS AND USEFUL FACTS
We employ standard definitions and facts from real analysis. We refer the readers to (Carothers) 2000
Rudin 1976) for precise definition and more details.
Injection, bijection, and surjection.
Consider function f - y. Then
f is injective, if for all æi, 32 x. x2
f( # f(x2). f is surjective, if for all y V, 3x X such that f(x) = y.
f is bijective or invertible, if f is both injective and surjective.
Also consider the following useful facts:
15

---

Published as a conference paper at ICLR 2025
1. f is injective if and only if there exist function g f(x) t such that g f(x) æ, Væ X, i.e., g o f is an identity function.
2. If f x+ is injective, then f X f(X) bijective.
3. If f og injective, and g is surjective, then f is injective.
4. If f
bijective, then is surjective and f is injective.
5. If f X + y is bijective, then f : + is injective, where y 2.
An important remark is that since injective functions can be "inverted" (fact above), there is no loss of information the input, i... the input information can "reconstructed" from the output. Hence, for the purpose of identifiability, it is sufficient to identify the latent variables upto injective transformations.
Simply Connected Sets.
First, a set (in X) is connected if and only if there does not exist any disjoint non-empty open sets A, S X such that AnC # o, p,Bre BCC & Ø, and C A U.B.
connected set is called simply connected if any simple closed curve on the set can be shrunk to point continuously while staying inside the set.
Homeomorphism.
A function f X + y homeomorphism if it is continuous bijection and has continuous inverse. there exists homcomorphism between and V, then X' and y are called homcomorphic.
Some useful properties are as follows:
R is not homeomorphic to RD for + M.
2. Open balls in RD are homeomorphic RD
3. Invariance of Domain: If X' is open subset of IRP and f X + RR is injective continuous map, then y = f(²) is open in RD, and f is homeomorphism between X and
y.
Covering dimension (Engelking. 1978
An open cover of set X' is collection of open sets such that X' lies inside their union.
refinement of a cover y of a set X' a new cover Z X such that every set in contained in some set in y.
The (Lebesgue) covering dimension, denoted by dim()), a set is the smallest number n such that for every open cover, there is refinement in which every point in X' lies in the intersection of no more than n + covering sets. For our purposes (subsets of Euclidean spaces), the covering dimension is the ordinary Euclidean deimension, e.g., for lines and curves, two for planes, and so on.
Some propoerties of the covering dimension follows:
1. Homeomorphic spaces have the same covering dimension.
2. If an open subset IR_, then dim(X' D.
3. For open sets X' RD, C R.M, dim(A' x 3) dim(X)) + dim()").
4. For a continuous funtion f and open set X, (f s dim(X').
5. The set Ag C. R of q-sparse vectors, i.c., A, = {x e |||||=/1o < /} has dimension dim (Ag) = q. This is because it is the union of a finite number (2) of q-dimensional
hyperplanes (i.e., hyperplanes with all elements 0, except elements at fixed positions).
B DETAILED DISCUSSIONS ON EXISTING IDENTIFIABILIT RESULTS
In this section, we review the existing content-styy style identifiability results from Sturma et al. (2023) Timilsina al. 2024 Kong al. (2022); Xie al. (0033
16

---

Published as a conference paper at ICLR 2025
B.1 IDENTIFIABILITY RESULT IN STURMA ET AL.
2023
The works of Timilsina et al. 2024 Sturma et al. model follows:
2023
considered a linear mixture-based generative
((())(()) Rdb and G!n) (R(do xxidett is an invertible mixing matrix for
2(1) = (c" {1,2).
(7)
where c E the nith domain.
Theorem B.1 (Identifiabilityy from Sturma et al. 2023), Under 7 assume that the following
are met: (i) The conditions for ICA identifabillty (Comon 1994) is met by each domain, including that the components of 2(n) = |e (s(m))T] are mutually statistically independent and contain at most one Gaussian variable. In addition, each 2!n) has unit variance; (ūi) P (())) # P. (a,) PP (mn) + -2 Vi,j E [dc d i i j. Then, assume that (m:imm) are obtained by ICA followed by cross domain matching distribution matching, i.e., enforcing
<1)) (d) x12) for ,,. Denote e(1) ee_ 20) and el?) = =e] 212) We have the
following:
sin)
ke (n)
((m) [dc].
(8)
where k E (+1,-1) and π is permutation of (l,...,,cc dc}.
Obviously, the result here relies the element wise independence among the entries of 2*))
B.2 IDENTIFIABILITY RESULT IN TIMILSINA ET AL. (2024)
Timilsina et al. (2024) proposed to solve the following problem order to extract the content and
style for model in
(n) E (Roe Q Rd x(do 1,2,
find
(9a)
subject to
(1) (d)
(2)
(9b)
n= 1,2,
(9c)
E
(Q( = n 1,2,
(9d)
(h) e (xl
(T) = n = 1,2,
(9c)
where a (d)
denotes that the distribution of a and b are the same. The following identifiability result was established of the solution of ():
Theorem B.2 (Identifiability from Timilsina al. (0224). Let Qc and Q3 denote the solution of (9. Under 7 assume that the following are met:
For any two linear subspaces p(n) C CRdc+dg" n = 1,2, with dim(pln)) = (1))
p(n) # x Rdb") and linearly independent vectors {v." (m) Rdo+d
R gdc n
1,2, the sets A(n) conv yi (n) ..... ,
(m) + p(n), n 1,2, such that if Pe.a(m) [A(ni] 0 for n or n 2, then there exists l: E such that the joint distributions P Pe.. [k) + P P [kAl))), where km)(n) {ka A(n)3.
2. One of the following assumptions is satisfied:
(a) The individual elements of the content components statistically independent (d) and non- Gon-Gaassian. In addition, e₁ + kej, Vi + j. IR and c; -c, Vi, i.e. the marginal distributions of the content elements cannot be matched with each other by mere scaling.
17

---

Published as a conference paper at ICLR 2025
(b) The support C, is hyper-rectangle. i.e., C [-a1, a1l x x I- ade. Adel (d)
(d)
Further, suppose that ci F kcz, + vk E R and c; # -c, Vi.
(n) x(n) Oc and Qs x(n) = n) (()), for some invertible ((n) for all
(m)
Then, we have Qe 1.2.
The first condition similar to the domain variability condition considered in Kong et al. (022) Xie al. (202)) and our Assumption 3.2 but with more specific form that is adjusted according the linear mixture model 7
B.3 IDENTIFIABILITY RESULT IN XIE ET AL. (0233 KONG ET AL. 2022)
Note that the results in Sec. B.1 and B.2 are only applicable in the linear mixing system (see (7). However, the linear model is hardly practical for complex data modalities like images. In the nonlinear case, Xie et al. (203) Kong al. (2022) showed the following:
Theorem B.3 (Identifiability from (Xie et al. 2023 Kong et al. 2022)). Denote = (c, 8) E x S and as the latent variables and an auxilaary variable" respectively. Assume the following assumptions hold:
(A1) (Smooth and Positive Density): log log p(=|u) is second-order differentiable and p(zlu) >
(A2) (Conditional independence): p(zlu) -1 ds p(z/lu).
(A3) (Linear independence): For any 8 E C RRds, there exists 2ds + values of u, i.e., Uj with j 0, 1,..,,.. ,dg such that the 2ds vectors w(s, u,) w(s, un) are linearly independent, where wss u.) - Deas/Onss 821//88f,.... 8us, /083g and qi =logp(s u.).
(A4) (Domain variability): For any set A E Z such that (i) P 22 [A] for any u' u and (ii) A cannot be expressed as B x S for any set B c C, there exist u1 u2 E U such that P P=/us [A] + IP (2/uu(A) where IP z/2,, [A] = SxEA P(zlu,)dz.
Then, by matching the model distribution in ( with the marginals IP Pain)) jointly, the component- wise and block-wise identifiabilit of and guaranteed, respectively, both up to invertible nonlinear transformations.
"The auxiliary variable is notion from the nICA literature (Hyvarinen & Morioka 2017 Hyvarinen
et al. 2019 Khemakhem et al. 2020). In the context of multi-domain learning. t olten represents the domain index.
The result interesting and insightful-partiiuaatte the domain variability condition (A4) is also used in analysis. However, some major challenges still remain: First, (A2) requires that all components of ((n) (c, s(x)) are statistically independent given (corresponding to u.) The use of this condition is because the identifiability proof is reminiscent of nICA (Khemakhem et al. 2020 Hyvarinen et al. 2019). But elementwise independence is nontrivial assumption. As shown analysis, for block-wise identifiability of and g(r)), elementwise independence among individual entries = (c, 8) is not necessary. In addition, (A3) needs that at least N = 2ds - domains exist (as represents the domain index in this context), which is hardly practical- many domain translation tasks were performed over N = 2 domains (Choi et al. 2020).
C PROOF OF THEOREM 3.3
We restate the theorem here:
Theorem 3.3 Under Eq. ( suppose that Assumptions 3.1 and 3.2 hold. Then, we have
f (m)) >(c) and (x)) = (n)()),, Vn E [N], where C Rdc and å S + Rds are injective functions.
Proof Outline. The proof pipeline of Theorem 3.3 can be divided into two parts: content identification and style identification. The content identification part includes two steps: (i) showing that f does
18

---

Published as a conference paper at ICLR 2025
not depend upon the style part, and thus is function (denoted by 7) of the content only and (ii) showing that y is injective. For the first step, we use the domain variability assumption and matched content distribution to conclude that the preimage sets (under f g) for any extracted content cover the entire style domain. This will imply that the extracted content invariant of the style part. In the second step, we use the property of the determinant of the jacobian of f o g to derive that Y should be injective. For style identification, we use the independence constraint and the result of content identification part to conclude that the extracted style does not depend upon the content part. And finally we use the injectivity of f to show that should also be injective.
Proof. The proof is as follows:
C.I CONTENT IDENTIFICATION
Let 2(1) (()) and 2 C S. Let aet fo(a)l)), An [M] and ((m) = fs((o(()), Vn
[N]. The proof consists of the following two steps:
Step 1. First, we show that under the assumptions in Theorem 3.3 ट does not depend upon (n))
Step 2. Next, we show that e(*) is transformed from the true shared component c via an injective function y : Rdc for all E [.V]
Step 1: We want to establish the following:
0c(m) m)
O, Vi [dcl.j [ds].
(10)
(n)
This can be shown using similar arguments in the domain adaptation work (Kong al. 2022). To be
specific, let h Z IRac+ be defined as follows:
h f
Let ē = hc(2). Due to the distribution matching constraint Ac = C:
2ah. the following holds for any
Paco [Ac] = [ac), Vi,j [N]
2()) [ (Ac)] 20))
(11)
where le (Ac) == {z (z) Ac} the preimage of hc(.) = [h(-)] The equivalence in (a) holds because Pa( [Ac] P (z())))) [Ac] = (P200) (hc Vn E [N].
Sufficient Condition of 110 In order show 10 holds almost surely, it suffices show that
VT Č, Bc + and Be CC
s.t. h 1 (7) = Be x S.
(12)
Eq. (12) implies that no matter what the value of the private variable a (m) is, as long as the shared variable E Be, the extracted è C. This implies that small changes v, E [ds], while keeping
all other variables fixed, does not result in any change in C, Vi whl-h means that (IO) holds.
Further, the following condition sufficient to ensure that (12) holds:
Ve E Ve 0, 3 Gc + & and Go S C,
he (N)( = 9c x S
(13)
where (o(e) fc' E ē ||e use contradiction. Suppose that z = (()) Z with
cl|2 t) is an open To how 133 implies (22
(13 does not imply 12) Then there exists an ñ E N and
? Bc (21dd he
19

---

Published as a conference paper at ICLR 2025
and 8 (i) S such that h2( # c. Since hc(-) is continuous function there exists é > such that
hc(²) (Na(c))
3 h h (Ni(e))
However, states that
hc (NG(a)) = Gc x S, for some Gc C C. Since Be S Gc by their respective definitions, it implies that č E Gc. And therefore,
C,
he (Wi(e)),
which is contradiction. Hence, prove
13
implies 12) Therefore, it is sufficient to show 13 in order to
Proving (T): Proving (33) (equivalently (TO)) can be accomplished using contradition. From L3 it is sufficient to show that he (Ac) Bc S for certain Bc C C.
Suppose that he (Ac:) Bc S for any Be C C. Then, we divide he (Ac) into two disjoint subsets, namely, 9 and F. The 9 part can be represented as Be x S for certain set Be S C (it is possible that G Φ) The F part cannot be written as De for any De C. Note that J + ø due the fact that h (Ac) * Bc S, and has strictly positive measure. Then, the following follows from
Pa( [he (Ac) 20 1 (Ac)
P [g] + P 220 [ [2)) + P2()) [F]
P Pa| P.tBBPPss. 8000 Bc] [F] = Pa[ (S
Px(0 [F]
P 2c00 [ = 20) [F]
(14)
where (a) holds because we have P ( 1, Vn [N]. However, (1 cannot hold due to Assumption 3.2 This is a contradication.
Step
Consider the Jacobian of h
(n)
Jne (s) Jng (8)
Jn(c, 8)
(15)
Jns (c)
In step I, we have shown that ho (s) 0. This implies that
(()) (c), Vn E [N]
(16)
for some function Y C Rdd Now, we want show that y injective. Note that h Z + Rdotds is differentiable injective function because it is composition of differentiable injection f and bijection g.
This implies that (cet(Jn(c,,)) = [det(Jnc (c)]llettt) (s)) * o, V(c, 8) e C x S. Hence [det(J he (c)) + 0, Vc E C, which in turn implies that is injective.
C.2 STYLE IDENTIFICATION
Our goal is to show that hs,c, s(m)) = 8(s(1)) for some injective function 8. As before, we first show that (e,c g(n)) is only a function of ((m) i.e., hs, ((1) 6(s!n)) for some function Then, show that 8 has to be injective.
To proceed, note that the statistical independence constraint 2n)) _1 implies the following:
2n0) (n)
p()"), (( = p(c p(((") ((((")
1( 0,
20

---

Published as a conference paper at ICLR 2025
where I(X;Y) denotes the mutual information between X and Y. Since we have already established ( ((c) for some injective function 7, we express y (() (where y 1 is the left inverse of see Appendix A.1 Then, g(x) + c(n)) y ((c is Markov chain. This is because, see when conditioned e.) c 7 ((( constant, which is independent with g(m), Next, the data processing inequality in information theory (Cover 1999 Theorem 2.8. 1) (and corollary following the theorem) implies that
, 2 (3" ( 1(3"))
As mutual information is always non-negative, we have I(3(");) This implies that hgs s(x) does not depend upon c. In other words, as pointed out in (Eastwood et al. 2023 Theorem 2, Step 3), we have (c, g s(")) 8(s!)), Vn for some function ô.
Finally, see that 6 is injective, we contradiction. Suppose that is injective. Then there exists 81, 82 E S and 81 # 82 such that
8(81) = 8(82).
(17)
However,
h(c, 81) = (hcle, 81) hs(c, 81)) = (v((),
Similarly,
h(c, = (hole, 82), hs(c, 82)) = (r(c),8($))
However, Eq.
implies that
h(c, hic, 82)
which is contradiction the injectivity of h. Hence, ô is an injective function.
D PROOF OF THEOREM 3.4
The theorem re-stated as follows:
Theorem 3.4 Assume that the conditions in Theorem 3.3 hold. Let f represent any solution
of Problem (4). Assume the following conditions hold: (a) de de and ds Z ds. (b)
0 < Patm) (z) < oo, Vz 2 C x S, Vn [N]. Then, there exists injective functions
Y : C + Rdo and 8 : S + IRds, Vn [N] such that è = fo( = v(c) and J (x) = 6(s(m)), An E [N].
Proof Outline. The proof of Theorem 3.4 follows the following pipeline
1. Content identification:
Step 1
Showing that the extracted content 2(m) does not depend upon the style part s(n). Step 2 Showing that the sparsity regularization forces the extracted style part to be free of any content information; and that, consequently, the extracted content has to be injective function of the true content.
2. Style identification: showing that the extracted style is an injective function of the true style.
We will fist show that unknown dimensionality does not affect Step I of content identification; i.e.. when de 2 dc, ((() will be independent of g(1) using the arguments from the proof of Theorem 3.3 The most important part of the proof is Step of content identification, which requires completely different analytical approaches compared that used for Theorem 3.3 For this, we first use contradiction to show that if were not injective, the injectivity of h will imply that hg depends upon e within set of strictly positive measure. This will imply that the extracted style contains information from both the style as well as the content part, which will require E [
[x) > dg. However, a feasible
solution to Problem can be constructed using the inverse of g which has [||( ds-
This will contradict our assumption that f is an optimal solution to Problem Hence, 7 should be injective. Finally, style identification follows the same proof as that of Theorem 3.3
21

---

Published as a conference paper at ICLR 2025
Proof. The proof is as follows:
Content Identification.
Step 1: We want to show that
0c n)
(n) O, vi [dcl.j [ds].
(18)
am
which will imply that (c 8) 7(c) for some function 7.
For this, the proof of Step I of content identifiability of Theorem 3.3 holds without any modification since all arguments of the proof are cimension-agnostiin
Step 2: We want to show that y is an injective function. For the sake of contradiction, assume that y is not injective. Then there exists č E C Cndd& č C, č & č such that 7(c) >(ē). This is equivalent to
he(c, s) = hc(c,s).
(19)
However, we know that h Z Rdo+ds an injective function, since is composition of injective and bijective function. This implies that
h(c, s) + h(c,s) s)
(20)
Eq.
and 20
imply that
(T, 8) + hs(c, 8)
(21)
Since hg is differentiable function, C, 8* E S, such that
21 implies that there exists & E [ds), E [de] and ct
Ihsle (c" 8*) + O. 8cr
(22)
Also, hs being differentiable implies that the above partial derivative is continuous. Hence, there exists O, such that the above non-equality holds (z S Z, where z" -c², s*). In addition, with small enough because the continuity of 0lksle [bs]a is strictly monotonic (either de,
increasing or decreasing) within the set (z*). with respect to Cr. This implies that [hs]e is locally monotonic function w.r.t its input e, at any E NN(x**)
However, this will imply [hcli, Vi E [de] cannot depend upon o, when restricted to Ne(2*), i.e.,
Vž e NN(2*) and all i E [dc].
dbccl (z) 0. dc,
(23)
This is because, if (lkeli (z) f 0 for some E (z and E [de], then there exists a 0 such Ber
that [he] strictly monotonic within (z), with respect to Cr.
Consider the non-empty open set R (N((2*) n (,,(2) 2. Due to the assumption that 0 < Pe00 (2) < oc, Vz E Z, has strictly positive measure. However, [he], and [hg]e are simultaneously changing monotonically everywhere in R with respect Cr. which implies that [hs] does not hold. This contradicts the independence constraint (2b). Hence (23) has to hold.
Now, Eq.
23 implies that
= =)
for any two points (C, 3) E (z*).. Here, ~_, represents all components of E excluding c, Next, the injectivity of indicates that for any (c. 3), (c. 3) E Ne(x*), if + 7, or 3 # 8, then
h Cr 3) * (c. G. 3). 3).
(24)
Next, show that Eq. (24) results in contradiction to the fact that f is optimal solution of Problem 4 This contradiction will conclude that y is injective, as Eq. (24) was obtained by the assumption that is not injective.
22

---

Published as a conference paper at ICLR 2025
Note that can construct feasible solution f of Problem
(4 satisfying the following:
[g "l((x("))
[g (1s(x
(X)
fc(a(())
E Rdd and fs(a(()))
E Rd vn E [N], E x(m)
which implies that Vn,
(xx dg.
(25)
This means that the optimal value of Problem (4b) is smaller than or equal to ds. as at least one solution can be constructed attain this value.
Now, consider our learned g(() [f g (1) lgg(x(())) We will show the following chain of inequalities Vn [A]]
Eg-P, ((m) [||3||||
8 dP gin)
hs(2)
|0||0 dlp g(n) + S InolNices) |0]|o g())
hg(2)VNs(((((E))
ds dl a(g + Insuncee)) (ds +1 dP ha ha(2)))a(A((()))
ds (IP gm)) hs [2) Pg0 (hs((V(=2)))) + (ds + 1)P g(n) (Ns(
ds + Ps (Ne
ds.
(26)
where (a) used the to-be- proven facts that |8|0 > ds almost everywhere on hs and that @ lo 2 ds almost everywhere on hs (N+(=*)). Note that if (26) holds, we reach a contradiction that f an optimal solution of Problem (4). as the solution in (25) can attain a smaller objective value. We prove the facts used in (a) in the following:
First, to see that ||@||o 2 ds, a.e. (on hg ()). suppose for the sake of contradiction that ||0||o < ds on of strictly positive measure, say Q hs( hs(2), i...
hns] #P
#P_(m) (n)) (2) > 0
P Pm(m h (x((()) 0.
Since P2())) admits a PDF with 0 < (2(m)(2)) < 0o, Vx E 2, h5 (Q) should contain an open ball
Z S Z. However dim (z) = de + ds since is homeomorphic to Ric+ (see Appendix A). Let
A, = (a E Rds+ |alla q). i.e., the set of all vectors with at most q non-zero elements. Note that dim(Ag) = 4. Then Q C Adg - Hence dim () S ds 1
Next, h is also injective function from Z to Q x xC). Note that dim()(C)) < de because continuous function and dim(C) = de- But this implies that dim( @ x (C)) diim ( dim(>(C)) de ds.
However, this contradicts that h is injective function from Z to Q x xC) because the domain of h has larger dimension than the co-domain
(C). Hence [hs] ((n) ( = 0 and || ds, a.e.
consider the set S x Cr, where C, is the support of c,. Eq. (4 implies that M = (88 C,) (2*) is homeomorphic to a subset of hs (N)((())) To see this, consider a fixed (c. (z*). Then, define a continuous function
Second, to show that || 2 ds + almost everywhere on hg (N
t(cr, s) = ho((err..r. cr,
ie., hy with fixed partial input c_.- Eq. (4) implies that is an injective function over M. Now, note that we have t(M) C hs(Aa((z))) and that & defines homeomorphism between M and t(M). Hence M is homeomorphic to a subset hg (x))) This implies that dim(hg(N dim(M). However, dim( ds + because M is open set in IRds+1 S is open
2
23

---

Published as a conference paper at ICLR 2025
set in Rds, C. is open set in R, and thus (8 C,) (x") is open set in R Hence, Gimmhs(WW (=*)) 2 dim(M) = ds + This concludes the proof of the inequalities in (26).
Finally Eg()) ||||||||| > ds, vn E [M] contradicts that f is an optimal solution to Problem Hence, Y an injective function.
Style Identification.
Finally, hs cannot depend upon e by the same reason as outlined in Sec. C.2 This implies that
hy (c, 8) = ((s), for some function S - Rds, Similarly, is injective by the same reason outlined in Sec.
This concludes the proof.
PROOF OF THEOREM 4.2
E. RELATIONSHIP BETWEEN THE DATA AND LATENT SPACES.
Before proceeding with the proof of Theorem 3.4 clarify the relationship between the data and latent spaces to provide a clearer understanding of the theorem and its proof.
First, note that C and S open sets in Rdo and Rdds respectively. Thus, x= 9(C S) forms de di-dimmesiooal manifold within Rd, In Theorem 4bb),, by assuming q C S + X is bijective, we effectively assume that x S is a de + ds-dimensional manifold within Rdc+ds, This holds because é x S q- o gC S), and since q a bijection. Consequently, although the ambient space Rdc+tds is higher dimensional than S, the spaces C S, C x S. and all have the same manifold dimension de + dg, allowing for the definition of bijective mappings between them.
composition of two bijections, is itself
E.2 THEOREM
The theorem is restated follows:
Theorem 4.2 Let Eg 2 d) be any differentiable optimal solution of Problem (6. Let C and S be simply connected open sets, Let 0 < Px(*) (z) < oz, Z = = x S. Under the assumptions Theorem 3.3] have the following:
(a) If de = de and ds ds and (6b) absent, then @ C S - X' is bijective and f = --
is also solution of Problem (2).
(b) If de > de and ds > ds and @ C.x S X is bijective, then J =q is also solution of Problem
E.3 PROOF OF PART (A).
One can see that problem (6a) is equivalent to the following optimization problem:
find
( q, ec, [q] #P
{es Jo=1
(27a)
subject to
ef = P Pg(,
(27b)
re ~ N(0 (o0,
(27c)
First, the solution of (6a) satisfies the distribution matching constraint (27b) because of (Goodfellow et al. 2014) [Theormm I]. Next, define:
(rc, (n)) = q(éc(rc), (r)")))
Consider the following theorem from Zimmermann et al. 2021
24

---

Published as a conference paper at ICLR 2025
Proposition E.1 Sec. A.5. "Effects of the Uniformity Loss", Proposition 5 (Zimmermann et al. 2021)). Let M and N be simply connected and oriented C¹ manifolds without boundaries and ::M + N be differentiable map. Further, let the random variable
M be distributed according to z E p(z) for regular function L...0 <. < co. If the push forward p*n( of through h is also regular density, ie., 0 < p@h 00, then h. is bijection.
Note that Proposition applies to a range of cases, including low dimensional manifolds M and N embedded in spaces with different dimensions, long M and N have the same manifold dimension (see application of Proposition E. Zimmermann et al., (02T) Corollary 1). The proof relies on showing that the determinant of the Jacobian of h cannot vanish. Here, the Jacobian of h at point corresponds to the matrix representation of the differential dh((p) MpMM + TM where T M. and [hipp:- are tangent spaces at point at point and hp() in M and N, respectively Lee & Lee 2012) [0h))/Ch 3]. Since the manifold dimensions of M and A are the same (required for h to be bijective), the Jacobian a square matrix.
In case, Z and X is simply connected and oriented C' manifold without boundaries. In addition, (n Rds+dc + X' is the differentiable map that we hope to use the above proposition to characterize. Every element of the random vector
(res is (")) follows the standard normal distribution. Hence, is readily seen that 0 < Pr(.) (r) < oo, Vr Rdc+da, In addition, the assumptions that () < Pg60 (z) 00, Vz E 2 with simply-connectee
2 and that is a differentiable bijection imply that < Pa(t)) (a) < oC, Vo E X, Vn and that X' is simply connected.
Following the above arguments, we can apply Proposition F.1 to conclude that (")), Vn E [N] are bijections. This, in turn, implies that ④ is surjection and êc and ēş are injections. However, since defined on the range of éc and Cg. q a bijection. Consequently, @c and ēg are also bijections.
To proceed, notice that (27b) implies that f = q satisfies the following:
Ce
#P
20
#P #P un Vi,j E [N], A9c
2-1 the output dimensions of that correspond the content part. To see the above where 9c
equality, let (()) (re)), (m Then,
qe (a (n) qe ((e,) (re).
Since (n) P Po this implies that
[elle Vn E [N]
Since the distribution we] is independent of n. Iae lac'l p Vn [N] are matched. This means that q satisfies the constraint (aa. Similarly, since é (e (m)) Vn [N],
qcc IP (in)
1 IHF (())> Vn E [N].
Hence also satisfies the constraint (2b). Therefore, q an optimal solution of Problem (
It remains to show that a solution to Problem (27) under our data generative model in ① exists.
When C C Rdc and S C Rds are open and simply connected, and Pe(c) 0 and Pg(n) (s) > 0, Vs E S, C, distributions, i.c., (/ Iac) and (0 J, ) to the content and style distributions, IP and PP (m) respectively. Such functions can be constructed by using Darmois construction (Darmois) 1 1951]
E [N], there exists a smooth bijective function that transforms standard normal
to first convert the normal distribution to uniform distribution, and then using inverse Darmois construction convert the uniform distribution to P. (and P a60) (see (Hyvärinen & Pajunen 1999) for more details), Let Ki (O Ie) + C and Kig (O 10,14 dg) + S denote the atorementioned
(n) Vn [NV] for the optimization functions. Then, solution Problem (77 is g, Kc. and Kig
functions q. ec, and es (r) E respectivll... Hence, problem is feasible. This concludes the proof.
25

---

Published as a conference paper at ICLR 2025
E.4
PROOF OF PART (B).
Problem ( is equivalent to the following problem:
N E
min a "
1
Σ
(m)
(rs
( [|l]
(28a)
N
==1
subject to
M [q]
=c (r)m)) =P =
(28b)
TC TC M de), r (n)
(os
(28c)
Following the same arguments as in the proof of Theorem Part (a), Eq.
28
implies that
1e0 I#F w(o)
[9c læ _ Vi,j E [N]
The above implies that
Mc #P ((n) = Pe vn E [N].
which means that 4c satisfies the constraint (4b). Similarly, since (rc) a()))
(r' (m) Vn [N].
(n)) 11 [9s
#F (())? An E [N].
Hence q also satisfies the constraint
Therefore, q is also a solution to Problem (4.
F BENCHMARKED MULTI-DOMAIN TRANSLATION SCHEMES
StarGAN and StarGAN v2. The representative multi-domain translation frameworks, namely, StarGAN (Choi et al. 2018) and StarGAN v2 (Choi al. 2020), learn common translation function that matches distribution between all pairs of domains. To be precise, let t x + X' represent the translation function that takes in a sample (=()) in the source domain s, and a style component (!t) from the larget domain and outputs the translated sample (8-++) from the target domain t
(x)
((x(1), (()).
In (Choi al. 2020 t learnt by optimizing the following criteria:
LstaGAN Lmatch Linvs
where Cine promotes invertibility of the translation function, and is composed of cycle-consistency, style diversity, and style invertibility terms, whereas Lmatch promotes distribution matching between all pairs of domains after translation by t. Specifically.
mattch
(29)
Σ
log di (x(+)) ++))+++ + log (1 ( (1 ((())))) (())
(30)
M [N]
where d; is the discriminator for the ith domain. Here, all pairs of domains, (s, t) [N] x [N], are considered for distribution matching. This can be potentially expensive for large N. The StarGAN loss matches N(N 1) pairs distributions for domains, which may not be affordable for large N.
Learning Pipeline of (Xie et al., 2023). The work (Xie et al. 2023) proposed the following pipeline. First, multi-domain generative model is trained, which is similar to our system. Second, instead of directly using the trained system, the work proposed to train a separate StarGAN with new regularization. To be specific, the work proposed to use synthetic paired samples (xx()), (*)) generated by their generative model to further regularize LStaGGAN which leads to the following loss:
LstarGAN
26

---

Published as a conference paper at ICLR 2025
where the second term is defined
Saup an1) [1((2()),
which Pa represents distribution the learned multi-domain generative model, and g()) is obtained from (+) by using a separate style encoder.
As we mentioned, one can show that the generators in fact invertible in the learned generative model (see Theorem 4.2). the above procedure can be circumvented. Using an off-the-shelf GAN inversion solver assists translation without retraining a regularization StarGAN.
G EXPERIMENTAL DETAILS
G.1 NEURAL NETWORK
We adopt the neural architecture of StyleGAN-ADA to represent the generator q as well as the discriminator d in Problem 6 Specifically, StyleGAN-ADA generator consists of a mapping network and a synthesis network (Karras et al. 2020p. We use the synthesis network to represent whereas the mapping network is discarded. For each of ec and ( Jn= use 3 layer MLP with 512 hidden units in each hidden layer. The input and output size of the MLPs for ec and {es (m) J de 384 and ds 128, respectively. We feed the output of ec the first 5 layers of the synthesis network (i.e., q). and the output of es Vn E [N] are fed to the remaining layers of (n) Yn are much more simplified since they are longer restricted
Compared (Xie al. 2023), ) es to component-wise inveriblee functions, which require special neural network design.
G.2 TRAINING HYPERPARAMETE SETTING
The hyperparameters used for GAN training in Problem 6 similar to those used in StyleGAN- ADA. Mainly, we use Adam (Kingma & Ba 2015) with an initial learning rate of 0.0025 with a batch size 16. The hyperparameters in Adam that control the exponential decay rates of first and second train the networks for 300,000 iterations. Sparsity regularization weight Problem (6) is set to 0.3 for all experiments. We also the data augmentation techniques proposed in (Karras al. 2020 for improved GAN training on limited data.
order moments set to 31 0 and 82 0.99, respectively. For all datasets,
G.3 GAN INVERSION SETTING
We follow the optimization procedure in (Karras al. 2020 for GAN Inversion. To summarize, recall the GAN inversion problem for the multi-domain translation task:
(c, 8) arg min div(g(c, .)). c.e
(31)
Eq. 31 is solved using gradient based optimization of c and s using Adam (Kingma & Ba 2015)
with an initial learning rate of 0.1. The hyperparameters of Adam are set to 31 0.9, /32 = 0.999.
The optimization is carried out for 400 steps. The div function is realized by using mean squared error in the feature space of pre-trained VGG16 (Simonyan & Zisserman 2014) neural network, i.c.,
div(a a, b) = IVGG(a)- VGG
MM where M is the output dimension of VGG(.).
G.4 BASELINES FOR MULTI-DOMAIN GENERATION
We use Transitional CGAN
I-GAN
Xie
et
al.
2023
Sty y BelleGAN-aa AD2 Karras et al. 2020). and
Shahbazi et al.
202
baselines for multi-domain generation task.
27

---

Published as a conference paper at ICLR 2025
Note that I-GAN is chosen because it is the most relevant to work, whereas AtyAeCAN-ADA is a very representative work among conditional GANs. Finally, Cransitional-CCatt ionallcGAN is a recent and representative conditional GAN for limited data regime (note that the datasets used in our work fall under limited data regime (Karras et al. 2020). For all the baselines, training is done with their default setting for all datasets. All the baselines are also trained for 300,000 iterations with batch size of 16 for a fair comparison, as well as to control the training time.
G.5 BASELINES FOR MULTI-DOMAIN TRANSLATION
We use St at 1 Choi et al. 2020). SmoothGAll Liu et al. 2021), I-GAN (Gen) Xic
translation task. Note that I CACAN (Gen) refers to the method that uses the multidomain generative model proposed
al., ) and I GAN (Tr) (ie et al. 2023) as baselines for the
by Xie et al. 2023 to carry out the translation procedure via GAN inversion described in Section
Whereas, I- GAN (Tr) is the separate domain translation system proposed by (Xie et al. 2023)
as described Appendix F Note that I-GAN (Tr) trained using the paired samples generated
by I-GAN (see Sec. G.4).
G.6 DATASETS DETAILS
AFHQ. We use the AFHQ (Choi et al. 2020) dataset for both
generation and translation tasks. The AFHQ dataset contains images animal faces in three domains: cat, dog, and wild with 5066, 4679, and 4594 training images, and 494, 492, and 484 testing images. We resize all images to 256 x 256 for training and testing.
CeIebA-HQ. The CelebA-HQ dataset (Karras al., 2018) for both multi-domaie generation and translation tasks. contains high-resolution images of celebrity faces along with attributes such as gender, hair color, etc. We split the dataset into domains based on gender. The male domain contains 18,875 images, whereas the female domain contains 1 11,025 images. We hold out 1000 images from each domain for testing and use the rest for training. Similar AFHQ, we resize all images 256 x 256 for training and testing.
CelebA. The CelebA dataset (Liu et al. 2015)) is used for multi-domain generation task. It contains 202.599 images of celebrity faces along with different attributes. We split the dataset into 7 domains based on the following attributes: "Black hair". "Blonde hair". "Brown hair". "Female", "Male", "Old", and "Young". We resize all images to 64 64 for training and testing.
G.7 REGARDING GAN TRAINING STABILITY
GAN training is known to be quite sensitive to hyperparameter setting, and even randomness in initialization. However, our experiments, we did not observe severe optimization issues or performance degradation with minor changes in hyperparameters, c.g., A, batch size, and random initialization. Nonetheless, we did observe that using overly deep fully connected networks (>5
(n) Vn could lead to convergence issues. Hence, we fixed the number of layers the content and style encoders to be for datasets except for the MNIST digits experiment presented in Appendix H.I
layers) for content encoder ec and style encoders eg
H ADDITIONAL EXPERIMENTAL RESULTS
H.1 ADDITIONAL DATASET: ROTATED AND COLORED MNIST
In this part, we present additional experiments on MNIST digits, where the domains are rotated MNIST and colored MNIST. Here, the style corresponds to the color and rotations for the colored and rotated MNIST, respectively. Whereas, the content is the identity of the digits.
Dataset. For the rotated MNIST domain, we apply rotation uniformly sampled from [-70, 70] degrees to all the MNIST digits. For the colored MNIST domain, we apply randomly sampled color the
28

---

Published as a conference paper at ICLR 2025
MNIST digit. Specifically. for a given MNIST digit image, we uniformly sample vector from [0, 1] x [o. 1] x [0, 1I, corresponding to the normalized RGB channel values. Then, we multiply each pixel of the given smage-which is also in the range [0, 1ith the color value. We resize the images in both domains into 32 x 32 resolution. Both domains contain 60,000 training samples each.
Neural Networks. Since MNIST digits are of very low resolution compared to the AFHQ and CelebA-HQ datasets, we use much smaller and simpler neural architectures to represent the generator
q, content encoder EC, style encoder as (m) and the discriminator d Mainly, both ec and es represented by linear layers, i.e., matrices of dimensions de x de and ds x ds. respectively. The architecture of q and d are shown in Table and In the tables, Conv represents convolutional layer, BN represents batch normalization, I-ReLU represents leaky ReLU activation with slope 0.2, and Up represents nearest neighbor upsampling layer, and dropout represents dropout with probability 0.25.
Layer
Description
Linear
Concatenates content and style, maps 128 2 128 filters of 3 x 3, stride 1, padding 1, upsample 2
Conv, BN, I-ReL.U, Up
Cony, BN, I-ReLU, Up
128 filters of 3 x 3, stride 1, padding 1, upsample
Conv, BN, I-ReLU, Up
64 filters of 3 x 3, stride padding I, upsample 2
Conv, BN, I-ReLU, Up
64 filters of 3 x 3, stride I, padding 1, upsample 2
Conv, Tanh
filters of 3 x 3, stride 1, padding
Table 3: Architecture of q.
Layer
Description
Embedding
Embeds domain index into 32 32 spatial dimensions
Concatenate
(input image and domain embedding) 4 x 32 x 32 tensor
Cony, I-ReLU, dropout
64 filters of x 3, stride 2. padding
Cony, I-ReLU, dropout, BN
128 filters of 3 x 3, stride padding
Conv, I-ReLU, dropout, BN
256 filters of 3 x 3, stride 2. padding
Linear, Sigmoid
Fully connected layer, maps to I output
Table 4: Architecture of (!).
Hyperparameter Setting. We use Adam optimizer with an initial learning rate of 0.0001 and parameters 31 = 0.5, 32 = 0.999. We use a batch size of 128. We use & regularization with = 0.15. The models are trained for 500 epochs.
Result. Fig. shows the result of generating samples across the two domains for the same content (i.e., Fig. (b)) Image in the nith row and ith column is generated by combining content e; with style s where n = 2. It can be observed that the proposed method generates the same digit for the same content information, with varying digit styles (color and rotation) in their respective domains. However, when the sparsity regularization is not used, the digit identity not preserved across domains due to the content-leakage issue (see, e.g., the first, third, and fifth columns).
input for the case of À= 0.15 (i.e., Fig. a)) and
Fig. 7 shows the result of image generation for the case of À = 0.15 (i.e., Fig. a b)) and = (i.e., Fig. 7c. d)) Image in the ith row and jth column is generated by combining content and (n) 8(")) for a given domain n. Hence each row contains the same content and each column contains the same style. Fig. a.b)) shows that content and style components are correctly learnt by the proposed method, i.e., changing the content corresponds to changing the digit, whereas changing the style corresponds to changing the color and rotation in colored MNIST and rotated MNIST, respectively. However, when the sparsity regularization is not used, Fig. cd.d shows the issue of content leakage into the extracted style information. Specifically, chaning the style component results in not only the change of color and rotation, but also the digit identity themselves. This validates Theorem 3.4 and the discussions in Sec. 3.2
29

---

Published as a conference paper at ICLR 2025
/
4
9
9
6
5
7
4
1
4
3
6
6
5
6
9
(a) Proposed (A 0.15)
9
4
8
7
7
2
8
6
0
3
6
6
7
9
8
2
9
(b) Without sparsity (A 0)
Figure 6: Result of sample generation across the domains when the content is fixed. Images in each column was generated using the same content (digit identity). The first row is expected to have various colors and the second various rotations. Ideally, the digits in the rows of the same column should be the same.
7
7
7
7
4
4
3
S
8
8
5
6
6
6
6
0
8
8
8
Content (digit idientit))
Content (digit identity)
Content (digie ideetity)
Content (digit identity)
3
3
3 0 5 S
3
7
5
5
6
5
6
6
0
O
6
6
3
5
8
3
5
5
5
5
3
3
3
3
5
0
5
5
<
S
S
5
X
X
4
3
3
6
3
Style (color)
Style (rotation)
Style (color)
Style (rotation)
(a) À = 0.15
(b) X = 0.15
(c) À = 0
(d) x = 0
Figure 7: Result of generation by varying both the content and style latent codes. Content is the digit identity, whereas style is the color and rotation in the colored MNIST and rotated MNIST, respectively. Average FID for the proposed method: 22.08; Average FID without sparsity regularization: 24.08.
H.2 MUL.TIDDOMAIN TRANSLATION.
In this section, we present qualitative results for multi-domain translation tasks by the proposed method and the basclines.
style
Style
(a) Proposed
(b) I-GAN (Gen)
Figure 8: Translation Task: Combining content (pose) with styles of the image in the top row (the cat domain)
30

---

Published as a conference paper at ICLR 2025
Content Image
Style image
Proposed StarGANv2 I-GAN (Tr) SmoothGAN
M Mtto
Lal
F M
Figure 9: Target image-guide translation between different pairs of domains. Content (pose) from the image in the first column combined with the style (appearance) of the image in the second to generate the translated images.
Fig. g (a) and (b) shows guided translation from wild domain (n 3) to cat (n 1). The first column contains two samples sz! (8) [2] in the wild domain, from which the content (( E [2] is (1) extracted using GAN inversion. The first row contains two samples x . [2] in the cat domain, from which the styles 8/") .j [2] are extracted. Finally, the translated image are synthesized using One can that the translated images appear consistent with their respective input contents (i.e., pose of wild) and styles (i.e., type of dog) for the proposed method, whereas I-GAN (Gen) seems ignore the style information.
q (3) $)
Fig. shows the result of target domain emage-guided translation between Male and Female of the CelebA-HQ datasets. Here, the first and second columns show the images in the source domain
(+))) and target domain (1( respectively. We, first, extract the content ((5) from m)(*) and style from st)) using GAN inversion. Then, the translated image (=*++)) is generated using (c())) s(f)). Fig. shows that the proposed method generates translations that are more consistent with the style (appearance) of the target domain images compared to the baselines (e.g., see first row).
Fig. 10 shows the result of randomly sampled translations between different pairs of domains in the AFHQ and CelebA-HQ datasets. Here, the first column shows the images in the source domain x(*),
We extract the content (( from x using GAN inversion. Then, style vector s(t) is randomly
sampled in the target domain t to generate the translated image x (8->)) (2( (f)).. One can see that the translation quality is competitive with the baselines. Moreover, in some cases, the translations produced by the proposed method appear more natural compared the baselines (e.g., see ears of wild animals in the "dog to wild" translation task).
H.3 MULTI-DOMAIN GENERATION.
In this section, we present additional qualitative results for multi-domain data generation by the proposed method and the baselines.
Fig. shows the generated images by varying content and style components various domains for the proposed method and the baseline. Image in the ith row and jth column is generated by
combining content e; and (n) for given domain n. Hence each row contains the same content and each column contains the same style. One can see that the proposed method has successfully learnt to
disentangle content (pose of the object) and style (appearance of the object) for all domains. Whereas,
for the baseline I-GAN, it seems that the style component is not properly learnt.
3 for the jth column associated with domain nj. we generate three different styles (mg) i E [3] Then, the image an in the ith row and jth column generated by qī, s (("))), using fixed content с. One can see that the proposed method generates content (pose) aligned samples in different domains with varying
Fig. 12 shows the qualitative results for the CelebA dataset. As in Fig.
31

---

Published as a conference paper at ICLR 2025
source
Proposed
StarGANv2
GAN (T)
SmOothGAN
cat dog
dog
dog to wild
cat wild wild to dog
(a) AFHQ dataset
source
Propused
StarGANv2
I-GAN (Tr)
SmmooothAAN
F t F
(b) CelebA-HO dataset. Male, Female
Figure 10: Randomly sampled style based translation between different pairs of domains, Content
(pose) from the image in the first column is combined with a randomly sampled style in the target domain generate the translated images.
32

---

Published as a conference paper at ICLR 2025
C
C
G
G
(a) I-GAN (Male)
(b) Proposed (Male)
6
G
G
G
(c) I I-GAN (Female)
(d) Proposed (Female)
G
G
G
(e) I-GAN (Dog)
(f) Proposed (Dog)
&
g I-GAN (Wild)
(h) Proposed (Wild)
Figure 11: Different contents c; combined with different styles (n) in different domains
8)²)

---

Published as a conference paper at ICLR 2025
Black Hair
Blond Hair
Brown hair
Female
Male
Old
Young
50
55
I-GAN
Black Hair
Blond Hair
Brown hair
Female
Male
Old
Young
SP
(b) Proposed
Figure 12: Qualitative results for the CelebA dataset. All images are generated using the same content è but different styles s(m)
[3],n n E [7]. i.e., image Fij in the (i,j)-th location in the grid is
xi = q( ( Vi
[3] x [7].
style. However, the baseline I-GAN seems generate the same image for given domain, in other experiments.
34

---

Published as a conference paper at ICLR 2025
H.4 OTHER SPARSITY PROMOTING REGULARIZATION
As mentioned in Section 6 any sparsity promoting regularization other than & can also be used in order avoid the content leakage problem when the latent dimensions are unknown. In this section, we show that using fp-no p-norm with < can be similarly effective. To that end we use X||es We set À = 0.3 and fix all other setting to be the same as described in Sec.
(m) (r' I|3-
Fig. 13 shows the result of image generation in the three domains (i.e., cat, dog, and wild) of the AFHQ dataset by varying content and style latent variables. Similar to the results obtained by using --normm based regularization, the content (pose) and the style (appearance) information is correctly learned.
Fig. 14 shows the result of image generation across different domains for the same content information.
Image in the nth row and ith column generated using content e and style s (n) One can observe that samples across different domains content-aligned.
Chirkry coteett
Chariig contet
Cat domain
(b) Dog domain
Wild domain
Figure 13: Generation result for fj -norm based sparsity regularization AFHQ dataset. FID: 6.61
Source
(content)
Transiation e1
Translation #2
Changing
Changing style
(a) fixed content across different domains
(b) Translation from wild to cat
Figure 14: (a) Generated images across different domains for the same content information in the AFHQ dataset. (b) Image translation from wild domain domain of the AFHQ dataset
H.5 CONDITIONAL IMAGE GENERATION.
Fig. 15 and 16 shows random samples from each domain. That is, each image is generated using randomly sampled content and styles for the proposed method and I-GAN. Whereas, for StyleGAN-AND ADA and Trans i tional CCGAN, each image in given domain is generated using
35

---

Published as a conference paper at ICLR 2025
randomly sampled latent vector. One see that the compared the class conditional genera- tive models StyleGAN- and Transit: Transitional-cGaal the proposed multi multi-domain generative model does not incur any loss in the visual quality. However, due to disentangled latent representa- tions, the proposed method enables content/style controlled generation and domain translation.
dog
wild
cal
dog
wild
(a) StyleGAle StyleGAN-ADA
(b) Transiticnal-on
dog
wild
(c) I-GAN
(d) Proposed
Figure 15: Class conditional image generation by all methods for AFHQ dataset.
H.6 UNDERESTIMATED LATENT DIMENSION
In this section, we provide some discussions on the implications of underestimated latent dimensions de and dg. Generally, when de de or ds < ds. solving Problem 44) cannot identify the content and the style parts. In addition, a feasible solution to Problem (4) may not even exist. To clarify, we provide some intuitive explanation for the following three cases:
1. do + ds < do + ds In this case, differentiable injective map f does not exist from de + dimennsoon manifold, X, to & Considering our implementation in Problem (6). the generated data may not be able to match the data distribution Pa(e) well since the
generated data a (ec(rc), (n) (rg") (n)) will have to live on a lower dimensional manifold than X.
2. de < de but de + ds 2 de + dg The content dimension will be insufficient to capture all the content information. As such the style part might be mixture of content and style.
36

---

Published as a conference paper at ICLR 2025
male
female
male
female
male
female
male
female
(a) Styl eGAN-ADA (b) Transitional-Cee
(c) 1-GAN
(d) Proposed
Figure 16: Class conditional image generation for CelebA-HQ dataset.
3. ds ds but de d 2 de + d The style dimension will be insufficient capture all the style information. But Assumption still ensures that the style does not leak into the extracted content part. Hence, the style variability may be captured by the generated images.
Fig. 17 shows generated samples by varying the content and style dimensions for the cat domain of the AFHQ dataset. We show results corresponding each of the above cases. Empirical tests showed that de > and dg > 8 are reasonable settings for the AFHQ dataset, which we will treat the "ground truth". For each row, fix the content part ec(rc) (i.e., pose of the cat) and (1) ~ (O, I to generate the images (11) q(c, ())). We set de = 8 and/or dz = 8 according to the specific case that we tested.
randomly sample different styles (1)) = eg (rl)) where r's
Fig. a.) corresponds to Case 1. One can that although content-style disentanglement appears satisfactory, the FID attained slightly worse than the other two cases, which could be due to insufficient latent dimension.
Fig. b) corresponds to Case 2. One can see that changing the style component seems to change both pose and appearance, which means that extracted style is mixture of the content and style.
Fig. c7(c) corresponds to Case 3. Here, changing the style component has very little effect on the appearance of the cat which corroborates intuition that the style information is not fully captured.
37

---

Published as a conference paper at ICLR 2025
Chanping content
Changing contert
Changing style
Changing style
(a) de 8, ds
8; FID: 6.33
(b) dc 8, dg 504; FID: 6.01
Canpaing content
nainging style
(c) de = 504, ds
8; FID: 6.08
Figure 17: Result of using small latent dimensions for AFHQ dataset.
38


Reviews:
[{"review_id": "wB441I5oo6", "reviewer": "Reviewer_wgDt", "summary": "This paper enhances the identifiability of latent content and style representations in unaligned multi-domain data by framing the problem within a distribution matching framework. The authors establish that, under relaxed conditions, their approach does not require restrictive assumptions like component-wise independence or known latent dimensions. By applying sparsity constraints, they ensure identifiable content-style separation, and they ultimately reformulate the distribution matching objective into a simpler GAN training framework. The authors empirically demonstrate improvement in image translation.", "questions": "- Do you believe that L1​ regularization provides a strong enough sparsity constraint to justify the Domain Variability assumption? Could you elaborate on why you chose L1 ​over potentially stricter sparsity-inducing methods, particularly given its central role in ensuring the injective property of the mapping function.\n- The model relies on some broad assumptions. Could you discuss the practical conditions that need to be met to ensure invertibility, and how sensitive the method might be to deviations from these conditions?\n- Have you observed scenarios where the L1 constraint may not be strong enough to prevent information leakage between content and style representations?\n- It appears from the results that the content representation often corresponds to pose, while the style representation captures semantic attributes. Is this always the case, or have you observed instances where content and style encode different aspects of the data? For example, in Figure 3(b), it seems that changing the domain also changes hair color.\n- It would be helpful to include experiments on simpler datasets, such as rotated MNIST, to provide a clearer demonstration of the model’s ability to separate content and style. For example, with rotated MNIST, the content could represent class information, while rotation and other stylistic aspects could be designated as style. Such examples could offer readers a more intuitive understanding of how content and style representations are disentangled and controlled within the model.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-structured and clearly written.\n- It introduces a unique and interesting distribution matching perspective for content and style transfer in image translation\n- By relaxing restrictive assumptions often required in previous work—such as the need for component-wise independence and pre-defined latent dimensions—the approach enhances practicality and scalability.\n- The paper provides both a theoretical foundation for content-style identifiability and promising qualitative results.", "weaknesses": "- The paper’s dimension-agnostic approach relies heavily on sparsity constraints to prevent information leakage between content and style representations, which may not be effective in all cases.\n- The paper relies on several broad assumptions, such as the invertibility of the function up to certain ambiguities, without fully detailing the specific conditions under which this assumption holds.\n- The L1 regularization appears to me to be too \"soft\" of a sparsity constraint to adequately justify the injective property of the mapping function $f$, particularly given that the model relies heavily on this constraint to ensure generative model is a valid invertible function.\n- The paper, in general, lacks a thorough ablation study to validate the author's claims.\n\n$\\textbf{Minor Remarks}$\n- Line 741, I think $\\mathcal{A}\\in\\mathcal{Y}$ rather than $\\mathcal{A}\\in\\mathcal{X}$.\n- This is a minor suggestion, but it may be worth considering alternative abbreviations for \"DM\" and \"LDM\" due to the common use of these terms in diffusion models and latent diffusion models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yMKsJmhINY", "reviewer": "Reviewer_r56a", "summary": "The authors try to attach the problem of identifying latent content and style variables from observations from un-aligned domains. The authors claim their main contribution as follows: (1) t restrictive assumptions such as component-wise independence of the latent variables can be removed; (2)  knowledge of the content and style dimensions is not necessary for ensuring identifiability. However, there are some inconsistencies between the main results and Theorem 4.1.", "questions": "1. In Theorem 4.1,  $q$ is a bijective and $f=q^{-1}$, thus trivially $f$ becomes an bijective, which somehow is inconsistent with the fact    that $f$ is a injective from $\\mathcal{X}$ to $\\mathbb{R}^{\\hat{d}_{c}+\\hat{d}_s}$ , \n   can the authors clarification? \n\n   Particularly on the relation between $\\mathcal{C}$, $\\mathcal{S}$ and $\\mathbb{R}^{\\hat{d}_{c}+\\hat{d}_s}$ as they are actually different spaces. \n2. In Assumption 3.1, $N$ is the dimension of style variables, but somehow in later part of the paper it has different meaning, can the authors clarify? Similarly, the notation $n$ also has different meanings in different part of the paper.\n3. Can you provide the correct citation for Proposition E.1?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The writing of the paper is good. Most parts of the paper are easy to follow and understand. \n2. The experimental results are good.", "weaknesses": "There may be some inconsistency between  the main results and Theorem 4.1. In the main results, it claims that $f$, $\\gamma$, and $\\delta$ are injective. However, in theorem, the $q$ is a bijective and $f=q^-1$, which implies that $f$ will be an bijective. As a result, $\\gamma$ and $\\delta$ becomes bijective and it implies the dimension of content and style is actually $\\hat{d}_c$ and $\\hat{d}_s$. This causes a contradiction in the theory.\n\nThe proof of Theorem 4.1 uses  Proposition E.1 from  (Zimmermann et al., 2021) https://arxiv.org/pdf/2102.08850. However, I did not find the Proposition in the reference. Can the authors fix the problem?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dWe7WHQhRC", "reviewer": "Reviewer_teKB", "summary": "This paper tackles the problem of identifiability of latent content and style variables from unaligned multi-domain data. A new approach based on cross-domain latent distribution matching (LDM) is presented, which removes restrictive assumptions from prior work regarding component-wise independence of latent variables and knowledge of content and style dimensions. The LDM formulation is implemented as a regularized multi-domain GAN loss with coupled latent variables. In addition to theoretical contributions on these topics, the authors present experimental results on image translation and generation tasks.", "questions": "Can the authors please add some remarks regarding the potential weaknesses mentioned above, regarding the suitability of their approach for text, and regarding the potential instability of adversarial GAN training used in their approach?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper provides a good overview of background and related work on multi-domain content-style modeling, content-style identifiability, and the challenge of knowing the content and style dimensions apriori. \n- The contributions in this paper appear to be novel and technically sound from both a theoretical and experimental standpoint.\n- The proposed approach in this paper appears to be reproducible, given the provided source code and experimental details in the paper.\n- The experimental results show that the proposed approach performs well on multi-domain image translation and generation tasks.", "weaknesses": "- Because the proposed approach is implemented as a GAN, it seems likely that this approach will be most suitable for images (and possibly audio), and may be problematic for discrete data modalities, particularly text. \n- The authors don’t directly mention or address potential instabilities that may result from adversarial training of their GAN-based approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6R0zWeoJcQ", "reviewer": "Reviewer_dub2", "summary": "Summary:\nThis paper studies causal identification of latent factors content and style in the cross-domain learning problems. Specifically, existing works assume that all latent components are independent to each other. Such an assumption is strict and could not be fulfilled in realistic scenarios. In this paper, the authors propose a cross-domain latent distribution matching framework which does not require strict independence between different latents. Moreover, the proposed framework can ensure content style identification can be free from the specific latent dimensions which further ease the assumption for tackling the problem. Through both theoretical and empirical justifications, the authors carefully testified the effectiveness of the proposed method.", "questions": "Please see the weaknesses part.", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths:\n- This paper is well-written, well-motivated, and easy-to-follow. Both the experiments and theoretical justification are intuitive and convincing.\n- The contribution of easing independent assumption is helpful for developing simple and efficient causal models.\n- A sparsity-regularized implementation is proposed to solve cross-modal learning efficiently.", "weaknesses": "Weaknesses:\n- The proposed target in Equation 4 aims to minimize the norm of style latents by enforcing distribution matching across content and block-independence between content and style. However, it could lead to a degenerate solution when there is misidentification between content and style, and some content features already achieves small norm. Could you explain how can the proposed identifiability result can avoid the problem from happening?\n- The computational efficiency of the framework is unknown. Only numerical result of generation is provided, there is no quantitative comparison to show the efficiency compared to other baseline methods. Moreover, why sparsity regularization is needed for training GAN needs a justification.\n- Missing related references:  \nHong et al., Improving Non-Transferable Representation Learning by Harnessing Content and Style, in ICLR 2024.  \nHuang et al., Harnessing Out-Of-Distribution Examples via Augmenting Content and Style, in ICLR 2023.  \nDai et al., Disentangling writer and character styles for handwriting generation, in CVPR 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GmFKIHO2RR", "reviewer": "Reviewer_EZhh", "summary": "This paper introduces an analytical framework via cross-domain latent distribution matching(LDM), which establishes content-style identifiability under substantially more relaxed conditions.", "questions": "1. Can you further explain the reasonability using Assumption 3.1? Can you relax it? If not, what are the difficulties of relaxing this assumption? Furthermore, in what real-world scenarios might this assumption be violated?\n2. Can you explain the gap between your method in 4(a) and the way you do implementation? What is the motivation for not using (4) directly? Any trade-offs or limitations of their implementation approach compared to directly using formulation (4).", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well written and the contribution is clear.", "weaknesses": "The author seems to slip the main result part (3.2) too quickly (with no more than half a page). I am unsure what the key difference between your approach and other low-rank techniques is. In addition, there seems to exist a gap between the optimization problem (4a)(4b)(4c) and the implementation. (I mean: why not to use the formulation in implementation section as the theoretical modeling approach in the methodology part?) There are some writing issues in the paper too, e.g. you can not directly quote (4) , you should use 'according to formulation (4)'.(e.g. the writing in Theorem 4.1)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)