Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
Authors: Zinan Lin
Enshu Liu
Xuefei Ning
Junyi Zhu
Wenyu Wang
Sergey Yekhanin
Venue: NeurIPS 2025 poster ()
Keywords: generative modeling
representation learning
classification
contrastive learning
rectified flow
diffusion models
latent zoning network
Primary area: general_machine_learning
TL;DR: A Unified Principle for Generative Modeling, Representation Learning, and Classification

Abstract: Generative modeling, representation learning, and classification are three core problems in machine learning (ML), yet their state-of-the-art (SoTA) solutions remain largely disjoint. In this paper, we ask: Can a unified principle address all three? Such unification could simplify ML pipelines and foster greater synergy across tasks. We introduce Latent Zoning Network (LZN) as a step toward this goal. At its core, LZN creates a shared Gaussian latent space that encodes information across all tasks. Each data type (e.g., images, text, labels) is equipped with an encoder that maps samples to disjoint latent zones, and a decoder that maps latents back to data. ML tasks are expressed as compositions of these encoders and decoders: for example, label-conditional image generation uses a label encoder and image decoder; image embedding uses an image encoder; classification uses an image encoder and label decoder. We demonstrate the promise of LZN in three increasingly complex scenarios: (1) LZN can enhance existing models (image generation): When combined with the SoTA Rectified Flow model, LZN improves FID on CIFAR10 from 2.76 to 2.59—without modifying the training objective. (2) LZN can solve tasks independently (representation learning): LZN can implement unsupervised representation learning without auxiliary loss functions, outperforming the seminal MoCo and SimCLR methods by 9.3% and 0.2%, respectively, on downstream linear classification on ImageNet. (3) LZN can solve multiple tasks simultaneously (joint generation and classification): With image and label encoders/decoders, LZN performs both tasks jointly by design, improving FID and achieving SoTA classification accuracy on CIFAR10. The code and trained models are available at https://github.com/microsoft/latent-zoning-networks. The project website is at https://zinanlin.me/blogs/latent_zoning_networks.html.

Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification
Zinan Lin' Microsoft Research Redmond, WA, USA
Enshu Liu Tsinghua University Beijing, China
Xuefei Ning Tsinghua University Beijing, China
Junyi Zhu KU Leuven Leuven, Belgium
Wenyu Wang Redmond, WA. USA
Sergey Yekhanin
Microsoft Research Redmond, WA, USA
Abstract
Generative modeling, representation learning, and classification are three core problems in machine learning (ML), yet their state-of-the-art (SoTA) solutions remain largely disjoint. this paper, we ask: Can a unified principle address all three? Such unification could simplify ML pipelines and foster greater synergy across tasks. We introduce Latent Zoning Network (LZN) as step toward this goal. At its information across all tasks. Each data type (e.g.. images, text, labels) is equipped with encoder that maps samples to disjoint latent zones, and decoder that maps latents back to data. MI. tasks are expressed as compositions of these encoders and decoders: for example, label-conditional image generation uses label encoder and image decoder; image embedding uses an image encoder; classification uses an image encoder and label decoder. We demonstrate the promise of LZN three increasingly complex scenarios: (1) LZN can enhance existing models (image generation): When combined with the SoTA Rectified Flow model, LZN improves FID on CIFAR10 from 2.76 to 2
LZN creates shared Gaussian latent space that encodes
without modifying the training objective. (2) LZN can solve tasks independently (representation learning): LZN can implement unsupervised representation learning without auxiliary loss functions, outperforming the seminal MoCo and SimCLR methods by 9.3% and 0.2%, respectively, on downstream linear classification on ImageNet. (3) LZN can solve multiple tasks simultaneously (joint generation and classification): With image and label encoders/decodders, LZN performs both tasks jointly by design, improving FID and achieving SoTA classification accuracy on CIFAR10. The code and trained models available at https:// //g //g thub. cww/micccccccccccco crosoft/ latent- patent-zoning--e zoning-networks networks. The project website at https:: :tap:://zinne in me// networks html.
blogs/latent_ coaitttooonnnnnootonoess
1 Introduction
Generative modeling, representation learning, and classification are three of the most widely used machine learning (ML) tasks. Generative models like DALLE (71, 70, 5] and GPT [68, 69, 1] power applications such as question answering and content creation. Representation learning, exemplified by CLIP [67], supports tasks like information retrieval. Classification central to tasks such as object recognition [17] and sentiment analysis [19, 54].
Notably, the state-of-the-art (SoTA) techniques for these tasks differ. For example, SoTA generative modeling relies on diffusion models [32, 76, 78] and auto-regressive transformers [68, 69, 7, 1]; SoTA
"Correspondence to: Zinan Lin (zinanl inêmi crosoft.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Label
Text
Label
Text
Text
Examples of tasks:
decoder
encoder
Unconditional image generation
Label encoder
Text decoder
Image embedding
conditioned image generation
Image classification
Shared latent space
Text embedding
Image-to text generation
Image
Image
(e.g. image captioning)
encoder
decoder
poxt-to-imant generation
Text-to Text-to-text generation (e.g. chatbot)
Image
Image
Figure I: Latent Zoning Network (LZN) connects multiple encoders and decoders through shared latent space, enabling a wide range of ML tasks via different encoder-decoder combinations or standalone encoders/decoders. The figure illustrates eight example tasks, but more could be supported. Only tasks 1-4 are evaluated in this paper, while the rest are for illustration.
The latent zone label
The latent zone of image
Latent computation
"Dog" "Bird"
"Dog" "Bird"
Samples
"Cat"
Encoder
Encoder 2
Anchor points
Follows Gaussian
Flow matching
Latent alignment
matching
distribution
Latent space
Latent zones
Figure 2: The latent space of LZN has two key properties: (1) Generative: It follows simple Gaussian prior, allowing easy sam- pling for generation tasks. (2) Unified: It serves as shared representation across all data types (e.g., image, text, label). Each data type induces a distinct partitioning of the latent space into latent zones, where each zone corresponds to a specific sample (e.g., individual image label). The latent space shown as a closed circle for illustration, but it is unbounded in practice.
Latent space
Latent space
Figure 3: Training and inference in LZN rely on two atomic operations: (1) Latent computation ($ 2.2.1):
Computes latent zones for data type by encoding sam- plcs into anchor points and using flow matching (FM) [53, 51] to partition the latent space. Conversely, any la- tent point can be mapped to sample via the decoder (not shown). (2) Latent alignment ($ 2.2.2): Aligns latent zones across data types by matching their FM processes. This figure also illustrates the approach for LZNI in joint conditional generative modeling and classification ($ 5).
representation learning employs contrastive loss [28, 10, 26]; and SoTA classification uses dedicated models trained with cross-entropy loss and its variants [46]. Although using distinct methods for these tasks has long been established and widely accepted in the community, we revisit this methodology from first principles and question its necessity. Specifically, we ask, out of curiosity:
Can a single principle unify generative modeling, representation learning, and classification?
Part of our motivation stems from Occam's 's Razor [83], which favors simpler solutions when possible. More importantly, while these tasks differ in formulation, they are fundamentally related and can benefit from one another; unified principle could facilitate such synergy.
In this paper, we reflect on the strengths and limitations of existing techniques and propose a new unified framework, Latent Zoning Network (LZN), illustrated in Figs. and At the core of our design is shared latent space that connects a series of encoders and decoders, each corresponding a specific data type (e.g., images, text, labels). Encoders map data into zone the latent space, and the corresponding decoder maps it back to the data. Different tasks can be interpreted as
"While auto auto-regreen regressive (AR) transformers with large-scale pre- pre-training provide one approach unify these tasks [68, 69, 7, 1J. SoTA transformer-base representation learning still relies on contrastive learning [45, 89]. More importantly, our approach can be viewed as an orthogonal layer on top of transformers and should be seen complementary rather than competing, See $ 2.4 for further discussion.

---

performing "translations" within the latent space either using encoder-decoder pairs or leveraging a single encoder or decoder. Compared to popular representation learning approaches, which place no constraint on the latent distribution [10], LZN's latent space is generative: it follows simple prior distribution for easy sampling. In contrast to modern generative modeling approaches, where different conditions (e.g., class labels, text) are treated as separate inputs [87], LZN maintains a single latent space that unifies different types of conditional information. Finally, unlike standard classification, where class labels are model outputs, LZN treats "class labels" as data type connected through the latent space like any other data type.
seice-either
To train and perform inference with LZN, we rely on two atomic operations (Fig. 3 and $ 2): (1) Latent computation: Given an encoder, we compute the latent zones for batch of samples. Specifically, we first use the encoder to compute each sample's anchor point, then apply flow matching [53, 51] to map these points to their corresponding latent zones. This procedure ensures that the resulting zones collectively follow a simple Gaussian distribution, facilitating generation tasks, while also guaranteeing that zones from different samples remain disjoint-allowing them to serve as unique latents for classification and representation learning. (2) Latent alignment: Aligning the latent zones produced by two different encoders to facilitate the tasks that require translations between encoders and decoders from different data types. This is fundamentally challenging task due to its discrete nature. To address it, we introduce a novel "soft" approximation that performs the alignment midway through the flow matching process, enabling effective and tractable training.
We demonstrate that, despite its simplicity and reliance on just two atomic operations, LZN is capable of supporting a wide range of seemingly diverse tasks. To illustrate its versatility and practical utility, we present three levels of applications:
LI: Enhancing one existing task ($ 3). Because LZN latents can be computed without su- pervision, they can be seamlessly integrated into existing models as an additional conditioning signal-without requiring any changes to the training loss or methods. In this setup, LZN latents hopefully can learn to improve the task performance. To demonstrate this, we incorporate LZN into rectified flow models [53]-a state-of of-the-art generative approach for images- and observe improved sample quality across CIFAR10, AFHQ- Cat, CelebA HQ, BSUNNBedroom datasets. Specifically, on CIFAR10, LZN closes the FID gap between conditional and unconditional generation by 59%.
L2: Solving one task independently ($ 4). LZN can also tackle tasks entirely on its own, without relying on existing methods. As a case study, we use LZN to implement unsupervised representation learning, a task traditionally addressed with contrastive loss. We find that LZN can even outperform the seminal methods such as MoCo [28] and SimCLR [10] by 9.3% and 0.2%, respectively, on downstream ImageNet linear classification.
L3: Solving multiple tasks simultaneously ($ 5). Pushing further, LZN is capable of handling multiple tasks at once. In particular, we employ two encoder-decoder pairs-ore one for images and one for labels-enablic within a single, unified framework. Built on rectified flow, this implementation outperforms the baseline conditional rectified flow model in generation quality, while also achieving state-of-the ofe art classification accuracy. Notably, the performance on both generation and classification exceeds that of training each task in isolation. This supports our core motivating intuition: seemingly distinct tasks can benefit from shared representations, and LZN provides a principled framework for enabling such synergy.
LZN to jointly support class-conditional generation and classification
While the early results are promising, many challenges, open questions, and exciting opportunities remain unexplored. In principle, as more encoder-decoder pairs are added to the shared latent space, the range of applications LZN can support should grow at least quadratically (Fig. 1). Whether LZN can scale gracefully and realize this potential remains to be seen. We hope this work opens new line of research toward this ambitious vision. See more discussions in $ 6.
2 Latent Zoning Network (LZN)
2.1 Overall Framework
Revisiting existing approaches. To motivate our design on a unified framework for diverse ML tasks, first analyze the strengths and limitations of existing approaches.
Generative modeling. Given samples x ~ p from an unknown distribution p. generative models aim to learn decoder D2 such that D2 (z) approximates p, where is random noise drawn
3

---

from a simple distribution." 3 While 2 can carry useful information for representation learning and classification [50], this pipeline has key limitations: (1) The mapping from 2 to x lacks flexibility. For example, in diffusion models, the optimal mapping is fixed once the distributions of and are fixed [78]. To introduce controllability, models often augment D with additional may overlap or conflict (e.g., text vs. label conditions in image generation), and the resulting representation (z, C1,..... ......... cu) becomes fragmented. (2) Inverting D to recover from sample æ is non-trivial for some SoTA generative models [36]. These issues limit the effectiveness of generative models for representation learning, as also observed in prior work [24].
condition inputs C1 c1.-.-e Cg. C1,,c)..... o+) [87]. This is
Unsupervised representation learning.' SoTA representation learning typically uses contrastive loss [10], where an encoder E maps related sample pairs-either from the same modality (c.g.. image augmentations) across modalities (e.g., image-text pairs)-to to similar embeddings, while pushing unrelated samples apart. These embeddings can perform zero-shot classification by comparing pairwise cosine similarities [67] or be adapted for classification using linear head trained with cross-entropy loss [10]. However, contrastive loss leads E to discard important details (e.g., augmentations. modalities), making the representations unsuitable for standalone generation. Moreover, with few exceptions [2], the representations lack distributional constraints, making them hard to sample from for generative tasks unless training an additional generative model [44].
The most common and SoTA classification approach trains a dedicated model with cross-entropy loss to map inputs class labels [19, 54]. Intermediate layer outputs can be used as representations [80]. However, because the objective focuses solely on classification, these representations tend to discard class-irrelevan information, limiting their utility for general- purpose representation or generation. As with contrastive learning, they also lack distributional constraints for generative tasks.
Classification.
While one could combine the above objectives and methods into a single training setup [60, 43), our focus is on designing a clean, unified framework that naturally integrates all these tasks.
Desiderata. We observe that all the above tasks can be framed as learning mappings between data and latent space. The main differences lie in: the mapping direction (e.g.. latent-to-data for generation,
for antreseetttiio/classsfittttooooi simple prior for generative models, none for others), and the amount of information encoded (e.g.. class labels for classification tasks, detailed reconstructions for generative models). To support all these tasks in single framework, we seek: (1) A unified latent space that captures all necessary information of all tasks; (2) A generative latent space that follows a simple distribution; and (3) Easy mappings between data and latent in both directions.
constraints on the latent space (e.g., a
Framework. To address the above desiderata, our key designs are (Fig. 2):
A unified latent space. In existing frameworks, sample like text can play inconsistent roles- appearing as input latent in text-to-imape generation or as output in text generation. This makes hard to define a unified latent space across tasks.
We address this by introducing a hypothetical foundation latent space that represents all possible samples in the world. Each foundation latent is abstract entity that appears through observations in different data types, such as images, text, and even class labels (e.g., "cat" or "dog").
Importantly, different latents can share the same observation (e.g., multiple cat images all labeled "cat" and described "a cat image"). ). As a result, each observed sample defines latent zone- subset of the latent space that produces the same observation in that data type. This provides a unified way to represent and connect all data types within the same latent space.
A generative latent space. We enforce the latent space follow Gaussian distribution, enabling easy unconditional sampling without constraining any data type. Our framework also supports easy conditional sampling from latent zone induced by an observed sample (e.g., a label).
Easy mappings. Given samples of a data type, compute their latent zones via the correspond- ing encoder. Conversely, a latent point can be decoded into a data type using its decoder.
Tasks. This design naturally supports variety of tasks (Fig. 1):
Foor diffusion models [32, 76, 78]. z the initial Gaussian noise in the sampling process. plus intermediate noise if using SDE sampling [79]. For AR transformers, 2 can be seen as the randomness in token sampling. Wee use "latent", "representation", and "embedding" interchangeably in the paper.
4

---

Single-module tasks. A standalone encoder or decoder can perform specific tasks independently. For instance, the image encoder alone produces image embeddings (representations), while the image decoder alone enables unconditional image generation.
Cross-module tasks. Any decoder pair defines a task. For example, label encoder image decoder enables class-conditional image generation, image encoder label decoder does classification, and text encoder + text decoder supports text generation.
We expect that tasks can benefit from each other through this unified framework (validated in 5). Each task contributes its core information to the latent space, making it increasingly expressive and powerful. Conversely, since all tasks interface through the same latent space, improvements in the latent representations can facilitate learning across tasks.
As the latent space partitions into zones, name this framework Latent Zoning Network (LZN).
2.2 Implementation of Atomic Operations
Training and inference in LZN rely on two operations (Fig. 3): latent computation and latent alignment. We will see in $ 3 to 5 that these two operations are sufficient to implement a variety of tasks.
2.2.1 Latent Computation
Desiderata. Latent computation important in both training and inference of LZN, Given samples
X' = [21, Tn } of the same data type (c.g., images, text, labels), the goal is to sample their latents 21..... 2n = = (2) with a random latent computation function C' such that: (1) Prior distribution
is Gaussian: the latent ~ Uniform { ................................ 2n. follows Gaussian distribution 'N' (0,1). and (2)
The latent zones of different samples are disjoint: Supp (z+) n Supp (2z) for *
Approach. To achieve this, we first use a deterministic encoder E, to map each sample to an anchor point a, = Ex (zz). We then apply the seminal flow matching (FM) method [53, 51], which establishes a one-to-one mapping between distributions, transform these anchor points into latent zones. Specifically, we define family of distributions Ti with endpoints vo = -0, I), the desired prior latent distribution, and Ti (8) 1 Σ 5(s a,), the distribution of the anchor points." 5 The intermediate distribution n, is induced by linearly interpolating between samples sp ~ To and $1 Ti via (ss. $1, t) = t)so ts (i.c., Ti is (1 t) no + tTı).. The velocity field in FM [53] can then be computed as
Eiz(aa -) exp exp
[a (a-ta,) )a
(pp(so, $1.() 15(50, 81, =8) (1-1) Ei= exp Ot
2(1 t)
(s, A ESO~NTO,
(a t0, 2(1-1)²
We obtain st by integrating along the FM trajectory: 8, FM, (50; - 8n + + V(S+ r)dr for 80 ~ Tp- It has been shown [53] that the distribution of 8, is TH- Similarly, integrating backward
= IFM, (s1-git) A $1-g + f V(s,, r)dr for $1-9 T1-9 (1 g)ml + (0, also yields Te, where g is a small constant," 6 With slight abuse of notation, we also write 5t = IFM, (a;, e;i t) to represent IFM,, (51-9it) with 81-g (1 g)a; + ge,. With these setups, define the latent computation as
= Cx), IFM, (a,, 0); where e, ~ (O,
Due to the discussed FM properties, we can see that this approach (approximately) satisfies the two desiderata by construction (see App. A.1 for more details).
Implementation. Computing C involves an integral, which we approximate using standard numerical solvers such as Fuler DPM-Solver [55, 56] that operates as in prior work on diffusion and RF models [53, 79]. key property of our approach is that all operations are differentiable, allowing gradients to backpropagate all the way from latent 2 to the encoders Ez during training. More details are deferred to App. A.2.
finite list of time steps 41,... br.
Efficiency optimization. Computing the velocity V requires access to all samples, making the memory and computation cost of C high. To address this, we introduce several optimization
58 denotes the Dirac delta function.
'FM is well-detined only when both No and nı have full support. However, in our case, ni is mixture of Dirac deltas and lacks full support. Therefore, the full-support distribution æ1-9 as the starting point.
5

---

techniques- latent parallelism, custom gradient checkpointing, and minibatch approximation -hat make the training of LZN scalable. See App. A.3 for details.
2.2.2 Latent Alignment
Desiderata. Following $ 2.2.1, latent zones from different data types are computed independently,
which undermines the purpose of a shared latent space. Many applications require these zones to be aligned. We consider two types of alignment: (1) Many-to-one (and one-to-many) alignment: for example, the latent zone of the "cat" label should cover all latent zones of all cat images. (2) One-to-one alignment: for example, in image-text datasets, paired image and text samples should share the same latent zone. Concrete examples will be shown in Š 4 and 5.
Formally, let X' (.1,...,,In and y (U1,.... Dm be two datasets from different data types. The pairing is defined by k; where Yi (e.g., a cat image) is paired with Tı, (e.g., the "cat" label). We aim to ensure Supp (2))
x 2 Supp (C' (D'),) for all E [m]] meaning the latent zone of Ik, covers that of V/- This formulation supports many-to-one alignments directly. For one-to-one alignment, a symmetric constraint can be added with x and y swapped.
Approach. Given the FM integral trajectories, alignment reduces to ensuring that the latent of yi. when mapped via the trajectory, matches the anchor point of Te. FM, (C ());; ; 1)= = F. (#k:).
discrete assignment is non-differentiable Before introducing our solution, we illustrate why the problem is nontrivial by examining strawman approaches. A natural idea is to directly minimize the distance: d (FM, (C)));;; 1), Es (IR,)) where d (-.-) is a distance metric. This approach fails because FM deterministically maps each latent to exactly one anchor point Eg (æz)) so the above objective effectively becomes minimizing the distance between anchor points. However, latent zone is influenced by all anchor points, not just its own. Therefore, reducing the distance between a pair of anchors does not necessarily improve zone-level alignment. More fundamentally, the core challenge is that FM induces a discrete assignment: each latent deterministically maps to one anchor. This discrete operation is non-differentiabl and cannot be directly optimized during training.
Challenge:
Technique 1: Soft approximation of alignment. To address this issue, our key idea is introduce soft approximation
the discrete anchor assignment process. Let us define of = FM, / (C ()),; and a = Eg (zz). By construction, the distribution सe is mixture of Gaussians: Te 1 - Wta,, (1 I)),, where the 1-th component corresponds to anchor a;. We define the (soft) probability that st is assigned to at as being proportional to the density of s; under the 1-th Gaussian component:
P (arls)) exp
exp
1nf-te,12)) 2(1-1)2
This formulation provides smooth, differentiable approximation of the otherwise discrete assignment. When t = O, the approximation is fully smooth, with F (ayls)) = 1/n for all i, I, reflecting a uniform assignment. As f increases toward 1, the assignment becomes sharper. In the limit as - 1, converges to the true discrete assignment, where sf deterministically maps to its assigned anchor.
From this, a straightforward idea maximize the assignment probability over all time steps such as Eng (t........ P (ax, (s)) (recall that 45 are solver time steps; see $ 2.2.1). However, our ultimate goal is only to ensure correct assignment at = 1. Even if this is achieved, the above objective would continue push the intermediate states st toward ak. which is unnecessary and potentially harmful.
Technique 2: Optimizing maximum assignment probability. To avoid this, we propose to maximize
max 4 IP (ux, Isi). This ensures that once the trajectory reaches the correct anchor near 1 = 1, the objective maximized (i.e., equals 1) and no further gradient is applied as desired.
Technique 3: Early step cutoff. However, this approach introduces a new issue: if st diverges from
as, early on, the maximum probability remains at the constant 1/n (attained at t = f1 0), yielding no training signal. To mitigate this, we truncate the set of time steps used the maximization, restricting it to the later stages of the trajectory: {lu,...,4p) J, where u is a hyperparameter that excludes early time steps. Putting all together, our proposed alignment objective is:
Align (X,)) maximize
Σ
max IP (ak, 1E(fu....tr
(2)
Please see App. B for more implementation details.

---

Latent space
Training
Generator
Draw a latent of the target image
condition input to the generator
Draw a latent from the whole latent space (standard Gaussian)
Generation
Generator
DS an extra condition imput to the generator
Figure 4: LZN for unconditional generative modeling ($ 3). During training, the LZN latent of each target image fed as an extra condition to the rectified flow (RF) model [53], making the RF learn conditional flows based on LZN latents. The objective remains the standard RF loss, and the LZN encoder is trained end-to-end within it. During generation, we sample LZN latents from a standard Gaussian and use them the extra condition. We illustrate the approach with RF, but since LZN latents require no supervision and are differentiable, the method could apply to other tasks by adding condition input for LZN latents to the task network.
2.3 Decoder
The decoder Da maps the LZN latent back to its corresponding sample: Da (=,) = Iz. As we will describe in more detail later, it can be implemented using either generative model ($ 3) or FM ($ 5).
2.4
Relationships Alternatives
A prominent alternative that can also unifies different ML tasks is the of AR transformers with large-scale pertaining, such as large language models (LLMs) [68, 69, 7, 1]. These models unify generation tasks by representing all data sequences of tokens and modeling them in AR manner. Classification tasks are cast as generation problems- for instance, prompting the model to complete the sentence "Which animal is in this image?" [69]. Additionally, prior work has shown that the intermediate outputs in these models can serve as strong representation for downstream tasks [72].
As such, this approach is generation-centric. the core formulation remains a generative modeling problem. Tasks that can be framed as generation-such as classification-can be unified naturally. However, for other tasks like representation learning, this approach must rely on surrogate methods, such as using intermediate model outputs. In contrast, LZN offers a new formulation that seamlessly unifies all these tasks within single framework, as we will demonstrate in the following sections.
More importantly, AR transformers (and other generative models) should be seen as orthogonal and complementary to LZN, rather than as competitors. In particular, LZN decoders that map latents to data samples can instantiated using any generative model. We demonstrate this in $ and This allows LZN to leverage the strengths of existing generative models within its unified framework.
2.5 Scope of Experiments
While the framework is general, this paper focuses specifically on the image domain. We present three studies: (1) generation ($ 3), (2) unsupervised representation learning ($ 4), and (3) joint classification and generation ($ 5). These case studies are arranged in order of increasing complexity: the first enhances single task, the second solves task using only LZN without any other external objectives, and the third tackles multiple tasks simultaneously within the same framework.
3 Case Study 1: Unconditional Generative Modeling
Approach (Fig. 4). Since LZN latents C (X²) can be computed using a standalone encoder without introducing new training objectives ($ 2.2.1), they can be easily integrated into existing pipelines as additional network inputs, without modifying the original loss function. We apply this idea to anconditional generative models, where the generator serves the LZN decoder. We modify the decoder by adding an extra input to accept the LZN latents. Both the encoder and decoder are trained jointly using the original generative loss. In this setup, latent alignment ($ 2.2.2) is not used. Importantly, the encoder is only used during training to compute C (X). During inference, latents are sampled from the Gaussian prior and passed directly to the decoder, maintaining the original model's 's inference efficiency. Please see App. C. for detailed pseudocode of the training and generation processes.
Why helps. LZN latents provide unique representations for each image. This pushes the generator (decoder) to depend more directly on the latent for reconstruction, making the image distribution conditioned on the latent more deterministie. As a result, the generative objective becomes easier to optimize. Our experiments later confirm that LZN latents indeed capture useful features for generation.
7

---

Table 1: Unconditional image generation quality scores across four datasets. The best results are in gray box Applying LZN to generative models improves RF on most image quality metries, RF SoTA method; due to space constraints, we omit additional methods see [53] for extensive comparisons between RF and others. Note that Inception Score (IS) is best suited for natural images
like CIFAR1O, though we report it for all datasets for completeness.
Algo.
CIFARIO (32 32
AFHO-Cat (206 x 256)
FIDL
aFID1
1S1
Precisson
Recall
CMMD]
Recon,
FID
sFIDS
1ST
Precision
Recalf
CMMD]
Recon]
RF
2.76
4.05
9.51
0.70
0.59
0.0360
0.83
6.08
49.60
1.80
0.86
0.28
0.5145
17.92
RT+L28
2.59
3.95
9.53
0.70
0.59
0.0355
0.41
5.68
49.32
1.96
0.87
0.30
0.3376
10.29
Algo.
CelebA-BQ
256)
LSUX-Bedroon (256
256)
FIDL
<FID]
1S1
Precision?
Recall"
CMMD]
Recon]
FID
xFID]
151
Precision!
Recall:
CMMD1
Recon]
RF
6.95
10.61
2.91
0.76
0.42
1.0276
26.20
6,25
16.22
2.18
0.60
0.40
0.5218
48.72
RF+L28
7.17
10.33
2.92
0.76
0.45
0.4901
15.90
5.95
17.84
2.22
0.59
0.41
0.4843
37.01
Figure 5: Generated images of RF+LZN on AFtt-Cat, CelebA-HO, LSUN-Bedroom More in App. C.
Related work. RCG [44] and Diffusion Autoencoder (DA) [66] also aim to improve image generation using unsupervised representations. RCG leverages pre-trained representation model, while DA uses an encoder trained jointly with the generator. Similar to LZN, these representations are provided as additional inputs to the generative model. However, because their representation lacks distributional constraints, both methods require training an auxiliary generative model over the representations to enable unconditional generation. In contrast, LZN enforces a Gaussian representation by construction, eliminating this extra step and allowing everything to be trained end-to-end in one stage.
Results. We plug LZN latents into the seminal rectified flow (RF) models [53], which is closely related to diffusion models [32, 76, 78]. RF achieved SoTA image generation performance on several datasets [53, 41], and has been used in some latest Stable Diffusion models [23] and AR models [52].
We follow the experimental setup of RF [53], evaluating on four datasets: CIFAR10, AFHQ-Cat, CelebA-HQ, and standard metrics-FID [31], sFID [58], IS [73], precision [40], recall [40], and CMMD [33]-we also report reconstruction error (the & distance between an image and its reconstruction). which is relevant for applications like image editing via latent manipulation [75, 50J. Results are shown in Tab. 1, with three key findings: (1) LZN latents improve image quality. During inference, the only difference in RF+LZN is the inclusion of an additional latent drawn from a Gaussian distribution. This improvement indicates that the decoder effectively leverages LZN latents for meaningful generative features. (2) LZN significantly reduces reconstruction error across all datasets, further confirming that its latents capture essential image information. (3) While unconditional generation is important, its image quality often trails that of conditional generation [44]. Compared to RF's CIFAR10 results in Tab. 4, find that LZN substantially reduces the FID gap between conditional and unconditional generation by 59 %, and even outperforms conditional RF in sFID and reconstruction error.
The latter three are high-resolution (256 x 256). In addition to
See Fig. 5 for some generated images. Due to space constraints, please refer to App. C for more details on implementation, datasets, metrics, and additional results such as more generated images and ablation studies.
4 Case Study 2: Unsupervised Representation Learning
Related work. Contrastive learning is a popular approach to unsupervised representation learning [10, 11, 28, 12, 14, 26, 13], which pulls similar images (e-g.. augmentations of the same image) together in the representation space. A central challenge is avoiding collapse, where all inputs are mapped to the same representation. Common solutions include pushing the representations of dissimilar samples from large batches [10] or memory banks [28] away, adopting architectural designs that prevent collapse [26, 13, 28]. Other unsupervised representation learning approaches [74, 42] include masked image modeling [43, 4, 27, 63] and training on other auxiliary tasks [84, 59, 65, 25].
Approach (Fig. 6). Inspired by contrastive learning, we train an image encoder using Align (X))) (8 2.2.2), where = {.:)²= and y = k= with mapping k; i contain image pairs (2,; Di) of random augmentations of the same image. Unlike traditional contrastive methods, our approach
inherently avoids collapse: different images are mapped to distinct latent zones by design, eliminating the need for large memory banks specialized architectures. Notably, only single LZN encoder for
8

---

Data augmentation
Latent space
Latent
Latent space
alignment
Figure 6: LZN for unsupervised representation learning ($ 4). During training, each image batch undergoes two sets of data augmentations, and latent zones for each set are computed using the same encoder. We then apply latent alignment ($ 2.2.2) to train the encoder. At inference, we can use the LZN latents, the encoder outputs (i.e., anchor points), or intermediate encoder outputs (App. D.4). The latter two options avoid the costly latent computation process.
Table 2:
Classification accuracy on ImageNet by training a linear classifier on the unsupervised representations. Methods marked with are based on contrastive learn- ing. 7 The horizontal line separates baselines that perform worse or better than our LZN. All methods use the ResNet-50 architecture [29] for fair comparison." 8
Table 3: Classification accuracy CIFAR10. Base- line results are from [39]. "RF+LZN (no gen)" refers "RF+LZN" with the RF loss for generation disabled. The horizontal line separates baselines that perform worse or better than our RF+LZN. Note that these re- sults refer to training purely on the CIFAR10 dataset (without pretraining or external data).
Algorithm
Acct
Algorithm
Top-1 I Acct
Top-5 Acc?
VGG16
92.64%
ResNet18
93.02%
InstDise [85]
54.0 [28]
ResNet50
93.62%
BigBiGAN [2]]
56.6 [28]
ResNet101
93.75%
LocalAgg [9]]
58.8 [28]
RegNetX 200MF
94.24%
MoCo [28]
60.2 [10]
RegNet -000FF
94.29%
PIRL [57]
63.6 [10]
MobileNetV2
94.43%
CPC v2 [30]
63.8 [10]
85.3 [10]
ResNeX(29(32x4
94.73%
CMC² [81]
66.2 [26]
87.0 [26]
Re5NeX(29(2x64d)
94.82%
[imSiam'[13)
68.1 [13]
SimpleDLA
94.89%
SimCLR [10]
69.3 [10]
89.0 [10]
DenseNet121
95.04%
MoCo v2 [12]
71.7 [12]
PreActResNet188
95.11%
SimCLR v2 [11]
71.7 [11]
DPN92
95.16%
BYOL$ [26]
74.3 [26]
91.6 [266
DLA
95.47%
DINO' 19]
75.3 19]
RF+LZN (no gen)
93.59%
L2N
69.5
89.3
RF+12N
94.47%
both X' and y is needed and decoders are not required. See App. D.I for pseudocode of the training process.
Results. We follow the canonical setting [10, 26, 28], where LZN is trained on the unlabelled ImageNet dataset using ResNet-50 [29] to learn representations. A linear classifier is then trained on top of these representations in a supervised manner, and its accuracy is evaluated on the ImageNett test set. The results are shown in Tab. 2. Note that these results are obtained without any pretraining use of external data. We observe that LZN matches outperforms several established methods in the field, including seminal approaches such as MoCo [28] (LZN outperforms by 9.3% ) and SimCLR [10] (LZN outperforms it by 0.2% This is remarkable given that LZN is new framework capable of supporting not only unsupervised representation learning but also other tasks ($ and 5). However, there remains a significant gap to SoTA performance. We emphasize that our current results are not fully optimized: (1) Training iteration. The performance of LZN continues to improve rapidly with ongoing training (App. D), so we expect the gap SoTA to narrow with full training. (2) Architecture. Prior work shows that more advanced architectures like ViT can improve the results significantly [14]. We leave these directions for future work.
Due to space constraints, please refer to App. D for more details on implementation, datasets, metrics, and additional results such representation visualization and ablation studies.
"Note that we use the term contrastive learning broadly to refer not only to methods employing the traditional contrastive loss, but to all approaches that encourage relevant images to share similar representations; see $ 4.
"It is known that better architectures [1]] or training on larger datasets [62] can yield stronger results. To ensure a fair comparison, we include only methods reporting results with the ResNet-50 50 architecture trained on the ImageNet dataset. This excludes potentially stronger methods lacking ResNet-50 results on ImageNett such DINOv2 [62]. See Tab, 6 for additional baselines using other architectures.

---

Table 4: Conditional image generation quality on CIFAR10.
The best results are in gray box Applying LZN to generative models improves or matches RF on all metrics.
Algo.
FID.
.FIDL
187
Precisiont
Recallf
CMMD)
Recon]
RF
2.47
4.05
9.77
0.71
0.58
0.0253
0.69
RF+LZN
2.40
3.99
9.88
0.71
0.58
0.0229
0.38
5 Case Study 3: Conditional Generative Modeling and Classification
Approach (Fig, 3). Building on $ 3, we consider X' = {=; and y {yr ) where each æ; class label and y/i is an image labeled as Fı,- In addition to the image encoder and decoder, we introduce a label encoder-decoder pair. Since labels come from finite set, both modules share a matrix A E RRxe of label anchor points, where q the latent dimension and is the number of classes. The encoder maps a one-hot label h to its anchor via Ah. The decoder recovers the class ID of a latent g by first applying FM to obtain its anchor A (g; 1), and then computing its corresponding class latent zone arg max A (g: 1)" A, where FMA denotes FM over anchors in A. The training objective extends that of $ by adding Align (X, 3"). After training, the model can perform both conditional and unconditional generation, as well as classification by design. See App. E.1 for detailed pseudocode of the training, generation, and classification processes.
Related work. Joint classification and conditional generation are often achieved by augmenting generative models with classification losses or networks [60, 73], treating label inputs to the generator and outputs from the classifier as separate components. In contrast, LZN unifies label inputs and outputs within shared latent space.
Results. Following the setting in $ 3, we conduct experiments on CIFAR10. Image quality metrics shown in Tab, 4, and classification accuracies are shown in Tab. 3. Key observations are: (1) LZN improves both image quality and reconstruction error. Similar to $ 3, this confirms that LZN latents capture useful features for generation. (2) LZN achieves classification accuracy par with SoTA. Tab. 3 includes SoTA classification accuracy from networks trained solely for classification. The fact that LZN, which jointly performs generation and classification and differs significantly from standard classification pipelines, can match SoTA performance is notable. Currently, LZN lags behind the best CIFAR10 result by 1%, potentially due to architectural factors: we use the RF encoder (App. E) without classification-specifi optimization. With a better architecture design (as in other methods in Tab. 3), LZN could likely improve further. (3) Joint training on generation and classification improves both. This is evident from: (i) RF+LZN in Tab. 4 showing better generation quality than in Tab. 1; and (ii) "RF+LZN" achieving higher classification accuracy than "RF+LZN (no gen)" in Tab. 3. These results support our motivation from $ that different ML tasks can benefit from each other, and demonstrate that LZN is promising unified framework for achieving this synergy.
Due to space constraints, please refer to App. E for more details on implementation, datasets, metrics, and additional results such generated images and ablation studies.
Limitations and Future Work
(1) Training efficiency. Training LZN requires backpropagating through the FM trajectory ($ 2.2. which is computationally expensive. In App. A, we describe several optimization strategies we implemented mitigate this cost. To further improve efficiency, we observe an interesting parallel between training LZN and training large language models (LLMs) (see App. G), suggesting that some efficient training techniques developed for L.MMs may be applicable here. (2) Pure generative modeling. While LZN is fundamentally capable of generative modeling without any auxiliary losses (see App. G), in 3, we only demonstrate how it can enhance existing generative models. Exploring how to fully leverage LZN for standalone generative modeling remains an open direction for future work. (3) Improving performance. Although LZN achieves competitive results in unsupervised representation learning ($ 4) and classification ($ 5), there remains gap to the SoTA. Bridging this gap is an interesting direction. One promising avenue is to incorporate well-established improvements from the literature that we have not yet adopted. such as more advanced architectural designs, as discussed in $ and 5. (4) Multi-modality and multi-tasks. In this paper, we focus primarily on image-based applications and at most two tasks simultaneously ($ 5). However, LZN is designed to be extensible: by incorporating additional encoders and decoders, can naturally support more modalities and perform multiple tasks concurrently (Fig. 1). We leave this exploration to future work.
10

---

Acknowledgement
The authors would like to thank Sangyun Lee for suggesting related work and for helpful discussions, as well as the anonymous reviewers for their valuable feedback. Xuefei Ning acknowledges the support by the National Natural Science Foundation China (No. 62506197). In addition, the authors gratefully acknowledge Cat Cookie and Cat Doudou for graciously lending their adorable faces for Figs. 2 to 4 and 6.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, 1lge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Devansh Arpit, Aadyot Bhatnagar, Huan Wang, and Caiming Xiong. Momentum contrastive autoencoder: Using contrastive learning for latent space distribution matching in wae. arXiv preprint arXiv:21 10303, 2021.
[3] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint- embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15619-1669,, 2023.
[4] Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Bert pre-training of image trans- formers. arXiv preprint Sedi::2006.0255550 08254, 2021.
[5] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/papers/dallle-3 pdf. 2(3):8, 2023.
[6] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint prrxi::109. 11066, 2018.
[7] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[8] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European conference computer vision (ECCV), pages 132-149, 2018.
[9] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference computer vision, pages 9650-9660, 2021.
[10] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PmLR, 2020.
[11] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey Hinton. Big self-supervised models are strong semi- conisuupevvised learners. Advances in neural information processing systems,
-222555, 2020.
[12] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning, arXiv preprint arXiv:2003.04297 2020.
[13] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750-15758, 2021.
From attps:///ww, http://www...om/aaaaatss///////a
doudou released in [47].

---

[14] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervise vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9640-9649, 2021.
[15] Yunjey Choi, Youngjung Uh Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 8188-8197, 2020.
[16] Quan Dao, Hao Phung. Binh Nguyen, and Anh Tran. Flow matching in latent space. arXiv preprint 23iv:23377008698, 2023.
[17] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 248-255 leee, 2009.
[18] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEFE signal processing magazine, 29(6):141-142 2012.
[19] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguisties: human language technologies, volume I (long and short papers), pages 4171 1771-4186, 2019.
[20] Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision, pages 1422-1430, 2015.
[21] Jeff Donahue and Karen Simonyan. Large scale adversarial representation learning. Advances neural information processing systems, 32, 2019.
[22] Alexey Dosovitskiy, Jost Tobias Springenberg, Martin Riedmiller, and Thomas Brox. Discrimi- native unsupervised feature learning with convolutional neural networks. Advances in neural information processing systems, 27, 2014.
[23] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[24] Michael Fuest, Pingchuan Ma, Ming Gui, Johannes Schusterbauer, Vincent Tao Hu, and Bjorn Ommer. Diffusion models and representation learning: A survey. arXiv preprint arXiv:2407.0077 00783, 2024.
[25] Spyros Gidaris, Praveer Singh, and Nikos Komodakis. Unsupervised representation learning by predicting image rotations. arXiv preprint 1 1803.07728, 2018
[26] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 13:21271-2111 2020.
[27] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on computer vision and putlern recognition, pages 16000-16009, 2022.
[28] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729-9738, 2020.
[29] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
12

---

[30] Olivier Henaff. Data-efficient image recognition with contrastive predictive coding. In Interna- tional conference on machine learning, pages #182-4192, PMLR, 2020.
[31] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler. and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[32] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[33] Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9307-9315, 2024.
[34] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017. 2017.
[35] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4401-4410, 2019.
[36] Tero Karras, Samuli Laine, Mïika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 81 11108119 2020.
[37] Durk Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018
[38] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers features from tiny images. 2009.
[39] kuangliu. Train cifar10 with pytorch. 2025.
[40] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
[41] Sangyun Lee, Zinan Lin, and Giulia Fanti. Improving the training of rectified flows, Advances in Neural Information Processing Systems, 37:63082-63109, 2024.
[42] Mufan Li, Mihai Nica, and Dan Roy. The neural covariance sde: Shaped infinite depth- and-width networks at initialization. Advances in Neural Information Processing Systems, 15:10795-10808, 2022.
[43] Tianhong Li, Huiwen Chang, Shlok Mishra, Han Zhang, Dina Katabi, and Dilip Krishnan. Mage: Masked generative encoder to unify representation learning and image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2142 1442-152, 2023.
[44] Tianhong Li. Dina Katabi, and Kaiming He. Return of unconditional generation: A self- supervised representation generation method. Advances in Neural Information Processing Systems, 37:125441-12544 2024.
[45] Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.cm 03281, 2023.
[46] Tsung-Yi Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. Proceedings of the IEEE international conference on computer vision, pages 2980-2988. 2017.
13

---

[47] Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, Harsha Nori, and Sergey Yekhanin. Dif- ferentially private synthetic data via foundation model apis 1: Images. arXiv preprint arXiv:2305. 15560, 2023.
[48] Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. Pacgan: The power of two samples in generative adversarial networks. Advances in neural information processing systems, 31, 2018.
[49] Zinan Lin, Vyas Sekar, and Giulia Fanti. Why spectral normalization stabilizes gans: Analysis and improvements. Advances in neural information processing systems, 34:9625-9638, 2021.
[50] Zinan Lin, Kiran Thekumparampil, Giulia Fanti, and Sewoong Oh. Infogan-cr and modelcen- trality: Self-supervised model training and selection for disentangling gans. In international conference on machine learning, pages 6127-6139 PMLR, 2020.
[51] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
[52] Enshu Liu, Xuefei Ning, Yu Wang, and Zinan Lin. Distilled decoding 1: One-step sampling of image auto-regressive models with flow matching. arXiv preprint 1711::24/2.17/53, 2024.
[53] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03:: 2000220000000,, 2022.
[54] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: robustly optimized bert pretraining approach. arXiv preprint 1907
2019.
[55] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787 2022.
[56] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.01095, 2022.
[57] Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invarian rep- resentations. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 6707-6717, 2020.
[58] Charlic Nash, Jacob Menick, Sander Dieleman, and Peter Battaglia. Generating images with sparse representations. arXiv preprint arXiv:2103.0388 2021.
[59] Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision, pages 69-84. Springer, 2016.
[60] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary classifier gans. International conference on machine learning, pages 2642-26511 PMLR, 2017.
[61] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03744 2018
[62] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07//93, 2023.
[63] Druv Pai, Ziyang Wu Wu, Sam Buchanan, Yaodong Yu, and Yi Ma. Masked completion via structured diffusion with white-box transformers. International Conference on Learning Representations, 2023.
[64] Gaurav Parmar, Richard Zhang, and Jun-Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1 11410-11420 2022.
14

---

[65] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536-2544, 2016.
[66] Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasor Suwajanakom. Diffusion autoencoders: Toward a meaningful and decodable representation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10619-10629, 2022.
[67] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages $748-8763. PmLR, 2021.
[68] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018.
[69] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 188):9, 2019.
[70] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06//5,, 1(2):3 2022.
[71] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and lyya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning. pages 8821-8831. Pmlr, 2021.
[72] Nils Reimers and Iryna Gurevych. Sentence- bert: Sentence embeddings using siamese bert- networks. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing.
Association for Computational Linguistics, 11 2019.
[73] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
[74] Michael Sander, Pierre Ablin, and Gabriel Peyré. Do residual neural networks discretize neural ordinary differential equations? Advances in Neural Information Processing Systems, 11336500-355333 2022.
[75] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou. Interpreting the latent space of gans for semantic face editing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9243-92285 2020.
[76] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265. pmlr, 2015.
[77] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
[78] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[79] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. baore-based generative modeling through stochastic differential equations. arXiv preprint arXiv: 13456, 2020.
[80] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Re- thinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818 18-2886, 2016.
[81] Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. In Com- puter Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI 16, pages 776-794, Springer, 2020.
15

---

[82] Arash Vahdat and Jan Kautz. Nvae: A deep hierarchical variational autoencoder, Advances in neural information processing systems, 33:19667-19679, 2020.
[83] Geoffrey Webb. Occam's 's Razor, pages 735-735, Springer US Boston, MA, 2010.
[84] Chen Wei, Haoqi Fan, Saining Xie, Chao- Yuan Wu, Alan Yuille, and Christoph Feichten- hofer. Masked feature prediction for self-supervised visual pre-training. In Proceedings the IEEE/CVE conference on computer vision and pattern recognition, pages 14668-14675 2022.
[85] Zhirong Wu, Yuanjun Xiong, Stella X Yu, and Dahua Lin. Unsupervised feature learning via non-parann non-parametric instance discrimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3733-3742, 2018.
[86] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv: /506 03365, 2015.
[87] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-iman image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836-3847, 2023.
[88] Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. In Computer Vision-ECCV 2016: 14th European Conference, Amsterlam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14, pages 649-666. Springer, 2016.
[89] Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang. Pengjun Xie, Fei Huang, et al. mgte: Generalized ong-context text represen- tation and reranking models for multilingual text retrieval. arXiv preprint arXiv:2407.19666 2024.
[90] Lin Zhao, Tianchen Zhao, Zinan Lin, Xuefei Ning, Guohao Dai, Huazhong Yang, and Yu Wang. Flasheval: Towards fast and accurate evaluation of texl-to-image diffusion generative models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 16131, 2024.
[91] Chengxu Zhuang, Alex Lin Zhai, and Daniel Yamins. Local aggregation for unsupervised learning of visual embeddings. In Proceedings of the IEEE/CVE international conference on computer vision, pages 6002-6012, 2019.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims are supported by experimental results in to 5.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are discussed in $
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification: See App. A.1.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The hyperparameters are reported Apps. to The code in https:
/ //github. com/ /inmmiieee waiicroooo/tttettt zoning--iinnnieeeieeeie- nettongs..
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification: The code is
in
attps://ggitma..cc///irorroo
latent- zoning -ooninngn-ewwrss. networks.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
httpg//dddee//eeeubmiii. oosiinccyyy) for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https:
wnips.ccppbblic. pec/pbblic/gue wttggg/ddccccssaa
ssionPolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Please see Apps. C to E and the attached code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our baseline results were taken from prior work, which did not include error bars. We followed the same setting and conducted each experiment only once. Additionally, LZN incurs high computational cost, which limits the feasibility of running multiple trials for each experiment within reasonable timeframe.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
19

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Apps. C to E reports the computation and memory cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that NeurIPS Code of Ethics https: / /ttp///eerrpp.eeeee cc httee/uulccceeeeeeeeeeee wtwpeieeeeettiiieeeeeeeeeeeeeeee
didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the
Answer:
[Yes]
Justification:
We followed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See App.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We will do responsible AI reviews before releasing the models.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: All the code and models of this work is created by
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The code comes with README.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not have such experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not have such experiments.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
22

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
The method development in this work does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https:// /ttps//://.e.p..eeeeeeeee hattp::/eearips..ccoofffee /ttp.//eeeeeeeece//22/////////// for what should or should not be described.
23

---

Appendix Contents
More Details on Latent Computation ($ 2.2.1)
25
A.1 The Desired Properties
25
A.2 More Implementation Details
25
A.3 Efficiency Optimizations
25
More Details on Latent Alignment ($ 2.2.2)
26
B.1 More Implementation Details
26
B.2 Efficiency Optimizations
26
C More Details and Results on Case Study 1
26
C.1 Algorithm Pseudocode
26
C.2 More Implementation Details
27
C,3 More Experimental Settings
27
C.4 More Results
30
More Details and Results on Case Study 2
36
D.1 Algorithm Pseudocode
36
D.2 More Implementation Details
36
D.3 More Experimental Settings
36
D.4 More Results
36
More Details and Results Case Study 3
39
E.1
Algorithm Pseudocode
39
E.2 More Implementation Details
39
E.3 More Experimental Settings
................................
40
E,4 More Results
Extended Discussions on Related Work
45
Extended Discussions on Limitations and Future Work
45
24

---

A More Details on Latent Computation ($ 2.2.1)
A.1 The Desired Properties
In this section, we explain in more detail why the construction in $ 2.2.1 (approximately) satisfies the two desired properties.
Prior distribution is Gaussian. By definition, the distribution of latent
Uniform {=1,..., 3 is induced by (1) drawing a Uniform {al,.... an) and e ~ N (0, I), (2) computing 81-9 (1 g)a + ge, and (3) computing IFM, (8) 0). In the above process, $1-9 ~ #1-g by definition. Due to the property of FM discussed 2.2.1, IFM, (#1-9i ~ mo when 81-g T1-9- Therefore, the latent
xo N (0,
Disjoint latent zones. Each latent point N (0, I) can be uniquely map to one of
through the defined FM. We define the latent zone i-th sample as the set of latents that map to ag: Z, { FM2 (z: 1) ),}. The probability that the latent computed through Eq. (1) falls in the incorrect latent zone can then be defined as P (IFM₂ (a, e; 0) E Zj) for i * j. where the that this probability arbitrarily small by choosing sufficiently small g. This is because that to make the latent fall in the incorrect latent zone, we need to have ||]| on the scale of [1=9)(;=a;} whose probability
probability is over the randomness of N (0, I). We
be made
0 when 9 - O. To make this intuition more precise, we give the closed form of this probability for a toy one-dimensional case below.
Theorem Assume that there are samples Iı, F2, with their anchor points 01 =1, 02 We have
( ).
P P(IM (01, c; Z2) = (IFM, (a2, c; c) Z,)
9
where & is the CDF function of the standard Gaussian distribution.
Proof. With slight abuse of notation, we define FM,, (s; &1, 12) s+ م V(s,, ( 7) as following the FM trajectory from to l2. We generalize the latent zone definition above all time steps as the latents at time step / that map to the anchor point a;: Z = {=: FM, (2; 4, = a:). Due to symmetry, we know that Zi (-o0, 0) and Z = (0, oo). Therefore, we have
P (IFM, (01, c) Z2) (-(1 g) ge 0)
1-9
9
where (-) is the CDF function of Gaussian distribution. Similarly, we can get that (IFM, 6; E Z1) [-1
A.2 More Implementation Details
We find that in the training or inference of some tasks benefit from using a more concentrated latent distribution:
24 C (#11 2n); A IFM (a,, ac;; 0),
where (0, I) and a E [0, 1) is the scaling factor. Similar techniques have been used in prior generative models for improving sample quality [35, 37, 6, 82].
A.3 Efficiency Optimizations
We introduce series of efficiency optimization techniques so that the training of LZN can scale to large models and large batch sizes.
Minibatch approximation (reducing memory and computation cost). By design, latent computa- tion requires using all samples, which is infeasible for large datasets. In practice, approximate this by using only the current minibatch as I,, which significantly reduces memory and computation cost.
Note that this approximation has nuanced implications on the two desired properties discussed in $ 2.2.1.
25

---

Minibatch approximation still preserves the Gaussian prior. LZN ensures that the latent distribution within each minibatch is approximately N (O. As result, the overall latent distribution becomes a mixture of Gaussians with the same parameters, which is still N (0, I). Therefore, the global prior remains valid under the minibatch approximation.
However, minibatch approximation violates the disjoint zone property. This is because the latent zones of each sample now depends on other samples in the same batch, which could change across different batches. Despite this approximation, our experiments show performs well.
In generative modeling ($ and 5). the latent does not need perfect zone disjointness- as long as it provides some information about the input sample, it can help reduce the variance needed to learn by the generative model (rectified flow our case) and improve the generation quality. In representation learning ($ 4), latent alignment occurs within a single batch. Thus, inconsis- tency across batches is irrelevant.
In classification ($ 5), we only need to map samples within a single batch to the latent zones of labels. Thus, inconsistency across batches is irrelevant. That said, larger batch sizes can improve the accuracy of latent zones and thus improve perfor-
mance ($ 5).
Custom gradient checkpointing (reducing memory cost). In PyTorch, forward passes store interme- diate results for use in backpropagation, incurring significant memory cost. Gradient checkpointing 10 reduces memory usage (with the cost of extra computation) by selectively discarding intermediates in the forward pass and recomputing them during the backward pass. This technique is typically applied within neural networks. In our case, discover that the main memory bottleneck lies in latent computation, which has memory complexity O(n²qr) (rr) where n is the number of samples, the latent dimension, and the solver steps. We design custom strategy that skips storing velocity computations and retains only the latent trajectories This reduces memory complexity to O(ngr). which makes the training far more manageable.
Latent parallelism (making training scalable with multi-GPU), For the same reason discussed above, the main computation overhead also lies in latent computation. A natural idea is to parallelize with multi-GPU. We partition the data samples across GPUs, and each GPU computes anchor points for its assigned subset. These anchor points are then broadcast to all GPUs, allowing each to compute latents for its own samples using the complete set of anchors. To ensure that gradients can propagate back correctly through the anchor points to the originating GPUs, we use the undoc- umented Py Torch function torch. distributer mn. con.fucciional wwteetiionnnaal...... aammallllllgaaaa....eeee all_gather, which-unlike the standard torch. distributedd all www.ggaatttattteee gradient flow to the original sources.
B
More Details on Latent Alignment ($ 2.2.2)
B.1 More Implementation Details
Optionally, we can apply a logarithm to the assignment probability to make the loss resemble a standard cross-entropy formulation. In that case, our proposed alignment objective is:
m
Align (X & max Σ
max (E(
log P (ax, ls:)
(5)
B.2 Efficiency Optimizations
We apply the same efficiency optimizations in App. A.3 in latent alignment.
C More Details and Results on Case Study 1
C.1 Algorithm Pseudocode
Fig. 7 and Fig. 8 show byde-by-side comparisons of the training and generation processes of RF and RF+LZN.
htttps:/ppytorch http//oc//ssaaaa org.///cs//tabbe//cccpoiiii/ html
26

---

Algorithm 2: RF+LZN training
Algorithm 1: RF training
Input
Training set:
Decoder: Dz
Input
Training set: X'
Encoder: Eg (used by C)
Decoder: Dz
Number of iterations: T
Number of iterations: T
Batch size: B
Batch size: B
for ieeation < 1, .,
. do
for iteration + ,,..., do
21...., BB Draw samples from X
2
F1.... BB Draw samples from X
(1....; CB f Gaussian noise
, t- C' (Z1,...,,22) TB)
B1.... tB t Random RF timesteps
4
61....; CB f- Gaussian noise
G (1 li)e 1.;;
Training using Da (E:)
5
l1,..., Random RF timesteps
6
& 4- 1,)e; + L.;;
Training using (5
Figure 7: Comparison between training processes of RF and RF+LZN. Left: A simplified illustration of the standard RF [53] training process. In each iteration, batch of real samples and batch of Gaussian noise are drawn and interpolated to produce noisy inputs, which are then passed through the decoder network compute the loss. Right: A simplified illustration of the RF+LZN training process. The key differences are highlighted in gray: we compute LZN latents for the samples using the method in 2.2.1, and provide these latents additional input the RF decoder.
Algorithm 4: RF+LZN generation
Algorithm 3: RF generation
Input Decoder: D₂
Input Decoodr: D2
Gaussian noise
{ <
Gaussian noise
Generated sample < Dr (E)
2 2 f
Gaussian noise
Generated sample Dx
Figure 8: Comparison between the generation processes of RF and RF+LZN. Left: A simple illustration of the standard RF generation process [53]. The decoder takes Gaussian noise as input and generates sample. The actual process is iterative, but we leave out the steps for simplicity and only show the starting input (Gaussian noise) and the final output (the generated image). Right: simple illustration of the RF+LZN generation process. The main differences are shown in gray: sample extra LZN latents from Gaussian noise and use them as additional inputs to the RF decoder during the iterative generation process.
C.2 More Implementation Details
Architecture.
Decoder.
The only change to the RF architecture [53] concatenating the LZN latent with the timestep embedding.
Encoder. We extend the UNet encoder in RF [53] by connecting the output of each ResNet block with latent transformation block. The sum of the outputs of the latent transformation blocks forms the LZN latent. Each latent transformation block consists of: (1) 1 x convolution that projects the ResNet output to 20 channels, reducing dimensionality; and (2) small MLP with d00esimmali hidden layer that outputs the latent from the flattened convolution output.
C.3 More Experimental Settings
Datasets.
CIFAR10 (32 x 32) [38] contains 50000 training images and 10000 test images of classes of objects. We only utilize the training set for this experiment.
AFHQ-Cat (256 x 256) [15] contains 5153 catimages.
27

---

CelebA-HQ (256 x 256) [34] contains 30000 face images. LSUN-Bedroom (256 x 256) [86] contains 3033042 bedroom images.
Metrics.
FID [31] and sFID [58] evaluate the similarity between real and generated images by projecting both into the latent space of pretrained network (e.g.. Inception-v3 [80]). fitting each set of latents with Gaussian distributions, and computing their Wasserstein-2 distance. The key difference between FID and sFID is the feature layer used: FID uses pooled features, while sFID uses intermediate features, making it more sensitive spatial details.
Inception score (IS) [73] measures image quality by assessing both the quality of each image (how confidently a classifier predicts a class) and diversity across all images (coverage over different classes). Since the classifier is trained on ImageNet [17], IS is best suited for natural image datasets like CIFAR10. We report IS for all datasets for completeness.
Precision and recall [40] evaluate the quality and coverage of generated images. Intuitively, precision measures the fraction of generated images that are close real ones, while recall measures the fraction of real images that are close to the generated ones.
CMMD [33] measure the MMD distances between the CLIP embeddings of the real images and generated images. Compared to FID, is reported align better with human preference and have better sample efficiency.
Reconstruction error measures how well generative model can reconstruct an input image. This reflects the model's representational power and is crucial for applications like image editing, where edits are made by modifying the image's latent representation []. For RF, we first apply the inverse ODE to map the image to its latent representation, then use the forward ODE to reconstruct the image, and compute the & distance between the original and reconstructed images. For RF+LZN, we add an initial step: compute the image's 's LZN latent C (.t and feed it into RF latent computation process an additional input.
Following the convention [53, 77, 48, 49), the metrics are all computed using the training set of the dataset.
For FID, sFID, IS, precision, recall, and CMMD, we subsample the training set and generate the same number of samples compute the metrics. The number of samples are:
CIFAR10: 50000 (the whole training set).
AFHQ-Cat: 5120, the largest multiple of the batch size (256) that is less than or equal to the
training set size (5153).
CelebA-HQ: 29952, the largest multiple of the batch size (256) that is less than equal to the
training set size (30000).
LSUN- Bedrromm: 29952, the largest multiple the batch size (256) that is less than or equal to 30000. We limit the number of samples to 30000 that the computation cost of the metrics reasonable.
For reconstruction error, randomly sample batch of images (2000 for CIFAR10 and 256 for the other datasets) from the training set. Each image is reconstructed 20 times (note that the LZN latents (X') have randomness). We report the average metric over all reconstructions.
Note that for all the random subsampling procedures mentioned above, we ensure the sampled sets are consistent between RF and RF+LZN, so that the resulting metrics are directly comparable.
Sampler. RF requires sampler to numerically solve the ODE (integral) trajectory for sample generation. For both RF and RF+LZN, we use the RK45 sampler from RF [53], which adaptively determines the number of steps. In App. C.4, also analyze the effect of varying the number of sampling steps using the Euler sampler [].
Hyperparameters.
CIFAR10
RFF
Batch size: 2000
Optimizer: Adam
Decoder learning rate: 0.001
28

---

Gradient clipping: 1.0
Number of parameters in decoder: 61 804419
RF+LZN:
Batch size: 2000
Optimizer: Adam
Decoder learning rate: 0.001
Encoder learning rate: 0.000025
Gradient clipping: 1.0
Latent dimension: 200
Number of parameters in decoder: 61906819
Number of parameters in encoder: 49790260
Aat
RF:
Batch size: 256
Optimizer: Adam
Decoder learning rate: 0.0002
Gradient clipping: 1.0
Number of parameters in decoder: 65574549
RF+LZN:
Batch size: 256
Optimizer: Adam
Decoder learning rate: 0.0002
Encoder learning rate: 0.000002
Gradient clipping: 1.0
Latent dimension: 200
Number of parameters in decoder: 65676949
Number of parameters in encoder: 87768896
CelebA-HQ
RF:
Batch size: 256
Optimizer: Adam
Decoder learning rate: 0.0002
Gradient clipping: 1.0
Number of parameters in decoder: 65574549
RF+LZN:
Batch size: 256
Optimizer: Adam
Decoder learning rate: 0.0002
Encoder learning rate: 0.000004
Gradient clipping: 1.0
Latent dimension: 200
Number of parameters in decoder: 65676949
Number of parameters in encoder: 87768896
RF:
Batch size: 256
Optimizer: Adam
Decoder learning rate: 0.0002
Gradient clipping: 1.0
Number of parameters in decoder: 65574549
RF+LZN:
Batch size: 256
29

---

Figure 9: Generated images of RF on CIFAR10.
Optimizer: Adam
Decoder learning rate: 0.0002
Encoder learning rate: 0.000002
Gradient clipping: 1.0
Latent dimension: 200
Number of parameters in decoder: 65676949
Number of parameters in encoder: 87768896
Computation cost. Excluding the computation cost of periodic evaluation (i.e., only counting the computation cost model training), each RF+LZN experiment takes:
CIFAR10: hours on 16 A100 (40 GB) GPUs.
AFHQ-Cat: 10 hours on 32 A100 (40 GB) GPUs.
CelebA-HQ: 58 hours on 32 A100 (40 GB) GPUs.
LSUN -edrooed 341 hours on 32 A100 (40 GB) GPUs.
C.4 More Results
Generated images. The generated images RF and RF+LZN are in Figs. to 16.
Ablation studies on FID implementation. It is known that subtle differences in FID implementation result in different results [64]. In our main experiments, we use the implementation in consistency models [77]. In Tab. 5, we additionally show the FID using two other implementations: RF [53] and clean FID [64]. We can that, while the numbers different, the relative ranking across all three implementations is consistent. Especially, RF+LZN achieves the best FID in three out of four datasets.
Ablation studies on sampling steps. In this experiment, we use the Fuler sampler with varying numbers sampling steps. As shown in Fig. 17, RF+L2N generally achieves better FID than the RF baseline across most settings. Notably, in the only case where RF+LZN performs worse than RF in
30

---

Figure 10: Generated images of RF+LZN on CIFAR10.
Table 5: FID with different implementations for unconditional image generation. "CM" denotes consistency models [77]; "RF" denotes Rectified Flow [53]; "clean" denotes clean FID [64]. The best results are gray box
CIFAR1O 32)
AFHQ-Cat 256)
Algo.
FID (clean)
FID (CM))
(clean))
FID (CM))
3.18
2.77
2.76
5.99
6.20
6.08
RF+L2N
3.05
2.61
2.59
5.66
5.69
5.68
CelebA- -HQ (256 x 256)
LSUN Bedroom (256 x 256)
Algo.
FTD (clean) FID (RT).
FID (CM).
FID (clean)
FID (RT).
FID (CM)L
RF
7.10
7.00
6.95
6.39
6.25
6.25
RF+L2N
7.31
7.23
7.17
5.88
5.87
5.95
Tab. 1, we observe that the underperformance occurs only at the highest number of sampling steps in the Euler sampler (Fig. 17c).
31

---

Figure 11: Generated images of RF on Aat.
Figure 12: Generated images of RF+LZN on AFHQ-Cat
32

---

Figure 13: Generated images of RF on CelebA-HQ.
Figure 14: Generated images of RF+LZN on CelebA-HQ.
33

---

Figure 15: Generated images of RF on LSUN-Bedrooo..
Figure 16: Generated images of RF+LZN on LSUN-Bedron
34

---

Rectified Flow
10
Rectified Flow
10a
Rectified Flow LZN
Rectified Flow LZN
2 10
6 x 10
FID
10a
x 10
3 x 10
10
102 NFE
10
10
102
10a
(a) CIFAR10.
(h) AFHQ-Cats
Rectified Flow
Rectified Flow LZN
Rectified Flow
10a
Rectified Flow LZN
FID
10a
10
6 109
10a
102 NFE
10a
10a
102 NFE
10
(c) CelebA-HQ. HQ.
(d) LSSUN-Bedroo
Figure 17: FID vs. number of sampling steps in the Euler sampler. RF+LZN outperforms RF in most cases.
35

---

D More Details and Results on Case Study 2
D.1 Algorithm Pseudocode
Alg. shows the pseudocode of the training process.
After training, the encoder E, can be used to obtain image representations. We provide several strategies for extracting these representations. Please App. D.4 for details.
D.2 More Implementation Details
Architecture. To remain consistent with prior work [28, 10, 26], we use the ResNet-50 architecture [29] as the encoder for LZN. The only modification make is replacing all batch normalization layers with group normalization. However, our early experiments indicate that this change does not lead to significant performance differences.
For the projection head following the ResNet-50 output, we use an MLP with one hidden layer, as in [10, 11].
Data augmentation. We follow the same data augmentation strategy in [11].
Representation. Following prior work [10, 11]. after training the ResNet-50, we discard the projection head and use only the ResNet-50 backbone extract representations for training the linear classifier. As result, obtaining representations from LZN in this way does not require going through the LZN latent computation process C, and thus has the same computational efficiency baseline methods.
Objective. We use the version with log (Eq. (5)).
D.3 More Experimental Settings
Datasets.
We the ImageNet dataset, which contains 1281167 training images and 50000 validation images. LZN is trained on the training set, and classification accuracy is evaluated on the validation set.
Hy perppaaraettrss.
Batch size: 8192
Optimizer: Adam
Learning rate: 8c-4
Gradient clipping: 1.0
Latent dimension: 256
Number of parameters: 24032832
a: 0.45
Computation cost. Excluding the computation cost of periodic evaluation (i.e., only counting the computation cost of model training), each LZN experiment takes 1800 hours on 128 A100 (40 GB) GPUs.
D.4 More Results
More baselines. Tab, 6 shows the result with more baselines that are not using the ResNet-50 architecture.
Visualizing the learned representations. We take images from randomly selected 20 classes from the validation set of ImageNet and computed their embeddings using the trained LZN model. We chose the validation set to ensure that the results are influenced by training set overfitting. We then projected these embeddings into 2D space using I-SNE-a widely used method for visualizing high-dimensional representations, following seminal works such as SimCLR [10]. The resulting
Note that we use the term contrastive learning broadly to refer not only to methods employing the traditional contrastive loss, but all approaches that encourage relevant images to share similar representations; 4.
36

---

Algorithm 5: Unsupervised representation learning with LZN
Input
Training set: X
Encoder: Ez
Number of iterations: T
Batch size: B
for ieration ,...., do
B1:..., FB f- Draw samples from X'
2/ Two random augmentations of I2 Training Ez using Align ((zi,...,,4) In) (,1....,333)) ,,})
Table 6: Classification accuracy on ImageNet by training a linear classifier on the unsupervised representations. Methods with are based on contrastive learning." The horizontal line separates baselines that perform worse or better than our LZN. "R" means "ResNet".
Algorithm
Architecture
Top-1 Acct
Top-5 Acct Acet
Colorization [88]
R101
39.6 [28]
Jigsaw [59]
R50w2x
44.6 [28]
Exemplar [22]
R50w3x
46.0 [28]
DeepCluster [8]
VGG
48.4 [28]
CPC v1 [61]
R101
48.7 [28]
RelativePosition [20]
R50w2
51.4 [28]
InsiDsse [85]
R50
54.0 [28]
Rotation [25]
Rv50w4 x
55.4 [28]
BigBiGAN |21|
R50
56.6 [28]
LoculAgg [91]
R50
58.8 [28]
MoCo [28]
R50
60.2 [10]
BigBiGAN [21]
Rv50w4x
61.3 [28]
81.9 [10]
PIRL [57]
R50
63.6 [10]
CPC v2f [30]
R50
63.8 [10]
85.3 [10]
CMC [81]
R50
66.2 [26]
87.0 [26]
SimSiam [13]
R50
68.1 [13]
SimCLR [10]
R50
69.3 [10]
89.0 [100
MoCo v25 [12]
R50
71.7 [12]
SimCLR v2 [1]]
R50
71.7 [1]]
BYOL [26]
R50
74.3 [26]
91.6 [26]
DINO [9]
R50
75.3 [9]
DINO 99]
VIT-S
77.0 [9]
DINO [9]
ViT-B/16
78.2 [9]
DINO [9]
VIT-S/8
79.7 [9]
DINO 99]
VIT-B/8
80.1 [9]
IJPEA [3]
ViT-B/16
72.9 [3]
IJPEA [3
VIT-L/16
77.5 [3]
IJPEA [3]
ViT-H/14
79.3 [3]
IJJEA [3]
ViT-H/16G4s
81.1 [3]
LZN
R50
69.5
89.3
t-SNE plot is in Fig, 18. We can see that the samples from different classes are well-clustered. This suggests LZN learns meaningful image representations.
Ablation studies on representation choice. Prior work [10, 11] has shown that the choice feature extraction layer significantly affects downstream performance. In particular, removing the projection head often improves results. Motivated by this, we explore various feature extraction strategies for LZN, which offers more flexibility due to its unique latent computation process ($ 2.2.1). Specifically, we compare the following methods:
With latent. Use the latent representation from LZN (see 2.2.1) to train the classifier.
With latent (a 0). Same above but with in the latent computation.
With head. Use the anchor point (i.e., encoder output before FM computation).
Without head. Use the ResNet backbone output (before the projection head).
37

---

993
859
298
553
231
306
706
496
558
-10
784
239
-0
55
906
-30
175
14
-20
20
40
77
Figure 18: t-SNE visualization of LZN representations projected into 2D for 20 randomly selected ImageNet validation classes. Images from the same class form distinct clusters, indicating that LZN learns meaningful image representations.
The first two methods are specific to LZN, while the last two follow the design commonly used in prior contrastive learning work [10, 11].
The results are shown in Fig. 19. We observe the following:
LZN latent achieves the lowest prediction accuracy. As discussed in App. A.2, the LZN latents are reliable only when computed over the full dataset. However, for efficiency, both training and inference rely on minibatches to approximate the latent representations. This approximation increases the size of the latent zones, leading to potential overlap between the zones of different samples across batches, which inevitably degrades downstream classification performance.
In comparison, LZN with a O yields significantly higher accuracy. This improvement can be attributed to the reduced likelihood of overlap between latent zones when a 0, making resulting representations more distinct and less noisy.
The final two methods, "with head" and "'without head", do not involve latent computation and are therefore more efficient. Consistent with findings from prior contrastive learning studies [10], we observe that "without head" performs substantially better. As explained in [10], the projection head often discards important information minimize the training loss. In contrast, layers preceding the head might retain richer and more discriminative features, which are more useful for downstream classification tasks.
as types data augmentation-
order to
Ablation studies the number of training steps. Fig. 20 shows classification accuracy training iterations. Accuracy continues to improve rapidly at the end of training, suggesting that with more training, the gap between LZN and the SoTA could be further reduced.
38

---

65.6
0.6
54.9
55.9
0.4
Top-1
0.3
0.2
11.9
0.1
0.0
latent
head
W/ latent head
(a m
Method
Figure 19: LZN's linear classification accuracy with different feature extraction methods. Note that this experiment uses fewer iterations (1060000) than the main experiment (5000000) and omits data augmentation when training the linear classifier (used in the main experiment), so the accuracies are lower than the main experiment.
0.70
Top-1 accuracy
0.66
0.64
0.62
0.60
0.58
Iterations
1e6
Figure 20: LZN'S linear classification accuracy training iteration. The accuracy is still improving at a fast rate at the end training. More training might further improve the result.
E
More Details and Results Case Study 3
E.1
Algorithm Pseudocode
Alg. 6, Fig. 21, and Alg. 9 show the algorithm pseudocode the training, generation, and classifica- tion process.
E.2 More Implementation Details
Architecture.
Image decoder. For RF, we modify the original architecture [53] to include a one-hot encoding of the class label as an additional input, concatenated with the timestep embedding. For RF+LZN, we apply the same modification on top of the architecture described in App. For unconditional generation, this one-hot encoding is deterministically derived from the LZN latent: given LZN latent, we use the class label decoder to predict the class and then encode it as a one-hot vector. As result, the decoder's output remains fully determined by the LZN and RF latents, consistent with $ 3. For conditional generation, this one-hot encoding is given as a condition, and LZN latents are sampled from the corresponding latent zone (as described in $ 2.2.1). Image encoder. Same as that of App. C.
Label FM. The method $ 2.2.2 implicitly assumes uniform distribution over class labels. However, due to sampling randomness during training, each batch may have imbalanced class
39

---

Algorithm 6: RF+LZN training (with class labels)
Input
Training set: labels A' and images y
Image decoder: Dy
Image encoder: Fyy (used by C)
Label anchors: A (used by Align)
Number of iterations: T
Batch size: B
for iteration ..... T do
M1...... - Draw images from y
21....1 2B .
(1...., CB + Gaussian noise
$1..... tB t Random RF timesteps
+ (1 -1--a -t,e,, t.yh
Training using Align (X, (yi,...,,1, (BB) and RF loss on Dy (Er: (a weighted loss between
the two)
Algorithm 8: RF+LZN generation (uncondi- tional)
Algorithm 7: RF+LZN generation (uncondi-
tional)
Input
:
Image decoder: Dy
Input :mage decoder: Dy
Label set: Ci....,Cp
Class ID: & (i.e., the class is c+)
{ Gaussian noise
Gaussian noise
5 t Gaussian noise
3 Generated sample- <
Dy (5:
2 2 + C' ((c1..... ,,)))
Generated sample Dy (E:
The generation process of RF+LZN (with class labels). In this case, RF+LZN can simultaneously support unconditional and conditional generation. Left: Unconditional generation, where the LZN latent is drawn from the prior Gaussian distribution, which is exactly the same as Fig. Right: Conditional generation, where the LZN latent is drawn from the latent zone of the corresponding class. The changes on top of unconditional generation are highlighted in gray.
Figure 21:
distribution. To address this, we modify the ग distribution when computing FM, (-) to be a weighted mixture of Dirac delta functions centered at Eg (z,). with weights corresponding to the fraction of class z; samples in the batch. During testing, we revert to a uniform prior, as the true class distribution of the batch not available.
Objective. We use the version without log (Eq. (2)). We also tried the version with log (Eq. (5)) and did not observe large difference in results.
E.3 More Experimental Settings
Datasets. We use the CIFAR10 dataset discussed App. C.
Metrics. In addition to the metrics discussed in App. C, we evaluate on CIFAR10 classification accuracy. The accuracy evaluated on CIFAR10 test
Sampler. Same as App. C.
Hyperparameters.
CIFAR10
RFF
Batch size: 2000
Optimizer: Adam
Decoder learning rate: 0.002
40

---

Algorithm 9: RF+LZN classification
Input
.Image encoder: D=
Label decoder: D
Images: y1,-...
......; .....,,.... 2B f- C (y .n)
D2 (zi)
Table FID with different implementations for conditional image generation on CIFAR10. "CM"
denotes consistency models [77]; "RF" denotes Rectified Flow [53]; "clean" denotes clean FID [64].
The best results are in gray box
Algo.
FID (clcan)
FID (RF)
FID (CM)
RF
2.85
2.50
2.47
RF+LZN
2.70
2.42
2.40
Gradient clipping: 1.0
Number of parameters in decoder: 61 609539
RF+LZN:
Batch size: 2000
Optimizer: Adam
Decoder learning rate: 0.002
Encoder learning rate: 0.00005
Label learning rate: 0.0001
Gradient clipping: 1.0
Latent dimension: 200
Number of parameters in decoder: 61911939
Number of parameters in encoder: 49790260
Computation cost. Excluding the computation cost of periodic evaluation (i.e., only counting the computation cost of model training), each RF+L2N experiment takes 31 hours on 16 A100 (40 GB) GPUs.
E,4 More Results
Generated images. The generated images of RF and RF+LZN are in Figs. 22 and 23.
Ablation studies on FID implementation. Same as App. we present the FID scores using three different implementations in Tab. 7. We see that, while the numbers are different, the relative ranking across all three implementations is consistent. Especially, RF+LZN achieves the best FID all implementations.
Ablation studies sampling steps. Following the experimental settings in App. C, Euler sampler with varying numbers of sampling steps. As shown in Fig. 24, RF+LZN generally achieves better FID than the RF baseline across settings.
the
A Ablaiion studies on classification techniques. Here, we discuss several techniques for improving the classification results.
Recall that latent computation (Eq. (1)) includes randomness from e, because each sample corresponds to a latent zone, not single point. Empirically, for classification tasks, using the "center" of the latent zone yields better performance. Concretely, we set in Eq. (4) when computing latents. This is intuitive, as the center is likely farther from zone boundaries and better represents the sample.
App. A.3 discusses that during training, we use batch samples rather than all samples to estimate latents for efficiency. However, during inference, where gradient computation is unnecessary and thus the overhead of large batch sizes less critical, we can use larger batch size to improve performance.
41

---

Figure 22: Generated images of RF on CIFAR10 (conditional generation). Every rows corresponds to one class in CIFAR10.
Fig. 25 shows that increasing the batch size and decreasing improve the classification accuracy. The best setting improves the default setting (batch size= 2000 and a = 1.0) by 2.9%.
Ablation studies on latent alignment hyperparameter. In latent alignment ($ 2.2.2), we introduced hyperparameter u that controls how many time steps are excluded from the latent alignment objective. In our main experiments, we set 20 (out of a total of 100 steps). Here, we conduct an ablation study by reducing that does not affect the results much, and the performance remains better than the baseline RF across most metries. This expected. Unlike common hyperparameters (such as loss weights) that influence the optimal solution, does not alter the optimal solution, which is the perfect alignment
(i.e., smaller). The results are shown in Tab. We can see
42

---

Figure 23: Generated images RF+LZN on CIFAR10(conditional generation). Every rows corre- sponds to one class in CIFAR10.
between two latent zones. Instead, this parameter is introduced solely to help avoid getting stuck in local optima ($ 2.2.2). We expect that any small but non-zero value of should be sufficient in practice.
43

---

10
Rectified Flow
Rectified Flow LZN
6 10
FID
4 10
3 x 10
104
10
10a
NFE
Figure 24:
FID number of sampling steps in the Fuler sampler CIFAR10 (conditional generation). RF+LZN outperforms RF in most cases.
0.940
2000
Batch size
4000
0.930
8000
00.0
64.24
2.344
90.00
0.925
10000
90.35
24:20
24:44
$4.47
94.45
54.44
1.0
0.8
0.6
0.4
0.2
0.0
0.920
Figure Classification accuracy with different hyperparameters on CIFAR10. Generally, increasing the batch size and decreasing a improve the classification accuracy.
Table 8: Conditional image generation quality and classification accuracy on CIFAR10. The best results are in gray box The hyperparameter does not impact the results much.
Algo.
FID.
sFID
ISt
Precision
Recallf
Recon
Accuracy!
RF
2.47
4.05
9.77
0.71
0.58
0.69
RF+LZN (u 20)
2.40
3.99
9.88
0.71
0.58
0.38
94.47
RF+LZN (u 5)
2.39
3.99
9.76
0.71
0.58
0.36
94.42
44

---

F Extended Discussions on Related Work
[16] proposes to conduct flow matching in the latent space. However, it has quite different goals and techniques from LZN.
Goals. The goal of [16] is to improve generation lasks. In contrast, goal is more ambitious: to develop a unified framework that supports generation, representation learning, and classification. This broader scope requires different design philosophy and technical approach, detailed next.
Techniques. [16] applies flow matching to the latent space of a pre-trained Stable Diffiusion autoencoeer, which is reasonable when focusing solely generation. However, such latent space is high-dimensiioall and retains spatial structure, limiting its suitability for classification and compact representation learning. To support our broader objectives, we introduce several novel techniques:
Match discrete distribution (i.c., the anchors) to a continuous one, as opposed to continuous-
to-continuous distribution matching in [16].
Use an adaptive latent space, since encoder and decoder are trained end-to-end, opposed to using a fixed wre-trained autoencoder and fixed latent space in [16].
Numerically solve the flow directly, as opposed to training an additional model to learn the flow
in [16].
Latent alignment between different data types (e.g., image and label), which is new in paper.
G Extended Discussions on Limitations and Future Work
Inference efficiency. It is important to note that while the training cost of LZN might be high, at inference time, LZN is often as efficient as existing approaches.
For image generation ($ and 5), we not need to compute the latent during inference. Instead, latents are sampled from the Gaussian prior and passed directly the decoder, making the generation speed comparable to the base model.
For representation learning (S 4), we find that dropping the final encoder layers during inference improves performance (App. D.4), similar to the observation in prior contrastive learning methods [10]. In this case, inference involves simply passing an image through the encoder without the latent computation process ($ 2.2.1), 1), just like in traditional contrastive learning methods.
Training efficiency. The main training bottleneck stems from the quadratic cost with respect the batch size. Notably, this is also the case for many contrastive learning methods, including the seminal works MoCo [28] and SimCLR [10), which compute pairwise similarities between examples in a batch.
The parallel between LLM training and LZN training. We observe an interesting parallel between the training of LLMs and LZN. Specifically, in LLM training, computing attention weights requires O (c) where e is the context length, d the attention dimension, and the number of layers. In LZN, computing the latents ($ 2.2.1) requires O(n2gr), where the number of samples in batch, q is the latent dimension, and r is the number of solver steps. Several parallels emerge:
Context length in LLMs (c) ++ Number of samples LZN (7))
Attention dimension in LLMs (d) Latent dimension in LZN (g)
Number of layers LLMs (v) ++ Number of solver steps in LZN (r)
Not only do these parameter pairs affect the time complexity in similar ways, but their computation flows are also analogous: in LLMs, the pairwise inner product of token features is computed to derive attention weights, and these weights are computed sequentially across layers. Similarly, in LZN, the pairwise distances between intermediate anchor points of samples are computed to derive velocity, and this velocity updated sequentially across solver steps.
While LLM training is known to be computationally expensive, recent advances have significantly improved efficiency. Given the structural similarities, we expect that such advances in L.M training could be adapted to enhance the training efficiency of LZN as well.
45

---

Using LZN solely to implement generative modeling. In theory, LZN can be used solely for generative modeling. By construction ($ 3), if the decoder is trained to map latents the corresponding data perfectly, then the generative distribution of LZN is exactly Di=1 0(s .i). i.e., the empirical distribution of the training set. We explored this approach in our early experiments. It performs well on simple datasets such as MNIST [18], but generates blurry images more complex datasets such as CIFAR10. We hypothesize that this may be due to the minibatch approximation (App. A.3), which can break the disjoint latent property. and/or the strict requirement that latent zones have no gaps between them. We leave deeper exploration of this direction to future work.
Societal impacts. Since LZN can be used to improve ML models, it has the potential for both beneficial and harmful applications. Positive use cases include creative content generation and improved information retrieval, while negative applications may involve the creation of fake or misleading content.
46


Reviews:
[{"review_id": "w3iIOgA9Zb", "reviewer": "Reviewer_71Fq", "summary": "This paper proposes Latent Zoning Network (LZN), a unified framework aiming to address three core ML tasks: generative modeling, representation learning, and classification through a shared Gaussian latent space. The key innovation is using flow matching to create disjoint latent zones for different samples, where each data type has dedicated encoders/decoders that map to/from this shared space. The authors demonstrate LZN's versatility through three case studies: enhancing existing generative models (improving RF's FID on CIFAR10 from 2.76 to 2.59), standalone unsupervised representation learning (outperforming MoCo by 5.4% on ImageNet), and joint generation+classification tasks.", "questions": "- How does the mini-batch approximation affect the theoretical guarantees, particularly the Gaussian prior and disjoint zone properties?\n- How does performance degrade as you add more tasks/modalities to the framework?\n- The soft alignment mechanism involves several hyperparameters. How sensitive is performance to these choices?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strength\n\n- The core idea of using flow matching to create a unified latent space with disjoint zones is interesting.\n- The theoretical framework connecting generative modeling, representation learning and classification is obvious but still conceptually appealing and potentially helpful to the community when drilling down the details like latent computation and latent alignment.\n- The soft alignment mechanism for latent zones (Section 2.2.2) provides a differentiable solution to an inherently discrete problem.\n- The experiments are comprehensive covering unconditional generation, representation learning and classification/conditional generation, which validates the framework's versatility.\n- The training and implementation details are detailed which would help the community reproducing the result.\n\n### Weakness\n\n- The paper doesn’t provide a comprehensive overview of the related work. For example, \"Flow Matching in Latent Space\" (Dao et al., 2023) proposes applying flow matching in the latent spaces as well. It will be helpful to do a more thorough comparison.\n- The scalability of the method might not be good as the O(n²qr) complexity for latent computation is prohibitive.\n- The mini-batch approximation fundamentally undermines the theoretical framework as the latent zones become batch-dependent rather than globally consistent\n- In classification tasks, LZN doesn’t perform as well as many other baselines, and the paper doesn't provide much justification/explanation.\n- The claim of the generality of the framework probably needs to be more humble if the experiments are only carried out in the image domain.", "was_revised": true, "final_justification": "I think the authors did a great job in addressing the main concerns I have for that paper, and I think it's an interesting unified framework in general with solid experiments and potential to generalize to more diverse tasks, so I recommend acceptance.", "rebuttal": "Thank you very much for these very insightful questions! Below, we address all your questions point-by-point.\n\n---\n\n---\n\n> ### Q1. Related work\n\nThank you very much for pointing out this interesting and relevant work. While we acknowledge its connection to our topic, we would like to clarify that both the **goals** and the **techniques** of the two papers differ significantly.\n\n**Goals**\n\n- The goal of the referenced paper is to improve *generation tasks*.\n- In contrast, our goal is more ambitious: to develop a **unified framework** that supports *generation*, *representation learning*, and *classification*. This broader scope requires a different design philosophy and technical approach, as detailed next.\n\n**Techniques**\n\n- The referenced paper applies flow matching to the **latent space of a pre-trained Stable Diffusion autoencoder**, which is reasonable when focusing solely on generation. However, such a latent space is **high-dimensional** and retains **spatial structure**, limiting its suitability for classification and compact representation learning.\n\n- To support our broader objectives, we introduce several novel techniques:\n  - Match a **discrete** distribution (i.e., the anchors) to a **continuous** one, as opposed to a continuous-to-continuous distribution matching in the referenced paper.\n  - Use an **adaptive latent space**, since our encoder and decoder are trained end-to-end, as opposed to using a fixed pre-trained autoencoder and fixed latent space in the referenced paper.\n  - **Numerically solve** the flow directly, as opposed to training an additional model to learn the flow in the referenced paper.\n  - **Latent alignment** between different data types (e.g., image and label), which is new in our paper.\n\nThat said, we agree that the referenced work is relevant and should be discussed. We will add the above points to the revision.\n\n---\n\n---\n\n> ### Q2. Scalability\n\nWe acknowledge the computational cost of our approach, as discussed in **Section 2** and **Section 6**. However, we would like to highlight several key points:\n\n- **The computation cost, while non-trivial, is comparable to many existing techniques.**  \n  - The main bottleneck stems from the quadratic cost with respect to the batch size (i.e., the $n^2$ term). Notably, this is also the case for many contrastive learning methods, including the seminal works **MoCo** [25] and **SimCLR** [8], which compute pairwise similarities between all examples in a batch.\n  - As discussed in **Appendix F**, the computational profile of LZN is similar to that of large language models (LLMs): LZN scales quadratically with the number of samples, whereas LLMs scale quadratically with the number of tokens. Given that modern LLMs often process millions of tokens, the computational cost of LZN is relatively modest in comparison.\n\n- **We propose several optimization strategies to improve LZN’s efficiency**, as detailed in **Appendix A.3**. With these optimizations, the training cost remains manageable across all our experiments.\n\n- **At inference time, LZN often does not incur this high computational cost.**\n  - For **image generation** (Case Studies 1 and 3), we do not need to compute the latent during inference. Instead, latents are sampled from the Gaussian prior and passed directly to the decoder, making the generation speed comparable to the base model. This is explained in **Lines 269–271**.\n  - For **representation learning** (Case Study 2), we find that dropping the final encoder layers during inference—similar to several contrastive learning methods (e.g., [8])—not only improves performance but also eliminates the $O(n^2)$ cost. In this case, inference involves simply passing an image through the encoder, just like in traditional contrastive learning methods. This is discussed in **Lines 1187–1192**.\n\n\n\n---\n\n---\n\n> ### Q3. Mini-batch approximation\n\nThank you for the great question!\n\n- **Mini-batch approximation still preserves the Gaussian prior.**  \n  LZN ensures that the latent distribution within *each mini-batch* is approximately $\\mathcal{N}(0, I)$. As a result, the overall latent distribution becomes a mixture of Gaussians with the same parameters, which is still $\\mathcal{N}(0, I)$. Therefore, the global prior remains valid under the mini-batch approximation.\n\n- **However, this approximation does violate the disjoint zone property.**  \n  To quantify this effect, we conducted the following experiment:  \n  We randomly sample one instance $x$ from the dataset and construct two batches with other randomly sampled instances: $\\\\{x, x_2, \\dots, x_n\\\\}$ and $\\\\{x, x_2', \\dots, x_n'\\\\}$, where $n$ is the batch size. We then compute the latent for $x$ using each batch and evaluate how often the latent of $x$ from one batch lies within the latent zone of $x$ in the other batch. The result shows that **only 49%** of the time does the latent remain in its intended zone, implying a 51% chance that it overlaps with other zones.\n\n- **Despite this limitation, the mini-batch approximation performs well in practice.**\n  - In **generative modeling** (Case Studies 1 and 3), the latent does not need perfect zone disjointness—as long as it provides *some* information about the input sample, it can help reduce the variance needed to learn by the generative model (rectified flow in our case) and improve the generation quality.\n  - In **representation learning** (Case Study 2), latent alignment occurs *within* a single batch. Thus, inconsistency across batches is irrelevant.\n  - In **classification** (Case Study 3), we only need to map samples *within a single batch* to the latent zones of labels. Thus, inconsistency across batches is irrelevant.\n\nGiven these points, we observe strong empirical results across all three case studies despite using the mini-batch approximation.\n\nThat said, developing a better method that is both efficient and preserves the disjoint latent zone property might further improve LZN’s performance. We leave this as an exciting direction for future work.\n\n---\n\n---\n\n> ### Q4. Performance on classification tasks\n\nThank you for the question.\n\n- **For the classification experiments in Case Study 2**, we would like to highlight that **our results have substantially improved since the submission**. By extending training to 4.5 million iterations, the top-1 accuracy improves to **69.3%** (+3.7%) and the top-5 accuracy improves to **89.1%** (+2.0%). With these updated results, LZN is now among the **top 4 state-of-the-art methods** for this benchmark. Importantly, these competing methods are designed **exclusively for representation learning**, whereas LZN supports **more tasks including generation**. Notably, LZN also **matches or outperforms seminal methods such as SimCLR**. We believe these are strong results.\n\n- **For the classification experiments in Case Study 3**:\n  - - Our method is only **1% below the best method**, even though LZN is designed to support **both classification and generation simultaneously within a unified model**—in contrast to the competing methods that are tailored solely for classification.\n  - We believe this gap is largely due to **architecture design**. The competing methods use architectures **explicitly tailored for classification**, while we use a **standard diffusion model architecture** without any classification-specific modifications (see Appendix E.2). This is supported by the fact that using our architecture solely for classification yields **only 93.59% accuracy** (Table 3).\n  - While adopting a better architecture could potentially close this gap, the more important takeaway is that **joint training for classification and generation via LZN leads to better performance than training each task in isolation** (Lines 354–358).\n\n\n---\n\n---\n\n> ### Q5. Soft alignment\n\nThe only hyperparameter introduced in soft alignment is **u**, which determines how many time steps are excluded from the latent alignment objective (Line 234). In our experiments, we set **u = 20** (out of a total of 100 steps) across all tasks.\n\nFollowing your suggestion, we conducted an ablation study in **Case Study 3** by reducing **u** to **5** (i.e., 4× smaller). The results are shown below:\n\n\n| Method       | FID ↓ | sFID ↓ | IS ↑  | Precision ↑ | Recall ↑ | Recon ↓ | Accuracy ↑ |\n|--------------|--------|--------|-------|--------------|-----------|----------|-------------|\n| RF           | 2.47   | 4.05   | 9.77  | 0.71         | 0.58      | 0.69     | -           |\n| RF+LZN (20)  | 2.40   | 3.99   | 9.88  | 0.71         | 0.58      | 0.38     | 94.47       |\n| RF+LZN (5)   | 2.39   | 3.99   | 9.76  | 0.71         | 0.58      | 0.36     | 94.42       |\n\nWe can see that this new parameter **u** does not affect the results much, and the performance remains better than the baseline **RF** across most metrics. This is expected. Unlike common hyperparameters (such as loss weights) that influence the optimal solution, **u** does **not** alter the optimal solution, which is the perfect alignment between two latent zones. Instead, this parameter is introduced solely to help avoid getting stuck in local optima (as discussed in Lines 231–236). We expect that any small but non-zero value of **u** should be sufficient in practice.\n\n---\n\n---\n\n\n> ### Q6. Claims about generality; more tasks.\n\nWe tried training on two tasks simultaneously in **Case Study 3**, where we did **not** observe any performance degradation. In fact, we saw an **improvement** compared to training on each task separately (Lines 354–358). \n\nApplying **LZN** to a broader set of tasks is indeed an important direction for future work, as we also discussed in **Section 6**. \n\nWe appreciate the suggestion and will refine the statements in the paper accordingly.\n\n\n---\n\n---\n\n\n> ### Q7. Table alignment\n\nThank you! We will fix these in the revision.\n\n---\n\n---"}, {"review_id": "cEZoGbr8XS", "reviewer": "Reviewer_dRMg", "summary": "The paper introduces the Latent Zoning Network, a new methodology for learning interpretable and cross-modality-aligned representations of data such as images, labels, captions, etc, possibly with a pre-trained model. They do this by requiring the marginal distribution of the image distribution, etc., to be a Gaussian, while images that correspond to different labels form disjoint subsets of the representation space. (A similar formalism exists for one-to-one identification, for example.) A flow matching (or traditional) decoder can map the representations back to the data space. Experiments on various image tasks show the performance of the network on generative modeling and zero-shot classification tasks.", "questions": "- Questions above re: mathematical formulation, especially unifying the geometry and topology of (very) differently behaved inputs.\n- Why does unconditional generation (without bootstrapping off a pre-trained model) not work (as discussed in Appendix F)?\n- Is the resulting LZN more interpretable? E.g. can clustering of the representations totally recover the class clusters? Is there any more learned geometric structure (such as can be revealed via PCA)?", "limitations": "Yes (Appendix F)", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n- The idea is creative; it uses flow matching as a mechanism for direct representation learning, which is a rather new or novel idea (cf [1]), and representation alignment, which is novel to the best of my knowledge.\n- The generality of the framework is appealing and indeed puts a unified view on classification and generation as claimed. The problem statement and overall methodology is clearly stated.\n- The performance seems very nice for a new paradigm, especially comparing to state of the art. But MoCo is a bit old, so it may be better to advertise comparisons against more recent methods.\n\nWeaknesses:\n- There are some parts that are slightly mathematically un-rigorous. For example, the set of labels is finite and small (compared to the set of images, say). When mapping this through a label encoder, you would only get a few points in the representation space, and would not necessarily be able to form a connected component with the same geometry or topology as the corresponding set of representations of images. It would be interesting to clarify if this matters (and if not, why not?).\n- Although the overall methodology is clearly stated and motivated, the heuristics (in the \"Latent Alignment\" section) to execute the method seem ad-hoc. It would be great to study these design choices and why they work (or if there are better alternatives).\n- While the paper is clear in many respects, it is rather dense.\n\n[1]: Pai, Druv, et al. \"Masked completion via structured diffusion with white-box transformers.\" International Conference on Learning Representations, 2023.", "was_revised": true, "final_justification": "I recommend acceptance (5, but also open to 6). The idea is novel and puts together a lot of nice tools, and the performance on standard benchmarks is pretty good -- a bit behind SoTA on each specific task with a unified model (other reviewers have pointed this out and tend to weight this higher than me). Moreover, there is lots of analysis of different properties of the system, including an amount of introspection about why certain things don't work and how to fix them, which is rare. Altogether it is a strong scientific work. The only concerns are with the performance (both raw numbers, and they did not use ViT to optimize their performance which is a bit unfortunate).", "rebuttal": "Thank you very much for appreciating the value of our work and for the thoughtful and interesting questions!  Below, we address each of your questions one by one.\n\n> ### Q1. Advertise against more recent methods than MoCo.\n\nThank you for the suggestion; we will revise it in the paper. \n\nWe would also like to take this opportunity to highlight that **our results have substantially improved since the submission**. By extending training to 4.5 million iterations, the top-1 accuracy improves to **69.3%** (+3.7%) and the top-5 accuracy improves to **89.1%** (+2.0%). With these updated results, LZN is now among the **top 4 state-of-the-art methods** for this benchmark. Importantly, these competing methods are designed **exclusively for representation learning**, whereas LZN supports **more tasks including generation**.\n\n> ### Q2. Unifying the geometry and topology of differently behaved inputs.\n\n- We would like to first clarify the concern regarding “connected components.”  \n\n  Our encoders map images and labels to *anchor points*. Since the dataset is finite, the set of anchor points for both images and labels is also finite. These anchor points are then passed through the latent computation process (via flow matching), which maps them into a shared Gaussian prior space, and each image and label is associated with a *latent zone* in this space.\n\n  Importantly, since flow matching is a continuous transformation, **each latent zone is always a connected region**. The latent alignment between zones occurs **in the continuous Gaussian prior space**, *not* directly on the discrete anchor points. Therefore, aligning zones in the Gaussian space is not mathematically infeasible. Indeed, in our experiments on CIFAR10 (Use Case 3), we observe that the **training alignment accuracy exceeds 95%**, demonstrating that the method can learn strong alignments in practice.\n\n- That said, it is possible that finite-capacity encoders may not achieve *perfect* alignment between zones of different data types (e.g., images and labels). However, **perfect alignment is not a necessity for meaningful learning**. As long as the objective encourages the encoder to align zones in the correct direction, the model can learn useful and effective representations. As an analogy, consider traditional classification with cross-entropy loss: a classifier of finite size might not perfectly map images to their labels, but it can still generalize well and achieve high performance. Similarly, in LZN, even imperfect latent alignment can lead to strong performance across tasks.\n\n> ### Q3. Design choices of latent alignment\n\nIn Section 2.2.2, we discussed several strawman alternatives to latent alignment and analyzed why they fail:\n\n- Directly minimizing the distance between the reconstructed anchor points and the target anchor point (Lines 206–214).\n- Soft alignment across all time steps (Lines 215–227).\n- Max soft alignment without cutoff (Lines 228–232).\n\nWe believe these discussions follow a natural progression from the most naive solution to the final one, as each failure motivated the refinement that led us to the final alignment strategy. In fact, these discussions honestly reflect the thought process we went through during the development of our method.  \n\nWe understand the reviewer's desire to see more detailed analysis of the design choice. To address this, we conducted an ablation study on the only hyperparameter introduced in the latent alignment algorithm: **u**, which determines how many time steps are excluded from the latent alignment objective (Line 234).\n\nIn our main experiments, we set **u = 20** (out of 100 total steps) across all tasks. Here, we perform an ablation in **Case Study 3**, reducing **u** to **5** (i.e., 4× smaller). The results are summarized in the table below:\n\n| Method       | FID ↓ | sFID ↓ | IS ↑  | Precision ↑ | Recall ↑ | Recon ↓ | Accuracy ↑ |\n|--------------|--------|--------|-------|--------------|-----------|----------|-------------|\n| RF           | 2.47   | 4.05   | 9.77  | 0.71         | 0.58      | 0.69     | -           |\n| RF+LZN (20)  | 2.40   | 3.99   | 9.88  | 0.71         | 0.58      | 0.38     | 94.47       |\n| RF+LZN (5)   | 2.39   | 3.99   | 9.76  | 0.71         | 0.58      | 0.36     | 94.42       |\n\nWe can see that this parameter **u** does not affect the results much, and the performance remains better than the baseline **RF** across most metrics. This is expected. Unlike common hyperparameters (such as loss weights) that influence the optimal solution, **u** does **not** alter the optimal solution, which is the perfect alignment between two latent zones. Instead, this parameter is introduced solely to help avoid getting stuck in local optima (as discussed in Lines 231–236). We expect that any small but non-zero value of **u** should be sufficient in practice.\n\n\n> ### Q4. Why unconditional generation does not work\n\nThank you for the great question! We believe there are two main reasons:\n\n- When using the minibatch approximation for latent computation (see Appendix A.3), the latent zone assigned to an image can vary across batches. As a result, the same latent could be asked to reconstruct different images in different batches, and this inconsistency makes unconditional generation with LZN infeasible.\n\n- Even without minibatching, generation remains challenging. To isolate this issue, we conducted an experiment where we reduced the dataset size so that all training images could fit into a single batch. This eliminates the need for minibatch approximation. However, even in this case, the generated images were not ideal. We hypothesize that the problem arises from the **strict requirement** that the Gaussian prior space be partitioned *exactly* among all images, with no gaps. This is a very strong constraint and could be difficult for the model to satisfy. In particular, regions where latent zones connect are extremely sensitive: a small movement across zone boundaries can drastically change the target image, making training hard.\n\nWe believe solving this issue is an important direction for future work.\n\n> ### Q5. Are LZN latents interpretable?\n\nThank you for the very interesting suggestion! Following your idea, we conducted the following experiment to examine the interpretability of LZN latents.\n\nWe take images from randomly selected 20 classes from the validation set of ImageNet and computed their embeddings using the LZN representation model from Use Case 2. We chose the validation set to ensure that the results are not influenced by training set overfitting. We then projected these embeddings into a 2D space using t-SNE—a widely used method for visualizing high-dimensional representations, following seminal works such as SimCLR.\n\nThe resulting t-SNE plot (not shown here due to rebuttal constraints) indicates that samples from different classes are well-clustered. To summarize this quantitatively, we present statistics below. The embeddings of different classes exhibit distinct means, and their standard deviations are small compared to the overall standard deviation across all samples. This suggests that the LZN latents are indeed clustered and show class-wise separation, indicating meaningful and interpretable structure in the learned representation.\n\n\n| Class ID | Mean of Dim 1 | Std of Dim 1 | Mean of Dim 2 | Std of Dim 2 |\n|----------|----------------|---------------|----------------|---------------|\n| 993      | 49.85          | 12.36         | 7.43           | 1.82          |\n| 859      | 1.63           | 3.07          | -22.02         | 4.57          |\n| 298      | -1.02          | 4.29          | 17.14          | 2.52          |\n| 553      | -6.33          | 2.43          | -29.51         | 3.91          |\n| 672      | -25.78         | 6.36          | -10.89         | 2.77          |\n| 971      | 0.82           | 8.38          | -1.36          | 3.97          |\n| 27       | 19.28          | 4.05          | 6.51           | 7.42          |\n| 231      | -26.77         | 3.18          | 25.73          | 3.43          |\n| 306      | 21.76          | 6.38          | -12.56         | 3.89          |\n| 706      | -19.77         | 3.07          | 0.88           | 3.22          |\n| 496      | 11.11          | 4.95          | -22.91         | 4.85          |\n| 558      | -4.62          | 5.44          | -3.53          | 8.61          |\n| 784      | 1.81           | 6.84          | -13.29         | 8.57          |\n| 239      | -27.98         | 5.01          | 16.55          | 4.69          |\n| 578      | -11.77         | 3.22          | -9.18          | 3.86          |\n| 55       | 18.21          | 2.26          | 17.89          | 6.79          |\n| 906      | -2.69          | 4.53          | -10.22         | 5.06          |\n| 175      | -18.12         | 3.09          | 25.56          | 3.96          |\n| 14       | 6.58           | 4.99          | 29.98          | 5.54          |\n| 77       | 28.15          | 5.04          | -1.59          | 4.10          |\n| **All**  | **0.61**       | **20.20**     | **0.41**       | **17.72**     |\n\nWe will add the experiment to the revision.\n\n> ### Q6. Related work\n\nThank you for pointing out the inspiring work *“Masked Completion via Structured Diffusion with White-Box Transformers.”* We appreciate the connection and will include a citation and discussion of it in the revised version of our paper."}, {"review_id": "cpZMxiqdGC", "reviewer": "Reviewer_PF16", "summary": "The paper describes a method to obtain and use a shared latent space for different tasks, such as image generation, classification or captioning.\nEach task is composed of an encoder and decoder to map data samples to and from an individual latent space.\nThe individual latent space is then transformed to a normalized Gaussian distribution (shared latent space) by using a flow matching method.\nThe shared latent space follows the rule that a single classification label, for example \"cat\", covers a latent zone of all \"cat\" images.\nThe method is compared on three case studies:\n- Image generation on CIFAR10, AFHQ-Cat, CelebA-HQ and LSUN-Bedroom.\n- Image classification on ImageNet and CIFAR10\n- Conditional image generation on CIFAR10", "questions": "1. Did you retrain / reevaluate the models from table 2 and 3 or used the reported values from the individual papers? Not retraining can lead to inaccurate results, as your training code might differ from the original one and thus lead to improvements or deterioration of the proposed method.\n2. How many images did you use for the FID calculation in table 1 and 4 and where they part of the training data? FID is known to have problems when using a too small test set. Maybe additionally evaluate with the proposed CMMD metric!?.\n3. The Recon. error if way lower using LZN then RF only with other scores being comparable. First, is there any reason why it should be better at reconstruction if the information should be the same as the task specific latent space?. And second, how did you reconstruct an image using RF as you did not report any conditioning in the appendix Fig. 5, algorithm 1.\n\nIn general, the reason \"space constraints\" is not a strong argument to not include a more complete evaluation. Suggestion: In Table 1, a few lines  can be saved by simply moving IS, precision and recall values to the appendix, as they only show a minor change compared to the baseline.", "limitations": "Limitations are present, but I am curious if the method can be used for variable length latent spaces (if even possible with flow matching). For example, token outputs of an LLM compared to image embeddings when using two different encoder structures.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n+ The paper describes an original approach to the problem of shared latent spaces between different computer vision tasks\n+ The method described is novel and solves a very interesting part of current AI application, namely the multi modality between different tasks.\n+ The formulas look mathematically correct and proofs are provided in the appendix\n+ The paper contains a lot of detailed information in the appendix\n\n**Weaknesses**\n- General writing could be clearer.\n- The paper lacks a more complete evaluation and discussion of the results, for example between LZN and Autoregressive methods. I am also curious why the proposed method schould improve the quality of generated images, as the method is mainly an adapter to transform from the shared latent space to the task specific one (my understanding).\n- Many tables in the paper violate the NeurIPS style guide: Sec. 4.4 Line 100: \"All tables must be centered, neat, clean and legible.\".\nFor example, table 4 - Line 102: \"Place one line space before the table title, one line space after the table title, and one line space after Table 2 and 3 are offset.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for the constructive comments, which are very helpful for us to improve the paper. Below, we respond to all your questions one by one.\n\n\n\n> ### Q1. FID Computation\n\nWe followed exactly the setting that the reviewer suggested: evaluating the FID score on the **training set** with a **large number of samples**. In fact, we went further to ensure a **fair and rigorous comparison**, especially on the FID implementation, which many prior works in this domain overlooked. Please find the details below.\n\n- We have reported the number of samples used for FID computation and the dataset split they are taken from in **Appendix C.3**, which we repeat here for clarity:\n  - Following the convention [47,67,42,80,43], the metrics are all computed using the training set of the dataset. \n  - For FID, sFID, IS, precision, and recall, we subsample the training set and generate the same number of samples to compute the metrics. The number of samples are:\n    - CIFAR10: 50000 (the whole training set).\n    - AFHQ-Cat: 5120, the largest multiple of the batch size (256) that is less than or equal to the training set size (5153).\n    - CelebA-HQ: 29952, the largest multiple of the batch size (256) that is less than or equal to the training set size (30000).\n    - LSUN-Bedroom: 29952, the largest multiple of the batch size (256) that is less than or equal to 30000. We limit the number of samples to 30000 so that the computation cost of the metrics are reasonable. \n\n- It is known that **different FID implementations** can produce **different results** [56]. However, many works in this space neglect this fact and **compare results from different FID implementations**, which can lead to **unfair comparisons**. We take this issue seriously. To ensure fairness:\n  - We evaluate **all models** using **three different FID implementations**, as shown in **Table 5** and **Table 7**.\n  - These evaluations are **run by us** (i.e., we do not use reported values from other papers) to ensure that the evaluation settings and code are aligned.\n  - Despite differences in absolute values, **all three implementations consistently support** that **LZN improves image generation quality** over the baselines.\n\nWe hope this clarifies our careful handling of FID evaluation and ensures the fairness and validity of our reported results.\n\n> ### Q2. Re-training the baselines\n\n- The baseline results reported in **Table 2** are taken directly from prior papers, with the source noted next to each number. While we agree that re-running all baselines would be ideal, we would like to clarify the rationale behind our approach:\n\n  - **Following standard convention**: There is a long-standing convention in the representation learning field to reuse numbers reported in prior works rather than re-running all baselines independently. This practice dates back to seminal works such as **MoCo (2019)** and **SimCLR (2020)**, if not earlier. We adhere to this convention and **responsibly cite** the source of each number to ensure transparency and reproducibility.\n\n  - **Re-running all baselines in our case may not be useful**: We believe re-running baselines is most valuable when the proposed method builds **directly on top of a baseline**. For example, in our **image generation experiments** (Tables 1 and 4), where **LZN is applied on top of Rectified Flow**, we **re-implement Rectified Flow ourselves**, ensuring identical training setups for a fair comparison.\n\n    However, in **Table 2**, our method (LZN) is fundamentally different from the baselines. We only share the **network architecture** (from SimCLR), but not the training algorithm or objective. In such cases, **hyperparameter or code alignment is not viable**, and **re-running the baselines would not offer additional insight**, unless there are concerns about the **trustworthiness of the original results**, which we assume are reliable.\n\n- Similarly, except for the “RF+LZN (no gen)” baseline, the other baseline results in **Table 3** are taken from **[35]**, as noted in the table caption. This follows the same reasoning: the baselines are **standard classifiers trained with cross-entropy losses**, whereas our method employs a **fundamentally different objective and training procedure**. Because there are no overlapping hyperparameters or training pipelines, **re-running these baselines would not yield new insights**. \n\n  We do, however, re-run the “RF+LZN (no gen)” baseline ourselves, as it serves as a direct ablation of our full method. In this case, aligning experimental settings is essential to draw valid conclusions about the contribution of each component.\n\nWe hope this clarifies our design choice and assures the reviewer that our methodology is both principled and consistent with field-wide practices.\n\n\n> ### Q3. CMMD metric. \n\nThank you for pointing out this metric. Following the suggestion, we compute the CMMD scores for all our image generation experiments. The images used for computing the metrics are the same as described in **Q1**.\n\nThe results are shown below:\n\n| Method   | Case Study 1: CIFAR10 | Case Study 1: AFHQ-Cat | Case Study 1: CelebA-HQ | Case Study 1: LSUN-Bedroom | Case Study 3: CIFAR10 |\n|----------|------------------------|--------------------------|---------------------------|-----------------------------|------------------------|\n| RF       | 0.0360                 | 0.5145                   | 1.0276                    | 0.5218                      | 0.0253                 |\n| RF+LZN   | 0.0355                 | 0.3376                   | 0.4901                    | 0.4843                      | 0.0229                 |\n\nWe can see that **RF+LZN consistently outperforms RF in all cases**. These results further support our claim that **LZN improves generative modeling quality across a variety of datasets**.\n\nWe will add the results to the revision.\n\n> ### Q4. Why LZN Can Improve Image Quality\n\nThank you for the question. Below, we explain why applying LZN on top of a generative model can improve image quality.\n\nGiven a dataset, the goal of a generative model (Rectified Flow in our case) is to learn the distribution $p(x)$ of samples $x$. The entropy (or diversity) of this distribution affects the difficulty of learning. For example,  \n- If $p(x)$ is highly deterministic (i.e., low entropy), it is easier for the model to learn.  \n- If $p(x)$ has a diverse support (i.e., high entropy), learning becomes more challenging.\n\nLZN helps by introducing a **strong conditioning signal** that reduces the effective entropy of the target distribution. Specifically, LZN maps images into structured regions of a latent space (see Section 2), where each latent code $z$ corresponds to a specific image or visually similar images (when using minibatch approximation in Appendix A.3). As a result, when having this z as an condition, the generative model only needs to learn $p(x|z)$ — a much simpler, more concentrated distribution — rather than the full, high-entropy distribution $p(x)$. This makes the learning problem easier, which leads to higher-quality image generation.\n\n> ### Q5. How to reconstruct images using RF, and why LZN reduces reconstruction error a lot\n\n- **How to reconstruct images using RF.**  \n  This process is described in Appendix C.3 (lines 1020–1026). Rectified Flow (RF) defines an ODE that maps between the data space (images) and a Gaussian latent space (note: this latent space is *different* from the LZN latent space). To reconstruct an image:\n  1. We first apply the **inverse ODE** to map the image to its corresponding Gaussian latent representation.\n  2. Then, we apply the **forward ODE** to map this latent back to the image space.\n\n  This reconstruction procedure is a standard technique already described in the original Rectified Flow paper (e.g., see Figure 12 therein), not something we introduced.\n\n- **Why LZN reduces reconstruction error.**  \n  This follows the same intuition as discussed in Q4. LZN assigns a unique latent $z$ to each image, thereby capturing the dominant variation in the data. As a result, the conditional distribution $p(x|z)$ that RF needs to learn becomes simpler and lower in entropy. This reduction in complexity allows the model to reconstruct images more accurately from their latent representations.\n\n> ### Q6. “The paper lacks a more complete evaluation and discussion of the results, for example between LZN and Autoregressive methods.”\n\nWe apologize, but we do not fully understand the question, as our current implementation does not rely on autoregressive methods. If the reviewer could point us to a specific paper or work related to this concern, we would be happy to review it and include it in our discussion.\n\n> ### Q7. Variable length latent spaces\n\nThis is a super insightful and interesting question! Our current approach does not support variable-length latent spaces. To handle token sequences of variable length, our current idea is to embed them into a fixed-dimensional vector, which can then be modeled with LZN. This is feasible given recent advances such as [arXiv:2310.06816], which show that variable-length sequences can be effectively encoded into fixed-size embeddings while preserving semantic information. We believe this is a promising direction and plan to explore it in future work.\n\n\n> ### Q8.  Table/figure formats.\n\nThank you for catching these formatting issues and the suggestions. We will correct them in the revision."}, {"review_id": "a9ENDuTosm", "reviewer": "Reviewer_9ZsW", "summary": "The manuscript proposes a unified framework for image generation, classification, and representation learning. The core concept behind the proposed framework is to model a structured modality-invariant latent space that organises semantic concepts into distinct regions. Then, the considered tasks could be solved by mapping inputs into the appropriate latent regions, optionally followed by decoding into the desired output format with a task-specific decoders. In practice, the proposed framework is implemented with two atomic operations: latent computation and latent alignment. Latent computation is performed by first mapping inputs into anchor points with a deterministic encoder. Then,  the anchor points are used to construct a distribution that is essentially a mixture of Kronecker deltas in the anchor points. This distribution is finally mapped into a Gaussian prior with distinct semantic regions by using flow matching. The latent alignment is conducted by minimising the distance between the latent representations of each element in the input pair while maximising the distance to other examples in the batch. This is performed along the flow matching trajectory. The initial experimental evaluation indicates that the proposed framework improves over the standard image generation with flow matching, while delivering representation learning and image classification that beats some of the baselines.", "questions": "See weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n\nS1. A single framework that solves core machine learning tasks is an interesting research topic.\n\nS2. Optimising for a well-structured latent space conceptually makes sense.\n\nS2. The proposed framework enables solving multiple tasks simultaneously.\n\n**Weaknesses:**\n\nW1. The manuscript presentation is disorganised and packed. For example, Section 2.1 is unnecessarily elaborate and fits more in a separate Related Work section. Some sections have 2 lines (see Section 2.3). Also, margins of (sub)sections are significantly reduced to fit the text. Instead of template manipulation, I recommend rewriting manuscript parts.\n\nW2. The presented experimental evaluation is unconvincing since 2/3 case studies reveal a significant performance gap compared to SotA task-specific methods. Thus, the proposed framework may not be a step in the right direction or may require further refinement.\n\nW3. The manuscript compares only to task-specific baselines. Comparison with previous works (or appropriately adapted baselines) that deal with task unification may strengthen the contributions.", "was_revised": true, "final_justification": "The manuscript proposes a unified framework for image generation, classification, and representation learning, which is an interesting research goal. My initial concerns regarding experimental evaluation have been successfully resolved. I still believe more effort has to be made in terms of paper presentation. Nonetheless, this concern is overpowered by the manuscript's strengths.", "rebuttal": "We sincerely appreciate your thoughtful and constructive feedback. Below, we provide detailed responses to each of your questions.\n\n> ### Q1. Performance.\n\nThank you for raising this important point. We would like to clarify our perspective and provide additional context.\n\n- In **Use Case 1** and **Use Case 3**, **LZN already outperforms the state-of-the-art in generative modeling** (see Tables 1 and 4). In **Use Case 3**, LZN also achieves classification accuracy that is **within 1% of the best-performing method**, despite being designed as a **unified model for both generation and classification**. In contrast, the baselines are optimized specifically for classification, not for multiple tasks.\n\n- For **Use Case 2**, which is the only case where LZN was initially behind by a larger margin, we are happy to report that **our updated results significantly improve upon the submission**. After extending training to 4.5 million iterations, the top-1 accuracy increases to **69.3%** (+3.7%) and top-5 accuracy to **89.1%** (+2.0%). With these results, LZN is now among the **top 4 methods** on this benchmark. Importantly, these competing methods are designed **exclusively for representation learning**, whereas LZN supports **more tasks including generation**. Notably, LZN also **matches or outperforms seminal methods such as SimCLR**. We believe these are strong results.\n\n- Finally, we would like to emphasize our broader perspective. **Even if one considers only Use Case 1, LZN already advances the state-of-the-art in image generation, which we believe is a substantial and meaningful contribution on its own.** Beyond that, LZN is a **general-purpose framework** capable of solving both generative and discriminative tasks within a unified design—yet it still performs competitively with specialized, task-specific models. In our view, this generality combined with strong performance across domains goes above and beyond typical expectations of papers. \n\n  We respectfully believe that emphasizing a modest performance gap in one setting, while overlooking areas where LZN achieves top results, would not offer a balanced or fair assessment.\n\n\n> ### Q2. Comparison with baselines that can deal with task unification\n\nThank you for the helpful suggestion. In our experiments, we have chosen to compare with the **state-of-the-art methods for each benchmark**, which we believe sets a **higher bar** than comparing against older or weaker baselines, regardless of whether they support multiple tasks. \n\nThat said, we are happy to include additional unified-task baselines in the revision—especially if the reviewer has any specific suggestions in mind. We greatly appreciate such pointers that can help us strengthen the comparisons.\n\n\n\n> ### Q3. Organization and formatting.\n\nThank you for the thoughtful suggestions. We agree that clarity and structure are important, and we appreciate your perspective.\n\nRegarding Section 2.1, we did consider alternative placements during the writing process. Ultimately, we decided to keep it early in the paper for the following reasons:\n\n- **Motivational context**: Section 2.1 explains why existing frameworks fall short of achieving our goal of a unified model for generation, representation learning, and classification. Without this context, readers may not fully appreciate the need for our proposed approach.\n- **Conceptual foundation**: It also provides essential background—such as the roles of encoders, decoders, and latent spaces—that our method builds upon. Skipping this section would make the introduction of LZN less accessible to a broader audience.\n\nThat said, we understand the concern about length and will work on **tightening the section** to improve flow and focus.\n\nWe will also address the other formatting issues you kindly pointed out in the revision. Thank you for catching that!"}]

Author rebuttal:

Meta-review: The rebuttal addressed the concerns raised by the reviewers and provided comprehensive experiments and analysis of the proposed method. Two reviewers provided Accepts, and one rated Borderline Accept. The Reviewer PF16 rated Borderline Reject, but didn’t interact with the authors. The authors actively provided detailed answers. The final version should include all reviewer comments, suggestions, and additional experiments from the rebuttal.
Decision: Accept (poster)