Title: Disentangled Representation Learning via Modular Compositional Bias
Venue: NeurIPS 2025
Accepted: yes
Keywords: Unsupervised Representation Learning

Disentangled Representation Learning

Compositionality Maximization
Abstract: Recent disentangled representation learning (DRL) methods heavily rely on factor-specific strategies—either learning objectives for attributes or model architectures for objects—to embed inductive biases. 
Such divergent approaches result in significant overhead when novel factors of variation do not align with prior assumptions, such as statistical independence or spatial exclusivity, or when multiple factors coexist, as practitioners must redesign architectures or objectives. 
To address this, we propose a compositional bias, a modular inductive bias decoupled from both objectives and architectures. 
Our key insight is that different factors obey distinct "recombination rules" in the data distribution: 
global attributes are mutually exclusive, *e.g.,* a face has one nose, while objects share a common support (any subset of objects can co-exist). We therefore randomly remix latents according to factor-specific rules, *i.e.,* a mixing strategy, and force the encoder to discover whichever factor structure the mixing strategy reflects through two complementary objectives: 
(i) a prior loss that ensures every remix decodes into a realistic image, and (ii) the compositional consistency loss introduced by Wiedemer et al., which aligns each composite image with its corresponding composite latent. 
Under this general framework, simply adjusting the mixing strategy enables disentanglement of attributes, objects, and even both, without modifying the objectives or architectures. Extensive experiments demonstrate that our method shows competitive performance in both attribute and object disentanglement, and uniquely achieves joint disentanglement of global style and objects.
Code is available at https://github.com/whieya/Compositional-DRL.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel, general method for disentangled representation learning (DRL), with the lens of (attribute and object) compositionality. The proposed compositional bias does not rely on leveraging specific attribute properties (such as, e.g., the geometry of the factors) and, thus, is very flexible. This is demonstrated by extensive experiments.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper is well written, it's high-level message easy to follow (thanks to Fig. 1 and the contribution list)\n- The experiments are extensive (and show superior performance or comparable to SOTA)\n- The authors address their works limitations (no theoretical guarantees, no handling of unknown factors)\n\n## Weaknesses\n\n### Major points\n**My main concern is the similarity (and even the same naming) to Wiedemer et al., 2024, which is not discussed in the paper (it is reference number 45, which is only briefly mentioned in the appendix, without a substantial discussion**. Especially that said paper has theoretical guarantees. I do notice that this paper has more extensive experimental evidence, and the formulation of the mixing strategy seems to be novel; however, that does not eliminate the need of the comparison to Wiedemer et al., 2024. I consider addressing this point as the necessary condition for the re-evaluation of my score. \n\n- **Abstract:** even though I work on compositional generalization, I could not really figure out the contributions from the abstract (particularly the mixing strategies part). After reading the paper, it became clear. I'd suggest rephrasing it (I am happy to iterate over suggestions during the rebuttal)\n- **Object and attribute latents:** please distinguish in the notation of the two types of latents better (I get that the super-/subscipts aim to indicate that). Particularly, I believe (2) might be incorrect: you cannot pick any object atributes (say, you pick the object color twice, but no shape). if this is excluded, please make it clearer.\n- (5): this formulation seems to be the same/very similar (depending on $d$) to the InfoNCE objective. Please refer to that to establish the connection\n\n### Minor points\n- L91: spatially non-exclusive factors could be a bt misleading, maybe consider \"global\" (my guess based on the exampels you provide)\n- L108: define $\\theta, \\phi$\n- **Limitations:** though the product space assumption is prevalent in the compositionality literature, all compositions might not imply valid configurations (e.g., hallucinations, or placing anachronistic objects in an image, e.g. a television into a medieval scene). I'd add this to either the limitations or to L52\n- L61: please reference the DCI paper\n- (4) - why do you calll it $\\mathcal{L}_{prior}?$ I believe it refers to the diffusion model, so maybe spell this out or call it the \"diffusion loss\" or something similar\n- L220 (typo?): do you mean the image decoder? I don't understand how an encoder would take features\n- Fig. 2: please add more information to the caption to make it self-contained (at least refer to your description in L255 and onward)\n- Fig. 3: the image shows $\\emptyset,$ but the caption shows $\\phi$ (al;so the green color is hard to see)\n- **Metrics**: could you please add a short description regarding how your metrics (FG-ARI mIoU, mBO, GRAM, etc) are defined? It's fine in the appendix\n- Could you plase add emphasis (bold/underline + explanation) to Tab. 4?\n\nQUESTIONS:\n1. Do you have an intuition why your method works worse on Cars3D?\n2. Could you please spell out with formulas/pseudocode your matching technique from A.7?\n3. For your baseline models, how much hyperparameter tuning did you do?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-23"}

comments: SUMMARY:
This paper learns disentangled representations of attributes and objects through self-supervised methods, which is a highly meaningful and interesting research direction. By designing different mixing strategies and learning strategies, the method achieves highly effective spontaneous decoupling, and demonstrates relatively obvious advantages over existing methods in multiple metrics across various datasets. Previous Disentangled Representation Learning was mainly based on the Slot Attention framework. This work designs new learning objectives and architectures, achieving better performance in some aspects, which I consider highly innovative and appealing.

STRENGTHS AND WEAKNESSES:
Strengths:
1. Propose attribute mixing strategy and object mixing strategy, which is the key to achieve attribute and object disentangle.

2. Propose novel learning objectives and the experimental results show its effectiveness.

3. Paper is well-organized, and the experiments are comprehensive.


Weeknesses:
1. Objective function is relatively complex, which may limit its potential and scaling to more data.

QUESTIONS:
Question: I would like to ask whether an object is represented by a single latent or multiple latents. My understanding is that the dimensions of the representation are grouped, and an object has multiple attribute representations. I wonder if my understanding is correct. If so, does it mean that specific channels of the representation encode specific attributes?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a disentanglement framework that decouples factor-specific inductive biases from learning objectives and architectures, enabling both attribute and object disentanglement under a single set of objectives and architectures. The framework derives mixing strategies as a compositional bias, embedding factor-specific biases in a modular way, and proposes specific strategies for attribute, object, and joint disentanglement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths \n\tThe concept of separating attributes and object disentanglement is straightforward but significant for the DRL domain.\n\tThe Mixing Strategy is clearly articulated, with well-defined designs for losses and objectives.\nWeaknesses\n\tTraining a DRL model using the commutative principle within Group theory is a common approach. This method exchanges specific factors between two different images or objects for reconstruction, encouraging the model to learn the attributes of each specific factor. The authors are encouraged to clearly highlight the differences between this work and existing Group-based DRL methods to demonstrate the novelty of their approach.\n\tIn lines 90-93, the authors claim distinctions between their object disentanglement strategy and other object-centric DRL models. To support these claims, more visual qualitative comparisons with state-of-the-art object-centric DRL models should be provided in Section 4.\n\tAlthough quantitative results on attribute disentanglement are presented in Table 1, a visual comparison with baseline methods is still necessary to validate the superior performance of the proposed model. This could be included as an extension of Figure 2, showcasing comparisons with baselines. \n\tThe experiments are currently conducted only on toy datasets such as CLEVR and Shape3D, which limits the contribution of this work to the DRL community. Given that the study aims to disentangle attributes and objects simultaneously, the authors should consider evaluating their method on real-world multi-object datasets to demonstrate its practicality.\n\nQUESTIONS:\nAccording to the weaknesses I previously highlighted, addressing the following points could potentially change my opinion:\n\tA thorough discussion clarifying the essential differences between the proposed “mixing strategy” and the commutative principle or Group-based DRL training methods.\n\tMore convincing visual comparisons with baseline models to substantiate the claims.\n\tExperimental results conducted on real-world multi-object datasets to demonstrate practical applicability.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is about disentangling attributes and objects in an image with a common framework. The idea is to optimize image reconstructions by mixing factors in the latent. The approach is assessed on public datasets and compared with existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper represents a nice reading, overall very clear (with few exceptions). The proposed idea shows a good level of novelty, and it may have a potential impact on different tasks where managing interpretable representations can be an add-on.\n\nOn the weaknesses, I want to mention the following points:\n- The introduction is not fully clear, and it fails to appropriately prepare the reader for the rest. Maybe, examples to give the intuitions behind attribute and object disentanglement could help to shape the story\n- On the related works, in my understanding, the class of disentanglement methods relying on weak-supervised (e.g. [1]) is not discussed. However, such methods may have the potential to address the challenges in this paper and might be of interest. \n- Again, on the related works, it would be useful to clarify the difference between the concepts of \"object disentanglement\" and \"object-centric learning\". To the unfamiliar reader, the impression is that the terms are sometimes used as synonyms\n- On the training, I am not sure I fully got the details. For instance, learning the disentanglement model adopted later for mixing is a step included in the training of the proposed approach, or is the idea to rely on pre-trained models?\n- The experiments are nice, but only refer to synthetic data, while in more general (possibly real) scenarios, conditions are more challenging  (for instance, we may have FoVs distributed on more than one component of the latent, or mixed). Some clarifications on the generalisation would help \n- Again, on the experiments, I think that weakly-supervised methods should be taken into account\n- Regarding the joint disentanglement, I have a couple of doubts. The first one is that while this is the core contribution of the paper, it is evaluated on one dataset only, and this is somehow a limitation. The second doubt refers to the fact that, as far as I understand, only one style attribute is considered, against multiple object-based properties. This leads me to another doubt that I have: in Shapes3D, why are we not considering floor and wall colours as style attributes? A clarification would help appreciate the difference\n- At different points, it is stated that the method can freely choose model architectures. If I'm corretly interpreting the statement, this would require showing experiments involving at least another architecture, to show this ability\n\n\n[1] https://arxiv.org/pdf/2002.02886\n\nQUESTIONS:\nIn addition to or as a complement to what was reported in the weaknesses section, I would like the authors to clarify the following points:\n- How does the field of object-centric learning relate to object disentanglement? Somehow, they seem to be used as synonyms\n- How does the proposed method compare to weakly-supervised learning approaches? Or, in alternative, why have such methods not been considered?\n- The differences with [20] (ref. in the main paper) should be more clearly stated. For instance, they also use an encoding-decoding (a UNet)\n- Rows. 137-139: I find this example misleading. Isn't this more related to object disentanglement, since it reasons on object parts?\n- Row 162: Is the separation of attribute types in the latent forced at training time? In general, some more details on the training would be appreciated\n- Row 186: Is the final targeted downstream task image generation?\n- Eq. (5); Shouldn't we have an average on the bottom part of the fraction? As it is now, my impression is that the numerator and denominator are not numerically comparable\n- Sec. 4.1 and 4.2: I am not sure what version of the method we are talking about here. For instance, are we evaluating in Sec. 4.1 the relevant portion of the latent (i.e. the first M attributes) jointly learnt with the rest, or is it a \"partial\" learning of only that part?  \n- As already mentioned, although what is presented in the experiments on joint disentanglement is nice, the fact that only one dataset and only one style attribute are used should be justified, as it looks like a limitation\n\nLIMITATIONS:\nIt is mentioned in Sec. 5 that limitations are discussed in the Appendix. However, there is no Appendix to the paper, while the information can be found in the Supplementary Material.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}