Paper: Multi-View Causal Representation Learning with Partial Observability (ICLR-2024)

Reviewer comment (Theory):
For definition 5.3, why assume l0 to be the same and not the two vectors are the same? For a given view, the content variables are fixed, right?

Author response:
### *Question 1:  why not assume two vectors are the same?*

  - **The shared content can be indexed at different positions**, especially when the number of views grows.  Since we consider a partially observable setup, each observation is generated by a subset of latents and, in general, does not have access to all of the latents. 

  + **For example**, say there are three underlying latent variables $z_1, z_2, z_3$. and two observations $x_1=f_1(z_1, z_2)\, x_2 = f_2(z_2, z_3)$. In this case, the optimal selecting vectors for each should be $\phi_1 = (0, 1), \phi_2 = (1, 0)$, as different views are generated by different latents requiring different indexing.

Reviewer comment (Experiments):
Could you provide a little more motivation as to why view-specific encoders are needed? I understand the theoretical reason. But in practice, one would expect to train with all views available, right?

Author response:
### *Question 2: motivation of view-specific encoders*
  - **Our work focuses on a multi-view and potentially multimodal setup**, where data are potentially collected by different measuring devices. Each view with a different modality would require a separate encoder. Even if two views are from the same modality, we can only share parameters if they measure the same latents.

  + We remark that the **joint intersection** (content by considering all available views) **differs from the intersection of all subsets of views** (and could be empty). **For example**, we have three partial views $x_1 = f_1(z_1, z_2)$,  $x_2 = f_2(z_2, z_3)$, $x_3 = f_3(z_1, z_3)$. In this case, the **joint content is empty** while the pairwise intersection would give us the latent variables of interest: e.g. we can identify $z_2$ from $(x_1, x_2)$; $z_3$ from $(x_2, x_3)$ and $z_1$ from $(x_1, x_3)$.

Reviewer comment (Theory):
The assumption that each view-specific generating function is invertible is strong, especially over multiple modalities. Intuitively, this requires the shared blocks to be duplicated multiple times and thus simplifies the identification problem. A recent work [1] has attempted to weaken this assumption.

Author response:
+ We also thank the reviewer for the reference, which we will discuss in the revised version. **The key difference** is that  [1] assumes structural conditions on the underlying causal graph (Condition 2.4) whereas our work **does not assume anything about the underlying causal structure**. Overall, we completely agree that adding structural assumptions while weakening the assumption on the mixing functions is an interesting direction for future exploration.

Reviewer comment (Novelty):
The recent work [1] also uses contrastive learning in CRL. Are there any resemblances to their approach, i.e. how do their loss function compare to content alignment?

Author response:
- **Relation with [1]**: Although [1] also focus on identifiability in causal representation learning, their setting is completely different in that they assume (unknown) **single-node** interventions which we believe is a very strict assumption. They generalize prior work that uses single-node interventions in the class of mixing functions. *Their cross-entropy loss is related to our alignment and entropy terms*. Incorporating the NOTEARS objective is something we did not explore and may be interesting to combine the causal representation learning with causal discovery. We will add a citation in the revised manuscript, highlighting this connection as interesting future works.

Reviewer comment (Novelty):
The works [3], [4] also study nonlinear ICA without auxiliary variables. I believe they use variants of VaDEs in their experiments, can this also be viewed as a soft alignment inductive bias?

Author response:
+ **Relation with [3-4]**: We fully agree with the reviewer that there is a connection with non-linear ICA with auxiliary variables, which we discussed in section 4, paragraph 4. We will add these references in this section for the next revision.

Reviewer comment (Theory):
Was this necessary in order to unify a wider range of existing identifiability results?

Author response:
We thank the reviewer for the positive feedback, and the compliment about the significance of the paper. In the following, we list the **practical benefits** of simultaneously identifying all the blocks:

- Our **intuitive motivation** is that we can measure the state of a system using multiple different sensors, each telling us something different. At the same time, we are interested in capturing the state of the system via high-level variables, and each block corresponds to a macro-variable (whose granularity depends on how fine-grained our sensors are). We believe this viewpoint will broaden the applicability of causal representation learning to more realistic and practical settings.

+ This is also **important for unifying results** from the disentangled representation literature, where the goal is to recover all factors of variation. With this, we can perfectly cover the theoretical results of (Locatello et al., 2020, Ahuja et al., 2022).

- Additionally, we provide an interpretation for recent disentanglement work (Fumero et al., 2023) that **scales to real-world data** in settings where factors of variations are not independent.

+ Overall, we maintain that this setup is very flexible, covers important special cases, already explains some practical algorithms and we hope will lead to more.

Reviewer comment (Evaluation):
If so, can you give a specific example how this generality helped?

Author response:
We thank the reviewer for the positive feedback, and the compliment about the significance of the paper. In the following, we list the **practical benefits** of simultaneously identifying all the blocks:

- Our **intuitive motivation** is that we can measure the state of a system using multiple different sensors, each telling us something different. At the same time, we are interested in capturing the state of the system via high-level variables, and each block corresponds to a macro-variable (whose granularity depends on how fine-grained our sensors are). We believe this viewpoint will broaden the applicability of causal representation learning to more realistic and practical settings.

+ This is also **important for unifying results** from the disentangled representation literature, where the goal is to recover all factors of variation. With this, we can perfectly cover the theoretical results of (Locatello et al., 2020, Ahuja et al., 2022).

- Additionally, we provide an interpretation for recent disentanglement work (Fumero et al., 2023) that **scales to real-world data** in settings where factors of variations are not independent.

+ Overall, we maintain that this setup is very flexible, covers important special cases, already explains some practical algorithms and we hope will lead to more.