Paper: What If You Were Not There? Learning Causally-Aware Representations of Multi-Agent Interactions (ICLR-2024)

Reviewer comment (Theory):
The proposed method is highly confined to the simulator. As the author already noted, counterfactual data is impractical to collect in real world scenario and they instead took a sim-to-real transfer learning approach. But the assumption that the underlying causal mechanisms stay the same is simply too fragile to be true. For example, in simulation, the author define the visibility range as 210 degree in front of the driver ignoring the fact that drivers also check rearview mirrors for road information. And the simulator itself focuses heavily on collision avoidance. However, human drivers are not perfectly rational and have various driving preferences. Instead of “causally-aware” representations, it’s more accurate to say the proposed method learns “collision-aware representations”. Rather than assuming “the causal mechanisms” satisfying some vague constraints, I would recommend the authors model the problem more rigorously with causal theories [1,2].

Author response:
* We acknowledge your concern and clarify that we do not assume all causal mechanisms stay the same. Instead, we assume they differ between the simulation and real-world in a sparse way (cf. sparse mechanism shift [1,2]). For the mechanisms shared across the two domains, we can first annotate them in the simulation and further transfer them to the real-world.
* This assumption aligns well with our considered setting, where the simulator is designed and calibrated to mimic pedestrian behaviors (not drivers) in the real-world, including a 210-degree field of view to match human visual perception [3].
* To the best of our knowledge, existing research in causal representation learning is still confined to synthetic/simulated settings, and struggles to scale to the real-world. Our proposed sim-to-real causal transfer provides a new perspective to bridge this gap and may hold great promise in broader settings beyond the considered application.

> From the current table, it seems that "ranking" is still not completely beating the baselines

* We appreciate your attention to the detailed results in our table. To clarify, our proposed ranking method yields consistent improvement over the baseline. In particular, the Autobots [1] trained by our method have shown comparable performance to the current state-of-the-art, EqMotion [2], on the ETH-UCY dataset, as detailed in Table 5 of our manuscript.
* Additionally, we expand our evaluation to include training on various proportions (25%, 50%, 100%) of the ETH-UCY dataset. The results, averaged over three random seeds and summarized in Fig 11&12 of our updated manuscript, corroborate with our findings earlier: the proposed ranking-based causal transfer significantly boosts sample efficiency, enabling the model to reach comparable or even better accuracy with significantly less real-world data.

> It seems that the sparse mechanism shift is defined under the Causal Markov Condition, which basically means that there will be no unobserved confounders. I am not sure if it is realistic to model the pedestrian behaviors assuming that there are no spurious correlations among them.

* To our knowledge, while the principle of sparse mechanism shift was derived from the distribution of the observables, its implication lies in more general properties of distribution shifts: `real-world distribution as a product of causal mechanisms. A change in such a distribution (e.g., when moving from one domain to a related one) … should usually not affect all mechanisms simultaneously` [3].
* We hypothesize that this principle applies to our considered context, even in the presence of unobserved confounders. In fact, we explicitly motivated our approach from the perspective of spurious correlations, as detailed in the social causality paragraph in Section 3.1, and designed OOD settings (e.g., density increase) to test this hypothesis.
* Finally, as stated in our manuscript, we consider our work as a first step towards causal representations of multi-agent interactions in practical settings. We acknowledge that our work is still subject to several limitations, e.g., a gap between our proposed sim-to-real causal transfer method and more formal causal modeling of the domain shift. Addressing these limitations, including formalizing and modeling indirect causal effects that existing representations still struggle to capture, can be fascinating directions for future research.

Reviewer comment (Evaluation):
Even if we only consider the collision-free driving style, the proposed method’s effectiveness is not convincing enough. Firstly, if you have done multiple runs with different random seeds for the same experiment, I am expecting to see confidence intervals or error bars in the results.

Author response:
> Could you extend the experiments to multiple random seeds and report the variances in the figures? 

Thank you for the suggestion. 
* We have updated Figure 6 in our manuscript to report the variance across 5 random seeds. 
* We are still running the experiments on different seeds for Figure 8. Below are the preliminary results on the full (100%) ETH-UCY dataset. We anticipate completing the experiments by next week and will update Figure 8 accordingly.

|             | ETH                       | Hotel                     | Univ                      | Zara1                     | Zara2                     |
| ----------- | ------------------------- | ------------------------- | ------------------------- | ------------------------- | ------------------------- |
| Baseline    | 0.932±0.02 / 1.870±0.032  | 0.322±0.008 / 0.646±0.026 | 0.540±0.005 / 1.154±0.006 | 0.420±0.002 / 0.909±0.006 | 0.344±0.004 / 0.766±0.004 |
| Ranking | 0.907±0.007 / 1.825±0.014 | 0.308±0.006 / 0.581±0.021 | 0.534±0.001 / 1.143±0.001 | 0.410±0.002 / 0.893±0.003 | 0.327±0.001 / 0.718±0.005 |

Reviewer comment (Presentation):
The author introduced three metrics for evaluation, Average Displacement Error (ADE), Final Displacement Error (FDE), and Average Causal Error (ACE). However, figure 6 is missing FDE and figure 8 is missing ACE.

Author response:
> Could you report all three metrics for figure 6 and 8?
* We’d like to clarify that it is not possible to evaluate ACE for Figure 8, due to the absence of causal annotations in the real-world ETH-UCY dataset.
* We have added FDE for Figure 6 into the appendix (Figure 9). We didn’t report it in the previous version, because the pattern of FDE is similar to that of ADE and does not provide much additional information.

Reviewer comment (Experiments):
There are notable improvements of the “rank” method over baselines, but it is unclear if it’s a fair comparison because the “rank” method actually provides way more labeling information than binary labels (causal/non-causal). It is uncertain whether those baselines also have access to such labels.

Author response:
> We do see notable improvements of the “rank” method over baselines but I am not sure if it’s a fair comparison because the “rank” method actually provides way more labeling information than binary labels (causal/non-causal).
* We’d like to clarify that the goal of the experiment is not about which method is better given the same annotations. Instead, we aim to examine the relative benefits of fine-grained annotations over coarse-grained labels, as noted at the beginning of the experiment section (Q3). 
* Our result shows that learning from fine-grained annotations through a ranking regularizer yields substantial improvements over the other counterparts. This result provides valuable guidance for causal annotation strategies in simulation and real-world scenarios.

Reviewer comment (Evaluation):
When using 100% data in the sim-to-real test, the proposed method (contrast) barely beats the baselines, leaving doubts on whether the performance gaps in 25% and 50% scenarios are due to learning efficiency differences instead of representation quality.

Author response:
> At last, when using 100% percent data in the sim-to-real test, the proposed method (contrast) barely beats those baselines leaving doubts on whether the performance gaps in 25% and 50% scenarios are due to learning efficiency difference instead of representation quality.
* We appreciate your observation regarding the performance gaps in different data scenarios. We’d like to clarify that we do not claim superiority of our contrastive variant over the augment baseline; their performance is comparable across most experiments. The main finding we’d like to highlight is the strength of our ranking-based variant, including its high learning efficiency in the low data regime, e.g., reaching similar or even better accuracies with 1/2 less of the real-world data.

Reviewer comment (Experiments):
In the annotation process of the proposed dataset, each $i$-th agent is labeled as non-causal, direct causal, or indirect causal based on the influence the ego agent $\mathcal{E}_i$, i.e., measured by removing a *single agent*. However, the causal effect of the agent (to the ego agent) could be different based on the other agent (as Fig 2 illustrates). For example, for indirect causal agents $i, j$ (i.e., $\mathcal{E}_i, \mathcal{E}_j \gg 0$), it is possible that removing both of them does not influence the ego agent, i.e., $\mathcal{E}_{ij} \simeq 0$. For another example, it is also possible that $\mathcal{E}_{ij} \gg 0$ when $\mathcal{E}_i, \mathcal{E}_j \simeq 0$ (e.g., when there are two bikes in the same lane in Fig 2, removing only one of them does not influence the ego agent but it does when removing them together).

Author response:
* Indeed, the causal effect in the multi-agent context can be studied at both the individual agent level (removing one agent at a time) and the group level (removing a group of multiple agents). Our focus on agent-level causal effects was motivated by two factors:
  * It aligns with the notion of Causal Agents, i.e., a neighboring agent has a certain causal relationship with the ego agent. This allows us to revisit the root cause of the robustness issues in the prior benchmark.
  * It permits efficient experiments, i.e., studying causal effects at the agent level presents a linear computational complexity O(N), whereas studying that at the group level could lead to an exponential increase in complexity, up to O(2^N).
* Having said that, we acknowledge that exploring the latter can be an exciting avenue for future research. We have updated our manuscript to reflect this perspective, framing our finding about `partially robust` in the context of agent-level causal effects and discussing its limitation in Appendix C.

Reviewer comment (Experiments):
The paper only considers a single dataset (ETH-UCY), and the proposed regularization strategy is only applied to a single backbone (AutoBots), which makes it difficult to assess its wide applicability. In other words, it is unclear how the proposed regularization strategy would work on other datasets and with other backbone methods.

Author response:
* We are currently running experiments on another backbone (D-LSTM) benchmarked in Tab 1. Our preliminary result shows larger performance gains on D-LSTM than on AutoBots. We anticipate completing the experiments soon and will update our response accordingly.

Reviewer comment (Experiments):
The paper lacks a detailed description of the proposed diagnostic dataset based on ORCA (in both the main body and the appendix). For example, what is the difference between OOD-Context and OOD-Density-Context? (They seem to be both dense.) Maybe a figure or illustration would be much appreciated. Also, as the one who is not directly working in this field, it is hard for me to understand the similarity and difference between ORCA simulation and ETH-UCY dataset, and thus it is difficult to assess how reasonable the proposed causal transfer strategy is.

Author response:
* The key difference between OOD-Context and OOD-Density-Context is that the former introduces major changes in scene contexts, whereas the latter introduces more significant increases in the number of causal agents as well as modest changes in scene contexts.
* As per your suggestion, we have added a more detailed description in the `dataset details` paragraph in Appendix B, along with animations in [our public repository](https://github.com/socialcausality/socialcausality#illustrations) for better illustration.

Reviewer comment (Evaluation):
The main issue I have is that the proposed approach seems to provide marginal improvements across all metrics. Perhaps this is down to the point made by the authors, that all methods are already pretty strong for non-causal agents, but not so much for causal agents.

Author response:
* We would like to highlight that our method was evaluated on a state-of-the-art baseline, where significant leaps in performance are inherently challenging. Despite this, our ranking-based method yields ~10% performance gain on several metrics, e.g., ACE in the low-quantile causal effect scenarios and ADE in the OOD density and context settings. Most notably, the final version of our method enables the model to reach better accuracy while requiring only half the real-world data.

Reviewer comment (Evaluation):
I think it would be much more interesting if the ACE metrics in Table 1 were replicated with the proposed approaches, showing how these values (particularly ACE-DC/IC) are presumably improved with the promotion of causal invariance.

Author response:
* The quantile split was chosen to better understand the strengths and limits of the proposed method in causal effect estimates. As shown in Fig 6a, the improvement over ACE is greater with smaller ground-truth causal effects. We conjecture this is in large part due to the imperfect decoding process.
  * When the causal effect of the neighboring agent is small (affects the ground-truth trajectory only locally), encoding the paired scenes into nearby embeddings results in similar prediction outputs and hence small ACE, canceling out the influence of the decoding error. 
  * In contrast, when the causal effect of the neighboring agent is large (affects the ground-truth trajectory at macro scale), even if encoding the paired scenes into properly separated embeddings, the decoding errors of each trajectory may compound and lead to large ACE. 
* Following your suggestion, we have extended our evaluation to include ACE-DC and ACE-IC metrics. The results are summarized in Figure 10 in our updated manuscript. Our proposed method yields comparable improvements on the two metrics. We conjecture this is because our proposed method treats DC and IC agents uniformly.

Reviewer comment (Experiments):
Why is the split done using quantiles here instead of splitting across the ACE-X metrics?

Author response:
* The quantile split was chosen to better understand the strengths and limits of the proposed method in causal effect estimates. As shown in Fig 6a, the improvement over ACE is greater with smaller ground-truth causal effects. We conjecture this is in large part due to the imperfect decoding process.
  * When the causal effect of the neighboring agent is small (affects the ground-truth trajectory only locally), encoding the paired scenes into nearby embeddings results in similar prediction outputs and hence small ACE, canceling out the influence of the decoding error. 
  * In contrast, when the causal effect of the neighboring agent is large (affects the ground-truth trajectory at macro scale), even if encoding the paired scenes into properly separated embeddings, the decoding errors of each trajectory may compound and lead to large ACE. 
* Following your suggestion, we have extended our evaluation to include ACE-DC and ACE-IC metrics. The results are summarized in Figure 10 in our updated manuscript. Our proposed method yields comparable improvements on the two metrics. We conjecture this is because our proposed method treats DC and IC agents uniformly.

Reviewer comment (Presentation):
In Figure 8, what is the difference between baseline and vanilla?

Author response:
* The baseline refers to training the model on real-world data only, whereas the vanilla sim-to-real refers to training the model on simulated and real-world data jointly. In our updated manuscript, we elaborate on the details of each method in the `baseline details` paragraph in Appendix B.

Reviewer comment (Presentation):
In relation to the previous point, I think all Figure captions should include a legend telling the reader the different variants. For example, Figure 6 has both a baseline and augment. But the text says that data augmentation is a baseline. This feels a little ambiguous. If I understood correctly, augment is the data augmentation baseline, and baseline is actually vanilla Autobots. Is this correct?

Author response:
* Yes, the baseline refers to the original Autobots throughout our manuscript, whereas the augment refers to the causal data augmentation technique in Roelofs el al. 2022. We have updated figure captions accordingly in our updated manuscript.

Reviewer comment (Evaluation):
I wonder if it would be interesting to also look at the number (or percentage) of ego-collisions in the sim-to-real experiments. It could give a better picture on the importance of promoting causal representations.

Author response:
* Thank you for the suggestion. Our method does not significantly reduce collision rates compared to the baseline. This is because the AutoBots baseline already exhibits exceptionally low collision rates on the ETH-UCY dataset (0.5%), leaving marginal room for improvements.

Reviewer comment (Experiments):
What exactly is the unseen context variant in the OOD experiments? This and high density should be clearly defined.

Author response:
* The training context comprises a simulated open area with randomized agent directions, whereas the OOD context features a simulated street with constrained agent movements. Please refer to more detailed description in the `dataset details` paragraph in Appendix B, along with animations in [our public repository](https://github.com/socialcausality/socialcausality#illustrations) for illustration.