reviews: 
Comments: The authors assess how VLMs represent space through the lens of “situated communication,” framing the meaning of spatial relations like “to the right of” under transformations, in different frames of reference, and others (situated communication).

They introduce the COMFORT dataset, tasks, and metrics to analyze spatial reasoning under these classes of transformations, and see which frame-of-reference preferences LMs have, and test whether these preferences are language-agnostic by extending their evaluation to multilingual settings. 

The images are rendered to enable simple dynamic generation of test samples. Their language queries that test the relations likewise have a simple structure and are programmatically produced, enabling translation into other languages. Testing the kinds of relations is simple using this methodology and allows them to test for continua such as how a model rates the “in front of” relation by angle between the two in the plane relative to the camera.

**Edit**: The authors did a great job adding some small case studies that **show** how their results generalize to other backgrounds and real images (although these are still pretty close to in-distribution to the test images). The use of a laptop rather than a ball in the real images in particular is a strong change. I have raised all the component scores to 4. **Although I would prefer an option to give a 9/10, since it isn't available I will raise from 8 to 10.**

These tests are then translated into metrics using the established concept of “acceptability regions”, which then allows them to score for correctness. Swapping object types, colors, etc in the same scene is easy using the synthetic pipeline and gives an evaluable sense of robustness.

They test a broad set of VLMs, including multilingual ones in 109 translated languages. Some models better represent egocentric vs object centric relations.

Overall, they find that the VLMs do have some acceptable egocentric understanding capabilities but they have severe robustness issues. They find that English notions of frame of reference are held across most test languages, not reflecting the diversity of notions.
Strengths: Deep utilization of synthetic pipeline to really probe spatial perception from several angles.

Demonstrations of anglewise judgements are compelling and surprising! (eg., figure 7)

Great to see the extra effort of testing 

Conclusions and discussion are thoughtful and well supported.
Weaknesses: ~Sole reliance on synthetic data.~ A brief ablation over natural images would be nice to see to give a sense of how well these findings generalize. They added the case study, the results generalize (though the case study is still close to ID the synth data)!

~Limited diversity of backgrounds. This is another form of variation that may have important implications for generalization.~ They added the case study, the findings generalize!

The multilingual experiments, while welcome, have limited time to breathe in the overall paper with just a few paragraphs and tables stuck in appendices. However, I think this is a great problem to have as the paper is densely packed with quality experimentation.
Recommendation: 10
Reviewer confidence: 4
Soundness/correctness: 4
Clarity: 4