Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper aims to tackle the out-of-domain generalization challenge for instance segmentation of 2D images and 3D point clouds. The proposed method combines the recent slot attention-based model for segmentation and the standard test-time-augmentation approach with the auxiliary reconstruction task. The proposed method significantly outperforms the prior art in the out-of-domain setting on standard benchmarks. The ablation studies show the importance of the proposed module and learning setting.
strength_and_weaknesses: Strength
- It's an interesting idea to take the slot-centric approach to do TTA with the image reconstruction auxiliary task, which is not done before.

- Strong performance on popular datasets from two modalities.

- Detailed ablation studies and model description for design justification and reproducibility.

Weakness
- Lack of novelty. The proposed method combines two existing works: slot-attention-based instance segmentation model and the TTA method with image reconstruction auxiliary loss, without providing much new insights.

- Biased experiment setting. For image instance segmentation, the benchmark dataset (sec. 7.1-7.2) has few instances in the training set and much more instances in the test set. Such experiment design favors instance segmentation models for crowded scenes (i.e., slot-attention based method) over popular maskformer models. Thus, it's unclear if the performance gain over the baseline (maskformer+reconstruction) majorly comes from the inductive bias of the backbone selection instead of the TTA framework. Also, It's unclear if the proposed method is indeed better on common images (e.g., MS-COCO). The paper result can be more convincing if it shows comparison on natural images.

- Naming. (1) Task: the proposed method is specifically designed for the "instance segmentation" task, but the paper uses "segmentation" in many places, which can be confusing. (2) Title: It's more accurate to use "image reconstruction" instead of "image synthesis". The title can be better to emphasize on its key insight: slot-centric reconstruction. (3) the model name. GFSNet is generic, as any TTA methods that require finetuning can be named as GFSNet.
clarity,_quality,_novelty_and_reproducibility: - Clarity&quality: it's easy to follow overall. The method section can be better with figures for the model architecture for three different settings.

- Novelty: It's incremental without much new insights.

- Reproducibility: good.
summary_of_the_review: Overall, the proposed method achieves strong performance on the selected multiple datasets. However, I have concerns on (1) its novelty as it simply combines two existing ideas; (2) usability in real world as the dataset images are heavily crowded with synthetic objects, which makes it unclear if the performance gains mostly comes from the TTA framework or the segmentation backbone that is better suited for the crowded scene. The paper will be much stronger if it provides results on popular 2D image instance segmentation benchmarks.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This work targets to segmenting scenes into objects and parts under an out-of-distribution setting. In an out-of-distribution setting, the statistical distribution of the testing example is different from that of the training examples. To handle the problem, the authors build a Generating Fast and Slow Network (GFS-Net) based on prior works, such as slot-centric generative models. The GFS-Net combined with test time adaption strategy, improves the segmentation performance on three different tasks. The authors conduct extensive experiments to demonstrate the efficacy of their solution.
strength_and_weaknesses: Strengths:

-- As far as I know, this is the first work exploring the scene understanding tasks by test-time adaptation.

-- The authors conduct extensive experiments on three different tasks with different kinds of input, i.e., posed multi-view 2D RGB image, single-view 2D RGB images, 3D point clouds. Correspondingly, the authors conduct modifications on the introduced network based on different inputs.

Weakness:

-- The technical contribution and novelty of this work seem not strong enough for ICLR. For example, the basic modules, and encoder/decoder structure of GFS-Nets, are all based on prior works (OSRT, spatial broadcast decoder, etc). It seems there is few new designs in the network structure.

-- For the test-time adaption part, it seems that the authors just directly borrow the idea and apply it to their tasks. Without a technical contribution, it is hard to claim there is a significant contribution in this part.

-- The submission is not self-contained and some formulations/terms need refined. For example, what is Hungarian matching? What is the occupancy value O_i? The authors should make the content self-contained as not all authors have related backgrounds. In Eq. 3, the formulation of each loss term is not provided in this work. Do the three tasks in this work use the same formulation as the l_{seg}? It should be {c_k, a_k} = Dec(p, S_k).
clarity,_quality,_novelty_and_reproducibility: The clarity and quality of this work, from my point of view, are not enough for ICLR. The novelty and technical contribution are limited. I believe this work is reproducible.
summary_of_the_review: In summary, this work is interesting and practical. However, the technical contribution is limited, making the novelty not strong enough. The writing and organization of this paper need significant improvement.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: NA
strength_and_weaknesses: NA
clarity,_quality,_novelty_and_reproducibility: NA
summary_of_the_review: NA
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes Generating Fast and Slow Networks (GFS-Nets), where the model has a slot-centric image or point-cloud rendering part, which is adpated per scene at test time through gradient decent. This adaptation at test time improves performance on out of distribution data.


strength_and_weaknesses: The paper has the following strenghts:

1) The paper is well-written the main ideas are effectively presented.
2) The paper tackles a problem of interest to the community where out of distribution segmentation task can be handled.
3) The slow-inference results are better on the benchmark datasets.


The weakness of the paper in my opinion are:
1) Some of hte limitation of the methods are discussed in the paper and the one which I think is most signifcant is the absence of the heirarchy of components of object. In segmentation the heirarcy and relations of components of objects is really important.

2) The fast infrence results are slightly lower than the benchmarks.


clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and the quality is good.

A novel idea of test-time adaptation is introduced in this paper for segmentation

The code and datasets are not shared.
summary_of_the_review: In my opinion the paper is well written and tackles a problem of interest to the community. I have, however, outlined some of the aspects which i think need some explanation in the strengths and weakness section.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes a domain adaptation method in test time. To do so, this paper proposes a semi-supervised slot-centric approach that combines slot attention and object scene representation transformer. The model adapts to a single test sample without supervision at test time. It is optimized for the self-supervised objective function. This paper shows that the loss for image synthesis is helpful for the test-time domain adaptation.

strength_and_weaknesses: 1. Strength
- The proposed test time domain adaptation method outperforms the normal method, which is never adapted at test time. 
- This paper compared not only the instance segmentation but also the 3D segmentation.

2. Weakness
- This paper combines two existing methods slot attention and the object scene representation transformer. Instance query-based matching is a common pipeline for semantic segmentation. 
- The contributions to the test time adaptation method are marginal. The only loss that is applied for adaptation is the reconstruction loss in eq 3. Reconstruction loss is common for self-supervised loss. Applying only reconstruction loss can degrade the performance of the other test datasets. 
- The proposed method is not densely evaluated. Only a small number of classes and datasets are utilized for the evaluation. 
- The proposed method has not been compared to the other test time domain adaptation methods [1-3].  If the self-supervised loss is effective, why not apply the loss to the other tasks such as semantic segmentation, and object detection? The proposed method only compared with the normal algorithm, which is not applied test time domain adaptation, such as mask2former. 


[1] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In ICLR, 2021.
[2] Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning, pp. 9229–9248. PMLR, 2020.
[3] Inkyu Shin, Yi-Hsuan Tsai, Bingbing Zhuang, Samuel Schulter, Buyu Liu, Sparsh Garg, In So Kweon, and Kuk-Jin Yoon. Mm-tta: Multi-modal test-time adaptation for 3d semantic segmentation. In CVPR, 2022.
clarity,_quality,_novelty_and_reproducibility: Overall, the contribution of this paper is too incremental, and the comparison to the state-of-the-art methods is missing. The overall quality of this paper is too weak to be accepted to ICLR.
summary_of_the_review: Because of (1) Incremental technical contribution and (2) Lack of comparison to the existing method, this paper is not adequate to be accepted to ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: in this paper, the authors proposed a test-time adaptation framework for some computer vision tasks. Though the setting studied in this paper is new, the proposed solution is incremental, and lacks technical novelty. Moreover, the experimental setting and results are not convincing. 

In summary, this paper is not ready for publication in ICLR based on its current shape. 
justification_for_why_not_higher_score: Incremental. Lacks technical novelty. Experiments are not convincing.
Decision: Reject