Venue: ICLR 2023 TinyPapers

Reviews:

final_score: rating: Clear, Correct, and Reproducible (CCR): a submission which meets the reviewing criteria
review_content: summary_of_contributions: This work develops a copy-and-concat scheme to augment each short training example to align with the length range of the long inference dataset,
strengths_and_weaknesses: Strengths:

1. This paper is interesting, though the problem is not urgent enough.
2. Clear motivations and extensive experimental results.
3. Solid theoretical analysis.

Weaknesses:

1. The idea is not novel.
2. The authors should compare their method with some recent works.
3. The written and presentation should be improved.
suggested_changes: Please refer to the weaknesses part.
review_title: null

final_score: rating: Great Start (GS): a submission which meets some of the reviewing criteria but has room for improvement
review_content: summary_of_contributions: This paper studies the robustness of the retrieval models. The authors empirically observe the vulnerability of the models on adversarial length attacks, and propose a simple length-based adversarial training to improve the robustness. The experimental results demonstrate the effectiveness of the proposed method from different points of view.
strengths_and_weaknesses: **Strengths:**

* The studied problem (robustness) is quite important.
* The motivation of this paper and the empirical findings (different similarity scores on semantically similar documents) are quite interesting.
* The authors analyze the effectiveness of the proposed method through three different views.

**Weakness:**

* The range of related work could be broader, including the works studying adversarial robustness in NLP.
* From my point of view, the proposed method is almost a simplified version of adversarial training, which augments the training data with adversarial examples (constructed longer documents without involving an inner maximization optimization). Therefore, the novelty of the proposed method is limited. Moreover, the proposed method (copy-and-concat) seems to be only effective against length attacks, which may limit the scope and impact of this paper.
* It is unclear what is the underlying reason for the text encoder's vulnerability on the longer documents. If the text encoder is only trained on small documents. It may be expected to observe this vulnerability due to its poor OOD generalization capability. Moreover, what is the connection between out-of-distribution generalization and adversarial robustness in the studied topic? It is suggested to provide further analyses and insights here.
* The authors emphasize that contrastive learning is prone to length overfitting, making it vulnerable to adversarial length attacks. However, it seems that the connection with contrastive learning is weak when I read the main paper. Is the ordinary training robust to length attack?

**Overall:**

* *Clarity:* Fair. The writing of this paper could be improved. Some confusing points exist. Please see the fourth point of weakness. Moreover, a broader review of relevant literature is needed.

* *Correctness:* Fair. The underlying reason for the empirical findings is unclear. Please see the third point of weakness.

* *Reproducibility:* Good. The proposed method is simple. It may be reproducible without much effort.

* *Follows basic requirements:* Yes, the authors follow the ICLR code of conduct.

In summary, despite several limitations, it is a great start for the authors to study this interesting and crucial research topic. Deeper analyses and insights are suggested to further improve the paper.
suggested_changes: Please see the weakness section.
review_title: null

Decision: Invite to archive