Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The problem of domain extension with language is addressed in this paper. The proposed method (LADS) uses a CLIP model's domain-level knowledge to learn a latent feature augmentation of the training set. It does not require any unseen domain samples and instead relies on written descriptions of the training and unseen domains. Domain alignment and class consistency losses are used to train the latent feature augmentation. Once trained, a simple linear classifier is trained on both the original and augmented image embeddings, resulting in improved in-domain and out-of-domain recognition performance. Experiments on DomainNet, CUB-Paintings, Colored MNIST, and Waterbirds demonstrate that LADS outperforms other methods.
strength_and_weaknesses: (Strengths)

- The proposed method does not require any unseen domain image samples. It significantly reduces the cost of data collection.

- LADS augments data at the image space level rather than the pixel level. This approach is useful for avoiding bottlenecks caused by the generative process's quality.

- The proposed method can be extended to address training data biases.

- Extensive experiments are carried out to demonstrate the performance of LADS in comparison to the state-of-the-art. Furthermore, the authors discuss the proposed method's limitations on the 'natural' distribution shift. 

(Weaknesses)

- The text embeddings of the domain descriptions guide the domain alignment loss. During training, the image embeddings space changes while the text embeddings space remains constant. It is suspected that the text embeddings function properly as guidance.
- Furthermore, the method is heavily reliant on the domain descriptions used. Taking the first two rows of table 3, the results are affected by the prompts. Considering the variations of the 'direction' instead of fixing a 'global direction' may be helpful.
- Though the ablation study investigated the role of each loss term, it is suggested that the role of $\alpha$ be investigated as well.

clarity,_quality,_novelty_and_reproducibility: The paper addresses a novel problem. The paper is simple to understand. The paper provides setting details that support reproducibility.
summary_of_the_review: The paper's contributions are novel. The technique is simple, but it works well. However, it is suggested that more research into domain alignment loss is required. Given the novelty of the paper, I would recommend a 'marginally above the acceptance threshold' rating.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposed a strategy to augment the image feature into different domains. The process is guided by CLIP to not just transfer to another domain but also keep the semantic content. In the experiment ,we can see that the proposed method, LADS, indeed performs well in both in-distribution and out-of-distribution. We can also qualitatively (Fig4) and quantitatively (Tab2) see that the model indeed learns to transfer the input feature from the training domain to a new domain, and at the same time keep the semantic meaning.
strength_and_weaknesses: ## Strength

1. The proposed method achieves good performance in both the in-distribution and the out-of-distribution data, without explicitly training on OoD labeled data.

1. The analyses demonstrate that the model indeed successfully learns to transfer the feature into another domain (contribution of $L_{DA}$) and keep the semantic meaning (contribution of $L_{CC}$.

## Weakness

1. The biggest concern the reviewer has is how to describe the unseen domain in the form of simple text. In this paper, the setting is rather simple: there is a clear distinction between training and testing domains, and the difference can be easily described in a few words. As also pointed out in Tab3, the model performance is quite sensitive to the prompt, or domain descriptions.

1. Related to the previous point, in Sec 3.1 the waterbird example, how to discover the additional prior knowledge that the "spurious correlation is the background" may not be trivial.

1. The author should also compare to other more general methods in domain generalization, where only the training domain is available and the domain descriptions of testing domains remain unknown.

1. Since the gap between CLIP-LP and LADS is not that big, how did the authors train the CLIP-LP model? For example, the ELEVATOR paper [1] pointed out that it is beneficial to initialize the clf layer weight by CLIP's language branch.

[1] https://openreview.net/forum?id=hGl8rsmNXzs
clarity,_quality,_novelty_and_reproducibility: Overall easy to follow. The experiments and analyses support the claims and the contributions for this paper.
summary_of_the_review: This paper did a good job in the proposed setting, where the unseen domain can be easily described in a few words and is given as prior knowledge during training. However, this seems like a rather contrived setting as described in the weakness section of the review.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes the Latent Augmentation using Domain Descriptions (LADS) framework to enhance the generalization ability of image classifiers across various domains. Specifically, LADS trains an augmentation module to transform the source domain image embeddings to multiple pre-known target domains, where domain alignment and class consistency are respectively constrained. After that, a linear classifier is trained with both source and target domain embeddings to achieve better generalization. Under both the settings of domain shift and dataset shift, authors demonstrate that LADS can achieve state-of-the-art performance on the extended domain (the collection of source and target domains).
strength_and_weaknesses: Strength:

1. This paper studies a practical problem that has been rarely explored, domain extension with language, where the domain gap is filled by utilizing the text descriptions of domains and classes instead of unlabeled target domain samples as in UDA.
2. The proposed method is technically sound. By utilizing pre-trained vision-language alignment models like CLIP, LADS can well represent different domains and classes and further transformer such knowledge to the image modality.
3. Extensive empirical results (performance comparison and ablation study) demonstrate the effectiveness of LADS on tackling domain and dataset shifts. 


Weakness:

1. The experimental results under domain shift are not strong enough. For image recognition under domain shift, people are always most interested in the performance on OOD domains. On the two datasets studied in this paper, LADS achieve gains on CUB-Paintings while underperforms zero-short CLIP on DomainNet. Therefore, it is interesting to study under what circumstances LADS is necessary against directly apply the zero-shot CLIP. The current experiments on two datasets are not sufficient to illustrate this. I suggest the authors to do performance comparison on a more diverse set of benchmark tasks under domain shift, e.g., the Office-Home dataset with moderate domain gaps and the VisDA dataset with a larger domain gap. 

clarity,_quality,_novelty_and_reproducibility: Quality:

In general, this paper is well-written. The motivation and derivation of proposed techniques are clear to the audience.


Novelty:

This paper shows decent novelty on the studied problem and the methodology used to solve it.


Reproducibility:

Authors did not submit source codes and are recommended to submit them (maybe upon acceptance) for the sake of reproducibility.

summary_of_the_review: In summary, I am convinced by the motivation of studying under the proposed setting and believe the technical soundness. I give an acceptance on the current version and suggest authors to do more studies under the domain shift setting during the response period.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Summary:  
This paper explores a language-based domain extension strategy. The proposed method, LADS, transforms the source domain image embeddings to multiple pre-known target domains guided by CLIP model’s knowledge. Experiments on several datasets demonstrate that LADS improves the performance on both in-domain and out-of-domain recognition settings, and outperforms other methods. 

Strengths:  
1. This paper addresses a rarely explored problem of language-guided domain extension which is well motivated and thought to be practically useful. 
2. The proposed method is technically sound and solid.
3. The proposed method achieves good performance in both the in-distribution and the out-of-distribution data, without explicitly training on OoD labeled data.
4. Analysis and discussion are good and convincing that the model indeed successfully learns to transfer the feature into another domain and keep the semantic meaning.
5. During the discussion period, the authors added new results showing that the proposed method is indeed better than simple linear probing of CLIP. 

Weaknesses:  
1. The method is based on the assumption that the unseen domain description is given and can be expressed in the form of relatively simple text. It is unclear how the method can be applied to more general cases. Also, debiasing spurious correlation would require further prior knowledge.


justification_for_why_not_higher_score: All reviewers basically acknowledge the novelty of the addressed problem and the proposed method as well as the appropriateness of the experimental validation. They are unanimous for the acceptance and the AC also agrees that the paper is well above the acceptance threshold. Meanwhile, the method has some obvious limitations due to some common problems in this kind of tasks (e.g., how to appropriately setup and utilize prior knowledge for linguistic guidance). From a practical viewpoint, the method is a little better than baseline CLIP-based probing but not a very big jump. In this sense, the method is good but not ground-breaking, so the AC believes it is a little short for oral presentation.
justification_for_why_not_lower_score: That being said (above), the paper does show a first concrete step showing the new direction in domain adaptation, i.e., language-guided domain extension. It would be valuable for the community by widely spreading its core idea in a spotlight.
Decision: Accept: notable-top-25%