Original: M ULTI -P ROMPT
Revised: M ULTI -P ROMPT

Original: A LIGNMENT FOR M ULTI -S OURCE
Revised: A LIGNMENT FOR M ULTI -S OURCE

Original: U NSUPERVISED D
Revised: U NSUPERVISED D

Original: OMAIN
Revised: OMAIN

Original: A DAPTATION
Revised: A DAPTATION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Most existing methods for multi-source unsupervised domain adaptation (UDA) rely on a common feature encoder to extract domain-invariant features.
Revised: Most existing methods for multi-source unsupervised domain adaptation (UDA) rely on a common feature encoder to extract domain-invariant features.

Original: However, learning such an encoder involves updating the parameters of the entire network, which makes the optimization computationally expensive, particularly when coupled with min-max objectives.
Revised: However, learning such an encoder involves updating the parameters of the entire network, which makes the optimization computationally expensive, particularly when coupled with min-max objectives.

Original: Inspired by recent advances in prompt learning that adapts high-capacity deep models for downstream tasks in a computationally economic way, we introduce M ulti-
Revised: Inspired by recent advances in prompt learning that adapts high-capacity deep models for downstream tasks in a computationally economic way, we introduce M ulti-

Original: P rompt A lignment (MPA), a simple yet efficient two-stage framework for multi-source UDA.
Revised: P rompt A lignment (MPA), a simple yet efficient two-stage framework for multi-source UDA.

Original: Given a source and target domain pair, MPA first trains an individual prompt to minimize the domain gap through a contrastive loss, while tuning only a small set of parameters.
Revised: Given a source and target domain pair, MPA first trains an individual prompt to minimize the domain gap through a contrastive loss, while tuning only a small set of parameters.

Original: Then, MPA derives a low-dimensional latent space through an auto-encoding process that maximizes the agreement of multiple learned prompts.
Revised: Then, MPA derives a low-dimensional latent space through an auto-encoding process that maximizes the agreement of multiple learned prompts.

Original: The resulting embedding further facilitates generalization to unseen domains.
Revised: The resulting embedding further facilitates generalization to unseen domains, making MPA naturally suitable for test time adaptation.

Original: Extensive experiments show that our method achieves state-of-the-art results on popular benchmark datasets while requiring substantially fewer tunable parameters.
Revised: Extensive experiments show that our method achieves state-of-the-art results on popular benchmark datasets while requiring substantially fewer tunable parameters.

Original: To the best of our knowledge, we are the first to apply prompt learning to the multi-source UDA problem and our method achieves the highest reported average accuracy of 54.1% on DomainNet, the most challenging UDA dataset to date, with only 15.9M parameters trained.
Revised: To the best of our knowledge, we are the first to apply prompt learning to the multi-source UDA problem and our method achieves the highest reported average accuracy of 54.1% on DomainNet, the most challenging UDA dataset to date, with only 15.9M parameters trained.

Original: More importantly, we demonstrate that the learned embedding space can be easily adapted to novel unseen domains.
Revised: More importantly, we demonstrate that the learned embedding space can be easily adapted to novel unseen domains with even fewer tuned parameters.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Deep learning has achieved remarkable progress in various computer vision tasks such as image classification (Krizhevsky et al., 2012;
Revised: Deep learning has achieved remarkable progress in various computer vision tasks such as image classification (Krizhevsky et al., 2012;

Original: He et al., 2016), object detection (Ren et al., 2015; Redmon et al., 2016;
Revised: He et al., 2016), object detection (Ren et al., 2015; Redmon et al., 2016;

Original: Liu et al., 2016) and image segmentation (Long et al., 2015a; Chen et al., 2017).
Revised: Liu et al., 2016) and image segmentation (Long et al., 2015a; Chen et al., 2017).

Original: However, these success relies on high capacity models trained in a supervised manner using a massive amount of manually labeled data, which are oftentimes expensive and time-consuming to collect.
Revised: However, these success relies on high capacity models trained in a supervised manner using a massive amount of manually labeled data, which are oftentimes expensive and time-consuming to collect.

Original: Furthermore, current deep models are brittle to the presence of domain shift (Quinonero-Candela et al., 2008; Torralba & Efros, 2011; Zhang et al., 2013) in the forms of different image styles, varied lighting conditions, diverse viewpoints, etc , between training and testing distributions.
Revised: Furthermore, current deep models are brittle to the presence of domain shift (Quinonero-Candela et al., 2008; Torralba & Efros, 2011; Zhang et al., 2013) in the forms of different image styles, varied lighting conditions, diverse viewpoints, etc. , between training and testing distributions.

Original: Unsupervised domain adaptation (UDA) is a popular strategy that mitigates domain discrepancies through transferring knowledge learned from a well-labeled source domain to an unlabeled target domain (Pan & Yang, 2010; Csurka, 2017; Wang & Deng, 2018).
Revised: Unsupervised domain adaptation (UDA) is a popular strategy that mitigates domain discrepancies through transferring knowledge learned from a well-labeled source domain to an unlabeled target domain (Pan & Yang, 2010; Csurka, 2017; Wang & Deng, 2018).

Original: While significant advances have been achieved, current approaches focus on a single source setting, where all the labeled training data share the same distribution.
Revised: While significant advances have been achieved, current approaches focus on the single source setting, where all the labeled training data share the same distribution.

Original: In practice, however, it is more common for the labeled data to be collected from multiple sources that are diverse in distribution.
Revised: In practice, however, it is more common for the labeled data to be collected from multiple sources that are diverse in distribution.

Original: Naturally, one could still tackle this problem by straightforwardly combining all the data into one single source and apply off-the-shelf
Revised: Naturally, one could still tackle this problem by straightforwardly combining all the data into one single source and apply off-the-shelf

Original: UDA methods.
Revised: UDA methods.

Original: However, directly applying single source UDA methods often results in a limited performance, as domain shift also exists among different source domains.
Revised: However, directly applying single source UDA methods often results in a limited performance, as domain shift also exists among different source domains.

Original: The integration of multiple source domains for improved adaptation results on the unlabeled target domain is generally known as multi-source unsupervised domain adaptation.
Revised: The integration of multiple source domains for improved adaptation results on the unlabeled target domain is generally known as multi-source unsupervised domain adaptation.

Original: Inspired by the theoretical analysis of Ben-David et al.
Revised: Inspired by the theoretical analysis of Ben-David et al.

Original: (2006), learning domain-invariant feature representations has become a prevailing paradigm for multi-source UDA.
Revised: (2006), learning domain-invariant feature representations has become a prevailing paradigm for multi-source UDA.

Original: One typical approach is to jointly learna common feature extractor together with domain-specific feature extractors and classifier heads. j-th classifier
Revised: One typical approach is to jointly learn j-th classifier

Original: C j ... C i target image source j image source i image
Revised: C j ... C i target image source j image source i image

Original: Conventional Methods
Revised: Conventional Methods

Original: Our Method
Revised: Our Method

Original: Common Feature Encoder prompt i prompt j
Revised: Common Feature Encoder prompt i prompt j

Original: P j P i i-th classifier
Revised: P j P i i-th classifier

Original: CLIP’s Text Encoder
Revised: CLIP’s Text Encoder

Original: CLIP’s Image Encoder ... ...
Revised: CLIP’s Image Encoder ... ...

Original: 1 1i IT 2 1i IT 3 1i IT 

Revised: 1 1i IT 2 1i IT 3 1i IT

Original: Ki I T 

Revised: Ki I T

Original: Ki I T 

Revised: Ki I T

Original: Ki I T ... ...
Revised: Ki I T ... ...

Original: 1 1j IT 1 Kj I T 2 1j IT 

Revised: 1 1j IT 1 Kj I T 2 1j IT

Original: Kj I T 3 1j IT 

Revised: Kj I T 3 1j IT

Original: Kj I T 

Revised: Kj I T

Original: I 2 I 3 I ...
Revised: I 2 I 3 I ...

Original: ... 1i T
Revised: ... 1i T

Original: Ki T 1j
Revised: Ki T 1j

Original: T
Revised: T

Original: Kj T 

Revised: Kj T

Original: Ki I T
Revised: Ki I T

Original: [Figure]
Revised: [Figure]

Original:  Various feature distance metrics (Long et al., 2015b; Sun & Saenko, 2016; Kang et al., 2019) or domain adversarial training (Tzeng et al., 2017) can be leveraged that serves as a preliminary alignment between source and target domains. Then different auxiliary lossesare carefully designed to further reduce the domain shift.
Revised: a common feature extractor together with domain-specific feature extractors and classifier heads. Various feature distance metrics (Long et al., 2015b; Sun & Saenko, 2016; Kang et al., 2019) or domain adversarial training (Tzeng et al., 2017) can be leveraged that serves as a preliminary alignment between source and target domains, followed by different auxiliary losses carefully designed to further reduce the domain shift.

Original: While these methods offer decent results, they require optimizing the entire set of parameters in the network and hence poses a significant challenge for optimization even without the widely used min-max objective (Zhao et al., 2018;
Revised: While these methods offer decent results, they require optimizing the entire set of parameters in the network and hence poses a significant challenge for optimization even without the widely used min-max objective (Zhao et al., 2018;

Original: Li et al., 2018; Hoffman et al., 2018).
Revised: Li et al., 2018; Hoffman et al., 2018).

Original: Such problem is further amplified if we wish to apply more advanced backbones such as
Revised: Such problem is further amplified if we wish to apply more advanced backbones such as

Original: Vision Transformer (Dosovitskiy et al., 2021) for improved performance.
Revised: Vision Transformer (Dosovitskiy et al., 2021) for improved performance.

Original: In this paper, we introduce a simple yet efficient approach for multi-source UDA without the need of retraining the entire network and the use of complicated min-max loss functions (See Figure 1 for a comparison).
Revised: In this paper, we introduce a simple yet efficient approach for multi-source UDA without the need of retraining the entire network and the use of complicated min-max loss functions (See Figure 1 for a comparison).

Original: In particular, we build upon prompt learning (Lester et al., 2021;
Revised: In particular, we build upon prompt learning (Lester et al., 2021;

Original: Wei et al., 2021) that has been designed to transfer knowledge learned from large pre-trained vision language models like CLIP (Radford et al., 2021).
Revised: Wei et al., 2021) that has been designed to transfer knowledge learned from large pre-trained vision language models like CLIP (Radford et al., 2021).

Original: In prompt learning, image representations are learned contrastively with a piece of language text termed as “prompt” that describes the class of the image.
Revised: In prompt learning, image representations are learned contrastively with a piece of language text termed as “prompt” that describes the class of the image.

Original: While recent studies (Ge et al., 2022; Ben-David et al., 2022) suggest  learnable prompts can be used forthe task of UDA, they are restricted to the single source scenario and directly generalizing them to the multisource setting produces limited results.
Revised: While recent studies (Ge et al., 2022; Ben-David et al., 2022) suggest that learnable prompts can be used for UDA, they are restricted to the single source scenario and directly generalizing them to the multisource setting produces limited results.

Original: Furthermore, while prompt learning provides an efficient alternative, we argue that the number of parameters needed to be tuned could be further reduced.
Revised: Furthermore, while prompt learning provides an efficient alternative, we argue that the number of parameters needed to be tuned could be further reduced.

Original: In light of this, we present a surprisingly easy framework, M ulti-
Revised: In light of this, we present a surprisingly easy framework, M ulti-

Original: P rompt A lignment (MPA), for multi-source UDA.
Revised: P rompt A lignment (MPA), for multi-source UDA.

Original: MPA is composed of two stages, one to learn an individual prompt by tuning a small set of parameters for each source and target domain pair and one to mine the relationships among learned prompts through deriving a shared embedding space.
Revised: MPA is composed of two stages, one to learn an individual prompt by tuning a small set of parameters for each source and target domain pair and one to mine the relationships among learned prompts through deriving a shared embedding space.

Original: The resulting embedding is expected to be domain-invariant and can generalize to unseen domains.
Revised: The resulting embedding is expected to be domain-invariant and can generalize to unseen domains.

Original: More specifically, for the first stage, given a source domain and a target domain, we use CLIP as our backbone and learn one prompt tailored for such a pair.
Revised: More specifically, for the first stage, given a source domain and a target domain, we use CLIP as our backbone and learn one prompt tailored for such a pair.

Original: We then align all the learned prompts in a latent space of a lower dimension d I .
Revised: We then align all the learned prompts in a latent space of a lower dimension d I .

Original: This is accomplished by a simple auto-encoder network with a reconstruction loss.
Revised: This is accomplished by a simple auto-encoder network with a reconstruction loss.

Original: Additionally, we incorporate an L 1 loss so that the reconstructed prompts agree on the classification of target images.
Revised: Additionally, we incorporate an L 1 loss so that the reconstructed prompts agree on the classification of target images.

Original: This is beneficial for prompts to handle situations in which the target data lies near the decision boundary.
Revised: This is beneficial for prompts to handle situations in which the target data lies near the decision boundary.

Original: We conduct extensive experiments on multiple benchmarks and the results  show that our method outperforms state-of-the-art methodsclearly in the multi-source setting.
Revised: We conduct extensive experiments on multiple benchmarks and the results clearly show that our method outperforms state-of-the-art methods in the multi-source setting.

Original: In particular, on DomainNet (Peng et al., 2019), the most challenging dataset for multi-source UDA so far, MPA surpasses all state-of-the-arts methods.
Revised: In particular, on DomainNet (Peng et al., 2019), the most challenging dataset for multi-source UDA so far, MPA surpasses all state-of-the-arts methods.

Original: More importantly, as the latent space is optimized with prompts from multiple source domains, it encodes knowledge shared by different domains and could potentially generalize to unseen domains by traversing the space.
Revised: More importantly, as the latent space is optimized with prompts from multiple source domains, it encodes knowledge shared by different domains and could potentially generalize to unseen domains by traversing the space.

Original: Consequently, we show how surprisingly easy it is to tune the learned low-dimensional embedding for deployment in unseen domains.
Revised: Consequently, we show how surprisingly easy it is to tune the learned low-dimensional embedding for deployment in unseen domains.

Revised: Since such tuning involves no source domain data, this can be naturally extended to test time adaptation problems.

Original: In summary, our contributions are three-fold:
Revised: In summary, our contributions are three-fold:

Original: • We introduce M ulti-
Revised: • We introduce M ulti-

Original: P rompt A lignment (MPA) for multi-source UDA.
Revised: P rompt A lignment (MPA) for multi-source UDA.

Original: MPA takes advantage of prompt learning, thus greatly reducing the number of parameters needed for training compared with alternative methods.
Revised: MPA takes advantage of prompt learning, thus greatly reducing the number of parameters needed for training compared with alternative methods.

Original: • MPA learns a latent embedding space by maximizing the consensus of multiple learned prompts and the resulting low-dimensional embeddingfacilitates surprisingly easy adaptation to novel unseen domains.
Revised: • MPA learns a latent embedding space by maximizing the consensus of multiple learned prompts. Consequently we provide a methodology that is able to solve test time adaptation problems by facilitating the adaptation of the resulting low-dimensional embedding to novel unseen domains.

Original: • MPA achieves state-of-the-art results on several popular benchmarks.
Revised: • MPA achieves state-of-the-art results on several popular benchmarks.

Original: Specifically, on the large-scale DomainNet dataset, MPA achieves the best reported average accuracy.
Revised: Specifically, on the largescale DomainNet dataset, MPA achieves the best reported average accuracy.

Original: Moreover, tuning the learned latent subspace offers comparable results with even fewer parameters when generalizing to unseen domains.
Revised: Moreover, tuning the learned latent subspace offers comparable results with even fewer parameters when generalizing to unseen domains.

Original:  R ELATED W ORK 2. M ULTI -S OURCE U NSUPERVISED D OMAIN A DAPTATION
Revised:  R ELATED W ORK 2. M ULTI -S OURCE U NSUPERVISED D OMAIN A DAPTATION

Original: First studied by Yang et al.
Revised: First studied by Yang et al.

Original: (2007), multi-source UDA has drawn increasing attention in the community.
Revised: (2007), multi-source UDA has drawn increasing attention in the community.

Original: Throughout the years, various methods have been studied.
Revised: Throughout the years, various methods have been studied.

Original: For example, in MDAN (Zhao et al., 2018) and DCTN (Xu et al., 2018), a discriminator applied with adversarial losses is trained so that the features from source and target domains are aligned.
Revised: For example, in MDAN (Zhao et al., 2018) and DCTN (Xu et al., 2018), a discriminator applied with adversarial losses is trained so that the features from source and target domains are aligned.

Original: MFSAN (Zhu et al., 2019) calculates and aligns the maximum mean discrepancy for each source and target pair.
Revised: MFSAN (Zhu et al., 2019) calculates and aligns the maximum mean discrepancy for each source and target pair.

Original: Similarly, M 3 SDA (Peng et al., 2019) aligns the moment distance for both target and source domains.
Revised: Similarly, M 3 SDA (Peng et al., 2019) aligns the moment distance for both target and source domains.

Original: All these methods require a shared feature extractor to obtain domain-invariant features, which is inevitably difficult to optimize and at the risk of losing semantic information as the number of domains increases.
Revised: All these methods require a shared feature extractor to obtain domain-invariant features, which is inevitably difficult to optimize and at the risk of losing semantic information as the number of domains increases.

Original: On the contrary, Rakshit et al.
Revised: On the contrary, Rakshit et al.

Original: (2019) adopt one domain-specific encoder for each source and target pair while Zhao et al.
Revised: (2019) adopt one domain-specific encoder for each source and target pair while Zhao et al.

Original: (2020) pretrain a classifier for each source domain and then adversarially map the target images into each trained feature space.
Revised: (2020) pretrain a classifier for each source domain and then adversarially map the target images into each trained feature space.

Original: The better alignment of these methods is at the cost of significantly increased number of parameters needed for training.
Revised: The better alignment of these methods is at the cost of significantly increased number of parameters needed for training.

Original: To overcome such a trade-off between performance and efficiency, we introduce prompt learning to multi-source UDA.
Revised: To overcome such a trade-off between performance and efficiency, we introduce prompt learning to multi-source UDA.

Original: Since each prompt contains substantially fewer parameters compared to a feature extractor, learning distinct prompts for each source and target pair is affordable.
Revised: Since each prompt contains substantially fewer parameters compared to a feature extractor, learning distinct prompts for each source and target pair is affordable.

Original: 2. P ROMPT L EARNING
Revised: 2. P ROMPT L EARNING

Original: Traditionally, given a pre-trained language model, a common approach in deep learning is finetuning the whole model or its task-specific heads to adjust to downstream tasks.
Revised: Traditionally, given a pre-trained language model, a common approach in deep learning is finetuning the whole model or its task-specific heads to adjust to downstream tasks.

Original: While effective, however, two main drawbacks exist.
Revised: While effective, however, two main drawbacks exist.

Original: First of all, as the model size keeps increasing, pre-training and fine-tuning has become more and more expensive.
Revised: First of all, as the model size keeps increasing, pre-training and fine-tuning is becoming more and more expensive.

Original: Secondly, for each diverse task, fine-tuning needs to be repeatedly conducted.
Revised: Secondly, for each diverse task, fine-tuning needs to be repeatedly conducted.

Original: Recently, researchers have shown that learned large-scale language models can handle a wide range of downstream tasks with only a few or even no samples by prepending instructions to the input text (Liu et al., 2021b).
Revised: Recently, researchers have shown that learned large-scale language models can handle a wide range of downstream tasks with only a few or even no samples by prepending instructions to the input text (Liu et al., 2021b).

Original: Such instruction texts are called prompts.
Revised: Such instruction texts are called prompts.

Original: Consequently, prompts can be tuned instead of the entire network for a more efficient adaptation to downstream tasks.
Revised: Consequently, prompts can be tuned instead of the entire network for a more efficient adaptation to downstream tasks.

Original: Originally, prompts are essentially sequences of manually designed language tokens that are mapped to an embedding space.
Revised: Originally, prompts are essentially sequences of manually designed language tokens that are mapped to an embedding space.

Original: To date, extensive research has demonstrated that training soft prompts, i.e. , prompts with their own parameters learned by deep models, is more effective (Li & Liang, 2021; Lester et al., 2021).
Revised: To date, extensive research has demonstrated that training soft prompts, i.e. , prompts with their own parameters learned by deep models, is more effective (Li & Liang, 2021; Lester et al., 2021).

Original: The success of prompt learning in NLP has also garnered attention in the vision community that motivated the establishment of many related work.
Revised: The success of prompt learning in NLP has also garnered attention in the vision community that motivated the establishment of many related work.

Original: To name a few, Zhou et al.
Revised: To name a few, Zhou et al.

Original: (2022) are the first to apply soft prompt learning to the image recognition task.
Revised: (2022) are the first to apply soft prompt learning to the image recognition task.

Original: Ju et al.
Revised: Ju et al.

Original: (2021) explore prompt learning for efficient and lightweight video understanding.
Revised: (2021) explore prompt learning for efficient and lightweight video understanding.

Original: While prompting in these studies are limited to the input of text encoders, Jia et al.
Revised: While prompting in these studies are limited to the input of text encoders, Jia et al.

Original: (2022) prepend token embeddings directly to the image patches.
Revised: (2022) prepend token embeddings directly to the image patches.

Original:  M ETHOD
Revised:  M ETHOD

Original: Our goal is to use multiple labeled source domains for improved performance on a target domain while only tuning a small set of parameters.
Revised: Our goal is to use multiple labeled source domains for improved performance on a target domain while only tuning a small set of parameters.

Original: To this end, we leverage prompt learning, which is an effective strategy by learning a small set of parameters to adapt a pretrainedlarge-scale model to different downstream tasks.
Revised: To this end, we leverage prompt learning, which is an effective strategy by learning a small set of parameters to adapt a pretrained model to different downstream tasks.

Original: In the following, we first review prompt learning in CLIP in Section 3.1 and then elaborate in Section 3.2 our proposed MPA method, which is a two-stage framework that seeks agreement from multiple learned prompts and derives a joint latent space with domain-invariant knowledge.
Revised: In the following, we first review prompt learning in CLIP in Sec. 3.1 and then elaborate in Sec. 3.2 our proposed MPA method, which is a two-stage framework that seeks agreement from multiple learned prompts and derives a joint latent space with domain-invariant knowledge.

Original: 3. A N O VERVIEW OF P ROMPT L EARNING IN CLIP
Revised: 3. A N O VERVIEW OF P ROMPT L EARNING IN CLIP

Original: CLIP consists of an image encoder and a text encoder, which are jointly trained with a contrastive loss on 400M image and text pairs.
Revised: CLIP consists of an image encoder and a text encoder that are jointly trained with a contrastive loss on 400M image and text pairs.

Original: The image encoder f , which can either be a ResNet (He et al., 2016) or a Vision Transformer (Dosovitskiy et al., 2021), maps raw images to an embedding space, while the text encoder g is a Transformer (Vaswani et al., 2017) that maps an input text sequence to the same embedding space.
Revised: The image encoder f , which can either be a ResNet (He et al., 2016) or a Vision Transformer (Dosovitskiy et al., 2021), maps raw images to an embedding space, and the text encoder g is a Transformer (Vaswani et al., 2017) that maps an input text sequence to the same embedding space.

Original: A prompt in CLIP usually exists in the form of “a photo of [CLS]” where [CLS] is a class token that can be replaced by a certain class name.
Revised: A prompt in CLIP usually exists in the form of “a photo of [CLS]” where [CLS] is a class token that can be replaced by a certain class name.

Original: This sequence of tokens is first converted into a lower cased byte pair encoding (BPE) representation, which is essentially a unique numeric ID (Zhou et al., 2022).
Revised: This sequence of tokens is first converted into a lower cased byte pair encoding (BPE) representation, which is essentially a unique numeric ID (Zhou et al., 2022).

Original: Then the numeric IDs are embedded to a 512 dimension vector that is further passed to the Transformer text encoder.
Revised: Then the numeric IDs are embedded to a 512 dimension vector that is further passed to the Transformer text encoder.

Original: In our work, instead of using manually crafted prompts, we train soft prompts that are directly embedded by the text encoder.
Revised: In our work, instead of using manually crafted prompts, we train soft prompts that are directly embedded by the text encoder.

Original: Given an image x and a text embedding w k for class k ∈ { 1 , 2 , ..., K } , where K is the total number of categories, CLIP aligns them in a contrastive manner so that:
Revised: Given an image x and a text embedding w k for class k ∈ { 1 , 2 , ..., K } , where K is the total number of categories, CLIP aligns them in a contrastive manner so that:

Original: [Equation]
Revised: [Equation]

Original: is maximized when the input image x indeed belongs to class k .
Revised: is maximized when the input image x indeed belongs to class k .

Original: Here < · , · > denotes the cosine similarity and T is a learnable temperature parameter.
Revised: Here < · , · > denotes the cosine similarity and T is a learnable temperature parameter.

Original: 3. M ULTI -P ROMPT A LIGNMENT
Revised: 3. M ULTI -P ROMPT A LIGNMENT

Original: Let N denote the total number of domains, where the first N − 1 domains are source domains and the N -th domain is the target domain.
Revised: Let N denote the total number of domains, where the first N − 1 domains are source domains and the N -th domain is the target domain.

Original: For all N − 1 source domains, both images and their labels are provided, while for the target domain, we only assume access to their images, as in the standard UDA setting.
Revised: For all N − 1 source domains, both images and their labels are provided, while for the target domain, we only assume access to their images, as in the standard UDA setting.

Original: For multi-source UDA, we wish to learn a domain-invariant latent space so that the domain shift among different source domains as well as the discrepancies between all the source and target domain pairs can be minimized.
Revised: For multi-source UDA, we wish to learn a domain-invariant latent space so that the domain shift among different source domains as well as the discrepancies between all the source and target domain pairs can be minimized.

Original: Unlike current multi-source UDA methods that require a significant amount of parameters for adaption, we leverage prompt learning for alignment among different domains.
Revised: Unlike current multi-source UDA methods that require a significant amount of parameters for adaption, we leverage prompt learning for alignment among different domains.

Original: In the following, we begin by introducing the prompt design, followed by multiprompt alignment.
Revised: In the following, we begin by introducing the prompt design, followed by learning individual prompts and multi-prompt alignment.

Original: Then we describe how to learn a joint embedding space that has the potential of generalizing to unseen domains.
Revised: Finally we describe how to learn a joint embedding space that has the potential of generalizing to unseen domains.

Original: Prompt Design.
Revised: Prompt Design.

Original: Following (Ge et al., 2022), our prompt for multi-source UDA includes a set of class-specific context vectors v ki , i ∈ { 1 , 2 , ..., M 1 } , k ∈ { 1 , 2 , ..., K } and another set of domainspecific vectors shared across all classes d dj , j ∈ { 1 , 2 , ..., M 2 } , d ∈ { s, t } .
Revised: Following (Ge et al., 2022), our prompt for multi-source UDA includes a set of class-specific context vectors v ki , i ∈ { 1 , 2 , ..., M 1 } , k ∈ { 1 , 2 , ..., K } and another set of domainspecific vectors shared across all classes d dj , j ∈ { 1 , 2 , ..., M 2 } , d ∈ { s, t } .

Original: See Figure 2 for an overview.
Revised: See Figure 2 for an overview.

Original: Here, M 1 and M 2 represent the number of tokens, K is the number of classes and s is short for source and t is short for target, resulting in a total of 2 K categories for training.
Revised: Here, M 1 and M 2 represent the number of tokens, K is the number of classes, s is short for source and t is short for target, resulting in a total of 2 K categories for training.

Original: In other words, each class prompt t dk ∈ R 1 × ( M 1 + M 2 ) × 512 is a concatenation of a “source prompt” and a “target prompt”.
Revised: In other words, each class prompt t dk ∈ R 1 × ( M 1 + M 2 ) × 512 is a concatenation of a “source prompt” and a “target prompt”.

Original: Therefore, the prompt for each source and target pair can be derived as:
Revised: Therefore, the prompt for each source and target pair can be derived as:

Original: [Equation]
Revised: [Equation]

Original:  These prompts serve as learnable parameters that help bridging the domain gap between a source domain and the target domain through a contrastive loss, as will be introduced below.
Revised: [Equation] [Equation] [Equation] These prompts serve as learnable parameters that help bridging the domain gap between a source domain and the target domain through a contrastive loss, as will be introduced below.

Original: Learning Individual Prompts.
Revised: Learning Individual Prompts.

Original: To apply prompt learning to multi-source UDA, we first train individual prompts for each source and target pair using the image and text encoders of CLIP.
Revised: To apply prompt learning to multi-source UDA, we first train individual prompts for each source and target pair using the image and text encoders of CLIP.

Original: Given an image x s sampled from the source domain D s whose label is y ∗ , we optimize the prompts so that the outputs from the image and text encoder are aligned. 
Revised: Given an image x s sampled from the source domain D s whose label is y ∗ , we optimize the prompts so that the outputs from the image and text encoder are aligned. For an image x t from the target

Original: For an image x t from the target domain D t whose label is unknown, we first leverage the strong zero-shot ability of CLIP to generate static pseudo-label ˆ y ∗ for image-text alignment. We only generate pseudo-labels for images whose maximum probability is larger than a fixed threshold τ using Equation 1. While one could use more sophisticated approaches like self-training to generate pseudo labels (Zou et al., 2018; Liu et al.,
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Revised: s T

Original: [Figure]
Revised: [Figure]

Original:  T
Revised: s T

Original: [Figure]
Revised: [Figure]

Original: sK T t  T 
Revised: sK T t Tt TtK T ..

Original: [Figure]
Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: 11 v 12 v

Revised: [Figure]

Revised: v 21 v 22 v

Original: t TtK [Figure] [Figure] 
Revised: [Figure] v

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]

Original: KM

Original: [Figure]
Revised: [Figure]

Original: d
Revised: v

Original: [Figure]
Revised: [Figure]

Original: d
Revised: v

Original: [Figure]
Revised: [Figure]

Original: sM [Figure] CLS s [Figure]
Revised: KM v

Original: d

Original: [Figure]
Revised: [Figure]

Original: s d
Revised: s d

Original: [Figure]

Original: sM d

Original: [Figure]
Revised: [Figure]

Original: CLS s t

Original: [Figure]

Original: s d
Revised: s d

Original: [Figure]
Revised: [Figure]

Original: s d

Original: [Figure] sM d [Figure] CLS sK t t d t  d 
Revised: sM d 2 CLS s t 1 s ds dsM d 2 CLS s ts ds dsM d 2 CLS sK t t d t dtM d 2 CLS t t 1 t dt dtM d 2 CLS t tt dt dtM d 2 CLS tK t ...

Original: [Figure]
Revised: [Figure]

Original: tM d 2 CLS t t 1 t d
Revised: 11 v 12 v

Original: [Figure]
Revised: [Figure]

Original: t d [Figure] tM d [Figure]
Revised: 21 v 22 v

Original: CLS t t

Original: [Figure]
Revised: [Figure]

Original: t d [Figure] t d
Revised: v

Original: [Figure]
Revised: [Figure]

Original: tM d

Original: [Figure]
Revised: [Figure]

Original: CLS tK t ... [Figure] [Figure] [Figure]
Revised: domain D t whose label is unknown, we first leverage the strong zero-shot ability of CLIP to generate static pseudo-label ˆ y ∗ for image-text alignment.

Original: [Figure]

Original: v [Figure] 21 v 22 v [Figure]
Revised: Pseudo-labels are only generated for images whose maximum probability is larger than a fixed threshold τ using Equation 1.

Original: 2021a) , we find that pseudo labels from CLIP are simple and effective.
Revised: While more sophisticated approaches like self-training could be leveraged to generate pseudo labels (Zou et al., 2018; Liu et al., 2021a), we find that pseudo labels from CLIP are simple and effective.

Original: We then train prompts  with cross-entropy loss functions. Figure 2 gives an overview of the process.
Revised: Finally, prompts are trained with cross-entropy loss functions and Figure 2 gives an overview of the process.

Original: More formally, for a prompt P i , i ∈ { 1 , 2 , ..., N − 1 } , the objective function for optimization follows:
Revised: More formally, for a prompt P i , i ∈ { 1 , 2 , ..., N − 1 } , the objective function for optimization follows:

Original: [Equation]
Revised: [Equation]

Original: Here, the probability P ( ·| x d ; P i ) of an image sample belonging to the k -th class is derived from a contrastive loss:
Revised: Here, the probability P ( ·| x d ; P i ) of an image sample belonging to the k -th class is derived from a contrastive loss:

Original: [Equation]
Revised: [Equation]

Original: where d ∈ { s, t } is a domain identifier indicating where the image comes from, T is a learnable temperature parameter, and f and g represents the image and text encoder in CLIP respectively, which are kept frozen during training.
Revised: where d ∈ { s, t } is a domain identifier indicating where the image comes from, T is a learnable temperature parameter, and f and g represents the image and text encoder in CLIP respectively, which are kept frozen during training.

Original: This specific design can push the prompts to learn disentangled representation of both class-invariant and class-specific semantic information to boost the performance of domain adaptation methods (Bousmalis et al., 2016; Liu et al., 2018).
Revised: This specific design can push the prompts to learn disentangled representation of both class-invariant and class-specific semantic information to boost the performance of domain adaptation methods (Bousmalis et al., 2016; Liu et al., 2018).

Original: Once prompts are learned, the predicted label of an image x t can be computed as:
Revised: Once prompts are learned, the predicted label of an image x t can be computed as:

Original: [Equation]
Revised: [Equation]

Original: Multi-Prompt Alignment.
Revised: Multi-Prompt Alignment.

Original: Thus far, we have obtained a prompt for each source and target domain pair.
Revised: Thus far, we have obtained a prompt for each source and target domain pair.

Original: However, the number of images as well as the noise level in each source domain varies, and hence these learned prompts might produce inconsistent results even for the same image.
Revised: However, the number of images as well as the noise level in each source domain varies, and hence these learned prompts might produce inconsistent results even for the same image.

Original: In the second stage, we aim to align predictions from different prompts and more importantly, we wish to find a domain-invariant latent space that minimizes the noise in learned prompts as well as can potentially generalize to unseen domains.
Revised: In the second stage, we aim to align predictions from different prompts and more importantly, we wish to find a domain-invariant latent space that minimizes the noise in learned prompts as well as can potentially generalize to unseen domains.

Original: To this end, we leverage auto-encoders that are trained to reconstruct the learned prompts.
Revised: To this end, we leverage auto-encoders that are trained to reconstruct the learned prompts.

Original: More formally, we use two separate auto-encoders, each consisting of a projection function Proj ( · ) and a back-projection function Proj b ( · ).
Revised: More formally, we use two separate auto-encoders, each consisting of a projection function Proj ( · ) and a back-projection function Proj b ( · ).

Original: The learned prompts P i are first projected into a latent subspace of a lower dimension d
Revised: The learned prompts P i are first projected into a latent subspace of a lower dimension d

Original: I by Proj ( · ) , followed by Proj b ( · ) projecting the vectors back into soft prompts ˆ P i .
Revised: I by Proj ( · ) , followed by Proj b ( · ) projecting the vectors back into soft prompts ˆ P i .

Original: Instead of projecting the whole prompt, we adjust P i to only contain the target token segment and feed its domain-specific and domaininvariant pieces to the two auto-encoders respectively.
Revised: Instead of reconstructing the whole prompt, we adjust P i to only contain the target token segment and feed its domain-specific and domain-invariant pieces to the two auto-encoders respectively.

Original: We posit that since these two feature vectors serve different purposes, using two separate auto-encoders will help with the alignment process.
Revised: We posit that since these two feature vectors serve different purposes, using two separate auto-encoders will help with the alignment process.

Original: The
Revised: The

Original: Proj ( · ) function is implemented by a one-layer feed forward network while Proj b ( · ) is implemented by a two-layer nonlinear perceptron:[Equation] and Proj b ( [Equation]
Revised: Proj ( · ) function is implemented by a one-layer feed forward network while Proj b ( · ) is implemented by a two-layer nonlinear perceptron:

Original: tanh ( W 2 d I + b

Original: [Equation]

Original: + b

Original: [Equation]

Original: [Equation]

Original: and we optimize a reconstruction loss

Original: [Equation]
Revised: [Equation]

Original: where ˆ P

Original: [Equation]

Original: Proj

Original: [Equation]

Original: P

Original: [Equation]

Original: ]: :, :, K [P i repeat K i
Revised: ]: :, :, K [P i repeat K i

Original: Pˆ 512 M K 

Revised: Pˆ 512 M K

Original:   11 v 21 v
Revised:   11 v 21 v

Original: K v12 v 1 1 M v 22 v 2 1 M v
Revised: K v12 v 1 1 M v 22 v 2 1 M v

Original: K v
Revised: K v

Original: KM v11 ˆ v 21 ˆ v
Revised: KM v11 ˆ v 21 ˆ v

Original: K v 1 ˆ 12 ˆ v 1 1 ˆ M v 22 ˆ v 2 1 ˆ M v
Revised: K v 1 ˆ 12 ˆ v 1 1 ˆ M v 22 ˆ v 2 1 ˆ M v

Original: K v 2 ˆ
Revised: K v 2 ˆ

Original: KM v 1 ˆ t dt dtM dt d 1 ˆ t d 2 ˆ tM d 2 ˆ
Revised: KM v 1 ˆ t dt dtM dt d 1 ˆ t d 2 ˆ tM d 2 ˆ

Original: M K 1 
Revised: M K 1 

Original:  I
Revised:  I

Original: d  1 KM
Revised: d  1 KM

Original: I d  2 M 512 M2   512 M2   (a) Reconstruction of prompts tune ˆ d tune ˆ v repeat K P~
Revised: I d  2 M 512 M2   512 M2   (a) Reconstruction of prompts tune ˆ d tune ˆ v repeat K P~

Original: PromptTuning
Revised: PromptTuning

Original: Feature
Revised: Feature

Original: Trainable parameters Fixed parameters 512 )
Revised: Trainable parameters Fixed parameters 512 )

Original: M M( K 2   

Revised: M M( K 2  

Original: M K 1   512 M2  
Revised: M K 1   512 M2  

Original: I d  1 KM
Revised: I d  1 KM

Original: I d  2 M 
Revised: I d  2 M (

Original: [Figure]
Revised: [Figure]

Revised: ) Generalize to novel domain

Revised: Here, d I represents a vector in the embedding space and we optimize a reconstruction loss

Revised: [Equation]

Original:  Intuitively, one would expect a certain target domain image to be classified as the same category for all the reconstructed prompts
Revised: where [Equation] Intuitively, one would expect a certain target domain image to be classified as the same category for all the reconstructed prompts

Original: ˆ P i .
Revised: ˆ P i .

Original: Inspired by this, we align the reconstructed prompts by introducing an additional L 1 loss to the objective function[Equation] |
Revised: Inspired by this, we align the reconstructed prompts by introducing an additional L 1 loss to the objective function

Original: [Equation]
Revised: [Equation]

Original: The overall loss function now becomes:
Revised: The overall loss function now becomes:

Original: [Equation]
Revised: [Equation]

Original: where L CLS is the cross entropy loss calculated using the reconstructed prompts ˆ P i and static pseudo labels.
Revised: where L CLS is the cross entropy loss calculated using the reconstructed prompts ˆ P i and static pseudo labels.

Original: Here α is a hyper-parameter controlling the weight of the L 1 loss.
Revised: Here α is a hyper-parameter controlling the weight of the L 1 loss.

Original: The whole training procedure is depicted in Figure 3a.
Revised: The whole training procedure is depicted in Figure 3a.

Original: Finally, for predicting the labels of target samples, we compute the average of the output logits using each ˆ P i .
Revised: Finally, for predicting the labels of target samples, we compute the average of the output logits using each ˆ P i .

Original: Generalizing to Unseen Domains.
Revised: Generalizing to Unseen Domains.

Original: Shown in Figure 3b, with multi-prompt alignment, we are able to derive a low-dimensional embedding space that potentially captures the relationships among different domains.
Revised: With multi-prompt alignment, we are able to derive a lowdimensional embedding space that potentially captures the relationships among different domains.

Original: With this, we introduce a L atent S ubspace T uning (LST) strategy.
Revised: With this, we introduce a L atent S ubspace T uning (LST) strategy.

Original: Given a novelunseen domain, a domain-invariant feature vector v tune ∈ R N × M 1 ×
Revised: Given a novel domain, a domain-invariant feature vector v tune ∈ R N × M 1 ×

Original: d
Revised: d

Original: I with a domain-specific feature vector d tune ∈ R 1 × M 2 ×
Revised: I with a domain-specific feature vector d tune ∈ R 1 × M 2 ×

Original: d I is randomly initialized and passed to the two learned back projection function Proj b ( · ) from MPA.
Revised: d I is randomly initialized and passed to the two learned back projection function Proj b ( · ) from MPA.

Original: Consequently, an entirely new prompt ˜ P can be constructed for predicting image labels on the “unseen” domain.
Revised: Consequently, an entirely new prompt ˜ P can be constructed for predicting image labels on the novel domain.

Original: This is achieved by minimizing the following loss function:
Revised: This is achieved by minimizing the following loss function:

Original: [Equation]
Revised: [Equation]

Original: By doing so, this further decreases the number of tunable parameters by a factor of at least ( N − 1)
Revised: By doing so, this further decreases the number of tunable parameters by a factor of at least ( N − 1)

Original: ×
Revised: ×

Original: d c d I times (for example, on the Office-Home dataset, d I = 150 , d c = 512 and N = 4 ) as against to MPA when adapting to novel domains and is thus much more computationally efficient.
Revised: d c d I times (for example, on the Office-Home dataset, d I = 150 , d c = 512 and N = 4 ) as against to MPA when adapting to novel domains and is thus much more computationally efficient.

Revised: Since data from the source domains are not required in the training of LST, this is in fact a problem of test time adaptation (Kundu et al., 2020;

Revised: Liang et al., 2020; Wang et al., 2021).

Revised: Considering that the domain of interest is not involved in the training of the latent subspace throughout the whole process, we dub it “unseen”.

Original:  E XPERIMENTS 4. E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 4. E XPERIMENTAL S ETUP

Original: Datasets and metrics.
Revised: Datasets and metrics.

Original: We conduct experiments on three popular benchmark datasets of UDA to evaluate the effectiveness of MPA, including ImageCLEF, Office-Home and DomainNet.
Revised: Experiments are conducted on three popular benchmark datasets of UDA to evaluate the effectiveness of MPA, including ImageCLEF, Office-Home and DomainNet.

Original: ImageCLEF is a small-scaled dataset consisting of 1,800 images from 12 object categories in 3 different domains: ImageNet ILSVRC 2012(I), Pascal VOC 2012(P), and Caltech-256 (C).
Revised: ImageCLEF is a small-scaled dataset consisting of 1,800 images from 12 object categories in 3 different domains: ImageNet ILSVRC 2012(I), Pascal VOC 2012(P), and Caltech-256 (C).

Original: Office-Home is a medium scaled dataset consisting of about 15,500 images from 65 categories in 4 different domains: Art, Clipart, Product and Real World.
Revised: Office-Home is a medium scaled dataset consisting of about 15,500 images from 65 categories in 4 different domains: Art, Clipart, Product and Real World.

Original: DomainNet is the largest dataset to date, consisting of about 0.6 million images from 345 categories in 6 different domains: clipart, infograph, painting, quickdraw, real and sketch.
Revised: DomainNet is the largest dataset to date, consisting of about 0.6 million images from 345 categories in 6 different domains: Clipart, Infograph, Painting, Quickdraw, Real and Sketch.

Original: The top-1 accuracyis used as our evaluation metric. Implementation Details.
Revised: We use top-1 accuracy as our evaluation metric and report results of the following settings: (1)

Original: For fair comparisons, we adopt a ResNet50 as our backbone on ImageCLEF and Office-Home and a ResNet101 on DomainNet. The weights for ResNet50 and ResNet are from CLIP and frozen through our experiments.
Revised: CLIP: zero-shot CLIP on the target domain, which can be regarded as a baseline of our method.

Original: The prompts and auto-encoders are trained using the mini-batch SGD optimizer with a learning rate of 0.003 and 0.005 while the learned subspace is tuned with 0.0005 learning rate. We use a batch size of 32 and adopt a cosine learning rate scheduler. For the hyper-parameters of stage one, token length M 1 and M 2 are both set to 16. Pseudo-label threshold τ is set to 0.4 for producing reliable labels.
Revised: (2) Source Combined: all source domains are combined into one single domain and applied with popular single-source UDA methods.

Original: As for stage two, α in Equation 9 is set to 500.
Revised: Specially in this setting, we adopt the prompting method from

Original: The weight matrix of the back projection function W 2 has a size of R 384 × d I , where d I is 100 for ImageCLEF, 150 for OfficeHome and 250 for DomainNet.
Revised: Zhou et al.

Original: Therefore, for generalizing to novel domains, only 0.02M, 0.17M and 1.47M parameters are tuned respectively. We report results of the following settings: (1) CLIP: zero-shot CLIP on the target domain, which can be regarded as the baseline of our method. (2) Source Combined: all source domains are combined into one single domain and then popular single-source UDA methods are applied.
Revised: (2022) to serve as another baseline named as “Simple Prompting”.

Original: (3) Multi-Source: results reported from other multi-source UDA methods.
Revised: (3) Multi-Source: results reported from other multi-source UDA methods.

Original: Note that we tried to re-implement state-of-the-art methods using backbone networks from CLIP, yet  the results are unsatisfactory.
Revised: Note that we tried to re-implement state-ofthe-art methods using backbone networks from CLIP, yet most of the results are unsatisfactory and thus we only report one such attempt.

Original: Similar trends are also found in (Devillers et al., 2021;
Revised: Similar trends are also found in (Devillers et al., 2021;

Original: Yang et al., 2022) when transferring CLIP to pure vision tasks.
Revised: Yang et al., 2022) when transferring CLIP to pure vision tasks.

Original: 4. C OMPARISON TO S TATE - OF - THE -A RT
Revised: 4. C OMPARISON TO S TATE - OF - THE -A RT

Original: We compare MPA with state-of-the-art methods on three datasets, including DAN (Long et al., 2015b), D-CORAL (Sun & Saenko, 2016), DANN (Ganin et al., 2016), MCD (Saito et al., 2018),

Original: DCTN (Xu et al.

Original: [Table]

Original: [Table] [Table] [Table] The results on ImageCLEF and Office-Home are shown in Table 1.
Revised: The results on ImageCLEF and Office-Home are shown in Table 1.

Original: For ImageCLEF, it is obvious that MPA outperforms other methods on every task with an average accuracy of 91.7%, where there is  a 3% increase when adapting to domain C and I. For Office-Home, MPA achieves the best results except when adapting to the domain Clipart.
Revised: For ImageCLEF, it is obvious that MPA outperforms other methods on every task with an average accuracy of 91.7%, where there is at least a 3% increase when adapting to domain C and I. For Office-Home, MPA achieves the best results except when adapting to the domain Clipart.

Original: Nevertheless, we achieve an accuracy of 75.4% on average, which is 1.3% higher than the second best method MFSAN.
Revised: Nevertheless, we achieve an accuracy of 75.4% on average, which is 1.3% higher than the second best method MFSAN.

Original: It is worth noting that compared to state-of-the-art method MFSAN on both datasets, MPA only trains 0.78M and 2.36M parameters, while MFSAN has a total of 51.75M and 51.80M parameters  for optimizing (66.3 and 21.9 times larger than ours).
Revised: It is worth noting that compared to state-of-the-art method MFSAN on both datasets, MPA only trains 0.78M and 2.36M parameters, while MFSAN has a total of 51.75M and 51.80M parameters needed for optimizing (66.3 and 21.9 times larger than ours).

Revised: Furthermore, to justify whether the performance gain is from a more powerful backbone, we conduct two different sets of experiments.

Revised: One is applying a simple prompt learning method (Zhou et al., 2022) to the source combined scenario; and again, as

Original: Table 2 shows that for DomainNet, MPA improves other multi-source UDA methods by more than 5%.
Revised: Table 1 suggests, while the Simple Prompting baseline is 1% on average better than zero shot CLIP, MPA still outperforms it with a significant margin. The other experiment is by directly switching MFSAN’s ResNet50 backbone pretrained on ImageNet to CLIP’s image encoder.

Revised: Surprisingly, when tested on the ImageCLEF dataset, the performance even drops by a small margin of 0.3%.

Revised: To conclude, both results demonstrate that CLIP’s backbone is not universally better and the majority of the performance gain of our method is from its domain adaptation ability.

Revised: [Table]

Revised: Table 2 shows that for DomainNet, MPA exceeds other multi-source UDA methods by more than 5%.

Original: To the best of our knowledge, this is the highest reported accuracy on this dataset so far with less than one third parameters optimized compared with most state-of-the-arts methods. 
Revised: To the best of our knowledge, this is the highest reported accuracy on this dataset so far with less than one third parameters optimized compared with most state-of-the-arts methods. Regard-

Original: Regarding individual adaptations, MPA achieves best results on most of the adaptation tasks but performs mediocre on the quickdraw domain.
Revised: [Table] ing individual adaptations, MPA achieves best results on most of the adaptation tasks but performs mediocre on the Quickdraw domain.

Original: Interestingly, the result is even a little worse than CLIP.
Revised: Interestingly, the result is even a little worse than CLIP.

Original: We hypothesize that this is because of the large domain gap between quickdraw and other domains.
Revised: We hypothesize that this is because of the large domain gap between Quickdraw and other domains.

Original: While
Revised: While

Original: CLIP exhibits strong performance on the DomainNet dataset, its performance on ImageCLEF and
Revised: CLIP exhibits strong performance on the DomainNet dataset, its performance on ImageCLEF and

Original: [Table]

Original: OfficeHome dataset is limited.
Revised: OfficeHome dataset is limited.

Original: On the contrary, MPA  achievesconsistent decent results regardless of the dataset assessed on.
Revised: On the contrary, MPA consistently achieves decent results regardless of the dataset assessed on.

Original: Specifically, for all three datasets, MPA surpasses CLIP by 6.2%, 4.0% and 3.6% respectively, demonstrating the success of our strategy on reducing domain discrepancies.
Revised: As a result, for all three datasets, MPA surpasses CLIP by 6.2%, 4.0% and 3.6% respectively. More importantly, the Simple Prompt baseline reached an average accuracy of 50.6%, 3.5% lower than MPA. All the above results further demonstrate the success of our strategy.

Original: Effectiveness of Generalizing to Unseen Domains Inspired by recent research that shows the generalization ability of large language models when adapting to unseen tasks presented withnatural language prompts (Sanh et al., 2022), we are interested in whether MPA also possess such ability.
Revised: Effectiveness of Generalizing to Unseen Domains Inspired by recent research that shows the generalization ability of large language models when adapting to unseen tasks presented with language prompts (Sanh et al., 2022), we are interested in whether MPA also possess such ability.

Original: Results from Table 1 and Table 2 are already strong indication of the success of MPA under conventional multi-source UDA setting.
Revised: Results from Table 1 and Table 2 are already strong indication of the success of MPA under conventional multi-source UDA setting.

Original: Now, we would like to investigate a more efficient framework, the
Revised: Now, we would like to investigate a more efficient framework, the

Original: LST strategy as mentioned in Section 3.2, for fast adaptation tasks.
Revised: LST strategy as mentioned in Section 3.2, for test-time adaptation tasks.

Original: We first present a concrete example.
Revised: We first present a concrete example that simulates a test-time adaptation scenario.

Original: On the Office-Home dataset, LST can be conducted by training MPA using Clipart and Product as source domains and Art as target domain, while fine-tuning the latent subspace on the unseen Real World domain.
Revised: On the OfficeHome dataset, LST can be conducted by training MPA using Clipart and Product as source domains and Art as target domain, while fine-tuning the latent subspace on the unseen Real World domain.

Original: Applying this to all domains from the OfficeHome and DomainNet dataset produces results shown in Table 3, where compared with MPA, the performance only dropped by 0.4% on DomainNet and 1.6% on Office-Home.
Revised: Applying this to all domains from the Office-Home and DomainNet dataset produces results shown in Table 3, where compared with MPA, the performance only dropped by 0.4% on DomainNet and 1.6% on Office-Home.

Original: Nevertheless, LST achieves higher accuracy compared to most baseline methods of Table 1 and Table 2.
Revised: Nevertheless, LST achieves higher accuracy compared to most baseline methods of Table 1 and Table 2.

Original: In particular, LST is still better than CLIP with an increase of 2.4% and 3.2% respectively for the two tested datasets.
Revised: In particular, LST is still better than CLIP with an increase of 2.4% and 3.2% respectively for the two tested datasets.

Original: Notably, only a total of 0.17M and 1.47M parameters are tuned, which is a further drop in the number of parameters (0.17M v.s. 2.36M, 1.47M v.s. 15.9M in MPA).
Revised: Notably, only a total of 0.17M and 1.47M parameters are tuned, which is a further drop in the number of parameters (0.17M v.s. 2.36M, 1.47M v.s. 15.9M in MPA).

Original: All of these are convincing evidence showing thestrong generalization ability of MPA.
Revised: All of these are convincing evidence showing the generalization ability of MPA.

Original: [Table]
Revised: [Table]

Original: Qualitative Results To obtain further insights about MPA, we visualize, in Figure 4, the distributions of prediction confidence of MPA and CLIP for  four target domains on the Office-Home dataset.
Revised: Qualitative Results To obtain further insights into MPA, we visualize, in Figure 4, the distributions of prediction confidence of MPA and CLIP for the four target domains on the Office-Home dataset.

Original: We see that for Art, Prouct and Real World, MPA produces highly confident predictions for the majority of samples.
Revised: We see that for Art, Prouct and Real World, MPA produces highly confident predictions for the majority of samples.

Original: For the Clipart domain where the recognition accuracy is lower than the other domains, MPA is clearly better than CLIP that produces a distribution skewed to the bottom(low confidence).
Revised: As for the Clipart domain where the recognition accuracy is lower than the other domains, MPA is clearly better than CLIP that produces a distribution skewed to the bottom

Original: 4. A BLATION S TUDY
Revised: [Table]

Original: In this section, we conduct extensive experiments to give a more detailed analysis of our method. Otherwise stated, all ablation studies are done on the Office-Home dataset for effiency purposes.
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original:  Effectiveness of Prompt
Revised: A BLATION S TUDY Effectiveness of Prompt

Original: Alignment The second stage of our method can be seen as a refinement of the prompt learned during stage one.
Revised: Alignment The second stage of MPA can be seen as a refinement of the prompt learned during stage one.

Original: Thus, we verify how the quality of thoseindividual prompts has changed and the results are shown in Table 4.
Revised: Thus, we verify how the quality of those prompts has changed and the results are shown in Table 4.

Original: It is worth pointing out that stage one of MPA improves the accuracy of CLIP by an average of 1.2%, while the second stage improves the first stage by an average of 2.8%, highlighting that the prompt has  been refined with better domain-invariant information. [Table] [Figure] [Equation]
Revised: It is worth pointing out that stage one of MPA improves the accuracy of CLIP by an average of 1.2%, while the second stage improves the first stage by an average of 2.8%, highlighting that the prompt has indeed been refined.

Original: [Figure]

Original: [Equation]

Original: [Figure]

Original: [Table]
Revised: [Table]

Original: Effectiveness of loss function To validate the effectiveness of our objective function Equation 9, two variants are evaluated: one without the L AE loss and one without the L 1 loss.
Revised: Effectiveness of loss function To validate the effectiveness of our objective function Equation 9, two variants are evaluated: one without the L AE loss and one without the L 1 loss.

Original: Table 5a shows that the removal of either piece will result in a performance degradation of about 0.7%.
Revised: Table 5a shows that the removal of either piece will result in a performance degradation of about 0.7%.

Original: In particular,
Revised: In particular,

Original: L 1 loss appears to have a larger impact on the overall performance. On the Clipart domain,  the accuracy dropped by a large margin of 1.5% without the L 1 loss, demonstrating the effectiveness of enforcing consistent predictions on different prompts.
Revised: L 1 loss exhibits a larger impact on the more difficult Clipart domain, where the accuracy dropped by a large margin of 1.5% without the L 1 loss, demonstrating the effectiveness of enforcing consistent predictions on different prompts.

Original: [Table]
Revised: [Table]

Original: Effectiveness of auto-encoders In addition to the objective function, we also tested the effectiveness of utilizing the auto-encoder structure in MPA.
Revised: Effectiveness of auto-encoders In addition to the objective function, we also tested the effectiveness of utilizing the auto-encoder structure in MPA.

Original: For details, zero auto-encoders means we completely discarded the auto-encoder structure and one auto-encoder is for testing the necessity of using two separate auto-encoders for reconstructing domain-specific and domain-invariant prompt.
Revised: For details, zero auto-encoders means we completely discarded the auto-encoder structure and one auto-encoder is for testing the necessity of using two separate auto-encoders for reconstructing domain-specific and domain-invariant prompt.

Original: Results from Table 5b show that in general, our design of the auto-encoder structure is beneficial to the overall performance.
Revised: Results from Table 5b show that in general, our design of the auto-encoder structure is beneficial to the overall performance.

Revised: Furthermore, we find that another benefit of applying the auto-encoder structure is that it helps stabilizing the training process.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we introduced prompt learning to multi-source UDA and proposed a simple MPA scheme to align the source and target domains.
Revised: In this paper, we introduced prompt learning to multi-source UDA and proposed a simple MPA scheme to align the source and target domains.

Original: MPA is composed of two stages.
Revised: MPA is composed of two stages.

Original: The first stage is to train individual source and target pair prompts and the second stage is to align them by autoencoder structures.
Revised: The first stage is to train individual source and target pair prompts and the second stage is to align them by auto-encoder structures.

Original: Extensive experiments showed that MPA achieved better results on various multi-source UDA tasks with substantially fewer parameters tuned.
Revised: Extensive experiments showed that MPA achieved better results on various multi-source UDA tasks with substantially fewer parameters tuned.

Original: Moreover, a latent subspace tuning strategy was introduced for generalization to novel unseen domains that further dropped the number of parameters needed for training.
Revised: Moreover, a latent subspace tuning strategy was introduced for generalization to novel unseen domains that further dropped the number of parameters needed for training and can be applied to test-time adaptation tasks.

Original: Hopefully, our work can inspire future work on applying prompt learning to transfer learning.
Revised: Hopefully, our work can inspire future work on applying prompt learning to transfer learning.

Original: A A PPENDIX A. MPA WITH LIMITED DATA
Revised: A A PPENDIX A. I MPLEMENTATION D ETAILS

Revised: For fair comparisons, we adopt a ResNet50 as our backbone on ImageCLEF and Office-Home and a ResNet101 on DomainNet.

Revised: The weights for ResNet50 and ResNet101 are from CLIP and frozen through our experiments.

Revised: The prompts and auto-encoders are trained using the mini-batch SGD optimizer with a learning rate of 0.003 and 0.005 while the learned subspace is tuned with 0.

Revised: learning rate.

Revised: We use a batch size of 32 and adopt a cosine learning rate scheduler.

Revised: For the hyperparameters of stage one, token length M 1 and M 2 are both set to 16.

Revised: Pseudo-label threshold τ is set to 0.4 for producing reliable labels.

Revised: As for stage two, α in Equation 9 is set to 500.

Revised: Ablation studies on hyper-parameter selection is presented in the following section.

Revised: The weight matrix W 2 of the back projection function in Equation 6 has a size of R 384 ×

Revised: d I , where d I is 100 for ImageCLEF,for OfficeHome and 250 for DomainNet.

Revised: Therefore, for generalizing to novel domains, only 0.02M, 0.17M and 1.47M parameters are tuned respectively.

Revised: A. H YPER - PARAMETER SELECTION

Revised: Experimental results for ablation studies on hyper-parameter selections are reported in Table 6.

Revised: [Table]

Revised: For both pseudo-label threshold τ and prompt token length M 1 , M 2 , three different choices τ ∈ { 0 . 3 , 0 . 6 , 0 . 8 } and M 1 , M 2 ∈ { 8 , 12 , 20 } (for simplicity we are setting M 1 and M 2 to be equal) are examined.

Revised: As τ increases, while the quality of the pseudo labels gets higher, fewer images will be fed into the model

Revised: [Table]

Revised: and Table 6a suggests that doing so hurts the overall performance.

Revised: On the contrary, shown in Table 6b

Revised: [Table]

Revised: the general trend for prompt token length is that the longer the prompt, the better the performance.

Revised: Consequently, we choose τ = 0 .

Original: {+4 and M 
+}
Revised: {+4 and M 
+}

Revised: = M 2 = 16 to balance the trade-off between performance and efficiency.

Revised: For α in Equation

Revised: [Table]

Revised: we examined four different choices α ∈ { 1 , 10 , 100 , 1000 } and the main reason we chose 500 for α is to balance all losses (in this case L 1 ) to be of the same order of magnitude.

Revised: Our experimental results from Table 6c also support such motivation.

Revised: A. A BLATION ON P ROMPT D ESIGN

Revised: We also conduct experiments on ablating the necessity of using class and domain specific tokens in our prompt design.

Revised: [Table]

Revised: Note that for a fair comparison, when excluding domain specific tokens in the prompt design, the total number of tokens remains 32, i.e. , M 1 = 32 , M 2 = 0 .

Revised: Still, results from Table 7 demonstrates that the reported prompt design exhibits a significant performance gain.

Revised: A. S IMILARITY BETWEEN RECONSTRUCTED PROMPTS

Revised: As is observed in Table

Revised: [Table]

Revised: the reconstructed prompts of different domains achieve almost the same results on the target domain and this is a further indication of the success of our alignment strategy.

Revised: To justify such statement

Revised: [Table]

Revised: the reconstructed prompts are tested without incorporating L 1 loss in the objective function and the results are shown in Table

Revised: [Table]

Revised: By comparing the standard deviation on the same target domain, it is clear that, without the L 1 loss, the reconstructed prompts achieve different results on the target domain

Original: Since prompt learning is also well-known for its strong few-shot ability,we design a limited data setting  where only a small portion of the data is used for training MPA while still tested on the full dataset to assess whether our method has inherited such ability.
Revised: Since prompt learning is also well-known for its strong few-shot ability, a limited data setting is designed where only a small portion of the data is used for training MPA while still testing it on the full dataset to assess whether our method has inherited such ability.

Original: We think that this framework is also practically meaningful as access to data might be limited in real-life scenario due to privacy, security and storage concerns (Ahmed et al., 2021).
Revised: We think that this framework is also practically meaningful as access to data might be limited in real-life scenario due to privacy, security and storage concerns (Ahmed et al., 2021).

Original: Specifically, no more than 50 images of each category are sampled using two distinct sampling strategies.
Revised: Specifically, no more than 50 images of each category are sampled using two distinct sampling strategies.

Original: The most direct one is by complete random sampling.
Revised: The most direct one is by complete random sampling.

Original: On the contrary, strategy two selects high quality images by utilizing CLIP’s zero shot inference ability similar to our pseudo label generating process.
Revised: On the contrary, strategy two selects high quality images by utilizing CLIP’s zero shot inference ability similar to our pseudo label generating process.

Original: Such setting is only experimented on the DomainNet dataset due to its large data scale.
Revised: Such setting is only experimented on the DomainNet dataset due to its large data scale.

Original: Despite the fact that the total number of data was reduced from 0.6M to roughly 0.1M, results from Tableshow that MPA is still capable of achieving a surprisingly good result where the recognition accuracy only dropped by an average of 0.7% compared with full data training and is still 2.9% better than our baseline CLIP.
Revised: Despite the fact that the total number of data was reduced from 0.6M to roughly 0.1M, results from Tableshow that MPA is still capable of achieving a surprisingly good result where the recognition accuracy only dropped by an average of 0.7% compared with full data training and is still 2.9% better than our baseline CLIP.

Original: We also want to point out that the quality of input images seems to be irrelevant to the performance, as results from the two sampling strategies don’t vary much. 
Revised: We also want to point out that the quality of input images seems to be irrelevant to the performance, as results from the two sampling strategies don’t vary much. [Table] CLIP

Original: [Table]
Revised: [Table]