Original: Cycle Class Consistency with Distributional Optimal Transport and Knowledge Distillation for Unsupervised Domain Adaptation
Revised: Cycle Class Consistency with Distributional Optimal Transport and Knowledge Distillation for Unsupervised Domain Adaptation

Original: Abstract
Revised: Abstract

Original: Unsupervised domain adaptation (UDA) aims to transfer knowledge from a model trained on a labeled source domain to an unlabeled target domain.
Revised: Unsupervised domain adaptation (UDA) aims to transfer knowledge from a model trained on a labeled source domain to an unlabeled target domain.

Original: To this end, we propose in this paper a novel cycle class-consistent model based on optimal transport (OT) and knowledge distillation.
Revised: To this end, we propose in this paper a novel cycle class-consistent model based on optimal transport (OT) and knowledge distillation.

Original: The model consists of two agents, a teacher and a student cooperatively working in a cycle process under the guidance of the distributional optimal transport and distillation manner.
Revised: The model consists of two agents, a teacher and a student cooperatively working in a cycle process under the guidance of the distributional optimal transport and distillation manner.

Original: The OT distance is designed to bridge the gap between the distribution of the target data and a distribution over the source class-conditional distributions.
Revised: The OT distance is designed to bridge the gap between the distribution of the target data and a distribution over the source class-conditional distributions.

Original: The optimal probability matrix then provides pseudo labels to learn a teacher that achieves a good classiﬁcation performance on the target domain.
Revised: The optimal probability matrix then provides pseudo labels to learn a teacher that achieves a good classiﬁcation performance on the target domain.

Original: Knowledge distillation is performed in the next step in which the teacher distills and transfers its knowledge to the student.
Revised: Knowledge distillation is performed in the next step in which the teacher distills and transfers its knowledge to the student.

Original: And ﬁnally, the student produces its prediction for the optimal transport step.
Revised: And ﬁnally, the student produces its prediction for the optimal transport step.

Original: This process forms a closed cycle in which the teacher and student networks are simultaneously trained to conduct transfer learning from the source to the target domain.
Revised: This process forms a closed cycle in which the teacher and student networks are simultaneously trained to conduct transfer learning from the source to the target domain.

Original: Extensive experiments show that our proposed method outperforms existing methods, especially the class-aware and OT-based ones on benchmark datasets including Ofﬁce-31, OfﬁceHome, and ImageCLEF-DA.
Revised: Extensive experiments show that our proposed method outperforms existing methods, especially the class-aware and OT-based ones on benchmark datasets including Ofﬁce-31, OfﬁceHome, and ImageCLEF-DA.

Original:  INTRODUCTION
Revised:  INTRODUCTION

Original: Unsupervised domain adaptation (UDA) allows us to transfer knowledge from a model trained on a source domain with labels to a target domain without any labels.
Revised: Unsupervised domain adaptation (UDA) allows us to transfer knowledge from a model trained on a source domain with labels to a target domain without any labels.

Original: To cope with structural data more efﬁciently and effectively, deep domain adaption (DDA) [Ganin and Lempitsky, 2015] has been proposed and extensively studied.
Revised: To cope with structural data more efﬁciently and effectively, deep domain adaptation (DDA) [Ganin and Lempitsky, 2015] has been proposed and widely studied [Nguyen et al., 2019, 2020, Phung et al., 2021].

Original: To tackle the data shift issue and learn domain-invariant features, DDA aims to bridge the distribution gap between the source and target domains in a latent space using a feature extractor.
Revised: To tackle the data shift issue and learn domain-invariant features, DDA aims to bridge the distribution gap between the source and target domains in a latent space using a feature extractor.

Original: Guided by this principle, most of the existing works in DDA propose minimizing a divergence between the source and target distributions in the latent space.
Revised: Guided by this principle, most of the existing works in DDA propose minimizing a divergence between the source and target distributions in the latent space.

Original: Popular choices of divergence include the Jensen-Shannon (JS) divergence [Ganin and Lempitsky, 2015, Tzeng et al., 2015, Shu et al., 2018], the maximum mean discrepancy (MMD) distance [Gretton et al., 2007, Long et al., 2015], and the Wasserstein (WS) distance [Shen et al., 2018, Lee et al., 2019].
Revised: Popular choices of divergence include the Jensen-Shannon (JS) divergence [Ganin and Lempitsky, 2015, Tzeng et al., 2015, Shu et al., 2018], the maximum mean discrepancy (MMD) distance [Gretton et al., 2007, Long et al., 2015], and the Wasserstein (WS) distance [Shen et al., 2018, Lee et al., 2019, Le et al., 2021a].

Original: Recently, Optimal transport (OT) [Villani, 2008, Santambrogio, 2015], a powerful tool in mathematics with rich and rigorous theories, has been widely applied in deep domain adaptation [Courty et al., 2017b,a, Damodaran et al., 2018,
Revised: Recently, Optimal transport (OT) [Villani, 2008, Santambrogio, 2015], a powerful tool in mathematics with rich and rigorous theories, has been widely applied in deep domain adaptation [Courty et al., 2017b,a, Damodaran et al., 2018,

Original: Redko et al., 2019, Lee et al., 2019, Xie et al., 2019, Xu et al., 2020].
Revised: Redko et al., 2019, Lee et al., 2019, Xie et al., 2019, Xu et al., 2020, Nguyen et al., 2021a,b, Le et al., 2021b, Nguyen et al., 2021c].

Original: From the conceptual perspective, OT-based methods encourage the target samples to move towards the source samples by minimizing a transportation cost.
Revised: From the conceptual perspective, OT-based methods encourage the target samples to move towards the source samples by minimizing a transportation cost.

Original: However, since the transportation cost usually engages the pairs of target and source samples without considering label information of the source samples, the movement of the target samples to the source domain seems to be unaware of the class regions in that domain, hence cannot resolve the label shift issue [Tachet des Combes et al., 2020].
Revised: However, since the transportation cost usually engages the pairs of target and source samples without considering label information of the source samples, the movement of the target samples to the source domain seems to be unaware of the class regions in that domain, hence cannot resolve the label shift issue [Tachet des Combes et al., 2020].

Original: Although
Revised: Although

Original: OT has been initially used for solving this problem [Courty et al., 2017b, Damodaran et al., 2018], the performance of the existing methods is still less satisfactory compared with the state-of-the-art ones.
Revised: OT has been initially used for solving this problem [Courty et al., 2017b, Damodaran et al., 2018], the performance of the existing methods is still less satisfactory compared with the state-of-the-art ones.

Original: In this paper, we propose a novel distributional OT that enables the incorporation of the source label information when engaging and matching target and source samples.
Revised: In this paper, we propose a novel distributional OT that enables the incorporation of the source label information when engaging and matching target and source samples.

Original: Speciﬁcally, in the source domain we consider that one label is associated with a conditional distribution over all the samples conditioned on that label.
Revised: Speciﬁcally, in the source domain we consider that one label is associated with a conditional distribution over all the samples conditioned on that label.

Original: Next, we deﬁne a distribution over these conditional distributions of all the labels in the source domain.
Revised: Next, we deﬁne a distribution over these conditional distributions of all the labels in the source domain.

Original: In the target domain where there are no labels, we also consider a distribution over all the target samples.
Revised: In the target domain where there are no labels, we also consider a distribution over all the target samples.

Original: With the two distributions for the source and target domains respectively, we formulate the
Revised: With the two distributions for the source and target domains respectively, we formulate the

Original: DA problem as the computation of the OT distance between the two distributions.
Revised: DA problem as the computation of the OT distance between the two distributions.

Original: The OT transport plan gives us the information of how a target sample related to the source samples by taking into account the source domain labels.
Revised: The OT transport plan gives us the information of how a target sample related to the source samples by taking into account the source domain labels.

Original: The challenge here is how to deﬁne the cost function, which indicates the transport cost of OT between a target sample and a source class-conditional distribution.
Revised: The challenge here is how to deﬁne the cost function, which indicates the transport cost of OT between a target sample and a source class-conditional distribution.

Original: To tackle this challenge, we propose a cycle class consistency framework in which we leverage the advantages of knowledge distillation (KD) which has recently obtained outstanding achievements [Tian et al., 2020, Zhao et al., 2020, Tejankar et al., 2021,
Revised: To tackle this challenge, we propose a cycle class consistency framework in which we leverage the advantages of knowledge distillation (KD) which has recently obtained outstanding achievements [Tian et al., 2020, Zhao et al., 2020, Tejankar et al., 2021,

Original: Feng et al., 2021].
Revised: Feng et al., 2021].

Original: We name our proposed approach C ycle
Revised: We name our proposed approach C ycle

Original: Class C O nsistency with O ptimal Transport and K nowledge
Revised: Class C O nsistency with O ptimal Transport and K nowledge

Original: Distillation for Unsupervised Domain Adaptation (COOK).
Revised: Distillation for Unsupervised Domain Adaptation (COOK).

Original: In summary, our contributions in this paper include:
Revised: In summary, our contributions in this paper include:

Original: • We propose a novel distributional OT which seeks the optimal matching between the target and source examples taking into account the source label information for reducing the label and data shift, two challenging problems of UDA.
Revised: • We propose a novel distributional OT which seeks the optimal matching between the target and source examples taking into account the source label information for reducing the label and data shift, two challenging problems of UDA.

Original: • We connect KD and OT to further improve the performance of class-aware UDA methods via proposing a cycle class consistency framework where the teacher and student networks cooperatively work in a distillation process and support to reduce the mismatch between the target distribution and the source class-conditional distributions.
Revised: • We connect KD and OT to further improve the performance of class-aware UDA methods via proposing a cycle class consistency framework where the teacher and student networks cooperatively work in a distillation process and support to reduce the mismatch between the target distribution and the source class-conditional distributions.

Original: • We conduct experiments to compare our proposed COOK with the existing standard UDA methods, especially class-aware UDA methods (e.g., RADA [Wang et al., 2019b] and CAN [Kang et al., 2019])
Revised: • We conduct experiments to compare our proposed COOK with the existing standard UDA methods, especially class-aware UDA methods (e.g., RADA [Wang et al., 2019b] and CAN [Kang et al., 2019])

Original: and OTbased UDA methods (e.g., DeepJDOT [Damodaran et al., 2018], ETD [Li et al., 2020], and RWOT [Xu et al., 2020]).
Revised: and OTbased UDA methods (e.g., DeepJDOT [Damodaran et al., 2018], ETD [Li et al., 2020], and RWOT [Xu et al., 2020]).

Original: The experimental results show that our proposed method surpasses the baselines on the benchmark datasets including Ofﬁce-31 , Ofﬁce -Home , and
Revised: The experimental results show that our proposed method surpasses the baselines on the benchmark datasets including Ofﬁce-31 , Ofﬁce-Home , and

Original: ImageCLEF-DA .
Revised: ImageCLEF-DA .

Original:  RELATED WORK 2.1 STANDARD DA
Revised:  RELATED WORK 2.1 STANDARD DA

Original: Deep domain adaptation has been intensively studied and shown appealing performance in various tasks and applications, notably in Ganin and Lempitsky [2015], Long et al.
Revised: Deep domain adaptation has been intensively studied and shown appealing performance in various tasks and applications, notably in Ganin and Lempitsky [2015], Long et al.

Original: [2015], et al.
Revised: [2015], et al.

Original: The core idea of DDA is to bridge the gap between source and target distributions in a joint space by minimizing a divergence between distributions induced from the source and target domains in this space.
Revised: The core idea of DDA is to bridge the gap between source and target distributions in a joint space by minimizing a divergence between distributions induced from the source and target domains in this space.

Original: Popular choices of divergence include JensenShannon divergence [Ganin and Lempitsky, 2015, Tzeng et al., 2015, Shu et al., 2018]; maximum mean discrepancy distance [Gretton et al., 2007, Long et al., 2015]; and 
Revised: Popular choices of divergence include JensenShannon divergence [Ganin and Lempitsky, 2015, Tzeng et al., 2015, Shu et al., 2018]; maximum mean discrepancy distance [Gretton et al., 2007, Long et al., 2015]; and WS distance

Original: WS distance [Shen et al., 2018, Lee et al., 2019].
Revised: [Shen et al., 2018, Lee et al., 2019, Le et al., 2021a].

Original: Some recent works have exploited different aspects of UDA for improving the performance [Kurmi et al., 2019, Wang et al., 2019a, Chen et al., 2019, Hu et al., 2020].
Revised: Some recent works have exploited different aspects of UDA for improving the performance [Kurmi et al., 2019, Wang et al., 2019a, Chen et al., 2019, Hu et al., 2020].

Original: Typically, CADA [Kurmi et al., 2019] considered the probabilistic certainty estimate of various regions and used these certainty estimate weights for improving the classiﬁer performance on the target dataset.
Revised: Typically, CADA [Kurmi et al., 2019] considered the probabilistic certainty estimate of various regions and used these certainty estimate weights for improving the classiﬁer performance on the target dataset.

Original: GSDA [Hu et al., 2020] introduced a novel method named Hierarchical Gradient Synchronization to model the synchronization relationship among the local distribution pieces and global distribution, aiming for more precise domain-invariant features.
Revised: GSDA [Hu et al., 2020] introduced a novel method named Hierarchical Gradient Synchronization to model the synchronization relationship among the local distribution pieces and global distribution, aiming for more precise domain-invariant features.

Original: 2.2 OPTIMAL TRANSPORT BASED DA
Revised: 2.2 OPTIMAL TRANSPORT BASED DA

Original: Optimal transport theory has been applied to domain adaptation in Courty et al.
Revised: Optimal transport theory has been applied to domain adaptation in Courty et al.

Original: [2017b,a], Damodaran et al.
Revised: [2017b,a], Damodaran et al.

Original: Redko et al.
Revised: Redko et al.

Original: [2019], Lee et al.
Revised: [2019], Lee et al.

Original: [2019], Xie et al.
Revised: [2019], Xie et al.

Original: [2019], Xu et al.
Revised: [2019], Xu et al.

Original: Particularly, Lee et al.
Revised: Particularly, Lee et al.

Original: [2019] proposed using sliced Wasserstein distance for domain adaption, whereas
Revised: [2019] proposed using sliced Wasserstein distance for domain adaption, whereas

Original: Xie et al.
Revised: Xie et al.

Original: [2019] proposed SPOT in which the optimal transport plan is approximated by a pushforward of a reference distribution, and cast the optimal transport problem into a minimax problem.
Revised: [2019] proposed SPOT in which the optimal transport plan is approximated by a pushforward of a reference distribution, and cast the optimal transport problem into a minimax problem.

Original: Recent OT-based DA work (RWOT)
Revised: Recent OT-based DA work (RWOT)

Original: [Xu et al., 2020] leveraged spatial prototypical information and intra-domain structures of image data to reduce the negative transfer caused by target samples near decision boundaries.
Revised: [Xu et al., 2020] leveraged spatial prototypical information and intra-domain structures of image data to reduce the negative transfer caused by target samples near decision boundaries.

Original: Moreover, Courty et al.
Revised: Moreover, Courty et al.

Original: [2017b] proposed an idea to connect the theory of optimal transport and domain adaptation, which later inspired an OT-based deep DA method (DeepJDOT)
Revised: [2017b] proposed an idea to connect the theory of optimal transport and domain adaptation, which later inspired an OT-based deep DA method (DeepJDOT)

Original: [Damodaran et al., 2018].
Revised: [Damodaran et al., 2018].

Original: Another recent work (ETD) [Li et al., 2020] tackled the bottlenecks of OT in UDA by developing an attention-aware OT distance to measure the domain discrepancy under the guidance of the prediction-feedback.
Revised: Another recent work (ETD) [Li et al., 2020] tackled the bottlenecks of OT in UDA by developing an attention-aware OT distance to measure the domain discrepancy under the guidance of the prediction-feedback.

Original: Our proposed approach is totally different from existing OT based DA approaches in which we examine an OT distance discrete distribution over source class-conditional distributions and the target data distribution.
Revised: Our proposed approach is totally different from existing OT based DA approaches in which we examine an OT distance discrete distribution over source class-conditional distributions and the target data distribution.

Original: By investigating this speciﬁc OT distance and minimizing it, we can guide target examples moving to an appropriate source class on the latent space for mitigating both data and label shifts.
Revised: By investigating this speciﬁc OT distance and minimizing it, we can guide target examples moving to an appropriate source class on the latent space for mitigating both data and label shifts.

Original: 2.3 CLASS-AWARE DA
Revised: 2.3 CLASS-AWARE DA

Original: Some recent approaches [Wang et al., 2019b, Kang et al., 2019] leverage the useful information from the label space to improve the quality of the alignment between the source and target domains.
Revised: Some recent approaches [Wang et al., 2019b, Kang et al., 2019] leverage the useful information from the label space to improve the quality of the alignment between the source and target domains.

Original: Wang et al.
Revised: Wang et al.

Original: [2019b] proposed a novel relationship-aware adversarial domain adaptation (RADA) algorithm.
Revised: [2019b] proposed a novel relationship-aware adversarial domain adaptation (RADA) algorithm.

Original: It ﬁrst uses a single multi-class domain discriminator to enforce the learning of inter-class dependency structure during domain-adversarial training.
Revised: It ﬁrst uses a single multi-class domain discrimi- nator to enforce the learning of inter-class dependency structure during domain-adversarial training.

Original: After that, it aligns this structure with the inter-class dependencies that are characterized from training the label predictor on source domain.
Revised: After that, it aligns this structure with the inter-class dependencies that are characterized from training the label predictor on source domain.

Original: Furthermore, the authors imposed a regularization term in order to penalize the structure discrepancy between the interclass dependencies estimated from domain discriminator and label predictor.
Revised: Furthermore, the authors imposed a regularization term in order to penalize the structure discrepancy between the interclass dependencies estimated from domain discriminator and label predictor.

Original: With this alignment, RADA makes the adversarial domain adaptation aware of the class relationships.
Revised: With this alignment, RADA makes the adversarial domain adaptation aware of the class relationships.

Original: Kang et al.
Revised: Kang et al.

Original: [2019] proposed a contrastive adaptation network (CAN) which optimizes a new metric modeling the intra-class domain discrepancy and the inter-class domain discrepancy.
Revised: [2019] proposed a contrastive adaptation network (CAN) which optimizes a new metric modeling the intra-class domain discrepancy and the inter-class domain discrepancy.

Original: In particular, the authors introduced a new contrastive domain discrepancy (CDD) objective to enable class-aware UDA.
Revised: In particular, the authors introduced a new contrastive domain discrepancy (CDD) objective to enable class-aware UDA.

Original: CAN aims to faciliate the optimization with CDD (established on maximum mean discrepancy (MMD)
Revised: CAN aims to faciliate the optimization with CDD (established on maximum mean discrepancy (MMD)

Original: [Long et al., 2015]).
Revised: [Long et al., 2015]).

Original:  BACKGROUND
Revised:  BACKGROUND

Original: In what follows, we present the background of OT for two discrete distributions, which is used in our work.
Revised: In what follows, we present the background of OT for two discrete distributions, which is used in our work.

Original: Consider two discrete distributions: P 1 = (cid:80) Mi =

Revised: Consider two discrete distributions: P 1 = (cid:80) Mi =

Original: π 1 i δ x 1 i and P 2 = (cid:80) Nj =

Revised: π 1 i δ x 1 i and P 2 = (cid:80) Nj =

Original: π 2 j δ x 2 j where π 1 = (cid:2)
Revised: π 2 j δ x 2 j where π 1 = (cid:2)

Original: π 1 i (cid:3) M i =1 and π 2 = (cid:2) π 2 j (cid:3) N j =1 are probability masses, (cid:8) x 1 i (cid:9) M i =1 and (cid:8) x 2 j (cid:9) N j =1 are the sets of atoms, and δ x is the Dirac delta distribution concentrated at the atom x .
Revised: π 1 i (cid:3) M i =1 and π 2 = (cid:2) π 2 j (cid:3) N j =1 are probability masses, (cid:8) x 1 i (cid:9) M i =1 and (cid:8) x 2 j (cid:9) N j =1 are the sets of atoms, and δ x is the Dirac delta distribution concentrated at the atom x .

Original: Let c (cid:0) x
Revised: Let c (cid:0) x

Original: i , x 2 j (cid:1) be a cost function.
Revised: i , x 2 j (cid:1) be a cost function.

Original: The OT distance between P 1 and P 2 w.r.t.
Revised: The OT distance between P 1 and P 2 w.r.t.

Original: the cost function c is deﬁned as
Revised: the cost function c is deﬁned as

Original: [Equation]
Revised: [Equation]

Revised: ij c (cid:0) x 1 i , x 2 j (cid:1) , (1) where

Original: where A =
Revised: A =

Original: [ a ij ] ∈ R M × N + of non-negative elements satisfying (cid:80) Nj =1 a ij =
Revised: [ a ij ] ∈ R M × N + of non-negative elements satisfying (cid:80) Nj =1 a ij =

Original: π 1 i , ∀ i ∈ { 1 , ..., M } and (cid:80) Mi
Revised: π 1 i , ∀ i ∈ { 1 , ..., M } and (cid:80) Mi

Original: =1 a ij = π 2 j , ∀ j ∈ { 1 , ..., N } .
Revised: =1 a ij = π 2 j , ∀ j ∈ { 1 , ..., N } .

Original: In addition, a ij ∈ [0; 1] is interpreted as the probability to match x 1 i and x 2 j or to transport x 1 i to x 2 j , which suffers the cost c (cid:0) x 1 i , x 2 j (cid:1) .
Revised: In addition, a ij ∈ [0; 1] is interpreted as the probability to match x 1 i and x 2 j or to transport x 1 i to x 2 j , which suffers the cost c (cid:0) x 1 i , x 2 j (cid:1) .

Original: Therefore, the sum (cid:80) Mi =1 (cid:80) Nj =1 a ij c (cid:0) x 1 i , x 2 j (cid:1) can be viewed as the total cost to match P 1 and P 2 or to transport P 1 to P 2 .
Revised: Therefore, the sum (cid:80) Mi =1 (cid:80) Nj =1 a ij c (cid:0) x 1 i , x 2 j (cid:1) can be viewed as the total cost to match P 1 and P 2 or to transport P 1 to P 2 .

Original: By solving the optimization problem in Eq.
Revised: By solving the optimization problem in Eq.

Original: (1), we aim to ﬁnd the optimal transportation matrix A ∗ which minimizes the total cost.
Revised: (1), we aim to ﬁnd the optimal transportation matrix A ∗ which minimizes the total cost.

Original:  DISTRIBUTIONAL OPTIMAL TRANSPORT APPROACH FOR CLASS-AWARE UDA 4.1 PROBLEM FORMULATION
Revised:  DISTRIBUTIONAL OPTIMAL TRANSPORT APPROACH FOR CLASS-AWARE UDA 4.1 PROBLEM FORMULATION

Original: We consider the standard setting of unsupervised domain adaptation in which we have a labeled dataset D S
Revised: We consider the standard setting of unsupervised domain adaptation in which we have a labeled dataset D S

Original: = (cid:8)(cid:0) x
Revised: = (cid:8)(cid:0) x

Original: S i , y S i (cid:1)(cid:9) N S i =1 from a source domain and an unlabeled dataset D T = (cid:8) x Ti (cid:9) N T i =1 from a target domain.
Revised: Si , y Si (cid:1)(cid:9) N S i =1 from a source domain and an unlabeled dataset D T = (cid:8) x Ti (cid:9) N T i =1 from a target domain.

Original: We assume that data examples x Si , x Ti ∈ R d and the categorical labels y Si ∈ { 1 , 2 , ..., M } where M is the number of classes.
Revised: We assume that data examples x Si , x Ti ∈ R d and the categorical labels y Si ∈ { 1 , 2 , ..., M } where M is the number of classes.

Original: For the sake of notion simpliﬁcation, we overload D S and D T to represent the empirical joint distributions of the source and target domains.
Revised: For the sake of notion simpliﬁcation, we overload D S and D T to represent the empirical joint distributions of the source and target domains.

Original: We denote P S and P T as the data distributions of the source and target domains respectively.
Revised: We denote P S and P T as the data distributions of the source and target domains respectively.

Original: Moreover, given a class m , we further denote P Sm as the mth class-conditional distribution of the source domain (i.e., the distribution with the density function p S ( x | y = m ) ).
Revised: Moreover, given a class m , we further denote P Sm as the mth class-conditional distribution of the source domain (i.e., the distribution with the density function p S ( x | y = m ) ).

Original: 4.2 MOTIVATION
Revised: 4.2 MOTIVATION

Original: For our proposed approach, we consider an OT distance of two discrete distributions.
Revised: For our proposed approach, we consider an OT distance of two discrete distributions.

Original: The ﬁrst one is the discrete distribution whose atoms are the target examples x T (i.e., x 1 i = x
Revised: The ﬁrst one is the discrete distribution whose atoms are the target examples x T (i.e., x 1 i = x

Original: Ti in Eq.
Revised: Ti in Eq.

Original: (1)), while the second one is the discrete distribution whose atoms are the source classconditional distributions
Revised: (1)), while the second one is the discrete distribution whose atoms are the source classconditional distributions

Original: P Sm (i.e., x 2 j = P Sm in Eq.
Revised: P Sm (i.e., x 2 j = P Sm in Eq.

Original: The cost c (cid:0) x Ti , P Sm (cid:1) is deﬁned as the negative log likelihood − log p
Revised: The cost c (cid:0) x Ti , P Sm (cid:1) is deﬁned as the negative log likelihood − log p

Original: Sm (cid:0) x Ti (cid:1) =
Revised: Sm (cid:0) x Ti (cid:1) =

Original: − log p S (cid:0) x
Revised: − log p S (cid:0) x

Original: Ti | y
Revised: Ti | y

Original: = m (cid:1) .
Revised: = m (cid:1) .

Original: Hence, if a target sample x Ti is more likely to be a sample from P Sm , the log likelihood log p
Revised: Hence, if a target sample x Ti is more likely to be a sample from P Sm , the log likelihood log p

Original: Sm (cid:0) x Ti (cid:1) is higher, meaning that the cost c (cid:0) x Ti , P Sm (cid:1)
Revised: Sm (cid:0) x Ti (cid:1) is higher, meaning that the cost c (cid:0) x Ti , P Sm (cid:1)

Original: = − log p
Revised: = − log p

Original: Sm (cid:0) x Ti (cid:1) becomes smaller.
Revised: Sm (cid:0) x Ti (cid:1) becomes smaller.

Original: As shown in Figure 1, by examining the OT distance between two aforementioned distributions, we aim to ﬁnd the best match between a given target sample x Ti and a source classconditional distribution P Sm .c ( x Ti , P Sm )
Revised: As shown in Figure 1, by examining the OT distance between two aforementioned distributions, we aim to ﬁnd the best match between a given target sample x Ti and a source classconditional distribution P Sm .

Original: = − log p
Revised: = − log p

Original: Sm (cid:0) x Ti (cid:1) =
Revised: Sm (cid:0) x Ti (cid:1) =

Original: − log p S (cid:0) x
Revised: − log p S (cid:0) x

Original: Ti | y
Revised: Ti | y

Original: = m (cid:1) .
Revised: = m (cid:1) .

Original: 4.3 DISTRIBUTIONAL OPTIMAL TRANSPORT
Revised: 4.3 DISTRIBUTIONAL OPTIMAL TRANSPORT

Original: We deﬁne P S
Revised: We deﬁne P S

Original: = (cid:80)
Revised: = (cid:80)

Original: Mm =

Revised: Mm =

Original: π m δ P
Revised: π m δ P

Original: Sm , where δ is the Dirac delta distribution and the mixing proportion π ∈ ∆ M := (cid:8) α ∈ R M : α ≥ 0 and (cid:107) α (cid:107) 1 = 1 (cid:9) with the number of classes M .
Revised: Sm , where δ is the Dirac delta distribution and the mixing proportion π ∈ ∆ M := (cid:8) α ∈ R M : α ≥ 0 and (cid:107) α (cid:107) 1 = 1 (cid:9) with the number of classes M .

Original: Obviously, P S is a discrete distribution of distributions wherein P S takes P Sm with the probability π m .
Revised: Obviously, P S is a discrete distribution of distributions wherein P S takes P Sm with the probability π m .

Original: As mentioned in the motivation section, we now examine an OT distance between P T and P S , we aim at matching target examples to the source class-conditional distributions in which a target example is absolutely guided to match the source class-conditional distribution corresponding to its ground-truth label.
Revised: As mentioned in the motivation section, we now examine an OT distance between P T and P S , we aim at matching target examples to the source class-conditional distributions in which a target example is absolutely guided to match the source class-conditional distribution corresponding to its ground-truth label.

Original: In the sequel, we inspect an OT distance between P T and
Revised: In the sequel, we inspect an OT distance between P T and

Original: P S in which we deﬁne the cost c (cid:0) x i , P Sm (cid:1) to match a target sample x i to P Sm as − log p
Revised: P S in which we deﬁne the cost c (cid:0) x i , P Sm (cid:1) to match a target sample x i to P Sm as − log p

Original: Sm ( x i ) .
Revised: Sm ( x i ) .

Original: Let us denote
Revised: Let us denote

Original: A =
Revised: A =

Original: [ a im ] ∈ R N T × M as the transportation matrix wherein a im represents the probability to match or transport x i to P
Revised: [ a im ] ∈ R N T × M as the transportation matrix wherein a im represents the probability to match or transport x i to P

Original: Sm .
Revised: Sm .

Original: The OT distance between P T and P S w.r.t.
Revised: The OT distance between P T and P S w.r.t.

Original: the cost function c and the mixing proportion π is deﬁned as:
Revised: the cost function c and the mixing proportion π is deﬁned as:

Original: [Equation]
Revised: [Equation]

Original: Similar to other DA works [Pan et al., 2008, Tzeng et al., 2015, Long et al., 2017], we employ a feature extractor G to map both source and target examples to a latent space.
Revised: Similar to other DA works [Pan et al., 2008, Tzeng et al., 2015, Long et al., 2017], we employ a feature extractor G to map both source and target examples to a latent space.

Original: We denote Q S , Q T , Q Sm , and Q S as the corresponding distributions over the latent space induced by P S , P T , P Sm , and
Revised: We denote Q S , Q T , Q Sm , and Q S as the corresponding distributions over the latent space induced by P S , P T , P Sm , and

Original: P S via the feature extractor G .
Revised: P S via the feature extractor G .

Original: The OT distance in Eq.
Revised: The OT distance in Eq.

Original: (2) is rewritten as:
Revised: (2) is rewritten as:

Original: [Equation]
Revised: [Equation]

Original: a [Equation] [Equation] To encourage the target examples G ( x i ) to move towards proper class regions of the source domain, we propose solving the following optimization problem (OP):
Revised: To encourage the target examples G ( x i ) to move towards proper class regions of the source domain, we propose solving the following optimization problem (OP):

Original: [Equation]
Revised: [Equation]

Original: With c (cid:0) G ( x i ) , Q Sm (cid:1) =
Revised: With c (cid:0) G ( x i ) , Q S m (cid:1) =

Original: − log p Sm ( x i ) , minimizing the OT distance in Eq.
Revised: − log p S m ( x i ) , minimizing the OT distance in Eq.

Original: (4) encourages the target example
Revised: (4) encourages the target example

Original: G ( x i ) to move towards a Q Sk (1 ≤ k ≤ M ) with a high likelihood and a i =
Revised: G ( x i ) to move towards a Q Sk (1 ≤ k ≤ M ) with a high likelihood and a i =

Original: [ a im ] m inspired to be close to the corresponding scaled one-hot vector 

Revised: [ a im ] m inspired to be close to the corresponding scaled one-hot vector

Original: N T 1 k .
Revised: N T 1 k .

Original: Here we denote 1 k as the one-hot vector with the k -th element being one.
Revised: Here we denote 1 k as the one-hot vector with the k -th element being one.

Original: 5.1 COST FUNCTION AND KNOWLEDGE DISTILLATION
Revised: 5.1 COST FUNCTION AND KNOWLEDGE DISTILLATION

Original: To deﬁne the cost function c (cid:0) G ( x i ) , Q Sm (cid:1) in Eq.
Revised: To deﬁne the cost function c (cid:0) G ( x i ) , Q Sm (cid:1) in Eq.

Original: (4), we build a classiﬁer h S over the latent space, and rely on its output to compute the cost values.
Revised: (4), we build a classiﬁer h S over the latent space, and rely on its output to compute the cost values.

Original: This classiﬁer is ﬁrst trained using the labeled source dataset D S
Revised: This classiﬁer is ﬁrst trained using the labeled source dataset D S

Original: = (cid:8)(cid:0) x
Revised: = (cid:8)(cid:0) x

Original: Si , y Si (cid:1)(cid:9) N S i =1 by minimizing the empirical loss:
Revised: Si , y Si (cid:1)(cid:9) N S i =1 by minimizing the empirical loss:

Original: [Equation]
Revised: [Equation]

Original: where σ denotes a softmax function and CE represents a cross-entropy loss.
Revised: where σ denotes a softmax function and CE represents a cross-entropy loss.

Original: Recap that given a target example x i , c (cid:0) G ( x i ) , Q Sm (cid:1) captures the matching extent of G ( x i ) and the class-conditional distribution Q Sm .
Revised: Recap that given a target example x i , [Equation] x i ) , Q Sm (cid:1) captures the matching extent of G ( x i ) and the class-conditional distribution Q Sm .

Original: Therefore, we can reasonably deﬁne c (cid:0) G ( x i ) , Q Sm 
Revised: Therefore, we can reasonably deﬁne c (cid:0) G ( x i ) , Q Sm (cid:1) =

Original: [Equation]
Revised: − log σ m (cid:0) h S ( G ( x i )) (cid:1) (i.e., σ m (cid:0) h S ( G ( x i ))

Original: (i.e., σ m (cid:0) h S ( G ( x i )) (cid:1) is the predicted probability of x i belonging to class m by classiﬁer h S ).
Revised: (cid:1) is the predicted probability of x i belonging to class m by classiﬁer h S ).

Original: However, we ﬁnd that h S is a well-trained classiﬁer on the source domain, and can generalize poorly on the target domain due to the data and label shifts.
Revised: However, we ﬁnd that h S is a well-trained classiﬁer on the source domain, and can generalize poorly on the target domain due to the data and label shifts.

Original: Therefore, instead of using only one classiﬁer trained to work well on both domains, we leverage knowledge distillation [Hinton et al., 2015, Tian et al., 2020, Tejankar et al., 2021] which includes the two-network architecture, a teacher h T and a student h S .
Revised: Therefore, instead of using only one classiﬁer trained to work well on both domains, we leverage knowledge distillation [Hinton et al., 2015, Tian et al., 2020, Tejankar et al., 2021] which includes the two-network architecture, a teacher h T and a student h S .

Original: The teacher h T aims to be an expert on the target domain, while the student h S , which classiﬁes accurately on the source domain, is also able to generalize on the target domain via distilling knowledge from its teacher.
Revised: The teacher h T aims to be an expert on the target domain, while the student h S , which classiﬁes accurately on the source domain, is also able to generalize on the target domain via distilling knowledge from its teacher.

Original: When the generalization ability of h S is improved, the cost c (cid:0) G ( x i ) , Q Sm (cid:1) is computed more accurately to solve the
Revised: When the generalization ability of h S is improved, the cost c (cid:0) G ( x i ) , Q Sm (cid:1) is computed more accurately to solve the

Original: OP in Eq.
Revised: OP in Eq.

Original: Inspired by the work of Hinton et al.
Revised: Inspired by the work of Hinton et al.

Original: [2015], we perform knowledge distillation from the teacher h T to the student h S in the target domain by minimizing a distillation loss L dl w.r.t.
Revised: [2015], we perform knowledge distillation from the teacher h T to the student h S in the target domain by minimizing a distillation loss L dl w.r.t.

Original: a temperature softmax function:
Revised: a temperature softmax function:

Original: [Equation]
Revised: [Equation]

Original:  where τ is a temperature parameter.
Revised: , (6) where τ is a temperature parameter.

Original: When setting τ > 1 , the teacher and student’s predictions become softer, from which the student can capture “dark knowledge”
Revised: When setting τ > 1 , the teacher and student’s predictions become softer, from which the student can capture “dark knowledge”

Original: [Hinton et al., 2015] from the teacher and effectively mimic the teacher’s behaviour.
Revised: [Hinton et al., 2015] from the teacher and effectively mimic the teacher’s behaviour.

Original: The student h S is now trained well in the source domain via
Revised: The student h S is now trained well in the source domain via

Original: Eq.
Revised: Eq.

Original: (5), and is possible to generalize on the target domain via Eq.
Revised: (5), and is possible to generalize on the target domain via Eq.

Original: To achieve this good generalization capability, we need to produce a teacher h T that is with good classiﬁcation performance on the target domain.
Revised: To achieve this good generalization capability, we need to produce a teacher h T that is with good classiﬁcation performance on the target domain.

Original: To this end, we propose minimizing a cross-entropy loss between the teacher’s prediction and pseudo labels computed via the optimal transportation matrix A ∗ after solving Eq.
Revised: To this end, we propose minimizing a cross-entropy loss between the teacher’s prediction and pseudo labels computed via the optimal transportation matrix A ∗ after solving Eq.

Original: [Equation]
Revised: [Equation]

Original: where ˆ y T are pseudo labels for unlabeled target samples.
Revised: where ˆ y T are pseudo labels for unlabeled target samples.

Original: It is worth noting that only a subset of target samples with high-conﬁdence pseudo labels is selected (i.e., n T < N T ).
Revised: It is worth noting that only a subset of target samples with high-conﬁdence pseudo labels is selected (i.e., n T < N T ).

Original: In the next section, we discuss on how to compute these pseudo labels and our framework.
Revised: In the next section, we discuss on how to compute these pseudo labels and our framework.

Original: 5.2 PSEUDO-LABEL SELECTION AND OUR FRAMEWORK
Revised: 5.2 PSEUDO-LABEL SELECTION AND OUR FRAMEWORK

Original: We now introduce the strategy to produce pseudo labels for unlabeled target samples.
Revised: We now introduce the strategy to produce pseudo labels for unlabeled target samples.

Original: Let us return to the Eq.
Revised: Let us return to the Eq.

Original: (3) where directly solving this OP is computationally expensive.
Revised: (3) where directly solving this OP is computationally expensive.

Original: Hence, we instead use an entropic regularized version to minimize:
Revised: Hence, we instead use an entropic regularized version to minimize:

Original: [Equation]
Revised: [Equation]

Original: ) where H ( A ) :
Revised: where H ( A ) :

Original: = − (cid:80) N T i =1 (cid:80)
Revised: = − (cid:80) N T i =1 (cid:80)

Original: Mm =1 a
Revised: Mm =1 a

Original: im log a im denotes an entropy of the transportation matrix A , and (cid:15) is the regularization rate.
Revised: im log a im denotes an entropy of the transportation matrix A , and (cid:15) is the regularization rate.

Original: During the training, we use Sinkhorn algorithm
Revised: During the training, we use Sinkhorn algorithm

Original: [Cuturi, 2013] to solve this OP and achieve A ∗ at every mini-batch.
Revised: [Cuturi, 2013] to solve this OP and achieve A ∗ at every mini-batch.

Original: Interestingly, the solution of Eq.
Revised: Interestingly, the solution of Eq.

Original: (8) also provides us (cid:80)
Revised: (8) also provides us (cid:80)

Original: Mm =1 a ∗ im = 1 N T or in other words,
Revised: Mm =1 a ∗ im = 1 N T or in other words,

Original: N T (cid:80) M m =1 a ∗ im = 1 .
Revised: N T (cid:80) M m =1 a ∗ im = 1 .

Original: Hence, we can deﬁne the pseudo label ˆ y Ti :=
Revised: Hence, we can deﬁne the pseudo label ˆ y Ti :=

Original: N T a ∗ i for a given target sample x i and it satisﬁes (cid:80)
Revised: N T a ∗ i for a given target sample x i and it satisﬁes (cid:80)

Original: Mm =1 ˆ y Tim = N T (cid:80) Mm =1 a ∗ im =
Revised: Mm =1 ˆ y Tim = N T (cid:80) Mm =1 a ∗ im =

Original: The deﬁnition of ˆ y Ti is then used for minimizing L pl in Eq.
Revised: The deﬁnition of ˆ y Ti is then used for minimizing L pl in Eq.

Original: One problem when choosing ˆ y Ti := N T a ∗ i is that the performance of the teacher h T can be reduced if some pseudo labels are incorrect, especially at the beginning of the training due to the data and label shifts between the source and target domains.
Revised: One problem when choosing ˆ y Ti := N T a ∗ i is that the performance of the teacher h T can be reduced if some pseudo labels are incorrect, especially at the beginning of the training due to the data and label shifts between the source and target domains.

Original: This issue also inﬂuences the distillation process since we aim to build a well-classiﬁed teacher h T on the target domain to transfer some of its aspects (e.g, its “dark knowledge”) to the student h S .
Revised: This issue also inﬂuences the distillation process since we aim to build a well-classiﬁed teacher h T on the target domain to transfer some of its aspects (e.g, its “dark knowledge”) to the student h S .

Original: To avoid this problem, inspired by Yang et al.
Revised: To avoid this problem, inspired by Yang et al.

Original: [2021], we propose only selecting highly conﬁdent pseudo labels (i.e., pseudo labels whose entropies are less than a threshold) using an entropy-based selection method.
Revised: [2021], we propose only selecting highly conﬁdent pseudo labels (i.e., pseudo labels whose entropies are less than a threshold) using an entropy-based selection method.

Original: The OP in Eq.
Revised: The OP in Eq.

Original: (7) is now minimized w.r.t.
Revised: (7) is now minimized w.r.t.

Original: the weights w i :
Revised: the weights w i :

Original: [Equation]
Revised: [Equation]

Original: ) where w i = I { H ( ˆ y Ti ) <
Revised: where w i = I { H ( ˆ y Ti ) <

Original: H ρ } with I C representing the indicator function for a statement C (i.e., I C returns 1 iff C is true), H (cid:0)
Revised: H ρ } with I C representing the indicator function for a statement C (i.e., I C returns 1 iff C is true), H (cid:0)

Original: ˆ y T
Revised: ˆ y T

Original: i (cid:1) :
Revised: i (cid:1) :

Original: = − (cid:80) M m =1 a
Revised: = − (cid:80) M m =1 a

Original: im log a im is the entropy of a pseudo label ˆ y Ti w.r.t.
Revised: im log a im is the entropy of a pseudo label ˆ y Ti w.r.t.

Original: a target example x i , and the threshold
Revised: a target example x i , and the threshold

Original: H ρ denotes the ρ -th percentile of H (cid:0)
Revised: H ρ denotes the ρ -th percentile of H (cid:0)

Original: ˆ y Ti (cid:1) .
Revised: ˆ y Ti (cid:1) .

Original: Additionally, when training our COOK, at each iteration, we sample a mini-batch of target examples and consider Q T as the distribution of latent representations corresponding to this mini-batch.
Revised: Additionally, when training our COOK, at each iteration, we sample a mini-batch of target examples and consider Q T as the distribution of latent representations corresponding to this mini-batch.

Original: Therefore, N T in Eq.
Revised: Therefore, N T in Eq.

Original: (8) is replaced by the batch size and the threshold H ρ denotes the ρ -th percentile of H (cid:0)
Revised: (8) is replaced by the batch size and the threshold H ρ denotes the ρ -th percentile of H (cid:0)

Original: ˆ y Ti (cid:1) in the mini-batch.
Revised: ˆ y Ti (cid:1) in the mini-batch.

Original: Finally, we present our framework in Figure 2 which includes three main steps: (i) the teacher is encouraged to be an expert on the target domain using the pseudo labeling technique; (ii) the teacher transfers its knowledge to the student via a distillation process to support the student to generalize well on the target domain; and (iii) the predicted probabilities of the student classiﬁer are utilized for minimizing W (cid:15)c, π (cid:0) Q T , Q S (cid:1) using Sinkhorn algorithm, and offering the optimal transportation matrix A ∗ to compute pseudo labels.
Revised: Finally, we present our framework in Figure 2 which includes three main steps: (i) the teacher is encouraged to be an expert on the target domain using the pseudo labeling technique; (ii) the teacher transfers its knowledge to the student via a distillation process to support the student to generalize well on the target domain; and (iii) the predicted probabilities of the student classiﬁer are utilized for minimizing W (cid:15)c, π (cid:0) Q T , Q S (cid:1) using Sinkhorn algorithm, and offering the optimal transportation matrix A ∗ to compute pseudo labels.

Original: The pseudo labels with low entropies are selected to train the teacher at the ﬁrst step.
Revised: The pseudo labels with low entropies are selected to train the teacher at the ﬁrst step.

Original: This process forms a closed cycle in which target samples are conﬁdently moved towards corresponding source class-conditional distributions Q Sm under the consistently cyclic guidance of the key factors including the distributional optimal transport and knowledge distillation, which motivates us to propose our COOK.
Revised: This process forms a closed cycle in which target samples are conﬁdently moved towards corresponding source class-conditional distributions Q Sm under the consistently cyclic guidance of the key factors including the distributional optimal transport and knowledge distillation, which motivates us to propose our COOK.

Original: 5.3 TRAINING PROCEDURE OF COOK
Revised: 5.3 TRAINING PROCEDURE OF COOK

Original: To strengthen h S for providing better predictions and accelerating matching target samples x T to source classconditional distributions Q S m , we enforce the clustering assumption to h S .
Revised: To strengthen h S for providing better predictions and accelerating matching target samples x T to source classconditional distributions Q Sm , we enforce the clustering assumption to h S .

Original: Inspired by applying clustering assumption in domain adaptation works [Shu et al., 2018, Kumar et al., 2018], we employ Virtual Adversarial Training (VAT)
Revised: Inspired by applying clustering assumption in domain adaptation works [Shu et al., 2018, Kumar et al., 2018], we employ Virtual Adversarial Training (VAT)

Original: [Miyato et al., 2019] in conjunction with minimizing entropy [Grandvalet and Bengio, 2005] of the prediction of 
Revised: [Miyato et al., 2019] in conjunction with minimizing entropy [Grandvalet and Bengio, 2005] of the prediction of h S (cid:0) G (cid:0) x T (cid:1)(cid:1) . VAT an local robustness [Nguyen-Duc et al., 2022, Phan et al., 2022]. At

Revised: , given a x a perturbation of that makes

Revised: S

Original: [Equation]
Revised: [Equation]

Original: where with H to be the entropy we have deﬁned:
Revised: where with H to be the entropy we have deﬁned:

Original: [Equation]
Revised: [Equation]

Original:  where D KL denotes the Kullback-Leibler divergence and θ is a hyperparameter set to a very small positive number.
Revised: σ (cid:0) h S ( G ( x (cid:48) )) (cid:1) (cid:19)(cid:21) , where D KL denotes the KullbackLeibler divergence and θ is a hyperparameter set to a very small positive number.

Original: The ﬁnal optimization problem of our COOK for ﬁnding h S , h T and G is as follows:
Revised: The ﬁnal optimization problem of our COOK for ﬁnding h S , h T and G is as follows:

Original: [Equation]
Revised: [Equation]

Original: where α, β, γ > 0 are trade-off parameters.
Revised: where α, β, γ > 0 are trade-off parameters.

Original: Conveniently, the cyclic process in Figure 2 is operated synchronously by simultaneously updating h S , h T and G during the training.
Revised: Conveniently, the cyclic process in Figure 2 is operated synchronously by simultaneously updating h S , h T and G during the training.

Original: Finally, we present the key steps of our COOK in Algorithm 1, while the overall architecture and motivation of component losses are depicted in Figure 3.
Revised: Finally, we present the key steps of our COOK in Algorithm 1 and the overall architecture and motivation of component losses are depicted in Figure 3.

Original: EXPERIMENTS

Original: In this section, we conduct experiments on benchmark datasets including Ofﬁce-31 , Ofﬁce-Home , and ImageCLEFDA to compare with existing baselines, especially OT-based and class-aware UDA methods.

Original: The information of experimental datasets is presented in the supplementary material due to the limit of space.

Original: Input: A source batch B S =
Revised: Input: A source batch B S x

Original: (cid:8)(cid:0) x Si , y Si (cid:1)(cid:9) b
Revised: Si , y i

Original: i =1 , a target batch
Revised: , a target batch

Original: BT = (cid:8) x Tj(cid:9) b j =1 ( b denotes the batch size).
Revised: B x Tj the batch size).

Original: Output: Classiﬁers h S ∗ , h T ∗ , generator G ∗ .
Revised: Output: Classiﬁers h S ∗ , h T ∗ , generator G ∗ .

Original: 1: for number of training iterations do 2:
Revised: 1: for number of training iterations do 2:

Original: Solve the OP in Eq.
Revised: Solve the OP in Eq.

Original: (8) using Sinkhorn algorithm to ﬁnd A ∗ .
Revised: (8) using Sinkhorn algorithm to ﬁnd A ∗ .

Original: Computeˆ y iT
Revised: Compute y iT

Original: Eq.
Revised: Eq

Original: (9) based  A ∗ .
Revised: (9) based on A ∗ .

Original: Compute w i  Eq.
Revised: Compute w i in Eq.

Original: (9) based  H ρ .
Revised: (9) based on H ρ .

Original: Update h S , h T G according Eq.
Revised: Update h S , h and G to Eq.

Original: : end for
Revised: 6: end for

Revised:  EXPERIMENTS

Revised: In this section, we conduct experiments on benchmark datasets including Ofﬁce-31 , Ofﬁce-Home , and ImageCLEFDA to compare with existing baselines, especially OT-based and class-aware UDA methods.

Original: 6.1 NETWORK ARCHITECTURE
Revised: 6.1 DATASETS

Original: In the experiments on the Ofﬁce-31 , Ofﬁce-Home  and 
Revised: Ofﬁce-31 [Saenko et al., 2010] is a well-known public dataset used for UDA. It consists of three domains including Amazon ( A ), Webcam ( W ) and Dslr ( D ) with 31 common classes and 4,110 images in total. Ofﬁce-Home [Venkateswara et al., 2017] is another and more challenging dataset for UDA which contains images from four different domains, namely Artistic ( Ar ), Clip

Revised: Art ( Cl ), Product ( Pr ) and Real-world images ( Re ).

Revised: This dataset consists of around 15,588 images in total withobject categories in ofﬁce and home scenes.

Revised: ImageCLEF-DA [Caputo et al., 2014] includes three domains including Caltech-256 ( C ), ImageNet ILSVRC( I ), and Pascal VOC 2012 ( P ), each of which has 12 classes with 50 images per class.

Revised: 6.2 IMPLEMENTATION DETAILS

Revised: In the experiments on the Ofﬁce-31 , Ofﬁce-Home and

Original: ImageCLEF-DA datasets, we use the extracted features ( 2048 dimensions) from ResNet-50 [He et al., 2016].
Revised: ImageCLEF-DA datasets, we use the extracted features ( 2048 dimensions) from ResNet-50 [He et al., 2016].

Original: The generator includes a fully connected layer that outputsdimensions.
Revised: The generator includes a fully connected layer that outputsdimensions.

Original: We use the same architecture for the student and teacher networks which consists of a fully connected layer for each network.
Revised: We use the same architecture for the student and teacher networks which consists of a fully connected layer for each network.

Original: Details of all network architecture are described in the supplementary material .

Original: 6.2 MODEL HYPERPARAMETERS

Original: Some hyperparameters substantially contributes to the performance of our COOK, namely the temperature τ in Eq.
Revised: Some hyperparameters substantially contributes to model performance, namely the temperature τ in Eq.

Original: (6), and the percentile ρ in Eq.
Revised: (6), and the percentile ρ in Eq.

Original: As suggested in the ablation study, we choose τ = 10 . 0 to effectively activate the knowledge distillation process from the teacher to the student.
Revised: As suggested in the ablation study, we choose τ = 10 . 0 to effectively activate the knowledge distillation process from the teacher to the student.

Original: The percentile ρ is important to measure how well the student h S can generalize on the target domain.
Revised: The percentile ρ is important to measure how well the student h S can generalize on the target domain.

Original: We empirically ﬁnd that ρ = 20 or in other words, choosing the 20 -th percentile of H (cid:0)
Revised: We empirically ﬁnd that ρ = 20 or in other words, choosing the 20 -th percentile of H (cid:0)

Original: ˆ y Ti (cid:1) is appropriate to select highconﬁdence pseudo labels.
Revised: ˆ y Ti (cid:1) is appropriate to select high-conﬁdence pseudo labels.

Original: Additionally, the regularization rate (cid:15) is set to 0 .
Revised: Additionally, setting (cid:15) less than or equal to 0 .

Revised: 1 can achieve better performance and we set (cid:15) to 0 .

Original: We also select the trade-off parameters α = β = 1 . 0 and γ = 0 .
Revised: We also select the trade-off parameters α = β = 1 . 0 and γ = 0 .

Original: 1 in our experiments as suggested in the ablation studies.
Revised: 1 in our experiments as suggested in the ablation studies.

Original: We apply Adam optimizer [Kingma and Ba, 2015] ( β 1 = 0 .
Revised: We apply Adam optimizer [Kingma and Ba, 2015] ( β 1 = 0 .

Original: 5 , β 2 = 0 .
Revised: 5 , β 2 = 0 .

Original: with Polyak averaging [Polyak and Juditsky, 1992], and the learning rate is set to 10 − 4 for Ofﬁceand Ofﬁce-Home , and 5 × 10 − 5 for ImageCLEF-DA .
Revised: with Polyak averaging [Polyak and Juditsky, 1992], and the learning rate is set to 10 − 4 for Ofﬁceand Ofﬁce-Home , and 5 × 10 − 5 for ImageCLEF-DA .

Original: For the baselines, we report the experimental results mentioned in the original papers.
Revised: For the baselines, we report the experimental results mentioned in the original papers.

Original: It is noticeable that in all experiments, we only train the feature extractor, and the performance of COOK can be further improved when ﬁne-tuning the backbone ResNet-50 is conducted.
Revised: It is noticeable that in all experiments, we only train the feature extractor, and the performance of COOK can be further improved when ﬁne-tuning the backbone ResNet-50 is conducted.

Original: 6.3 RESULT AND DISCUSSION
Revised: 6.3 RESULT AND DISCUSSION

Original: We compare our COOK with the standard baseline ResNet

Revised: We compare our COOK with the standard baseline ResNet

Original: [He et al., 2016] and existing works including DAN [Long et al., 2015], DANN [Ganin and Lempitsky, 2015],
Revised: [He et al., 2016] and existing works including DAN [Long et al., 2015], DANN [Ganin and Lempitsky, 2015],

Original: RTN [Long et al., 2016], iCAN [Zhang et al., 2018], CDANE [Long et al., 2018], CDAN-BSP [Chen et al., 2019],
Revised: RTN [Long et al., 2016], iCAN [Zhang et al., 2018], CDANE [Long et al., 2018], CDAN-BSP [Chen et al., 2019],

Original: CDAN-TransNorm
Revised: CDAN-T [Wang et al., 2019a], TPN [Pan et al., 2019], rRevGrad+CAT [Deng et al., 2019], CADA-P [Kurmi et al., 2019], SymNets [Zhang et al., 2019], especially class-aware

Original:  [Wang et al., 2019a], TPN [Pan et al., 2019], rRevGrad+CAT [Deng et al., 2019], CADA-P
Revised: DA and OT-based methods, namely RADA [Wang et al., [Table]

Original: [Kurmi et al., 2019], SymNets [Zhang et al., 2019], especially class-aware DA and OT-based methods, namely RADA [Wang et al., 2019b], CAN [Kang et al., 2019], DeepJDOT [Damodaran et al., 2018], ETD [Li et al., 2020], and RWOT [Xu et al., 2020].
Revised: 2019b] , CAN [Kang et al., 2019], DeepJDOT [Damodaran et al., 2018], ETD [Li et al., 2020] [Table] and RWOT [Xu et al., 2020].

Original: The results trained on Ofﬁce-31 are reported in Table 1.
Revised: The results trained on Ofﬁce-31 are reported in Table 1.

Original: In general, our proposed method achieves high results with four transfer tasks greater than 95% .
Revised: In general, our proposed method achieves high results with four transfer tasks greater than 95% .

Original: Except for the transfer task D → W  , our model signiﬁcantly outperforms others on almost adaptation tasks, and obtain 93 .
Revised: Except for the transfer tasks D → W and W → D , our model signiﬁcantly outperforms others on almost adaptation tasks [Table] and obtain 94 .

Original: 9% on average, which is a 3 . 1% increase compared to the runner-up result.
Revised: 1% on average, which is a 3 . 5% increase compared to the runner-up result.

Original: It is worth noting that our COOK outperforms the baselines by a large margin on challenging tasks, e.g., a 11 . 2% increase on D → A and W → A with a 7 . 1% improvement, in which the background of the training images between the two domains are totally dissimilar.
Revised: It is worth noting that our COOK outperforms the baselines by a large margin on challenging tasks, e.g., a 10 . 7% increase on D → A and W → A with a 9 . 2% improvement, in which the background of the training images between the two domains are totally dissimilar.

Original: We present the results trained on Ofﬁce-Home in Table 2.
Revised: We present the results trained on Ofﬁce-Home in Table 2.

Original: In this dataset, our COOK surpasses 7 over 12 transfer tasks compared with the baselines and achieves the best performance, making a 2 . 8% improvement on average.
Revised: In this dataset, our COOK surpasses 7 over 12 transfer tasks compared with the baselines and achieves the best performance, making a 2 . 8% improvement on average.

Original: More speciﬁcally, our model sees a remarkable improvement on more challenging adaptation tasks, namely Ar → Pr ( 3 . 6%) ,
Revised: More speciﬁcally, our model sees a remarkable improvement on more challenging adaptation tasks, namely Ar → Pr ( 3 . 6%) ,

Original: Cl → Pr ( 7 . 6%) , Cl → Re ( 4 . 1% ).
Revised: Cl → Pr ( 7 . 6%) , Cl → Re ( 4 . 1% ).

Original: We further evaluate our COOK on ImageCLEF-DA and report the classiﬁcation accuracy in Table 3.
Revised: We further evaluate our COOK on ImageCLEF-DA and report the classiﬁcation accuracy in Table 3.

Original: Our COOK outperforms 4 over 6 transfer tasks with an average accuracy of 90 . 8% , compared to ETD and RWOT with 89 .
Revised: Our COOK outperforms 4 over 6 transfer tasks with an average accuracy of 90 . 7% , compared to ETD and RWOT with 89 .

Original: 7% and 90 .
Revised: 7% and 90 .

Original: 3% , respectively.
Revised: 3% , respectively.

Original: [Table] 81.6 iCAN 92. 87.2 CDAN-E 94. 87.7 CDAN-BSP 93.
Revised: 6.4 ANALYSIS 6.4.1 Hyperparameter Sensitivity and Quantitative Evaluation

Original: 88.5 CDAN-TransNorm 95. 89.3 TPN 91. 87.1 rRevGrad+CAT 94.
Revised: We conduct experiments to evaluate hyperparameter sensitivity and quantitative result for our proposed COOK in

Original: 87.6 SymNets 90.
Revised: Figure 4.

Original: 88.4 DeepJDOT 88. 86.2 ETD 92. 86.2 RWOT 95.
Revised: Figure 4a experiences a decrease of the model per-

Original: 90.8 RADA 91.
Revised: [Table] [Figure]

Original: 87.3 CAN 94. 90.6 COOK 95. 6.4 ANALYSIS 6.4.1 Hyperparameter Sensitivity and Qualitative Evaluation We conduct experiments to evaluate hyperparameter sensitivity and qualitative results (e.g., t -SNE visualization) for our proposed COOK in Figure 4.
Revised: [Table]

Original: Figure 4a experiences a decrease of the model performance when twisting ρ in Eq.
Revised: [Table] [Table]

Original:  Our proposed COOK works well with ρ from 10 to 40 .
Revised: formance when twisting ρ in Eq. Our proposed COOK works well with ρ from 10 to 40 .

Original: Relying on this investigation, we pick ρ = 20 in our experiments.
Revised: Relying on this investigation, we pick ρ = 20 in our experiments.

Original: Similarly, Figure 4a shows results with the changes of τ .
Revised: Similarly, Figure 4a shows results with the changes of τ .

Original: We search τ in the grid of { 1 . 0 , 10 . 0 , 25 . 0 , 50 . 0 , 100 . 0 } and ﬁnd that setting τ = 10 . 0 achieves the best performance to perform knowledge distillation.
Revised: We search τ in the grid of { 1 . 0 , 10 . 0 , 25 . 0 , 50 . 0 , 100 . 0 } and ﬁnd that setting τ = 10 . 0 achieves the best performance to perform knowledge distillation.

Original: Furthermore, we investigate the Wasserstein distance W (cid:15)c, π (cid:0) Q T , Q S (cid:1) in Figure 4c, which sees a reduction during the training.
Revised: Furthermore, we investigate the Wasserstein distance W (cid:15)c, π (cid:0) Q T , Q S (cid:1) in Figure 4c, which sees a reduction during the training.

Original: This result shows the success of transporting target samples to their corresponding source class-conditional distributions.
Revised: This result shows the success of transporting target samples to their corresponding source class-conditional distributions.

Revised: We further evaluate the effects of the trade-off parameters α, β, γ on model performance by twisting their values.

Original: Finally, the source and target representations in the latent space are visualized in Figure 4d. The result shows that our COOK achieves a good alignment between the source andtarget domains with 31 clusters (corresponding to 31 classes of
Revised: Figure 5 shows results when we search α, β and γ in the grid of { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 , 5 . 0 , 10 . 0 } and report the test accuracy on two transfer tasks P → I ( ImageCLEF-DA ) and

Original:  Ofﬁce-31 ) and clear boundaries.
Revised: A → D ( Ofﬁce-31 ).

Original: The full visualization on transfer tasks A (cid:25) W and P (cid:25) C can be found in the supplementary material . 
Revised: The results show that the model yields the stable performance when α, β, γ from 0 . 001 to 1 . 0 . We ﬁnd that our COOK can achieve high performance when α = β = 1 . 0 and γ = 0 . 1 , hence we suggest picking these values on most of our experiments.

Original: 6.4.2 Effect of Losses
Revised: 6.4.2 Effect of Losses

Original: We investigate the effectiveness of the pseudo labelling loss
Revised: We investigate the effectiveness of the pseudo labelling loss

Original: L plw , the distillation loss L dl , and the clustering assumption loss L clus in Eq.
Revised: L plw , the distillation loss L dl , and the clustering assumption loss L clus in Eq.

Original: The experimental results are described in Table 4, which shows that all component losses contribute to the model performance since they participate  in the cyclic process and support to match target samples to the corresponding source regions.
Revised: The experimental results are described in Table 4, which shows that all component losses contribute to the model performance since they participate [Figure] [Figure] in the cyclic process and support to match target samples to the corresponding source regions.

Original: It is noticeable that the model performance is the best when all component losses are activated and participate in the training process.[Table] [Table] [Table]
Revised: It is noticeable that the model performance is the best when all component losses are activated and participate in the training process.

Original: [Table]

Original: 6.4.3 Effect of Knowledge Distillation
Revised: 6.4.3 Effect of Knowledge Distillation

Original: We further testify the contribution of KD to our proposed method in two different scenarios: Without KD and With KD .
Revised: We further testify the contribution of KD to our proposed method in two different scenarios: Without KD and With KD .

Original: For Without KD setting, we deploy a model where h S and h T are weight-sharing networks and train this model using the ﬁnal optimization problem where L dl = 0 .
Revised: For Without KD setting, we deploy a model where h S and h T are weight-sharing networks and train this model using the ﬁnal optimization problem where L dl = 0 .

Original: We compare the Without KD setting with our architecture COOK (a.k.a.
Revised: We compare the Without KD setting with our architecture COOK (a.k.a.

Original: With KD )
Revised: With KD )

Original: and report the accuracy score in Table 5.
Revised: and report the accuracy score in Table 5.

Original: The results show that our COOK with KD outperforms that without KD by nearly 1% , which demonstrates the effectiveness of KD for our framework.
Revised: The results show that our COOK with KD outperforms that without KD by nearly 1% , which demonstrates the effectiveness of KD for our framework.

Original: Due to  space limitations, we present additional studies in  the supplementary material .
Revised: We select transfer tasks A (cid:25) W ( Ofﬁce-31 ) and P (cid:25) C ( ImageCLEF-DA ) tasks to visualize their representation in the latent space using t -SNE [van der Maaten and Hinton, 2008]. The visualizations in Figure 6a and 6c show that after going through the backbone model ResNet-50, there is still a mismatch between the source and target distributions due to the data and label shifts. However, our proposed COOK (see Figure 6b and 6d) is trained to transport target samples to source samples, which closes this gap and achieves better alignment between the target and the source samples.

Original:  CONCLUSION
Revised:  CONCLUSION

Original: In this paper, we develop a novel framework for class-aware unsupervised domain adaptation.
Revised: In this paper, we develop a novel framework for class-aware unsupervised domain adaptation.

Original: In particular, our proposed method is based on the proposed distributional OT which quantiﬁes an OT distance between a distribution of target data and the source class-conditional distributions.
Revised: In particular, our proposed method is based on the proposed distributional OT which quantiﬁes an OT distance between a distribution of target data and the source class-conditional distributions.

Original: To efﬁciently train our model with the proposed distributional OT, we develop a novel model operating in a cyclic process.
Revised: To efﬁciently train our model with the proposed distributional OT, we develop a novel model operating in a cyclic process.

Original: By incorporating knowledge distillation and pseudo labelling technique into this process, our proposed COOK effectively tackles the data and label shifts problem by transporting the target samples to the corresponding source class-conditional distributions in a class-aware manner.
Revised: By incorporating knowledge distillation and pseudo labelling technique into this process, our proposed COOK effectively tackles the data and label shifts problem by transporting the target samples to the corresponding source class-conditional distributions in a class-aware manner.

Original: The experimental results show that COOK outperforms existing UDA methods, especially the class-aware and OT-based ones on the benchmark datasets.
Revised: The experimental results show that COOK outperforms existing UDA methods, especially the class-aware and OT-based ones on the benchmark datasets.

Original: IMPLEMENTATION SPECIFICATION 1.1 DATASETS

Original: Ofﬁce-31 [Saenko et al., 2010] is a well-known public dataset used for UDA.

Original: It consists of three domains including

Original: Amazon ( A ), Webcam ( W ) and Dslr ( D ) with 31 common classes and 4,110 images in total.

Original: Ofﬁce-Home [Venkateswara et al., 2017] is another and more challenging dataset for UDA which contains images from four different domains, namely Artistic ( Ar ), Clip

Original: Art ( Cl ), Product ( Pr ) and Real-world images ( Re ).

Original: This dataset consists of around 15,588 images in total withobject categories in ofﬁce and home scenes.

Original: ImageCLEF-DA [Caputo et al., 2014] includes three domains including Caltech-256 ( C ), ImageNet ILSVRC( I ), and Pascal VOC 2012 ( P ), each of which has 12 classes with 50 images per class.

Original: 1.2 NETWORK ARCHITECTURE

Original: The network architecture of COOK is described in Table 1.

Original: Our COOK includes a generator that jointly maps source and target examples into the latent space, while student and teacher classiﬁers are built on top of the generator without sharing weights.

Original: ADDITIONAL ABLATION STUDIES 2.1 HYPERPARAMETER SENSITIVITY
Revised: Acknowledgements

Original: We evaluate the effects of the trade-off parameters α, β, γ by twisting their values. Figure 1 shows results when we search α, β and γ in the grid of { 0 . 001 , 0 . 01 , 0 . 1 , 1 . 0 , 5 . 0 , 10 . 0 } and report the test accuracy on two transfer tasks P → I ( ImageCLEF-DA ) and A → D ( Ofﬁce-31 ). The results show that the model yields the stable performance when α, β, γ from 0 .

Original: 001 to 1 . 0 . We ﬁnd that our COOK can achieve high performance when α = β = 1 . 0 and γ = 0 . 1 , hence we suggest picking these values on most of our experiments. 2.2 FEATURE VISUALIZATION We select transfer tasks A (cid:25) W ( Ofﬁce-31 ) and P (cid:25) C ( ImageCLEF-DA ) tasks to visualize their representation in the latent space using t -SNE [van der Maaten and Hinton, 2008].
Revised: This work was supported by the US Air Force grant FA238621-1-4049.

Original: The visualizations in Figure 2a and 2c show that after going through the backbone model ResNet-50, there is still a mismatch between the source and target distributions due to the data and label shifts.

Original: However, our proposed COOK (see Figure 2b and 2d) is trained to transport target samples to source samples, which closes this gap and achieves better alignment between the target the source samples.