Original: Subsidiary Prototype Alignment for Universal Domain Adaptation
Revised: Subsidiary Prototype Alignment for Universal Domain Adaptation

Original: Abstract
Revised: Abstract

Original: Universal Domain Adaptation (UniDA) deals with the problem of knowledge transfer between two datasets with domain-shift as well as category-shift.
Revised: Universal Domain Adaptation (UniDA) deals with the problem of knowledge transfer between two datasets with domain-shift as well as category-shift.

Original: The goal is to categorize unlabeled target samples, either into one of the “known” categories or into a single “unknown” category.
Revised: The goal is to categorize unlabeled target samples, either into one of the “known” categories or into a single “unknown” category.

Original: A major problem in UniDA is negative transfer, i.e . misalignment of “known” and “unknown” classes.
Revised: A major problem in UniDA is negative transfer, i.e . misalignment of “known” and “unknown” classes.

Original: To this end, we first uncover an intriguing tradeoff between negative-transfer-risk and domaininvariance exhibited at different layers of a deep network.
Revised: To this end, we first uncover an intriguing tradeoff between negative-transfer-risk and domaininvariance exhibited at different layers of a deep network.

Original: It turns out we can strike a balance between these two metrics at a mid-level layer.
Revised: It turns out we can strike a balance between these two metrics at a mid-level layer.

Original: Towards designing an effective framework based on this insight, we draw motivation from Bag-of-visualWords (BoW).
Revised: Towards designing an effective framework based on this insight, we draw motivation from Bag-of-visualWords (BoW).

Original: Word-prototypes in a BoW-like representation of a mid-level layer would represent lower-level visual primitives that are likely to be unaffected by the category-shift in the high-level features.
Revised: Word-prototypes in a BoW-like representation of a mid-level layer would represent lower-level visual primitives that are likely to be unaffected by the category-shift in the high-level features.

Original: We develop modifications that encourage learning of word-prototypes followed by word-histogram based classification.
Revised: We develop modifications that encourage learning of word-prototypes followed by word-histogram based classification.

Original: Following this, subsidiary prototype-space alignment (SPA) can be seen as a closedset alignment problem, thereby avoiding negative transfer.
Revised: Following this, subsidiary prototype-space alignment (SPA) can be seen as a closedset alignment problem, thereby avoiding negative transfer.

Original: We realize this with a novel word-histogram-related pretext task to enable closed-set SPA, operating in conjunction with goal task UniDA.
Revised: We realize this with a novel word-histogram-related pretext task to enable closed-set SPA, operating in conjunction with goal task UniDA.

Original: We demonstrate the efficacy of our approach on top of existing UniDA techniques 1 , yielding state-of-the-art performance across three standard UniDA and Open-Set DA object recognition benchmarks.
Revised: We demonstrate the efficacy of our approach on top of existing UniDA techniques 1 , yielding state-of-the-art performance across three standard UniDA and Open-Set DA object recognition benchmarks.

Original:  Introduction
Revised:  Introduction

Original: Despite the success of deep networks trained on large-scale datasets, they are found to be brittle under input distribution shift i e .
Revised: Despite the success of deep networks trained on large-scale datasets, they are found to be brittle under input distribution shift i e .

Original: domain-shift [8].
Revised: domain-shift [8].

Original: Thus, adapting a trained model for a new target environment becomes challenging as data annotation is too expensive or time-consuming [9] for every new target dataset.
Revised: Thus, adapting a trained model for a new target environment becomes challenging as data annotation is too expensive or time-consuming [9] for every new target dataset.

Original: Unsupervised Domain Adaptation (DA) [13] is one of the solutions to this problem where knowledge is transferred from a labeled source domain to an unlabeled target domain.
Revised: Unsupervised Domain Adaptation (DA) [13] is one of the solutions to this problem where knowledge is transferred from a labeled source domain to an unlabeled target domain.

Original: While most works [14, 30, 23] focused on Closed-Set DA, where source and target label sets are shared ( C s = C t ), recent works introduced disjoint label set scenarios like Partial DA [58, 6] ( C t ⊂ C s ) and Open-Set DA [43, 21] ( C s ⊂ C t ).
Revised: While most works [14, 30, 23] focused on Closed-Set DA, where source and target label sets are shared ( C s = C t ), recent works introduced disjoint label set scenarios like Partial DA [58, 6] ( C t ⊂ C s ) and Open-Set DA [43, 21] ( C s ⊂ C t ).

Original: However, the most practical setting is Universal DA (UniDA) [55] where the relation between the source and target label sets is unknown i.e .
Revised: However, the most practical setting is Universal DA (UniDA) [55] where the relation between the source and target label sets is unknown i.e .

Original: with any number of shared, source-private and target-private classes.
Revised: with any number of shared, source-private and target-private classes.

Original: In UniDA, a model is trained to categorize unlabeled target samples into one of the shared classes (“known” classes) or into a single “unknown” class.
Revised: In UniDA, a model is trained to categorize unlabeled target samples into one of the shared classes (“known” classes) or into a single “unknown” class.

Original: The major problem in UniDA is negative-transfer [44] where misalignment between the shared and private classes degrades the adaptation performance.
Revised: The major problem in UniDA is negative-transfer [44] where misalignment between the shared and private classes degrades the adaptation performance.

Original: On the other hand, we have the domain-shift problem, which is usually handled by learning domain-invariant features [55, 22, 25].
Revised: On the other hand, we have the domain-shift problem, which is usually handled by learning domain-invariant features [55, 22, 25].

Original: So, we first
Revised: So, we first

Original: Pretext task to aid SPA
Revised: Pretext task to aid SPA

Original: [Figure]
Revised: [Figure]

Original: Realized via BoW
Revised: Realized via BoW

Original: [Figure]
Revised: [Figure]

Original: inspired framework
Revised: inspired framework

Original: [Figure]
Revised: [Figure]

Original: perform a control experiment to analyze the negative-transfer-risk (NTR) and domain-invariance-score (DIS) at different layers of the deep network (Fig. 2).
Revised: perform a control experiment to analyze the negative-transfer-risk (NTR) and domain-invariance-score (DIS) at different layers of the deep network (Fig. 2).

Original: NTR is measured through the class-specificity of the features via an entropy-based shared-vs-unknown binary classifier while DIS is measured as the inverse of the standard A -distance [1] that quantifies domain discrepancy.
Revised: NTR is measured through the class-specificity of the features via an entropy-based shared-vs-unknown binary classifier while DIS is measured as the inverse of the standard A -distance [1] that quantifies domain discrepancy.

Original: We observe that
Revised: We observe that

Original: NTR and DIS are at odds with each other i.e .
Revised: NTR and DIS are at odds with each other i.e .

Original: NTR increases while DIS decreases as we move from lower to higher-level (deeper) layers.
Revised: NTR increases while DIS decreases as we move from lower to higher-level (deeper) layers.

Original: Thus, we arrive at contradicting solutions where avoiding negative-transfer requires adaptation at a lower-level layer while effective domain-invariance requires adaptation at a deeper layer.
Revised: Thus, we arrive at contradicting solutions where avoiding negative-transfer requires adaptation at a lower-level layer while effective domain-invariance requires adaptation at a deeper layer.

Original: While a balance can be naively struck at a mid-level layer, we ask, can we further develop and constrain the mid-level representation space to avoid negative transfer?
Revised: While a balance can be naively struck at a mid-level layer, we ask, can we further develop and constrain the mid-level representation space to avoid negative transfer?

Original: Motivated by Bag-of-visual-Words (BoW) [56], we hypothesize that a BoW-like representation at a mid-level layer would represent lower-level visual primitives that are unaffected by the category-shift in the higher-level features.
Revised: Motivated by Bag-of-visual-Words (BoW) [56], we hypothesize that a BoW-like representation at a mid-level layer would represent lower-level visual primitives that are unaffected by the category-shift in the higher-level features.

Original: We illustrate this idea in Fig.
Revised: We illustrate this idea in Fig.

Original: 1 with a UniDA scenario where source and target have private and shared classes.
Revised: 1 with a UniDA scenario where source and target have private and shared classes.

Original: Note that, in most practical scenarios, the private categories are usually related to the shared categories ( e.g . a tractor and an excavator may be private classes in rural and urban scenes respectively, but they share some common visual attributes like their chassis).
Revised: Note that, in most practical scenarios, the private categories are usually related to the shared categories ( e.g . a tractor and an excavator may be private classes in rural and urban scenes respectively, but they share some common visual attributes like their chassis).

Original: Thus, in a word-prototype-space, different visual primitives can be shared across domains and classes (including unknown classes) and a closed-set alignment of target features with the primitives can be performed.
Revised: Thus, in a word-prototype-space, different visual primitives can be shared across domains and classes (including unknown classes) and a closed-set alignment of target features with the primitives can be performed.

Original: Next, we explain how to realize this subsidiary prototype-space alignment (SPA).
Revised: Next, we explain how to realize this subsidiary prototype-space alignment (SPA).

Original: First, we propose architecture modifications to introduce explicit word-prototypes and extract a wordhistogram output.
Revised: First, we propose architecture modifications to introduce explicit word-prototypes and extract a wordhistogram output.

Original: We analyze the alignment of feature vectors with different word-prototypes.
Revised: We analyze the alignment of feature vectors with different word-prototypes.

Original: Here, better prototype-space alignment would imply sparser word-histograms as a specific word-prototype would have a significant contribution to the word-histogram w.r.t.
Revised: Here, better prototype-space alignment would imply sparser word-histograms as a specific word-prototype would have a significant contribution to the word-histogram w.r.t.

Original: other prototypes.
Revised: other prototypes.

Original: Intuitively, with higher sparsity, the word-histogram space can better represent the intrinsic class structure (including unknown classes).
Revised: Intuitively, with higher sparsity, the word-histogram space can better represent the intrinsic class structure (including unknown classes).

Original: To enforce this sparsity i.e . the subsidiary prototype-space alignment (SPA), we minimize a self-entropy loss at the word-histogram level.
Revised: To enforce this sparsity i.e . the subsidiary prototype-space alignment (SPA), we minimize a self-entropy loss at the word-histogram level.

Original: However, this remains susceptible to word-level misalignment due to a lack of word-related priors.
Revised: However, this remains susceptible to word-level misalignment due to a lack of word-related priors.

Original: Thus, we seek a word-related prior that can be cast into a self-supervised pretext task so that both labeled source and unlabeled target can be used.
Revised: Thus, we seek a word-related prior that can be cast into a self-supervised pretext task so that both labeled source and unlabeled target can be used.

Original: Given that each instance can be represented as a word-histogram, we find a simple property based on grid-shuffling of image crops (Fig. 4).
Revised: Given that each instance can be represented as a word-histogram, we find a simple property based on grid-shuffling of image crops (Fig. 4).

Original: Here, the word-histogram entropy of a grid-shuffled image increases with the number of distinct instances that contribute crops.
Revised: Here, the word-histogram entropy of a grid-shuffled image increases with the number of distinct instances that contribute crops.

Original: Based on this, we create a novel pretext task to classify the number of instances used in an input grid-shuffled image.
Revised: Based on this, we create a novel pretext task to classify the number of instances used in an input grid-shuffled image.

Original: Intuitively, it encourages prototype alignment (SPA) as distinguishing different classes in this pretext task becomes easier with higher sparsity of word-histograms.
Revised: Intuitively, it encourages prototype alignment (SPA) as distinguishing different classes in this pretext task becomes easier with higher sparsity of word-histograms.

Original: To summarize, our contributions are as follows,
Revised: To summarize, our contributions are as follows,

Original: • We are the first to uncover and analyze the tradeoff between negative-transfer and domain-invariance in UniDA.
Revised: • We are the first to uncover and analyze the tradeoff between negative-transfer and domain-invariance in UniDA.

Original: While a naive balance can be struck, we introduce BoW-inspired word-prototypes and a subsidiary prototype-space alignment (SPA) objective to further alleviate negative-transfer.
Revised: While a naive balance can be struck, we introduce BoW-inspired word-prototypes and a subsidiary prototype-space alignment (SPA) objective to further alleviate negative-transfer.

Original: • We devise a word-related prior and cast it as a self-supervised pretext task to further improve SPA by avoiding word-level misalignment.
Revised: • We devise a word-related prior and cast it as a self-supervised pretext task to further improve SPA by avoiding word-level misalignment.

Original: • Our approach, coupled with existing UniDA approaches, yields state-of-the-art performance across three standard Open-Set DA and UniDA benchmarks for object recognition.
Revised: • Our approach, coupled with existing UniDA approaches, yields state-of-the-art performance across three standard Open-Set DA and UniDA benchmarks for object recognition.

Original:  Related work
Revised:  Related work

Original: Open-Set DA (OSDA) has been studied in several scenarios [43, 34, 3, 10].
Revised: Open-Set DA (OSDA) has been studied in several scenarios [43, 34, 3, 10].

Original: We focus on the case given by [43], where target domain contains private classes, unknown to source.
Revised: We focus on the case given by [43], where target domain contains private classes, unknown to source.

Original: [43] presented an adversarial learning method where feature generator learns known-unknown separation.
Revised: [43] presented an adversarial learning method where feature generator learns known-unknown separation.

Original: Other works focus on anomaly measurement [35] or learn to discriminate known and unknown samples [54, 39, 33].
Revised: Other works focus on anomaly measurement [35] or learn to discriminate known and unknown samples [54, 39, 33].

Original: ROS [3] uses rotation prediction to separate known and unknown samples whereas our word-based pretext task regularizes UniDA and implicitly improves known-unknown separation.
Revised: ROS [3] uses rotation prediction to separate known and unknown samples whereas our word-based pretext task regularizes UniDA and implicitly improves known-unknown separation.

Original: Universal DA (UniDA) [55] is a complex DA scenario that assumes no prior knowledge of the relationship between the source and target label spaces.
Revised: Universal DA (UniDA) [55] is a complex DA scenario that assumes no prior knowledge of the relationship between the source and target label spaces.

Original: Similar to Open-Set DA, UniDA also requires identification of target-private classes.
Revised: Similar to Open-Set DA, UniDA also requires identification of target-private classes.

Original: Prior works [12, 55, 44, 3] computed a confidence score for known classes, and data with a score below a threshold were considered unknown.
Revised: Prior works [12, 55, 44, 3] computed a confidence score for known classes, and data with a score below a threshold were considered unknown.

Original: [44] proposed neighbourhood clustering to understand the target domain structure and an entropy separation loss for feature alignment.
Revised: [44] proposed neighbourhood clustering to understand the target domain structure and an entropy separation loss for feature alignment.

Original: OVANet [42] used binary classifiers in a one-vs-all manner to identify unknown samples, and DCC [27] used domain consensus knowledge to find discriminative clusters in both shared and private data.
Revised: OVANet [42] used binary classifiers in a one-vs-all manner to identify unknown samples, and DCC [27] used domain consensus knowledge to find discriminative clusters in both shared and private data.

Original: In contrast to prior arts, we draw motivation from Bag-of-visual-Words and construct a pretext task to complement these works by enhancing their intrinsic domain structure.
Revised: In contrast to prior arts, we draw motivation from Bag-of-visual-Words and construct a pretext task to complement these works by enhancing their intrinsic domain structure.

Original: BoW-related works.
Revised: BoW-related works.

Original: Early works utilized Bag-of-visual-Words (BoW) representations for downstream applications like object recognition [51], object detection [2], image retrieval [46], etc .
Revised: Early works utilized Bag-of-visual-Words (BoW) representations for downstream applications like object recognition [51], object detection [2], image retrieval [46], etc .

Original: More recent work [15, 16] leveraged BoW prediction for self-supervised learning of representations for downstream tasks.
Revised: More recent work [15, 16] leveraged BoW prediction for self-supervised learning of representations for downstream tasks.

Original: To the best of our knowledge, we are the first to utilize BoW concepts in UniDA.
Revised: To the best of our knowledge, we are the first to utilize BoW concepts in UniDA.

Original:  Approach 3.1 Preliminaries
Revised:  Approach 3.1 Preliminaries

Original: Consider a labeled source dataset D s = { ( x s , y s ) : x s ∈ X , y s ∈ C s } where C s is the source label set,
Revised: Consider a labeled source dataset D s = { ( x s , y s ) : x s ∈ X , y s ∈ C s } where C s is the source label set,

Original: X is the input space, and x s is drawn from the marginal distribution p s .
Revised: X is the input space, and x s is drawn from the marginal distribution p s .

Original: Also consider an unlabeled dataset D t = { x t : x t ∈ X} where x t is drawn from the marginal distribution p t .
Revised: Also consider an unlabeled dataset D t = { x t : x t ∈ X} where x t is drawn from the marginal distribution p t .

Original: Let C t denote the target label set.
Revised: Let C t denote the target label set.

Original: In Universal DA [55], the relationship between C s and C t is unknown.
Revised: In Universal DA [55], the relationship between C s and C t is unknown.

Original: Without loss of generality, the shared label set is C =
Revised: Without loss of generality, the shared label set is C =

Original: C s ∩ C t and the private label sets for source and target are
Revised: C s ∩ C t and the private label sets for source and target are

Original: C s = C s \ C t and C t =
Revised: C s = C s \ C t and C t =

Original: C t \ C s respectively.
Revised: C t \ C s respectively.

Original: Next, we define two measures for our insights.
Revised: Next, we define two measures for our insights.

Original: Negative-Transfer-Risk (NTR).
Revised: Negative-Transfer-Risk (NTR).

Original: Negative transfer [50] is a major problem in DA where class-level misalignment occurs ( e.g . source class “dog" may get wrongly aligned with target class “cat" due to some similarities between the two classes).
Revised: Negative transfer [50] is a major problem in DA where class-level misalignment occurs ( e.g . source class “dog" may get wrongly aligned with target class “cat" due to some similarities between the two classes).

Original: The problem is aggravated in UniDA as both source and target may have private classes which may be wrongly aligned with the shared classes [55].
Revised: The problem is aggravated in UniDA as both source and target may have private classes which may be wrongly aligned with the shared classes [55].

Original: Thus, we introduce a negative-transfer-risk (NTR) γ NTR ( h ) for a given feature extractor h : X →Z , where Z is an intermediate feature-space.
Revised: Thus, we introduce a negative-transfer-risk (NTR) γ NTR ( h ) for a given feature extractor h : X →Z , where Z is an intermediate feature-space.

Original: NTR is computed as the target shared-vs-private classification accuracy via a self-entropy threshold on a source-trained linear task-classifier (see Suppl.
Revised: NTR is computed as the target shared-vs-private classification accuracy via a self-entropy threshold on a source-trained linear task-classifier (see Suppl.

Original: for complete details),
Revised: for complete details),

Original: [Equation]
Revised: [Equation]

Original: Here, ˆ y ( sp ) t is the prediction (shared or private) using a fixed self-entropy threshold and y ( sp ) t represents shared-private label (0 for shared, 1 for private).
Revised: Here, ˆ y ( sp ) t is the prediction (shared or private) using a fixed self-entropy threshold and y ( sp ) t represents shared-private label (0 for shared, 1 for private).

Original: We access the shared-private labels for a subset of target data only for analysis (not for training).
Revised: We access the shared-private labels for a subset of target data only for analysis (not for training).

Original: Intuitively, if the features h ( x ) are highly class-specific, then target-private samples would yield highly uncertain predictions (as they are unseen by the source-trained linear classifier) compared to shared-label samples.
Revised: Intuitively, if the features h ( x ) are highly class-specific, then target-private samples would yield highly uncertain predictions (as they are unseen by the source-trained linear classifier) compared to shared-label samples.

Original: Thus, target-private samples would be easily separable with the self-entropy threshold, leading to a high NTR.
Revised: Thus, target-private samples would be easily separable with the self-entropy threshold, leading to a high NTR.

Original: Domain-Invariance-Score (DIS).
Revised: Domain-Invariance-Score (DIS).

Original: The standard A -distance [1] measures the discrepancy between two domains.
Revised: The standard A -distance [1] measures the discrepancy between two domains.

Original: It is computed using the accuracy of a linear domain classifier (source vs target).
Revised: It is computed using the accuracy of a linear domain classifier (source vs target).

Original: Since we aim to measure domain invariance, i.e . the inverse of domain discrepancy, we use the inverse of
Revised: Since we aim to measure domain invariance, i.e . the inverse of domain discrepancy, we use the inverse of

Original: A -distance between the source and target datasets for the given feature extractor h ,
Revised: A -distance between the source and target datasets for the given feature extractor h ,

Original: [Equation]
Revised: [Equation]

Original: Here, d A ( ., . ) denotes A -distance computed using feature outputs of h .
Revised: Here, d A ( ., . ) denotes A -distance computed using feature outputs of h .

Original: Note that 0 ≤ d A ( ., . )
Revised: Note that 0 ≤ d A ( ., . )

Original: ≤ 2 .
Revised: ≤ 2 .

Original: 3.2 Balancing negative-transfer-risk and domain-invariance
Revised: 3.2 Balancing negative-transfer-risk and domain-invariance

Original: Given that UniDA is highly susceptible to negative-transfer due to the category-shift problem, we analyze the negative-transfer-risk at different layers of the deep network.
Revised: Given that UniDA is highly susceptible to negative-transfer due to the category-shift problem, we analyze the negative-transfer-risk at different layers of the deep network.

Original: Here, we consider that shallow layers encode low-level visual features like edges, corners, etc .
Revised: Here, we consider that shallow layers encode low-level visual features like edges, corners, etc .

Original: while deeper layers encode more abstract, class-specific features [57].
Revised: while deeper layers encode more abstract, class-specific features [57].

Original: In the context of UniDA, the feature space of the deeper layers for source and target would be more difficult to align due to the disjointness of the source and target label sets.
Revised: In the context of UniDA, the feature space of the deeper layers for source and target would be more difficult to align due to the disjointness of the source and target label sets.

Original: We empirically observe the same
Revised: We empirically observe the same

Original: , i.e . NTR increases as we go deeper in the model (solid blue curve in Fig. 2).
Revised: , i.e . NTR increases as we go deeper in the model (solid blue curve in Fig. 2).

Original: This suggests that adaptation should be performed at a shallower layer.
Revised: This suggests that adaptation should be performed at a shallower layer.

Original: [Figure]
Revised: [Figure]

Original: decreases
Revised: decreases

Original: While we have considered the category-shift problem to understand where to perform the adaptation, we cannot overlook the domain-shift problem.
Revised: While we have considered the category-shift problem to understand where to perform the adaptation, we cannot overlook the domain-shift problem.

Original: Most DA works [14] perform adaptation at deeper layers, which tend to learn increasingly more domain-specific features [47, 24].
Revised: Most DA works [14] perform adaptation at deeper layers, which tend to learn increasingly more domain-specific features [47, 24].

Original: The higher capacity of the deeper layers leads to unregularized domain-specific learning.
Revised: The higher capacity of the deeper layers leads to unregularized domain-specific learning.

Original: We also empirically observe that DIS (inverse of domain-specificity) decreases for the deeper layers (solid pink curve in Fig. 2).
Revised: We also empirically observe that DIS (inverse of domain-specificity) decreases for the deeper layers (solid pink curve in Fig. 2).

Original: This suggests that adaptation should be performed at a deeper layer to encourage domain-invariance.
Revised: This suggests that adaptation should be performed at a deeper layer to encourage domain-invariance.

Original: Thus, the problems of domain-shift and category-shift are contradictory in suggesting adaptation at a deeper and shallower layer respectively.
Revised: Thus, the problems of domain-shift and category-shift are contradictory in suggesting adaptation at a deeper and shallower layer respectively.

Original: In other words, there is a tradeoff between DIS and NTR (Fig. 2) where reducing the negative-transfer-risk negatively affects the domain-invariance and vice versa.
Revised: In other words, there is a tradeoff between DIS and NTR (Fig. 2) where reducing the negative-transfer-risk negatively affects the domain-invariance and vice versa.

Original: Thus, we define a criterion for minimal negative-transfer and maximal domain-invariance.
Revised: Thus, we define a criterion for minimal negative-transfer and maximal domain-invariance.

Original: Definition 1.
Revised: Definition 1.

Original: (Optimal tradeoff between negative-transfer and domain-invariance) Consider that adaptation is performed at the L th layer of the backbone h (let h L denote the backbone upto
Revised: (Optimal tradeoff between negative-transfer and domain-invariance) Consider that adaptation is performed at the L th layer of the backbone h (let h L denote the backbone upto

Original: L th layer).
Revised: L th layer).

Original: Then adaptation at h L will encounter minimal negative-transfer and exhibit maximum domain-invariance if with at least (1 − δ ) probability, γ NTR ( h L ) does not exceed ζ n by more than ε n , and γ DIS ( h L ) exceeds ζ d by no less than ε d , i.e .,
Revised: Then adaptation at h L will encounter minimal negative-transfer and exhibit maximum domain-invariance if with at least (1 − δ ) probability, γ NTR ( h L ) does not exceed ζ n by more than ε n , and γ DIS ( h L ) exceeds ζ d by no less than ε d , i.e .,

Original: [Equation]
Revised: [Equation]

Original: Thus, an optimal tradeoff requires NTR to be less than the threshold ζ n and DIS to be greater than the threshold ζ d simultaneously.
Revised: Thus, an optimal tradeoff requires NTR to be less than the threshold ζ n and DIS to be greater than the threshold ζ d simultaneously.

Original: Empirically, we find a good tradeoff at a mid-level layer (green vertical line in Fig.
Revised: Empirically, we find a good tradeoff at a mid-level layer (green vertical line in Fig.

Original: 2, e.g . Res3 block in ResNet), i.e . a compromise between the contradicting suggestions.
Revised: 2, e.g . Res3 block in ResNet), i.e . a compromise between the contradicting suggestions.

Original: Note that we re-use h to represent the backbone upto an optimal layer L instead of h L for simplicity.
Revised: Note that we re-use h to represent the backbone upto an optimal layer L instead of h L for simplicity.

Original: Why is low NTR desirable?
Revised: Why is low NTR desirable?

Original: High NTR implies known and unknown samples are well-separated.
Revised: High NTR implies known and unknown samples are well-separated.

Original: However, unsupervised adaptation at a higher NTR layer is more susceptible to misalignment between shared and private (unknown) classes because target-private classes get grouped into a single unknown cluster.
Revised: However, unsupervised adaptation at a higher NTR layer is more susceptible to misalignment between shared and private (unknown) classes because target-private classes get grouped into a single unknown cluster.

Original: In contrast, lower NTR feature space can better represent all the different classes without grouping the target-private classes into a single unknown cluster ( i.e . better intrinsic structure).
Revised: In contrast, lower NTR feature space can better represent all the different classes without grouping the target-private classes into a single unknown cluster ( i.e . better intrinsic structure).

Original: Hence, alignment in this space would better respect the separations of private classes than at a higher NTR feature space, which is necessary to avoid misalignment in UniDA.
Revised: Hence, alignment in this space would better respect the separations of private classes than at a higher NTR feature space, which is necessary to avoid misalignment in UniDA.

Original: For example, consider “hatchback" (compact car) and “SUV" (large-sized car) as a shared and target-private class, respectively, in an object recognition task.
Revised: For example, consider “hatchback" (compact car) and “SUV" (large-sized car) as a shared and target-private class, respectively, in an object recognition task.

Original: Before adaptation, at a high NTR layer, hatchback and SUV features would be well-separated as SUV is yet unseen to the source-trained model.
Revised: Before adaptation, at a high NTR layer, hatchback and SUV features would be well-separated as SUV is yet unseen to the source-trained model.

Original: However, during unsupervised adaptation, the similarities between hatchbacks and SUVs may align the single target-private cluster (containing SUV features) with the hatchback cluster.
Revised: However, during unsupervised adaptation, the similarities between hatchbacks and SUVs may align the single target-private cluster (containing SUV features) with the hatchback cluster.

Original: Due to this, other target-private classes also become closer to this hatchback class which increases the misalignment.
Revised: Due to this, other target-private classes also become closer to this hatchback class which increases the misalignment.

Original: In contrast, at a lower NTR layer, the target-private classes (including SUV) would not be grouped together.
Revised: In contrast, at a lower NTR layer, the target-private classes (including SUV) would not be grouped together.

Original: Hence, misalignment of hatchback and SUV clusters would not disturb other clusters unlike the higher NTR scenario.
Revised: Hence, misalignment of hatchback and SUV clusters would not disturb other clusters unlike the higher NTR scenario.

Original: 3.3 Conceptualizing Bag-of-visual-Words (BoW) for UniDA
Revised: 3.3 Conceptualizing Bag-of-visual-Words (BoW) for UniDA

Original: Now we ask whether a meaningful representation space can be developed at this mid-level layer to effectively mitigate negative-transfer.
Revised: Now we ask whether a meaningful representation space can be developed at this mid-level layer to effectively mitigate negative-transfer.

Original: To this end, we draw motivation from traditional Bag-of-visualWords (BoW) concepts.
Revised: To this end, we draw motivation from traditional Bag-of-visualWords (BoW) concepts.

Original: Consider a vocabulary V
Revised: Consider a vocabulary V

Original: =
Revised: =

Original: [v 1 , v 2 , . .
Revised: [v 1 , v 2 , . .

Original: . , v K ] where each word-prototype v k ∈ R N d ∀ k ∈ { 1 , 2 , . .
Revised: . , v K ] where each word-prototype v k ∈ R N d ∀ k ∈ { 1 , 2 , . .

Original: , K } ,
Revised: , K } ,

Original: i.e . K word-prototypes where each v k is a N d -dimensional vector.
Revised: i.e . K word-prototypes where each v k is a N d -dimensional vector.

Original: Here, the word-prototypes are representative of lower-level visual primitives [56] ( e.g . SIFT-like features).
Revised: Here, the word-prototypes are representative of lower-level visual primitives [56] ( e.g . SIFT-like features).

Original: Under the BoW idea [52], a histogram of word-prototypes ( word-histogram ) can be used as a representation of an image for downstream applications.
Revised: Under the BoW idea [52], a histogram of word-prototypes ( word-histogram ) can be used as a representation of an image for downstream applications.

Original: Through the following insight, we argue that BoW concepts can be leveraged in UniDA, for the problems arising from disjoint label sets.
Revised: Through the following insight, we argue that BoW concepts can be leveraged in UniDA, for the problems arising from disjoint label sets.

Original: Insight 1.
Revised: Insight 1.

Original: (Suitability of BoW concepts for UniDA)
Revised: (Suitability of BoW concepts for UniDA)

Original: Word-prototypes represent lower-level visual primitives which are largely unaffected by category-shift in the high-level features.
Revised: Word-prototypes represent lower-level visual primitives which are largely unaffected by category-shift in the high-level features.

Original: Thus, subsidiary closed-set word-prototype-space alignment assists UniDA with minimal negative-transfer-risk (NTR).
Revised: Thus, subsidiary closed-set word-prototype-space alignment assists UniDA with minimal negative-transfer-risk (NTR).

Original: Remarks.
Revised: Remarks.

Original: We conceptually illustrate this in Fig.
Revised: We conceptually illustrate this in Fig.

Original: Usually, the private categories are somewhat related to the shared categories.
Revised: Usually, the private categories are somewhat related to the shared categories.

Original: Thus, we hypothesize that a set of lower-level visual primitives (or word-prototypes) are capable of representing all categories (even unknown) in the word-prototypespace shared across domains and classes.
Revised: Thus, we hypothesize that a set of lower-level visual primitives (or word-prototypes) are capable of representing all categories (even unknown) in the word-prototypespace shared across domains and classes.

Original: In Fig.
Revised: In Fig.

Original: 1, before adaptation, different target features are scattered around the word-prototypes.
Revised: 1, before adaptation, different target features are scattered around the word-prototypes.

Original: Then, performing closed-set alignment between the features and word-prototypes can better capture the intrinsic class-structure to support UniDA.
Revised: Then, performing closed-set alignment between the features and word-prototypes can better capture the intrinsic class-structure to support UniDA.

Original: 3.4 Subsidiary Prototype-space Alignment (SPA)
Revised: 3.4 Subsidiary Prototype-space Alignment (SPA)

Original: An obvious question remains: How to realize the closed-set word-prototype-space alignment described in Insight 1?
Revised: An obvious question remains: How to realize the closed-set word-prototype-space alignment described in Insight 1?

Original: This is crucial because word-prototypes are abstract concepts which need to be explicitly realized.
Revised: This is crucial because word-prototypes are abstract concepts which need to be explicitly realized.

Original: To this end, we propose minor architectural modifications in the backbone.
Revised: To this end, we propose minor architectural modifications in the backbone.

Original: We insert a block ψ after the backbone h (see Fig. 3).
Revised: We insert a block ψ after the backbone h (see Fig. 3).

Original: First, a 1 × 1 -conv.
Revised: First, a 1 × 1 -conv.

Original: layer converts the N ddimensional spatial features to K -dimensional spatial features (same spatial size as a 1 × 1 filter is used with stride 1).
Revised: layer converts the N ddimensional spatial features to K -dimensional spatial features (same spatial size as a 1 × 1 filter is used with stride 1).

Original: We interpret its weight matrix ( V ∈ R N d × K ) as a vocabulary containing K number of N d -dimensional word-prototypes.
Revised: We interpret its weight matrix ( V ∈ R N d × K ) as a vocabulary containing K number of N d -dimensional word-prototypes.

Original: The softmax activation performs soft-quantization of the input features h ( x ) w.r.t.
Revised: The softmax activation performs soft-quantization of the input features h ( x ) w.r.t.

Original: the word-prototypes in V .
Revised: the word-prototypes in V .

Original: [Figure]
Revised: [Figure]

Revised: the features in terms of word-prototypes in V .

Original: Consider h u ( x ) ∈ R N d , the feature vector at a spatial location u ∈ { 1 , 2 , . .
Revised: Consider h u ( x ) ∈ R N d , the feature vector at a spatial location u ∈ { 1 , 2 , . .

Original: , U } (where U is the number of spatial locations in the features) and the vocabulary V =[v 1 , v 2 , . .
Revised: , U } (where U is the number of spatial locations in the features) and the vocabulary V =[v 1 , v 2 , . .

Original: . , v K ] where v k represents the k th word-prototype.
Revised: . , v K ] where v k represents the k th word-prototype.

Original: Then, the soft-quantization (soft equivalent of number of occurrences in a word-histogram) at a spatial location u for the k th word-prototype v k is,
Revised: Then, the soft-quantization (soft equivalent of number of occurrences in a word-histogram) at a spatial location u for the k th word-prototype v k is,

Original: [Equation]
Revised: [Equation]

Original: ϕ [Equation] Following this, we employ the remaining Res4-like conv-layers block (as Res3 was chosen for adaptation in Fig. 2) and global average pooling (GAP) to obtain a feature vector ψ ( x ) ∈ R K from the spatially dense features ϕ ( x ) .
Revised: Following this, we employ the remaining Res4-like conv-layers block (as Res3 was chosen for adaptation in Fig. 2) and global average pooling (GAP) to obtain a feature vector ψ ( x ) ∈ R K from the spatially dense features ϕ ( x ) .

Original: Thus, we repurpose a simple 1 × 1 conv.
Revised: Thus, we repurpose a simple 1 × 1 conv.

Original: layer to implement soft word-quantization with the layer weights as the word-prototypes.
Revised: layer to implement soft word-quantization with the layer weights as the word-prototypes.

Original: Now, we introduce a prototypealignment-score (PAS) to further motivate the effectiveness of BoW concepts.
Revised: Now, we introduce a prototypealignment-score (PAS) to further motivate the effectiveness of BoW concepts.

Original: Prototype-Alignment-Score (PAS).
Revised: Prototype-Alignment-Score (PAS).

Original: Consider the backbone feature vector h u ( x ) at spatial location u .
Revised: Consider the backbone feature vector h u ( x ) at spatial location u .

Original: We compute PAS, γ PAS , as k -means loss i.e . distance of h u ( x ) to the closest word-prototype in V ,γ PAS ( [Equation] ( x ) , [Equation]
Revised: We compute PAS, γ PAS , as k -means loss i.e . distance of h u ( x ) to the closest word-prototype in V ,

Original: [Equation]

Original: −

Original: [Equation]

Original: ∈

Original: [Equation]
Revised: [Equation]

Original: Here, ℓ cos denotes cosine-distance.
Revised: Here, ℓ cos denotes cosine-distance.

Original: Intuitively, γ PAS indicates how close the feature vector at u is to one of the word-prototypes in V .
Revised: Intuitively, γ PAS indicates how close the feature vector at u is to one of the word-prototypes in V .

Original: Since ϕ u ( x ) is a word-histogram (softmax probabilities), a higher γ PAS indicates that the closest prototype would have a much higher contribution in the word-histogram than the others i.e . a sparser word-histogram.
Revised: Since ϕ u ( x ) is a word-histogram (softmax probabilities), a higher γ PAS indicates that the closest prototype would have a much higher contribution in the word-histogram than the others i.e . a sparser word-histogram.

Original: Based on PAS, we arrive at the following insight.
Revised: Based on PAS, we arrive at the following insight.

Original: Crop-level
Revised: Crop-level

Original: Word-hist.
Revised: Word-hist.

Original: ClassEach crop yields a similarword-hist.
Revised: ClassEach crop yields a similarword-hist.

Original: Imagesamples
Revised: Imagesamples

Original: ClassClass-
Revised: ClassClass-

Original: Grid-shuffled images
Revised: Grid-shuffled images

Original: Entropy of word-hist increases with #inst.
Revised: Entropy of word-hist increases with #inst.

Original: #inst.
Revised: #inst.

Original: =
Revised: =

Original: #inst.
Revised: #inst.

Original: =
Revised: =

Original: #inst.
Revised: #inst.

Original: =
Revised: =

Original: ImagelevelWord- hist.
Revised: ImagelevelWord- hist.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: task use no. of instances for grid-shufﬂing (at least one crop from each of the instances)
Revised: task use no. of instances for grid-shufﬂing (at least one crop from each of the instances)

Original: Dataset
Revised: Dataset

Original: C l a ssC l a ssC l a ssC l a ssGrid-Size (4 here)
Revised: C l a ssC l a ssC l a ssC l a ssGrid-Size (4 here)

Original: [Figure]
Revised: [Figure]

Original: Insight 2.
Revised: Insight 2.

Original: (Encouraging Prototype-Alignment for UniDA)
Revised: (Encouraging Prototype-Alignment for UniDA)

Original: Since better prototype-alignment implies sparser word-histograms, the word-histogram-space ϕ
Revised: Since better prototype-alignment implies sparser word-histograms, the word-histogram-space ϕ

Original: ( x ) would better represent the intrinsic class structure (including private classes).
Revised: ( x ) would better represent the intrinsic class structure (including private classes).

Original: Thus, by encouraging higher PAS, DA at word-histogram level would exhibit lower negative-transfer-risk as word-prototypes are common across domains.
Revised: Thus, by encouraging higher PAS, DA at word-histogram level would exhibit lower negative-transfer-risk as word-prototypes are common across domains.

Original: Remarks.
Revised: Remarks.

Original: Consider Fig.
Revised: Consider Fig.

Original: 1 as an example.
Revised: 1 as an example.

Original: The shallower layers would extract generic shapes like rectangles, circles, lines, etc .
Revised: The shallower layers would extract generic shapes like rectangles, circles, lines, etc .

Original: while deeper layers would extract semantic shapes like windows, arms, chassis, etc.
Revised: while deeper layers would extract semantic shapes like windows, arms, chassis, etc.

Original: Intuitively, the word-histogram space at generic-shape-level cannot be sparse for the object recogition task while sparsity is desirable at deeper layers.
Revised: Intuitively, the word-histogram space at generic-shape-level cannot be sparse for the object recogition task while sparsity is desirable at deeper layers.

Original: Hence, we seek objectives that ensure word-histogram sparsity at a sufficiently high semantic level, catering to the UniDA problem.
Revised: Hence, we seek objectives that ensure word-histogram sparsity at a sufficiently high semantic level, catering to the UniDA problem.

Original: With this, the pre-classifier features better capture the class-level intrinsic structure that improves
Revised: With this, the pre-classifier features better capture the class-level intrinsic structure that improves

Original: UniDA performance.
Revised: UniDA performance.

Original: Note that a good intrinsic structure refers to a scenario where individual classes, including private classes, are well clustered in the feature space.
Revised: Note that a good intrinsic structure refers to a scenario where individual classes, including private classes, are well clustered in the feature space.

Original: Insight 2 encourages sparser word-histograms and a naive way to enforce this would be a self-entropy minimization objective on the word-histogram vectors ϕ u ( x ) ∀ u .
Revised: Insight 2 encourages sparser word-histograms and a naive way to enforce this would be a self-entropy minimization objective on the word-histogram vectors ϕ u ( x ) ∀ u .

Original: Intuitively, this objective would increase the contributions of the closest word-prototype in ϕ u ( x ) i.e . increase the PAS.
Revised: Intuitively, this objective would increase the contributions of the closest word-prototype in ϕ u ( x ) i.e . increase the PAS.

Original: Formally,
Revised: Formally,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: ]; where L uem
Revised: ]; where L uem

Original: [Equation]
Revised: [Equation]

Original: However, the self-entropy objective is susceptible to word-level misalignment due to a lack of constraints.
Revised: However, the self-entropy objective is susceptible to word-level misalignment due to a lack of constraints.

Original: For example, different classes may be mapped to the same word-prototype, or different same-class samples may be mapped to distinct word-prototypes.
Revised: For example, different classes may be mapped to the same word-prototype, or different same-class samples may be mapped to distinct word-prototypes.

Original: As the target domain is unsupervised with unseen private classes and the word-prototype concepts are abstract, it is very difficult to develop explicit constraints.
Revised: As the target domain is unsupervised with unseen private classes and the word-prototype concepts are abstract, it is very difficult to develop explicit constraints.

Original: Thus, we look for implicit constraints via self-supervision to enforce Insight 2.
Revised: Thus, we look for implicit constraints via self-supervision to enforce Insight 2.

Original: 3.5 BoW-based pretext task
Revised: 3.5 BoW-based pretext task

Original: We seek a word-related prior or property that can be used to encourage prototype-alignment and better word-prototypes V through a self-supervised pretext task.
Revised: We seek a word-related prior or property that can be used to encourage prototype-alignment and better word-prototypes V through a self-supervised pretext task.

Original: We hypothesize that different crops from the same image would yield similar word-histograms (Fig. 4A) assuming that all crops are extracted with some part of the object inside the crop.
Revised: We hypothesize that different crops from the same image would yield similar word-histograms (Fig. 4A) assuming that all crops are extracted with some part of the object inside the crop.

Original: This is reasonable as long as partially out-of-frame crops are not used.
Revised: This is reasonable as long as partially out-of-frame crops are not used.

Original: Based on this, we provide the following insight to formulate our pretext task.
Revised: Based on this, we provide the following insight to formulate our pretext task.

Original: Insight 3.
Revised: Insight 3.

Original: (Word-histogram self-entropy prior in grid-shuffled crops) Consider a grid-shuffled image where crops from y ins distinct instances are assembled (Fig. 4B).
Revised: (Word-histogram self-entropy prior in grid-shuffled crops) Consider a grid-shuffled image where crops from y ins distinct instances are assembled (Fig. 4B).

Original: The word-histogram selfentropy of the grid-shuffled image, say H e
Revised: The word-histogram selfentropy of the grid-shuffled image, say H e

Original: ( y ins ) , would increase with the number of distinct instances y ins involved in the construction of the grid-shuffled image, assuming a constant grid size N gs .
Revised: ( y ins ) , would increase with the number of distinct instances y ins involved in the construction of the grid-shuffled image, assuming a constant grid size N gs .

Original: [Equation]
Revised: [Equation]

Original: Remarks.
Revised: Remarks.

Original: Intuitively, combining crops from more number of distinct instances would increase the contributions of distinct word-prototypes (Fig. 4B), thereby increasing word-histogram entropy.
Revised: Intuitively, combining crops from more number of distinct instances would increase the contributions of distinct word-prototypes (Fig. 4B), thereby increasing word-histogram entropy.

Original: This behavior would be consistent when distinct instances also come from distinct task categories, as it ensures minimal overlap of word-prototypes among instances.
Revised: This behavior would be consistent when distinct instances also come from distinct task categories, as it ensures minimal overlap of word-prototypes among instances.

Original: The assumption of distinct categories for y ins instances is reasonable when goal task has a large number of categories, which usually holds in common DA benchmarks for object recognition.
Revised: The assumption of distinct categories for y ins instances is reasonable when goal task has a large number of categories, which usually holds in common DA benchmarks for object recognition.

Original: On the other hand, crops from the same image would only affect a small set of the same word-prototypes and word-histogram entropy can only be marginally affected.
Revised: On the other hand, crops from the same image would only affect a small set of the same word-prototypes and word-histogram entropy can only be marginally affected.

Original: While the actual word-histograms may vary with the instances used, the entropy
Revised: While the actual word-histograms may vary with the instances used, the entropy

Original: H e ( y ins ) for a given number of instances y ins would lie in a small range.
Revised: H e ( y ins ) for a given number of instances y ins would lie in a small range.

Original: The pretext task also encourages better prototype-alignment i.e . higher PAS.
Revised: The pretext task also encourages better prototype-alignment i.e . higher PAS.

Original: This is because separability of different pretext classes would improve with sparser word-histograms.
Revised: This is because separability of different pretext classes would improve with sparser word-histograms.

Original: Concretely, crops from multiple instances (in grid-shuffling) would each have a few distinct word-prototypes with significant contributions and Insight 3 would be better supported.
Revised: Concretely, crops from multiple instances (in grid-shuffling) would each have a few distinct word-prototypes with significant contributions and Insight 3 would be better supported.

Original: Then, the pretext classes (Fig. 4C) can be easily separated by word-histogram entropy.
Revised: Then, the pretext classes (Fig. 4C) can be easily separated by word-histogram entropy.

Original: Note that we use a learnable classifier since these intuitions may not hold at the start of training but pretext objectives can still guide the training.
Revised: Note that we use a learnable classifier since these intuitions may not hold at the start of training but pretext objectives can still guide the training.

Original: The pretext task helps avoid word-level misalignment, i.e . cases where different classes are aligned with the same word-prototype or different samples of a class are aligned to distinct word-prototypes.
Revised: The pretext task helps avoid word-level misalignment, i.e . cases where different classes are aligned with the same word-prototype or different samples of a class are aligned to distinct word-prototypes.

Original: Consider Fig.
Revised: Consider Fig.

Original: 4A with the worst-case of misalignment where all classes (class-1, class-2, class-3) are represented by the same word-histogram.
Revised: 4A with the worst-case of misalignment where all classes (class-1, class-2, class-3) are represented by the same word-histogram.

Original: Then, in Fig.
Revised: Then, in Fig.

Original: 4B, the image-level word-histograms would be identical for any no.
Revised: 4B, the image-level word-histograms would be identical for any no.

Original: of instances used for patch-shuffling and the pretext task of identifying no. of instances would fail.
Revised: of instances used for patch-shuffling and the pretext task of identifying no. of instances would fail.

Original: The above example shows, similar to a proof by contradiction, that the pretext task cannot allow word-level misalignment as misalignment would hurt the pretext task performance.
Revised: The above example shows, similar to a proof by contradiction, that the pretext task cannot allow word-level misalignment as misalignment would hurt the pretext task performance.

Original: Based on Insight 3, we construct an entropy-bin for each number of instances y ins ∈{ 1 , 2 , . . .
Revised: Based on Insight 3, we construct an entropy-bin for each number of instances y ins ∈{ 1 , 2 , . . .

Original: , N gs } for a novel pretext task of entropy-bin classification.
Revised: , N gs } for a novel pretext task of entropy-bin classification.

Original: Concretely, a pretext classifier f n : R K →C n operates on the output of ψ (Fig. 3) and is trained to predict y ins -class of input grid-shuffled images.
Revised: Concretely, a pretext classifier f n : R K →C n operates on the output of ψ (Fig. 3) and is trained to predict y ins -class of input grid-shuffled images.

Original: Procurement of pretext-task samples.
Revised: Procurement of pretext-task samples.

Original: We illustrate this process in Fig.
Revised: We illustrate this process in Fig.

Original: 4C. The same process is followed separately for both source dataset D s and target dataset D t where goal-task category labels are not required.
Revised: 4C. The same process is followed separately for both source dataset D s and target dataset D t where goal-task category labels are not required.

Original: First, the number of distinct instances y ins is sampled from a uniform distribution
Revised: First, the number of distinct instances y ins is sampled from a uniform distribution

Original: U (1 , N gs ) and serves as the pretext-label.
Revised: U (1 , N gs ) and serves as the pretext-label.

Original: Next, a batch of y ins instances is sampled from (say) the source dataset D s .
Revised: Next, a batch of y ins instances is sampled from (say) the source dataset D s .

Original: Now, crops are sampled randomly from the y ins instances such that each instance contributes at least one crop.
Revised: Now, crops are sampled randomly from the y ins instances such that each instance contributes at least one crop.

Original: Finally, these crops are randomly assembled into the grid-shuffled image denoted by x s,n .
Revised: Finally, these crops are randomly assembled into the grid-shuffled image denoted by x s,n .

Original: The same process can be performed for the target dataset to obtain x t,n .
Revised: The same process can be performed for the target dataset to obtain x t,n .

Original: In summary, a source-pretext dataset D s,n = { ( x s,n , y ins ) : x s,n ∈ X , y ins ∈ C n } and a target-pretext dataset D t,n = { ( x t,n , y ins ) : x t,n ∈ X , y ins ∈ C n } are extracted.
Revised: In summary, a source-pretext dataset D s,n = { ( x s,n , y ins ) : x s,n ∈ X , y ins ∈ C n } and a target-pretext dataset D t,n = { ( x t,n , y ins ) : x t,n ∈ X , y ins ∈ C n } are extracted.

Original: [Figure]
Revised: [Figure]

Original: Training algorithm.
Revised: Training algorithm.

Original: We aim to demonstrate that our proposed approach is complementary to existing UniDA methods.
Revised: We aim to demonstrate that our proposed approach is complementary to existing UniDA methods.

Original: Thus, we simply add our architecture modification ( i.e . ψ in Fig. 3) and the pretext classifier head f n while keeping the other components from an existing method like [42, 27].
Revised: Thus, we simply add our architecture modification ( i.e . ψ in Fig. 3) and the pretext classifier head f n while keeping the other components from an existing method like [42, 27].

Original: Consider an existing UniDA training algorithm denoted by UniDA-Algo ( D s , D t ) that trains the backbone h and goal-classifier f g : R K
Revised: Consider an existing UniDA training algorithm denoted by UniDA-Algo ( D s , D t ) that trains the backbone h and goal-classifier f g : R K

Original: → C s .
Revised: → C s .

Original: Additionally, we introduce the pretext-task objectives
Revised: Additionally, we introduce the pretext-task objectives

Original: L s,n = L ce ( f n ◦ h ( x s,n ) , y ins ) and L t,n (defined similarly) for the source and target data respectively (Fig.
Revised: L s,n = L ce ( f n ◦ h ( x s,n ) , y ins ) and L t,n (defined similarly) for the source and target data respectively (Fig.

Original: Here, L ce denotes the standard cross-entropy loss.
Revised: Here, L ce denotes the standard cross-entropy loss.

Original: Formally, the overall objective is,
Revised: Formally, the overall objective is,

Original: [Equation]
Revised: [Equation]

Original: Here, J ( . ) represents the objective or loss function and the third term is borrowed from Eq.
Revised: Here, J ( . ) represents the objective or loss function and the third term is borrowed from Eq.

Original: We update only the backbone h and goal-classifier f g using UniDA-Algo while the backbone h along with the word-prototype layer ψ and pretext-classifier f n are updated via the pretext-task objectives.
Revised: We update only the backbone h and goal-classifier f g using UniDA-Algo while the backbone h along with the word-prototype layer ψ and pretext-classifier f n are updated via the pretext-task objectives.

Original: Intuitively, the word-prototypes are learnt only through the pretext-task as they are the implicit constraints, as discussed under Insight 2.
Revised: Intuitively, the word-prototypes are learnt only through the pretext-task as they are the implicit constraints, as discussed under Insight 2.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Inference.
Revised: Inference.

Original: We discard the pretext-classifier f n and use only the goal-classifier f g at inference time.
Revised: We discard the pretext-classifier f n and use only the goal-classifier f g at inference time.

Original: For a fair comparison, we use the same known-unknown demarcation algorithm as UniDA-Algo .
Revised: For a fair comparison, we use the same known-unknown demarcation algorithm as UniDA-Algo .

Original:  Experiments
Revised:  Experiments

Original: Dataset.
Revised: Dataset.

Original: We report results on three different benchmarks.
Revised: We report results on three different benchmarks.

Original: Office-31 [41] contains three domains:
Revised: Office-31 [41] contains three domains:

Original: DSLR (D), Amazon (A), and Webcam (W).
Revised: DSLR (D), Amazon (A), and Webcam (W).

Original: Office-Home [49] is a more difficult benchmark, withclasses and 4 domains, Artistic (Ar), Clipart (Cl), Product (Pr), and Real-World (Rw).
Revised: Office-Home [49] is a more difficult benchmark, withclasses and 4 domains, Artistic (Ar), Clipart (Cl), Product (Pr), and Real-World (Rw).

Original: DomainNet [38] is the largest DA benchmark and the most challenging due to highly diverse domains and huge class-imbalance.
Revised: DomainNet [38] is the largest DA benchmark and the most challenging due to highly diverse domains and huge class-imbalance.

Original: Following [12], we use three subsets, namely Painting (P), Real (R), and Sketch (S).
Revised: Following [12], we use three subsets, namely Painting (P), Real (R), and Sketch (S).

Original: Evaluation.
Revised: Evaluation.

Original: We report the mean results of three runs for each experiment.
Revised: We report the mean results of three runs for each experiment.

Original: Following [42, 27], target-private classes are grouped into a single unknown class for both Open-Set and UniDA.
Revised: Following [42, 27], target-private classes are grouped into a single unknown class for both Open-Set and UniDA.

Original: We report H-score metric (HOS), i.e .
Revised: We report H-score metric (HOS), i.e .

Original: the harmonic mean of accuracy of shared and target-private samples.
Revised: the harmonic mean of accuracy of shared and target-private samples.

Original: Implementation Details.
Revised: Implementation Details.

Original: We use two recent prior arts, OVANet [42] and DCC [27], separately as UniDA-Algo (Fig. 5).
Revised: We use two recent prior arts, OVANet [42] and DCC [27], separately as UniDA-Algo (Fig. 5).

Original: We initialize ResNet50 with ImageNet-pretrained weights and retain other hyperparameters from the original baseline ( UniDA-Algo ).
Revised: We initialize ResNet50 with ImageNet-pretrained weights and retain other hyperparameters from the original baseline ( UniDA-Algo ).

Original: Unless otherwise mentioned,
Revised: Unless otherwise mentioned,

Original: UniDA-Algo will be OVANet (for most analysis experiments).
Revised: UniDA-Algo will be OVANet (for most analysis experiments).

Original: We also keep the optimizers, learning rates, and schedulers for both goal task iterations and pretext task iterations same as the baseline.
Revised: We also keep the optimizers, learning rates, and schedulers for both goal task iterations and pretext task iterations same as the baseline.

Original: We follow DCC [27] for the dataset and shared-private class splits.
Revised: We follow DCC [27] for the dataset and shared-private class splits.

Original: See Suppl.
Revised: See Suppl.

Original: for complete details.
Revised: for complete details.

Original: 4.1 Comparison with prior arts
Revised: 4.1 Comparison with prior arts

Original: Open-Set DA.
Revised: Open-Set DA.

Original: The benchmark comparisons for OSDA are presented in Table 3 and 4 for OfficeHome and Office-31 benchmarks respectively.
Revised: The benchmark comparisons for OSDA are presented in Table 3 and 4 for OfficeHome and Office-31 benchmarks respectively.

Original: Our method surpasses all current methods, even those tailored for OSDA [3, 43, 28].
Revised: Our method surpasses all current methods, even those tailored for OSDA [3, 43, 28].

Original: With a 2.2% H-score gain over OVANet [42] and a 4.6% gain over
Revised: With a 2.2% H-score gain over OVANet [42] and a 4.6% gain over

Original: DCC [27], we consistently outperform all OSDA baselines for Office-Home.
Revised: DCC [27], we consistently outperform all OSDA baselines for Office-Home.

Original: Similarly, our SPA on top of DCC and OVANet improves on Office-31 by 6.4% and 1.3% over the baselines.
Revised: Similarly, our SPA on top of DCC and OVANet improves on Office-31 by 6.4% and 1.3% over the baselines.

Original: Overall, we achieve a superior balance between shared class categorization and private sample identification.
Revised: Overall, we achieve a superior balance between shared class categorization and private sample identification.

Original: UniDA.
Revised: UniDA.

Original: On the Office-31 benchmark (Table 1), the proposed approach outperforms all other methods in terms of H-score.
Revised: On the Office-31 benchmark (Table 1), the proposed approach outperforms all other methods in terms of H-score.

Original: We improve upon the earlier state-of-the-art methods, DCC [27] and OVANet [42] by 7.0% and 0.4% respectively, again demonstrating a superior balance between shared and private sample identification.
Revised: We improve upon the earlier state-of-the-art methods, DCC [27] and OVANet [42] by 7.0% and 0.4% respectively, again demonstrating a superior balance between shared and private sample identification.

Original: Office-Home (Table 2) is a more difficult benchmark, with more
Revised: Office-Home (Table 2) is a more difficult benchmark, with more

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: private classes than the shared classes (55 vs. 10).
Revised: private classes than the shared classes (55 vs. 10).

Original: Our approach exhibits a stronger capability for the separation of shared and private classes in this extreme circumstance, benefiting from our proposed pretext task.
Revised: Our approach exhibits a stronger capability for the separation of shared and private classes in this extreme circumstance, benefiting from our proposed pretext task.

Original: On the Office-Home benchmark, our method combined with DCC yields a 4.7% improvement in H-score and an additional 1.4% gain with OVANet.
Revised: On the Office-Home benchmark, our method combined with DCC yields a 4.7% improvement in H-score and an additional 1.4% gain with OVANet.

Original: On the large-scale
Revised: On the large-scale

Original: DomainNet dataset (Table 1), our strategy improves OVANet [42] by 1.5%.
Revised: DomainNet dataset (Table 1), our strategy improves OVANet [42] by 1.5%.

Original: This shows that our approach outperforms prior arts in a variety of settings, i.e . varying degrees of openness.
Revised: This shows that our approach outperforms prior arts in a variety of settings, i.e . varying degrees of openness.

Original: 4.2 Discussion
Revised: 4.2 Discussion

Original: a) Ablation study.
Revised: a) Ablation study.

Original: We perform a thorough ablation study on Office-Home for the components of our approach (Table 5).
Revised: We perform a thorough ablation study on Office-Home for the components of our approach (Table 5).

Original: First, since our architecture modification presents a small increase in computation, we demonstrate that simply using the arch.
Revised: First, since our architecture modification presents a small increase in computation, we demonstrate that simply using the arch.

Original: mod.
Revised: mod.

Original: only marginally improves the performance (0.2% over OVANet and 0.6% over DCC).
Revised: only marginally improves the performance (0.2% over OVANet and 0.6% over DCC).

Original: Further, simply adding the word-histogram self-entropy loss
Revised: Further, simply adding the word-histogram self-entropy loss

Original: L em (Eq. 6) with the arch.
Revised: L em (Eq. 6) with the arch.

Original: mod.
Revised: mod.

Original: also yields fairly low improvements over the baseline (0.4% over
Revised: also yields fairly low improvements over the baseline (0.4% over

Original: OVANet and 1.2% over DCC).
Revised: OVANet and 1.2% over DCC).

Original: Next, we assess the improvement from the proposed pretext task (but without L em ) and observe gains of 1.1% over OVANet and 3.2% over DCC.
Revised: Next, we assess the improvement from the proposed pretext task (but without L em ) and observe gains of 1.1% over OVANet and 3.2% over DCC.

Original: Finally, including L em with the pretext task gives further gains of 0.5% over OVANet and 1.4% over DCC.
Revised: Finally, including L em with the pretext task gives further gains of 0.5% over OVANet and 1.4% over DCC.

Original: Thus, the arch.
Revised: Thus, the arch.

Original: mod.
Revised: mod.

Original: and L em independently give marginal gains.
Revised: and L em independently give marginal gains.

Original: However, they give significant improvements combined with our pretext task, underlining the importance of every component.
Revised: However, they give significant improvements combined with our pretext task, underlining the importance of every component.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: b) Evaluating clustering of target-private classes.
Revised: b) Evaluating clustering of target-private classes.

Original: To further support Insight 2, i.e . private class samples are better clustered with our approach, we apply linear evaluation protocol for the target-private classes on the word-prototype features ϕ ( x ) .
Revised: To further support Insight 2, i.e . private class samples are better clustered with our approach, we apply linear evaluation protocol for the target-private classes on the word-prototype features ϕ ( x ) .

Original: Here, we use the labels of target-private classes (only for this analysis) to train a linear classifier on the frozen features from ϕ ( x ) and compute the accuracy for target-private classes (Table 6).
Revised: Here, we use the labels of target-private classes (only for this analysis) to train a linear classifier on the frozen features from ϕ ( x ) and compute the accuracy for target-private classes (Table 6).

Original: We observe a significant gain (+5.6%) over the baseline
Revised: We observe a significant gain (+5.6%) over the baseline

Original: OVANet, which indicates that the target-private classes are better clustered with SPA.
Revised: OVANet, which indicates that the target-private classes are better clustered with SPA.

Original: c) Correlation between UniDA performance, pretext task performance, PAS and NTR.
Revised: c) Correlation between UniDA performance, pretext task performance, PAS and NTR.

Original: For
Revised: For

Original: UniDA on Office-Home, we study the correlation between goal task performance (HOS), pretext task performance, negative-transfer-risk (NTR) and prototype-alignment-score (PAS) (averaged over
Revised: UniDA on Office-Home, we study the correlation between goal task performance (HOS), pretext task performance, negative-transfer-risk (NTR) and prototype-alignment-score (PAS) (averaged over

Original: D t ) in Fig.
Revised: D t ) in Fig.

Original: 6C. We observe that goal task performance is positively correlated with both pretext task
Revised: 6C. We observe that goal task performance is positively correlated with both pretext task

Original: [Figure]
Revised: [Figure]

Original: B. Effect of varying grid
Revised: B. Effect of varying grid

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: C. Correlation of performance w/ NTR & PAS
Revised: C. Correlation of performance w/ NTR & PAS

Original: [Figure]
Revised: [Figure]

Original: P r e t e x t
Revised: P r e t e x t

Original: T a sk
Revised: T a sk

Original: [Figure]
Revised: [Figure]

Original: N
Revised: N

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: performance and PAS, as in Insight 2.
Revised: performance and PAS, as in Insight 2.

Original: Further, goal task performance is negatively correlated with
Revised: Further, goal task performance is negatively correlated with

Original: NTR which supports Insight 1.
Revised: NTR which supports Insight 1.

Original: Further, compared to the baseline OVANet [42] (dashed curves), our OVANet+SPA (solid curves) achieves better and faster convergence ( ∼ 100 vs. ∼ 300 iterations).
Revised: Further, compared to the baseline OVANet [42] (dashed curves), our OVANet+SPA (solid curves) achieves better and faster convergence ( ∼ 100 vs. ∼ 300 iterations).

Original: Further, we also observe that PAS does not change much when our pretext task is not employed.
Revised: Further, we also observe that PAS does not change much when our pretext task is not employed.

Original: [Table]
Revised: [Table]

Original: + Ours 65.
Revised: + Ours 65.

Original: c)
Revised: c)

Original: Comparison with other pretext tasks.
Revised: Comparison with other pretext tasks.

Original: We compare the effectiveness of our word-related pretext task, for Open-Set DA and Universal DA on Office-Home, with existing pretext tasks in Table 7.
Revised: We compare the effectiveness of our word-related pretext task, for Open-Set DA and Universal DA on Office-Home, with existing pretext tasks in Table 7.

Original: Note that we include our EM loss and architecture modification with each of the pretext tasks for a fair comparison.
Revised: Note that we include our EM loss and architecture modification with each of the pretext tasks for a fair comparison.

Original: First, we compare with dense-output based pretext tasks like colorization [59, 26] and inpainting [37].
Revised: First, we compare with dense-output based pretext tasks like colorization [59, 26] and inpainting [37].

Original: We observe marginal or no improvements with these tasks because the output spaces of these tasks are usually domain-dependent ( e.g . inpainting of rainy scenes must consider rain, a domain-specific factor), which hinders the adaptation.
Revised: We observe marginal or no improvements with these tasks because the output spaces of these tasks are usually domain-dependent ( e.g . inpainting of rainy scenes must consider rain, a domain-specific factor), which hinders the adaptation.

Original: Next, we consider non-dense classification tasks like rotation [31, 19], jigsaw [7] and patchlocation [48].
Revised: Next, we consider non-dense classification tasks like rotation [31, 19], jigsaw [7] and patchlocation [48].

Original: While these tasks achieve some marginal gains, our proposed pretext task outperforms them as it effectively aids UniDA and Open Set DA by encouraging prototype-alignment (Insight 2).
Revised: While these tasks achieve some marginal gains, our proposed pretext task outperforms them as it effectively aids UniDA and Open Set DA by encouraging prototype-alignment (Insight 2).

Original: d) Effect of varying degree of openness.
Revised: d) Effect of varying degree of openness.

Original: Fig.
Revised: Fig.

Original: 6A studies the effect of altering the degree of openness for UniDA in Office-Home.
Revised: 6A studies the effect of altering the degree of openness for UniDA in Office-Home.

Original: Following [42], we report the average results over five scenarios to cover a variety of domains while varying the amount of unknown classes.
Revised: Following [42], we report the average results over five scenarios to cover a variety of domains while varying the amount of unknown classes.

Original: Our approach exhibits a low sensitivity to the degree of openness and consistently outperforms other baselines.
Revised: Our approach exhibits a low sensitivity to the degree of openness and consistently outperforms other baselines.

Original: e) Effect of grid-size N gs used in grid-shuffling.
Revised: e) Effect of grid-size N gs used in grid-shuffling.

Original: For UniDA on Office-Home, we report a sensitivity analysis for grid-size N gs (Fig. 6B) which controls the number of entropy-bins i.e . number of pretext task classes.
Revised: For UniDA on Office-Home, we report a sensitivity analysis for grid-size N gs (Fig. 6B) which controls the number of entropy-bins i.e . number of pretext task classes.

Original: We observe a marginal decrease in the performance as the grid-size increases, but it outperforms the baseline (OVANet) across a wide range of grid-sizes (2x2 to 6x6).
Revised: We observe a marginal decrease in the performance as the grid-size increases, but it outperforms the baseline (OVANet) across a wide range of grid-sizes (2x2 to 6x6).

Original: Mitsuzumi et al.
Revised: Mitsuzumi et al.

Original: [32] show that, as grid-size increases beyond 4x4, shuffling the grid patches significantly alters domain information.
Revised: [32] show that, as grid-size increases beyond 4x4, shuffling the grid patches significantly alters domain information.

Original: Thus, beyond 4x4, performance drops as more domains are introduced which make adaptation difficult.
Revised: Thus, beyond 4x4, performance drops as more domains are introduced which make adaptation difficult.

Original: Another reason is that the pretext task itself becomes more difficult as the grid-size is increased.
Revised: Another reason is that the pretext task itself becomes more difficult as the grid-size is increased.

Original: Based on this, we choose grid-size 2x2 ( N gs =4) for our experiments.
Revised: Based on this, we choose grid-size 2x2 ( N gs =4) for our experiments.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we address the problem of Universal DA via closed-set Subsidiary Prototype-space
Revised: In this work, we address the problem of Universal DA via closed-set Subsidiary Prototype-space

Original: Alignment (SPA).
Revised: Alignment (SPA).

Original: First, we uncover a tradeoff between negative-transfer-risk and domain-invariance at different layers of a deep network.
Revised: First, we uncover a tradeoff between negative-transfer-risk and domain-invariance at different layers of a deep network.

Original: While a balance can be struck at a mid-level layer, we draw motivation from Bag-of-visual-Words (BoW) to introduce explicit word-prototypes followed by word-histogram based classification.
Revised: While a balance can be struck at a mid-level layer, we draw motivation from Bag-of-visual-Words (BoW) to introduce explicit word-prototypes followed by word-histogram based classification.

Original: We realize the closed-set SPA through a novel word-histogram based pretext task operating in parallel with UniDA objectives.
Revised: We realize the closed-set SPA through a novel word-histogram based pretext task operating in parallel with UniDA objectives.

Original: Building on top of existing UniDA works, we achieve state-of-the-art results on three UniDA and Open-Set DA benchmarks.
Revised: Building on top of existing UniDA works, we achieve state-of-the-art results on three UniDA and Open-Set DA benchmarks.

Original: Acknowledgements.
Revised: Acknowledgements.

Original: This work was supported by MeitY (Ministry of Electronics and Information
Revised: This work was supported by MeitY (Ministry of Electronics and Information

Original: Technology) project (No. 4(16)2019-ITEA), Govt. of India.
Revised: Technology) project (No. 4(16)2019-ITEA), Govt. of India.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Appendix B (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] See Appendix B (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Appendix C (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] See Appendix C (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments...
Revised: If you ran experiments...

Original: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] It will be available at https://sites.google.com/view/spa-unida .
Revised: [Yes] It will be available at https://sites.google.com/view/spa-unida .

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Sec. 4 and Appendix D. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: [Yes] See Sec. 4 and Appendix D. (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] See Appendix E (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [Yes] See Appendix E (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Appendix D. 4.
Revised: [Yes] See Appendix D. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] See Appendix D. (b) Did you mention the license of the assets?
Revised: [Yes] See Appendix D. (b) Did you mention the license of the assets?

Original: [Yes] See Appendix D. (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] See Appendix D. (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether
Revised: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] We used publicly available standard object recognition datasets that do not contain any personally identifiable information (PII).
Revised: [N/A] We used publicly available standard object recognition datasets that do not contain any personally identifiable information (PII).

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [N/A] See answer to 4d.
Revised: [N/A] See answer to 4d.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: Appendix
Revised: Appendix

Original: In this appendix, we provide more details of our approach, extensive implementation details, additional analyses, limitations and potential negative societal impact.
Revised: In this appendix, we provide more details of our approach, extensive implementation details, additional analyses, limitations and potential negative societal impact.

Original: Towards reproducible resesarch, we will publicly release our complete codebase and trained network weights.
Revised: Towards reproducible resesarch, we will publicly release our complete codebase and trained network weights.

Original: This supplementary is organized as follows:
Revised: This supplementary is organized as follows:

Original: • Section A: Notations (Table 8) • Section B: Limitations • Section C: Potential societal impact • Section D: Implementation details ◦ Baseline details ◦ Compute requirements ◦ Miscellaneous details (Fig. 7) • Section E: Analysis (Table 9, 10)
Revised: • Section A: Notations (Table 8) • Section B: Limitations • Section C: Potential societal impact • Section D: Implementation details ◦ Baseline details ◦ Compute requirements ◦ Miscellaneous details (Fig. 7) • Section E: Analysis (Table 9, 10)

Original: A Notations
Revised: A Notations

Original: We summarize the notations used throughout the paper in Table 8.
Revised: We summarize the notations used throughout the paper in Table 8.

Original: The notations are listed undergroups i.e . models, datasets, samples, spaces and measures.
Revised: The notations are listed undergroups i.e . models, datasets, samples, spaces and measures.

Original: B Limitations
Revised: B Limitations

Original: The proposed approach may be unsuitable for datasets with very less number of classes.
Revised: The proposed approach may be unsuitable for datasets with very less number of classes.

Original: When number of classes are low, our Insight 3 may not hold, making the pretext task very difficult to learn.
Revised: When number of classes are low, our Insight 3 may not hold, making the pretext task very difficult to learn.

Original: This may negatively impact the goal task performance as the backbone is shared between the two tasks.
Revised: This may negatively impact the goal task performance as the backbone is shared between the two tasks.

Original: While this is a limitation of the proposed implementation, a possible solution could be to
Revised: While this is a limitation of the proposed implementation, a possible solution could be to

Original: [Table]
Revised: [Table]

Original: merge some entropy-bins through clustering techniques to form more discriminative pretext classes.
Revised: merge some entropy-bins through clustering techniques to form more discriminative pretext classes.

Original: This limitation can also arise in case of large class-imbalance in the data, as this would also lead to overlapping entropy-bins, and a similar bin-merging solution may be used
Revised: This limitation can also arise in case of large class-imbalance in the data, as this would also lead to overlapping entropy-bins, and a similar bin-merging solution may be used

Original: [Table]
Revised: [Table]

Original: C Potential societal impact
Revised: C Potential societal impact

Original: Our findings may be used to train deep neural networks with minimal supervision by transferring knowledge from supplementary datasets.
Revised: Our findings may be used to train deep neural networks with minimal supervision by transferring knowledge from supplementary datasets.

Original: On many datasets with a large amount of annotated data, such as ImageNet, modern deep networks surpass humans [17].
Revised: On many datasets with a large amount of annotated data, such as ImageNet, modern deep networks surpass humans [17].

Original: In many cases where such large-scale related datasets are accessible, our proposed approach can be a proxy to supervision in the target data.
Revised: In many cases where such large-scale related datasets are accessible, our proposed approach can be a proxy to supervision in the target data.

Original: Our approach has a favorable impact as it can reduce the data collection effort for data-intensive applications.
Revised: Our approach has a favorable impact as it can reduce the data collection effort for data-intensive applications.

Original: This might make technology more accessible to organizations and individuals with limited resources.
Revised: This might make technology more accessible to organizations and individuals with limited resources.

Original: It can also aid applications where data is protected by privacy regulations and hence difficult to collect.
Revised: It can also aid applications where data is protected by privacy regulations and hence difficult to collect.

Original: The negative consequences might include making these systems more available to organizations or individuals who try to utilize them for illegal purposes.
Revised: The negative consequences might include making these systems more available to organizations or individuals who try to utilize them for illegal purposes.

Original: Our system is also vulnerable to adversarial attacks and lacks interpretability, as do all contemporary deep learning systems.
Revised: Our system is also vulnerable to adversarial attacks and lacks interpretability, as do all contemporary deep learning systems.

Original: While we demonstrate increased performance compared to the state-of-the-art, negative transfer is still possible in extreme cases of domain-shift or category-shift.
Revised: While we demonstrate increased performance compared to the state-of-the-art, negative transfer is still possible in extreme cases of domain-shift or category-shift.

Original: Thus, our technique should not be employed in critical applications or to make significant decisions without human supervision.
Revised: Thus, our technique should not be employed in critical applications or to make significant decisions without human supervision.

Original: D Implementation details
Revised: D Implementation details

Original: Here, we describe the implementation details excluded from the main paper due to the page limit.
Revised: Here, we describe the implementation details excluded from the main paper due to the page limit.

Original: D.1 Baseline details
Revised: D.1 Baseline details

Original: OVANet.
Revised: OVANet.

Original: Following prior works [44, 55], we use ResNet50 [18] as our backbone network, which has been pre-trained on ImageNet [40].
Revised: Following prior works [44, 55], we use ResNet50 [18] as our backbone network, which has been pre-trained on ImageNet [40].

Original: We add a new linear classification layer to replace the previous one.
Revised: We add a new linear classification layer to replace the previous one.

Original: We use inverse learning rate decay scheduling to train our models, as described in [44].
Revised: We use inverse learning rate decay scheduling to train our models, as described in [44].

Original: We set the weight for entropy minimization loss, λ =0 .
Revised: We set the weight for entropy minimization loss, λ =0 .

Original: 1 for all the settings.
Revised: 1 for all the settings.

Original: The value is calculated by
Revised: The value is calculated by

Original: [Table]
Revised: [Table]

Original: the outcome of Open-Set DA for Office-31 (Amazon to DSLR) following [44].
Revised: the outcome of Open-Set DA for Office-31 (Amazon to DSLR) following [44].

Original: For all experiments, the source and target batch size is 36.
Revised: For all experiments, the source and target batch size is 36.

Original: The starting learning rate for new layers is set to 0.01 and for backbone layers to 0.001.
Revised: The starting learning rate for new layers is set to 0.01 and for backbone layers to 0.001.

Original: Our method is implemented with PyTorch [36].
Revised: Our method is implemented with PyTorch [36].

Original: DCC.
Revised: DCC.

Original: We use ResNet50 [18] as the backbone, pretrained on ImageNet [40].
Revised: We use ResNet50 [18] as the backbone, pretrained on ImageNet [40].

Original: The classifier is made up of two linear layers, following [55, 12, 43, 5].
Revised: The classifier is made up of two linear layers, following [55, 12, 43, 5].

Original: We use Nesterov momentum SGD to optimize the model, which has a momentum of 0.9 and a weight decay of 5e-4.
Revised: We use Nesterov momentum SGD to optimize the model, which has a momentum of 0.9 and a weight decay of 5e-4.

Original: The learning rate decreases by a factor of (1 + α iN ) − β , where i and N represent current and global iteration, respectively, and we set α = 10 and β = 0 .
Revised: The learning rate decreases by a factor of (1 + α iN ) − β , where i and N represent current and global iteration, respectively, and we set α = 10 and β = 0 .

Original: We use a batch size of 36 and the initial learning rate is set as 1e-4 for
Revised: We use a batch size of 36 and the initial learning rate is set as 1e-4 for

Original: Office-31, and 1e-3 for Office-Home and DomainNet.
Revised: Office-31, and 1e-3 for Office-Home and DomainNet.

Original: We use PyTorch for implementation.
Revised: We use PyTorch for implementation.

Original: Existing code used.
Revised: Existing code used.

Original: • OVANet [42]: https://github.com/VisionLearningGroup/OVANet (MIT license) • DCC [44]: https://github.com/Solacex/Domain-Consensus-Clustering (MIT license)
Revised: • OVANet [42]: https://github.com/VisionLearningGroup/OVANet (MIT license) • DCC [44]: https://github.com/Solacex/Domain-Consensus-Clustering (MIT license)

Original: • PyTorch [36]: https://pytorch.org/ (BSD-style license)
Revised: • PyTorch [36]: https://pytorch.org/ (BSD-style license)

Original: Existing datasets used.
Revised: Existing datasets used.

Original: • DomainNet [38]: http://ai.bu.edu/M3SDA (Fair use notice)
Revised: • DomainNet [38]: http://ai.bu.edu/M3SDA (Fair use notice)

Original: • Office-Home [49]: https://www.hemanthdv.org/officeHomeDataset.html (Fair use notice)
Revised: • Office-Home [49]: https://www.hemanthdv.org/officeHomeDataset.html (Fair use notice)

Original: • Office-31 [41]: https://www.cc.gatech.edu/ ∼ judy/domainadapt (open source)
Revised: • Office-31 [41]: https://www.cc.gatech.edu/ ∼ judy/domainadapt (open source)

Original: D.2 Compute requirements
Revised: D.2 Compute requirements

Original: For our experiments, we used a local desktop machine with an Intel Core i7-6700K CPU, a single
Revised: For our experiments, we used a local desktop machine with an Intel Core i7-6700K CPU, a single

Original: Nvidia GTX 1080Ti GPU and 32GB of RAM.
Revised: Nvidia GTX 1080Ti GPU and 32GB of RAM.

Original: D.3 Miscellaneous details
Revised: D.3 Miscellaneous details

Original: Negative-Transfer-Risk (NTR).
Revised: Negative-Transfer-Risk (NTR).

Original: We introduce a negative-transfer-risk (NTR) γ NTR ( h ) for a given feature extractor h : X → Z , where Z is an intermediate feature-space.
Revised: We introduce a negative-transfer-risk (NTR) γ NTR ( h ) for a given feature extractor h : X → Z , where Z is an intermediate feature-space.

Original: First, the standard linear evaluation protocol [45] from transfer learning and self-supervised literature is applied on the feature extractor where a linear classifier f : Z → C s is trained on the feature h with the labeled source data.
Revised: First, the standard linear evaluation protocol [45] from transfer learning and self-supervised literature is applied on the feature extractor where a linear classifier f : Z → C s is trained on the feature h with the labeled source data.

Original: Next, following [44], NTR is computed as the known-unknown classification accuracy using a fixed entropy threshold ρ on the linear classifier prediction as: γ NTR ( h ) =
Revised: Next, following [44], NTR is computed as the known-unknown classification accuracy using a fixed entropy threshold ρ on the linear classifier prediction as: γ NTR ( h ) =

Original: E ( x,y unk )
Revised: E ( x,y unk )

Original: ∼D t 1 ( H t ( f ◦ h ( x ) , ρ )=
Revised: ∼D t 1 ( H t ( f ◦ h ( x ) , ρ )=

Original: y unk )
Revised: y unk )

Original: where H t ( f ◦ h ( x ) , ρ )=
Revised: where H t ( f ◦ h ( x ) , ρ )=

Original: (cid:26) 1; H ( f ◦ h ( x )) >
Revised: (cid:26) 1; H ( f ◦ h ( x )) >

Original: ρ 0; otherwise (9) where f = arg min f ′ E ( x s ,y s ) ∈D s CE ( f ′ ◦
Revised: ρ 0; otherwise (9) where f = arg min f ′ E ( x s ,y s ) ∈D s CE ( f ′ ◦

Original: h
Revised: h

Original: ( x s ) , y s ) is the learned source classifier on features from h .
Revised: ( x s ) , y s ) is the learned source classifier on features from h .

Original: Here, H ( . ) computes self-entropy, ρ is a fixed entropy threshold, log( | C s | ) / 2 , where | C s | represents the number of classes, following [44].
Revised: Here, H ( . ) computes self-entropy, ρ is a fixed entropy threshold, log( | C s | ) / 2 , where | C s | represents the number of classes, following [44].

Original: CE represents the standard cross-entropy loss, and y unk represents known-unknown label (0 for known, 1 for unknown).
Revised: CE represents the standard cross-entropy loss, and y unk represents known-unknown label (0 for known, 1 for unknown).

Original: We access the known-unknown labels y unk for a subset of target data only for analysis (not for training).
Revised: We access the known-unknown labels y unk for a subset of target data only for analysis (not for training).

Original: Pretext dataset procurement.
Revised: Pretext dataset procurement.

Original: We illustrate more examples in Fig.
Revised: We illustrate more examples in Fig.

Original: 7, based on the procedure given under Insight 3 and in Fig.
Revised: 7, based on the procedure given under Insight 3 and in Fig.

Original: 4C.
Revised: 4C.

Original: S o u r c e ( l a b e l e d )
Revised: S o u r c e ( l a b e l e d )

Original: [Figure]
Revised: [Figure]

Original: S o u
Revised: S o u

Original: r c e - p r e t e x t
Revised: r c e - p r e t e x t

Original: [Figure]
Revised: [Figure]

Original: T a r g e t ( u n l a b e l e d )
Revised: T a r g e t ( u n l a b e l e d )

Original: [Figure]
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: T a r g e t - p r e t e x t
Revised: T a r g e t - p r e t e x t

Original: [Figure]
Revised: [Figure]

Original: “class
Revised: “class

Original: [Figure]
Revised: [Figure]

Original: E Analysis
Revised: E Analysis

Original: Variance across different seeds.
Revised: Variance across different seeds.

Original: We highlight the significance of our results by reporting the mean and standard deviation of OS (overall accuracy) and OS* (known classes accuracy) over 3 runs with different random seeds in Table 9 for Open-Set DA on Office-31.
Revised: We highlight the significance of our results by reporting the mean and standard deviation of OS (overall accuracy) and OS* (known classes accuracy) over 3 runs with different random seeds in Table 9 for Open-Set DA on Office-31.

Original: We observe low variance with significant performance gains over the baseline.
Revised: We observe low variance with significant performance gains over the baseline.

Original: [Table]
Revised: [Table]

Original: Table 10 provides the details of the computational overhead caused by the extra parameters added in the BoW-inspired architecture modification (Sec. 3.4).
Revised: Table 10 provides the details of the computational overhead caused by the extra parameters added in the BoW-inspired architecture modification (Sec. 3.4).

Original: While ∼ 2M additional parameters are required, there is only a marginal increase in the
Revised: While ∼ 2M additional parameters are required, there is only a marginal increase in the

Original: MACS (number of multiply-accumulate operations).
Revised: MACS (number of multiply-accumulate operations).

Original: Further, simply adding the architecturemodification only marginally improves UniDA (also shown in Table 5, Sec. 4.2a).
Revised: Further, simply adding the architecturemodification only marginally improves UniDA (also shown in Table 5, Sec. 4.2a).