Paper: Noises are Transferable - An Empirical Study on Heterogeneous Domain Adaptation (ICLR-2024)

Reviewer comment (Evaluation):
Although the authors draw some interesting results, where the noise can be transferred from the source to the target domain, I think this result is still counterintuitive. If the noise is transferable in semi-supervised heterogeneous domain adaptation, it also can be transferred in the unsupervised heterogeneous domain adaptation, it is unclear why the authors limit this strong conclusion in the semi-supervised scenarios.

Author response:
**A.** This is a good question. **Although noises are transferable, they cannot be directly applied in unsupervised scenarios**. Indeed, **we generate source noises based solely on the number of categories**. For instance, let's assume that there are 10 categories in the target domain. We sample noises from 10 distinct Gaussian distributions and treat them as samples belonging to different categories. Accordingly, **those noises are task-agnostic, and their categories have no practical meaning**. Hence, **to avoid randomness in the learning process of a specific target task, having a small number of labeled target samples is essential. Also, their category indices are used to establish connections with those of noises**. Based on our findings, we can utilize domain adaptation mechanisms to jointly train all samples, including noises, and labeled and unlabeled target samples, to enhance target performance.

Reviewer comment (Experiments):
According to Section 5, the authors claim that the label information of the source sample might not be the primary factor that influences the performance of SHDA. Since the authors conduct the experiments with some large models like JMEA. It employs the ResNet-50, a very deep pre-trained network, which might contain some label information. Therefore, it cannot sufficiently reflect that the label information is useless. It is suggested that the authors should employ a lighter neural architecture like AlexNet to evaluate the proposed idea.

Author response:
**A.** We feel that there may be a misunderstanding about the baselines. In the SHDA problem, mainstream deep transformation approaches, including TNT, STN, SSAN, and JMEA, typically follow a two-step process. They first pre-process source and target samples by extracting heterogeneous features, e.g., $S_{800}$ and $D_{4096}$ (please refer to the Experimental Setup section in our manuscript). Then, to perform heterogeneous domain adaptation, **they train lightweight neural network architectures from scratch**. Accordingly, **JMEA does not contain any label information**. Also, we have used lightweight neural network architectures for experiments. Thus, based on the experimental results, we can infer that the label information of the source sample is not the primary factor influencing the performance of SHDA.

Reviewer comment (Experiments):
Since the authors evaluate the performance in the semi-supervised scenario, it is suggested that the authors should provide the experiment results of training with labeled target data. Because I doubt that some target-labeled data with a reasonable learning rate (to avoid overfitting) might be enough for the ideal performance.

Author response:
**A.** **We have reported the performance of supervised learning baselines, i.e., SVMt and NNt**. Specifically, SVMt and NNt only utilize the labeled target samples to train a support vector machine and a neural network, respectively (please refer to the Experimental Setup section in our manuscript). Also, in our manuscript, we can clearly see from Table 1 that, most SHDA methods are far better than SVMt and NNt. Specifically, on the task of A ($S_{800}$) $\rightarrow$ C ($D_{4096}$), the classification accuracy of STN is 88.41\%, which substantially exceeds the best supervised method, i.e., NNt, by 7.82\%. Furthermore, **it is difficult to achieve ideal performance solely through supervised learning approaches, which can be confirmed in many SHDA studies**.

Reviewer comment (Experiments):
In the part ‘Study on feature information of source samples’, the authors use features with different dimensions to denote different information, for example, D(4096) contains more information than D(800). This might be unreasonable since features with different dimensions might contain equal information.

Author response:
**A.** In our experiments, **not only do the dimensions differ, but the meaning of each dimension is also distinct**. Specifically, we follow the stand SHDA settings and adopt two different features, i.e., $S_{800}$ and $D_{4096}$, to represent images (please refer to the Experimental Setup section in our manuscript). They are completely different feature representations. Also, **we do not use the $D_{800}$ features to represent images**.

Reviewer comment (Theory):
Could you clarify the distinction between Semi-supervised Heterogeneous Domain Adaptation (SHDA) and Semi-supervised Domain Adaptation (SSDA) according to Definition 1? Both seem to align with Definition 1. It appears that $d_s$ and $d_t$ merely represent specific data dimensions and may not capture the heterogeneity in the nature or type of features between the source and target domains.

Author response:
**A.** Thanks for your suggestion, **we will revise Definition 1 by introducing the source and target feature spaces, i.e., $\mathcal{X}_s$ and $\mathcal{X}_t$. Under the SHDA setting, $\mathcal{X}_s \neq \mathcal{X}_t$, while in the SSDA problem, $\mathcal{X}_s = \mathcal{X}_t$**.

Reviewer comment (Experiments):
What was the rationale behind using fixed parameter settings for different SHDA tasks on the same dataset? Doesn't this approach risk not capturing the optimal performance for each task?

Author response:
**A.** Indeed, as suggested in the original papers of those baselines, they utilize the same parameter setting across different SHDA tasks on the same dataset. Consequently, adopting such settings could ensure optimal performance to some extent. **In our experiments, we solely modify the factors related to source samples, e.g., the number of source samples, to construct distinct SHDA tasks**. Thus, **to ensure a fair comparison, we adhere to the control variable principle, and maintain a consistent parameter setting for each baseline across different tasks on the same dataset**.

Reviewer comment (Experiments):
In the section of analysis on the transferable discriminability of source samples, when using $g_t(\cdot)$ as a single layer fully connected networks with the Leaky ReLU, is there any potential for underfitting the data?

Author response:
**A.** Since HCN is trained using extracted features (e.g., $DeCAF_6$) rather than raw samples, even with the adoption of lightweight networks, under-fitting is unlikely to occur.

Reviewer comment (Experiments):
I think the experiment of “label information” is meaningless. There is no doubt that the order of category indices would not affect the performance since they are just symbols without any semantics. Additionally, I do not think label information can be regarded as category indices.

Author response:
We feel that there may be a misunderstanding about the study on label information of source samples. **We want to emphasize that we keep the order of category indices unchanged for target samples and only modify them for source samples** (please refer to Figure 1 in our manuscript). As an example, **let's assume that there are three categories, i.e., cat, dog, and horse, and their ground-truth category indices are 1, 2, and 3, respectively**. **In the target domain, we keep the ground-truth category indices unchanged**. However, in the source domain, we randomly change the category index of samples belonging to the same category to that of another category. For instance, **we change the category indices of cat, dog, and horse to 2, 3, and 1, respectively**. As a result, **there is no one-to-one correspondence between the categories of the source and target domains**. Intuitively, this can affect target performance, especially in homogeneous scenarios where the source and target samples share the same feature extractor. In the SHDA, However, based on our experimental results, we observe that as the orders of category indices for source samples change, the accuracies of all methods remain almost unchanged. **This is a surprising and crucial finding**. Building upon this, we perform subsequent cross-dataset experiments to explore the impact of feature information on SHDA performance. **In those experiments, the categories of the source and target samples are completely distinct**. In addition, we can modify label information to category information in the revision.

Reviewer comment (Experiments):
In section 6, only one target domain is tested.

Author response:
Thanks for your suggestion. Indeed, **based on Table 1 and Figure 3 in our manuscript, we have noticed that in addition to the target domain of **S**, a similar phenomenon exists in other target domains**. That is, using noises or features from other datasets as source samples can enhance the corresponding target performance. Accordingly, we can infer that our conclusions are valid on distinct target domains. In addition, to empirically confirm this, we also add a target domain, i.e., **C** ($D\_{4096}$), to quantitatively analyze the potential sources of transferable knowledge in the SHDA. 

Firstly, we generate 10 noise domains, i.e., **NS$\_{300}\^{10}$**, **NS$\_{400}\^{10}$**, **NS$\_{500}\^{10}$**, **NS$\_{600}\^{10}$**, **NS$\_{700}\^{10}$**, **ND$\_{100}\^{10}$**, **ND$\_{200}\^{10}$**, **ND$\_{300}\^{10}$**, **ND$\_{400}\^{10}$**, and **ND$\_{500}\^{10}$**. Here, **NS$\_{300}\^{10}$** and **NS$\_{300}$** are generated using the identical technique, yet **NS$\_{300}\^{10}$** consists of 10 categories while **NS$\_{300}$** contains six. Others are similar to this. In the following table, we find that the performance of all baselines is almost constant as the number and dimensions of source samples change.

| $\mathcal{D}\_s \\rightarrow \mathcal{D}\_t$ | $\mathcal{TR}\_s$  | SVMt  | NNt | STN | SSAN  | JMEA |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| ND$\_{100}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.57 | 79.61 | 80.59 | 86.59 | 86.22 | 86.12 |
| ND$\_{200}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.51 | 79.61 | 80.59 | 87.15 | 87.78 | 85.55 |
| ND$\_{300}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.48 | 79.61 | 80.59 | 85.24 | 87.42 | 85.73 |
| ND$\_{400}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.48 | 79.61 | 80.59 | 87.16 | 87.18 | 85.54 |
| ND$\_{500}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.46 | 79.61 | 80.59 | 86.81 | 87.09 | 85.54 |
| - | - | - | - | - | - | - |
| NS$\_{300}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.48 | 79.61 | 80.59 | 85.86 | 86.94 | 85.88 |
| NS$\_{400}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.49 | 79.61 | 80.59 | 85.35 | 86.94 | 85.96 |
| NS$\_{500}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.48 | 79.61 | 80.59 | 85.24 | 87.42 | 85.73 |
| NS$\_{600}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.48 | 79.61 | 80.59 | 85.26 | 86.84 | 86.13 |
| NS$\_{700}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 0.49 | 79.61 | 80.59 | 85.31 | 87.22 | 86.04 |

Secondly, we create eight noise domains, i.e., **NK$\_7\^{10}$**, **NK$\_8\^{10}$**, **NK$\_9\^{10}$**, **NK$\_{10}\^{10}$**, **NL$\_{0.25}\^{10}$**, **NL$\_{0.35}\^{10}$**, **NL$\_{0.45}\^{10}$** and **NL$\_{0.55}\^{10}$**. Here, the former four are generated using the **Category Replicate** strategy, whereas the latter four are constructed through the adoption of the **Category Shift** mechanism. Also, the superscript denotes the total number of categories, and the subscript denotes the corresponding parameter. In the following table, we can observe again that the original discriminability of source samples is not the dominant factor affecting the performance of SHDA. Also, **in conjunction with the above table, it is evident that transferable discriminability stands as the primary factor influencing the performance of SHDA**.

| $\mathcal{D}\_s \\rightarrow \mathcal{D}\_t$ | LDA | $\mathcal{TR}\_s$ | SVMt | NNt | STN | SSAN | JMEA |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| NK$\_7\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 29.34 | 1.34 | 79.61 | 80.59 | 60.03 | 81.15 | 74.85 |
| NK$\_8\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 31.52 | 1.02 | 79.61 | 80.59 | 69.88 | 83.17 | 78.76 |
| NK$\_9\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 32.35 | 0.71 | 79.61 | 80.59 | 78.16 | 84.70 | 83.85 |
| NK$\_{10}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | **33.6** | **0.48** | **79.61** | **80.59** | **85.24** | **87.42** | **85.73** |
| - | - | - | - | - | - | - | - |
| NL$\_{0.25}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 35.38 | 2.59 | 79.61 | 80.59 | 57.00 | 76.63 | 80.59 |
| NL$\_{0.35}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 69.35 | 2.59 | 79.61 | 80.59 | 59.02 | 76.26 | 82.80 |
| NL$\_{0.45}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 114.64 | 2.60 | 79.61 | 80.59 | 61.48 | 76.91 | 82.22 |
| NL$\_{0.55}\^{10}$ $\\rightarrow$ C ($D\_{4096}$) | 171.26 | 2.63 | 79.61 | 80.59 | 62.32 | 76.11 | 82.36 |

Reviewer comment (Experiments):
In the experiment of “original discriminability”, I do not think category replicate and category shift are appropriate. Category replicate assigns the same category label to different category samples, which would damage the training. This damage is more serious when K is larger. So, we can not know whether the discriminability or damage causes the effect. Category shift does not alter the internal distribution. Instead, I regard the Gaussian with different means and variances to be better. Specifically, larger mean and variance differences between categories represent larger discriminability. Additionally, the conclusion that the primary source of transferable knowledge in SHDA tasks does not lie in the original discriminability of source samples is not convincing since the performance improves when LDA values increase in NK3,4,5 and 6.

Author response:
Firstly, there may be a misunderstanding about the **Category Replicate** strategy. **It first replicates $C - K$ copies of all samples from a specific category and then assigns the remaining categories to them individually**. Here, $C$ is the total number of categories in the target domain, and $K$ is the initial total number of categories in a noise domain. Thus, it does not assign the same category to different category samples.

Secondly, based on our experiments, **we find that achieving higher target performance becomes remarkably easy by using different means and variances to generate noises**. Hence, it does not reflect the trend in target performance changing with the original discriminability of source samples very well. As a result, we design the **Category Replicate** strategy.

Thirdly, we adopt the **Category Shift** strategy to design two counterexamples, i.e., **NL$\_{0.40}$** and **NL$\_{0.41}$**, which empirically demonstrate that the original discriminability of source samples is not the primary source of transferable knowledge in SHDA tasks. **Typically, if we encounter several counterexamples, then we can derive the above conclusion**.

Fourthly, we can also construct other counterexamples. For instance, we create two noise domains, i.e., **NL$\_{0.50}$** and **NL$\_{1.00}$**, each with a respective value of $\lambda$ as 0.50 and 1.00. Also, their LDA values are 90.07 and 360.28, respectively. In the following table, we can clearly see that although the LDA values of **NL$\_{0.50}$** and **NL$\_{1.00}$** are much higher than that of **NK$\_6$**, the performance of SHDA baselines on the **NL$\_{0.50}$ $\\rightarrow$ S** and **NL$\_{1.00}$ $\\rightarrow$ S** tasks is lower than that on the **NK$\_6$ $\\rightarrow$ S** task. Also, the $\mathcal{TR}\_s$ values of the **NL$\_{0.50}$ $\\rightarrow$ S** and **NL$\_{1.00}$ $\\rightarrow$ S** tasks are higher than that of the **NK$\_{6}$ $\\rightarrow$ S** task. This once again implies that the primary source of transferable knowledge in SHDA tasks does not lie in the original discriminability of source samples.

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | LDA | $\mathcal{TR}\_s$ | SVMt | NNt | STN | SSAN | JMEA |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| NK$_5$ $\rightarrow$ S | 55.77 | 0.69 | 57.24 | 60.34 | 66.75 | 64.65 | 63.07 |
| NK$_6$ $\rightarrow$ S | **60.57** | **0.27** | **57.24** | **60.34** | **69.91** | **67.11** | **69.33** |
| - | - | - | - | - | - | - | - |
| NL$_{0.40}$ $\rightarrow$ S | 57.64 | 1.73 | 57.24 | 60.34 | 63.82 | 60.24 | 64.92 |
| NL$_{0.41}$ $\rightarrow$ S | 60.56 | 1.72 | 57.24 | 60.34 | 63.71 | 61.13 | 64.53 |
| NL$_{0.50}$ $\rightarrow$ S | 90.07 | 1.70 | 57.24 | 60.34 | 64.94 | 59.41 | 64.29 |
| NL$_{1.00}$ $\rightarrow$ S | 360.28 | 1.66 | 57.24 | 60.34 | 64.82 | 59.03 | 65.22 |

**In summary, we have good reason to believe that our conclusion is convincing**.

Reviewer comment (Evaluation):
Why the metrics for measuring discriminability are different in the experiments of “original discriminability” and “transferable discriminability”? Concretely, one is LDA values, and the other is empirical risk.

Author response:
We are sorry for this confusion. Indeed, all of them can be used to measure the discriminability of source samples. **However, in the experiments, we find that the LDA values of source samples in the common space are very large. For instance, the LDA value of NK$\_6$ is 3912.9**. If we adopt the LDA values, we need to introduce a trade-off parameter for balancing the importance between the values of LDA and MMD. In addition, in the source space, there is no requirement for extra classifier training to calculate the empirical risk. Also, in the common space, there is no need for additional calculation of the LDA value. Both comply with the Occam's razor principle. Thus, we adopt the empirical risk of source samples to characterize their discriminability in the common space. We will make it more clearly in the revision.

Reviewer comment (Evaluation):
In Table 2, why report the average accuracy of seven methods instead of individual results as in previous experiments? It makes me feel suspicious.

Author response:
**As aforementioned, the principles of different methods are distinct, and there may be variations in performance trends. Hence, we observe the overall performance trends of all methods as source domains change, rather than focusing on individual methods**. To this end, we report the average accuracy of seven SHDA methods. Thus, we do not repeatedly list their classification accuracies. In addition, in the revision, we will include those results in the appendix for supplementary clarification.

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | NS$\_{300}$ $\\rightarrow$ S | NS$\_{400}$ $\rightarrow$ S | NS$\_{500}$ $\rightarrow$ S | NS$\_{600}$ $\rightarrow$ S | NS$\_{700}$ $\rightarrow$ S | ND$\_{100}$ $\rightarrow$ S | ND$\_{200}$ $\rightarrow$ S | ND$\_{300}$ $\rightarrow$ S | ND$\_{400}$ $\rightarrow$ S | ND$\_{500}$ $\rightarrow$ S |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| $\mathcal{TR}\_s$ | 0.27 | 0.27 | 0.27 | 0.28 | 0.27 | 0.31 | 0.28 | 0.27 | 0.27 | 0.26 |
| HCN$\_{acc}$ | 56.10 | 55.79 | 56.07 | 55.87 | 56.00 | 55.88 | 56.30 | 56.07 | 56.24 | 55.94 |
| SVMt | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 |
| NNt | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 |
| SHFA | 63.69 | 63.65 | 63.65 | 63.66 | 63.65 | 63.66 | 63.62 | 63.65 | 63.62 | 63.66 |
| CDLS | 63.52 | 63.57 | 63.56 | 63.57 | 63.60 | 63.54 | 63.53 | 63.56 | 63.64 | 63.58 |
| DDACL | 63.63 | 63.35 | 63.92 | 63.63 | 63.22 | 63.78 | 63.42 | 63.92 | 64.06 | 63.89 |
| STN | 69.69 | 70.53 | 69.91 | 69.94 | 69.88 | 70.98 | 69.11 | 69.91 | 69.65 | 68.57 |
| SSAN | 67.49 | 67.07 | 67.11 | 67.13 | 67.15 | 66.99 | 67.37 | 67.11 | 68.80 | 67.63 |
| JMEA | 68.87 | 69.13 | 69.33 | 68.96 | 69.47 | 68.88 | 68.03 | 69.33 | 68.98 | 65.89 |

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | NK$\_3$ $\rightarrow$ S | NK$\_4$ $\rightarrow$ S | NK$\_5$ $\rightarrow$ S | NK$\_6$ $\rightarrow$ S | NL$\_{0.40}$ $\rightarrow$ S | NL$\_{0.41}$ $\rightarrow$ S |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| $\mathcal{TR}\_s$ | 1.87 | 1.22 | 0.69 | 0.27 | 1.73 | 1.72 |
| HCN$\_{acc}$ | 56.25 | 56.31 | 55.81 | 56.07 | 55.74 | 55.56 |
| SVMt | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 | 57.24 |
| NNt | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 | 60.34 |
| SHFA | 48.32 | 49.50 | 56.13 | 63.65 | 53.65 | 55.80 |
| CDLS | 61.82 | 62.95 | 63.89 | 63.56 | 59.19 | 59.20 |
| DDACL | 51.42 | 53.73 | 57.20 | 63.92 | 38.78 | 39.04 |
| STN | 58.41 | 60.95 | 66.75 | 69.91 | 63.82 | 63.71 |
| SSAN | 59.09 | 61.28 | 64.65 | 67.11 | 60.24 | 61.13 |
| JMEA | 59.27 | 62.57 | 63.07 | 69.33 | 64.92 | 64.53 |

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | NG $\rightarrow$ S | NU $\rightarrow$ S | NL $\rightarrow$ S |
| ---- | ---- | ---- | ---- |
| $\mathcal{TR}\_s$ | 0.26 | 0.55 | 0.54 |
| HCN$\_{acc}$ | 55.88 | 55.73 | 55.81 |
| SVMt | 57.24 | 57.24 | 57.24 |
| NNt | 60.34 | 60.34 | 60.34 |
| SHFA | 63.69 | 64.14 | 64.18 |
| CDLS | 63.46 | 58.69 | 58.63 |
| DDACL | 64.77 | 66.07 | 65.71 |
| STN | 69.29 | 68.71 | 69.37 |
| SSAN | 65.10 | 64.30 | 63.55 |
| JMEA | 66.79 | 68.66 | 69.15 |

Reviewer comment (Evaluation):
The authors consider the noises to be transferable and the key factor to be the transferable discriminability. However, the comparison of the transferable discriminability of true samples and noises are not given in Table 2 and 3.

Author response:
Thanks for your suggestion. In the following table, we provide a comparison of the transferable discriminability between true samples and noises. From the results, **we can clearly see that the $\mathcal{TR}\_s$ values of all noises except NK$\_3$, NK$\_4$, NK$\_5$, NL$\_{0.40}$, and NL$\_{0.41}$ are close to those of true samples**. This once again confirms the significance of the transferable discriminability. We will include those results in the appendix.

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | NS$\_{300}$ $\rightarrow$ S | NS$\_{400}$ $\rightarrow$ S | NS$\_{500}$ $\rightarrow$ S | NS$\_{600}$ $\rightarrow$ S | NS$\_{700}$ $\rightarrow$ S | ND$\_{100}$ $\rightarrow$ S | ND$\_{200}$ $\rightarrow$ S | ND$\_{300}$ $\rightarrow$ S | ND$\_{400}$ $\rightarrow$ S | ND$\_{500}$ $\rightarrow$ S |
| ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- | ---- |
| $\mathcal{TR}\_s$ | 0.27 | 0.27 | 0.27 | 0.28 | 0.27 | 0.31 | 0.28 | 0.27 | 0.27 | 0.26 |
| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | **NK3 $\rightarrow$ S** | **NK4 $\rightarrow$ S** | **NK5 $\rightarrow$ S** | **NK6 $\rightarrow$ S** | **NL$\_{0.40}$ $\rightarrow$ S** | **NL$\_{0.41}$ $\rightarrow$ S** | **E $\rightarrow$ S** | **F $\rightarrow$ S** | **G $\rightarrow$ S** | **I $\rightarrow$ S** |
| $\mathcal{TR}\_s$ | 1.87 | 1.22 | 0.69 | 0.27 | 1.73 | 1.72 | 0.25 | 0.26 | 0.27 | 0.26 |

Reviewer comment (Experiments):
The effect of the number of labeled target samples has not been studied. For semi-supervised domain adaptation, the number of labeled target samples is a crucial factor for the adaptation performance. However, this study does not cover the investigation on this aspect.

Author response:
**A.** **We want to emphasize that, in this study, we do not propose a new SHDA approach. Instead, our focus is on exploring the essence of the SHDA**, i.e., What knowledge from a heterogeneous source domain is transferred to the target domain? Accordingly, we adhere to the control variable principle. For the same target domain, we explore the above problem by keeping the number of target samples unchanged while changing the source domain. Also, based on our experience in the SHDA, we believe that, within a certain range, as the number of target samples increases, it will only enhance the performance of all baselines without affecting our current conclusions. **Hence, we do not conduct experiments involving variations in the number of labeled target samples**. 

In addition, to empirically verify this, we report the classification accuracies of some approaches w.r.t. distinct numbers of labeled target samples per category in the following table. We can clearly observe that their performance is boosted with the increase in the number of labeled target samples. This is in line with our expectation. In the revision, we will include those results in the appendix for supplementary clarification.

| $\mathcal{D}_s \rightarrow \mathcal{D}_t$ | $l$ | NNt | STN | SSAN | JMEA |
| ---- | ---- | ---- | ---- | ---- | ---- |
| N$_6$ $\rightarrow$ S | 5 | 60.34 | 69.91 | 67.11 | 69.33 |
| N$_6$ $\rightarrow$ S | 10 | 68.02 | 72.57 | 72.86 | 73.04 |
| N$_6$ $\rightarrow$ S | 15 | 73.02 | 76.44 | 76.05 | 76.34 |
| N$_6$ $\rightarrow$ S | 20 | 74.57 | 78.47 | 77.55 | 78.66 |

Reviewer comment (Novelty):
The value of this study is kind of limited. Semi-supervised heterogeneous domain adaptation is a very small field (as can be seen from the literature) and a study on such a small field can only draw attention from a small group of audience. While this is valuable, the value is not very significant. We expect a study paper accepted at this conference to be of high scope so that a broad group of people can learn something from the study.

Author response:
**A.** **Thank you for acknowledging the value of our research. However, we respectfully and politely disagree that the value of research should be determined by the size of the field**. Please refer to the general response for details.