Paper: Towards Efficient Trace Estimation for Optimal Transport in Domain Adaptation (ICLR-2024)

Reviewer comment (Novelty):
The paper's contribution is limited. It's well established that Hutchinson's trace estimator reduces computational complexity, regardless of its downstream applications. Thus, it's certainly expected that adopting Hutchinson's will improve OTDA by Courty et al.

Author response:
Our primary contribution involves applying Hutchinson's trace estimator to OT-Laplace in domain adaptation. This trace optimization technique is particularly beneficial for OT-Laplace due to the Laplacian regularization term being the trace of a series of large matrix products. Our novel application addresses the critical challenge of balancing computational efficiency with accuracy in domain adaptation. These improvements are significant in scenarios where rapid processing or limited computational resources are critical factors.

Reviewer comment (Experiments):
The rest of the paper is focused on finding the best variants of Hutchinson's and optimal combinations of hyper-parameters that achieve a good balance between accuracy and time.

Author response:
Our primary contribution involves applying Hutchinson's trace estimator to OT-Laplace in domain adaptation. This trace optimization technique is particularly beneficial for OT-Laplace due to the Laplacian regularization term being the trace of a series of large matrix products. Our novel application addresses the critical challenge of balancing computational efficiency with accuracy in domain adaptation. These improvements are significant in scenarios where rapid processing or limited computational resources are critical factors.

Reviewer comment (Novelty):
What is the motivation of the work? Since DeepJDOT and other recent OT works have shown better ways of injecting OT techniques into domain adaptation algorithms, marginally improving OTDA's performance as the authors did in this paper doesn't seem to have a meaningful impact.

Author response:
The paper *Joint Distribution Optimal Transportation for Domain Adaptation* (JDOTDA) reports JDOT's accuracy as 91.54% versus OT-IT's 89.98% in the Caltech->Amazon task (as shown in Table 1 of JDOTDA). In contrast, the paper *Optimal Transport for Domain Adaptation* (OTDA) reports OT-IT at 37.75% and OT-Laplace at 38.96% in the same task (as shown in Table 3 of OTDA). These discrepancies in accuracy indicate differences in experimental setups and classification criteria. In the context of mean accuracy obtained under different groups of datasets in two papers, the paper JDOTDA reports mean accuracies for JDOT and OT-IT as 89.04% versus 86.02%, while the paper OTDA reports mean accuracies for OT-IT and OT-Laplace as 42.30% versus 43.20%. As both OT-Laplace and JDOT slightly outperform OT-IT and our modified OT-Laplace performs better than the original OT-Laplace, it is difficult to assert that JDOT is superior to our modified OT-Laplace without a direct comparison.

The paper *DeepJDOT: Deep Joint Distribution Optimal Transport for Unsupervised Domain Adaptation* demonstrates that DeepJDOT significantly improves performance in certain tasks. However, although experiments have not yet been conducted, we hypothesize that the computation time for our modified OT-Laplace will be significantly less than DeepJDOT as it requires learning a CNN layer. However, it is possible that our OT-Laplace might generally be less accurate. Additionally, the advantages of DeepJDOT may be limited to image recognition tasks.

Reviewer comment (Experiments):
Is the running time in Section 5 for Laplacian optimization alone or improved OT-Laplacian?

Author response:
Regarding the last question, we compared the running times of various variants of our improved OT-Laplace with the Control Group (i.e., the original OT-Laplace) in Section 5. All running times mentioned in the paper represent the durations of the entire domain adaptation algorithm process.