Paper: Online Feature Updates Improve Online (Generalized) Label Shift Adaptation (ICLR-2024)

Reviewer comment (Theory):
Although the application of SSL is reasonable, the authors do not provide theoretical guarantees on its performance of learning the implicit feature mapping.

Author response:
**Theoretical guarantee of implicit feature mapping**: Theoretical analysis on domain adaptation/feature learning is generally hard in the context of deep learning. Instead, we varied the severity of data shifts in the experiment and studied how OLS-OFU is influenced. Figure 2(b) shows the results under a mild severity of data shifts. Because the feature extractor is refined for the shift distribution and the violation of the label shift assumption is alleviated, OLS-OFU still benefits from the OLS module if we compare OLS-OFU with OFU. However, when the data is shifted with a higher level of severity, after the same feature refining, the violation of the label shift assumption cannot be controlled under a certain level so the OLS-OFU should not benefit from the OLS module. This is validated by the empirical results: as shown in Figure 3, OLS-OFU performs worse than the single OFU without any OLS module.

Reviewer comment (Novelty):
As discussed in the above weaknesses, the authors should explain the novelty of their method and theoretical results. Moreover, the authors should also discuss the theoretical guarantees of SSL.

Author response:
**Novelty and contributions**: Please see our general reply “Clarifications on Contribution”.

**Theoretical guarantee of implicit feature mapping**: Theoretical analysis on domain adaptation/feature learning is generally hard in the context of deep learning. Instead, we varied the severity of data shifts in the experiment and studied how OLS-OFU is influenced. Figure 2(b) shows the results under a mild severity of data shifts. Because the feature extractor is refined for the shift distribution and the violation of the label shift assumption is alleviated, OLS-OFU still benefits from the OLS module if we compare OLS-OFU with OFU. However, when the data is shifted with a higher level of severity, after the same feature refining, the violation of the label shift assumption cannot be controlled under a certain level so the OLS-OFU should not benefit from the OLS module. This is validated by the empirical results: as shown in Figure 3, OLS-OFU performs worse than the single OFU without any OLS module.

Reviewer comment (Experiments):
Experiments are weak in the current version.

Author response:
**Results presentation in the main paper**: Thank you for the suggestions. The results on several other datasets (STL10, CINIC, EuroSAT) are presented in Figure 5 and they actually show the same improvement pattern as the results for CIFAR10 (Figure 2). We agree that moving these results to the main paper will make the conclusion look more convincing. We will fix the pages in the revision.

**Ablation study**: Thank you for raising this concern. We actually have done the main ablations for our method implied by the main results (Figure 2). We explain these ablations below: 
- The ablation for the OFU module. The OFU module in OLS-OFU can be studied by comparing OLS-OFU and OLS in the main result Figure 2. We can observe consistent improvement across all OLS methods, which demonstrates the advantage of the OFU module introduced in this paper.
- The ablation for the OLS module. The OLS module in OLS-OFU can be studied by comparing OLS-OFU and OFU in the main result Figure 2. The improvement of OLS-OFU over OFU shows the superiority of the OLS module in the OLS problem. 

We will highlight these comparisons in the revision. Moreover, in the last paragraph of the experimental section, we also studied whether to "make predictions" first or "update $f_t$ to $f_{t+1}$" first as an ablation study for the overall framework of OLS algorithms (as shown in Figure 1).

Reviewer comment (Experiments):
The experimental results presented in the main paper provide limited information and lack many ablation experiments, which are crucial to supporting the conclusions of the paper. For example, how do self-supervised learning techniques improve the model performance?

Author response:
**Ablation study**: Thank you for raising this concern. We actually have done the main ablations for our method implied by the main results (Figure 2). We explain these ablations below: 
- The ablation for the OFU module. The OFU module in OLS-OFU can be studied by comparing OLS-OFU and OLS in the main result Figure 2. We can observe consistent improvement across all OLS methods, which demonstrates the advantage of the OFU module introduced in this paper.
- The ablation for the OLS module. The OLS module in OLS-OFU can be studied by comparing OLS-OFU and OFU in the main result Figure 2. The improvement of OLS-OFU over OFU shows the superiority of the OLS module in the OLS problem. 

We will highlight these comparisons in the revision. Moreover, in the last paragraph of the experimental section, we also studied whether to "make predictions" first or "update $f_t$ to $f_{t+1}$" first as an ablation study for the overall framework of OLS algorithms (as shown in Figure 1).

Reviewer comment (Theory):
The theoretical support in this work is not well verified by the empirical results. In Eq.(5) of the draft, the math showed that the online updates yield improvements. However, this point cannot be theoretically guaranteed. Furthermore, existing empirical results cannot reveal the relationship between the larger improvement and the better performance.

Author response:
**Evidence between Equation (5) and results**: Thanks for raising this point! We agree that although our algorithm is theoretically motivated by Equation (5), stronger evidence between Equation (5) and the results can provide stronger motivation. To show this, we calculate the Pearson coefficient between the improvement from Equation (5) (RHS - LHS) and the improvement from OLS to OLS-OFU (i.e. loss$^{\rm OLS}-$loss$^{\rm OLS-OFU}$) across all experimental setups (dataset, online shift pattern, SSL method).  Pearson coefficient is between [-1, 1] and if it is positive, it indicates their positive correlation. The coefficients as evaluated are 0.70, 0.76, 0.54, 0.63, 0.58, and 0.75 when OLS is set as FTFWH, FTH, ROGD, ATLAS, UOGD, and FLHFTL respectively, which all show strong positive correlations. We will include this analysis in the revision.

Reviewer comment (Theory):
About the theoretical analysis in Eq. (5). I believe that the motivation of introducing the online feature updates comes from the assumption in this inequality. And the authors tried to provide empirical evaluations in Sec. 4 (and Appendix D.6) to verify the holdness of this inequality. However, there is no further demonstrations of the quantitative relationship between the amount of feature representation refinement (or in other words the tightness of Eq. (5)) and the final performance. The reviewer reformulates the question mathematically: Suppose $\mathcal{M}$ denote the OLS methods (e.g., $\mathcal{M}\in${FLHFTL, FTH, ROGD,...}). Take $\mathcal{M}=$FLHFTL as an example. Let  
$$X^{\mathcal{M}}=\mathbb{E}[ \frac{1}{T} \Sigma_{t=1}^{T}\ell(f_{t}^{\mathcal{M}-ofu}; \mathcal{P}_{t}^{test})]$$  
$$Y=\mathbb{E}[ \frac{1}{T} \Sigma_{t=1}^{T}\ell(g(\cdot; f_{t}^{\prime\prime}, q_t/q_0); \mathcal{P}_{t}^{test})]$$  
and  
$$Z=\frac{1}{T} \Sigma_{t=1}^{T}\ell(g(\cdot; f_{0}, q_t/q_0); \mathcal{P}_{t}^{test})$$  
According to [1], Eq.(3) can be rewritten as  
$$ X^{\mathcal{M}} - Y  \leq \mathcal{O}(\frac{K^{1/6}V_{T}^{1/3}}{\sigma^{2/3}T^{1/3}} + \frac{K}{\sigma \sqrt{T}})$$  
If we define $\Delta = Z- Y$ as **the amount of improvement** ($\Delta \geq 0$ if we admit Eq. (5)), then introducing feature update can make the original bound in Eq. (4) tighter by $\Delta$:  
$$X^{\mathcal{M}} - Z \leq \mathcal{O}(\frac{K^{1/6}V_{T}^{1/3}}{\sigma^{2/3}T^{1/3}} + \frac{K}{\sigma \sqrt{T}}) - \Delta$$  
The larger improvement we make by feature update (in other words larger $\Delta$), the tighter bound we can derive from the original version. Thus, the question is: **could you please verify this point with empirical results to support your motivation in Eq. (5) in a quantitative manner?** Maybe we can fix an OLS method $\mathcal{M}$, and then choose different ways to obtain $f_{t}^{\prime\prime}$ (e.g., different $\ell_{ssl}$), then give a quantitative measure on the loss of $\mathcal{M}$-OFU (i.e., $X^{\mathcal{M}}$ defined above) with respect to the value of $\Delta$?

Author response:
**Evidence between Equation (5) and results**: Thanks for raising this point! We agree that although our algorithm is theoretically motivated by Equation (5), stronger evidence between Equation (5) and the results can provide stronger motivation. To show this, we calculate the Pearson coefficient between the improvement from Equation (5) (RHS - LHS) and the improvement from OLS to OLS-OFU (i.e. loss$^{\rm OLS}-$loss$^{\rm OLS-OFU}$) across all experimental setups (dataset, online shift pattern, SSL method).  Pearson coefficient is between [-1, 1] and if it is positive, it indicates their positive correlation. The coefficients as evaluated are 0.70, 0.76, 0.54, 0.63, 0.58, and 0.75 when OLS is set as FTFWH, FTH, ROGD, ATLAS, UOGD, and FLHFTL respectively, which all show strong positive correlations. We will include this analysis in the revision.

Reviewer comment (Presentation):
It seems the organization of the descriptions in Algorithm 1&2 is a little confusing and make the core steps of your methods less readable. For example, if we start Algorithm 1 from $t=1$, the step-1 of Algorithm 1 aims to return $f_{2}^{\prime} \leftarrow \textit{OLS-R}$. However, in the details of this step shown in Algorithm 2, it seems we need $f_{1}^{\prime\prime}$ returned by the step-3 of Algorithm of the previous loop. I failed to get what is the exact definition of $f_{1}^{\prime\prime}$.

Author response:
**Definition of $f_1''$**: Thank you for raising this clarification question. Both $f_1$ and $f_1''$ are defined as $f_0$ initially. We will address this in the revision.

Reviewer comment (Presentation):
The experimental results are presented in the form of graphs and charts. It might be beneficial to include some quantitative tables for better clarity and to facilitate comparisons by others.

Author response:
**Quantitative Results**: Thank you for the suggestions. We will include tables in the appendix for completeness in the revision.

Reviewer comment (Experiments):
The experimental dataset is relatively small. Conducting experiments on a larger dataset, such as ImageNet-C, would be more convincing.

Author response:
**Experiments on the Imagenet and Imagenet-C**: Current OLS algorithms might not work very well when the number of classes is large (e.g. 1000 in Imagenet), because it is hard to estimate the confusion matrix (e.g. 1000 by 1000 matrix on Imagenet) but the confusion matrix estimation is a very crucial step in all OLS algorithms. We will include this limitation discussion in the revision.

Reviewer comment (Theory):
Can the online generalized label shift scenario be simplified as a combination of domain shift and label shift?

Author response:
**Generalized label shift**: Yes, the definition of online generalized label shift implies it is a combination of domain shift and label shift, while domain shift is specified in this form: the two distributions conditioned on any label class can be aligned by the same mapping function $h$.

Reviewer comment (Experiments):
The selected self-supervised learning methods often require an additional branch, and their computational cost is not insignificant. Have you explored alternative lightweight methods for implementation, such as BN adaptation or entropy minimization?

Author response:
**Lightweight SSL methods**: Besides MoCo, we also experimented with other two lightweight SSL methods: entropy minimization and rotation degree prediction. From Figure 2, 6 and 7, we can observe that both entropy minimization and rotation degree prediction provides very consistent improvement and MoCo provides the most significant improvement overall among all three SSL methods.