Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper studies the problem of using online adaptation method in time-series prediction tasks. Existing approaches that feedback based on the latest prediction errors have the risk of forgetting past information. To address this challenge, the authors of this paper propose an approach that uses the critical data samples from the memory buffer to optimize the prediction model. The authors derive a bound of the prediction error, which gives the insight that the proposed approach will perform well if the ground-truth prediction functions are time-varying at a very slow rate. They also use experiments to show the proposed method performs better than existing approaches with and without adaptation.
strength_and_weaknesses: Strength:

This paper studies an interesting question that is impactful for the fields of online learning and online control.  The intuition behind the proposed change to existing online adaptation methods is convincing, and some intuitions have been verified in the first synthetic experiment. Specifically, I like the plot in Figure 1 (a) that shows how the performance of the proposed approach depends on the time-varying shift of the function.

Weakness:

I feel the major weakness of this work is on the theoretical part. If I understand correctly, the error bound in equation (8) is just an upper bound of the prediction error $e_{t+1}$. I don’t think one can claim feedforward is better than feedback because the $E[B_e^{ff}] \leq E[B_e^{fb}]$, because such claim can only be made when an upper bound is smaller than a lower bound. And it is also worth noticing that the proof only works for a specific special case. These two factors make me feel the theoretical guarantee is very weak.

Another weakness of this work is on the algorithm design. It has been shown both from the theory and the simulation that the proposed feedforward approach is prone to $\delta$, which characterize how fast the ground-truth prediction function changes. In other words, one needs to decide whether to use the feedforward approach based on $\delta$. However, the value of $\delta$ is not known before the online prediction process starts. Even if one can switch algorithms in the middle based on the historical experience, it is unclear how to estimate $\delta$ because the ground-truth prediction functions are unknown.

For the experiments, I’m uncertain about whether the improvement made on the real-world datasets are significant.
clarity,_quality,_novelty_and_reproducibility: This paper is well-written and easy to follow in general. However, a clarity issue in Algorithm 1 is that $s$ in line 4 is not defined. The authors might want to add a clarification that $s$ is chosen based on some rules.
summary_of_the_review: In summary, I feel the contributions made in this work is not significant enough for a conference publication. The theoretical contributions are not very meaningful, and the novelty of algorithm design is limited. I'm uncertain about how significant the empirical improvements on the real-world datasets are. Therefore, I would vote for reject with a low confidence score.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper studies the problem of online continual learning. The paper proposes a new algorithm which takes into account previously observed data samples instead of only the newly observed data sample.
strength_and_weaknesses: Strength: The idea of feedforward adaptation is interesting in online continual learning.

Weaknesses:

- The paper does not motivate the benefit of selecting the most similar data samples well. One simple solution is the learner adapt the model using all data samples in the buffer. Specifically, the paper does not consider the problem of online decision making and the learner only updates the model upon receiving new data sample without making decision for the newly observed data sample. In this case, the importance of adaptation is unclear.

- The paper does not present any theoretical results in form of a theorem or lemma to present the error bound. It would be useful if the paper presents the error bound in the paper and compares the error bound of the proposed algorithm with that of the case where all samples in the buffer are employed to update the model.
clarity,_quality,_novelty_and_reproducibility: Overall, the paper is well-written and also has some novelties although it is limited.
summary_of_the_review: In summary, the paper studies an important and interesting problem in online continual learning and proposes a new method for adaptivity of models in dynamic environments. However, the paper does motivate the importance of the proposed method well and to show the effectiveness of the proposed algorithm more analytical discussions should be added to the paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this study an online feed-forward adaptation approach is proposed to address distribution shift in  upcoming test data with respect to the training data. The core idea in feed-forward adaptation is finding a critical example from the memory, that is the most related to the current example, and adjust the model to predict the desired output of that critical example (in the past). This approach has been compared with the feedback adaptation approach in which the prediction model is updated according to the loss measured for the latest received ground truth. The authors also provided an analysis of the uncertainty of the prediction.

strength_and_weaknesses: The studied topic is interesting and it indeed seems to be the most viable learning scenario in practice in many applications. 
The contributions are clearly mentioned and the introduction motivates the reader to the topic. 
The proposed approach is interesting and specially introducing an approach uncertainty quantification can be very useful. 

However, when we get to the experimental results, the experiments lack the claims made in the paper. 
I would expect to see some experiments related to the uncertainty measurement for the real datasets. 
Also, for the real datasets, there is no experiment to study for each dataset, how much on average (or one single run), the critical selected examples, differ from the latest examples, and illustrate how this deviation relates to gaining in feed-forward in compare with feedback.  Also you could check for each dataset, how many times the critical examples, where actually the latest ones, or analyze the time invariance element empirically. 

In Page 3, “Let f_t  denotes the ground-truth predictor … that generates the ground-truth input-output tuples ”, please make it clear that it is only predicting the output not the tuple (not a generative approach).
 
In table 1, you could use bold face and underlined for the best and second best results, to make it easier to read.

There are typos and ambiguous sentences in the manuscript, such as:
 - Page 3: “Online adaptation continually learning from streaming data, which is belong…”  wrong sentence, please rewrite it.
- Page 4, the last sentence of the page is incomplete. “check the details in.”
- Page 5, eq(5) $\hat {K}$, is not defined.
- Several times in the main manuscript you referred to an equation that was in in appendix, whereas you had that same equation nearby too. Examples: Page 5, your referred to Eq. 29, instead of Eq. 8.
Page 6, Eq. 31 instead of Eq. 15. Page 7, Eq. 60, instead of Eq. 19 and Eq. 61 instead of Eq. 20. 
- In algorithm 2, line4, it should be s* in the subscript not s.
- In Lemma 2, B and $\hat {B}$ should be K and $\hat {K}$.
   

clarity,_quality,_novelty_and_reproducibility: The proposed approach is clearly explained and it seems novel. The quality of the study needs improvement. 
summary_of_the_review: In my opinion, this manuscript is not ready yet to be published at ICLR. However, it depends on how the authors address the raised issues.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: Authors have investigated to use online adaptation method in time-series prediction tasks. To address this challenge that catastrophic forgetting occurred in existing methods, it proposes a method to leverage critical data samples from the memory buffer to optimize the prediction model. They have conducted experiments to demonstrate that show the proposed method is effective. 

The theoretical analysis is not solid and algorithm design has a flaw.

justification_for_why_not_higher_score: Not so convincing in terms of theory and algorithm
Decision: Reject