From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This paper proposes to solve an online hypothesis testing procedure that seeks to determine whether a representation's difference from a constant is more significant than a predefined margin for streaming data. This paper takes a more classical statistical approach rather than a bandit approach as the title suggests. The reviewers found several shortcomings of the paper that I'll summarize here. 

1) Even though the title suggests that the authors are using the theory of multi-armed bandits, there is nothing in the approach that the authors are balancing between exploration and exploitation. Thus, the title is misleading. 

2) The novelty vis-a-vis Chen et al. (2022) is not clear. The test statistic was proposed in Chen et al. (2022) and its properties also examined therein. Theorem 3.1 is not well stated. It seems that \alpha_B is defined before \eta_B. It should be the other way round. 

3) The writing suffers from several deficiencies. A thorough proof-reading of the article is necessary. Corollary 3.1, a formal mathematical statement, contains an approximation whose accuracy is not quantified. 

4) My own observation is that in the absence of strong theory, for an ICLR submission to go through, the experimental evaluations must be convincing and strong. However, this paper does not contain any interesting experimental results. 
justification_for_why_not_higher_score: Reasons above. 
justification_for_why_not_lower_score: Can't go any lower
summary_of_AC-reviewer_meeting: NIL

From: ICLR.cc/2023/Conference/Paper1038/Authors
title: Response
comment: We sincerely thank the reviewers for their careful responses and valuable suggestions on our work. Below are our responses to your concerns. Please let us know if you have any further questions.

1. Question: It's not clear how bandit algorithms are applied in the proposed algorithm.

  Answer: 

Unlike conventional bandit algorithms, we need to emphasize that our motivation is not to obtain the maximum reward but to give statistical inference procedure of parameters in online learning field.

2. Question: It seems to me that the made decision (choosing arm L or R) does not affect the afterward estimation.

Answer: 

(1)	For the two armed bandit problem, we refer to the sampling strategy, which developed by Chen in [2], that generate the maximum average return. The sampling strategies are asymptotically optimal in terms of the maximization of the mean. 

(2)	As the results obtained in [2], since both the sample mean and sample fluctuation depend on the sampling strategy, the strategy we used in paper will be in terms of the maximal probabilities over all strategies. 

3. Question: The novelty of this paper is somehow limited.

Answer: 

(1) For online learning, this is completely new consideration to conduct statistical inference based on the two armed bandit model.

(2) Despite using the strategy-driven limit theorem in this paper, it is not necessary to consider the center of symmetry of the function. Additionally, we skillfully construct an opposite reward function based on randomness so that the proposed strategy is incorporates the “knowledge” from the null hypothesis, which is that the expectation of the left arm under our proposed strategy is greater than 0 when the null hypothesis is true. We have applied it for online representative similarity test problem under streaming data. 

(3) The proposed strategic aggregation learning method can be widely applied to streaming data and improves the testing power largely than classical normal distribution-based test. 

(4) Thank you for your professional suggestions and we will supplement the novelty of Theorem 3.1 in New Rebuttal Revision.

4. Contribution：

(1) The paper contributes to a representational similarity testing framework that can be applied to a variety of complex models for inferring statistical information from streaming data. 

(2) We introduce a new perspective under streaming data by adequately incorporating knowledge derived from the null hypothesis, which is the first contribution of this article.

From: ICLR.cc/2023/Conference/Paper1038/Authors
title: Response 
comment: Please accept our sincere thanks for your very helpful and professional comments, which have inspired us to enrich this work.
We apologize for the words error you have pointed out and we have changed the words in New Rebuttal Revision. Please kindly let us know if there are any further questions.
1. Question: It seems that Theorem 3.1 holds without any restriction on the batch size.

Answer: 

(1)	The reason why Theorem 3.1 holds without any restriction on the batch size is that the number of batches (i.e. B) tends to infinite.

(2)	Figure 4, Figure 6 and Figure 7 illustrate that increasing the batch-specific sample size can surely help test statistic to converge to the limited distribution.

(3)	But Figure 4 shows that the asymptotic distribution converges even if batch size is equal to 20 according to simulation. Because B also make a huge contribution to the convergence. We definitely hope that batch size is bigger and we will attach this part to the paper.


2. Question: What is the contribution of this paper?

Answer: Our main contributions include:

(1) The paper contributes to the development of a generalized procedure, which can be applied to a variety of complex models, for inferring statistical information from streaming data.

(2) This is the first consideration about the combination of knowledge-driven and data-driven learning procedures based on the two armed bandit model for statistical inference.

(3) The proposed test statistics can enhance the testing power largely in any settings compared with classical normal distribution-based test statistic, which is theoretically examined and illustrated in the Corollary 3.1 and intuitively showed in Figure 2.

(4) Thank you for your comments and we will again illustrate and highlight our contribution after introduction in New Rebuttal Revision.

3. Motivation and innovate: 

(1) Despite using the strategy-driven limit theorem in this paper, it is not necessary to consider the center of symmetry of the function. Additionally, we skillfully construct an opposite reward function based on randomness so that the proposed strategy is incorporates the “knowledge” from the null hypothesis, which is that the expectation of the left arm under our proposed strategy is greater than 0 when the null hypothesis is true. 

(2) The proposed online learning is actually speedy because we summarize the online updates by the simple average-aggregation and a strategy without access to the historical data.

From: ICLR.cc/2023/Conference/Paper1038/Authors
title: Response
comment: Thank you for your recognition and let me express my sincere gratitude for your review. Please kindly let us know if there are any further questions.

Motivation and innovate:

Unlike traditional test statistics, our proposed test statistic construction incorporates the “knowledge” from the null hypothesis, which is that the expectation of the left arm under our proposed strategy is greater than 0 when the null hypothesis is true. Therefore, the developed representational similarity test statistic, which incorporates the prior knowledge, which improves the testing power largely than classical normal distribution-based test statistic.

From: ICLR.cc/2023/Conference/Paper1038/Reviewer_7T3k
confidence: 1: You are unable to assess this paper and have alerted the ACs to seek an opinion from different reviewers.
summary_of_the_paper: I'm not able to review the paper since it is out of my expertise.
strength_and_weaknesses: na
clarity,_quality,_novelty_and_reproducibility: na
summary_of_the_review: na
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1038/Reviewer_UrZb
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper proposes a new statistical test method by using the two-arm-bandit model. Compared with classical statistics such as normal statistics, the proposed TAB statistic enjoys better power, which makes the hypothesis test more viable. The authors prove its asymptotic distribution to validate this claim. 
strength_and_weaknesses: Pros:
- The presentation is clear.
- The experiments are comprehensive, given that it is a statistics-related work.

Cons:
- There exists some writing issues in the current version of this paper. For instance, 

-- Page 2, they proposes -> propose

-- Page 2, the the strategy -> the strategy

-- Page 2, and aims at explore -> aim at exploring

-- Page 4, determine the next reward -> determines the next reward

-- Page 5, starting from 'According to...', the following texts should be removed from the Theorem environment.

-- Page 5, we concludes -> conclude

- The main theorems lack explanation. It seems that Theorem 3.1 holds without any restriction on the batch size. It seems counter-intuitive since a large batch size can indeed help the convergence to the asymtopic distribution. The authors could explain this. 
clarity,_quality,_novelty_and_reproducibility: The proposed method is novel for the hypothesis test.
summary_of_the_review: This paper proposes a new TAB-based statistic that can be applied to the hypothesis testing problem. I do not find any obvious mistake in both the algorithm or its proof. However, I can not tell what is the contribution of this paper, given the fact that the statistic is not originally proposed in this work (it is proposed in Chen et al. 2022 firstly, according to the authors). There also exist many writing issues which lower the quality of this work. Thus, I recommend a reject to this paper. 
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper1038/Reviewer_UZv9
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This paper formulates the statistical inference with incremental data as a bandit problem. The authors provide asymptotic theoretical results and simulation results. 
strength_and_weaknesses: ### Strength:

- Incorporating bandit with statistical inference and similarity tests sounds interesting and is new in literature to my knowledge.

### Weakness

- It's not clear how bandit algorithms are applied in the proposed algorithm. It is hard to find how the proposed algorithm is balancing the **exploration** and **exploitation** like conventional bandit algorithms. Also, different from other sequential decision-making problems, it seems to me that the made decision (choosing arm L or R) does not affect the afterward estimation 

- The author needs to highlight their contribution given [1, 2], especially the novelty of Theorem 3.1

[1] Zengjing Chen and Larry G Epstein. A central limit theorem for sets of probability measures. Stochastic Processes and their Applications, 2022.

[2] Zengjing Chen, Shui Feng, and Guodong Zhang. Strategy-driven limit theorems associated bandit problems. arXiv preprint arXiv:2204.04442, 2022.
clarity,_quality,_novelty_and_reproducibility: - This paper is overall well written and the experiments are reproducible. 

- The novelty of this paper is somehow limited: it seems that the algorithm is just combining the bandit structure and the incremental update in statistical inference and similarity test together. The key concerns in bandit problems, like exploration and exploitation, are not well addressed.
summary_of_the_review: This paper applied the bandit structure to online statistical inference and similarity tests. However, it seems to me that this combination is not that novel. For example, the key concerns in bandit problems, like exploration and exploitation, are not well addressed. It's hard to find why the proposed combination is a real sequential decision-making algorithm. For example, in the bandit problems, the decision made on the current step will affect the future estimation, however, here it seems that the similarity estimation $\hat \theta$ is not affected no matter whether arm L or R is pulled. 

Therefore, though this paper is overall well-written and supported, I would suggest a rejection unless the author could further highlight their contribution, especially the role in bandits and the comparison with (Chen et al, 2022)
correctness: 4: All of the claims and statements are well-supported and correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough