Title: Design-Based Bandits Under Network Interference: Trade-Off Between Regret and Statistical Inference
Venue: NeurIPS 2025
Accepted: yes
Keywords: Causal inference

Adversarial bandits

Network interference
Abstract: In multi-armed bandits with network interference (MABNI), the action taken by one node can influence the rewards of others, creating complex interdependence. While existing research on MABNI largely concentrates on minimizing regret, it often overlooks the crucial concern that an excessive emphasis on the optimal arm can undermine the inference accuracy for sub-optimal arms. Although initial efforts have been made to address this trade-off in single-unit scenarios, these challenges have become more pronounced in the context of MABNI. In this paper, we establish, for the first time, a theoretical Pareto frontier characterizing the trade-off between regret minimization and inference accuracy in adversarial (design-based) MABNI. We further introduce an anytime-valid asymptotic confidence sequence along with a corresponding algorithm, $\texttt{EXP3-N-CS}$, specifically designed to balance the trade-off between regret minimization and inference accuracy in this setting.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies an adversarial bandit problem with network interference where in each round, the decision maker must assign one of K arms to each of the N nodes in a network with some underlying graph structure. The authors study three (generally) competing objectives of regret minimization, validity of post hoc inference and average-treatment-effect estimation and establish Pareto optimality results. They also propose an anytime-valid asymptotic confidence sequence and based on it an EXP3-style algorithm that balances the trade-off between regret minimization and inference accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: The paper is well-written generally and makes progress on an important problem in causal learning. Addressing causal inference with interference in an online learning setting is quite relevant in practical applications like social networks and large-scale experimentation on platforms. The anytime-valid confidence sequences provided in this paper are important in this context. The authors also generalize the theoretical lower bound of (Simchi-Levi and Wang, 2024) to capture the complexity of the networked setting via a new implicit primitive (effective arm space size).\n\nThe following are not weaknesses, just some resistance I encountered while parsing the content: \nSome constructs (e.g., exposure mappings and clustering) are not common in traditional bandit literature and might need a little more context and/or instantiation via examples for readers not entirely familiar with the causal inference/bandit learning interface. I understand the paper is constrained to be within 9 pages, but it would be useful if the model development in S3.1 and S3.2 is pre-padded with a little more context in this regard. While not a weakness per se of the paper, it is less than ideal that the main algorithm itself is pushed to supplementary pages together with a high-level discussion of what it does. This leaves S5.2 somewhat obscure on the first read.\n\nI want to emphasize that I liked reading the paper but my conservative score is a consequence of being unable to place the methodological and analytical contributions in this work vis-a-vis prior art. In particular, though the networked model under interference is a significant generalization of the simple setting of (Simchi-Levi and Wang, 2024), the hardness of analyzing this setting is not very clear. I would be open to revising my scores pending a satisfactory rebuttal to this end.\n\nQUESTIONS:\nWhen N=1, does Theorem 4.1 reduce to Theorem 1 of Simchi-Levi and Wang [2024]? Or Condition 3.1 prohibits this reduction? Further to this point, it would be nice to know how much of the proof of Theorem 4.1 is adapted from the proof of Theorem 1 of Simchi-Levi and Wang [2024]. \n\nIn terms of the algorithm, Simchi-Levi and Wang [2024] also consider a variant of EXP3. Given this, it would be helpful for the readers to know what new technical/analytical contributions the authors make to overcome limitations of prior techniques.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nBriefly, the paper presents a modular framework for Multi-Armed Bandits with Network Interference (MABNI). It contributes the trade-off between regret minimization and inference accuracy through establishing a theoretical Pareto frontier for minimizing regret and ATE estimation error in adversarial networked bandit scenarios and also their EXP3 algorithm to address regret minimization, continual inference and ATE estimation error minimization. They manage the action space with exposure mapping and clustering.\n\nSTRENGTHS AND WEAKNESSES:\n### Quality: \nThe submission is somewhat technically sound.\nClaims are supported by theoretical analysis and experimental results.\nThe methods used are appropriate, each addressing different aspects in a modular fashion.\nThis is a complete piece of work, with some open problems as a future direction.\nThe authors are careful and honest about evaluating both the strengths and weaknesses of their work, but experiments section is a bit neglected.\n\n### Clarity: \nThe submission is mostly clearly written.\nIt is generally well organized.\nIt seems to adequately inform the reader by providing enough information to reproduce its results.\n\n### Significance: \nThe results are impactful for the community, with a well-establish trade-off between regret minimization and inference accuracy for consideration in critical practical applications.\nOthers (researchers or practitioners) are likely to use the ideas or build on them.\nThe submission addresses a difficult task in a better way than previous work, as demonstrated by the shortcomings of the past works.\nIt advances our understanding and knowledge on the topic in a demonstrable way. However, the importance of lower-bound is ambiguous and there exists an optimality gap.\nIt provides a unique theoretical modular framework.\n\n### Originality: \nThe work provides new insights and deepen understanding for MABNI trade-off with pareto frontier.\nIt is clear how this work differs from previous contributions, with relevant citations provided.\nThe work introduces novel methods that advance the field.\nThis work offers a novel combination of existing techniques to address different objectives, and the reasoning behind this combination is well-articulated.\n\nQUESTIONS:\nHow is \"overly prioritizing the optimal arm can compromise the inference accuracy of the sub-optimal arm\" neglected? Isn't this a version of exploration-exploitation trade-off, which is pretty standard in bandit research?\n\nThe phrases \"anytime-valid\" and asymptotic do not seem complementary. Please elaborate.\n\nIn (1), how is the adjacency matrix H both a possible argument for the mapping S and part of the domain bold italic S?\n\nLine 159: isn't the size reduced to a power of the size of $d_s$?\n\nThere seems to be mix-up with the notation in the first part of (2). Please properly define $S_t$.\n\nDefinition 3.2: explain ATE here.\n\n(3): what is the usage of the subscript in R? Adversarial nature? It is undefined.\n\nAlso, why is the lower-bound in Theorem 3.1 of importance? Why is the multiplication of square-root regret and ATE error is specifically chosen for lower bounding?\n\nThe experiments section can be improved. The figures only support the theoretical findings in a basic manner. Maybe replace the figures with tables to properly report various experimental results.\n\nLIMITATIONS:\nno, you can include some comments in conclusion.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the multi-armed bandit problem with network interference problem. The goal is to design an algorithm that continually satisfies two-fold objectives: regret minimisation, and accurate ATE estimation. The paper proposes first a lower bound on the product of regret and ATE estimation error. Then under known adjacency matrix assumption, it develops an adversarial bandit algorithm EXP3-N-CS, and proves the regret and ATE estimation upper bounds for it. Numerical experiments are provided to demonstrate efficiency of EXP3-N-CS.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The paper is cleanly and logically written. Works as a good introduction to the setup for the reader.\n2. The idea of deriving a \"Pareto optimality\" lower bound and upper bound in the setting as in [Simchi-Levi and Wang, 23] is interesting.\n3. The experimental evidence across networks validates the applicability of the algorithm.\n\nWeakness:\n1. Please add the algorithm pseudocode in the main paper. It is very hard to understand Sec 5.2. without going to and fro to the Appendix.\n2. It is not clear how to choose $\\alpha$ in practice. \n3. I understand why knowing H is helpful here and not knowing it would increase complexity. But I am not sure that it has to be known. The exposure mapping can be learned on-the-go too.\n4.  The paper does not show why the produce of two objectives is \"the\" way to look into Pareto optimality in this setting. It is not intuitive, in general. An explanation is needed.\n5. Line 362-365 were not necessary. You do not have to be slightly suboptimal or really optimal to write a science paper. Do remove this claim. Rather, Theorem 5.6. says that the gap between lower bound and upper bound grows with $T^{\\alpha/2}$. It is more useful to community if you explain from where this gap is arising. Is it a proof issue or natural to the algorithm? Also, experiments do not point to \"optimality\". There is no lower bound plot in experiments.\n\nQUESTIONS:\n1. Check the weaknesses and please address them.\n2. Why in theorem 5.5. $\\alpha$ has to be in the range of $[0,1/4)$? What is the technical bottleneck or is it some step in proof that leads to it? \n3. Can you provide a detailed ablation study on choice of $\\alpha$?\n4. Also, it would be interesting to see how the proposed algorithm performs against Simchi-Levi and Wang [2024] and Liang and Bojinov [2023] in network-less setup and Zhang and Wang [2024] in the networked setup. Specially, I am curious why to choose adversarial or stochastic algorithm in this setting.\n\nLIMITATIONS:\nExcept the known H assumption, limitations are not discussed much. Check pointers regarding $\\alpha$ and tightness claims.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper establishes theoretical results unifying statistical estimation and regret minimization in the multi-armed bandits with network interference setting. The paper also tests the trade-off with different network topologies and experimental setups.\n\nSTRENGTHS AND WEAKNESSES:\nStrength: \n1. Significance: I think the topic is very interesting to me, unifying different pieces from two topics: statistical estimation and regret minimization. The former is very broadly studied by statistics community and the latter by the online learning community. As shown in table 1, there is no previous work that considers the three objectives in a network interference setup. \n2. Quality: The theoretical results are complete and I also like the experiments.\nWeaknesses: \n1. Clarity: \n* Can you make sure the reference works in the main text as well as the appendix? In particular, figure 1 referred to theorem 5.2, 5.3 and 5.4 which are not in the text. \n* Some of the notations can be more clear. For example, in the definition of objective 3, the left hand side uses $\\nu$ which is a legitimate  instance but the right hand side not.\n\nQUESTIONS:\n1. The paper uses two common techniques to reduce the dimension of the potential outcomes, I wonder what is the role of the clustering assumption beyond just restricting the cardinality of $\\mathcal{U}_{\\epsilon}$. For example if I only have this exposure mapping but restrict the cardinality of $\\mathcal{U}_s$. I think all the results go through? \n2. I am a bit confused about the line 357, can you explain why you make that conjecture? Also, can you explain what makes line 339 and (3) so different? \n3. Is it possible to add covariates into this setup, for example your potential outcomes depend on the exposure mapping as well as personal characteristics.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}