SlowGuess/NeurIPS_2023_OCR huggingface dataset:
In this section, we'll discuss the limitations and future directions of our work. Firstly, the assumption of prior knowledge about non-stationarity following a linear form is strong. While it provides a starting point for understanding experiment design under structured non-stationarity, practical approaches are needed. One option is to use historical data for prior information, if there exists some. Alternatively, we can use data collected during the first phase of our L-OFUE algorithms to verify trends. Another question is model misspecification. What if the true underlying model is non-parametric, but the decision maker prefers using interpretable linear functions? Understanding how our design performs in such scenarios will provide valuable insights into its practical applicability. Furthermore, our framework and results are formulated based on a specific notion of non-stationarity. Exploring whether our framework can be adapted to different formulations of non-stationarity is another area of interest for future research. Another important direction for our future research is the exploration of instance-dependent regret bounds under linear and polynomial trends. In this paper, we focus on studying the worst-case optimal regret. However, in the field of classical stochastic bandits, there has been increasing attention on instance-dependent regret bounds (e.g., [11] [33][5]). It would be valuable to extend our analysis to incorporate instance-dependent parameters. Defining suitable instance-dependent parameters is not a trivial task, as the classical parameters such as arm gaps may not be directly applicable in our setting, where the parameters themselves change over time. Another more practical question that people may be of interest is how to choose  $\alpha$ . One possible approach is to adaptively end the first phase of the experiment. Ideally, continuous monitoring during Phase 1 would allow for the experiment to halt as soon as certain criteria are met, eliminating the need to manually select  $\alpha$ . However, reliable and efficient continuous monitoring of the switchback experiment poses significant challenges. Even in basic A/B testing without a linear trend, designing continuous monitoring requires substantial effort and delicate power analysis, as demonstrated in [45].
In this work, we present a comprehensive investigation into the problem of non-stationary experimental design with linear trends. Our focus is on two objectives: accurate estimation of dynamic treatment effects and minimizing welfare loss within the experiment. For scenarios involving linear trends, we propose an efficient design called L-E0FU. This design can be customized to achieve optimal estimation error rates, optimal regret rates, or strike a Pareto optimal trade-off between the two objectives. To establish the efficacy of our approach, we provide information-theoretical lower bounds that highlight the inherent challenges in estimating dynamic treatment effects and minimizing welfare loss. Additionally, we unveil the fundamental trade-off between the two objectives.
[1] Alberto Abadie and Jinglong Zhao. Synthetic controls for experimental design. arXiv preprint arXiv:2108.02196, 2021.
[2] Yasin Abbasi-Yadkori, David Pál, and Csaba Szepesvári. Improved algorithms for linear stochastic bandits. In Advances in Neural Information Processing Systems, pages 2312-2320, 2011.
[3] Karun Adusumilli. Risk and optimal policies in bandit experiments. arXiv preprint arXiv:2112.06363, 2021.
[4] Volkan E Akman and Adrian E Raftery. Asymptotic inference for a change-point poisson process. the Annals of Statistics, pages 1583-1590, 1986.
[5] Raman Arora, Teodor Vanislavov Marinov, and Mehryar Mohri. Corralling stochastic bandit algorithms. In International Conference on Artificial Intelligence and Statistics, pages 2116-2124. PMLR, 2021.