Effective Dimension in Bandit Problems under Censorship
Anonymous Author(s)
AfﬁliationAddress email
Abstract
In this paper, we study both multi-armed and contextual bandit problems in censored environments.
Our goal is to estimate the performance loss due to censorshipin the context of classical algorithms designed for uncensored environments.
Ourmain contributions include the introduction of a broad class of censorship modelsand their analysis in terms of the effective dimension of the problem – a naturalmeasure of its underlying statistical complexity and main driver of the regret bound.
In particular, the effective dimension allows us to maintain the structure of theoriginal problem at ﬁrst order, while embedding it in a bigger space, and thusnaturally leads to results analogous to uncensored settings.
Our analysis involves acontinuous generalization of the Elliptical Potential Inequality, which we believeis of independent interest.
We also discover an interesting property of decisionmaking under censorship: a transient phase during which initial misspeciﬁcationof censorship is self-corrected at an extra cost; followed by a stationary phase thatreﬂects the inherent slowdown of learning governed by the effective dimension.
Our results are useful for applications of sequential decision-making models wherethe feedback received depends on strategic uncertainty (e.g., agents’ willingness tofollow a recommendation) and/or random uncertainty (e.g., loss or delay in arrivalof information).
Introduction
Bandit problems are prototypical models of sequential decision-making under uncertainty.
Theyare widely studied due to their applications in recommender systems, online advertising, medicaltreatment assignment, revenue management, network routing and control [23, 35].
Our work ismotivated by settings in which the feedback received by the decision-maker in each round of decisionis censored by a stochastic process that depends on the current action as well as past history offeedbacks and actions.
For instance, in typical missing data problems, the decision-maker needs todeal with frequent losses of information (or delays in arrival of information) due to exogeneous failuressuch as faulty and/or unreliable communication.
Missing observations in dynamical interactionswith the environment are a common concern in diverse ﬁelds ranging from operations managementto health sciences to physical sciences [41, 17, 26].
In other settings, such as AI-driven platformsfor health alerts, route guidance, and product recommendations [7, 42], the reception of feedbackdepends on whether or not the decision (or recommendation) is adopted by strategic agents (e.g.patients, customers or drivers) with private valuations.
Thus, from the platform’s viewpoint, theadoption behavior of heterogeneous agents can be regarded as a stochastic censorship process.
In static environments, the bias induced by the presence of randomly missing information has beenthoroughly studied [26, 30].
However, in online settings, the dynamics of learning and acting areinherently coupled: since censorship mediates current information of the environment, it impacts the outcome of data-driven decision process; this in turn conditions the future decisions and futurecensored feedback, creating a complex and endogenous joint temporal dependency.
Our workcontributes to the analysis of such phenomena for a broad classes of decision and censorship models.
Importantly, it is the ﬁrst normative inquiry of how censorship impacts the statistical complexity ofbandit problems.
We develop an analysis approach that is useful for both estimating the performanceloss due to censorship and reﬁning the classical algorithms designed for uncensored environments.
1.1 Related Work
Within the extensive bandits literature, well-surveyed in [23, 35], our work is most closely relatedto stochastic delayed bandits.
Initially, this line of work focused on the joint evolution of actionsand information in settings where the reception of the latter is delayed [13].
Of particular interest isthe packet loss model recently introduced in [ 21 ] , which provides the regret bound O ( 1 p R T ) where
R T is the uncensored regret and p the censorship probability.
Analogous results have been shownin the context of Combinatorial Multi-Armed Bandits with probabilistically triggered arms; see forexample, [10] and [39].
Our work provides a systematic approach to study more general censorshipmodels, and sheds light on how the impact of coupled feedback and censorship realizations on theexpected regret can be evaluated in terms of the effective dimension of the problem.
Importantly, we also tackle the contextual bandit problems, where relatively few results are availableon the regret under missing or censored feedback.
A notable exception is the work of [38], who focuson a different information structure and obtain a scaling of 1 /p (see Remark 2).
A related contributionby [2] provides both a potential-based analysis of Upper Conﬁdence Bound algorithm (UCB) formulti-armed bandits and an algorithmic variant leveraging the Kaplan-Meier estimator, although theircensorship setting is different than ours.
In particular, our results are applicable to settings whendelay is signiﬁcantly large (possibly inﬁnite).
This is in contrast to prior results on bandits withdelayed information structure which assume either that the delay is constant , upper bounded , has aﬁnite mean , or simply provide regret guarantees that are linear in the cumulative delays up to time T[13, 19, 27, 43, 31].
Under such assumptions on delay, one usually gets an additive dependency ofthe regret in terms of delay parameters.
The abovementioned works primarily focus on modifying well-known bandit algorithms to accountfor delays, or propose new delay-robust algorithms which may be difﬁcult to implement in practice; anotable exception includes [40] but it is focusing on Thompson Sampling.
In our work, we insteadfocus on estimating the performance loss due to censorship and derive insights on the behavior of wellknown UCB class of algorithms [24, 11, 1].
These algorithms are widely used in practice; moreover,their theoretical study has been shown to be useful for analysis of broader class of algorithms (notably
Thompson Sampling [3, 33] and Information-Directed Sampling [34, 20]).
Our work contributes to the Generalized Linear Contextual Bandits literature [14, 25] in two ways:
Firstly, through the use of these models in a sequential decision-making framework on which theimpact of censorship is assessed in Sec. 4.
Secondly, by showing that our multi-threshold censorshipmodel MT induces, at ﬁrst order, a non-linear structure that closely mirrors such models.
Our resultsprovide new tools to study this structure.
It is useful to note that the notion of effective dimension hasbeen well-studied in the statistical learning and kernels literature [36, 37] (where it is deﬁned for a
Gram matrix K n and regularization  as d n eff (  ) =
tr ( K n ( K n +  I d )  1 ) ).
Our work shows that ananalogous quantity governs the regret bound of bandit problems in censored settings.
The literature on non-stochastic multi-armed bandit problems with delays problems [29, 9, 18] alsotackles multiplicative dependency, although in a different setting than ours.
Another related line ofwork is Partial Monitoring [6, 22] which deals with generic categorization of learnability, rather thana ﬁne-grained analysis of dimensionality in relation to censorship.
Finally, there is a rich literature onclassical missing and censored data problems, well surveyed in [26] or [30].
Our work contributes tothis broad ﬁeld since we deal with endogenous and sequentially generated censorship process.
1.2 Summary of Results
In Sec. 3, we consider Multi-Armed Bandit (MAB) models and prove that the regret scales as˜ O ( d e ↵ p T )
(Thm. 3.1), where d e ↵ is the effective dimension with value
P a 2 [ d ] 1 p a .
In doing so, werecover and generalize related results from [21, 10] to more complex regularized settings and noisemodels.
In particular, we prove that the effective dimension results from characterizing the so-called censored cumulative potential V ↵ .
Interestingly, we also show that the adaptive nature of censorshipon V ↵ plays only a second order role (Prop. 3.7), that is, impact of censorship can be treated in anofﬂine manner at ﬁrst order.
Importantly, our study of MAB under censorship instantiates an analysis framework which extendsto Linear Contextual Bandits (LCB)
(Sec. 4).
Our main result provides that regret is still governedby the effective dimension, but now with a dependency of ˜ O
(  p d · d e ↵ p T )
(Thm. 4.1).
To thebest of our knowledge, these regret bounds provide the ﬁrst theoretical characterization in LCB withcensorship, and contribute to the literature by evaluating the impact of censorship on the performanceof UCB-type algorithms.
Our second main contribution is identifying the effective dimension fora broad class of multi-threshold models MT as well as a precise understanding of the dynamicbehavior induced by these models (Thm. 4.5).
In particular, we ﬁnd that censorship introduces a twophase behavior: a transient phase during which the initial censoring misspeciﬁcation is self-correctedat an additional cost; followed by a stationary phase that reﬂects the inherent slowdown of learninggoverned by the effective dimension.
In extending our analysis from MAB to LCB, we also developa continuous generalization of the widely used Elliptical Potential Inequality (Prop. 4.3), which webelieve is also of independent interest.
Problem Setup and Background
Bandit Model: We successively consider stochastic multi-armed bandits (Sec. 3) and Linear
Contextual Bandits (LCB) (Sec. 4) in censored environments.
In both settings, at each round t  T ,the agent observes an action set A t ⇢ A .
She then selects an action a t 2 A t (i.e. an arm ) to 109 which a noisy feedback r ( a t )
+ ✏ t is associated, where r ( a t ) is a bounded reward and ✏ t is an i.i.d.sub-Gaussian noise of pseudo-variance  2 .
For action a , the sub-optimality gap at time t is denoted t ( a ) , max ˜ a 2 A t r (˜ a )  r ( a ) , and the maximal gap  max , max a,t  t ( a ) .
We now recall thespeciﬁcs of each model:
• MAB : There is a ﬁnite number of actions d , enumerated as A , [ d ] , each having a scalarreward ✓ ?
a .
Arms are independent : playing one arm gives no information about the others.
• LCB : The action set A t is a subset of the unit ball B d , possibly inﬁnite.
Non-stochastic
contexts are modeled by the fact that A t is drawn by an oblivious adversary.
Here one does
not need to rely on the typical i.i.d assumption on their generating process [43, 14].
Unless
explicitly mentioned, the reward is assumed to be linear with respect to
a latent unknown
vector ✓ ? 2 R d , i.e., r ( a ) =
h a, ✓ ? i .
Information Structure:
In the classical uncensored setting, the noisy feedback is immediatelyobserved post-decision and utilized to make decisions in the next round.
We introduce the followingcensorship model: an independent Bernoulli random variable of parameter p a t denoted as x a t isdrawn after each decision a t and the feedback is observed, i.e., realized , if and only if x a t = 1 ; elsethe feedback is said to be censored .
We recover the uncensored setting when p a ⌘ 1 .
Algorithm 1: Generic UCB
Input: Total time T , Regularization  , Precision  for t = 1 , . .
, T do
Provide reward estimator ˜ r  t verifying w.p.
1   : 8 a 2 A t , r ( a )  ˜ r  t ( a ) ;
Play action a t = argmax a 2 A t ˜ r  t ( a ) ; if ( a t , r ( a t ) + ✏ t ) is realized i.e. x a t = 1 then
Update ˜ r  t ; end end
Algorithms: To study the impact of censorship on bandit problems, we consider the class of high-probability index algorithms based on the optimism under uncertainty principle, commonly referred as UCB -algorithms.
Following [20], Algorithm 1 summarizes the generic UCB design framework.
We detail in Supplementary Information (SI) the speciﬁc instances of UCB for MAB (resp.
LCB) used in Sec.3 (resp. Sec.4).
Moreover, this family of algorithms strongly relies on regularized reward estimators ˜ r  t , where the regularizer is mostly used to prevent an artiﬁcial cold-start exploratory phase.
Performance Criterion: The frequentist performance of the agent is measured by the notion ofpseudo regret , i.e., the difference between the algorithm’s cumulative reward and the best total reward.
More formally, we introduce for any policy ⇡ 2 ⇧ :
[Equation]
We aim to provide guarantees on E [ R ( T, ⇡ )] with respect to the number of rounds T and quantitiesthat govern the complexity of the problem (for example number of arms, ambient dimension d ,parameters of censorship model or smoothness properties of the reward r ).
Here, the expectation iswith respect to the noise induced by the feedback, the censorship and a possibly randomized policy.2.1 Notations
Transpose of a vector u is denoted by u > , classical Euclidean inner product by h ., .
i and trace operatorby Tr .
For positive semi-deﬁnite matrix ⌃ 2 R d ⇥ d and for any vector u 2 R d , notation k u k ⌃ refersto p u > ⌃ u .
We use notation I d to denote the d ⇥ d identity matrix.
B d is the unit ball in R d .
[ n ]is the set of integers { 1 , 2 , · · · , n } .
For a given function f , we note f ( i ) the i th derivative of f .
Toavoid confusion with the dimension d , we use @ x instead of dx to denote an inﬁnitesimal increase ofx .
We use the asymptotic notations ⇠ , O , ⇥ and ˜ O
( O when log factors are removed).
Finally, for anevent H , we use ¬H to denote its complement.
Multi-Armed Bandits
3.1 Effective Dimension and Regret Bounds
The main result of this section is that censorship effectively enlarges the dimension of the problem.
We deﬁne the effective dimension as d e ↵ , P a 2 [ d ] 1 p a and our result (Thm. 3.1) shows that, at ﬁrstorder, the regret is guaranteed to be the same as the uncensored problem with d e ↵ arms instead of d .
Theorem 3.1.
Under censorship, the UCB algorithm with regularization  has an instanceindependent expected regret of:
[Equation]
Furthermore, we obtain analogous regret guarantees for instance-dependent cases where, at ﬁrst order,the uncensored dimension [Equation] enlarges to [Equation]
Proposition 3.2.
For a ﬁxed action set A t ⌘ [ d ] and for a-priori known action gap  a , max ˜ a ✓ ? ˜ a  151 ✓ ? a , the UCB algorithm with regularization  has the instance-dependent expected regret:
[Equation]
On one hand, a preliminary understanding of censorship posits an increase of the average " regret perinformation gain " [20] (as it takes longer on average to get the same amount of information) but doesnot change the underlying complexity of the problem.
One the other hand, our results (Thm. 3.1 and
Prop. 3.2) postulate that the censored problem is equivalent at ﬁrst order to a higher dimensionalproblem but explored with the same " regret per information gain ".
The abovementioned results extends to a-priori known heteroskedasticity (see SI).
For this generalsetting, the effective dimension for instance-independent (resp.
dependent) case is given by P [Equation] (resp.
P a 6 = a ?  2 a p a  a ), where  2 a is the variance proxy of arm a . Although the scaling in P
[Equation]
[Equation] p [Equation] was already mentioned in [21] for unregularized setting with homogeneous variance  2 and provento be optimal, our results generalize these ﬁndings.
3.2 Cumulative Censored Potential
We now provide a proof sketch of Thm. 3.1, and in doing so, we instantiate an analysis frameworkthat will be extended in Sec. 4.
This proof consists in the successive elimination of the noise induced by the feedback and censorship.
This leads to regret guarantees on a resulting deterministic quantityby characterizing worst-case learning conditions.
The ﬁrst step of the proof is a variant of the classicalreduction of the UCB regret to another quantity we refer to as the expected cumulative censoredpotential .
Before stating it, we deﬁne at the end of a round t 2 [ T ] , the random number of times anarm a has been pulled as ⌧ a ( t ) , P tl =1 1 { a l = a } .
Similarly, the number of times an action a hasbeen realized at the end of round t is denoted N a ( t ) , P tl =1 1 { a l = a, x a l = 1 } .
We then have:
Lemma 3.3.
Given an uniform regularization of  > 0 , the UCB algorithm veriﬁes:
[Equation]
where, for any ↵ > 0 and ⇡ 2 ⇧ , the cumulative potential under censorship is given by:
[Equation]
In contrast to the classical non-regularized analysis or to the LCB case of Sec. 4, we observe twodifferent orders of ↵ ( 1 / 2 and 1 ) coming from the use of the L 1 -norm instead of the L 2 -norm.
Takenindependently, they lead to respective contributions of O ( d e ↵ log( T )) and O ( p d e ↵ T ) .
Note that byworking with general ↵ , our analysis naturally extends beyond sub-Gaussian noise to more generalassumptions about the Laplace transform of noise (e.g., lighter or heavier tails).
To further study V ↵ ,we introduce the following property:
Proposition 3.4.
For all ↵ > 0 ,  2 ]0 , 1] and given ↵ a primitive of x 7! x  ↵ , we have:
[Equation]
The proof of this proposition involves two steps: ﬁrstly, we remove the stochastic dependence inducedby the censorship through concentration properties (See SI), and we then solve the resulting policymaximization problem (Lemma 3.5).
In the ﬁrst step, we consider for a given  2 ]0 , 1] the event:
[Equation]
where T 0 ( a ) , 24 log( T ) /p a and claim that P ( H CEN (  ))  4 d e ↵  2 T  12  2 , improving a result of[21].
Our second step makes use of the following lemma (also known as a water-ﬁlling process ininformation theory [12]):
Lemma 3.5.
For ↵ a primitive of x 7! x  ↵ where ↵ 2 ]0 , 1] , regularization (  a ) a 2 [ d ] 2 ( R > 0 ) dand censorship vector ( p a ) a 2 [ d ] , the solution of the optimization problem:
[Equation]
is given by ⌧ ? a = 1 p a [ C   a ] + , where C ensures the total budget constraint P
a 2 [ d ] ⌧ ? a = T .
Inparticular, with  eff , 1 d e ↵ P [Equation]
[ d ]  [Equation] p a and  0 a , d e ↵ (  a   eff ) , the optimal solution is given by⌧ ? a , 1 p a d e ↵ ( T   0 a ) for T  max a  0 a and the optimal value is d e ↵ ↵ ( Td e ↵ +  eff )
P a 2 [ d ] 1 p a ↵ (  a ) .
For unregularized algorithms, this framework can be easily applied to provide instances-dependentguarantees by adding constraints of type ⌧ a  f (  a ) within Lemma 3.5.
Optimal guarantees underregularization such as the ones given in Prop. 3.2 require however to consider both orders of V ↵ ( 1 / 2and 1 ) simultaneously and not independently, leading to slight variations as shown in the proof of
Prop. 3.2 in SI.
Next, we further discuss the properties of V ↵ given its importance in our analysis.
3.3 Evaluating Adaptivity Gain
In fact, through the introduction of H CEN (  ) and for any ↵ 2 [0 , 1] ,  2 ]0 , 1] , we showed in Prop.3.4 the upper bound d e
[Equation]
↵ ( T d e ↵ +
[Equation]
1   ) for max E [ V ↵ ( T, ⇡ )] where the maximum is taken overthe class of adaptive policies ⇧ adapt , i.e., measurable with respect to the censorship.
Note that theexact value of such maximum is notoriously difﬁcult to study due to the adaptive nature of censorshipinduced by the decision-making process.
Interestingly, we obtain a surprising result that the gain dueto adaptivity is not signiﬁcant.
To show this, we introduce ⇧ o ↵ , the class of policies that are notadaptive with respect to the censorship and we prove that :
Lemma 3.6.
For ↵ 2 ]0 , 1] and  > 0 , we have max ⇡
[Equation]
In other words, restricting attention to ofﬂine policies is sufﬁcient to obtain the correct scaling.
Thenext step to complete our claim is the asymptotic expansion:
Proposition 3.7.
For ↵ 2 ]0 , 1] , by denoting  ↵ ( p ) ,
[Equation]
, we have
[Equation]
[Equation]
Moreover, if for a given  2 ]0 , 1[ , we introduce ⇧ single (  T ) the policy class whose censorshipinformation set has a single updating at time b  T c , we have:
[Equation]
Thus,  ↵ ( p ) can be viewed as an adaptivity gain resulting from the continuous correction of thecumulative variance induced by the action selection process.
Essentially, it is closely related tothe Jensen Gap of an appropriate random variable and the proof involves the study of the Taylorexpansion of the potential function ↵ .
( ?? ) tells us that a single observation of the censorshiprealization is sufﬁcient to obtain a near-optimal gain in adaptivity .
We present a proof sketch of Prop.3.7 in SI.
This shows that censorship in MAB can be treated in an ofﬂine manner at ﬁrst order.
Contextual Bandit
In this section, we study Linear Contextual Bandits under censorship.
The regret analysis for thisgeneral setting under censorship is signiﬁcantly more complex than for the MAB model.
This is dueto the fact that different actions contribute differently to the information acquisition, leading to anon-linear phenomenon governing the trade-off between reward and information gain.
4.1 Multi-threshold Models and Regret Bounds
We now introduce the class of multi-threshold censorship models deﬁned as:
[Equation]
where (  j ) j  k +1 is an increasing {+sequence verifying  
 =  ⇡ 2 ,  k +1 = ⇡ 2 and u 2 R d is aunit vector. We assume that ( p j ) j  k is decreasing, i.e. the censorship is increasing with j indirection+} u .
Henceforth, we refer to the interval [sin(  j ) , sin(  j +1 )
[ as region j .
Note thatsimple models such as uniform censorship aresubsumed by this family (for k equals 0 ).
Furthermore, MT can be seen as a piecewise constant approximation of any generalized linearmodel [28].
Thus, we do not see this modelingabstraction as an inherently limiting factor onthe generality of our subsequent results.
Moreover, MT admits a natural behavioral interpretation: Such a distribution can be seenas induced by a population model of heterogeneous random-utility maximizers agents.
A single threshold model (i.e. k equals 1 ) correspondsto a given agent type, and the multi-threshold model naturally results from aggregate responses ofheterogeneous population [4].
We now state the main result of this section:
Theorem 4.1.
For a given multi-threshold censorship model MT , there exits d e ↵ such that the UCBalgorithm with regularization  has an instance-independent expected regret of:
[Equation]
Note that the mapping from the original dimension d to the enlarged p d · d e ↵ is more surprising 246 than previous dilation d 7! d e ↵ in the case of MAB problems.
An extension to Generalized 247 Linear Contextual Bandits is provided in the SI where we show that the dimension is governed byp d · d e ↵ /  , with  corresponding to a minimum of the derivative of the link function (encompassingthe smoothness of the generalized linear model at its maximum) [25, 14].
4.2 Generalized Cumulative Censored Potential
Analogous to the MAB case, we now introduce for LCB the random matrices corresponding tothe effective realization W Ct ,  I d +
P tn =
1 x a t a t a > t and the expected realization W t ,  I d +
P tn =1 p ( a t ) a t a > t .
We also introduce the continuous counterpart of W t deﬁned as W ( t ) ,  I d +
[-λ I d + 253 (cid:82) t u =
-]
R t u =0 p ( a ( u )) a ( u ) a ( u ) > @ u , where ( a ( u ))
u  T is an integrable deterministic path.
1 We emphasisethat the use of continuous counterpart is key in enabling our next results.
As in the MAB case, webound the regret although now using a generalization of V ↵ :
Lemma 4.2.
For all  2 ]0 , 1] , there exists a constant ˜   ( T ) = ⇥
( p d log( T )) such that
[Equation]
where, for ↵ > 0 and ⇡ 2 ⇧ , the linear extension of the cumulative censored potential is given by:
[Equation]
The proof idea is analogous (albeit more complex) than in the ﬁnite action case (see SI).
In orderto get a handle on V ↵ , we again leverage a two-step approach: ﬁrst we eliminate the randomnessdue to censorship (here, we utilize matrix martingale inequalities) and then optimize the resultingdeterministic quantity seen through a continuous lens.
The ﬁrst step requires the following result:
Proposition 4.3.
For any  2 ]0 , 1] ,  > 0 , ↵ > 0 and policy ⇡ 2 ⇧ , we have:
[Equation]
where C (  ) , 8(  + 1) max(log( d/  )) /  , 1) /  .
Remark 1.
The key idea of this result is to observe that the telescopic sum on which the classical
Elliptical Potential lemma [1, 32, 8] heavily relies on is, in fact, the discrete approximation of anintegral over a matrix path.
For the simpler case of classical uncensored environment, we obtain for↵ > 0 , ↵ 6 = 1 :
[Equation]
For ↵ = 1 , a similar reasoning is applied using the formula Tr(log
( A )) =
log(det A ) :
[Equation]
A deeper study of the eigenvalues of W 1  ↵ T then yields the worst-case upper bound d ↵ ( d  +
T ) 1  ↵ / (1  ↵ ) for ↵ < 1 and d  1  ↵ / ( ↵  1) for ↵ > 1 , recovering more naturally and extending the results of [8].
Thus, analogous to the water ﬁlling process highlighted in the MAB case in
Lemma 3.5, we now consider a spectral water-ﬁlling process [12] for the eigenvalues of ↵ ( W T )with a slight abuse of notations ( W 1  ↵ T and log W T in this discussion).
One of the main challenge introduced by the censorship is therefore to identify a suitable matrixoperator on which the spectral maximization can be performed.
Motivated by Lemma 4.2, wehenceforth focus on the case of ↵ = 1 for which Prop. 4.3 implies that for any policy:
[Equation]
4.3 Effective Dimension in Linear Settings
Again, the notion of effective dimension naturally appears.
We now highlight intuition provided bythis quantity, and then present its general study for the multi-threshold model MT .
Remark 2.
Let us consider an uniform censorship model p : a 7! ¯ p .
By leveraging the case ofequality in the Arithmetic-Geometric inequality applied to the eigenvalues of W T , we then simplydeduce the associated effective dimension d e ↵ , d/ ¯ p :
[Equation]
We next illustrate the logarithmic scaling of this quantity in the general case as well as the importanceof the leading dimension factor, crudely upper bounded by d/p min in the next lemma:
Lemma 4.4.
For any censorship function p , by introducing lower and upper bounds ( p min , p max ) ofp , we have:
[Equation]
Related problems in the generalized linear models literature [43, 25, 14] are implicitly solved inthe spirit of Lemma 4.4, where a minimum of the derivative of the link function plays the role ofp min above.
However, when the function p varies with action a , a more careful analysis is requiredto derive useful dimensional bounds.
Our next major result addresses this gap in the literature byimproving the bounds provided in Lemma 4.4:
Theorem 4.5.
For a multi-threshold censorship model MT , we have:
[Equation]
where d e ↵ is the effective dimension.
Furthermore, d e ↵ is characterized by two cases:
• Case 1: Single region j effective dimension d e ↵ = dp j .
• Case 2: Bi-region ( i, j ) effective dimension, with i < j :
[Equation]
where l ( i, j ) , sin 2 (  i ) sin 2 (  j ) and u ( i, j ) , cos 2 (  i ) cos 2 (  j ) .
Before moving to a discussion of the proof idea behind Thm.
4.5, we mention a few importantremarks.
First, a necessary condition for the bi-region ( i, j ) effective dimension to arise is theconstraint on p i p j :
[Equation]
In the limit p i p j ! r ?
( i, j ) , d e ↵ goes again to d/p j .
We interpret this limiting case as locally hardin the sense that censorship in region j is sufﬁciently important in comparison to all other regionsto impose a maximal effective dimension to the problem, irrespective of the values of p i , matching
Lemma 4.4.
On the other hand, for the other limiting case (under additional mild assumptions),we ﬁnd that d e ↵ also goes to d/p j , but now for a uniformly hard reason: that is, censorship isapproximately constant and equal to p j , recovering the Remark 2.
Note that in between these twoextremes lies the minimum effective dimension for a given value of p i
p j .
4.4 Temporal dynamics of W ( t )
We ﬁnd that the dynamics of the optimal policy of ( P ) are described through W ( t ) by two qualitativelydifferent regimes and conclude this section by outlining them.
We emphasize that our continuousapproach to analyzing cumulative censored potential is key to obtaining these results.
Transient Regime: There exists a decreasing sequence of censorship regions { i 1 = k, . .
, i l } oflength l 2 [ k +1] and associated time sequence { t 0 , 0 , t 1 , . . .
, t l } such that whenever t j  t  t j +1 314 for a given index j  l  1 , the evolution of W ( t ) is given by:
[Equation]
where W i denotes the d ⇥ d diagonal matrix diag ( cos 2 (  i ) d  1 , . .
, cos 2 (  i ) d  1 , sin 2 (  i )) .
Interestingly, theinitial misspeciﬁcation of censorship is self-corrected during this transient step but at an extra cost.
This characterization of transient regime highlights an important consequence of using classicalalgorithms in censored environments.
Steady State Regime: Post-transient regime, the dynamics of W ( t ) enter a steady state regime,where one of the two cases necessarily arise: 3 .
• Case 1: Single region i l .
This case arises when the last element of the time sequence t l is equal to + {+
+} and we have the single region evolution for all t  t l
[Equation]
The effective dimension corresponding to this dynamics is d/p i l , with the following equalityfor T  t l  1 :
[Equation]
• Case 2: Bi-region ( i l +1 , i l ) .
This case arises when the steady-state dynamics of W ( t ) span
the two regions ( i l +1 , i l ) with i l +1 < i l .
For all
t  t l we have the evolution:
[Equation]
where  ? and the proportionality factor are speciﬁed in SI.
The corresponding effectivedimension is given by ( D ) and the following equality holds for all T  t l :
[Equation]
Concluding Remarks
In this work, we demonstrate that the complexity of bandit learning under censorship is governed bythe notion of effective dimension.
To do so, we developed a novel analysis framework which enablesus to precisely estimate this quantity for a broad class of multi-threshold censorship models.
Animportant future work would be to extend our model and approach to Bayesian settings, which willlikely provide us with useful insights on the cumulative censored potential V ↵ , as initiated by [15].
Future work also includes the study of time-dependent censorship models such as Markov Decision
Processes.
We do not envision any negative societal impacts of our work other than that of bandits algorithmsdeployed in AI-driven platforms.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sec.
and 4 as well as SI.
(b) Did you include complete proofs of all theoretical results?
[Yes] See SI.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[N/A]
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e
g., typeof GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
) Did you mention the license of the assets?
[N/A]
) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
) Did you discuss whether the data you are using/curating contains personally identiﬁable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]