Effective Dimension in Bandit Problems under Censorship
Anonymous Author(s)
AffiliationAddress email
Abstract
In this paper, we study both multi-armed and contextual bandit problems in censored environments.
Our goal is to estimate the performance loss due to censorshipin the context of classical algorithms designed for uncensored environments.
Ourmain contributions include the introduction of a broad class of censorship modelsand their analysis in terms of the effective dimension of the problem – a naturalmeasure of its underlying statistical complexity and main driver of the regret bound.
In particular, the effective dimension allows us to maintain the structure of theoriginal problem at first order, while embedding it in a bigger space, and thusnaturally leads to results analogous to uncensored settings.
Our analysis involves acontinuous generalization of the Elliptical Potential Inequality, which we believeis of independent interest.
We also discover an interesting property of decision- making under censorship: a transient phase during which initial misspecificationof censorship is self-corrected at an extra cost; followed by a stationary phase thatreflects the inherent slowdown of learning governed by the effective dimension.
Our results are useful for applications of sequential decision-making models wherethe feedback received depends on strategic uncertainty (e.g., agents’ willingness tofollow a recommendation) and/or random uncertainty (e.g., loss or delay in arrivalof information).
Introduction
Bandit problems are prototypical models of sequential decision-making under uncertainty.
Theyare widely studied due to their applications in recommender systems, online advertising, medicaltreatment assignment, revenue management, network routing and control [21, 33].
Our work ismotivated by settings in which the feedback received by the decision-maker in each round of decisionis censored by a stochastic process that depends on the current action as well as past history offeedbacks and actions.
For instance, in typical missing data problems, the decision-maker needs todeal with frequent losses of information (or delays in arrival of information) due to exogeneous failuressuch as faulty and/or unreliable communication.
Missing observations in dynamical interactionswith the environment are a common concern in diverse fields ranging from operations managementto health sciences to physical sciences [39, 15, 24].
In other settings, such as AI-driven platformsfor health alerts, route guidance, and product recommendations [6, 40], the reception of feedbackdepends on whether or not the decision (or recommendation) is adopted by strategic agents (e.g.patients, customers or drivers) with private valuations.
Thus, from the platform’s viewpoint, theadoption behavior of heterogeneous agents can be regarded as a stochastic censorship process.
In static environments, the bias induced by the presence of randomly missing information has beenthoroughly studied [24, 28].
However, in online settings, the dynamics of learning and acting areinherently coupled: since censorship mediates current information of the environment, it impacts the outcome of data-driven decision process; this in turn conditions the future decisions and futurecensored feedback, creating a complex and endogenous joint temporal dependency.
Our workcontributes to the analysis of such phenomena for a broad classes of decision and censorship models.
Importantly, it is the first normative inquiry of how censorship impacts the statistical complexity ofbandit problems.
We develop an analysis approach that is useful for both estimating the performanceloss due to censorship and refining the classical algorithms designed for uncensored environments.
1.1 Related Work
Within the extensive bandits literature, well-surveyed in [21, 33], our work is most closely relatedto stochastic delayed bandits.
Initially, this line of work focused on the joint evolution of actionsand information in settings where the reception of the latter is delayed [12].
Of particular interest isthe packet loss model recently introduced in [ 19 ] , which provides the regret bound O ( 1 p R T ) where
R T is the uncensored regret and p the censorship probability.
Analogous results have been shownin the context of Combinatorial Multi-Armed Bandits with probabilistically triggered arms; see forexample, [9] and [37].
Our work provides a systematic approach to study more general censorshipmodels, and sheds light on how the impact of coupled feedback and censorship realizations on theexpected regret can be evaluated in terms of the effective dimension of the problem.
Importantly, we also tackle the contextual bandit problems, where relatively few results are availableon the regret under missing or censored feedback.
A notable exception is the work of [36], who focuson a different information structure and obtain a scaling of 1 /p (see Remark 2).
A related contributionby [2] provides both a potential-based analysis of UCB for multi-armed bandits and an algorithmicvariant leveraging Kaplan-Meier estimator, although their censorship setting is different than ours.
Inparticular, our results are applicable to settings when delay is significantly large (possibly infinite).
This is in contrast to prior results on bandits with delayed information structure which assume eitherthat the delay is constant , upper bounded , has a finite mean , or simply provide regret guarantees thatare linear in the cumulative delays up to time T [12, 17, 25, 41, 29].
Under such assumptions ondelay, one usually gets an additive dependency of the regret in terms of delay parameters.
The abovementioned works primarily focus on modifying well-known bandit algorithms to accountfor delays, or propose new delay-robust algorithms which may be difficult to implement in practice; anotable exception includes [38].
In our work, we instead focus on estimating the performance loss dueto censorship and derive insights on the behavior of well-known UCB class of algorithms [22, 10, 1].
These algorithms are widely used in practice; moreover, their theoretical study has been shownto be useful for analysis of broader class of algorithms (notably Thompson Sampling [3, 31] and
Information-Directed Sampling [32, 18]).
Our work contributes to the Generalized Linear Contextual Bandits literature [13, 23] in two ways:
Firstly, through the use of these models in a sequential decision-making framework on which theimpact of censorship is assessed in Sec. 4.
Secondly, by showing that our multi-threshold censorshipmodel MT induces, at first order, a non-linear structure that closely mirrors such models.
Our resultsprovide new tools to study this structure.
It is useful to note that the notion of effective dimension hasbeen well-studied in the statistical learning and kernels literature [34, 35] (where it is defined for a
Gram matrix K n and regularization λ as d n eff ( λ ) =
tr ( K n ( K n + λ I d ) − 1 ) ).
Our work shows that ananalogous quantity governs the regret bound of bandit problems in censored settings.
The literature on non-stochastic multi-armed bandit problems with delays problems [27, 8, 16] alsotackles multiplicative dependency, although in a different setting than ours.
Another related line ofwork is Partial Monitoring [5, 20] which deals with generic categorization of learnability, rather thana fine-grained analysis of dimensionality in relation to censorship.
Finally, there is a rich literature onclassical missing and censored data problems, well surveyed in [24] or [28].
Our work contributes tothis broad field since we deal with endogenous and sequentially generated censorship process.
1.2 Summary of Results
In Sec. 3, we consider Multi-Armed Bandit (MAB) models and prove that the regret scales as˜ O ( d eff √ T )
(Thm. 3.1), where d eff is the effective dimension with value (cid:80)
a ∈ [ d ] 1 p a .
In doing so, werecover and generalize related results from [19, 9] to more complex regularized settings and noisemodels.
In particular, we prove that the effective dimension results from characterizing the so-calledcensored cumulative potential V α .
Interestingly, we also show that the adaptive nature of censorship on V α plays only a second order role (Prop. 3.7), that is, impact of censorship can be treated in anoffline manner at first order.
Importantly, our study of MAB under censorship instantiates an analysis framework which extendsto Linear Contextual Bandits (LCB)
(Sec. 4).
Our main result provides that regret is still governedby the effective dimension, but now with a dependency of ˜ O
( σ (cid:112) d · d eff √ T )
(Thm. 4.1).
To thebest of our knowledge, these regret bounds provide the first theoretical characterization in LCB withcensorship, and contribute to the literature by evaluating the impact of censorship on the performanceof UCB-type algorithms.
Our second main contribution is identifying the effective dimension fora broad class of multi-threshold models MT as well as a precise understanding of the dynamicbehavior induced by these models (Thm. 4.5).
In particular, we find that censorship introduces a twophase behavior: a transient phase during which the initial censoring misspecification is self-correctedat an additional cost; followed by a stationary phase that reflects the inherent slowdown of learninggoverned by the effective dimension.
In extending our analysis from MAB to LCB, we also developa continuous generalization of the widely used Elliptical Potential Inequality (Prop. 4.3), which webelieve is also of independent interest.
Problem Setup and Background
Bandit Model: We successively consider stochastic multi-armed bandits (MAB) (Sec. 3) and
Linear Contextual Bandits (LCB) (Sec. 4) in censored environments.
In both settings, at each roundt ≤ T , the agent observes an action set A t ⊂ A .
She then selects an action a t ∈
A t (i.e. an arm ) towhich a noisy feedback r ( a t ) + ϵ t is associated, where r ( a t ) is a bounded reward and ϵ t is an i.i.d.sub-Gaussian noise of pseudo-variance σ 2 .
For action a , the sub-optimality gap at time t is denoted∆ t ( a ) ≜ max ˜ a ∈A t r (˜ a ) − r ( a ) , and the maximal gap ∆ max ≜ max a,t ∆ t ( a ) .
We now recall thespecifics of each model:
• MAB : There is a finite number of actions d , enumerated as A ≜ [ d ] , each having a scalar
reward θ ⋆a .
Arms are independent : playing one arm gives no information about the others.
• LCB : The action set A t is a subset of the unit ball B d , possibly infinite.
Non-stochastic
contexts are modeled by the fact that A t is drawn by an oblivious adversary.
Here one does
not need to rely on the typical i.i.d assumption on their generating process [41, 13].
Unless
explicitly mentioned, the reward is assumed to be linear with respect to
a latent unknown
vector θ ⋆ ∈ R d , i.e., r ( a ) =
⟨ a, θ ⋆ ⟩ .
Information Structure:
In the classical uncensored setting, the noisy feedback is immediatelyobserved post-decision and utilized to make decisions in the next round.
We introduce the followingcensorship model: an independent Bernoulli random variable of parameter p a t denoted as x a t isdrawn after each decision a t and the feedback is observed, i.e., realized , if and only if x a t = 1 ; elsethe feedback is said to be censored .
We recover the uncensored setting when p a ≡ 1 .
Algorithm 1: Generic UCB
Input: Total time T , Regularization λ , Precision δ for t = 1 , . .
, T do
Provide gap estimator ˜∆ λt verifying w.p. 1 − δ : ∀ a ∈
A t , ∆ t ( a ) ≤ ˜∆ λt ( a ) ;
Play action a t = argmin a ∈A t ˜∆ λt ( a ) ; if ( a t , r ( a t ) + ϵ t ) is realized i.e. x a t = 1 then
Update ˜∆ λt ; end end
Algorithms: To study the impact of censorship on bandit problems, we consider the class of high-probability index algorithms based on the optimism under uncertainty principle, commonly referred as UCB -algorithms.
Following [18], Algorithm 1 summarizes the generic UCB design framework.
We detail in Supplementary Information (SI) the specific instances of UCB for MAB (resp.
LCB) used in Sec.3 (resp. Sec.4).
Performance Criterion: The frequentist performance of the agent is measured by the notion ofpseudo regret , i.e., the difference between the algorithm’s cumulative reward and the best total reward.
More formally, we introduce for any policy π ∈ Π :
R ( T, π ) = T (cid:88) t = max a ∈A t r ( a ) − T (cid:88) t = r ( a t ) = T (cid:88) t
= ∆ t ( a t ) . We aim to provide guarantees on E [ R ( T, π )] with respect to the number of rounds T and quantitiesthat govern the complexity of the problem (for example number of arms, ambient dimension d ,parameters of censorship model or smoothness properties of the reward r ).
Here, the expectation iswith respect to the noise induced by the feedback, the censorship and a possibly randomized policy. Notations
Transpose of a vector u is denoted by u ⊤ , classical Euclidean inner product by ⟨ ., .
⟩ and trace operatorby Tr .
For positive semi-definite matrix Σ ∈ R d × d and for any vector u ∈ R d , notation ∥ u ∥ Σ refersto √ u ⊤ Σ u .
We use notation I d to denote the d × d identity matrix.
B d is the unit ball in R d .
[ n ]is the set of integers { 1 , 2 , · · · , n } .
For a given function f , we note f ( i ) the i th derivative of f .
Toavoid confusion with the dimension d , we use ∂x instead of dx to denote an infinitesimal increase ofx .
We use the asymptotic notations ∼ , O , Θ and ˜ O
( O when log factors are removed).
Finally, for anevent H , we use ¬H to denote its complement.
Multi-Armed Bandits
3.1 Effective Dimension and Regret Bounds
The main result of this section is that censorship effectively enlarges the dimension of the problem.
We define the effective dimension as d eff ≜ (cid:80) a ∈ [ d ] 1 p a and our result (Thm. 3.1) shows that, at firstorder, the regret is guaranteed to be the same as the uncensored problem with d eff arms instead of d .
Theorem 3.1.
Under censorship, the UCB algorithm with regularization of λ has an instanceindependent expected regret of:
[Equation]
Furthermore, we obtain analogous regret guarantees for instance-dependent cases where, at first order,the uncensored dimension (cid:80) a ̸ = a ⋆ σ 2 ∆ a enlarges to (cid:80) a ̸ = a ⋆ σ 2 p a ∆ a :
Proposition 3.2.
For a fixed action set A t ≡ [ d ] and for a-priori known action gap ∆ a ≜ max ˜ a θ ⋆ ˜ a −θ ⋆a , the UCB algorithm with regularization λ has the instance-dependent expected regret:
[Equation]
On one hand, a preliminary understanding of censorship posits an increase of the average " regret perinformation gain " [18] (as it takes longer on average to get the same amount of information) but doesnot change the underlying complexity of the problem.
One the other hand, our results (Thm. 3.1 and
Prop. 3.2) postulate that the censored problem is equivalent at first order to a higher dimensionalproblem but explored with the same " regret per information gain ".
The abovementioned results extends to a-priori known heteroskedasticity (see SI).
For this generalsetting, the effective dimension for instance-independent (resp.
dependent) case is given by (cid:80) a σ a p a(resp.
(cid:80) a ̸ = a ⋆ σ 2 a p a ∆ a ), where σ 2 a is the variance proxy of arm a .
Although the scaling in (cid:80) a 1∆ a p awas already mentioned in [19] for unregularized setting with homogeneous variance σ 2 and provento be optimal, our results generalize these findings.
3.2 Cumulative Censored Potential
We now provide a proof sketch of Thm. 3.1, and in doing so, we instantiate an analysis frameworkthat will be extended in Sec. 4.
This proof consists in the successive elimination of the noise induced by the feedback and censorship.
This leads to regret guarantees on a resulting deterministic quantityby characterizing worst-case learning conditions.
The first step of the proof is a variant of the classicalreduction of the UCB regret to another quantity we refer to as the expected cumulative censoredpotential .
Before stating it, we define at the end of a round t ∈ [ T ] , the random number of times anarm a has been pulled as τ a ( t ) ≜ (cid:80) tl =1 1 { a l = a } .
Similarly, the number of times an action a hasbeen realized at the end of round t is denoted N a ( t ) ≜ (cid:80) tl =1 1 { a l = a, x a l = 1 } .
We then have:
Lemma 3.3.
Given an uniform regularization of λ > 0 , the UCB algorithm verifies:
[Equation]
where, for any α > 0 and π ∈ Π , the cumulative potential under censorship is given by:
[Equation]
In contrast to the classical non-regularized analysis or to the LCB case of Sec. 4, we observe twodifferent orders of α ( 1 / 2 and 1 ) coming from the use of the L ∞ -norm instead of the L 2 -norm.
Takenindependently, they lead to respective contributions of O ( d eff log( T )) and O ( (cid:112) d eff T ) .
Note that byworking with general α , our analysis naturally extends beyond sub-Gaussian noise to more generalassumptions about the Laplace transform of noise (e.g., lighter or heavier tails).
To further study V α ,we introduce the following property:
Proposition 3.4.
For all α > 0 , δ ∈ ]0 , 1] and given ψ α a primitive of x (cid:55)→ x − α , we have:
[Equation]
The proof of this proposition involves two steps: firstly, we remove the stochastic dependence inducedby the censorship through concentration properties (See SI), and we then solve the resulting policymaximization problem (Lemma 3.5).
In the first step, we consider for a given δ ∈ ]0 , 1] the event:
[Equation]
where T 0 ( a ) ≜ 24 log( T ) /p a and claim that P ( H CEN ( δ )) ≤ 4 d eff δ 2 T − 12 δ 2 , improving a result of[19].
Our second step makes use of the following lemma (also known as a water-filling process ininformation theory [11]):
Lemma 3.5.
For ψ α a primitive of x (cid:55)→ x − α where α ∈ ]0 , 1] , regularization ( λ a ) a ∈ [ d ] ∈ ( R > 0 ) dand censorship vector ( p a ) a ∈ [ d ] , the solution of the optimization problem:
[Equation]
is given by τ ⋆a = 1 p a [ C − λ a ] + , where C ensures the total budget constraint (cid:80) a ∈
[ d ] τ ⋆a = T .
Inparticular, with λ eff ≜ 1 d eff (cid:80)
a ∈ [ d ] λ a p a and λ 0 a ≜ d eff ( λ a − λ eff ) , the optimal solution is given byτ ⋆ a ≜ 1 p a d eff ( T − λ 0 a ) for T ≥ max a λ 0 a and the optimal value is d eff ψ α
( Td eff + λ eff ) − (cid:80) a ∈ [ d ] 1 p a ψ α ( λ a ) .
For unregularized algorithms, this framework can be easily applied to provide instances-dependentguarantees by adding constraints of type τ a ≤ f (∆ a ) within Lemma 3.5.
Optimal guarantees underregularization such as the ones given in Prop. 3.2 require however to consider both orders of V α ( 1 / 2and 1 ) simultaneously and not independently, leading to slight variations as shown in the proof of
Prop. 3.2 in SI.
Next, we further discuss the properties of V α given its importance in our analysis.
3.3 Evaluating Adaptivity Gain
In fact, through the introduction of H CEN ( δ ) and for any α ∈ [0 , 1] , δ ∈ ]0 , 1] , we showed in Prop.3.4 the upper bound
[Equation]
for
[Equation]
π )] where the maximum is taken overthe class of adaptive policies Π adapt , i.e., measurable with respect to the censorship.
Note that theexact value of such maximum is notoriously difficult to study due to the adaptive nature of censorshipinduced by the decision-making process.
Interestingly, we obtain a surprising result that the gain dueto adaptivity is not significant.
To show this, we introduce Π off , the class of policies that are notadaptive with respect to the censorship and we prove that :
Lemma 3.6.
For α ∈ ]0 , 1] and λ > 0 , we have max π ∈ Π off E [ V α ( T, π )] ∼ d [Equation]
[Equation]
+ λ ) .
In other words, restricting attention to offline policies is sufficient to obtain the correct scaling.
Thenext step to complete our claim is the asymptotic expansion:
Proposition 3.7.
For α ∈ ]0 , 1] , by denoting γ α ( p ) ≜ α
[Equation]
− [Equation] (cid:88) a ∈ [ d ] [Equation]
(cid:88)
[Equation]
, we have:
[Equation]
Moreover, if for a given β ∈ ]0 , 1[ , we introduce Π single ( βT ) the policy class whose censorshipinformation set has a single updating at time ⌊ βT ⌋ , we have:
[Equation]
Thus, γ α ( p ) can be viewed as an adaptivity gain resulting from the continuous correction of thecumulative variance induced by the action selection process.
Essentially, it is closely related tothe Jensen Gap of an appropriate random variable and the proof involves the study of the Taylorexpansion of the potential function ψ α .
( ⋆⋆ ) tells us that a single observation of the censorshiprealization is sufficient to obtain a near-optimal gain in adaptivity .
We present a proof sketch of Prop.3.7 in SI.
This shows that censorship in MAB can be treated in an offline manner at first order.
Contextual Bandit
In this section, we study Linear Contextual Bandits under censorship.
The regret analysis for thisgeneral setting under censorship is significantly more complex than for the MAB model.
This is dueto the fact that different actions contribute differently to the information acquisition, leading to anon-linear phenomenon governing the trade-off between reward and information gain.
4.1 Multi-threshold Models and Regret Bounds
We now introduce the class of multi-threshold censorship models defined as:
[Equation]
ing, increasing [-sequence verifying  
 =  ⇡ 2 ,  k +1 = ⇡ 2 and u 2 R d is aunit vector. We assume that ( p j ) j  k is decreasing, i.e. the censorship is increasing with j indirection-]direction u .
Henceforth, we refer to the interval [sin( ϕ j ) , sin( ϕ j +1 )
[ as region j .
Note thatsimple models such as uniform censorship aresubsumed by this family (for k equals 0 ).
Furthermore, MT can be seen as a piecewise constant approximation of any generalized linearmodel [26].
Thus, we do not see this modelingabstraction as an inherently limiting factor onthe generality of our subsequent results.
Moreover, MT admits a natural behavioral interpretation: Such a distribution can be seenas induced by a population model of heterogeneous random-utility maximizers agents.
A single threshold model (i.e. k equals 1 ) correspondsto a given agent type, and the multi-threshold model naturally results from aggregate responses ofheterogeneous population [4].
We now state the main result of this section:
Theorem 4.1.
For a given multi-threshold censorship model MT , there exits d eff such that the UCBalgorithm with regularization λ has an instance-independent expected regret of:
[Equation]
Note that the mapping from the original dimension d to the enlarged (cid:112) d · d eff is more surprisingthan previous dilation d (cid:55)→ d eff in the case of MAB problems.
An extension to Generalized Linear Contextual Bandits is provided in the SI where we show that the dimension is governed by 247 (cid:112) d · d eff /κ , with κ corresponding to a minimum of the derivative of the link function (encompassingthe smoothness of the generalized linear model at its maximum) [23, 13].
4.2 Generalized Cumulative Censored Potential
Analogous to the MAB case, we now introduce for LCB the random matrices corresponding tothe effective realization W Ct ≜ λ
I d + (cid:80) tn
=1 x a t a t a ⊤ t and the expected realization W t ≜
λ I d + 252 (cid:80) tn
=1 p ( a t ) a t a ⊤ t .
We also introduce the continuous counterpart of W t defined as W ( t ) ≜
{+λ I d + 253 (cid:82) t u =
+}
p ( a ( u )) a ( u ) a ( u ) ⊤ ∂u , where ( a ( u ))
u ≤ T is an integrable deterministic path.
1 We emphasisethat the use of continuous counterpart is key in enabling our next results.
As in the MAB case, webound the regret although now using a generalization of V α :
Lemma 4.2.
For all δ ∈ ]0 , 1] , there exists a constant ˜ β δ ( T ) =
Θ( (cid:112) d log( T )) such that
[Equation]
where, for α > 0 and π ∈ Π , the linear extension of the cumulative censored potential is given by:
[Equation]
The proof idea is analogous (albeit more complex) than in the finite action case (see SI).
In orderto get a handle on V α , we again leverage a two-step approach: first we eliminate the randomnessdue to censorship (here, we utilize matrix martingale inequalities) and then optimize the resultingdeterministic quantity seen through a continuous lens.
The first step requires the following result:
Proposition 4.3.
For any δ ∈ ]0 , 1] , λ > 0 , α > 0 and policy π ∈ Π , we have:
[Equation]
where C ( δ ) ≜ 8( λ + 1) max(log( d/δ )) /λ, 1) /λ .
Remark 1.
The key idea of this result is to observe that the telescopic sum on which the classical
Elliptical Potential lemma [1, 30, 7] heavily relies on is, in fact, the discrete approximation of anintegral over a matrix path.
For the simpler case of classical uncensored environment, we obtain forα > 0 , α ̸ = 1 :
[Equation]
For α = 1 , a similar reasoning is applied using the formula Tr(log
( A )) =
log(det A ) :
[Equation]
A deeper study of the eigenvalues of W 1 − α T then yields the worst-case upper bound d α ( dλ +
T ) 1 − α / (1 − α ) for α < 1 and dλ 1 − α / ( α − 1) for α > 1 , recovering more naturally and extending the results of [7].
Thus, analogous to the water filling process highlighted in the MAB case in
Lemma 3.5, we now consider a spectral water-filling process [11] for the eigenvalues of ψ α ( W T )with a slight abuse of notations ( W 1 − α T and log W T in this discussion).
One of the main challenge introduced by the censorship is therefore to identify a suitable matrixoperator on which the spectral maximization can be performed.
Motivated by Lemma 4.2, wehenceforth focus on the case of α = 1 for which Prop. 4.3 implies that for any policy:
[Equation]
4.3 Effective Dimension in Linear Settings
Again, the notion of effective dimension naturally appears.
We now highlight intuition provided bythis quantity, and then present its general study for the multi-threshold model MT .
Remark 2.
Let us consider an uniform censorship model p : a (cid:55)→ ¯ p .
By leveraging the case ofequality in the Arithmetic-Geometric inequality applied to the eigenvalues of W T , we then simplydeduce the associated effective dimension d eff ≜ d/ ¯ p :
[Equation]
We next illustrate the logarithmic scaling of this quantity in the general case as well as the importanceof the leading dimension factor, crudely upper bounded by d/p min in the next lemma:
Lemma 4.4.
For any censorship function p , by introducing lower and upper bounds ( p min , p max ) ofp , we have:
[Equation]
Related problems in the generalized linear models literature [41, 23, 13] are implicitly solved inthe spirit of Lemma 4.4, where a minimum of the derivative of the link function plays the role ofp min above.
However, when the function p varies with action a , a more careful analysis is requiredto derive useful dimensional bounds.
Our next major result addresses this gap in the literature byimproving the bounds provided in Lemma 4.4:
Theorem 4.5.
For a multi-threshold censorship model MT , we have:
[Equation]
where d eff is the effective dimension.
Furthermore, d eff is characterized by two cases:
• Case 1: Single region j effective dimension d eff = dp j .
• Case 2: Bi-region ( i, j ) effective dimension, with i < j : [Equation] p [Equation]
( D
[Equation]
where l ( i, j ) ≜ sin 2 ( ϕ i ) sin 2 ( ϕ j ) and u ( i, j ) ≜ cos 2 ( ϕ i ) cos 2 ( ϕ j ) .
Before moving to a discussion of the proof idea behind Thm.
4.5, we mention a few importantremarks.
First, a necessary condition for the bi-region ( i, j ) effective dimension to arise is theconstraint on p i p j :
[Equation]
In the limit p i p j
→ r ⋆ ( i, j ) , d eff goes again to d/p j .
We interpret this limiting case as locally hardin the sense that censorship in region j is sufficiently important in comparison to all other regionsto impose a maximal effective dimension to the problem, irrespective of the values of p i , matching
Lemma 4.4.
On the other hand, for the other limiting case (under additional mild assumptions),we find that d eff also goes to d/p j , but now for a uniformly hard reason: that is, censorship isapproximately constant and equal to p j , recovering the Remark 2.
Note that in between these twoextremes lies the minimum effective dimension for a given value of p i
p j .
4.4 Temporal dynamics of W ( t )
We find that the dynamics of the optimal policy of ( P ) are described through W ( t ) by two qualitativelydifferent regimes and conclude this section by outlining them.
We emphasize that our continuousapproach to analyzing cumulative censored potential is key to obtaining these results.
Transient Regime: There exists a decreasing sequence of censorship regions { i 1 = k, . .
, i l } oflength l ∈ [ k +1] and associated time sequence { t 0 ≜ 0 , t 1 , . . .
, t l } such that whenever t j ≤ t ≤ t j +1for a given index j ≤ l − 1 , the evolution of W ( t ) is given by:
[Equation]
where W i denotes the d × d diagonal matrix diag ( cos 2 ( ϕ i ) d − 1 , . .
, cos 2 ( ϕ i ) d − 1 , sin 2 ( ϕ i )) .
Interestingly, theinitial misspecification of censorship is self-corrected during this transient step but at an extra cost.
This characterization of transient regime highlights an important consequence of using classicalalgorithms in censored environments.
Steady State Regime: Post-transient regime, the dynamics of W ( t ) enters a steady state regime,where one of the two cases necessarily arise: 3 .
• Case 1: Single region i l .
This case arises when the last element of the time sequence t l isequal to + [-
-]∞ and we have the single region evolution for all t ≥ t l − 1 :
[Equation]
The effective dimension corresponding to this dynamics is d/p i l , with the following equalityfor T ≥ t l − 1 : [Equation] T
[Equation]
• Case 2: Bi-region ( i l +1 , i l ) .
This case arises when the steady-state dynamics of W ( t span
the two regions ( i l +1 , i l ) with i l + <
i l For
all t ≥ t l , we have the evolution:
[Equation]
where λ ⋆ and the proportionality factor are specified in SI.
The corresponding effectivedimension is given by ( D ) and the following equality holds for all T ≥ t l :
[Equation]
Concluding Remarks
In this work, we demonstrate that the complexity of bandit learning under censorship is governed bythe notion of effective dimension.
To do so, we developed a novel analysis framework which enablesus to precisely estimate this quantity for a broad class of multi-threshold censorship models.
Animportant future work would be to extend our model and approach to Bayesian settings, which willlikely provide us with useful insights on the cumulative censored potential V α , as initiated by [14].
Future work also includes the study of time-dependent censorship models such as Markov Decision
Processes.
We do not envision any negative societal impacts of our work other than that of bandits algorithmsdeployed in AI-driven platforms.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Sec.
and 4 as well as SI.
(b) Did you include complete proofs of all theoretical results?
[Yes] See SI.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[N/A]
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[N/A]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)?
[N/A]
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[N/A]
b) Did you mention the license of the assets?
[N/A]
c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable
N
Did you describe any potential participant risks, with links to Institutional Review
) approvals, if applicable
[N
Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]