Greedy Approximation Algorithms for Active Sequential Hypothesis Testing

1 Introduction . Consider the problem of learning the true hypothesis from among a ( potentially large ) set of candidate hypotheses H . Assume that the learner is given a ( potentially large ) set of actions A , and knows the distribution of the noisy outcome of each action , under each potential hypothesis . The learner incurs a fixed cost each time an action is selected , and seeks to identify the true hypothesis with sufficient confidence , at minimum total cost . Finally , and most importantly , the learner is allowed to select actions adaptively . This well-studied problem is referred to as active sequential hypothesis testing , and as we will describe momentarily , there exists a broad set of results that tightly characterizes the optimal achievable cost under various notions of adaptivity . Unfortunately , the corresponding optimal policies are typically only characterized as the optimal policy to a Markov decision process ( MDP ) —thus , they remain computationally hard to compute when one requires a policy in practice . This deficiency becomes particularly apparent in modern applications where both the set of hypotheses and set of actions are large . As a concrete example , we will describe later on an application to cancer blood testing that has tens of hypotheses and billions of tests at full scale . Thus motivated , we provide the first approximation algorithms for ASHT . We study ASHT under two types of adaptivity : partial and full , where partial adaptivity requires the sequence of actions to be decided upfront ( with adaptively chosen stopping time ) , and full adaptivity allows the choice of action to depend on previous outcomes . For both problems , we propose greedy algorithms that run in O ( |A||H| ) time , and prove that their expected costs are upper bounded by a non-trivial multiplicative factor of the corresponding optimal costs . Most notably , these ⇤Equal contribution . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . approximation guarantees are independent of |A| ( contrast this with the trivially-achievable guarantee of O ( |A| ) ) and logarithmic in |H| ( the optimal cost itself is often ⌦ ( |H| ) ) . Our results rely on drawing connections to two existing problems : submodular function ranking ( SFR ) [ 5 ] and the optimal decision tree ( ODT ) problem [ 30 ] . These connections allow us to tackle what is arguably the primary challenge in achieving approximation results for ASHT , which is its inherent combinatorial nature . We will argue that existing heuristics from statistical learning fail precisely because they disregard this combinatorial difficulty—indeed , they largely amount to solving the completely non-adaptive version of the problem . At the same time , existing results for SFR and ODT fail to account for noise in a manner that would map directly to ASHT—this extension is among our contributions . Related Work Our work is closely related to three streams of research . Table 1 highlights the key differences between our contributions and those of the most relevant previous works . ( a ) Hypothesis Testing and Asymptotic Performance : In the classical binary sequential hypothesis testing problem , a decision maker is provided with one action whose outcome is stochastic [ 42 , 3 , 31 ] , and the goal is to use the minimum expected number of samples to identify the true hypothesis subject to some given error probability . The ASHT problem , first studied in [ 14 ] , generalizes this problem to multiple actions . Most related to our work is [ 34 ] , who formulated a similar problem as an MDP . We will postpone describing and contrasting their work until the experiments section . ( b ) Active Learning and Sample Complexity : In active learning , the learner is given access to a pool of unlabeled samples ( cheaply obtainable ) and is allowed to request the label of any sample ( expensive ) from that pool . The goal is to learn an accurate classifier while requesting as few labels as possible . Some nice surveys include [ 23 ] and [ 39 ] . Our model extends the classical discrete active learning model [ 17 ] in which outcomes are noiseless ( deterministic ) for any pair of hypothesis and unlabeled sample . When outcomes are noisy , the majority of provable guarantees are provided via sample complexity . [ 9 ] showed tight minimax classification error rates for a broad class of distributions . Other sample complexity results on noisy active learning include [ 43 , 36 , 7 , 4 , 24 ] . ( C ) Approximation Algorithms for Decision Trees : Nearly all optimal approximation algorithms for minimizing cover time are known in the noiseless setting [ 28 , 1 , 2 ] . When the outcome is stochastic , [ 21 ] proposed a framework for analyzing algorithms under the adaptive submodularity assumption . However , their assumption does not hold for many natural setups including ASHT . [ 13 ] considered a variant using ideas from the submodular max-coverage problem , and provided a constant factor approximation to the problem . Other works based on submodular function covering include [ 35 , 22 , 29 ] . [ 26 ] provided approximation ratios under the constraint that the algorithm may only terminate when it is completely confident about the outcome . 2 Model . We begin by formally introducing the problem . Let H be a finite set of hypotheses , among which exactly one is the ( unknown ) true hypothesis that we seek to identify . In this paper , we study the Bayesian setting , wherein this true hypothesis is drawn from a known prior distribution ⇡ over H . Let A be the set of available actions . Selecting an action yields a random outcome drawn independently from a distribution within a given family D = { D✓ } ✓2⇥ of distributions parameterized by ⇥ ✓ R. We are given a function µ : H ⇥A ! ⇥ such that if h 2 H is the underlying hypothesis and we select action a 2 A , then the random outcome is drawn independently from distribution Dµ ( h , a ) .2 An instance of the active sequential hypothesis testing problem is then fully specified by a tuple : ( H , A , ⇡ , µ , D ) . The goal is to sequentially select actions to identify the true hypothesis with “ sufficiently high ” confidence , at minimal expected cost , where cost is measured as the number of actions , and the expectation is with respect to the Bayesian prior and the random outcomes . The notion of sufficiently high confidence is encoded by a parameter 2 ( 0 , 1 ) , and requires that under any true h 2 H , the probability of erroneously identifying a different hypothesis is at most . An algorithm which satisfies this is said to have achieved -PAC-error . We focus on two important families of D✓ ’ s : the Bernoulli distribution Ber ( ✓ ) and the Gaussian distribution N ( ✓ , 2 ) where 2 is a known constant ( with respect to ✓ ) . By re-scaling , without loss of generality we may assume 2 = 1 . We require two additional assumptions to state our guarantees . The first assumption is needed for relating the sub-gaussian norm to the KL-divergence , in the partially adaptive version . It ensures that the parameterization ⇥ is a meaningful one , in the sense that if ✓ , ✓0 2 ⇥ are far apart , then the distributions D✓ and D✓0 are also “ far ” apart ( as measured by KL divergence ) . Assumption 1 is satisfied for Ber ( ✓ ) when ✓ 2 [ ✓min , ✓max ] for some constants 0 < ✓min < ✓max < 1 , and for N ( ✓ , 1 ) where ✓ lies in some bounded set in R. Assumption 1 . There exist C1 , C2 > 0 such that for any ✓ , ✓0 2 ⇥ , we have C1 ·KL ( D✓ , D✓0 ) ( ✓ ✓0 ) 2 C2 ·KL ( D✓ , D✓0 ) , where KL ( · , · ) is the Kullback-Leibler divergence . Our second major assumption simply ensures the existence of a valid algorithm , by assuming that every hypothesis is distinguishable via some action . Assumption 2 ( Validity ) . For all g , h 2 H where g 6= h , there exists a 2 A with µ ( g , a ) 6= µ ( h , a ) . In particular , we do not preclude the possibility that for a given action a , there exist ( potentially many ) pairs of hypotheses g , h such that µ ( g , a ) = µ ( h , a ) . In fact , eliminating such possibilities would effectively wash out any meaningful combinatorial dimension to this problem . On the other hand , any approximation guarantee should be parameterized by some notion of separation ( when it exists ) . For any two hypotheses g , h 2 H and any action a 2 A , define d ( g , h ; a ) = KL Dµ ( g , a ) , Dµ ( h , a ) . Definition 1 ( s-separated instance ) . An ASHT instance is said to be s-separated , if for any a 2 A and g , h 2 H , d ( g , h ; a ) is either 0 or at least s. Note that in real-world applications , the parameter s could be arbitrarily small , and we introduce the notion of s-separability for the sake of proofs . We will show in Section 6 how our algorithms can easily be modified to handle small s values . In this work , we will study two classes of algorithms that differ in the extent to which adaptivity is allowed . Definition 2 . A fully adaptive algorithm is a decision tree,3 each of whose interior nodes is labeled with some action , and each of whose edges corresponds to an outcome . Each leaf is labeled with a hypothesis , corresponding to the output when the algorithm terminates . Definition 3 . A partially adaptive algorithm ( , T ) is specified by a fixed sequence of actions = ( 1 , 2 , ... ) , with each i 2 A , and a stopping time T . In particular , under any true hypothesis h⇤ 2 H and for any t 1 , the event { T = t } is independent of the outcomes of actions t+1 , t+2 , . . . ( At the stopping time , the choice of which hypothesis to identify is trivial in our Bayesian setting—it is simply the one with the highest “ posterior ” probability ) . Note that a partially adaptive algorithm can be viewed as a special type of fully adaptive algorithm : it is a decision tree with the additional restriction that the actions at each depth are the same . Therefore , a fully adaptive algorithm may be far cheaper than any partially adaptive algorithm . However , there are many scenarios ( e.g. , content recommendation and web search [ 6 ] ) where it is desirable to fix the sequence of actions in advance . Furthermore , in many problems the theoretical analysis of partially adaptive algorithms turns out to be challenging ( e.g. , [ 27 , 12 ] ) . Thus , given an ASHT instance , there are two problems that we will consider , depending on whether the algorithms are partially or fully adaptive . In both cases , our goal is to design fast approximation 2In this noisy setting , an action can ( and often should ) be played for multiple times . 3By approximating D✓ ’ s with discrete distributions , we may assume each node has a finite number of children . algorithms—ones that are computable in polynomial4 time and that are guaranteed to incur expected costs at most within a multiplicative factor of the optimum . In the coming sections , we will describe our algorithms and approximation guarantees . Before moving on to this , it is worth noting that our problem setup is extremely generic and captures a number of well-known problems related to decision-making for learning including best-arm identification for multi-armed bandits [ 8 , 19 , 32 ] , group testing [ 18 ] , and causal inference [ 20 ] , just to name a few .

Summary: The problem considered in the paper regards the identification of the "correct one" (or true) hypothesis, chosen from a set of candidate hypotheses according to some given prior distribution. The identification works by applying tests/actions taken from a given set. The test returns a probabilistic result, according to a distribution that depends on the test and the hypothesis. The problem is to choose the sequence of tests so that from the results the correct hypothesis can be identified with probability bounded by some given parameter delta.  The model is a variant of the more classical combinatorial identification problems, where the outcome of the tests is a deterministic value. In fact, as, admittedly, the authors pointed out the problem remains inherently combinatorial and the solution provided here are tightly connected to the known approximation solutions in the non-statistical variants. The basic strategy is to apply the latter and repeat each test enough time to identify a signle outcome that have statistical significance. Then, proceed according to the deterministic strategy. Standard and some new statistical analytical tools are effectively employed to guarantee that from the random outcome a deterministic result is chosen that guarantess the separation (in the fully adaptive case) or coverage (in the nonadaptive case) leading to the overall cost bound.