SlowGuess/NeurIPS_2023_OCR huggingface dataset:
ecprice@cs.utexas.edu
Yihan Zhou
joeyzhou@cs.utexas.edu
For some hypothesis classes and input distributions, active agnostic learning needs exponentially fewer samples than passive learning; for other classes and distributions, it offers little to no improvement. The most popular algorithms for agnostic active learning express their performance in terms of a parameter called the disagreement coefficient, but it is known that these algorithms are inefficient on some inputs.
We take a different approach to agnostic active learning, getting an algorithm that is competitive with the optimal algorithm for any binary hypothesis class  $H$  and distribution  $\mathcal{D}_X$  over  $X$ . In particular, if any algorithm can use  $m^*$  queries to get  $O(\eta)$  error, then our algorithm uses  $O(m^* \log |H|)$  queries to get  $O(\eta)$  error. Our algorithm lies in the vein of the splitting-based approach of Dasgupta [2004], which gets a similar result for the realizable  $(\eta = 0)$  setting.
We also show that it is NP-hard to do better than our algorithm's  $O(\log |H|)$  overhead in general.
Active learning is motivated by settings where unlabeled data is cheap but labeling it is expensive. By carefully choosing which points to label, one can often achieve significant reductions in label complexity [Cohn et al., 1994]. A canonical example with exponential improvement is one-dimensional threshold functions  $h_{\tau}(x) \coloneqq 1_{x \geq \tau}$ : in the noiseless setting, an active learner can use binary search to find an  $\varepsilon$ -approximation solution in  $O\left(\log \frac{1}{\varepsilon}\right)$  queries, while a passive learner needs  $\Theta\left(\frac{1}{\varepsilon}\right)$  samples [Cohn et al., 1994, Dasgupta, 2005, Nowak, 2011].
In this paper we are concerned with agnostic binary classification. We are given a hypothesis class  $H$  of binary hypotheses  $h: \mathcal{X} \to \{0,1\}$  such that some  $h^* \in H$  has  $\mathrm{err}(h^*) \leq \eta$ , where the error
$$
\operatorname {e r r} (h) := \Pr_ {(x, y) \sim \mathcal {D}} [ h (x) \neq y ]
$$
is measured with respect to an unknown distribution  $\mathcal{D}$  over  $\mathcal{X} \times \{0,1\}$ . In our active setting, we also know the marginal distribution  $\mathcal{D}_X$  of  $x$ , and can query any point  $x$  of our choosing to receive a sample  $y \sim (Y \mid X = x)$  for  $(X,Y) \sim \mathcal{D}$ . The goal is to output some  $\widehat{h}$  with  $\mathrm{err}(\widehat{h}) \leq \eta + \varepsilon$ , using as few queries as possible.
The first interesting results for agnostic active learning were shown by Balcan et al. [2006], who gave an algorithm called Agnostic Active  $(\mathrm{A}^2)$  that gets logarithmic dependence on  $\varepsilon$  in some natural settings: it needs  $\widetilde{O}\left(\log \frac{1}{\varepsilon}\right)$  samples for the 1d linear threshold setting (binary search), as long as  $\varepsilon > 16\eta$ , and  $\widetilde{O}\left(d^{2}\log \frac{1}{\varepsilon}\right)$  samples for  $d$ -dimensional linear thresholds when  $\mathcal{D}_X$  is the uniform sphere and  $\varepsilon > \sqrt{d}\eta$ . This stands in contrast to the polynomial dependence on  $\varepsilon$  necessary in the passive setting. The bound's requirement that  $\varepsilon \gtrsim \eta$  is quite natural given a lower bound of  $\Omega\left(d\frac{\eta^2}{\varepsilon^2}\right)$
due to [Käärääinen, 2006, Beygelzimer et al., 2009], where  $d$  is the VC dimension. Subsequent works have given new algorithms [Dasgupta et al., 2007, Beygelzimer et al., 2010] and new analyses [Hanneke, 2007a] to get bounds for more general problems, parameterized by the "disagreement coefficient" of the problem. But while these can give better bounds in specific cases, they do not give a good competitive ratio to the optimum algorithm: see (Hanneke [2014], Section 8.2.5) for a realizable example where  $O\left(\log \frac{1}{\varepsilon}\right)$  queries are possible, but disagreement-coefficient based bounds lead to  $\Omega \left(\frac{1}{\varepsilon}\right)$  queries.
By contrast, in the realizable, identifiable setting  $(\eta = \varepsilon = 0)$ , a simple greedy algorithm is competitive with the optimal algorithm. In particular, Dasgupta [2004] shows that if any algorithm can identify the true hypothesis in  $m$  queries, then the greedy algorithm that repeatedly queries the point that splits the most hypotheses will identify the true hypothesis in  $O(m\log |H|)$  queries. This extra factor of  $\log |H|$  is computationally necessary: as we will show in Theorem 1.2, avoiding it is NP-hard in general. This approach can extend [Dasgupta, 2005] to the PAC setting (so  $\varepsilon > 0$ , but still  $\eta = 0$ ), showing that if any algorithm gets error  $\varepsilon$  in  $m^*$  queries, then this algorithm gets error  $8\varepsilon$  in roughly  $\widetilde{O}(m^* \cdot \log |H|)$  queries (but see the discussion after Theorem 8.2 of Hanneke [2014], which points out that one of the logarithmic factors is in an uncontrolled parameter  $\tau$ , and states that "Resolving the issue of this extra factor of  $\log \frac{1}{\tau}$  remains an important open problem in the theory of active learning").
The natural question is: can we find an agnostic active learning algorithm that is competitive with the optimal one in the agnostic setting?
Our Results. Our main result is just such a competitive bound. We say an active agnostic learning algorithm  $\mathcal{A}$  solves an instance  $(\bar{H},\mathcal{D}_X,\eta ,\varepsilon ,\delta)$  with  $m$  measurements if, for every distribution  $\mathcal{D}$  with marginal  $\mathcal{D}_X$  and for which some  $h^*\in H$  has  $\mathrm{err}(h^{*})\leq \eta$ , with probability  $1 - \delta$ ,  $\mathcal{A}$  uses at most  $m$  queries and outputs  $\widehat{h}\in H$  with  $\mathrm{err}\left(\widehat{h}\right)\leq \eta +\varepsilon$ . Let  $m^{*}(H,\mathcal{D}_{X},\eta ,\varepsilon ,\delta)$  be the optimal number of queries for this problem, i.e., the smallest  $m$  for which any  $\mathcal{A}$  can solve  $(H,\mathcal{D}_X,\eta ,\varepsilon ,\delta)$ .
Define  $N(H, \mathcal{D}_X, \alpha)$  to be the size of the smallest  $\alpha$ -cover over  $H$ , i.e., the smallest set  $S \subseteq H$  such that for every  $h \in H$  there exists  $h' \in S$  with  $\operatorname{Pr}_{x \sim \mathcal{D}_X}[h(x) \neq h'(x)] \leq \alpha$ . When the context is clear, we drop the parameters and simply use  $N$ . Of course,  $N$  is at most  $|H|$ .
Theorem 1.1 (Competitive Bound). There exist some constants  $c_{1}, c_{2}$  and  $c_{3}$  such that for any instance  $(H, \mathcal{D}_X, \eta, \varepsilon, \delta)$  with  $\varepsilon \geq c_1\eta$ , Algorithm 1 solves the instance with sample complexity
$$
m (H, \mathcal {D} _ {X}, \eta , \varepsilon , \delta) \lesssim \left(m ^ {*} \left(H, \mathcal {D} _ {X}, c _ {2} \eta , c _ {3} \varepsilon , \frac {9 9}{1 0 0}\right) + \log \frac {1}{\delta}\right) \cdot \log \frac {N (H , \mathcal {D} _ {X} , \eta)}{\delta}
$$
and polynomial time.
Even the case of  $\eta = 0$  is interesting, given the discussion in [Hanneke, 2014] of the gap in [Dasgupta, 2005]'s bound, but the main contribution is the ability to handle the agnostic setting of  $\eta >0$ . The requirement that  $\varepsilon \geq O(\eta)$  is in line with prior work [Balcan et al., 2006, Dasgupta, 2005]. Up to constants in  $\eta$  and  $\varepsilon$ , Theorem 1.1 shows that our algorithm is within a  $\log N\leq \log |H|$  factor of the optimal query complexity.
We show that it NP-hard to avoid this log  $N$  factor, even in the realizable  $(\eta = \varepsilon = \delta = 0)$  case:
Theorem 1.2 (Lower Bound). It is NP-hard to find a query strategy for every agnostic active learning instance within an  $c \log |H|$  for some constant  $c > 0$  factor of the optimal sample complexity.
This is a relatively simple reduction from the hardness of approximating SETCOVER [Dinur and Steurer, 2014]. The lower bound instance has  $\eta = \varepsilon = \delta = 0$ , although these can be relaxed to being small polynomials (e.g.,  $\varepsilon = \eta = \frac{1}{3|X|}$  and  $\delta = \frac{1}{3|H|}$ ).
Extension. We give an improved bound for our algorithm in the case of noisy binary search (i.e.,  $H$  consists of 1d threshold functions). When  $\eta = \Theta(\varepsilon), N(H, \mathcal{D}_X, \varepsilon) = \Theta\left(\frac{1}{\varepsilon}\right)$  and  $m^{*}(\eta, \varepsilon, .99) = O(\log \frac{1}{\varepsilon})$ . Thus Theorem 1.1 immediately gives a bound of  $O(\log^2 \frac{1}{\varepsilon \delta})$ , which is nontrivial but not ideal. (For  $\eta \ll \varepsilon$ , the same bound holds since the problem is strictly easier when  $\eta$  is smaller.) However, the bound in Theorem 1.1 is quite loose in this setting, and we can instead give a bound of
$$
O \left(\log {\frac {1}{\varepsilon \delta}} \log {\frac {\log {\frac {1}{\varepsilon}}}{\delta}}\right)
$$
for the same algorithm, Algorithm 1. This matches the bound given by disagreement coefficient based algorithms for constant  $\delta$ . The proof of this improved dependence comes from bounding a new parameter measuring the complexity of an  $H, \mathcal{D}_x$  pair; this parameter is always at least  $\Omega \left( \frac{1}{m^*} \right)$  but may be much larger (and is constant for 1d threshold functions). See Theorem 2.3 for details.
1.1 Related Work
Active learning is a widely studied topic, taking many forms beyond the directly related work on agnostic active learning discussed above [Settles, 2009]. Our algorithm can be viewed as similar to "uncertainty sampling" [Lewis, 1995, Lewis and Catlett, 1994], a popular empirical approach to active learning, though we need some modifications to tolerate adversarial noise.
One problem related to the one studied in this paper is noisy binary search, which corresponds to active learning of 1d thresholds. This has been extensively studied in the setting of i.i.d. noise [Burnashev and Zigangirov, 1974, Ben-Or and Hassidim, 2008, Dereniowski et al., 2021] as well as monotonic queries [Karp and Kleinberg, 2007]. Some work in this vein has extended beyond binary search to (essentially) active binary classification [Nowak, 2008, 2011]. These algorithms are all fairly similar to ours, in that they do multiplicative weights/Bayesian updates, but they query the single maximally informative point. This is fine in the i.i.d. noise setting, but in an agnostic setting the adversary can corrupt that query. For this reason, our algorithm needs to find a set of high-information points to query.
Another related problems is decision tree learning. The realizable, noiseless case  $\eta = \varepsilon = 0$  of our problem can be reduced to learning a binary decision tree with minimal depth. Hegedús [1995] studied this problem and gave basically the same upper and lower bound as in Dasgupta [2005]. Kosaraju et al. [2002] studied a split tree problem, which is a generalization of binary decision tree learning, and also gave similar bounds. Azad et al. [2022] is a monograph focusing on decision tree learning, in which many variations are studied, including learning with noise. However, this line of work usually allows different forms of queries so their results are not directly comparable from results in the active learning literature.
For much more work on the agnostic active binary classification problem, see Hanneke [2014] and references therein. Many of these papers give bounds in terms of the disagreement coefficient, but sometimes in terms of other parameters. For example, Katz-Samuels et al. [2021] has a query bound that is always competitive with the disagreement coefficient-based methods, and sometimes much better; still, it is not competitive with the optimum in all cases.
In terms of the lower bound, it is shown in Laurent and Rivest [1976] that the problem is NP-complete, in the realizable and noiseless setting. To the best of our knowledge, our Theorem 1.2 showing hardness of approximation to within a  $O(\log |H|)$  factor is new.
Minimax sample complexity bounds. Hanneke and Yang [2015] and Hanneke [2007b] have also given "minimax" sample complexity bounds for their algorithms, also getting a sample complexity within  $O(\log |H|)$  of optimal. However, these results are optimal with respect to the sample complexity for the worst-case distribution over  $y$  and  $x$ . But the unlabeled data  $x$  is given as input. So one should hope for a bound with respect to optimal for the actual  $x$  and only worst-case over  $y$ ; this is our bound.
We give the following example to illustrate that our bound, and indeed our algorithm, can be much better.
Example 1.3. Define a hypothesis class of  $N$  hypotheses  $h_1, \dots, h_N$ , and  $\log N + N$  data points  $x_1, \dots, x_{\log N + N}$ . For each hypothesis  $h_j$ , the labels of the first  $N$  points express  $j$  in unary and the labels of the last  $\log N$  points express  $j$  in binary. We set  $\eta = \varepsilon = 0$  and consider the realizable case.
In the above example, the binary region is far more informative than the unary region, but disagreement coefficient-based algorithms just note that every point has disagreement. Our algorithm will query the binary encoding region and take  $O(\log N)$  queries. Disagreement coefficient based algorithms, including those in Hanneke and Yang [2015] and Hanneke [2007b], will rely on essentially uniform sampling for the first  $\Omega(N / \log N)$  queries. These algorithms are "minimax" over  $x$ , in the sense that if you didn't see any  $x$  from the binary region, you would need almost as many samples as they use. But you do see  $x$  from the binary region, so the algorithm should make use of it to get exponential improvement.
Future Work. Our upper bound assumes full knowledge of  $\mathcal{D}_X$  and the ability to query arbitrary points  $x$ . Often in active learning, the algorithm receives a large but not infinite set of unlabeled sample points  $x$ , and can only query the labels of those points. How well our results adapt to this setting we leave as an open question.
Similarly, our bound is polynomial in the number of hypotheses and the domain size. This is hard to avoid in full generality—if you don't evaluate most hypotheses on most data points, you might be missing the most informative points—but perhaps it can be avoided in structured examples.
2 Algorithm Overview
Our algorithm is based on a Bayesian/multiplicative weights type approach to the problem, and is along the lines of the splitting-based approach of Dasgupta [2004].
We maintain a set of weights  $w(h)$  for each  $h \in H$ , starting at 1; these induce a distribution  $\lambda(h) := \frac{w(h)}{\sum_{h} w(h)}$  which we can think of as our posterior over the "true"  $h^*$ .
Realizable setting. As initial intuition, consider the realizable case of  $\eta = \varepsilon = 0$  where we want to find the true  $h^*$ . If  $h^*$  really were drawn from our prior  $\lambda$ , and we query a point  $x$ , we will see a 1 with probability  $\mathbb{E}_{h \sim \lambda} h(x)$ . Then the most informative point to query is the one we are least confident in, i.e., the point  $x^*$  maximizing
$$
r (x) := \min  \left\{\underset {h \sim \lambda} {\mathbb {E}} [ h (x) ], 1 - \underset {h \sim \lambda} {\mathbb {E}} [ h (x) ] \right\}.
$$
Suppose an algorithm queries  $x_{1},\ldots ,x_{m}$  and receives the majority label under  $h\sim \lambda$  each time. Then the fraction of  $h\sim \lambda$  that agree with all the queries is at least  $1 - \sum_{i = 1}^{m}r(x_{i})\geq 1 - mr(x^{*})$ . This suggests that, if  $r(x^{*})\ll \frac{1}{m}$ , it will be hard to uniquely identify  $h^*$ . It is not hard to formalize this, showing that: if no single hypothesis has  $75\%$  probability under  $\lambda$ , and any algorithm exists with sample complexity  $m$  and  $90\%$  success probability at finding  $h^*$ , we must have  $r(x^{*})\geq \frac{1}{10m}$ .
This immediately gives an algorithm for the  $\eta = \varepsilon = 0$  setting: query the point  $x$  maximizing  $r(x)$ , set  $w(h) = 0$  for all hypotheses  $h$  that disagree, and repeat. As long as at least two hypotheses remain, the maximum probability will be  $50\% < 90\%$  and each iteration will remove an  $\Omega \left(\frac{1}{m}\right)$  fraction of the remaining hypotheses; thus after  $O(m \log H)$  rounds, only  $h^*$  will remain. This is the basis for Dasgupta [2004].
Handling noise: initial attempt. There are two obvious problems with the above algorithm in the agnostic setting, where a (possibly adversarial)  $\eta$  fraction of locations  $x$  will not match  $h^*$ . First, a single error will cause the algorithm to forever reject the true hypothesis; and second, the algorithm makes deterministic queries, which means adversarial noise could be placed precisely on the locations queried to make the algorithm learn nothing.
To fix the first problem, we can adjust the algorithm to perform multiplicative weights: if in round  $i$  we query a point  $x_{i}$  and see  $y_{i}$ , we set
$$
w _ {i + 1} (h) = \left\{ \begin{array}{l l} w _ {i} (h) & \text {i f} h (x _ {i}) = y _ {i} \\ e ^ {- \alpha} w _ {i} (h) & \text {i f} h (x _ {i}) \neq y _ {i} \end{array} \right.
$$
for a small constant  $\alpha = \frac{1}{5}$ . To fix the second problem, we don't query the single  $x^{*}$  of maximum  $r(x^{*})$ , but instead choose  $x$  according to distribution  $q$  over many points  $x$  with large  $r(x)$ .
To understand this algorithm, consider how  $\log \lambda_i(h^*)$  evolves in expectation in each step. This increases if the query is correct, and decreases if it has an error. A correct query increases  $\lambda_i$  in proportion to the fraction of  $\lambda$  placed on hypotheses that get the query wrong, which is at least  $r(x)$ ; and the probability of an error is at most  $\eta \max_x \frac{q(x)}{\mathcal{D}_x(x)}$ . If at iteration  $i$  the algorithm uses query distribution  $q$ , some calculation gives that
$$
\underset {q} {\mathbb {E}} \left[ \log \lambda_ {i + 1} \left(h ^ {*}\right) - \log \lambda_ {i} \left(h ^ {*}\right) \right] \geq 0. 9 \alpha \left(\underset {x \sim q} {\mathbb {E}} [ r (x) ] - 2. 3 \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {x} (x)}\right). \tag {1}
$$
λ(h)	Values h(x)
h1	0.9	1111 1111
h2	0.1 - 10-6	1111 0000
h3	10-6	0000 1110
y	0000 1111
Figure 1: An example demonstrating that the weight of the true hypothesis can decrease if  $\lambda$  is concentrated on the wrong ball. In this example, the true labels  $y$  are closest to  $h_3$ . But if the prior  $\lambda$  on hypotheses puts far more weight on  $h_1$  and  $h_2$ , the algorithm will query uniformly over where  $h_1$  and  $h_2$  disagree: the second half of points. Over this query distribution,  $h_1$  is more correct than  $h_3$ , so the weight of  $h_3$  can actually decrease if  $\lambda(h_1)$  is very large.
The algorithm can choose  $q$  to maximize this bound on the potential gain. There's a tradeoff between concentrating the samples over the  $x$  of largest  $r(x)$ , and spreading out the samples so the adversary can't raise the error probability too high. We show that if learning is possible by any algorithm (for a constant factor larger  $\eta$ ), then there exists a  $q$  for which this potential gain is significant.
Lemma 2.1 (Connection to OPT). Define  $\| h - h' \| = \operatorname{Pr}_{x \sim \mathcal{D}_x}[h(x) \neq h'(x)]$ . Let  $\lambda$  be a distribution over  $H$  such that no radius-  $(2\eta + \varepsilon)$  ball  $B$  centered on  $h \in H$  has probability at least  $80\%$ . Let  $m^* = m^* \left( H, \mathcal{D}_X, \eta, \varepsilon, \frac{99}{100} \right)$ . Then there exists a query distribution  $q$  over  $\mathcal{X}$  with
$$
\underset {x \sim q} {\mathbb {E}} [ r (x) ] - \frac {1}{1 0} \eta \max  _ {x} \frac {q (x)}{\mathcal {D} _ {X} (x)} \geq \frac {9}{1 0 0 m ^ {*}}.
$$
At a very high level, the proof is: imagine  $h^* \sim \lambda$ . If the algorithm only sees the majority label  $y$  on every query it performs, then its output  $\widehat{h}$  is independent of  $h^*$  and cannot be valid for more than 80% of inputs by the ball assumption; hence a 99% successful algorithm must have a 19% chance of seeing a minority label. But for  $m^*$  queries  $x$  drawn with marginal distribution  $q$ , without noise the expected number of minority labels seen is  $m^* \mathbb{E}[r(x)]$ , so  $\mathbb{E}[r(x)] \gtrsim 1 / m^*$ . With noise, the adversary can corrupt the minority labels in  $h^*$  back toward the majority, leading to the given bound.
The query distribution optimizing (1) has a simple structure: take a threshold  $\tau$  for  $r(x)$ , sample from  $\mathcal{D}_x$  conditioned on  $r(x) > \tau$ , and possibly sample  $x$  with  $r(x) = \tau$  at a lower rate. This means the algorithm can efficiently find the optimal  $q$ .
Except for the caveat about  $\lambda$  not already concentrating in a small ball, applying Lemma 2.1 combined with (1) shows that  $\log \lambda(h^{*})$  grows by  $\Omega\left(\frac{1}{m^{*}}\right)$  in expectation for each query. It starts out at  $\log \lambda(h^{*}) = -\log H$ , so after  $O(m^{*}\log H)$  queries we would have  $\lambda(h^{*})$  being a large constant in expectation (and with high probability, by Freedman's inequality for concentration of martingales). Of course  $\lambda(h^{*})$  can't grow past 1, which features in this argument in that once  $\lambda(h^{*}) > 80\%$ , a small ball will have large probability and Lemma 2.1 no longer applies, but at that point we can just output any hypothesis in the heavy ball.
Handling noise: the challenge. There is one omission in the above argument that is surprisingly challenging to fix, and ends up requiring significant changes to the algorithm: if at an intermediate step  $\lambda_{i}$  concentrates in the wrong small ball, the algorithm will not necessarily make progress. It is entirely possible that  $\lambda_{i}$  concentrates in a small ball, even in the first iteration—perhaps  $99\%$  of the hypotheses in  $H$  are close to each other. And if that happens, then we will have  $r(x) \leq 0.01$  for most  $x$ , which could make the RHS of (1) negative for all  $q$ .
In fact, it seems like a reasonable Bayesian-inspired algorithm really must allow  $\lambda(h^{*})$  to decrease in some situations. Consider the setting of Figure 1. We have three hypotheses,  $h_1, h_2$ , and  $h_3$ , and a prior  $\lambda = (0.9, 0.099999, 10^{-6})$ . Because  $\lambda(h_3)$  is so tiny, the algorithm presumably should ignore  $h_3$  and query essentially uniformly from the locations where  $h_1$  and  $h_2$  disagree. In this example,  $h_3$  agrees with  $h_1$  on all but an  $\eta$  mass in those locations, so even if  $h^{*} = h_{3}$ , the query distribution can match  $h_1$  perfectly and not  $h_3$ . Then  $w(h_1)$  stays constant while  $w(h_3)$  shrinks.  $w(h_2)$  shrinks much faster, of course, but since the denominator is dominated by  $w(h_1)$ ,  $\lambda(h_3)$  will still shrink. However, despite  $\lambda(h_3)$  shrinking, the algorithm is still making progress in this example:  $\lambda(h_2)$  is shrinking fast, and once it becomes small relative to  $\lambda(h_3)$  then the algorithm will start querying points to distinguish  $h_3$  from  $h_1$ , at which point  $\lambda(h_3)$  will start an inexorable rise.
Our solution is to "cap" the large density balls in  $\lambda$ , dividing their probability by two, when applying Lemma 2.1. Our algorithm maintains a set  $S \subseteq H$  of the "high-density region," such that the capped
distribution:
$$
\overline {{\lambda}} (h) := \left\{ \begin{array}{l l} \frac {1}{2} \lambda (h) & h \in S \\ \lambda (h) \cdot \frac {1 - \frac {1}{2} \Pr [ h \in S ]}{1 - \Pr [ h \in S ]} & h \notin S \end{array} \right.
$$
has no large ball. Then Lemma 2.1 applies to  $\overline{\lambda}$ , giving the existence of a query distribution  $q$  so that the corresponding  $\overline{r}(x)$  is large. We then define the potential function
$$
\phi_ {i} \left(h ^ {*}\right) := \log \lambda_ {i} \left(h ^ {*}\right) + \log \frac {\lambda_ {i} \left(h ^ {*}\right)}{\sum_ {h \notin S _ {i}} \lambda_ {i} (h)} \tag {2}
$$