Title: No-Regret Online Prediction with Strategic Experts
Venue: NeurIPS 2023
Accepted: yes
Keywords: incentive-compatible

online prediction with expert advice

forecasting
Abstract: We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick $m\geq 1$ experts from a pool of $K$ experts and the overall utility is a modular or submodular function of the chosen experts. We focus on the setting in which experts act strategically and aim to maximize their influence on the algorithm's predictions by potentially misreporting their beliefs about the events. Among others, this setting finds applications in forecasting competitions where the learner seeks not only to make predictions by aggregating different forecasters but also to rank them according to their relative performance. Our goal is to design algorithms that satisfy the following two requirements: 1) \emph{Incentive-compatible}: Incentivize the experts to report their beliefs truthfully, and 2) \emph{No-regret}: Achieve sublinear regret with respect to the true beliefs of the best fixed set of $m$ experts in hindsight. Prior works have studied this framework when $m=1$ and provided incentive-compatible no-regret algorithms for the problem. We first show that a simple reduction of our problem to the $m=1$ setting is neither efficient nor effective. Then, we provide algorithms that utilize the specific structure of the utility functions to achieve the two desired goals.

No-Regret Online Prediction with Strategic Experts

Abstract

We study a generalization of the online binary prediction with expert advice framework where at each round, the learner is allowed to pick m ≥ 1 experts from a pool of K experts and the overall utility is a modular or submodular function of the chosen experts. We focus on the setting in which experts act strategically and aim to maximize their influence on the algorithm's predictions by potentially misreporting their beliefs about the events. Among others, this setting finds applications in forecasting competitions where the learner seeks not only to make predictions by aggregating different forecasters but also to rank them according to their relative performance. Our goal is to design algorithms that satisfy the following two requirements: 1) Incentive-compatible : Incentivize the experts to report their beliefs truthfully, and 2) No-regret : Achieve sublinear regret with respect to the true beliefs of the best-fixed set of m experts in hindsight. Prior works have studied this framework when m = 1 and provided incentive-compatible no-regret algorithms for the problem. We first show that a simple reduction of our problem to the m = 1 setting is neither efficient nor effective. Then, we provide algorithms that utilize the specific structure of the utility functions to achieve the two desired goals.

1 Introduction

Learning from a constant flow of information is one of the most prominent challenges in machine learning. In particular, online learning requires the learner to iteratively make decisions and at the time of making each decision, the outcome associated with it is unknown to the learner. The experts problem is perhaps the most well-known problem in online learning [1, 2, 3, 4]. In this problem, the learner aims to make predictions about a sequence of T binary events. To do so, the learner has access to the advice of K experts who each have internal beliefs about the likelihood of each event. At each round t ∈ [ T ] , the learner has to choose one among the advice of K experts and upon making her choice, the t -th binary event is realized and a loss bounded between zero and one is revealed. The goal of the learner is to have no regret, i.e., to perform as well as the best-fixed expert in hindsight. In many applications, however, the experts are strategic and wish to be selected by the learner as often as possible. To this end, they may strategically misreport their beliefs about the events. For instance, FiveThirtyEight 1 aggregates different pollsters according to their past performance to make a single prediction for elections and sports matches. To do so, FiveThirtyEight maintains publicly available pollster ratings 2 . A low rating can be harmful to the pollster's credibility and adversely impact their revenue opportunities in the future. Therefore, instead of maximizing their expected performance by reporting their predictions truthfully, the pollsters may decide to take risks and report more extreme beliefs to climb higher on the leaderboard. Therefore, it is important to design algorithms that not only achieve no-regret but also motivate the experts to report their true beliefs ( incentive-compatible ).

Otherwise, the quality of the learner's predictions may be harmed.

As we have mentioned in Section 1.1, all the previous works on this topic have focused on the standard experts problem where the goal is to choose a single expert among the K experts. In the offline setting, this is equivalent to a forecasting competition in which only the single highest-ranked forecaster wins and receives prizes. However, in many applications, a set of top-performing forecasters are awarded perks and benefits. For instance, in the Good Judgement Project, a recent geopolitical forecasting tournament, the top 2% of forecasters were given the 'superforecaster' status and received benefits such as paid conference travel and employment opportunities [5]. Similarly, in the latest edition of Kaggle's annual machine learning competition to predict the match outcomes of the NCAA March Madness college basketball tournament (called 'March Machine Learning Mania 2023' 3 ), the top 8 forecasters on the leaderboard received monetary prizes.

Variants of the m -experts problem have been previously studied in [6, 7, 8, 9], however, all of these works focused only on providing no-regret algorithms for the problem and the incentive compatibility considerations were not taken into account. To the best of our knowledge, this is the first work that focuses on the strategic m -experts problem where the experts may misreport their true beliefs to increase their chances of being chosen by the learner.

For the setting with modular utilities, perhaps the simplest approach to learning well compared to the best-fixed set of m experts while maintaining incentive compatibility is to run the incentivecompatible WSU algorithm of [10] for the standard experts problem (the setting with m = 1 ) over the set of ( K m ) meta-experts where each meta-expert corresponds to one of the sets of size m . This approach has two major drawbacks: 1) There are exponentially many meta-experts and maintaining weights per each meta-expert and running the WSU algorithm is computationally expensive, and 2) The dependence of the regret bound on m is sub-optimal. Therefore, for our setting, it is preferable to design algorithms that are tailored for the m -experts problem.

1.1 Related work

Prior works have studied the experts problem under incentive compatibility considerations for two feedback models: In the full information setting, the learner observes the reported prediction of all experts at each round. In the partial information setting, however, the learner is restricted to choosing a single expert at each round and does not observe the prediction of other experts. [11] considered algorithms that maintain weights over the experts and choose experts according to these weights. They assumed that experts' incentives are only affected by the unnormalized weights of the algorithm over the experts. However, since the probability of an expert being chosen by the learner equals her normalized weight, the aforementioned assumption might not be suitable. Later on, [10] made the assumption that at each round t ∈ [ T ] , incentives are tied to the expert's normalized weight (i.e., the probability of being chosen at round t +1 ) and studied this problem under both feedback models. They proposed the WSU and WSU-UX algorithms for the full information and partial information settings respectively where both algorithms are incentive-compatible and they obtained O ( √ T ln K ) and O ( T 2 / 3 ( K ln K ) 1 / 3 ) regret bounds for the algorithms. Then, [12] considered non-myopic strategic experts where the goal of each expert is to maximize a conic combination of the probabilities of being chosen in all subsequent rounds (not just the very next round). They showed that the well-known Follow the Regularized Leader (FTRL) algorithm with the negative entropy regularizer obtains a regret bound of O ( √ T ln K ) while being Θ( 1 √ T ) approximately incentive-compatible, i.e., it is a strictly dominated strategy for any expert to make reports Θ( 1 √ T ) distant from their true beliefs. For the non-strategic m -experts problem with modular utilities, [7] proposed the Component Hedge (CH) algorithm and obtained a regret bound of √ 2 mglyph[lscript] ∗ ln( K m ) + m ln( K m ) where glyph[lscript] ∗ is the cumulative loss of the best-chosen set in hindsight. They also gave a matching lower bound for this problem. [8] studied the FTPL algorithm with Gaussian noise distribution and provided an O ( m √ T ln( K m )) regret bound for this setting. For the non-strategic m -experts problem with submodular utility functions, [13] proposed the online distorted greedy algorithm (with dual averaging algorithm as the algorithm A i for i = 1 , . . . , m ) whose regret bound is O ( √ mT ln( K m )) . More recently, [9] studied the m -experts problem under various choices of the utility function (sum-reward, max-reward, pairwise-reward and monotone reward). In particular, for the setting with modular utilities (sum-reward), they proposed

an algorithm that matches the optimal regret bound of the CH algorithm while being computationally more efficient.

1.2 Contributions

In this paper, we focus on a generalization of the experts problem (called ' m -experts problem') where at each round, instead of picking a single expert, we are allowed to pick m ≥ 1 experts and our utility is either a modular or submodular function of the chosen experts. In particular, at round t ∈ [ T ] and for a set of experts S t ⊆ [ K ] , the utility function is defined as f t ( S t ) = | S t | m -1 m ∑ i ∈ S t glyph[lscript] i,t and f t ( S t ) = 1 -∏ i ∈ S t glyph[lscript] i,t in the modular and submodular cases respectively where glyph[lscript] i,t ∈ [0 , 1] is the loss of expert i at round t . The goal is to design algorithms that perform as well as the best-fixed set of m experts in hindsight ( no-regret ) and incentivize the experts to report their beliefs about the events truthfully ( incentive-compatible ).

Towards this goal, we build upon the study of the Follow the Perturbed Leader (FTPL) algorithm for the m -experts problem with modular utility functions by [8] and derive a sufficient condition for the perturbation distribution to guarantee approximate incentive compatibility. Furthermore, we show how this condition is related to the commonly used bounded hazard rate assumption for noise distribution. In particular, we show that while FTPL with Gaussian perturbations is not incentivecompatible, choosing Laplace or hyperbolic noise distribution guarantees approximate incentive compatibility.

Moreover, inspired by Algorithm 1 of [13] for online monotone submodular maximization subject to a matroid constraint, we first introduce a simpler algorithm (called the 'online distorted greedy algorithm') for the special case of cardinality constraints. This algorithm utilizes m incentivecompatible algorithms for the standard experts problem (i.e., m = 1 setting) and outputs their combined predictions. We provide (1 -c e ) -regret bounds for the algorithm where c ∈ [0 , 1] is the average curvature of the submodular utility functions. Therefore, applying the algorithm to the setting where the utility functions are modular (i.e., c = 0 ), the approximation ratio is 1 . For submodular utility functions, the algorithm achieves the optimal 1 -c e approximation ratio.

Finally, we validate our theoretical results through experiments on data gathered from a forecasting competition run by FiveThirtyEight in which forecasters make predictions about the match outcomes of the recent 2022 -2023 National Football League (NFL).

2 Preliminaries

Notation. The set { 1 , 2 , . . . , n } is denoted by [ n ] . For vectors x t , y ∈ R n , x i,t and y i denote their i -th entry respectively. Similarly, for a matrix A ∈ R n × n , we use A i,j to indicate the entry in the matrix's i -th row and j -th column. The inner product of two vectors x, y ∈ R n is denoted by either 〈 x, y 〉 or x T y . For a set function f , we use f ( j | A ) to denote f ( A ∪ { j } ) -f ( A ) . A set function f defined over the ground set V is monotone if for all A ⊆ B ⊆ V , f ( A ) ≤ f ( B ) holds. f is called submodular if for all A ⊆ B ⊂ V and j / ∈ B , f ( j | A ) ≥ f ( j | B ) holds. In other words, the marginal gain of adding the item j decreases as the base set gets larger. This property is known as the diminishing returns property. If equality holds in the above inequality for all A , B , and j , the function is called modular. As mentioned in Section 1.2, at round t ∈ [ T ] and for a set of experts S ⊆ [ K ] , the submodular utility function is defined as f t ( S t ) = 1 -∏ i ∈ S t glyph[lscript] i,t where glyph[lscript] i,t ∈ [0 , 1] is the loss of expert i at round t . To show that this function is submodular, note that we have:

where the inequality follows from glyph[lscript] i,t ∈ [0 , 1] for i ∈ B \ A . For a normalized monotone submodular set function f (i.e., f ( ∅ ) = 0 ), the curvature c f is defined as [14]:

It is easy to see that c f ∈ [0 , 1] always holds. c f ≤ 1 is due to monotonicity of f and c f ≥ 0 follows from f being submodular. Curvature characterizes how submodular the function is. If c f = 0 , the function is modular, and larger values of c f correspond to the function exhibiting a stronger diminishing returns structure.

3 m -experts problem

We introduce the m -experts problem in this section. In this problem, there are K experts available and each expert makes probabilistic predictions about a sequence of T binary outcomes. At round t ∈ [ T ] , expert i ∈ [ K ] has a private belief b i,t ∈ [0 , 1] about the outcome r t ∈ { 0 , 1 } , where r t and { b i,t } K i =1 are chosen arbitrarily and potentially adversarially. Expert i reports p i,t ∈ [0 , 1] as her prediction to the learner. Then, the learner chooses a set S t containing m of the experts. Upon committing to this action, the outcome r t is revealed, and expert i incurs a loss of glyph[lscript] i,t = glyph[lscript] ( b i,t , r t ) where glyph[lscript] : [0 , 1] ×{ 0 , 1 } → [0 , 1] is a bounded loss function. In this paper, we focus on the quadratic loss function defined as glyph[lscript] ( b, r ) = ( b -r ) 2 . The utility of the learner at round t is one of the following:

Modular utility function: f t ( S t ) = | S t | m -1 m ∑ i ∈ S t glyph[lscript] i,t = | S t | m -1 m ∑ i ∈ S t glyph[lscript] ( b i,t , r t ) .

Submodular utility function: f t ( S t ) = 1 -∏ i ∈ S t glyph[lscript] i,t = 1 -∏ i ∈ S t glyph[lscript] ( b i,t , r t ) .

It is easy to see that f t is monotone in both cases and f t ( S t ) ∈ [0 , 1] holds. Note that the utility at each round is defined with respect to the true beliefs of the chosen experts rather than their reported beliefs.

The goal of the learner is twofold:

1) Minimize the α -regret defined as α -R T = E [ α max S ⊆ [ K ]: | S | = m ∑ T t =1 f t ( S ) -∑ T t =1 f t ( S t ) ] , where the expectation is taken with respect to the potential randomness of the algorithm. For the modular utility function, α = 1 and for the submodular setting, we set α = 1 -c f e (where f = ∑ T t =1 f t ) which is the optimal approximation ratio for any algorithm making polynomially many queries to the objective function.

2) Incentivize experts to report their private beliefs truthfully. To be precise, at each round t ∈ [ T ] , each expert i ∈ [ K ] acts strategically to maximize their probability of being chosen at round t +1 and the learner's algorithm is called incentive-compatible if expert i maximizes this probability by reporting p i,t = b i,t . To be precise, we define the incentive compatibility property below.

Definition 1. An online learning algorithm is incentive-compatible if for every t ∈ [ T ] , every expert i ∈ [ K ] with belief b i,t , every report p i,t , reports of other experts p -i,t , every history of reports { p j,s } j ∈ [ K ] ,s<t , and outcomes { r s } s<t , we have:

where Bern ( b ) denotes a Bernoulli distribution with probability of success b and π i,t +1 is the probability of expert i being chosen at round t +1 .

In other words, an online learning algorithm is incentive-compatible if it is in the best interest of the experts to report their private beliefs truthfully to maximize the probability of being chosen for the next round.

As mentioned earlier, we focus on the quadratic loss function in this paper. The quadratic loss function is an instance of proper loss functions [15], i.e., the following holds:

glyph[negationslash]

i.e., each expert minimizes her expected loss (according to their true belief b ) by reporting truthfully.

3.1 Motivating applications

Several interesting motivating applications could be cast into our framework. We mention two classes of such applications below.

Forecasting competitions : In this problem, there are a set of K forecasters who aim to predict the outcome of sports games or elections (between two candidates). At each round t ∈ [ T ] , information on the past performance of the two opposing teams or candidates is revealed and forecasters provide a probabilistic prediction (as a value between [0 , 1] ) about which team or candidate will win. The learner can choose up to m forecasters at each round and her utility is simply the average of the utilities of chosen experts.

Online paging problem with advice [16]: There is a library { 1 , . . . , N } of N distinct files. A cache with limited storage capacity can store at most m files at any time. At each round t ∈ [ T ] , a user arrives and requests one file. The learner has access to a pool of K experts where each expert i ∈ [ K ] observes the user history and makes a probabilistic prediction p i,t ∈ [0 , 1] N for the next file

request (where 1 T p i,t = 1 ). For instance, p i,t = e j if expert i predicts the file j ∈ [ N ] where e j is the j -th standard basis vector. Also, r t = e j if the j -th file is requested at round t ∈ [ T ] . The learner can choose m of these experts at each round and put their predictions in the cache. The learner's prediction for round t is correct if and only if one of the m chosen experts has correctly predicted the file. Thus, the loss of expert i can be formulated as glyph[lscript] i,t = ‖ p i,t -r t ‖ 2 2 and the utility at round t could be written as f t ( S t ) = 2 -∏ i ∈ S t glyph[lscript] i,t which is exactly our submodular utility function. Note that this is a slight generalization of our framework where instead of binary outcomes, we consider nonbinary (categorical) outcomes. All our results could be easily extended to this setting as well.

3.2 Naive approach

The WSU algorithm of [10] for the standard experts problem is derived by drawing a connection between online learning and wagering mechanisms. The framework of one-shot wagering mechanisms was introduced by [17] and is as follows: There are K experts and each expert i ∈ [ K ] holds a belief b i ∈ [0 , 1] about the likelihood of an event. Expert i reports a probability p i ∈ [0 , 1] and a wager ω i ≥ 0 . A wagering mechanism Γ is a mapping from the reports p = ( p 1 , . . . , p K ) , wagers ω = ( ω 1 , . . . , ω K ) and the realization r of the binary event to the payments Γ i ( p, ω, r ) to expert i . It is assumed that Γ i ( p, ω, r ) ≥ 0 ∀ i ∈ [ K ] , i.e., no expert loses more than her wager. A wagering mechanism is called budget-balanced if ∑ K i =1 Γ i ( p, ω, r ) = ∑ K i =1 ω i . [17, 18] introduced a class of incentive-compatible budget-balanced wagering mechanisms called the Weighted Score Wagering Mechanisms (WSWMs) which is defined as follows: For a fixed proper loss function glyph[lscript] bounded in [0 , 1] , the payment to expert i is

The proposed algorithm in [10] is called Weighted-Score Update (WSU) and the update rule for the weights of the experts { π t } T t =1 is the following:

where π i, 1 = 1 K ∀ i ∈ [ K ] . In other words, the normalized weights of the experts at round t are interpreted as the wager of the corresponding expert, and the normalized weights at round t +1 are derived using a convex combination of the weights at the previous round and the payments in WSWM. Note that since WSWM is budget-balanced, the derived weights at each round automatically sum to one and there is no need to normalize the weights (which might break the incentive compatibility). Also, considering the incentive compatibility of WSWM, the WSU algorithm is incentive-compatible as well.

The update rule of WSU could be rewritten as follows:

where L i,t = glyph[lscript] i,t -∑ K j =1 π j,t glyph[lscript] j,t . Therefore, the WSU update rule is similar to that of the Multiplicative Weights Update (MWU) algorithm [19] with the relative loss L i,t instead of glyph[lscript] i,t in the formula.

A simple approach to solving the m -experts problem with modular utilities ( glyph[lscript] S,t = 1 m ∑ j ∈ S glyph[lscript] j,t ) is to define an 'expert' for each of the possible ( K m ) sets of size m and apply the incentive-compatible WSU algorithm of [10] for the standard experts problem to this setting. Note that we still define incentive compatibility with respect to individual experts (instead of the ( K m ) meta-experts). To be precise, we define π i,t = ∑ S : | S | = m,i ∈ S π S,t . We can show the following:

glyph[negationslash]

Given that π i,t +1 is linear in L i,t , L i,t = glyph[lscript] i,t -∑ K j =1 π j,t glyph[lscript] j,t is linear in glyph[lscript] i,t , and the loss function is proper, we can conclude that incentive compatibility holds in this setting as well. We summarize the result of this approach in the theorem below.

Theorem 1. If we apply the WSU algorithm of [10] to a standard experts problem with ( K m ) experts corresponding to each S with | S | = m , and set η = √ m ln( Ke m ) T , the algorithm is incentivecompatible and its regret is bounded as follows:

This approach has two major drawbacks:

1) Computational complexity of maintaining weights for each m feasible sets. In particular, we have to do exponentially many queries to the objective function at each round to update these weights. 2) The regret bound has a √ m dependence on the number of experts m that is suboptimal. In the subsequent sections, we propose two efficient algorithmic frameworks that exploit the modular or submodular structure of the utility function and obtain the desired regret and incentive compatibility guarantees.

( K )

4 Follow the Perturbed Leader (FTPL) algorithm

In this section, we study the well-known Follow the Perturbed Leader (FTPL) algorithm for the m -experts problem with modular utility functions and study its regret and incentive compatibility guarantees. The algorithm is as follows: At each round t ∈ [ T ] , we first take K i.i.d. samples { γ i,t } K i =1 from the noise distribution D . In particular, we focus on zero-mean symmetric noise distributions from the exponential family, i.e., f ( γ i,t ) ∝ exp ( -ν ( γ i,t )) where ν : R → R + is symmetric about the origin. At round t , we simply keep track of ∑ t -1 s =1 glyph[lscript] i,s + ηγ i,t for each i (where η is the step size) and pick the m experts for whom this quantity is the smallest. [8] previously studied the FTPL algorithm for a class of problems that includes the m -experts problem. However, they only focused on the setting with zero-mean Gaussian perturbations. In contrast, we not only extend this analysis to all zero-mean symmetric noise distributions from the exponential family, but we also analyze the incentive compatibility guarantees of the algorithm and determine a sufficient condition for the perturbation distribution under which the algorithm is approximately incentive-compatible. This condition is provided below.

Condition 1. For all z ∈ R , | ν ′ ( z ) | ≤ B holds for some constant B > 0 .

′

We have ν ( z ) = | z | for Laplace distribution. Therefore, ν ( z ) = sign ( z ) and B = 1 . For symmetric hyperbolic distribution, ν ( z ) = √ 1 + z 2 holds. So, | ν ′ ( z ) | = | z | √ 1+ z 2 ≤ 1 and B = 1 .

Condition 1 is closely related to a boundedness assumption on the hazard rate of the perturbation distribution. We first define the hazard rate below.

Definition 2. The hazard rate of D at z ∈ R is defined as

where f D and F D are the probability density function (pdf) and the cumulative density function (cdf) of the noise distribution D . The maximum hazard rate of D is haz D = sup z ∈ R haz D ( z ) .

The hazard rate is a statistical tool used in survival analysis that measures how fast the tail of a distribution decays. The theorem below shows the connection between Condition 1 and the bounded hazard rate assumption.

Theorem 2. If Condition 1 holds for the perturbation distribution D with the constant B > 0 , we have haz D ≤ B .

However, there are distributions with bounded hazard rates for which max z | ν ′ ( z ) | is unbounded (i.e., Condition 1 does not hold). For instance, consider the standard Gumbel distribution. In this case, ν ( z ) = z + exp ( -z ) . Therefore, we have ν ′ ( z ) = 1 -exp ( -z ) . So, if z → -∞ , | ν ′ ( z ) | → ∞ . Therefore, Condition 1 is strictly stronger than the bounded hazard rate assumption for the noise distribution D .

We show how Condition 1 guarantees an approximate notion of incentive compatibility for FTPL.

Theorem 3. For the FTPL algorithm with a noise distribution satisfying Condition 1 with a constant B > 0 , at round t ∈ [ T ] , for an expert i ∈ [ K ] , the optimal report from the expert's perspective p ∗ i,t is at most 2 B η -2 B away from her belief b i,t , i.e., the following holds:

Note that while we focused on the incentive structure in which at each round t ∈ [ T ] , experts wish to maximize their probability of being chosen at round t +1 , the same argument could be applied to a more general setting where the goal is to maximize a conic combination of probabilities of being chosen at all subsequent round s > t . Therefore, FTPL is approximately incentive compatible with respect to this more general incentive structure as well.

Theorem 3 allows us to bound the regret of the FTPL algorithm with respect to the true beliefs of the experts. First, note that FTPL obtains the following bound with respect to the reported beliefs of the experts.

Theorem 4. For the FTPL algorithm with noise distribution D satisfying Condition 1 with the constant B > 0 , if we set η = √ BT ln( K m ) , the following holds:

Using the result of Theorem 3, we have | p i,t -b i,t | = | p ∗ i,t -b i,t | ≤ 2 B η -2 B . Moreover, one can easily show that the quadratic loss function is 2 -Lipschitz. Therefore, for all t ∈ [ T ] and i ∈ [ K ] , we have:

Putting the above results together, we can obtain the following regret bound for the FTPL algorithm.

Given that η = √ BT ln( K m ) in Theorem 4, the expected regret bound is O ( √ BT ln( K m )) . This result is summarized in the following theorem.

Theorem 5. For the FTPL algorithm with noise distribution D satisfying Condition 1 with the constant B > 0 , if we set η = √ BT ln( K m ) , the regret bound is O ( √ BT ln( K m )) .

In order to ensure approximate incentive compatibility, the probability density function of the noise distribution f needs to be such that f ( z ) f ( z +1) does not grow to infinity for very large z . One way to enforce this condition is via a Lipschitzness assumption on ln f . Condition 1 implies that ln f is B -Lipschitz. That is why smaller values of B lead to better approximate incentive compatibility which in turn results in smaller regret bounds (given that the term 4 TC appears in the regret bound where C is the bound on the approximate incentive-compatibility derived in Theorem 3).

We can use the FTPL algorithm to obtain results for the partial information setting as well. [20] showed that if the hazard rate of the noise distribution is bounded by B , applying the FTPL algorithm to the partial information setting for the 1 -expert problem leads to O ( √ BKT ln K ) regret bounds. Using the result of Theorem 2, we know that if Condition 1 holds, the hazard rate is bounded. Therefore, if the noise distribution satisfies Condition 1, FTPL applied to the m -experts problem is approximately incentive-compatible and achieves O ( √ BKT ln( K m )) regret bound.

5 Online distorted greedy algorithm

In this section, we study the setting where the utility function is submodular. In this case, we have f t ( S t ) = 1 -∏ i ∈ S t glyph[lscript] i,t = 1 -∏ i ∈ S t glyph[lscript] ( b i,t , r t ) . The problem in this setting could be

written as an online monotone submodular maximization problem subject to a cardinality constraint of size m . [21] proposed the online greedy algorithm for this problem whose (1 -1 e ) -regret is bounded by O ( √ mT ln K ) . The algorithm works as follows: There are m instantiations A 1 , . . . , A m of no-regret algorithms for the 1 -expert problem. At each round t ∈ [ T ] , A i selects an expert v i,t ∈ [ K ] and the set S t = { v 1 ,t , . . . , v m,t } is selected. A i observes the reward f t ( v i,t | S i -1 ,t ) where S j,t = { v 1 ,t , . . . , v j,t } .

Inspired by Algorithm 1 of [13] for online monotone submodular maximization subject to a matroid constraint, we propose the online distorted greedy in Algorithm 1 for the special case of a cardinality constraint. The algorithm is similar to the online greedy algorithm of [21] discussed above. However, in the online distorted greedy algorithm, after choosing the set S t and observing the function f t , we first compute the modular lower bound h t defined as h t ( S ) = ∑ i ∈ S f t ( i | [ K ] \ { i } ) . We define g t = f t -h t . Note that g t is monotone submodular as well. The reward of A i for choosing v i,t at round t is (1 -1 m ) m -i g t ( v i,t | S i -1 ,t ) + h t ( v i,t ) (in case v i,t ∈ S i -1 ,t , we repeatedly take samples from the weight distribution of A i over the experts until we observe an expert not in the set S i -1 ,t ). This technique was first introduced by [22] for the corresponding offline problem and it allows us to obtain (1 -c f e ) -regret bounds (where f = ∑ T t =1 f t ) with the optimal approximation ratio (optimality was shown by [23]) instead of the (1 -1 e ) -regret bounds for the online greedy algorithm.

One particular choice for {A i } i m =1 is the WSU algorithm of [10]. We summarize the result for this choice in the theorem below.

Theorem 6. For all i ∈ [ m ] , let A i be an instantiation of the WSU algorithm of [10] for the 1-expert problem and denote f = ∑ T t =1 f t . The online distorted greedy algorithm applied to the m -experts problem obtains the following regret bound:

where R ( i ) T is the regret of algorithm A i . If c f = 0 , the algorithm is incentive-compatible.

If c f ∈ (0 , 1] , we can use the online greedy algorithm of [21] instead to ensure incentive compatibility while maintaining similar bounds for the (1 -1 e ) -regret. [10] provided O ( √ T ln K ) regret bounds for the WSU algorithm. If we plug in this bound in the result of Theorem 6, the regret bound of the online distorted greedy algorithm is O ( m √ T ln K ) . However, this bound depends linearly on m which is suboptimal. To remedy this issue, we first provide an adaptive regret bound for the WSU algorithm below.

Theorem 7. The regret of the WSU algorithm of [10] is bounded by O ( √ | L T | ln K +ln K ) where | L T | is the cumulative absolute loss of the algorithm.

Note that the bound in Theorem 7 adapts to the hardness of the problem. In the worst case, we have | L T | = T and recover the O ( √ T ln K ) bound proved in [10]. However, for smaller values of | L T | , the bound in Theorem 7 improves that of [10]. For the non-strategic setting with modular utilities, [7] proposed the Component Hedge (CH) algorithm and obtained a regret bound of √ 2 mglyph[lscript] ∗ ln( K m ) + m ln( K m ) where glyph[lscript] ∗ is the cumulative loss of the best-chosen set in hindsight. They also gave a matching lower bound for this problem. Applying the same analysis as in Theorem 7 to the setting of the naive approach with ( K m ) meta-experts, we can show that the regret bound of WSU matches the aforementioned lower bound.

We can use the above adaptive regret bound to improve the regret bound of the online distorted greedy algorithm. First, note that at round t ∈ [ T ] , the sum of the absolute value of losses incurred by {A i } i m =1 is bounded as follows:

Therefore, if we denote the cumulative absolute losses incurred by A i with | L ( i ) T | , we have:

Algorithm 1 Online distorted greedy algorithm

Using the result of Theorem 7, we know that the regret bound of the online distorted greedy algorithm is ∑ m i =1 √ | L ( i ) T | ln K + m ln K . ∑ m i =1 √ | L ( i ) T | is maximized when | L ( i ) T | = T m for all i ∈ [ m ] . Thus, in the worst case, the expected (1 -c f e ) -regret bound is O ( √ mT ln K + m ln K ). While we focused on submodular utility functions in this section, we can also apply the online distorted greedy algorithm to the setting with modular utilities. In this case, we have c f = 0 , and therefore, the algorithm is incentive-compatible and its 1 -regret is bounded by O ( √ mT ln K + m ln K ). Unlike the FTPL algorithm that is only approximately incentive-compatible, the online distorted greedy algorithm applied to modular utility functions is incentive-compatible. However, this comes at the price of an extra √ m term in the regret bound.

6 Experiments

In this section, we evaluate the performance of our proposed algorithms for modular utility functions on a publicly available dataset from a FiveThirtyEight forecasting competition 4 in which forecasters make predictions about the match outcomes of the 2022 -2023 National Football League (NFL). Before each match, FiveThirtyEight provides information on the past performance of the two opposing teams. Forecasters observe this information and make probabilistic predictions about the likelihood of each team winning the match. Considering that there are 284 different matches in the dataset, we set T = 284 . Out of the 9982 forecasters who participated in this competition, only 274 made predictions for every single match. We consider two cases: K = 20 and K = 100 . To reduce variance, for each case, we sample 5 groups of K forecasters from the 274 and run FTPL and Online Distorted Greedy (ODG) 10 times. We set m = 5 . Given that FTPL is only approximately incentivecompatible and according to the result of Theorem 3, the reported beliefs could be 2 B distant from the true beliefs, we add a uniformly random value in the range [ η -2 B , η -2 B ] to the true beliefs to model this fact. We use the standard Laplace distribution as the perturbation for FTPL. Hence, we set B = 1 . For both algorithms, the step size η is chosen according to our theoretical results. In Figure 1, we plot the average regret 1 t E [ max S ⊆ [ K ]: | S | = m ∑ t τ =1 f τ ( S ) -∑ t τ =1 f τ ( S τ ) ] of the two algorithms over time (along with error bands corresponding to 20 th and 80 th percentiles) along with that of the FiveThirtyEight aggregated predictions for K = 20 and K = 100 settings. Note that while our proposed algorithms choose m predictions at each round t ∈ [ T ] , the FiveThirtyEight aggregated prediction is a single scalar value. The plots suggest that while the regret of all three algorithms converges to zero as t gets larger, both our proposed algorithms have superior performance compared to that of the FiveThirtyEight predictions.

7 Conclusion and future directions

In this paper, we studied the m -experts problem, a generalization of the standard binary prediction with expert advice problem where at each round t ∈ [ T ] : 1) The algorithm is allowed to pick m ≥ 1 experts and its utility is a modular or submodular function of the chosen experts, and 2) The experts are strategic and may misreport their true beliefs about the t -th event to increase their probability of

η -2 B -2 B 2 B

(a)

(b)

le-1

le-1

Regret

1.0

ODG

Regret

1.75

ODG

FTPL

1.50

FTPL

538

538

0.8

1.25

Average

0.6

Averag

1.00

0.75

0.4

Running

6

Su!u

0.50

0.2

uui

0.25

E

R

0.0

0.00

0

50

100

150

200

250

0

50

100

150

200

250

being chosen at the next round (round t +1 ). The goal is to design algorithms that incentivize experts to report truthfully (i.e., incentive-compatible) and obtain sublinear regret bounds with respect to the true beliefs of the experts (i.e., no-regret). We proposed two algorithmic frameworks for this problem. In Section 4, we introduced the Follow the Perturbed Leader (FTPL) algorithm. Under a certain condition for the noise distribution (Condition 1), this algorithm is approximately incentivecompatible and achieves sublinear regret bounds for modular utility functions. Moreover, in Section 5, we proposed the online distorted greedy algorithm that applies to both modular and submodular utility functions. This algorithm is incentive-compatible but its regret bound is slightly worse than that of FTPL.

This work could be extended in several interesting directions. First, none of the algorithms discussed here or in prior works have taken into account the properties of the quadratic loss function. In particular, this loss function is exp-concave, and [4] showed that for exp-concave loss functions in the 1 -expert problem, the regret bound could be improved to O (ln K ) using the Hedge algorithm without the incentive compatibility property. Designing incentive-compatible algorithms with similarly improved regret bounds for the 1 -expert and m -experts problems is yet to be done. In order to obtain the O (ln K ) regret bound for the 1 -expert problem (with squared loss) using the Hedge algorithm, the algorithm makes a single prediction ∑ K i =1 π i,t p i,t at round t ∈ [ T ] and its loss is ( ∑ K i =1 π i,t p i,t -r t ) 2 . In other words, choosing an expert i ∈ [ K ] with probability π i,t at round t is not good enough to obtain the improved O (ln K ) regret bound. Moving on to the m -expert problem, the main challenge for obtaining regret bounds better than O ( √ T ) is to decide how to aggregate the K predictions as m scalar values. Second, while we focused on the particular choice of quadratic loss functions, the setting could be extended to other loss functions as well. It is not clear to what extent our results hold when moving beyond the quadratic loss function. Finally, [16] introduced a framework for augmenting online algorithms for various online problems with predictions or pieces of advice. An interesting future research direction is to extend this setting to the case where the predictions are given by strategic experts and study incentive compatibility guarantees for online problems beyond the m -experts problem.

Acknowledgments and Disclosure of Funding

This work was supported in part by the following grants: NSF TRIPODS II 2023166, NSF CIF 2212261, NSF CIF 2007036, NSF AI Inst 2112085.

References

Volodimir G Vovk. Aggregating strategies. In Annual Workshop on Computational Learning Theory: Proceedings of the third annual workshop on Computational learning theory, 1990 . Association for Computing Machinery, Inc, 1990.

Nicolo Cesa-Bianchi, Yoav Freund, David Haussler, David P Helmbold, Robert E Schapire, and Manfred K Warmuth. How to use expert advice. Journal of the ACM (JACM) , 44(3):427-485, 1997.

Nick Littlestone and Manfred K Warmuth. The weighted majority algorithm. Information and computation , 108(2):212-261, 1994.

Jyrki Kivinen and Manfred K Warmuth. Averaging expert predictions. In European Conference on Computational Learning Theory , pages 153-167. Springer, 1999.

Philip E Tetlock and Dan Gardner. Superforecasting: The art and science of prediction . Random House, 2016.

Manfred K Warmuth and Dima Kuzmin. Randomized online pca algorithms with regret bounds that are logarithmic in the dimension. Journal of Machine Learning Research , 9(Oct):22872320, 2008.

Wouter M Koolen, Manfred K Warmuth, Jyrki Kivinen, et al. Hedging structured concepts. In COLT , pages 93-105. Citeseer, 2010.

Alon Cohen and Tamir Hazan. Following the perturbed leader for online structured learning. In International Conference on Machine Learning , pages 1034-1042. PMLR, 2015.

Samrat Mukhopadhyay, Sourav Sahoo, and Abhishek Sinha. k-experts-online policies and fundamental limits. In International Conference on Artificial Intelligence and Statistics , pages 342-365. PMLR, 2022.

Rupert Freeman, David M Pennock, Chara Podimata, and Jennifer Wortman Vaughan. No-regret and incentive-compatible online learning. arXiv preprint arXiv:2002.08837 , 2020.

Tim Roughgarden and Okke Schrijvers. Online prediction with selfish experts. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30 , pages 1300-1310. Curran Associates, Inc., 2017.

Rafael Frongillo, Robert Gomez, Anish Thilagar, and Bo Waggoner. Efficient competitions and online learning with strategic forecasters. In Proceedings of the 22nd ACM Conference on Economics and Computation , pages 479-496, 2021.

Nicholas Harvey, Christopher Liaw, and Tasuku Soma. Improved algorithms for online submodular maximization via first-order regret bounds. Advances in Neural Information Processing Systems , 33:123-133, 2020.

Michele Conforti and Gérard Cornuéjols. Submodular set functions, matroids and the greedy algorithm: tight worst-case bounds and some generalizations of the rado-edmonds theorem. Discrete applied mathematics , 7(3):251-274, 1984.

Mark D Reid and Robert C Williamson. Surrogate regret bounds for proper losses. In Proceedings of the 26th Annual International Conference on Machine Learning , pages 897-904, 2009.

Thodoris Lykouris and Sergei Vassilvitskii. Competitive caching with machine learned advice. Journal of the ACM (JACM) , 68(4):1-25, 2021.

Nicolas S Lambert, John Langford, Jennifer Wortman, Yiling Chen, Daniel Reeves, Yoav Shoham, and David M Penno k. Self-financed wagering mechanisms for forecasting. In Proceedings of the 9th ACM conference on Electronic commerce , pages 170-179, 2008.

Nicolas S Lambert, John Langford, Jennifer Wortman Vaughan, Yiling Chen, Daniel M Reeves, Yoav Shoham, and David M Pennock. An axiomatic characterization of wagering mechanisms. Journal of Economic Theory , 156:389-416, 2015.

Sanjeev Arora, Elad Hazan, and Satyen Kale. The multiplicative weights update method: a meta-algorithm and applications. Theory of computing , 8(1):121-164, 2012.

Jacob D Abernethy, Chansoo Lee, and Ambuj Tewari. Fighting bandits with a new kind of smoothness. Advances in Neural Information Processing Systems , 28, 2015.

Matthew Streeter and Daniel Golovin. An online algorithm for maximizing submodular functions. Advances in Neural Information Processing Systems , 21, 2008.

Moran Feldman. Guess free maximization of submodular and linear sums. Algorithmica , 83(3):853-878, 2021.

Maxim Sviridenko, Jan Vondrák, and Justin Ward. Optimal approximation for submodular and supermodular optimization with bounded curvature. Mathematics of Operations Research , 42(4):1197-1218, 2017.

Appendix

A Missing proofs

A.1 Proof of Theorem 1

At time t = 1 , we set π S, 1 = 1 ( K m ) for each set S where | S | = m and update these weights as follows:

where L S,t = glyph[lscript] S,t -∑ S ′ : | S ′ | = m π S ′ ,t glyph[lscript] S ′ ,t and glyph[lscript] S,t = 1 m ∑ i ∈ S glyph[lscript] i,t . If we denote the optimal set as S ∗ , we can write:

Taking the natural logarithm of both sides, we have:

We can use the inequality ln(1 -x ) ≥ -x -x 2 for x ≤ 1 2 (we choose η later such that this inequality holds) to obtain:

Rearranging the terms and dividing both sides by η , we have:

Using the fact that R T = -∑ T t =1 L S ∗ ,t , the inequality ( K m ) ≤ ( Ke m ) m and | L S,t | ≤ 1 ∀ S, t , we can write:

Setting η = √ m ln( Ke m ) T , we obtain the regret bound O ( √ mT ln( K m )) . Note that the assumption ηL S ∗ ,t ≤ 1 2 (used in the proof) holds if T ≥ 4 m ln( Ke m ) . Therefore, we assume T is large enough to satisfy this inequality.

A.2 Proof of Theorem 2

We can show that if B = max z | ν ′ ( z ) | , the hazard rate of the distribution is bounded above by B as well. To see this, fix x ≥ 0 . Note that f ( -x ) 1 -F ( -x ) = f ( x ) 1 -F ( -x ) ≤ f ( x ) 1 -F ( x ) for a symmetric zero-mean distribution and therefore, we only need to bound the hazard rate at x > 0 to bound haz D . We can

write:

where we have used the mean-value theorem in (a) and z x is in the line segment between x and z . Also, note that since x, z > 0 , z x > 0 holds as well and therefore, ν ′ ( z x ) > 0 . We have used this fact along with x -z < 0 to obtain (b). Therefore, we have haz D ≤ B .

A.3 Proof of Theorem 3

Let's fix round t ∈ [ T ] and expert i ∈ [ K ] . For j = i , denote x ( t ) j, 0 = ∑ t -1 s =1 glyph[lscript] j,s + p 2 j,t + ηγ j,t as the total losses of expert j up to round t plus noise if r t = 0 . Similarly, we can define x ( t ) j, 1 = ∑ t -1 s =1 glyph[lscript] j,s +(1 -p j,t ) 2 + ηγ j,t . Define X ( t ) 0 ( X ( t ) 1 ) as the m -th smallest value in { x ( t ) j, 0 } j = i ( { x ( t ) j, 1 } j = i ). Note that | X ( t ) 0 -X ( t ) 1 | ≤ 1 holds because for each j = i , we have | x ( t ) j, 0 -x ( t ) j, 1 | ≤ 1 . Also, let L = ∑ t -1 s =1 glyph[lscript] i,s . If r t = 0 , expert i is chosen at round t +1 if and only if L + p 2 i,t + ηγ i,t < = X ( t ) 0 . Similarly, for the case r t = 1 , expert i is chosen if and only if L +(1 -p i,t ) 2 + ηγ i,t < = X ( t ) 1 . Rearranging the terms, we can write:

glyph[negationslash]

Given that f ( γ i,t ) ∝ exp ( -ν ( γ i,t )) , if we define Y 0 = ηγ i,t + L -X ( t ) 0 and Y 1 = ηγ i,t + L -X ( t ) 1 , we have:

Therefore, if F 0 and F 1 denote the cdf, we can write the probability of expert i being chosen at round t +1 as F 0 ( -p 2 i,t ) and F 1 ( -(1 -p i,t ) 2 ) for the cases r t = 0 and r t = 1 respectively. Putting the above results together, we can write the expected utility of expert i (according to her belief b i,t ) at round t as follows:

glyph[negationslash]

glyph[negationslash]

glyph[negationslash]

Taking the derivative and setting it to zero, we have:

Ideally, we want A to be as close to 1 as possible because if A = 1 , p ∗ i,t = b i,t and the algorithm would be incentive-compatible. In general, we have:

Note that this is not a closed-form solution for p ∗ i,t because A is also a function of p i,t . We can observe that for p i,t < p ∗ i,t , the derivative of the utility function is positive, and for p i,t > p ∗ i,t , the derivative is negative.

We can bound A as follows: Let g ( u ) = f ( -p 2 i,t -a -u ( a ′ -a ) η ) where a = L -X ( t ) 1 + 1 -2 p i,t , a ′ = L -X ( t ) 0 and f ( z ) = exp ( -ν ( z )) . Taking derivative of g with respect to u , we obtain:

Therefore, we can write:

We have a ′ -a = X ( t ) 1 -X ( t ) 0 + 2 p i,t -1 . Therefore, -2 ≤ a ′ -a ≤ 2 holds. Moreover, we have -B ≤ ν ′ ( -p 2 i,t -a -u ( a ′ -a ) η ) ≤ B due to Condition 1. Putting the above results together, the following holds:

Therefore, A ∈ [ exp ( -2 B η ) , exp ( 2 B η )] . Let h ( p i,t ) = p i,t -p ∗ i,t = p i,t -b i,t b i,t +(1 -b i,t ) A . Since p ∗ i,t ∈ [0 , 1] , we have h (0) < 0 and h (1) > 0 . Taking the derivative of h with respect to p i,t , we have:

Using Condition 1, we have 1 -2 Bb i,t (1 -b i,t ) A η ( b i,t +(1 -b i,t ) A ) 2 ≤ dh dp i,t . Therefore, we can write:

We choose η later such that η > B for the last inequality to hold. So, h is strictly increasing, there is exactly one solution p ∗ i,t , and the derivative is positive below it and negative above it.

If we replace A with something larger in p ∗ i,t , the value decreases and vice versa. Therefore, we can write:

Similarly, we can lower bound p ∗ i,t as follows:

Putting the above results together, we conclude that | p ∗ i,t -b i,t | ≤ 2 B η -2 B holds.

A.4 Proof of Theorem 4

Let η > 0 and X be the set of ( K m ) feasible sets of size m for this problem. Denote L t ∈ R K as the partial sum of losses for all experts before round t , i.e., [ L t ] i = ∑ t -1 s =1 glyph[lscript] i,s . The update rule of FTPL at round t ∈ [ T ] is π t = arg min x ∈X 〈 x, L t + ηγ t 〉 . First, we analyze the expected regret of the algorithm below.

where L ′ t is in the line segment between L t and L t +1 = L t + glyph[lscript] t . By definition, φ t is the minimum of linear functions and therefore, it is concave. Thus, ∇ 2 φ t ( L ′ t ) is negative semidefinite. Note that since γ t is simply K i.i.d. samples of the noise distribution for all t ∈ [ T ] , we have φ t ( θ ) = φ ( θ ) =

E γ ∼D K [min x ∈X 〈 x, θ + ηγ 〉 ] . Taking the sum over t ∈ [ T ] and rearranging the terms, we obtain:

Using Jensen's inequality, we can write:

Therefore, we can rearrange the terms and bound the expected regret as follows:

To bound the first term on the right hand side, we can use the fact that D is symmetric to write:

Now, we move on to bound the second term in the regret bound. Given that the losses of the experts at each round t ∈ [ T ] are bounded between 0 and 1, we have ‖ glyph[lscript] t ‖ ∞ ≤ 1 . Therefore, we have:

By definition, if we denote ˆ π ( θ ) = arg min x ∈X 〈 x, θ 〉 , we have:

Given the concavity of φ , diagonal entries of the Hessian ∇ 2 φ are non-positive. We can also use 1 T ˆ π ( θ ) = m to show that the off-diagonal entries are non-negative and each row or column of the Hessian sums up to 0. Therefore, we have ∑ i,j ∈ [ K ] |∇ 2 i,j φ ( L ′ t ) | = -2 ∑ K i =1 ∇ 2 i,i φ ( L ′ t ) . Putting the above results together, we can bound the regret as follows:

We can bound the first term as follows:

To bound the second term in the regret bound, we can use Condition 1 to write:

Putting the above results together, we have:

Therefore, if we set η = √ BT ln( K m ) , the regret bound is O ( √ BT ln( K m )) . In the proof of Theorem 3, we assumed that η is chosen such that η > B . So, we assume T > B ln( K m ) .

A.5 Proof of Theorem 6

For i ∈ [ m ] and t ∈ [ T ] , define:

For all i ∈ [ m ] , we can write

Taking the sum over t ∈ [ T ] , we obtain:

If the regret of A i is bounded above by R ( i ) T and the optimal benchmark solution is OPT = { v ∗ 1 , . . . , v ∗ m } , for all j ∈ [ m ] , we can write:

Putting the above inequalities together, we have:

We can use submodularity and monotonicity of g t to write:

We can combine the last two inequalities to write:

Taking the sum over i ∈ [ m ] , we have:

On the other hand, we have:

Combining the last two inequalities, we obtain:

where f = ∑ T t =1 f t . Rearranging the terms, we obtain the (1 -c f e ) -regret bound of the online distorted greedy algorithm as follows:

To see why the online greedy algorithm is incentive-compatible, note that at round t ∈ [ T ] , the loss of expert j ∈ [ K ] for the instance A i ; i ∈ [ K ] is:

If we use π ( i ) j,t +1 to denote the probability of expert j being chosen at round t +1 by A i , we can write:

π ( i ) j,t +1 is linear in glyph[lscript] j,t and expert j does not have control over the term ∏ k ∈ S i -1 ,t glyph[lscript] k,t (because S i -1 ,t is unknown to the expert). Therefore, to maximize π ( i ) j,t +1 , expert j can only aim to minimize E r t ∼ Bern ( b j,t ) [ glyph[lscript] j,t -∑ K s =1 π ( i ) s,t glyph[lscript] s,t ] . Given that the quadratic loss function is proper, we can conclude that A i is incentive-compatible. Moreover, since the online greedy algorithm simply outputs the predictions of {A i } K i =1 , this algorithm is incentive-compatible as well. For the case with c f = 0 , the loss of expert j ∈ [ K ] for each instance A i ; i ∈ [ K ] in both online greedy algorithm and online distorted greedy algorithm is simply 1 m ( glyph[lscript] j,t -1) and the same argument holds.

A.6 Proof of Theorem 7

First, assume that the losses are non-negative. Using the update rule of the algorithm, we can write:

We can use the inequalities 1 -ηx ≥ (1 -η ) x for x ∈ [0 , 1] and 1 -ηx ≥ (1 + η ) -x for x ∈ [ -1 , 0] to write:

Given the inequalities ln(1 -x ) ≥ -x -x 2 and ln(1 + x ) ≥ x -x 2 for x ≤ 1 2 , we have:

Given that L i ∗ ,t = glyph[lscript] i ∗ ,t -π T t glyph[lscript] t , we can write:

where | L ∗ T | and | L T | are the cumulative absolute loss of the benchmark and the algorithm respectively. Therefore, if we set η = min { 1 , √ ln K | L T | + | L ∗ T | } , we obtain an O ( √ max {| L T | , | L ∗ T |} ln K +ln K ) regret bound. Given that R T = L T -L ∗ T , if | L T | = L T < L ∗ T = | L ∗ T | , the regret is negative and if | L T | = L T ≥ L ∗ T = | L ∗ T | , the regret bound is O ( √ | L T | ln K +ln K ) . Therefore, the regret bound O ( √ | L T | ln K +ln K ) always holds.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission studies the problem of online decision-making using predictions given by experts with incentives to behave strategically. In particular, at each time-step, K experts hold a belief about a binary outcome. Each expert reports a prediction to a learner, who picks a set of m experts. The learner then suffers a loss which is a function of both the binary outcome and the chosen experts' true beliefs about the outcome. The goal of the learner is to achieve no-regret with respect to the best-in-hindsight choice of m experts while maintaining incentive compatibility, i.e. reporting their true belief as their prediction is a weakly-dominant strategy for each expert at every time-step. \n\nThe authors study two settings: one in which the learner has a modular utility function and one in which their utility function is submodular. An existing algorithm (WSU, [6]) for the 1-expert version of this problem (i.e. the version of the problem where the learner selects one expert at each time-step) is capable of getting sublinear regret in both settings, albeit at the cost of exponential runtime. For modular utility functions, authors show how to use a variant of the well-known Follow the Perturbed Leader (FTPL) algorithm under a sufficient condition for the perturbation distribution to guarantee both no-regret and approximate incentive compatibility. \n\nFor submodular utility functions, the authors use an \"online distorted greedy algorithm\" to obtain no-regret while maintaining incentive compatibility constraints exactly. At a high level, the online distorted greedy algorithm runs m algorithms for the strategic 1-expert problem concurrently. At each time-step, each of the m sub-algorithms selects an expert, and the learner uses these m experts as their selection. Based on the loss the learner receives, they set the loss of each sub-algorithm in a particular way. The authors use the WSU algorithm of [6] as their sub-algorithm. Along they way, they derive an adaptive regret bound for the WSU algorithm, which may be of independent interest. \n\nFinally, the authors empirically evaluate their two algorithms on a dataset from a FiveThrityEight forecasting competition to predict the outcomes of games in the 2022-2023 NFL football season. They find that both algorithms obtain similar performance in this setting.\n\nSTRENGTHS:\nWhile others have studied the (non-strategic) m-expert problem, as well as the strategic 1-expert problem, the authors are the first to study the strategic m-expert problem. This setting is well-motivated by applications such as forecasting competitions. While the algorithms presented by authors are not particularly novel, their application to the strategic m-expert problem requires non-trivial theoretical analysis, as well as some new ideas, such as (1) the sufficient condition for the perturbation distribution to guarantee approximate incentive compatibility in the modular setting, and (2) the adaptive regret bound for the WSU algorithm used as a subroutine for the submodular setting. \n\nWhile I would normally consider the experimental results on real-world data as a strength, their impact is limited due to the lack of relevant comparisons (see Weaknesses for more details).  \n\nFinally the writing is clear, which makes the authors' contributions easy to understand.\n\nWEAKNESSES:\nThe lack of any sort of lower bound is a weakness, especially since (as the authors point out) the loss function they consider is exp-concave, and thus the regret rates of Hedge in the non-strategic version of the problem where one expert is selected scale only logarithmically in the number of experts (and independently of the time horizon). While it may indeed be the case that going from constant-in-T to T^{1/2} regret rates is the price to pay for incentive compatibility in the m-expert problem, this submission presents no evidence either for or against this claim. \n\nIt would be nice to have provide some background on the current state of the (non-strategic) m-expert problem in the related work. \n\nFinally, the experimental results would be strengthened considerably if (1) the theoretical rates and (2) the regret of the naive application of the WSU algorithm were also plotted. Currently, it is not easy to see how well the algorithms are actually doing without the relevant baselines. Additionally, it would be interesting to see how closely the empirical performance of the algorithms match their corresponding theoretical bounds.\n\nQUESTIONS:\nWhat are the runtimes of both algorithms? \n\nWhat regret rates are obtained for the non-strategic m-experts problem with squared loss?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations of their work.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper generalizes the problem of online binary prediction with expert advice, where at each round the learner can pick m>1 experts and the overall utility is a modular or submodular utility of the chosen experts. The experts are strategic and wish to be selected by the algorithm as often as possible hence they may misreport their beliefs about the events. They design algorithms that are incentive-compatible and achieve sublinear regret in the hindsight.\nPrevious work has studied this problem for m=1. In the case of m>1, previous work have focused on designing no-regret algorithms and did not take into consideration the incentive-compatibility issues. \nTheir algorithm builds on a prior work that studies the FTPL algorithm for the m-expert problem with modular utility functions and derive conditions for the perturbation function to guarantee approximate incentive compatibility. In particular, they show that FTPL while Gaussian perturbations is not incentive-compatible, Laplace or hyperbolic noise distribution guarantees approximate incentive-compatibility.\n\nMoreover, they propose an algorithm that builds upon online monotone submodular maximization s.t. matroid constraints that takes m incentive-compatible for standard experts problem (i.e. m=1) and outputs their combined prediction. Compared to the first algorithm, this one achieves exact incentive-compatibility however, at the price of an extra \\sqrt{m} term in the regret bound.\n\nSTRENGTHS:\nThe paper is written nicely and is easy to follow. The problem seems to be well-motivated. I did not go through the proofs. The second algorithm is more straightforward, however, I think it is nice that they included it and compared it with their first algorithm.\n\nWEAKNESSES:\nIt might be helpful to add a discussion on the regret lower bound for this problem.\n\nQUESTIONS:\nMinor comments:\nThe citations need to be fixed, they do not show the authors' name, need to use \\citet.\nLine 268, can you expand on why this holds?\n\nQuestion : What is the lower bound for the regret of this problem? Can you give a lower bound that is stronger than the case for m=1?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-05"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\n- The paper studies the problem of online prediction with expert advice. In their setting, a learner at each round selects $m$ experts, and the loss is determined by a submodular or modular function of the beliefs reported by the selected experts. Each expert, being strategic, may intentionally misrepresent their beliefs to maximize their chances of being selected in the next round. The goal of the learner is to design no-regret algorithms that incentive experts to report their beliefs truthfully (also called IC).\n\n- To tackle this problem, the authors first present an inefficient reduction to the case when $m=1$. Subsequently, they propose two efficient algorithms that are for the specific types of loss functions considered.\n\n    - In the case of the modular loss function, the authors investigate Follow-The-Perturbed-Leader (FTPL) and propose a general condition for the perturbation distribution that ensures approximate IC.\n\n    - For the submodular loss function,they propose a simpler algorithm named the online distorted greedy algorithm. This algorithm not only guarantees exact IC but also attains the optimal approximation ratio.\n\nSTRENGTHS:\n- The paper studies an important and generic problem of online prediction with strategic experts that may find its applications in many scenarios.\n- The proposed algorithms in this paper are not only easy to implement and also enjoy good theoretical guarantees of being (approximate) IC and no regret.\n\nWEAKNESSES:\n1. Several claims may have some issues, see eg Questions 3, 5, 6.\n2. The presentation could be improved. For instance, Section 1.2 titled \"Contributions\" seems to contain some elements of background and motivations which might be more appropriately placed in the introduction of the paper. Moreover, it would be beneficial to include more discussions following important definitions (like Def 1) and theorems to provide more intuitions.\n3. In the experimental section, the authors plot the regret curve that is not normalized. This makes it hard to interpret, as unnormalized regret curves don't explicitly demonstrate the rate at which the algorithms learn. Additionally, to reflect the difference between reported and true beliefs, a uniformly random value within the range guaranteed by Theorem 3 is added, but this theorem doesn't ensure a uniform distribution for this difference. Finally, incorporating a comparison of the proposed algorithms with and aggregated predictions of FiveThirtyEight could potentially enrich the analysis.\n\nQUESTIONS:\n1. Regarding Def 1: \n    - Is this a new concept being proposed, or has it previously been studied in previous works? \n    - The experts are long-standing in this setting, but the focus of the paper is on myopic IC. Although the authors mentioned that the analysis of FTPL could be extended to the more general setting of maximizing a conic combination of probabilities of being chosen at all subsequent rounds, is it also true for the online distorted greedy algorithm? What's the main modification/challenge of extending to non-myopic experts?\n    - Would achieving IC become impossible if the definition is for all $r_t$, rather than the expectation over Bern( $b_{i,t}$ )?\n\n2. In the \"forecasting competitions\" example described in Section 3.1, it seems that the loss function is not captured by the modular function studied in this paper, given it is normalized by $|S_t|$ rather than $m$.\n\n3. About Thm 1, could you clarify why the reduction preserves IC? In particular:\n    - How is the belief of each meta-expert defined in this context? Is the new loss function $\\ell_S$ still proper? How to define IC when the belief of each expert is multi-dimensional?\n    - Even though WSU is IC for the $m=1$ problem, it only guarantees every single expert reports truthfully when fixing other's reports. In particular, it doesn't rule out the possibility that a group of experts form a coalition and misreport together to increase the sum of their probabilities of being selected, which is exactly what would happen when a single expert misreports in the $m$-expert problem -- it results in the simultaneous misreporting of $\\binom{K}{m-1}$ meta-experts.\n\n4. Thm 3 guarantees approximate IC in the sense of $|p^\\star\\_{i,t}-b\\_{i,t}|$ being small. However, a more common notion of approximate IC is through the difference of utility, which, applies to this setting, is the difference in $\\pi_{i,t+1}$ when $p^\\star_{i,t}$ and $b_{i,t}$ are reported respectively. Could the authors include a discussion about this alternative notion?\n\n5. Combining Thm 3 and Thm 5, it seems that as $B$ approaches 0, the FTPL algorithm becomes more IC and also yields a smaller regret. However, this contradicts the intuition that if the learner's decisions are entirely noise-driven (i.e., $B=0$), the regret should be high. Could the authors comment on this?\n\n6. On lines 297-298, you mention that the $(1-\\frac{c_f}{e})$ approximation factor is optimal. Is this claim supported by a lower bound? If so, could you point to the relevant references?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper broadly focuses on the design and analysis of no-regret and incentive-compatible algorithms for the m-experts problem (pick a subset $S_t \\subseteq [K]$ of $m$ experts in each round $t$ to obtain a utility value $f_t(S_t)$ which is a function of the losses $\\ell_{i,t} \\in [0,1]$ for all $i \\in S_t$), with specific utility functions (on the space of m-sized subsets of experts) in each round which are either modular or sub-modular.\n* They try to improve on the inefficient baseline of using the no-regret, incentive-compatible WSU algorithm (Freeman et all 2020) on the set of $\\binom{K}{m}$ meta-experts corresponding to sets of $m$ experts.\n* They analyze the FTPL algorithm in the modular utility function case where the perturbation distribution is zero-mean symmetric from an exponential family. They propose a sufficient-condition for this distribution (related to bounded hazard rate) which implies that FTPL would be no-regret and approximately incentive-compatible. In particular, they show that this condition is satisfied for the Laplace distribution and the symmetric hyperbolic distribution etc.\n* They propose and analyze an \"online distorted greedy algorithm\" (based on earlier work by (Harvey et al. 2020) on online submodular maximizination) for the submodular utility function case. They show that the algorithm has sublinear $\\alpha$-regret bounds and is $\\alpha$-approximately-incentive compatible. where $\\alpha$ may be $ < 1$ for general submodular functions (depending on the \"curvature\") while $\\alpha = 1$ in the modular case. This of course implies that the algorithm has no-regret in the usual sense and is fully incentive compatible in the modular case, but at the expense of more computation and a slightly worse regret bound compared to FTPL.\n\nSTRENGTHS:\n* The online $m$-experts problem is well-motivated in the introduction, as is the importance of incentive compatibility for the problem. \n* The algorithms proposed (FTPL and online distorted greedy) are natural for the problem and build nicely on previous work. All the appropriate details are given, along with the code and experiments.\n* The proposed sufficiency condition for the perturbation distribution of FTPL is fairly general, and may be of independent interest in the study of incentive compatibility. \n* Sufficient proof-sketches are given for quite a few of the results in the main paper itself, and more detail is provided in the supplementary material.\n\nWEAKNESSES:\n* The FTPL analysis applies to a single modular utility function defined in the paper (Line 136). Of course, this utility function is fairly natural for the problem. \n* The necessity of \"Condition 1\" for getting approximate-incentive-compatibility with FTPL is not explored.\n* The experimental evaluation provided is fairly limited (of course, the thrust of the work is theoretical).\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nNot applicable.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-09"}