Original: Peer Prediction for Learning Agents
Revised: Peer Prediction for Learning Agents

Original: Abstract
Revised: Abstract

Original: Peer prediction refers to a collection of mechanisms for eliciting information from human agents when direct verification of the obtained information is unavailable.
Revised: Peer prediction refers to a collection of mechanisms for eliciting information from human agents when direct verification of the obtained information is unavailable.

Original: They are designed to have a game-theoretic equilibrium where everyone reveals their private information truthfully.
Revised: They are designed to have a game-theoretic equilibrium where everyone reveals their private information truthfully.

Original: This result holds under the assumption that agents are Bayesian and they each adopt a fixed strategy across all tasks.
Revised: This result holds under the assumption that agents are Bayesian and they each adopt a fixed strategy across all tasks.

Original: Human agents however are observed in many domains to exhibit learning behavior in sequential settings.
Revised: Human agents however are observed in many domains to exhibit learning behavior in sequential settings.

Original: In this paper, we explore the dynamics of sequential peer prediction mechanisms when participants are learning agents.
Revised: In this paper, we explore the dynamics of sequential peer prediction mechanisms when participants are learning agents.

Original: We first show that the notion of no regret alone for the agents’ learning algorithms cannot guarantee convergence to the truthful strategy.
Revised: We first show that the notion of no regret alone for the agents’ learning algorithms cannot guarantee convergence to the truthful strategy.

Original: We then focus on a family of learning algorithms where strategy updates only depend on agents’ cumulative rewards and prove that agents’ strategies in the popular Correlated Agreement (CA) mechanism converge to truthful reporting when they use algorithms from this family.
Revised: We then focus on a family of learning algorithms where strategy updates only depend on agents’ cumulative rewards and prove that agents’ strategies in the popular Correlated Agreement (CA) mechanism converge to truthful reporting when they use algorithms from this family.

Original: This family of algorithms is not necessarily no-regret, but includes several familiar no-regret learning algorithms (e.g multiplicative weight update and Follow the Perturbed
Revised: This family of algorithms is not necessarily no-regret, but includes several familiar no-regret learning algorithms (e.g multiplicative weight update and Follow the Perturbed

Original: Leader) as special cases.
Revised: Leader) as special cases.

Original: Simulation of several algorithms in this family as well as the ϵ -greedy algorithm, which is outside of this family, shows convergence to the truthful strategy in the CA mechanism.
Revised: Simulation of several algorithms in this family as well as the ϵ -greedy algorithm, which is outside of this family, shows convergence to the truthful strategy in the CA mechanism.

Original:  Introduction
Revised:  Introduction

Original: A fundamental challenge in many domains is to elicit high-quality information from people when directly verifying the acquired information is not feasible, either because the ground truth is not available or because it’s too costly to obtain.
Revised: A fundamental challenge in many domains is to elicit high-quality information from people when directly verifying the acquired information is not feasible, either because the ground truth is not available or because it is too costly to obtain.

Original: Notable settings include asking people to label data for machine learning, having students perform peer grading in education and soliciting customer feedback for products and services.
Revised: Notable settings include asking people to label data for machine learning, having students perform peer grading in education and soliciting customer feedback for products and services.

Original: The peer prediction literature has made impressive progress on this challenge in the past two decades, with many mechanisms that have desirable incentive properties developed for this problem [20, 19, 27, 5, 22, 8, 14, 25, 17, 16, 21, 18, 23, 15].
Revised: The peer prediction literature has made impressive progress on this challenge in the past two decades, with many mechanisms that have desirable incentive properties developed for this problem [20, 19, 27, 5, 22, 8, 14, 25, 17, 16, 21, 18, 23, 15].

Original: The term peer prediction refers to a collection of reward mechanisms that solicit information from human agents and reward each agent solely based on how the agent’s reported information compares with that of the other agents, without having access to the ground truth.
Revised: The term peer prediction refers to a collection of reward mechanisms that solicit information from human agents and reward each agent solely based on how the agent’s reported information compares with that of the other agents, without having access to the ground truth.

Original: Under some assumptions, many peer prediction mechanisms [20, 19, 27, 22, 8, 16, 21] guarantee that every agent truthfully reporting their information is a game-theoretic equilibrium, and the more recent multi-task peer prediction mechanisms [5, 14, 25, 17, 18, 23, 15] further ensure that agents receive the highest expected payoff at the truthful equilibrium, compared with other strategy profiles.
Revised: Under some assumptions, many peer prediction mechanisms [20, 19, 27, 22, 8, 16, 21] guarantee that every agent truthfully reporting their information is a game-theoretic equilibrium, and the more recent multi-task peer prediction mechanisms [5, 14, 25, 17, 18, 23, 15] further ensure that agents receive the highest expected payoff at the truthful equilibrium, compared with other strategy profiles.

Original: While achieving truthful reporting as a highest-payoff equilibrium is a victory to declare for this challenging without-verification setting, there are however caveats associated with adopting the notion of equilibrium as a solution concept.
Revised: While achieving truthful reporting as a highest-payoff equilibrium is a victory to declare for this challenging without-verification setting, there are however caveats associated with adopting the notion of equilibrium as a solution concept.

Original: The equilibrium results rely on the assumption that participants are fully rational Bayesian agents.
Revised: The equilibrium results rely on the assumption that participants are fully rational Bayesian agents.

Original: Equilibrium is a static notion and doesn’t address how agents, who act independently, jump to play their equilibrium strategies.
Revised: Equilibrium is a static notion and does not address how agents, who act independently, jump to play their equilibrium strategies.

Original: Moreover, the equilibrium results of multi-task peer prediction mechanisms heavily depend on a consistent strategy assumption, that is, each agent is assumed to adopt a fixed strategy across all tasks that she participates.
Revised: Moreover, the equilibrium results of multi-task peer prediction mechanisms heavily depend on a consistent strategy assumption, that is, each agent is assumed to adopt a fixed strategy across all tasks in which she participates.

Original: All together these assumptions exclude the possibility that agents may explore and learn from previous experience, a behavior that’s not only commonly observed in practice but also has been modeled in studying other strategic settings [3, 6, 4].
Revised: All together these assumptions exclude the possibility that agents may explore and learn from previous experience, a behavior that’s not only commonly observed in practice but also has been modeled in studying other strategic settings [3, 6, 4].

Original: This paper is the first theoretical study on the dynamics of sequential peer prediction mechanisms when participants are learning agents.
Revised: This paper is the first theoretical study on the dynamics of sequential peer prediction mechanisms when participants are learning agents.

Original: The main question that we explore is whether and when in sequential peer prediction, learning agents will converge to all playing the truthful reporting strategy.
Revised: The main question that we explore is whether and when in sequential peer prediction, learning agents will converge to all playing the truthful reporting strategy.

Original: We first consider agents adopting no-regret learning algorithms and prove that the notion of no regret alone cannot guarantee convergence to truthful reporting.
Revised: We first consider agents adopting no-regret learning algorithms and prove that the notion of no regret alone cannot guarantee convergence to truthful reporting.

Original: We then define a natural family of reward-based learning algorithms where strategy updates only depend on agents’ cumulative rewards.
Revised: We then define a natural family of reward-based learning algorithms where strategy updates only depend on agents’ cumulative rewards.

Original: While algorithms in this family is not necessarily no-regret (e.g. the Follow the Leader algorithm), this family includes some familiar no-regret learning algorithms, including the Multiplicative Weight
Revised: While algorithms in this family are not necessarily no-regret (e.g. the Follow the Leader algorithm), this family includes some familiar no-regret learning algorithms, including the Multiplicative Weight

Original: Update and the Follow the Perturbed Leader algorithms.
Revised: Update and the Follow the Perturbed Leader algorithms.

Original: Our main result shows that, for the binarysignal setting, agents’ strategies in the popular Correlated Agreement (CA) mechanism [5] converge to truthful reporting when agents use any algorithm from this family.
Revised: Our main result shows that, for the binarysignal setting, agents’ strategies in the popular Correlated Agreement (CA) mechanism [5] converge to truthful reporting when agents use any algorithm from this family.

Original: To prove the result, we show the process has a self-fulfilling property: once Alice and Bob have large accumulated rewards for truth-telling, they are more likely to play truth-telling and resulting in larger accumulated rewards.
Revised: To prove the result, we show the process has a self-fulfilling property: once Alice and Bob have large accumulated rewards for truth-telling, they are more likely to play truth-telling and resulting in larger accumulated rewards.

Original: Theoretically, we carefully partition the process into three stages, bad, intermediate, and good events illustrated in fig.
Revised: Theoretically, we carefully partition the process into three stages, bad, intermediate, and good events illustrated in fig.

Original: 1, and use tools in martingale theory to argue the progress of the process.
Revised: 1, and use tools in martingale theory to argue the progress of the process.

Original: Finally, we simulate the strategy dynamics in the CA mechanism for several algorithms in this family as well as for the ϵ -greedy algorithm, which doesn’t belong to this family.
Revised: Finally, we simulate the strategy dynamics in the CA mechanism for several algorithms in this family as well as for the ϵ -greedy algorithm, which doesn’t belong to this family.

Original: We observe convergence to truthful reporting for all algorithms considered in our simulation, suggesting an interesting future direction to characterize all learning algorithms that converge to truthful reporting.
Revised: We observe convergence to truthful reporting for all algorithms considered in our simulation, suggesting an interesting future direction to characterize all learning algorithms that converge to truthful reporting.

Original: Related Works
Revised: Related Works

Original: This paper relates to two lines of work, information elicitation and mechanisms for learning agents.
Revised: This paper relates to two lines of work, information elicitation and mechanisms for learning agents.

Original: Information Elicitation Mechanisms
Revised: Information Elicitation Mechanisms

Original: The literature on information elicitation without verification focuses on capturing the strategic aspect of human agents.
Revised: The literature on information elicitation without verification focuses on capturing the strategic aspect of human agents.

Original: In multi-task settings, Dasgupta and Ghosh [5] proposed a seminal informed truthful mechanism, the Correlated Agreement (CA) mechanism, for binary positively correlated signals.
Revised: In multi-task settings, Dasgupta and Ghosh [5] proposed a seminal informed truthful mechanism, the Correlated Agreement (CA) mechanism, for binary positively correlated signals.

Original: A series of works then relaxed the binary and positively correlation assumptions [25, 17, 23, 15].
Revised: A series of works then relaxed the binary and positively correlation assumptions [25, 17, 23, 15].

Original: Additionally, Zheng et al.
Revised: Additionally, Zheng et al.

Original: [28] study the limitation of information elicitation in the multi-task setting.
Revised: [28] study the limitation of information elicitation in the multi-task setting.

Original: However, all of the above works assume agents using consistent strategies that are identical across all tasks.
Revised: However, all of the above works assume agents using consistent strategies that are identical across all tasks.

Original: The consistent strategy assumption excludes the possibility of agent learning.
Revised: The consistent strategy assumption excludes the possibility of agent learning.

Original: Our work removes the consistent strategy assumption and explicitly considers learning agents.
Revised: Our work removes the consistent strategy assumption and explicitly considers learning agents.

Original: We theoretically prove truthful convergence of the CA mechanism when agents using algorithms from a family of reward-based online learning algorithms.
Revised: We theoretically prove truthful convergence of the CA mechanism when agents using algorithms from a family of reward-based online learning algorithms.

Original: Our work of considering learning agents can be viewed as a way of testing the robustness of information elicitation mechanisms with respect to deviation from the rational Bayesian agent model.
Revised: Our work of considering learning agents can be viewed as a way of testing the robustness of information elicitation mechanisms with respect to deviation from the rational Bayesian agent model.

Original: From this perspective, Shnayder et al.
Revised: From this perspective, Shnayder et al.

Original: [26] is closely related to ours.
Revised: [26] is closely related to ours.

Original: They consider sequential information elicitation and empirically study if agents using replicator dynamics can converge to truth-telling in the CA mechanism and several other mechanisms.
Revised: They consider sequential information elicitation and empirically study if agents using replicator dynamics can converge to truth-telling in the CA mechanism and several other mechanisms.

Original: Our work theoretically proves that, besides replicator dynamics, learning agents can converge to truth-telling in the CA mechanism when they use a general family of learning algorithms.
Revised: Our work theoretically proves that, besides replicator dynamics, learning agents can converge to truth-telling in the CA mechanism when they use a general family of learning algorithms.

Original: Additionally, Schoenebeck et al.
Revised: Additionally, Schoenebeck et al.

Original: [24] designed an information elicitation mechanism that was robust against a small fraction of adversarial agents.
Revised: [24] designed an information elicitation mechanism that was robust against a small fraction of adversarial agents.

Original: Mechanisms for Learning Agents Several works in economics and computer science try to design mechanisms for learning agents, rather than for rational, Bayesian ones.
Revised: Mechanisms for Learning Agents Several works in economics and computer science try to design mechanisms for learning agents, rather than for rational, Bayesian ones.

Original: Braverman et al.
Revised: Braverman et al.

Original: [3] studied pricing mechanisms for learning agents with no external regrets.
Revised: [3] studied pricing mechanisms for learning agents with no external regrets.

Original: Their work was generalized by
Revised: Their work was generalized by

Original: Deng et al.
Revised: Deng et al.

Original: [6] to consider repeated Stackelberg games in full-information settings.
Revised: [6] to consider repeated Stackelberg games in full-information settings.

Original: Camara et al.
Revised: Camara et al.

Original: [4] further proposed counterfactual internal regrets (CIR) together with no-CIR assumption, which was proved to be a sufficient behavior assumption for no-regret principal mechanism design in repeated stage games.
Revised: [4] further proposed counterfactual internal regrets (CIR) together with no-CIR assumption, which was proved to be a sufficient behavior assumption for no-regret principal mechanism design in repeated stage games.

Original: However, all of these works focus on a single agent or full-information games, while peer prediction is an incomplete-information game with multiple agents.
Revised: However, all of these works focus on a single agent or full-information games, while peer prediction is an incomplete-information game with multiple agents.

Original: Finally, our goal is slightly different from that of most sequential mechanism design.
Revised: Finally, our goal is slightly different from that of most sequential mechanism design.

Original: Instead of maximizing the mechanism designer’s utility, our goal is to incentivize truthful reporting from agents.
Revised: Instead of maximizing the mechanism designer’s utility, our goal is to incentivize truthful reporting from agents.

Original:  Peer Prediction Settings
Revised:  Peer Prediction Settings

Original: For simplicity we consider two agents, Alice and Bob, who work on a sequence of tasks indexed by t ≥ 1 .
Revised: For simplicity we consider two agents 1 , Alice and Bob, who work on a sequence of tasks indexed by t ≥ 1 .

Original: For round t , both agents work on task t , Alice receives a signal X t = x t in { 0 , 1 } , and Bob a signal Y t =
Revised: For round t , both agents work on task t , Alice receives a signal X t = x t in { 0 , 1 } , and Bob a signal Y t =

Original: y t in { 0 , 1 } where X t and Y t denote random variables, and x t and y t are their realizations.
Revised: y t in { 0 , 1 } where X t and Y t denote random variables, and x t and y t are their realizations.

Original: Then Alice and Bob report ˆ X t = ˆ x t and ˆ Y t = ˆ y t in { 0 , 1 } .
Revised: Then Alice and Bob report ˆ X t = ˆ x t and ˆ Y t = ˆ y t in { 0 , 1 } .

Original: We define
Revised: We define

Original: X ≤ t = { X s : 1 ≤ s ≤ t } ∈ { 0 , 1 } t and ˆ X ≤ t = { ˆ X s : 1 ≤ s ≤ t } ∈ { 0 , 1 } t to denote Alice’s signal profile and report profiles until t -th round respectively, and define Y ≤ t and ˆ Y ≤ t for Bob similarly.
Revised: X ≤ t = { X s : 1 ≤ s ≤ t } ∈ { 0 , 1 } t and ˆ X ≤ t = { ˆ X s : 1 ≤ s ≤ t } ∈ { 0 , 1 } t to denote Alice’s signal profile and report profiles until t -th round respectively, and define Y ≤ t and ˆ Y ≤ t for Bob similarly.

Original: We use X , ˆ X , Y , and ˆ Y for the complete signal and report profiles.
Revised: We use X , ˆ X , Y , and ˆ Y for the complete signal and report profiles.

Original: Additionally, we consider the signals are generated from some distribution P that satisfies the following assumptions:
Revised: Additionally, we consider the signals are generated from some distribution P that satisfies the following assumptions:

Original: Assumption 2.1 (A priori similar tasks [5]) .
Revised: Assumption 2.1 (A priori similar tasks [5]) .

Original: Each pair of signal is identically and independently (i.i.d.) generated: there exists a distribution P X,Y over { 0 , 1 } 2 such that ( X t , Y t ) ∼ P X,Y for any t ∈ N + .
Revised: Each pair of signal is identically and independently (i.i.d.) generated: there exists a distribution P X,Y over { 0 , 1 } 2 such that ( X t , Y t ) ∼ P X,Y for any t ∈ N + .

Original: Moreover, we assume the distribution has full support, P X,Y ( x, y ) > 0 for all x, y ∈ { 0 , 1 } .
Revised: Moreover, we assume the distribution has full support, P X,Y ( x, y ) > 0 for all x, y ∈ { 0 , 1 } .

Original: Assumption 2.2 (Positively correlated signals) .
Revised: Assumption 2.2 (Positively correlated signals) .

Original: The distribution P X,Y is positively correlated, min { P X,Y (1 , 1) , P X,Y (0 , 0) }
Revised: The distribution P X,Y is positively correlated, min { P X,Y (1 , 1) , P X,Y (0 , 0) }

Original: > max { P X,Y (1 , 0) , P X,Y (0 , 1) } .
Revised: > max { P X,Y (1 , 0) , P X,Y (0 , 1) } .

Original: Now we introduce multi-task peer prediction mechanisms and sequential peer prediction mechanisms, and their relation.
Revised: Now we introduce multi-task peer prediction mechanisms and sequential peer prediction mechanisms, and their relation.

Original: We will focus on the sequential setting.
Revised: We will focus on the sequential setting.

Original: Multi-task peer prediction mechanisms work on a fixed number of tasks.
Revised: Multi-task peer prediction mechanisms work on a fixed number of tasks.

Original: Formally, a multi-task peer prediction mechanism on k tasks is a pair of payment functions ¯ M : { 0 , 1 } k
Revised: Formally, a multi-task peer prediction mechanism on k tasks is a pair of payment functions ¯ M : { 0 , 1 } k

Original: → [0 , 1] 2 .
Revised: → [0 , 1] 2 .

Original: For instance, the (multi-task) correlated agreement mechanism (CA mechanism) 2 [5, 25, 26] is ¯ M CA
Revised: For instance, the (multi-task) correlated agreement mechanism (CA mechanism) 2 [5, 25, 26] is ¯ M CA

Original: (ˆ x , ˆ y )
Revised: (ˆ x , ˆ y )

Original: =
Revised: =

Original: ( I [ˆ x 2 = ˆ y 2 ] − I [ˆ x 2 = ˆ y 1 ] , I [ˆ y 

Revised: ( I [ˆ x 2 = ˆ y 2 ] − I [ˆ x 2 = ˆ y 1 ] , I [ˆ y

Original: =
Revised: =

Original: ˆ x 2 ] − I [ˆ y 2 =
Revised: ˆ x 2 ] − I [ˆ y 2 =

Original: ˆ x 1 ]) for all ˆ x , ˆ y ∈ { 0 , 1 } 2 .
Revised: ˆ x 1 ]) for all ˆ x , ˆ y ∈ { 0 , 1 } 2 .

Original: Intuitively, the CA mechanism rewards agreement on the same task and punishes agreement on uncorrelated tasks.
Revised: Intuitively, the CA mechanism rewards agreement on the same task and punishes agreement on uncorrelated tasks.

Original: A sequential information elicitation mechanism is a sequence of payment functions M =
Revised: A sequential information elicitation mechanism is a sequence of payment functions M =

Original: { M t : t ≥ 1 } where M t : { 0 , 1 } 2 × t → [ − 1 , 1] for all t .
Revised: { M t : t ≥ 1 } where M t : { 0 , 1 } 2 × t → [ − 1 , 1] for all t .

Original: After Alice and Bob reporting ˆ x ≤ t and ˆ y ≤ t in round t , the mechanism computes ( r t , s t ) :
Revised: After Alice and Bob reporting ˆ x ≤ t and ˆ y ≤ t in round t , the mechanism computes ( r t , s t ) :

Original: = M t (ˆ x ≤ t , ˆ y ≤ t ) and pay r t to Alice and s t to Bob.
Revised: = M t (ˆ x ≤ t , ˆ y ≤ t ) and pay r t to Alice and s t to Bob.

Original: Here we assume M t can only depends on a constant k round of reports so that M t (ˆ x ≤ t , ˆ y ≤ t ) =
Revised: Here we assume M t can only depends on a constant k round of reports so that M t (ˆ x ≤ t , ˆ y ≤ t ) =

Original: M t (ˆ x t − k +1 , ˆ x t − k +1 , . . .
Revised: M t (ˆ x t − k +1 , ˆ x t − k +1 , . . .

Original: , ˆ x t , ˆ y t − k +1 , ˆ y t − k +1 , . . .
Revised: , ˆ x t , ˆ y t − k +1 , ˆ y t − k +1 , . . .

Original: , ˆ y t ) for all t , ˆ x ≤ t , and ˆ y ≤ t , and we call such M rank k mechanism .
Revised: , ˆ y t ) for all t , ˆ x ≤ t , and ˆ y ≤ t , and we call such M rank k mechanism .

Original: For instance, the (multi-task)
Revised: For instance, the (multi-task)

Original: CA mechanism can be adopted as a sequential rank 2 information elicitation mechanism: At round t , the payment is
Revised: CA mechanism can be adopted as a sequential rank 2 information elicitation mechanism: At round t , the payment is

Original: [Equation]
Revised: [Equation]

Original: where ˆ x 0 and ˆ y 0 are set as 0 .
Revised: where ˆ x 0 and ˆ y 0 are set as 0 .

Original: Similarly, we say a sequential information elicitation mechanism
Revised: Similarly, we say a sequential information elicitation mechanism

Original: M = ( M t ) t ≥ 1 is a sequential version of a multi-task information elicitation mechanism ¯ M if
Revised: M = ( M t ) t ≥ 1 is a sequential version of a multi-task information elicitation mechanism ¯ M if

Original: M t (ˆ x ≤ t , ˆ y ≤ t ) =
Revised: M t (ˆ x ≤ t , ˆ y ≤ t ) =

Original: ¯ M (ˆ x t − k
Revised: ¯ M (ˆ x t − k

Original: +1 , ˆ x t − k +1 , . .
Revised: +1 , ˆ x t − k +1 , . .

Original: , ˆ x t , ˆ y t − k +1 , ˆ y t − k +1 , . . .
Revised: , ˆ x t , ˆ y t − k +1 , ˆ y t − k +1 , . . .

Original: , ˆ y t ) for all ˆ x ≤ t , ˆ y ≤ t and t ≥ k .
Revised: , ˆ y t ) for all ˆ x ≤ t , ˆ y ≤ t and t ≥ k .

Original: The payment at round t ≥ k is ¯ M on the latest k reports.
Revised: The payment at round t ≥ k is ¯ M on the latest k reports.

Original: Conversely, a sequential information elicitation mechanism can be seen as a sequence of multi-task information elicitation mechanisms.
Revised: Conversely, a sequential information elicitation mechanism can be seen as a sequence of multi-task information elicitation mechanisms.

Original: Now we formally define agents’ strategies.
Revised: Now we formally define agents’ strategies.

Original: Due to symmetry, we introduce notation for Alice and omit Bob’s.
Revised: Due to symmetry, we introduce notation for Alice and omit Bob’s.

Original: Given an information elicitation mechanism M , at round t , Alice observes her signal x t and decides on her report ˆ x t .
Revised: Given an information elicitation mechanism M , at round t , Alice observes her signal x t and decides on her report ˆ x t .

Original: Thus, Alice has four options (pure strategies) : 1) opt 1 : report the private signal truthfully, 2) opt 2 : flip the private signal, 3) opt 3 : report 1 regardless of the signal, and 4) opt 4 : report 0 regardless of the signal.
Revised: Thus, Alice has four options (pure strategies) : 1) opt 1 : report the private signal truthfully, 2) opt 2 : flip the private signal, 3) opt 3 : report 1 regardless of the signal, and 4) opt 4 : report 0 regardless of the signal.

Original: We call opt 3 and opt 4 uninformative strategies.
Revised: We call opt 3 and opt 4 uninformative strategies.

Original: We use opt Xt to denote Alice’s pure strategy, and r t for her payoff at round t .
Revised: We use opt Xt to denote Alice’s pure strategy, and r t for her payoff at round t .

Original: At each round t , Alice knows her previous signals x ≤ t ∈ { 0 , 1 } t , her pure strategies opt X 1 , . .
Revised: At each round t , Alice knows her previous signals x ≤ t ∈ { 0 , 1 } t , her pure strategies opt X 1 , . .

Original: . , opt Xt − 1 , and Bob’s reports ˆ y ≤ t − 1 , so we use F t = { x ≤ t , opt X ≤ t − 1 , ˆ y ≤ t − 1 } to denote Alice knowledge at round t .
Revised: . , opt Xt − 1 , and Bob’s reports ˆ y ≤ t − 1 , so we use F t = { x ≤ t , opt X ≤ t − 1 , ˆ y ≤ t − 1 } to denote Alice knowledge at round t .

Original: Thus, Alice’s mixed strategy at round t is a stochastic mapping σ Xt from F t to { opt 1 , opt 2 , opt 3 , opt 4 } .
Revised: Thus, Alice’s mixed strategy at round t is a stochastic mapping σ Xt from F t to { opt 1 , opt 2 , opt 3 , opt 4 } .

Original: We’ll abuse our nation and also use σ Xt = opt Xt to represent the realized pure strategy.
Revised: We’ll abuse our nation and also use σ Xt = opt Xt to represent the realized pure strategy.

Original: Finally, a learning algorithm of Alice is a mapping from an information elicitation mechanism M to her strategies.
Revised: Finally, a learning algorithm of Alice is a mapping from an information elicitation mechanism M to her strategies.

Original: Strongly Truthful for Rational and Bayesian agents Previous works on information elicitation try to ensure truth-telling opt 1 is the best strategy for rational and strategic agents.
Revised: Strong Truthfulness for Rational and Bayesian agents Previous works on information elicitation try to ensure truth-telling opt 1 is the best strategy for rational and strategic agents.

Original: In particular, a mechanism is strongly truthful if Alice and Bob report truthfully is a Bayesian Nash Equilibrium (BNE) and they get strictly higher payment at this BNE than at any other non-permutation BNE.
Revised: In particular, a mechanism is strongly truthful if Alice and Bob report truthfully is a Bayesian Nash Equilibrium (BNE) and they get strictly higher payment at this BNE than at any other non-permutation BNE.

Original: We present the formal definitions in the appendix.
Revised: We present the formal definitions in the appendix.

Original: Informally, in a permutation BNE, every agent’s strategy on each round is a permutation/bijection from his/her signals to reports.
Revised: Informally, in a permutation BNE, every agent’s strategy on each round is a permutation/bijection from his/her signals to reports.

Original: However, the equilibrium results of previous mechanisms not only require assumption 2.1 but further assume agents using consistent strategies .
Revised: However, the equilibrium results of previous mechanisms not only require assumption 2.1 but further assume agents using consistent strategies .

Original: Specifically, Alice uses a consistent strategy if there is a fixed distribution on { opt 1 , opt 2 , opt 3 , opt 4 } so that opt Xt is generated from a fixed distribution that is independent of her private signals on other tasks and the round number.
Revised: Specifically, Alice uses a consistent strategy if there is a fixed distribution on { opt 1 , opt 2 , opt 3 , opt 4 } so that opt Xt is generated from a fixed distribution that is independent of her private signals on other tasks and the round number.

Original: For instance, when Alice and
Revised: For instance, when Alice and

Original: Bob are Bayesian and use consistent strategies under assumption 2.2 and 2.1, Dasgupta and Ghosh [5] show CA mechanism in eq.
Revised: Bob are Bayesian and use consistent strategies under assumption 2.2 and 2.1, Dasgupta and Ghosh [5] show CA mechanism in eq.

Original: (1) is strongly truthful.
Revised: (1) is strongly truthful.

Original: Intuitively, positive correlation assumption 2.
Revised: Intuitively, positive correlation assumption 2.

Original: guarantees that truthful reporting can maximize the chance of agreeing with the peer on the same task while avoiding agreeing on reports on other tasks.
Revised: guarantees that truthful reporting can maximize the chance of agreeing with the peer on the same task while avoiding agreeing on reports on other tasks.

Original: Furthermore, in appendix B we show CA mechanism merely has three types Bayesian Nash equilibria, at which both agents 1) play
Revised: Furthermore, in appendix B we show CA mechanism merely has three types Bayesian Nash equilibria, at which both agents 1) play

Original: truth-telling opt 1 , 2) flip the signal opt 2 , or 3) generate uninformative reports (mixture between opt 3 , opt 4 ) when agents use consistent strategies and assumptions 2.1 and 2.2 hold.
Revised: truth-telling opt 1 , 2) flip the signal opt 2 , or 3) generate uninformative reports (mixture between opt 3 , opt 4 ) when agents use consistent strategies and assumptions 2.1 and 2.2 hold.

Original: Truthful convergence for learning agents
Revised: Truthful Convergence for Learning Agents

Original: However, as we consider agents using a family of online learning algorithms to decide their strategies, standard solution concepts like Bayesian Nash equilibrium no longer apply.
Revised: However, as we consider agents using a family of online learning algorithms to decide their strategies, standard solution concepts like Bayesian Nash equilibrium no longer apply.

Original: Additionally, online learning algorithms often have exploration, so we cannot hope agents will always use the truth-telling strategy.
Revised: Additionally, online learning algorithms often have exploration, so we cannot hope agents will always use the truth-telling strategy.

Original: For learning agents, our goal is to test whether existing mechanisms can ensure that agents will converge to truthful reporting when they deploy certain learning behavior that goes beyond obliviously consistent strategies.
Revised: For learning agents, our goal is to test whether existing mechanisms can ensure that agents will converge to truthful reporting when they deploy certain learning behavior that goes beyond obliviously consistent strategies.

Original: We now formalize the convergence of algorithms to truthful reporting.
Revised: We now formalize the convergence of algorithms to truthful reporting.

Original: Because we want to elicit information without verification, it is information-theoretically impossible for us to separate permutation equilibrium, where all agents play opt 2 , from truthful equilibrium, where all agents play opt 1 , without any additional information [17].
Revised: Because we want to elicit information without verification, it is information-theoretically impossible for us to separate permutation equilibrium, where all agents play opt 2 , from truthful equilibrium, where all agents play opt 1 , without any additional information [17].

Original: However, if we have an additional bit of information on whether the prior of 0 is larger than 1 , we may tell apart these two equilibria.
Revised: However, if we have an additional bit of information on whether the prior of 0 is larger than 1 , we may tell apart these two equilibria.

Original: We hence define convergence to truthful reporting as the limits of both opt Xt and opt Yt being truth-telling (opt 1 ) or flipping (opt 2 ).
Revised: We hence define convergence to truthful reporting as the limits of both opt Xt and opt Yt being truth-telling (opt 1 ) or flipping (opt 2 ).

Original: Note that definition 2.3 requires almost surely convergence which is very strong convergence concept.
Revised: Note that definition 2.3 requires almost surely convergence which is very strong convergence concept.

Original: Definition 2.3.
Revised: Definition 2.3.

Original: An information elicitation mechanism M achieves truthful convergence for agents using algorithms A 1 and A 2 respectively if and only if both sequences of pure strategies converge to truth-telling or both flipping the reports.
Revised: An information elicitation mechanism M achieves truthful convergence for agents using algorithms A 1 and A 2 respectively if and only if both sequences of pure strategies converge to truth-telling or both flipping the reports.

Original: [Equation]
Revised: [Equation]

Original:  Online Learning Algorithms
Revised:  Online Learning Algorithms

Original: In this section, we explore candidates to model agents’ learning behavior to replace Bayesian agents’ consistent strategies in the literature.
Revised: In this section, we explore candidates to model agents’ learning behavior to replace Bayesian agents’ consistent strategies in the literature.

Original: We first show the conventional no-regret assumption is a necessary but not a sufficient condition for truthful convergence in section 3.1.
Revised: We first show the conventional no-regret assumption is a necessary but not a sufficient condition for truthful convergence in section 3.1.

Original: Then in section 3.2, we introduce a family of reward-based online learning algorithm to model agents’ learning behavior, and show that the family of reward-based online learning algorithms contains several common no-regret algorithms as special cases.
Revised: Then in section 3.2, we introduce a family of reward-based online learning algorithm to model agents’ learning behavior, and show that the family of reward-based online learning algorithms contains several common no-regret algorithms as special cases.

Original: 3.1 No-regret online learning algorithms
Revised: 3.1 No-regret Online Learning Algorithms

Original: We now investigate the relationship between no regret and truthful convergence.
Revised: We now investigate the relationship between no regret and truthful convergence.

Original: First, we show general no-regret algorithms may not ensure truthful convergence (theorem 3.1).
Revised: First, we show general no-regret algorithms may not ensure truthful convergence (theorem 3.1).

Original: However, we show the converse is almost true (theorem 3.2): If truthful convergence happens, the agents do not have regret when the sequential mechanism is a sequential version of a strongly truthful mechanism.
Revised: However, we show the converse is almost true (theorem 3.2): If truthful convergence happens, the agents do not have regret when the sequential mechanism is a sequential version of a strongly truthful mechanism.

Original: Given a sequential information elicitation mechanism M =
Revised: Given a sequential information elicitation mechanism M =

Original: ( M Xt , M Yt ) t ≥ 1 , signals x , y , and reports ˆ x , ˆ y , we define r
Revised: ( M Xt , M Yt ) t ≥ 1 , signals x , y , and reports ˆ x , ˆ y , we define r

Original: i,t = M Xt ( opt i ( x 1 ) , . .
Revised: i,t = M Xt ( opt i ( x 1 ) , . .

Original: , opt i ( x t ) , ˆ y ≤ t ) be the payoff when Alice uses strategy opt i and Bob’s choices are unchanged.
Revised: , opt i ( x t ) , ˆ y ≤ t ) be the payoff when Alice uses strategy opt i and Bob’s choices are unchanged.

Original: Then Alice’s regret is Reg X ( T ) =
Revised: Then Alice’s regret is Reg X ( T ) =

Original: max i (cid:80) t ≤ T r i,t − (cid:80) t ≤ T r t .
Revised: max i (cid:80) t ≤ T r i,t − (cid:80) t ≤ T r t .

Original: Finally, we say that Alice’s and Bob’s online learning algorithms are no regret (on M ) if
Revised: Finally, we say that Alice’s and Bob’s online learning algorithms are no regret (on M ) if

Original: E [ Reg X ( T )]
Revised: E [ Reg X ( T )]

Original: =
Revised: =

Original: E [ Reg Y ( T )]
Revised: E [ Reg Y ( T )]

Original: = o ( T ) over the randomness of signals and the algorithms, and we say
Revised: = o ( T ) over the randomness of signals and the algorithms, and we say

Original: Alice and Bob are no regret for short.
Revised: Alice and Bob are no regret for short.

Original: One may hope that no regret as a behavior assumption for agents is sufficient for achieving desirable outcome in a mechanism.
Revised: One may hope that no regret as a behavior assumption for agents is sufficient for achieving desirable outcome in a mechanism.

Original: However, the following theorem shows that we cannot have an information elicitation mechanism that achieves truthful convergence for all no-regret agents.
Revised: However, the following theorem shows that we cannot have an information elicitation mechanism that achieves truthful convergence for all no-regret agents.

Original: Theorem 3.1.
Revised: Theorem 3.1.

Original: For any sequential information elicitation mechanism M of rank k ∈ N , there exist no-regret algorithms for Alice and Bob so that M cannot achieve truthful convergence.
Revised: For any sequential information elicitation mechanism M of rank k ∈ N , there exist no-regret algorithms for Alice and Bob so that M cannot achieve truthful convergence.

Original: The main idea of the proof is that the no-regret assumption cannot prevent Alice and Bob from colluding.
Revised: The main idea of the proof is that the no-regret assumption cannot prevent Alice and Bob from colluding.

Original: In our counterexample, Alice and Bob decide on a no-regret sequence of reports regardless of their signals once the mechanism is announced.
Revised: In our counterexample, Alice and Bob decide on a no-regret sequence of reports regardless of their signals once the mechanism is announced.

Original: Technically, we use  probabilistic method to show the existence of a deterministic and no-regret sequence of strategies ( opt Xt , opt Yt ) t ≥ 1 that consists of reporting 1 or 0 regardless of private signal, i.e. opt 3 or opt 4 .
Revised: Technically, we use a probabilistic method to show the existence of a deterministic and no-regret sequence of strategies ( opt Xt , opt Yt ) t ≥ 1 that consists of reporting 1 or 0 regardless of private signals, i.e. opt 3 or opt 4 .

Original: The formal proof is in appendix C.1.
Revised: The formal proof is in appendix C.1.

Original: The notion of truthful convergence in definition 2.3 provides an ideal truthful guarantee to the mechanism designer.
Revised: The notion of truthful convergence in definition 2.3 provides an ideal truthful guarantee to the mechanism designer.

Original: Here we show that truthful convergence also ensures no regret for agents when the sequential mechanism is a sequential version of a strongly truthful one-shot multi-task mechanism.
Revised: Here we show that truthful convergence also ensures no regret for agents when the sequential mechanism is a sequential version of a strongly truthful one-shot multi-task mechanism.

Original: That is, when the one-shot mechanism admits truthful reporting as a highest-payoff BNE.
Revised: That is, when the one-shot mechanism admits truthful reporting as a highest-payoff BNE.

Original: For instance, if a pair of algorithms exhibits truthful convergence on the sequential CA mechanism (eq.
Revised: For instance, if a pair of algorithms exhibits truthful convergence on the sequential CA mechanism (eq.

Original: , they are also no regret (on the game).
Revised: , they are also no regret (on the game).

Original: Intuitively, if Bob converges to the truth-telling lim t →∞ opt Yt = opt 1 , the average expected gain of Alice deviating to opt i is equal to the expected gain of deviating to opt i when Bob always tells the truth.
Revised: Intuitively, if Bob converges to the truth-telling lim t →∞ opt Yt = opt 1 , the average expected gain of Alice deviating to opt i is equal to the expected gain of deviating to opt i when Bob always tells the truth.

Original: The gain is non positive because the
Revised: The gain is non positive because the

Original: CA mechanism is strictly truthful by lemma B.4.Therefore, the expected regrets E [ Reg X ( T )] and E [ Reg Y ( T )] are small.
Revised: CA mechanism is strictly truthful by lemma B.4.Therefore, the expected regrets E [ Reg X ( T )] and E [ Reg Y ( T )] are small.

Original: Theorem 3.2 formalizes and extends the above idea to any strongly truthful multi-task information elicitation mechanism.
Revised: Theorem 3.2 formalizes and extends the above idea to any strongly truthful multi-task information elicitation mechanism.

Original: Theorem 3.2.
Revised: Theorem 3.2.

Original: Let ¯ M be a strongly truthful multi-task information elicitation mechanism, and M be a sequential version of ¯ M .
Revised: Let ¯ M be a strongly truthful multi-task information elicitation mechanism, and M be a sequential version of ¯ M .

Original: If M achieve truthful convergence for Alice and Bob using algorithms Aand A 2 respectively, then Alice and Bob are no regret.
Revised: If M achieve truthful convergence for Alice and Bob using algorithms Aand A 2 respectively, then Alice and Bob are no regret.

Original: 3.2 Reward-based Online Learning Algorithms
Revised: 3.2 Reward-based Online Learning Algorithms

Original: As shown in theorem 3.1, the no-regret assumption allow does not guarantee truthful convergence.
Revised: As shown in theorem 3.1, the no-regret assumption allow does not guarantee truthful convergence.

Original: In this section, we introduce a general family of online learning algorithms, reward-based online learning algorithm , under the general full feedback bandit setting, and we will apply these algorithms in sequential information elicitation mechanisms later.
Revised: In this section, we introduce a general family of online learning algorithms, reward-based online learning algorithm , under the general full feedback bandit setting, and we will apply these algorithms in sequential information elicitation mechanisms later.

Original: Informally, a reward-based online learning algorithm decides each round’s strategy using a fixed update function that depends only on the accumulative reward.
Revised: Informally, a reward-based online learning algorithm decides each round’s strategy using a fixed update function that depends only on the accumulative reward.

Original: For simplicity, we only consider algorithms on four strategies { opt 1 , opt 2 , opt 3 , opt 4 } which can be extended easily.
Revised: For simplicity, we only consider algorithms on four strategies { opt 1 , opt 2 , opt 3 , opt 4 } which can be extended easily.

Original: Recall that the payoff of choosing option opt i , i ∈ [4] at round t is r i,j when others’ choices are unchanged.
Revised: Recall that the payoff of choosing option opt i , i ∈ [4] at round t is r i,j when others’ choices are unchanged.

Original: We denote the accumulated payoffs of these four options as R i,t = (cid:80) tj =1 r i,j for i ∈ [4] .
Revised: We denote the accumulated payoffs of these four options as R i,t = (cid:80) tj =1 r i,j for i ∈ [4] .

Original: Symmetrically, we use S i,t and s i,t to represent accumulated payoffs and the payoff of turning to choose option opt i in the t th round for Bob.
Revised: Symmetrically, we use S i,t and s i,t to represent accumulated payoffs and the payoff of turning to choose option opt i in the t th round for Bob.

Original: For example, for our specific peer prediction game using CA mechanism in eq.
Revised: For example, for our specific peer prediction game using CA mechanism in eq.

Original: (1), r 1 ,t
Revised: (1), r 1 ,t

Original: = I
Revised: = I

Original: [ x t = ˆ y t ] − I [ x t = ˆ y t − 1 ] and therefore,
Revised: [ x t = ˆ y t ] − I [ x t = ˆ y t − 1 ] and therefore,

Original: R 1 ,t = (cid:80) tj =1 ( I [ x t = ˆ y t ] − I [ x t = ˆ y t − 1 ]) for Alice.
Revised: R 1 ,t = (cid:80) tj =1 ( I [ x t = ˆ y t ] − I [ x t = ˆ y t − 1 ]) for Alice.

Original: We consider a family of reward-based online learning algorithms
Revised: We consider a family of reward-based online learning algorithms

Original: A that use an update function f : R 4 → △ 3 , and choose opt i with probability f i ( R 1 ,t − 1 , R 2 ,t − 1 , R 3 ,t − 1 , R 4 ,t − 1 ) for i ∈ [4] in the t th round, where f i is the i th coordinate of f .
Revised: A that use an update function f : R 4 → △ 3 , and choose opt i with probability f i ( R 1 ,t − 1 , R 2 ,t − 1 , R 3 ,t − 1 , R 4 ,t − 1 ) for i ∈ [4] in the t th round, where f i is the i th coordinate of f .

Original: A such mechanism based on accumulated payoffs is denoted by A f .
Revised: A such mechanism based on accumulated payoffs is denoted by A f .

Original: We have three assumptions for the update function f , which are all very natural.
Revised: We have three assumptions for the update function f , which are all very natural.

Original: The first two require that f is exchangeable and preserves ordering.
Revised: The first two require that f is exchangeable and preserves ordering.

Original: Assumption 3.3 (Exchangeability of f ) .
Revised: Assumption 3.3 (Exchangeability of f ) .

Original: For any R 1 , R 2 , R 3 , R 4 ∈ R and an arbitrary permutation of them R i 1 , R i 2 , R i 3 , R i 4 , f i j ( R 1 , R 2 , R 3 , R 4 ) =
Revised: For any R 1 , R 2 , R 3 , R 4 ∈ R and an arbitrary permutation of them R i 1 , R i 2 , R i 3 , R i 4 , f i j ( R 1 , R 2 , R 3 , R 4 ) =

Original: f j ( R i 1 , R i 2 , R i 3 , R i 4 ) for all j ∈ [4] .
Revised: f j ( R i 1 , R i 2 , R i 3 , R i 4 ) for all j ∈ [4] .

Original: Assumption 3.4 (Order preservation of f ) .
Revised: Assumption 3.4 (Order preservation of f ) .

Original: For any R 1 , R 2 , R 3 , R 4 ∈ R and suppose that
Revised: For any R 1 , R 2 , R 3 , R 4 ∈ R and suppose that

Original: R i 1 , R i 2 , R i 3 , R i 4 is a non-increasing order of them, for f we have f i 

Revised: R i 1 , R i 2 , R i 3 , R i 4 is a non-increasing order of them, for f we have f i

Original: ( R 1 , R 2 , R 3 , R 4 ) ≥ f i 2 ( R 1 , R 2 , R 3 , R 4 ) ≥ f i 3 ( R 1 , R 2 , R 3 , R 4 ) ≥
Revised: ( R 1 , R 2 , R 3 , R 4 ) ≥ f i 2 ( R 1 , R 2 , R 3 , R 4 ) ≥ f i 3 ( R 1 , R 2 , R 3 , R 4 ) ≥

Original: f i 4 ( R 1 , R 2 , R 3 , R 4 ) .
Revised: f i 4 ( R 1 , R 2 , R 3 , R 4 ) .

Original: Finally we consider the strategy chosen by the update function f when the accumulated payoff of an strategy is much higher than that of other strategies (assumption 3.5).
Revised: Finally we consider the strategy chosen by the update function f when the accumulated payoff of an strategy is much higher than that of other strategies (assumption 3.5).

Original: Appendix D.2 shows that the assumption is necessary for reward-based online learning algorithms to achieve no regret for any online decision problem.
Revised: Appendix D.2 shows that the assumption is necessary for reward-based online learning algorithms to achieve no regret for any online decision problem.

Original: Assumption 3.5 (Full exploitation of f ) .
Revised: Assumption 3.5 (Full exploitation of f ) .

Original: lim R 1 − max { R 2 ,R 3 ,R 4 }→ + ∞ f 1 ( R 1 , R 2 , R 3 , R 4 ) = 1 .
Revised: lim R 1 − max { R 2 ,R 3 ,R 4 }→ + ∞ f 1 ( R 1 , R 2 , R 3 , R 4 ) = 1 .

Original: Now we show that the family of reward-based online learning algorithms A satisfying assumptions 3.
Revised: Now we show that the family of reward-based online learning algorithms A satisfying assumptions 3.

Original: to 3.5 contains several classic no-regret online learning algorithms [10, 13].
Revised: to 3.5 contains several classic no-regret online learning algorithms [10, 13].

Original: Theorem 3.6.
Revised: Theorem 3.6.

Original: A contains Follow the Perturbed Leader (FPL) algorithm, and Multiplicative Weights algorithm as special cases.
Revised: A contains Follow the Perturbed Leader (FPL) algorithm, and Multiplicative Weights algorithm as special cases.

Original: Corresponding f
Revised: Corresponding f

Original: ’s for them are listed as below:
Revised: ’s for them are listed as below:

Original: Multiplicative Weight
Revised: Multiplicative Weight

Original: [Equation]
Revised: [Equation]

Original: FPL algorithm
Revised: FPL algorithm

Original: Given a noise distribution N
Revised: Given a noise distribution N

Original: [Equation]
Revised: [Equation]

Original: In the binary peer prediction problem using CA mechanism, we can further show that replicator dynamics and linear updating multiplicative weight algorithm [2] are both in A in appendix E.
Revised: In the binary peer prediction problem using CA mechanism, we can further show that replicator dynamics and linear updating multiplicative weight algorithm [2] are both in A in appendix E.

Original: Remark 3.7.
Revised: Remark 3.7.

Original: Our reward based online learning is very similar to the mean-based algorithm in [3] and both use the accumulated rewards to characterize the algorithm’s choice.
Revised: Our reward based online learning is very similar to the mean-based algorithm in [3] and both use the accumulated rewards to characterize the algorithm’s choice.

Original: Additionally, like the meanbased algorithms, our reward based online algorithm may contain algorithms with regret in genral game, e.g., follow the leader.
Revised: Additionally, like the meanbased algorithms, our reward based online algorithm may contain algorithms with regret in genral game, e.g., follow the leader.

Original: The family of reward based online algorithms uses an identical update function across all rounds.
Revised: The family of reward based online algorithms uses an identical update function across all rounds.

Original: Thus some no-regret algorithms, e.g., ϵ -greedy with time decreasing ϵ , doesn’t belong to family.
Revised: Thus some no-regret algorithms, e.g., ϵ -greedy with time decreasing ϵ , doesn’t belong to family.

Original: Finally, if a mean-based algorithm  uses the same update function in each round, then the mean-based algorithm belongs to reward based algorithms. Additionally, then the update function needs to satisfy assumption 3.5.
Revised: Finally, if a mean-based algorithm is also a reward based algorithm that uses the same update function in each round, then its update function satisfies assumption 3.5.

Original:  Truthful Convergence of CA Mechanism on Reward-based Algorithms
Revised:  Truthful Convergence of CA Mechanism on Reward-based Algorithms

Original: Now we present our main result.
Revised: Now we present our main result.

Original: We will show that the sequential CA mechanism can achieve truthful convergence if both agents use reward-based online learning algorithms from A .
Revised: We will show that the sequential CA mechanism can achieve truthful convergence if both agents use reward-based online learning algorithms from A .

Original: This convergence result suggests that the classical CA is robust even when agents deviate from Bayesian rational behavior and use a general family of online learning algorithms.
Revised: This convergence result suggests that the classical CA is robust even when agents deviate from Bayesian rational behavior and use a general family of online learning algorithms.

Original: Theorem 4.1.
Revised: Theorem 4.1.

Original: Under assumptions 2.1 and 2.2, the binary-signal, sequential CA mechanism as defined in eq.
Revised: Under assumptions 2.1 and 2.2, the binary-signal, sequential CA mechanism as defined in eq.

Original: (1) achieves truthful convergence when agents use reward-based algorithms
Revised: (1) achieves truthful convergence when agents use reward-based algorithms

Original: A f
Revised: A f

Original: and
Revised: and

Original: A g , where the update functions f
Revised: A g , where the update functions f

Original: and g satisfy assumptions 3.3 to 3.5.
Revised: and g satisfy assumptions 3.3 to 3.5.

Original: Note that given agents’ online learning algorithm and the payment function, the sequence of accumulated reward vector ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t , S 1 ,t , S 2 ,t , S 3 ,t , S 4 ,t ) t ≥ 1 forms a stochastic process and we define H t as the game history of the rewards and private signals in the first t round.
Revised: Note that given agents’ online learning algorithm and the payment function, the sequence of accumulated reward vector ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t , S 1 ,t , S 2 ,t , S 3 ,t , S 4 ,t ) t ≥ 1 forms a stochastic process and we define H t as the game history of the rewards and private signals in the first t round.

Original: Additionally, if the accumulated reward of truth-telling R 1 ,t , S 1 ,t is much larger than the others’, we can show
Revised: Additionally, if the accumulated reward of truth-telling R 1 ,t , S 1 ,t is much larger than the others’, we can show

Original: Alice and Bob converge to the truth-telling by assumption 3.4.
Revised: Alice and Bob converge to the truth-telling by assumption 3.5.

Original: Thus, it is sufficient for us to track the evolution of accumulated reward vector.
Revised: Thus, it is sufficient for us to track the evolution of accumulated reward vector.

Original: Though the process of accumulated reward vector is not a Markov chain because the payment function eq.
Revised: Though the process of accumulated reward vector is not a Markov chain because the payment function eq.

Original: (1) depends on reports in two rounds, we can still use ideas from semi-martingale to track the process.
Revised: (1) depends on reports in two rounds, we can still use ideas from semi-martingale to track the process.

Original: Before proving our main result, we first present two properties of CA mechanism for our binary signal peer prediction problem.
Revised: Before proving our main result, we first present two properties of CA mechanism for our binary signal peer prediction problem.

Original: Lemma 4.2 shows that the accumulated payoffs of uninformative strategies opt 3 , opt 4 R 3 ,t , R 4 ,t , S 3 ,t , and S 4 ,t are always bounded.
Revised: Lemma 4.2 shows that the accumulated payoffs of uninformative strategies opt 3 , opt 4 R 3 ,t , R 4 ,t , S 3 ,t , and S 4 ,t are always bounded.

Original: Lemma 4.2.
Revised: Lemma 4.2.

Original: Given the game defined in theorem 4.1, for any round t , the accumulated payoffs
Revised: Given the game defined in theorem 4.1, for any round t , the accumulated payoffs

Original: R 3 ,t , R 4 ,t , S 3 ,t , and S 4 ,t for two agents are bounded by [ − 1 , 1] .
Revised: R 3 ,t , R 4 ,t , S 3 ,t , and S 4 ,t for two agents are bounded by [ − 1 , 1] .

Original: The second one, Lemma 4.3, tells us that the summation of accumulated payoffs of opt 1 and opt 2 is always fixed and is equal to the summation of accumulated payoffs of uninformative ones opt 3 and opt 4 for both agents.
Revised: The second one, Lemma 4.3, tells us that the summation of accumulated payoffs of opt 1 and opt 2 is always fixed and is equal to the summation of accumulated payoffs of uninformative ones opt 3 and opt 4 for both agents.

Original: The proofs of these two lemmas are in appendix F.1.
Revised: The proofs of these two lemmas are in appendix F.1.

Original: Lemma 4.3.
Revised: Lemma 4.3.

Original: Given the game defined in theorem 4.1, for any round t , Alice has R 1 ,t + R 2 ,t =
Revised: Given the game defined in theorem 4.1, for any round t , Alice has R 1 ,t + R 2 ,t =

Original: R 3 ,t + R 4 ,t = 0 , and Bob has S 1 ,t + S 2 ,t = S 3 ,t + S 4 ,t = 0 .
Revised: R 3 ,t + R 4 ,t = 0 , and Bob has S 1 ,t + S 2 ,t = S 3 ,t + S 4 ,t = 0 .

Original: We now sketch the proof that consists of four steps.
Revised: We now sketch the proof that consists of four steps.

Original: Informally, using lemmas 4.2 and 4.3, the first step says that the uninformative strategies opt 3 and opt 4 can not completely dominate other strategies.
Revised: Informally, using lemmas 4.2 and 4.3, the first step says that the uninformative strategies opt 3 and opt 4 can not completely dominate other strategies.

Original: Specifically, both agents can not choose an option between opt 3 and opt 4 with a probability larger than 0 .
Revised: Specifically, both agents can not choose an option between opt 3 and opt 4 with a probability larger than 0 .

Original: With the first step, lemma 4.2 and lemma 4.3, we only need to focus on agents’ reward of the truthtelling opt 1 shown in fig.
Revised: With the first step, lemma 4.2 and lemma 4.3, we only need to focus on agents’ reward of the truthtelling opt 1 shown in fig.

Original: We partition the space into three types of events.
Revised: We partition the space into three types of events.

Original: Good events happen when R 1 , S 1 are both very large or very small, bad events happen when one of R 1 , S 1 is very large and one of them is very small, and intermediate states are the states between them.
Revised: Good events happen when R 1 , S 1 are both very large or very small, bad events happen when one of R 1 , S 1 is very large and one of them is very small, and intermediate states are the states between them.

Original: The second step removes the possibility that Alice and Bob continue using different reports opt 1 and opt 2 .
Revised: The second step removes the possibility that Alice and Bob continue using different reports opt 1 and opt 2 .

Original: Therefore, we can always escape "bad events" and enter "intermediate states".
Revised: Therefore, we can always escape "bad events" and enter "intermediate states".

Original: The third step further shows that if the game is in an intermediate state, there exists a constant probability that the game will get into a good events that leads to truthful convergence in a constant number of rounds.
Revised: The third step further shows that if the game is in an intermediate state, there exists a constant probability that the game will get into a good events that leads to truthful convergence in a constant number of rounds.

Original: Hence, after the game enters "good events", which leaves us the final step: showing their strategies converges to either both truth telling opt 1 or flipping opt 2 truthful convergence.
Revised: Hence, after the game enters "good events", which leaves us the final step: showing their strategies converges to either both truth telling opt 1 or flipping opt 2 truthful convergence.

Original: Now we discuss each steps in more details but defer all the proofs to appendix F.Step 1:
Revised: Now we discuss each steps in more details but defer all the proofs to appendix F.

Original: Choosing opt 1 , opt 2 with Nonzero Probability

Original: Combining Lemmas 4.2 and 4.3, we have the following Lemma 4.4, which completes step 1.
Revised: Combining Lemmas 4.2 and 4.3, we have the following Lemma 4.4, which completes step 1.

Original: Lemma 4.4.
Revised: Lemma 4.4.

Original: Given the game defined in theorem 4.1, for any round t ≥ 0 and i ∈ { 1 , 2 } , if (cid:80) tj =1 r i,j ≥ 0 , the probability for Alice to choose opt i is larger than 14 ; if (cid:80) tj =
Revised: Given the game defined in theorem 4.1, for any round t ≥ 0 and i ∈ { 1 , 2 } , if (cid:80) tj =1 r i,j ≥ 0 , the probability for Alice to choose opt i is larger than 14 ; if (cid:80) tj =

Original: 1 s i,t ≥ 0 , the probability for Bob to choose opt i is larger than 14 .
Revised: 1 s i,t ≥ 0 , the probability for Bob to choose opt i is larger than 14 .

Original: Step 2: Escaping Bad Events
Revised: Step 2: Escaping Bad Events

Original: Before introduce the formal statement of step 2, we define two "bad events".
Revised: Before introduce the formal statement of step 2, we define two "bad events".

Original: Given c 0 for all t ≥
Revised: Given c 0 for all t ≥

Original: E 1 , 2 t :=
Revised: E 1 , 2 t :=

Original: { R 1 ,t > c 0 , and S 2 ,t > c 0 } and E 2 , 1 t := { R 2 ,t > c 0 and S 1 ,t > c 0 } .
Revised: { R 1 ,t > c 0 , and S 2 ,t > c 0 } and E 2 , 1 t := { R 2 ,t > c 0 and S 1 ,t > c 0 } .

Original: We will specify clater.
Revised: We will specify clater.

Original: Intuitively, when c 0 is sufficiently large, E 1 , 2 t implies that Alice and Bob will choose opt 1 , optrespectively with a probability close to 1 in following rounds.
Revised: Intuitively, when c 0 is sufficiently large, E 1 , 2 t implies that Alice and Bob will choose opt 1 , optrespectively with a probability close to 1 in following rounds.

Original: For simplicity, we treat each event E as an indicator function, i.e., E happens if and only if E = 1 .
Revised: For simplicity, we treat each event E as an indicator function, i.e., E happens if and only if E = 1 .

Original: In order to prove that these two bad events cannot go on forever, we want to show that when E 1 , 2 t happens, R 1 ,t − R 2 ,t will tend to decrease at a rapid rate.
Revised: In order to prove that these two bad events cannot go on forever, we want to show that when E 1 , 2 t happens, R 1 ,t − R 2 ,t will tend to decrease at a rapid rate.

Original: Therefore, Alice will eventually deviate from opt 1 to choose opt 2 with a relatively high probability.
Revised: Therefore, Alice will eventually deviate from opt 1 to choose opt 2 with a relatively high probability.

Original: By assumption 3.5 and lemma 4.2, given δ > 0 there exists a constant c 1 such that when R 1 ,t > c 1 ,
Revised: By assumption 3.5 and lemma 4.2, given δ > 0 there exists a constant c 1 such that when R 1 ,t > c 1 ,

Original: Alice chooses opt 1 with probability larger than 1 − δ ; when S 2 ,t > c 1 , Bob chooses opt 2 with probability larger than 1 − δ .
Revised: Alice chooses opt 1 with probability larger than 1 − δ ; when S 2 ,t > c 1 , Bob chooses opt 2 with probability larger than 1 − δ .

Original: Let γ 1 =
Revised: Let γ 1 =

Original: P X,Y (1 , 1) + P X,Y (0 , 0)
Revised: P X,Y (1 , 1) + P X,Y (0 , 0)

Original: − P X,Y (1 , 0)
Revised: − P X,Y (1 , 0)

Original: − P X,Y (0 , 1) γ 2 =
Revised: − P X,Y (0 , 1) γ 2 =

Original: ( P X,Y (1 , 1) − P X,Y (0 , 0)) 2 − ( P X,Y (1 , 0) − P X,Y (0 , 1)) 2 .
Revised: ( P X,Y (1 , 1) − P X,Y (0 , 0)) 2 − ( P X,Y (1 , 0) − P X,Y (0 , 1)) 2 .

Original: Lemma 4.5.
Revised: Lemma 4.5.

Original: Given the game defined in theorem 4.1, there exists a δ > 0 and corresponding c 1 such that for any round t , E
Revised: Given the game defined in theorem 4.1, there exists a δ > 0 and corresponding c 1 such that for any round t , E

Original: [ r 1 ,t +1 − r 2 ,t +1 | S 1 ,t − 1 > c 1 + 1] ≥ γ 1 − γ 2> 0 .
Revised: [ r 1 ,t +1 − r 2 ,t +1 | S 1 ,t − 1 > c 1 + 1] ≥ γ 1 − γ 2> 0 .

Original: Given such δ and c 1 in lemma 4.5, we set c 

Revised: Given such δ and c 1 in lemma 4.5, we set c

Original: = c 1 + ⌈ 1000 γ 1 − γ 2 ⌉ + 1 .
Revised: = c 1 + ⌈ 1000 γ 1 − γ 2 ⌉ + 1 .

Original: Because in each round R 1 ,t , R 2 ,t and S 1 ,t , S 2 ,t vary by at most 1 , if E 1 , 2 t happens Alice chooses opt 1 and Bob chooses opt 2 with probability larger than 1 − δ independently for the next ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds.
Revised: Because in each round R 1 ,t , R 2 ,t and S 1 ,t , S 2 ,t vary by at most 1 , if E 1 , 2 t happens Alice chooses opt 1 and Bob chooses opt 2 with probability larger than 1 − δ independently for the next ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds.

Original: Similar argument holds for E 2 , 1 t .
Revised: Similar argument holds for E 2 , 1 t .

Original: We use the above observation to show lemma 4.6.
Revised: We use the above observation to show lemma 4.6.

Original: Lemma 4.6.
Revised: Lemma 4.6.

Original: Given the game defined in theorem 4.1, for all t and history H t ∈ E 1 , 2 t , we have
Revised: Given the game defined in theorem 4.1, for all t and history H t ∈ E 1 , 2 t , we have

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: This lemma formalizes the blue arrows in Fig.
Revised: This lemma formalizes the blue arrows in Fig.

Original: With this lemma, we get the main result of step two.
Revised: With this lemma, we get the main result of step two.

Original: Lemma 4.7.
Revised: Lemma 4.7.

Original: Given the game defined in theorem 4.1, Pr (cid:110) lim
Revised: Given the game defined in theorem 4.1, Pr (cid:110) lim

Original: sup t →∞ E 1 , 2 t
Revised: sup t →∞ E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: If we treat R 1 ,t − R 2 ,t as money of Alice, Lemma 4.7 is similar to the gambler’s ruin problem [7].
Revised: If we treat R 1 ,t − R 2 ,t as money of Alice, Lemma 4.7 is similar to the gambler’s ruin problem [7].

Original: More specifically, R 1 ,t − R 2 ,t has a negative expected growth each ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds by
Revised: More specifically, R 1 ,t − R 2 ,t has a negative expected growth each ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds by

Original: Lemma 4.6, so R 1 ,t − R 2 ,t will always become small enough to escape E 1 , 2 .
Revised: Lemma 4.6, so R 1 ,t − R 2 ,t will always become small enough to escape E 1 , 2 .

Original: Step 3: From Intermediate States to Good Events
Revised: Step 3: From Intermediate States to Good Events

Original: We define a series of "good events" at first.
Revised: We define a series of "good events" at first.

Original: For all u ∈ N + and t ≥ 1 , E 1 , 1 t ( u ) := { R 1 ,t ≥ u, and S 1 ,t ≥ u } and E 2 , 2 t ( u ) := { R 2 ,t ≥ u and S 2 ,t ≥ u } .
Revised: For all u ∈ N + and t ≥ 1 , E 1 , 1 t ( u ) := { R 1 ,t ≥ u, and S 1 ,t ≥ u } and E 2 , 2 t ( u ) := { R 2 ,t ≥ u and S 2 ,t ≥ u } .

Original: To the end, we want to show that ∨ t ∈ N + E 1 , 1 t ( u ) and ∨ t ∈ N + E 2 , 2 t ( u ) happen with probability 1 for any u ∈ N + .
Revised: To the end, we want to show that ∨ t ∈ N + E 1 , 1 t ( u ) and ∨ t ∈ N + E 2 , 2 t ( u ) happen with probability 1 for any u ∈ N + .

Original: Formally, we claim Lemma 4.8.
Revised: Formally, we claim Lemma 4.8.

Original: Lemma 4.8.
Revised: Lemma 4.8.

Original: Given the game defined in theorem 4.1, for all u there exists λ u so that for any
Revised: Given the game defined in theorem 4.1, for all u there exists λ u so that for any

Original: T with history
Revised: T with history

Original: H T ∈
Revised: H T ∈

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: T ∨ E
Revised: T ∨ E

Original: [Equation]
Revised: [Equation]

Original: , 1 T , we have
Revised: , 1 T , we have

Original: [Equation]
Revised: [Equation]

Original: This lemma generally says that when the agents are in an intermediate state that E 1 , 2 T ∨ E 2 , 1 T = 1 , they have a constant probability λ u to get into a good event in the next 4( u + c 0 ) + 100 rounds.
Revised: This lemma generally says that when the agents are in an intermediate state that E 1 , 2 T ∨ E 2 , 1 T = 1 , they have a constant probability λ u to get into a good event in the next 4( u + c 0 ) + 100 rounds.

Original: In order to prove this lemma, we define a nice event P T such that P T happens with probability larger than λ u and P T implies the good events happen in no more than 4( u + c 0 ) + 100 rounds.
Revised: In order to prove this lemma, we define a nice event P T such that P T happens with probability larger than λ u and P T implies the good events happen in no more than 4( u + c 0 ) + 100 rounds.

Original: Formally, event P T is defined as the following: First, for j = T, . .
Revised: Formally, event P T is defined as the following: First, for j = T, . .

Original: , T 1 until some round T 1 ≥ T such that S 1 ,T 1 ≥ 0 , x j +1 = 1 − ˆ y j , y j +

Revised: , T 1 until some round T 1 ≥ T such that S 1 ,T 1 ≥ 0 , x j +1 = 1 − ˆ y j , y j +

Original: = 1 − ˆ x j , Alice uses strategy opt 1 , Bob uses strategy opt 2 .
Revised: = 1 − ˆ x j , Alice uses strategy opt 1 , Bob uses strategy opt 2 .

Original: Then, Alice and
Revised: Then, Alice and

Original: Bob uses strategy opt 1 for 4 u + 50 rounds and signals are generated as x j +

Revised: Bob uses strategy opt 1 for 4 u + 50 rounds and signals are generated as x j +

Original: = y j +1 =
Revised: = y j +1 =

Original: − x j for j ≥ T 1 .
Revised: − x j for j ≥ T 1 .

Original: We can use Lemma 4.4 to prove that λ u
Revised: We can use Lemma 4.4 to prove that λ u

Original: = (min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .
Revised: = (min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .

Original: 25 2 ) 100+

Revised: 25 2 ) 100+

Original: ( u + c 0 ) is a feasible lower bound.
Revised: ( u + c 0 ) is a feasible lower bound.

Original: Step 4: Good Events Lead to Truthful Convergence
Revised: Step 4: Good Events Lead to Truthful Convergence

Original: Similar to step 2, we have lemma 4.9 to show that E 1 , 1 t ( u ) can lead to increasing of R 1 ,t − R 2 ,t at first.
Revised: Similar to step 2, we have lemma 4.9 to show that E 1 , 1 t ( u ) can lead to increasing of R 1 ,t − R 2 ,t at first.

Original: The idea is completely similar to Lemma 4.6 and it formalize the red arrows in Fig.
Revised: The idea is completely similar to Lemma 4.6 and it formalize the red arrows in Fig.

Original: Lemma 4.9.
Revised: Lemma 4.9.

Original: Given the game defined in theorem 4.1, for all u > 2 c 0 + 1 and t if H t ∈ E 1 , 1 t
Revised: Given the game defined in theorem 4.1, for all u > 2 c 0 + 1 and t if H t ∈ E 1 , 1 t

Original: [Equation]
Revised: [Equation]

Original: ⌋ (cid:1) , we have E (cid:20)(cid:80)
Revised: ⌋ (cid:1) , we have E (cid:20)(cid:80)

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: Using Lemma 4.9, we are able to prove that for any ε , there exists a u such that when E 1 , 1 t ( u ) or
Revised: Using Lemma 4.9, we are able to prove that for any ε , there exists a u such that when E 1 , 1 t ( u ) or

Original: E 2 , 2 t ( u ) happens, Alice and Bob will tend to choose ( opt 1 , opt 1 ) or ( opt 2 , opt 2 ) with increasingly higher probability.
Revised: E 2 , 2 t ( u ) happens, Alice and Bob will tend to choose ( opt 1 , opt 1 ) or ( opt 2 , opt 2 ) with increasingly higher probability.

Original: Formally, we propose Lemma 4.10.
Revised: Formally, we propose Lemma 4.10.

Original: Lemma 4.10.
Revised: Lemma 4.10.

Original: Given the game defined in theorem 4.1, for all ϵ > 0 there exists u ∈
Revised: Given the game defined in theorem 4.1, for all ϵ > 0 there exists u ∈

Original: N + such that given a history H T ∈
Revised: N + such that given a history H T ∈

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: ( u ) ∨ E
Revised: ( u ) ∨ E

Original: [Equation]
Revised: [Equation]

Original: 2 T ( u ) , we have
Revised: 2 T ( u ) , we have

Original: Pr
Revised: Pr

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: We design a sub-martingale { D i } i ∈ N that is proportional to R 1 ,T + i (cid:16) ⌈
Revised: We design a sub-martingale { D i } i ∈ N that is proportional to R 1 ,T + i (cid:16) ⌈

Original: [Figure]
Revised: [Figure]

Original: 2 ⌉ +1 (cid:17) −
Revised: 2 ⌉ +1 (cid:17) −

Original: R 2 ,T + i (cid:16) ⌈
Revised: R 2 ,T + i (cid:16) ⌈

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: 2 ⌉ +1 (cid:17) , and use Azuma-Hoeffding inequality to prove lemma 4.10.
Revised: 2 ⌉ +1 (cid:17) , and use Azuma-Hoeffding inequality to prove lemma 4.10.

Original:  Simulations
Revised:  Simulations

Original: We simulate the CA mechanism with various learning algorithms: the Hedge algorithms, follow the perturbed leader, follow the leader, and ϵ -greedy, and repeat the process 400 times with 800 rounds on each algorithm each time.
Revised: We simulate the CA mechanism with various learning algorithms: the Hedge algorithms, follow the perturbed leader, follow the leader, and ϵ -greedy, and repeat the process 400 times with 800 rounds on each algorithm each time.

Original: We define the converge proportion in round t as the fraction of the simulations where both agents report truthfully opt 1 (or both use opt 2 ) in all the subsequent rounds.
Revised: We define the converge proportion in round t as the fraction of the simulations where both agents report truthfully opt 1 (or both use opt 2 ) in all the subsequent rounds.

Original: In our simulations, we use the following private signal distribution that satisfies assumption 2.2:
Revised: In our simulations, we use the following private signal distribution that satisfies assumption 2.2:

Original: P X,Y (0 , 0) =
Revised: P X,Y (0 , 0) =

Original: P X,Y (1 , 1) = 0 .
Revised: P X,Y (1 , 1) = 0 .

Original: 4 , P X,Y (1 , 0) =
Revised: 4 , P X,Y (1 , 0) =

Original: P X,Y (0 , 1) = 0 .
Revised: P X,Y (0 , 1) = 0 .

Original: Moreover, Alice and Bob are using the same learning algorithms in our simulations that are listed below: First, Follow the
Revised: Moreover, Alice and Bob are using the same learning algorithms in our simulations that are listed below: First, Follow the

Original: Leader algorithm (FTL) chooses opt i with probability proportional to I
Revised: Leader algorithm (FTL) chooses opt i with probability proportional to I

Original: [ R i = max { R 1 , R 2 , R 3 , R 4 } ] .
Revised: [ R i = max { R 1 , R 2 , R 3 , R 4 } ] .

Original: Follow the perturbed leader (FPL*, where * can be 1, 4 or 8) adds a uniform random noise betweenand ∗ and choose strategy opt i with probability
Revised: Follow the perturbed leader (FPL*, where * can be 1, 4 or 8) adds a uniform random noise betweenand ∗ and choose strategy opt i with probability

Original: Pr (cid:110) R i + p i = max j ∈ [4] { R j + p j } (cid:12)(cid:12)(cid:12) p
Revised: Pr (cid:110) R i + p i = max j ∈ [4] { R j + p j } (cid:12)(cid:12)(cid:12) p

Original: j iid ∼ U [0 , ∗ ] (cid:111) .
Revised: j iid ∼ U [0 , ∗ ] (cid:111) .

Original: We consider FPL1, FPL4, and FPL8.
Revised: We consider FPL1, FPL4, and FPL8.

Original: Hedge algorithm 1 choose i with probability proportional to 3 R i / 2 that is an implementation by choosing ϵ = 0 .
Revised: Hedge algorithm 1 choose i with probability proportional to 3 R i / 2 that is an implementation by choosing ϵ = 0 .

Original: 5 for the multiplicative weights algorithm of Arora et al.
Revised: 5 for the multiplicative weights algorithm of Arora et al.

Original: Hedge algorithm 2 chooses i with probability proportional to e R i that is an implementation by choosing β = 1 of exponentially weighted averaged forecaster introduced by Freund and Schapire [10].
Revised: Hedge algorithm 2 chooses i with probability proportional to e R i that is an implementation by choosing β = 1 of exponentially weighted averaged forecaster introduced by Freund and Schapire [10].

Original: Finally, ϵ -greedy algorithm uses time varying ϵ = 1 ( t +1) 2 at round t .
Revised: Finally, ϵ -greedy algorithm uses time varying ϵ = 1 ( t +1) 2 at round t .

Original: Note that ϵ -greedy is not in A but still achieves truthful convergence.
Revised: Note that ϵ -greedy is not in A but still achieves truthful convergence.

Original: In Figure 2, all our algorithms converge to truth-telling.
Revised: In Figure 2, all our algorithms converge to truth-telling.

Original: First, all reward-based online learning algorithms (the Hedge algorithms, follow the perturbed leader, and follow the leader) exhibit truthful convergence that aligns with our theoretical result, theorem 4.1.
Revised: First, all reward-based online learning algorithms (the Hedge algorithms, follow the perturbed leader, and follow the leader) exhibit truthful convergence that aligns with our theoretical result, theorem 4.1.

Original: Moreover, although FTL is generally not no-regret, CA mechanism still works well with it.
Revised: Moreover, although FTL is generally not no-regret, CA mechanism still works well with it.

Original: Additionally, we observe that when an algorithm explores less (e.g. FPL4 vs FPL8), it converges faster, but very little exploration does not further improve the convergence rate (e.g. FTL vs FPL2).
Revised: Additionally, we observe that when an algorithm explores less (e.g. FPL4 vs FPL8), it converges faster, but very little exploration does not further improve the convergence rate (e.g. FTL vs FPL2).

Original: Finally, we also find that the ϵ -greedy with time decreasing ϵ , which is not a reward-based online learning algorithm, also shows truthful convergence.
Revised: Finally, we also find that the ϵ -greedy with time decreasing ϵ , which is not a reward-based online learning algorithm, also shows truthful convergence.

Original: This suggests that the CA mechanism may have truthful convergence beyond reward-based online learning algorithms.
Revised: This suggests that the CA mechanism may have truthful convergence beyond reward-based online learning algorithms.

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we study sequential peer prediction with learning agents and prove that the notion of no-regret alone is not sufficient for truthful convergence.
Revised: In this paper, we study sequential peer prediction with learning agents and prove that the notion of no-regret alone is not sufficient for truthful convergence.

Original: We then define a family of reward-based learning algorithms and show that the CA mechanism is able to achieve truthful convergence when agents use algorithms in this family.
Revised: We then define a family of reward-based learning algorithms and show that the CA mechanism is able to achieve truthful convergence when agents use algorithms in this family.

Original: Finally, we give a discussion on the converge rates of different learning agents based on simulations.
Revised: Finally, we give a discussion on the converge rates of different learning agents based on simulations.

Original: This is the first theoretical study on peer prediction with learning agents.
Revised: This is the first theoretical study on peer prediction with learning agents.

Original: There are many open problems and future directions to extend this work.
Revised: There are many open problems and future directions to extend this work.

Original: We believe similar proof techniques can be used to extend our results to settings where agents’ private signals are generated by a Markov chain with some assumptions on the transition matrix.
Revised: We believe similar proof techniques can be used to extend our results to settings where agents’ private signals are generated by a Markov chain with some assumptions on the transition matrix.

Original: Moreover, this work is only restricted to binary signals and it is still an open problem whether there exists a mechanism for non-binary settings that can promise truthful convergence.
Revised: Moreover, this work is only restricted to binary signals and it is still an open problem whether there exists a mechanism for non-binary settings that can promise truthful convergence.

Original: For the learning agents, one could consider a more general family of learning algorithms such as when f is time-varying.
Revised: For the learning agents, one could consider a more general family of learning algorithms such as when f is time-varying.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: The authors would like to thank the anonymous reviewers for their valuable comments and constructive feedback.
Revised: The authors would like to thank the anonymous reviewers for their valuable comments and constructive feedback.

Original: This work is partially supported by the National Science Foundation under Grant No.
Revised: This work is partially supported by the National Science Foundation under Grant No.

Original: IIS 2007887 and by the National Science Foundation and Amazon under Grant No. FAI 2147187.
Revised: IIS 2007887 and by the National Science Foundation and Amazon under Grant No. FAI 2147187.

Original: Appendix A Basic Math A.1 Martingale and Concentration
Revised: Appendix A Basic Math A.1 Martingale and Concentration

Original: In this section we will define martingales and some of its properties.
Revised: In this section we will define martingales and some of its properties.

Original: Definition A.1 (Martingale) .
Revised: Definition A.1 (Martingale) .

Original: Let F = {F t } t ∈ N be a filtration, which is an increasing sequence of σ -field.
Revised: Let F = {F t } t ∈ N be a filtration, which is an increasing sequence of σ -field.

Original: A martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F ( D t ∈ F t for all t ) that satisfies for any time t ,
Revised: A martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F ( D t ∈ F t for all t ) that satisfies for any time t ,

Original: [Equation]
Revised: [Equation]

Original: There are two extensions of a martingale that replace the equality of conditional probability by upper and lower bounds.
Revised: There are two extensions of a martingale that replace the equality of conditional probability by upper and lower bounds.

Original: Definition A.2 (Sub-martingale and super-martingale) .
Revised: Definition A.2 (Sub-martingale and super-martingale) .

Original: Let F = {F t } t ∈ N be a filtration, which is an increasing sequence of σ -field.
Revised: Let F = {F t } t ∈ N be a filtration, which is an increasing sequence of σ -field.

Original: A sub-martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F that satisfies for any time t ,
Revised: A sub-martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F that satisfies for any time t ,

Original: [Equation]
Revised: [Equation]

Original: A super-martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F that satisfies for any time t ,
Revised: A super-martingale with respect to F is a sequence D 0 , D 1 , D 2 , · · · adapted to F that satisfies for any time t ,

Original: [Equation]
Revised: [Equation]

Original: For a sub-martingale (or super-martingale), we have the Azuma–Hoeffding inequality [1], which is a concentration result for the values of martingales.
Revised: For a sub-martingale (or super-martingale), we have the Azuma–Hoeffding inequality [1], which is a concentration result for the values of martingales.

Original: Theorem A.3 (Azuma–Hoeffding inequality) .
Revised: Theorem A.3 (Azuma–Hoeffding inequality) .

Original: Suppose { D t } t ∈ N is a martingale (or supermartingale) and | D t − D t − 1 |
Revised: Suppose { D t } t ∈ N is a martingale (or supermartingale) and | D t − D t − 1 |

Original: ≤ c t almost surely.
Revised: ≤ c t almost surely.

Original: Then for any T ∈ N + and ϵ ∈ R + , we have
Revised: Then for any T ∈ N + and ϵ ∈ R + , we have

Original: [Equation]
Revised: [Equation]

Original: And symmetrically, if { D t } t ∈ N is a sub-martingale, we have
Revised: And symmetrically, if { D t } t ∈ N is a sub-martingale, we have

Original: [Equation]
Revised: [Equation]

Original: Now we define stopping time.
Revised: Now we define stopping time.

Original: This is intuitively a condition such that the "decision" whether to stop in the t th round should be based on information of the first t rounds, instead of any future information.
Revised: This is intuitively a condition such that the "decision" whether to stop in the t th round should be based on information of the first t rounds, instead of any future information.

Original: Definition A.4 (Stopping time) .
Revised: Definition A.4 (Stopping time) .

Original: τ is called a stopping time for a filtration F if and only if { τ = t } ∈
Revised: τ is called a stopping time for a filtration F if and only if { τ = t } ∈

Original: F t , ∀ t .
Revised: F t , ∀ t .

Original: A.2 Limit Inferior and Limit Superior
Revised: A.2 Limit Inferior and Limit Superior

Original: Limit inferior and limit superior are defined on sequences, representing limit bounds of a sequence.
Revised: Limit inferior and limit superior are defined on sequences, representing limit bounds of a sequence.

Original: To meet our needs, our definition focus on discrete metric.
Revised: To meet our needs, our definition focus on discrete metric.

Original: Definition A.5.
Revised: Definition A.5.

Original: Let {E t } t ∈ N + be a sequence of events.
Revised: Let {E t } t ∈ N + be a sequence of events.

Original: Limit inferior and limit superior of this sequence are
Revised: Limit inferior and limit superior of this sequence are

Original: [Equation]
Revised: [Equation]

Original: and
Revised: and

Original: [Equation]
Revised: [Equation]

Original: A.
Revised: A.

Original: -Cantelli Lemma
Revised: -Cantelli Lemma

Original: In this section, we formally introduce Borel-Cantelli lemma.
Revised: In this section, we formally introduce Borel-Cantelli lemma.

Original: Its proof can be found in [9].
Revised: Its proof can be found in [9].

Original: Theorem A.6 (Borel-Cantelli lemma) .
Revised: Theorem A.6 (Borel-Cantelli lemma) .

Original: Let E 1 , E 2 , · · · be a sequence of events in some probability space.
Revised: Let E 1 , E 2 , · · · be a sequence of events in some probability space.

Original: Borel-Cantelli lemma states that if (cid:80) + ∞ t =1 Pr {E t } < ∞ , then the probability that infinitely many of E t ’s occur is 0 , or more strictly,
Revised: Borel-Cantelli lemma states that if (cid:80) + ∞ t =1 Pr {E t } < ∞ , then the probability that infinitely many of E t ’s occur is 0 , or more strictly,

Original: [Equation]
Revised: [Equation]

Original: B Truthfulness of CA mechanism for Bayesian Agents
Revised: B Truthfulness of CA mechanism for Bayesian Agents

Original: In this section, we assume that both agents are using consistent strategies as previous works assume [5, 25, 17, 23, 15, 28].
Revised: In this section, we assume that both agents are using consistent strategies as previous works assume [5, 25, 17, 23, 15, 28].

Original: The formal definition of consistent strategy is given in definition B.1.
Revised: The formal definition of consistent strategy is given in definition B.1.

Original: Definition B.1 (Consistent strategy) .
Revised: Definition B.1 (Consistent strategy) .

Original: In a repeated game, a strategy profile σ is a consistent strategy if and only if for each agent i , she adopts σ i identically over each round of the game.
Revised: In a repeated game, a strategy profile σ is a consistent strategy if and only if for each agent i , she adopts σ i identically over each round of the game.

Original: In a sequential peer prediction game, for agent i , we set the average payoffs of agent i in T rounds as her utility.
Revised: In a sequential peer prediction game, for agent i , we set the average payoffs of agent i in T rounds as her utility.

Original: Then we introduce Bayesian Nash equilibrium.
Revised: Then we introduce Bayesian Nash equilibrium.

Original: A Bayesian Nash equilibrium (BNE) is strategies of agents that have the maximal expected payoff for each player given their beliefs on environments and others’ strategies [12].
Revised: A Bayesian Nash equilibrium (BNE) is strategies of agents that have the maximal expected payoff for each player given their beliefs on environments and others’ strategies [12].

Original: Definition B.2 (Bayesian Nash equilibiurm) .
Revised: Definition B.2 (Bayesian Nash equilibiurm) .

Original: A strategy profile σ is a Bayesian Nash equilibrium if and only if for every agent i , the expected payoff of using σ i for agent i is maximal keeping other agents’ strategies unchanged.
Revised: A strategy profile σ is a Bayesian Nash equilibrium if and only if for every agent i , the expected payoff of using σ i for agent i is maximal keeping other agents’ strategies unchanged.

Original: Moreover, A strategy profile σ is a strict Bayesian Nash equilibrium if and only if for every agent i , the expected payoff of using σ i for agent i is strictly larger than any other strategies keeping other agents’ strategies unchanged.
Revised: Moreover, A strategy profile σ is a strict Bayesian Nash equilibrium if and only if for every agent i , the expected payoff of using σ i for agent i is strictly larger than any other strategies keeping other agents’ strategies unchanged.

Original: Moreover, We say a peer prediction mechanism is strongly truthful if agents in truthtelling equilibrium get strictly higher payment than any other non-permutation equilibrium [25].
Revised: Moreover, We say a peer prediction mechanism is strongly truthful if agents in truthtelling equilibrium get strictly higher payment than any other non-permutation equilibrium [25].

Original: Here, a permutation equilibrium is the strategy profile that agents report a permutation of the signal.
Revised: Here, a permutation equilibrium is the strategy profile that agents report a permutation of the signal.

Original: Formally, we have definition B.3.
Revised: Formally, we have definition B.3.

Original: Definition B.3 (Strongly truthful) .
Revised: Definition B.3 (Strongly truthful) .

Original: In a peer prediction game, if agents are using consistent strategies, a mechanism is strongly truthful if and only if truthtelling is a BNE and also guarantees larger agent welfare than any non-permutation equilibrium.
Revised: In a peer prediction game, if agents are using consistent strategies, a mechanism is strongly truthful if and only if truthtelling is a BNE and also guarantees larger agent welfare than any non-permutation equilibrium.

Original: Here, welfare is defined by each agent’s expected payoff so that is to say, the expected payoff of each agent using truthtelling strategy profile is strictly higher than the expected payoff using non-permutation equilibrium.
Revised: Here, welfare is defined by each agent’s expected payoff so that is to say, the expected payoff of each agent using truthtelling strategy profile is strictly higher than the expected payoff using non-permutation equilibrium.

Original: Though Dasgupta and Ghosh [5], Shnayder et al.
Revised: Though Dasgupta and Ghosh [5], Shnayder et al.

Original: [25] have proved that CA mechanism is strongly truthful in non-sequential settings, our settings are slightly different from theirs and we rewrite the proof for binary sequential peer prediction settings.
Revised: [25] have proved that CA mechanism is strongly truthful in non-sequential settings, our settings are slightly different from theirs and we rewrite the proof for binary sequential peer prediction settings.

Original: More specifically, our CA mechanism uses the last round agreement term instead of average agreement term in the payoffs and we are focusing on average payoffs instead of total payoffs.
Revised: More specifically, our CA mechanism uses the last round agreement term instead of average agreement term in the payoffs and we are focusing on average payoffs instead of total payoffs.

Original: Before the complete proof, we have the following lemma.
Revised: Before the complete proof, we have the following lemma.

Original: Lemma B.4.
Revised: Lemma B.4.

Original: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if both agents are using consistent strategies, CA mechanism renders truthtelling strategy profile a strict Bayesian Nash equilibrium.
Revised: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if both agents are using consistent strategies, CA mechanism renders truthtelling strategy profile a strict Bayesian Nash equilibrium.

Original: Proof.
Revised: Proof.

Original: We know that Alice and Bob are Bayesian agents using consistent strategies.
Revised: We know that Alice and Bob are Bayesian agents using consistent strategies.

Original: Then we can suppose when Alice gets signal 1 , she reports 1 with probability p 1 ; when she gets signal 0 , she reports 0 with probability p 0 .
Revised: Then we can suppose when Alice gets signal 1 , she reports 1 with probability p 1 ; when she gets signal 0 , she reports 0 with probability p 0 .

Original: Similarly, Bob is using a fixed strategy that when he gets signal 1 , he reports 1 with probability q 1 ; when he gets signal 0 , he reports 0 with probability q 0 .
Revised: Similarly, Bob is using a fixed strategy that when he gets signal 1 , he reports 1 with probability q 1 ; when he gets signal 0 , he reports 0 with probability q 0 .

Original: Therefore, we can compute the first term in payoff of Alice and Bob (see eq.
Revised: Therefore, we can compute the first term in payoff of Alice and Bob (see eq.

Original: E
Revised: E

Original: [ I [ˆ x t = ˆ y t ]] , by
Revised: [ I [ˆ x t = ˆ y t ]] , by

Original: E
Revised: E

Original: [ I [ˆ x t = ˆ y t ]] =
Revised: [ I [ˆ x t = ˆ y t ]] =

Original: P X,Y (1 , 1)( p 1 q 1 + (1 − p 1 )(1 − q 1 ))
Revised: P X,Y (1 , 1)( p 1 q 1 + (1 − p 1 )(1 − q 1 ))

Original: + P X,Y (0 , 0)( p 0 q 0 + (1 − p 0 )(1 − q 0 ))
Revised: + P X,Y (0 , 0)( p 0 q 0 + (1 − p 0 )(1 − q 0 ))

Original: + P X,Y (1 , 0)( p 1 (1 − q 0 ) + (1 − p 1 ) q 0 ) + P X,Y (0 , 1)( q 1 (1 − p 0 )
Revised: + P X,Y (1 , 0)( p 1 (1 − q 0 ) + (1 − p 1 ) q 0 ) + P X,Y (0 , 1)( q 1 (1 − p 0 )

Original: + (1 − q 1 ) p 0 ) .
Revised: + (1 − q 1 ) p 0 ) .

Original: Also, we can compute the second term
Revised: Also, we can compute the second term

Original: [Equation]
Revised: [Equation]

Original: Similarly, E [ I [ˆ y t
Revised: Similarly, E [ I [ˆ y t

Original: = ˆ x t − 1 ]] also equals to this expression.
Revised: = ˆ x t − 1 ]] also equals to this expression.

Original: To prove BNE, we only need to prove that when q 0 = q 1 = 1 , we have
Revised: To prove BNE, we only need to prove that when q 0 = q 1 = 1 , we have

Original: [Equation]
Revised: [Equation]

Original: The other side when p 0 = p 1 = 1 is symmetrical.
Revised: The other side when p 0 = p 1 = 1 is symmetrical.

Original: Actually, we have
Revised: Actually, we have

Original: [Equation]
Revised: [Equation]

Original: , 1) − P X,Y (0 , 0) − P X,Y (1 , 0))
Revised: , 1) − P X,Y (0 , 0) − P X,Y (1 , 0))

Original: =
Revised: =

Original: 2 P X,Y (0 , 0) − 2( P X,Y (0 , 0)
Revised: 2 P X,Y (0 , 0) − 2( P X,Y (0 , 0)

Original: + P X,Y (0 , 1))
Revised: + P X,Y (0 , 1))

Original: ( P X,Y (0 , 0) + P X,Y (1 , 0))
Revised: ( P X,Y (0 , 0) + P X,Y (1 , 0))

Original: =
Revised: =

Original: 2( P X,Y (0 , 0) P X,Y (1 , 1) − P X,Y (0 , 1) P X,Y (1 , 0)) > 0 , and ∂ ( E
Revised: 2( P X,Y (0 , 0) P X,Y (1 , 1) − P X,Y (0 , 1) P X,Y (1 , 0)) > 0 , and ∂ ( E

Original: [ I [ˆ x t = ˆ y t ] | q 0 = q 1 = 1] − E
Revised: [ I [ˆ x t = ˆ y t ] | q 0 = q 1 = 1] − E

Original: [ I [ˆ x t = ˆ y t − 1 ] | q 0 =
Revised: [ I [ˆ x t = ˆ y t − 1 ] | q 0 =

Original: q 1 = 1])
Revised: q 1 = 1])

Original: ∂p= P X,Y (1 , 1) − P X,Y (1 , 0)
Revised: ∂p= P X,Y (1 , 1) − P X,Y (1 , 0)

Original: + ( P X,Y (1 , 1) + P X,Y (1 , 0))
Revised: + ( P X,Y (1 , 1) + P X,Y (1 , 0))

Original: ( P X,Y (0 , 0) + P X,Y (1 , 0) − P X,Y (1 , 1) − P X,Y (0 , 1))
Revised: ( P X,Y (0 , 0) + P X,Y (1 , 0) − P X,Y (1 , 1) − P X,Y (0 , 1))

Original: =
Revised: =

Original: 2 P X,Y (1 , 1) − 2( P X,Y (1 , 1) + P X,Y (1 , 0))
Revised: 2 P X,Y (1 , 1) − 2( P X,Y (1 , 1) + P X,Y (1 , 0))

Original: ( P X,Y (1 , 1) + P X,Y (0 , 1))
Revised: ( P X,Y (1 , 1) + P X,Y (0 , 1))

Original: =
Revised: =

Original: 2( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (0 , 1) P X,Y (1 , 0)) > 0 , which indicate eq.
Revised: 2( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (0 , 1) P X,Y (1 , 0)) > 0 , which indicate eq.

Original: Furthermore, we can find all the Nash equilibria for Bayesian agents under our CA mechanism in sequential peer prediction setting.
Revised: Furthermore, we can find all the Nash equilibria for Bayesian agents under our CA mechanism in sequential peer prediction setting.

Original: We have theorem B.5.
Revised: We have theorem B.5.

Original: Theorem B.5.
Revised: Theorem B.5.

Original: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if CA mechanism is used and agents adopt consistent strategies, there are three types of Nash equilibria for agents, which are
Revised: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if CA mechanism is used and agents adopt consistent strategies, there are three types of Nash equilibria for agents, which are

Original: truth-telling opt 1 , 2.
Revised: truth-telling opt 1 , 2.

Original: flip the signal opt 2 , 3. report regardless of private signals (uninformative reports).
Revised: flip the signal opt 2 , 3. report regardless of private signals (uninformative reports).

Original: Proof.
Revised: Proof.

Original: Using the same notations in the proof of lemma B.4, we deduce that ∂ ( E [ I [
Revised: Using the same notations in the proof of lemma B.4, we deduce that ∂ ( E [ I [

Original: [Equation]
Revised: [Equation]

Original: x t = ˆ y t ]] − E
Revised: x t = ˆ y t ]] − E

Original: [ I [ˆ x t = ˆ y t − 1 ]])
Revised: [ I [ˆ x t = ˆ y t − 1 ]])

Original: ∂p= ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1))(2 q 0 + 2 q 1 ) − 2( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1)) , and similarly, ∂ ( E [
Revised: ∂p= ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1))(2 q 0 + 2 q 1 ) − 2( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1)) , and similarly, ∂ ( E [

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: y t
Revised: y t

Original: [Equation]
Revised: [Equation]

Original: ]]) ∂p
Revised: ]]) ∂p

Original: [Equation]
Revised: [Equation]

Original: = ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1))(2 q 0 + 2 q 1 ) − ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0)
Revised: = ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0) P X,Y (0 , 1))(2 q 0 + 2 q 1 ) − ( P X,Y (1 , 1) P X,Y (0 , 0) − P X,Y (1 , 0)

Original: P X,Y (0 , 1)) .
Revised: P X,Y (0 , 1)) .

Original: Therefore, we know that when q 0 + q 1 > 1 , the best response of Alice is p 

Revised: Therefore, we know that when q 0 + q 1 > 1 , the best response of Alice is p

Original: = p 1 = 1 ; when q 0 + q 1 < 1 , the best response of Alice is p 

Revised: = p 1 = 1 ; when q 0 + q 1 < 1 , the best response of Alice is p

Original: = p 1 = 0 .
Revised: = p 1 = 0 .

Original: Symmetrically, we can deduce that when p 0 + p 1 > 1 , the best response of Alice is q 0 = q 1 = 1 ; when p 0 + p 1 < 1 , the best response of
Revised: Symmetrically, we can deduce that when p 0 + p 1 > 1 , the best response of Alice is q 0 = q 1 = 1 ; when p 0 + p 1 < 1 , the best response of

Original: Alice is q 0 =
Revised: Alice is q 0 =

Original: q 1 = 0 .
Revised: q 1 = 0 .

Original: Let p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 be a Nash equilibrium for this peer prediction game under CA mechanism.
Revised: Let p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 be a Nash equilibrium for this peer prediction game under CA mechanism.

Original: Then given p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 should be one of the best responses of Bob; given q ∗ 0 , q ∗ 1 , p ∗ 0 , p ∗ 1 should be one of the best responses of Alice.
Revised: Then given p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 should be one of the best responses of Bob; given q ∗ 0 , q ∗ 1 , p ∗ 0 , p ∗ 1 should be one of the best responses of Alice.

Original: If p ∗ 

Revised: If p ∗

Original: + p ∗ 1 > 1 , then the only best response of Bob is q 

Revised: + p ∗ 1 > 1 , then the only best response of Bob is q

Original: = q 1 = 1 , so q ∗ 

Revised: = q 1 = 1 , so q ∗

Original: =
Revised: =

Original: q ∗ 1 = 1 .
Revised: q ∗ 1 = 1 .

Original: When q ∗ 0 = q ∗ 1 = 1 , the only best response of Alice is p 

Revised: When q ∗ 0 = q ∗ 1 = 1 , the only best response of Alice is p

Original: = p 1 = 1 , so we can deduce that p ∗ 

Revised: = p 1 = 1 , so we can deduce that p ∗

Original: =
Revised: =

Original: p ∗ 1 = q ∗ 0 =
Revised: p ∗ 1 = q ∗ 0 =

Original: q ∗ 1 = 1 is the unique Nash equilibrium for this case.
Revised: q ∗ 1 = 1 is the unique Nash equilibrium for this case.

Original: If p ∗ 

Revised: If p ∗

Original: + p ∗ 1 < 1 , then the only best response of Bob is q 

Revised: + p ∗ 1 < 1 , then the only best response of Bob is q

Original: = q 1 = 0 , so q ∗ 

Revised: = q 1 = 0 , so q ∗

Original: = q ∗ 1 = 0 .
Revised: = q ∗ 1 = 0 .

Original: When q ∗ 0 = q ∗ 1 = 0 , the only best response of Alice is p 

Revised: When q ∗ 0 = q ∗ 1 = 0 , the only best response of Alice is p

Original: = p 1 = 0 , so we can deduce that p ∗ 

Revised: = p 1 = 0 , so we can deduce that p ∗

Original: =
Revised: =

Original: p ∗ 1 = q ∗ 0 =
Revised: p ∗ 1 = q ∗ 0 =

Original: q ∗ 1 = 0 is the unique Nash equilibrium for this case.
Revised: q ∗ 1 = 0 is the unique Nash equilibrium for this case.

Original: Symmetrically, when q ∗ 0 + q ∗ 1 ̸ = 1 , there are still only these two Nash equilibria.
Revised: Symmetrically, when q ∗ 0 + q ∗ 1 ̸ = 1 , there are still only these two Nash equilibria.

Original: Moreover, it is easy to verify that p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 that satisfies p ∗ 

Revised: Moreover, it is easy to verify that p ∗ 0 , p ∗ 1 , q ∗ 0 , q ∗ 1 that satisfies p ∗

Original: + p ∗ 1 = q ∗ 

Revised: + p ∗ 1 = q ∗

Original: + q ∗ 1 = 1 is Nash equilibrium.
Revised: + q ∗ 1 = 1 is Nash equilibrium.

Original: Therefore, all the uninformative reports are Nash equilibria.
Revised: Therefore, all the uninformative reports are Nash equilibria.

Original: These are exactly what we want to prove.
Revised: These are exactly what we want to prove.

Original: From the proof of lemma B.4, we can observe that when both agents use opt 1 , their expected payoffs are both
Revised: From the proof of lemma B.4, we can observe that when both agents use opt 1 , their expected payoffs are both

Original: [Equation]
Revised: [Equation]

Original: Also, we can observe that when both agents use uninformative reports, their expected payoffs are both 0 .
Revised: Also, we can observe that when both agents use uninformative reports, their expected payoffs are both 0 .

Original: According to theorem B.5, we know that the only non-permutation equilibrium is uninformative reports.
Revised: According to theorem B.5, we know that the only non-permutation equilibrium is uninformative reports.

Original: Therefore, we deduce that CA mechanism is strongly truthful.
Revised: Therefore, we deduce that CA mechanism is strongly truthful.

Original: Theorem B.6.
Revised: Theorem B.6.

Original: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if agents adopt consistent strategies, CA mechanism is strongly truthful.
Revised: For binary sequential signal peer prediction games under assumption 2.1 and assumption 2.2, if agents adopt consistent strategies, CA mechanism is strongly truthful.

Original: C Proofs and Details of Section 3. C.1 Impossibility of Truthful Convergence for No Regret Agents
Revised: C Proofs and Details of Section 3. C.1 Impossibility of Truthful Convergence for No Regret Agents

Original: Proof of theorem 3.1.
Revised: Proof of theorem 3.1.

Original: First, for any sequential information elicitation mechanism because Alice and Bob can use arbitrary no regret algorithm, there exist no-regret for Alice and Bob.
Revised: First, for any sequential information elicitation mechanism because Alice and Bob can use arbitrary no regret algorithm, there exist no-regret for Alice and Bob.

Original: If the truth-telling has regret on M , the statement trivially holds.
Revised: If the truth-telling has regret on M , the statement trivially holds.

Original: Otherwise, suppose truth-telling is no regret on M .
Revised: Otherwise, suppose truth-telling is no regret on M .

Original: We have for all T the expectations µ XT = max i (cid:80) t ≤ T E r i,t − (cid:80) t ≤ T E r t and µ Y T = max i (cid:80) t
Revised: We have for all T the expectations µ XT = max i (cid:80) t ≤ T E r i,t − (cid:80) t ≤ T E r t and µ Y T = max i (cid:80) t

Original: ≤ T E s i,t − (cid:80) t ≤ T E s t satisfy µ XT and µ YT =
Revised: ≤ T E s i,t − (cid:80) t ≤ T E s t satisfy µ XT and µ YT =

Original: o ( T ) .
Revised: o ( T ) .

Original: We will use probabilistic method to show the existence of a deterministic and no regret sequence of strategies ( opt Xt , opt Yt ) t =1 ,... that consists of reporting 1 and 0 regardless of private signal opt 3 , opt 4 .
Revised: We will use probabilistic method to show the existence of a deterministic and no regret sequence of strategies ( opt Xt , opt Yt ) t =1 ,... that consists of reporting 1 and 0 regardless of private signal opt 3 , opt 4 .

Original: As a result, when Alice and Bob use the sequence, the algorithm is no regret but M does not achieve truthful convergence on such algorithm.
Revised: As a result, when Alice and Bob use the sequence, the algorithm is no regret but M does not achieve truthful convergence on such algorithm.

Original: To find such sequence, it is sufficient for us to find a deterministic sequence (ˆ x ∗ t , ˆ y ∗ t ) t ≥ 

Revised: To find such sequence, it is sufficient for us to find a deterministic sequence (ˆ x ∗ t , ˆ y ∗ t ) t ≥

Original: so that for all T
Revised: so that for all T

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: because we can define an online learning algorithms so that Alice play opt X t = opt 2+ˆ x ∗ t and Bob play opt Yt = opt 2+ˆ y ∗ t for all t .
Revised: because we can define an online learning algorithms so that Alice play opt X t = opt 2+ˆ x ∗ t and Bob play opt Yt = opt 2+ˆ y ∗ t for all t .

Original: Additionally, if we can find (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t and T 0 that for all T ≥ T 0 , | max i (cid:88) r i,t − (cid:88) r t − µ XT | and | max i (cid:88) s i,t − (cid:88) s t − µ YT | ≤ T
Revised: Additionally, if we can find (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t and T 0 that for all T ≥ T 0 , | max i (cid:88) r i,t − (cid:88) r t − µ XT | and | max i (cid:88) s i,t − (cid:88) s t − µ YT | ≤ T

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: , (5) by eq.
Revised: , (5) by eq.

Original: (3) (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t satisfies eq.
Revised: (3) (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t satisfies eq.

Original: Because their signal mutually independent across different rounds and each round’s signals can only affect at most 2 k − 1 rounds of payoff, each pair of signals only changes max i (cid:80) t ≤ T r i,t − (cid:80) t ≤ T r t by 2(2 k − 1) ≤ 4 k .
Revised: Because their signal mutually independent across different rounds and each round’s signals can only affect at most 2 k − 1 rounds of payoff, each pair of signals only changes max i (cid:80) t ≤ T r i,t − (cid:80) t ≤ T r t by 2(2 k − 1) ≤ 4 k .

Original: Therefore, by Chernoff bound using method of bounded difference, we have for all T and ϵ >
Revised: Therefore, by Chernoff bound using method of bounded difference, we have for all T and ϵ >

Original: Pr
Revised: Pr

Original: [Table]
Revised: [Table]

Original: Thus, we can take ϵ = T − 1 / 3 and T 0 large enough and prove the random payoffs of truth-telling satisfy eq.
Revised: Thus, we can take ϵ = T − 1 / 3 and T 0 large enough and prove the random payoffs of truth-telling satisfy eq.

Original: (5) for all T ≥ T 0 with high probability by union bound.
Revised: (5) for all T ≥ T 0 with high probability by union bound.

Original: Therefore, by probabilistic method there exists a (determistic) sequence (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t so that eq.
Revised: Therefore, by probabilistic method there exists a (determistic) sequence (ˆ x ∗ t , ˆ y ∗ t ) 1 ≤ t so that eq.

Original: (5) holds for all T ≥ T 0 .
Revised: (5) holds for all T ≥ T 0 .

Original: C.2 Truthful Convergence Implies No regret
Revised: C.2 Truthful Convergence Implies No regret

Original: Proof of theorem 3.2.
Revised: Proof of theorem 3.2.

Original: Given reports ˆ x and signals x , let ˆ x t − k : t
Revised: Given reports ˆ x and signals x , let ˆ x t − k : t

Original: =
Revised: =

Original: (ˆ x t − k
Revised: (ˆ x t − k

Original: +1 , . .
Revised: +1 , . .

Original: , ˆ x t ) be a slice of reports ˆ x , and opt i ( x t − k : t )
Revised: , ˆ x t ) be a slice of reports ˆ x , and opt i ( x t − k : t )

Original: =
Revised: =

Original: ( opt i ( x t − k +1 ) , . . .
Revised: ( opt i ( x t − k +1 ) , . . .

Original: , opt i ( x t )) be a slice reports under consistent strategy opt i .
Revised: , opt i ( x t )) be a slice reports under consistent strategy opt i .

Original: Given T , we define four functions on the signals
Revised: Given T , we define four functions on the signals

Original: [Equation]
Revised: [Equation]

Original: for i = 1 , 2 , 3 , 4 .
Revised: for i = 1 , 2 , 3 , 4 .

Original: We want to show the value of F iT is small.
Revised: We want to show the value of F iT is small.

Original: Specifically, we bound the probability of the following good event
Revised: Specifically, we bound the probability of the following good event

Original: [Equation]
Revised: [Equation]

Original: First because ¯ M is strongly truthful, E σ X [ ¯ M X (ˆ x , y )] ≤ E [ ¯ M X ( x , y )] and E σ Y [ ¯ M Y ( x , ˆ y )] ≤ E [ ¯ M Y
Revised: First because ¯ M is strongly truthful, E σ X [ ¯ M X (ˆ x , y )] ≤ E [ ¯ M X ( x , y )] and E σ Y [ ¯ M Y ( x , ˆ y )] ≤ E [ ¯ M Y

Original: ( x , y )] , the expectation is non-positive
Revised: ( x , y )] , the expectation is non-positive

Original: E [
Revised: E [

Original: [Equation]
Revised: [Equation]

Original: ( x ,
Revised: ( x ,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: for all i .
Revised: for all i .

Original: Second, because each round’s signals are mutually independent and can only affect at most 2 k − 1 round of payoff, by Chernoff bound on F iT , we have for all i , Pr (cid:2)
Revised: Second, because each round’s signals are mutually independent and can only affect at most 2 k − 1 round of payoff, by Chernoff bound on F iT , we have for all i , Pr (cid:2)

Original: F iT ( x , y ) ≥ T 2 / 3 (cid:3) ≤
Revised: F iT ( x , y ) ≥ T 2 / 3 (cid:3) ≤

Original: Pr (cid:2) F iT ( x , y ) ≥ T 2 / 3 + E F iT (cid:3) ≤ exp (cid:0) − 2 16 k 2 T 1 / 3 (cid:1) .
Revised: Pr (cid:2) F iT ( x , y ) ≥ T 2 / 3 + E F iT (cid:3) ≤ exp (cid:0) − 2 16 k 2 T 1 / 3 (cid:1) .

Original: By union bound, we have
Revised: By union bound, we have

Original: [Equation]
Revised: [Equation]

Original: On the other hand, the truthful convergence consists of two disjoint events: both converging to truth telling opt 1 , and both converging to the flipping strategy opt 2 .
Revised: On the other hand, the truthful convergence consists of two disjoint events: both converging to truth telling opt 1 , and both converging to the flipping strategy opt 2 .

Original: By symmetric suppose the first event happens with a nonzero probability
Revised: By symmetric suppose the first event happens with a nonzero probability

Original: [Equation]
Revised: [Equation]

Original: Then there exists a random round t ∗ so that opt Xt = opt Yt = opt 1 for all t ≥ t ∗ given E .
Revised: Then there exists a random round t ∗ so that opt Xt = opt Yt = opt 1 for all t ≥ t ∗ given E .

Original: To bound the expected regret conditional on E , we consider two cases: If the converge time t ∗ is greater than
Revised: To bound the expected regret conditional on E , we consider two cases: If the converge time t ∗ is greater than

Original: T 2 / 3 , we use the truthful convergence to show the probability is small.
Revised: T 2 / 3 , we use the truthful convergence to show the probability is small.

Original: Otherwise if the converge time t ∗ is smaller than T 2 / 3 , we can ignore the first term.
Revised: Otherwise if the converge time t ∗ is smaller than T 2 / 3 , we can ignore the first term.

Original: Formally, Alice’s expected regret is
Revised: Formally, Alice’s expected regret is

Original: [Equation]
Revised: [Equation]

Original: For the first term, E (cid:2) 1 [ t ∗ > T 2 / 3 ] Reg X ( T ) | E (cid:3) ≤ T
Revised: For the first term, E (cid:2) 1 [ t ∗ > T 2 / 3 ] Reg X ( T ) | E (cid:3) ≤ T

Original: [Equation]
Revised: [Equation]

Original: 1 [ t ∗ > T 2 / 3 ] | E (cid:3) =
Revised: 1 [ t ∗ > T 2 / 3 ] | E (cid:3) =

Original: T Pr[ t ∗ > T 2 / 3 | E ] .
Revised: T Pr[ t ∗ > T 2 / 3 | E ] .

Original: Because Pr[ t ∗ > T 2 / 3 ] = o (1) as T increases due to truthful convergence, and E happens with nonzero probability, we have
Revised: Because Pr[ t ∗ > T 2 / 3 ] = o (1) as T increases due to truthful convergence, and E happens with nonzero probability, we have

Original: [Equation]
Revised: [Equation]

Original: On the other hand, when t ∗ ≤ T 2 / 3 happens,
Revised: On the other hand, when t ∗ ≤ T 2 / 3 happens,

Original: [Equation]
Revised: [Equation]

Original: ≤
Revised: ≤

Original: [Equation]
Revised: [Equation]

Original: max i
Revised: max i

Original: [Equation]
Revised: [Equation]

Original: ( r t and r i,t are in [0 , 1] )
Revised: ( r t and r i,t are in [0 , 1] )

Original: [Equation]
Revised: [Equation]

Original: i
Revised: i

Original: [Equation]
Revised: [Equation]

Original: adding addition
Revised: adding addition

Original: [Equation]
Revised: [Equation]

Original: Therefore,
Revised: Therefore,

Original: [Equation]
Revised: [Equation]

Original: To bound the second term, we partition the expectation by whether G happens or not
Revised: To bound the second term, we partition the expectation by whether G happens or not

Original: [Equation]
Revised: [Equation]

Original: (definition of G )
Revised: (definition of G )

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: eq
Revised: eq

Original: [Equation]
Revised: [Equation]

Original: Therefore, with eqs.
Revised: Therefore, with eqs.

Original: (7) and (8) we show E (cid:2)
Revised: (7) and (8) we show E (cid:2)

Original: Reg X ( T ) | E (cid:3) =
Revised: Reg X ( T ) | E (cid:3) =

Original: o ( T )+

Revised: o ( T )+

Original: ( T 2 / 3 + k )
Revised: ( T 2 / 3 + k )

Original: + O ( T 2 / 3 )
Revised: + O ( T 2 / 3 )

Original: =
Revised: =

Original: o ( T ) that completes the proof.
Revised: o ( T ) that completes the proof.

Original: D Justifications of Reward-Based Online Learning Algorithm Family A
Revised: D Justifications of Reward-Based Online Learning Algorithm Family A

Original: In this section, we have two subsections to give justifications for learning algorithm family A .
Revised: In this section, we have two subsections to give justifications for learning algorithm family A .

Original: In the first part, we introduce two common used learning algorithms and show that they are both reward-based online learning algorithms.
Revised: In the first part, we introduce two common used learning algorithms and show that they are both reward-based online learning algorithms.

Original: The second part gives justifications for assumption 3.5.
Revised: The second part gives justifications for assumption 3.5.

Original: D.1 Learning Algorithms in A
Revised: D.1 Learning Algorithms in A

Original: In this section, we do not focus on peer prediction problems but consider learning algorithms used on general online decision problems.
Revised: In this section, we do not focus on peer prediction problems but consider learning algorithms used on general online decision problems.

Original: D.1.1 Follow the Perturbed Leader
Revised: D.1.1 Follow the Perturbed Leader

Original: FPL algorithm is designed by [13].
Revised: FPL algorithm is designed by [13].

Original: In their work, they have proved that FPL algorithm achieves no best-in-hindsight regret in full-information online decision problems.
Revised: In their work, they have proved that FPL algorithm achieves no best-in-hindsight regret in full-information online decision problems.

Original: For simplicity, we consider FPL using on online decision problem with four options opt i , i ∈ [4] in total.
Revised: For simplicity, we consider FPL using on online decision problem with four options opt i , i ∈ [4] in total.

Original: The algorithm let the agent choose an arbitrary option among arg max opt i ∈{ opt 1 , opt 2 , opt 3 , opt 4 } ( R i,t − 

Revised: The algorithm let the agent choose an arbitrary option among arg max opt i ∈{ opt 1 , opt 2 , opt 3 , opt 4 } ( R i,t −

Original: + p i,t ) in the t th round of the game.
Revised: + p i,t ) in the t th round of the game.

Original: Here, p i,t ’s are i.i.d.
Revised: Here, p i,t ’s are i.i.d.

Original: sampled from a particular noise distribution N .
Revised: sampled from a particular noise distribution N .

Original: Because of variety of the noise distribution, FPL algorithm actually contains a large family of learning algorithms, i.e.,
Revised: Because of variety of the noise distribution, FPL algorithm actually contains a large family of learning algorithms, i.e.,

Original: Hannan’s algorithm [11] and Follow the Leader algorithm (FTL).
Revised: Hannan’s algorithm [11] and Follow the Leader algorithm (FTL).

Original: In formal, we have the following theorem.
Revised: In formal, we have the following theorem.

Original: Theorem D.1.
Revised: Theorem D.1.

Original: Algorithm 1 is a reward-based online learning algorithm included in A .
Revised: Algorithm 1 is a reward-based online learning algorithm included in A .

Original: Proof.
Revised: Proof.

Original: To prove that FPL algorithm is in our algorithm family A , we need to design a function f FPL such that
Revised: To prove that FPL algorithm is in our algorithm family A , we need to design a function f FPL such that

Original: [Equation]
Revised: [Equation]

Original: for i ∈ [4] .
Revised: for i ∈ [4] .

Original: Using this function based on cumulative payoffs, A f FPL is equivalent to algorithm 1.
Revised: Using this function based on cumulative payoffs, A f FPL is equivalent to algorithm 1.

Original: In detail, the probability of choosing opt i in the t th round in algorithm 1 is exactly
Revised: In detail, the probability of choosing opt i in the t th round in algorithm 1 is exactly

Original: [Equation]
Revised: [Equation]

Original: Now we only need to verify that f FPL satisfies assumption 3.3, assumption 3.5 and assumption 3.4.
Revised: Now we only need to verify that f FPL satisfies assumption 3.3, assumption 3.5 and assumption 3.4.

Original: Assumption 3.3 holds because the expression of probability
Revised: Assumption 3.3 holds because the expression of probability

Original: [Equation]
Revised: [Equation]

Original: is symmetrical with respect to i .
Revised: is symmetrical with respect to i .

Original: We know that for ∀ ϵ > 0 , we can find a large enough positive constant n such that Pr { p − p ′ ≥ n | p, p ′ iid ∼ N} < ϵ .
Revised: We know that for ∀ ϵ > 0 , we can find a large enough positive constant n such that Pr { p − p ′ ≥ n | p, p ′ iid ∼ N} < ϵ .

Original: Therefore, when R 1 > max { R 2 , R 3 , R 4 } + n , we can deduce that
Revised: Therefore, when R 1 > max { R 2 , R 3 , R 4 } + n , we can deduce that

Original: [Equation]
Revised: [Equation]

Original: i =
Revised: i =

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Pr (cid:110) R
Revised: Pr (cid:110) R

Original: [Table]
Revised: [Table]

Original: p
Revised: p

Original: [Table]
Revised: [Table]

Original: < R i + p i (cid:12)(cid:12)(cid:12) p i iid
Revised: < R i + p i (cid:12)(cid:12)(cid:12) p i iid

Original: [Table]
Revised: [Table]

Original: N , i ∈
Revised: N , i ∈

Original: [Table]
Revised: [Table]

Original: 4] (cid:111) ≥
Revised: 4] (cid:111) ≥

Original: [Table]
Revised: [Table]

Original: − (cid:88) i =
Revised: − (cid:88) i =

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Pr (cid:110) p 1 + n < p
Revised: Pr (cid:110) p 1 + n < p

Original: i (cid:12)(cid:12)(cid:12) p i
Revised: i (cid:12)(cid:12)(cid:12) p i

Original: [Table]
Revised: [Table]

Original: N , i ∈ [4] (cid:111) ≥ 1 − 3 ϵ.
Revised: N , i ∈ [4] (cid:111) ≥ 1 − 3 ϵ.

Original: Therefore, Assumption 3.5 holds.
Revised: Therefore, Assumption 3.5 holds.

Original: About Assumption 3.4, it is obvious by the definition of f FPL because each of p i ’s are sampled from an identical noise distribution.
Revised: About Assumption 3.4, it is obvious by the definition of f FPL because each of p i ’s are sampled from an identical noise distribution.

Original: D.1.2 Multiplicative Weight Algorithm
Revised: D.1.2 Multiplicative Weight Algorithm

Original: Multiplicative weights algorithm (or hedge algorithm) is first introduced in [10], which is called exponentially weighted averaged forecaster by them.
Revised: Multiplicative weights algorithm (or hedge algorithm) is first introduced in [10], which is called exponentially weighted averaged forecaster by them.

Original: It is also a no-regret algorithm in full-information online decision problem.
Revised: It is also a no-regret algorithm in full-information online decision problem.

Original: It can be written as Algorithm 2 for an online decision problem with four options opt i , i ∈ [4] .
Revised: It can be written as Algorithm 2 for an online decision problem with four options opt i , i ∈ [4] .

Original: In formal, we have the following theorem.
Revised: In formal, we have the following theorem.

Original: Theorem D.2.
Revised: Theorem D.2.

Original: Algorithm 2 is a reward-based online learning algorithm included in A .
Revised: Algorithm 2 is a reward-based online learning algorithm included in A .

Original: Proof.
Revised: Proof.

Original: To prove that algorithm 2 is in our algorithm family A , we can use a function f hedge 2 such that
Revised: To prove that algorithm 2 is in our algorithm family A , we can use a function f hedge 2 such that

Original: [Equation]
Revised: [Equation]

Original: for i ∈ [4] .
Revised: for i ∈ [4] .

Original: Using this function based on cumulative payoffs, A f hedge2 is equivalent to algorithm 2.
Revised: Using this function based on cumulative payoffs, A f hedge2 is equivalent to algorithm 2.

Original: In detail, the probability of choosing opt i in the t th round in algorithm 2 is exactly
Revised: In detail, the probability of choosing opt i in the t th round in algorithm 2 is exactly

Original: [Equation]
Revised: [Equation]

Original: Now we only need to verify that f hedge 2 satisfies assumption 3.3, assumption 3.5 and assumption 3.4.
Revised: Now we only need to verify that f hedge 2 satisfies assumption 3.3, assumption 3.5 and assumption 3.4.

Original: Assumption 3.3 holds because e
Revised: Assumption 3.3 holds because e

Original: [Equation]
Revised: [Equation]

Original: is symmetrical with respect to R 1 , R 2 , R 3 , R 4 .
Revised: is symmetrical with respect to R 1 , R 2 , R 3 , R 4 .

Original: Assumption 3.4 holds because β > 0 and exponential function is monotonic.
Revised: Assumption 3.4 holds because β > 0 and exponential function is monotonic.

Original: When R 1 − max { R 2 , R 3 , R 4 } > n > 0 , we have
Revised: When R 1 − max { R 2 , R 3 , R 4 } > n > 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, we deduce that
Revised: Therefore, we deduce that

Original: [Equation]
Revised: [Equation]

Original: + 3 e − βn
Revised: + 3 e − βn

Original: = 1 .
Revised: = 1 .

Original: Therefore, assumption 3.5 holds.
Revised: Therefore, assumption 3.5 holds.

Original: D.2 Justifications for Assumption 3. 
Revised: D.2 Justifications for Assumption 3.

Original: In this section, we prove that assumption 3.5 is a necessary condition for a reward-based online learning algorithm to be no-regret.
Revised: In this section, we prove that assumption 3.5 is a necessary condition for a reward-based online learning algorithm to be no-regret.

Original: Theorem D.3.
Revised: Theorem D.3.

Original: For a reward-based function f : R 4 → △ 3 , if
Revised: For a reward-based function f : R 4 → △ 3 , if

Original: [Equation]
Revised: [Equation]

Original: mechanism A f cannot be a no-regret algorithm for general online decision problem.
Revised: mechanism A f cannot be a no-regret algorithm for general online decision problem.

Original: Proof.
Revised: Proof.

Original: We design an online decision problem such that r 1 ,t = 2 , r 2 ,t
Revised: We design an online decision problem such that r 1 ,t = 2 , r 2 ,t

Original: = r 3 ,t = r 4 ,t = 1 for t ∈ N + .
Revised: = r 3 ,t = r 4 ,t = 1 for t ∈ N + .

Original: According to the description of f , there exists a n such that when R 1 − max { R 2 , R 3 , R 4 } ≥ n , f 1 ( R 1 , R 2 , R 3 , R 4 ) <
Revised: According to the description of f , there exists a n such that when R 1 − max { R 2 , R 3 , R 4 } ≥ n , f 1 ( R 1 , R 2 , R 3 , R 4 ) <

Original: 1 − c 2 .
Revised: 1 − c 2 .

Original: Notice that when t > n + 1 , we have R 1 ,t − 1 = 2 t > t + n = max { R 2 ,t − 1 , R 3 ,t − 1 , R 4 ,t − 1 } + n , therefore, when t > n +1 , the agent chooses opt 1 with probability at most 1 − c 2 in the t th round.
Revised: Notice that when t > n + 1 , we have R 1 ,t − 1 = 2 t > t + n = max { R 2 ,t − 1 , R 3 ,t − 1 , R 4 ,t − 1 } + n , therefore, when t > n +1 , the agent chooses opt 1 with probability at most 1 − c 2 in the t th round.

Original: Therefore, we can deduce that when T > n + 2 , E [ Reg ( T )] ≥ c 2 ( T − n − 1) , which is a linear function of T .
Revised: Therefore, we can deduce that when T > n + 2 , E [ Reg ( T )] ≥ c 2 ( T − n − 1) , which is a linear function of T .

Original: Therefore, A f is not no-regret.
Revised: Therefore, A f is not no-regret.

Original: E More Algorithms in A Applying on CA Mechanism Binary Sequential Peer Prediction
Revised: E More Algorithms in A Applying on CA Mechanism Binary Sequential Peer Prediction

Original: In appendix D.1, we have introduced two widely used algorithms that are reward-based online learning algorithms.
Revised: In appendix D.1, we have introduced two widely used algorithms that are reward-based online learning algorithms.

Original: In this section, we show that there are even more existing learning algorithms contained by A when the game is exactly binary sequential peer prediction using CA mechanism.
Revised: In this section, we show that there are even more existing learning algorithms contained by A when the game is exactly binary sequential peer prediction using CA mechanism.

Original: E.1 Replicator Dynamics
Revised: E.1 Replicator Dynamics

Original: Replicator dynamics track a set of agents in a repeating game and each agent chooses a pure strategy with a probability proportional to expected payoffs deviating to higher-payoff options.
Revised: Replicator dynamics track a set of agents in a repeating game and each agent chooses a pure strategy with a probability proportional to expected payoffs deviating to higher-payoff options.

Original: We use a similar implementation as [26] in a general discrete form here to show that replicator dynamics are also in A for the binary signal peer prediction problem.
Revised: We use a similar implementation as [26] in a general discrete form here to show that replicator dynamics are also in A for the binary signal peer prediction problem.

Original: To be more specific, during the repeating game, the agent maintains four probabilities q i,t , i ∈ [4] and choose opt i with probability q i,t , and then update them in the end of the t th round.
Revised: To be more specific, during the repeating game, the agent maintains four probabilities q i,t , i ∈ [4] and choose opt i with probability q i,t , and then update them in the end of the t th round.

Original: The updating rule is set as below:
Revised: The updating rule is set as below:

Original: [Equation]
Revised: [Equation]

Original: where h : R → R + is a monotonic function.
Revised: where h : R → R + is a monotonic function.

Original: Due to the variety of h , our discretized replicator dynamics contain extensive learning algorithms.
Revised: Due to the variety of h , our discretized replicator dynamics contain extensive learning algorithms.

Original: Common discretization of replicator dynamics set h as exponential function or linear function, but we consider general h functions here.
Revised: Common discretization of replicator dynamics set h as exponential function or linear function, but we consider general h functions here.

Original: ALGORITHM 3:
Revised: ALGORITHM 3:

Original: [Equation]
Revised: [Equation]

Original: We have the following theorem.
Revised: We have the following theorem.

Original: Theorem E.1.
Revised: Theorem E.1.

Original: When applying on binary sequential peer prediction using CA mechanism, algorithmis included in A .
Revised: When applying on binary sequential peer prediction using CA mechanism, algorithmis included in A .

Original: Proof.
Revised: Proof.

Original: According to eq.
Revised: According to eq.

Original: (9), we know that
Revised: (9), we know that

Original: [Equation]
Revised: [Equation]

Original: Here, eq.
Revised: Here, eq.

Original: (10) and eq.
Revised: (10) and eq.

Original: (11) are because there are only three realizations of ( r 1 ,i , r 2 ,i ) , which are (1 , − 1) , (0 , 0) and ( − 1 , 1) by the definition of CA mechanism and binary sequential peer prediction.
Revised: (11) are because there are only three realizations of ( r 1 ,i , r 2 ,i ) , which are (1 , − 1) , (0 , 0) and ( − 1 , 1) by the definition of CA mechanism and binary sequential peer prediction.

Original: Similarly for opt 3 , opt 4 , we can deduce that q 3 ,t +1 q 4 ,t +1 =
Revised: Similarly for opt 3 , opt 4 , we can deduce that q 3 ,t +1 q 4 ,t +1 =

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Moreover, we have
Revised: Moreover, we have

Original: [Equation]
Revised: [Equation]

Original: This is because ( r 1 ,i , r 2 ,i , r 3 ,i , r 4 ,i ) has only five possible realizations, which are (0 , 0 , 0 , 0) , (1 , − 1 , 1 , − 1) , (1 , − 1 , − 1 , 1) , ( − 1 , 1 , 1 , − 1) and ( − 1 , 1 , − 1 , 1) according to the definition of CA mechanism and binary sequential peer prediction for any i ∈ N + .
Revised: This is because ( r 1 ,i , r 2 ,i , r 3 ,i , r 4 ,i ) has only five possible realizations, which are (0 , 0 , 0 , 0) , (1 , − 1 , 1 , − 1) , (1 , − 1 , − 1 , 1) , ( − 1 , 1 , 1 , − 1) and ( − 1 , 1 , − 1 , 1) according to the definition of CA mechanism and binary sequential peer prediction for any i ∈ N + .

Original: According to lemma 4.3, we know that R 1 ,t + R 2 ,t = R 3 ,t + R 4 ,t .
Revised: According to lemma 4.3, we know that R 1 ,t + R 2 ,t = R 3 ,t + R 4 ,t .

Original: Therefore, we can deduce that for replicator dynamics applying on binary sequential peer prediction using CA mechanism, we have
Revised: Therefore, we can deduce that for replicator dynamics applying on binary sequential peer prediction using CA mechanism, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: R
Revised: R

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: h
Revised: h

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: h
Revised: h

Original: [Equation]
Revised: [Equation]

Original: 1) h ( − 1) (cid:17) R
Revised: 1) h ( − 1) (cid:17) R

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation]
Revised: [Equation]

Original: h (1) h ( − 1)
Revised: h (1) h ( − 1)

Original: [Equation]
Revised: [Equation]

Original: R 4 ,t
Revised: R 4 ,t

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: h (1) h ( − 1)
Revised: h (1) h ( − 1)

Original: [Equation]
Revised: [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: t
Revised: t

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: [Equation]
Revised: [Equation]

Original: R
Revised: R

Original: [Equation]
Revised: [Equation]

Original: h (
Revised: h (

Original: [Equation]
Revised: [Equation]

Original: (cid:17)
Revised: (cid:17)

Original: [Equation]
Revised: [Equation]

Original: Hence
Revised: Hence

Original: [Equation]
Revised: [Equation]

Original: now replicator dynamics behave the same as mechanism A f replicator such that
Revised: now replicator dynamics behave the same as mechanism A f replicator such that

Original: [Equation]
Revised: [Equation]

Original: It is easy to verify that f satisfies assumption 3.3, assumption 3.5 and assumption 3.4.
Revised: It is easy to verify that f satisfies assumption 3.3, assumption 3.5 and assumption 3.4.

Original: In detail, assumption 3.3 holds because f replicator is symmetrical obviously with respect to
Revised: In detail, assumption 3.3 holds because f replicator is symmetrical obviously with respect to

Original: R 1 , R 2 , R 3 , R 4 .
Revised: R 1 , R 2 , R 3 , R 4 .

Original: Moreover, we know that
Revised: Moreover, we know that

Original: [Equation]
Revised: [Equation]

Original: Therefore, when R 1 − max { R 2 , R 3 , R 4 } → + ∞ , we have
Revised: Therefore, when R 1 − max { R 2 , R 3 , R 4 } → + ∞ , we have

Original: [Equation]
Revised: [Equation]

Original: → 1 .
Revised: → 1 .

Original: Thus f replicator dynamics( R 1 , R 2 , R 3 , R 4 ) → 1 .
Revised: Thus f replicator dynamics( R 1 , R 2 , R 3 , R 4 ) → 1 .

Original: Hence, we have proved that assumption 3.5 holds.
Revised: Hence, we have proved that assumption 3.5 holds.

Original: Finally, assumption 3.4 also holds obviously according to the definition of f replicator and monotonicity of exponential function.
Revised: Finally, assumption 3.4 also holds obviously according to the definition of f replicator and monotonicity of exponential function.

Original: E
Revised: E

Original: An Alternating Version of Multiplicative Weights Algorithm
Revised: An Alternating Version of Multiplicative Weights Algorithm

Original: In appendix D.1, we have already introduced an exponential updating function form of multiplicative weights algorithm [10].
Revised: In appendix D.1, we have already introduced an exponential updating function form of multiplicative weights algorithm [10].

Original: There is another version multiplicative weights algorithm introduced in the survey [2], which can be written as algorithm 4 for our particular binary sequential peer prediction problem using CA mechanism.
Revised: There is another version multiplicative weights algorithm introduced in the survey [2], which can be written as algorithm 4 for our particular binary sequential peer prediction problem using CA mechanism.

Original: Update w t
Revised: Update w t

Original: +

Revised: +

Original: =
Revised: =

Original: 
Revised: 

Original:  (1 + β ) w i,t r i,t =w i,t r i,t =(1 − β ) w i,t r i,t = −.
Revised:  (1 + β ) w i,t r i,t =w i,t r i,t =(1 − β ) w i,t r i,t = −.

Original: 5 end
Revised: 5 end

Original: We only need to set h : R → R + to satisfy that h (1)
Revised: We only need to set h : R → R + to satisfy that h (1)

Original: = 

Revised: =

Original: + β , h (0)
Revised: + β , h (0)

Original: = 1 and h ( − 1)
Revised: = 1 and h ( − 1)

Original: = 1 − β .
Revised: = 1 − β .

Original: Then algorithm 3 behaves completely the same as algorithm 4.
Revised: Then algorithm 3 behaves completely the same as algorithm 4.

Original: Therefore, according to theorem E.1, we have theorem E.2 for this alternating version of multiplicative weights algorithm.
Revised: Therefore, according to theorem E.1, we have theorem E.2 for this alternating version of multiplicative weights algorithm.

Original: Theorem E.2.
Revised: Theorem E.2.

Original: When applying on binary sequential peer prediction using CA mechanism, algorithmis included in A .
Revised: When applying on binary sequential peer prediction using CA mechanism, algorithmis included in A .

Original: F Proofs and Details of SectionF.1 Proofs of Properties of CA Mechanism in Binary Signal Peer Prediction Games F.1.1 Proof of Lemma 4. 
Revised: F Proofs and Details of SectionF.1 Proofs of Properties of CA Mechanism in Binary Signal Peer Prediction Games F.1.1 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: By the definition of CA mechanism, we know that r 3 ,j = I [1 = ˆ y j ] − I [1 = ˆ y j − 1 ] for j ∈ N + .
Revised: By the definition of CA mechanism, we know that r 3 ,j = I [1 = ˆ y j ] − I [1 = ˆ y j − 1 ] for j ∈ N + .

Original: Therefore, we can deduce that
Revised: Therefore, we can deduce that

Original: R 3 ,t = t (cid:88)
Revised: R 3 ,t = t (cid:88)

Original: j = r 3 ,j = t (cid:88)
Revised: j = r 3 ,j = t (cid:88)

Original: j = ( I [1 = ˆ y j ] − I [1 = ˆ y j − 1 ]) =
Revised: j = ( I [1 = ˆ y j ] − I [1 = ˆ y j − 1 ]) =

Original: t (cid:88) j
Revised: t (cid:88) j

Original: =
Revised: =

Original: I [1 = ˆ y j ]
Revised: I [1 = ˆ y j ]

Original: [Equation]
Revised: [Equation]

Original: For strategy (cid:80) tj =1 p 4 ,j , (cid:80) tj =1 q 3 ,j , (cid:80) tj =1 q 4 ,j , the deductions are all similar.
Revised: For strategy (cid:80) tj =1 p 4 ,j , (cid:80) tj =1 q 3 ,j , (cid:80) tj =1 q 4 ,j , the deductions are all similar.

Original: F.1.2 Proof of Lemma 4. 
Revised: F.1.2 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: By the definition of CA mechanism, we know that r 1 ,j = I
Revised: By the definition of CA mechanism, we know that r 1 ,j = I

Original: [ x j = ˆ y j ] − I [ x j = ˆ y j − 1 ] and r 2 ,j = I [1 − x j = ˆ y j ] − I [1 − x j = ˆ y j − 1 ] .
Revised: [ x j = ˆ y j ] − I [ x j = ˆ y j − 1 ] and r 2 ,j = I [1 − x j = ˆ y j ] − I [1 − x j = ˆ y j − 1 ] .

Original: Therefore, we can deduce that
Revised: Therefore, we can deduce that

Original: R 1 ,t + R 2 ,t = t (cid:88)
Revised: R 1 ,t + R 2 ,t = t (cid:88)

Original: j
Revised: j

Original: = ( r 1 ,j + r 2 ,j ) =
Revised: = ( r 1 ,j + r 2 ,j ) =

Original: t (cid:88)
Revised: t (cid:88)

Original: j = ( I [ x j = ˆ y j ] − I [ x j = ˆ y j − 1 ]) + t (cid:88)
Revised: j = ( I [ x j = ˆ y j ] − I [ x j = ˆ y j − 1 ]) + t (cid:88)

Original: j = ( I [1 − x j = ˆ y j ] − I [1 − x j = ˆ y j − 1 ]) =
Revised: j = ( I [1 − x j = ˆ y j ] − I [1 − x j = ˆ y j − 1 ]) =

Original: t (cid:88)
Revised: t (cid:88)

Original: j = ( I [ x j = ˆ y j ]
Revised: j = ( I [ x j = ˆ y j ]

Original: + I [1 − x j = ˆ y j ]) − t (cid:88)
Revised: + I [1 − x j = ˆ y j ]) − t (cid:88)

Original: j = ( I [ x j = ˆ y j − 1 ]
Revised: j = ( I [ x j = ˆ y j − 1 ]

Original: + I [1 − x j = ˆ y j − 1 ])
Revised: + I [1 − x j = ˆ y j − 1 ])

Original: =
Revised: =

Original: t − t = 0 .
Revised: t − t = 0 .

Original: Similar deductions can be made for R 3 ,t + R 4 ,t , S 1 ,t + S 2 ,t and S 3 ,t + S 4 ,t , which completes the proof.
Revised: Similar deductions can be made for R 3 ,t + R 4 ,t , S 1 ,t + S 2 ,t and S 3 ,t + S 4 ,t , which completes the proof.

Original: F.2 Proofs of Truthful Convergence for CA mechanism on Reward-based Algorithms F.2.1 Proof of Lemma 4. 
Revised: F.2 Proofs of Truthful Convergence for CA mechanism on Reward-based Algorithms F.2.1 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: Because Alice and Bob are symmetric, we only consider Alice.
Revised: Because Alice and Bob are symmetric, we only consider Alice.

Original: By lemma 4.3, without loss of generality, we suppose that (cid:80) tj =1 r 1 ,j ≥ 0 .
Revised: By lemma 4.3, without loss of generality, we suppose that (cid:80) tj =1 r 1 ,j ≥ 0 .

Original: Then R 1 ,t ≥ 1 and R 2 ,t ≤ 0 or R 1 ,t = R 2 ,t = 0 because each r 1 ,j equals to − 1 , 0 or 1 , which is an integer.
Revised: Then R 1 ,t ≥ 1 and R 2 ,t ≤ 0 or R 1 ,t = R 2 ,t = 0 because each r 1 ,j equals to − 1 , 0 or 1 , which is an integer.

Original: If R 1 ,t ≥ 1 , then we can deduce that R 1 ,t ≥ max i =2 , 3 , 4 R i,t .
Revised: If R 1 ,t ≥ 1 , then we can deduce that R 1 ,t ≥ max i =2 , 3 , 4 R i,t .

Original: Therefore, f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ max i =2 , 3 , 4 f i ( R t, 1 , R 2 ,t , R 3 ,t , R 4 ,t ) according to assumption 3.4.
Revised: Therefore, f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ max i =2 , 3 , 4 f i ( R t, 1 , R 2 ,t , R 3 ,t , R 4 ,t ) according to assumption 3.4.

Original: We know that (cid:80) i ∈ [4] f i ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t )
Revised: We know that (cid:80) i ∈ [4] f i ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t )

Original: = 1 , so f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ 14 .
Revised: = 1 , so f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ 14 .

Original: If R 1 ,t = R 2 ,t = R 3 ,t = R 4 ,t = 0 , we can also deduce that R 1 ,t ≥ max i =2 , 3 , 4 R i,t .
Revised: If R 1 ,t = R 2 ,t = R 3 ,t = R 4 ,t = 0 , we can also deduce that R 1 ,t ≥ max i =2 , 3 , 4 R i,t .

Original: Therefore, f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ max i =2 , 3 , 4 f i ( R t, 1 , R 2 ,t , R 3 ,t , R 4 ,t ) according to assumption 3.4.
Revised: Therefore, f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ max i =2 , 3 , 4 f i ( R t, 1 , R 2 ,t , R 3 ,t , R 4 ,t ) according to assumption 3.4.

Original: We know that (cid:80) i ∈ [4] f i ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t )
Revised: We know that (cid:80) i ∈ [4] f i ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t )

Original: = 1 , so f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ 14 .
Revised: = 1 , so f 1 ( R 1 ,t , R 2 ,t , R 3 ,t , R 4 ,t ) ≥ 14 .

Original: Finally, we prove that { R 1 ,t = R 2 ,t = 0 , R 3 ,t = 1 , R 4 ,t = − 1 } and { R 1 ,t = R 2 ,t = 0 , R 3 ,t = − 1 , R 4 ,t = 1 } never occur in our game.
Revised: Finally, we prove that { R 1 ,t = R 2 ,t = 0 , R 3 ,t = 1 , R 4 ,t = − 1 } and { R 1 ,t = R 2 ,t = 0 , R 3 ,t = − 1 , R 4 ,t = 1 } never occur in our game.

Original: According to the definition of CA mechanism, we know that ( r 1 ,t , r 2 ,t , r 3 ,t , r 4 ,t ) has only five possibilities, which are (0 , 0 , 0 , 0) , (1 , − 1 , 1 , − 1) , (1 , − 1 , − 1 , 1) , ( − 1 , 1 , 1 , − 1) and ( − 1 , 1 , − 1 , 1) .
Revised: According to the definition of CA mechanism, we know that ( r 1 ,t , r 2 ,t , r 3 ,t , r 4 ,t ) has only five possibilities, which are (0 , 0 , 0 , 0) , (1 , − 1 , 1 , − 1) , (1 , − 1 , − 1 , 1) , ( − 1 , 1 , 1 , − 1) and ( − 1 , 1 , − 1 , 1) .

Original: Therefore, we can deduce that r 1 ,t − r 2 ,t + r 3 ,t − r 4 ,t can be divided by 4 .
Revised: Therefore, we can deduce that r 1 ,t − r 2 ,t + r 3 ,t − r 4 ,t can be divided by 4 .

Original: Hence, R 1 ,t − R 2 ,t + R 3 ,t − R 4 ,t = (cid:80) ti =1 ( r 1 ,t − r 2 ,t + r 3 ,t − r 4 ,t ) is also divided by 4 .
Revised: Hence, R 1 ,t − R 2 ,t + R 3 ,t − R 4 ,t = (cid:80) ti =1 ( r 1 ,t − r 2 ,t + r 3 ,t − r 4 ,t ) is also divided by 4 .

Original: This indicates that both { (cid:80) tj =1 r 1 ,j = (cid:80) tj =1 r 2 ,j = 0 , (cid:80) tj =1 r 3 ,j = 1 , (cid:80) tj =1 r 4 ,j = − 1 } and { (cid:80) t
Revised: This indicates that both { (cid:80) tj =1 r 1 ,j = (cid:80) tj =1 r 2 ,j = 0 , (cid:80) tj =1 r 3 ,j = 1 , (cid:80) tj =1 r 4 ,j = − 1 } and { (cid:80) t

Original: j =1 r 1 ,j = (cid:80) tj =1 r 2 ,j = 0 , (cid:80) t
Revised: j =1 r 1 ,j = (cid:80) tj =1 r 2 ,j = 0 , (cid:80) t

Original: j =1 r 3 ,j = − 1 , (cid:80) tj =1 r 4 ,j = 1 } cannot happen.
Revised: j =1 r 3 ,j = − 1 , (cid:80) tj =1 r 4 ,j = 1 } cannot happen.

Original: To sum up, the probability that Alice chooses opt 1 is larger than 14 , which is what we want.
Revised: To sum up, the probability that Alice chooses opt 1 is larger than 14 , which is what we want.

Original: F.2.2 Proof of Lemma 4. 
Revised: F.2.2 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: It is easy to verify that γ 1 > 0 .
Revised: It is easy to verify that γ 1 > 0 .

Original: Then under the situation that S 1 ,t − 1 > c 1 + 1 , for the next round, the expectation of r 1 ,t +1 − r 2 ,t +1 can be bounded as
Revised: Then under the situation that S 1 ,t − 1 > c 1 + 1 , for the next round, the expectation of r 1 ,t +1 − r 2 ,t +1 can be bounded as

Original: E [ r 1 ,t +1 − r 2 ,t +1 ] =
Revised: E [ r 1 ,t +1 − r 2 ,t +1 ] =

Original: E [( I [ x t +1 =
Revised: E [( I [ x t +1 =

Original: ˆ y t +1 ] − I [ x t +1 =
Revised: ˆ y t +1 ] − I [ x t +1 =

Original: ˆ y t ]) − ( I [1 − x t +1 =
Revised: ˆ y t ]) − ( I [1 − x t +1 =

Original: ˆ y t +1 ] − I [1 − x t +

Revised: ˆ y t +1 ] − I [1 − x t +

Original: = ˆ y t ])]
Revised: = ˆ y t ])]

Original: =
Revised: =

Original: E [ I [ x t +1 =
Revised: E [ I [ x t +1 =

Original: ˆ y t +1 ] − I [1 − x t +

Revised: ˆ y t +1 ] − I [1 − x t +

Original: =
Revised: =

Original: ˆ y t +1 ]] − E
Revised: ˆ y t +1 ]] − E

Original: [ I [ x t +1 =
Revised: [ I [ x t +1 =

Original: ˆ y t ] − I [1 − x t
Revised: ˆ y t ] − I [1 − x t

Original: +1 =
Revised: +1 =

Original: ˆ y t ]] ≥ Pr { opt Yt +1 = opt 1 } γ 1 − (cid:0) 1 − Pr { opt Yt +1 = opt 1 } (cid:1) − Pr { opt Yt = opt 1 } γ 2 + (1 − Pr { opt Yt = opt 1 } ) > (1 − δ ) γ 1 − δ − max { γ 2 , (1 − δ ) γ
Revised: ˆ y t ]] ≥ Pr { opt Yt +1 = opt 1 } γ 1 − (cid:0) 1 − Pr { opt Yt +1 = opt 1 } (cid:1) − Pr { opt Yt = opt 1 } γ 2 + (1 − Pr { opt Yt = opt 1 } ) > (1 − δ ) γ 1 − δ − max { γ 2 , (1 − δ ) γ

Original: 2 − δ
Revised: 2 − δ

Original: We know that
Revised: We know that

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we can always find δ such that
Revised: we can always find δ such that

Original: [Equation]
Revised: [Equation]

Original: which is what we want.
Revised: which is what we want.

Original: F.2.3 Proof of Lemma 4. 
Revised: F.2.3 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: Noticing that when E 1 , 2 t happens
Revised: Noticing that when E 1 , 2 t happens

Original: [Equation]
Revised: [Equation]

Original: Bob will choose opt 2 with probability larger than 1 − δ in the next ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds.
Revised: Bob will choose opt 2 with probability larger than 1 − δ in the next ⌈ 1000 γ 1 − γ 2 ⌉ + 1 rounds.

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we can deduce that
Revised: we can deduce that

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: such that
Revised: such that

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: [Figure]
Revised: [Figure]

Original: [Equation]
Revised: [Equation]

Original: H t such that E 1 , 2 t
Revised: H t such that E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: +
Revised: +

Original: [Equation]
Revised: [Equation]

Original: H t such that E 1 , 2 t =
Revised: H t such that E 1 , 2 t =

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 13) < − 100 .
Revised: 13) < − 100 .

Original: Here, eq.
Revised: Here, eq.

Original: (12) is because when j ≥ 2 , we have
Revised: (12) is because when j ≥ 2 , we have

Original: [Equation]
Revised: [Equation]

Original: H t such that E 1 , 2 t =
Revised: H t such that E 1 , 2 t =

Original: [Equation]
Revised: [Equation]

Original: t + j = ˆ y t + j − 1 ] − I [1 − x t + j = ˆ y t + j − 1 ] (cid:12)(cid:12)(cid:12) H t such that E 1 , 2 t = 1 (cid:105)
Revised: t + j = ˆ y t + j − 1 ] − I [1 − x t + j = ˆ y t + j − 1 ] (cid:12)(cid:12)(cid:12) H t such that E 1 , 2 t = 1 (cid:105)

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Pr { opt Yt + j = opt 1 } ) − Pr { opt Yt + j = opt 1 } γ− (1 − Pr { opt Yt + j − 1 = opt 1 } ) + Pr { opt Yt + j − 1 = opt 1 } γ< δ − (1 − δ ) γ 1 + max { γ 2 , (1 − δ ) γ 2 − δ } .
Revised: Pr { opt Yt + j = opt 1 } ) − Pr { opt Yt + j = opt 1 } γ− (1 − Pr { opt Yt + j − 1 = opt 1 } ) + Pr { opt Yt + j − 1 = opt 1 } γ< δ − (1 − δ ) γ 1 + max { γ 2 , (1 − δ ) γ 2 − δ } .

Original: Moreover, eq. (13) holds according to definition of δ in lemma 4.5.
Revised: Moreover, eq. (13) holds according to definition of δ in lemma 4.5.

Original: F.2.4 Proof of Lemma 4. 
Revised: F.2.4 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: First we show the complement of event lim sup t →∞ E 1 , 2 t ∨ E 2 , 1 t is lim inf
Revised: First we show the complement of event lim sup t →∞ E 1 , 2 t ∨ E 2 , 1 t is lim inf

Original: [Equation]
Revised: [Equation]

Original: ∨
Revised: ∨

Original: [Equation]
Revised: [Equation]

Original: According to the definition of lim sup , we can write lim sup t →∞ E 1 , 2 t ∨ E 2 , 1 t = 1 as ∧ ∞ t =1 (cid:16) ∨ ∞
Revised: According to the definition of lim sup , we can write lim sup t →∞ E 1 , 2 t ∨ E 2 , 1 t = 1 as ∧ ∞ t =1 (cid:16) ∨ ∞

Original: i =
Revised: i =

Original: t E 1 , 2 i ∨ E 2 , 1 i (cid:17) = 1 .
Revised: t E 1 , 2 i ∨ E 2 , 1 i (cid:17) = 1 .

Original: This is equivalent to ∨ ∞ t =1 (cid:16) ∧ ∞
Revised: This is equivalent to ∨ ∞ t =1 (cid:16) ∧ ∞

Original: i =
Revised: i =

Original: t (cid:16) E 1 , 2 i ∨ E 2 , 1 i (cid:17)(cid:17)
Revised: t (cid:16) E 1 , 2 i ∨ E 2 , 1 i (cid:17)(cid:17)

Original: = 

Revised: =

Original: by De Morgan’s law, which can be written as lim inf t →∞ (cid:16) E 1 , 2 t ∨ E 2 , 1 t (cid:17) = 0 .
Revised: by De Morgan’s law, which can be written as lim inf t →∞ (cid:16) E 1 , 2 t ∨ E 2 , 1 t (cid:17) = 0 .

Original: More concretely, according to the definition of lim inf , lim sup t →∞ E 1 , 2 t
Revised: More concretely, according to the definition of lim inf , lim sup t →∞ E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: Therefore, we only need to prove that Pr (cid:110) ∃ T ∈ N + , ∀ t ≥ T, E 1 , 2 t
Revised: Therefore, we only need to prove that Pr (cid:110) ∃ T ∈ N + , ∀ t ≥ T, E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: E
Revised: E

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: = 1 (cid:111)
Revised: = 1 (cid:111)

Original: = 

Revised: =

Original: that is the complement of eq.
Revised: that is the complement of eq.

Original: We denote all the game history before the t th round as H t .
Revised: We denote all the game history before the t th round as H t .

Original: Then we only need to prove that for any H T , the conditional probability Pr (cid:110) E 1 , 2 t
Revised: Then we only need to prove that for any H T , the conditional probability Pr (cid:110) E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: This is because
Revised: This is because

Original: [Equation]
Revised: [Equation]

Original: where the number of summed terms is countable and we know that the sum of countable infinite zeros is still zero.
Revised: where the number of summed terms is countable and we know that the sum of countable infinite zeros is still zero.

Original: Moreover, if E 1 , 2 t = 1 , E 2 , 1 t +1 ̸ = 1 according to the definition, we know that (cid:110) E 1 , 2 t = 1 , E
Revised: Moreover, if E 1 , 2 t = 1 , E 2 , 1 t +1 ̸ = 1 according to the definition, we know that (cid:110) E 1 , 2 t = 1 , E

Original: [Equation]
Revised: [Equation]

Original: 1 t +1 = 1 (cid:111) and (cid:110) E 2 , 1 t = 1 , E 1 , 2 t +1 = 1 (cid:111) always equals to zero.
Revised: 1 t +1 = 1 (cid:111) and (cid:110) E 2 , 1 t = 1 , E 1 , 2 t +1 = 1 (cid:111) always equals to zero.

Original: Therefore, (cid:110) ∀ t ≥ T, E 1 , 2 t
Revised: Therefore, (cid:110) ∀ t ≥ T, E 1 , 2 t

Original: [Equation]
Revised: [Equation]

Original: 1 (cid:12)(cid:12)(cid:12) H T (cid:111) is equivalent to (cid:8)
Revised: 1 (cid:12)(cid:12)(cid:12) H T (cid:111) is equivalent to (cid:8)

Original: [Equation]
Revised: [Equation]

Original: Moreover, suppose the event expressed as eq.
Revised: Moreover, suppose the event expressed as eq.

Original: (15) happens, we can deduce that for t ≥ T , if
Revised: (15) happens, we can deduce that for t ≥ T , if

Original: E 1 , 2 t = 1 , E 2 , 1 t +1 = 0 because (cid:110) E 1 , 2 t = 1 , E 2 , 1 t +1 = 1 (cid:111) = 0 , so E 1 , 2 t +1 still happens; if E 2 , 1 t = 1 ,
Revised: E 1 , 2 t = 1 , E 2 , 1 t +1 = 0 because (cid:110) E 1 , 2 t = 1 , E 2 , 1 t +1 = 1 (cid:111) = 0 , so E 1 , 2 t +1 still happens; if E 2 , 1 t = 1 ,

Original: E 1 , 2 t +1 = 0 because (cid:110) E 2 , 1 t = 1 , E 1 , 2 t +1 = 1 (cid:111) = 0 , so E 2 , 1 t +1 still happens.
Revised: E 1 , 2 t +1 = 0 because (cid:110) E 2 , 1 t = 1 , E 1 , 2 t +1 = 1 (cid:111) = 0 , so E 2 , 1 t +1 still happens.

Original: Thus, event in eq.
Revised: Thus, event in eq.

Original: (15) leads to (cid:110) ∀ t ≥ T, E 1 , 2 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) ∨ (cid:110) ∀ t ≥ T, E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) .
Revised: (15) leads to (cid:110) ∀ t ≥ T, E 1 , 2 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) ∨ (cid:110) ∀ t ≥ T, E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) .

Original: Conversely, (cid:110) ∀ t ≥ T, E 1 , 2 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) ∨ (cid:110)
Revised: Conversely, (cid:110) ∀ t ≥ T, E 1 , 2 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) ∨ (cid:110)

Original: ∀ t ≥ T, E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) is obviously included in the event expressed as eq.
Revised: ∀ t ≥ T, E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T (cid:111) is obviously included in the event expressed as eq.

Original: Hence, event (cid:110) ∀ t ≥ T, E 1 , 2 t ∨ E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T
Revised: Hence, event (cid:110) ∀ t ≥ T, E 1 , 2 t ∨ E 2 , 1 t = 1 (cid:12)(cid:12)(cid:12) H T

Original: [Equation]
Revised: [Equation]

Original: is equivalent to
Revised: is equivalent to

Original: [Equation]
Revised: [Equation]

Original: Therefore, without loss of generality, we only need to prove that Pr (cid:110) ∀ t ≥ T,
Revised: Therefore, without loss of generality, we only need to prove that Pr (cid:110) ∀ t ≥ T,

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: 2 t
Revised: 2 t

Original: [Equation]
Revised: [Equation]

Original: =for any game history H T .
Revised: =for any game history H T .

Original: We show this by using Borel-Cantelli lemma (theorem A.6) and AzumaHoeffding inequality (theorem A.3) to find a sub-sequence of tasks (cid:110) t i :=
Revised: We show this by using Borel-Cantelli lemma (theorem A.6) and AzumaHoeffding inequality (theorem A.3) to find a sub-sequence of tasks (cid:110) t i :=

Original: T + i
Revised: T + i

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: for some T so that E 1 , 2 t i only happens finitely often.
Revised: for some T so that E 1 , 2 t i only happens finitely often.

Original: For simplicity, we denote ⌈ 1000 γ 1 − γ 2 ⌉ by ζ .
Revised: For simplicity, we denote ⌈ 1000 γ 1 − γ 2 ⌉ by ζ .

Original: Let τ be a stopping time such that τ = min t>T { R 1 ,t ≤ c 0 or S 2 ,t ≤ c 0 } and i τ = ⌈ τ − T ζ +1 ⌉ .
Revised: Let τ be a stopping time such that τ = min t>T { R 1 ,t ≤ c 0 or S 2 ,t ≤ c 0 } and i τ = ⌈ τ − T ζ +1 ⌉ .

Original: We design a series of new variables { D i } i =0 , 1 , 2 , ··· , where
Revised: We design a series of new variables { D i } i =0 , 1 , 2 , ··· , where

Original: [Equation]
Revised: [Equation]

Original: Now we show that { D i } i =0 , 1 , 2 , ··· is a super-martingale with bounded difference, and we will use
Revised: Now we show that { D i } i =0 , 1 , 2 , ··· is a super-martingale with bounded difference, and we will use

Original: Azuma-Hoeffding inequality (theorem A.3) to show the value of D i (and thus R 1 ,t ) cannot be too big.
Revised: Azuma-Hoeffding inequality (theorem A.3) to show the value of D i (and thus R 1 ,t ) cannot be too big.

Original: Therefore, E 1 , 2 t can only happen finitely many times by Borel-Cantelli lemma (theorem A.6).
Revised: Therefore, E 1 , 2 t can only happen finitely many times by Borel-Cantelli lemma (theorem A.6).

Original: Actually, if H T + i ( ζ +1) satisfies that
Revised: Actually, if H T + i ( ζ +1) satisfies that

Original: I
Revised: I

Original: [ T + i ( ζ + 1)
Revised: [ T + i ( ζ + 1)

Original: < τ ] = 1 , we have E 1 , 2 T
Revised: < τ ] = 1 , we have E 1 , 2 T

Original: + i ( ζ +1)
Revised: + i ( ζ +1)

Original: = 1 .
Revised: = 1 .

Original: According to lemma 4.6, we have
Revised: According to lemma 4.6, we have

Original: [Equation]
Revised: [Equation]

Original: On the other hand, if H T + i ( ζ +1) satisfies that
Revised: On the other hand, if H T + i ( ζ +1) satisfies that

Original: I
Revised: I

Original: [ T + i ( ζ + 1)
Revised: [ T + i ( ζ + 1)

Original: < τ ] = 0 , we know that D i +

Revised: < τ ] = 0 , we know that D i +

Original: = D i .
Revised: = D i .

Original: Moreover, when i ≤ i τ , we know that | D i − D
Revised: Moreover, when i ≤ i τ , we know that | D i − D

Original: i − 1 | is bounded by 

Revised: i − 1 | is bounded by

Original: + 2 ζ for any i ∈ N + because | R 1 ,t + j − R 1 ,t | is bounded by j by definition for any t, j ∈ N .
Revised: + 2 ζ for any i ∈ N + because | R 1 ,t + j − R 1 ,t | is bounded by j by definition for any t, j ∈ N .

Original: When i >
Revised: When i >

Original: i τ , we know that
Revised: i τ , we know that

Original: D i
Revised: D i

Original: = D i − 1 .
Revised: = D i − 1 .

Original: It is worthy to notice that E 1 , 2 T
Revised: It is worthy to notice that E 1 , 2 T

Original: + i ( ζ +1)
Revised: + i ( ζ +1)

Original: = 1 implies R 1 ,
Revised: = 1 implies R 1 ,

Original: T
Revised: T

Original: + i ( ζ +1) >
Revised: + i ( ζ +1) >

Original: c 0 .
Revised: c 0 .

Original: Therefore, if E 1 , 2 T + j = 1 for all j > T , τ does not exist, so we have D i > 2 c 0 + 100 i .
Revised: Therefore, if E 1 , 2 T + j = 1 for all j > T , τ does not exist, so we have D i > 2 c 0 + 100 i .

Original: However, according to Azuma–Hoeffding inequality (theorem A.3), for i > R 1 ,T − R 2 ,T − 2 c 0, we have
Revised: However, according to Azuma–Hoeffding inequality (theorem A.3), for i > R 1 ,T − R 2 ,T − 2 c 0, we have

Original: [Equation]
Revised: [Equation]

Original: This upper bound of Pr { D i > 2 c 0 + 100 i } decays exponentially in i .
Revised: This upper bound of Pr { D i > 2 c 0 + 100 i } decays exponentially in i .

Original: By the Borel-Cantelli lemma (theorem A.6), the event { R 1 ,T + i ( ζ +1) > c 0 } will occur only finitely often almost surely.
Revised: By the Borel-Cantelli lemma (theorem A.6), the event { R 1 ,T + i ( ζ +1) > c 0 } will occur only finitely often almost surely.

Original: This is contradictory to E 1 , 2 T + j = 1 for all j > T , so Pr {E 1 , 2 t =
Revised: This is contradictory to E 1 , 2 T + j = 1 for all j > T , so Pr {E 1 , 2 t =

Original: [Equation]
Revised: [Equation]

Original: |
Revised: |

Original: [Equation]
Revised: [Equation]

Original: } =
Revised: } =

Original: [Equation]
Revised: [Equation]

Original: F.2.5 Proof of Lemma 4. 
Revised: F.2.5 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: By symmetry, we only consider the case that H T with R 1 ,T + S 1 ,T ≥ 0 and R 1 ,T ≥ S 1 ,T .
Revised: By symmetry, we only consider the case that H T with R 1 ,T + S 1 ,T ≥ 0 and R 1 ,T ≥ S 1 ,T .

Original: If
Revised: If

Original: R 1 ,T ≤ c 0 , then we have S 1 ,T ≥ − R 1 ,T ≥ − c 0 ; if R 1 ,T > c 0 , because E 1 , 2 T ∨ E 2 , 1 T = 1 , S 2 ,T ≤ c 0 , so S 1 ,T = − S 2 ,T ≥ − c 0 .
Revised: R 1 ,T ≤ c 0 , then we have S 1 ,T ≥ − R 1 ,T ≥ − c 0 ; if R 1 ,T > c 0 , because E 1 , 2 T ∨ E 2 , 1 T = 1 , S 2 ,T ≤ c 0 , so S 1 ,T = − S 2 ,T ≥ − c 0 .

Original: Therefore, we have R 1 ,T ≥ 0 together with S 1 ,T ≥ − c 0 .
Revised: Therefore, we have R 1 ,T ≥ 0 together with S 1 ,T ≥ − c 0 .

Original: We now propose a process P T with less than T + 4( u + c 0 ) + 100 of rounds, and we will prove that it happens with probability no less than a constant λ u given any H T as we suppose.
Revised: We now propose a process P T with less than T + 4( u + c 0 ) + 100 of rounds, and we will prove that it happens with probability no less than a constant λ u given any H T as we suppose.

Original: The process P T is defined as
Revised: The process P T is defined as

Original: If S 1 ,T ≥ 0 , skip this
Revised: If S 1 ,T ≥ 0 , skip this

Original: phase
Revised: phase

Original: Otherwise
Revised: Otherwise

Original: , x j +

Revised: , x j +

Original: = 1 − ˆ y j , y j +

Revised: = 1 − ˆ y j , y j +

Original: = 1 − ˆ x j from j = T , Alice uses strategy opt 1 , Bob uses strategy opt 2 until some round T 1 such that S 1 ,T 1 ≥ 0 . 2.
Revised: = 1 − ˆ x j from j = T , Alice uses strategy opt 1 , Bob uses strategy opt 2 until some round T 1 such that S 1 ,T 1 ≥ 0 . 2.

Original: Alice and Bob uses strategy opt 1 for 4 u + 50 rounds and signals are generated as x j +1 = y
Revised: Alice and Bob uses strategy opt 1 for 4 u + 50 rounds and signals are generated as x j +1 = y

Original: j +

Revised: j +

Original: =
Revised: =

Original: − x j for j ≥ T 1 .
Revised: − x j for j ≥ T 1 .

Original: Firstly, we prove that P T will stop in T + 4( u + c 0 ) + 100 rounds and in the round T 2 exactly after
Revised: Firstly, we prove that P T will stop in T + 4( u + c 0 ) + 100 rounds and in the round T 2 exactly after

Original: P T , the game will enter good events such that E 1 , 1 ( u ) ∨ E 2 , 2 ( u ) = 1 .
Revised: P T , the game will enter good events such that E 1 , 1 ( u ) ∨ E 2 , 2 ( u ) = 1 .

Original: This part can be proved simply according to the definition of P T .
Revised: This part can be proved simply according to the definition of P T .

Original: During this process, we claim that R 1 ,i and S 1 ,i are monotone.
Revised: During this process, we claim that R 1 ,i and S 1 ,i are monotone.

Original: If i ∈ [ T + 1 , T 1 ] , we have
Revised: If i ∈ [ T + 1 , T 1 ] , we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, R 1 ,T 1 ≥ R 1 ,T ≥ 0 so after the first phase, both R 1 ,T 1 and S 1 ,T 1 are non-negative.
Revised: Therefore, R 1 ,T 1 ≥ R 1 ,T ≥ 0 so after the first phase, both R 1 ,T 1 and S 1 ,T 1 are non-negative.

Original: If i ∈ [ T 1 + 1 , T 2 ] where T 2 = T 1 + 4 u + 50 , we have
Revised: If i ∈ [ T 1 + 1 , T 2 ] where T 2 = T 1 + 4 u + 50 , we have

Original: [Equation]
Revised: [Equation]

Original: Now we prove that T 1 ≤ T + 4 c 0 + 50 .
Revised: Now we prove that T 1 ≤ T + 4 c 0 + 50 .

Original: To prove this, we only need to show that S 1 ,i +4 − S 1 ,i ≥if T ≤ i ≤ T 1 − 4 , so S 1 ,T +4 c 0 ≥ c 0 + S 1 ,T ≥ 0 if T + 4 c 0 ≤ T 1 − 4 and thus, T 1 ≤ T + 4 c 0 .
Revised: To prove this, we only need to show that S 1 ,i +4 − S 1 ,i ≥if T ≤ i ≤ T 1 − 4 , so S 1 ,T +4 c 0 ≥ c 0 + S 1 ,T ≥ 0 if T + 4 c 0 ≤ T 1 − 4 and thus, T 1 ≤ T + 4 c 0 .

Original: Actually, we have
Revised: Actually, we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: which is what we want to prove.
Revised: which is what we want to prove.

Original: An example of the first phase is shown as table 1.
Revised: An example of the first phase is shown as table 1.

Original: Next, we prove that after the second phase, E 1 , 1 T 2 ( u ) = 1 .
Revised: Next, we prove that after the second phase, E 1 , 1 T 2 ( u ) = 1 .

Original: Actually, we have for any i ∈
Revised: Actually, we have for any i ∈

Original: [ T 1 + 2 , T 2 ] ,
Revised: [ T 1 + 2 , T 2 ] ,

Original: [Equation]
Revised: [Equation]

Original: Therefore, R 1 ,T 2 ≥ R 1 ,T 1 +1 + 4 u + 49 ≥ R 1 ,T 1 + 4 u + 49 > u and S 1 ,T 2 ≥ S 1 ,T 1 +1 + 4 u + 49 ≥
Revised: Therefore, R 1 ,T 2 ≥ R 1 ,T 1 +1 + 4 u + 49 ≥ R 1 ,T 1 + 4 u + 49 > u and S 1 ,T 2 ≥ S 1 ,T 1 +1 + 4 u + 49 ≥

Original: S 1 ,T 1 + 4 u + 49 > u .
Revised: S 1 ,T 1 + 4 u + 49 > u .

Original: Therefore, E 1 , 1 T 2 ( u ) = 1 .
Revised: Therefore, E 1 , 1 T 2 ( u ) = 1 .

Original: An example of the first phase is shown as table 2.
Revised: An example of the first phase is shown as table 2.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: =
Revised: =

Original: [Table]
Revised: [Table]

Original: Until now, we have proved that P T can lead to E 1 , 1 T 2 ( u ) = 1 where T 2 ≤ T + 4( c 0 + u ) + 100 .
Revised: Until now, we have proved that P T can lead to E 1 , 1 T 2 ( u ) = 1 where T 2 ≤ T + 4( c 0 + u ) + 100 .

Original: Then we lower-bound the probability of P T happens given H T such that R 1 ,T ≥ 0 , S 1 ,T ≥ − c 0 .
Revised: Then we lower-bound the probability of P T happens given H T such that R 1 ,T ≥ 0 , S 1 ,T ≥ − c 0 .

Original: Roughly speaking, the probability of each round in P is lower-bounded by a constant and we know that P T has a limited number of rounds, which implies the entire probability of P T is bounded by a power of the constant probability lower-bounding a single round in P T .
Revised: Roughly speaking, the probability of each round in P is lower-bounded by a constant and we know that P T has a limited number of rounds, which implies the entire probability of P T is bounded by a power of the constant probability lower-bounding a single round in P T .

Original: For a round i in the first phase, because signals are i.i.d and R 1 ,i ≥ 0 , S 2 ,i ≥ 0 , the probability of { x i = ˆ y i − 1 } ∧ { y i
Revised: For a round i in the first phase, because signals are i.i.d and R 1 ,i ≥ 0 , S 2 ,i ≥ 0 , the probability of { x i = ˆ y i − 1 } ∧ { y i

Original: = ˆ x i − 1 } ∧ { Alice chooses opt 1 } ∧ { Bob chooses opt 2 } given a consistent history H i − 1 is no less than min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .
Revised: = ˆ x i − 1 } ∧ { Alice chooses opt 1 } ∧ { Bob chooses opt 2 } given a consistent history H i − 1 is no less than min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .

Original: For a round i in the second phase, because signals are i.i.d and R 1 ,i ≥ 0 , S 1 ,i ≥ 0 , the probability of { x i = − x i − 1 } ∧ { y i = − x i − 1 } ∧
Revised: For a round i in the second phase, because signals are i.i.d and R 1 ,i ≥ 0 , S 1 ,i ≥ 0 , the probability of { x i = − x i − 1 } ∧ { y i = − x i − 1 } ∧

Original: { Alice chooses opt 1 } ∧ { Bob chooses opt 1 } given a consistent history H i − 1 is also no less than min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .
Revised: { Alice chooses opt 1 } ∧ { Bob chooses opt 1 } given a consistent history H i − 1 is also no less than min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .

Original: Therefore, for a history H T such that R 1 ,T ≥ 0 , S 1 ,T ≥ − c 0 , we have
Revised: Therefore, for a history H T such that R 1 ,T ≥ 0 , S 1 ,T ≥ − c 0 , we have

Original: [Equation]
Revised: [Equation]

Original: Therefore, we can set λ u as (min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .
Revised: Therefore, we can set λ u as (min i,j ∈{ 0 , 1 } { P X,Y ( i, j ) } × 0 .

Original: 25 2 ) 100+

Revised: 25 2 ) 100+

Original: ( u + c 0 ) , which is what we want to prove.
Revised: ( u + c 0 ) , which is what we want to prove.

Original: F.2.6 Proof of Lemma 4. 
Revised: F.2.6 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: By the definition of c 0 , we know that when E 1 , 1 t ( ⌊ u 2 ⌋ ) ∨ E 2 , 2 t ( ⌊ u 2 ⌋ )
Revised: By the definition of c 0 , we know that when E 1 , 1 t ( ⌊ u 2 ⌋ ) ∨ E 2 , 2 t ( ⌊ u 2 ⌋ )

Original: = 1 happens, Alice and
Revised: = 1 happens, Alice and

Original: Bob will choose opt 1 both with probability larger than 1 − δ in the next ζ
Revised: Bob will choose opt 1 both with probability larger than 1 − δ in the next ζ

Original: + 1 rounds.
Revised: + 1 rounds.

Original: Using this, we want to prove that E [ (cid:80) ζ +1 j =1 ( r 1 ,t + j − r 2 ,t + j ) |H
Revised: Using this, we want to prove that E [ (cid:80) ζ +1 j =1 ( r 1 ,t + j − r 2 ,t + j ) |H

Original: t such that E 1 , 1 t ( ⌊ u 2 ⌋ )
Revised: t such that E 1 , 1 t ( ⌊ u 2 ⌋ )

Original: = 1] ≥ 100 .
Revised: = 1] ≥ 100 .

Original: Actually, we can deduce that
Revised: Actually, we can deduce that

Original: [Equation]
Revised: [Equation]

Original: >
Revised: >

Original: [Equation]
Revised: [Equation]

Original: Here, eq.
Revised: Here, eq.

Original: (16) is because when j ≥
Revised: (16) is because when j ≥

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: j − 1 ] − I [1 − x t + j = ˆ y t + j − 1 ] (cid:12)(cid:12)(cid:12) H t such that E 1 , 1 t (cid:16) ⌊ u 2 ⌋ (cid:17) = 1 (cid:105) ) ≥ − (1 − Pr { opt Yt + j = opt 1 } )
Revised: j − 1 ] − I [1 − x t + j = ˆ y t + j − 1 ] (cid:12)(cid:12)(cid:12) H t such that E 1 , 1 t (cid:16) ⌊ u 2 ⌋ (cid:17) = 1 (cid:105) ) ≥ − (1 − Pr { opt Yt + j = opt 1 } )

Original: + Pr { opt Yt + j = opt 1 } γ+ (1 − Pr { opt Yt + j − 1 = opt 1 } ) − Pr { opt Yt + j − 1 = opt 1 } γ> − δ + (1 − δ ) γ 1 − max { γ 2 , (1 − δ )
Revised: + Pr { opt Yt + j = opt 1 } γ+ (1 − Pr { opt Yt + j − 1 = opt 1 } ) − Pr { opt Yt + j − 1 = opt 1 } γ> − δ + (1 − δ ) γ 1 − max { γ 2 , (1 − δ )

Original: γ 2 − δ } .
Revised: γ 2 − δ } .

Original: Moreover, eq.
Revised: Moreover, eq.

Original: (17) holds according to definition of δ in lemma 4.5.
Revised: (17) holds according to definition of δ in lemma 4.5.

Original: F.2.7 Proof of Lemma 4. 
Revised: F.2.7 Proof of Lemma 4.

Original: Proof.
Revised: Proof.

Original: Without loss of generality, we suppose H T satisfies E 1 , 1 T ( u ) , and what we need is to prove that ∃ u ∈ N + such that
Revised: Without loss of generality, we suppose H T satisfies E 1 , 1 T ( u ) , and what we need is to prove that ∃ u ∈ N + such that

Original: [Equation]
Revised: [Equation]

Original: for any ε > 0 .
Revised: for any ε > 0 .

Original: We choose u to be larger than 2 c 0 + 1 at first.
Revised: We choose u to be larger than 2 c 0 + 1 at first.

Original: Then we let τ =
Revised: Then we let τ =

Original: min t
Revised: min t

Original: =
Revised: =

Original: T
Revised: T

Original: + i ( ζ +1) ,i ∈ N + { R 1 ,t ≤
Revised: + i ( ζ +1) ,i ∈ N + { R 1 ,t ≤

Original: ⌊ u 2 ⌋ or S 1 ,t ≤ ⌊ u 2 ⌋} and i τ = ⌈ τ − T ζ +1 ⌉ .
Revised: ⌊ u 2 ⌋ or S 1 ,t ≤ ⌊ u 2 ⌋} and i τ = ⌈ τ − T ζ +1 ⌉ .

Original: We construct a sequence of random variables { D i } i ∈ N such that
Revised: We construct a sequence of random variables { D i } i ∈ N such that

Original: [Equation]
Revised: [Equation]

Original: ( i − 1) ( ζ + 1) ≥ τ ] .
Revised: ( i − 1) ( ζ + 1) ≥ τ ] .

Original: Now we show that { D i } i =0 , 1 , 2 , ··· is a sub-martingale with bounded difference, and we will use
Revised: Now we show that { D i } i =0 , 1 , 2 , ··· is a sub-martingale with bounded difference, and we will use

Original: Azuma-Hoeffding inequality (theorem A.3) to show the value of D i (and thus R 1 ,t ) cannot be too small.
Revised: Azuma-Hoeffding inequality (theorem A.3) to show the value of D i (and thus R 1 ,t ) cannot be too small.

Original: This implies that { D j < u + 2 j − 100 j } happens with a probability upper-bounded by a function of u .
Revised: This implies that { D j < u + 2 j − 100 j } happens with a probability upper-bounded by a function of u .

Original: Therefore, the probability of {∀ i ∈ N , E 1 , 1 T
Revised: Therefore, the probability of {∀ i ∈ N , E 1 , 1 T

Original: + i ( ζ +1) (cid:0)
Revised: + i ( ζ +1) (cid:0)

Original: ⌊ u 2 ⌋
Revised: ⌊ u 2 ⌋

Original: + i (cid:1)
Revised: + i (cid:1)

Original: = 1 |H T } can be lower-bounded by a decreasing function of u , which tends towards 1 when u tends towards infinity.
Revised: = 1 |H T } can be lower-bounded by a decreasing function of u , which tends towards 1 when u tends towards infinity.

Original: Actually, we firstly verify that { D i } i ∈ N is a sub-martingale.
Revised: Actually, we firstly verify that { D i } i ∈ N is a sub-martingale.

Original: If H T +( i − 1)( ζ +1) satisfies that I [ T + ( i − 1) ( ζ + 1) <
Revised: If H T +( i − 1)( ζ +1) satisfies that I [ T + ( i − 1) ( ζ + 1) <

Original: τ ] = 1 , we have R 1 ,T +
Revised: τ ] = 1 , we have R 1 ,T +

Original: ( i − 1)( ζ +1) >
Revised: ( i − 1)( ζ +1) >

Original: ⌊ u 2 ⌋ and S 1 ,T +( i − 1)( ζ +1) >
Revised: ⌊ u 2 ⌋ and S 1 ,T +( i − 1)( ζ +1) >

Original: ⌊ u 2 ⌋ .
Revised: ⌊ u 2 ⌋ .

Original: According to lemma 4.9, we have
Revised: According to lemma 4.9, we have

Original: E
Revised: E

Original: [Equation]
Revised: [Equation]

Original: D i
Revised: D i

Original: [Equation]
Revised: [Equation]

Original: T +( i − 1)( ζ +1) with D i − 1 , · · ·
Revised: T +( i − 1)( ζ +1) with D i − 1 , · · ·

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation]
Revised: [Equation]

Original: If I [ T + ( i − 1) ( ζ + 1) < τ ] = 0 , we know that D i
Revised: If I [ T + ( i − 1) ( ζ + 1) < τ ] = 0 , we know that D i

Original: = D i − 1 .
Revised: = D i − 1 .

Original: When i ≤ i τ , we know that | D i − D
Revised: When i ≤ i τ , we know that | D i − D

Original: i − 1 | = | 2 R 1 ,T + i ( ζ +1)
Revised: i − 1 | = | 2 R 1 ,T + i ( ζ +1)

Original: − 2 R 1 ,T +( i − 1)( ζ +1) | + 100 ≤ 102 + ζ .
Revised: − 2 R 1 ,T +( i − 1)( ζ +1) | + 100 ≤ 102 + ζ .

Original: When i > i τ , | D i − D
Revised: When i > i τ , | D i − D

Original: i − 1 | = 0 .
Revised: i − 1 | = 0 .

Original: Now according to Azuma-Hoeffding inequality (theorem A.3), we have
Revised: Now according to Azuma-Hoeffding inequality (theorem A.3), we have

Original: Pr
Revised: Pr

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation]
Revised: [Equation]

Original: <
Revised: <

Original: u
Revised: u

Original: [Equation]
Revised: [Equation]

Original: j − 100 j }
Revised: j − 100 j }

Original: [Equation]
Revised: [Equation]

Original: Pr
Revised: Pr

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation]
Revised: [Equation]

Original: D
Revised: D

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: − (98 j + u )
Revised: − (98 j + u )

Original: [Equation]
Revised: [Equation]

Original: (102 +
Revised: (102 +

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: (cid:33) .
Revised: (cid:33) .

Original: If ∧ j − 1 i =0 E 1 , 1 T + i ( ζ +1) (cid:0) ⌊ u 2 ⌋
Revised: If ∧ j − 1 i =0 E 1 , 1 T + i ( ζ +1) (cid:0) ⌊ u 2 ⌋

Original: + i (cid:1) =
Revised: + i (cid:1) =

Original: 1 and R 1 ,T + j
Revised: 1 and R 1 ,T + j

Original: ( ζ +1) ≤ ⌊ u 2 + j ⌋ , we can deduce that i τ > j and hence,
Revised: ( ζ +1) ≤ ⌊ u 2 + j ⌋ , we can deduce that i τ > j and hence,

Original: D j < u + 2 j − 100 j .
Revised: D j < u + 2 j − 100 j .

Original: We have
Revised: We have

Original: Pr (cid:110) ∧
Revised: Pr (cid:110) ∧

Original: j − 1 i =

Revised: j − 1 i =

Original: E 1 , 1 T
Revised: E 1 , 1 T

Original: + i ( ζ +1) (cid:16)
Revised: + i ( ζ +1) (cid:16)

Original: ⌊ u 2 ⌋
Revised: ⌊ u 2 ⌋

Original: + i (cid:17)
Revised: + i (cid:17)

Original: [Equation]
Revised: [Equation]

Original: Symmetrically, we have
Revised: Symmetrically, we have

Original: [Equation]
Revised: [Equation]

Original: Therefore
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: )2 j ( ζ + 102)(cid:33) .
Revised: )2 j ( ζ + 102)(cid:33) .

Original: Furthermore
Revised: Furthermore

Original: [Equation]
Revised: [Equation]

Original: we can bound the probability of Pr {∀ i
Revised: we can bound the probability of Pr {∀ i

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: − 1 decays exponentially in u .
Revised: − 1 decays exponentially in u .

Original: Until now, we can find a sufficient large u such that Pr {∀ i ∈ N , E 1 , 1 T +( ζ +1) i (cid:0)
Revised: Until now, we can find a sufficient large u such that Pr {∀ i ∈ N , E 1 , 1 T +( ζ +1) i (cid:0)

Original: ⌊ u 2 ⌋
Revised: ⌊ u 2 ⌋

Original: + i (cid:1) ∨ E 2 , 2 T +( ζ +1)
Revised: + i (cid:1) ∨ E 2 , 2 T +( ζ +1)

Original: i (cid:0) ⌊ u 2 ⌋
Revised: i (cid:0) ⌊ u 2 ⌋

Original: + i (cid:1)
Revised: + i (cid:1)

Original: = 1 |H T } ≥ 1 − ε for H T with
Revised: = 1 |H T } ≥ 1 − ε for H T with

Original: E 1 , 1 T ( u ) ∧ E 2 , 2 T ( u ) = 1 .
Revised: E 1 , 1 T ( u ) ∧ E 2 , 2 T ( u ) = 1 .

Original: F.2.8 Proof of Truthful Convergence (Theorem 4.1)
Revised: F.2.8 Proof of Truthful Convergence (Theorem 4.1)

Original: Before the proof of our final theorem, we introduce a lemma showing that good events happen for infinite many times with probability 1 .
Revised: Before the proof of our final theorem, we introduce a lemma showing that good events happen for infinite many times with probability 1 .

Original: Combining lemma 4.7 in step 2 and lemma 4.8, we use the martingale theory to get the following lemma.
Revised: Combining lemma 4.7 in step 2 and lemma 4.8, we use the martingale theory to get the following lemma.

Original: Lemma F.1.
Revised: Lemma F.1.

Original: Given the game defined in theorem 4.1, for all u we have Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨
Revised: Given the game defined in theorem 4.1, for all u we have Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨

Original: E 2 , 2 t ( u ) = 1 } = 1 .
Revised: E 2 , 2 t ( u ) = 1 } = 1 .

Original: Proof.
Revised: Proof.

Original: Initially, we prove that in order to prove Pr { lim sup t →
Revised: Initially, we prove that in order to prove Pr { lim sup t →

Original: + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to prove
Revised: + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to prove

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Actually, in order to prove Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to prove that for any u ∈ N + , Pr {∃ t ∈ N + , E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .
Revised: Actually, in order to prove Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to prove that for any u ∈ N + , Pr {∃ t ∈ N + , E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .

Original: More specifically, if this claim holds, we can always find a t 1 ∈ N + such that E 1 , 1 t 1 ( u 1 ) ∨ E 2 , 2 t 1 ( u 1 ) = 

Revised: More specifically, if this claim holds, we can always find a t 1 ∈ N + such that E 1 , 1 t 1 ( u 1 ) ∨ E 2 , 2 t 1 ( u 1 ) =

Original: where u 1 > u , a t 2 > tsuch that E 1 , 1 t 2 ( u 2 ) ∨ E 2 , 2 t 2 ( u 2 ) = 

Revised: where u 1 > u , a t 2 > tsuch that E 1 , 1 t 2 ( u 2 ) ∨ E 2 , 2 t 2 ( u 2 ) =

Original: where u 2 > u 1 , a t 3 > t 2 such that E 1 , 1 t 3 ( u 3 ) ∨ E 2 , 2 t 3 ( u 3 ) =where u 3 > u 2 and so on.
Revised: where u 2 > u 1 , a t 3 > t 2 such that E 1 , 1 t 3 ( u 3 ) ∨ E 2 , 2 t 3 ( u 3 ) =where u 3 > u 2 and so on.

Original: Hence, we can find a sequence t 1 , t 2 , · · · such that E 1 , 1 t i ( u ) ∨ E 2 ,
Revised: Hence, we can find a sequence t 1 , t 2 , · · · such that E 1 , 1 t i ( u ) ∨ E 2 ,

Original: 2 t i ( u ) =for every i ∈ N + , which indicates that Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .
Revised: 2 t i ( u ) =for every i ∈ N + , which indicates that Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .

Original: In order to prove Pr {∃ t ∈ N + , E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to show that Pr {∀ t ∈ N + , E 1 , 1 t ( u ) ∨
Revised: In order to prove Pr {∃ t ∈ N + , E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 , we only need to show that Pr {∀ t ∈ N + , E 1 , 1 t ( u ) ∨

Original: E 2 , 2 t ( u ) = 0 } = 0 .
Revised: E 2 , 2 t ( u ) = 0 } = 0 .

Original: We know from lemma 4.7 that with probability 1 , there are infinitely many t ∈ N + such that
Revised: We know from lemma 4.7 that with probability 1 , there are infinitely many t ∈ N + such that

Original: E 1 , 2 t ∨ E 2 , 1 t = 1 .
Revised: E 1 , 2 t ∨ E 2 , 1 t = 1 .

Original: For any H t such that E 1 , 2 t ∨ E 2 , 1 t = 1 , we can find a {E 1 , 1 i ( u ) ∨ E 2 , 2 i ( u ) = 1 } in the next 100 + 4( c 0 + u ) rounds with probability no less than λ u according to Lemma 4.8.
Revised: For any H t such that E 1 , 2 t ∨ E 2 , 1 t = 1 , we can find a {E 1 , 1 i ( u ) ∨ E 2 , 2 i ( u ) = 1 } in the next 100 + 4( c 0 + u ) rounds with probability no less than λ u according to Lemma 4.8.

Original: Therefore, we can create a sub-martingale
Revised: Therefore, we can create a sub-martingale

Original: [Equation]
Revised: [Equation]

Original: ) where T j is the smallest number such that
Revised: ) where T j is the smallest number such that

Original: [Equation]
Revised: [Equation]

Original: Now we show that { D i } i =0 , 1 , 2 , ··· is indeed a sub-martingale with bounded difference, and we will use Azuma-Hoeffding inequality (theorem A.3) to show the value of D i cannot be too big.
Revised: Now we show that { D i } i =0 , 1 , 2 , ··· is indeed a sub-martingale with bounded difference, and we will use Azuma-Hoeffding inequality (theorem A.3) to show the value of D i cannot be too big.

Original: Therefore, D i ≤ k − λ u i for any k ∈ N + such that k < iλu can only happen finitely many times by Borel-Cantelli lemma (theorem A.6).
Revised: Therefore, D i ≤ k − λ u i for any k ∈ N + such that k < iλu can only happen finitely many times by Borel-Cantelli lemma (theorem A.6).

Original: This implies lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens with probability 1 .
Revised: This implies lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens with probability 1 .

Original: This is because
Revised: This is because

Original: [Equation]
Revised: [Equation]

Original: Moreover, we know that | D
Revised: Moreover, we know that | D

Original: i +1 − D i | ≤ 4 c 0 + 4 u + 101 for i ∈ N + , therefore, according to
Revised: i +1 − D i | ≤ 4 c 0 + 4 u + 101 for i ∈ N + , therefore, according to

Original: Azuma-Hoeffding inequality (theorem A.3), we have
Revised: Azuma-Hoeffding inequality (theorem A.3), we have

Original: [Equation]
Revised: [Equation]

Original: which decays exponentially in i .
Revised: which decays exponentially in i .

Original: This holds for any k ∈ N +
Revised: This holds for any k ∈ N +

Original: and i > kλ u .
Revised: and i > kλ u .

Original: Therefore, by BorelCantelli lemma (theorem A.6), { D i ≤ k − λ u i } will happen only finitely often almost surely.
Revised: Therefore, by BorelCantelli lemma (theorem A.6), { D i ≤ k − λ u i } will happen only finitely often almost surely.

Original: If { D i ≤ k − λ u i } does not happen, we can deduce that E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens for at least k times in the first T i + 4 c 0 + 4 u + 100 rounds.
Revised: If { D i ≤ k − λ u i } does not happen, we can deduce that E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens for at least k times in the first T i + 4 c 0 + 4 u + 100 rounds.

Original: Hence, E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens with probability 1 and consequentially, Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .
Revised: Hence, E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) happens with probability 1 and consequentially, Pr { lim sup t → + ∞ E 1 , 1 t ( u ) ∨ E 2 , 2 t ( u ) = 1 } = 1 .

Original: Now combining lemma F.1 and lemma 4.10, we complete our final proof (theorem 4.1).
Revised: Now combining lemma F.1 and lemma 4.10, we complete our final proof (theorem 4.1).

Original: Proof.
Revised: Proof.

Original: First note that if E Convergence Condition T
Revised: First note that if E Convergence Condition T

Original: [Equation]
Revised: [Equation]

Original: E 2 ,
Revised: E 2 ,

Original: [Equation]
Revised: [Equation]

Original: (cid:0) ⌊ u 2 ⌋
Revised: (cid:0) ⌊ u 2 ⌋

Original: + i (cid:1) = 1 } happens for some T , we can deduce that lim t → + ∞ ( R 1 ,t − R 2 ,t ) =
Revised: + i (cid:1) = 1 } happens for some T , we can deduce that lim t → + ∞ ( R 1 ,t − R 2 ,t ) =

Original: lim t → + ∞ ( S 1 ,t − S 2 ,t ) = ±∞ , and Pr { lim t → + ∞ opt Xt =
Revised: lim t → + ∞ ( S 1 ,t − S 2 ,t ) = ±∞ , and Pr { lim t → + ∞ opt Xt =

Original: lim t →
Revised: lim t →

Original: + ∞ opt Yt = opt 1 } = 1 or
Revised: + ∞ opt Yt = opt 1 } = 1 or

Original: Pr { lim t → + ∞ opt Xt = lim
Revised: Pr { lim t → + ∞ opt Xt = lim

Original: t →
Revised: t →

Original: + ∞ opt Yt = opt 2 } = 1 .
Revised: + ∞ opt Yt = opt 2 } = 1 .

Original: Therefore, Pr {∨ + ∞ T =1 E Convergence Condition T } =implies theorem 4.1.
Revised: Therefore, Pr {∨ + ∞ T =1 E Convergence Condition T } =implies theorem 4.1.

Original: Otherwise
Revised: Otherwise

Original: [Equation]
Revised: [Equation]

Original: suppose Pr {∨ + ∞ T =1 E Convergence Condition T = 1 } = 1 − ε 2 < 1 .
Revised: suppose Pr {∨ + ∞ T =1 E Convergence Condition T = 1 } = 1 − ε 2 < 1 .

Original: Let ε 

Revised: Let ε

Original: = 12 ε 2 and u 1 satisfy lemma 4.
Revised: = 12 ε 2 and u 1 satisfy lemma 4.

Original: [Equation]
Revised: [Equation]

Original: then we can deduce that
Revised: then we can deduce that

Original: [Equation]
Revised: [Equation]

Original: F.1) ≥ 1 − ε
Revised: F.1) ≥ 1 − ε

Original: [Equation]
Revised: [Equation]

Original: lemma 4.10) which is a contradiction.
Revised: lemma 4.10) which is a contradiction.

Original: G Error Bars of Simulation in Section
Revised: G Error Bars of Simulation in Section

Original: We run our simulations single-threadedly on AMD Ryzen™ R7-5700U Processor at 3.60GHz with 16GB DDR4 SDRAM.
Revised: We run our simulations single-threadedly on AMD Ryzen™ R7-5700U Processor at 3.60GHz with 16GB DDR4 SDRAM.

Original: The total running time of all the simulations in fig.
Revised: The total running time of all the simulations in fig.

Original: 2 is 421 seconds.
Revised: 2 is 421 seconds.

Original: Code is available in https://github.com/fengtony686/peer-prediction-convergence .
Revised: Code is available in https://github.com/fengtony686/peer-prediction-convergence .

Original: We give error bars of converge rates of each algorithm shown in fig.
Revised: We give error bars of converge rates of each algorithm shown in fig.

Original: 2 one by one.
Revised: 2 one by one.

Original: Each error bar is drawn by computing converge proportion of each algorithm in 400 repeating simulations for 10 times.
Revised: Each error bar is drawn by computing converge proportion of each algorithm in 400 repeating simulations for 10 times.