Original: Distributed Online Convex Optimization with Compressed Communication
Revised: Distributed Online Convex Optimization with Compressed Communication

Original: Anonymous Author(s)

Original: AfﬁliationAddress email

Original: Abstract
Revised: Abstract

Original: We consider a distributed online convex optimization problem when streamingdata are distributed among computing agents over a connected communicationnetwork.
Revised: We consider a distributed online convex optimization problem when streaming data are distributed among computing agents over a connected communication network.

Original: Since the data are high-dimensional or the network is large-scale, communication load can be a bottleneck for the efﬁciency of distributed algorithms.
Revised: Since the data are high-dimensional or the network is large-scale, communication load can be a bottleneck for the efficiency of distributed algorithms.

Original: To tackle this bottleneck, we apply the state-of-art data compression scheme tothe fundamental GD-based distributed online algorithms.
Revised: To tackle this bottleneck, we apply the state-of-art data compression scheme to the fundamental GD-based distributed online algorithms.

Original: Three algorithms withdifference-compressed communication are proposed for full information feedback(DC-DOGD), one-point bandit feedback (DC-DOBD), and two-point bandit feedback (DC-DO2BD), respectively.
Revised: Three algorithms with difference-compressed communication are proposed for full information feedback (DC-DOGD), one-point bandit feedback (DC-DOBD), and two-point bandit feedback (DC-DO2BD), respectively.

Original: We obtain regret bounds explicitly in termsof the time horizon, compression ratio, decision dimension, agent number, andnetwork parameters.
Revised: We obtain regret bounds explicitly in terms of time horizon, compression ratio, decision dimension, agent number, and network parameters.

Original: Our algorithms are proved to be no-regret and match the sameregret bounds, w.r.t.
Revised: Our algorithms are proved to be no-regret and match the same regret bounds, w.r.t.

Original: the time horizon, with their uncompressed versions for bothconvex and strongly convex losses.
Revised: time horizon, with their uncompressed versions for both convex and strongly convex losses.

Original: Numerical experiments are given to validatethe theoretical ﬁndings and illustrate that the proposed algorithms can effectivelyreduce the total transmitted bits for distributed online training compared with theuncompressed baseline.
Revised: Numerical experiments are given to validate the theoretical findings and illustrate that the proposed algorithms can effectively reduce the total transmitted bits for distributed online training compared with the uncompressed baseline.

Original:  Introduction
Revised:  Introduction

Original: Online optimization has attracted considerable attention in recent decades, for its remarkable applications in machine learning tasks such as spam ﬁltering, dictionary learning, ad. selection andso on [1, 2, 3].
Revised: Online optimization has attracted considerable attention in recent decades, for its remarkable applications in machine learning tasks such as spam filtering, dictionary learning, advertising selection and so on [1, 2, 3].

Original: In such online tasks, data are revealed incrementally, and decisions must be madebefore all data are available.
Revised: In such online tasks, data are revealed incrementally and decisions must be made before all data are available.

Original: When the streaming data are collected at multiple agents, the distributedonline optimization over a multi-agent network is considered, where data storage and processing areperformed in the agents [4, 5].
Revised: When the streaming data are collected at multiple agents, the distributed online optimization over a multi-agent network is considered, where data storage and processing are performed in the agents [4, 5].

Original: It is often impractical to communicate data among different agentsfrom multiple concerns such as privacy and bandwidth utilization.
Revised: It is often impractical to communicate data among different agents from multiple concerns such as privacy and bandwidth utilization.

Original: Also, there is no center agent forglobal coordination.
Revised: Also, there is no central agent for global coordination.

Original: In such settings, each agent relies on its own data to run an algorithm whilecommunicating decisions with its immediate neighbors.
Revised: In such settings, each agent relies on its own data to run an algorithm while communicating decisions with its immediate neighbors.

Original: To be speciﬁc, this paper considers the distributed online convex optimization (DOCO) problem over an N -agent network.
Revised: To be specific, this paper considers the distributed online convex optimization (DOCO) problem over an N -agent network.

Original: The objective is to minimize the accumulated system-wide lossmin x ∈K (cid:80)
Revised: The objective is to minimize the accumulated system-wide loss min x ∈K (cid:80)

Original: Tt =1 (cid:80)
Revised: Tt =1 (cid:80)

Original: Ni =
Revised: Ni =

Original: 1 f ti ( x ) , where the local convex loss function f ti is formed by the data arrivingat time t in agent i , and K ⊂ R d is a convex feasible set.
Revised: 1 f ti ( x ) , where the local convex loss function f ti is formed by the data arriving at time t in agent i , and K ⊂ R d is a convex feasible set.

Original: Note that the loss information is revealed toagent i after its decision x ti is made.
Revised: Note that the loss information is revealed to agent i after its decision x ti is made.

Original: Generally, there are two basic types of information feedbackthat agents can possess.
Revised: Generally, there are two basic types of information feedback that agents can possess.

Original: One is the full information feedback, where agents have access to the lossfunctions.
Revised: One is the full information feedback, where agents have access to the loss functions.

Original: The other is the bandit feedback, where agents can only possess the values of the lossfunction at points around the decision.
Revised: The other is the bandit feedback, where agents can only possess the values of the loss function at points around the decision.

Original: At each time step, agents choose the decisions based on theirlocal feedback and neighbors’ information.
Revised: At each time step, agents choose the decisions based on their local feedback and neighbors’ information.

Original: To measure the performance of an algorithm, the ( regret is frequently used, which compares the cumulative loss of online decisions and the loss of the best decision chosen in hindsight through all the time horizons.
Revised: To measure the performance of an algorithm, the (static) regret is frequently used, which compares the cumulative loss of online decisions and the loss of the best decision chosen in hindsight through all the time horizons.

Original: The regret of node j ∈ V is deﬁnedas R( j, T ) =
Revised: The regret of node j ∈ V is defined as

Original: (cid:80) Tt =1 (cid:80) Ni =1 f ti ( x tj ) − min x ∈K (cid:80) Tt =1 (cid:80) Ni =
Revised: [Equation]

Original: 1 f ti ( x ) . An algorithm is called no-regret[6] if the average regret over T goes to zero as T is large, which means the online decision updatedby the streaming data is not far from the best decision chosen in hindsight.
Revised: An algorithm is called no-regret [6] if the average regret over T goes to zero, which means the online decision updated by the streaming data is not far from the best decision chosen in hindsight.

Original: Distributed no-regretonline algorithms have been widely studied in recent years [7, 8, 9].
Revised: Distributed no-regret online algorithms have been widely studied in recent years [4, 7, 8, 9, 10, 11].

Original: Although distributed algorithms are theoretically feasible, most of them are not practical as the modelsize gets large, since communication cost can be a bottleneck for efﬁciency.
Revised: Although distributed algorithms are theoretically feasible, most of them are not practical as the model size gets large, since communication cost can be a bottleneck for efficiency.

Original: In distributed trainingtasks, agents can be powful microcomputers, while their communication network may be with lowbandwidth.
Revised: In distributed training tasks, agents can be powful microcomputers, while their communication network may be with low bandwidth.

Original: The information exchange over the network is pretty slow compared with the computationtaking place in agents [10].
Revised: The information exchange over the network is pretty slow compared with the computation taking place in agents [12].

Original: Thus, communication compression techniques are of signiﬁcance forpractical implementations.
Revised: Thus, communication compression techniques are of significance for practical implementations.

Original: There have been many attempts to combinecompressors with distributed optimization algorithms.
Revised: There have been many attempts to combine distributed optimization algorithms with compressors.

Original: A straightforward idea is the direct compression scheme, while algorithms with this simple schemefail to converge even for the distributed average consensus problem [11, 12].
Revised: A straightforward idea is the direct compression scheme, while algorithms with this simple scheme fail to converge even for the distributed average consensus problem [13, 14].

Original: As an improvement,extrapolation compression scheme and difference compression scheme are proposed [13].
Revised: As an improvement, extrapolation compression scheme and difference compression scheme are proposed [15].

Original: Alongthis line, quite a number of studies successfully extend the distributed optimization algorithms withcompressors and meanwhile maintain the convergence rate [14, 15, 16].
Revised: Along this line, quite a number of studies successfully extend the distributed optimization algorithms with compressors and meanwhile maintain the convergence rate [16, 17, 18, 19].

Original: However, distributed onlineoptimization with compression is still an area that has not been fully exploited.
Revised: However, distributed online optimization with compression is still an area that has not been fully exploited.

Original: [17] proposed ECDAMSGrad algorithm that extended the AMSGrad to the distributed online setting with extrapolationcompression, while only empirical results were given without theoretical analysis.
Revised: [20] proposed ECD-AMSGrad algorithm that extended the AMSGrad to the distributed online setting with extrapolation compression, while only empirical results were given without theoretical analysis.

Original: The key openproblem in this area iswhether it is possible to design provably no-regret distributed online algorithmsthat work with compressors.
Revised: The key open problem in this area is whether it is possible to design provably no-regret distributed online algorithms that work with compressors.

Original: Contributions In this work, we answer the above question in the afﬁrmative.
Revised: Contributions In this work, we answer the above question in the affirmative.

Original: We apply thedifference compression scheme to the fundamental GD-based distributed online algorithms.
Revised: We apply the difference compression scheme to the fundamental GD-based distributed online algorithms.

Original: Althoughthe idea of such combination is simple, the underlying algorithm design and theoretical principle arechallenging since the compression error, projection error, and consensus error will be coupled.
Revised: Although the idea of such combination is simple, the underlying algorithm design and theoretical principle are challenging since the compression error, projection error, and consensus error will be coupled.

Original: Ourcontributions are summarized as follows:
Revised: Our contributions are summarized as follows:

Original: • We propose communication-efﬁcient distributed online algorithms, which consist of differencecompression, γ -gossip consensus, gradient descent, and projection, for the cases of full information feedback (DC-DOGD), one-point bandit feedback (DC-DOBD), and two-point banditfeedback (DC-DO2BD), respectively.
Revised: • We propose communication-efficient distributed online algorithms, which consist of difference compression, γ -gossip consensus, gradient descent, and projection, for the cases of full information feedback (DC-DOGD), one-point bandit feedback (DC-DOBD), and two-point bandit feedback (DC-DO2BD), respectively.

Original: We make the technical advance to combine the difference compression scheme with the projection scheme.
Revised: We make the technical advance to combine the difference compression scheme with the projection scheme.

Original: Through proper design, the errors can be estimated and controlled with the consensus stepsize γ and the gradient descent stepsizes.
Revised: Through proper design, the errors can be estimated and controlled with the consensus stepsize γ and the gradient descent stepsizes.

Original: • We analyze the regret bounds of the proposed algorithms for convex and strongly convex losses, respectively, which are established explicitly in terms ofthe time horizon T , compression ratio ω , decision dimension d , agent number N , andthe parameters of the communication graph G , as simpliﬁed and summarized in Table 1.
Revised: • We analyze the regret bounds of the proposed algorithms for convex and strongly convex losses, respectively, which are established explicitly in terms of time horizon T , compression ratio ω , decision dimension d , agent number N , and parameters of the communication graph G , as simplified and summarized in Table 1.

Original: The obtained regret bounds are in accordance with that of[18] w.r.t T, N, d .
Revised: The obtained regret bounds are in accordance with those of [11] w.r.t T, N, d .

Original: To the best of our knowledge, the proposed algorithms are the ﬁrst distributedonline algorithms with theoretical no-regret guarantees for ω − contracted compressors.
Revised: Our algorithms are no-regret with theoretical guarantees.

Original: • We give exhaustive experiments to illustrate the performance of the proposed algorithms.
Revised: • We give exhaustive experiments to illustrate the performance of the proposed algorithms.

Original: Compared with the uncompressed algorithm DAOL [7], the proposed algorithms can reduce the total transmitted bits for distributed online training.
Revised: Compared with the uncompressed algorithm DAOL [8], the proposed algorithms can reduce the total transmitted bits for distributed online training.

Original: Moreover, DC-DOGD and DC-DO2BD signiﬁcantly outperform the algorithm ECD-AMSGrad [17].
Revised: Moreover, DC-DOGD and DC-DO2BD significantly outperform the algorithm ECD-AMSGrad [20].

Original: [Table]
Revised: [Table]

Original: Related Work Distributed online convex optimization has received numerous attention in recentyears.
Revised: Related Work Distributed online convex optimization has received numerous attention in recent years.

Original: Many basic algorithms have been extended to distributed settings.
Revised: Many basic algorithms have been extended to distributed settings, referring to [7] and references therein.

Original: For example, [7] proposeda distributed online subgradient algorithm over a staticconnected directed network and achievedthe regrets O ( √ T ) and O (ln( T )) for convex and strongly convex losses, which is in line with theregrets ofclassical centralized online algorithms [19, 20]. Then [8] studied DOCO with long-termconstraints over a time-varying network and achievedthe regrets O ( T 3 / 4 ) and O ( T 2 / 3 ln 1 / 3 ( T )) forconvex and strongly convex losses in the one-point bandit feedback.
Revised: For instance, [8] proposed a distributed online subgradient algorithm over a static directed network and achieved the regrets O ( √ T ) and O (ln( T )) for convex and strongly convex losses, which are in line with those of centralized online algorithms [21, 22]. In the bandit feedback setting, by modifying the gradient to a randomized estimator, [9] proposed a distributed online bandit algorithm with one-point sampling over an undirected network and achieved O ( T 2 / 3 ln 1 / 3 ( T )) regret for strongly convex losses. Better bound could be achieved by two-point sampling, as [4] obtained

Original: As for  two-point bandit feedback,the regrets O ( √ T ) and O (ln( T )) were established in [9] for convex and strongly convex losses,which are the same as the centralized two-bandit algorithms [21].
Revised: O ( √ T ) regret for convex quadratic losses in case of bounded decision set. Also, [10] investigated the distributed online two-point bandit algorithm in dynamic environments and achieved O ( √ T ) and O (ln( T )) regrets for convex and strongly convex losses, which match those of centralized two-bandit algorithms [23].

Original: [18] comprehensively studied DOCO over Erd˝os-Rényi random networks in full gradient feedback, one-point bandit feedback, andtwo-point bandit feedback, and gave regret bounds.
Revised: [11] comprehensively studied DOCO over Erd˝os-Rényi random networks in full gradient feedback, one-point bandit feedback, and two-point bandit feedback, and gave regret bounds.

Original: Along the line of [18], this paper aims to furtherintroduce compressed communication strategies, while preserving the regret bounds.
Revised: Along the line of [11], this paper aims to further introduce compressed communication strategies, while preserving the regret bounds order-wisely.

Original: Recently, combining distributed optimization algorithms with compressors has seen a dramaticrise in interest.
Revised: Recently, combining distributed optimization algorithms with compressors has seen a dramatic rise in interest.

Original: Traditional compressors include the quantization  and sparsiﬁcation. Quantizationis to reduce the precision of each element, such as 1-bit SGD [22], SignSGD [23], GSGD [24],and so on. Sparsiﬁcation is to transmit only a few elements of the vectors, for instance, Top k[25], Rand k [26] and Threshold v [27].
Revised: Traditional compressors include the quantization [24, 25, 26], sparsification [27, 28, 29], and hybrid combining of them [30, 31, 32].

Original: Also there are hybrid compressors combining quantizationwith sparsiﬁcation, to name a few, SketchML [28], 3LC [29], Qsparse-local-SGD [30], etc.

Original: Theway to apply compressors is called a compression scheme.
Revised: The way to apply compressors is called a compression scheme.

Original: The most widely used compressionschemes in distributed optimization are extrapolation compression and difference compression [13].
Revised: The most widely used compression schemes in distributed optimization are extrapolation compression and difference compression [15].

Original: Extrapolation compression allows agents to compress the extrapolation between the last two localstates.
Revised: Extrapolation compression allows agents to compress the extrapolation between the last two local states.

Original: Decentralized PSGD with extrapolation compression (ECD-PSGD)
Revised: Decentralized PSGD with extrapolation compression (ECD-PSGD)

Original: [13] was proved toconverge sublinearly and match the rate of its uncompressed case (D-PSGD).
Revised: [15] was proved to converge sublinearly and match the rate of its uncompressed case (D-PSGD).

Original: Difference compression,which is also called CHOCO [14] or innovation compression [31], allows agents to add replicas ofneighboring states and compress the state-difference.
Revised: Difference compression (DC), which is also called CHOCO [16] or innovation compression [19], allows agents to add replicas of neighboring states and compress the state-difference.

Original: There have been extensive successful designscombining distribute optimization algorithms with difference compression, to name a few, DCDPSGD [13] (based on PSGD), CHOCO-SGD [14] (base on gossip SGD), SPARQ-SGD [15] (basedon event-trigger), C-GT [16] (base on gradient tracking), and COLD[31] (based on NIDS), etc. 
Revised: There have been extensive successful designs combining distribute optimization algorithms with DC, to name a few, DCD-PSGD [15] (based on PSGD), CHOCO-SGD [16] (base on gossip SGD), SPARQ-SGD [17] (based on event-trigger), C-GT [18] (base on gradient tracking), and COLD[19] (based on NIDS), etc. The difference compression scheme is different from the error-feedback scheme (EF) [24, 33] in what to be compressed, which results in different application fields.

Revised: DC compresses the difference between the current variable and the replica variable, which is widely used in distributed optimization where nodes tend to exchange state information whose limit is nonzero in general.

Revised: EF compresses the sum of the gradient and the residual error, which is widely used in federated learning where nodes tend to exchange gradient information whose limit is expected to be zero.

Revised: The insight is that successful designs have to compress something that goes to zero, otherwise the noise introduced by the compression will not vanish, which leads to the algorithm oscillation or even divergence.

Original: Also,the idea of difference compression have been widely adopted in federated learning [32, 33, 34].
Revised: It is worth noting that the idea of DC can also be applied to federated learning, such as EF21 [34] and

Original: Inthis work, we use difference compression scheme to design communication-efﬁcient distributedonline algorithms.
Revised: FedPAQ [35], which is another line of research. In this paper, we focus on applying DC to distributed online optimization.

Original: The results about the distributed online optimization with compression are quite limited.
Revised: The results about the distributed online optimization with compression are quite limited.

Original: [17]poposed the ECD-AMSGrad algorithm which combined AMSGrad with extrapolation compression.
Revised: [20] poposed the ECD-AMSGrad algorithm that combined AMSGrad with extrapolation compression.

Original: Actually, the AMSGrad algorithm may not be a good choice for DOCO algorithm design sincealthough AMSGrad itself is proved no-regret [35], a considerable performance gap still existsbetween AMSGrad and SGD [36].
Revised: Actually, the AMSGrad algorithm may not be a good choice for DOCO problem since although AMSGrad itself is proved no-regret [36], a considerable performance gap still exists between AMSGrad and SGD [37].

Original: Besides, the introduction of compression errors will furtherworsen the algorithm such that ECD-AMSGrad will lose the no-regret performance (seen Section 5).
Revised: Besides, the introduction of compression errors will further worsen the algorithm such that ECD-AMSGrad will lose the no-regret performance (see Section 5).

Original: In this paper, we focus on the fundamental GD-based algorithms and  give no-regret guarantees.
Revised: This paper focuses on the fundamental GD-based algorithms and will give no-regret guarantees.

Original:  Full Information Feedback
Revised:  Full Information Feedback

Original: In this section, we ﬁrst introduce the multi-agent network and the compressor we use, and thenpropose a communication-efﬁcient distributed online algorithm forthe DOCO with full informationfeedback.
Revised: In this section, we first introduce the multi-agent network and the compressor we use, and then propose a communication-efficient distributed online algorithm for DOCO with full information feedback.

Original: Expected regret bounds will be given for both convex and strongly convex losses.
Revised: Expected regret bounds will be given for both convex and strongly convex losses.

Original: Graph The multi-agent network is described by an undirected graph G ( V , E ) , where V ={ 1 , . . .
Revised: Graph The multi-agent network is described by an undirected graph G ( V , E ) , where V = { 1 , . . .

Original: , N } is the set of nodes, representing the set of agents, and E ⊂ V × V is the set of edges.
Revised: , N } is the set of nodes, representing the set of agents, and E ⊂ V × V is the set of edges.

Original: Let
Revised: Let

Original: A =
Revised: A =

Original: [ a ij ] ∈ R N × N be the connectivity matrix of G such that a ij = a ji .
Revised: [ a ij ] ∈ R N × N be the connectivity matrix of G such that a ij = a ji .

Original: If ( v i , v j ) ∈ E , then v i andv j can exchange information, and a ij = 0 otherwise.
Revised: If ( v i , v j ) ∈ E , then v i and v j can exchange information, and a ij = 0 otherwise.

Original: It is worth noting that the communication isnode-to-node in our distributed setting, and there is no central node.
Revised: It is worth noting that the communication is node-to-node in our distributed setting, and there is no central node.

Original: The graph in this paper satisﬁesthe following assumption.
Revised: The graph in this paper satisfies the following assumption.

Original: Assumption 1.
Revised: Assumption 1.

Original: The communication graph G is undirected and connected.
Revised: The communication graph G is undirected and connected.

Original: Its connectivity matrix
Revised: Its connectivity matrix

Original: A ∈ [0 , 1] N × N is a symmetric doubly stochastic matrix.
Revised: A ∈ [0 , 1] N × N is a symmetric doubly stochastic matrix.

Original: Compressor A compressor Q ( · ) :
Revised: Compressor A compressor Q ( · ) :

Original: R d → R d is a mapping whose output can be usually encodedwith fewer bits than its input.
Revised: R d → R d is a mapping whose output can be usually encoded with fewer bits than its input.

Original: In this paper, we consider a broad class of compressors with the following general property, which has been widely considered in distributed optimization withcompression [14, 15, 31].
Revised: In this paper, we consider a broad class of compressors with the following general property, which has been widely considered in distributed optimization with compression [16, 17, 19].

Original: Assumption 2.
Revised: Assumption 2.

Original: For some ω ∈ (0 , 1] , Q satisﬁes E Q (cid:107) Q ( x ) − x (cid:107) 2 ≤ (1 − ω ) (cid:107) x (cid:107) 2 , ∀ x ∈ R d , where 135 E
Revised: For some ω ∈ (0 , 1] , Q satisfies [Equation]

Original:  Q denotes the expectation over the internal randomness of Q .
Revised: where E Q denotes the expectation over the internal randomness of Q .

Original: Compressors satisfy the above assumption are called ω -contracted, which include many importantcompressors, such as Sparsiﬁcation Rand k and Top k [26], Random quantization  QSGD s [24],
Revised: Compressors satisfy the above assumption are called ω -contracted, which include many important biased or unbiased compressors, such as sparsification ( Rand k and Top k ) [28], random quantization ( QSGD s ) [26], random gossip ( RGossip p )

Original: Randomized gossip [14], etc.
Revised: [16], etc.

Original: 2.1 Algorithm design
Revised: 2.1 Algorithm design

Original: In the full information feedback, the loss function f ti is revealed to node i at time t after the decisionx ti is made.
Revised: In the full information feedback, the loss function f ti is revealed to node i at time t after the decision x ti is made.

Original: Then node i has access to the gradient value ∇ f ti ( x ti ) and can use g ti = ∇ f ti ( x ti ) tomake
Revised: Then node i has access to the gradient value ∇ f ti ( x ti ) and can use g ti = ∇ f ti ( x ti ) to make

Original: the next decision x t +1 i .
Revised: the next decision x t +1 i .

Original: We propose the DC-DOGD algorithm as shown in Algorithm 1, whichis based on DAOL [7] and Memory-efﬁcient CHOCO-GOSSIP [14].
Revised: We propose the DC-DOGD algorithm as shown in Algorithm 1, which is based on DAOL [8] and memory-efficient CHOCO-SGD [16].

Original: The DC-DOGD algorithmconsists of two main parts: difference compressed communication (steps 2 and 3) and local decisionupdate (steps 4 and 5). 
Revised: The DC-DOGD algorithm consists of two main parts: difference compressed communication (steps 2 and 3) and local decision update (steps 4 and 5). Algorithm

Original: Input: consensus = , time T
Revised: Input: consensus stepsize γ , GD stepsizes { η t } Tt =1 , time T

Original: Initialize
Revised: Initialize: set x 1 i = 0 , ˆ x 1 i = 0 , s 1 i = 0 , for each node i ∈ V .

Original:  T  do  V 2:
Revised: 1: for t = 1 to T − 1 do in parallel for each node i ∈ V 2:

Original: Compress the difference vector q ti = Q
Revised: Compress the difference vector q ti = Q

Original: ( x ti − ˆ x ti ) and update the replica
Revised: ( x ti − ˆ x ti ) and update the replica

Original: ˆ x t +1 i
Revised: ˆ x t +1 i

Original: =
Revised: =

Original: ˆ x ti + q ti .
Revised: ˆ x ti + q ti .

Original: Send q ti to its neighbors and receive q tj from all its neighbors j ∈ N i .
Revised: Send q ti to its neighbors and receive q tj from all its neighbors j ∈ N i .

Original: Update the estimate of the consensus decision by s t
Revised: Update the estimate of the consensus decision by s t

Original: +1 i
Revised: +1 i

Original: = s

Original: ti + (cid:80)

Original: [Equation]
Revised: [Equation]

Original: ∈N i a ij q tj .

Original: Receive the full information feedback and calculate g ti = ∇ f ti ( x ti ) .
Revised: Receive the full information feedback and calculate g ti = ∇ f ti ( x ti ) .

Original: Update its decision variable as follows
Revised: Update its decision variable as follows

Original: [Equation]
Revised: [Equation]

Original: where P K denotes the Euclidean projection, i.e., P K ( x ) = argmin y ∈K (cid:107) x − y (cid:107) .
Revised: where P K denotes the Euclidean projection, i.e., P K ( x ) = argmin y ∈K ∥ x − y ∥ .

Original: Output: { x [Equation]
Revised: Output: { x t i } T t =

Original: The insights of introducing the variables ˆ x ti andusing the difference compression are as below.
Revised: The insight of introducing the variable ˆ x ti and the difference compression are as below.

Original: Assume that x ∗ (cid:54) = 0 without loss of generality.
Revised: Assume that x ∗ ̸ = 0 without loss of generality.

Original: Then, if node i transmits the directly compressedinformation Q ( x ti ) to its neighbors, the compression error Q ( x ti ) − x ti will not vanish for t → ∞ .
Revised: Then, if node i transmits the directly compressed information Q ( x ti ) to its neighbors, the compression error Q ( x ti ) − x ti will not vanish for t → ∞ .

Original: The accumulation of compression errors makes the algorithm fail to converge.
Revised: The accumulation of compression errors makes the algorithm fail to converge.

Original: Instead, we compresssomething that goes to zero.
Revised: Instead, we compress something that goes to zero.

Original: Let node i and all its neighbors keep an auxiliary variable ˆ x ti locally,which acts as a replica of x ti .
Revised: Let node i and all its neighbors keep an auxiliary variable ˆ x ti locally, which acts as a replica of x ti .

Original: Whenever node i updates its decision variable x ti , node i calculates thedifference x ti − ˆ x ti , compresses the difference q ti = Q ( x ti − ˆ x ti ) , and sends the compressed informationq t i to its neighbors.
Revised: Whenever node i updates its decision variable x ti , node i calculates the difference x ti − ˆ x ti , compresses the difference q ti = Q ( x ti − ˆ x ti ) , and sends the compressed information q ti to its neighbors.

Original: After that, node i and all its neighbors update the local replica ˆ x t +1 i
Revised: After that, node i and all its neighbors update the local replica ˆ x t +1 i

Original: =
Revised: =

Original: ˆ x ti + q ti .
Revised: ˆ x ti + q ti .

Original: When all nodes are reaching a consensus optimal decision, the updates of local decisions are small,and the differences between the replica variables and the true decision variables are also small.
Revised: When all nodes are reaching a consensus optimal decision, the updates of local decisions are small, and the differences between the replica variables and the true decision variables are also small.

Original: Thenthe compression errors are expected to vanish.
Revised: Then the compression errors are expected to vanish.

Original: Local decision variables update throughthe gradient descent, γ -gossip, and projection, in order tominimize the local loss function, keep consensus with neighbors, and remain in the feasible set K ,respectively.
Revised: Local decision variables update through gradient descent, γ -gossip, and projection, in order to minimize the local loss function, keep consensus with neighbors, and remain in the feasible set, respectively.

Original: For each node i , s 1 i is initialized to 0 , and thus, s ti = (cid:80)
Revised: For each node i , s 1 i is initialized to 0 , and thus, s ti = (cid:80)

Original: Nj =1 a ij ˆ x tj .
Revised: Nj =1 a ij ˆ x tj .

Original: Recall that ˆ x ti tracksx ti , then s ti acts as node i ’s estimate of the consensus decision at time t .
Revised: Recall that ˆ x ti tracks x ti , then s ti acts as node i ’s estimate of the consensus decision at time t .

Original: The γ -gossip protocol isadopted to renovate the decision variable towards the consensus decision.
Revised: The γ -gossip protocol is adopted to renovate the decision variable towards the consensus decision.

Original: The consensus stepsizeγ ∈ (0 , 1] is tunable to control the consensus speed, which also plays a crucial role in controlling thecompression error.
Revised: The consensus stepsize γ ∈ (0 , 1] is tunable to control the consensus speed, and will also play a crucial role in controlling the compression error.

Original: If there is no compression, i.e., using exact communication, then ˆ x t +1 i turns out to be x ti , and s t +1 ibecomes (cid:80) Nj =1 a ij x tj .
Revised: If there is no compression, i.e., using exact communication, then ˆ x t +1 i turns out to be x ti , and s t +1 i becomes (cid:80) Nj =1 a ij x tj .

Original: Besides, take γ = 1 , and then (1) reduces tox t
Revised: Besides, take γ = 1 , and then (3) reduces to x t

Original: +1{++
+} i
Revised: {++
+} i

Original: [Equation]
Revised: [Equation]

Original:  which is the DAOL algorithm in [7].
Revised: , which is the DAOL algorithm in [8].

Original: Remark 1.
Revised: Remark 1.

Original: s ti is introduced for the memory-efﬁciency.
Revised: s ti is introduced for the memory-efficiency.

Original: Eq.
Revised: Eq.

Original: (1) is equivalent to the update rule
Revised: (3) is equivalent to the update rule

Original: [Equation]
Revised: [Equation]

Original: If we adopt the update rule (2) together with ˆ x t +1 j = ˆ x tj + q tj for j ∈ N i ∪ { i } instead of steps 3and 5, then each node have to store deg ( i ) 
Revised: If we adopt the update rule (4) together with ˆ x t +1 j = ˆ x tj [Equation] q tj for j ∈ N i ∪ { i } instead of stepsand 5, then each node have to store deg ( i ) + 2 vectors, namely, x i , ˆ x i and ˆ x j , j ∈ N i , which is memory-consuming.

Revised: Remark 2.

Original: + 2 vectors, namely, x i , ˆ x i and ˆ x j , j ∈ N i , which ismemory-consuming.
Revised: It is potential to extend our algorithm from a fixed graph to a time-varying graph G t = ( V , E t ) . We give the following two settings.

Original: 2.2 Regret bounds
Revised: • Random switching undirected networks. Our algorithm can be directly applied to the classic time-varying Erd˝os-Rényi random networks [38], where G t is generated over the prescribed graph G and { i, j } ∈ E t with a probability 0 < p < 1 for all { i, j } ∈ E . This setting is equivalent to performing random gossip compressor over the fixed graph G , that is, transmiting information with the probability p , which satisfies Assumption 2 with ω = p . • Deterministic switching undirected networks.

Revised: Our algorithm can be applied to this setting by modifying the connectivity matrix from A to A ( t ) .

Revised: With further assumptions on the switching networks such as Bounded Intercommunication Interval assumption [39], one can analyze the regret interval-wisely.

Revised: This is interesting but out of this paper’s focus, and we leave it for future study.

Revised: 2.2 Regret bounds

Original: We considerthe following assumptions, which are widely used in the studies of distributed onlineoptimization [1, 18, 37].
Revised: We consider following assumptions, which are widely used in the studies of distributed online optimization [1, 11, 40].

Original: Assumption 3.
Revised: Assumption 3.

Original: The convex set K is bounded with diameter D , i.e., (cid:107) x − y (cid:107) ≤ D, ∀ x, y ∈ K .
Revised: The convex set K is bounded with diameter D , i.e., ∥ x − y ∥ ≤ D, ∀ x, y ∈ K .

Original: Assumption 4.
Revised: Assumption 4.

Original: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is convex and differentiablewith bounded gradient over K , i.e., max i,t,x (cid:107)∇ f ti ( x ) (cid:107) ≤ G.
Revised: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is convex and differentiable with bounded gradient over K , i.e., max i,t,x ∥∇ f ti ( x ) ∥ ≤ G.

Original: Assumption 5.
Revised: Assumption 5.

Original: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is µ -strongly convex over Kwith the parameter µ > 0 , i.e., f ti ( x ) − f ti ( y ) ≥ (cid:104) x − y, ∇ f ti ( y ) (cid:105) + µ 2 (cid:107) x − y (cid:107) 2 , ∀ x, y ∈ K .
Revised: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is µ -strongly convex over K with the parameter µ > 0 , i.e., f ti ( x ) − f ti ( y ) ≥ ⟨ x − y, ∇ f ti ( y ) ⟩ + µ 2 ∥ x − y ∥ 2 , ∀ x, y ∈ K .

Original: Suppose that the eigenvalues of the symmetric doubly stochastic connectivity matrix A are 1 =| λ 1 ( A ) | > | λ 2 ( A ) | ≥ · · · ≥ | λ N ( A ) | .
Revised: Suppose that the eigenvalues of the symmetric doubly stochastic connectivity matrix A are 1 = | λ 1 ( A ) | > | λ 2 ( A ) | ≥ · · · ≥ | λ N ( A ) | .

Original: Deﬁne the spectral gap δ := 1 − | λ 2 ( A ) | ∈ (0 , 1] and thespectral radius of the Laplacian matrix β := (cid:107) I N − A (cid:107) 2 ∈ [0 , 2] .
Revised: Define the spectral gap δ := 1 − | λ 2 ( A ) | ∈ (0 , 1] and the spectral radius of the Laplacian matrix β := ∥ I N − A ∥ 2 ∈ [0 , 2] .

Original: Then we give the expected regretbounds of Algorithm 1 for convex and strongly convex losses, respectively.
Revised: Then we give the expected regret bounds of Algorithm 1 for convex and strongly convex losses, respectively.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let common Assumptions 1 and 2 hold.
Revised: Let common Assumptions 1 and 2 hold.

Original: Consider Algorithm 1 with the consensusstepsize
Revised: Consider Algorithm 1 with the consensus stepsize

Original: [Equation]
Revised: [Equation]

Original: (i) (Convex case) Under Assumptions 3 and 4, take the gradient descent stepsize η t =
Revised: (i) (Convex case) Under Assumptions 3 and 4, take the gradient descent stepsize η t =

Original: D G √ t + c for aconstant c ≥ 83 γδ .
Revised: D G √ t + c for a constant c ≥ 83 γδ .

Original: Then for each j ∈ V and T ≥ 1 ,
Revised: Then for each j ∈ V and T ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: (ii) (Strongly convex case)
Revised: (ii) (Strongly convex case)

Original: Under Assumptions 4 and 5, take the gradient descent stepsize η t = 1 µ ( t +c )for a constant c ≥ 163 γδ .
Revised: Under Assumptions 4 and 5, take the gradient descent stepsize η t = [Equation] [Equation] +

Revised: [Equation]

Revised: ) for a constant c ≥ 163 γδ .

Original: Then for each j ∈ V and T ≥ 1 ,
Revised: Then for each j ∈ V and T ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: The proof ideas are as follows.
Revised: The proof ideas are as follows.

Original: Firstly, we estimate the general regret bounds for each node, whichdepend on the consensus error, the projection error, the compression error, and the gradient descentstepsize.
Revised: Firstly, we estimate the general regret bounds for each node, which depend on the consensus error, the projection error, the compression error, and the gradient descent stepsize.

Original: Then comes the key points that we analyze the coupled relationship between the errors, andbound them with the consensus stepsize γ and the GD stepsize η t .
Revised: Then comes the key point that we analyze the coupled relationship between the errors, and bound them with the consensus stepsize γ and the GD stepsize η t .

Original: Finally, we choose proper γ andη t to obtain Theorem 1.
Revised: Finally, we choose proper γ and η t to obtain Theorem 1.

Original: Complete proofs are attached to Appendix B.
Revised: Complete proofs are attached to Appendix B.

Original: The consensus stepsize γ chosen in (3) depends on the compression ratio ω and the communicaitongraph paremeters δ and β .
Revised: The consensus stepsize γ chosen in (5) depends on the compression ratio ω and the communicaiton graph paremeters δ and β .

Original: Notice that γ is an increasing function with respect to ω , and γ | ω =

Revised: Notice that γ is an increasing function with respect to ω , and γ | ω =

Original: =0 , γ
Revised: = 0 , γ

Original: | ω =1 = 1 .
Revised: | ω =1 = 1 .

Original: Thus, γ ∈ (0 , 1] for ω ∈ (0 , 1] .
Revised: Thus, γ ∈ (0 , 1] for ω ∈ (0 , 1] .

Original: If there is no compression ( ω = 1 ) , then γ = 1 , and
Revised: If there is no compression ( ω = 1 ) , then γ = 1 , and

Original: Algorithm 1 exactly reduces to DAOL [7], as mentioned in the algorithm design.
Revised: Algorithm 1 exactly reduces to DAOL [8], as mentioned in the algorithm design.

Original: Theorem 1 shows that Algorithm 1 achieves the regret bounds O (( ω − 2 N 1 / 2 + ω − 4 ) N √ T ) and
Revised: Theorem 1 shows that Algorithm 1 achieves the regret bounds O (( ω − 2 N 1 / 2 + ω − 4 ) N √ T ) and

Original: O (cid:0)(cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) N ln( T  (cid:1) for convex losses and strongly convex losses, respectively.
Revised: O (cid:0)(cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) N ln( T ) (cid:1) for convex losses and strongly convex losses, respectively.

Original: Theresults suggest that
Revised: The results suggest that

Original: • Algorithm 1 is no-regret in both convex case and strongly convex case, since the time averagedregret E Q [R( j,T )] / T → 0 for T → ∞ .
Revised: • Algorithm 1 is no-regret in both convex case and strongly convex case, since the time averaged regret E Q [R( j,T )] / T → 0 for T → ∞ .

Original: The obtained regret bounds O ( √ T ) and O (ln( T )) are inaccordance with that of the centralized online algorithms in the respective cases [19, 20].
Revised: The obtained regret bounds O ( √ T ) and O (ln( T )) are in accordance with those of the centralized online algorithms in the respective cases [21, 22].

Original: The node averaged regret E Q [R( j,T )] / N increases with N , which in line with the results in [18].• As the compression ratio ω decreases, fewer bits are needed for node-to-node communication ineach iteration, while more iteration rounds are needed to reach the desired regret.
Revised: The node averaged regret E Q [R( j,T )] / N increases with N , which in line with the result in [11]. • As the compression ratio ω decreases, fewer bits are needed for node-to-node communication in each iteration, while more iteration rounds are needed to reach the desired regret.

Original: ω can be usedto balance the iteration rounds and the transmitted bits in each iteration from multiple concernssuch as the bandwidth and agent computation capability.
Revised: ω can be used to balance the iteration rounds and the transmitted bits in each iteration from multiple concerns such as the bandwidth and agent computation capability.

Original: In practice, we can choose a proper ωto minimize the total transmitted bits or minimize the overall training time.
Revised: In practice, we can choose a proper ω to minimize the total transmitted bits or minimize the overall training time.

Original:  One-point Bandit Feedback
Revised:  One-point Bandit Feedback

Original: In this section, we apply difference compression to DOCO with one-point bandit feedback.
Revised: In this section, we apply difference compression to DOCO with one-point bandit feedback.

Original: Wepropose DC-DOBD algorithm, which basically follows DC-DOGD, except for the gradient estimation.[Equation]
Revised: We propose DC-DOBD algorithm, which basically follows DC-DOGD, except for the gradient estimation.

Original: In the one-point bandit feedback, after making the decision x ti at time t , agent i can query the lossfunction value at one point around x ti and use the feedback to construct the gradient estimator g ti .
Revised: In the one-point bandit feedback, after making the decision x ti at time t , agent i can query the loss function value at one point around x ti and use the feedback to construct the gradient estimator g ti .

Original: Like the procedure in [38], let agent i choose a unit-norm vector u ti ∈ R d uniformly at random,query the value of f ti at the point y ti
Revised: Like the procedure in [41], let agent i choose a unit-norm vector u ti ∈ R d uniformly at random, query the value of f ti at the point y ti

Original: =
Revised: =

Original: x ti + (cid:15)u ti , and calculate g ti = d(cid:15) f ti ( y ti ) u ti .
Revised: x ti + ϵu ti , and calculate g ti = dϵ f ti ( y ti ) u ti .

Original: Since the lossfunction f ti is deﬁned in the set K , we slightly modify the projection in (1) as P (1 − ζ ) K to ensurethe query point y ti ∈ K .
Revised: Since the loss function f ti is defined in the set K , we slightly modify the projection in (3) as P (1 − ζ ) K to ensure the query point y ti ∈ K .

Original: Algorithm 2 actually performs the gradient descent on the functionˆ f ti ( x ) =
Revised: Algorithm 2 actually performs the gradient descent on the function ˆ f t i ( x ) =

Original: E u ∈B [ f ti ( x + (cid:15)u )] restricted to the convex set (1 − ζ ) K .
Revised: E u ∈B [ f t i ( x + ϵu )] restricted to the convex set (1 − ζ ) K .

Original: It has been shown by [38] that
Revised: It has been shown by [41] that

Original: E [ g ti ] = ∇ ˆ f ti ( x ti ) .
Revised: E [ g ti ] = ∇ ˆ f ti ( x ti ) .

Original: In the bandit setting, Assumptions 3 and 4 are modiﬁed as follows, which arecommonly used in online bandit optimization [38, 21, 18].
Revised: In the bandit setting, Assumptions 3 and 4 are modified as follows, which are common in online bandit optimization [11, 23, 41].

Original: Assumption 6.
Revised: Assumption 6.

Original: The convex set K contains the ball of radius r centered at the origin, and is containedin the ball of radius R , i.e., r B ⊆ K ⊆ R B , B = { u ∈ R d : (cid:107) u (cid:107) ≤ 1 } . 
Revised: The convex set K contains the ball of radius r centered at the origin, and is contained in the ball of radius R , i.e., r B ⊆ K ⊆ R B , B = { u ∈ R d : ∥ u ∥ ≤ 1 } . [Table] DC-DOBD)

Original: Assumption 7.
Revised: Input: consensus stepsize γ , GD stepsizes { η t } Tt =1 , time T , exploration parameter ϵ , shrinkage [Table] Initialize: set x 1 i = 0 , = 0 , s 1 i = 0 , for each node i ∈ V .

Revised: 1: for t = 1 to T − 1 do in parallel for each node i ∈ V 2:

Revised: Compress the difference vector q ti = Q

Revised: ( x ti − ˆ x ti ) and update ˆ x t

Revised: +1 i

Revised: =

Revised: ˆ x ti + q ti .

Revised: Spread q ti and receive q tj , j ∈ N i .

Revised: Update s t

Revised: +1 i

Original: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is convex and l -lipschitzcontinuous in K , i.e., | f ti ( x) − f ti ( y ) |
Revised: = s ti + (cid:80) j ∈N i a ij q tj . Receive the one-point bandit feedback and construct g ti = dϵ f ti ( x ti + ϵu ti ) u ti . Update the decision variable x t +1 i

Original: ≤ l (cid:107) x − y (cid:107) , ∀ x, y ∈ K {+(cid:0) x ti + γ ( s t +
 i − ˆ x t +1 i ) − η t g ti (cid:1)+} . 
Revised: = P (1 − ζ ) K {+(cid:0) x ti + γ ( s t +
 i − ˆ x t +1 i ) − η t g ti (cid:1)+} . Output: { x ti } Tt =

Revised: Assumption 7.

Revised: For each i ∈ V and t = 1 , 2 , ..., T , the loss function f ti is convex and l -Lipschitz continuous in K , i.e., | f ti ( x ) − f ti ( y ) | ≤ l ∥ x − y ∥ , ∀ x, y ∈ K .

Original: Assumptions 6 and 7 lead to an uniform upper bound on the function value, i.e., there exists a constant
Revised: Assumptions 6 and 7 lead to an uniform upper bound on the function value, i.e., there exists a constant

Original: B > 0 such that max x,i,t | f ti ( x ) | ≤ B .
Revised: B > 0 such that max x,i,t | f ti ( x ) | ≤ B .

Original: Then we establish the expected regret bounds of Algorithm 2for convex and strongly convex losses, respectively.
Revised: Then we establish the expected regret bounds of Algorithmfor convex and strongly convex losses, respectively.

Original: Theorem 2.
Revised: Theorem 2.

Original: Let common Assumptions 1, 2, 6 and 7 hold.
Revised: Let common Assumptions 1, 2, 6 and 7 hold.

Original: Consider Algorithm 2 with the consensusstepsize γ chosen in (3).
Revised: Consider Algorithm 2 with the consensus stepsize γ chosen in (5).

Original: Denote
Revised: Denote

Original: [Equation]
Revised: [Equation]

Original: (i) (Convex case) Take the gradient descent stepsize η t =
Revised: (i) (Convex case) Take the gradient descent stepsize η t =

Original: [Equation]
Revised: [Equation]

Original: dB √ t+ c for a constant c ≥ 83 γδ , (cid:15) =
Revised: t

Original: (cid:18)
Revised: [Equation]

Original: 1+4 H dBR l + Br
Revised: c for a constant c ≥

Original: [Equation]
Revised: [Equation]

Original: T +
Revised: , ϵ

Original: c )
Revised: [Equation]

Original:   Tand ζ = (cid:15)r .
Revised: [Equation]  Tand ζ = ϵr .

Original: Then for each j ∈ V and T ≥ 1 ,
Revised: Then for each j ∈ V and T ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: (ii) (Strongly convex case) With additional Assumptions 5, take the gradient descent stepsize η t =1 µ ( t + c ) for a constant c ≥ 163 γδ , (cid:15) = (cid:16)
Revised: (ii) (Strongly convex case) With additional Assumption 5, take the gradient descent stepsize η t = 1 µ ( t + c ) for a constant c ≥ 163 γδ , ϵ = (cid:16)

Original: [Equation]
Revised: [Equation]

Original: Br
Revised: ln( T

Original: [Equation]
Revised: [Equation]

Original: (cid:17) 13 and ζ = (cid:15)r
Revised: (cid:17) 13 and ζ = ϵr .

Original: Then for each j ∈ V and T ≥ 1 ,
Revised: Then for each j ∈ V and T ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: Theorem 2 shows that Algorithm 2 is also no-regret, and it achieves the regret bounds O ( d 1 / 2 N 5 / 4 T 3 / 4 ) and O ( d 2 / 3 N 7 / 6 T 2 / 3 ln 1 / 3 ( T )) for convex losses and strongly convexlosses, respectively, which match the bounds obtained by [18].
Revised: Theorem 2 shows that Algorithm 2 is also no-regret, and it achieves the regret bounds O ( d 1 / 2 N 5 / 4 T 3 / 4 ) and O ( d 2 / 3 N 7 / 6 T 2 / 3 ln 1 / 3 ( T )) for convex losses and strongly convex losses, respectively, which match the bounds obtained by [11].

Original: The regrets are scaled with(cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) 1 / 2 and (cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) 1 / 3 for convex and strongly convex losses, which indicates that the inﬂuence of the compression ratio ω on the regret bounds in the one-point banditsetting is less than that in the full information setting.
Revised: The regrets are scaled with (cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) 1 / 2 and (cid:0) ω − 2 N 1 / 2 + ω − 4 (cid:1) 1 / 3 for convex and strongly convex losses, which indicates that the influence of the compression ratio ω on the regret bounds in the one-point bandit setting is less than that in the full information setting.

Original:  Two-point Bandit Feedback
Revised:  Two-point Bandit Feedback

Original: In the two-point bandit feedback, agent i can query the loss function values at two points aroundx ti .
Revised: In the two-point bandit feedback, agent i can query the loss function values at two points around x ti .

Original: Like the procedure in [21], let agent i pick a unit-norm vector u ti ∈ R d uniformly at random,query the values of f ti at y ti,
Revised: Like the procedure in [23], let agent i pick a unit-norm vector u ti ∈ R d uniformly at random, query the values of f ti at y ti,

Original: 1 = x ti + 
Revised: 1 = x ti + ϵu ti

Original: (cid:15)u ti and y ti,

Original:  2 = x ti − (cid:15)u ti , and estimate the gradient asg ti =
Revised: and y ti, 2 = x ti − ϵu ti , and estimate the gradient as g ti =

Original: d 2 (cid:15) (cid:0) f ti ( y ti, 1 ) − f ti ( y ti, 2 ) (cid:1) u ti .
Revised: d 2 ϵ (cid:0) f ti ( y ti, 1 ) − f ti ( y ti, 2 ) (cid:1) u ti .

Original: Then we obtain DC-DO2BD as a variant of DC-DOBD.
Revised: Then we obtain DC-DO2BD as a variant of DC-DOBD.

Revised: In the two-point bandit setting, the regret of node j ∈ V is modified as

Revised: [Equation]

Revised: (DC-DO2BD)

Revised: Input: consensus stepsize γ , GD stepsizes { η t } Tt =1 , time T , exploration parameter ϵ , shrinkage

Revised: [Table]

Original: Spread q ti and receive q tj , j ∈ N i .
Revised: Initialize: set , for each 1: for to do parallel for i V 2:

Revised: Compress the difference vector q

Revised: Q (

Revised: −

Revised: ) and update ˆ x t +1 i

Revised: =

Revised: ˆ x ti + q ti .

Revised: Spread q ti and receive q tj , j ∈ N i .

Original: Update s t
Revised: Update s t

Original: +1 i
Revised: +1 i

Original: [Equation]
Revised: = s

Original:  Receive the two-point feedback and construct g ti =d 2 (cid:15) [Equation] f ti
Revised: ti + (cid:80) j ∈N i a ij q tj . Receive the two-point feedback and construct g ti =

Original: [Equation]

Original: x ti + [Equation] ti [Equation]
Revised: d 2 ϵ

Original:  f ti [Equation] x [Equation]
Revised: ( f ti ( x ti + ϵu ti ) − f ti ( x ti − ϵu ti ))

Original: − [Equation] u [Equation]
Revised: u ti .

Original: Update the decision variable x t 
Revised: Update the decision variable x t +1 i

Original: [Equation]{+= P (1 − ζ ) K (cid:0) x ti + γ ( s t +
+}
Revised: {+= P (1 − ζ ) K (cid:0) x ti + γ ( s t +
+}

Original: i [Equation]
Revised: i − ˆ x t +1 i ) − η t g ti (cid:1) .

Original: Output: { x ti } Tt =
Revised: Output: { x ti } Tt =

Original: In the two-point bandit setting, the regret of node j ∈ V is modiﬁed as R 2 ( j, T ) =(cid:80)

Original: [Equation]

Original: Theorem 3.
Revised: Theorem 3.

Original: Let common Assumptions 1, 2, 6 and 7 hold.
Revised: Let common Assumptions 1, 2, 6 and 7 hold.

Original: Consider Algorithm 3 with the consensusstepsize γ chosen in (3) and H deﬁned in (6).
Revised: Consider Algorithm 3 with the consensus stepsize γ chosen in (5) and H defined in (8).

Original: (i) (Convex case) Take the gradient descent stepsize η t =2 R dl √ t + c for a constant c ≥ 83 γδ , (cid:15)
Revised: (i) (Convex case) Take the gradient descent stepsize η t = R dl √ t + c for a constant c ≥ 83 γδ , ϵ = 1 √ T

Original: [-= 
 √-]
Revised: [-= 
 √-]and

Original: Tand ζ = (cid:15)r .
Revised: ζ = ϵr .

Original: Then for each j ∈ V and T ≥1 ,
Revised: Then for each j ∈ V and T ≥

Original: [Equation]
Revised: [Equation]

Original: (ii) (Strongly convex case) With additional Assumptions 5, take the gradient descent stepsize η t =1 µ ( t + c ) for a constant c ≥ 16 3 γδ , (cid:15) = ln( T ) T and ζ = (cid:15) r .
Revised: (ii) (Strongly convex case) With additional Assumption 5, take the gradient descent stepsize η t =µ ( t + c ) for a constant c ≥ 163 γδ , ϵ = ln( T ) T and ζ = ϵr .

Original: Then for each j ∈ V and T ≥ 1 ,
Revised: Then for each j ∈ V and T ≥ 1 ,

Original: [Equation]
Revised: [Equation]

Original: Theorem 3 shows that Algorithm 3 achieves O (( ω − 2 N 1 / 2 + ω − 4 )
Revised: Theorem 3 shows that Algorithm 3 achieves O (( ω − 2 N 1 / 2 + ω − 4 )

Original: Nd √ T ) and O (( ω − 2 N 1 / 2 +ω − 4 )
Revised: Nd √ T ) and O (( ω − 2 N 1 / 2 + ω − 4 )

Original: Nd 2 ln( T )) regret bounds for convex and strongly convex losses, respectively, which recoversthe regret bounds O ( √ T ) (convex) and O (ln( T )) (strongly convex) in the full information case,while the constants are larger than those of Theorem 1.
Revised: Nd 2 ln( T )) regret bounds for convex and strongly convex losses, respectively, which recovers the regret bounds O ( √ T ) (convex) and O (ln( T )) (strongly convex) in the full information case, while the constants are larger than those of Theorem 1.

Original:  Numerical Experiments
Revised:  Numerical Experiments

Original: In this section, we evaluate the three proposed algorithms on a real-world online problem.
Revised: In this section, we evaluate the three proposed algorithms on a real-world online problem.

Original: Aprominent example is the diabetes prediction task, which aims tobuild a model to diagnose diabetesthrough several risk factors.
Revised: A prominent example is the diabetes prediction task, which aims to diagnose diabetes through several risk factors.

Original: Consider the distributed online regularized logistic regression with thelocal loss function
Revised: Consider the distributed online regularized logistic regression with the local loss function

Original: [Equation]
Revised: [Equation]

Original:  where µ is the regularization parameter, and a batch data samples { ( a i,j , b i,j ) } Sj =1 are revealed toagent i at time t .
Revised: (14) where µ is the regularization parameter, and a batch data samples { ( a i,j , b i,j ) } S j =1 are revealed to agent i at time t .

Original: We adopt diabetes-binary-BRFSS2015 dataset with 70692 instances, 21 features,and 2 labels from Kaggle 1 .
Revised: We adopt diabetes-binary-BRFSS2015 dataset with 70692 instances, 21 features, and 2 labels from Kaggle.

Original: Here, a i,j ∈ R d with d = 21 , and b i,j ∈ {− 1 , 1 } .
Revised: Here, a i,j ∈ R d with d = 21 , and b i,j ∈ {− 1 , 1 } .

Original: We standardize thedata samples and distribute them evenly among N agents under the sorted setting, i.e., each agentonly gets data samples from one class.
Revised: We standardize the data samples and distribute them evenly among N agents under the sorted setting, i.e., each agent only gets data samples from one class.

Original: The connected communication network G ( N, M ) with Nnodes and M edges is generated randomly by tool NetworkX [39], and then we use the Metropolisrule [40] to construct the connectivity matrix A to satisfy Assumption 1.
Revised: The connected communication network G ( N, M ) with N nodes and M edges is generated randomly by tool NetworkX [42], and then we use the Metropolis rule [43] to construct the connectivity matrix A to satisfy Assumption 1.

Original: We repeat each experimentten times and depict the mean curve 2 .
Revised: We repeat each experiment ten times and depict the mean curve.

Original:  Thechoice of parameters is given in Appendix E.
Revised: 3 The parameters selection details are given in Appendix E.

Original: Comparison experiment We run our algorithms DC-DOGD, DC-DOBD, DC-DO2BD, and makecomparisons with ECD-AMSGrad [17], for the convex case ( µ = 0 ) and strongly convex case( µ = 1 ).
Revised: Comparison experiment We run our algorithms DC-DOGD, DC-DOBD, DC-DO2BD, and make comparisons with ECD-AMSGrad [20], for the convex case ( µ = 0 ) and strongly convex case ( µ = 1 ).

Original: The compressor type, the compression ratio, and the communication network are kept thesame.
Revised: The compressor type, the compression ratio, and the communication network are kept the same.

Original: Take the setting of  QSGD 2 with ω = 0 .
Revised: Take the setting of compressor QSGD 2 with ω = 0 .

Original: 3 over G (9 , 18) as an example.
Revised: 3 over G (9 , 18) as an example.

Original: We plot the timeaveraged maximum regret SR ( T ) :
Revised: We plot the time averaged maximum regret SR ( T ) :

Original: = max j R( j,T ) / T versus the time horizon T and versus the totalnumber of transmitted bits in Fig.
Revised: = max j R( j,T ) / T versus the time horizon T and versus the total number of transmitted bits in Fig.

Original: 1, where the best solution in the hindsight x ∗ is obtained by LogisticRegression optimizer from scikit-learn [41].
Revised: 1, where the best solution in the hindsight x ∗ is obtained by LogisticRegression optimizer from scikit-learn [44].

Original: Fig.
Revised: Fig.

Original: 1 shows that the time averaged regrets of DC-DOGD, DC-DOBD, and DC-DO2BD go tozero as T goes to inﬁnity, which is in agreement with the theoretical results that our algorithms areno-regret.
Revised: 1 shows that the time averaged regrets of DC-DOGD, DC-DOBD [Figure] and DC-DO2BD go to zero as T goes to infinity, which is in agreement with the theoretical results that our algorithms are no-regret.

Original: Among the three proposed algorithms, the one-bandit feedback has the worst performance,while using two-bandit information can improve the performance and even reach that of the fullinformation feedback.
Revised: Among the three proposed algorithms, the one-bandit feedback has the worst performance, while using two-bandit information can improve the performance and even reach that of the full information feedback.

Original: ECD-AMDGrad gets deteriorated in the ﬁrst few steps because the inverse ofthe second raw moment estimation is large and this algorithm does not have a projection to restrictvariables.
Revised: ECD-AMSGrad gets deteriorated in the first few steps because the inverse of the second raw moment estimation is large and this algorithm does not have a projection to restrict variables.

Original: Then, ECD-AMDGrad declines fast, while its time-average regret can not reach zero.
Revised: Then, ECD-AMSGrad declines fast, while its time-average regret can not reach zero.

Original: Clearly, DC-DOGD and DC-DO2BD signiﬁcantly outperform ECD-AMDGrad.
Revised: Clearly, DC-DOGD and DC-DO2BD significantly outperform ECD-AMSGrad.

Original: [Figure]
Revised: [Figure]

Original: Impact of compression ratio and compressor type Fixing the compressor type ( Top k ) and thegraph G (9 , 18) , we run DC-DOGD with different compression ratios ( ω = 0 .
Revised: Impacts of compression ratio and compressor type Fixing the compressor type ( Top k ) and the graph G (9 , 18) , we run DC-DOGD with different compression ratios ( ω = 0 .

Original: 05 , 0 . 1 , 0 . 5 ) for stronglyconvex losses 3 .
Revised: for strongly convex losses.

Original: As the baseline we consider DAOL [7], which is with exact communication and isthe special case of DC-DOGD with ω = 1 .
Revised: 4 We consider DAOL [8] as the baseline, which is with exact communication and is the special case of DC-DOGD with ω = 1 .

Original: The greater the compression degree  (less ω ), the moreiteration rounds are needed to reach the ε average regret as Fig. 2a shows, while the total transmittedbits are actually the fewer as Fig. 2b shows.
Revised: Fig. 2a shows that the greater the compression degree is (less ω ), the more iteration rounds are needed to reach a certain average regret, while the total transmitted bits are actually the fewer.

Original: DC-DOGD with ω = 0 .
Revised: DC-DOGD with ω = 0 .

Original: 5 performs almost as good as

Original: DAOL while using 2 × less total bits to reach the ε average regret.

Original: DC-DOGD with ω = 0 .

Original: 05 haveapproximately 10 × reduction on  bits to reach the ε average regret compared with DAOL.
Revised: 05 have approximately 8 × reduction on transmitted bits to reach a certain average regret compared with DAOL.

Original: Then, we ﬁx the compression ratio ( ω = 0 . 3 ) and the graph G (9 , 18) , and run DC-DOGD withdifferent compressors ( Top k , Rand k , RGossip p , GSGD s ) for strongly convex losses.
Revised: Then, we fix the compression ratio ( ω = 0 . 3 ) and the graph G (9 , 18) , and run DC-DOGD with different compressors ( Top k , Rand k , RGossip p , GSGD s ) for strongly convex losses.

Original: Figs.
Revised: Fig.

Original: 2c and2d show that Rand k and RGossip p have almost the same performance.
Revised: 2b shows that Rand k and RGossip p have almost the same performance.

Original: It is expected, sincein Rand k  each element of the vector  has the probability ω
Revised: It is expected, since Rand k allows each element of the vector to has the probability ω

Original: = k/d to be chosen to be transmitted, whichis equivalent to randomly transmitting the whole vector with the probability p = ω .
Revised: = k/d to be chosen to be transmitted, which is equivalent to randomly transmitting the whole vector with the probability p = ω .

Original: Besides, Figs.
Revised: Besides, Figs.

Original: 2c [Table] and 2d show that Top k has better performance than Rand k , which is in line with the intuition thatthe largest k coordinates contain more useful information than arbitrary k coordinates.
Revised: 2b shows that Top k has better performance than Rand k , which is in line with the intuition that the largest k coordinates contain more useful information than arbitrary k coordinates.

Original: In addition,quantization GSGD s performs better in reducing the total transmitted bits than sparsiﬁcation underthe same compression ratio.
Revised: In addition, quantization GSGD s performs better in reducing the total transmitted bits than sparsification under the same compression ratio.

Original: [Figure]
Revised: [Figure]

Original: 303 Impact of topology and node number The  network topology concerns the parameters δ and β ,which inﬂuence the choice of the consensus stepsize γ as well as the algorithm performance.
Revised: Impacts of topology and node number The communication network topology affects the parameters δ and β , which affect the choice of the stepsizes η t and γ as well as the algorithm performance.

Original: Asimpler topology with fewer edges needs fewer bits to transmit information in each iteration, whilemore iteration rounds are needed for decision consensus.

Original: Thus, there will be a tradeoff.

Original: We take DC-DOGD in the strongly convex case as an example.
Revised: We take DC-DOGD in the strongly convex case as an example.

Original: Fixing the compressor ( Top 1 ) with thecompression ratio ( ω = 0 . 05 ), we assess DC-DOGD over three basic topologies (ring, G ( N, 2 N ) ,full connected).
Revised: Fixing the compressor ( Top 1 ) and the compression ratio ( ω = 0 . 05 ), we assess DC-DOGD over three basic topologies, namely ring, G ( N, 2 N ) , and full connected.

Original: Fig.
Revised: Fig.

Original: 3a shows thatthe full connected graph uses the smallest iteration round to reachthe ε average regret, while Fig. 3b illustrates that the total numbers of transmitted bits to reach the εaverage regret are close.
Revised: 3a shows that full connected topology exhibits the best regret with respect to the time horizon, while G ( N, 2 N ) performs slightly better in the sense of tranmitted bits. A sparser topology with fewer edges uses fewer bits to transmit information in each iteration, while more iteration rounds are needed for decision consensus.

Original:  Finally, we let the node number N vary from to 10 to 50, and run DC-DOGD, DC-DOBD, DCDO2BD with the same compressor Top 2 , the same compression ratio ω = 0 . 1 , over the fullconnected graph, for convex losses and strongly convex losses.
Revised: Thus, there will be a tradeoff case by case. Finally, we let the node number N vary from to 10 to 50, and run DC-DOGD, DC-DOBD, DCDO2BD with the same compressor Top 2 , the same compression ratio ω = 0 . 1 , over the full connected graph, for convex losses and strongly convex losses.

Original: We plot the node averaged regret
Revised: We plot the node averaged regret

Original: AR ( T ) := max j R( j,T ) / N versus the node number N in Figs.
Revised: AR ( T ) := max j R( j,T ) / N versus the node number N in Fig.

Original: 3c and 3d.
Revised: 3b.

Original: [Figure]
Revised: [Figure]

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we considered DOCO with the full information feedback, one-point and two-pointsbandit feedback.
Revised: In this paper, we considered DOCO with the full information feedback, one-point and two-points bandit feedback.

Original: We designed provably no-regret distributed online algorithms that work with ωcontracted compressors.
Revised: We designed provably no-regret distributed online algorithms that work with ωcontracted compressors.

Original: The obtained regret bounds for both convex and strongly convex lossesmatched those of uncompressed algorithms in the literature.
Revised: The obtained regret bounds for both convex and strongly convex losses matched those of uncompressed algorithms in the literature.

Original: We further assessed the inﬂuence of thecompressor type and the compression ratio ω on the regrets, and showed that ω can be used to balancethe iteration rounds and the transmitted bits according to the bandwidth.
Revised: We further assessed the influence of the compressor type and the compression ratio ω on the regrets, and showed that ω can be used to balance the iteration rounds and the transmitted bits according to the bandwidth.

Original: The limitation of this work isthat the obtained regret bounds show high order inverse dependence on the compression ratio, whichare pretty conservative and may be further improved.
Revised: A limitation of this work is that the obtained regret bounds show high order inverse dependence on the compression ratio, which are pretty conservative and may be further improved.

Revised: Besides, the impact of topology sparsity is worth further investigation.

Original: We believe this paper is an important step inthis direction.
Revised: We believe this paper is an important step in this direction.

Original: Future research includes designing provably no-regret distributed online algorithmswith other compression schemes such as extrapolation compression.
Revised: Future research includes extensions from fixed graphs to time-varying graphs and combinations with other compression schemes such as extrapolation compression.

Revised: Acknowledgments and Disclosure of Funding

Revised: This work is supported in part by Shanghai Municipal Science and Technology Major Project under Grant 2021SHZDZX0100 and the National Natural Science Foundation of China under Grant 62173250, and in part by Australian Research Council under Grant DP190103615 and LP210200473.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 6 Conclusions.
Revised: [Yes] See Section 6 Conclusions.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [No] This work is a theoretical ﬁnding to solve the communication bottleneck of the distributed algorithms.
Revised: [No] This work is a theoretical finding to solve the communication bottleneck of the distributed algorithms.

Original: Therefore, this work does not present foreseeable societal impacts.
Revised: Therefore, this work does not present foreseeable societal impacts.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Assump- tions 1, 2, 3, 4, 5, 6, and 7.
Revised: [Yes] See Assumptions 1, 2, 3, 4, 5, 6, and 7.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] We present the proof ideas in the paper and put the complete proofs in supplemental materials due to spacelimitation.
Revised: [Yes] We present the proof ideas in the paper and put the complete proofs in supplemental materials due to space limitation.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] Codes are inthe supplemental material.
Revised: [Yes] Codes link is in the supplemental material.

Original: Data are from Kaggle with URL.
Revised: Data are from Kaggle with URL.

Original: Instructions are includedin Section 5.
Revised: Instructions are included in Section 5.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See Section 5 and Appendix E (c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
Revised: [Yes] See Section 5 and Appendix E (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] We repeat each experiment ten times and only depict the mean curves to make the ﬁgures easier to read.
Revised: [No] We repeat each experiment ten times and only depict the mean curves to make the figures easier to read.

Original: Actually, our experiment results withdifferent random seeds are similar.
Revised: Actually, our experiment results with different random seeds are similar.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See Section 5 footnote
Revised: [Yes] See Section 5 footnote4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] We use the tools NetworkX and scikit-learn with citation.
Revised: [Yes] We use the tools NetworkX and scikit-learn with citation.

Original: We use the open dateset from Kaggle with URL.
Revised: We use the open dateset from Kaggle with URL.

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See Appendix F (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: [Yes] See Appendix F (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [No] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [No] (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [No]5.
Revised: [No] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]