GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models
Authors: Wenbo Su
Yuanxing Zhang
Yufeng Cai
Kaixu Ren
Pengjie Wang
Huimin Yi
Yue Song
Jing Chen
Hongbo Deng
Jian Xu
Lin Qu
Bo Zheng
Venue: NeurIPS 2022 Accept (2022)
Keywords: recommendation model
synchronous training
asynchronous training
tuning-free switching
global batch size

Abstract: High-concurrency asynchronous training upon parameter server (PS) architecture and high-performance synchronous training upon all-reduce (AR) architecture are the most commonly deployed distributed training modes for recommendation models. Although synchronous AR training is designed to have higher training efficiency, asynchronous PS training would be a better choice for training speed when there are stragglers (slow workers) in the shared cluster, especially under limited computing resources. An ideal way to take full advantage of these two training modes is to switch between them upon the cluster status. However, switching training modes often requires tuning hyper-parameters, which is extremely time- and resource-consuming. We find two obstacles to a tuning-free approach: the different distribution of the gradient values and the stale gradients from the stragglers. This paper proposes Global Batch gradients Aggregation (GBA) over PS, which aggregates and applies gradients with the same global batch size as the synchronous training. A token-control process is implemented to assemble the gradients and decay the gradients with severe staleness. We provide the convergence analysis to reveal that GBA has comparable convergence properties with the synchronous training, and demonstrate the robustness of GBA the recommendation models against the gradient staleness. Experiments on three industrial-scale recommendation tasks show that GBA is an effective tuning-free approach for switching. Compared to the state-of-the-art derived asynchronous training, GBA achieves up to 0.2% improvement on the AUC metric, which is significant for the recommendation models. Meanwhile, under the strained hardware resource, GBA speeds up at least 2.4x compared to synchronous training.

GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models
Wenbo Su", Yuanxing Zhang; Yufeng Cai, Kaixu Ren, Pengjie Wang, Huimin Yi, Yue Song, Jing Chen', Hongho Deng, Jian Xu, Lin Qu', Bo Zheng!
Alibaba Group
(vincent. swb, yuanxing zyx, baike. cyf, kaixu. kaixu.rkx,
pengjie.wp wpj, huimin yhm, yhmm yue. song, dhb167148, xiyu.xj, bozheng/@alibaba-iine "ggonccheng.c) cj, hide.ql/@taobao.com
Abstract
High-concurrency asynchronous training upon parameter server (PS) architecture and high-performance synchronous training upon all-reduce (AR) architecture are the most commonly deployed distributed training modes for recommendation models. Although synchronous AR training is designed to have higher training efficiency, asynchronous PS training would be better choice for training speed when there stragglers (slow workers) in the shared cluster, especially under limited computing resources. An ideal way to take full advantage of these two training modes is to switch between them upon the cluster status. However, switch- ing training modes often requires tuning ayper-parametens which is extremely time- and essorcce-cossuming We find two obstacles the different distribution of the gradient values and the stale gradients from the stragglers. This paper proposes Global Batch gradients Aggregation (GBA) over PS, which aggregates and applies gradients with the same global batch size as the synchronous training. A token-control process is implemented assemble the gradients and decay the gradients with severe staleness. We provide the conver- gence analysis reveal that GBA has comparable convergence properties with the synchronous training, and demonstrate the robustness of GBA the recommenda- tion models against the gradient staleness. Experiments on three industrial-scale recommendation tasks show that GBA is an effective tuning-free approach for switching. Compared to the tfate-ofthe-an derived asynchronous training, GBA achieves up to 0.2% improvement on the AUC metric, which is significant for the recommendation models. Meanwhile, under the strained hardware resource, GBA speeds up at least 2.4x compared to synchronous training.
tuning-free approach:
1 Introduction
Nowadays, recommendation models with large volume of parameters and high computational complexity have become the mainstream in the deep learning communities
[3] Accelerating the
training of these recommendation models is trending issue, and recently synchronous training upon nigh-performmacc computing (HPC) has dominated the training speed records [18 16. [T]. The
resource requirements of the synchronous training upon AR are more rigorous than the asynchronous
These authors contrihuted equally to this work.
't Corresponding author
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

training upon PS M. In a shared training cluster with dynamic status [3, the synchronous training would be retarded by few straggling workers. Thus, its training speed may be even much slower than the high-concurrency asynchronous training.
Should it be possible switch the training mode according to the cluster status, we will have access to making full use of the limited hardware resources. Switching the training mode for a specific model usually demands tuning of the hyper-parameters for guarantees of accuracy. Re-tuning the syper-parameters is common in the one-shot training workloads (c.g., the general CV or NLP workloads) [20]. However, it is not applicable for the continual learning or the lifelong training of When
the recommendation models [TO], as tuning would be highly time- and resource-consuman switching the training mode of representative recommendation models, we confront three main challenges from our shared cluster: 1) Model accuracy may suffer from sudden drop after switching, requiring the model to be retrained on a large amount of data to reach the comparable accuracy before switching; 2) The distribution of gradient values is different between synchronous training and asynchronous training, making the models under two training modes difficult to reach the same accuracy by tuning the ) The cluster status imposes staleness on the asynchronous training, and staleness negatively impacts the aggregation of gradients, especially for the dense parameters.
We conduct a systematic investigation of the training workloads of recommendation models to tackle the above challenges. It is found that when the global batch size (i.e., the actual batch size of gradient aggregation) is the same, the distribution of gradient values of asynchronous training tends to be similar to that of synchronous training. Besides, we notice that due to the high sparsity, the embedding parameters in recommendation models are less frequently updated than the dense parameters, leading to stronger tolerance for staleness than the general CV or NLP deep learning models. Based on these insights, we propose Global Batch gradients Aggregation (GBA), which ensures the model keeps the same global batch size when switched between the synchronous and asynchronous training. GBA is implemented by a token-control mechanism, which resorts to bounding the staleness and making gradient aggregation 12]. The mechanism suppresses the staleness following a staleness decay strategy over the token index. The faster nodes would take more tokens without waiting, and thereby GBA trains as fast as the asynchromous mode. Furthermore, the convergence analysis shows that GBA has comparable convergence properties with the synchronous mode, even under high staleness for recommendation models. We conduct extensive evaluation three continual learning of recommendation tasks. The results reveal that GBA performs well on both accuracy and efficiency with the same hyper-parameters Particularly, GBA improves the AUC metric by 0.2% on average compared to the state-of-the-an training modes of asynchronous training. Besides, GBA presents at least 2.4x speedup over the synchronous AR training the cluster with strained hardware resources.
To the best of our knowledge, this is the first work to approach switching between synchronous and asynchronous training without tuning the syper-parametent training cluster. The tuning-free switching enables our users to dynamically change the training modes between GBA and the synchronous HPC training for the continual learning tasks. The overall training efficiency of these training workloads is thereby significantly improved, and the hardware utilization within the cluster also raised by a large margin.
GBA has been deployed in our shared
2 Related Work
Distributed training mode. PS 21 and AR [8] are two mainstream architectures for the training workloads of recommendation models, accompanied by the asynchronous training and synchronous training, respectively. Researchers are enthusiastic about the pipeline, communication, and compu- tation optimization for the AR architecture of recommender systems [31]. Meanwhile, to improve the training efficiency of the PS architecture, researchers propose a category of synchyyychssouus training mode [2]. For example, Hop-BS [24] restricts the gradient updates under the bounded staleness, and Hop-BW 24 ignores the gradients from the stragglers with the well-shuffled and redundancy data. Recently, a category of decentralized training has been proposed in many studies to scale out the AR architecture. Local all-reduce (25,, local update (26), exponential graph (28), and many topology-aware solutions have proven promising in the NL. and CV tasks. However, owing to the sparsity in recommendation models, the inconsistent parameters among workers and the dropped gradicnts of the scarce IDs would intolcrably degrade the accuracy. Besides, these training modes

---

Bync.
Asyte.
Apyil
Sync.
Sync.
Normalized value
CPU UHl.
Prague
DeepFM
0.8043
Sync.
SwamAdam
Async.
YouTubeDNNt 0,
AUC
Swittching
0.5
0.5
0.7660
6.00
12:00 Time in a day
23:00
0%
50% Training progress
100%
Figure 1: Normalized QPS of four training modes in training YouTubeDNN models in shared cluster, with CPU utilization in a day.
Figure 2: The AUC on the validation set of Criteo-4GB and Private by every 5% progress during training, switching at 50% progress.
hardly consider the requirements to switch to another training mode according to the cluster status, though switching beneficial to improve the training efficiency in the shared training clusters.
Staleness and noisy gradients. The indeterminate or even inferior model accuracy of asynchronous training is mainly attributed to the staleness 7 and the noisy gradients [27] caused by the small batch. Although prior research has pointed out that the converged giant model is less sensitive to staleness [] staleness is still negative factor in the accuracy of the continual recommendation training. Many efforts have been put into controlling staleness via Taylor expansion
[32) weighted penalty [35]. etc. Recent works present harge-baccl training paradigm with specially-designel optimizers scale the gradients before updating [29], and point out that it can reach the best accuracy by merely adjusting batch size [1 (0). There are also attempts change gradient aggregation strategies during the asynchronous training to achieve stable model accuracy [22). GBA generalizes the staleness control paradigm to the recommendation workloads by token-control mechanism, which finds the balance between bounding staleness and ignoring gradients. GBA runs with the same global batch size as the synchronous mode, ensuring the effective switching between GBA and synchronous training without tuning hyper- prpee-ppraatters..
Preliminaries
3.1 Distributed Training of Recommendation Models
Recommendation models usually comprise two modules: the sparse module contains the embedding layers with the embedding parameters, mapping the categorical IDs into numerical space; the dense module contains the computational blocks with the dense parameters, such as attention and MLP 43, to exploit the feature interactions. The main difference between the two kinds of parameters the occurrence ratio in each training batch. Each training batch needs all the dense parameters, yet only a tiny amount of embedding parameters are required according to the feature IDs in the data shard. The latest development of recommendation models introduces high complexity and large volume of parameters, making distributed training essential improve training efficiency. The synchronous HPC training mode usually adopts the AR architecture, where the dense parameters are replicated, and the embedding parameters are partitioned on each worker. HPC should be deployed by monopolizing few perforreform workers and making full use of the associated resources, which may be retarded by the slow workers [9] PS architecture is usually coupled with asynchronous high concurrency training where the parameters are placed on PSs, and the workers are responsible for the computation. On the one hand, the high concurrency mechanism activates the fragmentary resources in the training cluster by deploying hundreds of workers. On the other hand, the asynchronous training brings in gradient staleness, which occurs when the gradient is calculated based on the parameters of an old version and applied to the parameters of a new version.
3.2 Observations and Insights within a Shared Training Cluster
We investigate the training workloads of recommendation models from shared training cluster to observe the obstacles and necessities of switching training modes.
3

---

Sync. ... BSP-1K BSP-2K BSP-4K ... BSP-6K
0.25
10-
Proportion
Proportion
0.12
10
0
10-
6
12
80 Occurences per 100 batches
20
L2-norm of Gradients (x10-))
Figure 3: The distribution of L2-norm of gra- dients from the synchronous training and BSP with various size of aggregation.
Figure 4: The skewed distribution of ID occur- rences across batches, reflecting the frequency that an ID gets updated.
Observation 1: Cluster status determines the performance of training modes. Figure illlstrates the average CPU utilization within real shared cluster, and the corresponding samples/queries per second (QPS) of YouTubeDNN 66] model by the synchronous and asynchronous training mode. The utilization and QPS are normalized by their maximal value, respectively. When the cluster is relatively vacant, models trained in the synchronous mode can fully occupy the hardware resources, satisfying HPC conditions and presenting high efficiency. When there are plenty of heterogeneous workloads in the cluster, slow workers dominate the training speed, making the asynchronous training mode run much faster than the synchronous mode. We also implement two approaches of local all-reduce Since the status each device the cluster constantly changing, the local all-reduce-based mode would not work well when confronting resource shortages.
Observation 2: Directly switching training mode brings sudden drop on accuracy. We run DeepFM (33 over Criteo-4GB [15] (few parameters, fast convergence) and YouTubeDNN on Private dataset (trillions of parameters, slow convergence) the shared cluster. We tune the hyper-parameten from scratch for the best model accuracy of both asynchronous and synchronous mode, and denote the two sets of hyper-parameterr as set and set S, respectively. After training in one training mode, we evaluate the tendency of the training AUC after switching to the other training mode with set A or set S. Figure 2 illustrates that after switching from the synchronous mode to the asynchronous mode, the AUC encounters sudden drop and even decreases to 0.5. The AUC drop also appears in the opposite-side switching, indicating that this condition is irrelevant to whether the model had been converged. These observations imply that directly switching the training mode requires heavy effort in re-tuning the minima owing to the difference in batch size, learning rate and many other factors, which have already received in-depth theoretical research 2 17). We provide theoretical analysis to explain the sudden drop in Appendix D.
Inherently, training modes would lead to different convergence
We then probe into the insights from asynchronously training recommendation models.
Insight I: Distribution of the gradient values is related to the aggregated batch size. We attempt to investigate the reason for observation 2 from the gradient aspect. We implement asynchronous bulk synchronous parallel (BSP) on the YouTubeDNN recommendation task, which asynchronously aggregates K gradients from workers before applying the values to the parameters. Here, we set K to 100, the same as the number of workers. Besides, we compare the synchronous training in 6.4K local batch size (64 workers). Figure 3 plots the distribution of the 1.2-norm of gradient values from the synchronous training and BSP with various local batch sizes. It is evident that the batch size determines the mean and variance of the distribution. The distribution of BSP resembles synchronous training when the aggregation size is similar (i.e., BSP-4K). The result suggests that the same aggregation size could lead to a similar distribution of gradient values. However, there still a gap in model accuracy after equalizing the global batch size between the asynchronous training and the synchronous training, mainly induced by gradient staleness.
Insight 2: The gradient staleness imposes different impact on the embedding parameters and the dense parameters. Due to the skewed distribution, most IDs would merely appear in a small number of batches, as depicted in Fig. 4 It means that in the recommendation models, only a tiny portion of IDs would be involved every single batch, and the embedding parameters are less
'SwarmAdam is variant of SwarmGGD 26 with Adam optimizer. It is uncommon use SwarmAdam and Prague (25 in recommendation models they may lead accuracy loss which is not tolerable in the business.

---

Dense Parameters Embedding Parameters
M gradients in gradient buffer would be aggregated
Gradient Buffer
Gradlent Buffer
Staleees
2
D
M
token
token
gradient
gradient
(partition)
(partition)
tolien gradient
Update
Updat
PS
PS
Data List (Size Q)
worker sends token and gradient to PS
do d, d2
d
data and token
sent to workes
(Siee Q1
Worker
Worker
Worker 2
Worker N
0
Size
Size
Figure Illustration of the token-control mechanism in GBA: every M gradients would be aggregated in the buffer before the PSs apply them to the parameters; workers report gradients to the PSs along with token indicating the degree of data staleness.
frequently updated than the dense parameters. Therefore, the embedding parameters tend to more robust on the gradient staleness than the dense parameters (for example, considering a worker in training, there could be five updates for the dense parameters, yet only two updates for the embedding of specific ID).
4 Global Batch based Gradients Aggregation
4.1 Training Recommendation Models with GBA
Switching the distributed training mode for recommendation models should get rid of tuning the We introduce the concept of global batch size, which is defined as the actual batch size when gradients are aggregated and applied, and propose GBA for the tuning-free free switching. We denote the local batch size, i.e., the actual batch size on each worker and the number of workers, as B. and N, for the synchronous training. Ba and N₂ for the asynchronous training. Then, the global batch size in synchronous training, denoted by G., can be calculated as B, x N.. Following Insight 1, GBA remains the global batch size unchanged when switch the distributed training model from synchronous training to asynchronous training. For each step, all the dense parameters would be updated, and only small number of embedding parameters would be updated. Then the dense parameters and embedding parameters obtain different gradient staleness during training. Hence, define the data staleness the unified staleness in training recommendation models. The data staleness describes the gap between the global step when the worker begins to ingest a data batch and the global step when the calculated gradient is applied. Obviously, the data staleness in the synchronous training mode is constantly zero. Based on data staleness, we implement GBA by token-control mechanism the PS architecture cope with the sparsity and the dynamic cluster status.
Figure 5 illustrates the architecture of the proposed token-control mechanism. Over the canonical PS. we prepare a queue called data list to arrange the data (addresses) by batches. Given a dataset D, suppose can split into Q batches of size Ba, denoted by D = (do, dj...., dQ-1). Mcanwhile. we establish another queue called token list to yield the token of each individual batch. The token list contains Q tokens, denoted by (fo, 11, ....... tQ) each attached to one batch in the data list to indicate the global step when this batch is sent to a worker. The token value starts from zero, and each token value repeats M times in the token list. Here, M is the number of batches we use to aggregate gradients. Under this setting, we can deduce that there will be K Q gradient updates during the training. Then we set t; = 1 Vi (0, 10.1.....4 Q 1) to ensure that the token list yields the token value in ascending order. Apart from the two queues, we also employ a gradient buffer to receive the gradients calculated by the workers with the corresponding tokens of the gradients. To be consistent with the tokens, the capacity of the gradient buffer is set to M, and therefore the PSs would aggregate M gradients before applying them the variables. Note that each PS maintains an individual gradient buffer to deal with the gradients of the corresponding partitions of the variables.
5

---

During the training process, worker would pull the parameters from PS, batch from the data list, and a token from the token list simultaneously before ingesting the data and computing the gradient locally. When worker completes calculating the gradient of batch, the gradient and the corresponding token are sent to the gradient buffer on PS. Then, the worker immediately proceeds to work on the next batch. In this way, the fast workers can keep working without waiting for the slow ones. When the gradient buffer reaches the capacity of M pairs of gradients and tokens, all the gradients are aggregated to apply once, and at the same time, the buffer will be cleared. This is what we call finishing global step, and thereby the global batch size in GBA can be calculated as Ga Bg M. According to the design, we aim to keep the global batch size consistent in switching, that is. G. Ga- Hence, we can set the size of the gradient buffer to be M B, XNe, We would use B. M workers in GBA, i.e., N₂ M, to avoid the intrinsic gradient staleness led by the inconsistency between the number of workers and the number of batches to aggregate.
At the update of global step k. denote (m,, k) the m-th token in the gradient buffer. When we aggregate the gradients in the gradient buffer, we shall decay the gradients that suffer from severe staleness. GBA could employ different staleness decay strategies to mitigate the negative impact from the staleness according to the token index, and in this work we define as:
[0, k - T-, k) >
f(T(m, k))
11 k - T(m, k) S.
where is the threshold of tolerance. If f (T k), k) = 0, we exclude the m-th gradient in the buffer due to the severe staleness; otherwise, we aggregate the gradient. As we can see, tokens help identify whether the corresponding gradients are stale and how many stale steps are behind the current global step. In this case, although the token is designed over the data staleness, the negative impact from the canonical gradient staleness can also be mitigated.
4.2 Convergence Analysis
We have seen much research the convergence analysis of the synchronous and asynchronous training. Following the assumptions and convergence analysis in Dutta et al. M7 the expectation of error after & steps of gradient updates in the synchronous training can be deduced by:
Loo 2c.N B
B[F(w*)] F*
+1- (F(w)))) F
()-nc)
(2)
2o.N B
where Wi: denotes the parameter in step k. 17 denotes learning rate, is the Lipschitz. constant and a denotes the variance of gradients. F(w) is the empirical risk function that is strongly convex with parameter c. [[:)] - F is the expected gap of the risk function from its optimal value, used as gLa² the error after & steps. As mentioned in Eqn. (2. The first term in the right, i.c. 2.j, would be
the error floor, and (1 - v/c) is the decay rate. The proposed GBA is derived upon the asynchronous gradient aggregation. We assume that, for some 7 S 1,
(E]IVF((W)) V (F(
(3)
Here, 7 is a measure of gradients impact induced by the staleness; smaller value of y indicates that staleness makes less accuracy deterioration of the model. Besides, Ç indicates the average probability that any parameter in the model would be both updated in step and step T T(m, k).. Intuitively, C would be far below 1 in the recommendation models due to the strong sparsity. Then, the error of GBA after steps of aggregated updates would become (Appendix A presents the proof):
nLO²
TLo22
E [[F() - F+ < MB
+ +1 (1- (11m"c))(B(F(())) - F
(4)
207' Cer'MMn B.
where 1 and pa is lower bound on the conditional probability that the token equals to the global step, i.e., T(m, k) = k. Equation (4) proves the convergence of GBA. Considering the error floors of Eqn. (2) and Eqn. 44) M x B, should be set close to N, x B, to make GBA tuning-free. is exactly the global batch size we use in GBA, consistent with our main idea of keeping global batch size unchanged. Recall that with the embedding parameters, Ç < makes y lower than the training of general CV or NLP models. Consequently, the error floor remains low in GBA.

---

Syne.
GBA
Hop-BW
Hop-BS -
BSP
Asyne.
0.80
0.658
0.79
Last day
Average
+3.0211
+3.0002
AUC
Hop-BW
4.0012
4.0048
-0.0055
Hop 88
0.0215
0.0878
0.785
BBP
-0.0017
0.0045
-0.034
0.64
0.77
Asyne
-01553
4.
Test day
13
23
22
(a) Crite (from Sync.)
(b) Alimama (from Sync.)
(c) Private (from Sync.)
(g) Diff. from GBA (from Sync.)
0.80
0.665
0.79
doy
Last
Average
Symo:
+0.0011
+0.0001
10.0002
AUC
AUC
Hop-BWV
-0:0000
-9.0121
-0.000
Hop-B8
-00018
-0.0005
4.0009
0.79
BSP
-0.079
-0.0221
-0.0040
0.638
0.77p
Async.
-0.0000
-0.0980
-0.0575
Test day
23
(d) Criteo (to Sync.)
(e) Aliman (to Sync.)
(f) Private (to Sync.)
(h) Diff. from GBA (to Sync.)
Figure 6: The AUC tendencies on the test days of the three datasets after inheriting base model: (a-c) from the synchronous training modes and switching to the compared training modes; (d-f) from the compared training modes and switching to the synchronous training modes; (g-h) AUC difference between GBA and the other training modes after switching from/to synchronous training.
5
Evaluation
5.1 Settings
Table 1: Settings of the three continual recommendation tasks by the compared training modes.
Task
Model description
Data parts
Sample per day
Optimizer
rate
Learming
of workers
Local size
batch
Private
hyper-param.
(DepPPM)
Criteo
45B parais. 16 aog. dim.
19M(440K) FLOPS
12 days (base)
days (eval)
190M
Adam (Others)
Adagrad (Asyac.)
0.006 (Asyac.)
0,0011 (Others)
32 (Sync.)
(Others)
SK (Asyne.))
40K (Othen)
12 (GBA)
Hop BS (b =b)
BSP (b))20)
Hop-BW (bs)20)
GIA
(DIEN)
Alimama
112Mgx3K) FLOPS 160B params. mgg
days (base) days (eval)
90M
Adiagnad (Aayne.) 0.008 (Ayync)) 32 (Syne-) Adam (Others)
0.0015 (Others)
128 (Others)
0.75K (GBA)
3K (Ciliios)
IK (Async.) )
BSP (1=)20)
Hop-BS (b-2))
Hop-BW (la=20)
GBA (c=1)
Private
(YoNTube
24 avg. dim.
746Mix6 4K) FLOPS
L99 paramn.
14 days (base)
(exal)
2B
Adam (Others)
Adagrad (Asyac.)
0.001 (Asyac.)
0.0006 (Others)
64 (Sync.)
(Others)
6.1K (Others)
IK (GBA)
JK (Asyne)) )
BSP (bu)50)
Hop-BS th (h--2)
Hop-BW (by=100)
GBA (1-4)
We conduct systematical evaluations to examine the performance of GBA and make a fine-grained analysis. The evaluations involve three industrial-scale dataset (14), we implement DeepFM, where the recommendation tasks: On the Criteo- ITB on Criteo-4GB (AUC 0.8043) are utilized; On the Alimama dataset D we implement DIEN 34 and use the recommended saper-parameters in the original design; 3) On the Private dataset, we implement YouTubeeDNN, and
we tume the best pyper-paramette Tables. Detailed information on the dataset and the models are listed in Tab. [ We imitate the continual training without changing the hyper-parametee to the models, and ensure a similar cluster status for all evaluations. Inheriting from pre-trained checkpoint, we repeatedly train on the data of every day and evaluate the data of the subsequent day. The training cluster is equipped with a Tesla-V 1000S- GPU and Skylake CPU. We focus on AUC as the accuracy metric and global/local QPS (QPS of all/single workers) as the efficiency metric.
The models are implemented in DeepRec [5] with the expandable
We select several state-of- fftte-ofthhealte the-an art PS- based training modes: Bounded staleness (Hop-BS) restricts the maximal differences of gradient version between the fastest and the slowest workers, controlled by b;; Bulk synchronous parallel (BSP) aggregates pre-set number by of gradients when applying gradients to the parameters, regardless of the gradient version; Backup worker (Hop-BW) ignores the pre-set number by of gradients from the slowest workers during each gradient aggregation. We enumerate the specialized hyper-parameters of each training mode and record the statistics when reaching its best AUC.
7

---

Table 2: Global QPS of the compared training mode on the three tasks.
Syne.
Asyne.
Hop-BS
BSP
Hop-BW
GBA
Criteo
1,436K(1224K) 182K(4-52K)
3.253K(184K)
2.227
3,247K(193K)
3,240K 2.559K(1294K)
Alimama
403K(=33K)
217K(=65K)
403K(=33K)
288K(148K)
399K(135K)
Private
43K(H2IK)
90K 15K)
29
88K(+17K)
66K(+14K)
17K(
Table 3: Fine-grained analysis between GBA and the other training modes.
Local QPS
AUC
# drop
Avg. grad. staleness (max)
Async
GBA
Sync.
GBA
Hop-BW
GBA
Hop-BS
GBA
BSP
78K(=23K)
74K(+25K)
0,7864
0,7864
300K
1,454
0.06 (2)
0.21 (11)
2.61 (12)
95K(=15K) 87K(119K)
0,7864
0,7866
300K
898
0.04 (2)
0.15 (11)
1.92 (12)
99K(=12K)
18K(112K)
0,7864
0,7865
300K
786
0.03 (2)
0.12
1.62 (10)
5.2 Performance of Training Modes
We first examine the performance of GBA. Tigure[aa-o records the AUC tendencies after switching from synchronous to the other training modes over the three recommendation tasks. We mainly focus on the AUC at the first day and the last day, as well as the average AUC scores throughout the datasets. Although Hop-BW eliminates staleness, the ignorance of a large volume of data makes it perform the worst (also taken as evidence why we tend not to use local all-reduce in training recommender systems). The manipulation of global batch size contributes the best performance on both sides of switching. Compared to the best baselines (i.e., BW),BW), GBA imprrves AUC by at least 0.2% on average over the three datasets, which has the potential increase by 1% revenue in the real-we real-world business. Mcanwhile, after switching from synchronous training, GBA obtains immediate good accuracy (AUC at the first day), while there are explicit re-convergence on the other training modes, as depicted in Figure g)
Figure/ojd-.hh illustrates the AUCs of these training modes after switching to synchronous train- ing. We can see that GBA tends to obtain at least equal accuracy to the continuous synchronous training without switching. On the contrary, the models inherited from the other baselines require consuming more data to reach the desired accuracy of the synchronous training. The tendency of the AUC gaps between the synchronous training and the compared training modes reflects that the parameters trained by GBA are the most compatible with the synchronous training. It verifies that switching from GBA synchronous training also tuning-free.
We collect metrics of the training efficiency during the above experiments, and report their global QPS in Tab. 2 The results reveal that GBA performs similarly the asynchronous training. Although Hop-BS works better than BSP and Hop-BW in accuracy, it struggles to deal with the slow workers. indicates that when facing a resource shortage in the shared cluster, GBA can provide similar accuracy with synchronous training mode, while running as fast as the asynchronous training mode.
5.3 Fine-grained Analysis
We further probe into the performance of GBA. Here, we take the recommendation task on the Private dataset (the most complex model) as example, switching from the synchronous mode to GBA.
We first analyze how the cluster status affects the performance of GBA. The experiments are repeated in the cluster during different periods of day. We collect AUC, QPS, average gradient staleness on the dense parameters (for fair comparison among the basclines), and the number of excluded batches, as shown in Tab. From the results, we can infer that GBA properly finds the balance between the staleness and the excluded data (as defined in Eqn. (T). i.c., excluding fewer data compared to Hop-BW and suppressing the staleness to the same level of Hop-BS. GBA also shows strong robustness on the dynamic cluster status, obtaining stable performance on AUC.
We then examine the impact of the batch sizes in GBA. Figure depicts the average AUC score and the global QPS when we modify the local batch size (the number of workers is thereby changed) and keep the global batch size unchanged. Considering the hardware limitation communication overhead on PS, we vary the number of workers from 100 to 800. We can see a steady state of the AUC score (i.c., absolute difference less than 10 10-1), while the training achieves
worker and the
8

---

0.7866
0.7865
152g
AUG
AUG
86
Global
0.78
200
400
250
500
1K
2K
# of workers
Local batch size
Figure 7: The average AUC on the 8-day test sets and the training efficiency via GBA, varying the number of workers while maintaining the global batch size.
Figure 8: The range of AUC the 8- day test sets via GBA of 400 workers, varying the local batch size.
a significant efficiency boost when using more workers. can thereby be inferred that GBA has a good capability of scaling out. Besides, we fix the number of workers to 400 and change the local batch size for each worker, which means that the global batch size would differ. As shown in Fig. X the inconsistent global batch size with the synchronous training makes the training encounter lower AUC scores after switching. Although the larger global batch size may have the potential to achieve better accuracy (i.e., owing the stable and accurate gradient), the experiment indicates the model would hardly reach its best accuracy without tuning. These results verify that using the same global batch size in GBA in the synchronous training is necessary get rid of tuning when switching the training mode.
Conclusion
A tuning-free switching approach is demanded to take full advantage of the synchronous and asynchronous training, which can improve the training efficiency in shared cluster. We raise insights from the investigation and the gradient staleness are two main reasons to fail the switching regarding the model accuracy. Then GBA training mode is proposed for asynchronously training recommendation models via aggregating gradients by the global batch size. GBA enables switching between synchronous training and asynchronous training of the continual learning tasks with the accuracy and efficiency guarantees. With GBA, users can freely switch the training modes according to the status of the shared training clusters, without tuning hyper-pane peeee-aaameeers. GBA is implemented through token-control mechanism to ensure that the faster worker should contribute more gradients to the aggregation while the gradients from the straggling workers would be decayed. Evaluations of three representative continual training tasks of recommender systems reveal that GBA achieves similar accuracy with the synchronous training. while resembling the efficiency of the canonical asynchronous training. Currently, GBA requires the users to select the training mode according to their own judgment on the cluster status. In the future, we will attempt make GBA be adaptive the cluster status. The guidelines of automatic switching would be derived from more analyses upon the training trace logs. It could be formulated as an optimization problem under many control factors including but not limited the overall QPS, training cost, and task scheduling with priority.
the production training workloads that the inconsistent batch size
References
[1] B. Acun, M. Murphy, X. Wang. Nie, C.-. Wu, and K. Hazelwood. Understanding training efficiency of deep learning recommendation models at scale. In 202/ IEEE International Symposium on High-Performance Computer Architecture (HPCA), pages 802-814. IEEE, 2021.
[2] S. Barkai, I. Hakimi, and A. Schuster. Gap aware mitigation of gradient staleness. In ICLR, 2020.
[3] Y. Chen, J. Wang, Y. Lu, Han, Z. Lv, X. Min, H. Cai, W. Zhang, H. Fan, Li, al. Fangorn: adaptive execution framework for heterogeneous workloads on shared clusters. Proceedings of the VLDB Endowment, 14( (4(12))2972-2985, 2021.
[4] H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chandra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, M. Ispir, et al. Wide & deep learning for recommender systems. Proceedings of the 1st workshop deep learning for recommender systems, pages 7-10, 2016.

---

[5] D. community. Deeprec. https:// http://hhwbbcoaaaaaaaaaaaaeeeeee 2022.5.11.
[6] P. Covington, J. Adams, and E. Sargin. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems, pages 191-198, 2016.
[7] S. Dutta, G. Joshi. S. Ghosh, P. Dube, and Nagpurkar. Slow and stale gradients can win the race: Error-runtime trade-offs in distributed SGD. In International Conference on Artificial Intelligence and Statistics, pages 803-812. PMLR, 2018.
[8] S. Eliad, Hakimi, A. De Jagger, M. Silberstein, and A. Schuster. Fine-tuning giant neural networks on commodity hardware with automatic pipeline model parallelism. In 2021 USENIX Annual Technical Conference (USENIX ATC 2/), pages 381-396, 2021.
[9] A. Group. Ad display/click data on taobao.com. dataDetail ?dattal 566lan en us under CC 4.0, visited on 2022.5.11.
https: aliyunn com /at/sattet//
[10] Y. Guo, M. Liu, T. Yang, and T. Rosing. Improved schemes for episodic memory-base lifelong learning. Advances in Neural Information Processing Systems, 33:1023-1033, 2020.
[11] He, T. Liu, and Tao. Control batch size and learning rate to generalize well: Theoretical and empirical evidence. Advances in Neural Information Processing Systems, 32, 2019.
[12] Q. Ho. J. Cipar, H. Cui, S. Lee, J. K. Kim, P. B. Gibbons, G. A. Gibson, G. Ganger, and E. P. Xing. More effective distributed ml via stale synchronous parallel parameter server. Advances in neural information processing systems, 26, 2013.
[13] J. Hron, K. Krauth, M. Jordan, and N. Kilbertus. On component interactions in two-stage recommender systems. Advances Neural Information Processing Systems, 34, 2021.
[14] C. Inc. Critco 1tb click logs dataset.
https://atis criteo com/ visited on 2022.5.11.
[15] C. Inc. Kaggle display advertising challenge dataset. http: vaggle-display-Bne adverriiiii
/lbs/ criteo. www 2014/02/
visited on 2020.1.10,
[16] Y. Jiang, Y. Zhu, C. Lan, B. Yi, Y. Cui, and Guo. A unified architecture for accelerating distributed (DNN} training in heterogeneous (GPU/CPU) clusters. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), pages 463 479, 2020.
[17] N. S. Keskar, D. Mudigere, J. Nocedal, M. Smelyanskiy, and P. T. Tang. On large-batch training for deep learning: Generalization gap and sharp minima. In ICLR, 2017.
[18] S. Kim, G.-I. Yu, H. Park. S. Cho, E. Jeong, H. Ha. S. Lee, J. S. Jeong, and B. Chun. Parallax: Sparsity-aware data parallel training of deep neural networks. In Proceedings of the Fourteenth EuroSys Conference 2019, pages 1-15, 2019.
[19] A. Kumar, A. Beutel, Q. Ho, and E. Xing. Fugue: Slow-worker-agnosti distributed learning for big models on big data. In Artificial Intelligence and Statisties, pages 531-539. PMLR, 2014.
[20] L. Li, K. Jamieson, A. Rostamizadeh, E. Gonina, Ben-Tzur, M. Hardt, B. Recht, and A. Talwalkar. A system for massively parallel hyperparameter tuning. Proceedings of Machine Learning and Systems, 2:230 246, 2020.
[21] M. Li, D. G. Andersen, W. Park, A. J. Smola, A. Ahmed, Josifovski, J. Long, Shekita, and B.-Y. Su. Scaling distributed machine learning with the parameter server. In /1th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14), pages 583-598, 2014.
[22] S. Li, O. Mangoubi, I.. Xu, and T. Guo. Sync-switch: Hybrid parameter synchronization for distributed deep learning. In 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS), pages 528-538. IEEF, 2021.
[23] Lian, X. Zhou, F. Zhang, Z. Chen, X. Xie, and G. Sun. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1754-1763, 2018.
10

---

[24] Q. Luo, J. Lin, Y. Zhuo, and X. Qian. Hop: Heterogeneity-awatn aware decentralized training. In Proceedings of the Eventy-Fourrte International Conference on Architectural Support for Programming Languages and Operating Systems, pages 893-907, 2019.
[25] Q. Luo, J. He, Y. Zhuo, and X. Qian. Prague:
High-performance heterogeneity-awarr asyn- chronous decentralized training. Proceedings of the Twenty-Fiith International Conference on Architectural Support for Programming Languages and Operating Systems, pages 401-416, 2020.
[26] G. Nadiradze, A. Sabour, P. Davies, S. Li, and D. Alistarh. Asynchronous decentralized sgd with quantized and local updates. Advances in Neural Information Processing Systems, 34, 2021.
[27] J. Wu, W. Hu, H. Xiong, J. Huan, V. Braverman, and Z. Zhu. On the noisy gradient descent that generalizes as sgd. In International Conference on Machine Learning, pages 10367-10376. PMLR, 2020.
[28] B. Ying. K. Yuan, Y. Chen, H. Hu, P. Pan, and W. Yin. Exponential graph is provably efficient for decentralized deep training. Advances in Neural Information Processing Systems, 34, 2021.
[29] Y. You, J. Li, S. Reddi, J. Hseu, S. Kumar, S. Bhojanapalli, X. Song. J. Demmel, K. Keutzer, and C.-J. Hsieh. Large batch optimization for deep learning: Training bert in 76 minutes. In ICLR, 2020.
[30] H. Yu and R. Jin. On the computation and communication complexity of parallel sgd with dynamic batch sizes for stochastic non-convex optimization. In International Conference on Machine Learning, pages 7174-7183. PMLR, 2019.
[31] Y. Zhang, L. Chen, S. Yang, M. Yuan, H. Yi, J. Zhang, J. Wang. J. Dong, Y. Xu, Y. Song, al. Picasso: Unleashing the potential of gpu-centrie training for wide-and-deg, recommender systems. In 2022 IEEE International Conference on Data Engineering (ICDE), 2022.
[32] S. Zheng, Q. Meng. T. Wang, W. Chen, N. Yu, Z.-M. Ma, and T.-Y. Liu. Asynchronous stochastic gradient descent with delay compensation. In International Conference on Machine Learning, pages 4120-4122 PMLR, 2017.
[33] G. Zhou, X. Zhu, C. Song, Fan, H. Zhu, X. Ma, Yan, J. Jin, H. Li, and K. Gai. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD international conference knowledge discovery & data mining, pages 1059-1068, 2018
[34] G. Zhou, N. Mou, Y. Fan, Q. Pi, Bian, C. Zhou, X. Zhu, and K. Gai. Deep interest evolution network for click-througe rate prediction. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 5941-5948, 2019.
[35] Y. Zhou, Y. Yu, W. Dai, Y. Liang, and E. Xing. On convergence model parallel proximal gradient algorithm for stale synchronous parallel system. In Artificial Intelligence and Statistics, pages 713-722, PMLR, 2016

---

Paper Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Yes.
(b) Have you read the ethics review guidelines and ensured that your paper conforms to them? Yes.
(c) Did you discuss any potential negative societal impacts of your work?
Not applicable.
(d) Did you describe the limitations of your work?
Yes. We mentioned in the conclusion that the have manually select the training mode based on their own judgement of the status the training cluster.
you are including theoretical results...
Did you state the full set of assumptions of all theoretical results?
Yes.
Did you include complete proofs of all theoretical results?
Yes. The theories in this paper can be deduced in a straightforward manner. We provide the full proof' in the appendix.
If you ran experiments...
Did you include the code, data, and instructions needed reproduce the main experimental results (either in the supplemental material or as URL)?
The
code would be released
the HybridBackend project
Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Yes, as listed in Table 1
Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
No. The status of the training cluster and the asynchronous training manner introduce the randomness to the accuracy. The experiments were conducted on data of many days in the continual learning to show that the results convincing.
Did you include the amount compute and the type of resources used (e.g., type GPUs, internal cluster, or cloud provider)?
Yes, as listed in Table
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
If your work uses existing assets, did you cite the ereators? Yes.
Did you mention the license of the assets?
Not applicable. We provide the URLs of three out of the four datasets (Criteo-4GB, Criteo-1TH ITB, Alimama, Private) used in this paper and the last visited time.
Did you include any new assets either in the supplemental material or as a URL?
12

---

No.
Did you discuss whether and how consent obtained from people whose data you' 're ing/curating?
Not applicable.
Did you discuss whether the data you are using/eurating contains personally identifiable information or offensive content?
Not applicable. The data used in this paper does not contain personally identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects...
Not applicable.
13


Reviews:
[{"review_id": "j4kLHL7su2p", "reviewer": "Reviewer_JyKb", "summary": "This paper proposes a mechanism called global batch gradient aggregation (GBA) on parameter server architectures that enables effective switching between synchronous and asynchronous training of deep learning-based recommendation systems. ", "questions": "Figure 6 is hard to track, would it be possible to list the final AUC of each model for each task in a table?", "limitations": "Not applicable. ", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\n- The discovery of the phenomenon that directly switching between synchronous and asynchronous training of deep learning-based recommendation models leads to a significant drop in performances is interesting.  \n\nWeaknesses:\n\n- The explanation of the phenomenon is lack of technique depth. The insights presented in Section 3.2 are intuitive rather than analytical. It would make the paper stronger if there were some formal theoretical analysis on the drop in performance.  \n\n- The performance boost is marginal; for example, in Figure 6, it is very difficult to understand the performance gain w.r.t. the proposed method. To be more concrete, it seems that, for the Criteo dataset, synchronous training achieves the optimal AUC much ealier than GBA. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "PW-EByxcA13", "reviewer": "Reviewer_S16x", "summary": "The paper proposes a novel algorithm to switch between two common distributed training modes (abbreviated as PS and AR in the paper) without sacrificing accuracy and performance while also not tuning hyperparameters. The paper identifies three key observations that govern performance and accuracy and based on these, proposes a global batch aggregation scheme that mimics the global gradients in synchronous training to switch between PS to AR without any loss in accuracy. The paper shows a convergence analysis of the algorithm and evaluates its performance on three different tasks showing it has better performance and accuracy overall.\n", "questions": "- How does the performance vary with the number of workers? Since it relies on global batch aggregation, there may be some dependence on the number of workers?", "limitations": "The authors discussed the limitations of the work in the conclusion section.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The paper proposes a novel algorithm to effectively switch between two training modes without tuning hyper parameters which is costly.\n- he paper theoretically proves convergence.\n- The proposed algorithm performs better than state-of-the-art recommender training schemes on three different tasks.\n\nWeakness:\n- The paper should provide details on when to initiate switching based on cluster status and ablation studies that show how the model performs for different cluster statuses. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "6xJHTFesvss", "reviewer": "Reviewer_pSV7", "summary": "The authors of the paper \"GBA: A Tuning-free Approach to Switch between Synchronous and Asynchronous Training for Recommendation Models\" propose a new training method for the recommender systems that enables switching between two different modes: synchronous and asynchronous training benefiting from both approaches and allowing for continuous learning maintaining the same accuracy and efficiency without hyperparameter tuning. Users of such a system can select a suitable training mode according to their judgment of cluster workload. ", "questions": "I have only a couple of questions:\n* In Section 5.2 you write that \"We can see\n261 that the low gradient staleness helps Hop-BS receive a better accuracy than BSP\" (line 260-261). When I look at Figure 6 (a-c), I observe that HOP-BS performs the worst in the (a) setting dropping right after the beginning (Test day 14) while in other settings (b,c) it is not possible to distinguish any difference in AUC performance. BSP (dark blue) is performing clearly better. Next, you write that HOP-BW performs the worst, while I observe a relatively normal performance. I assume HOP-BS and HOP-BW got mixed up here. However, it doesn't explain that you say that GBA improves the AUC on (a) and (c) by at least 0.2% but I can't see this improvement in (a) and I don't think on (c) it gets to 0.2% difference. I would suggest double-checking this part as it confuses readers at this point. \n* In addition, I think Figure 6 description states (d-e) while it is supposed to be (d-f)\n* Another suggestion would be to highlight in each of the tables the best-performing metrics as you use quite a few of them and it becomes more difficult to check. This is a minor suggestion but can potentially improve readability. \n\n\n\n", "limitations": "The limitations were not explicitly discussed with the only suggestion for future work to include automatic switching between the training modes. I think the authors might want to consider unpacking the limitations of this work a bit more. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Post Rebuttal comments: After the changes provided by the authors, I've updated my score to accept\n\n------------------------------------------\nOriginality\n\nThe paper solves a problem that, though specific to recommender systems, can be very beneficial in a practical setting and valuable for all the industries attempting to train large recommendation systems under imperfect computing constraints. I would judge that the originality of the paper is high as, according to the authors, nobody has attempted to provide a hyperparameter tuning-free approach to the switching between synchronous and asynchronous training modes. \n\nQuality\n\nThe quality of this paper is also high - it provides a clear overview, and outlines several important insights and observations on the performance of the training modes that help to determine and address the gaps in the current work.  This provides clarity on the choices and explains the conducted experiments. \n\nClarity\n\nThe paper is well-structured and well-written. I enjoyed reading it. The only problem that undermines the clarity is Section 5.2 and more specifically, Figure 6. I will outline my confusion in the Questions section below. In summary, it seems that the methods in the figure and their performance do not correspond to the description in the text. \n\nSignificance\n\nI find that this work might have a medium impact in addressing important challenges specific to the recommender systems' properties in the industrial setting. I believe that this impact might translate to better resource utilization and, therefore, cost savings.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept