Original: Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing
Revised: Stimulative Training of Residual Networks: A Social Psychology Perspective of Loafing

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Residual networks have shown great success and become indispensable in today’sdeep models.
Revised: Residual networks have shown great success and become indispensable in today’sdeep models.

Original: In this work, we aim to re-investigate the training process of residualnetworks from a novel social psychology perspective of loafing, and further proposea new training strategy to strengthen the performance of residual networks.
Revised: In this work, we aim to re-investigate the training process of residualnetworks from a novel social psychology perspective of loafing, and further proposea new training strategy to strengthen the performance of residual networks.

Original: Asresidual networks can be viewed as ensembles of relatively shallow networks (i.e.,unraveled view ) in prior works, we also start from such view and consider thatthe final performance of a residual network is co-determined by a group of subnetworks.
Revised: Asresidual networks can be viewed as ensembles of relatively shallow networks (i.e.,unraveled view ) in prior works, we also start from such view and consider thatthe final performance of a residual network is co-determined by a group of subnetworks.

Original: Inspired by the social loafing problem of social psychology, we findthat residual networks invariably suffer from similar problem, where sub-networksin a residual network are prone to exert less effort when working as part of thegroup compared to working alone.
Revised: Inspired by the social loafing problem of social psychology, we findthat residual networks invariably suffer from similar problem, where sub-networksin a residual network are prone to exert less effort when working as part of thegroup compared to working alone.

Original: We define this previously overlooked problemas network loafing .
Revised: We define this previously overlooked problemas network loafing .

Original: As social loafing will ultimately cause the low individualproductivity and the reduced overall performance, network loafing will also hinderthe performance of a given residual network and its sub-networks.
Revised: As social loafing will ultimately cause the low individualproductivity and the reduced overall performance, network loafing will also hinderthe performance of a given residual network and its sub-networks.

Original: Referring to thesolutions of social psychology, we propose stimulative training , which randomlysamples a residual sub-network and calculates the KL-divergence loss between thesampled sub-network and the given residual network, to act as extra supervisionfor sub-networks and make the overall goal consistent.
Revised: Referring to thesolutions of social psychology, we propose stimulative training , which randomlysamples a residual sub-network and calculates the KL-divergence loss between thesampled sub-network and the given residual network, to act as extra supervisionfor sub-networks and make the overall goal consistent.

Original: Comprehensive empiricalresults and theoretical analyses verify that stimulative training can well handle theloafing problem, and improve the performance of a residual network by improvingthe performance of its sub-networks.
Revised: Comprehensive empiricalresults and theoretical analyses verify that stimulative training can well handle theloafing problem, and improve the performance of a residual network by improvingthe performance of its sub-networks.

Original:  Introduction
Revised:  Introduction

Original: Since ResNet [1] wins the first place at the ILSVRC-2015 competition, simple-but-effective residualconnections are applied in various deep networks, such as CNN, MLP and transformer.
Revised: Since ResNet [1] wins the first place at the ILSVRC-2015 competition, simple-but-effective residualconnections are applied in various deep networks, such as CNN, MLP and transformer.

Original: To explorethe secrets behind the success of residual networks, numerous studies have been proposed.
Revised: To explorethe secrets behind the success of residual networks, numerous studies have been proposed.

Original: He et.al [1] exploit the residual structure to avoid the performance degradation of deep networks.
Revised: He et.al [1] exploit the residual structure to avoid the performance degradation of deep networks.

Original: Further,
Revised: Further,

Original: He et.
Revised: He et.

Original: al [2] consider that such a structure can transfer any low level features to high level layers inforward propagation and directly transmit the gradients from deep to shallow layers in backwardpropagation.
Revised: al [2] consider that such a structure can transfer any low level features to high level layers inforward propagation and directly transmit the gradients from deep to shallow layers in backwardpropagation.

Original: Balduzzi et.
Revised: Balduzzi et.

Original: al [3] find that residual networks can alleviate the shattered gradientsproblem that gradients resemble white noise.
Revised: al [3] find that residual networks can alleviate the shattered gradientsproblem that gradients resemble white noise.

Original: In addition, Veit et.
Revised: In addition, Veit et.

Original: al
Revised: al

Original: [4] experimentally verify thatresidual networks can be seen as a collection of numerous networks of different lengths, namelyunraveled view .
Revised: [4] experimentally verify thatresidual networks can be seen as a collection of numerous networks of different lengths, namelyunraveled view .

Original: Following this view, Sun et.
Revised: Following this view, Sun et.

Original: al [5] further attribute the success of residual networksto shallow sub-networks, which may correspond to the low-degree term when regarding the neuralnetwork as a polynomial function.
Revised: al [5] further attribute the success of residual networksto shallow sub-networks, which may correspond to the low-degree term when regarding the neuralnetwork as a polynomial function.

Original: Since  unraveled view is supported both experimentally andtheoretically, we further investigate some interesting mechanism behind residual networks based onthis inspiring work.
Revised: Since the unraveled view is supported both experimentally andtheoretically, we further investigate some interesting mechanism behind residual networks based onthis inspiring work.

Original: Specifically, we treat a residual network as the ensemble of relatively shallowsub-networks and consider its final performance to be co-determined by a group of sub-networks.
Revised: Specifically, we treat a residual network as the ensemble of relatively shallowsub-networks and consider its final performance to be co-determined by a group of sub-networks.

Original: [Figure]
Revised: [Figure]

Original: In social psychology, working in a group is always a thing.
Revised: In social psychology, working in a group is always a thing.

Original: Compared with performing tasksalone, group members tend to make less efforts when working as part of a group, which is defined asthe social loafing problem [6, 7].
Revised: Compared with performing tasksalone, group members tend to make less efforts when working as part of a group, which is defined asthe social loafing problem [6, 7].

Original: Moreover, social psychology researches find that increasing thegroup size may aggravate social loafing for the decrease of individual visibility [8, 9].
Revised: Moreover, social psychology researches find that increasing thegroup size may aggravate social loafing for the decrease of individual visibility [8, 9].

Original: Inspired bythese, we find that the ensemble-like networks formed by residual connections also have a similarproblem and behavior.
Revised: Inspired bythese, we find that the ensemble-like networks formed by residual connections also have a similarproblem and behavior.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 1, we can see that, different residual networks invariablysuffer from the loafing problem, that is, the sub-networks working in a given residual network areprone to exert degraded performances than these sub-networks working individually.
Revised: 1, we can see that, different residual networks invariablysuffer from the loafing problem, that is, the sub-networks working in a given residual network areprone to exert degraded performances than these sub-networks working individually.

Original: For example, asub-network ResNet32 in the ResNet56 only has a 46.80% Top1 accuracy, much lower than ResNet32trained individually with accuracy of 92.63%.
Revised: For example, asub-network ResNet32 in the ResNet56 only has a 46.80% Top1 accuracy, much lower than ResNet32trained individually with accuracy of 92.63%.

Original: Moreover, the loafing problem of deeper residualnetworks inclines to be more severe than that of shallower ones, that is, the same sub-network indeeper residual networks constantly presents inferior performance than that in shallower residualnetworks.
Revised: Moreover, the loafing problem of deeper residualnetworks is inclined to be more severe than that of shallower ones, that is, the same sub-network indeeper residual networks constantly presents inferior performance than that in shallower residualnetworks.

Original: For example, ResNet20 within ResNet32 has a 55.28% Top1 accuracy, while ResNet20within ResNet56 (deeper than ResNet 32) only has a 23.17% Top1 accuracy.
Revised: For example, ResNet20 within ResNet32 has a 55.28% Top1 accuracy, while ResNet20within ResNet56 (deeper than ResNet 32) only has a 23.17% Top1 accuracy.

Original: Such problems have notbeen addressed in the literature so far as we know.
Revised: Such problems have notbeen addressed in the literature so far as we know.

Original: Hereafter, we define this previously overlookedproblem as network loafing .
Revised: Hereafter, we define this previously overlookedproblem as network loafing .

Original: As social psychology researches show that social loafing will ultimatelycause low productivity of each individual and the collective [10, 11], we consider that networkloafing may also hinder the performance of given residual network and all of its sub-networks.
Revised: As social psychology researches show that social loafing will ultimatelycause low productivity of each individual and the collective [10, 11], we consider that networkloafing may also hinder the performance of given residual network and all of its sub-networks.

Original: In social psychology, there are two commonly used solutions for preventing social loafing withingroups: 1) establishing individual accountability by increasing the individual supervision and 2)making tasks cooperative by setting up the overall goal [8, 9].
Revised: In social psychology, there are two commonly used solutions for preventing social loafing withingroups: 1) establishing individual accountability by increasing the individual supervision and 2)making tasks cooperative by setting up the overall goal [8, 9].

Original: Inspired by this, we propose anovel training strategy for improving residual networks, namely stimulative training .
Revised: Inspired by this, we propose anovel training strategy for improving residual networks, namely stimulative training .

Original: In details, foreach mini-batch during stimulative training, besides the main loss of the given residual network inconventional training, we will randomly sample a residual sub-network (individual supervision) andcalculate the KL-divergence loss between the sampled sub-network and the given residual network(consistent overall goal).
Revised: In details, foreach mini-batch during stimulative training, besides the main loss of the given residual network inconventional training, we will randomly sample a residual sub-network (individual supervision) andcalculate the KL-divergence loss between the sampled sub-network and the given residual network(consistent overall goal).

Original: This simple yet effective training strategy can relieve the loafing problem ofresidual networks, by strengthening the individual supervision of sub-networks and making the goalsof residual sub-networks and the given residual network more consistent.
Revised: This simple yet effective training strategy can relieve the loafing problem ofresidual networks, by strengthening the individual supervision of sub-networks and making the goalsof residual sub-networks and the given residual network more consistent.

Original: Comprehensive empirical analyses verify that stimulative training can solve the loafing problem ofresidual networks effectively and efficiently, thus improve both the performance of a given residualnetwork and all of its residual sub-networks by a large margin.
Revised: Comprehensive empirical analyses verify that stimulative training can solve the loafing problem ofresidual networks effectively and efficiently, thus improve both the performance of a given residualnetwork and all of its residual sub-networks by a large margin.

Original: Furthermore, we theoretically showthe connection of the proposed stimulative training strategy and the improved performance of a givenresidual network and all of its residual sub-networks.
Revised: Furthermore, we theoretically showthe connection of the proposed stimulative training strategy and the improved performance of a givenresidual network and all of its residual sub-networks.

Original: Besides, experiments on various benchmarkdatasets using various residual networks demonstrate the effectiveness of the proposed trainingstrategy.
Revised: Besides, experiments on various benchmarkdatasets using various residual networks demonstrate the effectiveness of the proposed trainingstrategy.

Original: The contributions of our work can be summarized as the following:
Revised: The contributions of our work can be summarized as the following:

Original: • We understand residual networks from a social psychology perspective, and find that differentresidual networks invariably suffer from the problem of network loafing.• We improve residual networks from a social psychology perspective, and propose a simplebut-effective stimulative training strategy to improve the performance of the given residualnetwork and all of its sub-networks.
Revised: • We understand residual networks from a social psychology perspective, and find that differentresidual networks invariably suffer from the problem of network loafing.• We improve residual networks from a social psychology perspective, and propose a simplebut-effective stimulative training strategy to improve the performance of the given residualnetwork and all of its sub-networks.

Original: • Comprehensive empirical and theoretical analysis verify that stimulative training can well
Revised: • Comprehensive empirical and theoretical analysis verify that stimulative training can well

Original: solve the loafing problem of residual networks.
Revised: solve the loafing problem of residual networks.

Original:  Related Works2.1 Unraveled View
Revised:  Related Works2.1 Unraveled View

Original: As one pioneer work to investigate residual networks, [4] experimentally shows that residual networkscan be seen as a collection of numerous networks of different length, namely unraveled view.
Revised: As one pioneer work to investigate residual networks, [4] experimentally shows that residual networkscan be seen as a collection of numerous networks of different length, namely unraveled view.

Original: Subsequently, [5] follows this view and further attributes the success of residual networks to shallowsub-networks first.
Revised: Subsequently, [5] follows this view and further attributes the success of residual networks to shallowsub-networks first.

Original: Besides, [5] considers the neural network as a polynomial function and correspondsshallow sub-network to low-degree term to explain the working mechanism of residual networks.
Revised: Besides, [5] considers the neural network as a polynomial function and correspondsshallow sub-network to low-degree term to explain the working mechanism of residual networks.

Original: Similar to [4] and [5], this paper also investigates residual networks from the unraveled view.
Revised: Similar to [4] and [5], this paper also investigates residual networks from the unraveled view.

Original: Differently, inspired by social psychology, we further reveal the loafing problem of residual networksunder the unraveled view.
Revised: Differently, inspired by social psychology, we further reveal the loafing problem of residual networksunder the unraveled view.

Original: Besides, we propose a novel stimulative training method to relieve thisproblem and further unleash the potential of residual networks.
Revised: Besides, we propose a novel stimulative training method to relieve thisproblem and further unleash the potential of residual networks.

Original: 2.2 Knowledge Distillation
Revised: 2.2 Knowledge Distillation

Original: As a classical method, knowledge distillation [12] transfers the knowledge from a teacher network toa student network via approximating the logits [12, 13, 14] or features [15, 16, 17] output.
Revised: As a classical method, knowledge distillation [12] transfers the knowledge from a teacher network toa student network via approximating the logits [12, 13, 14] or features [15, 16, 17] output.

Original: To avoidthe huge cost of training a high performance teacher, some works abandon the naive teacher-studentframework, like mutual distillation [18] making group of students learn from each other online,and self distillation [19] transferring knowledge from deep layers to shallow layers.
Revised: To avoidthe huge cost of training a high performance teacher, some works abandon the naive teacher-studentframework, like mutual distillation [18] making group of students learn from each other online,and self distillation [19] transferring knowledge from deep layers to shallow layers.

Original: Generally, allthese distillation methods need to introduce additional networks or structures, and employ fixedteacher-student pairs.
Revised: Generally, allthese distillation methods need to introduce additional networks or structures, and employ fixedteacher-student pairs.

Original: As a comparison, our method does not require any additional network orstructure, and the student network is a randomly sampled sub-network of a network.
Revised: As a comparison, our method does not require any additional network orstructure, and the student network is a randomly sampled sub-network of a network.

Original: Besides, ourmethod is essentially designed to address the loafing problem of residual networks, which is differentfrom knowledge distillation that aims to obtain a compact network with acceptable accuracy.
Revised: Besides, ourmethod is essentially designed to address the loafing problem of residual networks, which is differentfrom knowledge distillation that aims to obtain a compact network with acceptable accuracy.

Original: 2.3 One-shot NAS
Revised: 2.3 One-shot NAS

Original: One-shot NAS is an important branch of neural architecture search (NAS).
Revised: One-shot NAS is an important branch of neural architecture search (NAS).

Original: Along this direction, [20]trains an once-for-all (OFA) network with progressive shrinking and knowledge distillation to supportkinds of architectural settings.
Revised: Along this direction, [20]trains an once-for-all (OFA) network with progressive shrinking and knowledge distillation to supportkinds of architectural settings.

Original: Following this work, BigNAS [21] introduces several technologiesto train a high-quality single-stage model, whose child models can be directly deployed withoutextra retraining or post-processing steps.
Revised: Following this work, BigNAS [21] introduces several technologiesto train a high-quality single-stage model, whose child models can be directly deployed withoutextra retraining or post-processing steps.

Original: Both OFA and BigNAS aim at simultaneously training andsearching various networks with different resolutions, depths, widths and operations.
Revised: Both OFA and BigNAS aim at simultaneously training andsearching various networks with different resolutions, depths, widths and operations.

Original: Differently, theproposed method aims at improving a given residual network, thus can be seamlessly applied to thesearched model of NAS.
Revised: Differently, theproposed method aims at improving a given residual network, thus can be seamlessly applied to thesearched model of NAS.

Original: As OFA and BigNAS are not designed to solve the loafing problem, theirsampling space and supervision signal are also different with the proposed method.
Revised: As OFA and BigNAS are not designed to solve the loafing problem, theirsampling space and supervision signal are also different with the proposed method.

Original: More importantly,the social-psychology-inspired problem of network loafing may explain why OFA and BigNAS work.
Revised: More importantly,the social-psychology-inspired problem of network loafing may explain why OFA and BigNAS work.

Original: 2.4 Stochastic Depth
Revised: 2.4 Stochastic Depth

Original: As a regularization technique, Stochastic Depth [22] randomly disables the convolution layers ofresidual blocks, to reduce training time and test error substantially.
Revised: As a regularization technique, Stochastic Depth [22] randomly disables the convolution layers ofresidual blocks, to reduce training time and test error substantially.

Original: In Stochastic Depth, the reductionin test error is attributed to strengthening gradients of earlier layers and the implicit ensemble ofnumerous sub-networks of different depths.
Revised: In Stochastic Depth, the reductionin test error is attributed to strengthening gradients of earlier layers and the implicit ensemble ofnumerous sub-networks of different depths.

Original: In fact, its improved performance can be also interpretedas relieving the network loafing problem defined in this work.
Revised: In fact, its improved performance can be also interpretedas relieving the network loafing problem defined in this work.

Original: The theoretical analysis in this workcan be also applied to Stochastic Depth.
Revised: The theoretical analysis in this workcan be also applied to Stochastic Depth.

Original: Besides, for better solving the loafing problem, our methodsamples sub-networks with ordered depth, and uses an additional KL-divergence loss to provide amore achievable target and make the output of a given network and its sub-networks more consistent.
Revised: Besides, for better solving the loafing problem, our methodsamples sub-networks with ordered depth, and uses an additional KL-divergence loss to provide amore achievable target and make the output of a given network and its sub-networks more consistent.

Original:  Stimulative Training
Revised:  Stimulative Training

Original: 3.1 Motivation
Revised: 3.1 Motivation

Original: Social loafing is a social psychology phenomenon that individuals lower their productivity whenworking in a group.
Revised: Social loafing is a social psychology phenomenon that individuals lower their productivity whenworking in a group.

Original: Based on the novel perspective that a residual network behaves like an ensemblenetwork [4], we find various residual networks invariably exhibit loafing-like behaviors as shownin Fig.
Revised: Based on the novel perspective that a residual network behaves like an ensemblenetwork [4], we find various residual networks invariably exhibit loafing-like behaviors as shownin Fig.

Original: 1 and Fig.
Revised: 1 and Fig.

Original: 4, which we define as network loafing.
Revised: 4, which we define as network loafing.

Original: As social loafing is “a kind of socialdisease" that will harm the individual and collective productivity [10], we consider network loafing
Revised: As social loafing is “a kind of socialdisease" that will harm the individual and collective productivity [10], we consider network loafing

Original: 2 L f − 1 L f −
Revised: 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 77.3% 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 77.3% 2 L f − 1 L f −

Original: L f
Revised: L f

Original: Skip connection
Revised: Skip connection

Original: Residualmodule
Revised: Residualmodule

Original: Building block
Revised: Building block

Original: I m a g e
Revised: I m a g e

Original: L a b e l Mainloss 2 L f − 1 L f −
Revised: L a b e l Mainloss 2 L f − 1 L f −

Original: L f
Revised: L f

Original: Skip connection
Revised: Skip connection

Original: Residualmodule
Revised: Residualmodule

Original: Building block
Revised: Building block

Original: I m a g e
Revised: I m a g e

Original: L a b e l Mainloss
Revised: L a b e l Mainloss

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 60%(65%) 2 L f − 1 L f −
Revised: 60%(65%) 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f −

Original: [Table]
Revised: [Table]

Original: % 2 L f − 1 L f −
Revised: % 2 L f − 1 L f −

Original: L f
Revised: L f

Original: Skip connection
Revised: Skip connection

Original: Residualmodule
Revised: Residualmodule

Original: Building block
Revised: Building block

Original: I m a g e
Revised: I m a g e

Original: L a b e l Mainloss 70%(75%) 60%(65%) 2 L f − 1 L f −
Revised: L a b e l Mainloss 70%(75%) 60%(65%) 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f −

Original: L a b e l Mainloss
Revised: L a b e l Mainloss

Original: I m a g e
Revised: I m a g e

Original: KLloss
Revised: KLloss

Original: KLloss 2 L f − 1 L f −
Revised: KLloss 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f −

Original: L a b e l Mainloss
Revised: L a b e l Mainloss

Original: I m a g e
Revised: I m a g e

Original: KLloss
Revised: KLloss

Original: KLloss 80%(
Revised: KLloss 80%(

Original: [Table] [Table]
Revised: [Table] [Table]

Original: %) 2 L f − 1 L f −
Revised: %) 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f −

Original: L a b e l Mainloss
Revised: L a b e l Mainloss

Original: I m a g e
Revised: I m a g e

Original: KLloss
Revised: KLloss

Original: KLloss 80%(75%) 79%(65%) (a) Common training scheme suffers from severe network loafing problem (b) S timulative training scheme can relieve network loafing and improve the performance 2 L f − 1 L f −
Revised: KLloss 80%(75%) 79%(65%) (a) Common training scheme suffers from severe network loafing problem (b) S timulative training scheme can relieve network loafing and improve the performance 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 77.3% 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 77.3% 2 L f − 1 L f −

Original: L f
Revised: L f

Original: Skip connection
Revised: Skip connection

Original: Residualmodule
Revised: Residualmodule

Original: Building block
Revised: Building block

Original: I m a g e
Revised: I m a g e

Original: L a b e l Mainloss 70%(75%) 60%(65%) 2 L f − 1 L f −
Revised: L a b e l Mainloss 70%(75%) 60%(65%) 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f − 81.07% 2 L f − 1 L f −

Original: L f 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 2 L f − 1 L f − 2 L f −

Original: L a b e l Mainloss
Revised: L a b e l Mainloss

Original: I m a g e
Revised: I m a g e

Original: KLloss
Revised: KLloss

Original: KLloss 80%(75%) 79%(65%) may also hinder the performance of a residual network and its sub-networks.
Revised: KLloss 80%(75%) 79%(65%) may also hinder the performance of a residual network and its sub-networks.

Original: To alleviate networkloafing, it is intuitive to learn from social psychology.
Revised: To alleviate networkloafing, it is intuitive to learn from social psychology.

Original: There are two common methods for solvingsocial-loafing problem in sociology, namely, establishing individual accountability (i.e., increasingthe individual supervision) and making tasks cooperative (i.e., setting up the overall goal) [8, 9].
Revised: There are two common methods for solvingsocial-loafing problem in sociology, namely, establishing individual accountability (i.e., increasingthe individual supervision) and making tasks cooperative (i.e., setting up the overall goal) [8, 9].

Original: Inorder to increase individual supervision, we sample sub-networks in the whole network and provideextra supervision to train each sub-network sufficiently.
Revised: Inorder to increase individual supervision, we sample sub-networks in the whole network and provideextra supervision to train each sub-network sufficiently.

Original: For the overall goal, we adopt KL divergenceloss to constrain the output of sub-networks not far from that of the whole network, which aims atreducing the performance gap and driving the sampled sub-networks to develop cooperatively.
Revised: For the overall goal, we adopt KL divergenceloss to constrain the output of sub-networks not far from that of the whole network, which aims atreducing the performance gap and driving the sampled sub-networks to develop cooperatively.

Original: 3.2 Training Algorithm
Revised: 3.2 Training Algorithm

Original: In this subsection, we briefly illustrate the working scheme of the proposed stimulative trainingstrategy, and show its difference from the common training method.
Revised: In this subsection, we briefly illustrate the working scheme of the proposed stimulative trainingstrategy, and show its difference from the common training method.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 2, commontraining only focuses on optimization of main network, thus suffers from severe network loafing, thatis, sub-networks lower their performance when working in an ensemble.
Revised: 2, commontraining only focuses on optimization of main network, thus suffers from severe network loafing, thatis, sub-networks lower their performance when working in an ensemble.

Original: For example, as shown in
Revised: For example, as shown in

Original: Fig.
Revised: Fig.

Original: 2(a) sub-networks within the residual network only has an accuracy of 60% and 70%, much lowerthan their individual accuracy of 65% and 75%.
Revised: 2(a) sub-networks within the residual network only have an accuracy of 60% and 70%, muchlower than their individual accuracy of 65% and 75%.

Original: As a comparison, stimulative training optimizesthe main network and meanwhile uses it to provide extra supervision for a sampled sub-network ateach training iteration, thus well handles the loafing problem.
Revised: As a comparison, stimulative training optimizesthe main network and meanwhile uses it to provide extra supervision for a sampled sub-network ateach training iteration, thus well handles the loafing problem.

Original: In the test procedure, our method canadopt the main network or any sub-network as the inference model, thus requiring the same or lowermemory and inference cost compared with a given residual network.
Revised: In the test procedure, our method canadopt the main network or any sub-network as the inference model, thus requiring the same or lowermemory and inference cost compared with a given residual network.

Original: Formally, for a given residual network to be optimized, we define the main network as D m and thesub-network as D s .
Revised: Formally, for a given residual network to be optimized, we define the main network as D m and thesub-network as D s .

Original: All the sub-networks share weights with the main network, and make up thesampling space Θ
Revised: All the sub-networks share weights with the main network, and make up thesampling space Θ

Original: = { D s | D s = π
Revised: = { D s | D s = π

Original: ( D m ) } , where π is sampling operator, usually random sampling.
Revised: ( D m ) } , where π is sampling operator, usually random sampling.

Original: In the training process, we randomly sample a sub-network at each iteration to ensure the wholesampling space can be fully explored.
Revised: In the training process, we randomly sample a sub-network at each iteration to ensure the wholesampling space can be fully explored.

Original: To make the training more efficient and effective, we define anew sampling space obeying ordered residual sampling rule to be discussed in Section 3.3.
Revised: To make the training more efficient and effective, we define anew sampling space obeying ordered residual sampling rule to be discussed in Section 3.3.

Original: Denotingθ Dm and θ Ds as weights of the main network and the sampled network respectively, x as the minibatch training sample and y as its label, Z as the output of network, the total loss of stimulativetraining is computed as 
Revised: Denotingθ Dm and θ Ds as weights of the main network and the sampled network respectively, x as the minibatch training sample and y as its label, Z as the output of network, the total loss of stimulativetraining is computed as [Equation] [Table]

Original: [Equation]
Revised: [Equation]

Original: where the CE and KL are the standard cross entropy loss and KL divergence loss.
Revised: where the CE and KL are the standard cross entropy loss and KL divergence loss.

Original: λ is the balancedcoefficient, and CE loss denotes the main supervision for the main network.
Revised: λ is the balancingcoefficient, and CE loss denotes the main supervision for the main network.

Original: KL loss provides anextra supervision for the sampled sub-network, and guarantees all sub-networks to be optimized inthe same direction with the main network, which can be considered as setting a common goal for the“ensemble network".
Revised: KL loss provides anextra supervision for the sampled sub-network, and guarantees all sub-networks to be optimized inthe same direction with the main network, which can be considered as setting a common goal for the“ensemble network”.

Original: We use the standard stochastic gradient descent for updating the model, denotedas θ t +1 D m = θ tD m − η ∂ L ut ∂θ Dm , where η is the learning rate.
Revised: We use the standard stochastic gradient descent for updating the model, denotedas θ t +1 D m = θ tD m − η ∂ L ut ∂θ Dm , where η is the learning rate.

Original: Thanks to the weight-sharing between
Revised: Thanks to the weight-sharing between

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t O u t pu t
Revised: O u t pu t I npu t O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t O u t pu t
Revised: O u t pu t I npu t O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t O u t pu t
Revised: O u t pu t I npu t O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t O u t pu t
Revised: O u t pu t I npu t O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −
Revised: O u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −

Original: L f
Revised: L f

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −
Revised: u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −

Original: [Table]
Revised: [Table]

Original: f
Revised: f

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −
Revised: O u t pu t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −

Original: L f
Revised: L f

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t L f L f
Revised: O u t pu t L f L f

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t
Revised: O u t pu t I npu t

Original: O u t pu t
Revised: O u t pu t

Original: L f 1 L f −
Revised: L f 1 L f −

Original: L f
Revised: L f

Original: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f −
Revised: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f −

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: L f
Revised: L f

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: L f 1 L f −
Revised: L f 1 L f −

Original: L f
Revised: L f

Original: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f − 2.
Revised: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f − 2.

Original: Stochastic
Revised: Stochastic

Original: [Table]
Revised: [Table]

Original: Top1:
Revised: Top1:

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t 
Revised: O u t pu t I npu t O u t pu t

Original: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t I npu t O u t pu t
Revised: O u t pu t I npu t O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 1 L f −
Revised: O u t pu t 2 L f − 1 L f −

Original: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −
Revised: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −

Original: L f
Revised: L f

Original: L f L f
Revised: L f L f

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 1 L f −
Revised: O u t pu t 2 L f − 1 L f −

Original: L f 2 L f − 1 L f −
Revised: L f 2 L f − 1 L f −

Original: [Table]
Revised: [Table]

Original: −
Revised: −

Original: [Table]
Revised: [Table]

Original: −
Revised: −

Original: [Table]
Revised: [Table]

Original: −
Revised: −

Original: [Table]
Revised: [Table]

Original: f
Revised: f

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 1 L f −
Revised: O u t pu t 2 L f − 1 L f −

Original: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −
Revised: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −

Original: L f
Revised: L f

Original: L f L f
Revised: L f L f

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: npu t
Revised: npu t

Original: [Table]
Revised: [Table]

Original: u t
Revised: u t

Original: [Table]
Revised: [Table]

Original: t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −
Revised: t 2 L f − 2 L f − 2 L f − 1 L f − 1 L f −

Original: [Table]
Revised: [Table]

Original: f
Revised: f

Original: [Table]
Revised: [Table]

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: f
Revised: f

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: I npu t
Revised: I npu t

Original: [Table]
Revised: [Table]

Original: u t
Revised: u t

Original: [Table]
Revised: [Table]

Original: t
Revised: t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: L f 1 L f −
Revised: L f 1 L f −

Original: L f
Revised: L f

Original: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f − 2.
Revised: L f 2 L f − 1 L f − 1 L f − 2 L f − 2 L f − 1 L f − 2 L f − 2.

Original: Stochastic Residual Sampling Main Network Top1:80.53%
Revised: Stochastic Residual Sampling Main Network Top1:80.53%

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t
Revised: O u t pu t

Original: I npu t
Revised: I npu t

Original: O u t pu t 2 L f − 1 L f −
Revised: O u t pu t 2 L f − 1 L f −

Original: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −
Revised: L f 2 L f − 1 L f − 2 L f − 1 L f − 2 L f − 1 L f −

Original: L f
Revised: L f

Original: L f L f
Revised: L f L f

Original: Main Network Stage 2 L f − 1 L f −
Revised: Main Network Stage 2 L f − 1 L f −

Original: L f I npu t
Revised: L f I npu t

Original: O u t pu t
Revised: O u t pu t

Original: Reduce sampling space
Revised: Reduce sampling space

Original: Conform to neural differential equation ) ( ) 2 ( n O O n  sub-networks and the main network, we only need to update θ D m once for each iteration to optimizeboth the θ D m and θ D s .
Revised: Conform to neural differential equation ) ( ) 2 ( n O O n  sub-networks and the main network, we only need to update θ D m once for each iteration to optimizeboth the θ D m and θ D s .

Original: The pseudo code is shown in Appendix C.1.
Revised: The pseudo code is shown in Appendix C.1.

Original: 3.3 Ordered Residual Sampling
Revised: 3.3 Ordered Residual Sampling

Original: Network loafing is based on the novel view that residual networks can be seen as the ensemble ofnumerous different sub-networks[4].
Revised: Network loafing is based on the novel view that residual networks can be seen as the ensemble ofnumerous different sub-networks[4].

Original: It is intuitive to adopt raw unraveled view to design samplingspace, namely raw sampling.
Revised: It is intuitive to adopt raw unraveled view to design samplingspace, namely raw sampling.

Original: However, there exist two problems in raw sampling.
Revised: However, there exist two problems in raw sampling.

Original: First, it introducesa mass of single branch sub-networks which are hard to optimize, as shown in the left sub-figure of
Revised: First, it introducesa mass of single branch sub-networks which are hard to optimize, as shown in the left sub-figure of

Original: Fig.
Revised: Fig.

Original: Second, with the growth of network depth, the size of sampling space increases exponentially.
Revised: Second, with the growth of network depth, the size of sampling space increases exponentially.

Original: For a residual stage of n blocks, there are 2 n sub-networks, which constitutes a too large space toexplore.
Revised: For a residual stage of n blocks, there are 2 n sub-networks, which constitutes a too large space toexplore.

Original: Under limited computation resources, it is difficult to provide sufficient supervision to eachsub-network, causing insufficient stimulative training.
Revised: Under limited computation resources, it is difficult to provide sufficient supervision to eachsub-network, causing insufficient stimulative training.

Original: Therefore, in order to facilitate stimulative training, we re-design two different sampling ways:stochastic residual sampling and ordered residual sampling.
Revised: Therefore, in order to facilitate stimulative training, we re-design two different sampling ways:stochastic residual sampling and ordered residual sampling.

Original: Fig.
Revised: Fig.

Original: 3 shows a three-block residualnetwork as a simple illustration for three sampling ways.
Revised: 3 shows a three-block residualnetwork as a simple illustration for three sampling ways.

Original: For raw sampling, we obey the raw definitionof [4], that means each sub-network is composed of stacked convolution layers.
Revised: For raw sampling, we obey the raw definitionof [4], that means each sub-network is composed of stacked convolution layers.

Original: For stochastic residualsampling, we keep the basic structure of residual blocks and skip some blocks randomly.
Revised: For stochastic residualsampling, we keep the basic structure of residual blocks and skip some blocks randomly.

Original: For orderedresidual sampling, we also keep the residual structure but skip blocks orderly, (i.e., always skip thelast several blocks).
Revised: For orderedresidual sampling, we also keep the residual structure but skip blocks orderly, (i.e., always skip thelast several blocks).

Original: We implement these sampling methods for MobileNetV3 [23] on CIFAR-100.
Revised: We implement these sampling methods for MobileNetV3 [23] on CIFAR-100.

Original: Experimental results show that ordered residual sampling performs better than other two methods (by5.77% and 0.58% respectively).
Revised: Experimental results show that ordered residual sampling performs better than other two methods (by5.77% and 0.58% respectively).

Original: To explain the superiority of ordered residual sampling, we shouldpay attention to keeping residual structure and ordered sampling.
Revised: To explain the superiority of ordered residual sampling, we shouldpay attention to keeping residual structure and ordered sampling.

Original: For keeping residual structure, itcan drive the network training process easier and improve the final performance [1].
Revised: For keeping residual structure, itcan drive the network training process easier and improve the final performance [1].

Original: For orderedsampling, it can noticeably reduce the size of sampling space (from O (2 n ) to O ( n ) ), making itpossible to train each sub-network sufficiently.
Revised: For orderedsampling, it can noticeably reduce the size of sampling space (from O (2 n ) to O ( n ) ), making itpossible to train each sub-network sufficiently.

Original: Besides, all deep networks with residual connectionscan be approximated by neural differential equations [24], which treat discrete network layers ascontinuous and regard the layer output as the result of adding an integral to the layer input.
Revised: Besides, all deep networks with residual connectionscan be approximated by neural differential equations [24], which treat discrete network layers ascontinuous and regard the layer output as the result of adding an integral to the layer input.

Original: From thisperspective, ordered sampling will maintain the continuity of residual networks.
Revised: From thisperspective, ordered sampling will maintain the continuity of residual networks.

Original: Because of theseoutstanding features, we will adopt ordered residual sampling in the following experiments.
Revised: Because of theseoutstanding features, we will adopt ordered residual sampling in the following experiments.

Original:  Empirical Analysis
Revised:  Empirical Analysis

Original: In this section, we conduct extensive experiments to analyze and compare the proposed stimulativetraining and the common training strategy in detail.
Revised: In this section, we conduct extensive experiments to analyze and compare the proposed stimulativetraining and the common training strategy in detail.

Original: We first illustrate the loafing problem ofresidual networks with different training strategies.
Revised: We first illustrate the loafing problem ofresidual networks with different training strategies.

Original: We further demonstrate statistical characteristicsof the performance of the trained residual network and all of its sub-networks.
Revised: We further demonstrate statistical characteristicsof the performance of the trained residual network and all of its sub-networks.

Original: Then, we show
Revised: Then, we show

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: dataset.
Revised: dataset.

Original: Sampling 0 sub-networks denotes the common training strategy.
Revised: Sampling 0 sub-networks denotes the common training strategy.

Original: Stimulative training with more sub-networks can provide higher maximum, mean and lower standard deviation.
Revised: Stimulative training with more sub-networks can provide higher maximum, mean and lower standard deviation.

Original: the performance drop when destructing trained residual networks at test time, including deletingindividual layers, deleting more layers and permuting layers.
Revised: the performance drop when destructing trained residual networks at test time, including deletingindividual layers, deleting more layers and permuting layers.

Original: Besides, we also record the KL distancebetween a given residual network and all of its sub-networks in the whole training process.
Revised: Besides, we also record the KL distancebetween a given residual network and all of its sub-networks in the whole training process.

Original: For theconvenience of analysis, we always use a NAS-searched model MobileNetV3 [23] as the residualnetwork and train it on CIFAR100, unless otherwise specified.
Revised: For theconvenience of analysis, we always use a NAS-searched model MobileNetV3 [23] as the residualnetwork and train it on CIFAR100, unless otherwise specified.

Original: The detailed experimental settings canbe found in Appendix C.2.
Revised: The detailed experimental settings canbe found in Appendix C.2.

Original: 4.1 Experiment 1: Checking the loafing problem of residual networks with different trainingstrategies
Revised: 4.1 Experiment 1: Checking the loafing problem of residual networks with different trainingstrategies

Original: First, we find that the NAS-searched residual network MobileNetV3 still suffers from the loafingproblem.
Revised: First, we find that the NAS-searched residual network MobileNetV3 still suffers from the loafingproblem.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 4, when applying the common training strategy to MobileNetV3, thesub-networks from MobileNetV3 (blue solid circle) have much lower accuracies than the samesub-networks individually trained (green five-pointed star), and the performance gap increases whenthe size of the sub-network decreases.
Revised: 4, when applying the common training strategy to MobileNetV3, thesub-networks from MobileNetV3 (blue solid circle) have much lower accuracies than the samesub-networks individually trained (green five-pointed star), and the performance gap increases whenthe size of the sub-network decreases.

Original: Second, we show that the proposed stimulative training strategycan well handle the loafing problem.
Revised: Second, we show that the proposed stimulative training strategycan well handle the loafing problem.

Original: As shown in Fig. 4(c), when applying the stimulative trainingstrategy to MobileNetV3, the sub-networks from MobileNetV3 (red five-pointed star) have evenbetter performance than the same sub-networks individually trained (green five-pointed star).
Revised: As shown in Fig. 4(c), when applying the stimulative trainingstrategy to MobileNetV3, the sub-networks from MobileNetV3 (red five-pointed star) have evenbetter performance than the same sub-networks individually trained (green five-pointed star).

Original: Moreinterestingly, as shown in Fig. 4(a), even when we apply stimulative training with 6 sub-networks(red five-pointed star) to MobileNetV3, the performance of all the sub-networks (red solid circle) canbe also greatly improved compared to that of common training (blue solid circle).
Revised: Moreinterestingly, as shown in Fig. 4(a), even when we apply stimulative training with 6 sub-networks(red five-pointed star) to MobileNetV3, the performance of all the sub-networks (red solid circle) canbe also greatly improved compared to that of common training (blue solid circle).

Original: Besides, As shownin Fig. 4, with more sub-networks involved in stimulative training, the network loafing problem couldbe relieved better, and the performance of a given residual network and all of its sub-networks couldbe improved more significantly.
Revised: Besides, As shownin Fig. 4, with more sub-networks involved in stimulative training, the network loafing problem couldbe relieved better, and the performance of a given residual network and all of its sub-networks couldbe improved more significantly.

Original: More similar results of different datasets and residual networks canbe found in Appendix A.
Revised: More similar results of different datasets and residual networks canbe found in Appendix A.

Original: 4.2 Experiment 2: Collecting statistical characteristics of the performance of all residualsub-networks
Revised: 4.2 Experiment 2: Collecting statistical characteristics of the performance of all residualsub-networks

Original: To demonstrate that stimulative training can improve the performance of a given residual network andall of its sub-networks, we collect various statistical characteristics of the performance of all residual
Revised: To demonstrate that stimulative training can improve the performance of a given residual network andall of its sub-networks, we collect various statistical characteristics of the performance of all residual

Original: [Figure]
Revised: [Figure]

Original: sub-networks after the stimulative training and common training.
Revised: sub-networks after the stimulative training and common training.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 5, MobileNetV3with common training (i.e., sample 0 sub-networks) has the lowest maximum value, mean value, andthe highest standard deviation, which is mainly caused by the network loafing problem.
Revised: 5, MobileNetV3with common training (i.e., sample 0 sub-networks) has the lowest maximum value, mean value, andthe highest standard deviation, which is mainly caused by the network loafing problem.

Original: As a contrast,
Revised: As a contrast,

Original: MobileNetV3 with stimulative training (i.e., sample 144 sub-networks) has the highest maximumvalue, mean value, and the lowest standard deviation.
Revised: MobileNetV3 with stimulative training (i.e., sample 144 sub-networks) has the highest maximumvalue, mean value, and the lowest standard deviation.

Original: Moreover, we can see that, MobileNetV3 usingstimulative training with more sub-networks can usually provide higher maximum value, mean valueand lower standard deviation.
Revised: Moreover, we can see that, MobileNetV3 usingstimulative training with more sub-networks can usually provide higher maximum value, mean valueand lower standard deviation.

Original: All these results indicate that stimulative training can improve theperformance of all residual sub-networks effectively and efficiently.
Revised: All these results indicate that stimulative training can improve theperformance of all residual sub-networks effectively and efficiently.

Original: 4.3 Experiment 3: Destructing residual networks at test time
Revised: 4.3 Experiment 3: Destructing residual networks at test time

Original: In this experiment, we verify that stimulative training can provide stronger robustness in resistingvarious network destruction operations including deleting individual layers, deleting more layers andpermuting layers than common training.
Revised: In this experiment, we verify that stimulative training can provide stronger robustness in resistingvarious network destruction operations including deleting individual layers, deleting more layers andpermuting layers than common training.

Original: Deleting individual layers.
Revised: Deleting individual layers.

Original: We first show the performance drop when deleting individual layersfrom trained MobileNetV3 at test time.
Revised: We first show the performance drop when deleting individual layersfrom trained MobileNetV3 at test time.

Original: As shown in Fig. 6(a), when applying common training to
Revised: As shown in Fig. 6(a), when applying common training to

Original: MobileNetV3, the performance drop is non-negligible when some layers are deleted (especially forthe last several layers of each stage), and the trained MobileNetV3 is sensitive to the index of deletedlayer.
Revised: MobileNetV3, the performance drop is non-negligible when some layers are deleted (especially forthe last several layers of each stage), and the trained MobileNetV3 is sensitive to the index of deletedlayer.

Original: As a comparison, when applying stimulative training to MobileNetV3, no matter which layer isdeleted, the performance drop of stimulative trained model is always lower than that of the commontrained model, and the performance drop is always smaller than or equal to 1%.
Revised: As a comparison, when applying stimulative training to MobileNetV3, no matter which layer isdeleted, the performance drop of stimulative trained model is always lower than that of the commontrained model, and the performance drop is always smaller than or equal to 1%.

Original: Such results verifythat stimulative training can reduce the dependence between sub-networks significantly.
Revised: Such results verifythat stimulative training can reduce the dependence between sub-networks significantly.

Original: Deleting more layers.
Revised: Deleting more layers.

Original: We then show the performance drop when deleting more layers from trained
Revised: We then show the performance drop when deleting more layers from trained

Original: MobileNetV3 at test time.
Revised: MobileNetV3 at test time.

Original: Note that, deleting more layers may come in a variety of combinations,thus we show the statistical box plot of performance drops for all combinations.
Revised: Note that, deleting more layers may come in a variety of combinations,thus we show the statistical box plot of performance drops for all combinations.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 6(b),when applying common training to MobileNetV3, the medium value, minimum value and maximumvalue of performance drop for all combinations keeps continuously rising as the number of deletedlayers increases.
Revised: 6(b),when applying common training to MobileNetV3, the medium value, minimum value and maximumvalue of performance drop for all combinations keeps continuously rising as the number of deletedlayers increases.

Original: And the difference in performance drop for different combinations when removingthe same number of layers can be very large.
Revised: And the difference in performance drop for different combinations when removingthe same number of layers can be very large.

Original: For example, when deleting 4 layers of common trained
Revised: For example, when deleting 4 layers of common trained

Original: MobileNetV3, the Top1 accuracy drop for different combinations can vary from less than 5% tohigher than 35%.
Revised: MobileNetV3, the Top1 accuracy drop for different combinations can vary from less than 5% tohigher than 35%.

Original: Differently, when applying stimulative training, no matter how many layers aredeleted, the values of performance drop for all combinations always keep very low (i.e., no more than3%), indicating that stimulative training can bring strong robustness in resisting network destruction.
Revised: Differently, when applying stimulative training, no matter how many layers aredeleted, the values of performance drop for all combinations always keep very low (i.e., no more than3%), indicating that stimulative training can bring strong robustness in resisting network destruction.

Original: Permuting layers.
Revised: Permuting layers.

Original: We further show the performance drop when permuting layers in trained
Revised: We further show the performance drop when permuting layers in trained

Original: MobileNetV3 at test time.
Revised: MobileNetV3 at test time.

Original: Note that, only the layers in the same stage can be permuted since theresolution and width of different stages are varying, and we show the detailed experimental settings in
Revised: Note that, only the layers in the same stage can be permuted since theresolution and width of different stages are varying, and we show the detailed experimental settings in

Original: Appendix C.3.
Revised: Appendix C.3.

Original: As shown in Fig. 6(c), when applying common training to MobileNetV3, the mediumvalue, minimum value and maximum value of performance drop for all combinations increases withthe rising number of permuted layers, and the variance of performance drop for different combinationswhen permuting the same number of layers may be also very large.
Revised: As shown in Fig. 6(c), when applying common training to MobileNetV3, the mediumvalue, minimum value and maximum value of performance drop for all combinations increases withthe rising number of permuted layers, and the variance of performance drop for different combinationswhen permuting the same number of layers may be also very large.

Original: As a comparison, the values ofperformance drop for all permuting combinations always keep very low (i.e., less than 1%), whenstimulative training is applied.
Revised: As a comparison, the values ofperformance drop for all permuting combinations always keep very low (i.e., less than 1%), whenstimulative training is applied.

Original: This further verifies the superiority of stimulative training.
Revised: This further verifies the superiority of stimulative training.

Original: Experiment
Revised: Experiment

Original: : Recording the
Revised: : Recording the

Original: KL divergence
Revised: KL divergence

Original: between
Revised: between

Original: a residual network
Revised: a residual network

Original: and all of itssub-networks when training
Revised: and all of itssub-networks when training

Original: To study the consistency between the output of a given residual network and all of its sub-networks, we record their KL distance in the whole training process.
Revised: To study the consistency between the output of a given residual network and all of its sub-networks, we record their KL distance in the whole training process.

Original: As shown in Fig.
Revised: As shown in Fig.

Original: 7, when applying common training to MobileNetV3, the variance, maximum and medium value of KL distance between a given residual network and all of its sub-networks continuously increases with rising training epoch.
Revised: 7, when applying common training to MobileNetV3, the variance, maximum and medium value of KL distance between a given residual network and all of its sub-networks continuously increases with rising training epoch.

Original: For comparison, when stimulative training is applied to MobileNetV3, all values of KL distance between MobileNetV3 and all of its sub-networks keep very small (i.e., no more than 0.25) in the whole training process.
Revised: For comparison, when stimulative training is applied to MobileNetV3, all values of KL distance between MobileNetV3 and all of its sub-networks keep very small (i.e., no more than 0.25) in the whole training process.

Original: Such results imply that the proposed stimulative training can maintain the consistency between the output of a given residual network and all of its sub-networks.
Revised: Such results imply that the proposed stimulative training can maintain the consistency between the output of a given residual network and all of its sub-networks.

Original: [Figure]
Revised: [Figure]

Original:  Theoretical Analysis
Revised:  Theoretical Analysis

Original: Analysis 1: Why the performance of sub-networks can be improved.
Revised: Analysis 1: Why the performance of sub-networks can be improved.

Original: Based on Eq. 1, ourmethod can be treated as optimizing CE ( Z ( θ D m , x ) , y ) and KL ( Z ( θ D m , x ) , Z ( θ D s , x )) at thesame time.
Revised: Based on Eq. 1, ourmethod can be treated as optimizing CE ( Z ( θ D m , x ) , y ) and KL ( Z ( θ D m , x ) , Z ( θ D s , x )) at thesame time.

Original: According to the convergence of stochastic gradient descent, after enough number ofiterations, both losses can be bounded by a constant, which is formulated as
Revised: According to the convergence of stochastic gradient descent, after enough number ofiterations, both losses can be bounded by a constant, which is formulated as

Original: CE ( Z ( θ D m , x ) , y )
Revised: CE ( Z ( θ D m , x ) , y )

Original: = −
Revised: = −

Original: N (cid:88) i =
Revised: N (cid:88) i =

Original: y i log p mi
Revised: y i log p mi

Original: [Equation]
Revised: [Equation]

Original: where N is the number of categories.
Revised: where N is the number of categories.

Original: The ϵ 1 and ϵ 2 are the tiny constants for corresponding losses.
Revised: The ϵ 1 and ϵ 2 are the tiny constants for corresponding losses.

Original: Θ is the set of all sub-networks.
Revised: Θ is the set of all sub-networks.

Original: The p m and p s are the classification probability of main networkand sub-network, respectively.
Revised: The p m and p s are the classification probability of main networkand sub-network, respectively.

Original: Further, we can prove that the absolute difference of cross entropyof sub-networks and main network can be bounded by a constant (detailed proof can be found in
Revised: Further, we can prove that the absolute difference of cross entropyof sub-networks and main network can be bounded by a constant (detailed proof can be found in

Original: Appendix B), which can be formulated as
Revised: Appendix B), which can be formulated as

Original: [Equation]
Revised: [Equation]

Original: where ϵ 2 +log N e − ϵ+ ϵ 1 is a small constant after optimization.
Revised: where ϵ 2 +log N e − ϵ+ ϵ 1 is a small constant after optimization.

Original: Eq.
Revised: Eq.

Original: 4 demonstrates that the performancegap between sub-networks and the main network is constrained.
Revised: 4 demonstrates that the performancegap between sub-networks and the main network is constrained.

Original: Thus, as the rising of the mainnetwork performance, sub-networks performance will be improved step by step .
Revised: Thus, as the rising of the mainnetwork performance, sub-networks performance will be improved step by step .

Original: Analysis 2: Why the performance of given residual networks can be improved.
Revised: Analysis 2: Why the performance of given residual networks can be improved.

Original: Neural networktraining is in fact a form of polynomial regression, and any neural network with activation functionscan roughly correspond to a fitted polynomial [25].
Revised: Neural networktraining is in fact a form of polynomial regression, and any neural network with activation functionscan roughly correspond to a fitted polynomial [25].

Original: For simplifying notations, suppose that bothinput x and output y in a neural network have only one element.
Revised: For simplifying notations, suppose that bothinput x and output y in a neural network have only one element.

Original: Then, we can consider the trainingof a neural network as the fitting of a polynomial function.
Revised: Then, we can consider the trainingof a neural network as the fitting of a polynomial function.

Original: [Equation]
Revised: [Equation]

Original: (5) where c 1 , c 2 , . . .
Revised: (5) where c 1 , c 2 , . . .

Original: , c n , . .
Revised: , c n , . .

Original: . represent the polynomial coefficients.
Revised: . represent the polynomial coefficients.

Original: Based on training samples, we canestablish the polynomial (neural network), which denotes the mapping relationship between inputsand outputs.
Revised: Based on training samples, we canestablish the polynomial (neural network), which denotes the mapping relationship between inputsand outputs.

Original: For a test sample x , there always exists a point x 0 (e.g., a training sample) close enoughto x , which can obtain an accurate output via the trained model.
Revised: For a test sample x , there always exists a point x 0 (e.g., a training sample) close enoughto x , which can obtain an accurate output via the trained model.

Original: According to Taylor expansion, F ( x )at x 0 can be computed as :
Revised: According to Taylor expansion, F ( x )at x 0 can be computed as :

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: where R n ( x ) denotes the higher degree infinitesimal of ( x − x 0 ) n .
Revised: where R n ( x ) denotes the higher degree infinitesimal of ( x − x 0 ) n .

Original: According to Eq.
Revised: According to Eq.

Original: 5 and Eq.
Revised: 5 and Eq.

Original: 6, it isobvious that different terms of the fitted polynomial have different effects, and the low-degree termstend to have greater effects on the predicted performance.
Revised: 6, it isobvious that different terms of the fitted polynomial have different effects, and the low-degree termstend to have greater effects on the predicted performance.

Original: Based on this point, [5] further figuresout that shallow sub-networks in residual networks roughly correspond to low-degree polynomialsconsisting of low-degree terms, while deep sub-networks are opposite.
Revised: Based on this point, [5] further figuresout that shallow sub-networks in residual networks roughly correspond to low-degree polynomialsconsisting of low-degree terms, while deep sub-networks are opposite.

Original: Following this theory, theproposed stimulative training can not only strengthen the learning of different items of the polynomial(neural network), but also pay more attention to the learning of low-degree items .
Revised: Following this theory, theproposed stimulative training can not only strengthen the learning of different items of the polynomial(neural network), but also pay more attention to the learning of low-degree items .

Original: As a result, thefinal performance of given residual networks can be improved by stimulative training.
Revised: As a result, thefinal performance of given residual networks can be improved by stimulative training.

Original:  Verification on Various Datasets and Networks
Revised:  Verification on Various Datasets and Networks

Original: CIFAR implementation details.
Revised: CIFAR implementation details.

Original: CIFAR[26] is a classical image classification dataset consistingof 50,000 training images and 10,000 testing images.
Revised: CIFAR[26] is a classical image classification dataset consistingof 50,000 training images and 10,000 testing images.

Original: It includes CIFAR-100 in 100 categories and
Revised: It includes CIFAR-100 in 100 categories and

Original: CIFAR-10 in 10 categories.
Revised: CIFAR-10 in 10 categories.

Original: For MobileNetV3 and ResNet50, the data augmentations follow [27], weuse SGD optimizer and train the model for 500 epochs with a batch size of 64.
Revised: For MobileNetV3 and ResNet50, the data augmentations follow [27], weuse SGD optimizer and train the model for 500 epochs with a batch size of 64.

Original: The initial learningrate is 0.05 with cosine decay schedule.
Revised: The initial learningrate is 0.05 with cosine decay schedule.

Original: The weight decay is 3 × 10 − 5 and momentum is 0.9.
Revised: The weight decay is 3 × 10 − 5 and momentum is 0.9.

Original: ImageNet implementation details.
Revised: ImageNet implementation details.

Original: We implement our method on large-scale ImageNet[28] datasetcontaining 1.2 million training images and 50,000 validation images from 1,000 categories.
Revised: We implement our method on large-scale ImageNet[28] datasetcontaining 1.2 million training images and 50,000 validation images from 1,000 categories.

Original: For
Revised: For

Original: ResNet families, we use standard data augmentations for ImageNet, as done in [29, 30, 31].
Revised: ResNet families, we use standard data augmentations for ImageNet, as done in [29, 30, 31].

Original: Weutilize SGD optimizer to train the model for 100 epochs with a batch size of 512, and the learningrate is 0.2 with cosine decay schedule.
Revised: Weutilize SGD optimizer to train the model for 100 epochs with a batch size of 512, and the learningrate is 0.2 with cosine decay schedule.

Original: The weight decay is 1 × 10 − 4 and momentum is 0.9.
Revised: The weight decay is 1 × 10 − 4 and momentum is 0.9.

Original: Main network and sub-networks performance.
Revised: Main network and sub-networks performance.

Original: We further verify the effectiveness of stimulativetraining (ST) on various networks and datasets.
Revised: We further verify the effectiveness of stimulativetraining (ST) on various networks and datasets.

Original: Table.1 shows its comparison with common training(CT).
Revised: Table.1 shows its comparison with common training(CT).

Original: Besides the Top1 accuracy of main network, we also report the the mean value and standarddeviation of Top1 accuracy for all sub-networks.
Revised: Besides the Top1 accuracy of main network, we also report the the mean value and standarddeviation of Top1 accuracy for all sub-networks.

Original: It is obvious that stimulative training can noticeablyimprove the performance of various networks on various datasets compared to common training (e.g.,by 4.43% for ResNet50 on CIFAR-100 and 1.21% for ResNet101 on ImageNet).
Revised: It is obvious that stimulative training can noticeablyimprove the performance of various networks on various datasets compared to common training (e.g.,by 4.43% for ResNet50 on CIFAR-100 and 1.21% for ResNet101 on ImageNet).

Original: Moreover, the meanperformance of all sub-networks is dramatically improved and standard deviation greatly drops.
Revised: Moreover, the meanperformance of all sub-networks is dramatically improved and standard deviation greatly drops.

Original: Forexample, stimulative training for MobileNetV3 on CIFAR100 has a mean performance of 80.01%(much higher than common training of 55.26%) and a standard deviation of 0.59% (much lowerthan 13.37%).
Revised: Forexample, stimulative training for MobileNetV3 on CIFAR100 has a mean performance of 80.01%(much higher than common training of 55.26%) and a standard deviation of 0.59% (much lowerthan 13.37%).

Original: Such results validate that stimulative training can well alleviate the network loafingproblem, and improve the performance of the given residual network and all of its sub-networks.
Revised: Such results validate that stimulative training can well alleviate the network loafingproblem, and improve the performance of the given residual network and all of its sub-networks.

Original: Experimental results on various models and datasets verify the generalization of stimulative training,we advocate utilizing it as a general technology to train residual networks.
Revised: Experimental results on various models and datasets verify the generalization of stimulative training,we advocate utilizing it as a general technology to train residual networks.

Original:  Conclusions
Revised:  Conclusions

Original: The paper understands and improves residual networks from a social psychology perspective of loafing.
Revised: The paper understands and improves residual networks from a social psychology perspective of loafing.

Original: Based on the novel view that a residual network behaves like an ensemble network, we find that various residual networks invariably exhibit loafing-like behaviors that are consistent with the social loafing problem of social psychology.
Revised: Based on the novel view that a residual network behaves like an ensemble network, we find that various residual networks invariably exhibit loafing-like behaviors that are consistent with the social loafing problem of social psychology.

Original: We define this previously overlooked problem as network loafing.
Revised: We define this previously overlooked problem as network loafing.

Original: As the loafing problem hinders the productivity of each individual and the whole collective, we learn from social psychology and propose a stimulative training strategy to solve the network loafing problem.
Revised: As the loafing problem hinders the productivity of each individual and the whole collective, we learn from social psychology and propose a stimulative training strategy to solve the network loafing problem.

Original: Comprehensive empirical analyses show that stimulative training can improve the performance of a given residual network and all of its sub-networks and provide strong robustness in resisting various network destruction operations.
Revised: Comprehensive empirical analyses show that stimulative training can improve the performance of a given residual network and all of its sub-networks and provide strong robustness in resisting various network destruction operations.

Original: Furthermore, we theoretically show why the performance of a given residual network and its sub-networks can be improved.
Revised: Furthermore, we theoretically show why the performance of a given residual network and its sub-networks can be improved.

Original: Experiments on various datasets and residual networks demonstrate the effectiveness of the proposed method 
Revised: Experiments on various datasets and residual networks demonstrate the effectiveness of the proposed method Checklist

Revised: The checklist follows the references.

Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Revised: For example:

Revised: • Did you include the license to the code and datasets?

Revised: [Yes] See

Revised: Section ?

Revised: • Did you include the license to the code and datasets?

Revised: [No] The code and the data areproprietary.•

Revised: Did you include the license to the

Revised: Please do not modify the questions and only use the provided macros for your answers.

Revised: Note that the

Revised: Checklist section does not count towards the page limit.

Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Revised: For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Revised: contributions and scope?

Revised: [Yes](b) Did you describe the limitations of your work?

Revised: [No](c) Did you discuss any potential negative societal impacts of your work?

Revised: [No](d) Have you read the ethics review guidelines and ensured that your paper conforms to

Revised: them?

Revised: [Yes]

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [Yes]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi-

Revised: mental results (either in the supplemental material or as a URL)?

Revised: [Yes](b) Did you specify all the training details (e

Revised: g., data splits, hyperparameters, how theywere chosen)?

Revised: [Yes](c)

Revised: Did you report error bars (e

Revised: g., with respect to the random seed after running experiments multiple times)?

Revised: [No](d) Did you include the total amount of compute and the type of resources used (e

Revised: g., typeof GPUs, internal cluster, or cloud provider)?

Revised: [No]

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [Yes]

Revised: (b) Did you mention the license of the assets?

Revised: [Yes]

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [Yes]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [N/A]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: information or offensive content?

Revised: [N/A]

Revised: If you used crowdsourcing or conducted research with human subjects...

Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Revised: applicable?

Revised: [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?

Revised: [N/A]