Single-phase deep learning in cortico-cortical networks
Anonymous Author(s)
AffiliationAddress email
Abstract
The error-backpropagation (backprop) algorithm remains the most common solution to the credit assignment problem in artificial neural networks.
In neuroscience,it is unclear whether the brain could adopt a similar strategy to correctly modify itssynapses.
Recent models have attempted to bridge this gap while being consistentwith a range of experimental observations.
However, these models are either unableto effectively backpropagate error signals across multiple layers or require a multiphase learning process, neither of which are reminiscent of learning in the brain.
Here, we introduce a new model, bursting cortico-cortical networks (BurstCCN),which solves these issues by integrating known properties of cortical networksnamely bursting activity, short-term plasticity (STP) and dendrite-targeting interneurons.
BurstCCN relies on burst multiplexing via connection-type-specific STP topropagate backprop-like error signals within deep cortical networks.
These errorsignals are encoded at distal dendrites and induce burst-dependent plasticity as aresult of excitatory-inhibitory topdown inputs.
First, we demonstrate that our modelcan effectively backpropagate errors through multiple layers using a single-phaselearning process.
Next, we show both empirically and analytically that learning inour model approximates backprop-derived gradients.
Finally, we demonstrate thatour model is capable of learning complex image classification tasks (MNIST and
CIFAR-10).
Overall, our results suggest that cortical features across sub-cellular,cellular, microcircuit and systems levels jointly underlie single-phase efficient deeplearning in the brain.
Introduction
For effective learning, synaptic modifications throughout the brain should result in improved behavioural function.
This requires a process by which credit should be assigned to synapses giventheir contribution to behavioural output [11, 18, 19].
In multilayer networks, credit assignment isparticularly challenging as the impact of changing a synaptic connection depends on its downstreambrain areas.
Classical local Hebbian plasticity rules, even when coupled with global neuromodulatoryfactors, are unable to communicate enough information for effective credit assignment throughmultiple layers of processing [11].
In machine learning, the error-backpropagation (backprop) algorithm is the most successful solution to the credit assignment problem.
However, it relies on anumber of biologically implausible assumptions to compute gradient information used for synapticupdates.
Previous work has attempted to address these implausibilities but important issues remainopen in terms of mapping backprop to the neuronal physiology.
Earlier attempts relied on mappingbackprop to the brain by using single-compartment neuron models [10, 17].
This poses a problem assingle-compartment neurons are unable to simultaneously store the necessary inference and creditassignment signals.
One solution is to model neurons with an apical dendritic compartment thatseparately stores credit information [17, 20], supported by the electrotonic separation of the somaand apical dendrites [24].
These distal credit signals can then be communicated to the soma throughnon-linear dendritic events that trigger bursting at the soma [25], thereby inducing long-term synaptic plasticity [21].
In particular, two recent approaches, Error-encoding Dendritic Networks (EDNs) [20]and Burstprop [15], have demonstrated how such multi-compartment neuron models can be usedto construct networks capable of backprop-like credit assignment.
EDNs encode credit signals atapical dendrites resulting from the mismatch between dendritic-targeting interneuron activity anddownstream activity.
Burstprop uses the mechanism of bursting, controlled by dendritic excitability,to communicate credit signals.
However, these models still have major issues, such as the inabilityto effectively backpropagate error signals through many layers (EDNs) and the requirement for amulti-phase learning process (Burstprop).
Here, we propose a new model called the bursting cortico-cortical network (BurstCCN) as a solutionto the credit assignment problem which addresses several outstanding issues of current biologicallyplausible backprop research.
Our model builds upon prior multi-compartment neuron models [15, 20]:it encodes credit signals in distal dendritic compartments which trigger bursting activity at the somaticlevel to drive backprop-like synaptic updates.
We demonstrate that combining well-establishedproperties of cortical neurons such as bursting activity, short-term plasticity (STP) and dendritetargeting interneurons provides a biologically plausible mechanism for performing credit assignment.
In contrast to previous models, BurstCCN is highly effective at backpropagating credit signals inmulti-layer architectures while only requiring a single-phase learning process.
We implement multipleversions of the BurstCCN at different levels of abstraction in order to demonstrate some of its keyproperties and empirically confirm our theoretically motivated claims.
First, we used a spike-based implementation of the BurstCCN to demonstrate its ability to learnwithout the need for multiple phases.
We further show the importance of this single-phase learningby training a continuous-time rated-based version of the BurstCCN on an continuous-time nonlinear regression task.
Next, we show empirically and analytically that our model’s dynamics resultin learning that approximately follows backprop-derived gradients.
Finally, we use a simplifieddiscrete-time BurstCCN implementation to demonstrate that the model achieves good performancein non-trivial image classification tasks (MNIST and CIFAR-10), even in the presence of randomfeedback synaptic weights.
Bursting Cortico-Cortical Networks2.1 Burst Ensemble Multiplexing
Burst Ensemble Multiplexing (BEM) [14] refers to the idea that ensembles of cortical neurons arecapable of simultaneously representing multiple distinct signals within the patterns of their spikingactivity.
Typically, pyramidal cells receive top-down and bottom-up signals into their apical and basaldendrites, respectively.
Bottom-up basal inputs affect the rate of spiking and top-down apical inputsconvert these somatically induced spikes into high-frequency bursts.
Postsynaptic populations canthen use STP to decode these distinct signals from the overall spiking activity.
The BurstCCN uses the concept of BEM in a similar way to Burstprop [15] in which ensemblesof cells encode both feedforward inference signals and feedback error signals.
The model encodesthese signals as the rates of events and bursts , respectively, across the ensembles.
Here, the specificdefinition of a burst is a collection of spikes with interspike intervals less than 16ms and an event iseither a burst or a single isolated spike (i.e. a spike not followed or preceded by another within 16ms).
The burst probability of an ensemble is defined as the probability that an event at a given time is aburst and is computed as a ratio of the event rate ( e ) and burst rate ( b ): p = b / e .
2.2 Rate-based BurstCCN
In our discrete-time implementation of the rate-based BurstCCN, example input-output pairs areprocessed independently in discrete timesteps.
For each example, the event rates of the input layer,e 0 , encode the input stimulus.
The model then updates each layer consecutively, equivalent to thatof a standard feedforward artificial neural network (Fig. 1A).
Specifically, somatic potentials arecomputed by integrating basal input as v l =
W l e l − 1 where W l are short-term depressing (STD)feedforward weights from layer l − 1 to layer l .
The STD nature of these weights ensures that onlyevent rate information propagates forwards.
Each layer’s event rates are then computed by applyinga non-linear activation f function to the somatic potentials, e l = f ( v l ) .
These linear-nonlinear output teacher target layer layer layer (input ) (hidden ) (output ) short-
[Figure]
depression (events) short- [Figure] facilitation (bursts) layer layer
W
W
Q
Yvuv
IN layerlayer
W
Wvv
LTP
LTD layerlayer
W
Wvv
LTP LTD to ultimately obtain the output layer event rates,e L , where L denotes the total number of layers.
The desired target output of the network, e target , is compared to the output layer event rates toproduce a signed error, e target − e L , which is used as a teaching signal.
This error information is thenpropagated backwards through each layer in the network by altering the apical dendritic compartmentpotential and, as a result, the burst probability of each pyramidal ensemble.
At the output layer,the burst probability is computed directly as p L = p bL + p bL ⊙ ( e target − e L ) ⊙ h ( e L ) where ⊙denotes the element-wise product, p bL represents the baseline burst probability in the absence of anyteaching signal and h ( e l ) =
f ′ ( v l ) ⊙ e − 1 l .
These burst probabilities are used at the output layer( l = L ) to compute the burst rates as b l = e l ⊙ p l which are decoded and sent backwards to layerl − 1 apical dendrites by a set of short-term facilitating (STF) feedback weights, Y l − 1 .
The eventrates are also send backwards through a separate set of STD feedback weights, Q l − 1 , leading to aapical dendritic potential in the previous layer of u l − 1 =
Q l − 1 e l − Y l − 1 b l .
This determines thelayer’s burst probabilities which are computed as p l − 1 = ¯ σ ( u l − 1 ⊙ h ( e l − 1 )) where ¯ σ denotesthe sigmoid function, σ , with scaling and offset parameters, ¯ σ ( x ) = σ ( αx + β ) ([15]; see SM).
The same process is repeated backwards for each layer until the input layer to obtain their dendriticpotentials and burst probabilities.
We interpret the STF feedback connections as being provided via atype of dendrite-targeting interneuron and STD feedback as direct connections in line with recentexperimental studies [5, 9, 12, 13, 16, 26].
After the error information has been propagated backwards, feedforward synaptic weight changes arecomputed using a burst-dependent synaptic plasticity rule:
[Equation]
where η ( W ) is a learning rate and T is the transpose operation.
Importantly, the learning ruledepends on the change in burst probability from the predefined layer-wise baseline burst probability,p bl =
p bl (1 , . .
. , 1) T , which represents the signed error signal required for backprop-like learning.
Consequently, when we make both pre- and postsynaptic cells fire following Poisson statistics weobtain long-term depression and long-term potentiation for low and high firing rates, respectively(Fig.
1B).
This is in line with a large number of experimental studies at cortical synapses [3, 22].
It can be shown that the updates produced by this learning rule approximate those obtained by thebackpropagation algorithm in the weak-feedback case (see Section 3.3.1 and SM).
In the absence of a teaching signal, it is important for pyramidal ensembles to produce a baselinelevel of bursting such that no weight changes occur (cf.
Eq. 1).
This holds true for the output layeras there are no other inputs onto the apical dendrites.
However, for the hidden layers the event ratesignals through Q and the burst rate signals through Y need to exactly cancel each other out such thatthe apical dendritic potentials are at rest (i.e. u = 0 ).
For any Y weights, there is always an optimalset of Q weights that will produce this exact cancellation regardless of the event rates propagatingthrough the network.
Specifically, they must be set as Q l = p bl Y l which we refer to as the weightsbeing in a Q-Y symmetric state.
As it is not biologically plausible for the Q synapses to have directknowledge of Y , inspired by earlier work [20, 23] we use a learning rule for Q that provides thiscancellation:
[Equation]
which explicitly aims to silence the apical potentials (Fig. 1C).
In the absence of a teaching signalat the output layer, all Q weights will eventually converge to their optimal values and achieve asymmetric state (i.e. u = 0 ) under reasonable assumptions (see SM).
Note that we could similarlyhave added this learning rule on the Y feedback weights to cancel the Q weights, which producessimilar results (see SM).
When teaching signals are applied at the output layer, it is important to note that only the burstingactivity propagated through the Y connections changes because the event rates through Q areunaffected by the dendritic activity.
This enables single-phase learning as the symmetry in the twofeedback connection types ( Q and Y ) can be exploited to directly compare without teacher signals(i.e.
at baseline) to with teacher signals.
Details of the continuous time implementation can be found in the Supplementary Materials.
Spiking BurstCCN For a spiking implementation of BurstCCN, we adapted our burst-dependentsynaptic plasticity rule in Equation 1 (see SM).
Unlike the two rate-based implementations, the spiking BurstCCN more accurately models the internal neuron spiking dynamics instead of abstracting thesedetails away and only considering the ensemble-level behaviour.
Neurons are modelled with twocompartments corresponding to the soma and apical dendrites and spikes are generated when asomatic threshold potential is met.
Event and burst rates are no longer computed explicitly as separatequantities and are instead dictated by the patterns of bursting activity which emerge from the dynamics(see SM for more details).
Results
3.1 BurstCCN can learn with a single learning phase
A key motivation for developing the BurstCCN was to design a model capable of learning withoutthe need for separate learning phases, while being consistent with a range of cortical features acrossmultiple levels.
To demonstrate that our model can perform single-phase learning, we trained thespiking-version of our model on the XOR classification task and contrasted it with Burstprop, whichrequires a two-phase learning process (Fig. 2).
In both single and two-phase learning regimes, theinput stimulus is presented for a total of 8s before the next example is shown.
The two-phase learningregime has an initial prediction phase, lasting 7.2s for each input presentation, where plasticity isswitched off throughout the network and the output neurons do not receive any teaching signals(Fig.
2A).
This is followed by a teacher phase for the remaining 0.8s where plasticity is restored andteaching signals are delivered at the output.
The single-phase regime removes the initial predictionphase and extends the teacher phase to the full duration of the input stimulus (Fig. 2B).
Our results show that both models were capable of successfully learning the task in the two-phaseregime as indicated by the high output event rates in response to the (0 , 1) and (1 , 0) inputs and low event rates for the (0 , 0) and (1 , 1) inputs (Fig.
2C).
However, when training in the single-phase regime, only BurstCCN was able to learn the task (Fig.
2D).
The inability of Burstprop to learn the task can be explained by comparing the moving average of the burst probability ( p ) with the actualburst probability ( p ) which determines the sign of synaptic weight updates (Fig.
2D).
Burstpropfailed to learn in the single-phase learning setup due to the need to have a stable representation of pwithout a teaching signal, which becomes problematic in this case.
3.2 BurstCCN can learn with dynamic input-output
Typically, studies that have attempted to solve the credit assignment problem with biologicallyplausible implementations of backprop make an implicit assumption that during learning there isa period where the continuous-time input stream is fixed [15, 20].
This is required in most casesto allow the network to stabilise its activities before learning can take place.
With single-phaselearning, we can relax this assumption which enables learning in conditions where the inputs andtheir corresponding teaching signals are dynamically changing over time.
We assessed this ability bytraining the continuous-time BurstCCN (see SM) on an online non-linear regression task (Fig. 3).
Thistask consisted of three sinusoidal inputs, x i = sin ( α i t
+ β i ) , with random frequencies α i ∼ U (0 , π 20 )and phase offsets
β i ∼ U (0 , 2 π ) (Fig.
3A).
The network had a single output unit for which a nontrivial target was obtained by passing the same inputs to a 3-25-1 artificial neural network (ANN).
This approximates a setting in which a given cortical area learns to regress its input onto the activityof another cortical area.
The ANN weights were randomly initialised with w 1 ij ∼ U ( − √ 3 , √ 3) forthe first layer and w 2 ij ∼ U ( − 0 .
for the second layer.
Despite the BurstCCN initially producingoutputs that were significantly different to the target (Fig.
3C), the results showed that over training itlearned to produce output patterns that closely matched the non-linear and dynamic target (Fig.
3B,D).
This highlights that the BurstCCN is capable of adequately backpropagating useful error signalswhen both inputs and teaching signals are constantly changing.
[Figure]
Three sinusoidal waves random frequencies learn
3.3 Feedback plasticity rule facilitates alignment to backprop updates
Next, we wanted to understand how well our model approximates backprop.
As stated above, thepurpose of the learning rule for the feedback Q STD connections (Eq. 2) is to silence the apicalcompartment of every ensemble by cancelling activity through the feedback Y STF connections.
When a teaching signal is applied, this becomes important for computing the correct local errorsignal that is used for learning and backpropagated to previous layers.
Here we show analytically andempirically using the discrete version of the model how the errors computed by our model relate tobackprop.
3.3.1 BurstCCN with weak feedback approximates backpropagation algorithm
With some small assumptions, it can be shown analytically that the weight update term defined in Equation 1 approximates true backpropagated backpropagation; that is, ∆ w ij ≈ − ∂E task
∂w ij where E task =
|| e target − e L || 2 is the task error defined at the last layer.
In particular, if we focus onthe change in burst rate δ b l := ( p l − p b l ) ⊙ e l (as in right hand side of Eq. 1) and assume thatthe feedback weights are optimally aligned, i.e. Q l = p bl Y l , we can derive the following iterativerelationship:
[Equation]
Since δ b L = ∂E task ∂ v L (by construction), Equation 3 thus replicates the backpropagation of erroralgorithm to a third order 1 approximation with respect to the apical potentials u l when the feedforwardand feedback weights are perfectly aligned, Y l = − W Tl +1 .
We refer to this as the W-Y symmetricstate.
Applying the chain rule ∂E task ∂ W l = ∂E task ∂ v l ( ∂ v l ∂ W l ) T and noting that ∂ v l ∂ W l = e l − 1 , the linkbetween the weight update defined in Equation 1 to that used in error backpropagation then becomesclear:
[Equation]
[Figure] as To approach an equivalence to backpropagation, however, it remains to show that the apical potentialsu l are indeed appropriately small (so that || u 3 l || → 0 in Equation 3) for each layer l , which we canbe shown by induction.
Assuming that the apical potentials of the above layer u l +1 are small wecan derive the recursive relationship u l ≈ f ′ ( v l +1 ) ⊙ ( − Y l ) u l +1 .
This then highlights that if thederivative of the activation function f ′ is bounded by 1 (as is true for sigmoid and most activationfunctions) and the feedback weights Y l are reasonably small, then we have || u l || ≤ || u
l [-+
-]+1 ||
(see SM for full derivation). Therefore, if the original error gradient defined at the last layer ∂E task ∂ e N (andtherefore u L − 1 ) is appropriately small, our condition that u l is small for each layer l is satisfied and∆ W BurstCNN l ≈ ∆ W backprop l .
3.3.2 Learning Q feedback connections better approximates backprop-derived gradients
Next, we empirically evaluated our feedback plasticity rule by updating only the Q weights of arandomly initialised 5-layer discrete-time BurstCCN with all other weight types ( W and Y ) fixed.
We used multiple initialisations and training regimes to understand how the plasticity rule behaves indifferent scenarios.
The network was either initialised in the W - Y symmetric state or with randomfeedback weights (where Y l ̸ = − W Tl +1 ).
We computed the angle between the update calculatedfrom the feedforward plasticity rule (Eq. 1) and either backprop or feedback alignment [10] for thesymmetric and random configurations, respectively.
We examined both cases with a teaching signalat the output layer (Fig. 4A-D) and in the theoretically ideal case for learning Q , where no teachingsignal is present (Fig.
4E-H).
In all cases, as the alignment between the Q and Y connections improved (Fig.
4A,E), the apicalpotential decreased (Fig.
4B,F) and this resulted in updates that more closely aligned to backprop (Fig.
4C,G) and feedback alignment (Fig.
4D,H).
In the absence of a teaching signal, thisalignment angle to both backprop and feedback alignment eventually became very small which showsthat the model approximates backprop updates supporting our analytical results (Fig.
4C-D).
Despiteproducing less aligned feedforward updates in the presence of a teaching signal, the updates computedwere still informative since they were consistently well below 90 ◦ of the direction of steepest descent(Fig.
4G).
[Figure]
Next, to test whether our model can indeed perform backprop-like deep learning we trained a numberof (discrete-time) BurstCCN architectures on the MNIST handwritten digit classification task [7].
Wecompared the BurstCCN with Burstprop network [15] and EDNs [20] using similar architectures (see
SM).
We focus on the more biologically plausible case of using random fixed feedback weights (i.e.feedback alignment [10]; see SM for symmetric feedback weight case) with the remaining connectiontypes updated using their own plasticity rules as given by the different models.
To be able to test ourmodel in its idealised case we also tested a model for which the feedback STD weights ( Q ) were fixedin the Q - Y symmetric state (see Section 2.2), we denote this model as "BurstCCN ( Q - Y sym)".
Using 5-layer networks, the BurstCCN obtained a test error of 1.84 ± 0.01%, comparable to that of
Burstprop with 1.75 ± 0.01% and significantly outperforming the EDN with 10.65 ± 0.09% (Fig.
5A).
As the network depth increased, both BurstCCN and Burstprop retained high performances but the
EDN showed a substantial decay in performance with deeper networks (Fig. 5B).
We then comparedthe alignment between the models and backprop.
For the 5-layer networks, Burstprop’s updateswere most closely aligned to backprop, followed by the two BurstCCN models which all vastlyoutperformed the EDN (Fig.
5C).
As expected, the BurstCCN with Q - Y symmetry could betterpropagate error signals.
By increasing the network depth, we demonstrated that it was more difficultto produce updates that were closely aligned to backprop.
However, we show that the BurstCCN wasstill capable of backpropagating useful error signals in relatively deep networks (Fig. 5D).
3.4.2 CIFAR-10
Next, we investigated the capabilities of the BurstCCN on more challenging tasks that are commonlytested in deep learning.
We constructed a deep network consisting of three convolutional layersfollowed by a fully-connected hidden layer and output layer (Fig. 6A).
We trained BurstCCN and
Burstprop models using this network architecture on the CIFAR-10 image classification task [6].
BurstCCN ( Q - Y sym) was trained in the Q - Y symmetric regime whereas BurstCCN was initialisedin this state and Q weights were then updated using the corresponding plasticity rule.
All modeltypes were tested with two feedback weight regimes: W - Y symmetric and random fixed Y feedbackweights (i.e. feedback alignment).
After training, we observed a test error of 38.99 ± 0.18% for BurstCCN with random feedbackweights, similar to performances achieved by feedback alignment with an ANN (36.30 ± 0.16%)and Burstprop (41.32 ± 0.14% ) (Fig. 6B).
For the W - Y symmetric regime which most resemblesbackprop, BurstCCN (22.92 ± 0.03%) performed significantly better than all feedback alignmentsetups and, once again, obtained a similar error to the symmetric ANN (22.62 ± 0.10%) and Burstprop(24.15 ± 0.17%) cases.
The increased performance seen in the symmetric setups can be explained bythe improvement in alignment to backprop updates (Fig.
6C).
[Figure]
Conclusions and discussion
We introduced a new model capable of backprop-like credit assignment by integrating knownproperties of cortical networks.
We have shown that by combining specific biological mechanismssuch as bursting, STP and dendrite-targeting inhibition it is possible to construct a model that learnseffectively in a continuous setting that is reminiscent of learning in the brain.
Moreover, we havedemonstrated that such a model can learn complex image classification tasks in deep networks.
Our model proposed specific STP dynamics on the feedback connections and suggested a key rolefor dendrite-targeting interneurons.
There is evidence that SST-positive Martinotti cells receive
STF top-down connections whereas top-down projections onto pyramidal cells exhibit more STDdynamics as required by our model [5, 9, 12, 13, 16, 26]. This is in contrast with Burstprop whichrequires STF feedback connections.
In future work, it would be interesting to model the specificneuron types for each connection to satisfy Dale’s law and further increase biological plausibility.
A prediction from our model is that manipulations of these interneurons with STF connections wouldlead to disruptions in burst decoding from the layer (brain area) above thereby obstructing learningin the brain area below.
Additionally, as error signals alter the level of bursting in the network, themodel predicts that the variance in bursting activity and the distal dendritic potentials would correlatewith the level of error in the network during learning.
Although our model captures a wide range of biological features, some biological implausibilitiesremain.
Currently, we use feedback alignment to provide a solution to the weight transport problem [4]but this has a substantial impact on performance, particularly in more challenging tasks.
Therefore,it would be important to explore some of the recently introduced plausible feedback learning rules[1, 2, 8], that can be combined with our proposed feedback learning rule to outperform feedbackalignment [10].
Overall, our work provides a novel solution to the credit assignment problem and suggests that arange of cortical features from sub-cellular to the systems level jointly underlie single-phase efficientdeep learning in the brain.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .•
Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes](c)
Did you discuss any potential negative societal impacts of your work?
[N/A](d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See SM andmain text.
(b) Did you include complete proofs of all theoretical results?
[Yes] See SM.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
[Yes] See SM.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See SM.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)?
[Yes] Its also detailed in the captions how many seeds we used and what the error bars represent.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See SM.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] See SM.
(b) Did you mention the license of the assets?
[Yes] See SM.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A](e)
Did you discuss whether the data you are using/curating contains personally identifiableinformation or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]