Single-phase deep learning in cortico-cortical networks
Anonymous Author(s)
AffiliationAddress email
Abstract
The error-backpropagation (backprop) algorithm remains the most common solution to the credit assignment problem in artificial neural networks.
In neuroscience,it is unclear whether the brain could adopt a similar strategy to correctly modify itssynapses.
Recent models have attempted to bridge this gap while being consistentwith a range of experimental observations.
However, these models are either unableto effectively backpropagate error signals across multiple layers or require a multiphase learning process, neither of which are reminiscent of learning in the brain.
Here, we introduce a new model, bursting cortico-cortical networks (BurstCCN),which solves these issues by integrating known properties of cortical networksnamely bursting activity, short-term plasticity (STP) and dendrite-targeting interneurons.
BurstCCN relies on burst multiplexing via connection-type-specific STP topropagate backprop-like error signals within deep cortical networks.
These errorsignals are encoded at distal dendrites and induce burst-dependent plasticity as aresult of excitatory-inhibitory topdown inputs.
First, we demonstrate that our modelcan effectively backpropagate errors through multiple layers using a single-phaselearning process.
Next, we show both empirically and analytically that learning inour model approximates backprop-derived gradients.
Finally, we demonstrate thatour model is capable of learning complex image classification tasks (MNIST and
CIFAR-10).
Overall, our results suggest that cortical features across sub-cellular,cellular, microcircuit and systems levels jointly underlie single-phase efficient deeplearning in the brain.
Introduction
For effective learning, synaptic modifications throughout the brain should result in improved behavioural function.
This requires a process by which credit should be assigned to synapses given theircontribution to behavioural output [1–3].
In multilayer networks, credit assignment is particularlychallenging as the impact of changing a synaptic connection depends on its downstream brain areas.
Classical local Hebbian plasticity rules, even when coupled with global neuromodulatory factors, areunable to communicate enough information for effective credit assignment through multiple layersof processing [3].
In machine learning, the error-backpropagation (backprop) algorithm is the mostsuccessful solution to the credit assignment problem.
However, it relies on a number of biologicallyimplausible assumptions to compute gradient information used for synaptic updates.
Previous workhas attempted to address these implausibilities but important issues remain open in terms of mappingbackprop to the neuronal physiology.
Earlier attempts relied on mapping backprop to the brainby using single-compartment neuron models [4, 5].
This poses a problem as single-compartmentneurons are unable to simultaneously store the necessary inference and credit assignment signals.
One solution is to model neurons with an apical dendritic compartment that separately stores creditinformation [5, 6], supported by the electrotonic separation of the soma and apical dendrites [7].
These distal credit signals can then be communicated to the soma through non-linear dendritic eventsthat trigger bursting at the soma [8], thereby inducing long-term synaptic plasticity [9].
In particular, two recent approaches, Error-encoding Dendritic Networks (EDNs) [6] and Burstprop [10], havedemonstrated how such multi-compartment neuron models can be used to construct networks capableof backprop-like credit assignment.
EDNs encode credit signals at apical dendrites resulting fromthe mismatch between dendritic-targeting interneuron activity and downstream activity.
Burstpropuses the mechanism of bursting, controlled by dendritic excitability, to communicate credit signals.
However, these models still have major issues, such as the inability to effectively backpropagateerror signals through many layers (EDNs) and the requirement for a multi-phase learning process(Burstprop).
Here, we propose a new model called the Bursting Cortico-Cortical Network (BurstCCN) as asolution to the credit assignment problem which addresses several outstanding issues of currentbiologically plausible backprop research.
Our model builds upon prior multi-compartment neuronmodels [6, 10]: it encodes credit signals in distal dendritic compartments which trigger burstingactivity at the somatic level to drive backprop-like synaptic updates.
We demonstrate that combiningwell-established properties of cortical neurons such as bursting activity, short-term plasticity (STP)and dendrite-targeting interneurons provides a biologically plausible mechanism for performingcredit assignment.
In contrast to previous models, BurstCCN is highly effective at backpropagatingcredit signals in multi-layer architectures while only requiring a single-phase learning process.
Weimplement multiple versions of the BurstCCN at different levels of abstraction in order to demonstratesome of its key properties and empirically confirm our theoretically motivated claims.
First, we used a spike-based implementation of the BurstCCN to demonstrate its ability to learnwithout the need for multiple phases.
We further show the importance of this single-phase learningby training a continuous-time rated-based version of the BurstCCN on an continuous-time nonlinear regression task.
Next, we show empirically and analytically that our model’s dynamics resultin learning that approximately follows backprop-derived gradients.
Finally, we use a simplifieddiscrete-time BurstCCN implementation to demonstrate that the model achieves good performancein non-trivial image classification tasks (MNIST and CIFAR-10), even in the presence of randomfeedback synaptic weights.
Bursting Cortico-Cortical Networks2.1 Burst Ensemble Multiplexing
Burst Ensemble Multiplexing (BEM) [11] refers to the idea that ensembles of cortical neurons arecapable of simultaneously representing multiple distinct signals within the patterns of their spikingactivity.
Typically, pyramidal cells receive top-down and bottom-up signals into their apical and basaldendrites, respectively.
Bottom-up basal inputs affect the rate of spiking and top-down apical inputsconvert these somatically induced spikes into high-frequency bursts.
Postsynaptic populations canthen use STP to decode these distinct signals from the overall spiking activity.
The BurstCCN uses the concept of BEM in a similar way to Burstprop [10] in which ensemblesof cells encode both feedforward inference signals and feedback error signals.
The model encodesthese signals as the rates of events and bursts , respectively, across the ensembles.
Here, the specificdefinition of a burst is a collection of spikes with interspike intervals less than 16ms and an event iseither a burst or a single isolated spike (i.e. a spike not followed or preceded by another within 16ms).
The burst probability of an ensemble is defined as the probability that an event at a given time is aburst and is computed as a ratio of the event rate ( e ) and burst rate ( b ): p = b / e .
2.2 Rate-based BurstCCN
In our discrete-time implementation of the rate-based BurstCCN, example input-output pairs areprocessed independently in discrete timesteps.
For each example, the event rates of the input layer, e 0 ,encode the input stimulus.
The model then computes each subsequent layer’s activities, equivalent tothat of a standard feedforward artificial neural network (Fig. 1A).
Specifically, somatic potentials arecomputed by integrating basal input as v l =
W l e l − 1 where W l are short-term depressing (STD)feedforward weights from layer l − 1 to layer l .
The STD nature of these weights ensures that onlyevent rate information propagates forwards.
Each layer’s event rates are then computed by applying anon-linear activation function, f , to the somatic potentials, e l = f ( v l ) .
These linear and nonlinear output teacher target layer layer layer (input ) (hidden ) (output )
[Figure]
layer layer
W
W
Q
Yvuv
IN layerlayer
W
Wvv
LTP
LTD layerlayer
W
Wvv
LTP LTD obtain the output layer event rates,e L , where L denotes the total number of layers.
The desired target output of the network, e target , is compared to the output layer event rates toproduce a signed error, e target − e L , which is used as a teaching signal.
This error information is thenpropagated backwards through each layer in the network by altering the apical dendritic compartmentpotential and, as a result, the burst probability of each pyramidal ensemble.
At the output layer, theburst probability is computed directly as p L = p bL + p bL ⊙ ( e target − e L ) ⊙ h ( e L ) where ⊙ denotesthe element-wise product, p bL represents the baseline burst probability in the absence of any teachingsignal and h ( e l ) =
f ′ ( v l ) ⊙ e − 1 l .
These burst probabilities are used at the output layer ( l = L )to compute the burst rates as b l = e l ⊙ p l which are decoded and sent backwards to layer l − 1apical dendrites by a set of short-term facilitating (STF) feedback weights, Y l − 1 .
The event ratesare also send backwards through a separate set of STD feedback weights, Q l − 1 , leading to a apicaldendritic potential in the previous layer of u l − 1 =
Q l − 1 e l − Y l − 1 b l .
This determines the layer’sburst probabilities which are computed as p l − 1 = ¯ σ ( u l − 1 ⊙ h ( e l − 1 )) where ¯ σ denotes the sigmoidfunction, σ , with scaling and offset parameters, ¯ σ ( x ) = σ ( αx + β ) ([10]; see SM, Section B).
Thesame process is repeated backwards for each layer until the input layer to obtain their dendriticpotentials and burst probabilities.
We interpret the STF feedback connections as being provided via atype of dendrite-targeting interneuron and STD feedback as direct connections in line with recentexperimental studies [12–17].
After the error information has been propagated backwards, feedforward synaptic weight changes arecomputed using a burst-dependent synaptic plasticity rule:
[Equation]
where η ( W ) is a learning rate and T is the transpose operation.
Importantly, the learning ruledepends on the change in burst probability from the predefined layer-wise baseline burst probability, p bl =
p bl (1 , . .
. , 1) T , which represents the signed error signal required for backprop-like learning.
Consequently, when we make both pre- and postsynaptic cells fire following Poisson statistics weobtain long-term depression and long-term potentiation for low and high firing rates, respectively(Fig.
1B).
This is in line with a large number of experimental studies at cortical synapses [18, 19].
It can be shown that the updates produced by this learning rule approximate those obtained by thebackpropagation algorithm in the weak-feedback case (see Section 3.3.1 and SM, Section B).
In the absence of a teaching signal, it is important for pyramidal ensembles to produce a baselinelevel of bursting such that no weight changes occur (cf.
Eq. 1).
This holds true for the output layeras there are no other inputs onto the apical dendrites.
However, for the hidden layers the event ratesignals through Q and the burst rate signals through Y need to exactly cancel each other out such thatthe apical dendritic potentials are at rest (i.e. u = 0 ).
For any Y weights, there is always an optimalset of Q weights that will produce this exact cancellation regardless of the event rates propagatingthrough the network.
Specifically, they must be set as Q l = p bl Y l which we refer to as the weightsbeing in a Q-Y symmetric state.
As it is not biologically plausible for the Q synapses to have directknowledge of Y , inspired by earlier work [6, 20] we use a learning rule for Q that provides thiscancellation:
[Equation]
which explicitly aims to silence the apical potentials (Fig. 1C).
In the absence of a teaching signalat the output layer, all Q weights will eventually converge to their optimal values and achieve asymmetric state (i.e. u = 0 ) under reasonable assumptions (see SM, Section B.2).
Note that wecould similarly have added this learning rule on the Y feedback weights to cancel the Q weights,which produces similar results (Fig. S1).
When teaching signals are applied at the output layer, it is important to note that only the burstingactivity propagated through the Y connections changes because the event rates through Q areunaffected by the dendritic activity.
This enables single-phase learning as the symmetry in the twofeedback connection types ( Q and Y ) can be exploited to directly compare without teacher signals(i.e.
at baseline) to with teacher signals.
Details of the continuous time implementation can be found in the Supplementary Materials.
For a spiking implementation of BurstCCN, we adapted our burst-dependent synaptic plasticity rulein Equation 1 (see SM, Eq. 12).
Unlike the two rate-based implementations, the spiking BurstCCNmore accurately models the internal neuron spiking dynamics instead of abstracting these details awayand only considering the ensemble-level behaviour.
Neurons are modelled with two compartmentscorresponding to the soma and apical dendrites and spikes are generated when a somatic thresholdpotential is met (see Section A.2 for more details).
Results
3.1 BurstCCN can learn with a single learning phase
A key motivation for developing the BurstCCN was to design a model capable of learning withoutthe need for separate learning phases, while being consistent with a range of cortical features acrossmultiple levels.
To demonstrate that our model can perform single-phase learning, we trained thespiking version of our model on the XOR classification task and contrasted it with Burstprop, whichrequires a two-phase learning process (Fig. 2).
In both single- and two-phase learning regimes, theinput stimulus is presented for a total of 8s before the next example is shown.
The two-phase learningregime has an initial prediction phase, lasting 7.2s for each input presentation, where plasticity isswitched off throughout the network and the output neurons do not receive any teaching signals(Fig.
2A).
This is followed by a teacher phase for the remaining 0.8s where plasticity is restored andteaching signals are delivered at the output.
The single-phase regime removes the initial predictionphase and extends the teacher phase to the full duration of the input stimulus (Fig. 2B).
Our results show that both models were capable of successfully learning the task in the two-phaseregime as indicated by the high output event rates in response to the (0 , 1) and (1 , 0) inputs and lowevent rates for the (0 , 0) and (1 , 1) inputs (Fig.
2C).
However, when training in the single-phase regime, only BurstCCN was able to learn the task (Fig.
2D).
The inability of Burstprop to learn thetask can be explained by comparing the moving average of the burst probability ( p ) with the actualburst probability ( p ) which determines the sign of synaptic weight updates (Fig.
2D).
Burstpropfailed to learn in the single-phase learning setup due to the need to have a stable representation of pwithout a teaching signal, which becomes problematic in this case.
3.2 BurstCCN can learn with dynamic input-output
Typically, studies that have attempted to solve the credit assignment problem with biologicallyplausible implementations of backprop make an implicit assumption that during learning there isa period where the continuous-time input stream is fixed [6, 10].
This is required in most casesto allow the network to stabilise its activities before learning can take place.
With single-phaselearning, we can relax this assumption which enables learning in conditions where the inputs andtheir corresponding teaching signals are dynamically changing over time.
We assessed this ability bytraining the continuous-time BurstCCN (see SM, Section A.1) on an online non-linear regressiontask (Fig. 3).
This task consisted of three sinusoidal inputs, x i ( t ) = sin ( α i t
+ β i ) , with randomfrequencies α i ∼ U (0 , π 2 ) and phase offsets
β i ∼ U (0 , 2 π ) (Fig.
3A).
The network had a singleoutput unit for which a non-trivial target was obtained by passing the same inputs to a 3-25-1 artificialneural network (ANN).
This approximates a setting in which a given cortical area learns to regressits input onto the activity of another cortical area.
The ANN weights were randomly initialisedwith w 1 ij ∼ U ( −√ 3 , √ 3) for the first layer and w 2 ij ∼ U ( − 0 .
for the second layer.
Despitethe BurstCCN initially producing outputs that were significantly different to the target (Fig.
3C),the results showed that over training it learned to produce output patterns that closely matched thenon-linear and dynamic target (Fig.
3B,D).
This highlights that the BurstCCN is capable of adequatelybackpropagating useful error signals when both inputs and teaching signals are constantly changing.
[Figure]
random frequencies
3.3 Feedback plasticity rule facilitates alignment to backprop updates
Next, we wanted to understand how well our model approximates backprop.
As stated above, thepurpose of the learning rule for the feedback Q STD connections (Eq. 2) is to silence the apicalcompartment of every ensemble by cancelling activity through the feedback Y STF connections.
When a teaching signal is applied, this becomes important for computing the correct local errorsignal that is used for learning and backpropagated to previous layers.
Here we show analytically andempirically using the discrete version of the model how the errors computed by our model relate tobackprop.
3.3.1 BurstCCN with weak feedback approximates backpropagation algorithm
Under some small assumptions, we analytically show that the feedback pathway of BurstCCN isapproximately communicating the same error gradients that are computed by backprop. Specifically,we assume that the feedback weights are optimally aligned (i.e. Q l = p bl Y l ) and focus on the changein burst rate, δ b l := ( p l − p bl ) ⊙ e l .
If we let E task =
|| e L − e target ||
2 define the task errorthen, by construction, the change in burst rate at the output layer is equivalent to the negative errorgradient, δ b L = − ∂E task ∂ v L . For the hidden layers, we derive the following iterative relationship (see SM, Section B):
[Equation]
This approximates the same relationship present in backprop up to a third-order 1 term with respect tothe apical potentials u l if the feedback weights are set to be symmetric with the feedforward weights(i.e. Y l = − W Tl +1 ).
We refer to this as the W-Y symmetric state.
The link between weight updatesfrom simply performing gradient descent with backprop and the BurstCCN can be seen clearly:
[Equation]
It remains to be shown that the apical potentials, u l , of every layer are indeed appropriately small (sothat the approximation error, || u 3 l || , is small).
Under the assumption u l +1 is small, we can derive [Figure] as the recursive relationship u l ≈ f ′ ( v l +1 ) ⊙ ( − Y l ) u l +1 (see SM, Section B).
We show that if f ′ isbounded (as is the case for sigmoid and many activation functions) and the weights Y l are reasonablysmall then || u l || ≤ || u
l {++
+} || .
This means that if the error gradient at the output layer, ∂E task ∂ e N , isappropriately small then, by induction, u l is small for every layer and ∆ W BurstCNN l ≈ ∆ W backprop l .
3.3.2 Learning Q feedback connections better approximates backprop-derived gradients
We empirically evaluated our feedback plasticity rule by updating only the Q weights of a randomlyinitialised 5-layer discrete-time BurstCCN with all other weight types ( W and Y ) fixed.
We usedmultiple initialisations and training regimes to understand how the plasticity rule behaves in differentscenarios.
The network was either initialised in the W - Y symmetric state or with random feedbackweights (where Y l ̸ = − W Tl +1 ).
We computed the angle between the update calculated from thefeedforward plasticity rule (Eq. 1) and either backprop or feedback alignment [4] for the symmetricand random configurations, respectively.
We examined both cases: in the theoretically ideal case forlearning Q , where no teaching signal is present (Fig. 4A-D), and with a teaching signal at the outputlayer (Fig.
4E-H).
In all cases, as the alignment between the Q and Y connections improved (Fig.
4A,E), the apicalpotential decreased (Fig.
4B,F) and this resulted in updates that more closely aligned to backprop (Fig.
4C,G) and feedback alignment (Fig.
4D,H).
In the absence of a teaching signal, thisalignment angle to both backprop and feedback alignment eventually became very small which showsthat the model approximates backprop updates supporting our analytical results (Fig.
4C-D).
Despiteproducing less aligned feedforward updates in the presence of a teaching signal, the updates computedwere still informative since they were consistently well below 90 ◦ of the direction of steepest descent(Fig.
4G).
3.4 BurstCCN learns image classification tasks with multiple hidden layers3.4.1 MNIST
Next, to test whether our model can indeed perform backprop-like deep learning, we trained a numberof (discrete-time) BurstCCN architectures on the MNIST handwritten digit classification task [21].
We compared the BurstCCN with Burstprop [10] and EDNs [6] using similar architectures (see SM,
Section C.3.3).
We focus on the more biologically plausible case of using random fixed feedbackweights (i.e. feedback alignment [4]; see Fig. S2 for symmetric feedback weight case) with theremaining connection types updated using their own plasticity rules as given by the different models.
To be able to test our model in its idealised case we also tested a model for which the feedback STD weights ( Q ) were fixed in the Q - Y symmetric state (see Section 2.2), we denote this model as"BurstCCN ( Q - Y sym)".
Using 5-layer networks, the BurstCCN obtained a test error of 1.84 ± 0.01%, comparable to that of
Burstprop with 1.75 ± 0.01% and significantly outperforming the EDN with 10.65 ± 0.09% (Fig.
5A).
As the network depth increased, both BurstCCN and Burstprop retained high performances but the
EDN showed a substantial decay in performance with deeper networks (Fig. 5B).
In an idealised casefor the EDN, the disparity in performance and the effect of depth is less evident (Fig. S3).
We thencompared the alignment between the models and backprop.
For the 5-layer networks, Burstprop’supdates were most closely aligned to backprop, followed by the two BurstCCN models which allvastly outperformed the EDN (Fig.
5C).
As expected, the BurstCCN with Q - Y symmetry could betterpropagate error signals.
By increasing the network depth, we demonstrate that it was more difficult toproduce updates that were closely aligned to backprop.
However, we show that the BurstCCN wasstill capable of backpropagating useful error signals in relatively deep networks (Fig. 5D).
3.4.2 CIFAR-10
Finally, we investigated the capabilities of the BurstCCN on more challenging tasks that are commonlytested in deep learning.
We constructed a deep network consisting of three convolutional layersfollowed by a fully-connected hidden layer and output layer (Fig. 6A).
We trained BurstCCN and
Burstprop models using this network architecture on the CIFAR-10 image classification task [22].
BurstCCN ( Q - Y sym) was trained in the Q - Y symmetric regime whereas BurstCCN was initialisedin this state and Q weights were then updated using the corresponding plasticity rule.
All modeltypes were tested with two feedback weight regimes: W - Y symmetric and random fixed Y feedbackweights (i.e. feedback alignment).
After training, we observed a test error of 38.99 ± 0.18% for BurstCCN with random feedbackweights, similar to performances achieved by feedback alignment with an ANN (36.30 ± 0.16%)and Burstprop (41.32 ± 0.14% ) (Fig. 6B).
For the W - Y symmetric regime which most resemblesbackprop, BurstCCN (22.92 ± 0.03%) performed significantly better than all feedback alignmentsetups and, once again, obtained a similar error to the symmetric ANN (22.62 ± 0.10%) and Burstprop(24.15 ± 0.17%) cases.
The increased performance seen in the symmetric setups can be explained bythe improvement in alignment to backprop updates (Fig.
6C).
[Figure]
Conclusions and discussion
We introduced a new model capable of backprop-like credit assignment by integrating knownproperties of cortical networks.
We have shown that by combining specific biological mechanismssuch as bursting, STP and dendrite-targeting inhibition it is possible to construct a model that learnseffectively in a continuous setting that is reminiscent of learning in the brain.
Moreover, we havedemonstrated that such a model can learn complex image classification tasks in deep networks.
Our model proposes specific STP dynamics on the feedforward and feedback connections.
It requires STD on cortico-cortical projections onto pyramidal cells in line with experimental evidence [12–16]. In addition, it suggests a key role for dendrite-targeting interneurons such as SST-positive Martinotticells in the feedback pathway.
There is evidence that these interneurons receive STF top-downconnections whereas top-down projections onto pyramidal cells exhibit STD dynamics as required byour model [12–17].
In future work, it would be interesting to model the specific neuron types foreach connection to satisfy Dale’s law and further increase biological plausibility.
A prediction from our model is that manipulations of these interneurons with STF connections wouldlead to disruptions in burst decoding from the layer (brain area) above thereby obstructing learningin the brain area below.
Additionally, as error signals alter the level of bursting in the network, themodel predicts that the variance in bursting activity and the distal dendritic potentials would correlatewith the level of error in the network during learning.
Although our model captures a wide range of biological features, some biological implausibilities remain.
Currently, we use feedback alignment to provide a solution to the weight transport problem [23]but this has a substantial impact on performance, particularly in more challenging tasks.
Therefore,it would be important to explore some of the recently introduced plausible feedback learning rules[24–26] which could be used in conjunction with our proposed learning rules to outperform feedbackalignment [4].
Overall, our work provides a novel solution to the credit assignment problem and suggests that arange of cortical features from sub-cellular to the systems level jointly underlie single-phase, efficientdeep learning in the brain.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data areproprietary.•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes](c)
Did you discuss any potential negative societal impacts of your work?
[N/A](d)
Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]2.
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See SM andmain text.(b)
Did you include complete proofs of all theoretical results?
[Yes] See SM.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] See SM.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See SM.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] Its also detailed in the captions how many seeds we usedand what the error bars represent.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See SM.4.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes] See SM.
(b) Did you mention the license of the assets?
[Yes] See SM.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]