Learning on Arbitrary Graph Topologies via Predictive Coding
Abstract
Training with backpropagation (BP) in standard deep learning consists of two mainsteps: a forward pass that maps a data point to its prediction, and a backward passthat propagates the error of this prediction back through the network.
This process ishighly effective when the goal is to minimize a specific objective function.
However,it does not allow training on networks with cyclic or backward connections.
This isan obstacle to reaching brain-like capabilities, as the highly complex heterarchicalstructure of the neural connections in the neocortex are potentially fundamental forits effectiveness.
In this paper, we show how predictive coding (PC), a theory ofinformation processing in the cortex, can be used to perform inference and learningon arbitrary graph topologies.
We experimentally show how this formulation,called PC graphs , can be used to flexibly perform different tasks with the samenetwork by simply stimulating specific neurons.
This enables the model to bequeried on stimuli with different structures, such as partial images, images withlabels, or images without labels.
We conclude by investigating how the topology ofthe graph influences the final performance, and comparing against simple baselinestrained with BP.
Introduction
Classical deep learning has achieved remarkable results by training deep neural networks to minimizean objective function.
Here, every weight parameter gets updated to minimize this function usingreverse differentiation [1, 2].
However, in the brain, every synaptic connection is independentlyupdated to correct the behaviour of its post-synaptic neuron [3] using local information and itis unknown whether this process minimizes a global objective function.
The brain maintains aninternal model of the world, which constantly generates predictions of external stimuli.
When thepredictions differ from reality, the brain immediately corrects this error (difference between realityand prediction) by updating the strengths of the synaptic connections [4, 5, 6, 7].
This theory ofinformation processing, called predictive coding (PC) , is highly influential, despite experimentalevidence in the cortex being mixed [8, 9, 10, 11] and it is at the centre of a large amount of researchin computational neuroscience [12, 13, 14, 15, 16].
From the machine learning perspective, PC haspromising properties: it is able to achieve excellent results in classification [17] and memorization[18], and is able to process information in both a bottom up and a top down direction.
This lastproperty is fundamental for the functioning of different brain areas, such as the hippocampus [19, 18].
PC also shares the generalization capabilities of standard deep learning, as it is able to approximate
BP on any neural structure [20], and a variation of PC is able to exactly replicate the weight update of BP on any computational graph [21, 22].
Moreover, PC only uses local information to updatesynapses, allowing the network to be fully parallelized, and to train on networks with any topology.
[Figure]
Training on networks of any structure is notpossible in standard deep learning, where information only flows in one direction via the feedforward pass and then backpropagation (BP) isperformed in sequential steps backwards.
If acycle is present inside the computational graphof an artificial neural network (ANN), BP becomes stuck in an infinite loop.
More generally, the computational graph of any function
F : R d → R k is a poset, and hence acyclic.
While the problem of training on some specificcyclic structures has been partially addressedusing BP through time [24] on sequential data,the restriction to hierarchical architectures may present a limitation to reaching brain-like intelligence,since the human brain has an extremely complex and entangled neural structure that is heterarchicallyorganized with small-world connections [23]—a topology that is likely highly optimized by evolution.
This shape of structural brain networks, shown in Fig.
1, generates a unique communication dynamicsthat is fundamental for information processing in the brain, as different aspects of network topologyimply different communication mechanisms, and hence perform different tasks [23].
The heterarchical topology of brain networks has motivated research that aims to develop learning methods ongraphs of any topology.
A popular example is the assembly calculus [25, 26], a Hebbian learningmethod that can perform different operations implicated in cognitive phenomena.
In this work, we address this problem by proposing PC graphs , a structure that allows to train onany directed graph using the original (error-driven) framework by Rao and Ballard [7].
We thendemonstrate the flexibility of such networks by testing the same network on different tasks, whichcan be interpreted as conditional expectations on different neurons of the network.
Our PC graphsframework enables the model to be queried on stimuli with different structures, such as partialimages, images with labels, or images without labels.
This is significantly more flexible than thestrict input-output structure of standard ANNs, which are limited to scenarios when they are alwayspresented with data and labels in the same format.
Note that the main goal of this work is not to propose a specific architecture that achieves state-of-theart (SOTA) results on a particular task, but to present PC graphs as a new flexible and biologicallyplausible model, which can achieve good results on many tasks simultaneously.
In this work, westudy the simultaneous generation, classification, and associative memory capabilities of PC graphs,highlighting their flexibility and theoretical advantages over standard baselines.
Our contributions arebriefly as follows
• We present PC graphs, which generalize PC to arbitrary graph topologies, and show how a singlemodel can be queried in multiple ways to solve different tasks by simply altering the values ofspecific nodes, without the need for retraining when switching between tasks.
Particularly, wedefine two different techniques, which we call query by conditioning and query by initialization .•
We then experimentally show this in the most general case,
i
e., for fully connected PC graphs.
Here, we train different models on MNIST and FashionMNIST, and show how the two queries can
be used to perform different generation tasks.
Then, we test the model on classification tasks, andexplore its capabilities as an associative memory model.•
We next investigate how different graph topologies influence the performance of PC graphs ongeneration tasks, reproducing common network architectures such as feedforward, recurrent, andresidual networks as special cases of PC graphs, and investigate how the chosen structure influencesthe performance on generative tasks.
Finally, we also show how PC graphs can be used to derivethe popular assembly calculus [25].
[Figure]
Excitatory
[Figure]
PC Graphs
Let G =
( V, E ) be a directed graph, where V is a set of n vertices { 1 , 2 , . .
. , n } , and E
⊆ V × V isa set of directed edges between them, where every edge ( i, j ) ∈
E has a weight parameter θ i,j .
Theset of vertices V is partitioned into two subsets, the sensory and internal vertices .
External stimuliare always presented to the network via sensory vertices, which we consider to be the first d verticesof the graph, with d < n .
The internal vertices, on the other hand, are used to represent the internalstructure of the dataset.
Each vertex i encodes several quantities.
The main quantity is the values ofits activity, which change over time, and we refer to it as a value node x i,t .
We call the value nodesof the sensory vertices sensory nodes .
Additionally, each vertex computes the prediction µ i,t of itsactivity based on its input from value nodes of other vertices:
[Equation]
(1) where the summation is over all the vertices j connected to i via outgoing edges, and f is a nonlinearity.
Equivalently, it is possible to consider the summation on every j , and have θ i,j =
if( i, j ) ̸∈ E .
The error of every vertex at every time step t is then given by the difference between itsvalue node and its prediction, i.e., ε i,t = x i,t − µ i,t .
This local definition of error, which lies notonly in the output, but in every vertex of the network, is what allows PC graphs to learn using onlylocal information.
The value nodes x i,t and the weight parameters θ i,j are updated to minimize thefollowing energy function defined locally on every vertex:
[Equation]
A fully connected PC graph with 3 vertices is sketched in Fig.
2a, along with the operations thatdescribe the dynamics of the information flow, showing also how every operation can be representedvia inhibitory and excitatory connections.
Learning: When presented with a training point ¯ s taken from a training set, the value nodes ofthe sensory vertices are fixed to be equal to the entries of ¯ s for the whole duration of the trainingprocess, i.e., for every t .
A sketch of this is shown in Fig. 2b.
Then, the total energy of Eq.
(2) isminimized in two phases: inference and weight update .
During the inference phase, the weights arefixed, and the value nodes are continuously updated via gradient descent for T iterations, where T isa hyperparameter of the model.
The update rule is the following ( inference ):
[Equation]
where γ is the learning rate of the value nodes.
This process of iteratively updating the value nodesdistributes the output error throughout the PC graph.
When the inference phase is completed, thevalue nodes get fixed, and a single weight update is performed as follows ( weight update ):
[Equation]
where α is the learning rate of the weight update.
We now describe two different ways to query theinternal representation of a trained model, where the values of some sensory vertices are undefined,and have to be predicted.
In both cases, the weight parameters θ i,j are now fixed, and the total energy
E is continuously minimized using gradient descent on the re-initialized value nodes via Eq.
Query by conditioning: While each value node is randomly re-initialized, the value nodes ofspecific vertices are fixed to some desired value, and hence not allowed to change during the energyminimization process.
The unconstrained sensory vertices will then converge to the minimum of theenergy given the fixed vertices, thus computing the conditional expectation of the latent vertices giventhe observed stimulus.
Formally, let I =
{ i 1 , . . .
, i q } ⊂ { 1 , 2 , . . . , n } be a strict subset of vertices.
Assume now that we know that a subset of the value nodes corresponding to the vertices I is equalto a stimulus ¯ q ∈ R q .
Then, running inference until convergence allows to estimate the conditionalexpectation
[Equation]
where ¯ x T is the vector of value nodes at convergence.
Examples of tasks performed this way are (i)classification, where internal nodes are fixed to the pixels of an image, and the sensory nodes arefixed to a 1 -hot vector with the labels, (ii) generation, where the single value node encoding the classinformation is fixed, and the value nodes of the sensory nodes converge to an image of that class,and (iii) reconstruction, such as image completion, where a fraction of the sensory nodes are fixed tothe available pixels of an image, and the remaining ones converge to a reasonable completion of it.
A sketch of this process is shown in Fig. 2c.
Query by initialization: Again, every value node is randomly initialized, but the value nodes ofspecific nodes are initialized (for t = 0 only), but not fixed (for all t ), to some desired value.
Thisdiffers from the previous query, as here every value node is unconstrained, and hence free to changeduring inference.
The sensory vertices will then converge to the minimum found by gradient descent,when provided with that specific initialization.
Again, let I =
{ i 1 , . . .
, i q } ⊂ { 1 , 2 , . . . , n } be a strictsubset of vertices, and assume that we have an initial stimulus ¯ q ∈ R q .
Then, we can estimate theconditional expectation
[Equation]
Examples of tasks performed this way are (i) denoising, such as image denoising, where the sensoryneurons are initialized with a noisy version of an image, which is cleared during the energy minimization process, and (ii) reconstruction, such as image completion, where the fraction of missing pixelsis now not known a priori.
Experiments on Fully Connected PC Graphs
In this section, we perform experiments on a fully connected PC graph G =
( V, E ) , i.e., where
E = V × V .
Such PC graphs are fully general and encode no implicit priors on the structure of thedataset.
It is possible to obtain any possible graph topology by simply pruning specific weights of G .
Given a dataset D
= { ¯ s i } i<m , with ¯ s i ∈ R d , we train the PC graph as described in Section 2: Thefirst d neurons are fixed to the entries of a training point, and the energy function E t is minimizedvia inference and weight updates, via Eqs.
(3) and (4).
When the training is complete, we show thedifferent tasks that can be performed, without the need of retraining the model.
We use MNIST and
FashionMNIST [27], fixing the first d nodes to the data point, and show how to perform the tasks ofgeneration, denoising, reconstruction (without and with labels), and classification by querying the PCgraph as described in Section 2.
Setup:
For every dataset, we have trained 3 models: one for generation and classification tasks, onefor denoising and reconstructions, and one for associative memories.
The first two models consistof a fully connected graph with 2000 vertices, trained with 794 sensory vertices for classificationand generation tasks ( 784 pixels plus a 1-hot vector for the 10 labels), and 784 sensory verticesfor reconstruction and denoising.
Further details about other hyperparameters are given in thesupplementary material.
[Figure]
Generation:
To check the generation capabilities of a trained PC graph, we queried the model byconditioning on the labels:
Here, the value nodes dedicated to the 10 labels were fixed to each 1-hotvalue, and the energy of the model (Eq.
(2)) was minimized using Eq.
(3) until convergence.
Thegenerated images are then taken to be the value nodes of the unconstrained sensory nodes, whichwere originally fixed to the pixels of the images during training.
An example of the images generatedfor each label is given in Fig. 3a.
Reconstruction: We provide the PC graph with half of a test image, and ask it to reconstruct thesecond half.
This can be done using both queries: when querying by conditioning, half of the pixelsof a test image are fixed to the corresponding sensory nodes; when querying by initialization, thevalue nodes are simply initialized to the same values.
At convergence, we consider the value nodesof the unconstrained nodes, which should reconstruct the missing part of the image based on theinformation learned during training.
The results are given in Fig.
3b.
We have also replicated the sameexperiment using a network trained with the labels, and provided the label during the reconstruction.
This computes the distribution of the missing pixels knowing the available ones and the label.
Theresults in this case are visibly better and are given in Fig.
3d.
Denoising: We provide the PC graph with a corrupted image, obtained by adding zero-mean Gaussiannoise with variance 0 .
This is done by querying by initialization: before running inference, thevalue nodes of the sensory nodes are initialized to be equal to the pixels of the corrupted image.
Atconvergence, we consider the value nodes of the unconstrained nodes, which should reconstruct theoriginal image.
The results are given in Fig.
3c.
Results: As stated above, we picked a fully connected PC graph due to its generality, and not toobtain the best performance.
However, the results show that this framework is able to learn aninternal representation of a dataset, and that it can be queried to solve multiple tasks with a reasonableaccuracy.
The PC graph was in fact able to always generate the correct digit, and almost alwaysable to generate the correct clothing item in generation tasks, and always able to provide a noisy butreasonable reconstruction of incomplete test points.
The same happened with denoising experiments,as a cleaner (plausible) image was always produced.
In Section 4, we show how to improve all theseperformances by using different PC graph topologies.
Classification: We consider the same PC graph trained for the generation experiments.
To checkits generalization capabilities, we query by conditioning the pixels of every test image to the first784 sensory nodes, and run inference to reconstruct the 1 -hot label vector.
We do not expect toobtain results directly comparable with standard multilayer perceptrons for two reasons: firstly, the
[Table]
model does not contain any implicit hierarchy, which empirically appears crucial to obtaining goodclassification results.
Secondly, the PC graph is also simultaneously learning to generate the pixels,which are much more numerous than labels.
However, to check whether the obtained results wereacceptable, we tested against different learning algorithms that train on similar or equivalent fullyconnected architectures, such as Hopfield networks, unconstrained Boltzmann machines, and a localvariation of BP introduced in the late ′ 80 , called Almeida-Pineda, named after the two scientistswho independently invented it [28, 29].
As for Hopfield networks, we used the implementationprovided in [30].
The results, given in Table 1, show that our model outperforms every other learningalgorithm that can be trained on fully connected architectures.
Despite this, the results also show thatthe obtained test accuracy is not nearly comparable to the results obtained by multilayer perceptrons,as they are only slightly better than a linear classifier (obtaining 88% accuracy on MNIST).
However,this is not due to the learning rule of PC, which is well-known to be able to reach a competitiveperformance when provided with a hierarchical multilayer structure [17].
For the SVHN [31]experiment, we used models with 5000 vertices.
Associative memory: We now test whether PC graphs are able to memorize training images andretrieve them given a corrupted or incomplete version of it.
Particularly, we show that a fullyconnected PC graph is able to store complex data points, such as colored images, and retrieve themvia running inference.
To do that, we trained a novel fully connected PC graph on 100 data pointsof the MNIST, FashionMNIST, and CIFAR10 datasets.
We have used a model with 1000 verticesfor MNIST and FashionMNIST, and 3500 for SVHN and CIFAR10, and asked it to retrieve theoriginal memories by presenting it either only half of the original pixels, or a corrupted version with Gaussian noise variance 0 .
This task is similar to image reconstruction and denoising, with thenon-trivial difference that here we only use already seen data points, and hence no generalization isinvolved.
The results of these experiments are given in Fig.
3e, and show that our method is able tosuccessfully store and retrieve data points via energy minimization.
More details about the capacityof fully connected PC graphs are given in the supplementary material.
Extension to Different PC Graph Topologies
As well-known in deep learning, the performance of the trained model strongly depends on itsarchitecture: the number of vertices, layers, and their intrinsic structure.
In Section 3, we studied thegeneral architecture of fully connected PC graphs.
Here, we show how to reduce a fully connected
PC graph to lighter and even more powerful PC graphs.
Particularly, we show how to generatedifferent neural architectures by simply pruning specific edges of a fully connected PC graph
G =
( V, E ) .
In this case, the pruning is performed by applying a sparse mask M .
However, thereare multiple equivalent ways of implementing it.
Consider now the weight matrix ¯ θ ∈ R n × n , whereevery entry θ i,j represents the weight parameter connecting vertex i to vertex j .
To generate a neuralarchitecture that consists of a subset of the original connections, it suffices to mask the matrix ¯ θ viaentry-wise multiplication with a binary matrix M , where M i,j = 1 if the edge ( i, j ) exists in E , and
M i,j = 0 otherwise.
This allows the creation of hierarchical discriminative architectures such as a
PC equivalent of the multilayer perceptron (MLP) in Fig.
4a, or hierarchical generative networksin Fig.
4b, c. More generally, it creates a framework to generate and study architectures with anytopology, such as small-world networks inspired by brain regions [33], as shown in Fig.
4d.
Guidanceon which topology should be used depends on the tasks and dataset, and it is hence hard to propose ageneral theory (as it is with BP).
In what follows, however, we provide multiple examples.
[Figure]
Experiments: Here we study how the network topology influences the final performance, performingthe same experiments shown on the fully connected PC graph.
We expect the generated images to bevisibly better due to the enforced hierarchical structure of the PC graph.
Setup: We trained generative PC graphs, recurrent generative PC graphs, assemblies of neurons PCgraphs, and standard BP autoencoders with different numbers of hidden layers and hidden dimension,and report the best results.
For the generation results, we used the same setup, but added an inputlayer with 10 vertices, whose value nodes during training were initialized with the 1-hot label vector.
We performed a search across learning rates η and α , and on the number of iterations per batch T .
More details are given in the supplementary material, as well as a long discussion on how differentparameters influence the final performance of the architecture.
Results: The results are given in Fig.
5a and b. As expected, the hierarchical structure of the considered PC graphs improves over the fully connected PC graph, despite being comparable in the numberof parameters.
Compared against autoencoders (Fig. 5c), the standard ANN baseline trained with BP,the PC graph results are similar in image denoising, and better in image reconstruction.
Conditioning on Labels
[Figure]
Assume that we need to reconstruct a test imagefrom an incomplete version of it. As seen above,this can be done by querying the model by conditioning. However, assume now that this time weare also provided with the label of the corruptedimage.
It would be useful to be able use this extra information to obtain a better reconstruction.
In PC graphs, this is straightforward: it suffices
[Figure]
to simultaneously fix the value nodes representing the labels to the 1-hot vector of the providedlabel, and the sensory nodes to the pixels of the corrupted image.
We have already shown how thisimproves the reconstruction quality in Fig. 2d.
This method, however, has a second, more significant,application. Sometimes, it is difficult to infer to which class an incomplete image belongs, andproviding the label during the reconstruction allows the preferred label to influence the reconstruction.
Hence, we perform the following task: we provide images of digits that look similar when incomplete,and ask the model to reconstruct the missing half when giving the label information, i.e., use theadditional label information to correctly resolve the inherent ambiguity in the reconstruction task.
Experiments: We used the same PC graphs from above for generation tasks.
We provided the PCgraph the bottom 33% of random images representing 7 s or 9 s. Note that it is hard to distinguishbetween these two numbers when only this small portion of the image is available.
Then, we generatedthe missing 67% of the pixels by first giving 7 as a label, and then giving 9 .
We have repeated thesame task using 3 s and 5 s.
The results, available in Fig 6b, show that our model is able to performconditional inference, as the reconstructed digits always agree with the provided labels.
Assembly of Neurons
Recently, a model made by assemblies of neurons that are sparsely connected with each other hasbeen proposed to emulate brain regions [25].
This model consists of m ordered clusters of neurons( C 1 , . . .
, C m ) , and any two ordered neurons of the same cluster are connected by a synapse withprobability p , creating an Erdõs–Renyi graph G m,p .
Depending on the desired task, two clusters canbe connected via sparse connections following the same rule: if cluster C a is connected to cluster C b ,then, given a vertex v i ∈ C a and a vertex v j ∈ C b , there exists a synaptic connection connecting v ito v j with probability p .
Note that this structure is highly general, and allows to build networks suchas the one represented in Fig.
1b.
To conclude, at each time step, only the k neurons of every clusterwith the highest neural activity fire.
In the original work, the authors propose a Hebbian-like learningalgorithm, however, we show that it can also be trained using PC graphs.
A graphical representationon how to encode as a PC graph a network made by assemblies of neurons is given in Fig.
4d.
In thiscase, each dark block on the diagonal represents connections between neurons of the same region.
Unlike the other networks in the same figure, these are sparse matrices where every entry is eitherzero, or one with probability p .
As in the brain, not every region is connected with the other, andwhether two regions are directly connected has to be decided a priori when designing the architecture.
Again, two neurons between connected regions are directly connected with probability p .
In Fig.
4d,dark blocks off the diagonal represent the presence of directed connections between two regions C aand C b .
If situated below the diagonal, the connections go from C a to C b , with a < b ; if situatedabove the diagonal, they go from C b to C a .
Experiments: We replicated this structure, using 4 clusters with 3000 vertices each, connected in afeedforward way: the first cluster is connected with the second, which is connected with the third,and so on.
As sparsity and top-k constants, we used p = 0 .
1 and k = 0 .
2 , and performed the samegenerative experiments.
The results are given in Fig.
5c.
While the results look cleaner than the othermethods, note that they are specific to MNIST and FashionMNIST, as the top-k activation on the lastcluster well cleans the noise surrounding the reconstructions.
Related Work
Our work shares similarities and the final goal with a whole field of research that aims to improvecurrent neural networks by using techniques from computational neuroscience.
In fact, the biologicalimplausibility and limitations of BP highlighted in [34] have fueled research in finding a new learningalgorithm to train ANNs, with the most promising candidates being energy-based models such asequilibrium propagation [35, 36], Boltzmann machines [37] (including deep Boltzmann machines[38] and deep belief networks [39]), and PC [7].
The latter is the main field of interest in this paper.
The PC literature ranges from psychology to neuroscience and machine learning.
Particularly, itoffers a single mechanism that accounts for diverse perceptual phenomena observed in the brain,examples of which are end-stopping [7], repetition-suppression [40], illusory motions [41, 42],bistable perception [43, 44], and even attentional modulation of neural activity [45, 46], and it haseven been used to describe the retrieval and storage of memories in the human memory system [19].
Although inspired by neuroscience models of the cortex, the computational model introduced by
Rao and Ballard [7] still presents some implausibilities, with the main one being the presence ofsymmetric connections.
An implementation of PC with no symmetric connections that is able tosuccessfully learn image classification tasks has been presented in [47].
Even if not directly related to
PC, it has been shown that ANNs do not need symmetric connections to work on highly complexdatasets such as ImageNet [48, 49, 50].
Finally, there is a line of work focusing on building plausiblearchitectures divided into brain regions connected with each other [25, 26].
Here, the authors useplasticity and inhibition to build a model that describes complex cognitive phenomena.
Discussion
In this work, we have shown that PC is able to perform machine learning tasks on graphs of anytopology, called PC graphs.
Particularly, we have highlighted two main differences between ourframework and standard deep learning: flexibility in structure and query.
On the one hand, flexiblestructure allows for learning on any graph topology, hence including both classical deep learningmodels, and small-world networks that resemble sparse brain regions.
On the other hand, flexiblequery allows the model to be trained and tested on data points that carry different kind of information:supervised signals, unsupervised, and incomplete.
On a much broader level, this work strengthensthe connection between the machine learning and the neuroscience communities, as it underlinesthe importance of PC in both areas, both as a highly plausible algorithm to train brain-inspiredarchitectures, and as an approach to solve corresponding problems in machine intelligence.
The research of this paper (and current PC literature in general) is also of great importance fromanother perspective: training modern nets with BP has become computationally extremely expensive,making modern technologies inaccessible.
Biological neural networks, on the other hand, do nothave these drawbacks thanks to their biological hardware.
Recent breakthroughs in the developmentof neuromorphic and analog computing, such as the finding of the missing memristor [51], couldallow the training of deep models using only a tiny fraction of energy and time that modern GPUsneed.
To do this, however, we need to train networks end-to-end on the same chip, somethingthat is not possible using BP (or BP through time), due to the need of a control signal that passesinformation between different layers.
The energy formulation of neuroscience-inspired models allowsto overcome this limitation, making them perfect candidates to train deep models end-to-end on thesame chip [52].
This strongly motivates research in PC and other neuroscience-inspired algorithm,with a potentially huge long-term impact.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes]
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] The code andthe data are proprietary.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
[Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes]
(d) Did you include the total amount of compute and the type of resources used (e g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Supplementary material
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes]
(b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re [Table]
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[N/A]
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table] ) Did you include the full text of instructions given to participants and screenshots, if [Table]
applicable?
[N/A]
Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
Did you include the estimated hourly wage paid to participants and the total amount
[Table]
spent on participant compensation?
[N/A]
[Table]