dataesr/scientific-paragraphs-categorization huggingface dataset:
During learning, a task is sampled from T trn and N inner loop updates are made using batches of data sampled from D trn. The resulting parameters θ N are then used to make the outer loop update using the matching validation dataset D val. During each inner loop update, one or more SGD update steps are performed over a task-relevant loss function L: θ n+1 (θ n , D trn ) = θ n -α∇ θn L(f (x, θ n ), t), where {x, t} ∈ D trn for n = 1,. , N. (1) Here n is the number of inner loop adaptation steps, α is the inner loop learning rate. Note the dependence of θ n+1 on the initial parameter set θ 0 at the beginning of the inner loop through the recursion. ∇ θn here indicates the gradient over the inner loop loss L on the network using parameters θ n. The outer loop loss is defined as: L outer (θ 0 ) = T ∈P (T trn ) L(f (θ N (θ 0 , D trn ), D val )), (2) where T = {D trn , D val }.
Note that in practice the above expression is generally computed over a random subset of tasks rather than the full set T trn. Notice that the outer loop loss is computed over the validation dataset D val , whereas θ N (θ 0 ) is computed using the training dataset D trn , which is argued to improve generalization. The goal is to find the optimal θ 0 , denoted θ * 0 such that: θ * 0 = arg min θ 0 L outer (θ 0 ). (3) Provided the inner loop loss is at least twice differentiable with respect to θ 0 , the optimization can be performed via gradient descent over the initial parameters θ 0 , using a standard gradient-based optimizer using gradients ∇ θ 0 L outer (θ 0 ). Successive applications of the chain rule in the expression above results in second order gradients of the form ∇ 2 θn L(f (x, θ n ), t).
Datasets We benchmark our models on modifications of datasets collected using event-based vision sensors  (Lichtsteiner et al. , 2008; Posch et al. , 2011) , the Neuromorphic MNIST (N-MNIST)  (Orchard et al. , 2015) , and the American Sign Language Dynamic Vision Sensor (ASL-DVS)  (Bi et al. , 2019)  datasets. NMNIST consists of 32 × 32, 300ms long event data streams of MNIST images recorded with an ATIS Camera  (Posch et al. , 2011). The dataset contains 60,000 training event streams and 10,000 test event streams. From the N-MNIST dataset, we create Double N-MNIST datasets. Each event stream of Double N-MNIST is a combination of two N-MNIST event streams to make 64 × 32, 300ms long event data streams of doubledigit numbers that are downsampled to 32 × 16, 100ms long event data streams. Because there are ten digits in the original N-MNIST dataset, 100 different double-digit numbers can be created.
These 100 different numbers can be used to create a meta-dataset with K=100 tasks, where each double-digit number represents one task. We create N-shot K-way meta training, meta validation, and meta-test Double N-MNIST datasets from the training and test N-MNIST dataset. Each meta dataset consists of a subset of the 100 total possible tasks. The meta training dataset contains 64 tasks, the meta validation dataset contains 16 tasks, and the meta test dataset contains 20 tasks. The ASL-DVS dataset contains 24 classes corresponding to letters A-Y, excluding J, in American Sign Language recorded using a DAVIS 240C event-based sensor  (Brandli et al. , 2014). Data recording was performed in an office environment under constant illumination. The dataset contains 4200 240 × 180 100ms long event data streams of each letter, for a total of 100,800 samples.
Like Double N-MNIST, each event stream of Double ASL-DVS is a combination of two ASL-DVS event data streams to make 480 × 360, 100ms long event data streams of two letters that are downsampled to 80 × 30, 100ms long event data streams. Out of the 24 classes, 576 different tasks consisting of double ASL letter event streams can be used to create N-shot K-way meta datasets with each double ASL letter representing a task. From the ASL-DVS dataset, we create Double ASL-DVS N-shot K-way meta training, validation, and test datasets. The meta training dataset contains 369 tasks, the meta validation dataset contains 92 tasks, and the meta test dataset contains 115 tasks. Example images from the Double N-MNIST and Double ASL-DVS datasets are shown in Fig. 2. For all datasets, gradients were computed for the last 30ms of the sequence to reduce the memory footprint.
Few-shot Learning Performance on Double NMNIST and Double ASL Tasks For each dataset, we ran 1-shot 5-way learning experiments on models trained using MAML. The MAML models were meta-trained on the meta training tasks (T trn ) with the meta validation tasks (T val ) used to compute the loss gradient in the outer loop. The metatrained models were then tested on the meta-test tasks (T tst ). A summary of our results for each dataset is shown in Table  2. The results in Table  2  are obtained from averaging the inference performance of a meta-trained model over 10 trials on the test datasets (D tst ) of the meta validation and meta test tasks. Each trial has different random batches of data sampled from training tasks T trn and validation tasks T val respectively. All experiments only used a single inner loop gradient step (i.
e. in MAML N was set to 1). Additionally, we compare the results of the SNNs to equivalent non-spiking models meta-trained with MAML as well as SNNs trained with first-order MAML. First-order MAML ignores all terms involving the second-order gradients and is thus similar to the joint training of the tasks. This has the advantage of reducing the memory footprint required for learning but is known to reduce the accuracy of the meta-trained model  (Finn et al. , 2017). For the non-spiking models, the input data is first converted from the address event representation to static images by summing the events over the time dimension. The results show that both spiking and non-spiking MAML achieve 1-shot learning performance on the datasets comparable to the state-of-the-art performance shown by nonmeta models.
The state-of-the-art test accuracy for standard non-meta model training on the NMNIST dataset with SNNs is 99. 2 ± 0. 02% accuracy  (Shrestha and Orchard, 2018). The SNN achieves 98. 23 ± 1. 12% test accuracy on Double NMNIST in a one-shot learning scenario. Our SNN network achieves a test accuracy on the Double ASL-DVS dataset of 96. 04 ± 2. 31%. For both datasets, first-order MAML performed significantly worse, highlighting the importance of differentiable surrogate gradients for successful meta-training. On average MAML on SNNs tend to match or outperform non-spiking MAML on event- based datasets. This is likely because the dynamics of the SNN neurons are well suited for processing and learning the Spatio-temporal patterns of the event-based data streams the datasets are composed of.
Generalization of Learning Performance MAML requires selecting hyper-parameters such as the number of update steps and the learning rates. In real-world scenarios, the input constraints cannot be tightly controlled, leading to potential mismatches with the MAML hyper-parameters. For example, in a realtime gesture learning scenario, the parameter update schedule may not be tightly linked to the time the gesture is presented. Here we study the ability of MAML trained SNNs to generalize across different input conditions. The ability of MAML to generalize learning performance across its settings, such as the number of steps, has been previously documented for conventional ANNs  (Finn and Levine, 2018). Here, we demonstrate that this feature extends to our SNN. Using an SNN MAML network meta-trained on the Double NMNIST dataset, we varied the number of gradient steps during inner loop adaptation on test data.
The Fig. 3  shows how changing the number of gradient steps during inner loop adaptation affects the one-shot 5-way learning performance on each dataset. On both datasets, as the number of inner loop gradient steps increases the performance increases. Therefore there is a trade-off between the computational overhead of performing multiple gradient steps during one-shot learning and accuracy. We also show how the learning performance is affected when layers of the network are frozen during few-shot learning. Using a network meta-trained on the Double NMNIST dataset, we progressively froze layers of the network to observe the impact on performance, which is shown in Fig. 4. Even when all layers of the network were frozen, in this case, three, there is not a significant impact on performance.
MAML Few-shot Learning Relies on Few, Large Magnitude Updates To obtain adequate generalization, conventional deep learning relies on many, small magnitude updates across a large dataset. This is achieved using a relatively small learning rate. The usage of small learning rates is challenging on a physical substrate, as it requires high precision memory to accumulate the gradients across updates. This problem is further compounded by the fact that learning on a physical substrate cannot be easily performed using batches of samples. Interestingly, few-shot learning has the opposite requirements: few but large magnitude updates. This result is extremely relevant for neuromorphic hardware which uses low  precision parameters. Likewise, we observe that the SNN MAML model only needs a few, large magnitude parameter updates for few-shot learning.
During the inner loop adaptation, instead of always updating the parameters the update is gated by a threshold which is described in Eq. (  6 ). θ n k = θ k , if∆w > Θ, θ k-1 , if∆w < Θ (6) where θ n k are the parameters of the model, ∆w is the magnitude of the update, and Θ is the threshold. Thresholding the updates forces the parameters to be larger with fewer updates, which is shown in the Fig. 5. The threshold used in the Fig. 5was  equal to 5% of the value of the range of magnitude updates. A generalization problem involves learning a function, or model, whose behavior is constrained through a dataset that can make predictions about (i. e. learn features than can transfer to) other samples. A task domain consists of datasets that are related by a common domain, for example, datasets that all consist of double-digit numbers.
Discussion Neuromorphic hardware is particularly well suited for online learning at the edge. Here, we demonstrated how to pre-train SNNs to perform one-shot learning using MAML. The SNN MAML models used the surrogate gradients method to overcome non-linearities that occur in gradient-based training of SNNs. We demonstrated our results on combinations of event-based datasets recorded using a neuromorphic vision sensor. The effective batch size (=1), the precision required for learning from scratch, and the potential correlation in data samples in neuromorphic learning are serious obstacles to deploying neuromorphic learning in practical scenarios. Fortunately, learning from scratch on the device is generally not even desirable due to robustness and time-to-convergence issues, especially if the devices are intended for edge applications.
Non-gradient based meta-learning MAML is known to learn representations that are general across datasets rather than "learning to learn"  (Raghu et al. , 2020). The results of our experiments with freezing model layers showed this is the case for SNN MAML as well indicating the layers already contain good features at meta-initialization. Another meta-learning approach done with artificial recurrent neural networks is to train the optimizer itself modeled using an LSTM  (Andrychowicz et al. , 2016). The underlying mechanism relies on the recurrent cell states that capture knowledge that is common across the domain of tasks. The SNN based work  (Scherr et al. , 2020)  falls into this category. There, meta-learning was applied to SNNs trained using e-prop on arm reach and Omniglot tasks. The approach used for the meta-learning combined a Learning Network (LN) to carry out inner loop adaptation and a Learning Signal Generator (LSG) to carry out outer loop generalization that was both modeled by recurrent SNNs.
Meta-learning for Neuromorphic Learning Machines This work meta-optimized Hebbian-based STDP learning rules and local metaparameters on the Omniglot dataset to examine the performance of the model in few-shot learning tasks on the Tianjic neuromorphic hardware. Our work extends the one  (Wu et al. , 2020)  by directly optimizing a putative three-factor learning rule on event-based data and surrogate gradients that could be used for few-shot learning in neuromorphic hardware. Meta-learning and transfer learning techniques are already presenting themselves as key tools for neuromorphic learning machines. For example, transfer learning on the Intel Loihi Neuromorphic Research Chip was used to enable few-shot learning  (Stewart et al. , 2020a)  There, a gesture classification network was pre-trained using a functional simulator of the Loihi cores.
For comparison to the Double NMNIST SNN MAML model we first pretrained an SNN model on the 64 classes of the training dataset. Then we transferred the features to a new model that had an untrained last layer. The model was trained and tested on 20 of the remaining classes where all layers except the last layer were frozen. The model was trained on one shot of data at a time and then tested on 20 unseen shots of data. The few-shot transfer learning results are shown in Fig. 6. The results shown in the table were averaged over 10 trials. After the model trains on about 9 or 10 shots of data, the model achieves comparable accuracy to the SNN MAML model shown in Table.
The stratification splits the population into groups with similar regional fiber organization using manifold learning. We first successfully test the hypothesis that the main source of variability of the regional fiber organization is the variability of the regional folding pattern. Then, in each group, we proceed with the automatic identification of the most stable bundles, at a higher granularity level than what can be achieved with the non-stratified whole population, enabling the disentanglement of the very variable configuration of the short fibers. Finally, the method searches for bundle correspondence across groups to build a population level atlas. As a proof of concept, the atlas refinement achieved by this strategy is illustrated for the fibers that surround the central sulcus and the superior temporal sulcus using the HCP dataset.
Database The Human Connectome Project (HCP) database  ( Van Essen et al. , 2013 )  S900 release was used. This high resolution and high-quality database  ( Marcus et al. , 2013 )  allows a good reconstruction of the folding pattern and dMRI tractograms. We used the HARDI data from 897 subjects (28. 81 ± 3. 69 years old; 394 males and 503 females; 813 right-handed and 84 left-handed), chosen by the HCP consortium to represent healthy adults beyond the age of major neurodevelop-mental changes and before the onset of neurodegenerative changes  ( Van Essen et al. , 2012 ). All subjects provided written informed consent on forms approved by the Institutional Review Board of Washington University. All HCP subjects were scanned on a customized Siemens 3T "Connectome Skyra " housed at Washington University in St.
Whole-brain regularized streamline probabilistic tractography was performed  ( Perrin et al. , 2005 ) , with a forward step of 0. 2 mm and a maximum curvature angle of 30 •. This leads to tractography datasets, or tractograms, with an average of 27 million streamlines (later called fibers) per subject, with a length between 5 and 300 mm. Tractography preprocessing This preprocessing was done in order to filter out outlier fibers and to get a compressed representation of the tractograms in order to achieve affordable calculation times for some items of our method. Using all the fibers from the probabilistic tractography is beyond reach with conventional computers when performing inter-subject clustering. For each subject, the preprocessing relies on geometry-based fiber clustering  ( Guevara et al.
, 2017 )  applied in parallel to numerous partitions of the tractogram. The preprocessing performed in this paper is very standard for the field, and could be replaced by any similar one  ( Garyfallidis et al. , 2012; Guevara et al. , 2011b; Vázquez et al. , 2020 ). Therefore, the details are described in Section 1 of the Supplementary File. The output of the preprocessing is first a "clustered-fiber dataset " including all the fiber that have survived the outlier detection and second a "centroid dataset ". Each centroid represents the average trajectory of a specific fiber cluster. The items of our method that only require the shape of the cluster trajectory use the compressed centroid representation, while those that also require a representation of the shape of the cluster section orthogonal to the trajectory use the whole set of fiber trajectories.
The preprocessing results in an average of  3,221,726 ( ± 753,127)  streamlines and 10,406 centroids ( ± 2388) per hemisphere. To represent the cortex morphology, sulcus automatic labelling was first performed from the T1-weighted image. The sulci were extracted using the Morphologist toolbox of BrainVisa  ( Mangin et al. , 2004; Perrot et al. , 2011 ). The software provides for each sulci a representation based on a set of 3D points that sample the medial surface of the cerebrospinal fluid filling the fold. To control for the influence of variable brain sizes, the sulci representations are all normalized to the standard Talairach reference frame, by a 9-parameter affine transformation. In addition, the main gyri were identified using FreeSurfer, 2  which provides a 3D volume where the cortical mantle is split into a set of regions of interest (ROIs) defined by the Desikan-Killiany atlas  ( Desikan et al.
This value is used to fill the dimensional reduction input matrix (   ). the following, this sulcus-based Isomap will be noted   , while the subtractogram-based Isomap will be noted. In order to obtain the subject's subtractograms, for all of them, the centroids connecting the ROIs selected according to the sulcus under study were extracted from the centroid dataset using the FreeSurfer cortex parcellation. As there are two sulci under study: central sulcus and superior temporal sulcus, for each subject two subtractograms are obtained. For the central sulcus the subtractograms are defined by the centroids connecting the pre (PrC) and post central (PoC) gyri. For the superior temporal sulcus, we selected the superior temporal (ST), middle temporal (MT), inferior parietal (IP) and supramarginal (SM) gyri, to extract the centroids connecting the ST-MT, IP-SM and IP-IP pairs, the last pair corresponding to intra-regional fibers coursing around the posterior branches of the sulcus.
More details on the choice of the sampling percentage can be found in the Section 3 of the Supplementary File. A one-dimensional Isomap was computed using the scikit-learn 3 python library, which was slightly modified to add the possibility to use a distance matrix as input. The number of neighbors used by the Isomap algorithm to approximate the manifold-based geodesic distance by the distance in a graph of nearest neighbors was set to 20.
3 ), flipping the roles of the two subjects to get two symmetrical "bundle-subtractogram " distances that are then averaged. Therefore, we decompose the distance measure used to model the global variability of the population's subtractograms (   ) from a subtractogram-subtractogram distance into a bundle-subtractogram distance measure (see Fig. 5  ). For a specific bundle of the population atlas, this bundle-subtractogram distance measure is only computed for pairs of subjects with an occurrence of this bundle embedded in their group instance, namely with at least one centroid composing it. Since the subtractograms global distance measure (   ) is computed from a sampling of the subtractogram's fiber dataset rather than from the centroids, the bundle-subtractogram distance measure is also computed from the fibers.
Functional activations A key issue is the potential meaning of the bundle variability described by each   dimension relative to the underlying functional architecture. The relationship between the central sulcus or the superior temporal sulcus morphologies and the regional functional organization has already been explored using the Isomap strategy  ( Sun et al. , 2016 ). By means of an fMRI group analysis built upon the   population stratification, it was shown that whatever the highest knob position, it hosts the hand motor areas. In this spirit, we performed a similar group analysis using fMRI, but from the   stratification, to test whether the fiber geometry variability is also linked with functional variability. For this purpose, simple fMRI contrasts of the HCP dataset have been used.
Results for the central sulcus (CS) The first area of interest used to explore the potential of our methodology corresponds to the fibers that course around the central sulcus. We observed a strong correlation between the first dimensions of the   and the   (0. 58). This is particularly interesting since the first dimension of the   of the central sulcus is coding for a clear morphological feature: the variable localization of the hand knob along the central sulcus  ( Sun et al. , 2012; 2016 ). Our result stands for a link between the localization of the hand knob and the short fiber organization. The group-based sulcus average results highlight that the two Isomap code for a similar geometrical variation of the shape of the sulcus related to the position of the hand knob ( Fig. 7 a ). Group-based averages of the functional activation mapping the right-hand sensorimotor areas (HCP MOTOR 12 RH-AVG contrast) were also computed along the fiber Isomap.
Results for the superior temporal sulcus (STS) For the STS, the correlation between the first dimensions of both Isomaps was 0. 56 (  < 0. 001 ). Group-based averages were computed for each of the two stratifications leading to observe that both Isomaps encode a similar morphological change, which is clearer in the sulcusbased Isomap  ( Sun et al. , 2016 ). From the left to the right, the fork leading to the branches of the STS gets more developed and the main part of the STS gets shorter (see Fig. 8 b ). We also performed the comparison between the one group and the ten groups population-level atlases ( Fig. 8 c ). Note that most of the 10-group bundles without match in the one-group atlas (denoted by the grey color) are located in the STS branches region, namely the region with the highest morphological variability along this Isomap dimension.
, 2021 ). But this fasciculus is missing in the set of clusters inferred from the one single group experiment, when fiber clustering is performed from the whole population (see Fig. 7 c ). There is no doubt that the group definition plays an important role in our stratification-based strategy, and the manifold-based approach proposed in this paper has been just one possibility among many. When using a manifold, the choice of the ideal number of groups is an open issue that is related to a compromise between the disentangling effect of the population stratification and the need to keep enough subjects in each group to achieve a robust bundle inference. Thanks to the large size of the HCP dataset, we could explore the impact of the number of groups. Splitting the population into five groups along the Isomap first dimension already has led to the detection of the PPFM fasciculus in each of these groups, while it is missing in the whole population atlas.