dataesr/scientific-paragraphs-categorization huggingface dataset:
Datasets In this section, we describe the datasets used in our evaluation framework. The first one is a popular sentiment classification dataset, while the others are intent detection datasets. All datasets are public and in English. ARSC The Amazon Review Sentiment Classification dataset  (Blitzer et al. , 2007)  is composed of product reviews from 27 product categories. Each review belongs to one of the 27 domains, and contains a grade ranging from 1 to 5 stars. The usual setup  (Yu et al. , 2018b; Geng et al. , 2019)  to evaluate few-shot classification with this dataset is as follows: for each of p ≤ 27 product category and 2 ≤ t ≤ 5 score thresholds, E ARSC = p × t binary classification evaluation tasks are created. In each of these p × 4 tasks, a competitor model must learn to classify negative (< t) and positive (≥ t) reviews.
To build our test tasks, we consider the same product categories as previous works  (Yu et al. , 2018b; Geng et al. , 2019) , which are Books, DVD, Electronics, Kitchen, and t = 3 (thresholds are picked in the {2, 4, 5} set) -hence 12 binary classification test tasks in our benchmark for this dataset. Each of these twelve evaluation tasks comes with a number of support test samples (K = Q = 5 as stated previously). Nonetheless, in  (Yu et al. , 2018b)  the same 5 samples per testing class are fixed for all experiments 2  , which leads to a significant selection bias towards these 5 randomly selected samples used throughout the evaluation. In order to get more consistent results, we ran additional experimental runs, each of them selecting randomly new support samples. In the ARSC result table  (table 1 ), this corresponds to the last column (BERT + Sample shots).
OOS The Out Of Scope dataset 3   (Larson et al. , 2019)  is an intent detection dataset containing 150 equally-distributed classes. While initially used for out-of-scope prediction, it was also motivated by a high number of classes, a low number of examples per class (150), and its chatbot life-like style. In our experiments, we discard the out-of-scope class, keeping the remaining 150 classes to work with. Liu Introduced by  Liu et al. (2019) , this intent detection dataset consists in 54 classes. This dataset was collected on the Amazon Mechanical Turk platform, where workers were given an intent and had to formulate queries for this intent with their own words. It is highly imbalanced: the most common class (query) holds 5, 920 samples while the least common one (volume other) 24 samples.
Sentence Encoder In previous works comparing few-shot text classification methods, sentence encoders were not always the same. For example,  Yu et al. (2018b)  use a CNN on top of word embeddings, while Geng et al. (  2019 ) use a Bi-LSTM. Those differences make the results hard to compare since they do not use the same method to convert sentences into vectors. In our experiments, in order to reduce this selection bias, and since it is now the state-of-theart in many applications, we use a BERT  (Devlin et al. , 2018)  encoder, using models from the Hugging Face  (Wolf et al. , 2019)  team. For each dataset, instead of using an off-the-shelf pre-trained model, we fine-tune it on the masked language modeling task, as it greatly improves the quality of embeddings  (Sun et al. , 2019; Xie et al.
Baselines are surprisingly strong Few-shot learning methods were originally used to overcome data scarcity. In those situations, training a classifier on top of a small dataset -in our case, 5 samples per class -can be hard. However, our experiments on ARSC show that the Baseline and Baseline++, plain and simple classifiers, get surprisingly close to state-of-the-art results. Table 2 provides four correct and four incorrect classification examples for the Baseline model. While it fails to predict the correct text label for some shots, it is also able to correctly classify sentences such as What do I take home ? among the 50 test classes of the OOS dataset. On the ARSC dataset, it is also important to note that the Baseline++ model is significantly better than the Baseline, and is even on par with all other architectures, except Prototypical Networks.
Impact of switching to transformers One of the main contributions of our paper is to compare few-shot learning methods with the lowest bias possible (see Section 3. 3). On the ARSC dataset, using transformers drastically changes the   (Geng et al. , 2019)  (where applies), a dash (-) means that results for that encoder/metric pair were not reported, and denotes models only tested on computer vision tasks (first time applied to text in our contribution). The BERT column is our implementation using the same 5 shots as the first column but using a BERT encoder for all methods. The last column is also using BERT, but results are averaged over five runs, sampling different shots for each run. In the Configuration column, N/A means that the configuration criteria does not apply to the model. performances of all methods.
The curious case of induction networks When  Geng et al. (2019)  introduced Induction Networks, both the ARSC dataset and a private intent detection dataset were used for evaluation (publicly unavailable). Our experiments of this method on the ARSC dataset confirm those results in an acceptable range, even when trying to get more consistent results using multiple random seeds. Nonetheless, the performances of this method are underwhelming on all three intent detection datasets, even when matching the binary classification scenario using C = 2. Those poor performances were observed both on the test set as well as the train set, discarding the over-fitting argument. Such a big performance gap between sentiment and intent classification tasks show that Induction Networks, while suited for the former, are not directly applicable to any type of task.
On metric choice Prototypical Networks were originally designed to do better than Matching Networks. The two differences between them are the placement of the class average step, and the choice of the metric (cosine for Matching, euclidean for Prototypical). Our results show that metric choice yields a big gap in performances for both methods, this gap being larger than the gap caused by the model design. This hints that when using a pre-defined metric -excluding the case of metric learning -, choosing the right metric is of paramount importance. Moreover, while Matching Networks were designed to use the cosine distance, we found here that they perform significantly better when equipped with the Euclidean distance (on all datasets for all number of given test classes).
On architectural choices Overall, Prototypical Networks come on top of every intent detection dataset. More importantly, their gap between other competing approaches is wider as the number of classes increases. This result is important, as in practice, the number of classes is likely to be higher than what is used in the literature. The extended variant, proto++, obtains mixed results. While this shows that using unlabeled data can have some benefits, we also observe that the proto++ way of integrating this external knowledge is perfectible. Ultimately, note that our results do not mirror Computer Vision results. Since few-shot learning methods are used on top of embeddings, we could emit the hypothesis that they can be applied to any embeddings, regardless of the field. However, while Relation Networks, for example, were performing well in Computer Vision classification tasks -the tasks which they were originally designed for -as well as text classification -back in the days when transformers did not exist -, this is not the case anymore.
( 2h ) In this manuscript, we will consider system (2a)-(2h) for the slow-fast analysis and (1a)-(1h) for simulations under the stochastic input. We will be using the parameter set given in Table  1 , unless otherwise stated, for which δ = 0. 3 and ε = 0. 2. Numerical bifurcation analysis is performed in AUTO-07p software  [49]. The stochastic differential equations were integrated using the Euler-Murayama method with a step size dt = 10e-4 second in XPPAUT software  [50]. As noticed in  [48] , system (2a)-(2h) is a three-time-scale system written in fast form with (v 3 , y 8 ) being fast, (v 0 , y 5 , v 1 , y 6 ) slow and (v 2 , y 7 ) super-slow variables. Köksal Ersöz et al. [48]  have focused on electrophysiological pre-ictal bursting patterns recorded in human patients just before the onset of seizure.
The positioning of the electrodes is determined in each patient from hypotheses about the localization of the epileptogenic areas. Implantation accuracy   peri-operatively is verified by an X-ray CT scan. A post-operative CT scan without contrast product is then used to verify the precise 3D location of each electrode contact. After SEEG exploration, intracerebral electrodes are removed. An MRI is performed on which  the trajectory of each electrode remains visible. Finally, a CT-scan/MRI data fusion is performed to anatomically locate each contact along each electrode trajectory. The patient had electrodes implanted in the temporal region. For this study, signals were selected as they exhibited clear transitions in electrophysiological patterns. In particular, we selected pre-ictal events followed by a fast discharge typical of the seizure onset, which is one of the markers of the imbalanced relation of excitation and inhibition  [16, 71]  that involves excitability variations.
Acknowledgements We would like to thank  Prof. Fabrice Bartolomei  ( APHM, Timone Hospital, Clinical Neurophysiology, Marseille, France ) for providing the clinical data,  Mathieu Desroches  ( Inria Sophia Antipolis -Méditerranée, MathNeuro Team, Sophia Antipolis, France ) for his helpful comments and referees for their useful suggestions. Funding EKE was supported by  NIH  (Application number:  R01 NS092760-01A1 ;  18/02/2019  to  31/12/2020 ). She is member of the  Galvani  project ( ERC-SyG 2019 ; grant agreement No  855109 ) since  01/01/2021.
Abstract. Since the mid-1990s, a series of FES (finite element solution) global ocean tidal atlases has been produced and released with the primary objective to provide altimetry missions with tidal de-aliasing correction at the best possible accuracy. We describe the underlying hydrodynamic and data assimilation design and accuracy assessments for the latest FES2014 release (finalized in early 2016), especially for the altimetry de-aliasing purposes. The FES2014 atlas shows extremely significant improvements compared to the standard FES2004 and (intermediary) FES2012 atlases, in all ocean compartments, especially in shelf and coastal seas, thanks to the unstructured grid flexible resolution, recent progress in the (prior to assimilation) hydrodynamic tidal solutions, and use of ensemble data assimilation technique.
Introduction The FES2014 global ocean atlas is the latest release of a 20year effort to improve tidal predictions needed in satellite altimetry de-aliasing. It is based on the hydrodynamic modelling of tides (Toulouse Unstructured Grid Ocean model, further denoted T-UGOm) coupled to an ensemble data assimilation code (spectral ensemble optimal interpolation, denoted SpEnOI). It is a very significant upgrade compared to both the FES2004  (Lyard et al. , 2006)  and FES2012  (Stammer et al. , 2014)  atlases, thanks to the improvement of the assimilated data accuracy and the model performance. To some extent, FES2014 can be considered as an iterative step of the FES2012 atlas, mostly motivated by the overwhelming progress made in the hydrodynamic solutions accuracy toward the end of the FES2012 project and which could not be incorporated due to the project schedules.
As will be further mentioned in this publication, the efficiency of data assimilation increases significantly with prior solutions accuracy and for two main reasons. First, despite a rigorous theoretical framework, data assimilation relies on strong assumptions in which the choice of the vector norm chosen to build the penalty function is critical (the most commonly used norm is the L 2 norm, which is consistent with a Gaussianshaped error probability density assumption and which leads to easily resolved linear systems but also which tends to over-weight outliers in data or simulation values; see  Bennett, 1992, and Tarantola, 2005). Data assimilation must also be fed with quasi-empirical, partially subjective parameters, such as error covariances assigned to datasets. So while correcting prior (hydrodynamic) solution errors, it can also inject some methodological errors in the assimilation solutions, Published by Copernicus Publications on behalf of the European Geosciences Union.
more or less proportional to the prior distance between the observations and the numerical solutions. Second, as we use an ensemble technique to assess the prior modelling error covariances, and as those covariances will strongly dictate data assimilation innovation in model regions where assimilation data density is very sparse (sparse must be understood as compared to the tidal wavelength, hence being quite different in shallow-water seas compared to deep ocean regions), the prior hydrodynamic realism is critical to consistently propagate information from data locations (where data/prior model trade-off is actually solved) toward "remote" model regions. Therefore, considering the significant potential improvements, and thanks to the financial support of CNES (Centre National d'Etudes Spatiales), the decision was made to rapidly upgrade the FES2012 atlas toward the FES2014 atlas.
The FES2014 atlas denomination is quite misleading, as its final version has been delivered in early 2016. This has left time to the project team to precisely assess the FES2014 accuracy and performance in altimetry data de-aliasing correction and to make some final adjustments to guarantee the best possible quality at that time. It results in three available FES2014 releases. FES2014a is the first guess based on a data assimilation set where altimetry data were corrected from tidal loading provided by the GOTv8 model  (Desai and Ray, 2014). Its production allowed for internal verification checks and data assimilation adjustments and finally the production of the self-consistent FES2014a tidal loading atlas used within the FES2014b altimetry assimilation data processing. The FES2014a atlas was not intended to be widely distributed or advertised.
The objectives of our communication are to concisely present the FES2014 atlas main construction details, the validation diagnostics, and the available byproducts, and not to propose a dissertation about tidal science findings based on this atlas which would lead us much too far. Consequently, in the following sections, we intend to provide to the reader with information on the major ingredients of the FES2014 atlas production (hydrodynamic modelling, data processing, and data selection for assimilation and validation, assimilation processing) and a basic accuracy assessment overview. Complementary to the present publication, some additional information on present and earlier FES atlases and a link to the associated prediction software can be found on the Aviso+ website.
Model grid settings Since the first truly global ocean atlas (FES2004), the unstructured FES model mesh has been upgraded by using regional patches. The main meshing difficulty consists in dealing with the shoreline details. Present databases contain a high level of small-scale coastal details, much more than needed for a global ocean mesh. These small-scale details consequently need to be filtered out according to the targeted coastal resolution. Conversely, it is necessary to maintain and assemble together some packets of micro-islands that will form a macro-obstacle to the tidal propagation. Considering the tedious task of remeshing most of the ocean shorelines, automated tools have been developed to optimize the meshing operation. The targeted resolution for coastal areas is typically 10 km or less in terms of triangle-side length (shown in Fig.
An additional constraint was to limit the hydrodynamic solver memory use to 30 GB in order to keep computation load at a tractable level (at the time of production). Despite the large increase in resolution compared to FES2004, the FES2014 mesh resolution is still clearly not sufficient in some highly complex coastlines, with narrow channels of dynamical significance, or topographically trapped wave generation sites, and it could result in a loss of details/accuracy in such regions. This is, for instance, the case of the western Canadian and Alaska coastal regions (where the project failed to access any accurate bathymetry database at the time of production and so left resolution at a standard level), and it has resulted in a loss of details/accuracy in all of this area, especially away from assimilated data.
Model bathymetry When dealing with tides, bathymetry remains one of the most critical parameters. Several global ocean databases were available at the FES2014 production time: the General Bathymetric Chart of the Oceans (GEBCO, GEBCO compilation group, 2020), Earth Topography (ETOPO;  Amante and Eakins, 2009) ,  Smith and Sandwell (Smith and Sandwell, 1997) , etc. Their successive releases have shown tremendous improvements during the last 10 years. Unfortunately, none of those global databases have the effective resolution or the accuracy needed to be used directly in our global ocean tides modelling. For example, satellite-inverted bathymetry accu-racy is very limited on shelves and in coastal regions (Gibbs effect due to the spherical harmonic technique, uncertainties arising from sediment density, etc.
) and consequently should not be used in such locations except in some specific areas, namely in the absence of any other more accurate bathymetry. It must be noticed that the latest GEBCO distributions now include patches derived from inverted bathymetries, which is a serious issue for using recent GEBCO distributions in FES model bathymetry. Consequently, as for the earlier FES atlases, a composite bathymetry has been built from available global and regional databases. In some cases, a regional digital terrain model (DTM) has been specifically constructed from depth sounding and/or multi-beam data. A special treatment is applied to the Ross and Weddell seas, where the free water column depth must be processed by subtracting ice-shelf immersion to the bottom topography, using the RTopo-1 dataset  (Timmermann et al.
, 2010). Many regions of the world ocean are now quite well documented in terms of bathymetry; however, two major continental shelves, namely the Patagonian shelf and the Siberian shelf, do not match modern standards in any publicly available database. Bathymetry selection, reconstruction, and merging are tedious tasks, and they are quite uncertain because of the lack of independent validation data. Finally, the most practical way to assess bathymetry changes remains the examination of the tidal solutions computed from the candidate bathymetry. Naturally, this is not a perfect measure of accuracy, as errors in bathymetry can compensate some other modelling errors, but so far we have always found consistent results between improvements in bathymetry and tidal solutions. Thanks to the FD solver, extensive simulation testing can be performed, including the necessary recalibration loop needed when modifying significantly the model bathymetry, even on a regional level, as earlier calibration settings would be biased to compensate errors due to the former model bathymetry.
A global default value is locally used in regions not being targeted by the user-defined partition/polygon tuning list. Several simulations of the main tidal components (limited to M2, K1, S2, and O1 constituents) have been performed by extensively varying these two parameters (mostly globally except in a few regions for the internal tide drag coefficient), and each resulting simulation was compared to the altimetry and tide gauge (later denoted TG) validation databases. Figures  2  and 3  show the vector differences between the TP/J1/J2 (deep ocean) crossover point database and the hydrodynamic simulations of the FES2012 and FES2014 tidal models, for the M2 and K1 tidal components, respectively. Global values of vector differences are given in Table  1  for the same two hydrodynamic simulations plus FES2004.
3 Tidal harmonic constant data processing TG and altimetry-derived harmonic constant data have been used in validation of simulations and data assimilation steps. Concerning the TG data, preference was given to TGs for which the original time series were available and documented, and hence for which basic quality control could be performed by means of harmonic analysis and/or operational reports. In most cases, the time series were long enough so that a wide tidal spectrum could be analysed with the best possible accuracy. To some extent, TG selection (either for validation or data assimilation purposes) is more a question of how representative the tides captured by the instruments are (especially in coastal seas) and keeping a balanced distribution all over the ocean regions. Several tidal gauge databases have been used within the FES2014 project: a harmonic analysis was performed on time series from the GLOSS  (Holgate and al.
, 2013)  and SONEL  (Wöppelmannn and Marcos, 2015)  databases, GLOSS being a global observation network and SONEL providing measurements on all French territories; then, three validated databases provided by R. Ray have been used  (Ray, 2013)   (bottom pressure recorders), Shallow, and Coastal, hereafter and dedicated to deep ocean, shallow waters, and coastal regions, respectively. The altimetry-derived time series raise more processing and accuracy issues, with a strong dependence on the mission orbit and duration (which firstly determine the level of contamination of the tidal analysis by non-tidal ocean https://doi. org/10. 5194/os-17-615-2021 Ocean Sci. , 17, 615-649, 2021 Table  1. The rms of the vector differences (in cm) between the purely hydrodynamic solutions of FES2004, FES2012, and FES2014, and the TP/J1/J2 altimeter crossover points, for the M2 and K1 tidal components.
M2 tidal component K1 tidal component Crossover. Clearly, the 20-year and longer duration of the TOPEX/Poseidon (TP) and Jason series on a nearly 10 d repeat orbit allows for deriving outstandingly high-quality along-track and crossover datasets of tidal harmonic constants (TOPEX/Poseidon, Jason-1 (J1), and Jason-2 (J2) are three CNES/NASA satellites, successively launched, having exactly the same ground track and repetitivity, and similar on-board instruments and radar technologies. Since the FES2014 release, the series has been continued with the Jason-3 and Jason-CS satellites; at the end of their nominal missions, a satellite's orbit is changed toward an exactly interleaved ground track, hence doubling the mission spatial sampling until a possible move to a geodetic orbit or final decommissioning.
Interleaved track observations are not continuous in time and thus have shorter records compared to the nominal track records). Moreover, the altimetry dataset benefits from new altimeter standards, which allow a better observation of the tidal signals: GDR-D and REAPER orbits, ERA-Interim DAC for the European Remote-Sensing Satellite (ERS) and TOPEX missions, improved wet tropospheric, sea sate bias, and ionospheric corrections, and new mean sea surface profiles computed over a 20-year period  (Carrère and Lyard, 2003; Carrere et al. , 2016). The TOPEX-interleaved and Jason-1-interleaved track (denoted TPN/J1N) also provides an accurate crossover dataset but with larger uncertainties than the 20 years of TOPEX/Jason series, due to the shorter cumulative period of 6 years available. ERS/Envisat series and Geosat Follow-On (GFO) series do not have the same level of accuracy, as their orbits offer higher spatial coverage at the price of a lower temporal coverage (time sampling of 35 d for ERS/Envisat and 17 d for GFO).