dataesr/scientific-paragraphs-categorization huggingface dataset:
We implemented this framework into a simple numerical model. Boulder jamming was modeled using a binomial law which randomly generates boulders. Input for the binomial law was obtained from new field data in which the distribution of boulder sizes was characterized. We dealt with uncertain parameters, such as the initial clogging of the constriction due to woody debris, using an uncertainty propagation approach. The new model was used to analyze a proposed slit-dam, where field data for boulder size distributions was available. The analysis considered cases both with and without horizontal grillage bars. The new model showed that, absent significant jamming of large chunks of wood, a 6-m wide slit would probably release an excessive amount of debris flow material during projected events. The analysis suggests that adding bars across the constriction would drastically decrease the released volume during large events.
Acknowledgments Many thanks to  Cédric LEGAT  for the pictures of the canyons. We also want to thank the two anonymous referees and the Editor  Dr Mikael Attal , who together provided many helpful and encouraging comments to improve this paper. We thank  Sqomish Sea  to  Sky Developments  for allowing us to publish this work. Finally, the authors are prodigiously grateful for funding from the  MOPGA   2021/22  scheme ( MOPGA-976501H ), from the  ANR (Agence Nationale de la Recherche, France).
Data and Tools Most of the data are freely available online on the Association for Computational Linguistics (ACL) anthology website, and others are freely available in the International Speech Communication Association  (ISCA)  and Text Retrieval Conference (TREC) archives. IEEE International Conference on Acoustics, Speech and Signal Processing -Speech Track (ICASSP) and Transactions on Audio, Speech, and Language Processing (TASLP) articles have been obtained through the IEEE, and Language Resources and Evaluation (LRE) articles through Springer. For this study, we only considered the papers written in English and French. Most of the documents were available as textual data in PDF, whereas the eldest ones were only available as scanned images and had to be OCRized, which resulted in a lower quality.
Analysis of Papers' Citations Most Cited Papers Table  18  provides the list of the 20 most cited papers up to 2020 and a comparison with 2015. A number of 11 (55%) of the 20 most cited papers up to 2015 are still among the 20 most cited papers up to 2020, whereas it includes five newcomers and four papers published in or after 2015, with a special emphasis on word embedding and deep learning (Glove and BERT). The most cited paper up to 2015 is still the most cited up to 2020 (BLEU MT evaluation measure). The most cited papers are still mostly those related to language data (Penn Treebank, Wordnet, and Europarl), evaluation metrics (BLEU), language processing tools (Glove, BERT, Moses, SRILM), or methodology surveys (word representations, statistical alignment, statistical and neuronal machine translation).
Extraction of References In the internal NLP4NLP citation analysis, references were extracted through a highly reliable checking of titles, as we possess the knowledge of the NLP4NLP paper titles. We cannot use the same approach if we want to explore the citation of articles that were published in other sources than the NLP4NLP ones, as we do not have a list of the titles of all those articles. We therefore used a different strategy based on the use of the ParsCit software  (Councill et al. , 2008)  to identify the sources within the reference sections of articles for a limited set of NLP4NLP articles. This new process is resulted in a list of raw variants of source naming, which necessitated a manual cleaning, as it contained a lot of noise, followed by normalization and categorization in four categories (Conferences, Workshops, Journals, and Books).
We called "existence" 11 the fact that a term exists in a document and "presence" the percentage of documents where the term exists. We computed in that way the occurrences, frequencies, existences, and presences of the terms globally and over time  and also the average number of occurrences of the terms in the documents where they appear (Table  21 ). The ranking of the terms may slightly differ according to their frequency or to their presence. The most frequent term overall is "dataset," which accounts for 7. 6% of the terms and is present in 29% of the papers, whereas the most present term is "Speech Recognition," which is present in 31% of the papers while accounting for 5. 7% of the terms. The average number of occurrences of the terms in the documents where they appear varies a lot (from 4.
2 for "semantic" to more than 13 for "Deep Neural Network" or 12 for "Embedding"). We also compared the ranking with the 2015 one. A total of 17 of the 20 most frequent terms up to 2015 are still present 11 Also called "Boolean frequency" or "binary frequency". in this list, with few changes. We see a large progress in the terms associated with the neural network and machine-learning approaches ["dataset, " "embedding, " "neural network, " "DNN (Deep Neural Networks), " "decoder, " and "LSTM (Long Short-Term Memory)"] and a small decrease for the terms related to previous approaches ["HMM (Hidden Markov Models)", "GMM (Gaussian Mixture Models)", "SVM (Support Vector Machine)"].
Change in Topics The GapChart visualization tool 12  that we developed  (Perin et al. , 2016)  allows us to study the evolution of the terms over the years, based on their frequency or their presence. Figure  31  provides a glimpse of the evolution of topics over time, and we invite the reader to freely access the tool to get a better insight on the evolution of specific terms, such as those illustrated in the following figures. Figure  32  provides the evolution of the 25 most present terms in the past 10-year period  (2010) (2011) (2012) (2013) (2014) (2015) (2016) (2017) (2018) (2019) (2020). We see that some terms stay in this list over 10 years, such as "dataset," "metric," or "annotation," while terms related to neural network and machine-learning approaches (such as "embedding, " "encoder, " "BERT (Bidirectional Encoder Representations from Transformers), " "transformer, " "softmax, " "hyperparameter, " "epoch, " "CNN (Convolutional Neural Networks), " "RNN (Recurrent Neural Networks), " "LSTM, " and "DNN") made a large progression.
Tag Clouds for Frequent Terms Tag Clouds provide an estimate of the main terms used on a given year. For this purpose, we use TagCrowd 13  to generate Tag Clouds. We conducted experiments on full texts and on papers' abstracts and found that papers' abstracts provide a more meaningful analysis as they are more synthetic and contain a larger ratio of technical terms compared with general language. Figures 37A,B provide the Tag Clouds for 2015 and 2020. We clearly see the burst of terms related to machine learning ("BERT, " "CNN, " "decoder, " "embedding, " "encoder, " "pretraining, " "transformer") that were absent in 2015, and the sustainability of "neural network," "annotation," "metric," and "LM" ("Language Model").
Machine Learning for Time Series Prediction We explored the feasibility of predicting the research topics for the coming years based on the past  (Francopoulo et al. , 2016a). We selected in the time series plug-in of the Weka 14 machinelearning software package  (Witten et al. , 2011)  the Gaussian Processes algorithm with an 18-year window that provided the 14 www. cs. waikato. ac. nz/ml/weka best results on predicting the term frequency. We then applied this software to the full set of the NLP4NLP+5 corpus, year by year. Table  22  provides the ranking of the most frequent terms in 2018 and 2019 with their observed frequency, the topic predicted by the selected Weka algorithm for 2020 based on the smallest gap between predictions and observations on the past rankings and the ranking actually observed in 2020.
Prediction Reliability As we published such predictions for the years 2016-2020 in our previous paper, we were eager to verify whether these predictions were correct or not. We thus compared these predictions with the actual figures for these 5 years (Table  23 ). It appears that the predictions were quite reliable: the number of terms correctly  predicted to appear within the 10 top terms varies from 7 to 8 in the first 3 years  (2016, 2017, and 2018 ) that follow the year when the prediction was made (2015) and then decreases to 4 on the 4th year (given that BERT did not exist at the time of the prediction and therefore could not be predicted) and 3 in the 5th year, whereas one term ("dataset") was correctly predicted with the right ranking for the 5 years. It therefore confirms the assumption we made in our previous papers that predictions seem to be reasonable within a 3-year horizon (unless a major discovery happens in the meanwhile) in this research domain.
Use of Language Resources The LRE Map We have conducted an analysis of language resources (LR) as bricks that are used by the researchers to conduct their research investigations and develop their systems  (Francopoulo et al. , 2016b). We consider here language resources in the broad sense, embracing data (e. g. , corpus, lexicons, dictionaries, terminological databases, language models, etc. ), tools (e. g. , morpho-syntactic taggers, prosodic analyzers, annotation tools, algorithms, software packages, etc. ), system evaluation resources (e. g. , metrics, training, dry run or test corpus, evaluation packages, etc. ), and meta-resources (e. g. , best practices, guidelines, norms, standards, etc. ) that are mentioned in the LRE Map  (Calzolari et al. , 2012). This database is produced by the authors of papers at various conferences and workshops of the domain who are invited when submitting their paper to fill in a questionnaire which provides the main characteristics of the language resources produced or used in the research investigations that they report in their paper.
Language Resource Impact Factor We proposed to define the "Impact Factor" of a language resource as its existence, in recognition of the importance of the corresponding language resources for conducting research in NLP and of the researchers who provided these useful language resources, similar to the role of a citation index. Table  31  provides the impact factors for the language resources of the "Data, " "Evaluation, " and "Tools" types. We can notice the importance of quality measures (BLEU introduced for machine translation and ROUGE for text summarization) and of the recent burst of machine-learning toolkits (Word2Vec, GloVe, Weka, Seq2seq).
Kenny, Réda taslp 1,170 Verification Dehak, Pierre Dumouchel, Pierre Ouellet 8 0 Deep Contextualized Word Matthew E. Peters, Mark Neumann, Mohit naacl 1,166 Representations Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke S. Zettlemoyer 9 4 A Systematic Comparison of Various Franz Josef Och, Hermann Ney cl 1,079 855 Statistical Alignment Models 10 6 Statistical Phrase-Based Translation Philipp Koehn, Franz Josef Och, Daniel hlt, naacl 1,038 829 Marcu 11 7 The Mathematics of Statistical Machine Peter E. Brown, Stephen A. Della Pietra, cl 978 820 Translation: Parameter Estimation Vincent J. Della Pietra, Robert L. Mercer 12 0 Effective Approaches to Attention-based Thang Luong, Hieu Pham, Christopher D. emnlp 907 Neural Machine Translation Manning 13 8 Minimum Error Rate Training in Statistical Franz Josef Och acl 879 726 Machine Translation 14 >20 Convolutional Neural Networks for Yoon Chul Kim emnlp 862 Sentence Classification 15 0 Neural Machine Translation of Rare Words Rico Sennrich, Barry Haddow, Alexandra acl 836 with Subword Units Birch 16 >20 Wordnet: A Lexical Database For English George A.
Manning, Andrew Y. Ng, Christopher Potts 14 Librispeech: An ASR Corpus Based on Public icassps 2015 Vassil Panayotov, Guoguo Chen, Daniel Povey, 474 Domain Audio Books Sanjeev Khudanpur 15 Recurrent neural network-based language model isca 2010 Tomáš Mikolov, Martin Karafiát, Lukás Burget, Jan 472 Honza Cernocký, Sanjeev Khudanpur 16 Wordnet: A Lexical Database for English hlt 1992 George A. Miller 456 17 Get To The Point: Summarization with acl 2017 Abigail See, Peter J. Liu, Christopher D. Manning 455 Pointer-Generator Networks 18 Building a Large Annotated Corpus of English: The cl 1993 Mitchell P. Marcus, Beatrice Santorini, Mary Ann 447 Penn Treebank Marcinkiewicz 19 A large annotated corpus for learning natural emnlp 2015 Samuel R. Bowman, Gabor Angeli, Christopher 446 language inference Potts, Christopher D.
P300-BCIs are also relevant for fast stimulus recognition as both the amplitude and latency of this EEG component are linked to cognitive processes potentially triggered by those stimuli. In their original research paper, Sutaj et al. design and evaluate a novel P300-based real-time image ranking BCI. Their work demonstrates the relevance of BCIs for stimulus sorting and brings insights on the influence of stimulusrelated cognitive processes on the associated EEG patterns. Emotional reactions monitoring and classification is an important part of many real life applications involving physiological and affective computing technologies. Li et al. describe their open-source software toolbox MindLink-Eumpy that aims at recognizing emotions by integrating EEG and facial expression information. Besides online experiment conducted, the offline validation was done using DEAP  (Koelstra et al.
, 2011)  and MAHNOB-HCI  (Soleymani et al. , 2011)  datasets. As expected, the results show that multimodal methods outperform single-modal methods in both offline and online experiments. With the expansion of the BCI field comes the development of numerous applications. While they could be relevant and re-used, both for research purposes and out-of-the-lab applications (e. g. , entertainment, rehabilitation, etc. ), those applications most often remain confidential. Efforts to provide these application opensource, as in Woo et al. , will be most valuable for the community. Recently, functional near-infrared spectroscopy (fNIRS) has been for brain studies outside the lab or indoor conditions involving user physical activity like walking, as it is more robust to movement artifacts than EEG-based imaging.
FUNDING Partial support from the  IUCRC BRAIN  to JC-V is gladly recognized (NSF Award number  1650536 ). CJ has received partial support from the  French National Agency  (grant nos. ANR-20-CE38-0008-01 - ABCIS  project and  ANR-20-CE37-0012-02-BETAPARK project ). AV received partial support from the  Estonian Research Council  grant  PUT1518.
Tjernström et al. (2021)  later evaluated the vertical structure of the lower atmosphere in IFS using data from the Arctic Ocean 2018 campaign, showing that the warm bias, which had a deep vertical structure, arose from complex interactions between surface temperature, clouds, the energy fluxes, and vertical mixing. On the second front, APPLICATE also exploited the wealth of in situ observations from Arctic supersites (e. g. , from YOPPsiteMIP) for model error investigation, which involved the definition of process-based diagnostics for evaluating aspects as diverse as the surface energy budget or solid precipitation  (Day et al. 2020; Køltzow et al. 2020). This exercise also highlighted the importance of standardizing the formats and preprocessing of Arctic observations, as carried out within YOPP.
2020). Likewise, models can also assist the development of future observing systems, providing key information on the sampling locations providing more predictability  (Ponsoni et al. 2020; Sandu et al. 2021 ). Given the high potential shown by multisystem forecast ensembles to boost skill through the use of optimized ensemble aggregation techniques and the introduction of physical constraints to select members with larger skill, it is important to promote and further exploit existing multi-collaborative initiatives for retrospective and real-time prediction, such as it is currently done through DCPP, the Copernicus Climate Data Store, and SIPN2. Preliminary results from APPLICATE suggest that the high computational costs of enhancing horizontal resolution for seasonal climate prediction in the Arctic are still not worthwhile when compared to the benefits of larger ensembles, at least for the resolutions and models considered thus far.
Acknowledgments. PO's work was supported by the  Spanish  fellowship  RYC-2017-22772. The results discussed in this article were supported by the project  APPLICATE  ( 727862 ), funded by the  European Union 's  Horizon 2020 research and innovation programme. This is a contribution to the  Year of Polar Prediction , a flagship activity of the Polar Prediction Project, initiated by the  World Weather Research Programme (WWRP)  of the  World Meteorological Organization (WMO). We acknowledge the  WMO WWRP  for its role in coordinating this international research activity.
The main reason for this is that it reduces the number of degrees-of-freedom, but having the flexibility to choose a certain level of G-continuity independently of the finite element basis could be interesting, both from a user's perspective, as well as from a computational and numerical perspective. The NIMROD code  [27]  provides a study of MHD instabilities while varying the degree of the basis functions at fixed continuity level, but the ability to change the level of continuity at fixed polynomial degree could also be of interest. However, this would require a reformulation of the finite-element numerical structure inside JOREK, which is beyond the work addressed here.
This nodal formulation is essential for a practical implementation in the JOREK code and enables us to reduce the number of degrees-of-freedom, while imposing continuity by construction of the finite-elements, independently of the values of the degrees-of freedom themselves. Secondly, Aumann & Bentz only considers continuity between 2 Bezier patches (i. e. in one direction), whereas our case requires continuity between 4 patches (i. e. in two directions). Interestingly, Aumann & Bentz's formula is simultaneously more elaborate and simpler than our case: it is generalised for two patches of arbitrary polynomial order (whereas we require all patches to have the same odd polynomial order), but it does not provide a formulation for 4 patches meeting at the same corner node, requiring continuity in two directions, which our formulation provides.
Overview of the work In this paper, we present a generalised method for high-order Bezier finite-elements, which ensures that G-continuity increases linearly with the polynomial order of the elements. The method is defined, proven, and implemented in the JOREK code with several numerical tests and benchmark cases. Section-2 introduces the G2-continuous Bezier finite-elements, which serves as an introduction to the generalised method. Section-3 presents the definition the generalised Bezier formulation, together with the mathematical proofs to demonstrate G-continuity. Section-4 describes some of the key details that were required for the implementation of this new method in the JOREK code, and Section-5 presents the series of tests conducted to demonstrate the mathematical accuracy of the new method, together with the practical usability and advantages of using higher-order finite-element methods for non-linear MHD instabilities in tokamaks.
The JOREK code The above formulation is implemented in the JOREK code  [20] [21] [22] [23]. JOREK uses a finite-element grid in the poloidal plane, with finite Fourier series in the toroidal direction. The finite elements are quadrangular Bezier elements using a nodal formulation equivalent to the generalised formulation in Definition-1 and Definition-2 with polynomials of order 3, and thus G-continuity of order 1  [22]. The Fourier series used in the toroidal direction can be chosen with arbitrary toroidal periodicity, meaning that with 3 Fourier modes and a toroidal periodicity of 4, the physical toroidal mode numbers simulated would be n = 4, 8, 12. Typically, when producing linear stability scans, a single Fourier mode is used, while scanning the toroidal periodicity. For non-linear simulations, multiple Fourier modes are used, with a periodicity of 1 or 2.
For bi-quintic G2-continuous elements, this gives a total of: 4 nodes, times [(n + 1)/2] 2 basis vectors, times 6 derivatives (including zeroth derivative); hence 216 functions to calculate and include in the code. For elements of order n = 7, there are 384, and for n = 11, there are 864. Thus, although the basis functions and their derivatives can be calculated by hand for bi-quintic elements, at higher order this is not feasible without a considerable risk of human error. It is much safer to do this analytical calculation numerically, which can be done relatively easily with analytical algebra libraries like Python's symbols library, Sympy  [64]. Using this library, the Fortran module of JOREK that includes the basis function formulas is auto-generated for any polynomial order n. It is worth highlighting the fact that continuity is not 'imposed' in the matrix of the final linear system.
Gaussian integration The weak-formulation of JOREK requires integration over each element. This is done using Gaussian quadrature, for which the number of Gaussian integration points (and weights) needs to correspond to the polynomial order of the Bezier elements. Although trivially obtained from Python libraries like Numpy  [65] , this also requires the auto-generation of a small module in the JOREK code (for polynomial orders n > 7), to register these Gaussian integration points and weights.
The mesh generators and initial conditions The JOREK code includes a large variety of mesh generators, adapted to complex magnetic configurations in tokamaks, and with adaptive extensions to arbitrary wall structures  [66]. These grid generators had to be adapted to include higher order polynomials. The alignment of the grids, along the background magnetic flux surfaces, is done up to second derivatives, which is largely sufficient for most applications, and thus for polynomial orders n ≥ 7, the derivatives of order ≥ 3 are set to zero (only for the isoparametric spatial variables, not for the physics model variables). In the future, if higher accuracy is required, for Bezier elements of order n ≥ 7, these higher-order derivatives could be included in the grid generators. It is difficult to quantify accurately the effect of precision in the alignment of the 2nd grid, along flux-surfaces.
Thus, each matrix size is, at most m × m = (2 n n n d n v ) 2 (96) where the factor 2 comes from the sine and cosine component of each Fourier mode. Of course, these are sparse matrices, but in practice, their number of non-zero entries will scale with the square of the number of degrees of freedom, n 2 d. Since the number of degrees of freedom per node scales as the square of the polynomial basis order, n 2 , it means the size of the matrices will scale as n 4. In order to enable simulations with large spatial resolutions and large polynomial basis orders, parts of the JOREK code required modification to allow for a compressed sparse row (CSR) representation of the matrices using long-integers. Although straight-forward in most instances of the code, this also requires MPI communications of the matrix to be split into batches, since MPI communication indexing is restricted to short integers.