dataesr/scientific-paragraphs-categorization huggingface dataset:
Experimental Results In our experiments, all validation indices (Sections 3 and 4) are applied on all the datasets from Table  1. Moreover, since presenting all the results would take too much space, we retain only the best results for each index (even excluding P SF D). As shown in Table  1 , it is more difficult to predict an appropriate number of clusters for real-life datasets than for artificial datasets. Considering all indices, the average rate of success is indeed 21% in the case of real data, against 66% in the case of artificial data. Whatever the type of data, Visual TSFD outperforms the other indices, with 5 wins out of 10 in the case of real datasets, and 6 wins out of 7 in the case of artificial datasets. The worst results are obtained with V P C and V XB (0/10 and 4/7 wins each).
The other indices achieve intermediary results. In addition, when the value given by Visual TSFD is erroneous, it is quite close to the expected K, in contrast to V F S , our closest competitor (Table  1 ; Wine, Glass, Segmentation, Ecoli and Bensaid). For example, the optimal number of clusters should be 6 for the Glass dataset. V F S = 12, Visual TSFD's results are 5 and 7. Furthermore, we compare in Figures  2  and 3  Visual TSFD and the plot obtained with the Elbow Rule (which is labeled Elbow T SF D) with respect to K, on a sample of both real-life and artificial datasets bearing different characteristics, i. e. , Glass, Vehicle, Ecoli, Ruspini, Ruspini noised and E1071-5overlapped (Table  1 ). As is clearly visible from Figures  2  and 3   value. This makes our method more flexible that the existing ones in real-life scenarios.
Conclusion and Perspectives In this paper, we propose a novel quality index for FCM called Visual TSFD, which provides an overview of fuzzy clustering with respect to the number of clusters. We compare Visual TSFD to several clustering quality methods from the literature and experimentally show that it outperforms existing methods on various datasets. Furthermore, Visual TSFD can also be used in the case of categorical data with Fuzzy K-Medoids  [18]. Thus, Visual TSFD allows to deal with heterogeneous datasets, which makes our method a simple but noteworthy contribution, in our opinion. As a result, our next step is to design an ensem-ble fuzzy clustering method based on Visual TSFD that would deal with both numerical and categorical data.
Related Work The more direct way to perform in situ processing is called synchronous and consists in in-lining analytics code in the simulation code. The total execution time is the addition of simulation and analytics times, plus some possible overheads due to cache trashing. The analytics can directly access the simulation data structures, but more often a copy is performed to build a data structure adapted to the analytics needs  [6]. ParaView/Catalyst  [7]  and VisIt/Libsim  [8]  are both relying on this approach to enable in situ visualization. They recently worked on a unified in situ API for the simulation codes, called SENSEI  [9] , to switch between Catalyst, Libsim and the IO framework ADIOS  [10]  with very limited code modifications. Parallel simulations are almost never 100% efficient, some cores being idle during communication phases for instance or because some code sections do not provide enough parallelism to feed all the cores.
Goldrush detects sequential sections in the OpenMP code of the simulation to schedule the analytics processes. The simulation sends resume signals to the analytics during these sections while the analytics are suspended otherwise. Experiments show the simulation performance is improved compared to OS controlled scheduling or a synchronous approach. However, Goldrush does not enable overlapping simulation and analytics during short simulation sequential sections and weakly scalable parallel sections. All previously mentioned approaches applied to MPI or MPI+OpenMP simulations. New programming models are also developed as alternatives to message passing. StarPU  [16] , PaRSEC  [17] , Legion  [18]  and HPX  [19]  propose task-based runtime systems for distributed heterogeneous architectures. The program defines a directed acyclic graph where vertices are tasks and edges data dependencies between tasks.
Plugin System TINS aims at keeping the simulation and analytics codes well separated. We therefore developed a plugin system that allows to develop the analytics outside of the simulation code. A plugin is a code compiled as a shared library. At runtime, the analytics master thread scans the plugin directory provided by the user and loads the required analytics. This way, simulation and analytics tasks are scheduled by the same instance of the TBB scheduler. A plugin should meet the following requirements. First, it has to be developed using a MPI+TBB programming model and it should take as input a MPI communicator. Indeed, simulation and analytics may perform MPI communications simultaneously and they need to use distinct communicators for the messages not to be mixed. The analytics master thread therefore creates its own communicator that the plugins should use for their internal MPI communications.
ExaStamp Molecular Dynamics Code ExaStamp  [5]  is a molecular dynamics code dedicated to material sciences (condensed matter and shock physics). It is written in C++11 and uses MPI and TBB for the different levels of parallelism. ExaStamp is used as a production code and routinely runs on more than 4,000 cores to simulate the displacement of up to 1 billion particles in a 3D system. ExaStamp is well parallelized, leaving limited compute resources under-used: it shows an efficiency of 90% on one node varying the number of cores from 1 to 28, and of 85% when scaling from 1 to 512 nodes. For each particle, the parameters of interest are the index of the particle (idx), its type (type), its position along the three axes (rx, ry, rz) and its velocity along the three axes (vx, vy, vz). To ease the interoperability between ExaStamp and the analytics described below, we defined the ParticleInSitu data structure as a structure of arrays where each array contains nbPart elements, nbPart being the number of particles in the current MPI process.
The data structure is shared by the simulation code and the analytics implemented in the plugin system: the simulation produces and fills it and the plugins take it as input. Implementing the TINS approach in ExaStamp required about 50 extra lines of code. The analytics master thread is implemented as a C++ thread and the synchronization signals between the master threads are implemented with shared booleans. The master threads may perform MPI communications concurrently so we need a thread-safe implementation of MPI with MPI THREAD MULTIPLE. ExaStamp execution is parametrized through an input data file that defines the analytics to be executed, the analytics breakpoint frequency, the execution policy and the size of the arenas. No recompilation is required to change the configuration.
Experimental Setups Experiments run on the Cobalt supercomputer from the CCRT high performance computing center. Each   Experiments are conducted timing 32 consecutive iterations of ExaStamp, with the analytics performed after each timestep. In production codes, outputs are usually not produced at each timestep to avoid slowing down to much the execution. Here we stress the system by analyzing data at each iteration to make the overheads more visible. Tests are performed on simulations with 4,000,000 particles per MPI process and one MPI process per node. Simulation codes usually run several MPI processes per node, but we run only one MPI process per node to probe TBB scheduler with a larger pool of cores. We compared the performance of running ExaStamp with 1 process per node and 4 processes per node and measured only a 2% performance drop.
Conclusion Many previous works investigated how to perform asynchronous in situ processing at a process level for MPI applications. The helper core strategy emerged as the best approach to share the resources. In this paper, we propose the TINS approach that goes one step further by proposing a dynamic helper core strategy with a temporary thread isolation in a task-based programming model. The helper cores are assigned to analytics only when analytics tasks are available while they join the other threads for simulation processing instead. The TINS approach is a minimally intrusive method where it is easy to switch between static and dynamic helper core strategies without code recompilation and that is easy to use by the end-user. It enables use of both the simulation sequential regions and the part of the simulation that are not parallelized well enough.
The experiments conducted on up to 14,336 Broadwell cores on representative analytics codes show that the TINS framework implemented with the Intel R TBB library can be up to 40% faster than the Damaris and Goldrush approaches on the ExaStamp molecular dynamics code that shows a good MPI and TBB efficiency. In particular, when the analytics workload varies from an iteration to another, no fixed number of static helper cores is capable of ensuring the best performance while the dynamic helper core strategy proves more flexible. Experiments also show that the obtained performance are close to the raw simulation, demonstrating that our approach enables to perform analytics at a high frequency. Future work will investigate the behavior of TINS on real analytics use cases. We also plan to study how to port TINS on other task-based runtimes, OpenMP in particular.
Acknowledgements: MI thanks  Matteo Bonforte  for his comments to a preliminary version of this manuscript and  José A. Carrillo  for interesting discussions on this topic during a visit at Imperial College in 2015. FSP thanks the CNA at  Carnegie Mellon University  for their kind support. FS thanks  Imperial College  for their warm hospitality in 2017 when this work started, via a  CNRS-Imperial  fellowship.
Acknowledgment The research presented in this paper has been performed in the framework of the  AGILE  project  (Aircraft third Generation MDO for Innovative Collaboration of Heterogeneous Teams of Experts ) and has received funding from the  European Union   Horizon 2020 Programme  ( H2020-MG-2014-2015 ) under Grant Agreement No. 636202. This work has been also partially supported by the  French National Research Agency (ANR)  through the  ReBReD project  under grant  ANR-16-CE10-0002. These supports are gratefully acknowledged.
A. Hard Control Simulations based on All Collected Data All the collected data (unfiltered) during the experiment (when SMV6 was hard-curtailed) was used to execute a Wind-PRO based simulation. The simulation predicted an average production increase of almost 3% in SMV5 in the 200 • ± 20 • sector, as compared to normal operation. The weighted increase in each 10 • bin is shown in Figure  6. The greatest increase (1. 15%) is achieved in the 190 • -200 • bin. This was the prevailing wind direction during this experiment as evident from Figure  3. SMV5 is almost completely shadowed by SMV6 wake in the 190 • -200 • bin. Hence WindPRO predicted a higher gain for this bin. The smallest increase is in the bin 200 • -210 • where SMV5 is under full wake affects of SMV6. The wind blew less frequently in this bin, in comparison to the other three bins (see Figure  3 ).
X. CONCLUSION Preliminary results of the C P -based curtailment experiment from the SmartEOLE project are presented. The potential of coordinated control of wind farms, for increasing farm production, is investigated using live experiments. Two turbines in the SMV wind farm (SMV5 and SMV6) were chosen as the experiment turbines. The farm and turbines were equipped with modern LiDARs. SMV7 production was used as a reference for defining a two step hard curtailment strategy in the directional sector 200 • ± 20 •. The curtailment strategy reduced SMV6 production (using C P ) by a maximum of 20% for analysing the wake effect produced on SMV5 and the overall farm production. Data from different sources was analysed during the curtailment period. WindPRO predicted 3% increase in SMV5 production with the given hard curtailment strategy in the chosen directional sector when all the collected data was simulated.
ACKNOWLEDGMENTS The work of T. N. received support from the  Research Council of Norway ,  Fripro  Project No. 213453. The research of I. S. was supported in part by the  Research Council of Norway  (Contract  216699 ) and the  French National Research Agency  ( ANR-15-CHIN-0003 ). This research was supported in part by  NTNU  and the  Norwegian metacenter for High Performance Computing (NOTUR)  by the allocation of computer time. T. S. H. and T. N. contributed equally to this work.
The increased volumes of Arabic sources of data available on the Web has boosted the development of Natural Language Processing (NLP) tools over different tasks and applications. However, to take advantage from a vast amount of these applications, a prior segmentation task call Sentence Boundary Detection (SBD) is needed. In this paper we focus on SBD over Modern Standard Arabic (MSA) by comparing two different approaches based on Deep Neural Networks (DNN) using out-of-domain and in-domain training data with only lexical features (represented as character embedding) while conducting two scenarios based on a Convolutional Neural Network and a Recurrent Neural Network with attention mechanism architectures. While tuning a big out-of-domain dataset with a smaller in-domain dataset, improves the performance in general.
Introduction Arabic language is known to be challenging given its complex linguistic structure  [3]  and dialect variations  [14]. However, the development of Arabic Natural Language Processing (NLP) tools has increased these last years, creating a large set of state-of-the-art applications including POS taggers, syntactic parsers, information retrieval, machine translation, automatic speech recognition and synthesis systems  [9, 17]. Some NLP libraries and tools like Python NLTK  1  , OpenNLP 2   [5] , UIMA  3   [10] , LIMA  4  and NooJ 5   [26] , which were originally created for non Arabic texts now include Arabic extensions. By contrast, other libraries have been developed exclusively for Arabic. Althobaiti et al. developed AraNLP  [2] , which is focused on Arabic preprocessing. This library contains some tools covering tokenization, stemming, POS tagging, sentence detection, word segmentation, normalization and punctuation/diacritic deletion.
MADAMIRA  6  , developed by Pasha et al. , is an toolkit which combines two previous Arabic NLP systems: MADA  [13]  and AMIRA  [7]. It provides the following NLP tools for Modern Standard Arabic (MSA) and the Egyptian dialect: lemmatization, diacritization, glossing, POS tagging, morphological analysis, morphological disambiguation, stemming, tokenization, base phrase chunking, name entity recognition and word-level disambiguation  [23]. The Software Architecture For Arabic language pRocessing (SAFAR)  [27]  is a framework developed by Souteh et al. that aims to gather developed Arabic NLP tools within a single homogeneous architecture and create new ones if necessary. Implemented tools within SAFAR include morphological analyzers, stemmers, syntactic parsers, normalizers, tokenizers, sentence splitters, transliteration tools and question answering applications.
Works have also been made regarding unstructured and noisy Arabic texts, found in social media datasets like microblogs and tweets. Added to the common difficulties of working with structured Arabic texts, Arabic tweets carry other problems like high degree of ambiguity, spelling mistakes, etc. Mallek et al. [20]  implemented a phrase-based statistical machine translation system from MSA tweets into English. They conclude that a preprocessing step for this kind of noisy texts is very useful to improve the translation results. El-Masri et al. [8]  presented a sentiment analysis tool over Arabic tweets using a Naive Bayes learning approach. Their model detects the polarity of a group of tweet classifying it in four possible classes: positive, negative, both and neutral. Access to Internet multimedia platforms nowadays available like Youtube 7  , TED  8  and Dailymotion 9  opens a new universe for Arabic NLP tools.
Datasets One of the objectives of the current research is to analyze the impact of cross-domain datasets during the evaluation phase of SBD. The first dataset (TED) corresponds to the Multilingual Task 2017 proposed by The International Workshop on Spoken Language Translation (IWSLT)  11. It consists of 122 TED talks  12  manually transcribed containing 168,354 words. The second dataset (GW) is the Arabic Gigaword  13  , which gathers a series of different Arabic news wires containing around 938M words (after XML extraction). We used a sample of 70,261,169 words which corresponds to the Asharq Al-Awsat (aaw arb) news wire. As for the preprocessing, we applied a normalization and tokenization process over both datasets. During the normalization phase, all punctuation marks ( ? ! ,. : ;) were mapped to a common boundary symbol, which corresponds to the "BOUNDARY" class.
Based on the experiments performed by Alotaiby et al. [1] , where they observed that a big lexicon could be reduced about 24. 54%, we used the MADAMIRA toolkit to perform a tokenization over both datasets to reduce the dimensionality. The following proclitics and enclitics were separated, generating two or more tokens depending of the amount of clitics within the word: The final size of the datasets after normalization and tokenization as well as the training, validation and testing distribution can be observed in Table  1. We opted to omit the validation set for TED given its reduced size. It is important to remark the class distribution disparity. Given the nature of the dataset, where sentence boundaries are much less frequent that nonsentence boundaries, the "BOUNDARY" class oscillates between 6.
Character embeddings Word representation is an important topic to consider specially for morphology rich languages like Arabic. Common embedding strategies like Word2vec  [22]  or Glove  [24]  do not consider the internal structure of words, which is a limitation for morphology rich languages. FastText 14 , proposed by Bojanowski et al. [6] , is a word embedding representation where a vector is associated to each n-gram character; therefore, words are represented as the sum of their character vectors. We opted for FastText vectors to represent our datasets and conduct our experiments given its advantages concerning morphology rich languages. We performed a 300 dimension vector induction with the complete GW dataset obtaining 102,248 vectors.
Experiments and Evaluations We conducted two experimental scenarios for SBD over MSA using the neural models described in Section 3. 3. Lexical features for both DNN models correspond to the FastText character embeddings described in Section 3. 2. Vectors of OOV words from the embedding model are generated from the word's n-grams vectors, eliminating unknown vectors. The performance is measured as a binary classification task, where the "BOUNDARY" (BOUND) class corresponds to the words followed by a punctuation mark, while the "NO BOUNDARY" (NO BOUND) class corresponds to those words not followed by a punctuation mark. Given the unbalanced nature of the dataset, a global metric like Accuracy is likely to be biased by the larger class; thus we also compute Precision, Recall and F1 for each class.
Scenario 1 With this first scenario (S 1 ) we wanted to observe the impact of applying a SBD model trained with a big dataset collected from written sources over a spoken source dataset. We first conducted training, validation and test on both SBD Conv and SBD LS T M systems with GW train,valid,test for 3 and 7 epochs respectively. Number of epochs were dynamically decided with GW valid before overfitting. Then, we used TED test for evaluating the performance of the trained models over an out-of-domain dataset. Results for this scenario are shown in Table  3.
Conclusion In this paper we have studied the impact of using cross-domain datasets during the evaluation phase of two Sentence Boundary Detection systems over Modern Standard Arabic. The obtained results show that tuning a model that was originally trained with a big out-of-domain dataset with small in-domain dataset, in general, improves its performance. Both methods presented a similar behavior when the spoken language evaluation dataset was used. This may lead to think that spoken MSA maintain the same linguistic structures around SUs than written MSA, but also contains some constructions that written language does not contain. Our method based on LSTM showed to be less effective compared to the one based on CNNs. We think that letting the method to learn from more epochs will help to at least equal the performance.
The high Accuracy (0. 963) of SBD Conv over the GigaWord test dataset (SBD Conv(GW) ) may give an erroneous idea of the model performance. This behavior repeats over the rest of the models. The model is really good predicting the NO BOUND class a F1 of 0. 980. It is important to note almost 40% drop in Recall between both classes, which reflects model's difficulties to retrieve the corresponding instances of the BOUND class. In most values, SBD LS T M(GW) show a similar performance than SBD Conv(GW). The difference concerns the BOUND class Recall (0. 327) is 46. 57% smaller. Evaluation of SBD Conv over TED test (SBD Conv(T ED) ) shows a interesting behaviour when compared to SBD Conv(GW). Difference in Precision for both classes, as well as the NO BOUND class in Recall is very small compared to the Recall BOUND class, which falls about 23.
S 1 results over GW and TED evaluation datasets. The objective of the second scenario (S 2 ) is to measure the effect of adding a small in-domain spoken dataset over the models trained on S 1. For this scenario we continued training SBD Conv and SBD LS T M systems with TED train. TED dataset size is very small for NN training strategies to consider the creation of a validation set. For this reason, GW valid was used during validation phase and epoch control for both systems. The reduced size of TED train lead to a fast overfitting behaviour, SBD Conv was trained only for one epoch while X for SBD LS T M. Evaluation was performed over the same dataset of S 1 (TED test ). Table4shows the results for S 2. As in S 1 , accuracy values are very high given the class unbalanced. Concerning SBD Conv(T ED) , Precision and Recall for the NO BOUND class is almost the same with just a small difference of 0.
EXPERIMENTS All our experiments have been carried on a machine equipped with two Intel(R) Xeon(R) CPU E5-2660 v3 running at 2. 60GHz and with 250Go of DDR3 SDRAM. The Twitter dataset we used as well as our implementation in C of our algorithm have been made publicly available  2. Observe that for each tweet we retain only its timestamp and its GPS coordinates, due to privacy concerns. We evaluate the algorithms under a wide range of values for k, and the length of the sliding window W. In all experiments the default values are k = 20 and = 0. 1, unless otherwise specied. For our randomized algorithm, each result is the average among ten runs, unless otherwise specied.
Datasets We consider some datasets that are publicly available, as well as a dataset that we collected from Twitter. Twitter. We collected geotagged tweets by means of the Twitter API. We were able to collect 21M tweets between 9/09/2017 and 20/10/2017. Each tweet is associated with GPS coordinates, such as longitude and latitude, as well as, a timestamp. Flickr. The Yahoo Flickr Creative Commons 100 Million (YFCC100m) dataset 3  is a dataset containing the metadata of 100 Million picture posted on Flickr under the creative common licence between 2011 and 2015. Each picture is associated with GPS coordinate and a timestamp. We used the metadata of 47M pictures that possessed both a valid timestamp and GPS coordinate. Trajectories. This dataset contains trajectories performed by all the 442 taxis running in the city of Porto, in Portugal  4.