dataesr/scientific-paragraphs-categorization huggingface dataset:
Minimally intrusive approaches. A less intrusive approach  (Cantwell and Nielsen, 2019) , targeting time-dependent solvers, combines message-logging techniques and remote in-memory checkpointing to reduce this burden in often complex object-oriented applications. It exploits the nature of these codes by transparently logging MPI messages during the relatively short initialization phase of the application, when the data structures remain static, thereby allowing recovering processes to follow the normal unmodified program to reach the time-integration phase. For the time-integration phase, in which the solution data change frequently, remote in-memory checkpointing is used to maintain scalability. This approach is illustrated in Figure  2.
Fenix and MPI_ReInit. Fenix is a framework enabling MPI applications to recover nearly transparently from losses of data or computational resources manifested as observable errors. It is based on the premise that the MPI standard itself provides facilities for trapping and isolating such errors, allowing the application to retain control of remaining unaffected resources and obviating full program restart. Building on the success of Fenix's prototype implementation  (Gamell et al. , 2014; Teranishi and Heroux, 2014) , a significant extension has been made that provides APIs for online application rollback and data recovery together and serves many different application needs and styles with formal specifications  (Gamell et al. , 2016). For process recovery, Fenix_Init() specifies the point to rollback when failure is detected and the recovery schemes for the lost MPI ranks.
For data recovery, Fenix provides several data binding and recovery APIs that leverages the internal information of Fenix's process recovery interface for efficient data recovery. The current implementation of Fenix leverages ULFM, but the specification does not require it. Recently, Fenix has been modified to support asynchronous application recovery leveraging an on-node asynchronous many-task parallel runtime  (Paul et al. , 2020)  to eliminate the need for online rollback of all MPI ranks. MPI_ReInit  (Georgakoudis et al. , 2020; Laguna et al. , 2016 ) is a similar approach, but it does not expose any lowlevel APIs like MPI-ULFM. Instead, it focuses on efficient online rollback recovery, simplifying the low-level fault detection and notification mechanism accommodated by MPI-ULFM. MPI_ReInit does not specify any data recovery schemes, allowing the use of external software.
The latest implementation  (Georgakoudis et al. , 2020)  demonstrates better scalability than MPI-ULFM in the absence of failures. On data recovery for MPI programs, Global View Resilience (GVR,  Chien et al. , 2015)  and VeloC  (Nicolae et al. , 2019)  accommodate generic APIs for data persistency. Both Fenix and MPI_ReInit can leverage a combination of these, or others. Fenix chiefly aims at providing fast and in-memory redundant storage for efficient data recovery, whereas GVR and VeloC target secondary storage for data capacity. Nevertheless, VeloC addresses the efficiency issues using multi-level checkpointing that utilizes on-node memory and burst buffer (special staging I/O nodes), achieving scalable and asynchronous checkpointing for the state-of-art HPC platforms. (Bertagna et al.
Kokkos. Kokkos 3. 3. 5. Minimal ULFM and fallback. This approach is a much more lightweight wrapper around the ULFM specification, specifically targeted at Cþþ applications that intend to react to node losses and silent data corruption via Cþþ exceptions  (Engwer et al. , 2018). The wrapper ensures that in case of a failure an exception can be received on all surviving ranks if it happens inside a guarded block. The term surviving here means, that the rank is capable to continue the computation. We rely on two methods from the ULFM proposal: MPIX_Comm_revoke, which revokes the communicator for any communication and MPIX_Comm_agree to agree on the error state. Once a rank calls a communication method on a revoked communication an error is raised which is then mapped to an exception. A working communicator can be recovered by calling MPIX_Comm_shrink and its siblings, on which the computation can be continued, after the error state has been resolved.
This functionality is typically not available in default MPI installations on clusters yet, and thus, such a wrapper can be convenient in a transition phase. 3. 3. 6. Quality of service. In addition to the system reliability, quality of service (QoS) is increasingly important for NWP to guarantee the prediction code to complete within a reasonable time window. Recently, several papers pointed out the performance variability of large scale HPC systems due to multiple factors. For instance,  Bhatele et al. (2013 Bhatele et al. ( , 2020)) ,  Mubarak et al. (2017)  and  Yang et al. (2016)  observe performance interference by multiple concurrent jobs. This is due to the fact that most HPC systems allow multiple jobs to share interconnect network and I/O subsystem for various types of data transfers.
solution. Two examples are given below that can be applied when the fault translates to data that are lost and the location of the lost data is known. We make the assumption that a mechanism is able to notify the solver which data are lost, assuming a separate mechanism for fault detection. In the current software infrastructure, this corresponds well to node failure where all the data stored on the node are lost and MPI is able to detect the failure of the process running on that node. We describe numerical mitigations that can be implemented to ensure the continuation of the solution process with possibly some delay in the convergence compared to the non-faulty case. After data extraction, a well-chosen part of the missing data is regenerated through any of the described strategies, and is then used as meaningful input to restart the iterative scheme.
With this and other possible solutions, performance gains from fault recovery using spares and spawning, as well as savings in requeueing time and start-up cost of the NWP code, should be weighed against development overhead for the tools themselves. From the systems point of view, standards are already starting to provide resilience tools, an example being the potential inclusion of ULFM as an extension to version 4. x of MPI. In addition, performance and portability across multiple architectures with a single source code requires NWP models to shift to domain-specific-language (DSL)-based approaches, such as Kokkos  (Bertagna et al. , 2019) , Psyclone  (Adams et al. , 2019) , and Stella/GridTools  (Fuhrer et al. , 2014; Thaler et al. , 2019). Fault tolerance efforts requiring code modifications will more naturally interface with DSLs, see Section 3.
In fact, unlike faults on data, soft faults impacting instruction sets of hardware as well as integer arithmetic are not yet addressed by any of the methods in this survey, which have not yet been tested for a specific hardware configuration. Atos, an ESCAPE and ESCAPE-2 project member, takes part in MPI Forum, the MPI standardization body  (MPIForum, 2020). Currently there is a strong interest in the high-performance Fault Tolerance Interface (FTI) for hybrid systems  (Bautista-Gomez et al. , 2011) , and Atos is already delivering FTI as part of its software stack called Super Computer Suite. From the algorithms viewpoint, properties of the methods shown in this paper should be explored in more realistic NWP settings. In particular, interpolation-restart techniques could be deployed to fault-proof linear solvers in atmospheric dynamical cores under development such as the ECMWF's FVM  (Ku ¨hnlein et al.
This report has reviewed potential pathways to algorithmic fault tolerance and computational resilience in numerical weather and climate prediction. Existing numerical techniques have been surveyed, ranging from interpolation-restart and compressed checkpointing approaches for iterative solvers to in-memory checkpointing, ULFM-based process and data recovery for MPI, and advanced backup techniques. The methods were selected because of their proximity to NWP practice -iterative solvers for linear systems being a major component of semi-implicit methods employed in most operational dynamical cores worldwide, and MPI being the de facto standard for distributed-memory parallelism. In order to strengthen the case for hardware and software resilience, NWP scientists should reach out to other scientific communities confronted with related issues.
Funding The author (s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the  ESCAPE-2  project, European Union's  Horizon 2020 Research and Innovation Programme  (Grant Agreement No. 800897 ); the  ESiWACE2 Centre of Excellence, European Union 's  Horizon 2020 Research and Innovation Programme  (Grant Agreement No. 823988 ); and the  Deutsche Forschungsgemeinschaft  under  Germany's Excellence Strategy -EXC-2075  (Grant Agreement No. 390740016 ).
One message can be about voting while another is related to feeding ducks and both are tagged as #DuckTales. is issue can be addressed as variety from big data aspect and ambiguity from natural language processing aspect. Moreover, detection of a real-world event with large volume and velocity of data requires more research than finding an event on selected and filtered datasets  [6]. Another problem with this media is noisiness of posted tweets. ese tweets, unlike news articles and intellectual documents, are not well written and contain misspelling, grammatical errors, and even words or expressions like "yaaaaaay" that are not literary. Expressed problems of this media make TDT task much harder. Data mining and artificial intelligence community has seen many research works done in this scope which show promising leverage compared to each other.
Many of these works are based upon simple bag of words model while others keep searching on probabilistic topic models and still some of them look for sudden change in monitored properties. e common part of them all is the use of natural language processing techniques and methods instead of character level stochastic n-gram models. ese methodologies have come to aid in accomplishing the task of detecting and tracking events, and topics on social media streamlines are emerging to answer couple of questions such as the following: (i) What everybody talks about in a specific time? (ii) What is trending? (iii) What happens somewhere on Earth? (iv) Also, dynamic answered questions which have temporal and spatial properties with great increase of public interest. In order to find most related articles to this scope, we used Google Scholar academic search engine.
Users also add misspelling and lengthen to such words, which results in a very unpleasant issue. All of the mentioned problems are also added to big data 3-V model in which a large variety of velocity data along with big volume are generated and need to be processed just-intime to be monitored and tracked. is 3-V model is much more generalized than the 5-V model that is defined as follows: (i) store and retrieve any data with big data properties in large scales. Social data storage and retrieval require NoSQL databases to perform computing tasks  [11]. Other tools and programming languages can be used in this particular job, but the main properties of social big data require making use of the described tools as their relativity.
Categorization of Methods Existing methods for event detection and tracking task in Twitter can be categorized in different ways based on diverse points of view. One of these categorizations distinguishes between methods that only detect versus methods that detect and track events. Some of existing methods only detect while others track detected events and make storyline of detected topics based on timeline of tweets. e first one is also be known as topic detector while the other one realizing importance of tracking is an event detector and tracker, respectively, abbreviated as TD and EDT. Another categorization is raised when different methods use different Twitter data sources. Some use offline datasets for detection and/or tracking while others make use of online Twitter API. is distinction of data acquisition for training and testing part of algorithms raises a comparison error when comparing performance and results accuracy of existing methods.
Two other categories for event detection and tracking are known as retrospective event detection and new event detection. ese two are abbreviated as RED and NED. e main focus of RED is to discover previously unidentified events from offline datasets and documents while NED is focused on finding new events in online data streams. For TDT tasks, these two concepts are broadly investigated, and many research articles have been published to fulfill this task. From Twitter point of view, event discovery algorithm can be either NED or RED. Iterative clustering algorithms such as k-means are a common practice in RED category. Firstly, a document, sentence, or short tweet is selected as an entity and other entities are compared to the first one; if it is close enough in terms of distance in vector space, then both are merged to form bigger cluster; if not, a new cluster is created and this object is assigned to that new one.
In some cases, stemming is also applied for unification of processed words while non-target-language words are also vanished in this process. Elimination of non-target-language words helps improve extracted topic to be in a target language. Tokenization is also another part of preprocessing that gives unique tokens to each word in a tweet. is part of preprocessing is more crucial in TF-IDF (Term Frequency-Inverse Document Frequency) related models. Some methodologies like EvenTweet  [26]  use WordNet  [39]  check as part of their preprocessing. is WordNet dictionary lookup improves correctness of preprocessing output; thus, no non-English and incorrect words will be used for event detection task. Slang word translation is also used to translate user generated words into their formal meaning.
Tokenizing a text is a natural language processing task aimed   [12]  Naïve Bayes classifier ✓ ✓ Twitter API, handpicked users Hot news detection  [13]  BScore based BOW clustering ✓ ✓ Twitter API (offline) Disaster and story detection  [14]  BOW distance similarity ✓ ✓ Twitter API FSD (first story detection)  [15]  BNgram and TF-IDF ✓ ✓ Offline datasets Topic detection  [16]  Cross checking via Wikipedia ✓ ✓ Twitter API, Wikipedia Hot news detection  [17]  Formal concept analysis ✓ ✓ RepLab 2013 dataset Topic detection  [18]  FPM As discussed so far, preprocessing is an essential and inevitable part of any natural language processing algorithm, and in case of Twitter TDT task it is also demanded.
Based on ngram TF-IDF, all tweets are scored and, based on these scores, are then clustered into respective clusters. is scoring and clustering process is conducted in time windows, and in each time step, tweets related to a time window are compared to others that have been posted earlier. e proposed method has been trained on some handpicked datasets collected from Twitter API which were related to sports (the Cricket World Cup 2015), medicine (Swine Flu 2015), and bills (Land Acquisition Bill). Compared to frequent pattern mining methods, this method seems to be a simpler algorithm in terms of software implementation with good results in terms of output topics on some cases that shamefully are not expressed as F-measure, precision, recall, or any related metrics. e only social big data tool that this method uses is Lucene for keyword indexing.
is vector space modeling between tweets and Wikipedia pages checks the following: if any new event occurred, it is reflected as pick user page views in Wikipedia; if it was a false positive, no pick view on Wikipedia-related page happens. Another first story detection system is proposed in  [14]. is system makes use of an improved version of locality sensitive hashing (LSH) within a (1 + ε) × r distance of query point for Twitter first story detection. Time and space bounding narrow nearest neighbor finding problem. is problem arises when huge amount of user tweets are posted per day, and the goal is to find out if they point to a new story/event or a previously seen one; storing all of these data  6  Complexity and finding nearest neighbor between them are almost impossible. Time bounding refers to using a time window instead of computing all data from all times while space bounding points to solving this problem among limited number of tweets.
Probabilistic Models and Classifiers. Probabilistic topics models and classifiers that are described in this section are used to model and classify Twitter datasets or streamlines. One of these approaches that is presented in  [23]  uses a Naïve Bayesian classifier called NB-Text to satisfy this requirement. is probabilistic method is trained over 2,600,000 Twitter messages annotated by humans posted on 2010. is dataset is labeled for training and testing phases. Firstly, a classifier called RW-Tweet is trained to distinguish between real-world and non-real-world events. Weka toolkit  [43]  along with extracted cluster level features is used to train classification model. is Naïve Bayesian classifier treats all messages in a cluster as a single document and uses TF-IDF metric as features. Cluster level event features such as temporal, social, Twitter central, and topical features are utilized for this classifier.
Formal Concept Analysis. Formal concept analysis has been used by  [17]  in an unsupervised fashion. RepLab 2013 dataset  [44]  is used to evaluate this system. Formal concept as it is known from literature is an approach for finding relations between data that is almost hidden in its nature. is relation can be defined between objects and their attributes. Extent: if we see A as a set of objects (itemset), then it is called an extent Intent: if B is a set of all attributes of set A, then it is called intent Formal concept analysis in this way is formalization of extension and intention to find the most related items that possess important attributes in share. In  [17] , tweets are seen as objects and their terms are attributes, which makes this methodology very similar to the ones described in Section 5.
Frequent Pattern Mining Methods. Frequent pattern mining methods have been applied to TDT task in Twitter. Frequent pattern mining (FPM) as indicated by its name is concept of finding frequent itemsets in a database or any related data storage. A simple example of these frequently repeated patterns is described as a set of coffee and donuts which are in most of cases bought together  [46]. In  [19] , a FPM algorithm is introduced for Twitter offline dataset and compared to other relative studies. FP-growth algorithm with small modifications and utilization of similarity metric is applied to form a set of related tweets that form a topic. Cooccurrence patterns between terms that are larger than two constitute main contribution of this work. ree phases of topic extraction in this method are term selection, cooccurrence-vector formation, and postprocessing.
Macro and micro burst patterns are defined as bellow as main contributions of this work. Macro burst pattern is finding all clusters in BT in which BT is a burst topic set, and this task is accomplished with the use of a distance measure among features. Micro burst pattern is finding all subgraphs in user graph G such that sup G (GS) > treshold. is algorithm starts with finding set S that contains all frequent edges, and with use of DFS (Deep First Search algorithm), the subgraph extention algorithm eliminates nodes that do not satisfy the support threshold (τ). e subgraph extention algorithm is executed recursively to extend frequent subgraphs. Association rule mining (ARL) is another approach of frequent pattern mining in relational databases that has been used in  [18]  to detect events in Twitter.
ARL has two parts: antecedent and consequent. An antecedent is an item that is found in data while a consequent is an item found in combination with the antecedent  [47]. ese can be named as if/then (antecedent/consequent) patterns with help of criteria support to identify the strongest and most important relations between items in data. In  [18] , two main equations are used to match rules with regard to their similarity; they are adopted from  [48]. Emerging rules as a contribution of this work are proposed to identify breaking news. US Elections dataset has been used to evaluate the proposed methodology that shows good results in terms of F-measure, recall, and precision. Tracking dynamics of words in terms of graph, or converting sentences into graph representation and trying to understand the spikes inside, is a very useful method.
In this terminology, an event is described to be a change in the source of text stream which is a surprising rise in n-gram frequency. Computed p value for n-gram hypotheses gives a clear insight about the correctness of the null hypothesis that is stated to be "two individual textual datasets of two time frames are generated from one source. " Due to vast variety of n-grams, a suffix tree is also proposed to store the n-gram. Computed frequency is stored in this new data structure, and another algorithm runs over the tree to calculate and store p values along with it. Clustering of discrete wavelet signals of words generated from Twitter is also another approach that is used in  [50]. Unlike Fourier transform, wavelet transformations are  8  Complexity localized in both time and frequency domain and hence able to identify the time and the duration of a bursty event within the signal.
Deep Learning-Based Methods. Transfer learning in deep learning and specially NLP by using new methods and approaches such as Transformers enabled researchers to use pretrained models for various problems. Topic detection and tracking from Twitter is also one of these problems that researchers tried to solve by transfer learning based models such as BERT. TopicBERT is one these methods that utilizes BERT for semantic similarity combined with streaming graph mining  [33]. e proposed architecture is composed of a deep named entity recognition model  [52] , a graph database to store the nodes, and a semantic similarity extraction tool (SBERT). e whole system works in a combined manner in which the different parts constantly try to update the underlying graph database, and an extraction system using probability of clusters and probability of words gets the topics at each moment.
is system beats state-ofthe-art methods on three different datasets and is one of first methods that used Transformers for topic detection and tracking from Twitter. Combination of semantic vector representation of tweets with clustering algorithms is another methodology that is investigated in  [34]. e authors show that utilization of a good semantic feature extractor in form of a dense vector can be quiet useful when dealing with problems such as topic detection. ey have used COVID-19 dataset from Twitter and detected topics relatively. Another similar method for COVID-19 is proposed in  [36]. e authors propose to use Sentence BERT and Universal Sentence Encoder (USE) for sentiment analysis in combination with LDA based topic detection. Autoencoder based fuzzy c-means algorithm is presented by  [37].