dataesr/scientific-paragraphs-categorization huggingface dataset:
INTRODUCTION Response time is a critical metric in interactive systems such as search engines, payment platforms, or recommender systems (e. g. , a flight or hotel reservation system), where the latency in responding to a search query determines the quality of the user experience  [8, 10, 28, 1]. Each of these systems has to complete a different task: search engines compute potential matches to the search query, recommender systems identify the user's profile and activity to make suggestions as the user is browsing, while online payment systems detect fraudulent transactions before accepting payments. Nevertheless, the architecture used, whether on-premise installations or in the cloud, is similar in all cases: it involves a variety of databases and a pipeline of data processing stages where each stage runs on a cluster of machines  [18].
In the past, these pipelines involved mostly data intensive operations (e. g. , queries over databases). Nowadays, there is a widespread use of machine-learning classifiers and predictors, which are computationally heavier, as they have the ability to improve interactive systems in many ways. Thus, existing recommender systems  [6, 55]  and search engines  [12]  such as those used by Amazon, Netflix, LinkedIn, or YouTube, to mention a few, all employ machine-learning within their data processing pipelines. Similarly, the payment systems associated to the same platforms rely as well on machine learning for, e. g. , detecting bots or fraudulent transactions  [62, 4]. Using a real use case as motivation and baseline, in this paper we explore the interplay of data processing and machine learning in search engines pipelines.
Decision Trees (DT) Decision trees have been used in the past for, e. g. , data mining  [21, 22] , and are nowadays widely used for a range of machine-learning tasks such as classification (results are distinct labels) and regression (results are values). Implementations range from libraries as part of ML packages (H2O  [16] , SciKit  [44] , or XGBoost  [13] ), large-scale distributed implementations for cloud platforms  [43] , to database operators such as those available in Oracle Data Mining  [38]. While here we focus on data pipelines for search engines, our solution can be applied as-is in the context of databases or data warehouses that use FPGA acceleration  [54, 3, 32, 26, 59]. Decisions trees are constructed by recursively splitting the data into groups. The split is usually a greedy algorithm that considers all possible splits and decides which one is best at each step using some cost function determining the accuracy of each possible split.
Data Processing with FPGAs There is a substantial amount of work on using FPGAs for data processing, from databases  [56, 59, 26, 32]  and cloud systems  [14, 11, 2]  to machine-learning  [15, 23, 42, 25]. The vast body of existing research has focused on the compute capacity of an FPGA device, ignoring key performance overheads such as the data transfer cost between host and FPGA. Another aspect often ignored is the initiation cost of an FPGA accelerator (to start the processing), typically larger than that of a function call on the CPU. Later in the paper, we look in depth at these factors when considering the feasibility of the solution we propose.
Baseline Route Scoring The heuristics-based approach to selection in the Domain Explorer and the Route Selection often induces drastic and somewhat arbitrary cuts to the domain of solutions considered. Such cuts affect the quality of the results. Route Scoring (shown in Figure  3 ) was added to the pipeline to improve the situation. Route Scoring uses machine-learning to determine which of the proposed routes is most likely to be among the cheapest ones. The baseline implementation for the module uses the H2O framework on production requests to generate a GBDT model. GBDTs were chosen for three main reasons: a predominance of categorical features; the high likelihood of missing features; and a fast training cycle (new models can be generated and tested about every week). The training phase is based on production data.
Inference Engine Architecture Figure  4a  shows the inference engine architecture. The design is built out of many Processing Elements (PE) organized in Compute Units (CU) as illustrated in Figure  4b. A processing element can be programmed at runtime to process one or more decision trees in parallel. A Compute Unit consists of 28 PEs, and a tree of single precision floatingpoint adders that sums up the individual decision tree results produced from the 28 PEs. A Compute Unit is designed to process a complete decision tree ensemble. Hence, multiple Compute Units are allocated to parallelize the scoring of different routes. The Collector Unit then gathers individual route scoring results and writes them back to the FPGA memory. The Distributor is responsible for first replicating the tree ensemble to all CUs, then distributing incoming routes data in a round-robin fashion to CUs for processing.
The I/O   Unit first receives a request with a set of parameters through the command interface to perform a scoring query, then it loads the tree ensemble from the off-chip memory to the FPGA. Once all the trees are stored in the Compute Units, the engine becomes ready for data processing. The I/O Unit then starts reading the routes data from the FPGA memory and feeding it to the Compute Units (through the Distributor ) for processing. The engine can process 896 trees in parallel, either for tens of different routes (same trees processed for different routes as in small tree ensembles) or for just a few routes if the tree ensemble consists of hundreds of trees. Both the I/O Unit and the Collector have direct access to the FPGA on-board DDR memory. The command interface is connected to the PCIe interface of the FPGA card, such that the software running on the host CPU can write a command directly to start the scoring query.
For the CPU baseline, we reimplemented the H2O Java version in C++. Profile-based optimizations to the memory control and code streamlining were used to improve performance. The C++ implementation can score 60% more routes than the original H2O Java version. We did not optimize the data format and structures to maintain compatibility with the full specification of H2O. The FPGA design is implemented in SystemVerilog. The FPGA is configured with two inference engines. Each inference engine has four CUs and each CU contains 28 PEs. The Tree Memory in the PE is 8 KB in size, which can accommodate up to four trees of the model used for testing. This is more than enough to fit the trained GBDT model in a single compute unit. Performance metrics. We use scored routes per second as the performance metric.
As a comparison, on the Intel's HARP v2 platform, the inference engine on the FPGA can access the CPU main memory directly without the need to first transfer the data to the FPGA memory. This means the HARP v2 performs better than the AWS F1 and VCU1525 FPGA card when the cost of data transfers is considered even if it is nominally a smaller and slower FPGA. Another way to speed up data transfers is on FPGA with a network connection with sufficient bandwidth. The data can then be streamed directly to the scoring engines, eliminating the data transfer overhead and utilizing the full compute throughput. The existing solution at Amadeus implements the Route Scoring system using a total of 10 HP ProLiant servers, achieving 15 million routes per second in aggregate throughput. If deployed in the AWS Cloud, 25 CPU c5.
Scaling with request size An important aspect of the Route Scoring system to examine is the overhead of initiating a scoring request on the overall performance. Initiating a scoring request includes a kernel invocation as well as scheduling and initiating the necessary data transfers. In the previous Sections we used a scoring request of 1 million routes, which is large enough to hide the invocation overhead. In this Section we evaluate the performance for a range of request sizes (i. e. , number of routes to score). We used the same inference engine configuration and tree ensemble as in Section 6. 1, and considered a wide range of request sizes from a couple of hundreds of routes to hundreds of thousands. In these experiments, the FPGA device on the AWS F1 is programmed with one inference engine.
DISCUSSION The performance improvements achieved on the FPGA opens up several options for integrating the Route Scoring system in the search pipeline even if we are bound by existing architectural configurations. If the Route Scoring is embedded inside the Route Selection stage, the ten dedicated servers currently used for the Route Scoring system in a data center can be replaced by a single FPGA-based server. If the Route Scoring is deployed on a single FPGA server, the server must have sufficient network bandwidth to feed enough data to the FPGA to utilize its full compute capacity. To maintain the current throughput of all ten servers (15 million route per second), a minimum aggregate network bandwidth of 12 Gbps is required, within easy reach of modern data center networks. An alternative design that removes the need for an additional server for Route Scoring is to equip every Route Selec-  tion server with an FPGA implementing the route scoring engines, removing the network overhead (Figure  12 ).
RELATED WORK 8. 1 Low-Latency Data Processing Pipelines There have been many efforts both in academia and industry to use hardware accelerators such as FPGAs to improve the response time of latency-sensitive data processing pipelines. Microsoft initially developed the Catapult platform  [46, 11]  to accelerate the search queries of the Bing search engine, in particular the document ranking stage of the Bing data processing pipeline. The document ranking stage employs a series of feature computation, features synthesis, and machine-learning based document scoring. The FPGA acceleration of the ranking service reduced tail latency by 29% while maintaining equivalent throughput. Alibaba Cloud recently deployed an accelerated database system called X-DB to boost the performance of their services  [2].
In addition to accelerating SQL search queries, the FPGA performs the compaction operation, which merges multiple versions of the SSTables in the persistent storage and keeps the latest version. Alibaba was able to achieve up to 60% improvement on throughput and reducing significantly the performance jitters caused by the compaction operation competing with normal transaction processing on the CPU. Similarly, Baidu has deployed Software Defined Accelerators (SDA) based on Xilinx FPGAs in their datacenters to accelerate data analysis systems (e. g. , Spark)  [39]. The SDA offers a flexible hardware architecture implementing core SQL operators (join, select, filter, sort, group by, aggregate). A SQL plan is translated into a program to configure the SDA to perform a specific query. For certain benchmarks, the SDA achieves an improvement of one order of magnitude over CPU in terms of query response times.
Sidler et al. [53]  have proposed an FPGA solution for accelerating database pattern matching queries, the proposed solution reduces query response time by 70%. Similarly, Kara et al. [27] , demonstrated how offloading the partitioning operation of the SQL join operator to the FPGA can significantly improve performance and offer a robust solution. High-speed trading is another field where FPGAs have been used to significantly improve response time. Tang et al. [57]  developed a market-data processing library for FP-GAs that achieves an improvement of two orders of magnitude over CPU latency. All the prior efforts draw parallels to our effort in this work to improve the overall response of a given data processing pipeline. In all the mentioned accelerator solutions, as in ours, the developers strove to meet two conditions: lower latency and accelerator flexibility to adapt to changing workload characteristics and user requirements.
The Virtex Ultrascale+ FPGA we use offers nearly eight times the amount of resources of their FPGA, and is manufactured with 16 nm process-node. If we reimplement their design on our FPGA (we do not have access to their source code), adapt the design to our GBM trees and ensemble size which is nearly 16 times more complex in operations and memory requirements, we estimate that we will need two Ultrascale+ FPGAs to implement the trees. Running at 200 MHz their solution would deliver about 80 M routes/second. Compared to our solution, using one VCU1525 we achieve 94 M routes/second (computeonly). Integrating machine-learning in relational databases has been explored to benefit from the optimized data structures and operators inside the RDBMS. Sattler et al. [51]  proposed a set of SQL primitives as building blocks for implementing a decision tree classifier in relational databases.
To alleviate the overhead of the slow decision tree primitives in a database, Bentayeb et al. [9]  proposed an approach to represent an ID3 decision tree using SQL views. Recently, a large body of research work has explored a range of hardware accelerator solutions for Deep Neural Networks (DNNs), another machine-learning method. Microsoft Brainwave project  [15]  deploys an FPGA-based DNN processor in the Bing search engine. The FPGA solution meets the strict latency requirements of the Bing data processing pipeline offering an order of magnitude lower response time compared to a CPU-only solution. The Brainwave DNN processor harnesses the vast amount of FPGA resources to create wide parallel datapaths with a custom instruction set. As in our case, the approach must support the frequent updates to the DNN models.
Machine Learning on GPUs GPUs offer tremendous parallel processing capacity by allocating thousands of scalar processing elements equipped with double-precision arithmetic units. GPUs use a Single Instruction Multiple Data (SIMD) execution model, where parallel processing units concurrently crunch large amounts of sequential data. The GPU execution model and architecture is suitable for methods such as neural networks but it works less well for non-sequential control flow, low-density computations, and if a significant amount of random memory accesses is involved. These are, unfortunately, precisely the characteristics of decision tree inference  [48]. Existing work on using GPUs for decision trees focuses almost exclusively on training and learning of the trees, a task where GPUs excel. Only recently, NVidia described a library for inference over gradient-boosted trees (XGBoost models) that requires users to make the input dataset and the tree ensemble dense.
Introduction Argument component (AC) extraction typically involves addressing extremely complex high-level concepts, demanding significant amounts of world knowledge, natural language understanding, and reasoning to address  (Moens, 2018). These argument components may come from different datasets, different domains, and have varying tagsets (IOB -inside, outside or at the beginning of an entity), depending on the component and annotation criteria used  (Schulz et al. , 2018). Originally the field expanded its tagsets across tasks over time; however, due to the inherent difficulty, the field has contracted back to tackling much simpler tasks  (Moens, 2018). This difficulty is because performance across domains and tasks with limited resources makes training models extraordinarily difficult.
Recent work such as  Schulz et al. (2018)  uses single-task learning ("STL") and multi-task learning ("MTL") with character-level encodings and pre-trained GloVe word embeddings as inputs to a BiLSTM-CRF encoder to analyze this issue from a low resource standpoint, while other work approaches the task through the use of graph convolution networks (GCNs) with syntactic dependencies  (Morio and Fujita, 2019). However, both evaluate in terms of tag-level F1, including non-target O tags, rather than the more stringent span-based metric conventionally used to evaluate named entity recognition ("NER": Tjong Kim Sang and De Meulder (  2003 )). In this paper, we compare contemporary embedding approaches in STL and MTL contexts against  Schulz et al. (2018)  and achieve state-of-the-art results for the dataset, but more importantly, we demonstrate that under span-based evaluation, the current state-of-the-art is woefully low, calling into question whether argument component extraction as currently construed is feasible for current NLP methods.
Task Description Argument component (AC) extraction is the extraction of ACs such as factual premises and opinion-based claims from text, using a tag-based IOB system to extract the textual components as contiguous sequences of text as NER components  (Schulz et al. , 2018). The tasks are from a variety of disparate domains, with different IOB tagsets and associated distributions, some with simple claims or premises, others with more complex annotations  (Schulz et al. , 2018). The tasks are, as per previous work: var, wiki, news, essays, web and hotel  (Schulz et al. , 2018). These are NER tagged sentences that contain IOB tagged claims, premises, or more specific argument tags (with respect to the specific dataset annotation guidelines). They are sourced from various editorials/official documents/discussion boards, Wikipedia discussions, news comments, persuasive essays, web discourse, and hotel review domains respectively  (Schulz et al.
Analysis We find that in general, MTL often underperforms STL for individual tasks, which is in contrast to previous work  (Schulz et al. , 2018)  (see Figure  1 ). We hypothesize that this is due to the disparate domains, annotations, IOB distributions, and label sets of the various tasks. Therefore even with the extra supervision signal, MTL tends not to aid in the training process, especially with well-initialized pre-trained embeddings. We hypothesize that focusing training on sampling the core task with the pre-trained embeddings (with suitable regularisation -see Section 3. 3) will likely lead to better span extraction performance in lowresource, disparate domains (especially given the disparate label sets for the respective datasets), where the more robust and general performance of MTL is traded for higher performance in specific tasks.
We often find that in the STL/MTL cases there is a minimal improvement over the baseline CNNbased trained character embeddings and that the representational capacity of the pre-trained models is likely not sufficient to provide a significant improvement on these tasks. We find that in general F1 is substantially below much simpler tasks such as CoNLL NER, with the majority of our results well below an F1 of 0. 5 (see Figure  2 ), whereas CoNLL models trained equivalently produce results well in excess of 0. 9. In some cases such as the essays and hotel datasets, we see what we would expect with increasing pre-trained model complexity added to both STL and MTL tasks. However news, web and wiki all seem to exhibit highly variant baseline performance regardless of training methodology or pre-trained initialisation.
, 2018) , we were unable to produce results on par with tag-based evaluation. However span-based extraction provides a more realistic assessment of argument component extraction, with bert_stl generally providing the highest average score. We also validated our results against the CoNLL NER dataset for all ablations and found performance to be on par with existing SOTA systems  (Peters et al. , 2018). Thus more pre-trained, more diverse, and more integrated representations do help improve the performance across these tasks on average, but the performance for argumenta-tion component extraction leaves a lot to be desired under the span-based metric, suggesting that a usable extraction system is still well beyond the reach of current NLP models, based on the existing task formulation.
INTRODUCTION Block-diagram tools like SCADE Suite 1  and Simulink 2  are used to design control software. At their core are dataflow languages: operators apply point-wise to streams, state is encoded by unit delays, and subsystems are abstracted as stream functions. The Lustre synchronous language  [Caspi et al. 1987 ] epitomizes these ideas, but more sophisticated applications require more sophisticated constructs like state machines. State machines can be compiled into primitive constructs  [Colaço et al. 2005; Maraninchi and Rémond 2003] , foremost among which is the reset operator  [Hamon and Pouzet 2000] , which is a useful primitive in its own right and the subject of this article. In earlier work,  Bourke et al. 2017  describe the specification and verification in the Coq [Coq Development  Team 2019]  Interactive Theorem Prover (ITP) of a compiler for the dataflow kernel of Lustre.
We present its non-trivial extension to treat the reset operator, which permits a subsystem to be restarted in its initial state. We solve two main problems. The first is to reconcile the dataflow semantics of the core constructions with the imperative nature of the reset operator. Building on an earlier idea  [Hamon and Pouzet 2000 ] and preliminary work  [Bourke et al. 2018] , we show that the existing relational semantics can be extended by adding a single extra rule. The second challenge is to prove that this construct is compiled correctly. To do so we introduce a new intermediate language and associated semantics, adapt a model that specifies the manipulation of internal state, and combine both to show that a new compilation pass is semantics preserving. The overall result is an end-to-end proof showing that the assembly code generated by our compilation passes coupled with those of CompCert  [Leroy 2009 ] respects the extended dataflow semantics.
, ‹›, ‹11. 49›, ‹11. 49›,. , allows each value, and the corresponding computation, to be associated with a precise instant on a discrete time base. Moreover, the language is restricted to guarantee execution in bounded memory by forbidding recursion and limiting how operators may combine streams  [Caspi 1992 ]: for example, one of the two streams combined by a merge must be absent when the other is present. The result is a stream language suitable for programming real-time embedded systems. Another useful perspective is that Lustre is a language for composing transition systems that communicate by sampling  [Caspi and Pouzet 1998 ]. The when and merge operators allow for guarded activation with precise control over sampling, and restrictions on causality and clock types ensure determinism.
Nevertheless, the intuition of 'a fresh instance at each reset' is central to the semantic model presented in the next section. Not only is the reset operator convenient for programming and for compiling other control structures, it can be compiled to efficient, modular code by introducing a recursive reset function for each node as will be explained in section 3. While we focus on a subset of Scade 6, the principles studied are fundamental to the dataflow languages that underlie tools for Model-Based Design. Simulink, for instance, provides 'blocks' for resettable 5  and enabled 6  subsystems. In our context, they are simply compositions of ad hoc nodes with reset, sampling, and delay operators, as in the following equations. out1, out2 = (restart resettable_subsystem every rt) (in1, in2); out3, out4 = merge r ((restart enabled_subsystem every rt) ((in3, in4) when r)) (0.
DATAFLOW SEMANTICS Formal semantic models have been fundamental in the development of synchronous languages in general, and of Lustre and Scade 6 in particular. Compared to functional and imperative languages, however, less attention has been given to their mechanization within ITPs. As usual, definitions must capture intuitions about the language, act as specifications for compilation (the present focus), and serve as a base for program verification (not considered here). We build on the predicates introduced by  Bourke et al. 2017  and present an additional rule to account for the reset operator.