dataesr/scientific-paragraphs-categorization huggingface dataset:
Also, MadDM is the only tool that can easily provide results accurate up to n loops with custom models, while the other ones stop at the tree level. a  In this paper, we present the package DarkPACK, which aims at automatically generating a numerical library of scattering amplitudes to compute dark matter observables such as relic density, and which is interfaced with SuperIso Relic v4. 1. With DarkPACK, we want to propose a tool that can handle by itself all the steps from the definition of the model to the output of the sum of the squared amplitudes and the computation of dark matter related observables. DarkPACK is intended to be comprehensive, in the sense that we avoid unnecessary passages of input and output. Among the advantages of this approach, there is a larger flexibility in the code usage.
In fact, it is easier to modify an algorithm if it is written in a pre-established comprehensive framework. For such reasons we have chosen to rely on MARTY for the model building and the symbolic manipulations. Nevertheless, it is crucial to have modularity as a feature, since this enables an easy linking with external softwares if needed. In DarkPACK v1. 1 the generated library contains all the squared amplitudes of 2 NP particles into 2 Standard Model (SM) particles at the leading order and up to the one-loop level. In order to validate the results, we focus on the MSSM which is an adequate benchmark in terms of complexity and compare in particular the results of DarkPACK with those of SuperIso Relic v4. 1, which relies on a self-generated FormCalc  [5]  FORTRAN code. b  This article is structured as follows.
Setup of the package DarkPACK v1. 1 is available among the attachments of this article. Alternatively, the source code is hosted on the GitLab repository of the IN2P3, and it can be found at the address: a Some of them can have one-or multi-loop accuracy for the MSSM. b As explained in the conclusion, we foresee an upgrade of SuperIso Relic based on DarkPACK, since on one side DarkPACK is conceived to be modular, and on the other side the way SuperIso Relic v4. 1 computes amplitudes is not suited to be flexible and adaptable to compute new observables. https://gitlab. in2p3. fr/darkpack/darkpack-public. The version 1. 1, described in this work is available as a commit in the master branch with tag v1. 1. In order to use the code, the user needs a working setup of MARTY installed c. No further dependencies are needed.
e The script will work by using the files in auxiliary_library/<1> as the source code for the extra functions specific to the library and the files in auxiliary_library/script_<1> as the source code for the executable files. For standard use cases, the files in these two directories are the ones to be written specifically for any new library. Please see the content of section 5 for the description of the source files we provide, both general and library-specific. The content of the package is the following. The numerical library for the MSSM can be found in the folder mssm2to2, and external files for the library in the auxiliary_library folder. In order to generate the library from scratch f , the source file (e. g. MSSM. cpp for the MSSM) has to be compiled and the corresponding program launched.
Outlook of the code This code has been developed with the purpose of having a new tool to compute 2 to 2 scattering amplitudes, firstly in the MSSM at the leading order (LO). In this release, we provide the setup for the MSSM. g The idea behind this package is to use MARTY in order to define the desired model and quantities to compute and to export a numerical C++ library. With MARTY it is possible to compute symbolically scattering amplitudes up to the one-loop level. However, MARTY 1. 6 does not provide tools to apply automatically a renormalization scheme. As a consequence, a computation can be successfully performed at one loop if there are no divergent diagrams at the tree level, meaning that the one-loop level has to be the leading order. h In particular, in this release there is the example file MSSM.
Within this framework, more potentialities of MARTY will be exploited in the future. As far as input values are concerned, in this release we provide algorithms to read from. lha  [20, 21]  files. Decay widths at the tree level and mass spectra can be computed using MARTY if needed, otherwise, the necessary quantities have to be provided among the input parameters. In the auxiliary_library/script_mssm2to2 directory some example files are provided to show how to use the main features of the code in practice. The users are invited to contact the authors if they need some more examples or features. We provide: • example_1_single_process. cpp An example file that shows how to initialise a single process and do some calculations. • example_2_running. cpp An example file that shows how to easily run the Standard Model parameters.
Input reading and manipulation Since each model has its own parameters, the input management has to be handled by the users accordingly to their needs. In this paragraph, we describe the functions which we defined in order to work within the MSSM. The headers we use are: • leshouchesfromsuperiso. h: its functions are defined under the namespace mssm2to2::superisosupport, and they allow us to read from a. lha file calling SuperIso's routines; • leshouchesfrommarty. h: its functions are defined under the namespace bsm2to2::readmodule, and they allow us to read from a. lha file calling the lha extension native in the MARTY installation. For a general use, the library to refer to in order to modify the input management is leshouchesfrommarty. h. The library leshouchesfromsuperiso. h is useful while dealing with the MSSM only.
Handling the running of SM parameters Below we list the functions for the running of SM quantities, i. e. the strong coupling constant g s , and the top and the beauty quark masses. This is handled via a structure called RunningSM, accessible after including RunningSM. h. We used the code provided in SuperIso Relic v4. 1 to m By the appropriate replacement of the line using namespace mssm2to2::pdg2016Value; in auxiliary_library/macros. h, following the namespaces defined in auxiliary_library/params_new. h. n See for example https://marty. in2p3. fr/doc/marty-manual. pdf. define the methods of this class. Here we list its main public methods and elements: 1. RunningSM(const Param_t &input) This constructor builds the class variable starting from the values stored in a Param_t variable.
Comparison with other softwares We validated our code within the MSSM, testing 3430 processes of the type SUSY + SUSY → SM + SM. These processes are the ones that can be directly found in the mssm2to2 library in the repository. We compared our results for the sums of the squared amplitudes and the contribution to dW eff d cos(θ) with the ones from SuperIso Relic  [2] [3] [4] , and our results for total and differential cross sections with the ones from micrOMEGAs/CalcHEP  [11, 13]. In this way, we validated the behaviour of MARTY's MSSM library  [6]  and also we performed numerical tests. The comparison of the results of the total cross sections with micrOMEGAs/CalcHEP has shown that there are some cross sections (around 9 ) for which the integration algorithm produces numerically unstable results with any software.
If the number of intervals exceeds 256, the last value with the last o for the definition of the QCD models we refer to  [4]. error is taken as the result. p We analysed the behaviour of this method, concluding that the trapezoidal rule made converge 16% of the cross sections that do not converge with the Gauss-Legendre method, achieving the 9 of non-convergence mentioned earlier. The comparison with other softwares has been made by taking care of using the inputs in the same way. In particular, we have set the CKM matrix to the identity matrix, in order to compare the results with the SuperIso Relic/FormCalc code, and the mass of the muon and the electron to zero for the comparison with micrOMEGAs/CalcHEP. The outcome from the comparison with SuperIso Relic is that all the sums of the squared amplitudes are in agreement, except for the heights of some peaks.
Conclusions In this paper, we presented a new package to deal with sums of 2 to 2 squared amplitudes to the leading order in any New Physics scenario, as well as other dark matter related quantities, such as ⟨σv⟩ and the relic density. The development of such a code addresses the needs for modularity and flexibility which can be more challenging to achieve in already existing tools, such as FormCalc  [5]  and CalcHEP  [13] , which are the backends for SuperIso Relic v4  [4]  and micrOMEGAs5. 0  [11] , respectively. Using DarkPACK, the user can more easily parallelise the calculations, since global variables have been avoided as much as possible, achieving a better time performance. Additional features may be added upon request of the users. The part of this code relative to dark matter uses SuperIso Relic   H 0 (p 4 )   DarkPACK SuperIso   as a base, and adds more features and functionalities.
Statistical analysis and tools Data from analyzed images was processed using Genedata Screener (Genedata, Switzerland). Normalization was the percent of infection where neutral control is 0% inhibition of infection and "No-Virus" control is 100% inhibition of infection. Further chemoinformatic data visualizations were made with Certara D360 software, which is integrated with the CDD database. Excel (Microsoft, USA) was used to analyze the data and Prism (GraphPad, USA) was used to plot the data. Whenever comparing between two conditions, data was analyzed with two tailed student's t-test. Measurements are reported as mean of at least three biological repeats, and the error bars denote standard error of mean (SEM). Throughout the study, threshold for statistical significance was considered for p-values≤0.
Acknowledgements We thank members of the  Avinoam laboratory , and The  Wohl Drug Discovery Institute of the Nancy  and  Stephen Grand Israel National Center for Personalized Medicine (G-INCPM ) for discussions. We also thank  Yosef Shaul ,  Romano Strobelt  and  Julia Adler  for cell lines and plasmids, and  Benjamin Podbilewicz ,  Clari Valansi  and  Gideon Schreiber  for plasmids,  Galit Cohen  and  Shirly Valter  for assisting with compound plating. This research was supported by  Israel Science Foundation  (grant no. 3729/20 ),  Bina Nurture Program , and the  European Research Council (ERC)   Proof of Concept Grant  ( 101100758 -Inhibicov ). OA also acknowledges funding from the  Henry Chanoch Krenter Institute for Biomedical Imaging and Genomics , the  Schwartz Reisman Collaborative Science Program , and the  European Research Council (ERC)  under the  European Union 's  Horizon 2020 research and innovation program  (grant agreement no  851080 ).
, 2023). In this paper, we propose and evaluate two methodologies for determining the necessity and placement of citations within a sentence. These solutions can be adopted by a variety of tools for assisting students and researchers in their scientific writing endeavors. The first methodology conceptualizes the task as a Mask-Filling problem and leverages a generative model to solve it. The second strategy formulates it as a Named Entity Recognition task and aims to identify the tokens that should immediately precede a citation. We refined the outcomes of both methods by applying a set of NLP heuristics, further enhancing the accuracy of the predictions. In order to facilitate the fine-tuning of transformers models for this task, we constructed 2orc-9K, a novel open dataset including 9,000 articles where citations have been replaced with placeholders.
• We propose two new transformer-based methodologies based on these formalizations. • We developed a set of NLP heuristics that can significantly enhance the performance of both methodologies. • We released s2orc-9K, an open dataset for fine-tuning models on this task, and a new gold standard composed of 133 sentences annotated by three senior researchers. • We carried out a comparative evaluation of the two proposed methodologies against humans. We also present an in-depth analysis of the best method, investigating the impact of different parameters, training set sizes, and citation intents. • We present a statistical analysis demonstrating that one of the proposed generative models yields results that are not statistically different from those achieved by three senior researchers (𝑝 = 0.
58). • We provide the full codebase for implementing our methodologies, the relevant datasets, and the gold standard. 1   The remainder of the paper is organized as follows. Section 2 discusses related works and the differences with our methodologies. Section 3 contains the research objectives and, as such, formally defines the task we aim to solve. The datasets used for training and evaluating the methodologies are presented in Section 4. Section 5 discusses in detail the two new methodologies and the approaches we implemented for each of them. Section 6 reports the evaluation and the statistical analysis. Finally, Section 7 ends the paper with conclusions and future directions of research.
Related works In literature, various works have addressed the prediction or recommendation of citations for a given research paper. One of the first mentions of ''citation prediction'' was in the 2003 KDD cup  (Gehrke, Ginsparg, & Kleinberg, 2003) , a Data Mining and Knowledge Discovery competition organized by the ACM Special Interest Group on Knowledge Discovery and Data Mining (SIGKDD). The task consisted of the prediction of the number of citations that a paper would receive over time, therefore, it resembled a 1 https://github. com/Marcomurgia97/Citation-Prediction-by-Leveraging-Transformers-and-Natural-Language-Processing-Heuristics bibliometric exercise more than a recommendation one. One of the first works approaching citation recommendation from a textbased perspective is the one from  Tang and Zhang (2009).
CiTO  (Peroni & Shotton, 2012 ) is an ontology describing the nature of reference citations in scientific research articles and other scholarly works, providing more than 20 citation types. Lauscher, Glavaš, Ponzetto, and Eckert (2017)  studied the applicability of Convolutional Neural Networks and embeddings to the semantic classification of citation intents. GraphCite  (Berrebbi, Huynh, & Balalau, 2022)  used a hybrid graph and textual-based embedding method to classify citation intents into 6 different types on the ACL-ARC dataset  (Bird et al. , 2008). However, works addressing the task of determining if a sentence needs citations or the prediction of the position of a citation in a text are rare. This is a problem that only recently attracted some interest from the research community.
Boyack, van Eck, Colavizza, and Waltman (2018)  carried out a study over a large collection of research papers in different domains to identify when and where citations occur. They showed on average about 20% of sentences contain a reference, while less than 2% of sentences contain three or more mentions. They also found out that most citations occur within the first 5%-10% centiles of the studied documents. Cohan, Ammar, van Zuylen, and Cady (2019)  used a Multi-Layer Perceptron model to determine the ''citation worthiness'' of a paragraph. They also proposed a dataset (SciCite) for citation intent prediction that is larger than ACL-ARC. Vajdecka, Callegari, Xhura, and Ásmundsson (2023)  recently proposed a Large Language Model-based method to predict whether a citation should appear or not in a sentence, obtaining F1-scores between 75% and 89%.
The datasets To produce an accurate citation prediction model, it is advisable to employ a model that has been pre-trained on scientific article data. For this purpose, there are two possible strategies: (1) utilizing an existing transformer that was pre-trained on research papers or (2) pre-train a transformer on an available dataset. The first option could be more convenient and less expensive, but it is also less flexible. In this paper, we have explored both solutions. For the pre-trained transformer, we opted for arxiv-nlp, 2  a GPT-2 transformer that was pre-trained on arXiv-80MB, a dataset composed of 80 MB of plain text extracted from arXiv 3  papers about Computation and Language. We adopted arxiv-nlp as one of the generative models that we used to tackle citation prediction as a Mask-Filling problem.
Since this dataset was not specifically curated for citation prediction, it contained a variety of citation styles, such as square brackets with numbers or brief text, parentheses with citations expressed using the first author's name and publication year, and other formats. This is evident in the behavior of the resulting transformer, which generates citations in many different styles. In order to solve this issue, we created and made publicly available s2orc-9K, a novel dataset for citation prediction in which all citations have been normalized according to a single format. By providing a consistent citation format throughout the dataset, we aim to enhance a transformer's ability to comprehend and learn the underlying concept of citation more effectively. We generated the s2orc-9K dataset from the s2orc dataset  (Lo, Wang, Neumann, Kinney, & Weld, 2020) , a corpus containing about 80 million scientific articles from Semantic Scholar.
4  For each paper, s2orc includes several metadata, such as the title of the paper, the relevant venue, the year of publication, the URL on Semantic Scholar, the topic, the abstract, and the paper's sections. In order to produce s2orc-9K, we selected 9,000 papers from s2orc covering the Computer Science domain according to the topic field in the metadata. Subsequently, we standardized all citations by replacing them with the placeholder [ ]. The resulting dataset comprises approximately 43. 5 MB of text files. The s2orc-9K dataset has been used both to train an alternative transformer for the generative methodology and the NER approach.
Generative methodology The idea behind the generative methodology is to leverage the text-generation capabilities of transformer models. Currently, the most widely used transformers for this type of task is the GPT family of OpenAI. 5  We adopted the GPT2 transformer  (Radford et al. , 2019). GPT-2 is a generative language model pre-trained on a set of about 40 GB of unlabeled data. Given an input phrase or word, GPT2 predicts the next token. Specifically, we adopted the version of GPT-2 with 117 million parameters and 12 layers. 6  We chose GPT-2 because it is open and it is available in the most used deep learning libraries such as Transformers. 7   GPT-2 operates on a decoder-only architecture, setting it apart from other transformer models like BERT  (Devlin et al. , 2019). Indeed, the architecture of transformers generally consists of two main modules: encoder and decoder  (Vaswani et al.
, 2017). A decoder-only architecture is characterized by the absence of the encoder module and has different decoder blocks on top of each other. As shown in Fig. 1 , each layer of the transformer consists of a masked self-attention layer and a feed-forward neural network layer. GPT-2 generates text, token by token, one at a time, so it is possible to check each generated token and identify whether or not it is a citation. Basically, our methodology for citation prediction starts by giving as context the first token and checking whether the predicted token is a citation. If it is, we associate a citation to the first token. Then, we repeat the generation process by feeding the model a context of the first two tokens and so on. Whenever the model predicts a citation, we associate it with the preceding token.
As discussed in Section 4, the transformers used in our experiment were trained on two datasets: a set of non-preprocessed arXiv papers within the computational linguistics domain (arXiv-80MB) and a smaller dataset of about 43 MB papers (s2orc-9K) wherein all citations were replaced with a standardized placeholder. Since arXiv-80MB includes citations in various formats (as those indicated in Table  1 ), the corresponding transformer will generate a diverse array of citation styles. Consequently, there is a possibility that certain citations may adopt too many different styles, leading to potential challenges in their recognition. Conversely, the transformer pre-trained on the novel s2orc-9K dataset, introduced in this paper, will produce a single placeholder ([ ]). This facilitates significantly the identification of a citation.
NER-based methodology The second methodology we propose to tackle the citation prediction problem is a NER approach that aims to identify the token preceding a citation. To this purpose, we adopted the BERT transformer 9   (Devlin et al. , 2019). The pre-training of BERT was    We fine-tuned the BERT transformer for the NER task defined in Section 3 using the s2orc-9K dataset. To accomplish this, we initially pre-processed the dataset by annotating each token preceding a citation with the CITATION label, while assigning a NORMAL label to all other tokens. Additionally, the citation placeholder was removed. For example, let us consider the annotated sentence ''advanced NLP tasks [ ]''. In this case, the token [ ] is eliminated from the sentence, and the preceding word, ''tasks'' in this instance, is labeled as CITATION as depicted in Table  2.
Natural Language Processing heuristics We defined a set of NLP heuristics to further enhance the performance of the proposed methodologies. These heuristics are systematically applied to the outcomes obtained from both the generative and NER methodologies and are not involved in any phase of the training steps. The heuristics applied in this study are designed to adhere to standard rules governing the usage of citations. For instance, one heuristic aims to address cases where a predicted citation is situated between a series of uppercased tokens and the corresponding 10 https://huggingface. co/datasets/bookcorpus 11 https://huggingface. co/datasets/wikipedia acronym that refers to them. In such instances, the heuristic removes the predicted citation and places it directly after the acronym, as it is commonly acknowledged that this is the appropriate citation placement.
Evaluation This section reports the evaluation of the two methodologies introduced in Section 5, along with the datasets for fine-tuning detailed in Section 4 and the heuristics introduced in Section 5. 3. In the following, we first describe the generation of the gold standard and discuss the experimental design. We then report the performance of the different techniques in terms of precision, recall, and F1. Next, we present an analysis of the impact of the 𝑤 𝑔𝑒𝑛 parameter, the training set size, and citation intents  (Roman et al. , 2021)  on the performance of the best method. We conclude the section by conducting a rigorous statistical analysis to assess the consistency between senior researchers and the computational models.
Experiments This section outlines the tested methods and the metrics we adopted for their evaluation. We compared six alternative approaches that combine the previously discussed methodologies with the two datasets introduced in Section 4: • NER-s2orc, the NER-based methodology described in Section 5. 2 fine-tuned on s2orc-9K. • NER-s2orc-H, NER-s2orc making use of the heuristics described in Section 5. 3. • GM-arXiv, the generative methodology described in Section 5. 1 pre-trained on arxiv-nlp. • GM-arXiv-H, GM-arXiv making use of the heuristics. • GM-s2orc, the generative methodology pre-trained on s2orc-9K. • GM-s2orc-H, GM-s2orc making use of the heuristics. The last four methods were executed with window 𝑤 𝑔𝑒𝑛 of size 5, as this value produced the best outcome in preliminary experiments.