dataesr/scientific-paragraphs-categorization huggingface dataset:
Implementation: CTranslate CTranslate is the open-source inference engine for OpenNMT models (initially designed for Lua Torch). The code is implemented in raw C++ with minimal dependencies using the popular Eigen linear algebra library. CTranslate's goal is to offer a lightweight and embeddable solution for executing models. It benefits from Eigen efficiency and is about 20% faster than a Torch application using Intel MKL. Additionally, the use of C++ over a garbage-collected language ensures a predictable and reduced memory usage. All additional optimizations are built on top of this library.
We describe the architecture and functionalities of a C++ software framework, coined PARTONS, dedicated to the phenomenology of Generalized Parton Distributions. These distributions describe the three-dimensional structure of hadrons in terms of quarks and gluons, and can be accessed in deeply exclusive lepto-or photo-production of mesons or photons. PARTONS provides a necessary bridge between models of Generalized Parton Distributions and experimental data collected in various exclusive production channels. We outline the specification of the PARTONS framework in terms of practical needs, physical content and numerical capacity. This framework will be useful for physicists -theorists or experimentalists -not only to develop new models, but also to interpret existing measurements and even design new experiments.
However, a careful user should always get a correct result, even without knowing all details of the implementation. This means that all that can be automated has to be automated, and that physical modules should be designed in such a way that an inadequate use is forbidden, or indicated to the user with an explicit warning. Second, with respect to maintenance, we want to be sure that adding new functionalities or new modules will not do any harm to the existing pieces of code. This requires some non-regression tools to guarantee that the version n + 1 has (at least) all the functionalities of the version n. To trace back the results of the code (e. g. to be able to reproduce the results of a fit), it should be possible to save some computing scenarios for a later reference. The maintenance of the PARTONS code on a long-term perspective is one of the key element of its design, aimed at both the robustness and the flexibility.
However, cross sections and asymmetries are very complicated outputs, which integrate a lot of physical hypothesis and mathematical techniques. To properly estimate the importance of a given physical assumption or a numerical routine accuracy, it is necessary to handle intermediate outputs, like GPDs them-selves and CFFs. The modular structure of the PARTONS framework makes it possible. The output of each module is a well-defined object that can be stored in a database, if requested by the user running the code. Requests to the database allow post-processing of the data, either through plots or data tables. Ideally, it should have been possible to run the code through a web interface, in the spirit of Durham service for PDFs  [88, 89]. However, such a solution requires dedicated work to synchronize a database to the web page, to prevent it of any attack, to create a queue system if several users want to perform their computations at the same time and to handle large volumes of data, which can always happen with functions depending on (at least) three double variables x, ξ and t.
In particular, this means that a dedicated engineer has to take part of his time to tackle these problems and maintain all basic features. For now we let users download a client application to run the code on their own machines. We offer two possibilities -one can either download the source code of PARTONS and compile it by oneself, or download a preconfigured appliance of a virtual machine. The first way requires the availability of additional libraries, see Sect. 4, but in particular it allows to have PAR-TONS at computing farms. The second way only requires VirtualBox [90], which is one of the most popular virtualization suites. Our provided virtual machine allows to run PARTONS as it was out-of-the-box, independently of the user's operating system. Finally, let us mention that the field of 3D hadron structure has been witnessing in parallel a similar collaborative effort for the phenomenology of Tranverse Momentum Dependent parton distribution functions (TMDs): the tmdlib  [91, 92]  library offers an interface to various TMD models.
Code architecture The PARTONS framework is written in C++. This choice has been made for performances and to have a homogeneous product in terms of coding and programming languages. In particular, there is no wrapping of other third-party softwares written in a different programming language. The project considers two different communities: the developers, who have to understand the software architecture to use low-level functions, and the users, who can just use high-level functions ignoring the details of implementations. With the progress of automation, the users may run the code without writing a line of C++ code. In the community of developers, a crucial role is played by the software architect, who is responsible for the integration of new modules in the framework. He guarantees the robustness and homogeneity of the code being developed.
We have decided to depend as little as possible on third-party libraries to help its dissemination. Presently, the PARTONS code contains only one residual dependency on the CLN library [93], that is needed for one particular GPD module and can be easily suppressed later. Only the dependence on the cross-platform application framework Qt [94] is essential, because it manages the connections to different types of databases in a generic way (see Sect. 4. 5). The SFML library [95] is also needed to handle threads (see Sect. 4. 6). Information about the licenses is given in Sect. 7. From the software engineering point of view, the PAR-TONS project benefits from a layered and service-oriented architecture, which provides both the flexibility and the standardization. To the best of our knowledge, this architecture is original in the world of scientific computing, at least in nuclear and particle physics.
Layers Ideally, the code should not have to go through a major rewriting during the years dedicated to the analysis of Jefferson Lab and COMPASS exclusive data. One way to ensure this is to isolate potential modifications as well as possible. This is the reason for the layered architecture: every part of the architecture belongs to a layer, and a modification in one layer does not hinder other layers. The layered structure of the PARTONS software is shown in Fig. 4  and is made of seven parts. The Module layer is a collection of single encapsulated developments of various types. This layer contains the physics engine, like GPD models, but also computations of coefficient functions and cross sections with various physics assumptions. A module is fed by data and it produces results, which corresponds to the Data and Result layers, respectively.
In these two layers no treatment is made on data. These are just collections of containers (high-level objects) that make sure, for example, that each module receives an object that has been well-formed thanks to its constructor. For instance, instead of feeding a GPD module with 5 double variables (x, ξ , t, μ 2 F and μ 2 R ), which can be sent in an incorrect order after some minor editing of the code, we isolate all places where such kind of errors may happen. We trust the fact that the high-level object (here GPDKinematic) has been correctly constructed from those five double variables. The risk of an accidental manipulation (e. g. an exchange of μ 2 F and t) becomes much more limited. To illustrate this, we provide the code defining the GPDKinematic class: The class contains double variables used to store the value of x, ξ , t, μ 2 F and μ 2 R , and three methods to: -create a new GPDKinematic object from a set of five double variables, -create it from a generic list of parameters encoded in the ParameterList container (used by the automation), -return a std::string variable containing an alphanumeric representation of the object to be used e.
At last, three extra layers provide useful functionalities to the users. The Database layer contains tools to store results in local or remote databases. It is designed to optimize later requests and post-processing treatments, and to limit data redundancies in the used databases. The Automation layer is a collection of tools designed for the purpose of automation. A scenario, i. e. XML file containing all physical and mathematical assumptions on the computation to be performed, is parsed. With the XML file parsed, all relevant objects are created and evaluations are processed. The results are shown at the standard output and/or they can be stored in a database, including the associated scenario, either to trace back all hypothesis underlying the results, or to be able to evaluate them again later, e.
Modules The flexibility of the architecture is achieved through the class inheritance. The logical sequence of the code as a whole is centralized in classes that receive standardized inputs and return standardized outputs. All details of model descriptions, numerical precisions, etc. , are exclusively left to the child classes. An example is provided in Fig. 5 , which describes the actual implementation of GPD modules. The input is a GPDKinematic object described above. The output is an object called GPDResult, which contains GPDs provided Fig. 5  Modularity through class inheritance and standardized inputs and outputs by the considered model, with separate values for gluons and all available quark flavors, including singlet and non-singlet combinations. It also contains GPD kinematics and an identifier of the used GPD model, to trace back all conditions of the evaluation.
Finally, GPDResult also contains functions to filter the data (e. g. depending on the parton type) or to print the results. GPDModule is a collection of methods to compute various GPDs (e. g. H or E). There is no upper or lower limits on the number of GPD types that GPDModule should contain. A crucial part of the implementation of GPDModule class is shown here: // Computes GPDs with input parameters virtual PartonDistribution compute(double x, double xi, double t, double MuF2, double MuR2, GPDType::Type gpdType, bool evolution = true); // This method can be implemented in the child class to make the GPD H available for computations virtual PartonDistribution computeH(); Here, the variable gpdType selects the type of GPD to be computed, i. e. H , E, …, or all GPDs available in the considered model.
The addition of a new GPD is fairly simple. It suffices to inherit from a class (GPDModule or even its children), and to implement only the appropriate "compute" functions, e. g. computeHt for the GPD H. Such a new child class contains all specific implementation corresponding to e. g. the GK  [97] [98] [99]  or VGG  [7, 66, 100, 101]  models. Any model obeying this general structure can enter the PAR-TONS framework and benefit from all the other features. The example discussed above for GPD models can be extended to any other types of modules, such as QCD evolution modules, DVCS observable modules, etc.
Registry Adding a new child class does not require any modification of the existing code as long as this class inherits from an existing module. In particular, we can freely add as many GPD models as we want. On the contrary, if we wish to extend the functionalities of the PARTONS framework to the computation of e. g. TMDs, similarly to the tmdlib project  [91] , we will have to create all parent classes to define what TMDs are. Adding a new module simply consists in adding a new file to the whole project. The interoperability of the PARTONS structure is thus maintained all way long. This essential feature is provided by the Registry. The Registry is the analog of a phone book, which lists all available modules. From the software engineering point of view, the Registry corresponds to the singleton design pattern, which ensures that it may exist in the memory only as a unique object.
Factory The Registry lists everything that is available in the platform, but only one species of each. If one wants to use a module, one cannot take it from the Registry, otherwise it would not be available anymore. The solution consists in using the Factory design pattern, which gives to the user a pre-configured copy of an object stored in the Registry. The user can then manage the configuration of the module and its life cycle. The principle of the Factory is the following. We consider once again the example of GPDGK11. By construction, GPDGK11 is derived from GPDModule, which itself is derived from BaseObject to be stored generically in the Registry. As shown in Fig. 7 , when a user wants to use the GPD model identified by GPDGK11::classId, he asks ModuleObjectFactory to return him a pointer of GPDModule type.
ModuleObjectFactory asks BaseObjectFactory to provide a new instance of GPDModule identified by GPDGK11::classId. To this aim, BaseObjectFactory requests that the Registry gives back the reference to GPDGK11 already stored in the memory. The Registry goes through its internal list to find BaseObject with identifier GPDGK11::classId. Using the found reference, BaseObjectFactory clones 1 1 It is not a copy of a pointer, which would still points to the same object. It is a duplication of the object, referred to by a new pointer. the GPDGK11 object and provides Module Object Factory with a reference to the duplicated object. Finally, ModuleObjectFactory casts the pointer to this new object to the appropriate type GPDModule. What is needed to fit to the structure of the code is GPDModule (GPD models are all objects of the same type when seen from the exterior of a black box).
The specific implementation i. e. what defines a single model from the physics point of view (and what is in the black box from the software point of view) is in a child class, like GPDGK11. Through pointers and inheritance, the polymorphism feature of C++ allows the selection of a given module at the runtime. This is the basic sequence underlying the automation of the PARTONS code, discussed below in Sect. 4. 4. This works mutatis mutandis for all modules.
Services The Services hide the complexity of low-level functions to provide high-level features to the users. A single service is basically a toolbox for the user: the user is given tools to use the software without knowing details of its operating. 2 he Services demonstrate their relevance in computations that combine several different objects. Before the inclusion of our GPD codes in the PARTONS framework, we had to take the outputs from various objects, like e. g. some GPD values, to manually run an evolution code and then feed the code computing CFFs. These operations were hand-made, with all the risks this implies. In the PARTONS structure, the Services combine different modules and data sets to produce results in a transparent way. Among others, GPDService -The function computeTask is generic and is one of the building blocks of the automation in PARTONS.
-computeGPDModel evaluates all or only restricted types of GPDs for a single model and for a single kinematic configuration. -computeManyKinematicOneModel evaluates all or only restricted types of GPDs for a single model and for a list of kinematic configurations. If needed, the obtained results can be stored in a database. The insertion is identified by a unique computation id returned to the standard output. The kinematic configurations can be bunched into a set of packets, with the size of each packet defined via the configuration file of PARTONS. In such a case, each packet can be evaluated in a separate thread, see Sect. 4. 6. Thanks to the services, repetitive tasks are coded and validated only once, which saves many possible implementation errors.
Automation and scenario manager What the end-user really wants is just selecting the various models, kinematic configurations and observables to compute by specifying the necessary physical hypothesis. At the end of the day, this should be accomplished with a simple file, or through a web page. This would allow the PARTONS software to be used by physicists unfamiliar to C++, which represents a significant part of the theoretical physics community. We designed a functionality to run the code by sending the appropriate information, referred to as the scenario, through an XML file. This offers several advantages. First, the file can be read or manually written by a simple adaptation. Second, it can be easily generated by a web or graphical interface (such as the planned visualization tool). Third, the freedom in defining markups allows a structure very similar to that of the underlying C++ objects.
For example, the computation of beam-spin asymmetry A - LU defined in Sect. 2. 2. 3 reads: 1 <?xml version="1. 0" encoding="UTF-8" standalone="yes" ?> 2 3 <!--Scenario starts here --> 4 <!--For your convenience and for the bookkeeping you can provide creation date and a unique description --> 5 <scenario date="2017-07-18" description="DVCS observable evaluation for single kinematics example"> 6 7 <!--First task: evaluate DVCS observable for a single kinematics --> 8 <!--Indicate service and its methods to be used and indicate if the result should be stored in the database --> 9 <task service="ObservableService" method="computeObservable" storeInDB="0">  This XML file is parsed by an object named Scenario Manager, which now possesses a collection of string objects. The real difficulty is the creation of C++ objects from this collection of string objects.
The kinematic configuration at which the observable is evaluated has been converted and transmitted from top to bottom. The other way around returns sequentially the evaluation of the GPDs, of the CFFs, of the cross section and of the considered asymmetry. The final result can be stored in the used database (storeInDB switch). The automation file is totally generic: it is independent of the different modules or services. We can use a generic parser and a generic description of XML files. It is one more answer to our need of flexibility. The architecture of PARTONS can evolve without any modifications in the parser or in the XML file description.
Database: storage and transactions A database is needed for several reasons. We want to keep track of the results of our computations, and once a result is validated to keep it and not compute it again anymore. With a related database entry containing the XML file producing the result, it becomes easy to see how something was computed, even if we ask ourselves a long time after. It may also well be that the computational cost of some GPD model is prohibitive, in which case the predictions of this model can be computed separately on a dedicated cluster, stored in the database and then used for the computation of an observable. The structure of GPDModule is designed so as to make transparent to the user the fact that the GPD values come from a database instead of a direct numerical evaluation.
At last, we can also store experimental results in the database, and make systematic comparisons of experimental results and theoretical predictions. A database is optimized to make Fig. 8  Principle of automation: computation of a beam spin asymmetry data selections, in which case kinematic cuts (for example a cut on -t/Q 2 to probe the Bjorken limit) are simple and efficient. In the PARTONS architecture, one layer is dedicated to the transactions with databases, cf. Sect. 4. 1 and Fig. 4. At this level, we should ignore what is the explicit type of database (e. g. a local database on a laptop, or a database on a distant server). The PARTONS code manages the transactions by using Data Access Objects (DAOs) and the related services. For the sake of simplicity, we discuss the case of a single computation of GPD value.
We store both the GPD value and the associated kinematics. Thus, there are two DAO services involved, GPDKinematicDaoService and GPD ResultDaoService, which transform the corresponding C++ objects into a collection of simple types (int, double, string, etc. ) -it is the serialization step. The DAO services obey the same pattern as the other services in PARTONS: they receive as inputs and return as results high-level objects instead of simple types. The DAO services then call the necessary DAO objects. In our example, there are two of them: GPDKinematic Dao and GPDResultDao. In a DAO object we can define as many functions as there are requests to make to the database, e. g. in the case of GPDKinematicDao: insert (to add a kinematic configuration in the database), select (to read a kinematic configuration in the database), delete (to suppress a kinematic configuration in the database), etc.
Any type of requests can be implemented that way. The following excerpt shows the code underlying the insertion in the database, by the GPDKinematicDao object, of a single GPD kinematic configuration: PARTONS generates a SQL-like request, which is a simple string. This text is interpreted by Qt to replace the dynamical fields (here x, ξ , t, μ 2  F and μ 2 R ) by their actual values (here double variables). The Qt management of connectors  4  makes possible to send the same SQL request to databases of different types, like MySQL [102] and SQLite [103]. The connection of the PARTONS objects, specific to our needs, to the simple types in the database, is done once, and only once, whatever the type of the database is. The GPDKinematicDaoService performs the same tasks, but this time with a single or a list of GPDKinematic objects.
By default, PARTONS uses a transaction mechanism, which allows to "rollback" all modifications done during a single insertion session. This prevents in particular from a disintegration of the database content in a case of failed transaction. The way it is achieved is illustrated by the following code: QSqlDatabase::database(). commit(); 4 A connector is the library provided by the editors of the database which permits transactions with a database. This library is written in different languages, e. g. C++, Java, Python, … Note, that there is as many DAO classes as there are tables in the database. In that respect, the database structure reflects the architecture of the C++ code.
Threads Threads are sequences of instructions that can be independently managed by the user's operating system. On multiprocessor/multicore computers (basically all modern machines), threads can be executed simultaneously, in contrary to an execution of only one process at a time. This allows to exploit the full capacity of current computers (including those available at computing farms), which allows a significant reduction of the computation time. In PARTONS, threads are used exclusively by Services, but one thread is also reserved for the Logger, which processes human-understandable messages streamed from the code during its execution. The threads are managed by using the SFML library. In particular, this library is used to protect sensitive areas of the allocated memory from being modified by several threads at the same time.
Threads and services The PARTONS framework offers a possibility to use threads in Services, whenever one needs to make an evaluation for a large set of kinematic configurations. The mechanism is fully automated, so the use of threads is implicit for the users. We will illustrate how threads are used by the Services on an exemplary calculation of many GPD kinematic configurations. From the user's point of view, whenever one wants to use threads, one needs only to run the appropriate method of GPDService, that is computeManyKinematicOneModel that has been already described in Sect. 4. 3: In our example, gpdKinematicList contains many kinematic configurations to be evaluated by PARTONS. Note, that computeManyKinematicOneModel can be executed by the users explicitly if they code in C++, or implicitly in the automation if they process an input XML file.