dataesr/scientific-paragraphs-categorization huggingface dataset:
B. Translation Model PASCANI translates test module specifications into SCA component configurations and corresponding programming language implementations (i. e. , Java for this version of the framework). Furthermore, PASCANI provides an interface to deploy and execute the generated components using the FRASCATI middleware  [2]. From the information specified in the three blocks of a test module, PASCANI generates the corresponding SCA configuration as follows (cf. Fig. 2 ). The services of components specified in the system structure block are translated into SCA references that are bound to testcase and control components in order to be tested (cf. service mcm in component Matrix is bound to StrassenTest_Group. StrassenTest_checkCorre ctness_strassen_mcm); each of the test cases and the control block are translated into components (cf.
StrassenTest_checkCorrectness and StrassenTest_Control) and encapsulated together in the same composite (cf. StrassenTest_Group). Every generated component is equipped with two standard services: run, that is used for starting the execution of the component, and control, for controlling the logging functionalities (e. g. , the level of detail of the messages to log). In our example, component StrassenTest_Control has references to all of the testcase components within StrassenTest_Group, given that it must realize the test strategy as well as the control interface for the logging and traceability of test cases execution. Finally, component Logger records log messages, and component Runner allows the tester user to start the execution of the test module and control the logging level of detail through services Runner.
C. Execution Model The execution model of PASCANI relies on the standard SCA interfaces, services, and references that are defined for each component in the translation model. The execution model defines the way a test strategy is executed through the behavior of the generated components. The sequence diagram depicted in Fig. 3  represents the execution model of the StrassenTest module specified in Listing 1. The diagram is adapted to the SCA domain. That is, participants represent the components of the SCA configuration for the StrassenTest module, and messages correspond to the services that allow the communication among them. For simplicity, this sequence diagram focuses on the interactions among the components that realize the test strategy, and omits the ones associated with logging and control mechanisms.
The steps of the execution model for testing the classical implementation of Strassen's algorithm that multiplies two matrices (cf. component Matrix in Fig. 2 ) is as follows. A TestClient starts the execution of the test module by invoking service Runner. run, which in turn invokes service StrassenTest_Control. run to initiate the execution of the control, that is the test strategy. The control starts the execution of each test case, as specified in the control block of the test module, by calling the corresponding run methods. Afterwards, each test case component invokes the service or services to be tested. In the case of the StrassenTest module, the test strategy is composed of only one test case that is implemented in component StrassenTest_checkCorrectness. To perform the actual test, StrassenTest_checkCorrectness consumes service Matrix.
mcm, gets the result of the execution, and validates this result with respect to the specification to return the verdict of the test to the control block. Finally, the test ends by generating the test report. The execution model of PASCANI also comprises the monitoring of properties and behaviors of the testing components. PASCANI components expose services to allow the control of their logging mechanism by other SCA components, and through them by users. Furthermore, test components can be monitored by reading the generated log, which corresponds to the traceability of the test strategy execution. By default, PASCANI generates log registries at the most detailed level, that is for the test case execution. However, users can turn off the logging functionalities for all the components in a test module or for selected components of any type (i.
IV. COMPOSING TESTS WITH PASCANI From the testing perspective, the evaluation of a software application's quality depends not only on the results obtained from the execution of test cases, but also on the quality of the tests themselves  [11]. On the one hand, several testing techniques exist for evaluating the fulfillment of functional and extra-functional aspects of quality, such as regression testing and performance testing, respectively. On the other hand, the quality of a set of test modules is usually evaluated against selected test criteria, for instance by measuring the coverage (i. e. , percentage) of code that is executed when the test modules are applied. Even though the test module is the basic testing unit of reuse in PASCANI, the key factor for achieving composability and traceability of test specifications resides in its threeblocks structure.
More concretely, the reusability of PASCANI testing modules is achieved by three main characteristics: (i) integration of programming language statements within the control block test definition; (ii) modularity of tests; and (iii) composability of deployable testing components and the exposure of services for controlling test traceability and logging. This section illustrates the principles used in the design of the PASCANI language for defining the composition operators used to realize both testing strategies and tests quality evaluation. These operators compose testing components not only generated by PASCANI, but also other testing components from third parties, as long as they either conform to the SCA specification, or are exposed as testing services. Moreover, to support the composition of tests results, PASCANI defines the operators and, or, xor, and not, associated with its verdict primitive type.
C. Test Coverage The third form of composition supported by PASCANI targets test coverage. As mentioned previously, test coverage is used in classical testing for evaluating the thoroughness of a set of test artifacts, measured as the percentage of code whose execution is run-through by the test suites execution. More concretely, test coverage uses a control flow-based coverage criteria whose goal is to execute every statement in a program, or specified blocks of it. It is worth mentioning that several approaches for automating the generation of test data-sets have been proposed to achieve an exhaustive coverage of all control flow paths (e. g. ,  [12] ,  [13] ) even though it is known that this is unfeasible in general. In any case, automated generation of these data sets is outside the scope of this paper, as it addresses another dimensions of the testing problem.
In PASCANI, test coverage is evaluated as the thoroughness of a set of test modules, measured as the percentage of the services that are actually tested, with respect to the total number of provided services in all components subject to test (i. e. , imported in the system structure block of the PASCANI test module specification). The test modules are executed in sequence, avoiding repetitions in them and in service execution. The composition of several test modules T M 1 ,. , T M N for measuring their coverage is realized through the definition of the operator, denoted as ⊙, and expressed in the language as coverage{ T M 1 ,. , T M N }. where {s | ∀c in P ascani. T M i. Components(s in c. provided Services)} is the defined-by-comprehension set of the provided services s that are defined in all of the components imported in test module TM i ; and X is the cardinality of the set X ( count(testedServices) count(totalServices) * 100%).
That is, as with the performance-testing composition operator, PASCANI includes a record of the executed services in the attributes associated to every test module TM i , which is updated dynamically at runtime as the services involved in the test modules are executed. In this definition, Pascani. TM i. Components refers to all of the imported components in the system structure block of TM i , whereas Pascani. Services. markedAsExecuted refers to the services actually executed in the given execution. Thus, the semantics of the test-coverage composition operator not only computes the ratio between the number of actually tested services and the total number of them (expressed as a fraction of 100%), but also activates the recording of the executed services in the generated configuration. At runtime, the tests execution returns the percentage of test coverage, and as a side-effect, produces the record of the services that were actually tested (in cascade) from these tests execution.
Matrix-chain Multiplicator Matrixpair The fundamental motivation behind PASCANI is the composition and reuse of test modules, which is achieved through the specification of unit, integration, and system test modules. PASCANI allows the specification of integration and system tests by invoking the test cases of imported modules, and the composition of test modules through the regression (⊎), performance (⋓), and coverage (⊙) operators presented in Section IV. In light of this, each component of the MCM solution is equipped with one or several basic test modules that address different test objectives. We implemented basic test modules for testing the correctness and performance of several of the SCA components used in the MCM problem of our case study. For example, component Strassen is accompanied by a basic test module T M StrassenT est that tests its correctness.
The target of these two modules is unit testing, in its functional and extra-functional aspects, respectively. In this case study, PASCANI supports the service provider in the implementation and execution of the integration and system tests required to validate the correctness and performance of each of the fifteen MCM products that can be obtained from the variability model presented in Fig. 4. The explanations in this section use two of these possible products: M CM 1 and M CM 2. Assume that product M CM 1 is defined by component MatrixpairMultiplicator. Strassen (also referred as Strassen for the sake of simplicity), whereas product M CM 2 by components Strassen, Parenthesizer. Sequential and Scheduler. Sequential. The tester must specify test modules that call the test control blocks or selected test cases of the modules associated with the SCA components that define each product configuration.
For example, for testing the correctness of product M CM 2 , the tester must specify an integration module T M IntegrationT est whose execution produces a verdict on the integration. This verdict results from the and operation applied to the values returned by the basic test modules in charge of testing the individual correctness of its three components. 1) Regression Testing: Suppose now that the implementation of component Strassen is modified. Because of this change, which does not affect the component service interfaces, the tester must implement a regression test to guarantee that product M CM 2 still satisfies its functional or extra-functional requirements. For this, the tester must create a composed test module T M RegressionT est that specifies in its control block a regression test that composes the unit test of component Strassen with the integration test of product M CM 2 using the regression operator: T M StrassenT est ⊎ T M IntegrationT est.
That is, if the Strassen component implementation is changed (because e. g. of a new version or an evolutionary change of the algorithm base), it is necessary to retest that both the unit tests of the component correctness and its integration test with the Parenthesizer and Scheduler components are still successfully passed. Fig. 5    A regression test report in PASCANI typically includes, for each test module, its total execution time; a flag to indicate whether the module was executed in regression test mode; and for each of the services tested by its test components, a general flag to indicate whether the service was tested, and the execution time and verdict for each test case associated with the tested service. 2) Performance Testing: To test the performance of product M CM 2 the tester must compose the primitive modules that test the performance of components Strassen, Parenthesizer.
Sequential, and Scheduler. Sequential: T M P Strassen ⋓ T M P P arSeq ⋓ T M P SchedSeq ≤ T arget maxLimit. From this composed module, PASCANI generates the corresponding SCA configuration. The performance reports, not shown in this paper due to space constraints, contains the general verdict and performance measures obtained for product M CM 2 , as well as the verdicts and measures for each individual component. This level of detail in the logging of test results is possible thanks to the composition semantics of the performance tests operator, and the tracing and logging facilities of PASCANI. Table  I  shows the outputs obtained when using the performance composition operator to test products M CM 1 and M CM 2 for the multiplication of matrices of sizes 640 × 480, 480 × 320, 320 × 240, and 240 × 640 among others.
B. Engineering Effort Table  II  presents the relative gain of developer's effort when specifying tests with PASCANI in our case study. The results show an effort gain superior to 70% for all cases. This is mainly because PASCANI releases the developer from implementing functionalities of the SCA layer, such as protocol translation and message marshaling and unmarshaling. In this table, column Component refers to the service components we tested. For evaluation purposes, each test case was defined in a separate test module (cf. column Test Module) thus leaving the measurement at the coarsest level of granularity. Column PASCANI LOCs refers to the number of lines of code written in PASCANI by the developer. Column Generated LOCs includes two sub-columns: Project, which refers to the classes that are automatically generated in correspondence with the test modules; XML, which refers to the generated SCA composites.
Even though we found no evidence of research works focused on the traceability of tests at every level, in general logging in service testing is better instrumented for unit tests than in integration or system tests. Nevertheless, keeping track of external behaviours is still an open challenge for distributed environments. PASCANI not only provides logging mechanisms applicable to individual tests and their compositions, but also these mechanisms are automatically deployable since they are generated as part of the test modules. PASCANI also allows software testers to select the types of traces applicable in a particular situation, or decide on the level of logging (e. g. , by enabling logging functionalities for a selected set of test cases). Finally, with respect to test specification and expressiveness, we found only one domain specific language (DSL), based on ontologies  [19] , for the specification of tests in service-oriented applications.
However, we found no DSLs suitable for the specification of reusable, composable, and traceable service-oriented tests. In general, the focus is on generating test components or test logic from service specifications, in particular WSDLs, rather than on the specification of tests themselves  [17] ,  [20]. PASCANI defines composition operators whose semantics guarantee that the generated testing components automatically have these characteristics. PASCANI is a suitable and easy to use language intended for developers to specify composable testing artifacts for SOA/SCA applications. In this category, it is worth mentioning that most of the PASCANI language testing concepts were borrowed from TTCN-3 (cf. http://www. ttcn-3. org). TTCN-3 is a test specification and implementation language designed by the European Telecommunications Standards Institute (ETSI), for the specification of reactive system tests over a variety of communication interfaces.
Pairwise comparison of data vectors represents a large part of computational biology, especially with the continuous increase in genome-wide approaches yielding more information from more biological samples simultaneously. Gene clustering for function prediction as well as analyses of signalling pathways and the time-dependent dynamics of a system are common biological approaches that often rely on large dataset comparison. Different metrics can be used to evaluate the similarity between entities to be compared, such as correlation coefficients and distances. While the latter offers a more flexible way of measuring potential biological relationships between datasets, the significance of any given distance is highly dependent on the dataset and cannot be easily determined. Monte Carlo methods are robust approaches for evaluating the significance of distance values by multiple random permutations of the dataset followed by distance calculation.
INTRODUCTION To compare large data vectors, such as gene expression levels across multiple conditions, one needs to evaluate how similar they are. Various metrics are more or less commonly used that can be classified into two categories: correlation coefficients (e. g. Pearson, Spearman) and distances (e. g. Euclidean, Minkowski). As they do not yield identical results, they are intended to address different biological questions, and the appropriate metric to be used is highly dependent on both the working hypothesis and the original dataset  (1, 2). For instance, the Pearson correlation coefficient is the method of choice when dealing with absolute values (3), while Euclidean distance is described to perform better for the comparison of log2 ratio datasets  (3). The fact that correlation coefficients are not sensitive to scaling is also a critical aspect when selecting a comparison method.
Furthermore, distances allow for relative flexibility in the way the data are compared. Euclidean distance is more generally employed, but the use of the Minkowski distance with high power values, for example, has been mentioned to work better when analyzing high-dimensional datasets  (4). This is due to the 'curse of dimensionality'  (5) , which describes the loss of contrast between entities of high dimension that is caused by a narrowing of the distribution of distances relative to their average (in other words, the variance remains constant while the mean increases proportionally to the square [for the Euclidean distance] of the dimension)  (6). This phenomenon might be partially counteracted by increasing the power value of the Minkowski distance, which therefore appears to be a relevant approach for many biological, genome-scale datasets  (4).
Therefore, theoretical assumptions for parametric tests are rarely fulfilled, and it is theoretically invalid to use the classical t-or F-tests to analyze the experimental results. With the continuous increase in computing power, stochastic approaches, known to quickly become computerintensive, have emerged as robust and powerful ways of extracting meaningful information from random noise out of complex datasets  (7) , including distance matrices. Randomization tests belong to the broad class of Monte Carlo methods that rely on repeated random sampling to obtain numerical results. They are often used in computational biology  (8)  and physics  (9)  to address mathematical problems where it is difficult or impossible to obtain a closed-form expression or where a deterministic algorithm cannot be applied.
Density-based P-value computation. Direct estimation of the raw P-value possesses one major drawback. If the real (unknown) P-value is too small, as in a biological sequence randomization test  (16)  where the P-value can often range from 0. 1 to the order of magnitude of 10 -30 , the number of randomizations can be too small to estimate the raw Pvalue  (17). A classical method to avoid this drawback is to first directly estimate a density distribution function ρ(s) = d F d S (s) from the randomized statistic s * i by a kernel density estimation  (18, 19)  and then to estimate the P-value from numerical integration of ρ(s)  (20, 21) d( p, q) = n i =1 | p i -q i | Euclidean (Minkowski, P = 2) d( p, q) = n i =1 ( p i -q i ) 2 Although the Euclidean distance remains largely preferred in gene expression dataset analysis, the use of higher power values is becoming more frequent as a way to overcome the 'curse of dimensionality'.
On the other hand, lower power values (Manhattan distance or fractional values) allow for less sensitivity to outliers, and should be preferred in cases where they might interfere with the results. (4,5) Minkowski d( p, q) = k n i =1 | p i -q i | k Canberra d( p, q) = n i =1 | p i -q i | | p i |+|q i | The Canberra distance has been successfully used as a measure of stability for biological indicators identified from microarray data. (  11 ) Chebyshev (Minkowski, P = ∞) d( p, q) = max i (| p i -q i |) Seldom used in biology, the Chebyshev distance can be utilized in gene set analysis and gene selection, in particular when UPGMA approaches are conducted. ( ) 12 normalized datasets. On-the-fly computation of the appropriate density bandwidth for each distribution using Scott's rule of thumb  (22)  works effectively in most cases ('Nrd' bandwidth setting from the analysis options panel).
However, as it relies in part on the standard deviation of the distribution, very narrowly distributed distances after randomization (Figure  1A ) appear to yield extremely low bandwidth, resulting in an 'over-reactive' density kernel estimator (Figure  1B ) and therefore incorrect calculated P-values. To address this issue and prevent misinterpretation of the results, the R script implements a method for detecting such behavior and optimizing the density bandwidth (Figure  1C ) by iterative adjustment until the standard deviation of the density falls below 1 ('Optimized' bandwidth setting from the analysis options panel).
Clustering A very common step in a gene expression analysis workflow is the clustering of various genes or experimental conditions that are compared for function prediction  (23)  or identification of signalling cross-talk  (21). While this server does not implement all possible algorithms for the numerous clustering methods, the script includes a hierarchical clustering step using P-value-based single, average and complete linkage followed by tree cutting at the following heights: h = 0. 05, 0. 01, 10 -3 , 10 -4 , 10 -5 , 10 -6. The resulting clusters can be mapped to the output weighted matrix and network view by entity reordering and color-coding, respectively.
P-value-distance function The adjacency function that maps the interval of distances [0,∞] into [0,1] can be computed if the user has selected data normalization at the time the job is launched. The dataset then becomes usable in analysis frameworks for weighted networks assuming pairwise edges of values constrained to the interval [0,1] (24). The relationship P-value versus distance is therefore inferred by a logistic regression. Let R(x,y) be a possible relationship between two vectors x and y (genes, biological samples, etc. ). Then we can have R(x, y) = 1 if x and y are in relation (respectively, R(x, y) = 0 if x and y are not in relation). Let D(x, y) be a predictive variable (for instance, a distance computed between x and y). We have the a priori probability P(R(x, y) = 1) (respectively, P(R(x, y) = 0))) to take the value 1 (resp.