dataesr/scientific-paragraphs-categorization huggingface dataset:
Compared to the direct computation of the generalized random walk kernel, this method does not directly compute weight matrices of direct product graphs. Benefiting from the Kronecker product, only (normalized) adjacency matrices of original graphs are required, which have size of n 2 and can be pre-computed for each graph. Besides, computing q ⊤ × vec(M) requires O(N 2 n 2 ) time. Thus for N unlabeled graphs, the complexity of computing the corresponding Gram matrix is O(N 2 n 3 ), which is reduced compared to the direct computation. However, it has a strong drawback: libraries currently available to solve the generalized Sylvester equation, such as dlyap solver in MATLAB and control. dlyap function from Python Control Systems Library  (Murray et al. , 2018) , can only take d as 1, which means the solver is limited to edge-unlabeled graphs.
Experiments In this section, we first introduce several benchmark datasets corresponding to different types of graphs. These types include labeled and unlabeled graphs, with symbolic and non-symbolic attributes, different average vertex numbers, different average vertex degrees, linear, non-linear and cyclic patterns. Then we introduce methods to accelerate computation of kernels, and propose an analysis on applying these methods by choosing proper parameters. Finally, we perform each graph kernel on each dataset and analyze the performance and computational complexity according to the types of graphs, offer advice to choose graph kernels based on the type of datasets and discuss which ones work on particular graphs.
Real-world datasets We examine 7 well-known benchmark datasets, where the first two are for the regression task of estimating the boiling points of molecules and the others are classification tasks. "Substructures" are the sub-patterns that graphs contain; "Numbers of labels" include numbers of symbolic and non-symbolic vertex and edge labels, with ✗ for no label; "Directed" exhibits whether directed graphs are included; N is the number of graphs; n is the average number of graph vertices; m is the average number of edges; d is the average vertex degree; Tasks are either regression ("R") or classification ("C"). Alkane is a dataset of 150 alkanes with only carbon atoms, which are represented by acyclic unlabeled graphs  (Cherqaoui and Villemin, 1994). Acyclic has 183 acyclic molecules with hetero atoms  (Cherqaoui et al.
, 1994). MAO is a Monoamine Oxidase dataset of 68 cycle-included molecules labeled by two classes: 38 of them act as antidepressant drugs by inhibiting the monoamine oxidase and 30 do not  (Brun, 2018). PAH is a polycyclic aromatic hydrocarbon dataset composed of 94 cyclic unlabeled graphs  (Brun, 2018). Atoms are all carbons and chemical bounds are all aromatics. The associated target is to determine whether each molecule is cancerous or not. Mutag is composed of 188 mutagenic aromatic and heteroaromatic nitro compounds  (Debnath et al. , 1991). The classification task associated is to correctly determine whether each compound has a mutagenic effect. Letter-med involves graphs of 2250 distorted letter drawings of 15 capital letters of the Roman alphabet  (Riesen and Bunke, 2008). The "med" suffix indicates that the distortion is of medium strength.
The task is to classify each graph to the proper letter. Enzymes has 600 enzymes from the Brenda enzyme database  (Schomburg et al. , 2004) , which represents tertiary structures of protein  (Borgwardt et al. , 2005). The task is to predict the correct Enzyme Commission top-level class from all six classes. These datasets are chosen as they come from different fields, such as bioinformatics and handwriting recognition, and have different properties that allow to provide an extensive analysis of graph kernels. Table  2  outlines the properties of these datasets. shortest path adjacency matrix. Finally when comparing shortest paths, it retrieves comparison results of corresponding vertices from the matrix. This method reduces vertex comparison to at most n 1 n 2 times, which is only 1/(2n 1 n 2 ) of the direct implementation.
Parallelization Parallelization may significantly reduce the computational complexity. The basic concept of parallelization is to split a set of computation tasks into several pieces, and then carry them out separately on multiple computation units such as CPUs or GPUs. We implement parallelization with Python's multiprocessing. Pool module on all the graph kernels in two ways: To compute Gram matrices of graph kernels, parallelization is performed on pairs of graphs; during cross validation procedure, parallelization is carried out on the set of trials. Many factors may influence the efficiency of parallelization, such as the number of computing cores, the transmission bandwidth between these cores, the method to split the data, the computational complexity to tackle one piece of data, etc.
laptop and on CRIANN. The values of this ratio for large-scale datasets are around 3. 5, which turns out to be the inverse ratio of number of CPUs of the two machines (8 versus 28). It is worth noting that the Letter-med dataset has the largest number of graphs (but with the relatively "small" graphs), and Enzymes has the "average-largest" graphs (the second one being the PAH dataset). Ideally, parallelization is more efficient when more computing cores are applied. However this efficiency can be suppressed by the parallel procedure required to distribute data to computing cores and collect returned results. Paralleling relatively small graphs to a large number of computing cores can be more time consuming than nonparallel computation. For instance, Figure  7  shows that it takes almost the same time to compute the Gram matrix of the small dataset Acyclic on both machines, indicating that the time efficiency raised by applying more CPU cores on machine CRIANN is nearly neutralized by the cost to allocate these cores.
Choosing appropriate chunksize is an essential way to tackle this problem. Chunksize describes how many data are grouped together as one piece to be distributed to a single computing core. In Figure  8 , runtimes to compute Gram matrices of the shortest path kernel with different chunksize settings are compared, on 28 and 14 CPU cores. With both CPU core numbers, for almost all datasets, when chunksizes are too small, the computing time becomes slightly high, as the parallel procedure costs too much time; as chunksizes.
Performance analysis In this section, we compare and analyse performance of the kernels for all datasets. Table  3  gathers performance of all these kernels on all datasets. The column t gm is the time to compute Gram matrix/matrices in seconds. Note for kernels which need to tune hyper-parameters that are required to compute Gram matrices, multiple Gram matrices will be computed, and average time consumption and its confidence are obtained over the hyper-parameters grids, which are shown after the label "/". The time shown before "/" is the one spent on building the Gram matrix corresponding to the best test performance. Once hyper-parameters are fixed, learning is only performed on a single trix/matrices as well as to perform model selection for each kernel. For regression tasks (Acyclic and Alkane), the performances are given in terms of boiling points errors; for other datasets, the performances exhibit the classification accuracy in percentage.
Labeled and unlabeled graphs To study the influence of labeling on performance of graph kernels, we examine 3 datasets that have similar properties (e. g. n, m and d in Table  2 ), except for labeling: PAH is unlabeled, MAO has 3 symbolic vertex labels and 4 symbolic edge labels, and Mutag has 7 symbolic vertex labels and 11 symbolic edge labels. Figure  9  exhibits the accuracy of each kernel and the average time to compute each kernel between a pair of graphs. We can see that, for all kernels, the classification accuracy on dataset PAH are significantly lower and the confidence intervals around them are wider than the other two datasets, as PAH contains no labeling information. However, there is no significant relationship between the datasets and the runtime of the kernels, which implies that the influence of the graph labeling on runtime is minor on these datasets compared to other factors, such as the average number of graph vertices.
Graphs with symbolic and non-symbolic labels Non-symbolic labels are able to introduce continuous attributes to graphs. From all graph kernels, the shortest path kernel is able to tackle symbolic and non-symbolic vertex labels, whereas the conjugate gradient kernel, the fixed-point kernel and the struc-  tural shortest path kernel can deal with both non-symbolic labels of vertices and edges. From available datasets, Letter-med and Enzymes contain non-symbolic vertex labels. We compute accuracy and time complexity of each kernel on these 2 datasets, then we remove the non-symbolic labels from the datasets and compute the performance. Figure  10  shows that, with non-symbolic labels, classification accuracy of all kernels exceeds 90% on dataset Letter-med, and more than 60% on dataset Enzymes; these accuracies drop to about 35% when non-symbolic labels are removed, which are still better than random assignments because of the large numbers of competing classes (15 and 6, respectively).
It reveals how these graph kernels can take advantage of non-symbolic labels, which carry out essential information of dataset structures. This consequence is corroborated by the results revealed in Table  3  where graph kernels that cannot tackle non-symbolic labels work poorly on Letter-med and Enzymes, such as the common walk kernel and the marginalized kernel. As a result, non-symbolic labels should always be well examined before designing graph kernels. When only linear patterns are included, the shortest path kernel, the conjugate gradient kernel, the fixed-point kernel and the structural shortest path kernel would be the first to consider.
Graphs with different average vertex numbers The average vertex number of a graph dataset largely influences the time complexity of computing graph kernels. To examine it, we choose 3 datasets with relatively   Table  1  displays that the time complexity of only two kernels is directly affected by the average vertex degrees m, namely the structural shortest path kernel and the path kernel up to length h. As a result, Figure  11 (d) shows that the runtime for the path kernel up to length h increases as the growth of the average vertex degree. Runtime for most of the other kernels, however, is high for the first subset (when m = 3. 2), and stays stable after. Other than the average vertex degree of each subset, this runtime is mainly influenced by the average vertex number, which is much bigger for the first subset than the others.
Linear and non-linear graphs Graph kernels based on linear patterns are specific to linear structures in graphsdatasets. In this section, we examine the performance of these graph kernels on nonlinear structures. Datasets MAO and Mutag are employed, since the former contains more linear structures and less cycles, while the latter, composed of aromatic and heteroaromatic compounds, contains more cycles and less linear structures. Besides of that, they share comparable properties (e. g. vertex labels, n, m and d in Table  2 ). From Figure  12 , we can see that graph kernels based on linear patterns allow to explore nonlinear structures, without a significant impact on the accuracy. of all kernels on all classification datasets. Note that for datasets Letter-med and Enzymes, kernels that cannot tackle non-symbolic labels are omitted.
By portraying the confidence ellipses and their centers, we provide general conclusions on these graph kernel. From a global viewpoint, all kernels provide a good accuracy on all datasets. Examining the center of the ellipse of each kernel, we can see that the marginalized kernel has the worse runtime-accuracy, with some regular computational time. The structural shortest path kernel provides the best accuracy in general, the price to pay being its computational complexity. The Sylvester equation kernel, the spectral decom-position kernel and the path kernel up to length h have the good compromise between time complexity and accuracy. Based on this analysis, before choosing a graph kernel, one can have a rough expectation of its performances.
Conclusion In this paper, an extensive analysis of the graph kernels based on linear patterns was performed. Although graph kernels based on linear patterns are designed for linear structures, they were applied with success on datasets containing non-linear structures. We examined the influence of several factors, such as labeling, average vertex numbers and average vertex degrees, on the performance of graph kernels. The computational complexity -a major issue in designing and working with graph kernels -was extensively addressed in this paper. The average vertex numbers and average vertex degrees restrict kernels' scale abilities. Time complexity of all kernels are polynomial to the average vertex numbers, with the common walk kernel being the worst one, and thus it should be avoided for large-scale datasets.
Average vertex degrees had trivial influence on the time complexity, which remained low on all datasets. The application and generalization of some implementation tricks were rather helpful to reduce real-time computational complexity. Fast computation of the shortest path kernel was extended to any graph kernel that requires comparing vertices and/or edges explicitly. Parallelization was implemented for all kernels. Both strategies significantly reduced the computational time when dealing with real-world datasets, especially large ones. We implemented all graph kernels and applied these tricks to them. The code is available on GitHub as an open-source Python library named py-graph 1.
Graphs with different average vertex degreesAs vertex numbers, the vertex degrees play an important role in time complexity of computing graph kernels. Large vertex degrees indicate "dense" graphs where more edges and connections exist, leading to a much larger number of linear patterns inside graphs, such as walks and paths, and more time to explore them. Applying the same method as in Section 4. 4. 3, we choose a dataset with relatively wide range of vertex degrees, Enzymes, order it based on the vertex degree, split it into 5 subsets.
4. 4. 4. 44 Graphs with different average vertex degreesAs vertex numbers, the vertex degrees play an important role in time complexity of computing graph kernels. Large vertex degrees indicate "dense" graphs where more edges and connections exist, leading to a much larger number of linear patterns inside graphs, such as walks and paths, and more time to explore them. Applying the same method as in Section 4. 4. 3, we choose a dataset with relatively wide range of vertex degrees, Enzymes, order it based on the vertex degree, split it into 5 subsets.
https://doi. org/10. 5281/zenodo. 6915823. Raw early results submitted via the Unity Study Collaborators initiative are available on the following Zenodo link: https://zenodo. org/ communities/unity-sero-2021?page=1&size=20. DOIs and citations for the studies in this Zenodo community is included in Table A to C in S1 Materials. The Python code used for our automated estimate prioritization is available on the following GitHub link: https://github. com/serotracker/iitbackend/blob/ 8059e9b905395de997f28a1a2dff5def795276ad/ app/utils/estimate_prioritization/estimate_ prioritization. py.
Author summary Why was this study done? • Serosurveys, or studies capturing information on Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) antibody prevalence, help us understand true rates of infection, vaccination, and indicators of immunity in the population against the virus causing Coronavirus Disease 2019 (COVID-19) and inform public health decisionmaking. • Previous global systematic reviews of seroprevalence have highlighted a lack of standardization in study methods and fewer datasets in some regions like low-and middleincome countries. • Recently, in part via WHO's Unity studies, the quantity and quality of available seroprevalence data has increased, providing the opportunity to understand the true extent of exposure to SARS-CoV-2 and differences by demographic groups, region, and time.
Search strategy and study selection We conducted a systematic review of seroprevalence studies (hereafter "studies") published from January 1, 2020 to May 20, 2022, reported according to the Preferred Reporting Items Systematic review and Meta-Analyses (PRISMA) guideline  [13]  (File A in S1 Materials). We designed a primary search strategy in consultation with a health sciences librarian in MED-LINE, Embase, Web of Science, and Europe PMC using key terms such as SARS-COV-2, COVID-19, seroprevalence, and serology (full strategy and complete list of search terms in File B in S1 Materials). We attempted to mitigate possible publication bias by including both published articles and unpublished literature such as grey literature, preprints, institutional reports, and media reports. For our secondary search and article capture strategy, we invited submissions to our database through the open-access SeroTracker platform and recommendations from international experts, including literature compiled through the WHO Unity studies initiative.
In order to access timely evidence and mitigate challenges with publication delay, we also contacted WHO Unity study collaborators that had not yet made results available to the general public prior to our inclusion dates, to upload their aggregate results to the open access Zenodo research data repository  [14]. We accepted these templates up to 20 May 2022 in line with our primary search strategy and screened them according to the same criteria as other sources captured in our primary search. This systematic review and meta-analysis protocol was registered with PROSPERO (CRD42020183634) prior to the conduct of the review (File C in S1 Materials)  [15] , and searches and extractions conducted per the previously established SeroTracker protocol  [16]. Studies were screened, data extracted, and critically appraised in duplicate, with these tasks shared by a team of 13 study authors (listed in Research Contributions section under data curation).
We have study team members proficient in English, French, Portuguese, Spanish, and Cyrillic languages-articles in all other languages were translated using Google Translate where possible. Conflicts were resolved by consensus. Inclusion and exclusion criteria aligned with the SEROPREV standardized protocol for general population seroprevalence to minimize possible bias introduced by interstudy heterogeneity and other measures of study quality such as poor assay performance and/or sampling methods (full protocol criteria described in File D and E in S1 Materials). We included cross-sectional or longitudinal cohort studies with the objective of estimating SARS-CoV-2 seroprevalence in the general population. Restricting inclusion to direct population samples such as household surveys would have led to very little data in some regions and times, as these studies are expensive and difficult to conduct.
Data extraction, synthesis, and analysis From each study, we extracted seroprevalence estimates for the overall sample and stratified by age, sex, vaccination status, and timing of specimen collection according to the prespecified protocol. We extracted information on study population, laboratory assay used, any corrections made in estimating seroprevalence (e. g. , for population or assay performance), seroprevalence, and denominator. Standardized results uploaded to Zenodo by Unity study collaborators additionally included information on the proportion of asymptomatic seropositive individuals. Our procedure for the standardized, aggregate early data results submitted by Unity collaborators was to direct study authors to input their results into a formulated standard Excel template designed to match the same data extracted during routine published study extraction.
A blank version of the Excel template is available for reference  [23]. These templates were uploaded directly into R for analysis in tandem with other studies included in the meta-analysis. Templates were verified by 2 independent reviewers, and we conducted follow-up to complete information with study investigators where needed. In instances where data from early reporting templates we had received were published prior to May 20, 2022, or a partial dataset was previously published, we ensured to de-duplicate these data for analysis. We evaluated cases of duplicated results on a case-by-case basis, prioritizing the authors published version by default but made exceptions where data were more complete, robust, or up-to-date in the submitted templates. Once authors published or preprinted their results, a link to the full source was added to the Zenodo repository.
We critically appraised all studies using a modified version of the Joanna Briggs Institute (JBI) checklist for prevalence studies (File F in S1 Materials)  [24]. To assess risk of bias, a decision rule assigned a rating of low, moderate, or high risk of bias based on the specific combination of JBI checklist ratings for that study  [25]. This decision rule was developed based on guidance on estimating disease prevalence  [26, 27]  and was validated against overall risk of bias assessments derived manually by 2 independent reviewers for previously collected seroprevalence studies in the SeroTracker database, showing good agreement with manual review (intraclass correlation 0. 77, 95% CI 0. 74 to 0. 80; n = 2,070 studies)  [25]. Early results from templates were screened and evaluated for risk of bias using the same criteria as studies captured through routine screening processes.
We anchored each estimate to the date halfway between sampling start and end ("sampling midpoint date") to best reflect the time period of the study. To select the most representative and high-quality studies for analysis, we used only subnational or national studies rated low or moderate risk of bias to estimate seroprevalence in the general population over time and identify factors associated with seroprevalence (subdataset 1). We used only national studies rated low or moderate risk of bias to estimate case ascertainment (subdataset 2). To explore possible causes of heterogeneity among study results, we constructed a Poisson generalized linear mixed-effects model with log link function using the glmer function from the lme4 package in R  [30] [31] [32]. Independent predictors were defined a priori as WHO region, income group, geographic scope, sample frame, pandemic timing, age, cumulative confirmed cases, and average public health and social measure (PHSM) stringency index  [33].
We produced 95% confidence intervals for the mean seroprevalence estimate, reflecting uncertainty in the summary effect size  [36] , and 95% prediction intervals to give a range for the predicted parameter value in a new study. All numerical results presented are from this stage. To visualize the trend in regional and global estimates over time, we fit a smooth curve to these estimates using nonparametric regression (gam from R package mgcv)  [37]. We also summarized the relevant variant genome frequency in each region shared via the GISAID initiative  [38]. We also estimated to what extent laboratory confirmed SARS-CoV-2 cases  [39]  underestimated the full extent of infections based on seroprevalence. For studies that sampled participants in 2021, we used national seroprevalence estimates and vaccination rates  [40]  to calculate seroprevalence attributable to infection only.
The sensitivity and specificity values for correction were prioritized from the WHO SARS-CoV-2 Test Kit Comparative Study conducted at the NRL Australia  [19] , followed by a multicenter evaluation of 47 commercial SARS-CoV-2 immunoassays by 41 Dutch laboratories  [42] , and from independent evaluations by study authors where author-designed assays were used. Data were analyzed using R statistical software version 4. 1. 2 [32].