dataesr/scientific-paragraphs-categorization huggingface dataset:
ppOS The primary endpoint for this analysis was ppOS, defined as time from first PD to death from any cause, estimated using the Kaplan-Meier method. ppOS was evaluated using updated data in patients who had experienced PD at the time of updated data cutoff (BRIM-2: February 1, 2012; BRIM-3: July 8, 2015; BRIM-7: July 10, 2017; coBRIM: October 13, 2017). Prognostic subgroups identified by RPA were applied to the total pooled population and to pooled treatment cohorts (cobimetinib plus vemurafenib, vemurafenib monotherapy, and dacarbazine). Patients who did not die were censored at data cutoff.
RPA At the time of data cutoff for the RPA, 1004 patients had experienced PD; after excluding patients with missing data, 809 patients were included in the RPA. Demographic and clinical characteristics were comparable across the pooled population and pooled treatment cohorts (Table  1 ). Median postprogression follow-up durations were 5. 4 months (interquartile range [IQR], 2. 2-12. 5 months) for all patients, 4. 8 months (IQR, 1. 9-10. 7 months) for the cobimetinib plus vemurafenib cohort, 5. 2 months (IQR, 2. 2-12. 0 months) for the vemurafenib monotherapy cohort, and 6. 5 months (IQR, 2. 6-15. 6 months) for the dacarbazine cohort. In all pooled patients, the RPA identified baseline LDH, baseline disease stage, ECOG PS at PD, and initial ppRx as significant prognostic factors for ppOS, producing seven subgroups with distinct outcomes (Fig.
ppOS At the time of updated data cutoff for ppOS, 1027 patients had experienced PD; after excluding patients with missing data, 955 patients were included in analyses of ppOS based on RPA-defined prognostic subgroups. Demographic and clinical characteristics remained comparable across cohorts (Additional file 1: Table  S1 ). Median postprogression follow-up durations were 5. 5 months (IQR, 2. 2-14. 0 months) for all patients, 5. 2 months (IQR, 2. 1-14. 0 months) for the cobimetinib plus vemurafenib cohort, 5. 3 months (IQR, 2. 2-13. 2 months) for the vemurafenib monotherapy cohort, and 6. 5 months (IQR, 2. 6-15. 6 months) for the dacarbazine cohort. Among all patients, median ppOS was longest in the subgroup with normal baseline LDH, baseline disease stage M1c, and initial ppRx with immunotherapy or targeted therapy (12.
Further, the current analysis is largely based on data from randomized studies, wherein patients with similar baseline prognostic factors received a reference first-line and experimental treatment; this analysis is, therefore, less subject to the bias inherent in retrospective analyses of postprogression survival using data from patients treated in routine clinical practice where treatment decisions were based on physician and individual patient preferences  [19] [20] [21] [22]. However, as a retrospective exploratory analysis our analysis is also subject to limitations. The pooled dataset included patients enrolled in nonrandomized studies and original treatment assignment may be subject to selection bias. Also, choice of ppRx was not randomized or dictated by study protocols and may have been influenced in a nonuniform manner by changes in access to new treatments across the approximate 8-year time period over which the included studies were conducted.
Availability of data and materials Qualified researchers requesting access to de-identified patient-level data through the Clinical Study Data Request platform will be provided with accompanying clinical study documentation (protocol and any associated amendments, annotated case report form, reporting and analysis plan, dataset specifications, clinical study report). Researchers requesting access to clinical study documentation only, can do so via the following link: http://www. roche. com/resea rch_and_devel opmen t/who_we_are_how_we_work/clini cal_trial s/our_commi tment _to_data_shari ng/clini cal_study _docum ents_reque st_form. htm.
It is used to produce node embeddings for solving a classification task. InfoGraph  [36]  uses the same estimator to produce embeddings for entire attributed graphs. Graph2Vec  [29]  uses the same MI estimator in combination with Wesfeiler-Lehman procedure (see Section 3) to produce graph embeddings. It was originally inspired by languages models (especially Word2Vec) and considers node embedding as the vocabulary used to "write" a graph. Graph coarsening. The aim of graph coarsening is to produce a sequence of graphs of decreasing sizes; it can be done by node pooling, as with Loukas' algorithm  [19] , or by node decimation, like for example Kron reduction  [6, 1]. Coarsening can be combined with edge sparsification  [3, 6] , so as to reduce the density of coarsened graphs. In another context, DiffPool  [48]  performs graph coarsening using clustering but it learns the pooling function specific to each task in a supervised manner.
Graph2Vec [29] combines those two ideas to produce its graph embedding. The joint probability P XY = P X|Y P Y is constructed by sampling a graph g from the dataset, and then by sampling a label x from the sequence generated by WL from this graph. Minimizing the cross entropy with respect to θ leads to the following expression for the loss: L = E (x,g)∼P XY log σ(θ g • θ x ) + E (x,g)∼P X ⊗P Y log σ(-θ g • θ x ) (4) The discriminator function T θ (x, y) = θ y • θ x is taken as a dot product between a graph embedding θ g ∈ R d and a WL label embedding θ x ∈ R d , which are vector of parameters randomly initialized and optimized with SGD. There is one such vector for each graph g and each label x produced by WL. The resulting graph embedding is E(g) = θ g while θ x can be discarded. Optimizing this objective ensures to maximize the mutual information between the WL labels and the graph embeddings, which is a way to compress information about the distribution of WL labels into the embedding.
Since the algorithm is quite evolved and its explanation beyond the scope of our work, we refer to the original paper  [19]  for an explicit description of Loukas' method and an extensive list of properties and guarantees. The interest of this coarsening method is that, if two graphs g l-1 and h l-1 are close enough, the coarsened graphs g l is itself a satisfying coarsening for h l-1. By symmetry, the same result follows for h l and g l-1. Hence, one may hope that g l and h l share similar patterns (Figure  4 ), and it will be advantageous for the WL procedure. Such intuition can be confirmed experimentally. On four datasets (see Section 5 for their description), we sample two graphs g 0 and h 0 with independent and uniform probability. We measure their similarity using the Wasserstein distance between their spectra.
Such a function can be inferred directly from the dataset itself, simply by looking by the co-occurrences of some similar sub-graphs. Hence, the ability of GNN to learn features relevant for some specific task is an other advantage over WL. GCN  [15]  is as a baseline among this family of networks (see Section 2 for a brief survey). Unfortunately,  [23]  have shown that they behave badly when stacked. They propose Truncated Krylov GNN in replacement of GCN, correcting its flaws, with better theoretical guarantees and empirical evidence of their superiority over GCN. We chose it, encouraged by the promising results of early experiments. It amounts to consider, for a given node, a receptive field extended to the neighborhood of nodes at most at given distance a. The resulting layer uses the normalized adjacency matrix Ã = D -1 /2 (A + I)D -1 /2 (like in GCN) of the graph g l : g l+1 = Pool(tanh ([g l , Ãg l , Ã2 g l ,.
Hierarchical Negative Sampling Following the principles of Graph2Vec we aim to maximize mutual information between node embeddings and local neighborhood embeddings at each level. We achieve this by training a classifier to distinguish between positive and negative examples of such pairs of embeddings. Positive pairs are defined as node v in level l and its "super node" P(v) in level l + 1. More precisely, consider the l-th level of the hierarchical pyramid. The probability P Y l is built by sampling a graph g from the dataset, then by sampling a node u with uniform probability from this graph. P X l |Y l is obtained by sampling a node from P -1 (u). It gives a pair (x l (P -1 (u)), g l+1 (u)) sampled according to P X l Y l = P Y l P X l |Y l. The negative pairs are built as usual from the independent probability P X l ⊗ P Y l.
The magnitude of those matrices allows to handle graphs with hundred of nodes efficiently on modern GPUs, with embedding as big as 512 and up to 8 graphs per batch and 5 stages of coarsening. In practice, the bottleneck turn out to be the pre-computation of coarsening, which do not benefit of GPU speed up. Space. Note that for datasets with small graphs we have dim (E i ) = 2Ld > dim(X i ) + dim(A i ) = n|V | + |V | 2. However, when the number of nodes exceeds 50, the embedding size is always smaller than the adjacency matrix. Hence, this method is more suitable for big graphs with hundreds of nodes.
Evaluation The code for the proposed method and to reproduce the experiments can be found on: https://github. com/ Algue-Rythme/GAT-Skip-Gram. An experimental evaluation is conducted on various attributed graphs datasets, and the quality of the embeddings is assessed on supervised classification tasks in Section 5. 2. Our method is inductive: the model can be trained over a dataset and be used to embed graphs coming from another dataset. This property is analysed in Section 5. 3. HG2V differs from Graph2Vec by the usage of GNN and Loukas coarsening. The influence of those two elements is analysed with ablatives studies in Section 5. 4. Table  2 : Accuracy on classification tasks. HG2V is trained over both TrainVal+Test splits, without using labels due to its unsupervised nature. Model selection of C-SVM and hyper-parameters of HG2V have been done with 5-cross validation over TrainVal split.
Datasets Standard datasets. We use standard datasets from literature: PROTEINS, ENZYMES, D&D, NCI1, NCI109, MUTAG, IMDB (binary and multi) and PTC FR downloaded from  [14]. We also use the challenging REDDIT (binary and 5K versions). Synthetic datasets. Additionally, we introduce a novel dataset for the community: Diffusion Limited Aggregation (DLA). The attributed graphs are created by a random process that makes the graphs scale free, DLA being known to be fractal objects. The graphs have then an interesting property that justifies the creation of a new benchmark. In addition it provides natural features for the nodes: the coordinates in space (and this is why we prefer it to the well known and usual scale-free network model that is the Barabasi-Albert model). We refer to Appendix ?? for more details.
The code can be found on https://github. com/Algue-Rythme/DiffusionLimitedAgregation Image datasets. We convert MNIST and USPS popular datasets of the computer vision community into graphs, by removing blank pixels, and adding luminosity and (x, y) coordinates as node features. To solve the task, the method should be able to recognize the shape of the graph, which is a global property. Due to the size of these datasets, we restrain ourselves to a subset of 10, 000 images randomly sampled (instead of the 70, 000 available). Frankenstein dataset was created in  [32]  by replacing nodes labels of BURSI dataset with MNIST images. Pre-processing All the continuous node attributes are centered and normalized. The discrete node labels use one hot encoding. When there is no node feature, we use the degree of the node instead.
Training Procedure The embedding are trained over 10 epochs. At each step, 8 graphs are picked randomly from the dataset, and all the vocabulary from these graphs is used to produce positive and negative pairs (by cartesian product). Hence, the number of negative examples in each batch is higher than the number of positive examples. Consequently we normalize the loss of negative samples to reduce the unbalance. The optimizer is Adam  [5]  and the learning rate follows a geometric decay, being divided by approximately 1000 over the 10 epochs.
Baselines We compare our work to various baselines of the literature: Kernel Methods All the results reported are extracted from the corresponding papers  [18, 30] , giving an idea of the best possible performance achievable by WL-Optimal Assignment  [18]  and the Wasserstein Wesfeiler-Lehman  [37]  graph kernels. It almost always outperform inductive methods based on neural networks. However, like many kernel-based method, they have quadratic time complexity in the number of graphs, which is prohibitive for dealing with large datasets. Due to its very high scores and its sensibly different design, we consider them apart and we never highlight their results (even if the best). DiffPool, GIN We report the results of the rigorous benchmarks of  [7] , including the popular DiffPool  [48]  and GIN  [47].
Results We note that get substantial improvements over Graph2Vec baseline for many datasets, more specifically when the graph are big and carry high dimensional features. For FRANKENSTEIN, if we connect a randomly initialized 2-layer CNN to the input of the model for better feature extraction, the results are improved and reach 66. 5 ± 0. 4% which is a noticeable improvement. Thanks to the end-to-end differentiability of the model, the CNN is trained with backpropagation, benefiting from the unsupervised loss signal. On the notably difficult REDDIT-B and REDDIT-5K we reach high results, comparable to SOTA. The coarsening operation is beneficial to these datasets, considering the size of the graphs. On datasets with smaller graphs, the results are less significant.
Inductive learning The method is inductive: it allows us to train HG2V on a dataset and test on another dataset. The training set is used to extract relevant features, that are expected to be seen again during inference. In previous section 5. 2 we showed that when the domain of training set and inference set are the same (different splits of the same dataset) it provides good results. In this section, we prove that even when the training set and the inference set are disjoint, the model is still able to produce good representations. We emphasize that this property is specific to unsupervised inductive methods. Hence, it is not possible to perform such experiment on Graph2Vec (transductive), or on models trained with supervised loss (e. g GIN, DiffPool). To the best of our knowledge, no such domain adaption experiments have been done in graph classification tasks.
Hyper-parameters The best hyper-parameters found in the previous section are kept as is without further hyperparameter tuning. The goal was not to reach the best possible result, but ensures that we can reuse trained weights to produce quickly good embeddings. USPS is a dataset of handwritten digits similar to MNIST, hence we expect that a model trained on it can also embed MNIST graphs. FASHION MNIST have been introduced in  [46]  is built similarly to MNIST: a set of 28 × 28 grayscale images. We only work on a subset of 10, 000 training examples.
Results We see that the model can easily transfer from one domain to another. MNIST seems to be a better prior than USPS, which is a behavior previously observed in transfer learning. On the other datasets, the accuracy drops compared to the baseline: it is expected, since the new graphs are out of distribution of the training set. Nevertheless, even in this unfavorable setting, the accuracy remains within a comparable range to the baseline. We conclude that the inductivity property is not only theoretical, but also a property that can be checked in practice, without specific hyper-parameter tuning or costly manual adaptation.
C. 1 DLA The DLA dataset has been artificially generated with Diffusion Limited Aggregation (  [43] ), a random process that creates cluster of particles following a Brownian motion. Particles are added one by one, and when two particles touch, they can aggregate with some probability of stickiness p. The resulting structure is a tree, each particle being a node and each link corresponding to a bond. The resulting graphs have scale free properties  [44]. The degree distribution of the nodes and their position in space will depend on p. We generated a total of 1000 graphs with 500 nodes each. This dataset is splited into two classes, one with stickiness p = 1 and the other with stickiness p = 0. 05. The attributes are the x and y coordinates of the particles following a 2D Brownian motion for simplicity.
C. 2 MNIST and USPS We produce graphs from MNIST (resp. USPS) handwritten digits dataset. The graphs are created by removing all the pixel with luminosity equals to 0 (resp. less than 0. 3), by mapping each remaining pixel to a node, and then by adding the x and y coordinate of each node to the vector of attributes. Due to the size of MNIST (70, 000 images in total) we kept only the test split (10, 000 images) to train the embeddings, and kept the whole dataset for USPS (9298 images). However these datasets and their graphs remain way larger than the other standard benchmarks.
1 epoch already provides a strong baseline for molecule datasets, and lasts less than 1 minute, using GTX 1080 GPU. The most challenging dataset was REDDIT-MULTI-5K, trained on V100 GPU, with 5000 graphs, an average of 508 nodes and 595 edges per graph. The pre-computation of Loukas's coarsening required 40 minutes HG2V Graph2Vec+GNN Graph2Vec+Loukas Graph2Vec DATASET #nodes Accuracy Delta Accuracy Delta Accuracy Delta IMDB-b 20 70. 85 +7. 75 70. 70 +7. 60 57. 5 -5. 60 63. 10 NCI1 30 77. 97 +4. 75 75. 40 +2. 18 65. 45 -7. 77 73. 22 MNIST 151 95. 83 +39. 56 91. 05 +34. 78 72. 5 +16. 23 56. 27 D&D 284 78. 01 +19. 37 79. 26 +13. 16 66. 10 +7. 45 58. 64 REDDIT-B 430 91. 95 +16. 23 OOM 82. 50 +6. 78 75. 72 Table.
Again, the estimation of the attenuation effect of clouds and the variability of solar radiation at several locations over WA (in different climatic zones) will provide an added value to previous studies (e. g. ,  [11, 21] ) that focused on single locations. This approach could help develop specific plans tailored for managing solar energy production in different areas within the WA region. It is worth noting that dust has important effects on the incoming solar radiation, but we focus only on clouds in this study. The article is organized as follows. Section 2 presents the dataset, study area, and methodology. Section 3 presents and discusses the findings of the study. In Section 4, we conclude and provide insights for the future.
Data In this study, cloud cover and incoming surface solar radiation flux data are analyzed and the attenuation of the solar radiation by clouds is determined. The cloud cover and radiation data are extracted from the fifth generation of the state-of-the-art European Centre for Medium-range Weather Forecasts (ECMWF) Re-Analysis, which is also known as ERA5 (Detailed documentation at  [22]. In  [23] , the ERA5 dataset was evaluated and used to analyze the synergies between renewable energy sources including solar energy in WA. The dataset is available at an hourly resolution with a horizontal resolution of 0. 25 • × 0. 25 • on a regular grid. For this study, we analyzed only daytime hours for a 10-year period spanning from 2006 to 2015. ERA5 provides 2D (and 3D) distributions of different cloud properties integrated for all levels within some predefined layers in the atmosphere.
Here, we analyze the cloud fraction, which includes the total cloudiness (TC), low-level cloudiness (LLC), mid-level cloudiness (MLC), and high-level cloudiness (HLC). In the ERA5 dataset, TC, LLC, MLC, and HLC are clouds integrated from the surface to the top of the atmosphere (TOA), the surface to 800 hPa, 800 hPa to 450 hPa, and 450 hPa to TOA, respectively  [24]. In addition to the cloud fraction, ERA5 provides the surface downwelling shortwave radiation fluxes in all-sky and clear-sky conditions. Table  1  provides a summary of all the input data used for this study.
Data Evaluation A necessary condition for using the ERA5 dataset for such a study is to first evaluate its ability to reproduce the actual observations in the region of interest. In a recent study by  [19]  in WA, the ERA5 cloud fraction was compared with some synoptic in situ observations (and satellite data) from the region obtained from the Integrated Surface Database  [25]. The comparison showed a satisfactory agreement between the reanalysis and in situ dataset (see Appendix S1 in  [19] ). Here, we evaluate the surface downwelling shortwave radiation flux with in situ observations in WA. For this, we use surface measurements from two sites of the AMMA-CATCH observatory (African Monsoon Multidisciplinary Analysis-Couplage de l'Atmosphère Tropicale et du Cycle eco-Hydrologique  [26] ). The two sites (red triangles in Figure  1 ) are located at Wankama South (2.
6334 • E and 13. 6476 • N) in Niger and Nalohou (1. 6049 • E and 9. 7448 • N) in Benin and provide complete surface solar radiation flux measurements for the period of study. Thus, solar radiation data from the closest grid points to the two sites in the ERA5 dataset are used for the evaluation complemented by the satellite-based Surface Radiation Data Set-Heliosat (SARAH) hourly solar irradiance time series  [27] , extracted from EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Atmosphere 2020, 11, x FOR PEER REVIEW 4 of 18 the region obtained from the Integrated Surface Database  [25]. The comparison showed a satisfactory agreement between the reanalysis and in situ dataset (see Appendix S1 in  [19] ). Here, we evaluate the surface downwelling shortwave radiation flux with in situ observations in WA.
For this, we use surface measurements from two sites of the AMMA-CATCH observatory (African Monsoon Multidisciplinary Analysis-Couplage de l'Atmosphère Tropicale et du Cycle eco-Hydrologique  [26] ). The two sites (red triangles in Figure  1 ) are located at Wankama South (2. 6334° E and 13. 6476° N) in Niger and Nalohou (1. 6049° E and 9. 7448° N) in Benin and provide complete surface solar radiation flux measurements for the period of study. Thus, solar radiation data from the closest grid points to the two sites in the ERA5 dataset are used for the evaluation complemented by the satellite-based Surface Radiation Data Set-Heliosat (SARAH) hourly solar irradiance time series  [27] , extracted from EUMETSAT Satellite Application Facility on Climate Monitoring (CM-SAF). Figure  2  presents two Taylor diagrams showing the performance of ERA5 and SARAH in the evaluation with the surface observations in Benin and Niger, respectively.
Summary and Conclusions In this paper, we used the state-of-the-art ERA5 reanalysis dataset to show the occurrence of cloud cover in WA and assess its impacts on surface solar radiation in the context of planned large-scale solar energy generation in the region. The goal was to contribute to answering some key relevant questions to provide useful information for stakeholders involved in the planning of large-scale solar energy projects in the region. The results showed that the attenuation of incoming solar radiation by cloud cover is strongest in the summer months during the core of the monsoon season in all parts of WA. Analysis of the year-to-year variability of the surface solar radiation also revealed a higher uncertainty in the received solar radiation during the summer (wetter) months as compared to the drier months.