dataesr/scientific-paragraphs-categorization huggingface dataset:
We again use the five copies of the dataset described in RQ1 and compute the various evaluation metrics. Table  3  shows that the performance of PatchNet degrades if we ignore any one of the considered types of information. Accuracy, precision, recall, F1 score, and AUC drop by 19. 39%, 15. 41%, 21. 26%, 18. 34%, and 16. 06% respectively if we ignore commit messages. They drop by 16. 96%, 14. 62%, 16. 58%, 14. 76%, and 14. 21% respectively if we ignore code changes. And they drop by 11. 08%, 12. 62%, 16. 43%, 13. 86%, and 11. 98% respectively if we ignore function names. Thus, each kind of information contributes to PatchNet's performance. Additionally, the drops are greatest if we ignore commit messages, indicating that they are slightly more important than the other two to PatchNet's performance.
RQ4: What are the results of PatchNet on the complete set of Linux kernel patches? For RQ1, we use a dataset collected such that the number of stable and non-stable patches is roughly balanced. Among the 267,251 patches that meet the selection criteria, we picked 42,408 stable patches and 39,995 non-stable patches to build our dataset. To investigate the results of PatchNet on the complete set of patches from Linux v3. 0-v4. 12 having at most 100 lines (and accepted by our preprocessor), we randomly divide the remaining 184,481 non-stable patches into five sets and merge each of them with each of the five test sets described in RQ1. After this process, we have a new collection of five test sets. In each test set, there are around 8. 4K stable patches and 44. 8K non-stable patches. For each new test set, we use the corresponding model trained for RQ1.
We repeat this five times, and then average the results to get the aggregated AUC score. PatchNet achieves an average AUC of 0. 808. Since the new five test sets are highly imbalanced (only 15. 79% patches are stable patches), we omit the other metrics (i. e. , accuracy, precision, recall, and F1)  [49] ,  [53] ,  [60]. We also trained PatchNet on a whole training dataset (i. e. , 42,408 stable patches and 39,995 non-stable patches) and evaluated it on 184,481 non-stable patches. We find that PatchNet can correctly label them as non-stable 81. 32% of the time. We also check the effectiveness of PatchNet on patches that have more than 100 lines of code (i. e. , long patches). As mentioned earlier, we omit those patches from our training dataset as they do not meet the selection criteria of Linux kernel.
We collect 52,415 long patches from July 2011 to July 2017. Among them, there are 3,376 long stable patches and 49,039 long non-stable patches. 21. 33% of these patches contain the "Cc: stable" tag. The others may have been manually selected for stable versions despite not having a tag or may come from the release candidates. We again train PatchNet on the whole training dataset and evaluate the effectiveness of PatchNet on the 52,415 long patches. PatchNet achieves an AUC score of 0. 805. Again we only use AUC as this dataset is highly imbalanced  [49] ,  [53] ,  [60]. Finally, we also check whether there is a difference of performance in classifying patches containing a "Cc: stable" tag and patches that do not containing a "Cc: stable" tag. Among the 42,408 stable patches, there are 15,410 stable patches with a stable tag and 26,998 stable patches with no stable tag.
e. , vmwgfx_kms. c). We also see the difference between removed lines and added lines when the author committed his code. By representing the removed code and the added code as two three-dimensional matrices (each dimension represents the number of hunks, the number of removed or added code lines, and the number of words in each removed or added code line), PatchNet uses the removed code module and the added code module to construct the embedding vector of the removed code and added code, respectively (see Section 3. 3). The two embedding vectors are then concatenated On the other hand, we find that none of the other baselines are able to classify the patch in Figure  12  as a stable patch. Keyword is a heuristic approach that only looks at whether the content of a commit message includes "bug" or "fix".
Unsuccessful Case Next, we present a patch that PatchNet fails to classify correctly as a stable patch. This example serves to provide an understanding of cases in which PatchNet may not perform well. Figure  13  shows a stable patch that was not recognized by PatchNet. Its commit message does not contain any keywords (i. e. , "bug" or "fix") that suggest whether the patch is a stable patch. The code changes only include one removed line and the removed line contains only three words: vma, vm_flags, and VM_IO. As there is very little information in both the commit message and the code changes, PatchNet is unable to predict the patch in Figure  13  as a stable patch. We find that the other baselines (i. e, keywords, LS-CNN, and LPU+SVM), except F-NN, also fail to classify the patch as a stable patch.
THREATS TO VALIDITY Internal validity. Threats to internal validity relate to errors in our experiments and experimenter bias. We have double checked our code and data, but errors may remain. In the baseline approach by Tian et al. [63] , commits were labeled by an author with expertise in Linux kernel code, which may introduce author bias. In this work, none of the authors label the commits. External validity. Threats to external validity relate to the generalizability of our approach. We have evaluated our approach on more than 80,000 patches. We believe this is a good number of patches. Still, the results may differ if we consider other sets of Linux kernel patches. Similar to the evaluation of Tian et al. [63] , we only investigated Linux kernel patches, although PatchNet can be applied to patches of other systems, if labels are available.
Similar to LS-CNN, PatchNet also learns semantic representations of both code and text. However, different from LS-CNN, PatchNet includes a new representation learning architecture for commit code comprising the representations of removed code and added code of each affected file in a given patch. The representation of removed code and added code is able to capture the sequential nature of the source code inside a code change, and it is learned following a CNN-3D architecture  [26]  instead of LSTM. Our results in Section 4 show that PatchNet can achieve an 11. 24% improvement in terms of F1 over the LS-CNN model.
CONCLUSION In this paper, we propose PatchNet, a hierarchical deep learning-based model for identifying stable patches in the Linux kernel. For each patch, our model constructs embedding vectors from the commit message and the set of code changes. The embedding vectors are concatenated and then used to compute a prediction score for the patch. Different from existing deep learning techniques working on the source code  [16] ,  [17] ,  [24] ,  [36] ,  [44] ,  [66] ,  [68] , our hierarchical deep learning-based architecture takes into account the structure of code changes (i. e. , files, hunks, lines) and the sequential nature of source code (by considering each line of code as a sequence of words) to predict stable patches in the Linux kernel. We have extensively evaluated PatchNet on a new dataset containing 82,403 recent Linux kernel patches.
On this dataset, PatchNet outperforms four baselines including two also based on deep-learning. In particular, for a wide range of values in the precision-recall curve, PatchNet obtains the highest recall for a given precision, as well as the highest precision for a given recall. For example, PatchNet achieves a 14. 9% higher recall (0. 786) at a high precision level (0. 950) and a 41. 2% higher precision (0. 603) at a high recall level (0. 950) compared to the best-performing baseline. In future work, we want to investigate ways to improve our approach further, e. g. , by incorporating additional data such as more kinds of names and type information. Another issue is to identify the stable versions to which a patch should be applied. We plan to investigate whether machine learning can help with this issue.
It would also be interesting to apply our approach that learns patch embeddings to other related problems, e. g. identification of valid/invalid patches in automated program repair  [69] , assignment of patches to developers for code review  [62] ,  [71] , etc. Dataset and Code. The dataset and code for PatchNet are available at https://github. com/hvdthong/PatchNetTool. A video demonstration of PatchNet is available at https: //goo. gl/CZjG6X.
Methods To test the validity of our derivation, we compared the state of a size distribution at a given χ in different contexts. We used our new code Ishinisan, a coagulation algorithm inspired by the Dustdap code  (Guillet et al. 2007 ), that we make public with the publication of this paper. We emphasize that Ishinisan computes the (discrete version of) Smoluchowski equation (  1 ) and not the derived equation (  8 ). The code is able to reproduce the coagulation of grains using the constant and additive kernels for which analytical solutions exist. The aim of these tests is to show the validity of our derivation that the coagulated size distribution only depends on the value of χ. Appendix B details the methods we used that are implemented in the code. We took an MRN distribution with a min = 5 nm and a max = 250 nm as our initial condition.
Comparing various environments Our reference environment was taken from a numerical simulation of a protostellar collapse using the RAMSES code  (Teyssier 2002). We tracked the density and temperature history of a tracer particle during the first phases of the collapse of a uniform M = 1 M sphere of gas until its arrival at the first  Larson (1969)  core. The trajectory is mostly isothermal at T = 10 K, with the temperature rising as the density reaches n H ≈ 2 × 10 11 cm -3. The initial density was n H = 1. 4 × 10 6 cm -3 , and we stopped the simulation when the particle was well settled into the first core at n H = 2. 4 × 10 12 cm -3. The integration along the path gives χ = 8. 12 × 10 17 cgs. We used Ishinisan to compute the coagulation along the trajectory of the particle and to compare it with environments at constant temperature and density (physically realistic or not) that are summarized in Table  1.
We adapted the model of DS87 by reducing it to a single equation that can be numerically solved to rapidly compute the average charge of grains as a function of their size, as well as the ion and electron density. Using a Newton-Raphson scheme, the solution of this equation is fast enough to be performed on the fly in numerical simulations. Two upcoming papers will focus on the application of these methods to the formation of dense molecular cores and in pro-tostellar collapse simulations. These tools can be used together to rapidly and self-consistently calculate the nonideal MHD resistivities locally. The code Ishinisan is publicly available at https://bitbucket. org/pmarchan/ishinisan. P. Marchand et al. :  Fast methods for tracking grain coagulation and ionization. I. Analytic derivation (λ = -3.
Acknowledgements. P. M. acknowledges financial support by the  Kathryn W. Davis Postdoctoral Fellowship  of the  American Museum of Natural History. U. L. acknowledges financial support from the  European Research Council (ERC)  via the  ERC Synergy Grant  ECOGAL (grant  855130 ). M. -M. M. L. acknowledges partial support from  NSF  grant  AST18-15461  and  NASA   Theory Program  grant  NNX17AH80G.
Satellite image time series (SITS) collected by modern Earth Observation (EO) systems represent a valuable source of information that supports several tasks related to the monitoring of the Earth surface dynamics over large areas. A main challenge is then to design methods able to leverage the complementarity between the temporal dynamics and the spatial patterns that characterize these data structures. Focusing on land cover classification (or mapping) tasks, the majority of approaches dealing with SITS data only considers the temporal dimension, while the integration of the spatial context is frequently neglected. In this work, we propose an attentive spatial temporal graph convolutional neural network that exploits both spatial and temporal dimensions in SITS. Despite the fact that this neural network model is well suited to deal with spatio-temporal information, this is the first work that considers it for the analysis of SITS data.
e. , Laplacian graph extraction  [9]. To deal with the SITS land cover mapping task, with the aim of explicitly integrating the segments spatial correlation in the underlying analysis, we propose an attentive Spatial Temporal Graph Convolutional Neural Network, named STEGON. STEGON leverages attention at two different stages. Firstly, the spatial neighborhood of a SITS segment is automatically aggregated weighting the contribution of each neighbor according to its importance. Secondly, an attention mechanism combines the information coming from the target SITS segment and its neighborhood. Overall, the contributions of our work are as follows: • We propose a novel spatial temporal graph convolutional approach to deal with SITS land cover mapping task. • We equip STEGON with several attention modules, allowing the model to automatically weight the contribution of the spatial neighborhood at several stages of the processing pipeline.
• We adopt spatial graph convolutions  [9]  to work directly on the raw data avoiding global graph analysis like eigenvalues and/or eigenvectors computation; this allows STEGON to scale up on real world large study areas conversely to all the previous spatial temporal graph CNN methods  [9]. • We perform extensive experiments on two different benchmarks covering two large study areas; the results confirm the quality of our model consistently with respect to all the competing methods. To validate our proposal, we consider benchmarks representing two different study areas exhibiting contrasted land cover landscapes and state of the art machine learning and deep learning approaches commonly employed in the task of land cover mapping from SITS data.
IV. METHOD OVERVIEW In this section we introduce STEGON , our attention-based spatial convolutional graph neural network especially tailored to deal with the land cover mapping task. Figure  3  visually depicts the proposed framework. Given a target segment with its spatial neighborhood set as input, STEGON processes on one side the time series information associated to the target segment (the red node) and, on the other side the time series associated to the spatial neighborhood set (the blue nodes). In both cases, a one dimensional convolutional neural network is employed as encoder network to extract the segment embeddings. This encoder network operates on the temporal dimension of the SITS data explicitly modeling the sequential information it contains. For this encoder we use the one dimensional CNN introduced in  [23]  as embedding extractor for the segment SITS.
V. SATELLITE IMAGE TIME SERIES DATA AND GROUND TRUTH The analysis is carried out on the Reunion Island dataset (a French Overseas department located in the Indian Ocean) and the Dordogne dataset (a French department located in the Southwest). The Reunion Island (resp. Dordogne) dataset   2  and preprocessed in surface reflectance via the MACCS-ATCOR Joint Algorithm  [27]  developed by the National Centre for Space Studies (CNES). For all the Sentinel-2 images we only considers band at 10m: B2,B3,B4 and B8 (resp. Blue, Green, Red and Near-Infrared). A preprocessing was performed to fill cloudy observations through a linear multi-temporal interpolation over each band (cfr. Temporal Gapfilling,  [14] ). Two additional indices: NDVI 3  (Normalized Difference Vegetation Index) and NDWI  4  (Normalized Difference Water Index), are also calculated.
Finally, each Sentinel-2 image has a total of six channels. The spatial extent of the Reunion Island dataset is 6 656 × 5 913 pixels corresponding to 3 935 Km 2 whereas the extent of the Dordogne site 5 578 × 5 396 pixels corresponding to 3 010 Km 2. Figure  4  and Figure  5  depicts the Reunion Island and Dordogne study site, respectively, with the associated ground truth polygons. As regards the Reunion Island dataset  [28] , the ground truth (GT) was built from various sources: (i) the Registre Parcellaire Graphique (RPG)  5  reference data for 2014, (ii) GPS records from June 2017 and (iii) visual interpretation of very high spatial resolution (VHSR) SPOT6/7 images (1,5-m)  completed by a field expert with the knowledge of territory to distinguish natural and urban areas. Regarding the Dordogne dataset  [29] , the GT was obtained via (i) the Registre Parcellaire Graphique (RPG) reference data for 2014 as the Reunion Island dataset and (ii) the Topographic database (BD-TOPO) 6 provided by the French National Geographic Institute (IGN).
For both datasets, the GT comes in GIS vector file format containing a collection of polygons each attributed with a unique land cover class label. In addition, to ensure a precise spatial matching with image data, all geometries have been suitably corrected by hand using the corresponding Sentinel-2 images as reference. Successively, the GIS vector file containing the polygon information has been converted in raster format at the Sentinel-2 spatial resolution (10m). Table  3   To analyze data at object level and exploit their spatial context, a segmentation was provided by field experts on each study site, according to one of the Sentinel-2 images of the time series that they considered pertinent for the adopted nomenclature. The selected image was segmented using the SLIC algorithm  [8]  available via the scikit-image toolkit  [30].
A. REGION ADJACENCY GRAPH STATISTICS The Reunion-Island dataset's RAG has 59 335 nodes, 186 374 edges and an average degree of 6. 28, whereas the Dordogne dataset's RAG has 103 514 nodes, 315 027 edges and an average degree of 6. 08. Moreover, considering the adjacency graph, Figures  6a  and 6b  show the cumulative distribution functions (in terms of number of neighbors) for the Reunion-Island and Dordogne study site, respectively, distinguishing between the whole set of segments (Nodes) and the set of segments with associated label information (Labeled Nodes). We can observe that the RAGs corresponding to the study sites have some differences, the maximum neighborhood size on the Reunion-Island is equal to 32 while on Dordogne study site the maximum neighborhood has a size equals to 21. We can also note that the majority (at least 98% of the objects considering both distributions) has a number of neighbors no greater than 8 for the Reunion-Island study site while, considering the Dordogne benchmark, the same value is no greater than 10.
5  summarize the average quantitative performances obtained by the different competing approaches on the Reunion-Island and Dordogne datasets, respectively. We can note that, according to the three evaluation metrics, STEGON clearly outperforms all the competitors on both study sites. Regarding the best competing method (CNN), our framework achieves more than 5 points of gain in F-Measure, demonstrating the added value derived by integrating the spatial surrounding information for the land cover mapping task. In addition, the comparison between STEGON and its ablations demonstrates that: i) automatically weighting, by means of the attention mechanism, the importance of the spatial context is more effective than an uniform weighting of all the neighborhood objects (STEGON vs STEGON noGAT ) and ii) integrating the auxiliary classifier into the training stage (i.
1, 0. 3, 0. 5, 0. 7, 0. 9}. Table  8  reports the performances of STEGON coupled with different activation functions for the graph attention mechanism. The subscript indicates which is the employed activation function. We can observe that, on both benchmarks, the LeakyReLU activation function achieves the best average performances in terms of F-Measure, Kappa and Accuracy. These results confirm the choice made by the authors in  [21]  that equip the graph attention mechanism with the LeakyReLU activation function. Figure  7  depicts the sensitivity analysis of STEGON w. r. t. the hyper-parameter λ (the importance related to the auxiliary classifier loss). We can observe that, generally, the proposed approach is quite stable regarding this hyperparameter. For the case of Reunion-Island, the performances, in terms of F-Measure, varying between 93.
C. QUALITATIVE EXPERIMENTAL RESULTS To further investigate the behavior of STEGON , we conduct two additional qualitative studies. First, we visualize and compare the representation learnt by STEGON w. r. t. the representations learnt by some of the competing deep learning methods so as to analyze their internal behavior and, second, we discuss a sample from the land cover map generated by STEGON. We deploy such evaluations on the Dordogne study site. Figure  8  depicts the visual projection (via T-SNE  [36] ) of the embeddings extracted by the competing deep learning approaches. The Dordogne dataset includes seven land cover classes, that are listed in the legend of the figure with the associated color mapping. We can note that STEGON (Figure  8f ) clearly recovers a more visible cluster structure with respect to the one exhibited by the embeddings extracted by the competitors.
VII. CONCLUSION Satellite image time series (SITS) data constitutes a valuable source of information to assess the Earth surface dynamics. Applications range from food production estimation to natural resources mapping and biodiversity monitoring. How to get the most out of such rich information source, leveraging simultaneously both spatial and temporal dimensions, is one of the main current challenges in the remote sensing community. To tackle it, in this work we have presented a novel attentive spatial temporal graph convolutional neural network to analyze SITS data in the context of land cover mapping. Our framework is equipped with a spatial attention mechanism that allows the network to automatically aggregate the spatial context information surrounding a target segment. Quantitative and qualitative evaluations demonstrate the effectiveness of our method with respect to state of the art competitors that do not integrate the spatial dimension in their analysis.
Dino Ienco  (This work was supported in part by the  French National Research Agency through the Investments for the Future Program , under Grant  ANR-16-CONV-0004  (DigitAg), in part by the  GEOSUD Project  under Grant  ANR-10-EQPX-20 , in part by the  French Ministry of agriculture Agricultural and Rural Development Trust Account , and in part by the  PARCELLE  Project funded by the  French Space Agency  under Grant  DAR CNES 2019.
The human cerebral hemispheres show a left-right asymmetrical torque pattern, which has been claimed to be absent in chimpanzees. The functional significance and developmental mechanisms are unknown. Here, we carried out the largest-ever analysis of global brain shape asymmetry in magnetic resonance imaging data. Three population datasets were used, UK Biobank (N = 39 678), Human Connectome Project (N = 1113), and BIL&GIN (N = 453). At the population level, there was an anterior and dorsal skew of the right hemisphere, relative to the left. Both skews were associated independently with handedness, and various regional gray and white matter metrics oppositely in the two hemispheres, as well as other variables related to cognitive functions, sociodemographic factors, and physical and mental health.