dataesr/scientific-paragraphs-categorization huggingface dataset:
With the present work we achieve for image-based PLI a language diversity comparable to the maximum one supported by image-based PLI approaches and also higher accuracy. Heuristic-based programming language recognition tools like sloccount  (Wheeler, 2001 ) and cloc  (Danial, 2006)  have been available for a long time. Linguist  (GitHub, Inc. , 2011)  is a state-of-the-art PLI tool developed by GitHub, with a self-reported  (Ganesan & Foti, 2019)  accuracy of 85%. Several PLI studies rely on Linguist as ground truth, as we do in this work. In terms of language diversity, Linguist supports 573 languages, but it requires file extensions-without which performance drops crucially  (Ganesan & Foti, 2019)  to an F1 score < 0. 05-which are not always. In particular, file extensions cannot be relied upon when trying to identify the programming language of code snippets and in all other situations in which only limited code portions are available.
Dataset The experimental dataset consists of a large set of code snippets images obtained by rendering real-world code, an approach shared with other image-based PLI works in the literature. We started from a dataset released by the Software Heritage project  (Di Cosmo & Zacchiroli, 2017) , consisting of all versions of all source code files collected from the 1000 most popular GitHub repositories on 2019-10-08 (https://annex. softwareheritage. org/ public/dataset/content-samples/2019-10-08-github-top1k/). This dataset contains both the actual files (≈ 25 million distinct files) and the associated filenames. We used file extensions as a first indication about the file types (that it is going to be refined later). Initially the dataset contained ≈ 6000 different extensions, from which we excluded all uncommon ones by filtering out those that occur with a frequency < 10 -5 within the dataset, thus obtaining 717 remaining extensions.
As the main goal of this work is the recognition of programming languages, we could not directly use file extensions as labels for supervised machine learning. Hence, we run Linguist (GitHub, Inc. , 2011) (a popular choice for source of truth of PLI works in the literature) on all source code files having one of the remaining extensions, while excluding all the files for which Linguist was unable to predict the language. At the end of this step 212 languages remained in the dataset, together with the associated source code files. From this point on classification labels are programming languages (as detected by Linguist) and no longer file extensions.
Code snippets The amount of both files and source lines of code (SLOCs) in the dataset at this point were highly unbalanced across languages. For example, popular programming languages such as Python or JavaScript occurs in thousands of source code file examples, whereas other languages only had 100 examples or less. As unbalanced datasets are a well-known issue for supervised machine learning we wanted to mitigate this issue, while at the same time avoiding both high file repetition rate (oversampling) and the exclusion of too many languages (downsampling). Avoiding downsampling is particularly relevant here, since our goal is to assess the feasibility of high-accuracy image-based PLI with many languages. We exploited the fact that we need code snippets images rather than entire source code files.
Whereas we did have enough data in our dataset to start from scratch, training cost remains an important concern in PLI because, due to the rapid evolution of source code artifacts in the target domain, one has to add to the initial training cost that of periodic retraining. This domain-specific aspect of the problem led us to the decision of using transfer learning. We compared the performances of three classifiers based on three different CNN architectures pre-trained for images recognition: ResNet34  (He et al. , 2016) , MobileNetv2  (Howard et al. , 2017)  and AlexNet  (Krizhevsky, Sutskever & Hinton, 2017). The first two have about 30 layers each, while the latter has 8 layers. We show in Fig. 2  the architecture of AlexNet, as it is the simplest to fully depict; the other two architecture are similar but.
Training As a preliminary step in the model training we replaced the classification layer (or head) of each CNN-initially composed of 1000 output neurons, as required by the ImageNet classification task-with a head composed of 149 output neurons, corresponding to the cardinality of our set of programming languages to recognize. We applied a (usual) 80/20% split to our dataset twice to obtain the training, validation, and test sets. First we kept aside 20% of the obtained code bundles for testing and then further split the rest to obtain the training and validation sets. This resulted in an overall partition of all code bundles in three sets: 64% for training, 16% for validation, and 20% for testing. We then applied a two-step training procedure to all three CNN architecture. As a first step the weights of the CNN (the body) have been frozen so that training could only affect the substituted head.
RESULTS We implemented in Python the three classifiers described in the previous section, using the PyTorch framework  (Paszke et al. , 2019). The three implementations are available as part of the complete replication package for this paper (see 'Conclusion'). For each architecture the training phase has been performed on the original nonscrambled dataset and then repeated for each classifier and for each scrambled version of the dataset. In order to be able to compare results among the different architectures we used a fixed PRNG (pseudorandom number generator) seed to make sure that images were processed in the same order during both training and evaluation.
Image-based PLI performances with many languages On the non-scrambled dataset, after 8 epochs of the first training phase-in which only the weights of the classifier's head were able to be updated-the ResNet-and MobileNetbased classifiers reached ≈ 90% accuracy on the validation set, while the AlexNet-based model reached only ≈ 60%. Performances improved for all models after the fine-tuning phase-when all weights could be updated, although at a lower learning rate. After 7 epochs of fine-tuning ResNet accuracy reached ≈ 92% (+2%), MobileNet ≈ 93% (+3%), and AlexNet ≈ 84% (a significant +24%, but still the worst performing classifier overall). Several performance measures have been computed to evaluate the performance of the three classifiers: precision (P), recall (R) and F1 measure (F1). Since PLI classification is a single-label, multi-class problem, P, R and F1 are computed on the confusion matrix obtained by applying the trained models to the test set separately for each class, as follows: P i = C ii 149 j=0 C ji = TP i TP i + FP i (1) R i = C ii 149 j=0 C ij = TP i TP i + FN i (2) F 1 i = 2P i R i P i + R i (3) where: the generic element C i,j of the confusion matrix represents the amount of samples of the ith class that the model classified as jth class instances; TP i is the amount of true positives (i.
Second, the ResNet-and MobileNet-based classifiers significantly outperform the AlexNet-based one, to better generalize to the PLI dataset within a limited number of training epochs. Performances of the ResNet-and MobileNetclassifiers are almost as good (-1. 5%) as the state-of-the-art in image-based PLI, in spite of a ×15-time increase in language diversity and of reduced training costs. A detailed breakdown of performance results for each language is given in Appendix A. We notice from it that most languages perform very well, close to the overview given by the aggregate performance metrics. Most of the languages that perform poorly still perform well above 80% precision with the best performing classifiers. The languages that perform the worst tend to have common syntactic characteristics either among them or with other languages included in the dataset.
What makes code snippets visually recognizable? The three classifiers trained on the original trainset have been then tested on the scrambled versions too. Performance results are presented in Table  3  for each architecture on the various dataset versions. The results provides some insights on what makes a code snippet visually recognisable, as we discuss below. We can see that randomly scrambling decimal digits alone (dataset ''N'') induces no degradation in PLI performances for the three classifiers. Scrambling alphabetic characters Table  3  PLI performances with and without scrambling. From left to right: no scrambling (Orig), scrambling of alphabetic characters (A), digits (N), symbols (S), combinations of them (AN, AS, NS, ANS) and substitution of all non-blank characters for x (X).
alone (dataset ''A'') has a mild performance impact, degrading precision and recall by 3-11%, depending on the CNN, with AlexNet being the most affected one. Scrambling symbol characters alone (punctuation, operators, parentheses, etc; dataset ''S'') on the other hand is enough to have a dramatic effect on the performances of every considered architecture, inducing an impressive drop in both precision and recall in the 2-4 × range. This degradation is likely due to the syntactic (and hence visual) importance that punctuation characters play in programming languages and the highly different usage of them across different languages. Scrambling several character classes at once (datasets: ''AN'', ''AS'', ''NS'', and ''ANS'') appears to simply combine the effects of scrambling individual character classes.
Orig Performances for the ''X'' dataset, where all non-blank characters have been replaced by x, are below 10% for both precision and recall in most cases, reaching as low as 1% for the precision of the ResNet-based classifier. It appears that the code ''layout'' alone, as captured by indentation, is nowhere near enough to make programming languages visually recognizable. 1 These maps can be seen as heatmaps, with black/blue colors indicating low temperatures and the scale going up to yellow for high temperature in the usual way. High temperature in our case means high contribution to the final classifier decision. approach. Using larger code bases  (Abramatic, Di Cosmo & Zacchiroli, 2018; Ma et al. , 2019)  as training datasets it should be possible to achieve even higher language diversity without significant reductions in identification accuracy.
We intuitively expected the ''layout'' of the code alone, as captured by indentation, to perform better than what we have observed with the ''all x'' scrambled dataset, which performed terribly with all classifiers in our experiments. In particular, we expected indentation to be a tell for at least the languages where indentation is syntactically meaningful, such as Python. That factor is probably contrasted by the fact that proper indentation is a coding best practice for all languages, and certainly so in a dataset assembled from popular open source projects. This could make good indentation a common trait of all languages and snippets, from which nothing can be learned by a trained classifier. To get a qualitative feeling of where the classifiers gather the most relevant information for image-based PLI, we show in Fig.
5  the class activation map (CAM)  (Zhou et al. , 2015)  heatmaps for selected snippets. We have generated CAMs from the ResNet-based classifier, using PyTorch hooks just after the last convolutional layer. Colors in CAMs highlight which image parts contributed the most to the final classifier decision, helping with the understandability of machine learning models. 1  A few observations about these CAMs are in order. First, CAMs highlight the fact that the beginning of code lines is very relevant for classifier decisions. This relates to the importance of symbol characters-which are often found at the beginning of each line, like parentheses, and are also highlighted by CAMs elsewhere in code snippets-but it appears to go deeper than that. For instance it seems that for several languages the CNN has learned to recognize full language keywords, such as "def" for the Python programming language and other keywords for Dockerfiles and Visual Basic.
Such a multimodal approach could be naturally applied to PLI, by interpreting code snippets as both images and text. The empirical validation of the applicability of side-tuning to PLI is left as future work. on StackOverflow or publishing a video tutorial being much higher than that of pushing code to GitHub). In terms of dataset size, and according to a recent literature review  (Kiyak et al. , 2020) , our experiments have been conducted over 12-15 × more snippets than the largest empirical studies in image-based PLI. Rather, our dataset size is inline with the largest datasets used for text -based PLI. It is worth noting that we have arguably amplified the number of snippets that form our experimental dataset, in the sense that we have extracted several snippets from each source code file.
CONCLUSION As a first contribution we have shown that image-based programming language identification (PLI) can be performed with high precision and recall even when many different programming languages (149 in our experiments) are considered. This corresponds to a 15 × increase in language diversity with respect to the state-of-the-art, incurring no significant loss (≤ 1. 5%) in accuracy. We established this result on a dataset consisting of ≈ 300,000 code snippets extracted from 1,000 popular GitHub repositories. We have used three classifiers based on convolutional neural networks (CNNs) pretrained for images (ResNet34, MobileNetv2, and AlexNet) then adapted to the PLI domain using transfer learning, with significant reductions in terms of training cost with respect to the common approach in the PLI literature of training from scratch.
The ResNet-based classifier performed best, achieving 92% in both precision and recall. Our second contribution is a quantitative exploration of what makes programming languages visually recognizable. By selectively scrambling different character classes in the original dataset we found evidence that symbolic characters such as punctuation, arithmetic operators, and parentheses contribute the most to visual programming language recognition, followed (by far) by alphabetic characters, with no to negligible impact by decimal digits and indentation in our experiments. A preliminary qualitative exploration of class activation maps(CAMs) supports these findings and also suggests that CNN-based classifiers for code snippet images might be able to autonomously learn to spot language keywords, without any a priori knowledge of programming language grammars.
Fine-grained analyses are needed to identify which code parts matter the most for CNN-based classifiers. CAM-based observations seem to hint at the importance of keywords, which we intend to explore more, together with finer-grained scrambling and spatial layout analyses. If a clear understanding of what makes code visually recognizable could be reached, it is possible that dedicated feature engineering could lead to even more accurate image-based PLI classifiers. Finally, even though we have been able to visually recognize almost 150 programming languages, we intend to try pushing further this limit and reach hundreds of recognized languages, retaining high accuracy. The main difficulty here is the lack of a suitably balanced dataset, a task that we intend to address in the near future by perusing large source code archives.
Data Availability The following information was supplied regarding data availability: A replication package for the experiments, with the code that generates the code snippet images, are available in the Supplemental Files and at Zenodo: Francesca Del Bonifro, Maurizio Gabbrielli, Antonio Lategano, & Stefano Zacchiroli. (  2021 ). Image-based Manylanguage Programming Language Identification -Replication Package (Version 1. 0) [Data set]. IEEE Software. Zenodo. http://doi. org/10. 5281/zenodo. 4986391.
DISCUSSION Based on the obtained results we can now affirmatively answer RQ1: it is possible to automatically recognize up to 149 different programming languages in brief code snippet images, with high accuracy. This is a significant step forward in the state-of-the-art of image-based PLI, which was limited to recognize up to 10 languages. This result paves the way to use of image-based PLI in real-world settings, where much more than a handful of programming languages need to be handled. It is worth noting that we stopped at 149 languages only to avoid dataset imbalance, not due to intrinsic limitations in the proposed.
Moreover, works that use ''real'' images rely for the most part on video frames extracted from programming tutorials. Those tutorials are generally produced as screencasts, in which programming editors and IDEs are recorded directly from the desktop environment (rather than, say, from a physical camera pointed at the screen), resulting in very high-quality video frame images. Therefore we do not expect our performances to be significantly impacted by the switch from synthetic images to screencast video frames. Visual recognition of actual real-world images-e. g. , pictures of billboards showing code or movie frames of screens showing code-would be a different matter, but it is a challenge we share with most works in the image-based PLI literature. We relied on Software Heritage as data source (instead of ad hoc crawling) and used a dataset corresponding to code retrieved from the most popular GitHub repositories.
In this paper we will extend the 1D modeling and experimental analysis done by Hughes et al. [8]  using the SOLPS-ITER model, a coupled fluid-plasma/kinetic Monte Carlo neutral simulation code  [11]. This will allow us to investigate the poloidal distribution of the neutrals and the changes that occur when pedestal opaqueness increases. As the neutral density is not poloidally symmetric, one aspect which is not well understood is how changes in opaqueness might affect poloidally asymmetric fueling  [12]. In the following section we will briefly review the C-Mod experiments that form the base of our investigation, as well as go over the SOLPS-ITER code and the setup of our simulations. In the third section we will review the neutral density information gleaned from these simulations, as well as the process with which we calculate the local penetration length of the neutrals.
SOLPS modeling setup The impact of neutral fueling and plasma transport on the density pedestal structure is investigated using the SOLPS-ITER code  [11]. The setup of our simulations involves specifying all vessel walls as molybdenum, with a recycling coefficient of 1. 0 (metal walls tend to not absorb incoming plasma) set at all surfaces with exception of the two targets, which are set to 0. 99, and the upper cryopump entrance surface, which is set to 0. 5 (determined from a wide scan of pump surface coefficient values). Deuterium gas puff ports are specified by designating specific wall elements as the puffing sources. The levels and locations are chosen to match the puffing carried out in the various experiments (See Fig. 1  and Table  1 ). Fixed particle and energy flux boundary conditions are set at the core boundary of the simulations (  and  , ) using experimentally derived values for the heat flux.
Acknowledgments The views and opinions expressed herein do not necessarily reflect those of the  ITER Organization. Work supported by  US DoE  under  DE-SC0007880 ,  DE-SC0014264 , and  DE-AC05-00OR22725. The authors acknowledge  William & Mary Research Computing  for providing computational resources and/or technical support that have contributed to the results reported within this paper. URL: https://www. wm. edu/ it/rc.
Background: Network-based analysis of gene expression through co-expression networks can be used to investigate modular interactions occurring between genes toward different biological functions. An extended description of the network modules is therefore a critical step to understand the underlying processes contributing to a disease or a phenotype. Biological integration, topology study and conditions comparison (e. g. wild vs mutant) are the main methods to do so, but to date no tool combines them all into a single pipeline. Results: Here we present GWENA, a new R package that integrates gene co-expression network construction and a whole characterization of the detected modules through gene set enrichment, phenotypic association, hub genes detection, topological metric computation, and differential co-expression.
Conclusion: GWENA is an R package available through Bioconductor (https://bioconductor. org/packages/release/bioc/html/GWENA. html) developed to perform extended analysis of gene co-expression networks. Thanks to biological and topological information as well as conditions comparison, it eases the understanding of genes interactions involved in diseases or phenotypes. Going beyond actual packages to perform co-expression analysis, GWENA includes new tools to fully characterize modules, such as differential co-expression, additional enrichment databases, and network visualization.
Background The study of biological functions through discrete genes analysis methods has allowed the description of numerous pathways and the understanding of genedisease associations  [1]. The full comprehension of the complex interactions taking place in cellular processes requires methods able to grasp the connections between the genes involved  [2]. To address this issue, biological networks have been used as a framework to represent and study interactions between genes. In a gene network, a node represents a gene and an edge joining two nodes represents their interaction. Among the measures of interaction, weighted co-expression is one of the most widely used thanks to the popularity of the WGCNA R package  [3]  where the interactions are quantification (weight) instead of only a presence/absence information.
Moreover, no available tool includes differential co-expression, exploit the potential of other topological metrics such as connectivity, or enables analysis to be carried out with other R packages or software as easily. In a desire to meet all these needs, we developed an R package for Gene Whole co-Expression Network Analysis (GWENA) available on Bioconductor (https://bioconductor. org/packages/release/bioc/ html/GWENA. html). Based on a modified version of WGCNA for the network construction and modules detection, GWENA is a modular pipeline that provides ORA enrichment on 9 biological sources, phenotypic association, hub genes detection, and differential co-expression between multiple conditions. These come with a set of descriptive visualizations to help the user to understand and interpret the results.
In order to demonstrate the capabilities of our tool, we applied it to investigate skeletal muscle aging using publicly available gene expression data from donors spanning different age ranges from the GTEx database (ref). Skeletal muscle aging is indeed a major source of mobility loss in the elderly, resulting in a high fall ratio, depression, and therefore an increased mortality  [17]. This decrease in the regenerative capacity of skeletal muscles and their progressive atrophy (sarcopenia)  [18]  gradually leads to a reduction of the contractile force and thus a loss of autonomy of the individuals  [19]. Recent studies have made progress in finding factors associated to progression of sarcopenia  [20, 17] , such as body weight  [21] , but the understanding of their intricate molecular mechanisms is still lacking.