dataesr/scientific-paragraphs-categorization huggingface dataset:
2 sequence for the -s parameter. This assembly led to 32,219 transcripts and to a gtf file showing their mapped location in the PN40024 12X. 2 sequence. The two transcript datasets were formatted in gff format compatible with the Artemis Browser so that the predicted gene structures of the cytochromes P450 could be compared with the transcripts and edited if needed. The command maskFastaFromBed v2. 19. 1 from the bedtools package  [49]  was used to mask the regions of the PN40024 12X. 2 sequence where we annotated cytochrome P450 exons after having reformatted the gff file of the annotations into a bed file. We performed TBLASTN analyses of the four grape cytochrome P450 datasets against the masked PN40024 12X. 2 sequence and parsing analyses using the same parameters and cutoffs as previously described.
This step allowed identifying the region of the grape genome for which a cytochrome P450 similarity was missed during the manual curation. To validate the set of complete genes of cytochromes P450 that we annotated, a BLAST against non-redundant sequence database (NR) was performed and only the genes for which the best hit was a cytochrome P450 were kept. For the pseudogenes, a BLASTX was performed against the set of complete P450 genes that we annotated and we kept only the ones that aligned over at least 30% of the query length with the percentage identity of 50%. The presence of physical clusters of cytochrome P450s in the grape genome was tested based on the following definition of a cluster. Two consecutive P450 annotations are part of a cluster if they are separated by 200kb and 8 non-P450 genes at the most  [50, 51].
The two annotations also have to be located on the same scaffold, which guaranties a precise estimation of the intergenic distances. A bootstrap test was performed to check whether the cytochromes P450 were more clustered than what is randomly expected. A homemade script was developed with R version 3. 0. 2 [52]. Ten thousand sampling without replacement of 579 (number of P450 annotations) or 279 features (number of complete P450 genes) were performed on the genome annotation V1 stored in Grape Genome Database hosted at CRIBI counting 29,971 features. The percentage of features organized in clusters was computed using the same protocol as for cytochromes P450. The p-value was calculated by counting each time a percentage equal of greater than the percentage of P450 in clusters divided by 10000 (number of iterations).
Sequence similarity within and between clusters was analyzed by performing a BLASTP search of translated complete P450 genes against themselves. Only the genes that aligned over at least 70% of the query length with the percentage identity of 40% were kept. The Circos software  [53]  was used to draw the figure. Clusters that contained less than two complete genes were excluded from this analysis (i. e. clusters that contained partial genes, pseudogenes and putative pseudogenes with less than 2 complete genes). The dotter software version 4. 23  [54]  was used to draw the sequence similarity graphs of the cluster 190 with its fasta sequence and annotations in a gff format as an input.
Gene expression We retrieved raw grape RNA-Seq data from NCBI SRA public database (http://www. ncbi. nlm. nih. gov/sra). Fifty-nine sequence files generated in the framework of six different experiments  [42, 43, 45, 46, 63, 64]  and four RNA-Seq datasets submitted in the frame of this study were used. The data were formatted in the fastq format using the fastq-dump command from the SRA Toolkit package version 2. 3. 4 (http://www. ncbi. nlm. nih. gov/books/NBK158900). Alignments of these reads against the PN40024 12X. 2 sequence were then performed using GSNAP version 2013-11-27  [65]  with the following parameters: -B 4, -N 1, -n 3,-nofails and the quality protocol according to the experiment. These files were parsed to keep the best, unique and paired (if paired-end reads) alignments using a homemade script.
The number of fragments aligned on each annotation from the genome annotation V1 stored in Grape Genome Database hosted at CRIBI and the cytochromes P450 was counted using the command htseq-count from the HTSeq framework version 0. 6. 0 [66]  with the following parameters: -m intersection-nonempty and -s no. Using a homemade script, FPKMs (Fragments Per Kilo base of exon per Million fragments mapped) were calculated for every annotation. Using all non-zero FPKM values, the 33th and 66th quantiles were calculated to assign the expression values to one of the four levels of expression chosen: no, low, average and high expression. The experiments were grouped into six categories regarding the conditions under which the samples were obtained. These categories were: leaves, downy mildew (Plasmopara viticola) infected leaves, powdery mildew (Erysiphe necator) infected leaves, flowers, young berries and ripe berries.
An average expression per category was then calculated for each gene and assigned to one of the four levels of expression regarding its value: no expression if the average was zero, low expression between zero and the 33% quantile, average expression between 33% and 66% quantile and high expression for averages higher than the 66% quantile. The average expression values for each P450 annotation were used to perform a clustering analysis using HCE version 3. 5  [67]  with a complete linkage method and a Pearson's correlation as distance measure. The cut-off to define the clusters was set at a Pearson's correlation coefficient of 0. 656. The heatmap was drawn using the package "pheatmap"  [68]  after row normalization ((FPKM valuerow minimum) / row maximum) in R  [52]. A RNA-Seq dataset of 48 conditions for berries at four developmental stages (bbch75 = pea size; bbch77 prior to veraison; bbch85 at the end of veraison; bbch89 ripe) for four grapevine varieties (Sangiovese, Barbera, Negro amaro and Refosco) in triplicate, published by Palumbo and coworkers  [69]  was used to perform an analysis of differentially expressed genes.
The reads were aligned using STAR  [70]  and counted using featureCounts  [71]  on the grapevine reference genome PN40024 12X. 2 and the VCost. v3 annotation  [72]  supplemented with the cytochrome P450 annotations. The analysis of differentially expressed genes across the four varieties was performed on the whole gene set of the grapevine genome using the script AskoR_DE. R (https://github. com/askomics/askoR) with the parameter cpm > 0. 5.
Expression profiles of grapevine P450s To identify P450 genes with potential roles in pathogen resistance or biosynthesis of berry metabolites we analyzed the expression of the 579 P450 sequences. Pseudogenes were included in the analysis of expression to account for recently pseudogenized sequences that may still be expressed to some extent. We used 59 RNA-Seq datasets (S2 Table ), which describe gene expression in different tissues (flowers, berries, leaves), different stages of berry development, and pathogen infection (downy and powdery mildews). To enable a meaningful comparison of gene expression between different experiments we calculated fragment number per kilobase of transcript per million mapped reads (FPKM) for each P450 sequence (S1 File). The majority For each circle, the grey bars correspond to the 19 grape chromosomes and the "Unknown chromosome".
6, median FPKM = 0. 4) compared to pseudogenes and putative pseudogenes (mean FPKM = 2. 4, median FPKM = 0) or partial genes (mean FPKM = 1. 7, median = 0. 1). To simplify the dataset, we grouped the 59 experiments in six categories: leaves, downy mildew (Plasmopara viticola) infected leaves, powdery mildew (Erysiphe necator) infected leaves, flowers, young berries and ripe berries. "Flowers", "young" and "ripe berries" categories were significantly enriched in expressed P450 sequences whereas "leaves" and "downy mildew" categories were depleted (chi 2 test, p-value = 1E-15). In addition, we grouped the expression levels into four classes (no, low, medium or high expression). The "powdery mildew" category was found to be significantly enriched for highly expressed P450 sequences whereas the "leaves" category was depleted (chi 2 test, p-value = 9E-16).
Our annotation represents an improvement over the existing dataset for several reasons. The assembly of PN40024 genome is of better quality compared to the Pinot noir genome: it contains fewer gaps and a higher fraction of anchored contigs. The homozygosity of the genome not only enabled a better quality of the assembly, but also assured that most of the annotated sequences are individual loci and not allelic variants. This can partially explain a lower number of cytochrome P450 genes in our annotation-279-compared to the 315 genes reported on the Cytochrome P450 homepage. Additionally, the annotation on the Cytochrome P450 homepage classifies the sequences in only two categories, genes and pseudogenes, whereas we employed a more stringent classification into genes, partial genes, putative pseudogenes and pseudogenes.
The authors acknowledge the support of the  French Agence Nationale de la Recherche  to the  InteGrape  project ( ANR-13-BSV6-0010 ). The doctoral fellowship of  Tina Ilc  was funded by the  People Programme  ( Marie Curie Actions ) of the European Union's  7th Framework Programme  (FP7/2007-2013) under  REA Grant Agreement 289217. The doctoral fellowship of  Gautier Arista  was funded by  INRA  and the  Region Alsace. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Implemented Move Classes and Parameter Settings The methods presented in this paper have been implemented in a software package called MoMA (for Molecular Motion Algorithms), which includes modeling tools and algorithms to sample conformations and transition paths of biomolecules. The open-source code of MoMA is not available yet. Nevertheless, binaries can be provided upon request. We have implemented four move classes. Three of them, producing fixed-end moves, are based on the proposed approach. The other one is simply based on the variation of a single bond torsion at each iteration. More precisely: - The simplest class of trial moves, largely applied to sample the conformation of chain-like molecules, consists of perturbing a randomly selected bond torsion and then propagating the motion toward the end of the chain.
Test Systems We have chosen two different types of proteins to evaluate the performance of the move classes. The first one is a SH3 domain  [26]. More precisely, we have used the SH3 domain of obscurin, represented in Figure  2a. This is a small globular protein composed of 68 amino-acid residues. It presents a relatively rigid beta-barrel-like core and two flexible loops. Its crystal structure is available in the Protein Data Bank (PDB ID: 1V1C). The second test system is an intrinsically disordered protein (IDP). More precisely, we have considered a disordered region of the Sic1 protein  [27] , containing 77 residues. A model of this protein region is shown in Figure  2b. The model was generated using the Flexible-Meccano method  [28]  for sampling a statistically probable backbone conformation, and SCWRL4  [29]  for placing the side-chains.
Additional Results for Ubiquitin We performed additional tests on human ubiquitin, a well studied system with experimental and computational methods  [31]. The structure of this protein (PDB ID: 1UBQ) is shown in Figure  7. Our main goal was to compare, qualitatively, the results obtained with the methods proposed in this paper to those presented by Bottaro et al. [22]. The authors of this related work propose an extension of the concerted rotation move class, called CRISP (Concerted Rotations Involving Self-consistent Proposals), and use ubiquitin to demonstrate the ability of their enhanced MC method to reproduce structural variations around the native conformation, with a performance comparable to molecular dynamics simulations. Note that the CRISP move class is compatible with the approach presented in this paper, and could be implemented using the proposed framework.
The method applied in this work for solving the IK problem for a general 6R serial kinematic chain has been adapted from the solver proposed by Renaud  [40, 41]. This solver is based on algebraic elimination theory, and develops an ad hoc resultant formulation inspired by the work of Lee and Liang  [42, 43]. Starting from the matrix equation representing the IK problem, the elimination procedure leads to an 8-by-8 quadratic polynomial matrix in one variable. The problem can then be treated as a generalized eigenvalue problem, as was previously proposed by Manocha and Canny  [44] , for which efficient and robust solutions are available  [45]. Our implementation applies the Schur factorization from the linear algebra package LAPACK  [46]. Further details on the applied IK solver are provided in a technical report by  Renaud [41].
Copy-Paste-Modify versus Manual Abstraction To understand why copy-paste-modify might be easier, consider the function costFunction1 from Figure  1. This function (adapted from the OpenAge 4  project) computes the Chebyshev distance of two 2-dimensional coordinates. The implementation consists of a function header with formal parameters, a computation for the intermediate values dx and dy, and finally a computation of the actual Chebyshev distance from dx and dy. At some point, a developer decides that a different distance function is needed, describing the beeline distance between two points (i. e. , dx 2 + dy 2 ). Computing this distance requires almost the same steps as implemented in costFunction1, except for calling the standard library function std::hypot instead of std::max. At this point, the developer faces a choice: she can copy and paste the existing code into a new function (requiring only a copy, paste, and rename action) and modify the call from std::max to std::hypot (a trivial one-word edit), or she can manually transform the function costFunction1 into a more abstract version, such as costFunctionM (depicted on the bottom right in Figure  1 ).
Resolution Patterns for Merging Method-Level Code Clones To understand the commonly occurring differences in such clones in C++ code and to motivate specific approaches to merging method-level clones, we conducted a study of clone groups from top trending Open Source GitHub repositories. In this section, we describe the set-up of the study and report on our findings and insights. As we were not able to find a method-level clone detector for C++, we chose to implement a simple clone detector of our own based on the robust tree edit distance algorithm RTED  (Pawlik and Augsten, 2011). The tree edit distance represents a metric for the amount of edits required to transform one tree into another, which intuitively represents a basis for a metric of similarity for copy-paste-modified code. We adapted the existing implementation,  5  which works on trees of strings, to support the AST nodes of the Eclipse CDT.
In the following, we describe the top three types of differences we observed, using real examples from our sample set and the methods of merging we propose for those differences. For readability, we illustrate each case using very small examples that would probably not be worthwhile to merge in practice. We present more realistic examples in our evaluation in Sections 7 and 8. In the examples that follow, the code differences and the manually generated parameters and code segments are highlighted with a gray background. Difference type 1 -Constants. Consider the following functions from Google's Protobuf (normalized tree edit distance: 0. 25): 7 // Return the name of the A s s i g n D e s c r i p t o r s () // function for a given file. string G l o b a l A s s i g n D e s c r i p t o r s N a m e (const string & filename ) { return "protobuf AssignDesc " + F i l e n a m e I d e n t i f i e r ( filename ); } // Return the name of the ShutdownFile () // function for a given file.
string G l o b a l S h u t d o w n F i l e N a m e (const string & filename ) { return "protobuf ShutdownFile " + F i l e n a m e I d e n t i f i e r ( filename ); } These functions differ only in the string "protobuf AssignDesc " or "protobuf -ShutdownFile ", used to make up the beginning of the return value. As both strings are constants, we can create a merged version of these functions, in which the differences are resolved using a global variable or an extra parameter. The following code shows the result when using a global variable. The code includes both the merged function and the original functions modified to use the new merged version. return mergedMethod ( fileName , "protobuf ShutdownFile "); } Difference type 2 -Types. The following code shows another pair of functions from Google's Protobuf 8  that differ in a parameter type (normalized tree edit distance: 0.
25): string TextFormat :: F i e l d V a l u e P r i n t e r :: PrintInt32 (int32 val) const { return SimpleItoa ( val ); } string TextFormat :: F i e l d V a l u e P r i n t e r :: PrintUInt32 (uint32 val) const { return SimpleItoa ( val ); } The code below merges these functions using a template argument: template <typename T> string TextFormat :: F i e l d V a l u e P r i n t e r :: PrintInt (T val) const { return SimpleItoa ( val ); } string TextFormat :: F i e l d V a l u e P r i n t e r :: PrintInt32 (int32 val) const { return PrintInt<int32>( val ); } string TextFormat :: F i e l d V a l u e P r i n t e r :: PrintUInt32 (uint32 val) const { return PrintInt<uint32>( val ); } Difference type 3 -Statements. The following code shows a pair of functions from Facebook's RocksDB 9  that differ in various aspects of a statement containing a function call (normalized tree edit distance: 0.
To reduce the number of differences that our algorithm must resolve (and thereby produce less complex merged code in the end), we first align the node positions in all trees, based on the edit list. The procedure AlignPositions (Figure  4 ) normalizes the ASTs by finding insertions and shifting the positions of all nodes whose positions will be affected by the insertion to the right, if necessary. AlignPositions takes as input the set of ASTs and the edit list of each pair of ASTs, as obtained from RTED. AlignPositions outputs a data structure Position, which is the position list of every node in every AST. AlignPositions aligns the position of every node that was impacted by an insertion in some AST. AlignPositions needs to make sure the recalculation does not happen more than once for the same insertion when comparing one AST with multiple ASTs.
g. , if external libraries might reference them. This ensures that our approach is safe even when we lack whole-program information. Alternatively, an IDE could offer merge-substitution with automatic inlining, though we have not explored this option in detail. Below, we discuss the resolution patterns that we have implemented to evaluate our approach, and illustrate them with examples taken from open source projects hosted at GitHub. For each resolution pattern, we describe the merge resolution and the fix-up mechanism. We picked these four patterns based on the dominant kinds of differences that we observed in the samples from our earlier study in Section 3. We found these patterns to be sufficient to cover the merges that we had identified for all of these samples. In the examples below, the nodes highlighted in red (light gray in a black and white view) indicate the unique nodes in each function and the nodes highlighted in blue (dark grey in a black and white view) indicate the nodes produced by our merge resolution.
Pull requests and user study We have evaluated our approach by exploring the following research question: RQ: Are the abstractions performed by our algorithm of sufficient quality for production level code? In order to evaluate this question, we first looked for clone group candidates to merge. We explored top trending GitHub repositories, identified potential candidates for merging using our RTED-inspired clone detector, and used our approach to abstract the identified candidates. We finally submitted the abstracted code back to the developers through pull requests, to see how many of them were of sufficient quality to be introduced back into production code. We performed a total of 18 abstractions of clone groups from the top trending GitHub repositories that we identified previously and sent pull requests to the repositories from which we got the code.
Behavior preservation In order to validate if the refactored code broke the behavior of the repositories, we ran unit tests after the application of our tool on the selected clone groups, whenever possible. Out of the ten repositories that we submitted to, two had automatic unit tests that ran on our pull requests without observing any errors. In addition, we explored the unit test suites shipped with the software. For two of the projects, Google-protobuf and Cocos-2d, we found test suites that we were able to build and run and that exercised our merged code (which we verified by instrumenting the code); neither test suite observed any errors. Since the automatic pre-pull request unit testing and the manual unittesting post-merge did not cover all the accepted repositories, we also checked whether our changes had survived in the projects' code base between the pull request submission in December 2014 (for Phase 1, or February 2015 for Phase 2) and November 2017.
We set both systems to a detection threshold of 30%, the maximum number of differing AST nodes in our detector to 100, and the maximum number of differing lines to 10 in NiCad. NiCad detected 1382 clone pairs, while our approach detected 2042 clone pairs, including 987 (more than 70%) of the clone pairs reported by NiCad. We hypothesize that the difference in the results primarily stems from differences in accounting for thresholds (i. e. , lines of code vs. number of AST nodes). 8 Full Repository Evaluation -GIT While our earlier two sets of experiments illustrated the utility that our tool provides in realistic scenarios, we biased our selection through the use of a clone detector whose similarity metric is closely related to our merging algorithm. To explore whether this bias is a concern in practice, we ran a third experiment with a mainstream off-the-shelf clone detector.
Since we were not aware of any method-level clone detector for C++, we targeted our experiment to C code, allowing us to use NiCad  (Cordy and Roy, 2011)  as a clone detector. Since our system is based on the Eclipse CDT, we can also use it on C code, as long as we disable abstraction patterns that require C++-only language features. As target program we therefore selected one of the top trending C repositories on github, the Git 18 revision control system. At the time of our experiment, Git had a total of 6251 functions. We configured NiCad for our experiment as follows: Granularity: functions Max difference threshold: 30% Clone size: 20 -2500 lines NiCad detected 5 clone groups. In the following, we describe each clone group, as well as the results of merging the functions in each of these clone groups, and the insights that we obtained from each merge.
Clone Management Other work on clone management include tracking tools such as CloneBoard  (de Wit et al, 2009)  and Clone tracker  (Duala-Ekoko and Robillard, 2008). While CloneBoard provides the ability to organize clones and to some extent the ability to suggest the types of clones and possible resolution mechanisms, it lacks the ability to actually perform the resolution. Another approach to handling clones is linked editing  (Toomim et al, 2004) , which maintains the clones as they are, but allows editing of multiple clones simultaneously. This has the advantage of preserving code 'as is', but the disadvantage of requiring continued tool use for future evolution. Linked editing shares our view that copy-paste-modify is an effective way to evolve software, but disagrees on how clones should be managed; it is an open question which approach is more effective for long-term software maintenance.