dataesr/scientific-paragraphs-categorization huggingface dataset:
B. RESULTS OF COMMENTS RELATED TO ISSUES AND PRS The Issues and PRs of our dataset have 96366 and 133865 comments in total, respectively. We find that 1296 Issues (represents 3. 7% (1296/34732) of Issues) and 8 PRs do not have any comments. The reason is that the PRs usually go through a review and discussion process represented by comments in the PRs, but this process is not mandatory to have happened for Issues. Another reason is that the corresponding contributors have the privilege of skipping the review and discussion stage because they are trusted enough by the community (e. g. , having enough expertise). On the other hand, we identify that some projects have a large number of comments, e. g. , 7185 comments related to Issues and 3206 ones related to PRs. This means that the corresponding Issues and PRs have got long discussions between the contributors to define their solutions.
D. RESULTS OF COMMITS RELATED TO ISSUES AND PRS The total number of commits attached to the identified PRs is 90740. We note that only 11 PRs do not have any commits. For instance, an Issue demands executing test cases. In this case, the results of these test cases are attached in the body of the corresponding PR, without changing the repository content (i. e. , do not need to add a commit). However, these can be neglected as they do not impact the quality of our dataset compared to 90740 commits. On the other hand, for other projects, we find a huge number of commits. For instance, we identify 3206 commits in one project. On average, we have 15. 82 commits connected to PRs per project. However, we have a large variation in terms of the number of commits per project based on the standard deviation (73.
VI. FEATURE SELECTION In our dataset, we identify four main features that can be used to distinguish between PR-Issue links. These features are Title, Body, Comment and Label of Issues and PRs. We select these features because they include much information in terms of the textual representation of their Issues and PRs. For instance, Titles provide short-term descriptions. Bodies include long-term descriptions. Comments represent the communication between the contributors. Labels refer to the types of Issues and PRs. In this section, we present the text size for each feature to better understand their information amounts. We calculate the text size in terms of word count. To do so, we first pre-processed the text by cleaning up text (e. g. removing unwanted words and punctuation marks). Then we count the number of words represented by each feature text.
A. FEATURE PRE-PROCESSING We apply the following standard pre-processing steps to clean the dataset. 1) Tokenization: we tokenize (split) each feature text to a list of words. The tokenization strategy considers space as a separator and discards punctuation marks and unnecessary symbols. To do so, we use the "spaCy" library to end up with useful words. 2) Stopword Removal: in this step, we remove all stopwords that frequently occur in the word list but do not carry any meaning (e. g. the). We use "NLTK " (Natural Language Toolkit) to remove all stop-words. 3) Stemming: it is a process of reducing words to their root form. We use one of the most popular stemming algorithms, "Porter stemmer", provided by NLTK library. Moreover, we convert the entire text into lower case characters VOLUME 10, 2022.
2) Body Word Count Figure  9  illustrates the distribution of the Body word sizes for Issues and PRs in our dataset. We can see that the Body sizes of the Issues are relatively vast bigger than the Body sizes of the PRs. The median value is 52 in the case of Issue, whereas the median value is 21 in the case of PR. The reason behind this size variation between Issue and PR is that the contributors need much more words to explain better a new feature or bug (Issue ) while needing fewer words to explain its solution (PR). in case of Issue, whereas commenting in many places in case of many PRs.
4) Label Word Count Figure  11  shows the distribution of the Label sizes for Issues and PRs in our dataset. As same as the Title sizes, the Label sizes are small compared with the Body and the Comment Sizes. The Label size range from one to 6 and from one to 3 for labelled Issue and PR, respectively. Figure  12  represents the top 20 Labels occurrences that occurred in more than 1000 times for all Issues and PRs in our dataset. We can obviously observe that the "bug" label has the biggest share (13518 times) among the other Labels. The next two Labels "enhancement", and "closed" occurred more than 5000 times, and the next two Labels "android", and "feature" occurred more than 3000 times. Finally, the rest Labels occurred between 1000 and 2000 times.
1) Title Similarity Results The results show that Levenshtein and BERT have a similar distribution as their box-plots are almost symmetric and identical. Based on their results, the titles of Issues and their linked PRs have similarity scores of 55. 3% and 56. 3% on average, respectively. As Levenshtein similarity is sensitive to the lengths of the texts, its results show that titles of the Issues and their linked PRs have akin lengths for most of the PR-Issue links in our dataset. Jaccard has a similar distribution as Levenshtein and BERT but with lower scores. Although Q1, Q2 and Q3 have different values, the number of elements (i. e. , similarity scores) between Q1-Q2 and Q2-Q3 are almost the same. The small difference in the position of the Jaccard box-plot compared to Levenshtein and BERT ones is because Jaccard ignores similarities coming from word frequency.
A. INTERNAL THREATS -In GitHub, many (sub)projects may be hosted in the same repository (called monorepo). If the monorepo includes non-Android projects in addition to an Android project, the proposed scraper tool fetches all projects together, and stores them in the dataset as target projects. Therefore, some PR-Issue links in the dataset are not from Android projects. However, after a focused inspection, this case does not frequently occur where only 6% of all PR-Issue links are from non-Android projects. -In our dataset, there are many projects with only one PR-Issue link which give an impression that such links are the reason behind the promising obtained results. However, these links form just 8% of the dataset's links which will not impact the obtained results. Also, either single and multiple PR-Issue link projects are from the same subject (Android projects), which means that textual information of single PR-Issue links have similar vocabularies with others.
B. EXTERNAL THREATS -The ground-truth dataset is collected around only Android applications in GitHub. In fact, we limit ourselves to Android applications because domain-based dataset support establishing a recommender system in that subject (Android) in contrast to the general dataset. Moreover, the domain-based dataset is to guarantee a biasfree dataset for their uses (e. g. especially in Machine Learning) because it shares a lot of similar vocabularies. -In this paper, the proposed scrapper tool is used to collect PR-Issue links only from Android applications. This gives the first-glance impression that the proposed scrapper tool can not be generalize to other types of applications. In fact, the proposed scraper tool works with any type of application hosted on GitHub.
A. IDENTIFY MISSING PR-ISSUE LINK In GitHub, PR-Issue links are usually manually established by contributors (developers), specifically in large projects. However, many incidents show that contributors might forget or be lazy to establish these links. There are only a tiny share of PR-Issue links are established, but a large portion of links are missed in the development history. However, manually recovering PR-Issue links from evolutionary development history is a challenging, time-consuming, and error-prone task, even for expert developers. For this challenge, our dataset can be used with ML and deep-learning approaches to extract the missing PR-Issue links. It can be used to feed the learning models and make them able to predicate the missing links. For example, our dataset can be used to determine the appropriate features that are useful to build a PR-Issue clustering model that aims to group PRs and Issues that should be linked with each other.
B. IDENTIFY DOCUMENTATION-CODE TRACEABILITY LINK Recovering the traceability links between documentation and source code, especially in legacy systems, is important for various software engineering tasks. It is mandatory for software comprehension, software maintenance and analysis, systematic software reverse, and software collaboration. The major challenge is to develop a methodology to recover these types of links which are rarely explicit. However, manually recovering Documentation-Code links from large software projects is a boring, challenging, time-consuming, and errorprone task. In this case, our dataset can help to define the relationships between the code and its documentation. It can be used to learn from ground-truth Documentation-Code links to recover the missing ones. We can consider that the textual description of our PR-Issue links represents the documentation and the code represented by commits.
C. RECOMMENDATION SYSTEM SOCIAL CODING PLATFORMS On a practical level, our dataset can be used by social coding platforms (e. g. GitHub) to recommend PR-Issue links for current software development. Our dataset can be helped to develop integrated tools for these social coding platforms that can assist the developer to easily establish these links during development. This kind of tool will help to reduce the number of missing links that contain much valuable information about software development and the evolution of current development, and document experiences and knowledge for future development.
X. RELATED WORK Links can be recovered from various software artefacts like requirements, source code and documentation  [25]. Many approaches are proposed to recover the links between software artefacts based on bug tracking (e. g. , Bugzilla) and version control (e. g. , Git) systems. There are many platforms that store these links between different software artefacts like GitHub and Bitbucket. These platforms record much information like PR-Issue links, Issue-Commit links, Bug-Commit links, and Documentation-Code links. Consequently, many research and tools have been proposed to identify this type of information. To the best of our knowledge, it has yet to be proposed a ground-truth dataset for PR-Issue links. In this section, we review the most closely related work in two aspects: the ground truth datasets for the links between software artefacts, and the applications and tools proposed based on these kinds of ground truth datasets.
A. GROUND-TRUTH DATASET A ground-truth dataset for linking bugs with their bug-fixes code (commits) is proposed by Bachmann et al. [26]. The dataset was extracted from The Apache HTTP web server project manually based on heuristic methods. It contains 256 bugs that are linked with 472 commits. However, the dataset is considered small and it was extracted only from one project. Therefore, the features that are extracted from this dataset have not been evaluated on other Bug-Commit links placed on other projects. Bird et al. [27]  expand the dataset by adding Bug-Commit links from another two Android projects, ZXing and OpenIntents. They also manually read change logs and the corresponding code commits to establish links between bugs and their linked commits. The added Android projects result from 236 new Bug-Commit links.
However, the dataset still considers small and confined to a few sample projects. Mehdi et al. presented a dataset for bots' comments in GitHub Issue and PR based on a manual analysis  [28]. The dataset represents 527 bots accounts with at most 100 comments for each account from 5,000 distinct Github accounts. A diverse dataset of Java Bugs is proposed by  [18]. The dataset comprises 1,158 bugs and patches classified into eight categories extracted from eight large and popular GitHub Java projects. However, the extraction process of the dataset from GitHub projects is evaluated manually by a few people (at most three persons) that are not involved in these projects. Therefore, it could have faulty information due to the evaluation decisions. Yue et al. [29]  proposed a dataset of duplicate PRs in GitHub using a semi-automatic approach.
The data contains 2,323 pairs of duplicate PRs collected from 26 popular opensource projects hosted on GitHub. Moreover, it includes duplicate relations between PRs, the meta-data and reviews of duplicate PRs. Another dataset for duplicate repositories on GitHub was proposed by Spinellis et al. [30]. The dataset presents 30 thousand duplicate, cloned, and forked GitHub projects. Fry et al. presented a dataset for real and alias developer IDs  [31]. The data shows that 14. 8 million alias developer IDs extracted from 38 million belong to 5. 4 million different developers (2 per developer). This data might be helpful in analyzing developer behaviour. However, the real and alias developer dataset are classified based on some assumptions that could not be valid for some portions of the dataset.
All of the aforementioned ground-truth datasets are extracted manually and based on heuristic methods. However, manually extracted datasets are time-consuming, costly, and error-prone. Moreover, the links between software artefacts are extracted from projects by another person who is not involved in the development of these projects. They are assumed to be links based on heuristic methods but not based on explicit notation provided by projects' contributors. Therefore, some of these links could be faulty. However, our ground-truth dataset is automatically extracted and based on explicit notation by the involved contributors themselves. This makes the links represented by our dataset Reliable and error-free.
B. GROUND-TRUTH DATASET APPLICATION A full-automation tool named ReLink was proposed by Wu et al. [32]. ReLink recovered Bug-Commit links based on three heuristics: the bug and its commit resolving time should be close to each other; the similarity of the bug and its commit reports; and the bug and its commit contributor should be the same. The ReLink results have been evaluated based on the Bug-Commit links ground-truth dataset (mentioned in the previous subsection) that is proposed by  [26]  and  [27]. Further improvements of ReLink were proposed by Bissyande et al. [33]  and tested with the same ground-truth dataset. Several approaches have been proposed to recover Issuecommit links using machine learning techniques. A heuristic approach named MLink proposed by Nguyen et al. [34]  used code heuristics to recover Issue-Commit links by analyzing the similarity of code fragments depicted in Issue reports and commits.
MLink also used the ground-truth dataset that is proposed by  [26]  to measure the accuracy of the results. Another approach proposed by Le et al. [35]  introduced RCLinker, a clustering tool to predict Issue-Commit links based on the metadata and the textual similarity for Issues and commits. For this purpose, the authors collected 609 Issues-Commits links dataset that was extracted from six Apache software projects. All these projects use JIRA as the issue-tracking system and Git as the version control. Many approaches have been proposed that use the same RCLinker dataset. Sun et al. proposed FRLink in  [36]  and PULink in  [37]  to recover Issue-commit links. FRLink filtered out irrelevant source code related to an Issue and added non-source documents as a feature to recover missing similarity links compared with other approaches.
PULink used positive links (the corresponding commit fixes the Issue) and unlabeled links (links from commits that are not found in the positive links) to achieve better recovery performance. Moreover, Likewise, Rath et al. [38]  recovered Issue-commit links by using different metadata and textual similarity features. For evaluation, they extracted their own dataset from different domains that utilized Git and JIRA. The dataset contains 4,182 Issue-commit links that are manually extracted from six projects.
XI. CONCLUSION AND FUTURE WORK In this paper, we proposed a novel approach to extract a ground-truth dataset for the links between PRs and Issues in collaborative software development repositories on GitHub. We developed the PR-Issue-Scraper tool that extracted 50369 PRs explicitly linked to 34732 Issues from 5742 Android projects on GitHub. We select four features, Title, Body, Comment, and Label, to better understand the relationships between the Issues and their linked PRs. After that, we analyzed the selected features by using four textual similarity metrics frequently used in NLP. We compute the lexical and semantic similarity between each feature pair of Issue and their linked PRs using Jaccard, Cosine, Levenshtine, and BERT. The results show that some feature similarities are sensitive to the text length, whereas other feature similarities are sensitive to the term frequency.
The number of issues in projects This helps to make our dataset not biased. Concerning the multiple cardinality relationships, the results show that roughly 32% (1 -(6. 04 ÷ 8. 77)) of Issues are linked to multiple PRs. This means that the solutions to these Issues have been proposed by multiple contributors at multiple levels. FIGURE 6: The distribution of Issue count over projects. Figures 6 and Figures 7 show the distributions of Issues andPRs over projects, respectively. These distributions show that we have 3105 and 3101 projects having only a single Issue or PR, respectively. However, these Issues and PRs only represent 8% (3105/34732) of Issues and 6% (3101/50369) of PRs of our dataset. On the other hand, the dataset includes some projects that have large numbers of Issues and PRs. For example, we have a project that includes 974 Issues and 1570 PRs.
These represent 2. 8% (974/34732) and 3. 1% (1570/50369) of Issues and PRs of our dataset, respectively. Considering another example from the data centralization, we have 51 projects that have between 100 and 200 PRs. In total, their PRs are 7022 which makes 13. 9% (7022/50369) of all PRs. Referring to the mean values, the average number of Issues and PRs are 6. 04 Issues and 8. 77 PRs, respectively. As a result, our dataset is representative for different types of PR-Issue links in terms of their co-occurrences together within the same projects starting from singular occurrences up to a large number of co-occurrences (e. g. , 947 co-occurrence Issues).
| RNA extraction and sequencing Total RNA was extracted from a 0. 5 to 1-cm-long branchlet piece. We coupled the TRIzol™ Plus RNA Purification Kit (Invitrogen) or the NZYol reagent (Nzytech) for tissue homogenization with the RNeasy Mini Kit (Qiagen) for RNA purification following the manufacturer's instructions. Residual DNA was removed using TurboDNAse (Invitrogen) or DNAse I (Qiagen). The quality, purity, and yield of the RNA was assessed using gel electrophoresis and a Qubit® Fluorometer and/or Nanodrop spectrophotometer. RNA-seq library preparation and sequencing was outsourced to BGI Tech Solutions in Hong Kong (100 bp paired-end DNBseq). A total of 34 individuals were processed for RNA-seq and the sequences deposited in NCBI's Sequence Read Archive (BioProject ID: PRJNA847883; BioSample accessions SAMN28899278-SAMN28899331).
| Sequence quality control and filtering Raw reads were preprocessed by BGI to trim adaptor sequences and low-quality reads (<Q20) using filter_fq (https://github. com/ great fireb all/filter_fq). The quality of filtered sequence data was then evaluated with two complementary quality control tools, FastQC v0. 11. 6 (https://www. bioin forma tics. babra ham. ac. uk/proje cts/fastq c/) and FastqPuri (Pérez-  Rubio et al. , 2019). Because these analyses detected the presence of overrepresented sequences corresponding to ribosomal RNAs (rRNAs), contaminant rRNA reads were filtered with FastqPuri using the bloom filter approach implemented in the function trimFilterPE (for details see  Pérez-Rúbio et al. , 2019  and the FastqPuri documentation page: https://github. com/jenge lmann/ Fastq Puri).
the assembled contigs based on evidence from the reads used to generate the assembly, allowing the filtration of "bad" (i. e. , poorly assembled) contigs from the assembly  (Smith-Unna et al. , 2016). Second, using the "good" contigs obtained with TransRate, we identified putative open reading frames (ORFs) with FragGeneScan  (Rho et al. , 2010)  and clustered the ORFs at 97% nucleotide sequence identity using VSEARCH  (Rognes et al. , 2016). Finally, the contigs of the clustered assemblies were aligned against NCBI's nonredundant database for Anthozoa (Taxonomy ID: 6101) using DIAMOND in BLASTX mode  (Buchfink et al. , 2014). Contigs with a top query hit were retained in the final transcriptome assemblies, with the remainder discarded as potential contaminants (e. g. , bacteria). The completeness of the final, curated transcriptome assemblies was evaluated with BUSCO v2/v3 by screening the recovery of single-copy orthologs (queried as predicted proteins) that are expected to be present across higher taxonomic groups  (Simão et al.
, 2015). The analysis was performed against the Metazoa reference database (OrthoDB 9) using the online platform gVolante v1. 2. 1 available at https://gvola nte. riken. jp/ (Nishimura et al. , 2017). Given that the transcriptome assembly for P. cf. grayi had the highest TransRate and BUSCO scores (Section 3), all subsequent analyses were conducted using this assembly as the reference.
Allele sequences were obtained from the resulting. vcf files with vcfutils. pl and seqkit fq2fa, aligned with mafft and trimmed with Gblocks in TranslatorX using a custom python script (https://github. com/cymon). The final set of loci used contained no gaps or missing data. The input. bam files were generated by mapping reads onto the reference transcriptome using the RSEM (v1. 2. 31) wrapper script and Bowtie2  (Langmead & Salzberg, 2012; Li & Dewey, 2011). Locus alignments were assigned to "pseudoalleles" (i. e. alleles derived from in silico phasing of short read data stored in the alignment files) and analyzed by maximum likelihood in IQ-TREE 2  (Minh, Schmidt, et al. , 2020). A partitioned analysis was performed by fitting a separate evolutionary model of sequence evolution for each locus using ModelFinder and based on BIC score.
Gene and site concordance factor (CF) analysis, which estimates the percentage of gene trees (gCF) or sites (sCF) which agree with the consensus species tree, was performed following  Minh, Hahn, and Lanfear (2020)  and the recommendations at http:// www. iqtree. org/doc/. Species delimitation on the same multi-individual dataset as used for the species tree was performed using Species bOundry Delimitation using Astral method (SODA;  Rabiee & Mirarab, 2020)  based on quartet tree analysis with  ASTRAL-III (v5. 7. 3; Rabiee et al. , 2019; Zhang et al. , 2018). The input ML gene trees from IQ-TREE 2 were used, and a species tree was estimated with ASTRAL-III. SODA uses the mapping of quartet tree constellations onto an ASTRAL species tree with all individuals to assess if/where coalescence of the tree topology is completely random.
The species Paramuricea biscaya was used as an outgroup in all the analyses. We downloaded RNA-seq data for four samples from NCBI's SRA (Accessions: SRX4389727-SRX4389730;  DeLeo et al. , 2018). Sequence data were filtered and mapped as described above. Finally, to extend the taxonomic coverage of the analysis to other Paramuricea spp. , we conducted an ML tree reconstruction in IQ-TREE 2 with the mitochondrial gene mt-mutS. Sequence data were retrieved from NCBI Genbank and from. bam files generated here by mapping the RNA-seq reads onto the reference mitogenome of P. clavata using RSEM (Genbank Accession: NC_034749) for a subset of the samples (for additional information see Figure  S1 ). New mt-mutS sequences were deposited in NCBI GenBank (Accession numbers: ON804207-ON804214). (Boavida, Assis, et al.