dataesr/scientific-paragraphs-categorization huggingface dataset:
We consider Jaccard Similarity Index (JSI), tf-idf (TF), Doc2Vec (DW), Fast-Text (FT), BERT, and USE for computing requirements similarity. Then, we consider the software that implements similar requirements by means of explicit trace links, and we use JPLag to measure the similarity between the software. Finally, we compute the correlation between the different requirements similarity measures and the software similarity. It is worth noting that no data about the actual similarity between requirements and software pairs, as evaluated by humans, were available beforehand as ground truth. The only data available in our dataset are the requirements, the software realizing those requirements, and the trace links between requirements and their implementation. We use the trace links to identify if, given two highly similar requirements pairs, the software linked to the requirements is also similar.
Data collection procedure (RQ1) Figure  2  shows a high-level view of the procedure followed for data collection. One project manager from the company was involved in validating our procedure. Two requirements documents belonging to two projects (shown in Fig. 2 , project A and B, in the following) of the PPC team were considered for this study. The projects were selected based on convenience of the project manager to represent a potential scenario of requirements-based software reuse. The requirement documents of the two projects were exported from a requirements management tool, and therefore, non-requirement-related information, such as headings and definitions, were also included. As shown in Fig. 2 , the documents were subjected to cleaning to remove entries that are not requirements but are additional supporting information, such as headings and definitions.
As a result, we consider all of the 254 requirements-112 from project A and 142 from project B-selected out of 265 entries. Table  1  outlines the data about the two projects with information on requirements and lines of code. As shown in Fig. 2 , the requirements were used as an input to the language models for similarity computation with and without pre-processing. Pre-Processing. The pre-process pipeline takes the requirements text and removes English stop-words from it. After the removal of the stop-words, each token of the requirement text is tagged with Part-of-speech (POS) tags to guide the lemmatization. The pre-trained spaCy model 2  is used to lemmatize the text of the requirement. The output of this pipeline is the pre-processed text of the requirement. The dataset before and after pre-processing is shown in In the remainder of this paper, the language model's variants where pre-processing is applied are referred to as "p" followed by the name of the language model.
Language Models In the following, we report the settings for the language models applied in our study and the specific implementations adopted in the case of pre-trained models. -JSI The computation of the index has been implemented by the authors. -TF The model is configured to build the term-document matrix on project B and then uses principal component analysis (PCA)  [53]  to select the top features based on the explained variance of 95% from the matrix. The minimum and maximum document frequencies are set to 6 and 0. 5, respectively. We consider n-grams ranging from 1 to 8. -DW the pre-trained Doc2Vec model available in Gensim data  3. is used. The model has a vector size of 300, with a minimum frequency set to 2. The model is trained on the English Wikipedia documents resulting in a vocabulary size of 35,556,952.
-FT we use the pre-trained FT model available in Gensim data. The model has a vector size of 100 with a minimum frequency set to 1. The model is trained on the English Wikipedia documents on the sub-word level. This results in a vocabulary size of 2,519,370. Both FT and DW are based on the skip-gram neural network architecture  [64] , known for contextual word prediction. -BERT we use the uncased pre-trained BERT model by Google Research  [30]. The model has 12 layers and a vector size of 768. We use the BERT implementation available in BERT-as-a-service  4. -USE we use the English pre-trained model available in the TensorFlow hub  5. The model is trained on a variety of data sources and produces a feature vector of 512 dimensions. The model does not require the input text to be pre-processed.
Requirements Similarity Computation We compute the similarity and identify similar requirements based on the different language models with and without pre-processing. Note that for DW, FT, USE, and BERT, the hyper-parameters are not in our control and come from the original pre-trained models. The input to each language model is a requirement, and the output is a vector. The similarity between each pair of requirements' vectors is calculated using the cosine similarity metric with the scipy implementation available in the library scipy  [77]. This procedure applies to all language models, except JSI. For JSI, a pair is created by computing the index directly on the text of the requirements. A pair of requirements is created by retrieving the most similar requirement from project B for each requirement in project A, as shown in Fig.
2  as Similar Pairs of Reqs. We have chosen this approach as project B was developed before project A. Therefore, this setting mimics a software retrieval scenario in which requirements from project A are regarded as queries, and those from project B are database items to be retrieved. It should be noted that, as in a real retrieval scenario, requirements considered as "most similar" according to a particular metric may not actually be similar in practice from the user's viewpoint. As in this study, we are concerned with automatic requirements similarity; this aspect is not currently addressed. Given the total number of requirements, we select the top-50 similar pairs using cosine similarity. We choose the top-50 pairs because it is a suitable number for a sample size on which to apply statistical tests, and, at the same time, does not cover all possible, likely unrelated, requirements pairs which would not be relevant in a retrieval scenario, (see Sect.
5. 1 for a more detailed explanation). Code Generation. In the studied projects, the requirements are realized in Simulink models, and code is generated from the models for deployment. In our setting, we compute software similarity on the generated code. Computing software similarity directly on the Simulink models might lead to interesting results and possibly different results. However, the tool support for computing Simulink model similarity is limited. In addition, in many scenarios such as safety certification and deployment, the central artifact is the generated source code, as this will be the one that will actually run on the system. Furthermore, in the studied settings, engineers use Simulink Embedded Coder with the MinGW64 gmake tool-chain to generate code from the models for deployment and testing.
Therefore, as shown in Fig. 2 , we mimic the same setup and generated code 6 from the Simulink model to be considered for software similarity computation. The related code realizing each requirement was traced and moved to directories tagged with the requirement's identifiers to ease the Source Code Similarity computation (shown in Fig. 2 ). In the future, we also aim to extend this analysis to the similarity computed on Simulink models. Source Code Similarity Computation. Our software similarity pipeline takes pairs of requirement identifiers as input and copies each pair's code to separate folders 7. The pipeline then uses JPLag to compute the similarity between the pair of source codes. To compute the similarity between the source code of the two requirements, we use the JPLag's Java ARchive (JAR) with C/C++ as a language parameter  [75].
JPLag was originally designed to detect plagiarism in students' assignments and thus is able to detect semantically similar code. Note that JPLag ignores code comments and white spaces and scans and parses the input programs to convert the programs into string tokens. JPLag then uses a greedy version of the string tiling algorithm to compute the similarity between the tokens of the source code. The similarity number is basically the percentage of similar tokens in the pairs of source codes. The output of this pipeline is the software similarity values between 0 and 100, later converted to a range between 0 and 1 for the input pairs. The requirements similarity between pairs of requirements and the software similarity for the corresponding software modules act as the quantitative data collected to answer RQ1.
The requirements similarity is reported on the X-axis, while the software similarity is plotted on Y-axis and is calculated using JPLag. The blue line is the trendline between the two variables, giving insights into the relationship between them. In all cases, as can be seen from the trendlines, there could be a positive association between the two variables. However, for some language models such as BERT, and FT, the variation between its similarity and software similarity is very high. For example, many pairs with varying software similarities can be seen with a requirements similarity of 0. 97 in FT. The trend line for USE and pJSI seems to show more association compared to others. In Fig. 6 , we also visualize the interquartile range (IQR), mean, and outliers in our variables. The boxplot shows that the software similarity for most requirement pairs stays above 70%.
The boxplot also gives an overview of the requirements similarity ranges (embedding space) for the different language models. For instance, BERT tends to produce a cosine similarity value between 90% and 100%, while for FastText (FT), the embedding space ranges between 75% to 100%. This indicates that for some metrics, and especially DW and BERT, one may need to apply some scaling on the similarity values, if they want to use them to clearly distinguish between similar and non-similar requirements. On the contrary, for JSI and pJSI, requirements similarity is particularly low and shows a wider range (between 0. 3 and 0. 6). This measure captures exclusively the lexical similarity in terms of word overlaps, but this is nevertheless sufficient to identify similar software, as software similarity is still in the high ranges.
The Average rho column is the average correlation coefficient for the top-30 to top-50 pairs, which ranges from -1 to 1. As it can be observed, there is a positive association between the requirements similarity and software similarity for all the language models, and the results are significant for = 0. 05. Further- more, for JSI, pTF, FT, pBERT, and USE, the correlation is moderately high (i. e. , between 0. 5 and 0. 7)  [49]. The highest average correlation ( = 0. 60 ) is achieved with pBERT, the transformer-based BERT model with pre-processing, followed by word2vec-based FT (with average = 0. 59 ), and then the traditional tf-idf with pre-processing (with average = 0. 58 ), USE (0. 522), and JSI (0. 521). As anticipated by the analysis of the descriptive statistics, DW shows the lowest correlation ( = 0.
Some requirements which might be very similar but in the context they are used, might be a small part of a function with many other different requirements affecting the implementation. Theme 2. Perception of similar software. All participants agree that software modules are perceived to be similar if they have identical inputs/interfaces, do analogous processing on the inputs, and provide similar outputs. As for analogous processing, it was clarified that they intend that the Simulink models appear to perform the same manipulation on the data, regardless of the type of representation (e. g. , state machines vs. Simulink block diagrams). We look for the identical input signals, the same description of processing and output, if so, then probably the software are very similar. Theme 3. Association between similar requirements and similar software.
RQ1. To what extent is requirements similarity correlated to the similarity of their linked software in the context of requirements-based software reuse? The correlation analysis (presented in Table  2 ) shows that for all language models, we were able to find a positive correlation between requirements similarity and software similarity. In particular, the average across the top-30 to top-50 pairs shows a moderately positive correlation between the two variables for JSI, pTF, FT, USE, and pBERT (shown in bold text in Table  2 ). Results also show that pre-processing improves the correlation for some language models except JSI, DW, FastText and USE. Looking at the individual language models, we can make the following observations. The emerging pre-trained deep learning transformer-based language model pBERT outperforms other state-of-the-art language models in terms of average correlation coefficient, with an average of 0.
60 with respect to software similarity. It is followed closely by the word2vec-based pre-trained FastText language model with an average of 0. 59. This suggests that these semantic-rich language models are both appropriate for our context. On the other hand, the traditional term frequency-based IR approach tf-idf with pre-processing also shows a promising a correlation with an average of 0. 58. Surprisingly, the simple stringlevel JSI lexical similarity approach also shows a moderately positive correlation (with an average of 0. 52) with software similarity. The good performance of JSI and tf-idf can be explained by the limited vocabulary and limited terms used in the requirements of the two projects, as typical in the RE domain  [41]. Also, the projects come from the same team, and the issues of synonyms previously observed may have less prominence in the considered documents.
Therefore, lexical and IR metrics, which give relevance to single terms, can be effective in this context and might have performance that is only slightly lower compared to those of more semantically rich models. However, in tasks where requirements might be sharing fewer terms-e. g. , in case of comparison between high-level customer requirements and low-level specifications-the benefit of language models capturing semantics could become more evident. The worst performance is obtained with Doc2Vec (DW and pDW). This language model works well with long documents and might not be a good candidate for RE tasks, as single requirements are typically short, but maybe beneficial in contexts where the comparison is performed between entire requirements documents. Overall, the observed performance of the BERT language model further justifies its increasing use in RE tasks  [48, 59] , and suggests that, together with FastText, it is an appropriate choice for requirements retrieval.
RQ1. There is a positive correlation between automatically computed requirements similarity and software similarity measured with JPLag. On average, the BERT language model with preprocessing (pBERT) is the one that best represents software similarity in the considered context, followed by FastText and tf-idf with pre-processing (pTF). Other observations in relation to RQ1 can be derived from the analysis of the descriptive statistics. The trendlines in Fig. 5  visually confirm that the results from all the language models could have a positive association with software similarity. Furthermore, from the results shown in Fig. 4 , it can be seen that even in worst cases, requirements-based code retrieval would result in retrieving some code with a high software similarity (that is more than 80%), which can be therefore a good candidate for reuse.
Though the correlation between requirements similarity and software similarity is lower with respect to other metrics ( = 0. 52 ), the top-50 similar requirements pairs are linked to highly similar software. Indeed, in the considered sample, no software pairs are retrieved with a similarity that is lower than 60%, as shown in Fig. 4. This suggests that, even in case of lower requirements similarity, a retrieval system based on USE may identify relevant software. Further research considering manual annotations is needed to further assess these results and understand how to better exploit this characteristic of USE. Observation. Different language models tend to exhibit different behaviors when measuring requirements similarity and also when used to retrieve software. FT and pBERT appear to identify nuanced semantic requirements similarities, but could in principle, also retrieve software RQ2.
We therefore foresee that approaches for detecting inconsistencies in the granularity of trace links could improve the requirements-driven software reuse process. pre-trained models that are heavily dependent on the quality of the training dataset. The quality of the results might differ if different pre-trained language models are considered. To mitigate potential threats to construct validity, we selected a diverse set of approaches (see Sect. 3) to represent the semantics of the requirements. Other construct validity threats might be relevant to the design of our focus group instrument. To mitigate potential threats to construct validity, we designed the focus instrument using terminologies known to the participant. The instrument was refined over several iterations and through a pilot focus group session.
Furthermore, in our study, we mimic the original process at the company and generate code from Simulink models instead of using manually written code. Thus, when referring to software or code, we consider automatically generated one. Analyzing the relationship between requirements similarity and software similarity computed on manually written code might produce different results. Indeed, automatically generated code tends to appear as code always written by the same programmer  [38] , which can lead to a higher similarity between automatically generated software pairs than manually written ones. On the other hand, railway companies follow specific internal coding guidelines to ensure high-quality code. Therefore, the similarity of manually written code could also be higher with respect to other, less regulated contexts.
External Validity. Our results are based on data provided by one company using a dataset of two projects developed by one team. The qualitative results are based on the perceptions of experts from two teams of the same company, and are limited to the viewpoint of five experts. Five participants are within the recommended number of participants for focus groups in software engineering, which is 3 to 12 participants  [55]. Furthermore, the two teams considered for this study use different requirements engineering practices Observation. The reuse process can be enhanced with improved requirements similarity computation, but also with other automated practices. These include requirements extraction from call for tenders, variability mining, extraction of requirements dependencies, and trace link recovery.
The company follows a MATLAB/Simulink-based modeldriven development process, similar to other companies and domains  [39, 62] , where the system is modeled, and code is generated automatically to reduce development efforts. We argue that similar results can be obtained in domains with highly structured and waterfall-like processes, as the railway one. Further studies are needed, considering other abstraction levels of requirements and in different companies and domains, to generalize the results. Reliability. Finally, we address the threats to the reliability of our results by providing enough details on the experimental setup and implementation. We designed the study following well-established guidelines, involving authors from diverse backgrounds. In addition, we also provide the R script and the similarity values between the pairs for replication purposes 11.
Conclusion and future work Content-based recommender systems for code retrieval typically use requirements as queries to identify previously developed requirements, and in turn, reuse their implementation. These systems take the operational assumption that similar requirements can be used as proxies to retrieve similar code that can be reused with limited adaptation. This paper presents an empirical investigation on the relationship between requirements similarity and code similarity in the context of a large railway company. The goal of the work is to explore to which extent similar requirements can be considered as a proxy to retrieve similar code. We consider two related projects in the company. We use different seminal NLP-based language models to represent the requirements and support similarity computation.
org/ 10. 5281/ zenodo. 49160 71. provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. org/licenses/by/4. 0/.
with the number of SLOC) implementing these requirements shown in the SLOC column. The considered requirements are relatively long in terms of words-about 64 words, on average. They are indeed composed of more than one sentence, and they can be considered to have a medium degree of abstraction. They are technical requirements, so more lowlevel compared to business requirements. At the same time, they are still at the system level of abstraction, and they are not broken down yet into module-level requirements, which have a closer relation with the code. We consider these requirements because they are the ones that the engineers of the PPC team typically use for identifying software reuse opportunities. A real requirement from the dataset before and after preprocessing is shown below.
Accurate determination of skeletal maturation indicators is crucial in the orthodontic process. Chronologic age is not a reliable skeletal maturation indicator, thus physicians use bone age. In orthodontics, the treatment timing depends on Cervical Vertebral Maturation (CVM) assessment. Determination of CVM degree remains challenging due to the limited annotated dataset, the existence of significant irrelevant areas in the image, the huge intra-class variances, and the high degree of inter-class similarities. To address this problem, researchers have started looking for external information beyond current available medical datasets. This work utilizes the domain knowledge from radiologists to train neural network models that can be utilized as a decision support system. We proposed a novel supervised learning method with a multi-scale attention mechanism, and we incorporated the general diagnostic patterns of medical doctors to classify lateral X-ray images as six CVM classes.
CVM stages can be estimated by the morphological description of vertebrae spines (C2, C3, and C4). CVM stages have been described into six stages correlating with morphological modifications of the vertebral shapes and estimated time lapse from the mandibular growth peak. Manual analysis is time-consuming and demanding for expert graders, which is also prone to yield subjective results. Consequently, an automatic and reliable CVM classification is required for efficient diagnosis. Automatic CVM stage estimation can decrease diagnosis time and treatment cost. In the medical field there are more challenges due to three main reasons: 1-In comparison to popular natural image datasets such as ImageNet, the medical image dataset's size is too small. This problem can cause overfitting. 2-Medical images are noisy, their boundaries are ambiguous, and ROI is located in a small part of the image.