esa-sceva/satcom-chunk-collection huggingface dataset:
# An Empirical Study on Software Defect Prediction Using CodeBERT Model  
[PERSON]  
1 The Key Laboratory on Reliability and Environmental Engineering Technology, Beihang University, Beijing 100191, China; [EMAIL_ADDRESS] (M.L.); xubiacorms@buaa.  
and [PERSON] [27]. Recently in 2019, BERT [28] improves natural language pre-training using mask-based objectives and a Transformer-based architecture. BERT has successfully improved many state-of-the-art results for various natural language tasks.  
Along with the development of the natural language model, recent research also shows great success in applying natural language models to artificial languages, especially programming languages. Such programming language models mainly embed local and global context [29], abstract syntax trees (ASTs) [30; 31], AST paths [32], memory heap graphs [33], and the combination of ASTs and data-flows [34; 35]. CodeBERT [36] is one of the latest released powerful language models that is a transformer pre-trained with a Bert-like architecture on open source repositories. It could support paired natural language and multi-lingual programming language tasks, such as code search and code documentation generation. Despite the great success of fine-tuning CodeBERT in downstream tasks, it remains unclear if CodeBERT could improve results in software defect prediction.  
In this paper, we investigate the feasibility of using the CodeBERT model for software defect prediction. Specifically, we propose four models based on CodeBERT: CodeBERT-NT, CodeBERT-PT, CodeBERT-PS, and CodeBERT-PK. We perform experiments to investigate the performance of CodeBERT-based models that \"understand\" source code syntax and semantic and design new prediction patterns that are specially designed for pre-trained language models like CodeBERT. The experimental data are available at [[https://gitee.com/penguinc/applsci-code-bert-defect-prediciton](https://gitee.com/penguinc/applsci-code-bert-defect-prediciton)]([https://gitee.com/penguinc/applsci-code-bert-defect-prediciton](https://gitee.com/penguinc/applsci-code-bert-defect-prediciton)). (accessed on 23 May 2021).  
Our contributions are as follows:  
* We are the first to introduce pre-trained programming language models for software defect prediction.
* We propose new forms of prediction patterns specially designed for pre-trained language models in software defect prediction.
* We discuss the reason why new forms of prediction patterns work in software defect prediction.
## 4 Layered KNN-SVM Methodology
### Continuous Variable Prediction Using KNN
#### 4.2.2 SVM for Binary Classification  
Supposing a training set \(Z=\{(\mathbf{x}_{i},y_{i}),\mathbf{x}_{i}\in\mathbb{R}^{N\times p},y_{i}\in\{\text{-}1, +1\},i=1,2,\ldots,m\}\), an optimal hyperplane separates the space formed by \(\{x_{i}\}_{i=1}^{m}\) into two subspaces, where \(\mathbf{x}_{i}\) can be divided into two classes in the space. SVM is the method to seek the separating hyperplane \(\omega^{T}\mathbf{x}+b\)= 0 with the largest margin, which can be expressed as an optimization problem:  
\[\begin{array}{ll}\min&\frac{1}{2}\|\mathbf{\omega}\|^{2}+\xi\sum_{i=1}^{m} \varepsilon_{i}\\ s.t.&y_{i}(\omega^{T}\mathbf{x}_{i}+b)\geq 1-\varepsilon_{i},\\ &\varepsilon_{i}\geq 0,\\ &i=1,2,\ldots,m,\end{array} \tag{11}\]  
where \(\xi\) is the penalty factor to balance the minimization of error cost and maximization of margin, and \(\varepsilon_{i}\) is the distance from \(\mathbf{x}_{i}\) to \(\mathbf{\omega}^{T}\mathbf{x}+b=y_{i}\).  
Problem (11) can be rewritten as Equation (12) using Lagrange multipliers \(\alpha_{i}\) and \(\beta_{i}\)  
\[L(\mathbf{\omega},\varepsilon,b,\alpha_{i},\beta_{i})=\frac{1}{2}\|\mathbf{\omega}\|^{ 2}\xi\sum_{i=1}^{m}\varepsilon_{i}-\sum_{i=1}^{m}\alpha_{i}\Big{[}y_{i}\Big{(} \mathbf{\omega}^{T}\phi(x_{i})+b\Big{)}-1+\varepsilon_{i}\Big{]}+\sum_{i=1}^{m} \beta_{i}\varepsilon_{i}, \tag{12}\]  
where \(\phi(\mathbf{x}_{i})\) is a mapping function that maps \(\mathbf{x}_{i}\) into a high dimensional space. SVM can efficiently perform a non-linear classification using \(\phi(\mathbf{x}_{i})\).  
Figure 4: Directed acyclic graph (DAG) for classifying \(L\) classes with (\(L-1\)) binary classifiers.  
After partial derivative of Equation (12), problem (11) can be transformed into the dual optimization problem:  
\[\begin{array}{ll}\min&-\sum\limits_{i=1}^{m}{{{i_{1}}}{{{i_{i}}}}}+\frac{1}{2} \sum\limits_{i,j=1}^{m}{{{\alpha_{i}}}{{{i_{j}}}}{{{y_{i}}}{{y_{j}}}}}\kappa({{ x_{i}}},{{x_{j}}})\\ s.t.&0\leq{{{\alpha_{i}}}}\leq C,\\ &\sum\limits_{i=1}^{m}{{{\alpha_{i}}}{{y_{j}}}}=0,\\ &i=1,2,\ldots,m,\end{array} \tag{13}\]  
where \(\kappa({x_{i}},{x_{j}})\) is the kernel function, \(\kappa({x_{i}},{x_{j}})=\phi({x_{i}})^{\mathsf{T}}\phi({x_{j}})\). Radial basis function is the popular kernel function used in SVM,  
\[\kappa({x_{i}},{x_{j}})=\exp\Bigl{[}-\|{x_{i}}-{x_{j}}\|^{2}/(2{\sigma^{2}}) \Bigr{]}. \tag{14}\]  
Problem (13) is efficiently solvable by quadratic programming algorithms. Then, \(\boldsymbol{\omega}\) can be solved by  
\[\boldsymbol{\omega}=\sum\limits_{i=1}^{m}{{{\alpha_{i}}}{{y_{i}}}\phi({x_{i}} )}. \tag{15}\]  
The SVM classifier function can be written as:  
\[C({\boldsymbol{x}^{\prime}})=\text{sgn}\left(\sum\limits_{i=1}^{m}{{{y_{i}}} {{\alpha_{i}}}\kappa({x_{i}},{\boldsymbol{x}^{\prime}})}+b\right) \tag{16}\]  
The calculation results of \(C({\boldsymbol{x}^{\prime}})\) is \(\pm 1\), where \({\boldsymbol{x}^{\prime}}\) can be classified into one of two classes. Then (\(L{-}1\)) SVM classifiers with DAG can be used in Figure 3 for categorical variable prediction in FR.
## 1 Inherited Retinal Dystrophies
Firstly, it is highly accessible for imaging and surgery, secondly it is enclosed allowing the administration of small amounts of the therapeutic product, and lastly, it is sequestered from the systemic circulation by the blood-retinal-barrier [11], thus providing it with an immuno-privileged status. For all the aforementioned reasons, retinal gene therapy has been the focus of much interest over the last ten years. This field has progressed the fastest for autosomal recessive IRDs, which are amenable to straightforward gene supplementation therapy [12]. However, it is important to now address the challenges associated with autosomal dominant disorders, and the evolution in genome-editing strategies, provides hope for the treatment of these IRD forms. Here, we focus on the autosomal dominant forms of the most common rod-cone dystrophy, retinitis pigmentosa (adRP). We present an overview of the more prevalent adRP causative genes and mutations, and discuss the most suitable genome-editing strategies. These examples serve as models that can be extrapolated beyond adRP to other autosomal dominant IRDs.
# Cooperative Power-Domain NOMA Systems: An Overview
## 1 Introduction
### Scope  
Most studies in the literature have emphasised on code-domain NOMA (CD-NOMA) [21; 22; 23; 24; 25], or power-domain NOMA (PD-NOMA) [22; 26], albeit without cooperation between users or devices. However, to the best of our knowledge, no comprehensive survey or an overview has taken into account a machine learning-based PD-NOMA, RIS-based PD-NOMA, and transmit antenna selection-based PD-NOMA systems. Hence, the present survey includes a majority of NOMA variants within the broad category of PD-NOMA and cooperative communication in PD-NOMA.
# Line-Detection Based on the Sum of Gradient Angle Differences
## 4 Line-Detection Experiments with Simulated Images
### Special Simulation Tests
In addition, in a comparison of the EC- and SGAD-based methods, shown in Figure 6a-e,g, the line-detection performances of the two methods are comparable for \(w=2,4,6\) with \(\sigma_{n}=10/255\); for \(w=2,4\) with \(\sigma_{n}=30/255\); and for \(w=2\) with \(\sigma_{n}=60/255\). However, as shown in the other panels of Figure 6, the performance of the SGAD-based method is better than that of the EC-based method in terms of the RMSEs of the detected line pixel locations.
## 5 Conclusions  
Our study recorded forty post-operative complications. The temporomandibular joint disorders and mandibular paraesthesia were the most common. The complications that differed most from the available literature are represented by nerve damage (22.41% versus 50% or more) [6; 11; 20]. The evaluation of the results obtained from other studies and/or reported by the most recent literature reviews indicates that despite a large number of possible complications associated with orthognathic surgery procedures, the vast majority of them do not affect aesthetic and functional results [21; 49]. Literature data suggests how these complications can be solved thanks to the joint work of surgeons, orthodontists, and operational teams [50; 51; 52; 53; 54; 55; 56]. The analysis of the literature and our clinical experience also led to evidence that the piezoelectric device might provide an innovative safe and effective technique compared to other rotating instruments due to the absence of macro-vibrations; moreover, it is easy to be used and may provide safer control and cutting, especially in complex anatomical areas [8; 11]. Its physical and mechanical properties have clinical advantages: precise cutting, saving of vital structures, and better visualization of the surgical field with time reduction [11]. Another point that we want to emphasize is that many reviews have indicated that most of the studies were obtained through case reports or case series [7; 40]. These types of studies currently do not provide reliable evidence. Therefore, more controlled clinical trials and good quality randomized controlled trials are required to provide better evidence in this field. Finally, the complication rate might be further reduced with an in-depth understanding of the pato-physiological aspects of the possible complications, careful patient assessment and treatment planning, and the use of proper surgical technique and instrumentation.  
**Author Contributions:** Conceptualization, G.R. and D.B.; Methodology, M.A.; Software, D.D.; Validation, L.C.C., G.R., and D.D.; Formal analysis, R.N.; Investigation, A.L.; Resources, A.L.; Data curation, A.L.; Writing--original draft preparation, G.R.; Writing--review and editing, L.C.C.; Visualization, R.N.; Supervision, M.A.; Project administration, A.L. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.  
**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and was approved by the Ethical Committee of San Raffaele Hospital, Milan, Italy ethics and the regional Ethical Review Board of Verona approved the study (107/1 NT/2017).  
**Informed Consent Statement:** Written informed consent has been obtained from all the patients to publish this paper.  
**Conflicts of Interest:** The authors declare no conflict of interest.
# A Low Phase Noise Frequency Synthesizer with a Fourth-Order RLC Loop Filter
## 4 Simulation Analysis
### Loop Capture Time Analysis  
Equation (12) describes the loop capture time \(T_{p}\). It indicates when the loop reaches the locked state from the starting unlocked state.  
\[T_{p}=\frac{\Delta\omega_{0}^{2}}{2\xi\omega_{n}^{3}} \tag{12}\]  
where \(\Delta\omega\) is the capture belt. It refers to the maximum initial frequency difference that can be locked through the frequency traction loop. \(\omega_{n}\) is the bandwidth of the loop filter, and \(\xi\) is the system damping coefficient. With a constant damping factor and capture band, the loop capture time is reduced by reducing the loop bandwidth. Figure 8 shows the loop capture time of the simulation.  
Figure 7: Phase noise simulation results: (**a**) passive fourth-order RC filter and (**b**) passive fourth-order RLC filter.  
As can be seen in Table 5, the capture times are the same at about 30 us. That is because both loops have the same bandwidth [23]. However, this is the ideal case of capturing time. Due to many external factors, the actual capture time can become larger.
# A Data Processing Framework for Polar Performance Diagrams
## 2 Problem Formulations and Theoretical Backgrounds
### Weather Routing -- An Application of Polar Diagrams
#### 2.5.1 Graph Based Algorithms  
The main idea of graph-based algorithms is to use a predefined set of positions (often a grid representation of a flat map) as graph nodes and a set of rules declaring between which nodes the vessel can travel to generate the edges(see Figure 4 or Figure 2 in [14] for an example).  
After assigning weights to the edges which correspond to a cost function, this results in a weighted graph. On this graph, standard shortest path algorithms, such as Dijkstra or A\({}^{\star}\) can be applied to find an optimal path.  
The question how to assign weights to the edges occurs naturally. Most of the established tools assume one can travel from A to B with constant weather conditions given at the start of the tour and calculate the costs under this assumption. We aim to include the calculations of edge weights for general cost functions in our framework considering the weather conditions at each point.
## 2 Development of a Prototype of Turist Visitor Office Room Using Spatial.io
### Selection of the Metaverse Platform: Spatial.io  
To develop and test an experimental online tourist office for the city of Valencia (Spain) as an extension of the existing tourist phone service, the authors of this work conducted a thorough analysis of popular Metaverse platforms (refer to Table 1) to determine the most suitable platform for the project.  
Spatial.io was selected as the platform for developing and testing an online tourist information office due to its impressive features and advantages over competitors. One of the significant strengths of the platform is its extensive device compatibility, supporting a wide range of devices such as PCs, smartphones, tablets, and VR hardware. This ensuresthat the online tourist office can be accessed by a broad user base, expanding its reach and accessibility.  
A standout feature of Spatial.io is its ability to generate realistic full-body 3D avatars from 2D images. This enhances the immersive experience for users, making their interaction with the online tourist office more engaging and lifelike. The platform's approach to avatar customization and the creation of shared 3D spaces is user-friendly, simplifying the process for non-technical users. This ensures that the online tourist office is not only immersive and captivating, but also easy to navigate and user-friendly.  
Additionally, Spatial.io enables the creation of curated social spaces, which is ideal for hosting the pilot touristic service. These features combined make Spatial.io an optimal choice for creating a virtual tourist office.  
In summary, Spatial.io stands out for its realistic and customizable avatars generated from user photos. It offers a user-friendly interface compatible with various devices and browsers. Furthermore, it provides comprehensive tools for hosts to design interactive and collaborative 3D experiences. Therefore, Spatial.io is considered the ideal choice for creating a virtual tourist office.  
Figure 1: Proposed Metaverse room design methodology for tourist visitor offices.
their sensory properties. In saffron, 3-Cyclohexadiene-1-carboxaldehyde, 2,6,6-trimethyl-, known as safranal, is the major volatile compound contributing to saffron aroma [4].  
One of the greatest interests of these bioactive compounds is due to their high antioxidant capacity and free radical scavenging activity. The generation of reactive oxygen species is a normal process in cells, but the uncontrolled generation and concomitant increase in reactive oxygen species (ROS) level in the body results in \"oxidative stress\", which is considered to be the main cause of various diseases [5]. The antioxidant substances play a role in protecting biological systems against the effects of oxidative processes on macromolecules. Many of those substances are plant-derived natural molecules, such as carotenoids or polyphenic compounds, that could contribute to preventing and fighting against several diseases in which ROS are involved [6].  
Currently, some studies have reported that petals, considered waste, contain valuable bioactive compounds such as flavonoids and polyphenols with potential antioxidant activity [7]. Therefore, saffron floral by-products could be natural antioxidant sources to be used in food, increasing the resource efficiency and the competitiveness of this sector. This waste raw material could be valorized through the development of innovative high added-value food ingredients, and this could potentially increase saffron demand on the market.  
The main objective of this study was to value both saffron and its floral by-products through their use as a source of bioactive extracts, studying the in vitro antioxidant capacity, the metal chelating activity, the composition of volatile compounds by GC-MS/MS, and the determination of crocetins esters and total crocins by HPLC in saffron and its floral by-products. As far as we know, no research has been published to date that delves into the composition of the volatile fraction in saffron floral by-products and into the changes in the volatile profile depending on the drying treatment used. Moreover, the results of this research would support the production of saffron as a more sustainable and profitable agronomic resource.
## 5 Conclusions  
In the study, a deep self-supervised learning model named M-SSL, based on encoder and contrast methods, is proposed on multisource remote sensing datasets that combine to represent learning and self-supervised discrimination methods to extract features for tree species classification in HSI and MSI more effectively than other methods. Inspired by encoder and generator methods, features extracted with MVAE and MAAE models, respectively, as pretext tasks in HSI and MSI, are discriminated to fuse by DCAM modules, so that effective information is enhanced. Then, the fused features from the different modules are applied to compare the learning characteristics of multisources, which were filtered through negative clustering samples, for the sake of reducing computing resources of the contrast learning method. The accuracy of the experimental results with M-SSL is close to 80%, which is 5% higher than that of other advanced methods. M-SSL can learn more representative information. It shows that the deep self-supervised learning method still has great potential in forest classification, especially for multisource remote sensing datasets.  
**Author Contributions:** X.W.: Conceptualization, methodology, software, data curation, funding acquisition, writing--original draft, writing--review and editing; N.Y.: methodology, writing--review and editing, supervision; E.L.: writing--review and editing, supervision; W.G.: writing--review and editing, supervision; J.Z.: funding acquisition, supervision; S.Z.: supervision; G.S.: supervision; J.W.: supervision. All authors have read and agreed to the published version of the manuscript.  
**Funding:** This research was supported by Basic Scientific Research Project of Heilongjiang Provincial Universities (Grant number 145109219), Basic Scientific Research Project of Heilongjiang Provincial Universities (Grant number 135409422). All the works were conducted at Forestry Intelligent Equipment Engineering Research Center.  
**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to still being used in a proceeding project.  
**Conflicts of Interest:** Authors declare no conflict of interest.
## 2 Materials and Methods  
### Data Description  
The source of information used for this research are the SCADA and alarm datasets as inputs to the model, and the maintenance task logs as ground-truth material to evaluate the effectiveness of the methodology. Two years of operation data for more than 100 turbines rated 2 MW and different manufacturers was available. Data has been received directly from the windfarm operator in the form of comma-separated values (csv) and text archives and uploaded in a SQL database.  
The dataset was split into a training and test set, maintaining a train/test split ratio of 70-30%. The last 9 months of data have been used as the test dataset, and the remaining data was used for training the algorithms. The utilized data is a real-life dataset of various windfarms operating under common conditions, it is not the results of a simulation. As a consequence, the data required thorough cleaning and pre-processing to get rid of inconsistencies due to sensors' errors and communication malfunctions.
## 3 Fundamentals of GLI-Split
### Communication between Upgraded GLI-Nodes  
In this section, we describe the communication between two GLI-nodes with upgraded networking stacks and how networking details are hidden from the transport layer. In the examples, GLI-node 1 establishes communication with another GLI-node with the DNS name hostX.exampleY.net. GLI-node 1 queries the DNS and obtains an IPv6 address. As the prefix of the returned address indicates a GLI-address, GLI-node 1 extracts the ID from that address. We distinguish whether both GLI-nodes are in the same domain or in different domains.
## 4 Results  
### Evaluation Metrics  
The prediction of the occurrence of strong earthquakes involves a two-class classification problem, and the evaluation of its prediction results typically employs a confusion matrix (Table 2).  
Specific judgments regarding the performance of earthquake occurrence prediction models are typically made by calculating four evaluation indicators based on the confusion matrix:  
\[Accuracy=\frac{TP+TN}{TP+TN+FP+FN} \tag{7}\]  
\[Precision=\frac{TP}{TP+FP} \tag{8}\]  
\[Recall=\frac{TP}{TP+FN} \tag{9}\]  
\[F1=\frac{2*Precision*Recall}{Precision+Recall} \tag{10}\]  
Additionally, ROC curves and area under curve (AUC) are used to illustrate the relationship between the aforementioned indicators and present the results.  
For magnitude prediction, mean square error (\(MSE\)), mean absolute error (\(MAE\)), and root mean square error (\(RMSE\)) are calculated to evaluate the prediction accuracy of the model. \(MSE\) represents the prediction error, \(MAE\) represents the average absolute error between the predicted value and the observed value, and \(RMSE\) reflects the degree of deviation between the predicted value and the true value. The formula to calculate each index is as follows:  
\[MSE=\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2} \tag{11}\]  
\[MAE=\frac{1}{n}\sum_{i=1}^{n}|y_{i}-\hat{y}_{i}| \tag{12}\]  
\[RMSE=\sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_{i}-\hat{y}_{i})^{2}} \tag{13}\]  
where \(n\) is the number of predicted values, \(y_{i}\) is the true value, and \(\hat{y}_{i}\) is the predicted value.  
\begin{table}
\begin{tabular}{c c c} \hline  & **Predicted Condition Is** & **Predicted Condition Is** \\  & **Positive** & **Negative** \\ \hline Actual condition is positive & True Positive (TP) & False Negative (FN) \\ Actual condition is negative & False Positive (FP) & True Negative (TN) \\ \hline \end{tabular}
\end{table}
Table 2: Confusion matrix of binary classification problem.