Reviewerly/exHarmony huggingface dataset:
Automating Code-Related Tasks Through Transformers: The Impact of Pre-training

Transformers have gained popularity in the software engineering (SE) literature. These deep learning models are usually pre-trained through a self-supervised objective, meant to provide the model with basic knowledge about a language of interest (e.g., Java). A classic pre-training objective is the masked language model (MLM), in which a percentage of tokens from the input (e.g., a Java method) is masked, with the model in charge of predicting them. Once pre-trained, the model is then fine-tuned to support the specific downstream task of interest (e.g., code summarization). While there is evidence suggesting the boost in performance provided by pre-training, little is known about the impact of the specific pre-training objective(s) used. Indeed, MLM is just one of the possible pre-training objectives and recent work from the natural language processing field suggest that pre-training objectives tailored for the specific downstream task of interest may substantially boost the model's performance. For example, in the case of code summarization, a tailored pre-training objective could be the identification of an appropriate name for a given method, considering the method name to generate as an extreme summary. In this study, we focus on the impact of pre-training objectives on the performance of trans-formers when automating code-related tasks. We start with a systematic literature review aimed at identifying the pre-training objectives used in SE. Then, we pre-train 32 transformers using both (i) generic pre-training objectives usually adopted in SE; and (ii) pre-training objectives tailored to specific code-related tasks subject of our experimentation, namely bug-fixing, code summarization, and code completion. We also compare the pre-trained models with non pre-trained ones and show the advantage brought by pre-training in different scenarios, in which more or less fine-tuning data are available. Our results show that: (i) pre-training helps in boosting performance only if the amount of fine-tuning data available is small; (ii) the MLM objective is usually sufficient to maximize the prediction performance of the model, even when comparing it with pre-training objectives specialized for the downstream task at hand.
Statistical evidence and algorithmic decision-making

Abstract The use of algorithms to support prediction-based decision-making is becoming commonplace in a range of domains including health, criminal justice, education, social services, lending, and hiring. An assumption governing such decisions is that there is a property Y such that individual a should be allocated resource R by decision-maker D if a is Y. When there is uncertainty about whether a is Y, algorithms may provide valuable decision support by accurately predicting whether a is Y on the basis of known features of a . Based on recent work on statistical evidence in epistemology this article presents an argument against relying exclusively on algorithmic predictions to allocate resources when they provide purely statistical evidence that a is Y. The article then responds to the objection that any evidence that will increase the proportion of correct decisions should be accepted as the basis for allocations regardless of its epistemic deficiency. Finally, some important practical aspects of the conclusion are considered.
SensorGAN: A Novel Data Recovery Approach for Wearable Human Activity Recognition

Human activity recognition (HAR) and more broadly, activities of daily life recognition using wearable devices, have the potential to transform a number of applications including mobile healthcare, smart homes, and fitness monitoring. Recent approaches for HAR use multiple sensors on various locations on the body to achieve higher accuracy for complex activities. While multiple sensors increase the accuracy, they are also susceptible to reliability issues when one or more sensors are unable to provide data to the application due to sensor malfunction, user error, or energy limitations. Training multiple activity classifiers that use a subset of sensors is not desirable since it may lead to reduced accuracy for applications. To handle these limitations, we propose a novel generative approach that recovers the missing data of sensors using data available from other sensors. The recovered data is then used to seamlessly classify activities. Experiments using three publicly available activity datasets show that with data missing from one sensor, the proposed approach achieves accuracy that is within 10% of the accuracy with no missing data. Moreover, implementation on a wearable device prototype show that the proposed approach takes about 1.5 ms for recovering data in the w-HAR dataset, which results in an energy consumption of 606 μ J. The low energy consumption ensures that SensorGAN is suitable for effectively recovering data in tinyML applications on energy-constrained devices.
Adaptive weighted fuzzy clustering based on intra-cluster data divergence

Fuzzy C-means clustering (FCM) approach is an effective method for clustering and has been successfully applied in numbers of real-world problems. In this paper, we propose an improving adaptive weighted FCM based on data divergence, with the merits of three aspects: 1) to avoid randomization of cluster centers, we propose a new cluster centers initialization method; 2) we present an adaptive parameter reflecting the changes of intra-cluster data divergence in the process of cluster formation from iteration to iteration for correcting the unreasonable factors resulting from the changes timely; 3) we propose a new data weighting method. By integrating the adaptive parameter and feature weighting method, we propose a novel adaptive objective function, by which the updating iterative formulas of the membership degrees, the feature weights and the cluster centers are obtained. Experimental results have shown that the novel clustering approach put forward can improve the clustering performance effectively.
Generating Realistic and Diverse Tests for LiDAR-Based Perception Systems

Autonomous systems rely on a perception component to interpret their surroundings, and when misinterpretations occur, they can and have led to serious and fatal system-level failures. Yet, existing methods for testing perception software remain limited in both their capacity to efficiently generate test data that translates to real-world performance and in their diversity to capture the long tail of rare but safety-critical scenarios. These limitations are particularly evident for perception systems based on LiDAR sensors, which have emerged as a crucial component in modern autonomous systems due to their ability to provide a 3D scan of the world and operate in all lighting conditions. To address these limitations, we introduce a novel approach for testing LiDAR-based perception systems by leveraging existing real-world data as a basis to generate realistic and diverse test cases through mutations that preserve realism invariants while generating inputs rarely found in existing data sets, and automatically crafting oracles that identify potentially safety-critical issues in perception performance. We implemented our approach to assess its ability to identify perception failures, generating over 50,000 test inputs for five state-of-the-art LiDAR-based perception systems. We found that it efficiently generated test cases that yield errors in perception that could result in real consequences if these systems were deployed and does so at a low rate of false positives.
Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation

Deep learning (DL) plays a more and more important role in our daily life due to its competitive performance in industrial application domains. As the core of DL-enabled systems, deep neural networks (DNNs) need to be carefully evaluated to ensure the produced models match the expected requirements. In practice, the de facto standard to assess the quality of DNNs in the industry is to check their performance (accuracy) on a collected set of labeled test data. However, preparing such labeled data is often not easy partly because of the huge labeling effort, i.e., data labeling is labor-intensive, especially with the massive new incoming unlabeled data every day. Recent studies show that test selection for DNN is a promising direction that tackles this issue by selecting minimal representative data to label and using these data to assess the model. However, it still requires human effort and cannot be automatic. In this paper, we propose a novel technique, named Aries, that can estimate the performance of DNNs on new unlabeled data using only the information obtained from the original test data. The key insight behind our technique is that the model should have similar prediction accuracy on the data which have similar distances to the decision boundary. We performed a large-scale evaluation of our technique on two famous datasets, CIFAR-10 and Tiny-ImageNet, four widely studied DNN models including ResNetl0l and DenseNetl21, and 13 types of data transformation methods. Results show that the estimated accuracy by Aries is only 0.03% - 2.60% off the true accuracy. Besides, Aries also outperforms the state-of-the-art labeling-free methods in 50 out of 52 cases and selection-labeling-based methods in 96 out of 128 cases.
Measuring Secure Coding Practice and Culture: A Finger Pointing at the Moon is not the Moon

Software security research has a core problem: it is impossible to prove the security of complex software. A low number of known defects may simply indicate that the software has not been attacked yet, or that successful attacks have not been detected. A high defect count may be the result of white-hat hacker targeting, or of a successful bug bounty program which prevented insecurities from persisting in the wild. This makes it difficult to measure the security of non-trivial software. Researchers instead usually measure effort directed towards ensuring software security. However, different researchers use their own tailored measures, usually devised from industry secure coding guidelines. Not only is there no agreed way to measure effort, there is also no agreement on what effort entails. Qualitative studies emphasise the importance of security culture in an organisation. Where software security practices are introduced solely to ensure compliance with legislative or industry standards, a box-ticking attitude to security may result. The security culture may be weak or non-existent, making it likely that precautions not explicitly mentioned in the standards will be missed. Thus, researchers need both a way to assess software security practice and a way to measure software security culture. To assess security practice, we converted the empirically-established 12 most common software security activities into questions. To assess security culture, we devised a number of questions grounded in prior literature. We ran a secure development survey with both sets of questions, obtaining organic responses from 1,100 software coders in 59 countries. We used proven common activities to assess security practice, and made a first attempt to quantitatively assess aspects of security culture in the broad developer population. Our results show that some coders still work in environments where there is little to no attempt to ensure code security. Security practice and culture do not always correlate, and some organisations with strong secure coding practice have weak secure coding culture. This may lead to problems in defect prevention and sustained software security effort.
Learning Deep Semantics for Test Completion

Writing tests is a time-consuming yet essential task during software development. We propose to leverage recent advances in deep learning for text and code generation to assist developers in writing tests. We formalize the novel task of test completion to automatically complete the next statement in a test method based on the context of prior statements and the code under test. We develop TECo-a deep learning model using code semantics for test completion. The key insight underlying TECO is that predicting the next statement in a test method requires reasoning about code execution, which is hard to do with only syntax-level data that existing code completion models use. Teco extracts and uses six kinds of code semantics data, including the execution result of prior statements and the execution context of the test method. To provide a testbed for this new task, as well as to evaluate TECO, we collect a corpus of 130,934 test methods from 1,270 open-source Java projects. Our results show that Teco achieves an exact-match accuracy of 18, which is 29% higher than the best baseline using syntax-level data only. When measuring functional correctness of generated next statement, Teco can generate runnable code in 29% of the cases compared to 18% obtained by the best baseline. Moreover, Teco is sianificantly better than prior work on test oracle generation.
Strategies, Benefits and Challenges of App Store-inspired Requirements Elicitation

App store-inspired elicitation is the practice of exploring competitors' apps, to get inspiration for requirements. This activity is common among developers, but little insight is available on its practical use, advantages and possible issues. This paper aims to empirically analyse this technique in a realistic scenario, in which it is used to extend the requirements of a product that were initially captured by means of more traditional requirements elicitation interviews. Considering this scenario, we conduct an experimental simulation with 58 analysts and collect qualitative data. We perform thematic analysis of the data to identify strategies, benefits, and challenges of app store-inspired elicitation, as well as differences with respect to interviews in the considered elicitation setting. Our results show that: (1) specific guidelines and procedures are required to better conduct app store-inspired elicitation; (2) current search features made available by app stores are not suitable for this practice, and more tool support is required to help analysts in the retrieval and evaluation of competing products; (3) while interviews focus on the why dimension of requirements engineering (i.e., goals), app store-inspired elicitation focuses on how (i.e., solutions), offering indications for implementation and improved usability. Our study provides a framework for researchers to address existing challenges and suggests possible benefits to fostering app store-inspired elicitation among practitioners.
OSSFP: Precise and Scalable C/C++ Third-Party Library Detection using Fingerprinting Functions

Third-party libraries (TPLs) are frequently used in software to boost efficiency by avoiding repeated developments. However, the massive using TPLs also brings security threats since TPLs may introduce bugs and vulnerabilities. Therefore, software composition analysis (SCA) tools have been proposed to detect and manage TPL usage. Unfortunately, due to the presence of common and trivial functions in the bloated feature dataset, existing tools fail to precisely and rapidly identify TPLs in C/C++ real-world projects. To this end, we propose OSSFP, a novel SCA framework for effective and efficient TPL detection in large-scale real-world projects via generating unique fingerprints for open source software. By removing common and trivial functions and keeping only the core functions to build the fingerprint index for each TPL project, OSSFP significantly reduces the database size and accelerates the detection process. It also improves TPL detection accuracy since noises are excluded from the fingerprints. We applied OSSFP on a large data set containing 23,427 C/C++ repositories, which included 585,683 versions and 90 billion lines of code. The result showed that it could achieve 90.84% of recall and 90.34% of precision, which outperformed the state-of-the-art tool by 35.31% and 3.71%, respectively. OSSFP took only 0.12 seconds on average to identify all TPLs per project, which was 22 times faster than the other tool. OSSFP has proven to be highly scalable on large-scale datasets.
Hierarchical Manta Ray Foraging Optimization with Weighted Fitness-Distance Balance Selection

Abstract Manta ray foraging optimization (MRFO) tends to get trapped in local optima as it relies on the direction provided by the previous individual and the best individual as guidance to search for the optimal solution. As enriching population diversity can effectively solve this problem, in this paper, we introduce a hierarchical structure and weighted fitness-distance balance selection to improve the population diversity of the algorithm. The hierarchical structure allows individuals in different groups of the population to search for optimal solutions in different places, expanding the diversity of solutions. In MRFO, greedy selection based solely on fitness can lead to local solutions. We innovatively incorporate a distance metric into the selection strategy to increase selection diversity and find better solutions. A hierarchical manta ray foraging optimization with weighted fitness-distance balance selection (HMRFO) is proposed. Experimental results on IEEE Congress on Evolutionary Computation 2017 (CEC2017) functions show the effectiveness of the proposed method compared to seven competitive algorithms, and the proposed method has little effect on the algorithm complexity of MRFO. The application of HMRFO to optimize real-world problems with large dimensions has also obtained good results, and the computational time is very short, making it a powerful alternative for very high-dimensional problems. Finally, the effectiveness of this method is further verified by analyzing the population diversity of HMRFO.
Perception of avatars nonverbal behaviors in virtual reality

Abstract Virtual reality has shown great potential in many fields, especially in business and psychology. By immersing someone in a new computer‐generated reality, it is possible to create realistic, safe, and controllable simulations for research and training, as well as new three‐dimensional‐enriched consumer experiences and services. Most of these environments, especially in the metaverse, rely on virtual representations of people called avatars. The design and non‐verbal behaviors of these avatars must be carefully crafted to provide a realistic and truly immersive experience. This paper aims to understand how avatar nonverbal behaviors (i.e., body posture, facial expression, and head movement) are perceived by users immersed in a virtual reality context, a very common situation encountered in many simulations and especially during training. Therefore, the first objective of this study is to validate, through an experiment with 125 participants, how the audience's levels of emotional valence and arousal are perceived in virtual reality. Based on these results, a library of audience non‐verbal behaviors corresponding to different arousal and valence levels is now available for future applications. The experiment also examines the benefits of using low‐end versus high‐end virtual reality headsets, and photo‐realistic versus cartoon avatars. The results have implications for the design of realistic, challenging, and interactive virtual audiences.
An optimized Extreme Learning Machine by Evolutionary Computation for River Flow Prediction and Simulation of Water Pollution in Colorado River Basin, USA

Humanity is witnessing scientific advances and since one of the vital concerns has been accessing water for drinking and agriculture, scientific innovations have been used to solve water challenges. Recently, Machine Learning (ML) is one of the most promising developments, therefore, for utilization of this innovation in simulating the Water Quality-Quantity Assessment (WQA) issues, the author has developed the Extreme Learning Machine (ELM) as the stand-alone model and its combination with evolutionary algorithms (EA) to optimize the modeling of the WQA parameters. So, the Particle Swarm Optimization (PSO), Genetic Algorithm (GA), and Imperialist Competitive Algorithm (ICA) are implemented to improve the performance of WQA parameters prediction. Then, the three novel models are developed for this purpose simultaneously with the stand-alone ML model. The novel models are GAELM, PSELM, and ICELM. The study area was the Colorado River basin in the USA, which was the input dataset extracted from the US Geological Survey. The considered parameters were Power of hydrogen (pH), river flow or Debi, Electrical Conductivity (EC), and Dissolved Oxygen (DO). To over- and under-fitting ML models, the input dataset was detrended and randomized by K-fold cross-validation. Furthermore, this study used seven evaluation methods for determining the models' performances. Based on the evaluation metrics, the ICELM model was the best in the EC, DO, and pH modeling. Also, the ELM model was superior in Debi prediction. Additionally, the discrepancy percent charts for all models in simulations were drawn. Moreover, the error percent plots also were drawn for all modeling. Finally, the Wilson method analyzed the prediction errors and related uncertainties.
Unsupervised Prostate Cancer Histopathology Image Segmentation via Meta-Learning

We propose a novel unsupervised meta-learning based segmentation algorithm for histopathology images. The proposed algorithm does not require any kind of patch-level annotations and relies solely on image labels, corresponding to any classification task, and direct feedback from a classifier. Furthermore, instead of simply segmenting histopathology images into different types of tissue, our algorithm determines the relative importance of each tissue region. After thresholding, the produced segmentations can also be used as regions of interest for various machine learning based diagnosis systems. We have tested our approach on Prostate cANcer graDe Assessment (PANDA) dataset and obtained 0.79 AUC, when testing the segmentation performance at patch-level, and 0.432 Dice coefficient, when testing precise segmentation, which is comparable to 0.446, described in related work which performed a supervised segmentation with U-Net. Note that no pixel level annotations were used.
Early prediction of the risk of ICU mortality with Deep Federated Learning

Intensive Care Units usually carry patients with a serious risk of mortality. Recent research has shown the ability of Machine Learning to indicate the patients' mortality risk and point physicians toward individuals with a heightened need for care. Nevertheless, healthcare data is often subject to privacy regulations and can therefore not be easily shared in order to build Centralized Machine Learning models that use the combined data of multiple hospitals. Federated Learning is a Machine Learning framework designed for data privacy that can be used to circumvent this problem. In this study, we evaluate the ability of deep Federated Learning to predict the risk of Intensive Care Unit mortality at an early stage. We compare the predictive performance of Federated, Centralized, and Local Machine Learning in terms of AUPRC, F1-score, and AUROC. Our results show that Federated Learning performs equally well as the centralized approach (for 2, 4, and 8 clients) and is substantially better than the local approach, thus providing a viable solution for early Intensive Care Unit mortality prediction. In addition, we demonstrate that the prediction performance is higher when the patient history window is closer to discharge or death. Finally, we show that using the F1-score as an early stopping metric can stabilize and increase the performance of our approach for the task at hand.