Reviewerly/exHarmony huggingface dataset:
Revisiting Inter-Class Maintainability Indicators

Over the last few decades, a variety of static code metrics have been published and promoted to measure the maintainability of software systems.This study evaluates 12 common static code metrics for their correlation with observed maintenance efforts. Leveraging modern repository mining techniques, we examine the historical data of three large open-source software systems with a combined size of over 1M LOC and over 10k classes. We automatically identify maintenance activities and measure the effort needed to perform them through revised lines of code. Then, we investigate if the state of the system as captured by these metrics is an indicator for the required maintenance effort.In contrast to earlier research, our results could not validate a general correlation between any of the examined metrics and maintainability. Instead, all evaluated metrics showed positive and negative correlations with maintenance effort depending on the considered time interval. Strong correlations only hold for specific projects, and within these projects, only for limited time spans. Across the project history, however, all metrics showed moderate correlations at most.As no metric was found to be a good indicator for high maintenance efforts in all contexts, we advocate against using any of the evaluated metrics without project-specific validation. If metrics are to be used to monitor the maintainability of a system, either directly or through models based on these metrics, engineers have to validate their applicability not just for the project at hand, but also for the current time span.
Improving Fine-tuning Pre-trained Models on Small Source Code Datasets via Variational Information Bottleneck

Small datasets are common in software engineering tasks such as linguistic smell detection and code runtime complexity prediction, as crafting these datasets often involves expert knowledge. Prior work usually applies machine learning algorithms (e.g., logistic regression and SVM) with hand-crafted features to tackle them, which could outperform neural models such as CNN. Recently, researchers have employed fine-tuning large pre-trained code models on various code-related tasks thanks to their transferability. However, it might be still instable and overfitting when fine-tuning on small datasets. In this paper, we firstly conduct an empirical study to fine-tune CodeBERT(a) on four code-related small datasets and observe the instability phenomenon. This could be induced by over-capacity and irrelevant features inherent in these large pre-trained code models with respective to those small datasets. To address this issue, we leverage variational information bottleneck to filter out irrelevant features when fine-tuning the models. The experiments demonstrate the out-performance of our method compared to standard fine-tuning and regularization method such as dropout and weight decay. We also experimentally study the stability of our method through varying dataset sizes. Our code and data are available at https://github.com/little-pikachu-hash/VIBCodeBERT.
A Comprehensive Study of WebAssembly Runtime Bugs

WebAssembly runtime is the infrastructure for executing WebAssembly, which is widely used as an execution engine by web browsers or blockchain platforms. Bugs in the WebAssembly runtime can lead to unexpected behavior and even security vulnerabilities in any application that relies on it. Therefore, to aid developers in understanding the WebAssembly runtime, a thorough investigation of bugs in the WebAssembly runtime should be conducted. To accomplish this, we carry out the first empirical analysis of 867 real bugs across four popular WebAssembly runtimes (V8, SpiderMonkey, Wasmer, and Wasmtime). We analyze the WebAssembly runtime bug characteristics based on their root causes, symptoms, bug-fixing time, and the number of files and lines of code involved in the bug fixes. Here are a few major research findings: 1) Incorrect Algorithm Implementation accounts for 25.49% of WebAssembly runtime bugs, the most prevalent of all root causes; 2) The most prevalent symptom is Crash, which accounts for 56.86% of WebAssembly runtime bugs; 3) At the median, the bug-fixing time are 13, 4, 5, and 6 days for V8, SpiderMonkey, Wasmer, and Wasmtime respectively; 4) Over 50% of bug fixes in the four WebAssembly runtimes involve only one file, while more than 90% of bug fixes involve no more than 8 files; 5) The median source code lines for bug fixes for V8, SpiderMonkey, Wasmer, and Wasmtime are 18.5, 14, 26, and 36 lines, respectively. Overall, our research summarizes 18 findings and discusses the broad implications for WebAssembly runtime bug detection, localization, debugging, and repair based on the key findings.
Investigating Software Testing and Maintenance of Open-Source Distributed Ledger

A distributed ledger is the backbone of all blockchain solutions. It provides a shared database spreading across a network of nodes. The number of DL solutions and their implementations has grown in recent years. Besides the architectural and performance promises of thesesolutions, organizations seekingto implement DL also need to consider the overall quality of the software available and its ecosystem. Particularly, previous research has identified the need to better understand the testing and maintenance practices behind these types of technologies. This paper investigates the testing and maintenance of 18 different open-source projects that implement distributed ledgers. We perform a manual inspection of test artefacts and mine the history of commits, issues and contributors of the chosen projects to understand the landscape of testing and maintenance in these projects. Our findings suggest that unit and integration tests are present in most projects, they do not follow a holistic system testing approach. Moreover, projects rely on a small team of core contributors (5 on average). While the projects are continuously maintained, larger changes are uncommon. Our results can be used for benchmarking and pinpointing areas of improvement for the development of distributed ledgers.
Evaluating Learning-to-Rank Models for Prioritizing Code Review Requests using Process Simulation

In large-scale, active software projects, one of the main challenges with code review is prioritizing the many Code Review Requests (CRRs) these projects receive. Prior studies have developed many Learning-to-Rank (LtR) models in support of prioritizing CRRs and adopted rich evaluation metrics to compare their performances. However, the evaluation was performed before observing the complex interactions between CRRs and reviewers, activities and activities in real-world code reviews. Such a pre-review evaluation provides few indications about how effective LtR models contribute to code reviews. This study aims to perform a post-review evaluation on LtR models for prioritizing CRRs. To establish the evaluation environment, we employ Discrete-Event Simulation (DES) paradigm-based Software Process Simulation Modeling (SPSM) to simulate real-world code review processes, together with three customized evaluation metrics. We develop seven LtR models and use the historical review orders of CRRs as baselines for evaluation. The results indicate that employing LtR can effectively help to accelerate the completion of reviewing CRRs and the delivery of qualified code changes. Among the seven LtR models, LambdaMART and AdaRank are particularly beneficial for accelerating completion and delivery, respectively. This study empirically demonstrates the effectiveness of using DES-based SPSM for simulating code review processes, the benefits of using LtR for prioritizing CRRs, and the specific advantages of several LtR models. This study provides new ideas for software organizations that seek to evaluate LtR models and other artificial intelligence-powered software techniques.Data&materials: https://figshare.com/s/a033e99cd2a61e64c8bc.
Gripping Aerial Topology Optimized Robot (GATOR)

This paper introduces the design, modeling, manufacturing, and testing of a Gripping Aerial Topology Optimized Robot (GATOR). The airframe of this unmanned aerial vehicle (UAV) is designed to be lightweight, structurally stiff, modular, and multi-functional. A Level-Set Topology Optimization (LSTO) method defines the external geometry of the frame, while the frame infill is controlled using a variable thickness latticing technique based on Finite Element Analysis (FEA) results. The UAV incorporates a soft robotic gripper, allowing the vehicle to collect delicate samples from the environment and perch for low-power use for extended periods. The bio-inspired design and fabrication of a mountable soft robotic gripper are presented and the associated kinematics are derived for controls. To further decrease the weight of the designs a novel volume-changing material was introduced following careful characterization through Scanning Electron Microscopy (SEM) and tensile testing. The resulting platform leverages additive manufacturing using material extrusion technology and can be swiftly instrumented with propulsion and flight control systems. The presented modular design methodology can be applied to the rapid prototyping of a broad range of aerial platforms and lightweight structures.
A Multi-Step Learning Approach to Assist Code Review

Modern code review is a process for early detection and reduction of issues, which assists in ensuring the quality of the source code, detecting anomalies, and identifying potential improvements. However, this is a highly manual activity that requires a lot of resources and time. Recent research has addressed these problems by attempting to entirely automate this task (i.e., generating code reviews). However, we do believe that dismissing the reviewer from this process is not the best option in terms of its optimal functioning, especially considering the high error rates in the proposed approaches. Furthermore, this full automation is still too far to achieve given the complexity of the task that requires human intelligence. In this work, we aim to assist the reviewer in the code review process. We propose an approach for detecting the type of issue and locating parts of the code that need to be revised by developers. In the first phase, we propose a meta-learner that combines a learning-based model and a knowledge-based model to predict the type of issue from the review comment. Then, we use this component to create and label a large dataset composed of quadruplets <original code, review, issue type, revised code>. We use this data set to finetune a pre-trained language model to predict the types of issues (e.g., naming, resource handling, etc.), that need to be addressed in the original code snippet. Furthermore, we fine-tune another pre-trained language model to locate these issues in the source code submitted by developers. We evaluate the performance of our approach using a test set not considered during the training. Our results show that our model accurately locates and predicts the types of issues.
An Optimized Concurrent Proof of Authority Consensus Protocol

Security and reliability in Blockchain software systems is a major challenge in Blockchain Oriented Software Engineering. One of the most critical components to address at the architectural level is the consensus protocol, as it serves as the mechanism for accepting valid transactions and incorporating them into the ledger history. Given that this process is executed by specific blockchain nodes, it is crucial to consider them as a key point of focus for ensuring the integrity of the entire blockchain history. This paper addresses the major challenge of security and reliability in Blockchain software systems by proposing a new protocol for Permissioned Concurrent Proof of Authority (CPoA). This protocol involves selecting a group of nodes as authority nodes, responsible for validating new identities, blocks, and transactions. The protocol is integrated with a framework that subjects validators to a unique eligibility criterion and a combination of reputation, security score, online aging, and general performance indicators related to node reliability, significantly reducing the risk of validator misbehavior and enhancing security, reliability and confidentiality of the entire blockchain compared to other existing approaches.
Blockchain oracles for document certification: A case study.

The use of blockchain oracles is becoming increasingly widespread as it responds to the problem of providing the blockchain with external data automatically. In this paper, we focus on the use of blockchain oracles for the certification of articulated data. In particular, we show how it is possible to design and implement a DApp capable of providing documents on request without these data permanently residing on the blockchain. We present a case study of the on-demand provision of university curriculum, fully managed via a blockchain oracle. In this system, the smart contract will not record data permanently but only for the time necessary for its use. The system is intended to allow a user to request and obtain data only about herself. The paper also presents a classification of the types of blockchain oracles and their most common applications. The case study will be presented as blockchain-oriented software, and in terms of architecture and dynamics using UML diagrams, the benefits and drawbacks of the approach are discussed.
Documentation-Guided API Sequence Search without Worrying about the Text-API Semantic Gap

Developers often search for application programming interfaces (APIs) and their usage patterns to speed up the efficiency of software development. This paper focuses on the API sequence search task, which refers to using a function-relevant textual query to search for API sequences mined from open-source software repositories that can implement this function. However, the severe semantic gap between text and API makes it challenging to discover the correspondence between natural language queries and desired API sequences. Therefore, we propose a method called documentation-guided API sequence search (DGAS), through which we do not need to worry about the semantic gap between text and API. Specifically, DGAS consists of documentation-guided cross-modal attention (DGCA) and documentation-guided cross-modal matching (DGCM). DGCA calculates the cross-modal attention map using features extracted from the same modality (i.e., API documentation sequence and textual query) instead of from different modalities (i.e., API sequence and textual query) to bridge the semantic gap during the cross-modal attention phase. Besides, DGCM takes API documentation as supplementary information of API sequence to bridge the semantic gap during the cross-modal matching phase. We use the API documentation to extend the existing dataset for API sequence generation to construct a dataset for API sequence search to evaluate DGAS. Experimental results show that DGAS outperforms the baseline methods.
EcoDialTest: Adaptive Mutation Schedule for Automated Dialogue Systems Testing

With the rapid growth of Artificial Intelligence, dialogue systems have become increasingly powerful. Though Recurrent Neural Network power the dialogue systems, it also bring challenges to the systems’ testing. In order to ensure the safety of these systems, which we must pay attention to, DialTest showed up. DialTest broke the traditional test methods, it made innovation at many levels. We have to acknowledge this great contribution. However, DialTest has a smattering of shortcomings. It treats all seeds as equal, implying that it cannot adjust the energy assignment quickly, resulting in energy waste. Moreover, DialTest’s mutant sentences generated by a few original seed sentences in the late stage of variation. This paper presents an improved DialTest with an adaptive mutation schedule, we called it EcoDialTest. EcoDialTest divides all the seed into three states, different states have different energy distribution strategies. We devise a new mutation strategy to improve the effectiveness and dependability of the seeds in the transformed seed set. All of these were implemented based on DialTest, we still adopt DeepGini impurity as the main guidance to guide the test generation process and utilize the three mutation operators as it does. Through ATIS, Snips and Facebook datasets, EcoDialTest was evaluated by two state-of-the-art models in the experiment. According to the result, we found that EcoDialTest attained lower values in both intent accuracy and slot accuracy than DialTest.
How Dynamic Features Affect API Usages? An Empirical Study of API Misuses in Python Programs

Incorrect usages of Application Programming Interfaces (APIs) may lead to unexpected problems during the software development process. Although there have been many attempts to address API-misuse issues, most of them are mainly for static languages. In contrast, API misuses in dynamic languages are rarely covered, mostly due to challenges about dynamic features. In this paper, we develop the first-ever comprehensive study of API misuses for Python programs. To accomplish this, we manually analyze 79,096 commits of six popular open-source Python projects on GitHub to collect true-positive cases. Based on the validation, we develop a classification of Python API Misuses, called PAM, and a dataset, PAMBench, containing 670 validated real-world API-misuse cases in popular Python programs. For each API-misuse case, we explore its root cause, symptom, program issue and repair method. Specifically, we pay attention to the effect of dynamic features on API usages in Python. The systematic study on PAMBench shows that, most importantly, dynamic features, especially type dynamics, have a non-negligible impact on API usages in Python, mainly related to incorrect assumptions about the type, callable state, attribute and existence of caller object, method call itself, passed argument(s) and return value during an API invocation. Our root-cause analysis reveals the importance of correct design, implementation, annotation, checking and recording about the types and states of all parts of API method calls during Python program development. Finally, we present possible solutions for more secure, reliable and maintainable API usages in Python.
Analysis and Propagation of Feature Revisions in Preprocessor-based Software Product Lines

Preprocessor-based software product lines (SPLs) are used to deal with evolution in space, in which features (so-called configuration options)—annotated in source code with #ifdefs—are included, removed, and systematically reused. Inevitably, feature implementations also evolve over time, i.e., when existing features are revised. Nowadays, Version control systems (VCSs) are well-integrated into SPL development processes for versioning support of releases. Changes to existing features in one version, a.k.a. release of an SPL, usually developed in a branch, frequently need to be propagated to other active releases. However, there is no automated support for analyzing and propagating features in SPL releases. For instance, VCSs can only propagate changes at the commit level, but miss support at the feature level, i.e., the building blocks of SPLs. Manually analyzing and propagating a version of a feature, i.e., a feature revision, through #ifdefs is risky, time-consuming, and error-prone because a feature can be interacting with multiple features and it can be spread in multiple blocks of code across different files. We thus present a novel and tool-supported approach for the analysis and propagation of feature revisions. We evaluated our approach quantitatively by computing its correct behavior and runtime. Our approach analyzes and propagates a feature implementation in ≈63 seconds, with, on average, precision and recall of 99%. In total, we propagated 3,134 features in space and time between 200 pairs of releases on four real-world preprocessor-based SPLs. In addition, we qualitatively evaluated the usefulness of our tool support by conducting interviews with five experienced core developers of three popular preprocessor-based SPLs. The qualitative results confirm that our tool support is useful to speed up the analysis and propagation of feature revisions.
Architecture Decisions in AI-based Systems Development: An Empirical Study

Artificial Intelligence (AI) technologies have been developed rapidly, and AI-based systems have been widely used in various application domains with opportunities and challenges. However, little is known about the architecture decisions made in AI-based systems development, which has a substantial impact on the success and sustainability of these systems. To this end, we conducted an empirical study by collecting and analyzing the data from Stack Overflow (SO) and GitHub. More specifically, we searched on SO with six sets of keywords and explored 32 AI-based projects on GitHub, and finally we collected 174 posts and 128 GitHub issues related to architecture decisions. The results show that in AI-based systems development (1) architecture decisions are expressed in six linguistic patterns, among which Solution Proposal and Information Giving are most frequently used, (2) Technology Decision, Component Decision, and Data Decision are the main types of architecture decisions made, (3) Game is the most common application domain among the eighteen application domains identified, (4) the dominant quality attribute considered in architecture decision-making is Performance, and (5) the main limitations and challenges encountered by practitioners in making architecture decisions are Design Issues and Data Issues. Our results suggest that the limitations and challenges when making architecture decisions in AI-based systems development are highly specific to the characteristics of AI-based systems and are mainly of technical nature, which need to be properly confronted.
Identifying Emergent Leadership in Open Source Software Projects Based on Communication Styles

In open source software (OSS) communities, existing leadership indicators are dominantly measured by code contribution or community influence. Recent studies on emergent leadership shed light on additional dimensions such as intellectual stimulation in collaborative communications. This paper aims to mine communication styles and identify emergent leadership behaviors in OSS communities, using issue comments data. We start with the construction of 6 categories of leadership behaviors based on existing leadership studies. Then, we manually label leadership behaviors in 10,000 issue comments from 10 OSS projects, and extract 304 heuristic linguistic patterns which represent different types of emergent leadership behaviors in flexible and concise manners. Next, an automated algorithm is developed to merge and consolidate different pattern sets extracted from multiple projects into a final pattern ranking list, which can be applied for the automatic leadership identification. The evaluation results show that iLead can achieve a median precision of 0.82 and recall of 0.78, outperforming ten machine/deep learning baselines. We argue that emergent leadership behaviors in issue discussion should be taken into consideration to broaden existing OSS leadership viewpoints.
TemLock: A Lightweight Template-based Approach for Fixing Deadlocks Caused by ReentrantLock

ReentrantLock, an alternative to Synchronized, is provided in Java5 to handle the conflicts of memory accesses in concurrent programs. However, falsely using ReentrantLock may introduce deadlocks. To fix deadlocks caused by ReentrantLock, in this paper, we propose TemLock, an approach that can detect and fix deadlocks in Java programs based on the fix templates. We detect and fix deadlocks based on the predefined templates by searching and modifying the node information in AST of the program. Experimental results show that TemLock can fix 156 out of 177 deadlocks caused by ReentrantLock in Java projects, indicating its effectiveness. The URL of this tool is https://github.com/yyc36/TemLock_ReentrantLock/tree/master. The video of our demo is available at https://www.youtube.com/watch?v=LIRcRF99ApY.