Reviewerly/exHarmony huggingface dataset:
An Empirical Study of Predicting Fault-prone Components and their Evolution

Predicting fault-prone components at an early stage is useful for any organization to ensure quality software delivery. Prioritizing tests becomes easier with the prediction of faultproneness of the components since developers can allocate more time and resources to the “High” fault-prone components. Furthermore, test prioritization helps reduce the cost of regression and allows developers to take careful decisions regarding the sensitive components. This paper performs an empirical study to predict fault-prone components and their evolution on two popular open-source projects, Chromium web browser and ProFTPD. First, we construct and compare two prediction models: Random Forest (RF) and Support Vector Machine (SVM) for classifying components as “High” or “Low” fault prone. Second, we analyze the evolution of the fault proneness of 22 components of Chromium in 42 releases and 12 components of ProFTPD in 15 releases. Chromium has a median of 3. 9k commits per release with a standard deviation of lk while ProFTPD has 578 commits per release with a standard deviation of 654. We consider the total churns, bug-fix commits, bug-fix churns, rush period changes, number of developers, and number of files modified as the measures to construct our prediction models. Our models are able to successfully predict the fault-proneness of the components having the Random Forest outperforming SVM with an accuracy of 96% and 95% for Chromium and ProFTPD respectively. We found that the majority of the Chromium components are high fault-prone. For ProFTPD, components are found to be high and low fault-prone interchangeably. The fault proneness of the components has evolved over the releases where “Chromecast” in Chromium seems to be gradually turning into a low fault-prone component.
Comparison of Developer’s Work Efficiency between Different Editors

In this paper, as an example of comparing developer’s work efficiency between different editors, we propose a method to collect and compare self-evaluations and quantitative evaluations of developer’s work efficiency for each editor. We also practiced the proposed method on Visual Studio Code and Eclipse, and confirmed the applicability of the method.
Quality assurance study with mismatched data in sentiment analysis

Considering mismatched data have harmful influence on the quality assurance in sentiment analysis, therefore this paper proposed an effective method to detect these mismatches. This study considered data adequacy and model’s confidence together, and proposed to use the surprise adequacy metric for mismatch detection. Experiments were implemented on Amazon.com review data. Performances of mismatched data detection and model retraining were evaluated. The proposed method using Mahalanobis-distance-based surprise adequacy was verified feasible and effective to detect mismatched data in the studied dataset. Moreover, after mismatch detection, retrained model was illustrated useful to improve AI model’s quality.
Threshold Concepts and Skills in Software Architecture: Instructors’ Perspectives

Context: Software Architecture is an important subject and core course in the Software Engineering degree that teaches multiple co-mingled concepts and skills. The academic community believes that students find the course difficult to grasp and master. However, to revise the course curriculum, identification of threshold concepts and skills can help prioritize topics ensuring alignment with course learning objectives. Objective: The aim of the study was to identify threshold concepts and skills in Software Architecture to help instructors focus on redesigning the curriculum and improving didactics. Method: We applied the Delphi technique to identify threshold concepts and skills from instructors with teaching experience in university-level Software Architecture courses. Results: We identified eleven threshold concepts and nine threshold skills with more than 80% agreement among the participants. Six out of twenty-one threshold concepts and skills achieved 100% agreement from participants indicating high consensus. Furthermore, all participants agreed that applying skills to design Software Architecture is more difficult than understanding the underlying theoretical concepts. Conclusion: The Software Architecture course is demanding, and the industry expects graduating students are prepared to design solutions for complex systems. The identified threshold concepts and skills can help academics to redesign Software Architecture courses, focus on hard to grasp topics, and offer support for skills that are difficult to master. Often theoretical concepts are considered more important than the skills required to apply them in practice. However, instructors agreed that students struggle to apply theoretical concepts in designing solutions. Thus, skills development should be equally emphasized.
A new measure to assess the systematicity of the abstracts of reviews self-identifying as systematic reviews

Systematic reviews are crucial for various stakeholders since they allow them to make evidence-based decisions without being overwhelmed by a large volume of research. The abstract is one of the most important systematic review’s components since it usually reflects the content of the review. It may be the only part of the review that most of thereaders will read when needing to form an opinion on a given topic. Besides, the content of an abstract is usually the main information readers use to decide if they want to access the full content of the review or not. Since an abstract usually summarizes a review, readers may therefore mostly rely on that abstract to judge the quality of the review as well as its methodological rigor. However, abstracts are usually poorly written and may therefore give a misleading and even harmful picture of the reviews’ contents. To assess abstracts, we propose a measure that allows quantifying the systematicity of reviews’ abstracts i.e., the extent to which these abstracts exhibit good reporting quality. Experiments on 151 reviews published in the software engineering field showed that these reviews’ abstracts exhibit a suboptimal systematicity.
Preliminary Analysis of Review Method Selection Based on Bandit Algorithms

To enhance the reliability of software, it is important is to review all software artifacts (e.g., design documents) to remove defects as earlier as possible. There are various review methods available, and project managers face the challenge of choosing a suitable method for their current projects. One of approaches to support the selection of review methods is to evaluate review methods beforehand, to identify the most effective method on average. However, past studies have not evaluated review methods thoroughly as the process can be time-consuming. We propose a bandit-algorithm (BA) based method to evaluate and then dynamically select a suitable review method (from a list of candidates). In our experiments, we assume that the proposed method is applied to design document review on basic design phase. We performed experiments based on a simulation, instead of using an actual dataset. On our simulation, when a review method is selected by our BA method, productivity (i.e., total development time) was improved by about 1.25 times, and it was the second highest among candidates of review methods.
Supporting Implementation of Virtual Reality in Engineering Design by Structured Reflection

Multifaceted advantages are reported for the application of Virtual Reality (VR) technologies in engineering design. However, there are different challenges for productive use of VR in practice, caused by a lack of systematic information about potentials, technical and organizational requirements, limited support to guide the implementation process and additional efforts for preparation and post-processing activities. In this paper the usefulness of structured reflection to explore benefits and challenges on long- and short-term levels and with regard to social, process and goal aspects in highlighted. Therefore, a framework and guiding questions are presented and a case study of reflection on VR implementation in a lecture setting is reported. Based on the findings essential fields for further research are formulated.
Incremental Relational Topic Model for Duplicate Bug Report Detection

In software development, bug fixing is a timeconsuming, yet unavoidable task. A bug is occasionally reported by more than one reporters, resulting in duplicate bug reports. Detecting duplicate bug reports is crucial because it helps reduce the maintenance efforts from developers as well as provides more information in the bug fixing process. In this paper, we propose an automatic approach to this problem. In our approach, a bug report is considered as a textual document describing one or more technical aspects of a software system, in which some of them might be erroneously implemented. The reports similarly describing the same erroneous technical aspects are considered as duplicate ones. We utilize Relational Topic Model (RTM), a probabilistic, generative topic model, to formulate the probabilistic structures of technical aspects in a collection of bug reports and the duplication indicators among them. Trained with historical data including identified duplicate reports, the model can be used to detect other not-yet-identified duplicate ones. To support software evolution, we extend RTM into incremental RTM(iRTM) in which the trained model can be quickly updated without spending a large amount of time for complete re-training when new reports are filed or additional duplication information is available. Our empirical evaluation on large systems shows that iRTM outperforms the state-of-the-art approaches, achieving up to 90% top-10 accuracy with up to 8 times faster in updating its internal model as new reports arrive.
Goal-oriented Knowledge Reuse via Curriculum Evolution for Reinforcement Learning-based Adaptation

Reinforcement learning is a powerful methodology that enables self-adaptive systems to relearn and update their adaptation policy when dealing with unforeseen changes. To update the policy more efficiently, several knowledge reuse approaches have been proposed to speed up relearning. However, the current studies treat and reuse the knowledge integrally, which may result in increased relearning costs if the reused knowledge is inappropriate in the changed situation. Generally, some localized pieces of the knowledge are still appropriate for reuse if they are not related to the changes, while some pieces may become inappropriate for reuse if they are affected by the changes. This paper proposes a goal-oriented curriculum evolution method to realize finer-grained knowledge reuse, combining goal-oriented modeling and curriculum learning. The method is twofold: (1) at design time, we apply goal-oriented modeling to design a curriculum in which an RL problem is decomposed into sub-problems, so that knowledge can be decomposed into several pieces of localized knowledge for sub-problems, and (2) at runtime, we evolve the curriculum to reflect changes (i.e., update the sub-problems related to the changes), so that the affected pieces of knowledge can be locally updated to make them appropriate for reuse in the changed situation. The evaluation based on a cleaning robot shows that the relearning time was shortened, demonstrating the effectiveness of our method.
Visualizing Contributor Code Competency for PyPI Libraries: Preliminary Results

Python is known to be used by beginners to professional programmers. Python provides functionality to its community of users through PyPI libraries, which allows developers to reuse functionalities to an application. However, it is unknown the extent to which these PyPI libraries require proficient code in their implementation. We conjecture that PyPI contributors may decide to implement more advanced Pythonic code, or stick with more basic Python code. Are complex codes only committed by few contributors, or only to specific files? The new idea in this paper is to confirm who and where complex code is implemented. Hence, we present a visualization to show the relationship between proficient code, contributors, and files. Analyzing four PyPI projects, we are able to explore which files contain more elegant code, and which contributors committed to these files. Our results show that most files contain more basic competency files, and that not every contributor contributes competent code. We show how our visualization is able to summarize such information, and opens up different possibilities for understanding how to make elegant contributions.
Multi-Perspective Alignment Mechanism for Code Search

Developers often tend to search and reuse code snippets from large code repositories to improve their programming skills. To support code reuse, early code search models used information retrieval (IR) techniques to index a large corpus of code and then returned relevant code based on search statements. However, IR-based models can cause information loss due to keyword matching. To solve this problem, developers applied deep learning (DL) techniques to encode search models. However, these models either learn isolated representations of codes and queries or learn interdependent representations of codes and queries in a single way, which both limit the effectiveness of the models.In this study, we propose a code search model MpaCS, which can interact from multiple perspectives to enable cross-language retrieval between code and query statements. MpaCS extracts code and query information from the code textual features (i.e., method name, and tokens), the code structural feature (i.e., abstract syntax tree), and the query feature (i.e., tokens). We first implement the alignment between code features and query features at the fragment level. Then we propose a context matching module to achieve a deeper interaction between the code and the query. Finally, we propose a new attention network to explore the maximum alignment relationship between code and query from another perspective. We evaluate the performance of MpaCS on two existing large-scale datasets with 69k and 105k code snippets, respectively. Experimental results show that MpaCS outperforms four state-of-the-art models DeepCS [1],UNIF [2], MPCAT [3] and CARLCS-CNN [4].
AMF: A Field to Realize the Values of Agile Manifesto through Retrospectives

In recent decades, companies and other organizations have been increasingly focusing on agile software development methods, and Scrum has been increasingly introduced into software development projects. However, teams in the early stages of Scrum adoption tend to focus on adhering to development processes and practices, often neglecting the four essential values declared in the Agile Manifesto. This paper proposes a field of retrospective called Agile Manifesto Farm (AMF) as a method to promote awareness of the four values and to progressively formalize and establish the knowledge obtained through the activities of each sprint. As a case study, the proposed method is applied in a project-based learning class.
SCAR: Smart Contract Alarm Ranking

In 2016, the famous attack on the smart contract named The DAO led to a financial loss of 60 million dollars. Since then, attacks on smart contracts have increased. Thus, the security issue of smart contracts has attracted more and more attention from the community, putting pressure on developers to discover security issues in smart contracts before deploying them. To address this problem, many researchers have developed smart contract analyzers to early detect potential vulnerabilities. However, a common problem with these tools is a large number of alarms with a high false positive rate. Consequently, developers need to spend much time and effort investigating the alarms which are falsely detected as vulnerable. In this paper, we propose SCAR, a novel approach to prioritize the alarms of static analysis tools. Based on the intuition that alarms with similar contexts tend to have the same labels (true positive or false positive), SCAR is built with two deep learning models to capture the patterns associated with the contexts of the labeled alarms. After that, for new alarms, SCAR calculates their likelihood to be true positives and ranks them according to the predicted scores. SCAR is evaluated on a large data set of 14,184 alarms from 47,518 realworld smart contracts. The results show that the programmers can productively find up to two-thirds of the actual vulnerabilities by investigating only 20% of the ranked alarms.
Why Visualize Data When Coding? Preliminary Categories for Coding in Jupyter Notebooks

Data visualization becomes a crucial component in data analytics, especially data exploration, understanding, and analysis. Effective data visualization impacts decision-making and aids in discovering and understanding relationships. It leads to benefits in data-intensive software development tasks e.g., feature engineering in machine learning-based software projects. However, it is unknown how visualizations are used in competitive programming. The idea of this paper is to report early results on what visualizations are prevalent in competitive programming. Grandmasters are the highest level reached in competitions (novice, expert, master, and grandmaster). Analyzing the visualizations of 7 high-rank competitors (i.e., Grandmaster) in Kaggle, we identify and present a catalog of visualizations used to both tell a story from the data, as well as explain the process and pipelines involved to explain their coding solutions. Our taxonomy includes nine types from over 821 visualizations in 68 instances of Jupyter notebooks. Furthermore, most visualizations are for data analysis for distribution (DA Distribution), and frequency (DA Frequency) are most used. We envision that this catalog can be useful to better understand different situations in which to employ these visualizations.
Two-Stage Patch Synthesis for API Migration from Single API Usage Example

Third-party libraries are widely used and constantly evolving. When migrating client code to a new API, a major challenge is editing client code to adapt to incompatible changes of the API. Some tools provide automated migration that synthesizes a generic patch from migration samples in a pair of beforeand after-migration snippets. However, they still have limited applicability due to the difficulty in retrieving an adequate pair in which before- and after-migration snippets invoke the sourceand destination-API of the migration respectively. In this paper, we present AUTOMIG, which addresses the problem by a patch synthesis from a single API usage example that invokes the destination-API. Due to the absence of a beforemigration version of the API usage example, it is nontrivial to find out the invariant context of API invocation through the migration. To synthesize a patch that preserves the context in the client code, AUTOMIG synthesizes patches in two-stage patch synthesis procedure. The first stage synthesizes a generic patch to describe a replacement of API. The second stage modmes the generic patch to preserve the context of the API invocation in the client code. Our experiment using a dataset that provides real-world Java codes shows that AUTOMIG synthesized a correct patch for 92.2% of the code on average. Our manual investigation on synthesized patches shows that AUTOMIG can synthesize highquality patches that preserve the context of API invocation of each client code.
Multifaceted Hierarchical Report Identification for Non-Functional Bugs in Deep Learning Frameworks

Non-functional bugs (e.g., performance-or accuracy-related bugs) in Deep Learning (DL) frameworks can lead to some of the most devastating consequences. Reporting those bugs on a repository such as GitHub is a standard route to fix them. Yet, given the growing number of new GitHub reports for DL frameworks, it is intrinsically difficult for developers to distinguish those that reveal non-functional bugs among the others, and assign them to the right contributor for investigation in a timely manner. In this paper, we propose MHNurf -an end-to-end tool for automatically identifying non-functional bug related reports in DL frameworks. The core of MHNurf is a Multifaceted Hierarchical Attention Network (MHAN) that tackles three unaddressed challenges: (1) learning the semantic knowledge, but doing so by (2) considering the hierarchy (e.g., words/tokens in sentences/statements) and focusing on the important parts (i.e., words, tokens, sentences, and statements) of a GitHub report, while (3) independently extracting information from different types of features, i.e., content, comment, code, command, and label.To evaluate MHNurf, we leverage 3,721 GitHub reports from five DL frameworks for conducting experiments. The results show that MHNurf works the best with a combination of content, comment, and code, which considerably outperforms the classic HAN where only the content is used. MHNurf also produces significantly more accurate results than nine other state-of-the-art classifiers with strong statistical significance, i.e., up to 71% AUC improvement and has the best Scott-Knott rank on four frameworks while 2nd on the remaining one. To facilitate reproduction and promote future research, we have made our dataset, code, and detailed supplementary results publicly available at: https://github.com/ideas-labo/APSEC2022-MHNurf.
UTCache: Updatable Transactional Cache For Database-Driven Applications

Most popular applications rely on the remote database server for data storage. In response to users’ requests, applications interactively read or update the data in databases through DML, like SQL. Caching database results can eliminate the network latency of remote access but also may violate transaction isolation provided by the database. To avoid anomalies caused by the cache, existing works designed transactional caches, which can manage concurrent access to cache entries and invalidate stale ones automatically. For some OLTP workloads with frequent updates, invalidation will reduce the cache hit ratio and hurt performance. To handle these workloads better, we propose UTCache, a transactional cache that supports updating stale entries. UTCache caches database results in different ways according to the characteristic of queries. After the data in the database is modified, it will automatically update the affected cache entries to serve future queries, if possible. Our evaluation shows that in typical OLTP workloads, TPC-C and SmallBank, UTCache can outperform invalidation-based cache a lot and achieve higher throughput with lower latency and network traffic.
How students design simple programs: a study on the effort and perceptions of the design activity

Software design seeks to contribute to developing quality software. During the design phase, the requirements are translated into a representation of the software (also known as design), whose quality can be evaluated and improved. For graduating students, software design is difficult to understand and produce. In fact, producing a good design seems to require a certain level of cognitive development that few students achieve. The aims of this study are, on the one hand, to know the effort devoted by students to software design, how is the effort variation throughout the different programs and how is the effort variation between students. On the other hand, we analyzed the software quality during the programs and the students’ perception of the problems they related to design. We found that students spend at least 3 times as much time on coding as on design. Also, regarding design, students have highly variable behavior among themselves and also highly variable behavior when developing different programs. We believe this may be due to a lack of knowledge on design, a lack of knowledge of the importance of design in software quality and an important lack of habits and discipline when developing software. Also, we found that students did not produce better quality products in the latest programs compared with the first ones.