Reviewerly/exHarmony huggingface dataset:
Does the Stream API Benefit from Special Debugging Facilities? A Controlled Experiment on Loops and Streams with Specific Debuggers

Java's Stream API, that massively makes use of lambda expressions, permits a more declarative way of defining operations on collections in comparison to traditional loops. While experimental results suggest that the use of the Stream API has measurable benefits with respect to code readability (in comparison to loops), a remaining question is whether it has other implications. And one of such implications is, for example, tooling in general and debugging in particular because of the following: While the traditional loop-based approach applies filters one after another to single elements, the Stream API applies filters on whole collections. In the meantime there are dedicated debuggers for the Stream API, but it remains unclear whether such a debugger (on the Stream API) has a measurable benefit in comparison to the traditional stepwise debugger (on loops). The present papers introduces a controlled experiment on the debugging of filter operations using a stepwise debugger versus a stream debugger. The results indicate that under the experiment's settings the stream debugger has a significant ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{p} &lt; .001$</tex> ) and large, positive effect <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\eta_{p}^{2}=.899;\ \frac{M_{stepwise}}{M_{stream}} \sim 204\%)$</tex> . However, the experiment reveals that additional factors interact with the debugger treatment such as whether or not the failing object is known upfront. The mentioned factor has a strong and large disordinal interaction effect with the debugger ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{p} &lt; .001; \eta_{p}^{2}=.928$</tex> ): In case an object is known upfront that can be used to identify a failing filter, the stream debugger is even less efficient than the stepwise debugger <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\frac{M_{stepwise}}{M_{stream}}\sim 72\%)$</tex> . Hence, while we found overall a positive effect of the stream debugger, the answer whether or not debugging is easier on loops or streams cannot be answered without taking the other variables into account. Consequently, we see a contribution of the present paper not only in the comparison of different debuggers but in the identification of additional factors.
On the Applicability of Language Models to Block-Based Programs

Block-based programming languages like Scratch are increasingly popular for programming education and end-user programming. Recent program analyses build on the insight that source code can be modelled using techniques from natural language processing. Many of the regularities of source code that support this approach are due to the syntactic overhead imposed by textual programming languages. This syntactic overhead, however, is precisely what block-based languages remove in order to simplify programming. Consequently, it is unclear how well this modelling approach performs on block-based programming languages. In this paper, we investigate the applicability of language models for the popular block-based programming language Scratch. We model Scratch programs using n-gram models, the most essential type of language model, and transformers, a popular deep learning model. Evaluation on the example tasks of code completion and bug finding confirm that blocks inhibit predictability, but the use of language models is nevertheless feasible. Our findings serve as foundation for improving tooling and analyses for block-based languages.
Semi-Automatic, Inline and Collaborative Web Page Code Curations

Software developers spend about a quarter of their workday using the web to fulfill various information needs. Searching for relevant information online can be time-consuming, yet acquired information is rarely systematically persisted for later reference. In this work, we introduce SALI, an approach for semi-automated inline linking of web pages to source code locations. SALI helps developers naturally capture high-quality, explicit links between web pages and specific source code lo-cations by recommending links for curation within the IDE. Through two laboratory studies, we examined the developer's ability to both curate and consume links between web pages and specific source code locations while performing software development tasks. The studies were performed with 20 subjects working on realistic software change tasks from widely-used open-source projects. Results show that developers continuously and concisely curate web pages at meaningful locations in the code with little effort. Additionally, we found that other developers could use these curations while performing new and different change tasks to speed up relevant information gathering within unfamiliar codebases by a factor of 2.4.
The untold story of code refactoring customizations in practice

Refactoring is a common software maintenance practice. The literature defines standard code modifications for each refactoring type and popular IDEs provide refactoring tools aiming to support these standard modifications. However, previous studies indicated that developers either frequently avoid using these tools or end up modifying and even reversing the code automatically refactored by IDEs. Thus, developers are forced to manually apply refactorings, which is cumbersome and error-prone. This means that refactoring support may not be entirely aligned with practical needs. The improvement of tooling support for refactoring in practice requires understanding in what ways developers tailor refactoring modifications. To address this issue, we conduct an analysis of 1,162 refactorings composed of more than 100k program modifications from 13 software projects. The results reveal that developers recurrently apply patterns of additional modifications along with the standard ones, from here on called patterns of customized refactorings. For instance, we found customized refactorings in 80.77% of the Move Method instances observed in the software projects. We also investigated the features of refactoring tools in popular IDEs and observed that most of the customization patterns are not fully supported by them. Additionally, to understand the relevance of these customizations, we conducted a survey with 40 developers about the most frequent customization patterns we found. Developers confirm the relevance of customization patterns and agree that improvements in IDE's refactoring support are needed. These observations highlight that refactoring guidelines must be updated to reflect typical refactoring customizations. Also, IDE builders can use our results as a basis to enable a more flexible application of automated refactorings. For example, developers should be able to choose which method must handle exceptions when extracting an exception code into a new method.
Developer-Intent Driven Code Comment Generation

Existing automatic code comment generators mainly focus on producing a general description of functionality for a given code snippet without considering developer intentions. However, in real-world practice, comments are complicated, which often contain information reflecting various intentions of developers, e.g., functionality summarization, design rationale, implementation details, code properties, etc. To bridge the gap between automatic code comment generation and real-world comment practice, we define Developer-Intent Driven Code Comment Generation, which can generate intent-aware comments for the same source code with different intents. To tackle this challenging task, we propose DOME, an approach that utilizes Intent-guided Selective Attention to explicitly select intent-relevant information from the source code, and produces various comments reflecting different intents. Our approach is evaluated on two real-world Java datasets, and the experimental results show that our approach outperforms the state-of-the-art baselines. A human evaluation also confirms the significant potential of applying DOME in practical usage, enabling developers to comment code effectively according to their own needs.
(Partial) Program Dependence Learning

Code fragments from developer forums often migrate to applications due to the code reuse practice. Owing to the incomplete nature of such programs, analyzing them to early determine the presence of potential vulnerabilities is challenging. In this work, we introduce NeuralPDA, a neural network-based program dependence analysis tool for both complete and partial programs. Our tool efficiently incorporates intra-statement and inter-statement contextual features into statement representations, thereby modeling program dependence analysis as a statement-pair dependence decoding task. In the empirical evaluation, we report that NeuralPDA predicts the CFG and PDG edges in complete Java and C/C++ code with combined F-scores of 94.29% and 92.46%, respectively. The F-score values for partial Java and C/C++ code range from 94.29%-97.17% and 92.46%-96.01%, respectively. We also test the usefulness of the PDGs predicted by NeuralPDA (i.e., PDG*) on the downstream task of method-level vulnerability detection. We discover that the performance of the vulnerability detection tool utilizing PDG <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">*</sup> is only 1.1% less than that utilizing the PDGs generated by a program analysis tool. We also report the detection of 14 real-world vulnerable code snippets from StackOverflow by a machine learning-based vulnerability detection tool that employs the PDGs predicted by NeuralPDA for these code snippets.
Reliability Assurance for Deep Neural Network Architectures Against Numerical Defects

With the widespread deployment of deep neural networks (DNNs), ensuring the reliability of DNN-based systems is of great importance. Serious reliability issues such as system failures can be caused by numerical defects, one of the most frequent defects in DNNs. To assure high reliability against numerical defects, in this paper, we propose the RANUM approach including novel techniques for three reliability assurance tasks: detection of potential numerical defects, confirmation of potential-defect feasibility, and suggestion of defect fixes. To the best of our knowledge, RANUM is the first approach that confirms potential-defect feasibility with failure-exhibiting tests and suggests fixes automatically. Extensive experiments on the benchmarks of 63 real-world DNN architectures show that RANUM outperforms state-of-the-art approaches across the three reliability assurance tasks. In addition, when the RANUM-generated fixes are compared with developers' fixes on open-source projects, in 37 out of 40 cases, RANUM-generated fixes are equivalent to or even better than human fixes.
APICAD: Augmenting API Misuse Detection through Specifications from Code and Documents

Using API should follow its specifications. Otherwise, it can bring security impacts while the functionality is damaged. To detect API misuse, we need to know what its specifications are. In addition to being provided manually, current tools usually mine the majority usage in the existing codebase as specifications, or capture specifications from its relevant texts in human language. However, the former depends on the quality of the codebase itself, while the latter is limited to the irregularity of the text. In this work, we observe that the information carried by code and documents can complement each other. To mitigate the demand for a high-quality codebase and reduce the pressure to capture valid information from texts, we present APICAD to detect API misuse bugs of C/C++ by combining the specifications mined from code and documents. On the one hand, we effectively build the contexts for API invocations and mine specifications from them through a frequency-based method. On the other hand, we acquire the specifications from documents by using lightweight keyword-based and NLP-assisted techniques. Finally, the combined specifications are generated for bug detection. Experiments show that APICAD can handle diverse API usage semantics to deal with different types of API misuse bugs. With the help of APICAD, we report 153 new bugs in Curl, Httpd, OpenSSL and Linux kernel, 145 of which have been confirmed and 126 have applied our patches.
Revisiting Learning-based Commit Message Generation

Commit messages summarize code changes and help developers understand the intention. To alleviate human efforts in writing commit messages, researchers have proposed various automated commit message generation techniques, among which learning-based techniques have achieved great success in recent years. However, existing evaluation on learning-based commit message generation relies on the automatic metrics (e.g., BLEU) widely used in natural language processing (NLP) tasks, which are aggregated scores calculated based on the similarity between generated commit messages and the ground truth. Therefore, it remains unclear what generated commit messages look like and what kind of commit messages could be precisely generated by existing learning-based techniques. To fill this knowledge gap, this work performs the first study to systematically investigate the detailed commit messages generated by learning-based techniques. In particular, we first investigate the frequent patterns of the commit messages generated by state-of-the-art learning-based techniques. Surprisingly, we find the majority (~90%) of their generated commit messages belong to simple patterns (i.e., addition/removal/fix/avoidance patterns). To further explore the reasons, we then study the impact of datasets, input representations, and model components. We surprisingly find that existing learning-based techniques have competitive performance even when the inputs are only represented by change marks (i.e., “+”/“-”/“ ”), It indicates that existing learning-based techniques poorly utilize syntax and semantics in the code while mostly focusing on change marks, which could be the major reason for generating so many pattern-matching commit messages. We also find that the pattern ratio in the training set might also positively affect the pattern ratio of generated commit messages; and model components might have different impact on the pattern ratio.
Detecting Exception Handling Bugs in C++ Programs

Exception handling is a mechanism in modern programming languages. Studies have shown that the exception handling code is error-prone. However, there is still limited research on detecting exception handling bugs, especially for C++ programs. To tackle the issue, we try to precisely represent the exception control flow in C++ programs and propose an analysis method that makes use of the control flow to detect such bugs. More specifically, we first extend control flow graph by introducing the concepts of five different kinds of basic blocks, and then modify the classic symbolic execution framework by extending the program state to a quadruple and properly processing try, throw and catch statements. Based on the above techniques, we develop a static analysis tool on the top of Clang Static Analyzer to detect exception handling bugs. We run our tool on projects with high stars from GitHub and find 36 exception handling bugs in 8 projects, with a precision of 84%. We compare our tool with four state-of-the-art static analysis tools (Cppcheck, Clang Static Analyzer, Facebook Infer and IKOS) on projects from GitHub and handmade benchmarks. On the GitHub projects, other tools are not able to detect any exception handling bugs found by our tool. On the handmade benchmarks, our tool has a significant higher recall.
Efektivitas Blended learning dan Diskusi-Presentasi Terhadap Hasil Belajar Ditinjau dari Kemampuan Verbal-Lingustic Mahasiswa

Blended learning is a new method and deemed in accordance with the current ICT developments. In addition to the need for study on the method, blended learning is also felt able to answer the problem of the lack of learning in university that still rely discussion-presentation teaching. This study has the aim to determine whether the results of student learning with blended learning method is better than the method of discussion-presentation viewed from verbal-linguistic ability. This study uses a quantitative approach to the type of quasi- experimental, by trying out of a treatment. Techniques hypothesis testing using analysis of variance (ANOVA) two ways with a 2 x 2 factorial (α = 5%) and the prerequisite test used is the average similarity test, test of normality and homogenity test. The results obtained are (1) the results of student learning with blended learning method is better than the results of student learning with the method of discussion-presentation. (2) There is significant influence levels of verbal-linguistic toward student learning outcomes and (3) the results of student learning with blended learning method is better than the the results of student learning with the method of discussion-presentation, both for students with high or low verbal-linguistic abilities.
Utilizing Implicit Feedback for User Mainstreaminess Evaluation and Bias Detection in Recommender Systems

Bias and fairness issues have attracted considerable attention in recommender systems. From the user’s perspective, intentions to stay or leave heavily depend on the degree of satisfaction with the received recommendation results. Mainstream bias refers to the phenomenon that recommendation algorithms favor mainstream users and provide inferior results to non-mainstream users, which harms user fairness. In recent work, Zhu et al. [24] explore several approaches to evaluate the mainstreaminess of users and show the existence of mainstream bias using implicit feedback data. However, they omit the factor of profile size, which can greatly influence the evaluation. In this paper, we complete the data preprocessing steps missing in the original paper and reproduce the evaluation experiments. In particular, we redesign the setup and present a simple and intuitive evaluation approach with high interpretability. Experimental results show that our method outperforms others with better effectiveness in measuring users’ mainstream level. Finally, we validate the wide existence of mainstream bias and assess its impact on recommendations. Our source code and results are available at https://github.com/Xaiver97/mainstream_evaluation .
Sentiment Analysis of Mobile Apps Using BERT

In this paper, we focus on identifying issues in mobile app updates that adversely impact the opinion in user reviews by analyzing the sentiment of the reviews. We use sentiment analysis using BERT to evaluate the performance of mobile apps and the sentiment distribution of reviews for identifying the cause of sentiment shifts. Using our method, developers can correctly locate the period of specific sentiment and review the sentences and keywords used in reviews to identify the problems and complaints in recent updates. An increase in negative sentiments after any major update can help identify the exact issue causing the problem. Our experimental analysis shows the effectiveness of the proposed method in recognizing issues and identifying any potential problematic updates.
Intent Understanding for Automatic Question Answering in Network Technology Communities Based on Multi-task Learning

In the realm of automatic question-answering (Q&A) for technical communities, accurately perceiving and predicting user intent is a crucial step towards improving Q&A system performance by integrating user intention with answer reasoning processes. We conducted research into intent understanding at the sentence level, aiming to clarify the function of each sentence in technical Q&A communities and improve the system's response accuracy. To address the shortcomings of existing research, which typically ignores information such as speaker type and sentence position, we propose a multi-task learning framework to effectively utilize this information for sentence representation learning. By doing so, the model can acquire richer interactive question-answer language features, thereby enhancing the performance of intent label classification. Within this framework, we present two models: BA-multi and CCR-multi. Our validation experiments on the MSDialog-Intent dataset demonstrate that the multi-task learning model significantly outperforms both the baseline and feature extension models, achieving state-of-the-art performance.
A Survey on Automated Code Evaluation Systems and Their Resources for Code Analysis

The automated code evaluation system is designed to reliably evaluate user-submitted code. Code is first compiled and then tested on a homogeneous surface using defined input and output test cases. Automated code evaluation systems are gaining popularity due to their wide range of applications and valuable accumulated resources. The success of machine learning techniques emboldens researchers to use them for source code analysis tasks, and a large number of real-life solution codes from automated evaluation systems adds significant value. In this paper, we review the state-of-the-art of automated code evaluation systems and their resources for code analysis tasks using machine learning. We classify these code evaluation systems into several categories, including programming contests, programming learning, recruitment, online compilers, and additional modules of other systems. We research the datasets available in these systems for code analysis. Moreover, we summarize the machine learning-based code assessment tasks, including error detection, code comprehension, review, search and representation, refactoring, and repair using these datasets.
Accelerating Deep Reinforcement Learning with Fuzzy Logic Rules

Most deep reinforcement learning methods focus on the agent’s learning performance but ignore the uncertainty in the environment, which would lead to a lot of redundant interaction data and slow learning speed. We propose two novel policy frameworks that extract human knowledge through fuzzy inference systems and incorporate it into deep reinforcement learning to accelerate agent training. We also compared agent learning for two different combinations. Our frameworks allow the agent to choose an action based on a new probability distribution that reduce the negative impact of future events. Thereby helping the agent reduce invalid interaction data and improve learning efficiency. We conduct experiments in multiple game environments, and the result exhibits that our method can effectively reduces training episodes, and achieve a better learning result in the mid-term.
Bootless Application of Greedy Re-ranking Algorithms in Fair Neural Team Formation

Team formation aims to automate forming teams of experts who can successfully solve difficult tasks, which have firsthand effects on creating organizational performance. While existing neural team formation methods are able to efficiently analyze massive collections of experts to form effective collaborative teams, they largely ignore the fairness in recommended teams of experts. Fairness breeds innovation and increases teams’ success by enabling a stronger sense of community, reducing conflict, and stimulating more creative thinking. In this paper, we study the application of state-of-the-art deterministic greedy re-ranking algorithms to mitigate the potential popularity bias in the neural team formation models based on demographic parity. Our experiments show that, first, neural team formation models are biased toward popular experts. Second, although deterministic re-ranking algorithms mitigate popularity bias substantially, they severely hurt the efficacy of teams. The code to reproduce the experiments reported in this paper is available at https://github.com/fani-lab/Adila/tree/bias23 ( , a feminine Arabic given name meaning just and fair.)