Reviewerly/exHarmony huggingface dataset:
A case study of fairness in generated images of Large Language Models for Software Engineering tasks

Bias in Large Language Models (LLMs) has significant implications. Since they have revolutionized content creation on the web, they can lead to more unfair outcomes, lack of inclusivity, reinforcement of stereotypes and ethical and legal concerns. Notably, OpenAI has recently made claims they have introduced a new technique to ensure that DALL-E-2 generates images of people accurately reflect the diversity of the world’s population. In order to investigate bias within the field of Software Engineering, the study utilized DALL-E-2 image generation to assess 56 tasks related to software engineering. Another objective was to determine the impact of OpenAI’s new measures on the generated images for these specific tasks. Two sets of experiments were conducted. In one set, the tasks were prefixed with the clause "As a Software Engineer," while in the other set, only the tasks themselves were used. The tasks were presented in a gender-neutral manner, and the AI was instructed to generate images for each task 20 times. For a female-dominant task of doing administrative tasks, 40 more images were generated. The study revealed a large gender bias in the 2,280 images generated. For instance, in the subset of experiments with prompts explicitly incorporating the phrase "As a software engineer," only 2% of the generated images portrayed female protagonists. In all the images in this setting, male protagonists were dominant and in 45 tasks 100% of the protagonists were male. Notably, images generated without the prefixed clause only had more female protagonists in ‘provide comments on project milestones’ and ‘provide enhancements’, while other tasks did not exhibit a similar pattern. The findings emphasize unsuitability of implemented guardrails and the importance of further research on LLMs assessments. Further research is needed in LLMs to find out where their guardrails fail so companies can address them properly.
Integrating Visual Aids to Enhance the Code Reviewer Selection Process

Modern Code Review (MCR) is an integral part of a software development strategy that accelerates product quality by identifying defects, code smells, and other harmful practices. However, assigning appropriate reviewers to evaluate changed code during the review process remains challenging. While automated tools for reviewer assignments have limited impact in practice, the process often relies on manual investigation of project histories to retrieve knowledge of team members and their activities. Therefore, in this study, we present an approach to automatically assemble developers’ information and visualize it meaningfully, which helps to choose appropriate reviewers. First, we propose three metrics that measure developers’ collaboration, reviewers’ expertise, and reviewers’ workload and visualize them through networks. Second, we perform a case study of three popular open-source projects, where we compute and visualize each developers’ information according to the proposed metrics. Finally, we conducted two online surveys to assess the developers’ perceptions of the proposed visual benefits. The results show that the proposed method can assist in identifying relevant reviewers and be immensely helpful to new developers. Additionally, survey respondents expressed reliance on the efficacy of the visual aids in workload balancing and reducing review time.
CCBERT: Self-Supervised Code Change Representation Learning

Numerous code changes are made by developers in their daily work, and a superior representation of code changes is desired for effective code change analysis. Recently, Hoang et al. proposed CC2Vec, a neural network-based approach that learns a distributed representation of code changes to capture the semantic intent of the changes. Despite demonstrated effectiveness in multiple tasks, CC2Vec has several limitations: 1) it considers only coarse-grained information about code changes, and 2) it relies on log messages rather than the self-contained content of the code changes. In this work, we propose CCBERT (Code Change BERT), a new Transformer-based pre-trained model that learns a generic representation of code changes based on a large-scale dataset containing massive unlabeled code changes. CCBERT is pre-trained on four proposed self-supervised objectives that are specialized for learning code change representations based on the contents of code changes. CCBERT perceives fine-grained code changes at the token level by learning from the old and new versions of the content, along with the edit actions. Our experiments demonstrate that CCBERT significantly outperforms CC2Vec or the state-of-the-art approaches of the downstream tasks by 7.7%–14.0% in terms of different metrics and tasks. CCBERT consistently outperforms large pre-trained code models, such as CodeBERT, while requiring 6–10× less training time, 5–30× less inference time, and 7.9× less GPU memory.
Finding an Optimal Set of Static Analyzers To Detect Software Vulnerabilities

Software vulnerabilities are ubiquitous and costly. To detect vulnerabilities earlier during development, organizations deploy a set of static analyzers to locate and eventually fix these vulnerabilities before releasing their software. Due to the prohibitive cost of running all available analyzers, organizations must run only a subset of all possible analyzers on their codebases. Choosing this set deterministically leaves recognizable gaps of vulnerability coverage. To overcome these challenges, we present Randomized Best Response (RBR), a method that computes an optimal randomization over size-bounded sets of available static analyzers. RBR models the relationship between malicious users and organizations as a leader-follower Stackelberg security game. Our solution focuses on software vulnerabilities due to their security implications when exploited by malicious users. Using 8 static analyzers for C/C++ and 8 Common Weakness Enumeration (CWE) vulnerability types, we show that RBR outperforms a set of natural baselines by always picking analyzers that achieve a higher benefit to the defender. Through a case study of a large system at Oracle, we show how RBR may be used in a real-world scenario.
Exploring Security Commits in Python

Python has become the most popular programming language as it is friendly to work with for beginners. However, a recent study has found that most security issues in Python have not been indexed by CVE and may only be fixed by "silent" security commits, which pose a threat to software security and hinder the security fixes to downstream software. It is critical to identify the hidden security commits; however, the existing datasets and methods are insufficient for security commit detection in Python, due to the limited data variety, non-comprehensive code semantics, and uninterpretable learned features. In this paper, we construct the first security commit dataset in Python, namely PySecDB, which consists of three subsets including a base dataset, a pilot dataset, and an augmented dataset. The base dataset contains the security commits associated with CVE records provided by MITRE. To increase the variety of security commits, we build the pilot dataset from GitHub by filtering keywords within the commit messages. Since not all commits provide commit messages, we further construct the augmented dataset by understanding the semantics of code changes. To build the augmented dataset, we propose a new graph representation named CommitCPG and a multi-attributed graph learning model named SCOPY to identify the security commit candidates through both sequential and structural code semantics. The evaluation shows our proposed algorithms can improve the data collection efficiency by up to 40 percentage points. After manual verification by three security experts, PySecDB consists of 1,258 security commits and 2,791 non-security commits. Furthermore, we conduct an extensive case study on PySecDB and discover four common security fix patterns that cover over 85% of security commits in Python, providing insight into secure software maintenance, vulnerability detection, and automated program repair.
Bugsplainer: Leveraging Code Structures to Explain Software Bugs with Neural Machine Translation

Software bugs cost the global economy billions of dollars each year and take up ≈50% of the development time. Once a bug is reported, the assigned developer attempts to identify and understand the source code responsible for the bug and then corrects the code. Over the last five decades, there has been significant research on automatically finding or correcting software bugs. However, there has been little research on automatically explaining the bugs to the developers, which is essential but a highly challenging task. In this paper, we propose Bugsplainer, a novel web-based debugging solution that generates natural language explanations for software bugs by learning from a large corpus of bug-fix commits. Bugsplainer leverages code structures to reason about a bug and employs the fine-tuned version of a text generation model – CodeT5 – to generate the explanations.Tool video: https://youtu.be/xga-ScvULpk
Developers and Modern Communication Mediums

As part of their daily work, software developers use information that has been traditionally sourced from code, version control repositories, colleagues, emails, and online documentation. However, recently developers have started using information from modern communication mediums like online forums, videos, and chat platforms. The rising usage of these platforms, fueled by online learning and remote work, offers vast benefits but also necessitates significant effort to locate task-specific knowledge.Modern communication mediums such as online videos and instant messaging platforms have become more prevalent among developers than ever before. Although these mediums provide a large array of benefits to developers, they can also require a significant amount of effort spent searching and browsing through the information they contain to find the knowledge that is relevant to the developers’ current tasks and needs.My work aims to design mechanisms and systems that assist developers in extracting information from videos and instant messaging. In particular, i) use information retrieval algorithms to produce meaningful tags for software development videos and ii) present evidence on how developers interact with instant messaging platforms.
An Enhancing Semi-Supervised Federated Learning Framework for Internet of Vehicles

With the growing computing power in the Internet of Vehicles (IoV), machine learning is increasingly utilized. Yet, IoV faces challenges like privacy, security, and trust issues between vehicles and infrastructure, hindering efficient information usage and machine learning. This paper introduces Semi-Supervised Federated Learning (SSFL) for object recognition in IoV. The proposed approach is designed to enhance the generalization capability and improve the performance of the algorithm by adapting the SSFL framework to the specific characteristics of IoV data deployment and algorithms. A teacher-student structured approach leverages labeled and unlabeled data, and a deployment scheme optimizes training. Results surpass traditional methods, promising improved IoV object recognition accuracy and efficiency.
Quality-of-Trust in 6G: Combining Emotional and Physical Trust through Explainable AI

Wireless networks like many multi-user services have to balance limited resources in real-time. In 6G, increased network automation makes consumer trust crucial. Trust is reflect in both a personal emotional sentiment as well as a physical understanding of the transparency of AI decision making. Whilst there has been isolated studies of consumer sentiment to wireless services, this is not well linked to the decision making engineering. Likewise, limited recent research in explainable AI (XAI) has not established a link to consumer perception.Here, we develop a Quality-of-Trust (QoT) KPI that balances personal perception with the quality of decision explanation. That is to say, the QoT varies with both the time-varying sentiment of the consumer as well as the accuracy of XAI outcomes. We demonstrate this idea with an example in Neural Water-Filling (N-WF) power allocation, where the channel capacity is perceived by artificial consumers that communicate through Large Language Model (LLM) generated text feedback. Natural Language Processing (NLP) analysis of emotional feedback is combined with a physical understanding of N-WF decisions via meta-symbolic XAI. Combined they form the basis for QoT. Our results show that whilst the XAI interface can explain up to 98.9% of the neural network decisions, a small proportion of explanations can have large errors causing drops in QoT. These drops have immediate transient effects in the physical mistrust, but emotional perception of consumers are more persistent. As such, QoT tends to combine both instant physical mistrust and long-term emotional trends.
An Automated Code Update Tool For Python Packages

The adoption of libraries provides developers with pre-existing functionality that is both robust and easy to use. As a code base grows over time, it is natural that the libraries become stale and require updates in order to sustain innovation. Nonetheless, updating a library comes at a cost; it can potentially introduce breaking changes in the code. Thus it may require a large portion of developer time for maintenance. To alleviate this, we propose to use artificial intelligence to parse release notes documentation and automatically recommend code updates to become compatible with new versions. The solution comes in the form of an IDE plugin that can automatically detect deprecated library usages in live code bases and suggest the recommended fixes in a user-friendly way. The system architecture is comprised of three components: a web crawler that sources and pre-processes library deprecation texts, a deprecations parser that transforms those texts into a structured form, and an IDE plugin that finds and updates the parsed deprecations in a code base. In order to validate our approach, we collect and annotate a dataset of 426 API deprecations from 7 popular Python libraries and obtain an overall average weighted subtree overlap of 31.3 and 45.9 on method deprecations. Finally, we demo our tool in the form of a plugin for the industry leading Python IDE PyCharm and test it on 33 internal repositories at J.P Morgan Chase.
Knowledge Graph based Explainable Question Retrieval for Programming Tasks

Developers often seek solutions for their programming problems by retrieving existing questions on technical Q&A sites such as Stack Overflow. In many cases, they fail to find relevant questions due to the knowledge gap between the questions and the queries or feel it hard to choose the desired questions from the returned results due to the lack of explanations about the relevance. In this paper, we propose KGXQR, a knowledge graph based explainable question retrieval approach for programming tasks. It uses BERT-based sentence similarity to retrieve candidate Stack Overflow questions that are relevant to a given query. To bridge the knowledge gap and enhance the performance of question retrieval, it constructs a software development related concept knowledge graph and trains a question relevance prediction model to re-rank the candidate questions. The model is trained based on a combined sentence representation of BERT-based sentence embedding and graph-based concept embedding. To help understand the relevance of the returned Stack Overflow questions, KGXQR further generates explanations based on the association paths between the concepts involved in the query and the Stack Overflow questions. The evaluation shows that KGXQR outperforms the baselines in terms of accuracy, recall, MRR, and MAP and the generated explanations help the users to find the desired questions faster and more accurately.
Predicting Defective Visual Code Changes in a Multi-Language AAA Video Game Project

Video game development increasingly relies on using visual programming languages as the primary way to build video game features. The aim of using visual programming is to move game logic into the hands of game designers, who may not be as well versed in textual coding. In this paper, we empirically observe that there are more defect-inducing commits containing visual code than textual code in a AAA video game project codebase. This indicates that the existing textual code Just-in-Time (JIT) defect prediction models under evaluation by Electronic Arts (EA) may be ineffective as they do not account for changes in visual code. Thus, we focus our research on constructing visual code defect prediction models that encompass visual code metrics and evaluate the models against defect prediction models that use language agnostic features, and textual code metrics. We test our models using features extracted from the historical codebase of a AAA video game project, as well as the historical codebases of 70 open source projects that use textual and visual code. We find that defect prediction models have better performance overall in terms of the area under the ROC curve (AUC), and Mathews Correlation Coefficient (MCC) when incorporating visual code features for projects that contain more commits with visual code than textual code.
Capturing Contextual Relationships of Buggy Classes for Detecting Quality-Related Bugs

Quality concerns are critical for addressing system-wide issues related to reliability, security, and performance, among others. However, these concerns often become scattered across the codebase, making it challenging for software developers to effectively address quality bugs. In this paper, we propose a holistic approach to detecting and clustering quality-related content hidden within the codebase. By leveraging the Hierarchical Dirichlet Process (HDP) and complementary techniques such as information retrieval and machine learning, including structural and textual analysis, we create a meaningful hierarchy that detects classes containing relevant information for addressing quality bugs. This approach allows us to uncover rich synergies between complex structured artifacts and infer bug-fixing classes for repairing quality bugs. The reported results show that our approach improves over the state-of-the-art achieving a high precision of 83%, recall of 82%, and F1 score of 83%.
Automated Extraction of Security Profile Information from XAI Outcomes

Security applications use machine learning (ML) models and artificial intelligence (AI) to autonomously protect systems. However, security decisions are more impactful if they are coupled with their rationale. The explanation behind an ML model's result provides the rationale necessary for a security decision. Explainable AI (XAI) techniques provide insights into the state of a model's attributes and their contribution to the model's results to gain the end user's confidence. It requires human intervention to investigate and interpret the explanation. The interpretation must align system's security profile(s). A security profile is an abstraction of the system's security requirements and related functionalities to comply with them. Relying on human intervention for interpretation is infeasible for an autonomous system (AS) since it must self-adapt its functionalities in response to uncertainty at runtime. Thus, an AS requires an automated approach to extract security profile information from ML model XAI outcomes. The challenge is unifying the XAI outcomes with the security profile to represent the interpretation in a structured form. This paper presents a component to facilitate AS information extraction from ML model XAI outcomes related to predictions and generating an interpretation considering the security profile.
A lightweight attention-guided semantic segmentation network using a group convolution pyramid

Semantic segmentation is a key part of computer vision tasks, but its application is greatly limited by memory capacity and computational cost. This paper proposes a lightweight attention-guided semantic segmentation network (LAGNet) that adopts a joint group convolution pyramid strategy. Specifically, we introduced a lightweight symmetric double attention module (LS-DAM) and an adaptive selection interaction module (ASIM). In the LS-DAM, the self-attention adopts adaptive maximum pooling and average pooling instead of its previous form, in which the pixel-by-pixel similarity is calculated. Moreover, The ASIM has the capability to seamlessly integrate both high-level semantic information and low-level geometric information, leading to a remarkable enhancement in the performance of semantic segmentation. We evaluate our proposed LAGNet model using mean intersection over union (mIoU) on PASCAL VOC 2012 and Cityscapes, two commonly used datasets in the field of semantic segmentation, and finally achieve state-of-the-art performance of 81.73% and 81.90%, respectively.
GPTERS: GPT with Entity Relation Enhanced for Abstractive Text Summarization

In this paper, we propose GPTERS: GPT with Entity Relation Enhanced for Abstractive Text Summarization to improve the performance of text summary pre-trained model by combining article, abstract and entity relation (new) simultaneously as inputs to the model, and by adding the Entity Relation fine-tuning task with the Abstract fine-tuning task to fully learn the structural differences between the article, the abstract, and the entity relation, the abstract generation can maintain factual consistency to a certain extent, thus improving the performance of the abstract generation. Finally, the experimental data show that the model improves the accuracy of summary results and shows good results when the features of entity relations are incorporated.
Modelling Adaptive Autonomic Cloud Application Utility Using Template Functions

Autonomic decisions are necessary for persistent Cloud application adaptation due to dynamic execution context and changing workload. However, it is often difficult to accurately model the utility to represent the application owner's preferences as a mathematical function linking the utility with the monitored information from the application. We propose a systematic approach for utility function modelling for adaptive applications in the Cloud continuum. This method exploits a set of utility function templates, automated quality checks, and the impact of measurements and performance indicators values on the utility value range. The method is evaluated with an illustrative example of a Cloud application that is compared to an existing manually modelled utility function.
Towards a Catalog of Refactorings for Elixir

Elixir is an emerging functional programming language that is gaining popularity in the industry. However, to the best of our knowledge, no study has yet presented a specialized catalog of refactoring strategies specifically tailored for this language. Therefore, this paper aims to address this research gap by conducting a systematic literature review to explore whether there are existing refactoring strategies, compatible with Elixir, that have been proposed for other functional languages or languages that served as inspiration for the development of Elixir. Our preliminary results indicate that there are 54 refactoring strategies compatible with Elixir code, thus forming a comprehensive catalog of refactoring techniques tailored specifically for this language. To illustrate the application of these refactoring strategies, we have provided code examples that show the transformations resulting from each refactoring. Additionally, these refactorings have been categorized into three distinct groups based on the specific programming features required for the respective code transformations.