Reviewerly/exHarmony huggingface dataset:
Learning to Locate and Describe Vulnerabilities

Automatically discovering software vulnerabilities is a long-standing pursuit for software developers and security analysts. Since detection tools usually provide limited information for vulnerability inspection, recent work turns the attention to identify fine-grained vulnerabilities, i.e., vulnerable statements. However, existing work for vulnerability localization struggles to capture long-range and integral dependency information due to the bottleneck of Graph Neural Networks (GNNs). Moreover, little research has been done to help developers understand detected vulnerabilities, leaving vulnerability diagnosis a challenging task. In this paper, we propose VulTeller, a deep learning-based approach that can automatically locate vulnerable statements in a function and more importantly, can describe the vulnerability. Our approach focuses on extracting precise control and data dependencies in the code, achieved through modeling control flow paths and employing taint analysis. We design a novel neural model that encodes the control flows and taint flows which reside in the control flow paths, and decodes them via node classification and an attentional decoder for the two tasks respectively. We conduct extensive experiments with real-world vulnerabilities to evaluate the proposed approach. The evaluation results, including quantitative measurement and human evaluation, demonstrate that our approach is highly effective and outperforms state-of-the-art approaches. Our work for the first time formulates the problem of vulnerability description generation, and makes one step further towards automated vulnerability diagnosis.
Optimizing Continuous Development by Detecting and Preventing Unnecessary Content Generation

Continuous development (CD) helps developers quickly release and update their software. To enact CD, developers customize their CD builds to perform several tasks, including compiling, testing, static analysis checks, etc. However, as developers add more tasks to their builds, the builds take longer to run, therefore slowing down the entire CD process. Furthermore, developers may unknowingly include tasks into their builds whose results are not used (e.g., generating coverage files that are never read or uploaded anywhere), therefore wasting build runtime doing unnecessary tasks. We propose OptCD, a technique to dynamically detect unnecessary work within CD builds. Our intuition is that unnecessary work can be identified by the generation of files that are not used by any other task within the build. OptCD runs alongside a CD build, tracking the generated files during the build and which files are read/written. Files that are written to but are never read from are unnecessary content from a build. Based on the names of the unnecessary files, OptCD then maps the files to the specific build tasks responsible for generating or writing to those files. Finally, OptCD leverages ChatGPT to suggest changing the build configuration to disable generating these unnecessary files. Our evaluation of OptCD on 22 open-source projects finds that 95.6% of projects generate at least one unused directory, a directory whose contents are all unnecessarily generated. OptCD identifies the correct task that generates 92.0% of the unused directories. Further, OptCD can produce a patch for the CD configuration file to prevent generating 72.0% of the unused directories. Using the patches, we reduce the runtime by 7.0% on average for the projects we studied. We submitted 26 pull requests for the unused directories that we could disable. Developers have accepted 12 of them, with five rejected, and nine still pending.
ASTER: Automatic Speech Recognition System Accessibility Testing for Stutterers

The popularity of automatic speech recognition (ASR) systems nowadays leads to an increasing need for improving their accessibility. Handling stuttering speech is an important feature for accessible ASR systems. To improve the accessibility of ASR systems for stutterers, we need to expose and analyze the failures of ASR systems on stuttering speech. The speech datasets recorded from stutterers are not diverse enough to expose most of the failures. Furthermore, these datasets lack ground truth information about the non-stuttered text, rendering them unsuitable as comprehensive test suites. Therefore, a methodology for generating stuttering speech as test inputs to test and analyze the performance of ASR systems is needed. However, generating valid test inputs in this scenario is challenging. The reason is that although the generated test inputs should mimic how stutterers speak, they should also be diverse enough to trigger more failures. To address the challenge, we propose Aster, a technique for automatically testing the accessibility of ASR systems. Aster can generate valid test cases by injecting five different types of stuttering. The generated test cases can both simulate realistic stuttering speech and expose failures in ASR systems. Moreover, Aster can further enhance the quality of the test cases with a multi-objective optimization-based seed updating algorithm. We implemented Aster as a framework and evaluated it on four open-source ASR models and three commercial ASR systems. We conduct a comprehensive evaluation of Aster and find that it significantly increases the word error rate, match error rate, and word information loss in the evaluated ASR systems. Additionally, our user study demonstrates that the generated stuttering audio is indistinguishable from real-world stuttering audio clips.
VD-Guard: DMA Guided Fuzzing for Hypervisor Virtual Device

Virtualization has been widely used in various scenarios, such as cloud computing. As its core technology, virtualization hypervisor brings up the efficiency of sharing the physical machine's resources via virtual devices. However, virtualization hypervisor also introduces significant security risks due to defective design or implementation schemes on virtual devices. Although several methods have been proposed to detect vulnerabilities in virtual devices, they still cannot effectively discover them because of missing critical information related to the MMIO/PIO and DMA operations to guide their dynamic methods. In this paper, we propose a hybrid method, VD-GUARD, to detect vulnerabilities in virtual devices. Specifically, it first leverages static control flow analysis to track call traces from various data entry points of virtual devices (MMIO/PIO functions) to the critical dispatcher points (DMA functions), and generate seeds that can trigger this call trace via static analysis and limited fuzzing test. And then, it takes these seeds as input and leverages DMA guided fuzzing to discover bugs. To verify the effectiveness of Vd-guard, we build a dataset, including 10 bugs in QEMU, based on previous works, and Vd-guardoutperforms the state-of-the-art hypervisor fuzzer Morphuzz. Vd-guardalso has found 4 new vulnerabilities in QEMU and VirtualBox, all of which have been confirmed and fixed (have been assigned 3 CVE IDs).
RegAgg: A Scalable Approach for Efficient Weight Aggregation in Federated Lesion Segmentation of Brain MRIs

Federated learning (FL), a distributed machine learning paradigm, enables multiple clients to train models locally on their data while only sharing model updates with the central server. However, this process can be computationally intensive and resource-consuming, leading to communication overheads and scalability issues. In this study, we investigate the robustness and reliability of Regularized Weight Aggregation (RegAgg) and compare its performance with an additional regularization layer while aggregating model weights for the federated segmentation of cancer tumors in brain MRIs. Our method is evaluated using a large set of brain image data which consists of multi-institutional multi-parametric magnetic resonance imaging (mpMRI) scans. Through extensive experimentation, we demonstrate that the performance of federated lesion segmentation is significantly improved with RegAgg, surpassing the results obtained when an additional regularization process is introduced. The findings highlight the efficacy of RegAgg as a powerful strategy for advancing brain tumor segmentation in FL settings. Notably, RegAgg provides scalability, cost-effectiveness, and the ability to adapt to collaborators with heterogeneous non-IID characteristics. It enables efficient resource allocation and collaborator selection, contributing to enhanced performance in dynamic brain tumor segmentation. Overall, this research contributes to the development of efficient weight aggregation techniques for dynamic brain tumor segmentation in FL. The proposed approach demonstrates its potential to address the challenges associated with collaborative training on distributed brain MRI data while maintaining accuracy and privacy.
A Shared Memory SMC Sampler for Decision Trees

Modern classification problems tackled by using Decision Tree (DT) models often require demanding constraints in terms of accuracy and scalability. This is often hard to achieve due to the ever-increasing volume of data used for training and testing. Bayesian approaches to DTs using Markov Chain Monte Carlo (MCMC) methods have demonstrated great accuracy in a wide range of applications. However, the inherently sequential nature of MCMC makes it unsuitable to meet both accuracy and scaling constraints. One could run multiple MCMC chains in an embarrassingly parallel fashion. Despite the improved run-time, this approach sacrifices accuracy in exchange for strong scaling. Sequential Monte Carlo (SMC) samplers are another class of Bayesian inference methods that also have the appealing property of being parallelizable without trading off accuracy. Nevertheless, finding an effective parallelization for the SMC sampler is difficult, due to the challenges in parallelizing its bottleneck, redistribution, in such a way that the workload is equally divided across the processing elements, especially when dealing with variable-size models such as DTs. This study presents a parallel SMC sampler for DTs on Shared Memory (SM) architectures, with an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$O(log_{2} N)$</tex> parallel redistribution for variable-size samples. On an SM machine mounting 32 cores, the experimental results show that our proposed method scales up to a factor of 16 compared to its serial implementation, and provides comparable accuracy to MCMC, but 51 times faster.
Distributed Edge Machine Learning Pipeline Scheduling with Reverse Auctions

Scheduling distributed machine learning pipelines in edge environments is a growing area of research as developers work to bring large, high-accuracy models to relatively low-powered devices. Edge environment dynamics, such as device availability and connectivity, make distributed scheduling a more challenging problem than in traditional cloud environments. Existing approaches usually require significant a priori knowledge of the environment and make assumptions about model availability, both of which are impractical in real edge deployments. We address this problem by proposing a simple and efficient reverse auction algorithm, where a device that wants to distribute a large machine learning workload requests bids from available resources in the environment to construct connected pipelines. We implement our reverse auction scheduling on an existing distributed machine learning pipeline framework and perform an empirical evaluation using a real distributed edge computing testbed. We prove that scheduling distributed pipelines without repeating devices is an NP-complete problem, but that finding good latency or throughput pipelines is tractable for fixed device orderings.
Enhancing Code Safety in Quantum Intermediate Representation

Quantum Intermediate Representation (QIR) is an LLVM-based intermediate representation developed by Microsoft for quantum program compilers. QIR is designed to offer a universal solution for quantum program compilers, decoupled from both front-end languages and back-end hardware, thereby eliminating the need for redundant development of intermediate representations and compilers. However, the lack of a formal definition and reliance on natural language descriptions in the current state of QIR result in interpretational ambiguity and a dearth of rigor in implementing quantum functions. In this paper, we present formal definitions for QIR's data types and instruction sets to establish correctness and safety assurances for operations and intermediate code conversions within QIR. To demonstrate the effectiveness of our approach, we provide examples of unsafe QIR codes where errors can be identified with our method.
Scene-Driven Exploration and GUI Modeling for Android Apps

Due to the competitive environment, mobile apps are usually produced under pressure with lots of complicated functionality and UI pages. Therefore, it is challenging for various roles to design, understand, test, and maintain these apps. The extracted transition graphs for apps such as ATG, WTG, and STG have a low transition coverage and coarse-grained granularity, which limits the existing methods of graphical user interface (GUI) modeling by UI exploration. To solve these problems, in this paper, we propose SceneDroid, a scene-driven exploration approach to extracting the GUI scenes dynamically by integrating a series of novel techniques including smart exploration, state fuzzing, and indirect launching strategies. We present the GUI scenes as a scene transition graph (SceneTG) to model the GUI of apps with high transition coverage and fine-grained granularity. Compared with the existing GUI modeling tools, SceneDroid has improved by 168.74% in the coverage of transition pairs and 162.42% in scene extraction. Apart from the effectiveness evaluation of SceneDroid, we also illustrate the future potential of SceneDroid as a fundamental capability to support app development, reverse engineering, and GUI rearession testing.
Better Patching Using LLM Prompting, via Self-Consistency

Large Language models (LLMs) can be induced to solve non-trivial problems with “few-shot” prompts including illustrative problem-solution examples. Now if the few-shots also include “chain of thought” ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{C}oT$</tex> ) explanations, which are of the form problem-explanation-solution, LLMs will generate a “explained” solution, and perform even better. Recently an exciting, substantially better technique, self-consistency [1] ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{S}-C$</tex> ) has emerged, based on the intuition that there are many plausible explanations for the right solution; when the LLM is sampled repeatedly to generate a pool of explanation-solution pairs, for a given problem, the most frequently occurring solutions in the pool (ignoring the explanations) tend to be even more likely to be correct! Unfortunately, the use of this highly-performant <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{S}-C$</tex> (or even <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{C}oT$</tex> ) approach in software engineering settings is hampered by the lack of explanations; most software datasets lack explanations. In this paper, we describe an application of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathcal{S}-C$</tex> approach to program repair, using the commit log on the fix as the explanation, only in the illustrative few-shots. We achieve state-of-the art results, beating previous approaches to prompting-based program repair, on the MODIT dataset; we also find evidence suggesting that the correct commit messages are helping the LLM learn to produce better patches.
A Federated Learning Benchmark on Tabular Data: Comparing Tree-Based Models and Neural Networks

Federated Learning (FL) has lately gained traction as it addresses how machine learning models train on distributed datasets. FL was designed for parametric models, namely Deep Neural Networks (DNNs). Thus, it has shown promise on image and text tasks. However, FL for tabular data has received little attention. Tree-Based Models (TBMs) have been considered to perform better on tabular data and they are starting to see FL integrations. In this study, we benchmark federated TBMs and DNNs for horizontal FL, with varying data partitions, on 10 well-known tabular datasets. Our novel benchmark results indicates that current federated boosted TBMs perform better than federated DNNs in different data partitions. Furthermore, a federated XGBoost outperforms all other models. Lastly, we find that federated TBMs perform better than federated parametric models, even when increasing the number of clients significantly.
Are We Ready to Embrace Generative AI for Software Q&amp;A?

Stack Overflow, the world's largest software Q&A (SQA) website, is facing a significant traffic drop due to the emergence of generative AI techniques. ChatGPT is banned by Stack Overflow after only 6 days from its release. The main reason provided by the official Stack Overflow is that the answers generated by ChatGPT are of low quality. To verify this, we conduct a comparative evaluation of human-written and ChatGPT-generated answers. Our methodology employs both automatic comparison and a manual study. Our results suggest that human-written and ChatGPT-generated answers are semantically similar, however, human-written answers outperform ChatGPT-generated ones consistently across multiple aspects, specifically by 10% on the overall score. We release the data, analysis scripts, and detailed results at https://github.com/maxxbw54/GAI4SQA.
Genetic Algorithm-Based Dynamic Backdoor Attack on Federated Learning-Based Network Traffic Classification

Federated learning enables multiple clients to collaboratively contribute to the learning of a global model orchestrated by a central server. This learning scheme promotes clients' data privacy and requires reduced communication overheads. In an application like network traffic classification, this helps hide the network vulnerabilities and weakness points. However, federated learning is susceptible to backdoor attacks, in which adversaries inject manipulated model updates into the global model. These updates inject a salient functionality in the global model that can be launched with specific input patterns. Nonetheless, the vulnerability of network traffic classification models based on federated learning to these attacks remains unexplored. In this paper, we propose GABAttack, a novel genetic algorithm-based backdoor attack against federated learning for network traffic classification. GABAttack utilizes a genetic algorithm to optimize the values and locations of backdoor trigger patterns, ensuring a better fit with the input and the model. This input-tailored dynamic attack is promising for improved attack evasiveness while being effective. Extensive experiments conducted over real-world network datasets validate the success of the proposed GABAttack in various situations while maintaining almost invisible activity. This research serves as an alarming call for network security experts and practitioners to develop robust defense measures against such attacks.
Security and Privacy Threat Analysis for Solid

This paper provides an in-depth security and privacy analysis of the Solid protocol. Solid is a specification that allows user data to be stored decentralized in a personal online datastore (pod) independent from the application. This allows users to easily migrate to a different service and have more control over who data is shared with. We provide a comprehensive overview of the authentication, identification, and authorization protocols within Solid. We make use of the SPARTA threat modeling tool to assess the security and privacy aspects of Solid by modeling a realistic finance analytics application envisioned by the Solid community. This concrete use case allowed us to prioritize the residual threats in Solid. We employ methodologies such as STRIDE and LINDDUN for robust security and privacy threat modeling. The findings highlight the existence of several critical threats in the Solid specification. This is especially the case for privacy threats, which although it is an essential aspect of Solid, has so far not yet received enough attention, as our results indicate. These findings can be employed in future work to prioritize which residual threats to address and mitigate first.
Understanding and Enhancing Issue Prioritization in GitHub

GitHub has become a prominent platform for open source software development, facilitating collaboration and communication among a diverse group of contributors. Efficient issue tracking is a crucial aspect of managing projects on GitHub, and labels serve as one of the primary mechanisms for issue prioritization, while various other issue features are also utilized by issue handlers for the same purpose. However, in large projects, prioritizing issues remains a challenge, and the efficacy of using labels or other issue features for prioritization is not well understood. To address this knowledge gap, we conduct a comprehensive empirical study that investigates the role of labels in GitHub issue prioritization, examines the influence of various issue features on prioritization, and assesses the performance of different ranking algorithms based on these impactful features. Our study, conducted on a dataset comprising data from over 1.5 million issues across diverse GitHub projects, provides valuable insights for issue handling in open source platforms and offers guidance for future research in this domain. Specifically, the study reveals the limited effectiveness of labels in issue prioritization, highlights the significance of certain issue features in the prioritization process, and compares the performance of various ranking algorithms for issue prioritization to support issue handlers.
PRICAR: Privacy Framework for Vehicular Data Sharing with Third Parties

In-vehicle infotainment (IVI) platforms are getting increasingly connected. Besides Original Equipment Manufacturer (OEM) apps and services, the next generation of IVI platforms are expected to allow integration of third-party apps, such as usage-based insurance (UBI). Under this anticipated business model, vehicular sensor and event data can be collected and shared with selected third-party apps. However, third-parties can be malicious and have easier access to the driver’s data. Several research projects and commercial products also show the possibility of leaking sensitive private information such as vehicle location via seemingly benign vehicular sensors, which can, in turn, harm the driver’s privacy. Furthermore, increasing privacy regulations worldwide, such as GDPR, make privacy a major issue for the automotive industry. To overcome these problems, we present PRICAR, a framework for privacy-preserving vehicular data collection and sharing with third-parties. It enforces three data privacy goals — minimization, anonymization and sanitization — while focusing on the last one. We describe and evaluate how to sanitize user data before sharing it with a third-party by adapting two well-known techniques from anomaly detection, Change-Point Detection (CPD) and Entropy-Based Detection (EBD).