bluuebunny/crossref_metadata_2025 huggingface dataset:
Natural Products from Brazilian Biodiversity Explored as Anti-EBV Drug Candidates: In-Silico Database Mining, Docking Computations, Molecular Dynamics and DFT Calculations
Authors: Hassan Alaa M. A. , Sidhom Peter A. , Mekhemer Gamal A. H. , El-Tayeb Mohamed A. , Hegazy Mohamed-Elamir F. , Ibrahim Mahmoud A. A.
Published: July 2024

A DNA tumor virus called Epstein-Barr virus (EBV) is the cause of 1-2% of human cancers. EBV-associated carcinogenesis is caused by a persistent latent infection. The shortage of antiviral medications or vaccinations to control the virus led to a significant percentage of critically sickness individuals needing to be hospitalized. In the absence of an effective vaccine and approved drug, there is an immediate need for treatments that can combat EBV infection. Epstein Barr nuclear antigen 1 (EBNA1) is continually expressed in all malignancies linked to EBV; it is a desirable target for curative interference. Herein, natural product compounds from the Nuclei of Bioassays, Ecophysiology and Biosynthesis of Natural Products (NuBBE) database were mined to identify potent EBNA1 inhibitors. The performance of the molecular docking technique was initially assessed in expecting the ligand-target binding pose according to the available experimental data. On the basis of the estimated docking protocol, the NuBBE database was virtually screened toward the EBNA1 protein, and the natural compounds with binding scores less than KWG (calc. [Formula: see text] kcal/mol) were subjected to molecular dynamics (MD) simulations followed by binding energy ([Formula: see text]) calculation using MM/GBSA approach. Based on binding energy computations, NuBBE1460 revealed superior [Formula: see text] with a value of [Formula: see text] kcal/mol, compared to KWG (calc. [Formula: see text] kcal/mol). Post-MD analyses displayed a high steadiness of NuBBE1460 within the EBNA1 protein binding pocket. Furthermore, the physicochemical, pharmacokinetic and toxicity features of the identified compound were predicted, demonstrating high degrees of its oral bioavailability. The energetical and geometrical properties of NuBBE1460 were calculated using the DFT method. Conclusively, this study offers NuBBE1460 obtained from natural sources as a lead EBNA1 inhibitor for future investigation and development as a therapeutic for EBV.
Inhibition of β-lactamase by Novel Benzothiazole-Coupled Azetidinone Derivatives: A Comprehensive Study Using an In silico and In vitro Approaches Against Multi Drug Resistant Bacteria
Authors: Rajesh Gupta Dheeraj , Koshy Abel John , Akshay Sadanand Dangari , Dwivedi Prarambh S. R. , Ashtekar Harsha , Rehman Niyas , Kumar Pankaj
Published: July 2024

Anti-microbial resistance has emerged as the leading cause of death worldwide with the rise of multidrug-resistant (MDR) bacteria, the need for novel anti-microbials to treat serious illnesses has become a great necessity. Hence, in this study, we aimed to design and synthesize benzothiazole-coupled azetidinone derivatives (GD1-GD12) as novel antibacterial agents. The synthesized compounds were elucidated by nuclear magnetic resonance, infrared and mass spectroscopy. The antibacterial activities of these compounds were tested against antibiotic-susceptible and MDR strains of bacteria. In addition, we performed ADME profiling, molecular docking and molecular dynamic simulations, principle component analysis, dynamic cross-correlation map and free-energy landscape to assess the binding of synthesized compounds with the [Formula: see text]-lactamase. Among the synthesized compounds GD6 and GD5 displayed minimum inhibitory concentrations of 12.5 [Formula: see text]g/mL and 50 [Formula: see text]g/mL against MDR strains of E. coli; more effective than standard. The molecular docking of designed molecules was performed against [Formula: see text]-lactamase enzyme and the stability of the complex was vali-dated. The pharmacokinetic profile displayed the compounds to possess druggable properties within the suitable ranges. The in silico approach displayed compound GD6 to be stable with [Formula: see text]-lactamase enzyme; indicating the mechanism for these compounds to be via inhibition of [Formula: see text]-lactamase. The novel anti-microbial compounds assessed against susceptible and MDR strains of bacteria possess antibacterial potential via the inhibition of [Formula: see text]-lactamase. The aforementioned data will be crucial to the development of novel broad-spectrum antibacterial compounds.
COMPARATIVE STUDY ON AAOIFI FAS 1 AND IAS 1 AND EXPLORING THE COMPLIANCE OF AAOIFI FAS 1 FOR ISLAMIC FINANCIAL INSTITUTIONS (IFIS) IN BRUNEI AND UNITED ARAB EMIRATES (UAE)
Authors: SAMRA NORIZATUL AQILAH BINTI , BESAR MOHD HAIRUL AZRIN BIN HAJI
Published: July 2024

This paper has two main objectives. First, it aims to compare the Accounting and Auditing Organization for Islamic Financial Institutions (AAOIFI) Financial Accounting Standard 1 (FAS 1) with International Accounting Standard 1 (IAS 1). Second, it seeks to assess the level of compliance of Islamic Financial Institutions (IFIs) in the United Arab Emirates (UAE) and Brunei with the AAOIFI standard on General Presentation and Disclosures (FAS 1). To achieve these goals, the author conducted a content assessment of annual reports from selected financial institutions, using checklists designed for the AAOIFI standard on General Presentation and Disclosures. The study found that UAE IFIs displayed greater compliance with AAOIFI FAS 1 compared to their counterparts in Brunei. In 2021, Emirates National Bank of Dubai (Emirates NBD) achieved a compliance rate of 74.3%, while BIBD recorded a lower compliance rate of 70.8%. In summary, this paper sheds light on the differences between AAOIFI FAS 1 and IAS 1, offering valuable insights into compliance levels among IFIs in the UAE and Brunei. However, it acknowledges certain limitations, such as limited generalizability, reliance on secondary data, subjective content analysis, absence of primary research, evolving regulatory frameworks, and resource constraints. Overall, this research contributes to the field of Islamic accounting by employing content analysis techniques to compare AAOIFI FAS 1 and IAS 1 and assess compliance in IFIs in Brunei and the UAE. It aligns with previous studies on International Financial Reporting Standards for IFIs and aims to advance the understanding of Islamic accounting and financial reporting practices.
ChangeRCA: Finding Root Causes from Software Changes in Large Online Systems
Authors: Yu Guangba , Chen Pengfei , He Zilong , Yan Qiuyu , Luo Yu , Li Fangyuan , Zheng Zibin
Published: July 2024

In large-scale online service systems, the occurrence of software changes is inevitable and frequent. Despite rigorous pre-deployment testing practices, the presence of defective software changes in the online environment cannot be completely eliminated. Consequently, there is a pressing need for automated techniques that can effectively identify these defective changes. However, the current abnormal change detection (ACD) approaches fall short in accurately pinpointing defective changes, primarily due to their disregard for the propagation of faults. To address the limitations of ACD, we propose a novel concept called root cause change analysis (RCCA) to identify the underlying root causes of change-inducing incidents. In order to apply the RCCA concept to practical scenarios, we have devised an intelligent RCCA framework named ChangeRCA. This framework aims to localize the defective change associated with change-inducing incidents among multiple changes. To assess the effectiveness of ChangeRCA, we have conducted an extensive evaluation utilizing a real-world dataset from WeChat and a simulated dataset encompassing 81 diverse defective changes. The evaluation results demonstrate that ChangeRCA outperforms the state-of-the-art ACD approaches, achieving an impressive Top-1 Hit Rate of 85% and significantly reducing the time required to identify defective changes.
Component Security Ten Years Later: An Empirical Study of Cross-Layer Threats in Real-World Mobile Applications
Authors: Lian Keke , Zhang Lei , Yang Guangliang , Mao Shuo , Wang Xinjie , Zhang Yuan , Yang Min
Published: July 2024

Nowadays, mobile apps have greatly facilitated our daily work and lives. They are often designed to work closely and interact with each other through app components for data and functionality sharing. The security of app components has been extensively studied and various component attacks have been proposed. Meanwhile, Android system vendors and app developers have introduced a series of defense measures to mitigate these security threats. However, we have discovered that as apps evolve and develop, existing app component defenses have become inadequate to address the emerging security requirements. This latency in adaptation has given rise to the feasibility of
cross-layer exploitation
, where attackers can indirectly manipulate app internal functionalities by polluting their dependent data. To assess the security risks of cross-layer exploitation in real-world apps, we design and implement a novel vulnerability analysis approach, called CLDroid, which addresses two non-trivial challenges. Our experiments revealed that 1,215 (8.8%) popular apps are potentially vulnerable to cross-layer exploitation, with a total of more than 18 billion installs. We verified that through cross-layer exploitation, an unprivileged app could achieve various severe security consequences, such as arbitrary code execution, click hijacking, content spoofing, and persistent DoS. We ethically reported verified vulnerabilities to the developers, who acknowledged and rewarded us with bug bounties. As a result, 56 CVE IDs have been assigned, with 22 of them rated as ‘critical’ or ‘high’ severity.
Efficiently Detecting Reentrancy Vulnerabilities in Complex Smart Contracts
Authors: Wang Zexu , Chen Jiachi , Wang Yanlin , Zhang Yu , Zhang Weizhe , Zheng Zibin
Published: July 2024

Reentrancy vulnerability as one of the most notorious vulnerabilities, has been a prominent topic in smart contract security research. Research shows that existing vulnerability detection presents a range of challenges, especially as smart contracts continue to increase in complexity. Existing tools perform poorly in terms of efficiency and successful detection rates for vulnerabilities in complex contracts.

To effectively detect reentrancy vulnerabilities in contracts with complex logic, we propose a tool named SliSE. SliSE’s detection process consists of two stages:
Warning Search
and
Symbolic Execution Verification
. In Stage 1, SliSE utilizes program slicing to analyze the
Inter-contract Program Dependency Graph
(I-PDG) of the contract, and collects suspicious vulnerability information as warnings. In Stage 2, symbolic execution is employed to verify the reachability of these warnings, thereby enhancing vulnerability detection accuracy. SliSE obtained the best performance compared with eight state-of-the-art detection tools. It achieved an F1 score of 78.65%, surpassing the highest score recorded by an existing tool of 9.26%. Additionally, it attained a recall rate exceeding 90% for detection of contracts on Ethereum. Overall, SliSE provides a robust and efficient method for detection of Reentrancy vulnerabilities for complex contracts.
Shadows in the Interface: A Comprehensive Study on Dark Patterns
Authors: Nie Liming , Zhao Yangyang , Li Chenglin , Luo Xuqiong , Liu Yang
Published: July 2024

As digital interfaces become increasingly prevalent, a series of ethical issues have surfaced, with dark patterns emerging as a key research focus. These manipulative design strategies are widely employed in User Interfaces (UI) with the primary aim of steering user behavior in favor of service providers, often at the expense of the users themselves. This paper aims to address three main challenges in the study of dark patterns: inconsistencies and incompleteness in classification, limitations of detection tools, and inadequacies in data comprehensiveness. In this paper, we introduce a comprehensive framework, called the Dark Pattern Analysis Framework (DPAF). Using this framework, we construct a comprehensive taxonomy of dark patterns, encompassing 64 types, each labeled with its impact on users and the likely scenarios in which it appears, validated through an industry survey. When assessing the capabilities of the detection tools and the completeness of the dataset, we find that of all dark patterns, the five detection tools can only identify 32, yielding a coverage rate of merely 50%. Although the four existing datasets collectively contain 5,566 instances, they cover only 32 of all types of dark patterns, also resulting in a total coverage rate of 50%. The results discussed above suggest that there is still significant room for advancement in the field of dark pattern detection. Through this research, we not only deepen our understanding of dark pattern classification and detection tools, but also offer valuable insights for future research and practice in this field.
Predicting Configuration Performance in Multiple Environments with Sequential Meta-Learning
Authors: Gong Jingzhi , Chen Tao
Published: July 2024

Learning and predicting the performance of given software configurations are of high importance to many software engineering activities. While configurable software systems will almost certainly face diverse running environments (e.g., version, hardware, and workload), current work often either builds performance models under a single environment or fails to properly handle data from diverse settings, hence restricting their accuracy for new environments.
In this paper, we target configuration performance learning under multiple environments. We do so by designing SeMPL — a meta-learning framework that learns the common understanding from configurations measured in distinct (meta) environments and generalizes them to the unforeseen, target environment. What makes it unique is that unlike common meta-learning frameworks (e.g., MAML and MetaSGD) that train the meta environments in parallel, we train them sequentially, one at a time. The order of training naturally allows discriminating the contributions among meta environments in the meta-model built, which fits better with the characteristic of configuration data that is known to dramatically differ between different environments.
Through comparing with 15 state-of-the-art models under nine systems, our extensive experimental results demonstrate that SeMPL performs considerably better on 89% of the systems with up to 99% accuracy improvement, while being data-efficient, leading to a maximum of 3.86× speedup. All code and data can be found at our repository: https://github.com/ideas-labo/SeMPL.
R2I: A Relative Readability Metric for Decompiled Code
Authors: Eom Haeun , Kim Dohee , Lim Sori , Koo Hyungjoon , Hwang Sungjae
Published: July 2024

Decompilation is a process of converting a low-level machine code snippet back into a high-level programming language such as C. It serves as a basis to aid reverse engineers in comprehending the contextual semantics of the code. In this respect, commercial decompilers like Hex-Rays have made significant strides in improving the readability of decompiled code over time. While previous work has proposed the metrics for assessing the readability of source code, including identifiers, variable names, function names, and comments, those metrics are unsuitable for measuring the readability of decompiled code primarily due to i) the lack of rich semantic information in the source and ii) the presence of erroneous syntax or inappropriate expressions. In response, to the best of our knowledge, this work first introduces R2I, the Relative Readability Index, a specialized metric tailored to evaluate decompiled code in a relative context quantitatively. In essence, R2I can be computed by i) taking code snippets across different decompilers as input and ii) extracting pre-defined features from an abstract syntax tree. For the robustness of R2I, we thoroughly investigate the enhancement efforts made by (non-)commercial decompilers and academic research to promote code readability, identifying 31 features to yield a reliable index collectively. Besides, we conducted a user survey to capture subjective factors such as one’s coding styles and preferences. Our empirical experiments demonstrate that R2I is a versatile metric capable of representing the relative quality of decompiled code (e.g., obfuscation, decompiler updates) and being well aligned with human perception in our survey.
Abstraction-Aware Inference of Metamorphic Relations
Authors: Nolasco Agustín , Molina Facundo , Degiovanni Renzo , Gorla Alessandra , Garbervetsky Diego , Papadakis Mike , Uchitel Sebastian , Aguirre Nazareno , Frias Marcelo F.
Published: July 2024

Metamorphic testing is a valuable technique that helps in dealing with the oracle problem. It involves testing software against specifications of its intended behavior given in terms of so called
metamorphic relations
, statements that express properties relating different software elements (e.g., different inputs, methods, etc). The effective application of metamorphic testing strongly depends on identifying suitable domain-specific metamorphic relations, a challenging task, that is typically manually performed. This paper introduces MemoRIA, a novel approach that aims at automatically identifying metamorphic relations. The technique focuses on a particular kind of metamorphic relation, which asserts equivalences between methods and method sequences. MemoRIA works by first generating an object-protocol abstraction of the software being tested, then using fuzzing to produce candidate relations from the abstraction, and finally validating the candidate relations through run-time analysis. A SAT-based analysis is used to eliminate redundant relations, resulting in a concise set of metamorphic relations for the software under test. We evaluate our technique on a benchmark consisting of 22 Java subjects taken from the literature, and compare MemoRIA with the metamorphic relation inference technique SBES. Our results show that by incorporating the object protocol abstraction information, MemoRIA is able to more effectively infer meaningful metamorphic relations, that are also more precise, compared to SBES, measured in terms of mutation analysis. Also, the SAT-based reduction allows us to significantly reduce the number of reported metamorphic relations, while in general having a small impact in the bug finding ability of the corresponding obtained relations.
TraStrainer: Adaptive Sampling for Distributed Traces with System Runtime State
Authors: Huang Haiyu , Zhang Xiaoyu , Chen Pengfei , He Zilong , Chen Zhiming , Yu Guangba , Chen Hongyang , Sun Chen
Published: July 2024

Distributed tracing has been widely adopted in many microservice systems and plays an important role in monitoring and analyzing the system. However, trace data often come in large volumes, incurring substantial computational and storage costs. To reduce the quantity of traces, trace sampling has become a prominent topic of discussion, and several methods have been proposed in prior work. To attain higher-quality sampling outcomes, biased sampling has gained more attention compared to random sampling. Previous biased sampling methods primarily considered the importance of traces based on diversity, aiming to sample more edge-case traces and fewer common-case traces. However, we contend that relying solely on trace diversity for sampling is insufficient, system runtime state is another crucial factor that needs to be considered, especially in cases of system failures. In this study, we introduce TraStrainer, an online sampler that takes into account both system runtime state and trace diversity. TraStrainer employs an interpretable and automated encoding method to represent traces as vectors. Simultaneously, it adaptively determines sampling preferences by analyzing system runtime metrics. When sampling, it combines the results of system-bias and diversity-bias through a dynamic voting mechanism. Experimental results demonstrate that TraStrainer can achieve higher quality sampling results and significantly improve the performance of downstream root cause analysis (RCA) tasks. It has led to an average increase of 32.63% in Top-1 RCA accuracy compared to four baselines in two datasets.
Fast Graph Simplification for Path-Sensitive Typestate Analysis through Tempo-Spatial Multi-Point Slicing
Authors: Cheng Xiao , Ren Jiawei , Sui Yulei
Published: July 2024

Typestate analysis is a commonly used static technique to identify software vulnerabilities by assessing if a sequence of operations violates temporal safety specifications defined by a finite state automaton. Path-sensitive typestate analysis (PSTA) offers a more precise solution by eliminating false alarms stemming from infeasible paths. To improve the efficiency of path-sensitive analysis, previous efforts have incorporated sparse techniques, with a focus on analyzing the path feasibility of def-use chains. However, they cannot be directly applied to detect typestate vulnerabilities requiring temporal information within the control flow graph, e.g., use-to-use information.

In this paper, we introduce FGS, a Fast Graph Simplification approach designed for PSTA by retaining multi-point temporal information while harnessing the advantages of sparse analysis. We propose a new multi-point slicing technique that captures the temporal and spatial correlations within the program. By doing so, it optimizes the program by only preserving the necessary program dependencies, resulting in a sparser structure for precision-preserving PSTA. Our graph simplification approach, as a fast preprocessing step, offers several benefits for existing PSTA algorithms. These include a more concise yet precision-preserving graph structure, decreased numbers of variables and constraints within execution states, and simplified path feasibility checking. As a result, the overall efficiency of the PSTA algorithm exhibits significant improvement.

We evaluated FGS using NIST benchmarks and ten real-world large-scale projects to detect four types of vulnerabilities, including memory leaks, double-frees, use-after-frees, and null dereferences. On average, when comparing FGS against ESP (baseline PSTA), FGS reduces 89% of nodes, 86% of edges, and 88% of calling context of the input graphs, obtaining a speedup of 116x and a memory usage reduction of 93% on the large projects evaluated. Our experimental results also demonstrate that FGS outperforms six open-source tools (IKOS, ClangSA , Saber, Cppcheck, Infer, and Sparrow) on the NIST benchmarks, which comprises 846 programs. Specifically, FGS achieves significantly higher precision, with improvements of up to 171% (42% on average), and detects a greater number of true positives, with enhancements of up to 245% (52% on average). Moreover, among the ten large-scale projects, FGS successfully found 105 real bugs with a precision rate of 82%. In contrast, our baseline tools not only missed over 42% of the real bugs but also yielded an average precision rate of just 13%.
Syntax Is All You Need: A Universal-Language Approach to Mutant Generation
Authors: Deb Sourav , Jain Kush , van Tonder Rijnard , Le Goues Claire , Groce Alex
Published: July 2024

While mutation testing has been a topic of academic interest for decades, it is only recently that “real-world” developers, including industry leaders such as Google and Meta, have adopted mutation testing. We propose a new approach to the development of mutation testing tools, and in particular the core challenge of
generating mutants
. Current practice tends towards two limited approaches to mutation generation: mutants are either (1) generated at the bytecode/IR level, and thus neither human readable nor adaptable to source-level features of languages or projects, or (2) generated at the source level by language-specific tools that are hard to write and maintain, and in fact are often abandoned by both developers and users. We propose instead that source-level mutation generation is a special case of
program transformation
in general, and that adopting this approach allows for a single tool that can effectively generate source-level mutants for essentially
any
programming language. Furthermore, by using
parser parser combinators
many of the seeming limitations of an any-language approach can be overcome, without the need to parse specific languages. We compare this new approach to mutation to existing tools, and demonstrate the advantages of using parser parser combinators to improve on a regular-expression based approach to generation. Finally, we show that our approach can provide effective mutant generation even for a language for which it lacks any language-specific operators, and that is not very similar in syntax to any language it has been applied to previously.
CodePlan: Repository-Level Coding using LLMs and Planning
Authors: Bairi Ramakrishna , Sonwane Atharv , Kanade Aditya , C. Vageesh D. , Iyer Arun , Parthasarathy Suresh , Rajamani Sriram , Ashok B. , Shet Shashank
Published: July 2024

Software engineering activities such as package migration, fixing error reports from static analysis or testing, and adding type annotations or other specifications to a codebase, involve pervasively editing the entire repository of code. We formulate these activities as
repository-level coding
tasks. Recent tools like GitHub Copilot, which are powered by Large Language Models (LLMs), have succeeded in offering high-quality solutions to localized coding problems. Repository-level coding tasks are more involved and cannot be solved directly using LLMs, since code within a repository is inter-dependent and the entire repository may be too large to fit into the prompt. We frame repository-level coding as a planning problem and present a task-agnostic, neuro-symbolic framework called CodePlan to solve it. CodePlan synthesizes a multi-step chain-of-edits (plan), where each step results in a call to an LLM on a code location with context derived from the entire repository, previous code changes and task-specific instructions. CodePlan is based on a novel combination of an incremental dependency analysis, a change may-impact analysis and an adaptive planning algorithm (symbolic components) with the neural LLMs. We evaluate the effectiveness of CodePlan on two repository-level tasks: package migration (C#) and temporal code edits (Python). Each task is evaluated on multiple code repositories, each of which requires inter-dependent changes to many files (between 2–97 files). Coding tasks of this level of complexity have not been automated using LLMs before. Our results show that CodePlan has better match with the ground truth compared to baselines. CodePlan is able to get 5/7 repositories to pass the validity checks (i.e., to build without errors and make correct code edits) whereas the baselines (without planning but with the same type of contextual information as CodePlan) cannot get any of the repositories to pass them. We provide our (non-proprietary) data, evaluation scripts and supplementary material at https://github.com/microsoft/codeplan.
Towards AI-Assisted Synthesis of Verified Dafny Methods
Authors: Misu Md Rakib Hossain , Lopes Cristina V. , Ma Iris , Noble James
Published: July 2024

Large language models show great promise in many domains, including programming. A promise is easy to make but hard to keep, and language models often fail to keep their promises, generating erroneous code. A promising avenue to keep models honest is to incorporate formal verification: generating programs’ specifications as well as code so that the code can be proved correct with respect to the specifications. Unfortunately, existing large language models show a severe lack of proficiency in verified programming.

In this paper, we demonstrate how to improve two pretrained models’ proficiency in the Dafny verification-aware language. Using 178 problems from the MBPP dataset, we prompt two contemporary models (GPT-4 and PaLM-2) to synthesize Dafny methods. We use three different types of prompts: a direct Contextless prompt; a Signature prompt that includes a method signature and test cases, and a Chain of Thought (CoT) prompt that decomposes the problem into steps and includes retrieval augmentation generated example problems and solutions. Our results show that GPT-4 performs better than PaLM-2 on these tasks and that both models perform best with the retrieval augmentation generated CoT prompt. GPT-4 was able to generate verified, human-evaluated, Dafny methods for 58% of the problems, however, GPT-4 managed only 19% of the problems with the Contextless prompt, and even fewer (10%) for the Signature prompt. We are thus able to contribute 153 verified Dafny solutions to MBPP problems, 50 that we wrote manually, and 103 synthesized by GPT-4.

Our results demonstrate that the benefits of formal program verification are now within reach of code generating large language models. Likewise, program verification systems can benefit from large language models, whether to synthesize code wholesale, to generate specifications, or to act as a "programmer’s verification apprentice", to construct annotations such as loop invariants which are hard for programmers to write or verification tools to find. Finally, we expect that the approach we have pioneered here — generating candidate solutions that are subsequently formally checked for correctness — should transfer to other domains (e.g., legal arguments, transport signaling, structural engineering) where solutions must be correct, where that correctness must be demonstrated, explained and understood by designers and end-users.
COSTELLO: Contrastive Testing for Embedding-Based Large Language Model as a Service Embeddings
Authors: Jiang Weipeng , Zhai Juan , Ma Shiqing , Zhang Xiaoyu , Shen Chao
Published: July 2024

Large language models have gained significant popularity and are often provided as a service (i.e., LLMaaS). Companies like OpenAI and Google provide online APIs of LLMs to allow downstream users to create innovative applications. Despite its popularity, LLM safety and quality assurance is a well-recognized concern in the real world, requiring extra efforts for testing these LLMs. Unfortunately, while end-to-end services like ChatGPT have garnered rising attention in terms of testing, the LLMaaS embeddings have comparatively received less scrutiny. We state the importance of testing and uncovering problematic individual embeddings without considering downstream applications. The abstraction and non-interpretability of embedded vectors, combined with the black-box inaccessibility of LLMaaS, make testing a challenging puzzle. This paper proposes COSTELLO, a black-box approach to reveal potential defects in abstract embedding vectors from LLMaaS by
contrastive testing
. Our intuition is that high-quality LLMs can adequately capture the semantic relationships of the input texts and properly represent their relationships in the high-dimensional space. For the given interface of LLMaaS and seed inputs, COSTELLO can automatically generate test suites and output words with potential problematic embeddings. The idea is to synthesize contrastive samples with guidance, including positive and negative samples, by mutating seed inputs. Our synthesis guide will leverage task-specific properties to control the mutation procedure and generate samples with known partial relationships in the high-dimensional space. Thus, we can compare the expected relationship (oracle) and embedding distance (output of LLMs) to locate potential buggy cases. We evaluate COSTELLO on 42 open-source (encoder-based) language models and two real-world commercial LLMaaS. Experimental results show that COSTELLO can effectively detect semantic violations, where more than 62% of violations on average result in erroneous behaviors (e.g., unfairness) of downstream applications.
How Does Simulation-Based Testing for Self-Driving Cars Match Human Perception?
Authors: Birchler Christian , Mohammed Tanzil Kombarabettu , Rani Pooja , Nechita Teodora , Kehrer Timo , Panichella Sebastiano
Published: July 2024

Software metrics such as coverage or mutation scores have been investigated for the automated quality assessment of test suites. While traditional tools rely on software metrics, the field of self-driving cars (SDCs) has primarily focused on simulation-based test case generation using quality metrics such as the out-of-bound (OOB) parameter to determine if a test case fails or passes. However, it remains unclear to what extent this quality metric aligns with the human perception of the safety and realism of SDCs. To address this (reality) gap, we conducted an empirical study involving 50 participants to investigate the factors that determine how humans perceive SDC test cases as safe, unsafe, realistic, or unrealistic. To this aim, we developed a framework leveraging virtual reality (VR) technologies, called SDC-Alabaster, to immerse the study participants into the virtual environment of SDC simulators. Our findings indicate that the human assessment of safety and realism of failing/passing test cases can vary based on different factors, such as the test's complexity and the possibility of interacting with the SDC. Especially for the assessment of realism, the participants' age leads to a different perception. This study highlights the need for more research on simulation testing quality metrics and the importance of human perception in evaluating SDCs.
Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLM
Authors: Ryan Gabriel , Jain Siddhartha , Shang Mingyue , Wang Shiqi , Ma Xiaofei , Ramanathan Murali Krishna , Ray Baishakhi
Published: July 2024

Testing plays a pivotal role in ensuring software quality, yet conventional Search Based Software Testing (SBST) methods often struggle with complex software units, achieving suboptimal test coverage. Recent work using large language models (LLMs) for test generation have focused on improving generation quality through optimizing the test generation context and correcting errors in model outputs, but use fixed prompting strategies that prompt the model to generate tests without additional guidance. As a result LLM-generated testsuites still suffer from low coverage.

In this paper, we present SymPrompt, a code-aware prompting strategy for LLMs in test generation. SymPrompt’s approach is based on recent work that demonstrates LLMs can solve more complex logical problems when prompted to reason about the problem in a multi-step fashion. We apply this methodology to test generation by deconstructing the testsuite generation process into a multi-stage sequence, each of which is driven by a specific prompt aligned with the execution paths of the method under test, and exposing relevant type and dependency focal context to the model. Our approach enables pretrained LLMs to generate more complete test cases without any additional training. We implement SymPrompt using the TreeSitter parsing framework and evaluate on a benchmark challenging methods from open source Python projects. SymPrompt enhances correct test generations by a factor of 5 and bolsters relative coverage by 26% for CodeGen2. Notably, when applied to GPT-4, SymPrompt improves coverage by over 2x compared to baseline prompting strategies.
Enhancing Code Understanding for Impact Analysis by Combining Transformers and Program Dependence Graphs
Authors: Yan Yanfu , Cooper Nathan , Moran Kevin , Bavota Gabriele , Poshyvanyk Denys , Rich Steve
Published: July 2024

Impact analysis (IA) is a critical software maintenance task that identifies the effects of a given set of code changes on a larger software project with the intention of avoiding potential adverse effects. IA is a cognitively challenging task that involves reasoning about the abstract relationships between various code constructs. Given its difficulty, researchers have worked to automate IA with approaches that primarily use coupling metrics as a measure of the "connectedness" of different parts of a software project. Many of these coupling metrics rely on static, dynamic, or evolutionary information and are based on heuristics that tend to be brittle, require expensive execution analysis, or large histories of co-changes to accurately estimate impact sets.

In this paper, we introduce a novel IA approach, called ATHENA, that combines a software system's dependence graph information with a conceptual coupling approach that uses advances in deep representation learning for code without the need for change histories and execution information. Previous IA benchmarks are small, containing less than ten software projects, and suffer from tangled commits, making it difficult to measure accurate results. Therefore, we constructed a large-scale IA benchmark, from 25 open-source software projects, that utilizes fine-grained commit information from bug fixes. On this new benchmark, our best performing approach configuration achieves an mRR, mAP, and HIT@10 score of 60.32%, 35.19%, and 81.48%, respectively. Through various ablations and qualitative analyses, we show that ATHENA's novel combination of program dependence graphs and conceptual coupling information leads it to outperform a simpler baseline by 10.34%, 9.55%, and 11.68% with statistical significance.
Java JIT Testing with Template Extraction
Authors: Zang Zhiqiang , Yu Fu-Yao , Thimmaiah Aditya , Shi August , Gligoric Milos
Published: July 2024

We present LeJit, a template-based framework for testing Java just-in-time (JIT) compilers. Like recent template-based frameworks, LeJit executes a template---a program with holes to be filled---to generate concrete programs given as inputs to Java JIT compilers. LeJit automatically generates template programs from existing Java code by converting expressions to holes, as well as generating necessary glue code (i.e., code that generates instances of non-primitive types) to make generated templates executable. We have successfully used LeJit to test a range of popular Java JIT compilers, revealing five bugs in HotSpot, nine bugs in OpenJ9, and one bug in GraalVM. All of these bugs have been confirmed by Oracle and IBM developers, and 11 of these bugs were previously unknown, including two CVEs (Common Vulnerabilities and Exposures). Our comparison with several existing approaches shows that LeJit is complementary to them and is a powerful technique for ensuring Java JIT compiler correctness.
BRF: Fuzzing the eBPF Runtime
Authors: Hung Hsin-Wei , Amiri Sani Ardalan
Published: July 2024

The eBPF technology in the Linux kernel has been widely adopted for different applications, such as networking, tracing, and security, thanks to the programmability it provides. By allowing user-supplied eBPF programs to be executed directly in the kernel, it greatly increases the flexibility and efficiency of deploying customized logic. However, eBPF also introduces a new and wide attack surface: malicious eBPF programs may try to exploit the vulnerabilities in the eBPF subsystem in the kernel.

Fuzzing is a promising technique to find such vulnerabilities. Unfortunately, our experiments with the stateof-the-art kernel fuzzer, Syzkaller, show that it cannot effectively fuzz the eBPF runtime, those components that are in charge of executing an eBPF program, for two reasons. First, the eBPF verifier (which is tasked with verifying the safety of eBPF programs) rejects many fuzzing inputs because (1) they do not comply with its required semantics or (2) they miss some dependencies, i.e., other syscalls that need to be issued before the program is loaded. Second, Syzkaller fails to attach and trigger the execution of eBPF programs most of the times.

This paper introduces the BPF Runtime Fuzzer (BRF), a fuzzer that can satisfy the semantics and dependencies required by the verifier and the eBPF subsystem. Our experiments show, in 48-hour fuzzing sessions, BRF can successfully execute 8× more eBPF programs compared to Syzkaller (and 32× more programs compared to Buzzer, an eBPF fuzzer released recently from Google). Moreover, eBPF programs generated by BRF are much more expressive than Syzkaller’s. As a result, BRF achieves 101% higher code coverage. Finally, BRF has so far managed to find 6 vulnerabilities (2 of them have been assigned CVE numbers) in the eBPF runtime, proving its effectiveness.
Metamorphic Testing of Secure Multi-party Computation (MPC) Compilers
Authors: Li Yichen , Xiao Dongwei , Liu Zhibo , Pang Qi , Wang Shuai
Published: July 2024

The demanding need to perform privacy-preserving computations among multiple

data owners has led to the prosperous development of secure multi-party

computation (MPC) protocols. MPC offers protocols for parties to jointly compute

a function over their inputs while keeping those inputs private. To date, MPC

has been widely adopted in various real-world, privacy-sensitive sectors, such

as healthcare and finance. Moreover, to ease the adoption of MPC, industrial and academic

MPC compilers have been developed to automatically translate

programs describing arbitrary MPC procedures into low-level MPC executables.

Compiling high-level descriptions into high-efficiency MPC executables is

challenging: the compilation often involves converting high-level languages into

several intermediate representations (IR), e.g., arithmetic or boolean circuits,

optimizing the computation/communication cost, and picking proper MPC protocols (and

underlying virtual machines) for a particular task and threat model. Various

optimizations and heuristics are employed during the compilation procedure to

improve the efficiency of the generated MPC executables.

Despite the prosperous adoption of MPC compilers by industrial vendors and

academia, a principled and systematic understanding of the correctness of

MPC compilers does not yet exist. To fill this critical gap, this paper

introduces MT-MPC, a metamorphic testing (MT) framework specifically designed for

MPC compilers to effectively uncover erroneous compilations. Our approach

proposes three metamorphic relations (MRs) that are tailored for MPC programs to

mutate high-level MPC programs (compiler inputs). We then examine if MPC

compilers yield semantics-equivalent MPC executables regarding the original and

mutated MPC programs by comparing their execution results.

Real-world MPC compilers exhibit a high level of engineering quality.

Nevertheless, we detected 4,772 inputs that can result in erroneous

compilations in three popular MPC compilers available on the market. While the

discovered error-triggering inputs do not cause the MPC compilers to crash

directly, they can lead to the generation of incorrect MPC executables,

jeopardizing the underlying dependability of the computation.

With substantial manual effort and help from the MPC compiler developers, we

uncovered thirteen bugs in these MPC compilers by debugging them using the

error-triggering inputs. Our proposed testing frameworks and findings can be

used to guide developers in their efforts to improve MPC compilers.
An Analysis of the Costs and Benefits of Autocomplete in IDEs
Authors: Jiang Shaokang , Coblenz Michael
Published: July 2024

Many IDEs support an autocomplete feature, which may increase developer productivity by reducing typing requirements and by providing convenient access to relevant information. However, to date, there has been no evaluation of the actual benefit of autocomplete to programmers. We conducted a between-subjects experiment (N=32) using an eye tracker to evaluate the costs and benefits of IDE-based autocomplete features to programmers who use an unfamiliar API. Participants who used autocomplete spent significantly less time reading documentation and got significantly higher scores on our post-study API knowledge test, indicating that it helped them learn more about the API. However, autocomplete did not significantly reduce the number of keystrokes required to finish tasks. We conclude that the primary benefit of autocomplete is in providing information, not in reducing time spent typing.
Decomposing Software Verification using Distributed Summary Synthesis
Authors: Beyer Dirk , Kettl Matthias , Lemberger Thomas
Published: July 2024

There are many approaches for automated software verification, but they are either imprecise, do not scale

well to large systems, or do not sufficiently leverage parallelization. This hinders the integration of software

model checking into the development process (continuous integration). We propose an approach to decompose

one large verification task into multiple smaller, connected verification tasks, based on blocks in the program

control flow. For each block, summaries (block contracts) are computed — based on independent, distributed,

continuous refinement by communication between the blocks. The approach iteratively synthesizes preconditions

to assume at the block entry (computed from postconditions received from block predecessors, i.e., which

program states reach this block) and violation conditions to check at the block exit (computed from violation

conditions received from block successors, i.e., which program states lead to a specification violation). This

separation of concerns leads to an architecture in which all blocks can be analyzed in parallel, as independent

verification problems. Whenever new information (as a postcondition or violation condition) is available from

other blocks, the verification can decide to restart with this new information. We realize our approach on

the basis of configurable program analysis and implement it for the verification of C programs in the widely

used verifier CPAchecker. A large experimental evaluation shows the potential of our new approach: The

distribution of the workload to several processing units works well, and there is a significant reduction of the

response time when using multiple processing units. There are even cases in which the new approach beats

the highly-tuned, existing single-threaded predicate abstraction.