Reviewerly/exHarmony huggingface dataset:
Parser Weakness Enumeration: Definition and Preliminary Assessment

Parsing structured data into meaningful content is fundamental in machine to human communication. However, this area persistently contains critical exploitable vulnerabilities directly controllable by an attacker. Developing secure parsers without security bugs are a significant concern. Language Theoretic Security (LangSec) has defined language formalisms in safeguarding untrusted input into parsers. Taking the safety lessons learned in designing specification compliant language grammar, we present a preliminary Parser Weakness Enumeration (PWE). We identify seven anti-patterns, which occur when modeling data or file formats into structured code during parser development. PWE describes the weakness in MITRE’s Common Weakness Enumeration (CWE) terminology to encourage best practices when designing and developing parsers without the benefit of a strict language grammar. PWE captures parsing semantics not identified by current CWEs security semantics. PWE definitions are actionable, concise, and include potential mitigation guidelines to help securely model format specification with unsafe code. Our preliminary results include verifying one PWE on one vulnerability on a single program as a proof-of-concept through static analysis, showing the potential of the approach for automated analysis. Our list of weaknesses are a starting point for further experimentation. Writing secure parsers has unique insecurities captured by our PWE and their inclusion into the discussion of software weaknesses would benefit developers.
Fork Entropy: Assessing the Diversity of Open Source Software Projects' Forks

On open source software (OSS) platforms such as GitHub, forking and accepting pull-requests is an important approach for OSS projects to receive contributions, especially from external contributors who cannot directly commit into the source repositories. Having a large number of forks is often considered as an indicator of a project being popular. While extensive studies have been conducted to understand the reasons of forking, communications between forks, features and impacts of forks, there are few quantitative measures that can provide a simple yet informative way to gain insights about an OSS project's forks besides their count. Inspired by studies on biodiversity and OSS team diversity, in this paper, we propose an approach to measure the diversity of an OSS project's forks (i.e., its fork population). We devise a novel fork entropy metric based on Rao's quadratic entropy to measure such diversity according to the forks' modifications to project files. With properties including symmetry, continuity, and monotonicity, the proposed fork entropy metric is effective in quantifying the diversity of a project's fork population. To further examine the usefulness of the proposed metric, we conduct empirical studies with data retrieved from fifty projects on GitHub. We observe significant correlations between a project's fork entropy and different outcome variables including the project's external productivity measured by the number of external contributors' commits, acceptance rate of external contributors' pull-requests, and the number of reported bugs. We also observe significant interactions between fork entropy and other factors such as the number of forks. The results suggest that fork entropy effectively enriches our understanding of OSS projects' forks beyond the simple number of forks, and can potentially support further research and applications.
ZC<sup>3</sup>: Zero-Shot Cross-Language Code Clone Detection

Developers introduce code clones to improve programming productivity. Many existing studies have achieved impressive performance in monolingual code clone detection. However, during software development, more and more developers write semantically equivalent programs with different languages to support different platforms and help developers translate projects from one language to another. Considering that collecting cross-language parallel data, especially for low-resource languages, is expensive and time-consuming, how designing an effective cross-language model that does not rely on any parallel data is a significant problem. In this paper, we propose a novel method named ZC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> for Z_ero-shot Cross-language Code Clone detection. ZC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> designs the contrastive snippet prediction to form an isomorphic representation space among different programming languages. Based on this, ZC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> exploits domain-aware learning and cycle consistency learning to further constrain the model to generate representations that are aligned among different languages meanwhile are diacritical for different types of clones. To evaluate our approach, we conduct extensive experiments on four representative cross-language clone detection datasets. Experimental results show that ZC <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> outperforms the state-of-the-art baselines by 67.12%, 51.39%, 14.85%, and 53.01% on the MAP score, respectively. We further investigate the representational distribution of different languages and discuss the effectiveness of our method.
Curbing the Vulnerable Parser: Graded Modal Guardrails for Secure Input Handling

Parsers written in an ad hoc manner are vulnerable to exploitation, but writing a secure parser in a general purpose language often requires blending custom data validation with existing parsing libraries, and for high assurance this will often entail a separate formal verification effort. The language-theoretic security research area has produced a collection of security properties of safe parsers and recommendations for safe design patterns that help enforce these properties. We illustrate that we can place these tools in the hands of developers, empowering them via type annotations to establish guardrails against insecure input handling. Following the practice of language-based security, we leverage Granule’s graded modal type system to provide strong static guarantees for secure parsing.
ArduinoProg: Towards Automating Arduino Programming

Writing code for Arduino poses unique challenges. A developer 1) needs hardware-specific knowledge about the interface configuration between the Arduino controller and the I/Ohardware, 2) identifies a suitable driver library for the I/O hardware, and 3) follows certain usage patterns of the driver library in order to use them properly. In this work, based on a study of real-world user queries posted in the Arduino forum, we propose ArduinoProg to address such challenges. ArduinoProg consists of three components, i.e., Library Retriever, Configuration Classifier, and Pattern Generator. Given a query, Library Retriever retrieves library names relevant to the I/O hardware identified from the query using vector-based similarity matching. Configuration Classifier predicts the interface configuration between the I/O hardware and the Arduino controller based on the method definitions of each library. Pattern Generator generates the usage pattern of a library using a sequence-to-sequence deep learning model. We have evaluated ArduinoProg using real-world queries, and our results show that the components of ArduinoProg can generate accurate and useful suggestions to guide developers in writing Arduino code. Demo video: bit.ly/3Y3aeBe Tool: https://huggingface.co/spaces/imamnurby/ArduinoProg Code and data: https://github.com/imamnurby/ArduProg
Virtual Patient in Medical Education

This study aims to demonstrate the importance of Virtual Patients (VPs) in influencing future medical education. This study investigates the potential of integrating virtual avatars into learning opportunities, engaging students, and ensuring patient safety. The study discusses practical considerations and approaches for incorporating VPs into medical education contexts, along with guidance on creating training materials, integrating technology, and addressing ethical dilemmas. In order to improve learning quality, virtual patient and immersive technologies are being used to promote cutting-edge teaching and learning strategies.
DeFiWarder: Protecting DeFi Apps from Token Leaking Vulnerabilities

Decentralized Finance (DeFi) apps have rapidly proliferated with the development of blockchain and smart contracts, whose maximum total value locked (TVL) has exceeded 100 billion dollars in the past few years. These apps allow users to interact and perform complicated financial activities. However, the vulnerabilities hiding in the smart contracts of DeFi apps have resulted in numerous security incidents, with most of them leading to funds (tokens) leaking and resulting in severe financial loss. In this paper, we summarize Token Leaking vulnerability of DeFi apps, which enable someone to abnormally withdraw funds that far exceed their deposits. Due to the massive amount of funds in DeFi apps, it is crucial to protect DeFi apps from Token Leaking vulnerabilities. Unfortunately, existing tools have limitations in addressing this vulnerability. To address this issue, we propose DeFiWarder, a tool that traces on-chain transactions and protects DeFi apps from Token Leaking vulnerabilities. Specifically, DeFiWarder first records the execution logs (traces) of smart contracts. It then accurately recovers token transfers within transactions to catch the funds flow between users and DeFi apps, as well as the relations between users based on role mining. Finally, DeFiWarder utilizes anomaly detection to reveal Token Leaking vulnerabilities and related attack behaviors. We conducted experiments to demonstrate the effectiveness and efficiency of DeFiWarder. Specifically, DeFi-Warder successfully revealed 25 Token Leaking vulnerabilities from 30 Defi apps. Moreover, its efficiency supports real-time detection of token leaking within on-chain transactions. In addition, we summarize five major reasons for Token Leaking vulnerability to assist DeFi apps in protecting their funds.
Understanding Resolution of Multi-Language Bugs: An Empirical Study on Apache Projects

Background: In modern software systems, more and more systems are written in multiple programming languages (PLs). There is no comprehensive investigation on the phenomenon of multi-programming-language (MPL) bugs, which resolution involves source files written in multiple PLs. Aim: This work investigated the characteristics of bug resolution in MPL software systems and explored the reasons why bug resolution involves multiple PLs. Method: We conducted an empirical study on 54 MPL projects selected from 655 Apache OSS projects, of which 66,932 bugs were analyzed. Results: (1) the percentage of MPL bugs (MPLBs) in the selected projects ranges from 0.17% to 42.26%, and the percentage of MPLBs for all projects as a whole is 10.01%; (2) 95.0% and 4.5% of all the MPLBs involve source files written in 2 and 3 PLs, respectively; (3) the change complexity resolution characteristics of MPLBs tend to be higher than those of single-programming-language bugs (SPLBs); (4) the open time for MPLBs is 19.52% to 529.57% significantly longer than SPLBs regarding 9 PL combinations; (5) the reopen rate of bugs involving the PL combination of JavaScript and Python reaches 20.66%; (6) we found 6 causes why the bug resolution involves multiple PLs and identified 5 cross-language calling mechanisms. Conclusion: MPLBs are related to increased development difficulty.
Leveraging Evidence Theory to Improve Fault Localization: An Exploratory Study

Background: Fault localization in software maintenance and debugging can be a costly process. Spectrum-Based Fault Localization (SBFL) is a widely-used method for fault localization. It assigns suspicion scores to code elements based on tests, indicating the likelihood of defects in specific code lines. However, the effectiveness of SBFL approaches varies depending on the subject code. Aims: In this paper, our aim is to present an approach that combines multiple SBFL formulae using evidence theory. Method: We first introduce a taxonomy of SBFL techniques. Then, we describe how we fuse suspiciousness scores obtained from a set of SBFL formulae. We also introduce a concept of fuzzy windows, and describe how they can enhance localization accuracy and how they can be tuned to further refine results. Results: We present an empirical evaluation of our approach using the Defects4J dataset. Our results demonstrate improvements in fault localization accuracy over existing statement-level SBFL techniques. Specifically, by fusing three SBFL methods, our approach reduces code inspection effort by up to 34.5 % with a size-4 window and increases the hit rate for the top 10% most suspicious lines by 27.9 % using a size-7 window. Moreover, in multi-line bug scenarios, our approach reduces code inspection effort by up to 35.6% and achieves a maximum increase of 43.2% in the hit rate of the top 10% most suspicious lines. Additionally, our approach outperforms state-of-the-art machine learning-based method-level fusion approaches in terms of top rank fault localization accuracy. Conclusions: Our study highlights the applicability of evidence theory in addressing fault localization as an uncertain and ambiguous information fusion problem involving multiple SBFL techniques. The combination of SBFL formulae using evidence theory, along with the use of fuzzy windows, shows promise in enhancing fault localization accuracy.
Automating Bias Testing of LLMs

Large Language Models (LLMs) are being quickly integrated in a myriad of software applications. This may introduce a number of biases, such as gender, age or ethnicity, in the behavior of such applications. To face this challenge, we explore the automatic generation of tests suites to assess the potential biases of an LLM. Each test is defined as a prompt used as input to the LLM and a test oracle that analyses the LLM output to detect the presence of biases.
Botnet Detection Using Label Propagation and Batch K-means Clustering for Securing IoT Networks

A profound devastation is experienced when privacy is violated, and personal information is compromised or lost. Now, consider the situation where all these penetrations occur simultaneously and remain oblivious until a considerable time has passed. This scenario described the Internet of Things (IoT) network when attacked by a Botnet. Existing machine-learning Botnet detection methods tend to increase the number of incorporated features along with increasing the complexity of the detection algorithms, which can impede timely Botnet detection and hinder optimal defense, especially in large-scale environments. This work introduces an intelligent Botnet detection system constructed based on batch k-means and label propagation algorithms. Despite its simplicity, the system exhibits noticeable efficacy in identifying Botnet traffic. Moreover, the system is highly able to learn the underlying behavioral patterns of Botnet flows despite being constructed with a narrow zone of flow-based features solely relying on packet-length distribution. The proposed system can serve the dual purpose of clustering and classifying Botnet traffic where it is designed to detect HTTP-based Botnet attacks belonging specifically to the Virut Botnet family. Using the ISCX-Botnet dataset, the system has achieved accuracy and F1-score of 98.25% and 93.23% respectively. These findings are on par with (or superior) to those of state-of-the-art complex techniques, demonstrating the proposed system's capability to be deployed in timely, large-scale botnet defense architectures.
Contextuality of Code Representation Learning

Advanced machine learning models (ML) have been successfully leveraged in several software engineering (SE) applications. The existing SE techniques have used the embedding models ranging from static to contextualized ones to build the vectors for program units. The contextualized vectors address a phenomenon in natural language texts called polysemy, which is the coexistence of different meanings of a word/phrase. However, due to different nature, program units exhibit the nature of mixed polysemy. Some code tokens and statements exhibit polysemy while other tokens (e.g., keywords, separators, and operators) and statements maintain the same meaning in different contexts. A natural question is whether static or contextualized embeddings fit better with the nature of mixed polysemy in source code. The answer to this question is helpful for the SE researchers in selecting the right embedding model. We conducted experiments on 12 popular sequence-/tree-/graph-based embedding models and on the samples of a dataset of 10,222 Java projects with +14M methods. We present several contextuality evaluation metrics adapted from natural-language texts to code structures to evaluate the embeddings from those models. Among several findings, we found that the models with higher contextuality help a bug detection model perform better than the static ones. Neither static nor contextualized embedding models fit well with the mixed polysemy nature of source code. Thus, we develop Hycode, a hybrid embedding model that fits better with the nature of mixed polysemy in source code.
Information Retrieval-Based Fault Localization for Concurrent Programs

Information retrieval-based fault localization (IRFL) techniques have been proposed as a solution to identify the files that are likely to contain faults that are root causes of failures reported by users. These techniques have been extensively studied to accurately rank source files, however, none of the existing approaches have focused on the specific case of concurrent programs. This is a critical issue since concurrency bugs are notoriously difficult to identify. To address this problem, this paper presents a novel approach called BLCoiR, which aims to reformulate bug report queries to more accurately localize source files related to concurrency bugs. The key idea of BLCoiR is based on a novel knowledge graph (KG), which represents the domain entities extracted from the concurrency bug reports and their semantic relations. The KG is then transformed into the IR query to perform fault localization. BLCoiR leverages natural language processing (NLP) and concept modeling techniques to construct the knowledge graph. Specifically, NLP techniques are used to extract relevant entities from the bug reports, such as the word entities related to concurrency constructs. These entities are then linked together based on their semantic relationships, forming the KG. We have conducted an empirical study on 692 concurrency bug reports from 44 real-world applications. The results show that BLCoiR outperforms existing IRFL techniques in terms of accuracy and efficiency in localizing concurrency bugs. BLCoiR demonstrates effectiveness of using a knowledge graph to model the domain entities and their relationships, providing a promising direction for future research in this area.
Robin: A Novel Method to Produce Robust Interpreters for Deep Learning-Based Code Classifiers

Deep learning has been widely used in source code classification tasks, such as code classification according to their functionalities, code authorship attribution, and vulnerability detection. Unfortunately, the black-box nature of deep learning makes it hard to interpret and understand why a classifier (i.e., classification model) makes a particular prediction on a given example. This lack of interpretability (or explainability) might have hindered their adoption by practitioners because it is not clear when they should or should not trust a classifier's prediction. The lack of interpretability has motivated a number of studies in recent years. However, existing methods are neither robust nor able to cope with out-of-distribution examples. In this paper, we propose a novel method to produce Robust interpreters for a given deep learning-based code classifier; the method is dubbed Robin. The key idea behind Robin is a novel hybrid structure combining an interpreter and two approximators, while leveraging the ideas of adversarial training and data augmentation. Experimental results show that on average the interpreter produced by Robin achieves a 6.11% higher fidelity (evaluated on the classifier), 67.22% higher fidelity (evaluated on the approximator), and 15.87x higher robustness than that of the three existing interpreters we evaluated. Moreover, the interpreter is 47.31% less affected by out-of-distribution examples than that of LEMNA.
An Integrated Program Analysis Framework for Graduate Courses in Programming Languages and Software Engineering

Program analysis, verification and testing are important topics in programming languages and software engineering. They aim to produce engineers who are not only capable of empirically evaluating but, also formally reasoning on the correctness of software systems. We propose a specialized framework, Chiron, designed to teach graduate-level courses on these topics. Chiron has a small code base for easy understanding, uses a unified intermediate representation across all its analysis modules, maintains a modular architecture for plugging in new algorithms and uses a “fun” programming language to provide a gamified experience. Currently, it packages a dataflow analysis engine for driving compiler optimizations, an abstract interpretation engine for verification, a symbolic execution engine, a fuzzer and an evolutionary test generator for program testing, and a spectrum based statistical bug localization module. Within Chiron, program analysis tasks are posed in an unconventional setting (as adventures of a turtle) to provide a gamified experience; the accompanying animations (showing the movements of the turtle) allow the student to understand the underlying concepts better, and the detailed logs allow the teaching assistants in their grading activities. Chiron has been used in two offerings of a graduate level course on program analysis, verification and testing. In response to our survey questionnaire, all the students unanimously held the opinion that Chiron was extremely helpful in aiding their learning, and recommended its use in similar courses.