Reviewerly/exHarmony huggingface dataset:
The <i>Why</i>, <i>When</i>, <i>What,</i> and <i>How</i> About Predictive Continuous Integration: A Simulation-Based Investigation

Continuous Integration (CI) enables developers to detect defects early and thus reduce lead time. However, the high frequency and long duration of executing CI have a detrimental effect on this practice. Existing studies have focused on using CI outcome predictors to reduce frequency. Since there is no reported project using predictive CI, it is difficult to evaluate its economic impact. This research aims to investigate predictive CI from a process perspective, including why and when to adopt predictors, what predictors to be used, and how to practice predictive CI in real projects. We innovatively employ Software Process Simulation to simulate a predictive CI process with a Discrete-Event Simulation (DES) model and conduct simulation-based experiments. We develop the Rollback-based Identification of Defective Commits (RIDEC) method to account for the negative effects of false predictions in simulations. Experimental results show that: 1) using predictive CI generally improves the effectiveness of CI, reducing time costs by up to 36.8% and the average waiting time before executing CI by 90.5%; 2) the time-saving varies across projects, with higher commit frequency projects benefiting more; and 3) predictor performance does not strongly correlate with time savings, but the precision of both failed and passed predictions should be paid more attention. Simulation-based evaluation helps identify overlooked aspects in existing research. Predictive CI saves time and resources, but improved prediction performance has limited cost-saving benefits. The primary value of predictive CI lies in providing accurate and quick feedback to developers, aligning with the goal of CI.
An Empirical Study of Refactoring Rhythms and Tactics in the Software Development Process

It is critical for developers to develop high-quality software to reduce maintenance cost. While often, developers apply refactoring practices to make source code readable and maintainable without impacting the software functionality. Existing studies identify development rhythms (i.e., weekly development patterns) and their relationship with various metrics, such as productivity. However, existing studies focus entirely on development rhythms. There is no study on refactoring rhythms and their relationship with code quality. Moreover, the existing studies categorize the refactoring tactics (i.e., long-term refactoring patterns) into two general concepts of consistent and inconsistent refactoring. Nevertheless, the existence of other tactics and their relationship with code quality is not explored. In this paper, we conduct an empirical study on the refactoring practices of 196 Apache projects in the early, middle, and late stages of development. We aim to identify (1) existing refactoring rhythms, (2) further refactoring tactics, and (3) the relationship between the identified tactics and rhythms with code quality. The recognition of existing refactoring strategies and their relationship with code quality can assist practitioners in recognizing and applying the appropriate and high-quality refactoring rhythms or tactics to deliver a higher quality of software. We find two frequently used refactoring rhythms: work-day refactoring and all-day refactoring. We also identify two deviations of floss and root canal refactoring tactics as: intermittent root canal, intermittent spiked floss, frequent spiked floss, and frequent root canal. We find that root canal-based tactics are correlated with less increase in the code smells (i.e., higher quality code) compared to floss-based tactics. Moreover, we find that refactoring rhythms are not significantly correlated with the quality of the code. Furthermore, we provide detailed information on the relationship of each refactoring tactic to each code smell type.
Studying the Influence and Distribution of the Human Effort in a Hybrid Fitness Function for Search-Based Model-Driven Engineering

Search-Based Software Engineering (SBSE) offers solutions that efficiently explore large complex problem spaces. To obtain more favorable solutions, human participation in the search process is needed. However, humans cannot handle the same number of solutions as an algorithm. We propose the first hybrid fitness function that combines human effort with human simulations. Human effort refers to human participation for providing evaluations of candidate solutions during the search process, whereas human simulations refer to recreations of a scenario in a specific situation for automatically obtaining the evaluation of candidate solutions. We also propose three variants for the hybrid fitness function that vary in the distribution of human effort in order to study whether the variants influence the performance in terms of solution quality. Specifically, we leverage our hybrid fitness function to locate bugs in software models for the video games of game software engineering. Video games are a fertile domain for these hybrid functions because simulated players are naturally developed as part of the video games (e.g., bots in First-Person Shooters). Our evaluation is at the scale of industrial settings with a commercial video game (Play Station 4 and Steam) and 29 professional video game developers. Hybridizing the fitness function outperforms the results of the best baseline by 33.46% in F-measure. A focus group confirms the acceptance of the hybrid fitness function. Hybridizing the fitness function significantly improves the bug localization process by reducing the amount of tedious manual work and by minimizing the number of bugs that go unnoticed. Furthermore, the variant that obtains the best results is a counter-intuitive result that was under the radar of the interactive SBSE community. These results can help not only video game developers to locate bugs, but they can also inspire SBSE researchers to bring hybrid fitness functions to other software engineering tasks.
Toward a Unified Transformer-Based Framework for Scene Graph Generation and Human-Object Interaction Detection

Scene graph generation (SGG) and human-object interaction (HOI) detection are two important visual tasks aiming at localising and recognising relationships between objects, and interactions between humans and objects, respectively. Prevailing works treat these tasks as distinct tasks, leading to the development of task-specific models tailored to individual datasets. However, we posit that the presence of visual relationships can furnish crucial contextual and intricate relational cues that significantly augment the inference of human-object interactions. This motivates us to think if there is a natural intrinsic relationship between the two tasks, where scene graphs can serve as a source for inferring human-object interactions. In light of this, we introduce SG2HOI+, a unified one-step model based on the Transformer architecture. Our approach employs two interactive hierarchical Transformers to seamlessly unify the tasks of SGG and HOI detection. Concretely, we initiate a relation Transformer tasked with generating relation triples from a suite of visual features. Subsequently, we employ another transformer-based decoder to predict human-object interactions based on the generated relation triples. A comprehensive series of experiments conducted across established benchmark datasets including Visual Genome, V-COCO, and HICO-DET demonstrates the compelling performance of our SG2HOI+ model in comparison to prevalent one-stage SGG models. Remarkably, our approach achieves competitive performance when compared to state-of-the-art HOI methods. Additionally, we observe that our SG2HOI+ jointly trained on both SGG and HOI tasks in an end-to-end manner yields substantial improvements for both tasks compared to individualized training paradigms.
Adoption of Transparency and Openness Promotion (TOP) guidelines across journals

The Transparency and Openness Promotion (TOP) guidelines include eight modular standards, with three tiers of increasing stringency, which journals can adopt to promote open science (Nosek et al. 2015). Standards evaluate journal policies on data citation, data transparency, material transparency, code transparency, design and analysis, study preregistration, analysis preregistration, and replication.To get an idea about whether TOP guidelines are being adopted by journals, we analysed the policies of 1,219 journals available on the website of the Center for Open Science. Journals were categorized by the three stringency tiers of eight TOP standards - were analyzed. 68% of the journals have their policies aligned to at least one of the TOP standards, nearly 46% have their policies aligned to data citation standards, and only 10% have a policy for pre-registration of analysis plans or study preregistration. If a journal policy adopts TOP guidelines, it is most likely of the stringency Level 1 (less stringent) across all eight standards (median=70%). However, policies for the “Analysis code transparency” standard are more “stringent” compared to other standards, and half of the journals require Level 2 and 3 stringency, e.g. storage in an open repository and/or code reproducibility. In the future, we are planning to complement these preliminary results with information about the journal policy requirements stratified across different scientific disciplines and to add a component of time – the evolution of requirements from publishers for adoption of open science practices.Our results already suggest two ways to improve implementation of open science practices:1) adoption of policies that promote open science for journals that still have not adopted them, and2) increasing the stringency of requirements for open science practices for journals that have.
cuAlign: Scalable Network Alignment on GPU Accelerators

Given two graphs, the objective of network alignment is to find a one-to-one mapping of vertices in one graph (A) to vertices in the other (B), such that the number of overlaps is maximized. We say that edges (i, j) ∈ A and (i′, j′) ∈ B are overlapped if i is mapped to i′ and j is mapped to j′. Network alignment is an important optimization problem with several applications in bioinformatics, computer vision and ontology matching. Since it is an NP-hard problem, efficient heuristics and scalable implementations are necessary. However, a combination of combinatorial and algebraic kernels within the network alignment algorithm poses significant hurdles for parallelization. Further, load imbalance and irregular DRAM traffic limit achievable performance on GPUs. In this work, we introduce a novel framework (cuAlign) that combines intra-network proximity using node (vertex) embedding, sparsification for computational efficiency, and belief propagation (BP) and approximate weighted matching for alignment. We demonstrate qualitative improvements up to over state-of-the-art approaches. We provide a scalable implementation targeting modern GPU accelerators. Our novel approach identifies and exploits unique structural properties of the BP-based algorithm and employs code fusion to reduce data movement between different steps of the algorithm. Using a diverse set of inputs, we demonstrate up to 19 × speedup for belief propagation, 3 × speedup for approximate weighted matching, and 15 × total, relative to a state-of-the-art multi-threaded implementation.
Reducing Memory Requirements for the IPU using Butterfly Factorizations

High Performance Computing (HPC) benefits from different improvements during last decades, specially in terms of hardware platforms to provide more processing power while maintaining the power consumption at a reasonable level. The Intelligence Processing Unit (IPU) is a new type of massively parallel processor, designed to speedup parallel computations with huge number of processing cores and on-chip memory components connected with high-speed fabrics. IPUs mainly target machine learning applications, however, due to the architectural differences between GPUs and IPUs, especially significantly less memory capacity on an IPU, methods for reducing model size by sparsification have to be considered. Butterfly factorizations are well-known replacements for fully-connected and convolutional layers. In this paper, we examine how butterfly structures can be implemented on an IPU and study their behavior and performance compared to a GPU. Experimental results indicate that these methods can provide 98.5% compression ratio to decrease the immense need for memory, the IPU implementation can benefit from 1.3x and 1.6x performance improvement for butterfly and pixelated butterfly, respectively. We also reach to 1.62x training time speedup on a real-word dataset such as CIFAR10.
Prototype of a Batched Quantum Circuit Simulator for the Vector Engine

State-of-the-art quantum circuit simulators have mostly focused on scaling the number of qubits. However, we argue that studying current noisy quantum computers and variational quantum algorithms benefits from high-throughput simulation of intermediate-scale quantum circuits. We present the first implementation and evaluation of a batched quantum simulator on the NEC Vector Engine (VE), a vector processor with massive memory bandwidth that is ideal for memory-intensive state vector simulation. To take advantage of the long-vector architecture of VE, we design a parallelization strategy and memory layout suited for batched state vector simulation. Our evaluation shows that the latest-generation VE Type 30A achieves equal performance as the NVIDIA A100 80 GB model for single-qubit gates, but slightly underperforms A100 for two-qubit gates. Furthermore, our vector-friendly noise gate implementation offers a large speedup compared to that of cuStateVec. When simulating a noisy 16-qubit random circuit with our simulator, VE30 is up to 1.95 × faster than A100.
RISA: Round-Robin Intra-Rack Friendly Scheduling Algorithm for Disaggregated Datacenters

Recent trends see a move away from a fixed-resource server-centric datacenter model to a more adaptable “disaggregated” datacenter model. These disaggregated datacenters can then dynamically group resources to the specific requirements of an incoming workload, thereby improving efficiency. To properly utilize these disaggregated datacenters, workload allocation techniques must examine the current state of the datacenter and choose resources that not only optimize the current workload request, but future ones. Since disaggregated datacenters are severely bottlenecked by the available network resources, our work proposes a heuristic-based approach called RISA, which significantly reduces the network usage of workload allocations in disaggregated datacenters. Compared to the state-of-the-art, RISA reduces the power consumption for optical components by and reduces the average CPU-RAM round-trip latency by . Additionally, RISA significantly outperforms the state-of-the-art in terms of execution time.
Analyzing the Performance Impact of HPC Workloads with Gramine+SGX on 3rd Generation Xeon Scalable Processors

Intel SGX, which provides a strongly isolated execution environment for running applications to protect their code and data from privileged adversaries, is hopeful for increased integrity and confidentiality in multi-user systems. However, SGX is rarely used in the field of HPC despite the increasing importance of data protection. This is because SGX had limited memory resources and required supercomputing users to modify their code for execution. Lately, both 3rd generation Xeon scalable processors, which have many advanced data protection features including increased SGX memory, and Gramine 1.0, which is the first production-ready version of a library OS for the execution of unmodified applications on SGX, were released. However, the impact of these state-of-the-art technologies on HPC workloads is not well investigated.
Dynamic Selective Protection of Sparse Iterative Solvers via ML Prediction of Soft Error Impacts

Soft errors occur frequently on large computing platforms due to the increasing scale and complexity of HPC systems. Various resilience techniques (e.g., checkpointing, ABFT, and replication) have been proposed to protect scientific applications from soft errors at different levels. Among them, system-level replication often involves duplicating or even triplicating the entire computation, thus resulting in high resilience overhead. This paper proposes dynamic selective protection for sparse iterative solvers, in particular for the Preconditioned Conjugate Gradient (PCG) solver, at the system level to reduce the resilience overhead. For this method, we leverage machine learning (ML) to predict the impact of soft errors that strike different elements of a key computation (i.e., sparse matrix-vector multiplication) at different iterations of the solver. Based on the result of the prediction, we design a dynamic strategy to selectively protect those elements that would result in a large performance degradation if struck by soft errors. An experimental evaluation demonstrates that our dynamic protection strategy is able to reduce the resilience overhead compared to existing algorithms.
A Performance Model for Estimating the Cost of Scaling to Practical Quantum Advantage

We present a simple performance model to estimate the qubit-count and runtime associated with large-scale error-corrected quantum computations. Our estimates extrapolate current usage costs of quantum computers and show that computing the ground state of the 2D Hubbard model, which is widely believed to be an early candidate for practical quantum advantage, could start at a million dollars. Our model shows a clear cost advantage of up to four orders of magnitude for quantum processors based on superconducting technology compared to ion trap devices. Our analysis shows that usage costs, while substantial, will not necessarily block the road to practical quantum advantage. Furthermore, the combined effects of algorithmic improvements, more efficient error correction codes, and R&D cost amortization are likely to lead to orders of magnitude reductions in cost.
Accelerating Deep Neural Network guided MCTS using Adaptive Parallelism

Deep Neural Network guided Monte-Carlo Tree Search (DNN-MCTS) is a powerful class of AI algorithms. The DNN operations are highly parallelizable, but tree search operations are sequential and often become the system bottleneck. Existing MCTS parallel schemes on CPU platforms either exploit data parallelism but sacrifice memory access latency, or take advantage of local cache for low-latency accesses but constrain the search to a single thread. This work analyzes the tradeoff of these parallel schemes, and proposes an adaptive parallel scheme that optimally chooses the MCTS component’s parallel scheme on the CPU. Additionally, an efficient method for searching the optimal communication batch size when the CPU interfaces with DNN operations on an accelerator(GPU) is proposed. Using a DNN-MCTS algorithm on board game benchmarks, we show that our work is able to adaptively generate the best-performing parallel implementation, leading to a range of 1.5 × −3 × speedup compared with the baseline methods.
QArchSearch: A Scalable Quantum Architecture Search Package

The current era of quantum computing has yielded several algorithms that promise high computational efficiency. While the algorithms are sound in theory and can provide potentially exponential speedup, there is little guidance on how to design proper quantum circuits to realize the appropriate unitary transformation to be applied to the input quantum state. In this paper, we present QArchSearch, an AI based quantum architecture search package with the QTensor library as a backend that provides a principled and automated approach to finding the best model given a task and input quantum state. We show that the search package is able to efficiently scale the search to large quantum circuits and enables the exploration of more complex models for different quantum applications. QArchSearch runs at scale and high efficiency on high-performance computing systems using a two-level parallelization scheme on both CPUs and GPUs, which has been demonstrated on the Polaris supercomputer.
Optimized Uncertainty Estimation for Vision Transformers: Enhancing Adversarial Robustness and Performance Using Selective Classification

Deep Learning models often exhibit undue confidence when encountering out-of-distribution (OOD) inputs, misclassifying with high confidence. The ideal outcome, in these cases, would be an "I do not know" verdict. We enhance the trustworthiness of our models through selective classification, allowing the model to abstain from making predictions when facing uncertainty. Rather than a singular prediction, the model offers a prediction distribution, enabling users to gauge the model’s trustworthiness and determine the need for human intervention. We assess uncertainty in two baseline models: a Convolutional Neural Network (CNN) and a Vision Transformer (ViT). By leveraging these uncertainty values, we minimize errors by refraining from predictions during high uncertainty. Additionally, we evaluate these models across various distributed architectures, including new AI architectures, Cerebras CS-2, and SambaNova SN30.
LibPressio-Predict: Flexible and Fast Infrastructure For Inferring Compression Performance

Over recent years, substantial efforts have gone into developing systems to infer compression performance without running compressors. These efforts have driven down the error in the estimates, reduced their runtimes, and improved their generality. However, these efforts are uncoordinated increasing the efforts required to perform comparisons between them. There may be subtle differences in sampling approaches, and nuances to the interfaces requiring efforts to port applications between them and to reproduce experiments. Additionally, many of these methods call for substantial amounts of training data to produce reliable estimates, as well as scalable codes to perform the training. In this work, we present LibPressio-Predict – a scalable library for use in applications using predictions of compression performance and a scalable tool LibPressio-Bench to run these experiments quickly at scale. We use this tool to evaluate 3 recent compression prediction approaches systematically with all 48 timesteps and 13 fields from the Hurricane Issable dataset.
Using Azure Quantum Resource Estimator for Assessing Performance of Fault Tolerant Quantum Computation

The automatic resource estimation tools provided by Azure Quantum and Microsoft Quantum Development Kit are described, which enable the evaluation of the resources required to run algorithms on fault-tolerant quantum computers. An example is given of obtaining resource estimates for quantum fault-tolerant implementations of several multiplication algorithms.
Filtering Wasteful Vertex Visits in Breadth-First Search

Breadth-First Search (BFS) is a common building block for several graph processing algorithms today. In this work, we highlight that a large fraction of vertex visits across the network in distributed BFS results in wasteful work. We investigate methods to identify and filter such wasteful cross-network vertex visits to save network bandwidth for energy and performance improvements. We analyze the metadata requirements to perform such filtering in modern hierarchical distributed architectures and identify the tradeoffs between storage and filtering rate. We perform our experiments using the graph500 benchmark and provide a model to scale results to larger graphs. Finally, we propose heuristics to reduce the storage for a BFS message filter and explore the design space for implementing such filtering logic in software, hardware, or a combination.