Root Cause Localization for Unreproducible Builds via Causality Analysis Over System Call Tracing Localization of the root causes for unreproducible builds during software maintenance is an important yet challenging task, primarily due to limited runtime traces from build processes and high diversity of build environments. To address these challenges, in this paper, we propose RepTrace, a framework that leverages the uniform interfaces of system call tracing for monitoring executed build commands in diverse build environments and identifies the root causes for unreproducible builds by analyzing the system call traces of the executed build commands. Specifically, from the collected system call traces, RepTrace performs causality analysis to build a dependency graph starting from an inconsistent build artifact (across two builds) via two types of dependencies: readwrite dependencies among processes and parentchild process dependencies, and searches the graph to find the processes that result in the inconsistencies. To address the challenges of massive noisy dependencies and uncertain parentchild dependencies, RepTrace includes two novel techniques: (1) using differential analysis on multiple builds to reduce the search space of readwrite dependencies, and (2) computing similarity of the runtime values to filter out noisy parentchild process dependencies. The evaluation results of RepTrace over a set of realworld software packages show that RepTrace effectively finds not only the root cause commands responsible for the unreproducible builds, but also the files to patch for addressing the unreproducible issues. Among its Top10 identified commands and files, RepTrace achieves high accuracy rate of 90.00% and 90.56% in identifying the root causes, respectively.

SCMiner: Localizing SystemLevel Concurrency Faults from Large System Call Traces Localizing concurrency faults that occur in production is hard because, (1) detailed field data, such as user input, file content and interleaving schedule, may not be available to developers to reproduce the failure; (2) it is often impractical to assume the availability of multiple failing executions to localize the faults using existing techniques; (3) it is challenging to search for buggy locations in an application given limited runtime data; and, (4) concurrency failures at the system level often involve multiple processes or event handlers (e.g., software signals), which can not be handled by existing tools for diagnosing intraprocess(threadlevel) failures. To address these problems, we present SCMiner, a practical online bug diagnosis tool to help developers understand how a systemlevel concurrency fault happens based on the logs collected by the default system audit tools. SCMiner achieves online bug diagnosis to obviate the need for offline bug reproduction. SCMiner does not require code instrumentation on the production system or rely on the assumption of the availability of multiple failing executions. Specifically, after the system call traces are collected, SCMiner uses data mining and statistical anomaly detection techniques to identify the failureinducing system call sequences. It then maps each abnormal sequence to specific application functions. We have conducted an empirical study on 19 realworld benchmarks. The results show that SCMiner is both effective and efficient at localizing systemlevel concurrency faults.