<figure><img src="image_1.png" /></figure>

HAL
open science

# Divergence-Driven Debugging: Understanding Behavioral Changes Between Two Program Versions

Rémi Dufloer, Imen Sayar, Anne Etien, Steven Costiou

► To cite this version:

Rémi Dufloer, Imen Sayar, Anne Etien, Steven Costiou. Divergence-Driven Debugging: Understanding Behavioral Changes Between Two Program Versions. ICPC 2025 - 33rd IEEE/ACM International Conference on Program Comprehension, Apr 2025, Ottawa, Canada. hal-05114444

HAL Id: hal-05114444

https://inria.hal.science/hal-05114444v1

Submitted on 16 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Divergence-Driven Debugging: Understanding Behavioral Changes Between Two Program Versions

Rémi Dufloer*, Imen Sayar†, Anne Etien*, Steven Costiou*

*Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 CRISTAL, Lille, France

†Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRISTAL, Lille, France

***Abstract***—Daily software changes present an inherent risk of introducing bugs. To understand such bugs, it is crucial to understand why specific changes introduced the bugs. This is difficult and time-consuming. We propose Divergence-Driven Debugging, an approach using a debugger for detecting execution divergences between two versions of the same program. Developers can explore both versions' execution in full and compare them based on the detected divergences. We evaluate our approach on a bug scenario and show that our debugger highlights the live state and behavior required to understand why the changes produce the bug and to identify its root cause.

***Index Terms***—program change, bug, behavior, divergence, debugging.

## I. INTRODUCTION AND BACKGROUND

Developing and maintaining software require constant code modifications. Developers change their code for evolving their software [1], for bug fixing [2] and for improving their code [3]. These changes carry an inherent risk of introducing bugs or unexpected behaviors into their software [4]. In this case, developers need to understand why their changes lead in behavioral differences between the *reference* version (before changes) and the *modified* version (after changes) [5].

One approach consists in analyzing both versions of the source code to try understanding the impact of the changes, such as code reviews [6] or code impact analysis [1]. This is difficult [5], especially if done manually due to the size and complexity of modern systems [7]. Recent work provide additional support for code review, for example through visualizations [8]. These techniques however only provide static representations of the changes' impact, and are not appropriate to understand the impact on the behavior of the *reference* and *modified* programs. Researchers therefore completed code changes information with dynamic data from real executions [9] and found out that dynamic data helped developers better understand the impact of changes.

Another approach consists in debugging the modified code [1] to try understanding the execution differences between the two versions. Debugging consists in identifying and correcting errors within software code [2]. It is a difficult and costly activity [10, 11, 2, 12] that requires program understanding [13, 14], an activity that developers perform more frequently than any other [15]. While detecting the execution differences can help to understand the new behavior [16], finding these differences is difficult [16] because a deep knowledge of the code is required to understand the changes'

impact. Standard debugging tools are therefore insufficient, and specialized techniques and tools were developed.

*Delta debugging* [17, 18, 19, 20] automates the detection and correction of bugs. The delta debugging algorithms take as input a program containing a bug and a test which detects an error. It provides as output a minimal code change that passes the test, therefore highlighting the part of the code that introduces the bug. However, it does not help further in understanding why these changes lead to a bug [19].

*Relative debugging* [21] uses an interactive debugger to identify execution differences between two versions of the same program. Developers manually define assertions to check the state of key data structures in both versions. The debugger then runs both versions and provides side-by-side reports (one view for each execution) of any discrepancies in the data structures' states during execution. However, manually defining assertions makes relative debugging cumbersome and prone to errors. An enhanced version [22] allows for the automatic generation of assertions once the developer manually identifies the corresponding variables in both versions. Both standard and automated relative debugging focus solely on differences in data structure states, overlooking variations in control flow, even though control flow differences are crucial for understanding program behavior [2, 23].

In contrast, *Echo debugging* [24] automatically detects all control flow differences and reports them using a dual debugger, but it does not account for deviations in execution state between the *reference* and the *modified* versions. Furthermore, the Echo debugger operates on an ad hoc basis, without clearly defining the criteria for comparing the two executions, making it difficult to replicate its approach.

In these solutions [9, 22, 24], divergences reveal differences in either state or control flow, but not both. We argue that combining both types of divergences offers more comprehensive insight into the impact of code changes and leads to a deeper understanding of the program's behavior.

To address these challenges, we propose *Divergence-Driven Debugging*, a tool-based approach for detecting divergences between the executions of two versions of the same program. We introduce $D^3$, an experimental time-traveling debugger built in Pharo [25]. We designed $D^3$ to capture and expose execution divergences, in order to help understanding why and how specific changes impact the behavior of a program. We evaluate $D^3$ and the divergence-driven debugging approach
on a debugging scenario of a well-documented bug from the literature and show that D³ exposes the critical information required to understand the bug.

## II. DIVERGENCE-DRIVEN DEBUGGING WITH D³

We present our approach and the D³ debugger, designed to help understanding execution differences between two versions of the same program.

### A. Divergence-Driven Debugging in a Nutshell

The D³ debugger (Figure 1) compares the *reference* and *modified* programs' executions. To do so, D³ uses the concepts of execution divergence and convergence.

a) **Definitions:** An *execution divergence* [26] occurs when two executions that initially follow the same instructions deviate by taking different execution paths. We define a divergence between two executions as a difference of control flow (i.e., they follow different paths), or as a difference of state (values in their data structures are different). A *reconvergence* means the two executions share a common state and return to an identical trajectory [27]. We define a convergence as the end of a divergence, i.e., both executions synchronize their execution paths and state after having diverged.

b) *The divergence-driven debugging approach:* While state divergences help in identifying bugs [9], we argue that flow divergences help to understand why two executions have state divergences. Therefore, to understand why two executions yield different results and behaviors, developers explore interactively the control flow and state divergences detected by the debugger (Figure 1, A). Each divergence leads to the associated stack frames of both execution (Figure 1, B1 & B2) with the corresponding source code (Figure 1, C1 & C2) where developers can observe the instructions that diverge. The debugger is interactive and the live state of each variable can be inspected (not visible in Figure 1). Developers can time-travel back in time to the detection of a divergence by using the *jump* action of a divergence. They can execute one or both programs executions step by step to investigate the reasons of their divergences.

### B. Execution Model and Comparison

To detect divergences and convergences between the *reference* and *modified* executions, we capture and compare every message send (for control flow divergences/convergences) and every state modification (for state divergences/convergences). To do so, we rely on *Time Travelling Queries* (TTQ) [28, 29]. A TTQ is a query over a program execution that selectively collects information from that execution. It is then possible to restore the execution context from which information was collected, in a live and interactive debugger. The Seeker debugger [29] provides a list of ready-to-use queries, in particular one for finding all messages sends and one for finding all assignments occurring during an execution. We execute these two TTQ over the two program versions, and do a divergence/convergence analysis on the TTQ results.

a) *Execution model:* TTQs provide an execution model that reifies run-time information and returns it as a list of TTQ results. In our case, these reifications represent message sends and assignments and contains:

* An abstract syntax tree (AST) node matching the TTQ (i.e., a message node or an assignment node). It corresponds to the executed instruction.

* A program counter, representing the number of the executed instruction at the time of the capture.

* The receiver: the object executing the instruction.

* For assignments: the assigned value.

* For messages: the message receiver and its arguments.

Message nodes contain a message selector and assignment nodes contain a variable name to which assignment are made. These reifications make it simple to compare both executions on message-send/assignment basis through a divergence/convergence analysis.

b) *Identifying divergences:* We identify divergences by comparing elements reified from TTQs on both executions. We consider as divergence all TTQ results for which at least one reification is not equivalent between both executions. Control flow divergences occur if any of these conditions are met:

* The program counter differs between both executions, i.e. one or more actions were executed in one execution. E.g., code was added between two message sends in one program version.

* The AST is a message node and the nodes differ between both executions. E.g., a message send changed in one program version.

State divergences occur if the receiver, the message receiver, the message and method arguments, the variable name or value differ between both executions. E.g., the assigned variable changed in one program version.

A divergence can be detected as both a state and control flow divergence. When detected, a divergence is added to a list of divergences and we look for a convergence point to delimitate the divergence.

c) *Identifying Convergences:* To identify convergences, we compare receivers and message sends:

* For control flow convergences, we check whether the same message node appears later in the execution of the other version, and if this node is associated with an equivalent receiver.

* For state convergence, we check if the same assignment occurs later for the same receiver.

## III. EVALUATION

We present a debugging scenario of a real bug, introduced in a system through a single change whose impact difficult to understand through conventional means. We show that our debugger is able retrieve state and flow divergences between the two versions of the program under study, and that exposing these divergences exposes the root cause of the bug.
<figure><img src="image_3.png" /><figcaption>Fig. 1. D³ debugger: an interactive time-traveling debugger showing execution divergences (A), stacks of compared executions (B1 and B2) with the executed source code (C1 and C2). We can time-travel between divergences (jump) and then execute both executions step-by-step to understand how they diverge.</figcaption></figure>

## A. The Pillar Bug

Pillar [30] is a document generator for the Pharo [25] system. It is a complex software relying on many external projects. In 2017, a simple refactoring (adding an instance variable and its accessors) provoked unit tests failures in code unrelated to the refactoring. Pillar developers were never able to solve the bug and they chose to completely rebuild the incriminated part of the codebase from scratch. This infamous bug was then labeled as the pillar bug<fnref n="1" /> [31]. Researchers experimented advanced debugger prototypes on the bug and finally explained it [32, 33, 34] The bug was later simplified for explanation and simplification [24]. In the following, we explain and use this simplified version of the Pillar bug.

The bug description [31]: Pillar uses configuration objects to stores properties for document generation. To store and access properties, the reference version of Pillar used a dictionary. Properties were accessed using reflection: the code contained unknown message sends that, when trying to access a property, triggered a system exception intercepted by Pillar at the meta-level. Configuration objects then performed an internal lookup for finding the property into their dictionaries.

Pillar developers wanted to remove this meta-level property lookup by adding instance variables to hold properties in configurations, and methods to access and modify these variables. This provoked a bug whose main symptom was that a configuration entirely lost the value of its properties, leading to tests failures. However, tests failed in part of the code that seemed unrelated and distant from the configuration classes that in which the modification occured.

Figure 2 shows the Pharo code<fnref n="2" /> of the unit test under debug. A first configuration `c1` is instantiated line 3, and is configured with a property `mySetting` line 4. A second configuration `c2` is instantiated line 5, and is configured as a child configuration of `c1` line 6. The test asserts that the value of the `mySetting` property of `c2` should be equal to the value of the same property in the parent configuration `c1`. The test passes in the *reference* version using reflective code, but fails in the *modified* version where we introduced a `mySetting` instance variable to the `PCBConfig` class with accessor methods.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Code of the failing test.</figcaption></figure>

## B. Solving the Pillar Bug

We used D³ to execute two versions of the Pillar program: one that works (**reference**) and one that fails (**modified**) in which we introduced an instance variable and its accessors. D³ detects **34** divergences, which are not all necessarily related to the bug under investigation. In the following, we explain only the divergences that helped in understanding the bug.

<footnote n="2">²For readers unfamiliar with Pharo, a comparison with Java-like syntax:
- Variables are declared between pipes `|` and `;`, assignments use `:=`
- The message-send notation uses spaces: `a foo()` is equivalent to `a.foo()`
- Arguments are specified by colons: `a includes b` is equivalent to `a.include(b)`</footnote>

<footnote n="1">¹https://github.com/guillep/pillar-bug</footnote>
<table><caption>Fig. 3. The first detected control flow divergence.</caption><thead><tr><th>Reference execution</th><th>Modified execution</th></tr></thead><tbody><tr><td>PC Context</td><td>PC Context</td></tr><tr><td>20 PCBTest&gt;>testChildConfigurationLooksUpParent</td><td>20 PCBTest&gt;>testChildConfigurationLooksUpParent</td></tr><tr><td>21 PCBConfig&gt;>doesNotUnderstand: #mySetting:</td><td>21 PCBConfig&gt;>mySetting:</td></tr></tbody></table>

The first control flow divergence concerns the `mySetting` message send (Figure 3), which execution takes a different path in each version. In the *reference* version, this message is not implemented and triggers a `doesNotUnderstand` (DNU) exception. This exception is passed to the Pillar domain code that starts executing a DNU method (Figure 3, line 21 of the *reference* execution stack). In the *modified* version, we added the `mySetting` message as a setter for the instance variable `mySetting` (Figure 3, line 21 of the *modified* execution stack).

D<fnref n="3" /> then detects state divergences in the *reference* execution, on the instance variable `propertyDictionary` of the configuration object `c1` (Figure 2, line 4). In the *modified* execution, only the `mySetting` variable is assigned the value 0. The control-flow divergence is resolved (i.e., both executions converge) when the `PCBConfig new` instruction is executed (Figure 2, line 5).

D<fnref n="3" /> identifies another control flow divergence when the `mySetting` message is sent to the `c2` configuration (Figure 2, line 7). This message remains unimplemented same as `mySetting`: in the *reference* version, and returns 0 in the *reference* execution (i.e., the correct value) and `nil` in the *modified* execution (hence the test fails).

To explore this divergence, we used the time-travel capabilities of D<fnref n="3" /> to reset the execution before the divergence occurs. There, we stepped both executions concurrently. We observed that the DNU message called a lookup algorithm that checked whether the dictionary of the current configuration contains a key matching the selector of the unknown message (i.e., `mySetting`). If the key is not found, the lookup continues its search in the dictionary of the `propertyDictionary` variable of the `parentConfig`, in this case, `c1` (line 6 in Figure 2), where the value associated with `mySetting` is 0. This value is then returned.

In the *modified* version, this message acted as a getter for the `mySetting` instance variable, returning `nil` because that variable was never assigned for the sub-configuration. The *modified* version lost a part of the *reference* behavior where a property not found was automatically looked up in the parent configuration, while this mechanism does not exist with standard getters and setters.

## C. Discussion and Further Evaluation

Both control flow and stated divergence helped us understanding the differences of behavior between the two program versions. State divergences enabled us to track and check the state differences of both program versions at each step. Flow divergences helped us to find a point of interest to explore in

details, for which we used time-travel to reset the execution before the divergence and dive in using step-by-step execution.

To explore further the applicability of our approach and its debugger, we conducted the same experiment on another complex scenario, the *Moose Critics bug<fnref n="3" />*. Moose is a complex software analysis platform for modeling and analyzing software systems, offering functionalities for querying, manipulation, transformation, and visualization of these systems. Moose-Critics is a tool provided by Moose to create and manage architectural and programming rules to help analyzing models. The Moose critics bug appears when loading rules: two different ways of importing the same rules produces a different, unexpected treatment of the rules.

We explored this bug in a similar way than to the Pillar bug, considering each way of importing the rules as a different program version – but this time we applied the debugger on real software and not a simplified bug reproduction. For clarity and space reasons, we do not report the full investigation of that bug. However, while we had no knowledge of Moose, D<fnref n="3" /> showed us control flow and state divergence helping us understanding the root cause of the bug (in this case, the type of the rules were different depending how they were loaded, which later triggered a critical control flow divergence). This shows that our approach and debugger can help in understanding larger systems.

# IV. CONCLUSION AND FUTURE WORK

We introduced our *Divergence-Driven Debugging* approach and its accompanying time-travel debugger D<fnref n="3" />, designed to help developers understand how specific changes impact a program's behavior. Our approach detects divergences in both control flow and state between a *reference* and a *modified* executions of the same program. To evaluate the D<fnref n="3" /> debugger, we applied it to the simplified version of the Pillar bug, a well-known case in the literature. We identified critical control flow and state divergences between the two program versions, and used time-travel debugging to jump directly to the corresponding execution points in each version. This allowed us to replay in details the critical points in both executions where we could observe the bug root cause. D<fnref n="3" /> offered valuable insights into behavioral and state changes that helped us understanding the bug. Furthermore, we used D<fnref n="3" /> to successfully understand a similar bug in Moose, a real-world, large system.

The divergence-driven debugging approach and D<fnref n="3" /> are promising, and our future work will focus on two key areas. First, we plan to conduct empirical studies to assess how D<fnref n="3" /> supports the comprehension of *modified* program versions in real-world settings. Second, we aim to address new challenges, such as changes occurring in system environments where changes are considerably larger than for a single program's change (e.g., transitioning from Pharo 10 to Pharo 13).

## ACKNOWLEDGMENT

This work was supported by the Région Hauts de France.

<footnote n="3">³https://modularmoose.org/posts/2022-08-08-moosecritics</footnote>
REFERENCES

[1] B. Li, X. Sun, H. Leung, and S. Zhang, "A survey of code-based change impact analysis techniques," *Software Testing, Verification and Reliability*, vol. 23, 2013.

[2] A. Zeller, *Why programs fail: a guide to systematic debugging*. Elsevier, 2009.

[3] M. Fowler, *Refactoring: Improving the Design of Existing Code*. Boston: Addison-Wesley Professional, 2018.

[4] X. Zhu and M. Böhme, "Regression greybox fuzzing," in *SIGSAC Conference on Computer and Communications Security*, 2021, pp. 2169–2182.

[5] Y. Tao, Y. Dang, T. Xie, D. Zhang, and S. Kim, "How do software engineers understand code changes? an exploratory study in industry," in *International symposium on the foundations of software engineering*, 2012.

[6] F. Ebert, F. Castor, N. Novielli, and A. Serebrenik, "An exploratory study on confusion in code reviews," *Empirical Software Engineering*, 2021.

[7] S. Lehnert, "A taxonomy for software change impact analysis," in *International Workshop on Principles of Software Evolution and the 7th annual ERCIM Workshop on Software Evolution*, 2011, pp. 41–50.

[8] T. Kanda, K. Shimari, and K. Inoue, "Didiffff: A viewer for comparing changes in both code and execution traces," in *International Conference on Program Comprehension*, 2022, pp. 528–532.

[9] K. Etemadi, A. Sharma, F. Madeiral, and M. Monperrus, "Augmenting diffs with runtime information," *Transactions on Software Engineering*, 2023.

[10] G. Tassey, "The economic impacts of inadequate infrastructure for software testing," *National Institute of Standards and Technology*, 2002.

[11] I. Vessey, "Expertise in debugging computer programs: An analysis of the content of verbal protocols," *Transactions on Systems, Man, and Cybernetics*, vol. 16, 1986.

[12] D. H. O'Dell, "The debugging mindset: Understanding the psychology of learning strategies leads to effective problem-solving skills." *Queue*, 2017.

[13] J. Sillito, G. Murphy, and K. De Volder, "Asking and answering questions during a programming change task," *IEEE Transactions on Software Engineering*, 2008.

[14] A. J. Ko, "Asking and answering questions about the causes of software behavior," Ph.D. dissertation, Carnegie Mellon University, 2008.

[15] R. Minelli, A. Mocci, and M. Lanza, "I know what you did last summer-an investigation of how developers spend their time," in *International conference on program comprehension*, 2015, pp. 25–35.

[16] D. Spinellis, "Modern debugging: The art of finding a needle in a haystack," *Commun. ACM*, 2018.

[17] A. Zeller, "Automated debugging: Are we close?" *Computer*, 2001.

[18] —, "Isolating cause-effect chains from computer programs," *SIGSOFT Software Engineering Notes*, 2002.

[19] G. Misherghi and Z. Su, "Hdd: hierarchical delta debugging," in *International conference on Software engineering*, 2006, pp. 142–151.

[20] G. Wang, R. Shen, J. Chen, Y. Xiong, and L. Zhang, "Probabilistic delta debugging," in *European Software Engineering Conference and Symposium on the Foundations of Software Engineering*, 2021, pp. 881–892.

[21] D. Abramson, C. Chu, D. Kurniawan, and A. Searle, "Relative debugging in an integrated development environment," *Software: Practice and Experience*, 2009.

[22] A. J. Searle, "Automatic relative debugging," Ph.D. dissertation, Queensland University of Technology, 2006.

[23] D. Spinellis, *Effective debugging: 66 specific ways to debug software and systems*. Addison-Wesley Professional, 2016.

[24] T. Dupriez, S. Costiou, and S. Ducasse, "First infrastructure and experimentation in echo-debugging," in *IWST*, 2020.

[25] A. P. Black, S. Ducasse, O. Nierstrasz, D. Pollet, D. Cassou, and M. Denker, *Pharo by Example*. Kehrsatz, Switzerland: Square Bracket Associates, 2009. [Online]. Available: http://books.pharo.org

[26] H. Palikareva, T. Kuchta, and C. Cadar, "Shadow of a doubt: testing for divergences between software versions," in *International Conference on Software Engineering*, 2016, pp. 1181–1192.

[27] B. Buhse, T. Wei, Z. Zang, A. Milicevic, and M. Gligoric, "Veddebug: Regression debugging tool for java," in *International Conference on Software Engineering: Companion Proceedings*, 2019, pp. 15–18.

[28] M. Willembrinck, S. Costiou, A. Etien, and S. Ducasse, "Time-Traveling Debugging Queries: Faster Program Exploration," in *International Conference on Software Quality, Reliability, and Security*, 2021.

[29] M. I. Willembrinck Santander, "An interactive debugging approach based on time-traveling queries," *Thesis*, Université de Lille, Nov. 2023.

[30] T. Arloing, Y. Dubois, D. Cassou, and S. Ducasse, "Pillar: A versatile and extensible lightweight markup language," in *IWST*, Aug. 2016.

[31] T. Dupriez, G. Polito, and S. Ducasse, "Analysis and exploration for new generation debuggers," in *IWST*, 2017.

[32] T. Dupriez, G. Polito, S. Costiou, V. Aranega, and S. Ducasse, "Sindarin: A versatile scripting api for the pharo debugger," in *International Symposium on Dynamic Languages*, 2019, pp. 67–79.

[33] S. Costiou, M. Kerboeuf, C. Toullec, A. Plantec, and S. Ducasse, "Object miners: Acquire, capture and replay objects to track elusive bugs," *Journal of Object Technology*, vol. 19, no. 1, pp. 1:1–32, Jul. 2020.

[34] S. Costiou, "Unanticipated behavior adaptation: application to the debugging of running programs," Ph.D. dissertation, Université de Bretagne occidentale, 2018.