Repository: gem5/gem5
Pull request #2161: util: Add feature to update debug flags in external hypercall util
Author: login: Harshil2107
Labels: name: util
description: Utilities for gem5. Typically found in "util"
Created: 2025-04-09T00:27:07Z | Merged: 2025-04-30T18:26:16Z

**Add utility to update debug flags during simulation**

This PR introduces a utility that enables users to dynamically update debug flags during a running simulation using the `hypercall_external_signal` utility.

The utility is executed via `orchestrator-request.py` located in `util/hypercall_external_signal`. To use it, run the following command:

```
python3 ./orchestrator-request.py update_debug_flags --debug-flags-to-update=Flag1,Flag2,...
```

If you want to disable an already enabled debug flag, simply prepend the flag with a `-`. For example, passing `-Exec` will disable the `Exec` debug flag.

### Example Usage

1. **Start a simulation:**
   ```bash
   ./build/ALL/gem5.opt configs/example/gem5_library/x86-ubuntu-run.py
   ```

2. **To enable the `Exec` debug flag mid-simulation:**
   ```bash
   python3 ./orchestrator-request.py update_debug_flags --debug-flags-to-update=Exec
   ```

3. **To disable the `Exec` flag and enable the `Fetch` debug flag:**
   ```bash
   python3 ./orchestrator-request.py update_debug_flags --debug-flags-to-update=-Exec,Fetch
   ```

This utility makes it easy to modify debug flags without the need to restart the simulation.


Diff:
diff --git a/src/python/gem5/simulate/exit_handler.py b/src/python/gem5/simulate/exit_handler.py
index 74c9eaf5215..ca571f28f19 100644
--- a/src/python/gem5/simulate/exit_handler.py
+++ b/src/python/gem5/simulate/exit_handler.py
@@ -392,12 +392,41 @@ def _get_status(self, simulator: "Simulator") -> Dict[str, str]:
             "instruction_count": simulator.get_instruction_count(),
         }
 
+    def _add_debug_flags(self, debug_flags: List[str]) -> Dict[str, str]:
+        from m5 import debug
+
+        flags_disabled = []
+        flags_enabled = []
+        invalid_flags = []
+        for flag in debug_flags:
+            off = False
+            if flag.startswith("-"):
+                flag = flag[1:]
+                off = True
+
+            if flag not in debug.flags:
+                invalid_flags.append(flag)
+                continue
+
+            if off:
+                flags_disabled.append(flag)
+                debug.flags[flag].disable()
+            else:
+                flags_enabled.append(flag)
+                debug.flags[flag].enable()
+
+        return {
+            "flags_enabled": str(flags_enabled),
+            "flags_disabled": str(flags_disabled),
+            "invalid_flags": str(invalid_flags),
+        }
+
     @overrides(ExitHandler)
     def _process(self, simulator: "Simulator") -> None:
         try:
             socket_path = self._payload.get("response_socket")
             function = self._payload.get("function", "status")
-
+            arguments = self._payload.get("arguments")
             if socket_path:
                 sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
                 sock.connect(socket_path)
@@ -407,6 +436,9 @@ def _process(self, simulator: "Simulator") -> None:
                 elif function == "get_stats":
                     stats = simulator.get_stats()
                     response = json.dumps(stats)
+                elif function == "update_debug_flags":
+                    debug_flags = arguments.split(",")
+                    response = json.dumps(self._add_debug_flags(debug_flags))
                 else:
                     response = json.dumps(
                         {"error": f"Unknown function: {function}"}
diff --git a/util/hypercall_external_signal/orchestrator-request.py b/util/hypercall_external_signal/orchestrator-request.py
index b0ed862178e..8db2c5575cb 100755
--- a/util/hypercall_external_signal/orchestrator-request.py
+++ b/util/hypercall_external_signal/orchestrator-request.py
@@ -171,7 +171,9 @@ def receive_full_message(conn: socket.socket, timeout: float = 30.0) -> str:
     return b"".join(chunks).decode()
 
 
-def send_and_receive_hypercall(pid: int, function: str) -> str:
+def send_and_receive_hypercall(
+    pid: int, function: str, arguments: str = None
+) -> str:
     """
     Send a hypercall to gem5 and wait for response.
 
@@ -180,8 +182,10 @@ def send_and_receive_hypercall(pid: int, function: str) -> str:
 
     :param pid: Process ID of gem5 instance
     :type pid: int
-    :param function: Hypercall function to execute ('status'|'get_stats')
+    :param function: Hypercall function to execute ('status'|'get_stats'|'update_debug_flags')
     :type function: str
+    :param arguments: This parameter contains the arguments for the function. The functions 'status' and 'get_stats' dont expect any arguments. The 'update_debug_flags' function expects a ',' separated string of debug flags to update on the simulation. If the debug flag starts with '-' then the flag will be disabled.
+    :type debug_flags_to_update: str
     :return: JSON response from gem5
     :rtype: str
     :raises TimeoutError: If gem5 doesn't respond within timeout
@@ -202,7 +206,11 @@ def send_and_receive_hypercall(pid: int, function: str) -> str:
 
     try:
         payload = json.dumps(
-            {"function": function, "response_socket": socket_path}
+            {
+                "function": function,
+                "arguments": arguments,
+                "response_socket": socket_path,
+            }
         )
         send_signal(pid, 1000, payload)
 
@@ -241,7 +249,14 @@ def main():
         "(auto-detected if not specified)",
     )
     parser.add_argument(
-        "function", choices=["status", "get_stats"], help="Function to execute"
+        "function",
+        choices=["status", "get_stats", "update_debug_flags"],
+        help="Function to execute",
+    )
+    parser.add_argument(
+        "--debug-flags-to-update",
+        metavar="FLAG1,FLAG2,..",
+        help="Sets the flags for debug output. This flag will only work if positional argument 'update_debug_flags' is passed. To disable a flag you can pass the flag name starting with '-' for example -FLAG1. If an invalid flag is passed the flag will be skipped.",
     )
     parser.add_argument(
         "--output", type=str, help="Write response to specified file"
@@ -250,7 +265,9 @@ def main():
 
     try:
         pid = args.pid if args.pid is not None else find_gem5_pid()
-        response = send_and_receive_hypercall(pid, args.function)
+        response = send_and_receive_hypercall(
+            pid, args.function, args.debug_flags_to_update
+        )
         if args.output:
             write_response_to_file(response, args.output)
         else:


Reviews:
user: login: powerjg
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-04-09T15:39:12Z
user: login: powerjg
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-04-29T22:57:31Z

Review comments:
diff_hunk: @@ -392,12 +392,40 @@ def _get_status(self, simulator: "Simulator") -> Dict[str, str]:
             "instruction_count": simulator.get_instruction_count(),
         }
 
+    def _add_debug_flags(self, debug_flags: [List]) -> Dict[str, str]:
path: src/python/gem5/simulate/exit_handler.py
user: login: powerjg
body: I think you mean

```suggestion
    def _add_debug_flags(self, debug_flags: List[str]) -> Dict[str, str]:
```
created_at: 2025-04-09T15:34:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -171,7 +171,9 @@ def receive_full_message(conn: socket.socket, timeout: float = 30.0) -> str:
     return b"".join(chunks).decode()
 
 
-def send_and_receive_hypercall(pid: int, function: str) -> str:
+def send_and_receive_hypercall(
+    pid: int, function: str, debug_flags_to_add: str = None
path: util/hypercall_external_signal/orchestrator-request.py
user: login: powerjg
body: I don't think it makes sense to have an argument specific to one function in the generic function. This can be solved by either (1) making it a generic "arguments" or (2) making a new function to send/recv "add_debug_flags".

Also, why is it called "add_debug_flags" if you can both enable and disable flags? Maybe it should be "update_debug_flags" or something like that.
created_at: 2025-04-09T15:37:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -180,8 +182,10 @@ def send_and_receive_hypercall(pid: int, function: str) -> str:
 
     :param pid: Process ID of gem5 instance
     :type pid: int
-    :param function: Hypercall function to execute ('status'|'get_stats')
+    :param function: Hypercall function to execute ('status'|'get_stats'|'add_debug_flags')
     :type function: str
+    :param debug_flags_to_add: Contains a ',' separated string of debug flags to add to the simulation. This field will be only work if the value of the 'function' argument is 'add_debug_flags'.
path: util/hypercall_external_signal/orchestrator-request.py
user: login: powerjg
body: Make it clear that `-` at the beginning of the flag means to remove it.
created_at: 2025-04-09T15:37:53Z
author_association: CONTRIBUTOR
diff_hunk: @@ -241,7 +249,14 @@ def main():
         "(auto-detected if not specified)",
     )
     parser.add_argument(
-        "function", choices=["status", "get_stats"], help="Function to execute"
+        "function",
+        choices=["status", "get_stats", "add_debug_flags"],
+        help="Function to execute",
+    )
+    parser.add_argument(
+        "--debug-flags-to-add",
+        metavar="FLAG1,FLAG2,..",
+        help="Sets the flags for debug output. This flag will only work if positional argument 'add_debug_flags' is passed",
path: util/hypercall_external_signal/orchestrator-request.py
user: login: powerjg
body: Again, make it clear that `-` means to remove it. Also, specify what happens if a flag is "invalid"
created_at: 2025-04-09T15:38:19Z
author_association: CONTRIBUTOR

PR comments:
user: login: powerjg
created_at: 2025-04-25T22:31:02Z
body: This looks good. Can you extend the description (at least on GitHub) with an example of how you can use this? Also, an explanation of what you can do and why it's useful would be good to have in the description.

Finally, do you mean for this to have 2 commits, or do you want a single commit? If a single commit, then you need to update the commit message.
author_association: CONTRIBUTOR
user: login: powerjg
created_at: 2025-04-29T17:26:07Z
body: @Harshil2107 can you make these small changes so we can get this merged?
author_association: CONTRIBUTOR
user: login: Harshil2107
created_at: 2025-04-29T18:42:14Z
body: I have updated the PR description with more details and an example. We can squash this PR since we don't need two commits. This will allow the PR description to become the commit message, providing more context in the commit history.
author_association: CONTRIBUTOR