Repository: pwndbg/pwndbg
Pull request: Auto-update dependencies after a git pull
I updated the `gdbinit.py` code which checks for stored hash value, and compares it with current one. Based on the result appropriate `poetry install` command will be executed to install the required dependencies (if any) or completely skipped if the hashes match. I also updated the `setup-dev.sh` so as to add an additional file to mark the installation via `setup-dev.sh`. This Fixes #2138


Problem statement:
Auto-update dependencies after a `git pull`
After doing a `git pull`, it's possible that there were changes to system dependencies in `setup.sh` or Python dependencies in `pyproject.toml`/`poetry.lock` that need to be updated. If those dependencies don't get updated, things may work fine, or things may break in subtle and hard to reproduce ways. Python dependency updates are much more common than `setup.sh` updates, so I think this issue should just focus on that.

On startup, `pwndbg` should check for a file called `$PWNDBG_VENV_PATH/poetry.lock.hash` or something similar. If it doesn't exist it should create it with with the hash of `poetry.lock` and run `poetry install`. If it does exist, it should check if the hash in that file matches the actual hash of `poetry.lock`. if they differ, run `poetry install` and update the hash stored in `poetry.lock.hash`.

For developers, we need to run `poetry install -with dev` to update dev dependencies. We could maybe store an additional file in `$PWNDBG_VENV_PATH` marking the user as a developer (created in `setup-dev.sh`).

The one issue with this is that both normal and dev dependencies are stored in `poetry.lock`, so if I update a dev dependency the `poetry.lock` hash will change and `poetry install` will still get run, even though it wouldn't update anything for regular users. This won't happen very often, and `poetry install` is pretty fast anyway, so maybe this won't be a problem.

Hints from issue discussion:

Commit message: Fixed Linting issues

Patch to review:
diff --git a/gdbinit.py b/gdbinit.py
index ca277046dd1..2f770cccd92 100644
--- a/gdbinit.py
+++ b/gdbinit.py
@@ -1,8 +1,10 @@
 from __future__ import annotations
 
 import cProfile
+import hashlib
 import os
 import site
+import subprocess
 import sys
 import time
 from glob import glob
@@ -16,8 +18,46 @@
     _start_time = time.time()
     _profiler.enable()
 
+
 # Get virtualenv's site-packages path
 venv_path = os.environ.get("PWNDBG_VENV_PATH")
+
+
+def calculate_hash(file_path):
+    return hashlib.file_digest(open(file_path, "rb"), "sha256").hexdigest()
+
+
+def run_poetry_install(dev=False):
+    command = ["poetry", "install"]
+    if dev:
+        command = ["poetry", "install", "--with", "dev"]
+    subprocess.run(command, check=True)
+
+
+PWNDBG_VENV_PATH = os.getenv("PWNDBG_VENV_PATH")
+POETRY_LOCK_PATH = os.path.join(os.path.dirname(__file__), "poetry.lock")
+POETRY_LOCK_HASH_PATH = os.path.join(PWNDBG_VENV_PATH, "poetry.lock.hash")
+DEV_MARKER_PATH = os.path.join(PWNDBG_VENV_PATH, "dev.marker")
+
+# verify virtual environment path exists
+os.makedirs(PWNDBG_VENV_PATH, exist_ok=True)
+
+current_hash = calculate_hash(POETRY_LOCK_PATH)
+if os.path.exists(POETRY_LOCK_HASH_PATH):
+    with open(POETRY_LOCK_HASH_PATH, "r") as f:
+        stored_hash = f.read().strip()
+else:
+    stored_hash = None
+
+# checks if dev.marker exists
+dev_mode = os.path.exists(DEV_MARKER_PATH)
+
+# if hashes doesn't match, run the appropriate command based dev.marker file
+if current_hash != stored_hash:
+    run_poetry_install(dev=dev_mode)
+    with open(POETRY_LOCK_HASH_PATH, "w") as f:
+        f.write(current_hash)
+
 if venv_path == "PWNDBG_PLEASE_SKIP_VENV" or path.exists(path.dirname(__file__) + "/.skip-venv"):
     pass
 else:
diff --git a/setup-dev.sh b/setup-dev.sh
index 8ee1ab523eb..cdade051707 100755
--- a/setup-dev.sh
+++ b/setup-dev.sh
@@ -228,4 +228,9 @@ if linux; then
 
     source "${PWNDBG_VENV_PATH}/bin/activate"
     ~/.local/bin/poetry install --with dev
+
+    # Create a developer marker file
+    DEV_MARKER_PATH="${PWNDBG_VENV_PATH}/dev.marker"
+    touch "${DEV_MARKER_PATH}"
+    echo "Developer marker created at ${DEV_MARKER_PATH}"
 fi


Code review comments:
diff_hunk: @@ -16,8 +18,46 @@
     _start_time = time.time()
     _profiler.enable()
 
+
 # Get virtualenv's site-packages path
 venv_path = os.environ.get("PWNDBG_VENV_PATH")
+
+
+def calculate_hash(file_path):
+    return hashlib.file_digest(open(file_path, "rb"), "sha256").hexdigest()
+
+
+def run_poetry_install(dev=False):
+    command = ["poetry", "install"]
+    if dev:
+        command = ["poetry", "install", "--with", "dev"]
+    subprocess.run(command, check=True)
+
+
+PWNDBG_VENV_PATH = os.getenv("PWNDBG_VENV_PATH")
+POETRY_LOCK_PATH = os.path.join(os.path.dirname(__file__), "poetry.lock")
+POETRY_LOCK_HASH_PATH = os.path.join(PWNDBG_VENV_PATH, "poetry.lock.hash")
+DEV_MARKER_PATH = os.path.join(PWNDBG_VENV_PATH, "dev.marker")
+
+# verify virtual environment path exists
+os.makedirs(PWNDBG_VENV_PATH, exist_ok=True)
path: gdbinit.py
text: @user1:
I think we can get rid of this if we move this code below the `if not os.path.exists(venv_path):` check at line 71.

diff_hunk: @@ -16,8 +18,46 @@
     _start_time = time.time()
     _profiler.enable()
 
+
 # Get virtualenv's site-packages path
 venv_path = os.environ.get("PWNDBG_VENV_PATH")
+
+
+def calculate_hash(file_path):
+    return hashlib.file_digest(open(file_path, "rb"), "sha256").hexdigest()
+
+
+def run_poetry_install(dev=False):
+    command = ["poetry", "install"]
+    if dev:
+        command = ["poetry", "install", "--with", "dev"]
+    subprocess.run(command, check=True)
+
+
+PWNDBG_VENV_PATH = os.getenv("PWNDBG_VENV_PATH")
path: gdbinit.py
text: @user1:
Looks like the `venv_path` variable already exists, let's us that instead of making our own. I also think lower case variables are fine for all these names, as they're not constants.

@author:
I am sorry. The duplication was unintentional. When integrating the new functionality for checking and updating dependencies, I mistakenly redefined `venv_path`, not realizing it was already declared.