Repository: FastLED/FastLED (2026-07-31T05:37:17Z)

Issue:
build: src/ and tests/ metadata caches still use the mtime hash that caused #3761
Follow-up to #3761 / PR #3771, deliberately left out of that PR to keep it reviewable.

#3761 was caused by `compute_example_files_hash` keying on `f"{rel_path}:{stat.st_mtime:.6f}:{stat.st_size}"`. Every `git checkout`, `git worktree add` and branch switch rewrites mtimes with byte-identical content, so each one invalidated the cache and forced a ~20-30s reconfigure. PR #3771 fixed that one call site by hashing content instead.

**Two byte-identical copies of the same scheme remain:**

- `ci/meson/src_metadata_cache.py` — `compute_src_files_hash` (consumed by the top-level `--check`)
- `tests/test_metadata_cache.py` — `compute_test_files_hash` (consumed by the tests `--check`)

Both still miss on every checkout with unchanged content, so they re-run `src/` and `tests/` discovery inside each configure for no reason. The #3761 root cause survives in two of the three call sites, and will have to be diagnosed and patched twice more — with the fix diverging each time.

## Suggested fix

Extract one shared `compute_tree_content_hash(files, root)` helper and have all three use it. Related existing helpers worth folding in or reconciling: `ci/compiler/build_utils.py` `compute_content_hash(path)` (per-file SHA256 of content — exactly what #3771 re-implements inline) and `ci/iwyu_cache.py` `compute_source_tree_hash` (tree-level, also mtime-based).

Note the pruning subtlety #3771 hit: the hashed file set must match what the mtime fast path scans (`should_skip_scan_dir`), or build artifacts land in the hash and re-create the same false-positive class from a different direction.

## Separately: `detect_test_file_changes` conflates a missing cache with a change

`ci/meson/meson_setup_phases.py` — `except FileNotFoundError: cached_test_files = []` feeds the `current_test_files != cached_test_files` compare, so a fresh or cleaned build dir prints "Detected test file changes: Added: 284 test files" when nothing was added. Same misleading-diagnosis cost as #3761.

It does **not** self-perpetuate (the same branch rewrites the cache and watermark, so it heals on the first run) and does not use the mtime hash, so it only needs the missing-vs-changed reason split that #3771 applied to the example detector — not the same restructuring.


Discussion:

Patch:
diff --git a/ci/meson/cache_utils.py b/ci/meson/cache_utils.py
index 731516157b..7730bfdd74 100644
--- a/ci/meson/cache_utils.py
+++ b/ci/meson/cache_utils.py
@@ -8,11 +8,67 @@
 decisions. They must remain free of any non-stdlib imports.
 """
 
+import hashlib
 import json
 import os
 import platform
 from pathlib import Path
-from typing import Optional
+from typing import Iterable, Optional
+
+
+# ---------------------------------------------------------------------------
+# Content hashing
+# ---------------------------------------------------------------------------
+
+
+def compute_files_content_hash(files: Iterable[Path], root: Path) -> str:
+    """
+    Hash each file's path plus a digest of its contents.
+
+    Deliberately *content*-based rather than mtime-based. ``git checkout``,
+    ``git worktree add`` and branch switches rewrite mtimes with byte-identical
+    content, so an mtime hash treats every one of those as a source change and
+    forces the expensive rediscovery the cache exists to avoid (issue #3761 for
+    ``examples/``, #3773 for ``src/`` and ``tests/``).
+
+    Callers pass their own file list: the three metadata caches select files
+    very differently, and folding that selection in here would be how a real
+    source directory quietly stops being hashed.
+
+    Args:
+        files: Files to hash, in a stable order (the order is significant).
+        root: Directory paths are reported relative to. Files outside it fall
+            back to their absolute path so they still contribute uniquely.
+
+    Returns:
+        SHA256 hex digest over the (path, content) pairs.
+    """
+    digest = hashlib.sha256()
+    for f in files:
+        if not f.is_file():
+            continue
+        try:
+            rel_path = f.relative_to(root).as_posix()
+        except ValueError:
+            # Outside *root*: use the ABSOLUTE path. A bare f.as_posix() would
+            # keep a relative path, which can collide with an identically-named
+            # relative path inside root when the contents match -- and a
+            # collision here means an external file's change is invisible to
+            # the cache.
+            rel_path = f.absolute().as_posix()
+        try:
+            content_digest = hashlib.sha256(f.read_bytes()).digest()
+        except OSError:
+            # Unreadable (AV/indexer lock, permissions). Use a distinct marker
+            # so it cannot collide with a genuinely empty file, and so the hash
+            # still differs from the readable version -- failing toward a
+            # rediscovery rather than silently reporting "unchanged".
+            content_digest = b"<unreadable>".ljust(32, b"\0")
+        digest.update(rel_path.encode())
+        digest.update(b"\0")
+        digest.update(content_digest)
+        digest.update(b"\n")
+    return digest.hexdigest()
 
 
 # ---------------------------------------------------------------------------
diff --git a/ci/meson/example_metadata_cache.py b/ci/meson/example_metadata_cache.py
index 988592960e..05527fe985 100644
--- a/ci/meson/example_metadata_cache.py
+++ b/ci/meson/example_metadata_cache.py
@@ -28,7 +28,10 @@
 import sys
 from pathlib import Path
 
-from ci.meson.cache_utils import should_skip_scan_dir
+from ci.meson.cache_utils import (
+    compute_files_content_hash,
+    should_skip_scan_dir,
+)
 
 
 CACHE_FILENAME = "example_metadata.cache"
@@ -124,29 +127,9 @@ def compute_example_files_hash(examples_dir: Path) -> str:
     Returns:
         SHA256 hash of all example file paths + contents
     """
-    digest = hashlib.sha256()
-    for f in iter_example_files(examples_dir):
-        if not f.is_file():
-            continue
-        # Use absolute path for scripts outside examples_dir
-        try:
-            rel_path: str = f.relative_to(examples_dir).as_posix()
-        except ValueError:
-            rel_path = f.as_posix()
-        try:
-            content_digest = hashlib.sha256(f.read_bytes()).digest()
-        except OSError:
-            # Unreadable (Windows AV/indexer lock, permissions). Use a distinct
-            # marker so it cannot collide with a genuinely empty file, and so
-            # the hash still differs from the readable version — failing toward
-            # a reconfigure rather than silently reporting "unchanged".
-            content_digest = b"<unreadable>".ljust(32, b"\0")
-        digest.update(rel_path.encode())
-        digest.update(b"\0")
-        digest.update(content_digest)
-        digest.update(b"\n")
-
-    return digest.hexdigest()
+    # Paths outside examples_dir (the discovery script) fall back to their
+    # absolute path inside the helper, so they still contribute uniquely.
+    return compute_files_content_hash(iter_example_files(examples_dir), examples_dir)
 
 
 def load_cache(build_dir: Path) -> dict[str, str | float] | None:
diff --git a/ci/meson/src_metadata_cache.py b/ci/meson/src_metadata_cache.py
index 3530b028dd..b700b540a4 100644
--- a/ci/meson/src_metadata_cache.py
+++ b/ci/meson/src_metadata_cache.py
@@ -2,7 +2,7 @@
 Source file metadata caching for Meson build system.
 
 This module provides hash-based caching to avoid re-running source discovery scripts
-when source files haven't changed. It tracks all .cpp file metadata (path + mtime + size)
+when source files haven't changed. It tracks all .cpp file paths and contents
 and invalidates the cache only when sources are added, deleted, or modified.
 
 Usage:
@@ -17,18 +17,20 @@
 """
 
 import argparse
-import hashlib
 import json
 import sys
 from dataclasses import dataclass
-from os import stat_result
 from pathlib import Path
 
 from typeguard import typechecked
 
+from ci.meson.cache_utils import compute_files_content_hash
+
 
 CACHE_FILENAME = "src_metadata.cache"
-CACHE_VERSION = 2
+# Bumped to 3: the hash switched from mtime-based to content-based (#3773),
+# so entries written by an older version are not comparable.
+CACHE_VERSION = 3
 
 
 @typechecked
@@ -44,33 +46,30 @@ class CacheEntry:
 
 def compute_src_files_hash(src_dir: Path, pattern: str) -> str:
     """
-    Compute a hash of all source file metadata (path + mtime + size).
+    Compute a hash of all source file paths and contents.
 
     This provides a fast way to detect if any source files have been added,
     deleted, or modified without re-parsing all source files.
 
+    Content-based, not mtime-based: ``git checkout``, ``git worktree add`` and
+    branch switches rewrite mtimes with byte-identical content, and an mtime
+    hash treated every one of those as a source change, re-running discovery
+    inside each configure for nothing (issue #3773; #3761 was the same bug in
+    the examples cache).
+
+    The file selection is deliberately left as a plain ``rglob`` with no
+    build-artifact pruning: ``src/fl/build/`` is a real source directory (see
+    ``library.json`` -> ``build.srcFilter``), so a name-based "skip build dirs"
+    filter here would silently stop hashing shipped sources.
+
     Args:
         src_dir: Root directory containing source files
         pattern: File pattern to match (default: "*.cpp")
 
     Returns:
-        SHA256 hash of all source file metadata
+        SHA256 hash of all source file paths + contents
     """
-    # Find all files matching pattern recursively
-    source_files = sorted(src_dir.rglob(pattern))
-
-    # Create hash input from file metadata
-    hash_input: list[str] = []
-    for f in source_files:
-        if f.is_file():
-            stat: stat_result = f.stat()
-            rel_path: str = f.relative_to(src_dir).as_posix()
-            # Include path, mtime, and size in hash
-            hash_input.append(f"{rel_path}:{stat.st_mtime:.6f}:{stat.st_size}")
-
-    # Compute SHA256 hash
-    hash_str = "\n".join(hash_input)
-    return hashlib.sha256(hash_str.encode()).hexdigest()
+    return compute_files_content_hash(sorted(src_dir.rglob(pattern)), src_dir)
 
 
 def load_cache(build_dir: Path) -> CacheEntry | None:


Test patch:
diff --git a/ci/tests/test_content_hash.py b/ci/tests/test_content_hash.py
new file mode 100644
index 0000000000..c50c582c4a
--- /dev/null
+++ b/ci/tests/test_content_hash.py
@@ -0,0 +1,188 @@
+"""Tests for the shared content-hash helper (issue #3773, follows #3761).
+
+The three metadata caches (src/, tests/, examples/) used to key on
+``path:mtime:size``. ``git checkout``, ``git worktree add`` and branch switches
+rewrite mtimes with byte-identical content, so every one of those invalidated
+the cache and forced the rediscovery the cache exists to avoid. #3761 fixed the
+examples cache; this covers the extracted helper all three now share.
+"""
+
+from __future__ import annotations
+
+import os
+import unittest
+from pathlib import Path
+from tempfile import TemporaryDirectory
+from unittest import mock
+
+from ci.meson.cache_utils import compute_files_content_hash
+
+
+def _tree(root: Path) -> list[Path]:
+    (root / "pkg").mkdir(parents=True)
+    a = root / "pkg" / "a.cpp"
+    b = root / "pkg" / "b.h"
+    a.write_text("int a() { return 1; }\n")
+    b.write_text("#pragma once\n")
+    return [a, b]
+
+
+class TestContentHash(unittest.TestCase):
+    def test_mtime_churn_does_not_change_the_hash(self) -> None:
+        """The whole point: a git checkout must not invalidate the cache."""
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            before = compute_files_content_hash(files, root)
+
+            for f in files:
+                st = f.stat()
+                os.utime(f, (st.st_atime + 10_000, st.st_mtime + 10_000))
+
+            self.assertEqual(before, compute_files_content_hash(files, root))
+
+    def test_content_edit_changes_the_hash(self) -> None:
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            before = compute_files_content_hash(files, root)
+            files[0].write_text("int a() { return 2; }\n")
+            self.assertNotEqual(before, compute_files_content_hash(files, root))
+
+    def test_rename_changes_the_hash(self) -> None:
+        """Paths participate, so identical content under a new name differs."""
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            before = compute_files_content_hash(files, root)
+            renamed = files[0].with_name("renamed.cpp")
+            files[0].rename(renamed)
+            self.assertNotEqual(
+                before, compute_files_content_hash([renamed, files[1]], root)
+            )
+
+    def test_order_is_significant_so_callers_must_sort(self) -> None:
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            self.assertNotEqual(
+                compute_files_content_hash(files, root),
+                compute_files_content_hash(list(reversed(files)), root),
+            )
+
+    def test_dropping_a_file_changes_the_hash(self) -> None:
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            self.assertNotEqual(
+                compute_files_content_hash(files, root),
+                compute_files_content_hash(files[:1], root),
+            )
+
+    def test_missing_files_are_skipped_not_fatal(self) -> None:
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            files = _tree(root)
+            expected = compute_files_content_hash(files, root)
+            with_ghost = files + [root / "pkg" / "does_not_exist.cpp"]
+            self.assertEqual(expected, compute_files_content_hash(with_ghost, root))
+
+    def test_file_outside_root_still_contributes(self) -> None:
+        """The examples cache hashes its discovery script, which lives outside."""
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp) / "inside"
+            root.mkdir()
+            files = _tree(root)
+            outside = Path(tmp) / "outside.py"
+            outside.write_text("# discovery script\n")
+
+            base = compute_files_content_hash(files, root)
+            with_outside = compute_files_content_hash(files + [outside], root)
+            self.assertNotEqual(base, with_outside)
+
+            # And its *content* matters, not just its presence.
+            outside.write_text("# discovery script, changed\n")
+            self.assertNotEqual(
+                with_outside, compute_files_content_hash(files + [outside], root)
+            )
+
+    def test_relative_path_outside_root_cannot_collide_with_one_inside(self) -> None:
+        """An outside-root file is keyed by its ABSOLUTE path.
+
+        Keying it by the relative path it was passed as lets it collide with an
+        identically-named path inside *root* whenever the contents match, and a
+        collision means the external file's edits are invisible to the cache.
+        """
+        with TemporaryDirectory() as tmp:
+            work = Path(tmp) / "work"
+            root = work / "inside"
+            root.mkdir(parents=True)
+            files = _tree(root)  # root/pkg/a.cpp, root/pkg/b.h
+
+            # Same relative path, same content, but OUTSIDE root.
+            (work / "pkg").mkdir()
+            (work / "pkg" / "a.cpp").write_bytes(files[0].read_bytes())
+
+            # The collision only exists when the caller passes a *relative*
+            # path, because Path.as_posix() on an absolute path is already
+            # absolute. Reproducing it therefore requires the cwd to be `work`
+            # so that "pkg/a.cpp" names the outside file -- os.path.relpath()
+            # from the ambient cwd yields a long "../.." form that collides
+            # with nothing and would make this test vacuous.
+            #
+            # cwd is restored in finally. Note this is process-global: it is
+            # safe under pytest's default in-process serial execution and under
+            # xdist (separate worker processes), but do not make this class
+            # thread-parallel without revisiting it.
+            prev = os.getcwd()
+            os.chdir(work)
+            try:
+                outside_rel = Path("pkg") / "a.cpp"
+                self.assertFalse(outside_rel.is_absolute())
+                self.assertNotEqual(
+                    compute_files_content_hash([files[0]], root),
+                    compute_files_content_hash([outside_rel], root),
+                )
+            finally:
+                os.chdir(prev)
+
+    def test_unreadable_file_is_distinct_from_an_empty_one(self) -> None:
+        """An unreadable file must not hash the same as a genuinely empty one.
+
+        If it did, a file locked by AV or an indexer would hash identically to
+        an empty file, and a later edit made while it was still unlocked-but-
+        empty-looking would read as "unchanged". Drives the helper against a
+        real read failure rather than asserting against a hand-built digest,
+        so it would catch the helper skipping the file or hashing it as b"".
+        """
+        with TemporaryDirectory() as tmp:
+            root = Path(tmp)
+            (root / "pkg").mkdir()
+            target = root / "pkg" / "empty.cpp"
+            target.write_text("")
+
+            empty_hash = compute_files_content_hash([target], root)
+
+            real_read_bytes = Path.read_bytes
+
+            def deny(self: Path) -> bytes:
+                if self == target:
+                    raise PermissionError(13, "locked by another process")
+                return real_read_bytes(self)
+
+            with mock.patch.object(Path, "read_bytes", deny):
+                unreadable_hash = compute_files_content_hash([target], root)
+
+            self.assertNotEqual(
+                empty_hash,
+                unreadable_hash,
+                "an unreadable file must not hash like an empty one",
+            )
+
+            # And it must still be hashed, not silently dropped: a skipped file
+            # would collapse to the digest of an empty file list.
+            self.assertNotEqual(unreadable_hash, compute_files_content_hash([], root))
+
+
+if __name__ == "__main__":
+    unittest.main()
diff --git a/tests/test_metadata_cache.py b/tests/test_metadata_cache.py
index 2150ba1281..a34d1afa72 100644
--- a/tests/test_metadata_cache.py
+++ b/tests/test_metadata_cache.py
@@ -2,7 +2,7 @@
 Test metadata caching for Meson build system.
 
 This module provides hash-based caching to avoid re-running organize_tests.py
-when test files haven't changed. It tracks all test file metadata (path + mtime + size)
+when test files haven't changed. It tracks all test file paths and contents
 and invalidates the cache only when tests are added, deleted, or modified.
 
 Usage:
@@ -20,29 +20,36 @@
 """
 
 import argparse
-import hashlib
 import json
 import sys
 from itertools import chain
 from pathlib import Path
 from typing import Dict, List, Set
 
+from ci.meson.cache_utils import compute_files_content_hash
+
 
 CACHE_FILENAME = "test_metadata.cache"
 
 
 def compute_test_files_hash(tests_dir: Path) -> str:
     """
-    Compute a hash of all test file metadata (path + mtime + size).
+    Compute a hash of all test file paths and contents.
 
     This provides a fast way to detect if any test files have been added,
     deleted, or modified without re-parsing all test files.
 
+    Content-based, not mtime-based: ``git checkout``, ``git worktree add`` and
+    branch switches rewrite mtimes with byte-identical content, and an mtime
+    hash treated every one of those as a test-tree change, re-running discovery
+    inside each configure for nothing (issue #3773; #3761 was the same bug in
+    the examples cache).
+
     Args:
         tests_dir: Root directory containing test files
 
     Returns:
-        SHA256 hash of all test file metadata
+        SHA256 hash of all test file paths + contents
     """
     # Find all *.cpp / *.ino files recursively, same logic as discover_tests.py
     from discover_tests import TEST_SOURCE_GLOBS
@@ -70,18 +77,7 @@ def compute_test_files_hash(tests_dir: Path) -> str:
 
     unique_files = test_files  # already deduplicated by rglob + sorted
 
-    # Create hash input from file metadata
-    hash_input = []
-    for f in unique_files:
-        if f.is_file():
-            stat = f.stat()
-            rel_path = f.relative_to(tests_dir).as_posix()
-            # Include path, mtime, and size in hash
-            hash_input.append(f"{rel_path}:{stat.st_mtime:.6f}:{stat.st_size}")
-
-    # Compute SHA256 hash
-    hash_str = "\n".join(hash_input)
-    return hashlib.sha256(hash_str.encode()).hexdigest()
+    return compute_files_content_hash(unique_files, tests_dir)
 
 
 def load_cache(build_dir: Path) -> dict | None: