Repository: spack/spack

Issue:
Changing a patch, leads to NoSuchPatchError
`astyle` is a random package (chosen because of its lack of dependencies)

Add some patch to it
```
diff --git a/README.md b/README.md
index 98d8a6c..01754eb 100755
--- a/README.md
+++ b/README.md
@@ -7,3 +7,5 @@ The file **astyle.html**' contains information on using Artistic Style.
 
 The files **news.html** and **notes.html** contain information on changes
 made to the various releases.
+
+aaa
```
do `spack install astyle && spack uninstall astyle`, change the patch, in any way
```
% spack install astyle                           
==> Installing astyle
==> Searching for binary cache of astyle
==> No binary for astyle found: installing from source
==> Using cached archive: /Users/healther/Documents/git.nosync/spack/var/spack/cache/astyle/astyle-3.1.tar.gz
==> Error: NoSuchPatchError: Couldn't find patch with sha256: 10caab73785f3cddcaf8365d4734261e999301bf702c7207bf5a688d21febcc2

/Users/healther/Documents/git.nosync/spack/lib/spack/spack/package.py:1075, in do_fetch:
       1072            self.stage.check()
       1073
       1074        self.stage.cache_local()
  >>   1075
       1076        for patch in self.spec.patches:
       1077            patch.fetch(self.stage)
```

For ease of reproduction: https://github.com/spack/spack/commit/b55db69c6d911dfceef18b192c8a8e9cdd63ff2f
do the `spack install` on the two commits to reproduce the problem


Imo this should not happen, as the `patch` directive does not require a checksum for repo-internal patches and as such should not fail for something it remembered. Granted that only hits developers who should know about `spack clean -a`, but still.


Discussion:
This problem just hit me too.  What should I do to get around it?  I don't want to clean out all cache state of all types, every time I modify my patch!

I worked around this problem by deleting `~/.spack/cache/patches/builtin-index.json`.  It "solved" the problem, but took an awfully long time to regenerate.  Deleting individual entries in the file did NOT help.

How can we tell / coax Spack into regenerating patch index just for the package I'm working on `petsc`?  Even better... if Spack cannot find a patch in the index, why doesn't it regenerate that part of the index on the fly?

@scheibelp?
I have also run into this issue with my own repo index. As you said deleting the index does fix it, and when it's for a smaller index the regeneration time isn't that bad. That said, this is definitely a bug because as soon as I delete the index and it regenerates, Spack finds the patch correctly.

What I don't understand, is how it didn't find the patch to begin with. Every time I've run into this the patch file has never changed, it's always in the same place with the same sha256 hash. In my case, the package wasn't even in the patch cache at all when it couldn't find the patch. 

So, when/how does this cache get generated and how could a package that the environment knew it had to install slip through the cracks? My only guess is that the builtin repo has interfered because the package does not have a patch in that index but it does in mine.
I've also just faced this issue after copying some packages from a read-only "system" spack distribution to my own dir to apply some edits, and had to delete the whole dir `~/.spack/cache/patches/` in order to "solve".
... and I'm still figuring out how to fix this properly when inside an environment. In my case I had to change the version range when to apply a patch and Spack ignored my changes to the file completely when inside the environment. The only thing which worked so far was to install the packages outside the environment and then doing a `spack concretize -f && spack install` inside the environment.
I just encountered this as well (I'm modifying a patch while trying to debug a one-off problem). Very confusing!  I'm glad to have found this issue and @citibeth 's suggestion to delete that json file.  That was a successful workaround.
I've encountered this many times over the past week iterating on patches. I've been changing the name of the patch file to get around it, essentially with a version number. This way the patch file doesn't change.
I think one aspect that can be confusing (at least it was to me) is that running `spack concretize` also locks down the patches to be applied, irrespective of later edits to the `package.py`.
At the same time, we are used to edit the `package.py` to make changes to things like configure / cmake args  / ... and have them apply "on the fly" during `spack install`.

I guess in principle the patches follow directly from the concretized version, variant, etc., which may be why I didn't think of them being part of the concretization problem. I guess the reason they are part of the concretization stage is for computing the hash (?).

A first step here could simply be to make the error message
```
Error: NoSuchPatchError: Couldn't find patch with sha256: 10caab73785f3cddcaf8365d4734261e999301bf702c7207bf5a688d21febcc2
```
more helpful by explaining where this information comes from, and what the user needs to do to fix it (e.g. `concretize` in order for changes to patches in `package.py` to take effect).


A second step could in principle be to re-run the 'determine patches' algorithm for the `package.py` in question shortly before the point where the above error is raised, check whether the result is still the same as the one from the concretization, and warn the user when this is not the case.
It seems that if package version and variants, etc. are known, that might be relatively easy to do (but I may be wrong).

Resolving patch:
diff --git a/lib/spack/spack/patch.py b/lib/spack/spack/patch.py
--- a/lib/spack/spack/patch.py
+++ b/lib/spack/spack/patch.py
@@ -312,21 +312,19 @@ def from_json(cls, stream, repository):
     def to_json(self, stream):
         sjson.dump({"patches": self.index}, stream)
 
-    def patch_for_package(self, sha256, pkg):
+    def patch_for_package(self, sha256: str, pkg):
         """Look up a patch in the index and build a patch object for it.
 
         Arguments:
-            sha256 (str): sha256 hash to look up
+            sha256: sha256 hash to look up
             pkg (spack.package_base.PackageBase): Package object to get patch for.
 
         We build patch objects lazily because building them requires that
-        we have information about the package's location in its repo.
-
-        """
+        we have information about the package's location in its repo."""
         sha_index = self.index.get(sha256)
         if not sha_index:
-            raise NoSuchPatchError(
-                "Couldn't find patch for package %s with sha256: %s" % (pkg.fullname, sha256)
+            raise PatchLookupError(
+                f"Couldn't find patch for package {pkg.fullname} with sha256: {sha256}"
             )
 
         # Find patches for this class or any class it inherits from
@@ -335,8 +333,8 @@ def patch_for_package(self, sha256, pkg):
             if patch_dict:
                 break
         else:
-            raise NoSuchPatchError(
-                "Couldn't find patch for package %s with sha256: %s" % (pkg.fullname, sha256)
+            raise PatchLookupError(
+                f"Couldn't find patch for package {pkg.fullname} with sha256: {sha256}"
             )
 
         # add the sha256 back (we take it out on write to save space,
@@ -405,5 +403,9 @@ class NoSuchPatchError(spack.error.SpackError):
     """Raised when a patch file doesn't exist."""
 
 
+class PatchLookupError(NoSuchPatchError):
+    """Raised when a patch file cannot be located from sha256."""
+
+
 class PatchDirectiveError(spack.error.SpackError):
     """Raised when the wrong arguments are suppled to the patch directive."""
diff --git a/lib/spack/spack/spec.py b/lib/spack/spack/spec.py
--- a/lib/spack/spack/spec.py
+++ b/lib/spack/spack/spec.py
@@ -74,6 +74,7 @@
 import spack.deptypes as dt
 import spack.error
 import spack.hash_types as ht
+import spack.patch
 import spack.paths
 import spack.platforms
 import spack.provider_index
@@ -3906,7 +3907,15 @@ def patches(self):
                 for sha256 in self.variants["patches"]._patches_in_order_of_appearance:
                     index = spack.repo.PATH.patch_index
                     pkg_cls = spack.repo.PATH.get_pkg_class(self.name)
-                    patch = index.patch_for_package(sha256, pkg_cls)
+                    try:
+                        patch = index.patch_for_package(sha256, pkg_cls)
+                    except spack.patch.PatchLookupError as e:
+                        raise spack.error.SpecError(
+                            f"{e}. This usually means the patch was modified or removed. "
+                            "To fix this, either reconcretize or use the original package "
+                            "repository"
+                        ) from e
+
                     self._patches.append(patch)
 
         return self._patches


Test patch: