Repository: spack/spack

Issue:
Installation issue: interproscan
### Steps to reproduce the issue

```console
% spack spec -I interproscan
 
Input spec
--------------------------------
 -   interproscan

Concretized
--------------------------------
 -   interproscan@5.61-93.0%gcc@8.5.0 build_system=generic patches=6a81cbb,70c0de2 arch=linux-rocky8-x86_64
[+]      ^maven@3.8.4%gcc@8.5.0 build_system=generic arch=linux-rocky8-x86_64
[+]      ^openjdk@11.0.17_8%gcc@8.5.0 build_system=generic arch=linux-rocky8-x86_64
[+]      ^perl@5.36.0%gcc@8.5.0+cpanm+open+shared+threads build_system=generic arch=linux-rocky8-x86_64
[+]          ^berkeley-db@18.1.40%gcc@8.5.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-rocky8-x86_64
[+]          ^bzip2@1.0.8%gcc@8.5.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]              ^diffutils@3.6%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^gdbm@1.23%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^zlib@1.2.13%gcc@8.5.0+optimize+pic+shared build_system=makefile arch=linux-rocky8-x86_64
[+]      ^python@3.10.10%gcc@8.5.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,7d40923,f2fd060 arch=linux-rocky8-x86_64
[+]          ^expat@2.5.0%gcc@8.5.0+libbsd build_system=autotools arch=linux-rocky8-x86_64
[+]              ^libbsd@0.11.7%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]                  ^libmd@1.0.4%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^gettext@0.21.1%gcc@8.5.0+bzip2+curses+git~libunistring+libxml2+tar+xz build_system=autotools arch=linux-rocky8-x86_64
[+]              ^libiconv@1.17%gcc@8.5.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]              ^libxml2@2.10.3%gcc@8.5.0~python build_system=autotools arch=linux-rocky8-x86_64
[+]              ^tar@1.30%gcc@8.5.0 build_system=autotools zip=pigz arch=linux-rocky8-x86_64
[+]          ^libffi@3.4.4%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^libxcrypt@4.4.33%gcc@8.5.0~obsolete_api build_system=autotools arch=linux-rocky8-x86_64
[+]          ^ncurses@6.4%gcc@8.5.0~symlinks+termlib abi=none build_system=autotools arch=linux-rocky8-x86_64
[+]          ^openssl@1.1.1t%gcc@8.5.0~docs~shared build_system=generic certs=mozilla arch=linux-rocky8-x86_64
[+]              ^ca-certificates-mozilla@2023-01-10%gcc@8.5.0 build_system=generic arch=linux-rocky8-x86_64
[+]          ^pkgconf@1.4.2%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^readline@8.2%gcc@8.5.0 build_system=autotools patches=bbf97f1 arch=linux-rocky8-x86_64
[+]          ^sqlite@3.40.1%gcc@8.5.0+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-rocky8-x86_64
[+]          ^util-linux-uuid@2.38.1%gcc@8.5.0 build_system=autotools arch=linux-rocky8-x86_64
[+]          ^xz@5.4.1%gcc@8.5.0~pic build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
```


### Error message

<details><summary>Error message</summary>
There isn't an error message as such, but the package builds with tiny, vestigial fake databases containing a handful of HMMs.  These databases are _just_ sufficient to allow the package to pass the post-installation tests, making the package initially _look_ functional (at least if you don't know what annotations the test sequences should get.)  However, the package cannot be used for actual analyses without the full databases, and examination of even the included test sequence will show that they are underannotated.  

Package filesizes are consistent with the presence of only truncated databases. Looking at Pfams:  While the pfam_a.hmm database when compressed is currently ~290 MB (https://pfam-docs.readthedocs.io/en/latest/ftp-site.html), "pfam_a.hmm" is <1MB:

```
% ls -la
total 448
drwxrwsr-x. 2 user group   4096 Aug  3 15:15 .
drwxrwsr-x. 3 user group  4096 Aug  3 15:15 ..
-rw-rw-r--. 1 user group    650 Feb 21 10:14 pfam_a.dat
-rw-rw-r--. 1 user group 389903 Feb 21 10:14 pfam_a.hmm
-rw-rw-r--. 1 user group  45780 Feb 21 10:14 pfam_a.seed
-rw-rw-r--. 1 user group  1229 Feb 21 10:14 pfam_clans
```

And the .hmm file contains only 4 HMMs:

```
% grep PF pfam_a.hmm                                
ACC   PF00004.32
ACC   PF12574.11
ACC   PF09847.12
ACC   PF00220.20
```

(There are similar issues with every other database, as far as I can tell, although I only manually checked a few like the Pfam and TIGRfam databases; only the non-database tools (like disorder prediction) are unaffected.)

Meanwhile full interproscan releases (including databases) are ~5.5 GB when compressed:
(https://github.com/ebi-pf-team/interproscan/releases), but the compressed spack interproscan package is <100 MB (the last file, which has a name matching the sha256 hash for the current spack interproscan version, ending in ...288e4.tar.gz.

```
% ls -la --block-size=M
total 104M
drwxrwsr-x.   2 user group  1M Aug  3 14:25 .
drwxrwsr-x. 218 user group  1M Jul  6 16:25 ..
-rw-rw-r--.   1 user group  4M May 26 23:46 70230aa70a2c6f844fc41dd93e5f62af6859dfed390026ae58f223637e5283ca.tar.gz
-rw-r--r--.   1 user group  9M Jun 26 12:44 704aed49b19eb5a7178b34b2873620ec299db08752d6a8574f95d41879ab8851.tar.gz
-rw-rw-r--.   1 user group  1M Jun  1 13:55 7072beb1f680a2fe3f9e535b797c146d22528990c72f63ddb49d2f350a3653ed.tar.xz
-rw-rw-r--.   1 user group  1M May 25 21:06 707cdc75038c70fe91779b888ac050f128565d3967ba96680e1b1c7cc9733875.tar.gz
-rw-r--r--.   1 user group 92M Aug  3 14:25 70aca3b14983733fe5119b6978cb707156d006d7f737aa60ce6c9addd6c288e4.tar.gz
```

The readme.txt only says:

```
For more information on downloading, installing, running and configuring InterProScan 5 please see:
https://interproscan-docs.readthedocs.io
```

And `spack info interproscan` says nothing about requirements for separate database downloads either.

So it is not stated to the user before, during, or after installation that the full databases are not present, making it initially unclear why only a handful of genes get annotated during use.   (The interproscan@5.56-89.0 package has equivalent issues.). There are also no guides to manually downloading the databases.  Since the non-spack interproscan package builds with the databases, their resources do not include this information either.

I suppose it's possible this is somehow limited to my institution's spack installation, but I have not encountered similar issues with other packages, and components like the sha256 hash for the version seem consistent?
</details>


### Information on your system

% spack debug report                                                                                                                                                                                                         137 ↵
* **Spack:** 0.21.0.dev0 (8eef458cea65902b8d751992455791f18c1e0e2e)
* **Python:** 3.6.8
* **Platform:** linux-rocky8-skylake_avx512
* **Concretizer:** clingo

### Additional information

Note: no files for build failure because in one sense, the build did not fail, the package contents were just incomplete - but since the package incompleteness leaves the package not usable as-is, "build error" seems like a better fit than some of the other options.

In any case, if the full databases can't included in the spack package, it should be made clear that they need to be downloaded separately.  Additionally, instructions/download locations should be included somehow - perhaps in the readme.txt file, or in the package description?  (If the full databases _can_ be included, that would be preferable, though.)

cc: @snehring (identified via `spack maintainers interproscan`)

### General information

- [X] I have run `spack debug report` and reported the version of Spack/Python/Platform
- [X] I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
- [X] I have uploaded the build log and environment files
- [X] I have searched the issues of this repo and believe this is not a duplicate


Discussion:
Let me check the build to make sure I didn't goof anything up severely during the refactor, but I don't think the build downloads the databases.

I believe historically we've just directed our users to download the data archive and put it somewhere when they need it (we don't want to store databases with our binaries). But I think I can probably add a variant that includes that with the build for folks that want it, or at the very least add directions on how to do so. 

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/interproscan/package.py b/var/spack/repos/builtin/packages/interproscan/package.py
--- a/var/spack/repos/builtin/packages/interproscan/package.py
+++ b/var/spack/repos/builtin/packages/interproscan/package.py
@@ -10,12 +10,16 @@ class Interproscan(Package):
     """InterProScan is the software package that allows sequences
     (protein and nucleic) to be scanned against InterPro's signatures.
     Signatures are predictive models, provided by several different
-    databases, that make up the InterPro consortium."""
+    databases, that make up the InterPro consortium. By default this
+    does not download all the databases you likely want, you'll either
+    want to build with +databases or download the data archive for the
+    version you installed to a separate location and modify interproscan.properties"""
 
     homepage = "https://www.ebi.ac.uk/interpro/interproscan.html"
     url = "https://github.com/ebi-pf-team/interproscan/archive/5.36-75.0.tar.gz"
     maintainers("snehring")
 
+    version("5.63-95.0", sha256="3d7babd09e64da3d7104c58f1e5104a298d69425e3210952331bc3f1ddf89ca6")
     version("5.61-93.0", sha256="70aca3b14983733fe5119b6978cb707156d006d7f737aa60ce6c9addd6c288e4")
     version("5.56-89.0", sha256="75e6a8f86ca17356a2f77f75b07d6d8fb7b397c9575f6e9716b64983e490b230")
     version("5.38-76.0", sha256="cb191ff8eee275689b789167a57b368ea5c06bbcd36b4de23e8bbbbdc0fc7434")
@@ -26,6 +30,41 @@ class Interproscan(Package):
         url="ftp://ftp.ebi.ac.uk/pub/software/unix/iprscan/4/RELEASE/4.8/iprscan_v4.8.tar.gz",
     )
 
+    resource(
+        when="@5.63-95.0 +databases",
+        name="databases",
+        url="https://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/5.63-95.0/alt/interproscan-data-5.63-95.0.tar.gz",
+        sha256="6048eabe2eeaa4630b7a6a0b34d8c5a1724b0d22bba318c04c43777368e16cc4",
+    )
+
+    resource(
+        when="@5.61-93.0 +databases",
+        name="databases",
+        url="https://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/5.61-93.0/alt/interproscan-data-5.61-93.0.tar.gz",
+        sha256="064aa4b4c3b2e27b457298359087878e48fc785bff801c95691f090d1b83867d",
+    )
+
+    resource(
+        when="5.56-89.0 +databases",
+        name="databases",
+        url="https://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/5.56-89.0/alt/interproscan-data-5.56-89.0.tar.gz",
+        sha256="49cd0c69711f9469f3b68857f4581b23ff12765ca2b12893d18e5a9a5cd8032d",
+    )
+
+    resource(
+        when="5.38-76.0 +databases",
+        name="databases",
+        url="https://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/5.38-76.0/alt/interproscan-data-5.38-76.0.tar.gz",
+        sha256="e05e15d701037504f92ecf849c20317e70df28e78ff1945826b3c1e16d9b9cce",
+    )
+
+    resource(
+        when="5.36-75.0 +databases",
+        name="databases",
+        url="https://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/5.36-75.0/alt/interproscan-data-5.36-75.0.tar.gz",
+        sha256="e9b1e6f2d1c20d06661a31a08c973bc8ddf039a4cf1e45ec4443200375e5d6a4",
+    )
+
     resource(
         when="@:4.8",
         name="binaries",
@@ -33,6 +72,12 @@ class Interproscan(Package):
         sha256="551610a4682b112522f3ded5268f76ba9a47399a72e726fafb17cc938a50e7ee",
     )
 
+    variant(
+        "databases",
+        default=False,
+        description="Fetch and include databases in the install. Greatly increases install size.",
+    )
+
     depends_on("java@8.0:8.9", type=("build", "run"), when="@5:5.36-99.0")
     depends_on("java@11", type=("build", "run"), when="@5.37-76.0:")
     depends_on("maven", type="build", when="@5:")
@@ -63,6 +108,10 @@ def install(self, spec, prefix):
         target = join_path("core", "jms-implementation", "target", "interproscan-5-dist")
         install_tree(target, prefix)
 
+        if spec.satisfies("+databases"):
+            remove_directory_contents(prefix.data)
+            install_tree(f"interproscan-{self.spec.version}/data", prefix.data)
+
         # link the main shell script into the PATH
         symlink(join_path(prefix, "interproscan.sh"), join_path(prefix.bin, "interproscan.sh"))
 


Test patch: