Repository: spack/spack

Issue:
Cannot Build Albacore - Will Submit PR After!
Hi Everyone

I promise I will submit a pull request after this gets resolved!

I am trying to install Albacore software into Spack.  It's a python basecaller.

The error:

```
sysadmin@pascal[~/repos/spack]$ spack install ont-albacore
==> py-ont-fast5-api is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-ont-fast5-api-0.3.2-b63i6txaetczgxkuvizroqlomvftvwu7
==> python is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/python-3.5.2-ahsjckbsufoenrtx5icl5hkoowvlpppm
==> py-pip is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-pip-9.0.1-hlncmuvaxq7buelw5epfe3bxmanm6ghp
==> py-dateutil is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-dateutil-2.5.2-pvd5ur3hyrndht34vg54p7euq7ur3gr2
==> py-numpy is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-numpy-1.12.1-y2lzjuwl23iqrubvzve3rdeuenaha4td
==> py-six is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-six-1.10.0-a4geypfh5s6ds3ta2wvu7j23mvonv6tu
==> py-setuptools is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-setuptools-34.4.1-5jenwqxxne5qjllagj36t27wy72rzv27
==> py-h5py is already installed in /pbtech_mounts/homes027/sysadmin/repos/spack/opt/spack/linux-rhel6-x86_64/gcc-4.4.6/py-h5py-2.6.0-o5utrgzddp7z5uyvufi4ltnqugeocvuo
==> Installing ont-albacore
==> Error: AttributeError: 'NoneType' object has no attribute 'lstrip'
/pbtech_mounts/homes027/sysadmin/repos/spack/lib/spack/spack/package.py:705, in _make_root_stage:
     703      def _make_root_stage(self, fetcher):
     704          # Construct a mirror path (TODO: get this out of package.py)
  >> 705          mp = spack.mirror.mirror_archive_path(self.spec, fetcher)
     706          # Construct a path where the stage should build..
     707          s = self.spec
     708          stage_name = "%s-%s-%s" % (s.name, s.version, s.dag_hash())
     709  
     710          def download_search():
     711              dynamic_fetcher = fs.from_list_url(self)
     712              return [dynamic_fetcher] if dynamic_fetcher else []
     713  
     714          stage = Stage(fetcher, mirror_path=mp, name=stage_name, path=self.path,
     715                        search_fn=download_search)
     716          return stage
```

```
sysadmin@pascal[~/repos/spack]$ spack info ont-albacore
Package:    ont-albacore
Homepage:   https://nanoporetech.com

Safe versions:  
    1.1.0    https://mirror.oxfordnanoportal.com/software/analysis/ont_albacore-1.1.0-cp35-cp35m-manylinux1_x86_64.whl

Variants:
    None

Installation Phases:
    install

Build Dependencies:
    py-dateutil  py-h5py  py-numpy  py-ont-fast5-api  py-pip  py-setuptools  py-six  python

Link Dependencies:
    None

Run Dependencies:
    py-dateutil  py-h5py  py-numpy  py-ont-fast5-api  py-pip  py-setuptools  py-six  python

Virtual Packages: 
    None

Description:
    The Albacore basecaller!
```

```
    depends_on('python@3.5.2',        type=('build', 'run'))
    depends_on('py-setuptools',        type=('build', 'run'))
    depends_on('py-six',        type=('build', 'run'))
    depends_on('py-numpy',        type=('build', 'run'))
    depends_on('py-dateutil',        type=('build', 'run'))
    depends_on('py-h5py',        type=('build', 'run'))
    depends_on('py-ont-fast5-api',        type=('build', 'run'))
    depends_on('py-pip',        type=('build', 'run'))
```

Do you know what could be causing this error?


Discussion:
The problem is the download URL. Spack takes your URL and tries to determine the compression scheme of the file. If it ends in `.tar.gz`, it uses tar to extract the download. If it ends in `.zip`, it uses unzip. In this case, it uses neither. We don't generally use wheels in Spack, so if there is a source code version, you should use that. If they only distribute wheels (maybe it's some proprietary software?) you can get around this by adding `expand=False` to the `version()` directive.

Alternatively (and this is more for other users who run into this error message), if the URL is for a normal tarball but the URL doesn't contain the string `.tar.gz`, you can add `extension='.tar.gz'` to the `version()` directive.

I'll write a PR that tries to make the error message a little more... helpful.
@adamjstewart 

Thanks!

Would you accept the wheel in PR if there's no source from the vendor?
Yes! If that's the only way to install it, that shouldn't be a problem. I don't know much (literally anything) about how wheels work, but if it's possible to install the package to its own installation prefix instead of to the Python installation prefix, that would be ideal.
It seems version() only takes two arguments.
You have to specify the parameter name. Here are other packages that do this:
```
$ grep expand=False */package.py
ack/package.py:    version('2.16', '7085b5a5c76fda43ff049410870c8535', expand=False)
ack/package.py:    version('2.14', 'e74150a1609d28a70b450ef9cc2ed56b', expand=False)
astra/package.py:    version('2016-11-30', '17135b7a4adbacc1843a50a6a2ae2c25', expand=False,
cuda/package.py:    version('8.0.61', '33e1bd980e91af4e55f3ef835c103f9b', expand=False,
cuda/package.py:    version('8.0.44', '6dca912f9b7e2b7569b0074a41713640', expand=False,
cuda/package.py:    version('7.5.18', '4b3bcecf0dfc35928a0898793cf3e4c6', expand=False,
cuda/package.py:    version('6.5.14', '90b1b8f77313600cc294d9271741f4da', expand=False,
miniconda2/package.py:    version('4.3.11', 'd573980fe3b5cdf80485add2466463f5', expand=False)
miniconda3/package.py:    version('4.3.11', '1924c8d9ec0abf09005aa03425e9ab1a', expand=False)
nextflow/package.py:            expand=False)
nextflow/package.py:            expand=False)
nextflow/package.py:            expand=False)
nextflow/package.py:            expand=False)
py-entrypoints/package.py:    version('0.2.2', '73bd7ce92c19b25dc5a20aff41be996a', expand=False)
vizglow/package.py:    version('2.2alpha20', '2bef890c66f3a44aaf96f7c96788c89e', expand=False,
vizglow/package.py:    version('2.2alpha17', '1de268564363e0ee86f9ffff1c3b82e1', expand=False,
vizglow/package.py:    version('2.2alpha15', 'be2b5044f30f2b2c3bbe87a0037bf228', expand=False,
```
Thanks @adamjstewart!

That worked!

How can I tell pip to install the wheel?  Obviously make does not work since it's not source:

```
    #def install(self, spec, prefix):
        # FIXME: Unknown build system
        #make()
        #make('install')
```
Try something like this:
```python
import glob

def install(self, spec, prefix):
    wheel = glob.glob('*.whl')[0]
    pip = which('pip')
    pip('install', '--prefix={0}'.format(prefix), wheel)
```
I'm not positive on the syntax, but that should be enough to get you there.

Resolving patch:
diff --git a/lib/spack/spack/mirror.py b/lib/spack/spack/mirror.py
--- a/lib/spack/spack/mirror.py
+++ b/lib/spack/spack/mirror.py
@@ -53,13 +53,33 @@ def mirror_archive_filename(spec, fetcher, resourceId=None):
         if fetcher.expand_archive:
             # If we fetch with a URLFetchStrategy, use URL's archive type
             ext = url.determine_url_file_extension(fetcher.url)
-            ext = ext or spec.package.versions[spec.package.version].get(
-                'extension', None)
-            ext = ext.lstrip('.')
+
+            # If the filename does not end with a normal suffix,
+            # see if the package explicitly declares the extension
             if not ext:
-                raise MirrorError(
-                    "%s version does not specify an extension" % spec.name +
-                    " and could not parse extension from %s" % fetcher.url)
+                ext = spec.package.versions[spec.package.version].get(
+                    'extension', None)
+
+            if ext:
+                # Remove any leading dots
+                ext = ext.lstrip('.')
+
+            if not ext:
+                msg = """\
+Unable to parse extension from {0}.
+
+If this URL is for a tarball but does not include the file extension
+in the name, you can explicitly declare it with the following syntax:
+
+    version('1.2.3', 'hash', extension='tar.gz')
+
+If this URL is for a download like a .jar or .whl that does not need
+to be expanded, or an uncompressed installation script, you can tell
+Spack not to expand it with the following syntax:
+
+    version('1.2.3', 'hash', expand=False)
+"""
+                raise MirrorError(msg.format(fetcher.url))
         else:
             # If the archive shouldn't be expanded, don't check extension.
             ext = None


Test patch: