Repository: spack/spack

Issue:
OpenSSL checks are too slow
I often work on systems that can't see the outside world.  On such systems, everytime spack queries the OpenSSL package, a 2 minute delay is added to my command.  For example:

```
% time spack spec openssl%intel@16.0.3
<output trimmed>
==> Warning: This installation depends on an old version of OpenSSL, which may have known security issues. 
==> Warning: Consider updating to the latest version of this package.
==> Warning: More details at http://www.openssl.org
<output trimmed>

real    2m9.064s
user    0m0.632s
sys 0m0.216s
```

Also:

```
% spack -p spec openssl%intel@16.0.3
  ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1  127.316  127.316  127.316  127.316 {method 'connect' of '_socket.socket' objects}
        3    0.689    0.230    0.689    0.230 {built-in method poll}
<output trimmed>

```

Is there any way to disable this check? or make it time-out after 15-20 seconds?  For now, I just comment out this section of openssl/package.py.



Discussion:
Even when you are connected to the outside world this check can cause a noticeable slowdown. How necessary is it? Could we just remove all old versions of OpenSSL from the package and call it good enough? @alalazo @citibeth

I really want to remove this check.  It adds time to anything that uses the spec, and it gets executed for commands that don't need it like `spack spec`.  I don't think the benefits outweigh the cost.  I think users need to be responsible for pulling their own updates, at least until we can do something better.  I also don't know of another package manager that pings the OpenSSL site like this.

I agree 100% with Todd.

We've discussed possible better things in the past --- how to deprecate
outdated/insecure packages, remove them from the Spack installation, and
then rebuild around them.  It brings up a lot of thorny issues.  I don't
think that in the OpenSSL package is the right place to tackle them.

Also... packages should never have to download stuff.  In case they do need
to download stuff, you should make two packages: package (a) downloads the
stuff and "installs" by copying the tarballs, then package (b) uses the
tarballs from package (a).  Then make (b) depend on (a).

I think the right place for a check like this would be as a phase of some sort in @alalazo's #1186 PR, preferably one that is disable-able in a config file somewhere.  Right now it's sort of instrumented into `url_for_version`, which is the wrong place for it.  I'll make a PR to get rid of the check and we can talk about where it might make sense to put it back.

@KineticTheory: if you want to override this in the meantime you can change `url_for_version` to `_url_for_version` in `var/spack/repos/builtin/packages/openssl/package.py`


Resolving patch:
diff --git a/var/spack/repos/builtin/packages/openssl/package.py b/var/spack/repos/builtin/packages/openssl/package.py
--- a/var/spack/repos/builtin/packages/openssl/package.py
+++ b/var/spack/repos/builtin/packages/openssl/package.py
@@ -22,7 +22,6 @@
 # License along with this program; if not, write to the Free Software
 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
 ##############################################################################
-import urllib
 import llnl.util.tty as tty
 
 from spack import *
@@ -50,61 +49,15 @@ class Openssl(Package):
     parallel = False
 
     def url_for_version(self, version):
-        # This URL is computed pinging the place where the latest version is
-        # stored. To avoid slowdown due to repeated pinging, we store the URL
-        # in a private class attribute to do the job only once per version
-        openssl_urls = getattr(Openssl, '_openssl_url', {})
-        openssl_url = openssl_urls.get(version, None)
-        # Same idea, but just to avoid issuing the same message multiple times
-        warnings_given_to_user = getattr(Openssl, '_warnings_given', {})
-        if openssl_url is None:
-            if self.spec.satisfies('@system'):
-                # The version @system is reserved to system openssl. In that
-                # case return a fake url and exit
-                openssl_url = '@system (reserved version for system openssl)'
-                if not warnings_given_to_user.get(version, False):
-                    tty.msg('Using openssl@system: '
-                            'the version @system is reserved for system openssl')
-                    warnings_given_to_user[version] = True
-            else:
-                openssl_url = self.check_for_outdated_release(
-                    version, warnings_given_to_user)  # Store the computed URL
-            openssl_urls[version] = openssl_url
-            # Store the updated dictionary of URLS
-            Openssl._openssl_url = openssl_urls
-            # Store the updated dictionary of warnings
-            Openssl._warnings_given = warnings_given_to_user
-
-        return openssl_url
-
-    def check_for_outdated_release(self, version, warnings_given_to_user):
-        latest = 'ftp://ftp.openssl.org/source/openssl-{version}.tar.gz'
-        older = 'http://www.openssl.org/source/old/{version_number}/openssl-{version_full}.tar.gz'  # NOQA: ignore=E501
-        # Try to use the url where the latest tarballs are stored.
-        # If the url does not exist (404), then return the url for
-        # older format
-        version_number = '.'.join([str(x) for x in version[:-1]])
-        try:
-            openssl_url = latest.format(version=version)
-            urllib.urlopen(openssl_url)
-        except IOError:
-            openssl_url = older.format(
-                version_number=version_number, version_full=version)
-            # Checks if we already warned the user for this particular
-            # version of OpenSSL. If not we display a warning message
-            # and mark this version
-            if not warnings_given_to_user.get(version, False):
-                tty.warn(
-                    'This installation depends on an old version of OpenSSL, '
-                    'which may have known security issues. ')
-                tty.warn(
-                    'Consider updating to the latest version of this package.')
-                tty.warn('More details at {homepage}'.format(
-                    homepage=Openssl.homepage))
-                warnings_given_to_user[version] = True
-
-        return openssl_url
-
+        if '@system' in self.spec:
+            return '@system (reserved version for system openssl)'
+        else:
+            return super(Openssl, self).url_for_version(self.spec)
+
+    def handle_fetch_error(self, error):
+        tty.warn("Fetching OpenSSL failed. This may indicate that OpenSSL has "
+                 "been updated, and the version in your instance of Spack is "
+                 "insecure. Consider updating to the latest OpenSSL version.")
 
     def install(self, spec, prefix):
         # OpenSSL uses a variable APPS in its Makefile. If it happens to be set


Test patch: