Repository: spack/spack

Issue:
New packages postgresql-client & py-psycopg, enhance postgresql
We introduce a new package `postgresql-client` which allows for the installation of only client executables and libraries. It provides virtual packages `postgresql-c`, `postgresql-python`, `postgresql-perl`, `postgresql-tcl` and adds other variants (`lineedit`, `gssapi`) which weren't available in the original postgresql package. The postgresql package itself is updated to inherit from the client package and is therefore upgraded to understand the same variants, *etc.*, as the client. As a side effect this fixes the problems in the original package spec caused by a misspelled override (`configure_arg()` vs `configure_args()`).
We also introduce a new package `py-psycopg` which `depends_on(postgresql-python)`. It also depends on `postgresql-c` but here appears to be a problem with the current version of spack if a recipe `depends_on()` two virtual packages provided by the same concrete package. Advice is sought on whether this should be reported as an issue or understood as a limitation of the system.


Discussion:
I believe all requested changes have been made: please review.
@adamjstewart I'm confused. I thought this worked briefly, but now I'm getting:
```
==> Error: NameError: global name 'make' is not defined
NameError: NameError: global name 'make' is not defined

/home/greenc/work/cet-is/sources/spack/var/spack/repos/builtin/packages/postgresql-client/package.py:104, in install:
     2             """
     3             for subdir in ('src/bin', 'src/include', 'src/interfaces', 'src/pl'):
     4                 with working_dir(subdir):
  >> 5                     make('install')
```
It really is beginning to look like I need the inspect, because I have no idea where the `make()` global function is supposed to be pulled from. Could you explain this bit of magic to me, please?
`make()` is added to the module scope of the package here:
https://github.com/spack/spack/blob/develop/lib/spack/spack/build_environment.py#L358

I'm not sure why you would be seeing that error message. Perhaps because the package is used as a base class? @alalazo would know more than me on this one. If that's the case, you may need to keep the inspect.
There's an extra layer (literally). I had an overriding spec of the same name in a higher-priority repo that had:
```
from spack.pkg.builtin.postgresql_client import PostgresqlClient \
    as BuiltinPackage
```
Followed by:
```
class PostgresqlClient(BuiltinPackage):
  # Override some variants with different defaults ...
  ...
```
The `install()` method was inherited from `BuiltinPackage`, and its reference to `make()` cannot be resolved without the use of `inspect`. I'm not quite sure why this is the case, but I think `inspect` needs to be used in `postgresql-client` for the same reason it needs to be used in (say) `AutoToolsPackage`.
Since being a base (_e.g._ of `postgresql`) is the intent of `postgresql-client`, I'd like to put the use of `inspect` back in. Speaking more generally though, use of `inspect` would seem to be a best practice as overriding variant defaults is currently a (the?) way to solve the problem of needing site-local variant defaults because noone propagates variants upwards. If I build `python` with `+ucs4`, for example, I'm out of luck if I try to build a high level package the (say) `depends_on(py-setuptools)` because I'm going to get a default-built `python`.
@scheibelp 
@chissg generally `provides` is used for packages which provide a "well-known" API (e.g. mpi), I get the impression you are using it here to create sensible (but personally-administered) categorizations of provided functionality. If that is true, would you mind removing the `provides` declarations? Happy to chat on it more if that didn't make sense.
@scheibelp Per my https://github.com/spack/spack/pull/7729#issuecomment-396252815, this PR has one of the same motivations as #7729, namely that Fermilab has the need to distribute and install server-less packages wherever possible. Requiring the `postgresql` package when one only needs the C or Perl (say) client bindings, or requiring specifically the client when one might already have the full package built for other reasons, would seem to be a waste. I'm having trouble seeing this use of virtual packages as being materially different to, say, `tbb` from the point of view of use by dependent packages. I thought about having a variant `~server` on the single package, but the parallels with `mysql` and `tbb`, and the wish to avoid waste, led me in this direction. If there is no use of virtual packages here, then there is no motivation for two different recipes.
@chissg please updated the license headers on this PR

Resolving patch:
diff --git a/var/spack/repos/builtin/packages/postgresql/package.py b/var/spack/repos/builtin/packages/postgresql/package.py
--- a/var/spack/repos/builtin/packages/postgresql/package.py
+++ b/var/spack/repos/builtin/packages/postgresql/package.py
@@ -14,22 +14,98 @@ class Postgresql(AutotoolsPackage):
 
     homepage = "http://www.postgresql.org/"
     url      = "http://ftp.postgresql.org/pub/source/v9.3.4/postgresql-9.3.4.tar.bz2"
+    list_url = "http://ftp.postgresql.org/pub/source"
+    list_depth = 1
 
+    version('11.2',   sha256='2676b9ce09c21978032070b6794696e0aa5a476e3d21d60afc036dc0a9c09405')
+    version('11.1',   sha256='90815e812874831e9a4bf6e1136bf73bc2c5a0464ef142e2dfea40cda206db08')
+    version('11.0',   sha256='bf9bba03d0c3902c188af12e454b35343c4a9bf9e377ec2fe50132efb44ef36b')
+    version('10.7',   sha256='bfed1065380c1bba927bfe51f23168471373f26e3324cbad859269cc32733ede')
+    version('10.6',   sha256='68a8276f08bda8fbefe562faaf8831cb20664a7a1d3ffdbbcc5b83e08637624b')
+    version('10.5',   sha256='6c8e616c91a45142b85c0aeb1f29ebba4a361309e86469e0fb4617b6a73c4011')
+    version('10.4',   sha256='1b60812310bd5756c62d93a9f93de8c28ea63b0df254f428cd1cf1a4d9020048')
     version('10.3', '506498796a314c549388cafb3d5c717a')
     version('10.2', 'e97c3cc72bdf661441f29069299b260a')
-    version('9.3.4', 'd0a41f54c377b2d2fab4a003b0dac762')
+    version('10.1',   sha256='3ccb4e25fe7a7ea6308dea103cac202963e6b746697366d72ec2900449a5e713')
+    version('10.0',   sha256='712f5592e27b81c5b454df96b258c14d94b6b03836831e015c65d6deeae57fd1')
+    version('9.6.12', sha256='2e8c8446ba94767bda8a26cf5a2152bf0ae68a86aaebf894132a763084579d84')
+    version('9.6.11', sha256='38250adc69a1e8613fb926c894cda1d01031391a03648894b9a6e13ff354a530')
     version('9.5.3', '3f0c388566c688c82b01a0edf1e6b7a0')
+    version('9.3.4', 'd0a41f54c377b2d2fab4a003b0dac762')
 
-    depends_on('openssl')
-    depends_on('readline')
-
+    variant('client_only', default=False,
+            description='Build and install client only.')
     variant('threadsafe', default=False, description='Build with thread safe.')
+    variant('lineedit', default='readline',
+            values=('readline', 'libedit', 'none'), multi=False,
+            description='Line editing library')
+    variant('python', default=False, description='Enable Python bindings.')
+    variant('perl', default=False, description='Enable Perl bindings.')
+    variant('tcl', default=False, description='Enable Tcl bindings.')
+    variant('gssapi', default=False,
+            description='Build with GSSAPI functionality.')
 
-    def configure_arg(self):
+    depends_on('readline', when='lineedit=readline')
+    depends_on('libedit', when='lineedit=libedit')
+    depends_on('openssl')
+    depends_on('tcl', when='+tcl')
+    depends_on('perl', when='+perl')
+    depends_on('python', when='+python')
+
+    def configure_args(self):
         config_args = ["--with-openssl"]
+
         if '+threadsafe' in self.spec:
-            config_args.append("--enable-thread-safety")
+            config_args.append('--enable-thread-safety')
         else:
-            config_args.append("--disable-thread-safety")
+            config_args.append('--disable-thread-safety')
+
+        if self.spec.variants['lineedit'].value == 'libedit':
+            config_args.append('--with-libedit-preferred')
+        elif self.spec.variants['lineedit'].value == 'none':
+            config_args.append('--without-readline')
+
+        if '+gssapi' in self.spec:
+            config_args.append('--with-gssapi')
+
+        if '+python' in self.spec:
+            config_args.append('--with-python')
+
+        if '+perl' in self.spec:
+            config_args.append('--with-perl')
+
+        if '+tcl' in self.spec:
+            config_args.append('--with-tcl')
 
         return config_args
+
+    def install(self, spec, prefix):
+        if '+client-only' in self.spec:
+            for subdir in ('bin', 'include', 'interfaces', 'pl'):
+                with working_dir(os.path.join('src', subdir)):
+                    make('install')
+        else:
+            AutotoolsPackage.install(self, spec, prefix)
+
+    def setup_environment(self, spack_env, run_env):
+        spec = self.spec
+
+        if '+perl' in spec:
+            run_env.prepend_path('PERL5LIB', self.prefix.lib)
+        if '+tcl' in spec:
+            run_env.prepend_path('TCLLIBPATH', self.prefix.lib)
+        if '+python' in spec:
+            run_env.prepend_path('PYTHONPATH', self.prefix.lib)
+
+    def setup_dependent_environment(self, spack_env, run_env, dependent_spec):
+        spec = self.spec
+
+        if '+perl' in spec:
+            spack_env.prepend_path('PERL5LIB', self.prefix.lib)
+            run_env.prepend_path('PERL5LIB', self.prefix.lib)
+        if '+tcl' in spec:
+            spack_env.prepend_path('TCLLIBPATH', self.prefix.lib)
+            run_env.prepend_path('TCLLIBPATH', self.prefix.lib)
+        if '+python' in spec:
+            spack_env.prepend_path('PYTHONPATH', self.prefix.lib)
+            run_env.prepend_path('PYTHONPATH', self.prefix.lib)


Test patch: