Repository: mesonbuild/meson

Issue:
Improve curses detection
Currently, meson offers custom lookup for the `curses` library by searching for `ncurses` and `ncursesw` via pkg-config. And if that was enough, the world would be a better place. Unfortunately, the curses library has a long and twisted history that makes it a nightmare to use. There's different types of curses in use today (taken from [this](https://github.com/autoconf-archive/autoconf-archive/blob/master/m4/ax_with_curses.m4) autoconf script):

- Standard curses conforming to [X/Open Curses](https://publications.opengroup.org/c094) and System V UNIX, release 4 (SVr4).
- Enhanced curses with wide char support.
- Curses with the obsolete bits i.e. functions used on old BSDs and SVr4.
- "new" curses with support for colors and mice and panels and such.

As for implementations, there's a handful (I think):

- SVr4 / X/Open curses, still used on solaris/omnios (though omnios includes ncurses in base).
- ncurses, which implements SVr4 curses plus the colors and stuff.
- ncursesw, superset of ncurses that supports widechar things. May or may not be split out as a seperate library/header.
- PDcurses, which supports wide stuff and colours. Haven't looked deep into it.
- NetBSD curses, which is its own [can of worms](https://invisible-island.net/ncurses/ncurses-netbsd.html).

Then there's the header file. Because of all these libraries, you can expect to find the "right" one in various places:

```c
#include <ncursesw/ncurses.h>
#include <ncurses/ncurses.h>
#include <ncurses/curses.h>
#include <ncurses.h>
#include <curses.h>
```

Sometimes the bottom headers are just symlink/include wrappers around the top.

As mentioned, OS maintainers may split ncursesw into its own lib, so now you have to check these:

- `-lncursesw`
- `-lncurses`
- `-lcursesw`

But that's not all, you can also see ncurses 5 and 6 on the same machine due to backwards incompatible
changes. So append a 5 or 6 to each one.

ncurses started generating pkg-config files c. [2009](https://invisible-island.net/ncurses/NEWS.html#index-t20090103) to fix this, but some OSs don't include them. Sometimes they'll bundle ncurses-config to help, but this also affected by both the wide char split and the 5/6 split, so you have to search for `ncursesw?(5-6)?-config`.

So yeah, its a little confusing. I'm actually hitting this because of htop-dev/htop#80, and I resorted to making [this monstrosity](https://git.sgregoratto.me/ncurses-meson-dep/file/ncurses/meson.build.html):

```meson
# Ultimate ncurses detection 100% real no fake
nc_libs = [
	'ncursesw',
	'ncurses',
	'curses'
]
nc_headers = [
	'ncursesw/ncurses.h',
	'ncursesw.h',
	'ncurses/ncurses.h',
	'ncurses/curses.h',
	'ncurses.h',
	'curses.h',
]
nc_progs = [
	'ncursesw6',
	'ncursesw5',
	'ncurses6',
	'ncurses5',
]
wide_incantations = [
	'-D_XOPEN_SOURCE_EXTENDED',
	'-DNCURSES_WIDECHAR',
]

found = false
foreach h : nc_headers
	if cc.has_header(h)
		found = true
		conf.set('CURSES_HEADER', h)
		break
	endif
endforeach
if not found
	error('ncurses header not found')
endif

foreach l : nc_libs
	curses = dependency(l, required: false)
	if curses.found()
		subdir_done()
	endif
endforeach
foreach p : nc_progs
	# Method is whitelisted for specific libraries, darn.
	#curses = dependency(p, method: 'config', required: false)
	#if curses.found()
	#	subdir_done()
	#endif
	name = p + '-config'
	prog = find_program([name,
		'/bin' / name,
		'/usr' / 'bin' / name,
		'/usr' / 'local' / 'bin' / name],
		required: false
	)
	if not prog.found()
		continue
	endif
	libs	= run_command(prog, '--libs-only-l').stdout().strip()
	cflags	= run_command(prog, '--cflags').stdout().strip()
	version = run_command(prog, '--version').stdout().strip()

	cflags = ' '.join([cflags] + wide_incantations)
	curses	= declare_dependency(
		compile_args: cflags,
		link_args: libs,
		version: version,
	)
	subdir_done()
endforeach
foreach l : nc_libs
	curses = cc.find_library(l, required: false)
	if curses.found()
		subdir_done()
	endif
endforeach

error('ncurses library not found')
```

And even then it will probably break on some system. As @eli-schwartz put it

> honestly this cries out for the meson builtin dependency resolver for curses to be improved.


Discussion:
This is one of those cases where after detection the build file will likely need to know which headers should be used. We don't currently have a way to get that info from the [dependency object](https://mesonbuild.com/Reference-manual.html#dependency-object).

It can then also integrate with the `has_headers` kwarg to `cc.find_library()`, to fetch the list of headers that were found (which would be all of them if the dependency was found).
Even just detecting the right pkg-config target should help as that could then be used as a dependency in has_header() manual checks.

Maybe more generally it would be nice to have a shortcut syntax to check for a header and define configuration_data() entries for the HAVE_HEADER_H common pattern.
@nirbheek, I have an MR that would give us a way to get that from the get_variable method, if you think that's a good approach
@dcbaker I am personally not a big fan of adding additional functionality to an existing API unless it makes sense and is intuitive in the scope of the API. As far as I understand it, getting a list of found headers would not be a variable lookup and should IMHO not be exposed with `get_variable`.
@mensinda, I'm just concerned about the long term usability of dependencies if every system dependency implements a new public method that only works for that one dependency. Especially if it cannot be implemented by a subproject replacing that dependency.

Which for threads might not be an issue, but it definitely could be for curses.
On the other hand, feeding a magic string to `get_variable` (which, without further knowledge, I would assume is how it is implemented) feels really clunky and also only works for a select few dependencies.

As far as I understand the issue, the user needs to know more specific information about the dependency (a list of headers in this case). This could also be implemented for other dependencies (boost would be a good candidate). I am thinking of a general `dep.get_resource(what: str) -> T.List[str]` API that would accept a select few documented strings (`includes`, `libraries`, `prefix`, etc.) as an argument.

The difference to `get_variable` would be that this would work for all dependency types, without the need to specify multiple variables for the different backends. The advantage of this approach is that system dependencies could also easily use this interface. `declare_dependency` could also be updated to provide this information.
Sure, a get_resource interface seems like a good solution, just something generic and extensible

Resolving patch:
diff --git a/mesonbuild/dependencies/dev.py b/mesonbuild/dependencies/dev.py
--- a/mesonbuild/dependencies/dev.py
+++ b/mesonbuild/dependencies/dev.py
@@ -480,7 +480,7 @@ def __init__(self, name: str, environment: 'Environment', kwargs: T.Dict[str, T.
             for lib in libs:
                 l = self.clib_compiler.find_library(lib, environment, [])
                 h = self.clib_compiler.has_header('zlib.h', '', environment, dependencies=[self])
-                if l and h:
+                if l and h[0]:
                     self.is_found = True
                     self.link_args = l
                     break
diff --git a/mesonbuild/dependencies/misc.py b/mesonbuild/dependencies/misc.py
--- a/mesonbuild/dependencies/misc.py
+++ b/mesonbuild/dependencies/misc.py
@@ -32,7 +32,7 @@
 
 if T.TYPE_CHECKING:
     from ..environment import Environment, MachineChoice
-    from .base import DependencyType  # noqa: F401
+    from .base import DependencyType, Dependency  # noqa: F401
 
 
 @factory_methods({DependencyMethods.PKGCONFIG, DependencyMethods.CMAKE})
@@ -404,16 +404,101 @@ def get_methods():
         return [DependencyMethods.SYSTEM, DependencyMethods.PKGCONFIG]
 
 
-@factory_methods({DependencyMethods.PKGCONFIG})
+class CursesConfigToolDependency(ConfigToolDependency):
+
+    """Use the curses config tools."""
+
+    tool = 'curses-config'
+    # ncurses5.4-config is for macOS Catalina
+    tools = ['ncursesw6-config', 'ncursesw5-config', 'ncurses6-config', 'ncurses5-config', 'ncurses5.4-config']
+
+    def __init__(self, name: str, env: 'Environment', kwargs: T.Dict[str, T.Any], language: T.Optional[str] = None):
+        super().__init__(name, env, kwargs, language)
+        if not self.is_found:
+            return
+        self.compile_args = self.get_config_value(['--cflags'], 'compile_args')
+        self.link_args = self.get_config_value(['--libs'], 'link_args')
+
+
+class CursesSystemDependency(ExternalDependency):
+
+    """Curses dependency the hard way.
+
+    This replaces hand rolled find_library() and has_header() calls. We
+    provide this for portability reasons, there are a large number of curses
+    implementations, and the differences between them can be very annoying.
+    """
+
+    def __init__(self, name: str, env: 'Environment', kwargs: T.Dict[str, T.Any]):
+        super().__init__(name, env, kwargs)
+
+        candidates = [
+            ('pdcurses', ['pdcurses/curses.h']),
+            ('ncursesw',  ['ncursesw/ncurses.h', 'ncurses.h']),
+            ('ncurses',  ['ncurses/ncurses.h', 'ncurses/curses.h', 'ncurses.h']),
+            ('curses',  ['curses.h']),
+        ]
+
+        # Not sure how else to elegently break out of both loops
+        for lib, headers in candidates:
+            l = self.clib_compiler.find_library(lib, env, [])
+            if l:
+                for header in headers:
+                    h = self.clib_compiler.has_header(header, '', env)
+                    if h[0]:
+                        self.is_found = True
+                        self.link_args = l
+                        # Not sure how to find version for non-ncurses curses
+                        # implementations. The one in illumos/OpenIndiana
+                        # doesn't seem to have a version defined in the header.
+                        if lib.startswith('ncurses'):
+                            v, _ = self.clib_compiler.get_define('NCURSES_VERSION', '#include <{}>'.format(header), env, [], [self])
+                            self.version = v.strip('"')
+                        if lib.startswith('pdcurses'):
+                            v_major, _ = self.clib_compiler.get_define('PDC_VER_MAJOR', '#include <{}>'.format(header), env, [], [self])
+                            v_minor, _ = self.clib_compiler.get_define('PDC_VER_MINOR', '#include <{}>'.format(header), env, [], [self])
+                            self.version = '{}.{}'.format(v_major, v_minor)
+
+                        # Check the version if possible, emit a wraning if we can't
+                        req = kwargs.get('version')
+                        if req:
+                            if self.version:
+                                self.is_found = mesonlib.version_compare(self.version, req)
+                            else:
+                                mlog.warning('Cannot determine version of curses to compare against.')
+
+                        if self.is_found:
+                            mlog.debug('Curses library:', l)
+                            mlog.debug('Curses header:', header)
+                            break
+            if self.is_found:
+                break
+
+    @staticmethod
+    def get_methods() -> T.List[DependencyMethods]:
+        return [DependencyMethods.SYSTEM]
+
+
+@factory_methods({DependencyMethods.PKGCONFIG, DependencyMethods.CONFIG_TOOL, DependencyMethods.SYSTEM})
 def curses_factory(env: 'Environment', for_machine: 'MachineChoice',
-                   kwargs: T.Dict[str, T.Any], methods: T.List[DependencyMethods]) -> T.List['DependencyType']:
-    candidates = []  # type: T.List['DependencyType']
+                   kwargs: T.Dict[str, T.Any], methods: T.List[DependencyMethods]) -> T.List[T.Callable[[], 'Dependency']]:
+    candidates = []  # type: T.List[T.Callable[[], Dependency]]
 
     if DependencyMethods.PKGCONFIG in methods:
-        pkgconfig_files = ['ncurses', 'ncursesw']
+        pkgconfig_files = ['pdcurses', 'ncursesw', 'ncurses', 'curses']
         for pkg in pkgconfig_files:
             candidates.append(functools.partial(PkgConfigDependency, pkg, env, kwargs))
 
+    # There are path handling problems with these methods on msys, and they
+    # don't apply to windows otherwise (cygwin is handled seperately from
+    # windows)
+    if not env.machines[for_machine].is_windows():
+        if DependencyMethods.CONFIG_TOOL in methods:
+            candidates.append(functools.partial(CursesConfigToolDependency, 'curses', env, kwargs))
+
+        if DependencyMethods.SYSTEM in methods:
+            candidates.append(functools.partial(CursesSystemDependency, 'curses', env, kwargs))
+
     return candidates
 
 


Test patch:
diff --git a/test cases/frameworks/31 curses/meson.build b/test cases/frameworks/31 curses/meson.build
--- a/test cases/frameworks/31 curses/meson.build	
+++ b/test cases/frameworks/31 curses/meson.build	
@@ -1,9 +1,13 @@
 project('curses', 'c')
 
-curses = dependency('curses', required: false)
+curses = dependency('curses', required: false, method : get_option('method'), version : '>= 0')
 if not curses.found()
   error('MESON_SKIP_TEST: Curses library not found')
 endif
 
 exec = executable('basic', 'main.c', dependencies: curses)
 # didn't run the test because in general graphics fail on CI
+
+# this should fail
+not_found = dependency('curses', required: false, method : get_option('method'), version : '> 1000000')
+assert(not_found.found() == false)
diff --git a/test cases/frameworks/31 curses/meson_options.txt b/test cases/frameworks/31 curses/meson_options.txt
new file mode 100644
--- /dev/null
+++ b/test cases/frameworks/31 curses/meson_options.txt	
@@ -0,0 +1,6 @@
+option(
+    'method',
+    type : 'combo',
+    choices : ['pkg-config', 'config-tool', 'system'],
+    value : 'pkg-config',
+)
diff --git a/test cases/frameworks/31 curses/test.json b/test cases/frameworks/31 curses/test.json
new file mode 100644
--- /dev/null
+++ b/test cases/frameworks/31 curses/test.json	
@@ -0,0 +1,11 @@
+{
+  "matrix": {
+    "options": {
+      "method": [
+        { "val": "pkg-config" },
+        { "val": "config-tool" },
+        { "val": "system" }
+      ]
+    }
+  }
+}