Repository: mesonbuild/meson

Issue:
Add a way to disable subproject using the new `feature` options
In `gst-build` in particular it would be very useful to be able to pass a `feature` option to `subproject()` so that it is taken into account and failure in `auto` case is ignored.

Also in the case where we have a disabled `subproject('subproject')` then if another subproject tries to `dependency('subproject', fallback: ['subproject', 'dep'])` it sould fail (or if `required: auto` it should just ignore the dep).


Discussion:
Currently, `subproject()` calls are unconditional, and can't fail. In the case of `dependency()`, we return a not-found dependency, but I'm not sure how that would work for `subproject()`.
> Currently, subproject() calls are unconditional, and can't fail. In the case of dependency(), we return a not-found dependency, but I'm not sure how that would work for subproject().

I know and that is what this issue is about, allowing to have conditionnal subproject, not failing if a dependency is not found and the subproject is "auto" fail if it is "enabled" and do not run the subproject at all if it is "disabled".
That would be neat, yes.
The main issue is that the usage of `dependency()` is very straightforward, but with `subproject()` you run `get_variable()` to fetch arbitrary variables that we don't know the type of. For instance, it can be a list of sources.

The current workaround I've been using everywhere is: `dependency('', fallback ['foo', 'foo_dep'], required: false)`. Maybe that can work for gst-build?
> The main issue is that the usage of dependency() is very straightforward, but with subproject() you run get_variable() to fetch arbitrary variables that we don't know the type of. For instance, it can be a list of sources.

I am not sure I understand what you mean,  we are already able to handle that for dependency fallbacks, how is it different for `subproject()`? Basically a `subproject` object that was not built will not be usable and we need to add a method to check its status, like `.found()` for dependencies.

> The current workaround I've been using everywhere is: dependency('', fallback ['foo', 'foo_dep'], required: false). Maybe that can work for gst-build?

Yeah in some cases I could, not in the cases where we do not `declare_dependency` in the subproject though...
> Basically a subproject object that was not built will not be usable and we need to add a method to check its status, like .found() for dependencies.

Hmm, I see, we *can* add `subproject(required: false)` and then you need to check the return value before trying to use it. We can't change the `required: true` behaviour though, because that breaks existing build files.
> Basically a subproject object that was not built will not be usable and we
> need to add a method to check its status, like .found() for dependencies.
>
> Hmm, I see, we *can* add subproject(required: false) and then you need to
> check the return value before trying to use it. We can't change the required:
> true behaviour though, because that breaks existing build files.
>

There is now `required` arg in `subproject` yet is there?

> There is now `required` arg in `subproject` yet is there?

No, but the default will continue to be `required: true`, unless we want to treat that differently than not-specified.
Sure, yes, default has to be `required: true` for backward compat.

Resolving patch:
diff --git a/mesonbuild/interpreter.py b/mesonbuild/interpreter.py
--- a/mesonbuild/interpreter.py
+++ b/mesonbuild/interpreter.py
@@ -383,6 +383,9 @@ def __init__(self, dep, pv):
                              'partial_dependency': self.partial_dependency_method,
                              })
 
+    def found(self):
+        return self.found_method([], {})
+
     @noPosargs
     @permittedKwargs({})
     def type_name_method(self, args, kwargs):
@@ -916,16 +919,30 @@ def get_name(self):
 
 class SubprojectHolder(InterpreterObject, ObjectHolder):
 
-    def __init__(self, subinterpreter):
+    def __init__(self, subinterpreter, subproject_dir, name):
         InterpreterObject.__init__(self)
         ObjectHolder.__init__(self, subinterpreter)
+        self.name = name
+        self.subproject_dir = subproject_dir
         self.methods.update({'get_variable': self.get_variable_method,
+                             'found': self.found_method,
                              })
 
+    @noPosargs
+    @permittedKwargs({})
+    def found_method(self, args, kwargs):
+        return self.found()
+
+    def found(self):
+        return self.held_object is not None
+
     @permittedKwargs({})
     def get_variable_method(self, args, kwargs):
         if len(args) != 1:
             raise InterpreterException('Get_variable takes one argument.')
+        if not self.found():
+            raise InterpreterException('Subproject "%s/%s" disabled can\'t get_variable on it.' % (
+                self.subproject_dir, self.name))
         varname = args[0]
         if not isinstance(varname, str):
             raise InterpreterException('Get_variable takes a string argument.')
@@ -1838,7 +1855,7 @@ def get_cross_property_method(self, args, kwargs):
                     'both_libraries': known_library_kwargs,
                     'library': known_library_kwargs,
                     'subdir': {'if_found'},
-                    'subproject': {'version', 'default_options'},
+                    'subproject': {'version', 'default_options', 'required'},
                     'test': {'args', 'depends', 'env', 'is_parallel', 'should_fail', 'timeout', 'workdir', 'suite'},
                     'vcs_tag': {'input', 'output', 'fallback', 'command', 'replace_string'},
                     }
@@ -2219,7 +2236,16 @@ def func_subproject(self, nodes, args, kwargs):
         dirname = args[0]
         return self.do_subproject(dirname, kwargs)
 
+    def disabled_subproject(self, dirname):
+        self.subprojects[dirname] = SubprojectHolder(None, self.subproject_dir, dirname)
+        return self.subprojects[dirname]
+
     def do_subproject(self, dirname, kwargs):
+        disabled, required, feature = extract_required_kwarg(kwargs, self.subproject)
+        if disabled:
+            mlog.log('\nSubproject', mlog.bold(dirname), ':', 'skipped: feature', mlog.bold(feature), 'disabled')
+            return self.disabled_subproject(dirname)
+
         default_options = mesonlib.stringlistify(kwargs.get('default_options', []))
         default_options = coredata.create_options_dict(default_options)
         if dirname == '':
@@ -2237,7 +2263,13 @@ def do_subproject(self, dirname, kwargs):
             incpath = ' => '.join(fullstack)
             raise InvalidCode('Recursive include of subprojects: %s.' % incpath)
         if dirname in self.subprojects:
-            return self.subprojects[dirname]
+            subproject = self.subprojects[dirname]
+
+            if required and not subproject.found():
+                raise InterpreterException('Subproject "%s/%s" required but not found.' % (
+                                           self.subproject_dir, dirname))
+
+            return subproject
         subproject_dir_abs = os.path.join(self.environment.get_source_dir(), self.subproject_dir)
         r = wrap.Resolver(subproject_dir_abs, self.coredata.wrap_mode)
         try:
@@ -2249,22 +2281,35 @@ def do_subproject(self, dirname, kwargs):
             # promotion...
             self.print_nested_info(dirname)
 
-            msg = 'Subproject directory {!r} does not exist and cannot be downloaded:\n{}'
-            raise InterpreterException(msg.format(os.path.join(self.subproject_dir, dirname), e))
+            if required:
+                msg = 'Subproject directory {!r} does not exist and cannot be downloaded:\n{}'
+                raise InterpreterException(msg.format(os.path.join(self.subproject_dir, dirname), e))
+
+            mlog.log('\nSubproject ', mlog.bold(dirname), 'is buildable:', mlog.red('NO'), '(disabling)\n')
+            return self.disabled_subproject(dirname)
+
         subdir = os.path.join(self.subproject_dir, resolved)
         os.makedirs(os.path.join(self.build.environment.get_build_dir(), subdir), exist_ok=True)
         self.global_args_frozen = True
         mlog.log()
         with mlog.nested():
-            mlog.log('\nExecuting subproject', mlog.bold(dirname), '\n')
-            subi = Interpreter(self.build, self.backend, dirname, subdir, self.subproject_dir,
-                               self.modules, default_options)
-            subi.subprojects = self.subprojects
-
-            subi.subproject_stack = self.subproject_stack + [dirname]
-            current_active = self.active_projectname
-            subi.run()
-            mlog.log('\nSubproject', mlog.bold(dirname), 'finished.')
+            try:
+                mlog.log('\nExecuting subproject', mlog.bold(dirname), '\n')
+                subi = Interpreter(self.build, self.backend, dirname, subdir, self.subproject_dir,
+                                   self.modules, default_options)
+                subi.subprojects = self.subprojects
+
+                subi.subproject_stack = self.subproject_stack + [dirname]
+                current_active = self.active_projectname
+                subi.run()
+                mlog.log('\nSubproject', mlog.bold(dirname), 'finished.')
+            except Exception as e:
+                if not required:
+                    mlog.log(e)
+                    mlog.log('\nSubproject', mlog.bold(dirname), 'is buildable:', mlog.red('NO'), '(disabling)')
+                    return self.disabled_subproject(dirname)
+                else:
+                    raise e
 
         if 'version' in kwargs:
             pv = subi.project_version
@@ -2274,7 +2319,7 @@ def do_subproject(self, dirname, kwargs):
         self.active_projectname = current_active
         self.build.subprojects[dirname] = subi.project_version
         self.subprojects.update(subi.subprojects)
-        self.subprojects[dirname] = SubprojectHolder(subi)
+        self.subprojects[dirname] = SubprojectHolder(subi, self.subproject_dir, dirname)
         self.build_def_files += subi.build_def_files
         return self.subprojects[dirname]
 
@@ -2805,6 +2850,13 @@ def check_subproject_version(wanted, found):
 
     def get_subproject_dep(self, name, dirname, varname, required):
         try:
+            subproject = self.subprojects[dirname]
+            if not subproject.found():
+                if not required:
+                    return DependencyHolder(NotFoundDependency(self.environment), self.subproject)
+
+                raise DependencyException('Subproject %s was not found.' % (name))
+
             dep = self.subprojects[dirname].get_variable_method([varname], {})
         except InvalidArguments as e:
             if required:
@@ -2826,6 +2878,9 @@ def _find_cached_fallback_dep(self, name, dirname, varname, wanted, required):
         dep = self.get_subproject_dep(name, dirname, varname, required)
         if not dep:
             return False
+        if not dep.found():
+            return dep
+
         found = dep.version_method([], {})
         # Don't do a version check if the dependency is not found and not required
         if found == 'none' and not required:
diff --git a/run_unittests.py b/run_unittests.py
--- a/run_unittests.py
+++ b/run_unittests.py
@@ -2771,6 +2771,16 @@ def test_vcs_tag_featurenew_build_always_stale(self):
         msg = '.*WARNING:.*feature.*build_always_stale.*custom_target.*'
         self.assertMesonDoesNotOutput(vcs_tag, msg, meson_version='>=0.43')
 
+    def test_missing_subproject_not_required_and_required(self):
+        self.assertMesonRaises("sub1 = subproject('not-found-subproject', required: false)\n" +
+                               "sub2 = subproject('not-found-subproject', required: true)",
+                               """.*Subproject "subprojects/not-found-subproject" required but not found.*""")
+
+    def test_get_variable_on_not_found_project(self):
+        self.assertMesonRaises("sub1 = subproject('not-found-subproject', required: false)\n" +
+                               "sub1.get_variable('naaa')",
+                               """Subproject "subprojects/not-found-subproject" disabled can't get_variable on it.""")
+
 
 class WindowsTests(BasePlatformTests):
     '''


Test patch:
diff --git a/test cases/common/206 subproject with features/meson.build b/test cases/common/206 subproject with features/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/meson.build	
@@ -0,0 +1,17 @@
+project('proj', 'c')
+
+auto_subproj = subproject('sub', required: get_option('use-subproject'))
+assert(auto_subproj.found(), 'Subproject should always be buildable and thus found')
+
+auto_dep = dependency('', fallback: ['sub', 'libSub'], required: true)
+assert(auto_dep.found() == true, 'Subproject is required and foundable, dependency should be found.')
+
+disabled_subproj = subproject('disabled_sub', required: get_option('disabled-subproject'))
+assert(disabled_subproj.found() == false, 'Disabled subproject should be NOT found')
+
+disabled_dep = dependency('', fallback: ['disabled_sub', 'libSub'], required: false)
+assert(disabled_dep.found() == false, 'Subprojetc was disabled, it should never be built.')
+nothing = executable('nothing', 'nothing.c', dependencies: [disabled_dep])
+
+subproj_with_missing_dep = subproject('auto_sub_with_missing_dep', required: get_option('auto-sub-with-missing-dep'))
+assert(subproj_with_missing_dep.found() == false, 'Subproject with required=auto and missing dependency should be NOT found')
diff --git a/test cases/common/206 subproject with features/meson_options.txt b/test cases/common/206 subproject with features/meson_options.txt
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/meson_options.txt	
@@ -0,0 +1,3 @@
+option('use-subproject', type : 'feature', value : 'auto')
+option('disabled-subproject', type : 'feature', value : 'disabled')
+option('auto-sub-with-missing-dep', type : 'feature', value : 'auto')
diff --git a/test cases/common/206 subproject with features/nothing.c b/test cases/common/206 subproject with features/nothing.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/nothing.c	
@@ -0,0 +1,4 @@
+int main(int argc, char const *argv[])
+{
+    return 0;
+}
\ No newline at end of file
diff --git a/test cases/common/206 subproject with features/subprojects/auto_sub_with_missing_dep/meson.build b/test cases/common/206 subproject with features/subprojects/auto_sub_with_missing_dep/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/auto_sub_with_missing_dep/meson.build	
@@ -0,0 +1,3 @@
+project('sub', 'c')
+
+dependency('no_way_this_exists', required: true)
\ No newline at end of file
diff --git a/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/meson.build b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/meson.build	
@@ -0,0 +1,3 @@
+lib = static_library('sub', 'sub.c')
+
+libSub = declare_dependency(include_directories: include_directories('.'), link_with: lib)
\ No newline at end of file
diff --git a/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.c b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.c	
@@ -0,0 +1,5 @@
+#include "sub.h"
+
+int sub() {
+    return 0;
+}
diff --git a/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.h b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.h
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/disabled_sub/lib/sub.h	
@@ -0,0 +1,6 @@
+#ifndef SUB_H
+#define SUB_H
+
+int sub();
+
+#endif
diff --git a/test cases/common/206 subproject with features/subprojects/disabled_sub/meson.build b/test cases/common/206 subproject with features/subprojects/disabled_sub/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/disabled_sub/meson.build	
@@ -0,0 +1,3 @@
+project('disabled_sub', 'c')
+
+subdir('lib')
\ No newline at end of file
diff --git a/test cases/common/206 subproject with features/subprojects/sub/lib/meson.build b/test cases/common/206 subproject with features/subprojects/sub/lib/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/sub/lib/meson.build	
@@ -0,0 +1,2 @@
+lib = static_library('sub', 'sub.c')
+libSub = declare_dependency(include_directories: include_directories('.'), link_with: lib)
diff --git a/test cases/common/206 subproject with features/subprojects/sub/lib/sub.c b/test cases/common/206 subproject with features/subprojects/sub/lib/sub.c
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/sub/lib/sub.c	
@@ -0,0 +1,5 @@
+#include "sub.h"
+
+int sub() {
+  return 0;
+}
diff --git a/test cases/common/206 subproject with features/subprojects/sub/lib/sub.h b/test cases/common/206 subproject with features/subprojects/sub/lib/sub.h
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/sub/lib/sub.h	
@@ -0,0 +1,6 @@
+#ifndef SUB_H
+#define SUB_H
+
+int sub();
+
+#endif
diff --git a/test cases/common/206 subproject with features/subprojects/sub/meson.build b/test cases/common/206 subproject with features/subprojects/sub/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/common/206 subproject with features/subprojects/sub/meson.build	
@@ -0,0 +1,3 @@
+project('sub', 'c')
+
+subdir('lib')
\ No newline at end of file