Repository: mesonbuild/meson

Issue:
meson subprojects command does not consider subproject_dir option
**Describe the bug**
When a project configures an alternative directory for subprojects `project( ... , subproject_dir: 'lib', ...)`, `meson subprojects ...` fails with `Directory . does not seem to have subprojects`

Looking at the source, looks like the subproject dir is hard-coded in:
https://github.com/mesonbuild/meson/blob/8369dbbfecafa87629f0624e6dc7c9cd235043a4/mesonbuild/msubprojects.py#L692-L696

This looks like a fairly easy fix (keyword looks, I'm not familiar with the code), I should be able to tackle it if the maintainers wish/need, let me know

I'll note that I realize `subproject_dir` is supplied as a compatibility option, and ideally the projects build-system would be "fixed" to use the recommended directory, but that's not always ideal, and I think this is undesired behavior and warrants a fix

**To Reproduce**
A very simple project with a [any] subproject, and alternate `subproject_dir` should be enough
```
project/
├── meson.build
└── lib/
    └── libusb.wrap
```

`meson.build`
```
project('project', subproject_dir: 'lib')

libusb = subproject('libusb')
```

`libusb.wrap`
```
[wrap-git]
url = https://github.com/dragonCodecs/libusb
revision = blackmagic/meson
clone-recursive = false
```

Running the following commands
```bash
cd project/
meson subprojects download
```

**Expected behavior**
`meson subprojects ...` should either parse the `subproject_dir` from the source meson.build, or more simply, support an argument for an alternative directory like `--subproject-dir='lib'`, and behave exactly like it does for the default `subprojects`

i.e. when I rename `lib` to `subprojects`
``` bash
$ meson subprojects download

Cloning into 'libusb'...
remote: Enumerating objects: 16939, done.
remote: Counting objects: 100% (3107/3107), done.
remote: Compressing objects: 100% (434/434), done.
remote: Total 16939 (delta 2714), reused 2707 (delta 2670), pack-reused 13832
Receiving objects: 100% (16939/16939), 5.11 MiB | 1.28 MiB/s, done.
Resolving deltas: 100% (12218/12218), done.
branch 'blackmagic/meson' set up to track 'origin/blackmagic/meson'.
Switched to a new branch 'blackmagic/meson'
Download libusb...
  -> done
```

**system parameters**
* meson version `1.1.1`

I don't think anything matters here other than the meson version but for the sake of completeness
* operating system `Arch Linux`
* python version `Python 3.11.3`
* ninja version `1.11.1`



Discussion:
> This looks like a fairly easy fix

Quote of the year ahah :D

More seriously, the difficulty is `meson subprojects` works without a builddir which means it does not have the `subproject_dir` value without parsing `meson.build` file, at least the first `project()` function call. Nothing impossible of course, PR welcome :)
Yep I realize that, which is why I mentioned, and is probably more that enough for the purpose while keeping it simple, the use of a argument/flag, this is probably the extent of what I can contribute too, someone more familiar with the code might see a path to "properly" fixing this by parsing the `meson.build` :)
`meson configure` parses the project() call. I would look through the code in that path
I think `meson configure` does way too much, it will parse and interpreter the whole project.

Some pointers if you want to work on this:
- See `InterpreterBase.load_root_meson_file()` to parse the root `meson.build` file into an AST.
- See  `Interpreter.handle_meson_version_from_ast()` to extract a kwarg from the top `project()` function.
At the core, this is just running an "introspection" interpreter, which is like running a real `meson setup` except that all functions are stubbed out to do no real work, and depending on function it will generate some lightweight metadata.

As noted:
> I think `meson configure` does way too much, it will parse and interpreter the whole project.

For `configure` / `introspect` / `rewrite`, we do need that to an extent as we need to check subprojects too, or generally print/manipulate a full AST. For `meson subprojects download`, definitely not. :D 

The implementation for the AST interpreter is simple enough:
```python
    def analyze(self) -> None:
        self.load_root_meson_file()
        self.sanity_check_ast()
        self.parse_project()
        self.run()
```

Parsing the project should be enough here. We do not need to do the kind of hacks we do in handle_meson_version_from_ast, which exists solely to ensure we can have the `project()` function itself generate version-based warnings and must therefore know the meson_version before running project(), a chicken-and-egg problem.

Resolving patch:
diff --git a/mesonbuild/msubprojects.py b/mesonbuild/msubprojects.py
--- a/mesonbuild/msubprojects.py
+++ b/mesonbuild/msubprojects.py
@@ -14,6 +14,7 @@
 import zipfile
 
 from . import mlog
+from .ast import IntrospectionInterpreter, AstIDGenerator
 from .mesonlib import quiet_git, GitException, Popen_safe, MesonException, windows_proof_rmtree
 from .wrap.wrap import (Resolver, WrapException, ALL_TYPES, PackageDefinition,
                         parse_patch_url, update_wrap_file, get_releases)
@@ -685,15 +686,20 @@ def add_arguments(parser: argparse.ArgumentParser) -> None:
     p.set_defaults(subprojects_func=Runner.packagefiles)
 
 def run(options: 'Arguments') -> int:
-    src_dir = os.path.relpath(os.path.realpath(options.sourcedir))
-    if not os.path.isfile(os.path.join(src_dir, 'meson.build')):
-        mlog.error('Directory', mlog.bold(src_dir), 'does not seem to be a Meson source directory.')
+    source_dir = os.path.relpath(os.path.realpath(options.sourcedir))
+    if not os.path.isfile(os.path.join(source_dir, 'meson.build')):
+        mlog.error('Directory', mlog.bold(source_dir), 'does not seem to be a Meson source directory.')
         return 1
-    subprojects_dir = os.path.join(src_dir, 'subprojects')
-    if not os.path.isdir(subprojects_dir):
-        mlog.log('Directory', mlog.bold(src_dir), 'does not seem to have subprojects.')
+    with mlog.no_logging():
+        intr = IntrospectionInterpreter(source_dir, '', 'none', visitors = [AstIDGenerator()])
+        intr.load_root_meson_file()
+        intr.sanity_check_ast()
+        intr.parse_project()
+    subproject_dir = intr.subproject_dir
+    if not os.path.isdir(os.path.join(source_dir, subproject_dir)):
+        mlog.log('Directory', mlog.bold(source_dir), 'does not seem to have subprojects.')
         return 0
-    r = Resolver(src_dir, 'subprojects', wrap_frontend=True, allow_insecure=options.allow_insecure, silent=True)
+    r = Resolver(source_dir, subproject_dir, wrap_frontend=True, allow_insecure=options.allow_insecure, silent=True)
     if options.subprojects:
         wraps = [wrap for name, wrap in r.wraps.items() if name in options.subprojects]
     else:
@@ -714,7 +720,7 @@ def run(options: 'Arguments') -> int:
         pre_func(options)
     logger = Logger(len(wraps))
     for wrap in wraps:
-        dirname = Path(subprojects_dir, wrap.directory).as_posix()
+        dirname = Path(subproject_dir, wrap.directory).as_posix()
         runner = Runner(logger, r, wrap, dirname, options)
         task = loop.run_in_executor(executor, runner.run)
         tasks.append(task)


Test patch: