Repository: mesonbuild/meson

Issue:
Fortran 2008 submodule
Fortran 2008 added `submodule`, which is quite useful for good structure and fast builds of Fortran programs. The present lack of `submodule` support in Meson is a key blocker for Fortran use.

Meson does not appear to understand the `submodule` statement. A self-contained single `.f90` file will of course compile correctly, but almost all use cases for `submodule` are where the submodule is in a different `.f90` file from the `module` containing the `submodule`.

Here is a simple example consisting of two files.

file `basic.f90`

```fortran
module demo
real, parameter :: pi = 4.*atan(1.)
real :: tau

interface
  module subroutine hello(pi,tau)
    real, intent(in) :: pi
    real, intent(out) :: tau
  end subroutine hello
end interface
contains 
end module demo

program sm
use demo
call hello(pi, tau)
print *,'pi=',pi, 'tau=', tau
end program
```

file `basic_sub.f90`
```fortran
submodule (demo) hi
contains
module procedure hello
  tau = 2*pi
end procedure hello
end submodule hi
```

file `meson.build`
```meson
executable('basic', ['basic.f90', 'basic_sub.f90'])
```


Discussion:
log output for the above example.  Changing the order of the .f90 files in the `executable()` statement does not change the error/output (the order of files shouldn't matter anyway)
```
The Meson build system
Version: 0.49.0
Source dir: ~/code/fortran2018-examples
Build dir: ~/code/fortran2018-examples/bin
Build type: native build
Project name: Fortran2018examples
Project version: 1.0.0
Native Fortran compiler: gfortran (gcc 7.3.0 "GNU Fortran (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Build targets in project: 7
Found ninja-1.8.2 at ~/miniconda3/bin/ninja
[1/3] Compiling Fortran object 'submodule/d8566c5@@basic@exe/basic_sub.f90.o'.
FAILED: submodule/d8566c5@@basic@exe/basic_sub.f90.o 
gfortran -Isubmodule/d8566c5@@basic@exe -Isubmodule -I../submodule -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -g  -Jsubmodule/d8566c5@@basic@exe   -o 'submodule/d8566c5@@basic@exe/basic_sub.f90.o' -c ../submodule/basic_sub.f90
f951: Fatal Error: Module file ‘demo.smod’ has not been generated, either because the module does not contain a MODULE PROCEDURE or there is an error in the module.
compilation terminated.
[2/3] Compiling Fortran object 'submodule/d8566c5@@basic@exe/basic.f90.o'.
ninja: build stopped: subcommand failed.
```
Here is the verbose output of CMake building this executable correctly:
```
[  8%] Building Fortran object CMakeFiles/basic.dir/basic.f90.o
/usr/bin/gfortran   -c ~/code/fortran2018-examples/submodule/basic.f90 -o CMakeFiles/basic.dir/basic.f90.o
[ 16%] Building Fortran object CMakeFiles/basic.dir/basic_sub.f90.o
/usr/bin/gfortran    -c ~/code/fortran2018-examples/submodule/basic_sub.f90 -o CMakeFiles/basic.dir/basic_sub.f90.o
[ 25%] Linking Fortran executable basic
/usr/bin/gfortran   CMakeFiles/basic.dir/basic.f90.o CMakeFiles/basic.dir/basic_sub.f90.o  -o basic 
[ 25%] Built target basic
```

Here is how one can build the executable manually. The compiler commands must be issued in this order--the module must be compiled before the submodules.

1. `gfortran -c basic.90`  this creates files: `basic.o  demo.mod  demo.smod`. 
2. `gfortran -c basic_sub.f90 -o basic_sub.o`  this creates files: `basic_sub.o demo@hi.smod`.
3. `gfortran basic.o basic_sub.o -o basic`  creating executable `basic`

For simplicity, here is a table of the module and submodule files generated by several popular Fortran compilers:

Compiler   | module files       | submodule files
-----------|--------------------|----------------
`gfortran` | demo.mod demo.smod | demo@hi.smod
`flang`    | demo.mod           | demo-hi.mod
`pgf90`    | demo.mod           | demo-hi.mod
`ifort`    | demo.mod           | demo@hi.smod
`xlf2008`  | demo.mod           | demo_hi.smod
How do modules and submodules behave with respect to each other? Can there be multiple modules and/or submodules in a single file?

Also your post above the table says the output file name is `demod.mod` rather than `demo.mod`. Which one of those is correct or does it depend?
it's `demo.mod`, I made a copy-paste error.
Fortran `module` can use an arbitrary number of `submodule`. The `interface` are defined in the `module`, and the `submodule` point back to the `module` they're used in. In this way, changing the code inside a submodule only requires rebuilding the submodule, since the interface (variables in/out) remains the same.

Yes there can be multiple modules and submodules in a single file, though more often for clarity they are put in separate files.

This gives an excellent description of [Fortran submodule hierarchy](https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-submodule)

The times when you need to use `submodule` are when the code is complex, hence one looks at the examples and thinks "why is this needed".  In the HPC world, say I have a complex program with `io.f90` as the "ancestor" module.  The children are `io_hdf5.f90` `io_netcdf.f90`, where there are common procedures in `io.f90` (say, gathering MPI variables) and then a user selects HDF5 or NetCDF4 at compile time via Meson or CMake command line options.

I put some example code with `meson.build` and `CMakeLists.txt` in https://github.com/scivision/fortran-submodule

Resolving patch:
diff --git a/mesonbuild/backend/ninjabackend.py b/mesonbuild/backend/ninjabackend.py
--- a/mesonbuild/backend/ninjabackend.py
+++ b/mesonbuild/backend/ninjabackend.py
@@ -11,7 +11,7 @@
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
-
+from typing import List
 import os
 import re
 import shlex
@@ -29,7 +29,7 @@
 from .. import mlog
 from .. import dependencies
 from .. import compilers
-from ..compilers import CompilerArgs, CCompiler, VisualStudioCCompiler
+from ..compilers import CompilerArgs, CCompiler, VisualStudioCCompiler, FortranCompiler
 from ..linkers import ArLinker
 from ..mesonlib import File, MachineChoice, MesonException, OrderedSet
 from ..mesonlib import get_compiler_for_source, has_path_sep
@@ -1826,7 +1826,8 @@ def scan_fortran_module_outputs(self, target):
         if compiler is None:
             self.fortran_deps[target.get_basename()] = {}
             return
-        modre = re.compile(r"\bmodule\s+(\w+)\s*$", re.IGNORECASE)
+        modre = re.compile(r"\s*\bmodule\b\s+(\w+)\s*$", re.IGNORECASE)
+        submodre = re.compile(r"\s*\bsubmodule\b\s+\((\w+:?\w+)\)\s+(\w+)\s*$", re.IGNORECASE)
         module_files = {}
         for s in target.get_sources():
             # FIXME, does not work for Fortran sources generated by
@@ -1849,11 +1850,23 @@ def scan_fortran_module_outputs(self, target):
                                 'two files %s and %s.' %
                                 (modname, module_files[modname], s))
                         module_files[modname] = s
+                    else:
+                        submodmatch = submodre.match(line)
+                        if submodmatch is not None:
+                            submodname = submodmatch.group(2).lower()
+                            if submodname in module_files:
+                                raise InvalidArguments(
+                                    'Namespace collision: submodule %s defined in '
+                                    'two files %s and %s.' %
+                                    (submodname, module_files[submodname], s))
+                            module_files[submodname] = s
+
         self.fortran_deps[target.get_basename()] = module_files
 
-    def get_fortran_deps(self, compiler, src, target):
+    def get_fortran_deps(self, compiler: FortranCompiler, src: str, target) -> List[str]:
         mod_files = []
         usere = re.compile(r"\s*use\s+(\w+)", re.IGNORECASE)
+        submodre = re.compile(r"\s*\bsubmodule\b\s+\((\w+:?\w+)\)\s+(\w+)\s*$", re.IGNORECASE)
         dirname = self.get_target_private_dir(target)
         tdeps = self.fortran_deps[target.get_basename()]
         with open(src) as f:
@@ -1879,9 +1892,23 @@ def get_fortran_deps(self, compiler, src, target):
                     # the same name.
                     if mod_source_file.fname == os.path.basename(src):
                         continue
-                    mod_name = compiler.module_name_to_filename(
-                        usematch.group(1))
+                    mod_name = compiler.module_name_to_filename(usename)
                     mod_files.append(os.path.join(dirname, mod_name))
+                else:
+                    submodmatch = submodre.match(line)
+                    if submodmatch is not None:
+                        parents = submodmatch.group(1).lower().split(':')
+                        assert len(parents) in (1, 2), (
+                            'submodule ancestry must be specified as'
+                            ' ancestor:parent but Meson found {}'.parents)
+                        for parent in parents:
+                            if parent not in tdeps:
+                                raise MesonException("submodule {} relies on parent module {} that was not found.".format(submodmatch.group(2).lower(), parent))
+                            if tdeps[parent].fname == os.path.basename(src):  # same file
+                                continue
+                            mod_name = compiler.module_name_to_filename(parent)
+                            mod_files.append(os.path.join(dirname, mod_name))
+
         return mod_files
 
     def get_cross_stdlib_args(self, target, compiler):
diff --git a/mesonbuild/compilers/fortran.py b/mesonbuild/compilers/fortran.py
--- a/mesonbuild/compilers/fortran.py
+++ b/mesonbuild/compilers/fortran.py
@@ -179,7 +179,7 @@ def compute_parameters_with_absolute_paths(self, parameter_list, build_dir):
 
         return parameter_list
 
-    def module_name_to_filename(self, module_name):
+    def module_name_to_filename(self, module_name: str) -> str:
         return module_name.lower() + '.mod'
 
     def get_std_shared_lib_link_args(self):


Test patch:
diff --git a/test cases/fortran/12 submodule/a1.f90 b/test cases/fortran/12 submodule/a1.f90
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/a1.f90	
@@ -0,0 +1,25 @@
+module a1
+implicit none
+
+interface
+module elemental real function pi2tau(pi)
+  real, intent(in) :: pi
+end function pi2tau
+
+module real function get_pi()
+end function get_pi
+end interface
+
+end module a1
+
+program hierN
+
+use a1
+
+pi = get_pi()
+
+tau = pi2tau(pi)
+
+print *,'pi=',pi,'tau=',tau
+
+end program
diff --git a/test cases/fortran/12 submodule/a2.f90 b/test cases/fortran/12 submodule/a2.f90
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/a2.f90	
@@ -0,0 +1,10 @@
+submodule (a1) a2
+
+contains
+
+module procedure pi2tau
+  pi2tau = 2*pi
+end procedure pi2tau
+
+
+end submodule a2
diff --git a/test cases/fortran/12 submodule/a3.f90 b/test cases/fortran/12 submodule/a3.f90
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/a3.f90	
@@ -0,0 +1,10 @@
+submodule (a1:a2) a3
+
+contains
+
+module procedure get_pi
+  get_pi = 4.*atan(1.)
+end procedure get_pi
+
+
+end submodule a3
diff --git a/test cases/fortran/12 submodule/child.f90 b/test cases/fortran/12 submodule/child.f90
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/child.f90	
@@ -0,0 +1,10 @@
+submodule (mother) daughter
+
+contains
+
+module procedure pi2tau
+  pi2tau = 2*pi
+end procedure pi2tau
+
+end submodule daughter
+
diff --git a/test cases/fortran/12 submodule/meson.build b/test cases/fortran/12 submodule/meson.build
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/meson.build	
@@ -0,0 +1,7 @@
+project('submodule single level', 'fortran')
+
+hier2 = executable('single', 'parent.f90','child.f90')
+test('single-level hierarchy', hier2)
+
+hierN = executable('multi', 'a1.f90', 'a2.f90', 'a3.f90')
+test('multi-level hierarchy', hierN)
diff --git a/test cases/fortran/12 submodule/parent.f90 b/test cases/fortran/12 submodule/parent.f90
new file mode 100644
--- /dev/null
+++ b/test cases/fortran/12 submodule/parent.f90	
@@ -0,0 +1,23 @@
+module mother
+real, parameter :: pi = 4.*atan(1.)
+real :: tau
+
+interface
+module elemental real function pi2tau(pi)
+  real, intent(in) :: pi
+end function pi2tau
+end interface
+
+contains 
+
+end module mother
+
+
+program hier1
+use mother
+
+tau = pi2tau(pi)
+
+print *,'pi=',pi, 'tau=', tau
+
+end program