Repository: mesonbuild/meson

Issue:
issue with i18n.merge_file
I have the following rule:
```
i18n.merge_file(
  'org.freedesktop.systemd1.policy',
  po_dir : po_dir,
  input : policy_in,
  output : 'org.freedesktop.systemd1.policy',
  install : true,
  install_dir : polkitpolicydir)
```

This fails like this:
```
FAILED: src/core/org.freedesktop.systemd1.policy 
'msgfmt' '--xml' '--template' 'src/core/org.freedesktop.systemd1.policy.in' '-d' '/home/zbyszek/src/systemd/po' '-o' 'src/core/org.freedesktop.systemd1.policy'
msgfmt: cannot locate ITS rules for src/core/org.freedesktop.systemd1.policy.in
```
I have no idea what "ITS rules" are ;(

Doing it "manually" works:
```
policy_in = configure_file(
    input : 'org.freedesktop.systemd1.policy.in.in',
    output : 'org.freedesktop.systemd1.policy.in',
    configuration : substs)

custom_target(
   'org.freedesktop.systemd1.policy',
  input : policy_in,
  output : 'org.freedesktop.systemd1.policy',
  command : intltool_command,
  install : true,
  install_dir : polkitpolicydir)
```
I saw similar message in https://github.com/mesonbuild/meson/issues/1441#issuecomment-283585493, it might be related...


Discussion:
In order for gettext to know what items to translate in an XML file it needs an its file to figure that out. intltool was a custom tool on top of gettext that did this in another method (prefixing with _). So the solution to this is writing an its file, they are very simple and you can see existing ones in `/usr/share/gettext/its` and http://itstool.org. They can be used locally by passing the `data_dirs` kwarg.
Thanks for the reply. I cannot get it to work though. I copied the polkit.its and polkit.loc files from https://cgit.freedesktop.org/polkit/commit/?id=c78819245ff8a270f97c9f800773e727918be838 and installed them into /usr/share/gettext/its. This seems to have no effect — I get the same failure as before. The command that is being run is:
```
FAILED: src/core/org.freedesktop.systemd1.policy 
'msgfmt' '--xml' '--template' 'src/core/org.freedesktop.systemd1.policy.in' '-d' '/home/zbyszek/src/systemd/po' '-o' 'src/core/org.freedesktop.systemd1.policy'
msgfmt: cannot locate ITS rules for src/core/org.freedesktop.systemd1.policy.in
```

I tried changing `pattern="*.policy"` to `pattern="*.policy.in"`, but that has no effect.
Dunno, I'm probably doing something stupid, some help would be appreciated.

I tried also adding `data_dirs` to my `i18n.merge_file` rule, but that results in a warning about unknown arg. Where is this supposed to go?
> I tried also adding data_dirs to my i18n.merge_file rule, but that results in a warning about unknown arg. Where is this supposed to go?

That might be something overlooked but `i18n.gettext()` accepts this to extract translations, might have to add it to `merge_file()` too.

> . I copied the polkit.its and polkit.loc files and installed them into /usr/share/gettext/its. This seems to have no effect

Hmm, I'm not sure what the problem is there.
Ha, so I had the wrong dir: `msgformat` uses `/usr/share/gettext/its/`, but there's also `/usr/share/itstool/its/` used by `itstool`. I put the files in the second directory erroneously. Once that's fixed, translations happen.

So this almost works, but I think in current state it's not usable for us, because:
1. we need to carry the polkit.{its,loc} files in our source tree, since we want to support compilation with older polkit-devel which does not provide them. But w/o `data_dirs`, there's no way to tell `merge_file` about them.
2. I think we want to keep compatibility with the autotools based build system, so we'd need to remove the "_" prefixes by hand. `merge_file` of course does not remove them...
> But w/o data_dirs, there's no way to tell merge_file about them.

For now you can make a custom `msgfmt` wrapper but I'll work on fixing that in Meson too.

> I think we want to keep compatibility with the autotools based build system, so we'd need to remove the "_" prefixes by hand. merge_file of course does not remove them...

Yes you should port that to use upstream gettext also.

Resolving patch:
diff --git a/mesonbuild/mesonmain.py b/mesonbuild/mesonmain.py
--- a/mesonbuild/mesonmain.py
+++ b/mesonbuild/mesonmain.py
@@ -222,6 +222,9 @@ def run_script_command(args):
     elif cmdname == 'gtkdoc':
         import mesonbuild.scripts.gtkdochelper as abc
         cmdfunc = abc.run
+    elif cmdname == 'msgfmthelper':
+        import mesonbuild.scripts.msgfmthelper as abc
+        cmdfunc = abc.run
     elif cmdname == 'regencheck':
         import mesonbuild.scripts.regen_checker as abc
         cmdfunc = abc.run
diff --git a/mesonbuild/modules/i18n.py b/mesonbuild/modules/i18n.py
--- a/mesonbuild/modules/i18n.py
+++ b/mesonbuild/modules/i18n.py
@@ -49,6 +49,12 @@
 
 class I18nModule(ExtensionModule):
 
+    @staticmethod
+    def _get_data_dirs(state, dirs):
+        """Returns source directories of relative paths"""
+        src_dir = path.join(state.environment.get_source_dir(), state.subdir)
+        return [path.join(src_dir, d) for d in dirs]
+
     def merge_file(self, state, args, kwargs):
         podir = kwargs.pop('po_dir', None)
         if not podir:
@@ -60,8 +66,15 @@ def merge_file(self, state, args, kwargs):
         if file_type not in VALID_TYPES:
             raise MesonException('i18n: "{}" is not a valid type {}'.format(file_type, VALID_TYPES))
 
-        kwargs['command'] = ['msgfmt', '--' + file_type,
-                             '--template', '@INPUT@', '-d', podir, '-o', '@OUTPUT@']
+        datadirs = self._get_data_dirs(state, mesonlib.stringlistify(kwargs.pop('data_dirs', [])))
+        datadirs = '--datadirs=' + ':'.join(datadirs) if datadirs else None
+
+        command = [state.environment.get_build_command(), '--internal', 'msgfmthelper',
+                   '@INPUT@', '@OUTPUT@', file_type, podir]
+        if datadirs:
+            command.append(datadirs)
+
+        kwargs['command'] = command
         ct = build.CustomTarget(kwargs['output'] + '_merge', state.subdir, kwargs)
         return ModuleReturnValue(ct, [ct])
 
@@ -72,7 +85,7 @@ def gettext(self, state, args, kwargs):
             raise coredata.MesonException('Can not do gettext because xgettext is not installed.')
         packagename = args[0]
         languages = mesonlib.stringlistify(kwargs.get('languages', []))
-        datadirs = mesonlib.stringlistify(kwargs.get('data_dirs', []))
+        datadirs = self._get_data_dirs(state, mesonlib.stringlistify(kwargs.get('data_dirs', [])))
         extra_args = mesonlib.stringlistify(kwargs.get('args', []))
 
         preset = kwargs.pop('preset', None)
diff --git a/mesonbuild/scripts/msgfmthelper.py b/mesonbuild/scripts/msgfmthelper.py
new file mode 100644
--- /dev/null
+++ b/mesonbuild/scripts/msgfmthelper.py
@@ -0,0 +1,35 @@
+# Copyright 2016 The Meson development team
+
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+
+#     http://www.apache.org/licenses/LICENSE-2.0
+
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import subprocess
+import os
+
+parser = argparse.ArgumentParser()
+parser.add_argument('input')
+parser.add_argument('output')
+parser.add_argument('type')
+parser.add_argument('podir')
+parser.add_argument('--datadirs', default='')
+
+
+def run(args):
+    options = parser.parse_args(args)
+    env = None
+    if options.datadirs:
+        env = os.environ.copy()
+        env.update({'GETTEXTDATADIRS': options.datadirs})
+    return subprocess.call(['msgfmt', '--' + options.type, '-d', options.podir,
+                            '--template', options.input,  '-o', options.output],
+                           env=env)


Test patch: