Repository: mesonbuild/meson

Issue:
regression with string escaping/formatting and cc.run()
#3169 seems to have caused a regression with the GLib meson build, which does the following thing (simplified):
```
  templ = '''
#include<stdio.h>
int main(int argc, char **argv) {
  printf("%d\n", (int)@0@);
  return 0;
}'''
value_FOO = cc.run(templ.format('FOO'), name : 'FOO value').stdout().strip()
```

Let's ignore for now that this is not cross-compile friendly and should be using `cc.compute_int()` or such instead.

Since the above commit 36aab0f4b2a05a1ae4c8603e57b4c3c684fe8ea8 this now fails with `Checking if "POLLIN value" runs: DID NOT COMPILE`

The reason this fails to compile is because the code to be compiled is now (excerpt from log file):

```
Code:
#include<stdio.h>
int main(int argc, char **argv) {
  printf("%d
", (int)FOO);
  return 0;
}
Compiler stdout:
 
Compiler stderr:
 /tmp/tmpguawnoj0/testfile.c: In function ‘main’:
/tmp/tmpguawnoj0/testfile.c:5:10: warning: missing terminating " character
   printf("%d
```

As you can see, the `\n` in the multi-line string has been turned into an actual newline here. Before the commit, the code would have a backslash and an `n` character there instead.


Discussion:
Would it be enough to only not expand \n in triple quoted strings or do we need to revert fully and come up with something else?
To be fully backward compatible I guess we should introduce a new type of string that does escaping. Maybe use "some\nstring" or escape_string('some\nstring').
No double quotes. And the latter does not work because the escaping is already done by the time the method is called and you can't reliably undo it because some linefeeds might come from actual newlines.
otoh, the doc previously said:

> Similarly `\n` gets converted to a newline

So one could argue that glib's code should never have worked.
@jpakkane or just like python has `b'binary string'` we could have `e'escaped string'`
The real question is how many projects would break with this change? If it's only one or two we can break it but not if it affects many or popular projects.

Maybe we could detect strings with those newline settings, print a warning for a release or two and then change it to the current behaviour or something like that?
> And the latter does not work because the escaping is already done by the time the method is called

Actually not, \n was not replaced by newline otherwise glib code would never have worked. So we could add in function that does additional replacements.
This is a well documented breakage. If we don't want to introduce any breakage at all we simply can't change the escaping rules.. I think we are early enough in mesons development that we should be able to fix this kind of "soundness" issues. IMO it is broken behavior that `\n` does not convert to a newline.

But if it is your opinion to introduce a new  string type I'm open for that as well. I could also output a warning for every single escape sequence.
How about introducing raw strings like `r'nothing is escaped here'`?
That doesn't fix the backwards compatibility issue though, does it?

It's also unclear to me why anyone would ever want to see a literal `\n` in a multi-line string constant be turned into an actual newline. If they wanted a newline they'd just put an actual newline there, it's a multi-line string constant after all.
For consistency, but we can remove all escape sequences in triplequoted strings to keep some backwards compatibility.
We do need something like Python's raw strings. However, in the absence of `r''` and other notations, I like the idea of having `'''` mean no escaping is done.
You like it until you want to put in a \t or a unicode character. But sure then we can have `e''''''` to be escaped version of triple-qouted string.
> You like it until you want to put in a \t or a unicode character

We can mix `'''` and `'` with `+` and `@0@`, etc, but yes, it was a mistake to not have a consistent plan for escaping and raw strings.

Resolving patch:
diff --git a/mesonbuild/mparser.py b/mesonbuild/mparser.py
--- a/mesonbuild/mparser.py
+++ b/mesonbuild/mparser.py
@@ -28,18 +28,6 @@
     | \\[\\'abfnrtv]   # Single-character escapes
     )''', re.UNICODE | re.VERBOSE)
 
-# This is the regex for the supported escape sequences of a multiline string
-# literal, like '''abc\x00'''. The only difference is that single quote (')
-# doesn't require escaping.
-ESCAPE_SEQUENCE_MULTI_RE = re.compile(r'''
-    ( \\U........      # 8-digit hex escapes
-    | \\u....          # 4-digit hex escapes
-    | \\x..            # 2-digit hex escapes
-    | \\[0-7]{1,3}     # Octal escapes
-    | \\N\{[^}]+\}     # Unicode characters by name
-    | \\[\\abfnrtv]    # Single-character escapes
-    )''', re.UNICODE | re.VERBOSE)
-
 class MesonUnicodeDecodeError(MesonException):
     def __init__(self, match):
         super().__init__("%s" % match)
@@ -187,10 +175,6 @@ def lex(self, subdir):
                     elif tid == 'multiline_string':
                         tid = 'string'
                         value = match_text[3:-3]
-                        try:
-                            value = ESCAPE_SEQUENCE_MULTI_RE.sub(decode_match, value)
-                        except MesonUnicodeDecodeError as err:
-                            raise MesonException("Failed to parse escape sequence: '{}' in string:\n{}".format(err.match, match_text))
                         lines = match_text.split('\n')
                         if len(lines) > 1:
                             lineno += len(lines) - 1


Test patch:
diff --git a/test cases/common/32 multiline string/meson.build b/test cases/common/32 multiline string/meson.build
--- a/test cases/common/32 multiline string/meson.build	
+++ b/test cases/common/32 multiline string/meson.build	
@@ -23,3 +23,13 @@ quote2 = '\''
 if quote1 != quote2
   error('Single quote quoting is broken.')
 endif
+
+cc = meson.get_compiler('c')
+prog = '''
+int main(int argc, char **argv) {
+  int num = 1;
+  printf("%d\n", num);
+  return 0;
+}'''
+
+assert(cc.compiles(prog), 'multline test compile failed')
diff --git a/test cases/common/42 string operations/meson.build b/test cases/common/42 string operations/meson.build
--- a/test cases/common/42 string operations/meson.build	
+++ b/test cases/common/42 string operations/meson.build	
@@ -78,15 +78,17 @@ assert('"1.1.20"'.strip('".') == '1.1.20', '". badly stripped')
 assert('"1.1.20"   '.strip('" ') == '1.1.20', '". badly stripped')
 
 bs_c = '''\c'''
-bs_bs_c = '''\\\c'''
+bs_bs_c = '''\\c'''
 nl = '''
 '''
-bs_n = '''\\n'''
+bs_n = '''\n'''
 bs_nl = '''\
 '''
-bs_bs_n = '''\\\\n'''
-bs_bs_nl = '''\\\\
+bs_bs_n = '''\\n'''
+bs_bs_nl = '''\\
 '''
+bs_bs = '''\\'''
+bs = '''\'''
 
 assert('\c' == bs_c, 'Single backslash broken')
 assert('\\c' == bs_c, 'Double backslash broken')
@@ -97,3 +99,5 @@ assert('\\n' == bs_n, 'Double backslash broken before n')
 assert('\\\n' == bs_nl, 'Three backslash broken before n')
 assert('\\\\n' == bs_bs_n, 'Four backslash broken before n')
 assert('\\\\\n' == bs_bs_nl, 'Five backslash broken before n')
+assert('\\\\' == bs_bs, 'Double-backslash broken')
+assert('\\' == bs, 'Backslash broken')