templater: avoid recursive evaluation of string literals completely

Changeset 3d8bfe2ecf6d (released with 2.8.1) fixed "recursively
evaluate string literals as templates" problem (issue4103) by
introducing "_evalifliteral()".

But some parts in template expressions below are still processed by
the combination of "compiletemplate()" and "runtemplate()", and may
cause same problem unexpectedly.

  - 'init' and 'hang' of 'fill(text, width, init, hang)'
  - 'expr' of 'sub(pat, repl, expr)'
  - 'label' of 'label(label, expr)'

This patch processes them by "_evalifliteral()" instead of the
combination of "compiletemplate()" and "runtemplate()" to avoid
recursive evaluation of string literals completely.

--HG--
branch : stable
templater: apply "stringify()" on sub expression to get string correctly

Templating syntax allows nested expression to be specified as parts
below, but they are evaluated as a generator and don't work correctly.

  - 'sep' of 'join(list, sep)'
  - 'text' and 'chars' of 'strip(text, chars)'

In the former case, 'sep' returns expected string only for the first
separation, and empty one for the second or later, because the
generator has only one element.

In the latter case, templating is aborted by exception, because the
generator doesn't have 'strip()' method (as 'text') and can't be
passed as the argument to 'str.strip()' (as 'chars').

This patch applies "stringify()" on these sub expression to get string
correctly.

--HG--
branch : stable
templater: make strings in template expressions be "string-escape"-ed correctly

Changeset 64b4f0cd7336 (released with 2.8.1) fixed "recursively
evaluate string literals as templates" problem (issue4102) by moving
the location of "string-escape"-ing from "tokenizer()" to
"compiletemplate()".

But some parts in template expressions below are not processed by
"compiletemplate()", and it may cause unexpected result.

  - 'expr' of 'if(expr, then, else)'
  - 'expr's of 'ifeq(expr, expr, then, else)'
  - 'sep' of 'join(list, sep)'
  - 'text' and 'style' of 'rstdoc(text, style)'
  - 'text' and 'chars' of 'strip(text, chars)'
  - 'pat' and 'repl' of 'sub(pat, repl, expr)'

For example, '\n' of "{join(extras, '\n')}" is not "string-escape"-ed
and treated as a literal '\n'. This breaks "Display the contents of
the 'extra' field, one per line" example in "hg help templates".

Just "string-escape"-ing on each parts above may not work correctly,
because inside expression of nested ones already applies
"string-escape" on string literals. For example:

  - "{join(files, '\n')}" doesn't return "string-escape"-ed string, but
  - "{join(files, if(branch, '\n', '\n'))}" does

To fix this problem, this patch does:

  - introduce "rawstring" token and "runrawstring" method to handle
    strings not to be "string-escape"-ed correctly, and

  - make "runstring" method return "string-escape"-ed string, and
    delay "string-escape"-ing until evaluation

This patch invokes "compiletemplate()" with "strtoken=exp[0]" in
"gettemplate()", because "exp[1]" is not yet evaluated. This code path
is tested via mapping ("expr % '{template}'").

In the other hand, this patch invokes it with "strtoken='rawstring'"
in "_evalifliteral()", because "t" is the result of "arg" evaluation
and it should be "string-escape"-ed if "arg" is "string" expression.

This patch doesn't test "string-escape"-ing on 'expr' of 'if(expr,
then, else)', because it doesn't affect the result.

--HG--
branch : stable