test/dump-restore: new tests for empty files and leading comments/whitespace.

Three of these are marked broken; the third is a regression test,
since it passes by virtue of batch-tag being the default input format.
notmuch-restore: handle empty input file, leading blank lines and comments.

This patch corrects several undesirable behaviours:

1) Empty files were not detected, leading to buffer read overrun.

2) An initial blank line cause restore to silently abort

3) Initial comment line caused format detection to fail
restore: Make missing messages non-fatal (again)

Previously, restore would abort if a message ID in the dump was
missing.  Furthermore, it would only report this as a warning.  This
patch makes it distinguish abort-worthy lookup failures like
out-of-memory from non-fatal failure to find a message ID.  The former
is reported as an error and causes restore to abort, while the latter
is reported as a warning and does not cause an abort.

This restores 0.14's non-fatal handling of missing message IDs in
restore (though 0.14 also considered serious errors non-fatal; we
retain the new and better handling of serious errors).
util: Factor out boolean term quoting routine

This is now a generic boolean term quoting function.  It performs
minimal quoting to produce user-friendly queries.

This could live in tag-util as well, but it is really nothing specific
to tags (although the conventions are specific to Xapian).

The API is changed from "caller-allocates" to "readline-like".  The
scan for max tag length is pushed down into the quoting routine.
Furthermore, this now combines the term prefix with the quoted term;
arguably this is just as easy to do in the caller, but this will
nicely parallel the boolean term parsing function to be introduced
shortly.

This is an amalgamation of code written by David Bremner and myself.
util: Function to parse boolean term queries

This parses the subset of Xapian's boolean term quoting rules that are
used by make_boolean_term.  This is provided as a generic string
utility, but will be used shortly in notmuch restore to parse and
optimize for ID queries.
dump: Disallow \n in message IDs

When we switch to using regular Xapian queries in the dump format, \n
will cause problems, so we disallow it.  Specially, while Xapian can
quote and parse queries containing \n without difficultly, quoted
queries containing \n still span multiple lines, which breaks the
line-orientedness of the dump format.  Strictly speaking, we could
still round-trip these, but it would significantly complicate restore
as well as scripts that deal with tag dumps.  This complexity would
come at absolutely no benefit: because of the RFC 2822 unfolding
rules, no amount of standards negligence can produce a message with a
message ID containing a line break (not even Outlook can do it!).

Hence, we simply disallow it.
dump/restore: Use Xapian queries for batch-tag format

This switches the new batch-tag format away from using a home-grown
hex-encoding scheme for message IDs in the dump to simply using Xapian
queries with Xapian quoting syntax.

This has a variety of advantages beyond presenting a cleaner and more
consistent interface.  Foremost is that it will dramatically simplify
the quoting for batch tagging, which shares the same input format.
While the hex-encoding is no better or worse for the simple ID queries
used by dump/restore, it becomes onerous for general-purpose queries
used in batch tagging.  It also better handles strange cases like
"id:foo and bar", since this is no longer syntactically valid.
man: Update notmuch-dump(1) and notmuch-restore(1)

Describe the new batch-tag format.  For notmuch-restore, rather than
half-heartedly duplicating the description, we now cite notmuch-dump.
emacs: Use the minibuffer for CLI error reporting

We recently switched to popping up a buffer to report CLI errors, but
this was too intrusive, especially for transient errors and especially
since we made fewer things ignore errors.  This patch changes this to
display a basic error message in the minibuffer (using Emacs' usual
error handling path) and, if there are additional details, to log
these to a separate error buffer and reference the error buffer from
the minibuffer message.  This is more in line with how Emacs typically
handles errors, but makes the details available to the user without
flooding them with the details.

Given this split, we pare down the basic message and make it more
user-friendly, and also make the verbose message even more detailed
(and more debugging-oriented).
fix line breaks in one comment paragraph in generated .notmuch-config file

While one comment in generated .notmuch-config file looked good in the
source file notmuch-config.c, the generated output was inconsistently
wide -- even breaking the 80-column boundary.
tag-util: factor out rules for illegal tags, use in parse_tag_line

This will allow us to be consistent between batch tagging and command
line tagging as far as what is an illegal tag.
notmuch-tag.c: convert to use tag-util

Command line parsing is factored out into a function
parse_tag_command_line in tag-util.c.

There is some duplicated code eliminated in tag_query, and a bunch of
translation from using the bare tag_op structs to using that tag-utils
API.
cli: add support for batch tagging operations to "notmuch tag"

Add support for batch tagging operations through stdin to "notmuch
tag". This can be enabled with the new --batch command line option to
"notmuch tag". The input must consist of lines of the format:

+<tag>|-<tag> [...] [--] <query> [...]

Each line is interpreted similarly to "notmuch tag" command line
arguments. The delimiter is one or more spaces ' '. Any characters in
<tag> MAY be hex encoded with %NN where NN is the hexadecimal value of
the character. Any ' ' and '%' characters in <tag> and MUST be hex
encoded (using %20 and %25, respectively). For future-proofing, any
'"' characters in <tag> SHOULD be hex-encoded.

Any characters that are not part of <tag> or
MUST NOT be hex encoded.

<query> is passed verbatim to Xapian

Leading and trailing space ' ' is ignored. Empty lines and lines
beginning with '#' are ignored.

Signed-off-by: Jani Nikula <jani@nikula.org>

Hacked-like-crazy-by: David Bremner <david@tethera.net>
test/tagging: add test for error messages of tag --batch

This is based on the similar test for notmuch restore, but the parser
in batch tagging mode is less tolerant of a few cases, in particular
those tested by illegal_tag.
test/tagging: add basic tests for batch tagging functionality

This tests argument parsing, blank lines and comments, and basic hex
decoding functionality.
test/tagging: add tests for exotic tags

We test quotes seperately because they matter to the query escaper.
test/tagging: add test for exotic message-ids and batch tagging

The (now fixed) bug that this test revealed is that unquoted
message-ids with whitespace or other control characters in them are
split into several tokens by the Xapian query parser.
man: document notmuch tag --batch, --input options
test/tagging: add test for naked punctuation in tags; compare with quoting spaces.

This test also serves as documentation of the quoting
requirements. The comment lines are so that it exactly matches the man
page. Nothing more embarrassing than having an example in the man page
fail.
string-util: Disallow empty prefixes in parse_boolean_term

Xapian doesn't consider ":abc" to be a prefixed term.  This makes
parse_boolean_term similarly reject queries with an empty prefix.