giscanner: improve wording of inline documentation
giscanner: use "if in [a, b]" instead of "if == a or if == b"
giscanner: make it clear DocOptions() also has a position attribute

Lost a couple of hours hunting down a failed test case (from the
annotationparser work about to land) due to this not being clear...
giscanner: use re.match() instead of re.search()

From Python's documentation:
Python offers two different primitive operations based on
regular expressions: match checks for a match only at the
beginning of the string, while search checks for a match
anywhere in the string (this is what Perl does by default).

The different behavior between match() and search() doesn't
matter in our case and conceptually we are checking if a
piece of text matches a certain pattern, so match() is a
better fit.
giscanner: add AnnotationParser tests

The tests in giscanner/annotationpatters.py only test the regular
expression programs used when parsing GTK-Doc comment blocks but
do not test the structure of the resulting "parse tree".

This patch adds 193 GTK-Doc comment blocks and the expected
results AnnotationParser should return (with it's current level
of understanding of GTK-Doc comment block syntax). These are
compared by tests/scanner/annotationparser/test_parser.py which
complains with a diff on failure.
giscanner: move unittests from giscanner/annotationpatters.py...

... to tests/scanner/annotationparser/test_patterns.py
giscanner: construct list of possible tag names for TAG_RE from _ALL_TAGS...

... instead of having a secondary hard-coded list. This results
in the removal of giscanner/annotationpatterns.py to prevent a
circular import.
giscanner: treat the GTK-Doc Description: tag like any other tag

and add paragraph break tests.
giscanner: Correctly detect invalid GTK-Doc comment block end markers
giscanner: remove re.MULTILINE usage from annotationparser

These are remnants from the original prototype and are no
longer used.
giscanner: update annotationparser to most recent gtkdoc-mkdb bug fixes.

This patch adds a test for and implements a more generic solution to
GTK-Doc commit 47abcd53b8489ebceec9e394676512a181c1f1f6
Update GLib annotations to master

Which at the time of writing is GLib commit cd8ec3359b4b8afd5cce48bf218ecd793f8064eb
giscanner: fix DocBlock().comment

If there is no comment block description, DocBlock().comment
should be None. This results in the removal of unneeded blank
lines in the output of DocBlock().to_gtk_doc and hence the .c
files generated by misc/update-glib-annotations.py
giscanner: don't continue parsing after muliline descriptions
giscanner: deprecate top level g-i specific GTK-Doc tags.

Rationale
---------

The first wave of improvements on AnnotationParser intended
to fix bug http://bugzilla.gnome.org/show_bug.cgi?id=672254
(multiline descriptions on tags not working correctly, polluting
the comment block description instead) had an unforseen side
effect reported in bug http://bugzilla.gnome.org/show_bug.cgi?id=674658.

Analysing this problem we found that even though comment blocks
having g-i specific tags before the comment block description
are technically malformed, g-i specific tags never had multiline
descriptions to begin with (but the basic set of GTK-Doc tags do).
During this analysis, a secondary bug report
(http://bugzilla.gnome.org/show_bug.cgi?id=676133) was filed with
the GTK-Doc project because it was found that the same malformed
comment blocks did not result in the expected html output.
All hell broke loose...

In short, we learned that these g-i specific top level tags should
never have existed in the first place, preferring annotations on the
identifier instead. As a result, this patch deprecates g-i specific
top level tags and implements them as annotations on the identifier
instead. Without breaking support for malformed comment blocks still
using g-i specific top level tags.

This means that all annotated code "out there" should continue to work
just fine with this version of g-i, but when a developer decides to
fix deprecation warnings in his/her comment blocks, the dependency
on g-i needs to be raised to a version that contains this patch.

Overview of changes
-------------------

1) giscanner/annotationparser.py and giscanner/maintransformer.py
   have been adjusted to deprecate g-i specific top level tags
   and now implement these in terms of annotations on the identifier.
   The Attributes: tag and it's new (attributes) annotation
   counterpart continue to be broken (regression introduced in
   commit 182fdfec261f36fa095484352b9b834861036907), which will be
   addressed in a followup patch.

2) AnnotationParser().parse_options() has learned to correctly parse
   multiple closing parentheses, which is for example required for the
   new syntax in tests/scanner/annotationparser/gi/annotation_type.xml:
   * AnnotationObject:hash-table: (type GLib.HashTable(utf8,gint8))

3) comment blocks in tests/*.c and tests/*.h had to be adjusted as
   tests are not allowed to emit warnings. As a result, these test
   cases now no longer use g-i specific top level tags but annotations
   on the identifier instead.

This hopefully satisfies https://bugzilla.gnome.org/show_bug.cgi?id=676133
while fixing http://bugzilla.gnome.org/show_bug.cgi?id=674658 as a side effect.
giscanner: fix annotations pattern

The "annotations" part of our regular expression programs failed
on multiple closing parentheses like (element-type GLib.List(utf8)).
This patch adds tests and fixes the issue.

Thanks to Colin Walters for the scrutiny and voicing his concerns on irc!
giscanner: fix multiline annotation warnings

Emit a warning when multiline annotations are detected on the
identifier and only emit a warning when a parameter or tag
lack a description.
giscanner: move AnnotationParser() warning tests

The following patch (add support for deprecated @Varargs
parameter notation) would require adding a new expected
warning to tests/warn/unknown-parameter.h before all
other warnings, even those from unrelated comment blocks.
This because AnnotationParser() warnings are emitted *before*
MainTransformer() warnings.

This patch extracts all warning tests for AnnotationParser()
into the test_parser.py testsuite instead.
giscanner: add support for deprecated @Varargs parameter notation

Transparantly transform @Varargs into @... and add tests.
giscanner: don't scan identical comment blocks from the same file multiple times

Now that annotationparser is used independently by
`g-ir-annotationtool --fix`, we don't want to waste time scanning
identical comment blocks from the same file multiple times...