tests: validate GTK-Doc test files

This is more of a handy developer tool than something
that is always expected to be executed as part of the
test suite, so we don't add a hard dependency on xmllint.

https://bugzilla.gnome.org/show_bug.cgi?id=688897
giscanner: tiny MainTransformer cleanup
giscanner: rename "options" to "annotations"

Makes it easier to understand the difference between
an annotation and an option, which are written as
"(annotation option1=value1 option2=value2 ...)" in
GTK-Doc comment blocks.
giscanner: rename "colon" to "delimiter"

It servers as a GTK-Doc "field" separator, so delimiter seems
like a better name. This also makes the descintion between the
colon used in property and signal names and the GTK-Doc "field"
delimiter a bit more clear.
giscanner: rename AnnotationParser() to GtkDocCommentBlockParser()

Clarify the purpose of what up until now was know as the
AnnotationParser() class, as it does more than just extracting
annotations, it parses the complete GTK-Doc comment block.
giscanner: rename the parse() method to parse_comment_blocks()

We already have a parse_comment_block() method parsing a single
GTK-Doc comment block so it only seems natural to have a plural
parse_comment_blocks() to go along with that.
giscanner: expand parse_comment_block() parameters

Makes it consitent with the parse_comment_blocks() and
_parse_comment_block() methods.
giscanner: use "\s" instead of "[^\S\n\r]"

The original idea was to only match non breaking space, but
there's really no reason to do things differently than gtkdoc-mkdb.
giscanner: always use re.UNICODE mode

should prevent us from running into locale specific issues
wrt what's whitespace, alpha, numeric, etc.
giscanner: define __slots__ on some classes

For example, when creating Gtk-3.0.gir, thousands of instances
of these classes get created. This makes us waste less space
for these instances by not having their __dict__ being created.
giscanner: complete parsing varargs parameters

Normalize deprecated @Varargs and named __VA_ARGS__ notations
in as single location, add tests.
giscanner: fix GTK-Doc comment block parsing

- GTK-Doc and current g-ir-scanner do not allow tags to be written before the
  comment block description but old version of g-ir-scanner parsed that just
  file. Fix this regression, but emit when tags are encountered at the wrong
  location. #674658

- During this analysis, a secondary bug report was filed with the GTK-Doc
  project because it was found that the same malformed tags did not result
  in the expected html output.

  After discussing this with the GTK-Doc maintainers, we learned
  that our g-i specific top level tags should never have existed
  in the first place. The prefered notation for annotations that
  apply to the identifier should be written on the identifier line,
  for example like what we already do with (skip).

  As a result, this patch deprecates g-i specific top level tags and
  implements them as annotations on the identifier instead and at
  the same time adds support for malformed comment blocks still
  using g-i specific top level tags.

  This means that all annotated code "out there" will continue to work
  just fine with this version of g-i, but when a developer decides to
  fix deprecation warnings in his/her comment blocks, the dependency
  on g-i needs to be raised to a version that contains at least this
  patch. #676133

- I previously (and foolishly) removed parentheses matching with
  a regex, which off course is not possible when there are multiple
  nested levels of parens (which we will now see more with annotations
  on the identifier, for example with the type annotation). Remove
  the "annotation parens matching" regexes again and add in more
  warnings on unbalanced/unexpected parentheses. #688897

https://bugzilla.gnome.org/show_bug.cgi?id=674658
https://bugzilla.gnome.org/show_bug.cgi?id=676133
https://bugzilla.gnome.org/show_bug.cgi?id=688897
giscanner: Fix description field storage in .gir files

GTK-Doc description fields for tags can contain multiple lines and
even multiple paragraphs. Whitespace cannot be preserved in XML
attributes, so we move the "deprecated" description text into
a "<doc-deprecated />" element right next to where we already have
the "<doc />" element. Keep the "deprecated" attribute around for
backwards compatibility though, but set its value to "1" (analogous
to the "writable", "contruct", etc attributes) if the annotated
symbol is marked as deprecated.

While at it, add <doc-version /> and <doc-stability /> which
was not yet available in the .gir files...

This takes care of the "Since:", "Stability:" and "Deprecated:"
GTK-Doc tags. Nothing needs to be done the "Returns:" tag as
as we already write a "<doc />" child element on "<return-value />".
Add a tool fixing GTK-Doc comment blocks in .c and .h files

Just a standalone script for now, should probably eventually
go into g-ir-doctool or g-ir-annotation-tool...
docs: document some annotationparser.py hacking hints

https://bugzilla.gnome.org/show_bug.cgi?id=688897
Add giscanner package documentation

Using Sphinx [1] markup for now. We eventually might want
to add GTK-Doc support for python and dogfood g-ir-doctool
on this package.

[1] De facto standard Python documentaton tool: http://sphinx-doc.org

https://bugzilla.gnome.org/show_bug.cgi?id=688897
g-ir-compiler: remove references to --code, remove --no-init options

The "--code" option was removed years ago in ac81f3e8c5f1c380d16677232d67466e739da283
so remove references to it from README and g-ir-compiler(1)

Remove the "--no-init" option from g-ir-compiler and g-ir-compiler(1)
as it was documented to "can only be used if --code is also specified",
so no reason to keep it around.
build: actually require libffi >= 3.0.0

as documented in README...
README: fix the Tools section a bit
[WIP] docs: improve layout