lsegal/yard — pull request #668 by kyrylo (2013-04-27 21:28:17+00:00)
Title: CParser: make sure structs are not mixed up with struct retvals

```
Fix issue #666 (Cannot properly parse Hash from Ruby 1.9.3-p392)

The underlying problem is that YARD mixes up C struct definitions with
functions that return structs. This results in improper parsing of C
files.

STRUCT | FUNCTION
------------------------------------------------------
struct foreach_safe_arg { | struct st_table *
/* | rb_hash_tbl(VALUE hash)
* Blah-blah-blah. | {
*/ | /*
}; | * Yadda-yadda-yadda.
| */
| }

So, what's the real problem? Well, YARD indeed cannot properly parse
'hash.c' from Ruby 1.9 (and some other files like 'thread.c') but
parsing 'hash.c' from Ruby 2.0 works as expected. And the reason for
that is quite amusing. Note that the STRUCT definition terminates
with a semicolon while FUNCTION isn't. So given both cases to be
parsed, `consume_until(';')` from `CParser` would try to find a
semicolon. But in the case of FUNCTION it won't find it. And the loop
will keep advancing until a top-level semicolon is found. And in case
of Ruby 1.9 the first "good" semicolon is found somewhere near line 820.
It means that YARD just skipped like a quarter of useful docs.

Why is it possible to successfully parse 'hash.c' from Ruby 2.0, then?
Actually, it's not. Ruby 2.0 is lucky with regard to this aspect. The
first top-level semicolon can be found almost immediately after the
function, which returns a pointer to a struct. And between them there
are no any docs.

The fix is just to slightly amend the regexp for a top-level statement.
I also had to remove the `\b` from the regexp, but the good news is that
we don't use `$1` value anywhere, so there was no real purpose in the
`\b`.
```