Added :ignore tokens, including docs.

Also switched to a non-recursive parser to defend against possible
DoS attacks that try to consume the C stack using lots of __start__ tags.
Furthermore, the non-recursive parser turns out easier to read and
modify IMHO.