Enhance tokenizer test program
Rename append_tag -> push_tag and similar

And remove an unnecessary helper.
Implement comment states
Reformat shorthand macros
Implement doctype states
Implement BogusComment state
Move remaining state into the Tokenizer struct

This brings a number of benefits:

    - support a queue of buffers to process
    - bugfix and cleanup of reconsuming
    - erase distinction between process_char states and others
    - eliminate variable capture in go!(error)
    - local bindings for 'c' improve readability
Make a library crate and add temporary build script