tests: added file for general API tests
tokenizer: added specialized characterState function
tokenizer: reconsume characters in ifElseState()

fixes handling of <![CD> and friends
tokenizer: fixed boundaries of legacy entities
test: added/extended test cases
tokenizer: reconsume last token when not in CDATA