mozilla/webvtt — pull request #356 by RickEyre (2013-04-15 18:34:18+00:00)
Title: Tokenizer test fixes

Fixed up some of the tokenizer tests fixes.

- Make NonAlphaNumeric more understandable.
Updated EscapeStatTokenizer.NonAlphaNumeric to be more
understandable. Instead of using a '<' character which
serves the purpose of denoting tags I've switched it
for an '!' which makes what the test is testing more clear.

Change tokenizer unit tests to not use \0.
- I've changed the tokenizer unit tests to not rely on \0
when testing. \0 in a lot of cases terminates the tokenizer
functions so checking for them at the end of the test might
result in not catching some errors where it is incorrectly
but still ended up on the \0.
- I've updates those unit tests with text to test that is
more like what it would receive in real situations.

Also fixed a few spelling mistakes.