Added sanitisation early on the the tokenizer so that tokens containing special character (which are used as separators later) are not let through. Expanded tests to verify dependencies are right.