Added ability to override the tokenizer.

Can control both the compile-time and runtime tokenizers.
Merge commit '6fcc0eec7029880cfe0c0a40d0cf192ed4571d24'

Conflicts:
	scan/src/parse.rs
	scan/src/plugin.rs
Removed duplicate line.

How did I even?