Better error reporting from tokenize
Message explaining why I'm not worried about suboptimal union
Moved testing and benchmarking into its own module.
A long c file for testing
Core doesn't recognize \f \v as whitespace
Added character constants
Convenience functions added and testing removed