Moved tokenisation-related code into its own module as part of work on the reduce function.
Reworked reduce report method in order to be correct.