Add tokeniser test mode to view tokeniser output on stdout