Tokenisation and flat-file encoding