Split out tokenisation and make it a little more robust