Use sentinel-terminated token array

Instead of an unterminated array with a separate length variable,
terminate the array with a TOKEN_NONE token.