added flexibility in token parsing