moved test into comments
combined sentence and token parsing logic