Improve token_and_options regex and test

1. Strip an erroneous double quote from the params split regex
2. Add test cases to test the token and option extraction functionality