Improve token_and_options regex and test

1. Strip an erroneous double quote from the params split regex
2. Add test cases to test the token and option extraction functionality
Yes. It is better to include a negative unit test to this.