instead of long if-elif-elif-elif lists, use a dictionary and some decorators to dispatch tokens to the methods that handle them.