see same comment about preprocessToken for line 67.