Did you test the speed of this array-based approach against the simpler 
if (char==terminator)
 return TERMINATOR;
else if (char==quote)
  return QUOTE;

etc 
approach ?

Just curious really. Modern processors are funny things, and the array would need to stay in l1 cache to be effective.