Use sets O(1) to avoid repeated O(n) operations during char scans

This amounts to a significant performance boost.