You're right. Harry says using arrays instead of the Token object is faster and this might be why. I like the syntactic sugar but if getting rid of it means I get to keep token objects then I will.

I'll try this and benchmark it.