Use arraycopy to fill in the tags.
Rayification.

So odd that you'd pre bit-shift a few things to be 1, 2, 4, 8...

Also: some of the other magic numbers should be cleaned up in here, along
with documentation of what every bit in a 'key' integer means.

And perhaps the modifier key stuff should be an enum anyway.
Aren't the values opaque to a user of this class? Ints are just begging
for mistakes, especially for things like the method hasModifierKey(int, int)