Revert "Shrink Key structure, implement __sizeof__"

Major slowdown! Figure it out later.

This reverts commit a006c0f422cada9a984161a81d6b0d1448a8bbc4.