Use WeakKeyDictionary instead of WeakSet

That makes it compatible with Python 2.6 again.

(cherry picked from commit 9e57ab4ad6ce8b083c2d7fc688f3fc1571db286d)

Change-Id: I9745165cb372a3e2589dfc289182d668eccf0253