Remove the user checking for the keymanager.

The existing keymanager tests for specific users are skipped, but not removed.
I expect that sometime in the near future we will have keys stored for users, but we are not doing that just yet.