Remove the user checking for the keymanager.

The existing keymanager tests for specific users are skipped, but not removed.
I expect that sometime in the near future we will have keys stored for users, but we are not doing that just yet.

We currently don't test for this. Do we need to?