Remove the user checking for the keymanager.

The existing keymanager tests for specific users are skipped, but not removed.
I expect that sometime in the near future we will have keys stored for users, but we are not doing that just yet.

It is extremely hard to get that line to error out without mocking it out.  More than we care about I think.