Don't use cache in {Public,Private}KeyFile

It's cheap to compute those strings again, and they are used
very infrequently (in one-time code and error messages).