all: Avoid error when hostkey is not readable to user

Was previously merged in ad4a20942d1cb61cb77b981961ef2d4086233fd9 however the fix was subsequently lost in 4c5f7f538c027e1c226f70f54ca6f4fbed0f8e80.