Ensure only users with ssh access are added with keys