As mentioned on IRC, this is necessary for when pki_setup is not run (in production envs..)