Ensure pki_setup not run before configuration complete

Change-Id: I81bee474866c0a3c3786fcf1ab025163cbdc3ac9