Start setting jenkins_path prefix instead of doing weird things with post_config