Also remove old puppet certs