Remove host from hosts_to_roles if no roles match.

Otherwise, host will fail follow on validation check.