Only create PagerDuty alerts if given service name

[#50018413]