Emit alerts json that mimics config structure