Make sure yaml data loaded for role is valid before using it

Fixes #4322