Sadly `ERROR` directives are not parsed during the `run-make` tests. I would recommend storing the output of `rustc` and then invoking `grep` to make sure that it printed a vaguely correct message.