Run on failure flag

Trying this again with branch rebased on rtomac's master.