Add switch to pause failing test

Currently IDE tests continue and after last test IDE is closed no matter what. It could be handy to have support to pause test execution after test fails.