Yes, it stops if a test fails. Because there are so many tests it can be easy to miss a failing test in the logs, so I figure it's best to stop the tests there.