I wonder if this is the right thing. I can imagine having errors in `src/main/test`, and you'd probably want to still run your application. I think the current Java-way (that lets the user decide) is more useful.