Prevent Bamboo from failing job when failing tests are moved to quarantine
Asked by pbaranski:

On Bamboo I have MSBuild job for building and running tests.

It works fine - but I want to use test quarantining option so when I have some test moved to quarantine (and this moved tests are only what fail) job will be marked as successful

I moved one failing test to quarantine but job is marked as failed

Configuration:

Job in Tasks I have Command named Test runner:

Executable = VSTest.Console

Argument = "WebServiceTestClient.dll" /Logger:trx

And after running job with failing test quarantined I get log
> 01-Apr-201410:59:44Total tests:62.Passed:61.Failed:1.Skipped:0.  
> 01-Apr-201410:59:44TestRunFailed.01-Apr-201410:59:44Test execution time:10.1296Minutes  
> 01-Apr-201410:59:44Failing task since return code of \[C:\\Program Files(x86)\\Microsoft VisualStudio11.0\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow\\VSTest.Console.exe "WebServiceTestClient.dll"/Logger:trx\] was 1while expected 0  
> 01-Apr-201410:59:44Finished task 'Test runner'  
> 01-Apr-201410:59:44Starting task 'Parse test logs' of type 'com.atlassian.bamboo.plugin.dotnet:mstest'  
> 01-Apr-201410:59:44Parsing test results...  
> 01-Apr-201410:59:45Failing task since 1 failing test cases were found

Job is marked failed but then:

> 01-Apr-201410:59:45ChangingTaskResult to SUCCESS as all failed tests were quarantined.01-Apr-201410:59:45Finished task 'Parse test logs'01-Apr-201410:59:45Running post build plugin 'NCover Results Collector'01-Apr-201410:59:45Running post build plugin 'Clover Results Collector'01-Apr-201410:59:45Running post build plugin 'Artifact Copier'01-Apr-201410:59:45Finalising the build...

Unfortunately the output is:

|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| ![enter image description here](http://i.stack.imgur.com/lEl3P.png) This question was asked here <http://stackoverflow.com/questions/22803557/prevent-bamboo-from-failing-job-when-failing-tests-are-moved-to-quarantine>but hopefully somebody here give me right answer. |


Answers:
author: Przemek Bruski
body: A fixed plugin is available here:

[](https://maven.atlassian.com/public/com/atlassian/bamboo/plugins/dotnet/atlassian-bamboo-plugin-dotnet/5.5.2/atlassian-bamboo-plugin-dotnet-5.5.2.jar)<https://maven.atlassian.com/public/com/atlassian/bamboo/plugins/dotnet/atlassian-bamboo-plugin-dotnet/5.5.2/atlassian-bamboo-plugin-dotnet-5.5.2.jar>

You can install it on older Bamboo versions, but first, remove the old plugin from bundled-plugins.zip

comments: author: Pawel Skierczynski
body: There is newer version fixing additional problem when validadtion error occur. Not related to this but you may want to install the newest one.

<https://maven.atlassian.com/public/com/atlassian/bamboo/plugins/dotnet/atlassian-bamboo-plugin-dotnet/5.5.3/atlassian-bamboo-plugin-dotnet-5.5.3.jar>


author: Jason Monsorno
body: We had this problem 2 weeks ago, I just had it run through a script then used the MSTest Parser instead. Yes, admittedingly it is a hack for functionality that should be builtin but it's not a show stopper anymore.

comments: author: Matt Block
body: Would you mind providing more details about this script and configuration? We are admittedly a \*nix shop but maintaining a Windows app during a migration process, so we defintely aren't windows scripting experts. How do you force the script to return 0?


author: Jason Monsorno
body: The script always return 0 by default, you have to do extra work to make it fail a build. It just runs MSTest with testContainer, resultsFile, \& category command line option. Then I use the MSTest Parser task to pickup the .trx file I just generated.


author: Matthew Block
body: I'm going to need a little more detail. I tried doing this and it is definitely still returning an error code and it doesn't even make it to the MSTest Parser task. Here is the inline script I was using...

```
echo offecho Running Tests writing results to testresults-%BAMBOO_BUILD_NUMBER%.trx"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" /testcontainer:./Source/UnitTest.DAL/bin/Debug/DSC.Dal.UnitTest.dll /resultsfile:testresults-%BAMBOO_BUILD_NUMBER%.trx
```


author: Matt Block
body: It has to do with the fact that the MSTest Runner script is still returning non-zero result because there is a failing test. Bamboo sees this non-zero result from the script and assumes there is an issue. I'm surprised this doesn't work since Atlassian provides the MSTest Runner script. In my opinion this is a bug so I'm creating a support request with Atlassian.