Selenium and Bamboo
Asked by Bernard O'Flynn:

I've seen a couple of people posting about this on StackExchange and I have similar questions and thought I'd post it here.  

We want to run selenium html tests in our bamboo build and integrate the results with Bamboo.  

I know that Atlassian uses Selenium to test Jira, etc. so I was hoping that there was a plug-in or some automated way to integrate Bamboo and Selenium  

In particular taking the html result file from Selenium and displaying it in Bamboo.  

Does anyone know how we can do this? Apparently Bamboo requires its results in jUnit format?


Answers:
author: Gary Weaver
body: You could call Selenium RC with Junit: <http://www.seleniumwiki.com/tag/junit/>

That should produce results in Junit format.

comments: author: Bernard O'Flynn
body: we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?


author: Gary Weaver
body: You might want to remove this answer and instead add comments to each of the answers individually to keep comments about answers together.


author: Bernard O'Flynn
body: we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?


author: Bernard O'Flynn
body: Good point - I've deleted my post and added things as comments instead...

we're already adding the html result as an artifact, but I would have hoped that we could integrate it into Bamboo like other build servers allow you to do.

A lot of our tests are in the html format as they were done by business testers. My understanding of Selenium is that to call it using jUnit you would need to convert them all to java so that's not an option. Please correct me if I'm wrong?


author: Joe Clark
body: This is the approach we take at Atlassian - most of our Selenium tests are invoked via JUnit and/or TestNG.

We don't have (m)any business users who write acceptance tests/functional tests.


author: Gary Weaver
body: You could convert selenium tests to junit, even though you said you didn't want to:

* In IDE use File -\> Export Test Case As
* Batch conversion: <http://stackoverflow.com/questions/5951136/automated-way-of-converting-selenium-html-tests-to-junit> and plugin at <https://addons.mozilla.org/en-US/firefox/addon/test-suite-batch-converter-sel/?src=api>

Atlassian's Nick Menere and Scott Farquhar in [Selenium - Is It Worth the Pain](http://blogs.atlassian.com/developer/2007/08/selenium_is_the_pain_worth_it.html) said that Atlassian uses Bamboo + Cargo + Selenium for its tests. More here: [Confluence has a Selenium Build](http://blogs.atlassian.com/developer/2008/05/confluence_has_a_selenium_buil.html "Confluence has a Selenium Build") and there are [more posts on Selenium in Atlassian Developer Blogs](http://www.atlassian.com/search/?resource=blogs.atlassian.com&query=selenium)

There was also a [talk about Selenium and Bamboo](http://www.atlassian.com/summit/2010/presentations/development-speed/automated-testing-selenium-bamboo.jsp) in Atlassian Summit 2010 by Dyn Inc.

You could also use the ant task or maven plugin, but I don't know how results would be integrated into bamboo results:

* <http://mojo.codehaus.org/selenium-maven-plugin/>
* <http://release.seleniumhq.org/selenium-remote-control/0.9.0/doc/server/org/openqa/selenium/server/ant/SeleneseAntTask.html>

You could probably also do something like this calling as bamboo script, but I don't know how results would be integrated into bamboo results:

* <http://stackoverflow.com/questions/1231975/i-want-to-run-selenium-test-case-file-from-command-line>

comments: author: Bernard O'Flynn
body: We're a .Net team so while we understand the syntax of Java, converting all the html tests that were created by our QA team to Java isn't really an option.

The ideal solution is to either conver the html test result file which is just html into the jUnit format OR enable Bamboo to read the html test result file.

It seems to me that Bamboo has a bit of a gap here compared to other CI products (like CC.Net) which allow easy plugging in of results files to builds.


author: Gary Weaver
body: For .Net, NUnit works with Bamboo and Selenium RC can be used with NUnit:

* <http://www.theautomatedtester.co.uk/tutorials/selenium/selenium_two_csharp_nunit.htm>
* <http://lostechies.com/christaylor/2009/02/10/how-to-configure-selenium-rc-for-use-in-c-nunit-tests/>


author: Gary Weaver
body: Also [create a ticket](https://jira.atlassian.com/) and tell Atlassian what you'd like them to do to Bamboo to make it easier and post the link here so others can vote on it.


author: Gary Weaver
body: And if you wanted to write an NUnit test that started Selenium tests via command-line, you could [use Process in .Net](http://stackoverflow.com/questions/206323/how-to-execute-command-line-in-c-get-std-out-results). In JUnit/java you could use Runtime.getRuntime().exec("my command"). Also, here is a [note](http://www.jadeliquid.com/liquidtest/docs/doku.php?id=qa:selenium:running_selenium_tests_from_the_command_line_on_windows:home) about running Selenium from command-line in Windows.


author: Bernard O'Flynn
body: Thanks for all these. We've looked at all of that and it still means that our QA team (mostly business type users) would need to learn c# - its just not going to happen.

So the end result is that Bamboo (like [cc.net](http://cc.net) and TeamCity and Hudson all do) should allow you to configure it so that it can display html result files.

I've opened a request for this <https://jira.atlassian.com/browse/BAM-9179>

Please vote for it.


author: Gary Weaver
body: Sure thing, and it would help my karma if you voted my answer/comments up. Thanks!


author: Stefan Kohler
body: Would the Sauce OnDemand plugin do what you want? <https://plugins.atlassian.com/plugin/details/30134>

As last resort, you can always add the HTML file as an artifact to your build, that way it isn't lost after a build and easy accessable for viewing.

comments: author: Bernard O'Flynn
body: The Sauce OnDemand plugin isn't free and runs tests in the cloud which isn't an option for us.


author: EddieW
body: Will NUnit - <http://www.nunit.org/> not meet your needs?

Along with Bamboo's provided .NET plugin you should be able to export Selenium test cases to an NUnit format and run those right in bamboo along side your unit tests. It is exactly as a Java project would go the route, but using .NET.

<http://codebetter.com/jeremymiller/2006/05/15/automated-web-testing-with-selenium-driven-by-net/>

comments: author: Gary Weaver
body: Eddie, appreciate your answer. Just to note, if you down mod other people's answers like mine and Stefan's for this question, please provide a reason why in the comments so that we and others know what was wrong with the answer. Note that everything including changes to karma are public, also.


Tags: .net

selenium