Package: robotsuite
Python unittest test suite for Robot FrameworkThis is an experimental package
for wrapping Robot Framework test suites into Python unittest suites
to make it possible to run Robot Framework tests
asplone.testing’s layered test suites:importunittestfromplone.testingimportlayeredfromrobotsuiteimportRobotTestSuitefrommy_package.testingimportACCEPTANCE_TESTINGdeftest_suite():suite=unittest.TestSuite()suite.addTests([layered(RobotTestSuite('mysuite.txt'),layer=ACCEPTANCE_TESTING),])returnsuiteRobotTestSuitesplits Robot Framework test suites into separate
unittest test cases so that Robot will be run once for every test
case in every test suite parsed from the given Robot Framework
test suite.
Because of that, each Robot will generate a separate test report
for each test.
Each report will have it’s own folder,
which are created recursively
reflecting the structure of the given test suite.RobotTestSuite’s way of wrapping tests into
unittest’s test suite is similar to how doctest-module’s
DocTestSuite does its wrappings.
See the documentation of DocTestSuite for
possible common parameters (e.g. for how to pass a test suite from a
different package).The main motivation behind this package is to make
Robot Framework support existing test fixtures and test isolation
when testingPlone.
Yet, this should help anyone wanting to use Robot Framework withzope.testrunneror other Python unittest compatible test runner.If this works for you, please contribute at:http://github.com/collective/robotsuite/Setting robot variables from environment variablesRobot Framework supports overriding test variables from command-line, which
is not-available when running tests as robotsuite-wrapped with other test
runners. That’s why robotsuite supports settings variables as environment
variables so that everyROBOT_-prefixed environment variable will be
mapped into corresponding test variable without theROBOT_-prefix.Declaring tests non-critical by given set of tagsNoteCriticality is no-longer supported in Robot Framework >= 4.0 and has been
replaced with SKIP status. Robotsuite does not take a stance on SKIP status yet.Robot Framework supports declaring tests with given tags asnon-criticalto prevent their failing to fail the complete build on CI. This is supported
as keyword argument forRobotTestSuiteas follows:deftest_suite():suite=unittest.TestSuite()suite.addTests([layered(RobotTestSuite('mysuite.txt',noncritical=['non-critical-tag']),layer=ACCEPTANCE_TESTING),])returnsuiteSetting zope.testrunner-levelzope.testrunnersupports annotating test suites with levels to avoid
slow test being run unless wanted:deftest_suite():suite=unittest.TestSuite()suite.addTests([layered(RobotTestSuite('mysuite.txt'),layer=ACCEPTANCE_TESTING),])suite.level=10returnsuiteRetry failing testsYou can retry a failed test.
This can be useful for flaky robot browser tests.
Warning: this may not be good for all types of test.
For example any changes that were done in the test until the first failure, may persist.You can enable retries in two ways:Set an environment variableROBOTSUITE_RETRY_COUNT=X.Override this by passingretry_count=Xto aRobotTestSuitecall.The default is zero: no retries.
The retry countexcludesthe original try.deftest_suite():suite=unittest.TestSuite()suite.addTests([robotsuite.RobotTestSuite('test_example.robot',retry_count=3),robotsuite.RobotTestSuite('test_variables.robot'),robotsuite.RobotTestSuite('test_setups',retry_count=2)])returnsuiteAppending test results to existing test reportWhen running Robot Framework through robotsuite, its test reports are created
into the current working directory with filenamesrobot_output.xml,robot_log.htmlandrobot_report.html. The default behavior is to
override the existingrobot_output.xml(and also the other report files
generated from that).To merge test results from separate test runs into the same test report, set
environment variableROBOTSUITE_APPEND_OUTPUT_XML=1to prevent robotsuite
from overriding the existing test results, but to always append to the existingrobot_output.xml.Filtering test execution errorsSet environment variableROBOTSUITE_LOGLEVEL=ERRORto filter all top levelTest Execution Errorsbelow the given log level (e.g. ERROR) from the merged
test report. This is useful when unnecessary warnings are leaking from the
tested code into Robot Framework logs.Including or skipping all RobotTestSuite-wrapped testsRobot Framework is often used withSelenium2Libraryto write acceptance test
using the Selenium-framework. Yet, because those test may be slow to run, one
might want sometimes (e.g. on CI) to run everything except the robotsuite
wrapped tests, and later only the robotsuite wrapped tests.This can be achieved for sure, with injecting a custom string into the names
of robotsuite-wrapped tests withROBOTSUITE_PREFIX-environment variable
and then filter the test with that string.E.g. run everything except the robotsuite wrapped tests with:$ROBOTSUITE_PREFIX=ROBOTSUITEbin/test--all-t\!ROBOTSUITEand the other way around with:$ROBOTSUITE_PREFIX=ROBOTSUITEbin/test--all-tROBOTSUITERe-using test suites from other packagesSometime it could be useful to re-use acceptance test from some upstream
package to test your slightly tailored package (e.g. with a custom theme).
This can be done with by defining the test lookup location withpackage-keyword argment forRobotTestSuite:deftest_suite():suite=unittest.TestSuite()suite.addTests([layered(leveled(robotsuite.RobotTestSuite('robot',package='Products.CMFPlone.tests'),),layer=PLONE_APP_MOSAIC_NO_PAC_ROBOT),])returnsuiteChangelog2.3.2 (2023-09-09)Fix to support screenshots / images below sub directories
[datakurre]2.3.1 (2022-05-23)Let tests be critical by default.
FixesPlone issue 3537.  [maurits]2.3.0 (2022-05-02)Add support for RF 4 and RF 5
[Asko Soukka]Drop support in criticality witH RF 4 or later
[Asko Soukka]Restore support for RF 3.2
[Asko Soukka]2.2.1 (2020-02-14)Revert support for Robot Framework 3.2b1 due to observee
regression between 2.0.0 and 2.1.1
[datakurre]2.1.1 (2020-02-13)Add to release as universal wheel
[datakurre]2.1.0 (2020-02-13)Add optional support for retrying a failed test.
Set environment variableROBOTSUITE_RETRY_COUNT=X.
Or override this by passingretry_count=Xto aRobotTestSuitecall.
Default is zero: no retries.  The retry countexcludesthe original try.
[maurits]Add support for Robot Framework 3.2b1
[datakurre]Drop robotframework-python3 requirement,
robotframework itself is already python 3 compatible.
[gforcada]2.0.0 (2016-12-22)Do not support python 2.6 anymore
[gforcada]Remove unittest2 references
[gforcada]1.7.0 (2015-07-23)Add support for Robot Framework 2.9
[datakurre]1.6.1 (2014-10-01)Add ROBOTSUITE_LOGLEVEL environment variable for filtering text execution
errors during log merge
[datakurre]1.6.0 (2014-06-29)Add support for ROBOTSUITE_PREFIX-env for injecting custom string into all
robotsuite test names
[datakurre]Add UTF-8 support for robot variables [#6]
[Tom Gross]
[Vincent Fretin]Added Python 3.4 support
[Vincent Fretin]1.5.0 (2014-04-13)Restore original behavior to override the exiting robot_output.xml. Set
environment variable ROBOTSUITE_APPEND_OUTPUT_XML=1 to append test results to
the existing robot_output.xml.
[datakurre]1.4.3 (2014-01-27)Fix issue where test suites with sub-suites were not supported
[datakurre]1.4.2 (2013-11-22)Fix issue where suite setups were run for suites without tests
wit robotframework >= 2.8.0
[datakurre]1.4.1 (2013-10-13)Fix regression in 1.4.0, which dropped relative path names from robot test
cases
[datakurre]Fix RobotTestCase to mimic DocTestCase enough to allow plone.testing to set
‘layer’ for it
[datakurre]Fix to set given test ‘setUp’ and ‘tearDown’ methods properly as
instancemethods (will break existing methods not accepting ‘self’ parameters)
[datakurre]1.4.0 (2013-06-19)Supports only roboframework >= 2.8.
Use robotsuite 1.3.x with robotframework 1.7.x
[datakurre]1.3.3 (2013-06-05)Fix to support custom screenshot filenames with paths [fix #2]
[datakurre]1.3.2 (2013-06-03)Remove screenshot capturing from functional test
[datakurre]1.3.1 (2013-06-03)Pin robotframework < 2.8rc1, because robotsuite is not yet
compatible with robotframework 2.8.x
[datakurre]1.3.0 (2013-04-09)Allow passing arguments critical/noncritical
to only raise an AssertionError if the test is critical
[reinhardt]1.2.2 (2013-04-08)Fix to copy all screenshots (not only selenium-screenshot*.png) when mergin
test reports1.2.1 (2013-03-08)Revert setting the default zope.testrunner level and support setting it via
environment variable ROBOTSUITE_LEVEL (e.g. ROBOTSUITE_LEVEL=2).1.2.0 (2013-03-08)Override test suite __module__ test case id() to support
collective.xmltestreportAdd support for zope.testrunner test suite levels (with default value of 2 to
allow easily to run all but robot tests with zope.testrunner1.1.0 (2013-02-20)Change of behavior to not remove existingrobot_output.xml, but always
just merge new test data into it.robot_output.xmlcan be reseted by
deleting it manually between tests.1.0.4 (2013-02-19)Fix issue in merging of test reports from tests form two or more separate
test suites1.0.3 (2013-02-18)Fix the example test to read ZSERVER_PORT -environment variable properly1.0.2 (2013-02-09)Fix to replace spaces in test report folder names1.0.1 (2013-02-09)Add newline before stdout dump on failing tests1.0.0 (2013-02-08)Stop replacing spaces with underscores in testnames0.9.0 (2012-09-21)Added injecting logged errors into captured stdout to be dumped with robot
output when a test fails.0.8.0 (2012-09-13)Added support for setting pybot cli-variables from ROBOT_-prefixed
environment variables.0.7.0 (2012-09-11)Implemented custom merge for Robot Framework reports to create better
concatenated test report and fully support Robot Framework plugin for
Jenkins.Fixed to copy (selenium-)screenshots into the root test directory to
make it easier to publish them in Jenkins.0.6.0 (2012-08-21)Fixed screenshot paths in concatenated report.Fixed test names to include suite filename instead of classname.Fixed to allow dash in normalized test names.0.5.0 (2012-07-23)Included robotframework test tags as part of tests’ string representation
to allow test be filtered with tags (e.g. withzope.testrunner).Added creation of concatenated log file.0.4.0 (2012-06-04)Fixed to rename test method of each test after the actual robot framework
test run by the test case.0.3.1 (2012-06-03)Updated README.0.3.0 (2012-06-03)Enabled robotframework reports for tests. Each test suite and each test
will have their own outputdir, recursively.0.2.0 (2012-06-02)Support for directories as (recursive) suites.0.1.0 (2012-05-30)Proof of concept.