Package: robotframework-zebrunner
python-agent-robotThe official Zebrunner Robotframework agent provides the reporting functionality. It can automatically track Selenium sessions
and send the info about session details to Zebrunner backend. It can be easily integrated into a project by just installing the library
and adding the configuration file.Including reporting into your project is easy - just install the agent and provide minimal valid configuration for reporting.Installationpip install robotframework-zebrunnerConfigurationAfter the installation, reporting is disabled by default. It won't send any data to the Zebrunner service without a valid configuration.It is currently possible to provide the configuration via:Environment variablesYAML filepyproject.toml,command argumentsare in plans for future releases.Environment variablesThe following configuration parameters are recognized by the agent:REPORTING_ENABLED- enables or disables reporting. The default value istrue.REPORTING_SERVER_HOSTNAME- mandatory if reporting is enabled. It is Zebrunner server hostname. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section;REPORTING_SERVER_ACCESS_TOKEN- mandatory if reporting is enabled. Access token must be used to perform API calls. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Token' section;REPORTING_PROJECT_KEY- optional value. It is the project that the test run belongs to. The default value isDEF. You can manage projects in Zebrunner in the appropriate section;REPORTING_RUN_DISPLAY_NAME- optional value. It is the display name of the test run. The default value isDefault Suite;REPORTING_RUN_BUILD- optional value. It is the build number that is associated with the test run. It can depict either the test build number or the application build number;REPORTING_RUN_ENVIRONMENT- optional value. It is the environment where the tests will run;REPORTING_SEND_LOGS- Sends test logs to Zebrunner. Default:true;REPORTING_NOTIFICATION_NOTIFY_ON_EACH_FAILURE- optional value. Specifies whether Zebrunner should send notification to Slack/Teams on each test failure. The notifications will be sent even if the suite is still running. The default value isfalse;REPORTING_NOTIFICATION_SLACK_CHANNELS- optional value. The list of comma-separated Slack channels to send notifications to. Notification will be sent only if Slack integration is properly configured in Zebrunner with valid credentials for the project the tests are reported to. Zebrunner can send two type of notifications: on each test failure (if appropriate property is enabled) and on suite finish;REPORTING_NOTIFICATION_MS_TEAMS_CHANNELS- optional value. The list of comma-separated Microsoft Teams channels to send notifications to. Notification will be sent only if Teams integration is configured in Zebrunner project with valid webhooks for the channels. Zebrunner can send two type of notifications: on each test failure (if appropriate property is enabled) and on suite finish;REPORTING_NOTIFICATION_EMAILS- optional value. The list of comma-separated emails to send notifications to. This type of notification does not require further configuration on Zebrunner side. Unlike other notification mechanisms, Zebrunner can send emails only on suite finish;REPORTING_RUN_TREAT_SKIPS_AS_FAILURES- optional value. If value is false all test-runs with skipped & passed tests
would be marked as passed.Agent also recognizes.envfile in the resources root folder.Yaml fileAgent recognizes agent.yaml or agent.yml file in the resources root folder. It is currently not possible to configure an alternative file location.Below is a sample configuration file:reporting:enabled:trueproject-key:DEFsend-logs:trueserver:hostname:localhost:8080access-token:<token>notification:notify-on-each-failure:trueslack-channels:automation, dev-teamms-teams-channels:automation, qa-teamemails:example@example.comrun:treat-skips-as-failures:falsedisplay-name:Nightly Regression Suitebuild:1.12.1.96-SNAPSHOTenvironment:TEST-1reporting.enabled- enables or disables reporting. The default value istrue;reporting.server.hostname- mandatory if reporting is enabled. Zebrunner server hostname. Can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section;reporting.server.access-token- mandatory if reporting is enabled. Access token must be used to perform API calls. Can be obtained in Zebrunner on the 'Account & profile' page under the 'Token' section;reporting.project-key- optional value. The project that the test run belongs to. The default value isDEF. You can manage projects in Zebrunner in the appropriate section;reporting.send-logs- Sends test logs to Zebrunner. Default:truereporting.run.display-name- optional value. The display name of the test run. The default value is Default Suite;reporting.run.build- optional value. The build number that is associated with the test run. It can depict either the test build number or the application build number;reporting.run.environment- optional value. The environment in which the tests will run.reporting.notification.notify-on-each-failure- optional value. Specifies whether Zebrunner should send notification to Slack/Teams on each test failure. The notifications will be sent even if the suite is still running. The default value isfalse;reporting.notification.slack-channels- optional value. The list of comma-separated Slack channels to send notifications to. Notification will be sent only if Slack integration is properly configured in Zebrunner with valid credentials for the project the tests are reported to. Zebrunner can send two type of notifications: on each test failure (if appropriate property is enabled) and on suite finish;reporting.notification.ms-teams-channels- optional value. The list of comma-separated Microsoft Teams channels to send notifications to. Notification will be sent only if Teams integration is configured in Zebrunner project with valid webhooks for the channels. Zebrunner can send two type of notifications: on each test failure (if appropriate property is enabled) and on suite finish;reporting.notification.emails- optional value. The list of comma-separated emails to send notifications to. This type of notification does not require further configuration on Zebrunner side. Unlike other notification mechanisms, Zebrunner can send emails only on suite finish;reporting.run.treat_skips_as_failures- optional value. If value is false all test-runs with skipped & passed tests
would be marked as passed.If the required configurations are not provided, there is a warning displayed in logs with the problem description and the names of options
which need to be specified. Parameter names are case-insensitive and can be written in upper and lower registers.Activate listenerThere is two options for activation of Zebrunner listener.Command line argument:robot --listener robotframework_zebrunner.ZebrunnerListenerImport Zebrunner library into your project:Library  robotframework_zebrunner.ZebrunnerLibCollecting captured screenshotSometimes it may be useful to have the ability to track captured screenshots in scope of Zebrunner Reporting. The agent comes
with the API allowing you to send your screenshots to Zebrunner, so that they could be attached to the test.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

With screenshot
    ...
    Create File  ${OUTPUTDIR}/selenium-screenshot-1.png
    Capture Page Screenshot	${OUTPUTDIR}/selenium-screenshot-1.png
    Attach Test Screenshot  ${OUTPUTDIR}/selenium-screenshot-1.png
    ...Additional reporting capabilitiesTracking test maintainerYou may want to add transparency to the process of automation maintenance by having an engineer responsible for
evolution of specific tests or test classes. Zebrunner comes with a concept of a maintainer - a person that can be
assigned to maintain tests. In order to keep track of those, the agent comes with themaintainertag.See a sample test bellow:*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

With maintainer
    [Tags]  maintainer=<maintainer_name>
    Should Be True    5 + 5 == 10Attaching labelsIn some cases, it may be useful to attach some meta information related to a test. The agent comes with a concept of a label.
Label is a key-value pair associated with a test. The key and value are represented by astr. Labels can be attached to
tests and test runs.There is a tag that can be used to attach labels to a test. There is also an API to attach labels during test or execution.
The agent has functions that can be used to attach labels.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

With labels
    [Tags]  labels: some_label=123, other_label=234

    Attach Test Label   some_test_label  value
    Attach Test Run Label  some_test_run_label  value
    Should Be True    5 + 5 == 10The test from the sample above attaches 3 test-level labels.Reverting test registrationIn some cases it might be handy not to register test execution in Zebrunner. This may be caused by very special circumstances of test environment or execution conditions.Zebrunner agent comes with a convenient method revert_registration() from CurrentTest class for reverting test registration at runtime. The following code snippet shows a case where test is not reported on Monday.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

Revert test
     Current Test Revert Registration
     Should Be True    5 + 5 == 10Overriding run attributesThis section contains information on agent APIs allowing to manipulate test run attributes during runtime.Setting build at runtimeAll the configuration mechanisms listed above provide possibility to declaratively set test run build. But there might be cases when actual build becomes available only at runtime.For such cases Zebrunner agent has a special method that can be used at any moment of the suite execution:*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Current Test Run Set Build  0.0.1Setting localeIf you want to get full reporting experience and collect as much information in Zebrunner as its possible, you may want to report the test run locale.For this, Zebrunner agent has a special method that can be used at any moment of the suite execution:*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Current Test Run Set Locale  ENOverriding platformA test run in Zebrunner may have platform associated with the run. If there is at least one initiatedRemotedriver session within the test run, then its platform will be displayed as a platform of the whole test run. Even if subsequentRemotedrivers are initiated on another platform, the very first one will be displayed as the run platform.In some cases you may want to override the platform of the firstRemotedriver session. Another problem is that it is not possible to specifyAPIas a platform.Zebrunner provides two special methods to solve both of these problems:Current Test Run Set PlatformandCurrent Test Run Set Platform Version.
In the example below, the hook sets the API as a platform associated with the current test run.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Current Test Run Set Platform  APICollecting additional artifactsIn case your tests or an entire test run produce some artifacts, it may be useful to track them in Zebrunner.
The agent comes with a few convenient methods for uploading artifacts in Zebrunner and linking them to the currently running test or test run.
Artifacts and artifact references can be attached using functions. Together with an artifact
or artifact reference, you must provide the display name. For the file, this name must contain the file extension that
reflects the actual content of the file. If the file extension does not match the file content, this file will not be
saved in Zebrunner. Artifact reference can have an arbitrary name.Attaching artifact to test*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

With artifact
    Attach Test Artifact  requirements.txt
    ...Attaching artifact reference to test*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

With artifact reference
    Attach Test Artifact Reference  google  https://google.com
    ...Attaching artifact to test run*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

Attach Test Run Artifact  requirements.txtAttaching artifact reference to test run*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib

Attach Test Run Artifact Reference  google  https://google.comArtifact uploading process is performed in the foreground now, so it will block the execution thread while sending.
The background uploading will be available in upcoming releases.Syncing test executions with external TCM systemsZebrunner provides an ability to upload test results to external test case management systems (TCMs) on test run finish. For some TCMs it is possible to upload results in real-time during the test run execution.This functionality is currently supported for TestRail, Xray, Zephyr Squad and Zephyr Scale.TestRailFor successful upload of test run results in TestRail, two steps must be performed:Integration with TestRail is configured and enabled for Zebrunner projectConfiguration is performed on the tests sideConfigurationZebrunner agent has a specialTest Railkeywords:Test Rail Set Suite Id  <id>:   Mandatory. The method sets TestRail suite id for current test run. This method must be invoked before all tests.Test Rail Set Case Id  <id>ortest_rail_case_id=<id>tag
:   Mandatory. Using these mechanisms you can set TestRail's case associated with specific automated test. It is highly recommended using thetest_rail_case_idtag instead of keyword. Use the keyword only for special casesTest Rail Disable Sync:   Optional. Disables result upload. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsTest Rail Include All Test Cases In New Run:   Optional. Includes all cases from suite into newly created run in TestRail. Same as#set_suite_id(str), this method must be invoked before all testsTest Rail Enable Real Time Sync:   Optional. Enables real-time results upload. In this mode, result of test execution will be uploaded immediately after test finish. This method also automatically invokesTest Rail Include All Test Cases In New Run. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsTest Rail Set Run Id  <id>:   Optional. Adds result into existing TestRail run. If not provided, test run is treated as new. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsTest Rail Set Run Name  <name>:   Optional. Sets custom name for new TestRail run. By default, Zebrunner test run name is used. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsTest Rail Set Milestone  <milestone>:   Optional. Adds result in TestRail milestone with the given name. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsTest Rail Set Assignee <assignee>:   Optional. Sets TestRail run assignee. Same asTest Rail Set Suite Id, this keyword must be invoked before all testsBy default, a new run containing only cases assigned to the tests will be created in TestRail on test run finish.ExampleIn the example below, a new run with name "Best run ever" will be created in TestRail on test run finish. Suite id is321and assignee is "Deve Loper". Results of theawesome_test1will be uploaded as result of cases with id10000,10001,10002. Results of theawesome_test2will be uploaded as result of case with id20000.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Run Keywords  Test Rail Set Suite Id  1
...          AND           Test Rail Set Run Name  Best run ever
...          AND           Test Rail Set Assignee  Deve Loper

With Case Id
  [Tags]  test_rail_case_id=1
  ... 

With Another Case id
  Test Rail Set Case Id  2XrayFor successful upload of test run results in Xray two steps must be performed:Xray integration is configured and enabled in Zebrunner projectXray configuration is performed on the tests sideConfigurationZebrunner agent has a specialXraykeywords to control results upload:Xray Set Execution Key  <key>:   Mandatory. The method sets Xray execution key. This method must be invoked before all tests.Xray Set Test Key  <key>orxray_test_key=<key>:   Mandatory. Using these mechanisms you can set test keys associated with specific automated test. It is highly recommended using thexray_test_keytag instead of keyword. Use the keyword only for special casesXray Disable Sync:   Optional. Disables result upload. Same asXray Set Execution Key, this method must be invoked before all testsXray Enable Real Time Sync:   Optional. Enables real-time results upload. In this mode, result of test execution will be uploaded immediately after test finish. Same asXray Set Execution Key, this method must be invoked before all testsBy default, results will be uploaded to Xray on test run finish.ExampleIn the example below, results will be uploaded to execution with keyZBR-42. Results of theawesome_test1will be uploaded as result of tests with keyZBR-10000,ZBR-10001,ZBR-10002. Results of theawesome_test2will be uploaded as result of test with keyZBR-20000.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Xray Set Execution Key  ZBR-42


With test key
  [Tags]  xray_test_key=ZBR-10000  xray_test_key=ZBR-10001  xray_test_key=ZBR-10002
  ... 

With Another Test Key
  Xray Set Test Key  ZBR-20000ZephyrFor successful upload of test run results in Zephyr two steps must be performed:Zephyr integration is configured and enabled in Zebrunner projectZephyr configuration is performed on the tests sideDescribed steps work for both Zephyr Squad and Zephyr Scale.ConfigurationZebrunner agent has a specialZephyrkeywords to control results upload:Zephyr Set Test Cycle Key  <key>:   Mandatory. The method sets Zephyr test cycle key. This method must be invoked before all tests.Zephyr Set Jira Project Key  <key>:   Mandatory. Sets Zephyr Jira project key. Same asZephyr Set Test Cycle Key, this method must be invoked before all testsZephyr Set Test Case Key  <key>orzephyr_test_case_key=<key>:   Mandatory. Using these mechanisms you can set test case keys associated with specific automated test. It is highly recommended using thezephyr_test_case_key=1tag instead of keyword. Use the keyword only for special casesZephyr Disable Sync:   Optional. Disables result upload. Same asZephyr Set Test Cycle Key, this method must be invoked before all testsZephyr Enable Real Time Sync:   Optional. Enables real-time results upload. In this mode, result of test execution will be uploaded immediately after test finish. Same asZephyr Set Test Cycle Key, this method must be invoked before all testsBy default, results will be uploaded to Zephyr on test run finish.ExampleIn the example below, results will be uploaded to test cycle with keyZBR-R42from project with keyZBR. Results of theawesome_test1will be uploaded as result of tests with keyZBR-T10000,ZBR-T10001,ZBR-T10002. Results of theawesome_test2will be uploaded as result of test with keyZBR-T20000.*** Settings ***
Library     robotframework_zebrunner.ZebrunnerLib
Suite Setup  Run Keywords  Zephyr Set Test Cycle Key  ZBR-R42
...          AND           Zephyr Set Jira Project Key  ZBR


With test key
  [Tags]  zephyr_test_case_key=ZBR-T10000  zephyr_test_case_key=ZBR-T10001  zephyr_test_case_key=ZBR-T10002
  ... 

With Another Test Key
  Zephyr Set Test Case Key  ZBR-T20000


#conftest.py
from pytest_zebrunner.tcm import Zephyr

@pytest.hookimpl(trylast=True)
def pytest_sessionstart(session: Session) -> None:
    Zephyr.set_test_cycle_key("ZBR-R42");
    Zephyr.set_jira_project_key("ZBR");Selenium WebDriver supportThe Zebrunner test agent is capable of tracking tests along with remote Selenium WebDriver sessions.Capturing session artifactsZebrunner supports 3 types of test session artifacts:Video recordingSession logVNC streamingTest agent itself does not capture those artifacts since it has no control over underlying Selenium Grid implementation, however, it is possible to attach appropriate artifact references by providing specially designed set of driver session capabilities (enabling capabilities) - see the table below for more details. Only theTruevalue is considered as trigger to save the link.ArtifactDisplay nameEnabling capabilityDefault referenceReference overriding capabilityVideo recordingVideoenableVideoartifacts/test-sessions/<session-id>/video.mp4videoLinkSession logLogenableLogartifacts/test-sessions/<session-id>/session.loglogLinkVNC streamingenableVNC<provider-integration-host>/ws/vnc/<session-id>vncLinkThedisplay nameis the name of the artifact that will be displayed on Zebrunner UI. This value is predefined and unfortunately can not be changed at the moment.Thedefault referenceis a reference to a location, where artifact isexpected to residein S3-compatible storage once created by test environment - it is important that it stays in sync with test environment configuration. It is possible to override references if needed by providingreference overriding capabilities. Note, that special<session-id>placeholder is supported and can be used as part of the value of those capabilities allowing runtime session id (generated by WebDriver) to be included into actual reference value.VNC streamingVNC is an artifact of a special type. They don't have a name and are not displayed among other artifacts. They are displayed in the video section on Zebrunner UI during session execution and are dropped off on session close.Default reference to the VNC streaming is based onprovidercapability. Value of this capability will be converted to preconfigured integration fromTest Environment Providergroup. The resolved integration must have a filled in URL property and be enabled in order to save the link to VNC streaming. The<provider-integration-host>placeholder of the default link will be replaced by the host of the obtained integration URL. Also, thehttpprotocol in the VNC streaming url will be automatically replaced byws, andhttpsprotocol will be replaced bywss. Currently, we only support Selenium, Zebrunner and MCloud integrations.ContributionTo check out the project and build from the source, do the following:git clone git://github.com/zebrunner/python-agent-pytest.git
cd python-agent-pytestLicenseZebrunner reporting agent for PyTest is released under version 2.0 of theApache License.