Package: pytest-railflow-testrail-reporter
pytest-railflow-testrail-reporterpytest-railflow-testrail-reporteris a custom reporter which generates JSON report file containing some TestRail-related data.The reporter helps you to integrate your pytest test with TestRail easily by producing JSON test report files which can be uploaded intoTestRailby using powerfulRailflow NPM CLItool or native plugins forJenkins and TeamCity.RequirementsIn order to use pytest-railflow-testrail-reporter plugin, following
prerequisites should be met.Python 2.7, 3.4 or greaterPytestInstallationUsing PipTo install the pytest-railflow-testrail-reporter plugin using pip,run
the following command in terminal:pip install pytest-railflow-testrail-reporterThis will install the plugin to the system.UsageTo add Railflow markers to the tests '@pytest.mark.railflow' marker is used.The plugin supports the following attributes:Railflow configuration attributes descriptionAttribute NameDescriptiontitleName of the test suite or test casecase_typeCase type in TestRail, e.g.: Automated, Compatibility, etc...case_priorityCase type in TestRail, e.g.: Critical, High, etc...case_fieldsValues for custom case fields in TestRail in the following format: ['field1=value1','field2=value2']result_fieldsValues for result fields in TestRail in the following format: ['field1=value1','field2=value2']jira_idsJira IDs.These values will be populated as a case field 'refs'. Should input as an array of strings, e.g.: ['jid1','jid2']testrail_idsIDs of test cases in TestRail. Should input as an array of integers, e.g.: [1,2,3]smart_failure_assignmentArray of TestRail users to automatically assign failed test cases. Should input as a string array, e.g.: ['aaa@test.com','bbb@test.com']These attributes can be either used in class level or function level.Function level attributesClass level attributesjira_idscase_fieldscase_fieldsresult_fieldsresult_fieldscase_typetestrail_idscase_prioritycase_typesmart_failure_assignmentcase_prioritytitletitleTo run the test, execute the following command in the terminal from the test directory:pytest --jsonfile output.jsonAdding arbitrary attachments into test reportThetestrail_add_screenshotfixture can be used to add arbitrary attachments into the test report:deftest_add(testrail_add_screenshot):a=3b=2c=a+bassertc==5testrail_add_screenshot("screenshot path")Adding test steps to test reportThetestrail_add_test_stepfixture can be used to add test steps into test report:The fixture has the following parameters:OrderDescriptionIs Mandatory1Step NameYes2Status NameYes3Actual ValueNo4Expected ValueNodeftest_multiply(testrail_add_test_step):res=8*4testrail_add_test_step("multiply 8 and 4","PASSED",res,"32")ExamplesExample 1 : pytest tests within a classInstall 'pytest-railflow-testrail-reporter' .$pipinstallpytest-railflow-testrail-reporterAdd pytest test with Railfllow marker on class level.test_calculation.pyimportpytest

@pytest.mark.railflow(jira_ids=["301219"],case_fields=[{"name":"Required text field","value":"Class"}],result_fields=[{"name":"Custom field","value":"Result from class"}],case_type="Automated",case_priority="Critical",smart_failure_assignment=["user1@gmail.com, user2@gmail.com"])classTestClass:deftest_add(self):a=3b=2c=a+bassertc==5deftest_subtract(self):a=3b=2c=a-bassertc==0@pytest.mark.railflow(title="method title",jira_ids=["11111"],case_fields=[{"name":"Required text field","value":"method"}],result_fields=[{"name":"Custom fieLD","value":"Result from method"}],case_type="Compatibility",case_priority="High",smart_failure_assignment=["user3@gmail.com"])@pytest.mark.parametrize("a,b,c",[(22,11,2),(64,8,8),(9,3,3)])deftest_divide(self,a,b,c):asserta/b==cRun tests and generate reportpytest--jsonfileoutput.jsontest_calculation.pyView report fileReport file generated at the same directory where the test executed.output.json[{"class_name":"TestClass","markers":"","file_name":"test_sample2","attachments":[],"tests":[{"test_name":"test_add","details":null,"markers":"","result":"PASSED","duration":0.00012004900054307655,"timestamp":"2022-03-09T15:53:53","message":null},{"test_name":"test_subtract","details":null,"markers":"","result":"FAILED","duration":0.00017888799993670546,"timestamp":"2022-03-09T15:53:53","message":"self = <test_sample2.TestClass object at 0x7f659f8af310>\n\n    def test_subtract(self):\n        a = 3\n        b = 2\n        c = a - b\n>       assert c == 0\nE       assert 1 == 0\n\ntest_sample2.py:33: AssertionError"},{"test_name":"test_divide","details":null,"markers":"parametrize","parameters":[{"name":"a","value":22},{"name":"b","value":11},{"name":"c","value":2}],"result":"PASSED","duration":0.0001058529996953439,"timestamp":"2022-03-09T15:53:53","message":null,"railflow_test_attributes":{"title":"method title","jira_ids":["11111"],"case_fields":[{"name":"Required text field","value":"method"}],"result_fields":[{"name":"Custom field","value":"Result from method"}],"case_type":"Compatibility","case_priority":"High","smart_failure_assignment":["user3@gmail.com"]}},{"test_name":"test_divide","details":null,"markers":"parametrize","parameters":[{"name":"a","value":64},{"name":"b","value":8},{"name":"c","value":8}],"result":"PASSED","duration":0.00013493100050254725,"timestamp":"2022-03-09T15:53:53","message":null,"railflow_test_attributes":{"title":"method title","jira_ids":["11111"],"case_fields":[{"name":"Required text field","value":"method"}],"result_fields":[{"name":"Custom field","value":"Result from method"}],"case_type":"Compatibility","case_priority":"High","smart_failure_assignment":["user3@gmail.com"]}},{"test_name":"test_divide","details":null,"markers":"parametrize","parameters":[{"name":"a","value":9},{"name":"b","value":3},{"name":"c","value":3}],"result":"PASSED","duration":0.00020506200235104188,"timestamp":"2022-03-09T15:53:53","message":null,"railflow_test_attributes":{"title":"method title","jira_ids":["11111"],"case_fields":[{"name":"Required text field","value":"method"}],"result_fields":[{"name":"Custom field","value":"Result from method"}],"case_type":"Compatibility","case_priority":"High","smart_failure_assignment":["user3@gmail.com"]}}],"railflow_test_attributes":{"jira_ids":["301219"],"case_fields":[{"name":"Required text field","value":"Class"}],"result_fields":[{"name":"Custom field","value":"Result from class"}],"case_type":"Automated","case_priority":"Critical","smart_failure_assignment":["user1@gmail.com, user2@gmail.com"]}}]Install Railflow CLInpminstallrailflowRun Railflow CLI and upload test results into TestRailnpxrailflow-kABCDE-12345-FGHIJ-67890-urlhttps://testrail.your-server.com/-utestrail-username-ptestrail-password-pr"Railflow Demo"-pathsection1/section2-fpytest-railflow-rpytest_example/*.json-smpathWhere:KeyDescriptionExample-k, --key(online activation) License key. Can be set with RAILFLOW_LICENSE environment variable-k XXXXX-XXXXX-XXXXX-XXXXX-url, --urlTestRail instance URL-urlhttps://example.testrail.io-u, --usernameTestRail username. Can be set with RAILFLOW_TR_USER environment variable-u test-username-p, --passwordTestRail password or API Key. Can be set with RAILFLOW_TR_PASSWORD environment variable-p XtpHXiPLEODyhF-pr, --projectTestRail project name-pr "example project"-path, --test-pathTestRail test cases path-path "Section1/subsection2/ShoppingCart-f, --formatReport format: 'pytest-railflow' (case insensitive)-f junit-r, --report-filesThe file path(s) to the test report file(s) generated during the build. User can pass multiple values separated with spaces. Ant-style patterns such as**/reports/*.xmlcan be used.E.g. usetarget/reports/*.xmlto capture all XML files intarget/reportsdirectory.-r target/surefire-reports/*.xml target/failsafe-reports/*.xml-sm, --search-modeSpecifies the test case lookup algorithm.name:search for test case matching the name within the entire test suite. If test case found, update the test case. If test case not found, create a new test case within specified-pathpath:search for test case matching the name within the specified-path. If test case found, update the test case. If test case not found, create a new test case within specified-path-sm pathPlease seeRailflow NPM documentationfor the all the details about Railflow CLI options.View results in TestRailTest runTest result detailsParameterized tests detailsExample 2 : pytest tests without a class (using pytest splinter)Install 'pytest-splinter' .$pipinstallpytest-splinterAlso, make sure that chromedriver or Firefox driver is installed in your system for running splinter.For Firefox driver: [Splinter Documentation on Firefox Driver](https://splinter.readthedocs.io/en/latest/drivers/firefox.html)

For Chrome driver : [SPlinter Documentation on Chrome Driver](https://splinter.readthedocs.io/en/latest/drivers/chrome.html)Add pytest test.test_browser.pyimportpytest

@pytest.mark.railflow(jira_ids=["301219"],case_fields=[{"name":"Required text field","value":"Class"}],result_fields=[{"name":"Custom field","value":"Result from class"}],case_type="Automated",case_priority="Critical",smart_failure_assignment=["user1@gmail.com, user2@gmail.com"])deftest_google(browser):"""Test using real browser."""url="https://www.google.com"browser.visit(url)browser.fill('q','splinter - python acceptance testing for web applications')# Find and click the 'search' buttonbutton=browser.find_by_name('btnK')# Interact with elementsbutton.click()assertbrowser.is_text_present('splinter.cobrateam.info'),"splinter.cobrateam.info wasn't found... We need to"' improve our SEO techniques'Run tests and generate report (using chrome driver)
if '--splinter-webdriver' argument is not provided , it will use firefox as default web driverpytest--splinter-webdriverchrome--jsonfileoutput.jsontest_browser.pyView report fileoutput.json[{"class_name":null,"test_name":"test_google","details":"Test using real browser.","markers":"","result":"FAILED","duration":8.737954783000077,"timestamp":"2022-03-06T10:51:51","message":"browser = <splinter.driver.webdriver.chrome.WebDriver object at 0x7f55c6f50670>\n\n    @pytest.mark.railflow(\n         jira_ids=[\"301219\"],\n        case_fields=[\n            {\n                \"name\": \"ReQuired text field\",\n                \"value\": \"Class\"\n            }\n        ],\n        result_fields=[\n            {\n                \"name\": \"Custom fIeLD\",\n                \"value\": \"Result from class\"\n            }\n        ],\n        case_type=\"Railflow\",\n        case_priority=\"Critical\",\n        smart_failure_assignment=[\"user1@gmail.com, user2@gmail.com\"]\n    )\n    def test_google(browser):\n        \"\"\"Test using real browser.\"\"\"\n        url = \"https://www.google.com\"\n        browser.visit(url)\n        browser.fill('q', 'splinter - python acceptance testing for web applications')\n        # Find and click the 'search' button\n        button = browser.find_by_name('btnK')\n        # Interact with elements\n        button.click()\n>       assert browser.is_text_present('splinter.cobrateam.info'), \"splinter.cobrateam.info wasn't found... We need to\"\nE       AssertionError: splinter.cobrateam.info wasn't found... We need to\nE       assert False\nE        +  where False = <bound method BaseWebDriver.is_text_present of <splinter.driver.webdriver.chrome.WebDriver object at 0x7f55c6f50670>>('splinter.cobrateam.info')\nE        +    where <bound method BaseWebDriver.is_text_present of <splinter.driver.webdriver.chrome.WebDriver object at 0x7f55c6f50670>> = <splinter.driver.webdriver.chrome.WebDriver object at 0x7f55c6f50670>.is_text_present\n\ntest_browser.py:30: AssertionError","file_name":"test_browser","attachments":[" /home/projects/pytest/example_tests/railflow_pytest_examples/railflow_pytest_examples.test_browser/test_google-browser.png"],"railflow_test_attributes":{"jira_ids":["301219"],"case_fields":[{"name":"Required text field","value":"Class"}],"result_fields":[{"name":"Custom field","value":"Result from class"}],"case_type":"Railflow","case_priority":"Critical","smart_failure_assignment":["user1@gmail.com, user2@gmail.com"]}}]Run Railflow CLI and upload test results into TestRailnpxrailflow-kABCDE-12345-FGHIJ-67890-urlhttps://testrail.your-server.com/-utestrail-username-ptestrail-password-pr"Railflow Demo"-pathsection1/section2-fpytest-railflow-rpytest_example/*.jsonPlease checkexamplesfor more information and sample tests.LicenseThis software is licensed under theMIT licenseSeeLicense filefor more information.