Package: pytest-analyzer
Testomat.io plugin for pytestuses Testomat.io API:https://testomatio.github.io/check-tests/https://testomatio.github.io/reporter/Installationpipinstallpytest-analyzerconfigurationCreate environment variableTESTOMATIOand set your testomat.io API key.
Linux:exportTESTOMATIO=<key>Windows (cmd):setTESTOMATIO=<key>Run groups parameterThere is environment variableTESTOMATIO_RUNGROUP_TITLEthat can be used to specify run tests with specific group title.pytest.iniIn case you are using private testomat.io service, createpytest.inifile in your project root directory. Specify
testomat.io url in it[pytest]testomatio_url=https://app.testomat.ioUsageRun pytest with analyzer add parameter to analyze your tests, send them to testomat.io and get back test id. Tests will
not be executedpytest--analyzeraddRun pytest with analyzer remove parameter to remove all test ids from your tests. Tests will not be executedpytest--analyzerremoveRun pytest with analyzer sync parameter to execute tests and send the execution status to testomat.io.Sync can be executed even without marking tests with ids. If testomat.io failed to match tests by title, it will create
new tests for the runpytest--analyzersyncRun pytest with analyzer debug parameter to get test data collected in metadata.json filepytest--analyzerdebugSubmitting Test Run Environmentto configure test environment, you can use additional option:pytest--analyzersync--testRunEnvwindows11,chrome,1920x1080Eny environments used in test run should be placed in comma separated list, NO SPACES ALLOWED.Submitting Test ArtifactsTestomat.io does not store any screenshots, logs or other artifacts.In order to manage them it is advised to use S3 Buckets (GCP Storage).https://docs.testomat.io/usage/test-artifacts/Analyser needs to be aware of the cloud storage credentials.
There are two options:EnableShare credentials with Testomat.io Reporteroption in testomat.io Settings -> Artifacts.Use environment variablesACCESS_KEY_ID, SECRET_ACCESS_KEY, ENDPOINT, BUCKETYou would need to decide when you want to upload your test artifacts to cloud storageUpload page screenshot when test fails, using fixturesreference# content of conftest.pyimportpytestfromtypingimportDictfrompytestimportStashKey,CollectReportfromplaywright.sync_apiimportPagephase_report_key=StashKey[Dict[str,CollectReport]]()@pytest.hookimpl(wrapper=True,tryfirst=True)defpytest_runtest_makereport(item,call):rep=yielditem.stash.setdefault(phase_report_key,{})[rep.when]=repreturnrep@pytest.fixture(scope="function")defhandle_artifacts(page:Page,request):yieldreport=request.node.stash[phase_report_key]if("call"notinreport)orreport["setup"].failedorreport["call"].failed:random_string=''.join(random.choices(string.ascii_letters+string.digits,k=8))filename=f"{random_string}.png"screenshot_path=os.path.join(artifacts_dir,filename)page.screenshot(path=screenshot_path)# file_path - required, path to file to be uploaded# file_bytes - required, bytes of the file to be uploaded# key - required, file name in the s3 bucket# bucket_name - optional,name of the bucket to upload file to. Default value is taken from Testomatio.ioartifact_url=pytest.testomatio.upload_file(screenshot_path,filename)# or# artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)pytest.testomatio.add_artifacts([artifact_url])page.close()⚠️ Please take into account s3_connector available only afterpytest_collection_modifyitems()hook is executed.If you prefer to use pytest hooks - addpytest_runtest_makereporthook in yourconftest.pyfile.defpytest_runtest_makereport(item,call):artifact_url=pytest.testomatio.upload_file(screenshot_path,filename)pytest.testomatio.add_artifacts([artifact_url])Clarificationstests can be synced even without@patest.mark.testomatio('@T96c700e6')decorator.test title in testomat.io == test name in pytesttest suit title in testomat.io == test file name in pytestExample of testTo make analyzer experience more consistent, it uses standard pytest markers.Testomat.io test id is a string value that starts with@Tand has 8 symbols after.importpytest@pytest.mark.testomatio('@T96c700e6')deftest_example():assert2+2==4Compatibility table withTestomatio check-testsActionCompatibilityMethodImporting test into Testomatiocompletepytest --analyzer addExclude hook code of a testN/AN/AInclude line number code of a testN/AN/AImport Parametrized Testscompletedefault behaviourDisable Detached Testscompletepytest --analyzer add --no-detachedSynchronous Importcompletedefault behaviourAuto-assign Test IDs in Source Codecompletedefault behaviourKeep Test IDs Between Projectscompletepytest --analyzer add --createClean Test IDscompletepytest --analyzer removeImport Into a BranchN/AN/AKeep Structure of Source Codecompletepytest --analyzer add --keep-structureDelete Empty Suitescompletepytest --analyzer add --no-emptyImport Into a Specific SuiteN/AN/ADebuggingparitypytest --analyzer debugChange log1.6 - Fixes nested suitesTestomaito not longer supports nested test suites. Suites could be only in a folder.Add helped to attach test artifactsExpose environment variables to provide access to cloud storageUpdate readme1.5.0 - Fixes artifacts in fixtures lifecycleEarlier, artifacts added in pytest fixtures where scipped by analyser1.4.0 - Fixes artifacts and test sync with TestomatioFixes artifacts uploadsFixes test id resolution when syncing local test with TestomatioFixes test id when sending test into test runAdds--create,--no-detached,--keep-structure,--no-empty,  for compatibility with original Testomatio check-testsImproves file update so it doesn't cause code style changes1.3.0 - added artifacts support connectorissue 5- connection issues not blocking test execution anymore1.2.8 - fixed issuesissue 4- Analyzer's execution order1.2.5 - fixed issuesadded env variableTESTOMATIO_RUNGROUP_TITLEto group test runsfixed serialization issue for update test status example1.2.4 - improved parametrized tests reportingnow parameters are passed to example attribute in the report1.2.3 - fixed issue with test artifactsif no credentials provided, test artifacts will not be uploaded and no issue raised1.2.0 - major update after testomat.io reviewcode refactoredsimplified authentication. Only API key neededmoved API key from pytest.ini to environment variableS3 credentials now read from testomat.io API, no local configuration neededPrettified test names in testomat.io1.1.0 - added artifacts support connectorthere is possibility to add artifacts (screenshots, logs) to test report1.0.9 - first public releasetest analyzer able to sync tests with testomat.iotest analyzer able to add test ids to teststest analyzer able to submit test results to testomat.ioRoadmapimprove logging