Package: robotstatuschecker
IntroductionInstallation instructionsUsageDefining expected test statusDefining expected log messagesIntroductionStatusChecker is a tool for validating that executedRobot Frameworktest cases have expected statuses and log messages. It is mainly useful
for Robot Framework test library developers who want to use Robot
Framework to also test their libraries. StatusChecker 1.3 and newer are
compatible both with Python 2 and Python 3.StatusChecker project is hosted atGitHuband downloads are atPyPI.Installation instructionsThe easiest way to install StatusChecker is by usingpip:pip install robotstatuscheckerAlternatively you can get the code by cloning the project fromGitHubor downloading the source distribution fromPyPIand
extracting it. After that you can install the tool with:python setup.py installUsageFrom the command line:python -m robotstatuschecker infile [outfile]Programmatically:fromrobotstatuscheckerimportprocess_outputprocess_output('infile.xml','outfile.xml')If an output file is not given, the input file is edited in place.Defining expected test statusBy default, all test cases are expected toPASSand have no
message. Changing the expected status toFAILis done by having
the wordFAIL(in uppercase) somewhere in the test case
documentation. The expected error message must then follow
theFAILmarker.For robotframework version 4 you can also change the expected status
toSKIPby adding the wordSKIPin the test case documentation.
Like Fail, the expected skip message must follow the wordSKIP.
If a test documentation contains the wordsFAILandSKIP,SKIPwill be ignored and the expected status will beFAIL.If a test is expected toPASSwith a certain message, the wordPASSmust be added to its documentation explicitly and the
expected message given after that.If a message check should happen in test setup or teardown, that check
must be prefixed withSETUPorTEARDOWNword.The expected message can also be specified as a regular expression by
prefixing it withREGEXP:. The specified regular expression
must match the error message fully. Having spaces between the status,
the message and the possible regular expression prefix is optional.An alternative to using regular expressions is using glob patterns where*matches anything (including newline) and?matches any single
character. This is can be accomplished by starting the expected message
withGLOB:.Finally, it is possible to test that the message starts with something
by prefixing the expected message withSTARTS:.The following examples illustrate different ways to define test
statuses and messages:*** Test Cases ***Simple failure[Documentation]FAIL Expected error messageStepsCheck in test setup is done by SETUP marker[Documentation]LOG SETUPThis first log message in test setup[Setup]Test specific setupStepsExclude documentation before marker[Documentation]This text is ignored FAIL Expected error messageStepsRegexp example[Documentation]FAIL REGEXP: (IOError|OSError): .*StepsGlob example[Documentation]FAIL GLOB: ??Error: *StepsStart example[Documentation]FAIL STARTS: IOError:StepsPassing without messageStepsPassing with message[Documentation]PASS Expected messageStepsDefining expected log messagesThe expected keyword log messages can also be defined in the test case
documentation using a syntax such as:LOG x.y:z LEVEL Actual messageThe part before the colon specifies the keyword to check. For
example,1means first keyword,1.2is the second child
keyword of the first keyword, and so on.The part after the colon species the message. For example,1:2means the second message of the first keyword and1.2:3is
the third message of the second child keyword of the first keyword.
The message index is optional and defaults to1.
The message index also supports wildcard*. For example1:*matches any message of the first keyword.Message level is specified before the actual message, and it can be
any of the valid log levels in capital letters. If the level is not
given it defaults toINFO. Starting from 1.4 release alsoERRORlevel is supported. The message level also supports wildcardANYwhich will match all log levels.Possible leading and trailing whitespace is ignored both in the expected
and in the actual log message.This syntax can be used multiple times to test multiple messages.  It
also works together with specifying the expected error message withFAIL, but it that caseFAILand the expected error must
be first.It is also possible to give the message as a regular expression or glob
pattern or to give just the start of the message. This is accomplished
by prefixing the message withREGEXP:,GLOB:orSTARTS:,
respectively, exactly like whendefining expected test status.Finally, to check that a keyword does not have a certain message, it
is possible to useNONEin the place of the message.*** Test cases ***Simple example[Documentation]LOG 1Hello, world!StepsNested keywords[Documentation]LOG 2.11st child of 2nd kwStepsMessage index[Documentation]LOG 2:22nd msg of 2nd kwStepsNested and index[Documentation]LOG 3.1:22nd msg of 3rd kw's 1st childStepsLog levels[Documentation]LOG 2DEBUG Debug-level message...LOG 1.2:3WARN WarningStepsMultiple messages[Documentation]LOG 1First tested message...LOG 1.2Second tested message...LOG 2.2.1DEBUG Third tested messageStepsStatus and log[Documentation]FAILExpected error message...LOG 1.2Expected log messageStepsRegexp message[Documentation]LOG 1REGEXP: (Hello|Hi) world!StepsGlob message[Documentation]LOG 1GLOB: * world!StepsStart of the message[Documentation]LOG 1STARTS: Hello wStepsNo message[Documentation]LOG 1:1Test that we have only 1 msg...LOG 1:2NONEStepsCount Messages[Documentation]LOG 4 COUNT: 2# Fourth keyword should have excatly 2 messages.Steps