Package: mbed-greentea
Development movedThe development of Greentea has been moved into thembed-os-toolspackage. You can continue to use this module for legacy reasons, however all further development should be continued in the new package.Greentea - test automation for mbedGenericregressionenvironment fortestautomationIntroductionGreentea is the automated testing tool for mbed OS development. It automates the process of flashing mbed boards, driving the test and accumulating test results into test reports. Developers use it for local development as well as for automation in a Continuous Integration environment.This document should help you start using Greentea. Please see thehtrun documentation, the tool Greentea uses to drive tests, for the technical details of the interactions between the platform and the host machine.Because Greentea is an open source project, we accept contributions! Please see ourcontributing documentfor more information.PrerequistesGreentea requiresPython version 2.7. It supports the following OSes:WindowsLinux (Ubuntu preferred)OS X (experimental)InstallingTools that depend on Greentea usually install it. Determine if Greentea is already installed by running:$ mbedgt --version
1.2.5You can also install it manually via pip.pip install mbed-greenteaTest specification JSON formatThe Greentea test specification format decouples the tool from your build system. It provides important data, such as test names, paths to test binaries and the platform on which the binaries should run.Greentea automatically looks for files calledtest_spec.jsonin your working directory. You can also use the--test-specargument to direct Greentea to a specific test specification file.When you use the-t/--targetargument with the--test-specargument, you can select which "build" should be used. In the example below, you could provide the arguments--test-spec test_spec.json -t K64F-ARMto only run that build's tests.Example of test specification fileIn the below example, there are two defined builds:BuildK64F-ARMfor NXPK64Fplatform compiled withARMCCcompiler.BuildK64F-GCCfor NXPK64Fplatform compiled withGCC ARMcompiler.{"builds":{"K64F-ARM":{"platform":"K64F","toolchain":"ARM","base_path":"./BUILD/K64F/ARM","baud_rate":9600,"tests":{"tests-mbedmicro-rtos-mbed-mail":{"binaries":[{"binary_type":"bootable","path":"./BUILD/K64F/ARM/tests-mbedmicro-rtos-mbed-mail.bin"}]},"tests-mbed_drivers-c_strings":{"binaries":[{"binary_type":"bootable","path":"./BUILD/K64F/ARM/tests-mbed_drivers-c_strings.bin"}]}}},"K64F-GCC":{"platform":"K64F","toolchain":"GCC_ARM","base_path":"./BUILD/K64F/GCC_ARM","baud_rate":9600,"tests":{"tests-mbedmicro-rtos-mbed-mail":{"binaries":[{"binary_type":"bootable","path":"./BUILD/K64F/GCC_ARM/tests-mbedmicro-rtos-mbed-mail.bin"}]}}}}}The examples below use the above test specification file.Command-line usageThis section highlights a few of the capabilities of the Greentea command-line interface. For a full list of the available options, please runmbedgt --help.Assume for the examples below that the abovetest_spec.jsonfile is in the current directory.Listing all testsYou can use the-largument to list all available tests:$ mbedgt -l
mbedgt: greentea test automation tool ver. 1.2.5
mbedgt: using multiple test specifications from current directory!
        using 'BUILD\tests\K64F\ARM\test_spec.json'
        using 'BUILD\tests\K64F\GCC_ARM\test_spec.json'
mbedgt: available tests for built 'K64F-GCC_ARM', location 'BUILD/tests/K64F/GCC_ARM'
        test 'tests-mbedmicro-rtos-mbed-mail'
mbedgt: available tests for built 'K64F-ARM', location 'BUILD/tests/K64F/ARM'
        test 'tests-mbed_drivers-c_strings'
        test 'tests-mbedmicro-rtos-mbed-mail'Executing all testsThe default action of Greentea usingmbedgtis to execute all tests that are found intest_spec.jsonfiles. You can also add-Vto make the output more verbose.Limiting testsYou can select test cases by name using the-nargument. This command executes all tests namedtests-mbedmicro-rtos-mbed-mailfrom all builds in the test specification:$ mbedgt -n tests-mbedmicro-rtos-mbed-mailWhen using the-nargument, you can use the*character at the end of a test name to match all tests that share a prefix. This command executes all tests that start withtests*:$ mbedgt -n tests*You can use the-targument to select which build to use when finding tests. This command executes the testtests-mbedmicro-rtos-mbed-mailfor theK64F-ARMbuild in the test specification:$ mbedgt -n tests-mbedmicro-rtos-mbed-mail -t K64F-ARMYou can use a comma (,) to separate test names (argument-n) and build names (argument-t). This command executes the teststests-mbedmicro-rtos-mbed-mailandtests-mbed_drivers-c_stringsfor theK64F-ARMandK64F-GCC_ARMbuilds in the test specification:$ mbedgt -n tests-mbedmicro-rtos-mbed-mail,tests-mbed_drivers-c_strings -t K64F-ARM,K64F-GCC_ARMSelecting platformsYou can limit which boards Greentea uses for testing by using the--use-tidsargument.$ mbedgt --use-tids 02400203C3423E603EBEC3D8,024002031E031E6AE3FFE3D2Where02400203C3423E603EBEC3D8and024002031E031E6AE3FFE3D2are the target IDs of platforms attached to your system.You can view target IDs using thembed-lstool (installed with Greentea).$ mbedls
+--------------+---------------------+------------+------------+-------------------------+
|platform_name |platform_name_unique |mount_point |serial_port |target_id                |
+--------------+---------------------+------------+------------+-------------------------+
|K64F          |K64F[0]              |E:          |COM160      |024002031E031E6AE3FFE3D2 |
|K64F          |K64F[1]              |F:          |COM162      |02400203C3423E603EBEC3D8 |
|LPC1768       |LPC1768[0]           |G:          |COM5        |1010ac87cfc4f23c4c57438d |
+--------------+---------------------+------------+------------+-------------------------+In this case, you won't test one target, the LPC1768.Testing on Fast Model FVPsFast Models FVPs are software models for Arm reference design platfromGreentea supports running test on Fast Models. Andmbed-fastmodel-agentmodule is required for this purpose.The "--fm" option only available when thembed-fastmodel-agentmodule is installed :You can run tests for FVP_MPS2_Cortex-M3 models, by '--fm' option:$ mbedgt --fm FVP_MPS2_M3:DEFAULTWhereFVP_MPS2_M3is the platfrom name for theFVP_MPS2_Cortex-M3models in mbed OS.AndDEFAULTis the config to the Fast Model, you can find out usingmbedfmcommandCreating reportsGreentea supports a number of report formats.HTMLThis creates an interactive HTML page with test results and logs.mbedgt --report-html html_report.htmlJUnitThis creates an XML JUnit report, which you can use with popular Continuous Integration software, such asJenkins.mbedgt --report-junit junit_report.xmlJSONThis creates a general JSON report.mbedgt --report-json json_report.jsonPlain textThis creates a human-friendly text summary of the test run.mbedgt --report-text text_report.textHost test detectionWhen developing with mbed OS, Greentea detects host tests automatically if you place them in the correct location. All tests in mbed OS are placed under aTESTSdirectory. You may place custom host test scripts in a folder namedhost_testsin this folder. For more information about the mbed OS test directory structure, please see thembed CLI documentation.Common issuesIOERR_SERIALerrorsPossible causes:Another program is using the serial port. Be sure all terminals and other instances of Greentea are closed before trying again.The mbed interface firmware is out of date. Please see the platform's page ondeveloper.mbed.orgfor details about how to update it.