Package: shouldget
should-- Test command-line applications through.shouldfilesshouldis a single-file program to test command-line applications on Unix-like systems.
It checks the standard output, or possibly another file, and parses for exact or regular expressions,
possibly while counting them and checking their number of occurrences.
It also parses and tests JSON data.shouldoutputs reports in.tapformatand in JUnit-like XML.shouldis written in Python with no external dependencies except from Python >= 3.4
and is intended to work on any command-line application
-- should your application outputs something, you can test it!DownloadThe archiveshould-3.0.0.tar.gzalso includes this documentation with demo files.
In your projetcts, you only need theshould.pyfile.Basic usageThedemo/hello.shouldexample covers the basic functionality :# A .should file launches any command it encounters.# Every line starting with a `#` is a comment.echo"hello, world"# Lines containing a `:` are test lines.# The `test expression` is what is found at the right of the `:`.# It should be found in the stdout, at least one time.:world
:lo,wo# What is at the left of the `:` are modifiers.# One can specify the exact number of times the test expression has to appear.1:hello0:Bye-bye# Lines beginning by `$` give a name to the following tests$Checkspacebeforeworld1:,wo

$Checkmisspellings0:wrld0:helo# The `r` modifier triggers Python regular expressions$Twoo´s
r:o.*o

$Amoreflexibletestr:[Hh]ello,\s+worldThe test is then run by callingshouldon the.shouldfile:>./shoulddemo/hello.should$LAUNCHER=demo/hello.should
**echo"hello, world"**stdout-->1linesstderr-->0lines

ok
ok
ok
ok
ok-Checkspacebeforeworld
ok-Checkmisspellings
ok-Checkmisspellings
ok-Twoo´s
ok-Amoreflexibletestok-Exitcodeis0==>ok-ok:10total:10tests

demo/hello.should==>ok-ok:10total:10testsshouldcan be run on several.shouldfiles at once. In this case, it furthers shows statistics
on all tests. Here 39 out of 40 tests passed from 8.shouldfiles.>./shoulddemo/*.should(...)Summary==>ok-ok:8total:8filesSummary==>ok-ok:39TODO:1total:40testsDocumentationThe documentation is completed by the files indemo/.Thedemo/cal.shouldexample shows
matching on several lines (l), counting inside lines (w),
ignoring whitespace differences (b), expecting a test to fail (f)
or allowing a test to fail (a),
requiring less than or more than a given number of expressions (</>).demo/commands.shouldshows that several commands can be used into a same.shouldfile. Tests are flushed after each set of commands.demo/exit-codes.shouldshows how to require a particular exit code with!EXIT_CODE.demo/unicode.shouldshows that unicode characters can be put
in test comment and strings, provided that a UTF-8 locale has been set up.demo/variables.shouldshows how to define and use variables.demo/launcher.shouldshows how to define a "launcher" that is appended before every command,
for example to launch tools likevalgrindon a test set.demo/extra.shouldshow how to add an argument to every tested command.demo/json.shouldshows how to test JSON data (keys and values).gitOptions and modifiersusage:should[--cdPATH][--cd-same][--launcherCMD][--extraARG][--modMODIFIERS][--varNAME=value][--timeoutTIMEOUT][--shuffle][--no-a][--no-f][--only-a][--only-f][--retry][--retry-warned][--log][--tap][--xml][-v][-q][--fail-a][-h][--version]should-file[should-file...]Testcommand-lineapplicationsthrough.shouldfiles

positionalarguments:should-fileinputfiles(.should)runningtests(canalsobesetpertestin!OPTIONS):--cdPATHdirectoryfromwhichtorunthetestcommands--cd-samerunthetestcommandsfromthesamedirectoryasthe.shouldfiles--launcherCMDlauncherprecedingeachcommand(orreplacing$LAUNCHER)--extraARGextraargumentafterthefirstwordofeachcommand(orreplacing$EXTRA)--modMODIFIERSglobalmodifiers(uppercaseletterscancelpreviousmodifiers)f/Fconsiderthatthetestshouldfaila/Aconsiderthatthetestisallowedtofailr/Rconsiderasaregularexpressionw/Wcountalloccurrences,evenonasamelinei/Iignorecasechangesb/Bignorewhitespacedifferencesassoonasthereisatleastonespace.Implies'r'l/Lsearchonalltheoutputratherthanoneverylinez/Zkeepleadingandtrailingspacesj/Jinterpretjsondata.Implies'lw'>requiresthattheexpressionoccursstrictlymorethanthegivennumber<requiresthattheexpressionoccursstrictlylessthanthegivennumber--varNAME=valuevariabledefinition(thenuse$NAMEin.shouldfiles)--timeoutTIMEOUTDelay(inseconds)afterwhichthetaskisstopped(default:120)selectingteststoberun:--shuffleshufflethetests--no-adonotlaunch'a'tests--no-fdonotlaunch'f'tests--only-alaunchesonly'a'tests--only-flaunchesonly'f'tests--retrylaunchesonlythelastfailedtests--retry-warnedlaunchesonlythelastfailedorwarnedtests

controllingoutput:--logstorestheoutputinto.logfiles--tapoutputs.tapfiles--xmloutputsJUnit-likeXMLintoshould.xml-v,--verboseincreaseverbosity-q,--quietverbositytozero--fail-afailonpassing'a'tests-h,--helpshowthishelpmessageandexit--versionshowprogramversionnumberandexitOutput.By default,shouldonly writes to the standard output.
The--log,--tapand--xmloptions enable to store the actual output of the tested commands
as well as.tapfilesand a JUnitshould.xmlfile.Exit code.should.pyreturns0when all the tests passed (or have been skipped, or marked asfailed-with-ALLOWwitha).
As soon as one regular test fails, or as soon as a test marked withTODOpass,should.pyreturns1.AlternativesAruba: Testing with Cucumber, RSpec, Minitest --https://github.com/cucumber/arubaCram: It's test time --https://bitheap.org/cram/Bats: Bash Automated Testing System --https://github.com/sstephenson/batsExpect --http://core.tcl.tk/expectTush: Literate testing for command-line programs --https://github.com/darius/tushSee alsohttps://stackoverflow.com/questions/353198/best-way-to-test-command-line-tools.Historyshouldis a refactor of an earlier shell script that was developed and heavily used since 2013
to test two bioinformatics programs,CRAC(mapping of RNA sequences, discovery of transcriptomic and genomic variants),Vidjil-algo(clustering and analysis of lymphocyte clusters from their DNA sequences,
contains700+shouldtestsin about 100 files).