Package: klara
KlaraKlara is a static analysis tools to automatic generate test case, based
on SMT (z3) solver, with a powerful ast level inference system. Klara
will take python file as input and generate corresponding test file in
pytest format, that attempt to cover all return values. For example,
following function in filetest.pydeftriangle(x:int,y:int,z:int)->str:ifx==y==z:return"Equilateral triangle"elifx==yory==zorx==z:return"Isosceles triangle"else:return"Scalene triangle"will generateimporttestdeftest_triangle_0():asserttest.triangle(0,0,0)=='Equilateral triangle'asserttest.triangle(0,0,1)=='Isosceles triangle'asserttest.triangle(2,0,1)=='Scalene triangle'See the Klara's documentation athttps://klara-py.readthedocs.ioNote: Klara is still in early experimental stage, notable missing features are loop, comprehension, module import, exceptions and many more.
Seelimitationsfor full list. It probably will not run on real world projects, so it's best
to cherry-pick a few interesting functions to generate the corresponding test case.InstallingKlara can be installed viapiptool by using:pip install klaraUsageWe can invokeklaraon any python source file, and it will generate a
corresponding pytest test file.$catsource.py
deffoo(x:int,y:int,z:str):ifx+y>2:returnx+y+12elifx<y:returnx+yelif(z+"me")=="some":returnz+"thing"else:returnx-y

$klarasource.py
$cattest_source.py
importcontract_test


deftest_foo_0():assertcontract_test.foo(0,3,\'\')==15assertcontract_test.foo(0,1,\'\')==1assertcontract_test.foo(0,0,\'so\')==\'sothing\'assertcontract_test.foo(0,0,\'\')==0Consult thequick startmanual for more examples and
guidance. To use it as a static analysis library, go toInference.Why Klara?Klara works on ast level and it doesn't execute user code in any way,
which is a very important difference compared to similar tool likeCrosshairandPynguinthat utilize concolic
symbolic execution that required user code execution that might cause
unwanted side effects. Klara work on ast level, combine with data flow analysis
that utilize Control Flow Graph(CFG), Static Single Assignment(SSA), use-def chain, etc... to build a
powerful python inference system that leverages Z3-solver for
constraints solving and path feasibility check. Because of this, Klara
is able to operate on both python2/3 source code with the help oftyped_ast. To specify the source
code is in python 2, pass in-py 2argument. It's python 3 by
default.Klara can also be used as a static analysis tool, allow user to define
custom rule to identify programming bugs, error or enforcing coding
standard. With SMT solver support, analysis will be more accurate and
greatly reduce false-positive case. For exampleimportklaratree=klara.parse("""def foo(v1: int):if v1 > 4:if v1 < 3:z = 1else:z = 2else:z = 3s = z""")withklara.MANAGER.initialize_z3_var_from_func(tree.body[0]):print(list(tree.body[0].body[-1].value.infer()))Will print out:[2, 3]Becausez = 1is not possible due tov1 > 4andv1 < 3is unsatisfiableThe inference system architecture and api is largely inspired byAstroid, a static inference library
used byPylint.Klara utilize the inference system to generate test case, in other
words, itgenerate test case for all possible return values of the
function, instead of generate test case for all control path of the
function.To illustrate the point, consider the function below, withdivide by zerovulnerabilities at line 3deffoo(v1:int,v2:float):ifv1>10000:s=v1/0# unused statementifv1>v2:s=v1else:s=v2returnsKlara will generate test inputs belowimportcontract_testdeftest_foo_0():assertcontract_test.foo(0,-1.0)==0assertcontract_test.foo(0,0.0)==0.0It doesn't generate inputv1 > 10000, so the test case would not be
able to find out the exceptions. This is because thesat
line 3 is unused in the return value.If we modify the second if statement toelif, which we'll
be able to return the [s]{.title-ref} at line 3, klara will generate
test inputs that coverv1 > 10000case.This is an important distinction with other automatic test case
generation available now, because by only generate test case for return
values, we can generate a minimal test case, and it's easier to
customize how do Klara cover the function.For example, say we are composing a complex systemdefmain(number:int,cm:int,dc:int,wn:int):mc=0ifwn>2:ifnumber>2andnumber>2ornumber>2:ifnumber>0:ifwn>2orwn>2:mc=2else:mc=5else:mc=100else:mc=1nnn=number*cmifcm<=4:num_incr=4else:num_incr=cmn_num_incr=nnn/num_incrnnn_left=dc*num_incr*(n_num_incr/2+n_num_incr%2)nnn_right=nnn-nnn_leftis_flag=nnn_rightifis_flag:cell=Component(nnn_right,options=[mc])else:cell=Component(nnn_right)returncellIt isn't immediately clear to us how many possible return values there
are. But we can utilize Klara to generate inputs instantly, below is the
generated testimportcontract_testdeftest_main_0():assertcontract_test.main(2,4,1,3)isnotNoneassertcontract_test.main(2,4,-1,6)isnotNoneassertcontract_test.main(2,4,1,4)isnotNoneassertcontract_test.main(-2,4,3,4)isnotNoneassertcontract_test.main(-1,-1,-1,2)isnotNoneassertcontract_test.main(0,0,0,3)isnotNoneassertcontract_test.main(0,0,0,6)isnotNoneassertcontract_test.main(0,0,0,4)isnotNoneassertcontract_test.main(-2,0,0,4)isnotNoneassertcontract_test.main(0,0,0,0)isnotNoneAbove generated 10 total results, which is product ofnnn_rightwhich have 2 possibilities andmcwhich have 5 possibilities.Suppose that 10 tests input is too much, and we have determine that theoptionsargument toComponentis redundant to test, we
can use Klara's custom plugin to selectively determine which part to
ignore in test generation. Go tocustomize coverage
strategyfor more information.After we have setup the plugin, Klara will generate following testimportcontract_testdeftest_main_0():assertcontract_test.main(1,3,0,0)isnotNoneassertcontract_test.main(0,0,0,0)isnotNoneWhich is only 2 combinations ofnnn_rightBecause Klara can't dynamically execute the code, it will provide extension to specify how to infer
specific ast node or user defined type to make Klara 'smarter'. It's described inextending,extending user
typeandcustomize coverage
strategy.ContributingWe usePoetryto manage dependencies.
After poetry is installed, run:$ poetry shell
$ poetry installTo run the test case, do:$ poetry run pytest testAcknowledgementsThe architecture of the inference system is largely inspired byAstroid.Special thanks to Dr. Poh for guiding the early stages of the project.LicenseThis project is licensed under the terms of the GNU Lesser General
Public License.