Package: islearn
ISLearnISLearn is a system for learning ISLa constraints from a set of patterns.
Patterns are instantiated according to sample inputs (that can also automatically
be generated based on a given program property), filtered, and combined to
a disjunctive normal form (conjunctions of disjunctions of literals, where
literals are possibly negated, instantiated patterns). ISLearn output the result
ranked by estimates forspecificityandrecallof the results: Top-ranked
invariants were found to be best suitable for distinguishing positive (with
respect to a program property) and negative inputs.ExampleConsider a grammar of a simple assignment programming language (e.g., "x := 1 ; y := x"):importstringLANG_GRAMMAR={"<start>":["<stmt>"],"<stmt>":["<assgn>","<assgn> ; <stmt>"],"<assgn>":["<var> := <rhs>"],"<rhs>":["<var>","<digit>"],"<var>":list(string.ascii_lowercase),"<digit>":list(string.digits)}For learning input invariants, we need a property of the inputs satisfying that
invariant. If the goal is to learn properties about (non-context-free) syntactical
correctness of a programming language, this is a function returningTrueif, and
only if, a statement is executed without raising a specific class of errors. For
our language described byLANG_GRAMMAR, we define aneval_langfunction that raises
an error if an identifier is not defined;validate_langturns this into a property.fromtypingimportDictfromislearn.parse_tree_utilsimportdfs,get_subtree,tree_to_stringfromisla.languageimportDerivationTreefromisla.parserimportEarleyParserfromisla.type_defsimportParseTreedefeval_lang(inp:str)->Dict[str,int]:defassgnlhs(assgn:ParseTree):returntree_to_string(get_subtree(assgn,(0,)))defassgnrhs(assgn:ParseTree):returntree_to_string(get_subtree(assgn,(2,)))valueMap:Dict[str,int]={}tree=list(EarleyParser(LANG_GRAMMAR).parse(inp))[0]defevalAssignments(tree):node,children=treeifnode=="<assgn>":lhs=assgnlhs(tree)rhs=assgnrhs(tree)ifrhs.isdigit():valueMap[lhs]=int(rhs)else:valueMap[lhs]=valueMap[rhs]dfs(tree,evalAssignments)returnvalueMapdefvalidate_lang(inp:DerivationTree)->bool:try:eval_lang(str(inp))returnTrueexceptException:returnFalseISLearn can learn this property based on the "Def-Use (reST Strict)" pattern from
the catalog (the standard catalog is insrc/islearn/patterns.toml). You call
the learner as follows:fromislearn.learnerimportInvariantLearnerfromisla.languageimportISLaUnparser,FormulafromtypingimportDict,Tupleresult:Dict[Formula,Tuple[float,float]]=InvariantLearner(LANG_GRAMMAR,prop=validate_lang,activated_patterns={"Def-Use (reST Strict)",# Optional; leads to quicker results},).learn_invariants()print("\n".join(map(lambdap:f"{p[1]}: "+ISLaUnparser(p[0]).unparse(),{f:pforf,pinresult.items()ifp[0]>.0}.items())))The expected result is(1.0, 1.0): forall <rhs> use_ctx="{<var> use}" in start:
  exists <assgn> def_ctx="{<var> def} := <rhs>" in start:
    (before(def_ctx, use_ctx) and
    (= use def))That invariant has full specificity (first value in the tuple) and recall
(second value), since it holdsexactlyfor all valid inputs.Resources / Important FilesYou find the pattern catalog insrc/islearn/patterns.toml.The evaluation scripts are in the directoryevaluations/.The most important files of our implementation aresrc/islearn/learner.py,
containing the learning incl. candidate generation, filtering, etc., and
andsrc/islearn/language.py, which defines the abstract ISLa langauge for
the patterns catalog.Configuration ParametersISLearn (the classInvariantLearner) has a number of optional configuration parameters that can be passed to the
constructor; many of them can be used to reduce or expand the search space, and thus either lead to more / longer
invariants or to quicker results.ParameterDefaultDescriptionpropNoneThe program property for distinguishing valid from invalid inputs.positive_examplesNoneA set of valid sample inputs. Optional if prop is present.negative_examplesNoneA set of invalid sample inputs. Optional. Needed to estimate specificifity if prop is not present.min_recall.9What is the minimum target recall value?min_specificity.6What is the minimum target specificity value?max_disjunction_size1Size of disjunctions to be generated. As default, no disjunctions (size 1). Potentially increases recall.include_negations_in_disjunctionsFalseAlso include negations in literals within disjunctions. Use with case; can lead to "spurious" invariants.max_conjunction_size2Size of conjunctions to be generated. As default, conjunctions of size 2. Potentially increases specificity.activated_patternsNoneA set of pattern names from the catalog that should be selected. As a default, all patterns are selected.deactivated_patternsNoneA set of pattern names from the catalog that shouldnotbe selected.pattern_fileNoneA file name to a pattern catalog (TOML file). Standard issrc/islearn/patterns.toml.patternsNoneA set of patterns (abstract formulas) to be considered if no catalog should be used.target_number_positive_samples10How many positive samples should be created using fuzzing?target_number_negative_samples10How many negative samples should be created using fuzzing?target_number_positive_samples_for_learning10How many positive samples should be generated specifically for learning? <=positive_examplesreduce_inputs_for_learningTrueIfTrue, learning samples are reduced, keeping k-path coverage and validity.reduce_all_inputsFalseIfTrue, all inputs are reduced. Reduction of learning inputs can be specifically disabled.generate_new_learning_samplesTrueOnly ifTrue, the fuzzers are used to generate further learning examples.do_generate_more_inputsTrueOnly ifTrue, any further examples (positive, negative) are generated by the fuzzers.filter_inputs_for_learning_by_kpathsTrueIfTrue, learning inputs are filtered k-paths: Retain only those with new k-path coverage information.mexpr_expansion_limit1To what depth should the learner search for instantiations of match expressions?max_nonterminals_in_mexprNoneHow many nonterminals are allowed in match expressions?Nonemeans no restriction.exclude_nonterminalsNone"Irrelevant" grammar nonterminals. Reduces search space. Example: White space nonterminals.k3Thekfromk-Paths. Used by the input generators and filters.Install, Build, TestISLearn depends on Python 3.10 and the Python header files. To compile all of ISLearns's dependencies, you need
gcc, g++ make, and cmake. To check out the current ISLearn version, git will be needed. Furthermore,
python3.10-venv is required to run ISLearn in a virtual environment.OnAlpine Linux, all dependencies can be installed usingapkaddpython3.10python3.10-devpython3.10-venvgccg++makecmakegitInstallTo install ISLearn, a simplepip install islearnsuffices. We recommend installing ISLearn inside a virtual
environment (virtualenv):python3.10-mvenvvenvsourcevenv/bin/activate
pipinstall--upgradepip
pipinstallislearnBuildISLearn is built locally as follows:gitclonehttps://github.com/rindPHI/islearn.gitcdislearn/

python3.10-mvenvvenvsourcevenv/bin/activate

pipinstall--upgradepip
pipinstall--upgradebuild
python3-mbuildThen, you will find the built wheel (*.whl) in thedist/directory.Testing & DevelopmentFor running the ISLearn tests, you additionally needclang,racket, andgraphviz. On Alpine Linux, you
can install those byapkupdate
apkupgrade
apkaddclangracketgraphvizThen, you can run the ISLearn tests as follows:gitclonehttps://github.com/rindPHI/islearn.gitcdislearn/

python3.10-mvenvvenvsourcevenv/bin/activate
pipinstall--upgradepip# Run testspipinstall-e.[test]python3-mpytest-n16tests