Package: rialtic-klee-py
Rialtic Klee OverviewThis library should serve to replace our loose scriptsbuild_test_cases.python,test_engine.python,smoke_test.pythonin almost all cases.You are still able to write an interim scripts using library calls,
when new engines create yet unaddressed requirements.
However, this should be kept to a minimum.The Command Line UtilityThis library provides a command line utility in order to conduct local tests (with automatic building), and remote (smoke) tests.klee build [<optional_test_cases>]klee test [<optional_test_cases>]klee smoke [<optional_test_cases>]note: could we perhaps reconcile the diverged logic of SPEs/MPEs?The Library APIFile:klee.buildCommand:klee build [<optional_test_cases>]With the following arguments,-p <plan-type>, default:'default'-c <claims-dir>, default:'test_claims'-o <output-dir>, default:test_casesIt will outputInsightEngineTestCaseobjects to<output_dir>/json_cases,
andHistoryClaimarrays to<output_dir>/history.File:klee.plansClass Interfaceclass<LegacyTestPlan|MPETestPlan>(EngineTestPlan):def__init__(self,claims_dir='test_claims',output_dir='',history_dir=''):...defbuild_all_cases(self)->Dict[str,InsightEngineTestCase]:...defbuild_test_case(self,node:Union[str,KleeTestClaim])->InsightEngineTestCase:...defbuild_node_labels(self,node_labels:List[str])->Dict[str,InsightEngineTestCase]:...File:klee.testCommand:klee test [<optional_test_cases>]With the following arguments,-p <plan_type>, default:'default'-c <claims_dir>, default:'test_claims'-o <output-dir>, default:test_cases--pytest="...", default:'-vvv'-ld, --local_defenseClass InterfaceclassLocalTest(PyTestUtility):def__init__(self,args:Iterable[str]=tuple()):...definvoke_cases(self,test_cases:Dict[str,InsightEngineTestCase]):...Command:klee smoke [<optional_test_cases>]With the following arguments,-p <plan_type>, default:'default'-c <claims_dir>, default:'test_claims'-o <output-dir>, default:'test_cases'--pytest="...", default:'-vvv'Class InterfaceclassSmokeTest(PyTestUtility):def__init__(self,args:Iterable[str]=tuple()):...definvoke_cases(self,test_cases:Dict[str,InsightEngineTestCase]):...Library StructureSupported Test Plan FormatsFile:klee.plans.*ExposesclassMPETestPlan(EngineTestPlan):passclassLegacyTestPlan(EngineTestPlan):passclassEngineTestPlan(TestCaseBuilder):defclaim_line(self,claim:KleeTestClaim)->int:...defget_history(self,claim:KleeTestClaim)->List[HistoryClaim]:...defget_defense(self,case:InsightEngineTestCase)->str:...defvalidate_case(self,case:InsightEngineTestCase)->bool:...Supported Test Claims FormatsFile:klee.claimsExposesclassClaimsDirectory:...classKleeTestClaim:...Test Claim/Tab Naming ConventionsSupported Insight/Defenses FormatsFile:klee.insightsExposesclassInsightDict(dict):...classInsightText:...defload_insights()->InsightDict:...defread_engine_result()->InsightDict:...defread_local_files()->InsightDict:...Flow:result.EngineResult.insight_<type|text|defense>><insights_and_defenses|insights|defense>.csv><insights|defense>.json>test_plan.csv + test_dataset.csv ???Library File UtilsFile:klee.filesExposesclassKleeFile:...classInspection:...defsave_json(data,target):...defread_claim(target)->Claim:...defread_json(target)->Dict[Any,Any]:...defread_str(target)->str:...defread_csv(target,normalize=False,**kwargs):...deffhir_to_json(obj:domainresource.DomainResource)->Dict:...File:klee.internalExposesclassStructure:root_dir:strclaim_dir:stroutput_dir:strinternal:strreports:strlogs:strhistory:strbinary:strsearch:strtimestamp:intjson_enabled:boolMass TestingWe could potentially create aGithub Actionto test (by a yet to be decided trigger) against a range of engine repos.VersioningAs the design of this library evolves over time, engines will be written depending on certain assumptions made by this test suite. To address these,  we will be using semantic versioning (https://semver.org/).
If features/tests are removed, it should be considered amajorbreaking change.
If features/tests are added, it should be considered aminorapi change.
If there is a bug introduced, a bugfixpatchshould be released.In the format ofv<MAJOR>.<MINOR>.<PATCH>, which can be used with Pipenv's compatibility operator to accept all non breaking changes asrialtic-klee-python ~= <MAJOR>.<MINOR>.Repository BranchesBranches:developmentserves as our development/specification discussion headv<MAJOR>.<MINOR>should be used for retroactive fixesTags:v<MAJOR>.<MINOR>.<PATCH>for each release