Package: circa-clue
CIRCAThis project contains the code of baselines and simulation data generation for the KDD '22 paper,Causal Inference-Based Root Cause Analysis for Online Service Systems with Intervention Recognition.
Experiment results can be found infigshare, where the code is corresponding to the commit 1522ddd7efd16db55e9f351fd70324501ce9134e.UsageThis repository contains aDockerfileto describe the necessary steps to setup the environment.
To install this project as a package withpip, R packagepcalghas to be installed manually.Simulation Data Generationpython-mcirca.experimentgenerateSimulation Study# Explore parameter combinationspython-mcirca.experiment--max-workers16--model-paramsparams-sim-tune.jsontune# Explore all the datasets with pre-defined parameterspython-mcirca.experiment--model-paramsparams-sim-run.jsonrun# Robustness evaluationpython-mcirca.experimentrobustnessExecuteRscript img/draw.sim.Rto produce summaries underimg/output.params-sim-run.jsonis created according toimg/output/best-sim-tuning.texTo create parameter template, execute the following commandpython-mcirca.experimentparams>default.jsonToolboxCIRCA is designed as a toolbox with a set of interfaces.BasicEach root cause analysis algorithm is separated into two steps, namelygraph constructionandscoring.The graph construction step should implementcirca.graph.GraphFactory.GraphFactory.createtakes data for analysis (an instance ofcirca.model.case.CaseData) and timestamp (float) when the algorithm is triggered.
The output is a graph (an instance ofcirca.model.graph.Graph) for the fault under analysis.The scoring step contains a sequence of scorers (instances ofcirca.alg.base.Scorer).Scorer.scoreof each scorer needs the following information:The graph produced in the graph construction step,data for analysis (an instance ofcirca.model.case.CaseData),timestamp (float) when the algorithm is triggered, and(optional) output of the previous scorer.Scorer.scorewill generate a mapping from a node in the input graph to its score (circa.alg.base.Score).
The design of the scorer sequence enables reusing scorers, i.e., two algorithms can share one scorer as a common step.
Note that a scorer may drop some nodes in the input graph, performing as a filter.circa.alg.commonprovides some common utilizations.
For example,circa.alg.common.Modelcombines a graph factory and a sequence of scorers as a whole with optional names.Model.analyzewill forward data and timestamp for them and produce an ordered sequence of scores.circa.alg.common.evaluatewill further evaluate a model with a set of cases (instances ofcirca.model.case.Case, each of which combines data and the corresponding answers)."""An example showing the basic usage of CIRCA"""fromcollectionsimportdefaultdictfromtypingimportDictfromtypingimportSequencefromtypingimportTupleimportnetworkxasnxfromsklearn.linear_modelimportLinearRegressionfromcirca.alg.ciimportRHTScorerfromcirca.alg.ci.anmimportANMRegressorfromcirca.alg.commonimportModelfromcirca.graph.commonimportStaticGraphFactoryfromcirca.model.caseimportCaseDatafromcirca.model.data_loaderimportMemoryDataLoaderfromcirca.model.graphimportMemoryGraphfromcirca.model.graphimportNodelatency=Node("DB","Latency")traffic=Node("DB","Traffic")saturation=Node("DB","Saturation")# circa.model.graph.MemoryGraph is derived from circa.model.graph.Graphgraph=MemoryGraph(nx.DiGraph({traffic:[latency,saturation],saturation:[latency],}))# 1. Assemble an algorithm# circa.graph.common.StaticGraphFactory is derived from circa.graph.GraphFactorygraph_factory=StaticGraphFactory(graph)scorers=[# circa.alg.ci.RHTScorer is derived from circa.alg.common.DecomposableScorer,# which is further derived from circa.alg.base.ScorerRHTScorer(regressor=ANMRegressor(regressor=LinearRegression())),]model=Model(graph_factory=graph_factory,scorers=scorers)# 2. Prepare datamock_data={latency:(10,12,11,9,100,90),traffic:(100,110,90,105,200,150),saturation:(5,4,5,6,90,85),}mock_data_with_time:Dict[str,Dict[str,Sequence[Tuple[float,float]]]]=defaultdict(dict)fornode,valuesinmock_data.items():mock_data_with_time[node.entity][node.metric]=[(index*60,value)forindex,valueinenumerate(values)]data=CaseData(# circa.model.data_loader.MemoryDataLoader is derived from# circa.model.data_loader.DataLoader, which manages data with configurationsdata_loader=MemoryDataLoader(mock_data_with_time),sli=latency,detect_time=240,lookup_window=4,detect_window=2,)# 3. Conduct root cause analysis one minute after a fault is detectedprint(model.analyze(data=data,current=data.detect_time+60))Advancedcirca.experimentsupports comparison among models and parameter exploration, as mentioned for the simulation study.
To conduct experiments with your own dataset, start from the following code namedexample.py.
Executepython -m example -s run-new --output-dir output/test --report-dir report/testand find the report inreport/test/report.csv.
Find more command line parameters withpython -m example -h."""An example showing the advanced usage of CIRCA"""importargparseimportloggingimportosfromtypingimportListfromcirca.experimentimportcomparisonfromcirca.experiment.comparison.modelsimportget_modelsfromcirca.experiment.__main__importget_parserfromcirca.graph.structuralimportStructuralGraphfromcirca.model.caseimportCasefromcirca.utilsimportsilence_third_partyBASE_DIR=os.path.abspath(os.path.dirname(__file__))defrun(args:argparse.Namespace):"""Evaluate multiple models"""data_dir:str=args.data_dirreport_dir:str=args.report_diros.makedirs(report_dir,exist_ok=True)logger=logging.getLogger(__package__)logger.info("Loading from%s",data_dir)# TODO: Prepare your data with answers herecases:List[Case]=[]models,graph_factories=get_models(# TODO: Configure your own structural graph here# structural_graph_params=dict(#     structural_graph=StructuralGraph(filename="tests/alg/sgraph/index.yml"),# ),params=args.model_params,seed=args.seed,cuda=args.cuda,max_workers=1,)logger.info("Start running on%swith #models=%d",data_dir,len(models))comparison.run(models=models,cases=cases,graph_factories=graph_factories,output_dir=args.output_dir,report_filename=os.path.join(report_dir,"report.csv"),max_workers=1ifargs.cudaelseargs.max_workers,)defwrap_parsers(subparsers:argparse._SubParsersAction):"""Add argparser for your own experiments"""parser_params=dict(formatter_class=argparse.ArgumentDefaultsHelpFormatter)parser=argparse.ArgumentParser(add_help=False,**parser_params)parser.add_argument("--data-dir",type=str,default=os.path.join(BASE_DIR,"data"),help="Data directory",)parser_run:argparse.ArgumentParser=subparsers.add_parser("run-new",parents=[parser],help="Explore all combinations of model parameters",**parser_params,)parser_run.add_argument("--output-dir",type=str,default="output",help="Output directory")parser_run.add_argument("--report-dir",type=str,default="report",help="Report directory")parser_run.set_defaults(func=run)def_main():parser,subparsers=get_parser()wrap_parsers(subparsers)parameters=parser.parse_args()ifparameters.S:logging.basicConfig(level=logging.ERROR)elifparameters.V:logging.basicConfig(level=logging.DEBUG)else:logging.basicConfig(level=logging.INFO)ifnotparameters.V:silence_third_party()if"func"inparameters:parameters.func(parameters)else:parser.print_usage()if__name__=="__main__":_main()