Package: tree-climber
tree-climberProgram analysis tools built ontree-sitter.
Currently supports only C.Try it outInstall from pip:pipinstalltree_climberor run from source:# install depspipinstall-rrequirements.txt# run on a test program :)pythontree_climbertests/data/example.c--draw_ast--draw_cfg--draw_ducFor Fedora 36:sudo dnf install graphviz-devel python3-tkinterFeel free to open a PR with other platform-specific instructions.Seedevelopers.mdfor developer setup instructions.Table of contentsFeaturesVisualize ASTConstruct and visualize Control-flow graph (CFG)Monotonic dataflow analysisConstruct and visualize Def-use chain (DUC)Construct and visualize Code Property Graph (CPG)ContributeFeaturesExamples shown ontests/data/example.c.intmain(){intx=0;x=x+1;if(x>1){x+=5;}else{x+=50;}x=x+2;for(inti=0;i<10;i++){x--;}x=x+3;while(x<0){x++;x=x+1;}x=x+4;returnx;}Visualize ASTVisualize AST without concrete tokens included in tree-sitter:pythonmain.pytests/data/example.c--draw_astExample:Construct and visualize Control-flow graph (CFG)Convert tree-sitter AST to CFG for C programs.
AST -> CFG algorithm is based on Joern, specificallyCfgCreator.scala.Visualize CFG:pythonmain.pytests/data/example.c--draw_cfgExample:Monotonic dataflow analysisSeedataflow_solver.py.Construct and visualize Def-use chain (DUC)Visualize DUC:pythonmain.pytests/data/example.c--draw_ducExample:Construct and visualize Code Property Graph (CPG)CPG composes AST + CFG + DUC into one graph for combined analysis.
Eventual goal is feature parity with Joern's usage in ML4SE.Visualize CPG (edges are color-coded - black = AST, blue = CFG, red = DUC):pythonmain.pytests/data/example.c--draw_cpgExample:ContributeOpen issues on GithubStress test (Jun 16 2022, outdated)Fileparse.shruns Joern and tree-sitter side by side to compare performance.
Usejoern-install.shto install Joern first.Benchmark 1: long stupid file - 10,000 lines ofx++.
Output 2022-06-15 19:44, v1.1.891 of Joern:(tree-sitter-py38)benjis@AM:~/code/ts$bashtests/vs-joern/parse.sh--joerntests/data/10000.c
executing/home/benjis/code/ts/tests/vs-joern/get_func_graph.scalawithparams=Map(filename->tests/data/10000.c)Compiling/home/benjis/code/ts/tests/vs-joern/get_func_graph.scala
creatingworkspacedirectory:/home/benjis/code/ts/workspace
Creatingproject`10000.c`forcodeat`tests/data/10000.c`movingcpg.bin.ziptocpg.binbecauseitisalreadyadatabasefile
CreatingworkingcopyofCPGtobesafe
LoadingbaseCPGfrom:/home/benjis/code/ts/workspace/10000.c/cpg.bin.tmp
Codesuccessfullyimported.Youcannowqueryitusing`cpg`.
Foranoverviewofallimportedcode,type`workspace`.
AddingdefaultoverlaystobaseCPG
Thegraphhasbeenmodified.Youmaywanttousethe`save`commandtopersistchangestodisk.Allchangeswillalsobesavedcollectivelyonexitscriptfinishedsuccessfully
Some(())real0m14.143s
user0m44.302s
sys0m1.260s(tree-sitter-py38)benjis@AM:~/code/ts$bashtests/vs-joern/parse.sh--tree-sittertests/data/10000.c

real0m1.503s
user0m1.385s
sys0m0.111sBenchmark 2:Linux kernel 5.18.4Output 2022-06-15 21:51, v1.1.891 of Joern:(tree-sitter-py38)benjis@AM:~/code/ts$timepythonmain.pylinux-5.18.4--cfg--file>output_treesitter.txt

real9m47.570s
user9m4.308s
sys0m5.854s(base)benjis@AM:~/code/ts$time./joern/joern-cli/joern--script./tests/vs-joern/get_func_graph.scala--paramsfilename=linux-5.18.4
executing/home/benjis/code/ts/tests/vs-joern/get_func_graph.scalawithparams=Map(filename->linux-5.18.4)[34mCompiling/home/benjis/code/ts/tests/vs-joern/get_func_graph.scala[39m
creatingworkspacedirectory:/home/benjis/code/ts/workspace
Creatingproject`linux-5.18.4`forcodeat`linux-5.18.4`Killed
Errorrunningshellcommand:List(/home/benjis/code/ts/joern/joern-cli/c2cpg.sh,linux-5.18.4,--output,/home/benjis/code/ts/workspace/linux-5.18.4/cpg.bin.zip)Exceptioninthread"main"java.lang.AssertionError:scripterrored:atio.joern.console.ScriptExecution.runScript(BridgeBase.scala:253)atio.joern.console.ScriptExecution.runScript$(BridgeBase.scala:229)atio.joern.joerncli.console.AmmoniteBridge$.runScript(AmmoniteBridge.scala:5)atio.joern.console.BridgeBase.runAmmonite(BridgeBase.scala:164)atio.joern.console.BridgeBase.runAmmonite$(BridgeBase.scala:146)atio.joern.joerncli.console.AmmoniteBridge$.runAmmonite(AmmoniteBridge.scala:5)atio.joern.joerncli.console.AmmoniteBridge$.delayedEndpoint$io$joern$joerncli$console$AmmoniteBridge$1(AmmoniteBridge.scala:7)atio.joern.joerncli.console.AmmoniteBridge$delayedInit$body.apply(AmmoniteBridge.scala:5)atscala.Function0.apply$mcV$sp(Function0.scala:39)atscala.Function0.apply$mcV$sp$(Function0.scala:39)atscala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)atscala.App.$anonfun$main$1(App.scala:76)atscala.App.$anonfun$main$1$adapted(App.scala:76)atscala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)atscala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)atscala.collection.AbstractIterable.foreach(Iterable.scala:926)atscala.App.main(App.scala:76)atscala.App.main$(App.scala:74)atio.joern.joerncli.console.AmmoniteBridge$.main(AmmoniteBridge.scala:5)atio.joern.joerncli.console.AmmoniteBridge.main(AmmoniteBridge.scala)Causedby:io.joern.console.ConsoleException:Errorcreatingprojectforinputpath:`linux-5.18.4`real499m56.583s
user1193m14.686s
sys7m26.020s