Package: tree-sitter
py-tree-sitterThis module provides Python bindings to thetree-sitterparsing library.InstallationThis package currently only works with Python 3. There are no library dependencies,
but you do need to have a C compiler installed.pip3installtree_sitterUsageSetupFirst you'll need a Tree-sitter language implementation for each language that you
want to parse. You can clone some of theexisting language reposorcreate your own:gitclonehttps://github.com/tree-sitter/tree-sitter-go
gitclonehttps://github.com/tree-sitter/tree-sitter-javascript
gitclonehttps://github.com/tree-sitter/tree-sitter-pythonUse theLanguage.build_librarymethod to compile these into a library that's
usable from Python. This function will return immediately if the library has
already been compiled since the last time its source code was modified:fromtree_sitterimportLanguageLanguage.build_library(# Store the library in the `build` directory"build/my-languages.so",# Include one or more languages["vendor/tree-sitter-go","vendor/tree-sitter-javascript","vendor/tree-sitter-python"],)Load the languages into your app asLanguageobjects:GO_LANGUAGE=Language("build/my-languages.so","go")JS_LANGUAGE=Language("build/my-languages.so","javascript")PY_LANGUAGE=Language("build/my-languages.so","python")Basic ParsingCreate aParserand configure it to use one of the languages:parser=Parser()parser.set_language(PY_LANGUAGE)Parse some source code:tree=parser.parse(bytes("""def foo():if bar:baz()""","utf8",))If you have your source code in some data structure other than a bytes object,
you can pass a "read" callable to the parse function.The read callable can use either the byte offset or point tuple to read from
buffer and return source code as bytes object. An empty bytes object or None
terminates parsing for that line. The bytes must encode the source as UTF-8.For example, to use the byte offset:src=bytes("""def foo():if bar:baz()""","utf8",)defread_callable(byte_offset,point):returnsrc[byte_offset:byte_offset+1]tree=parser.parse(read_callable)And to use the point:src_lines=["def foo():\n","    if bar:\n","        baz()"]defread_callable(byte_offset,point):row,column=pointifrow>=len(src_lines)orcolumn>=len(src_lines[row]):returnNonereturnsrc_lines[row][column:].encode("utf8")tree=parser.parse(read_callable)Inspect the resultingTree:root_node=tree.root_nodeassertroot_node.type=='module'assertroot_node.start_point==(1,0)assertroot_node.end_point==(3,13)function_node=root_node.children[0]assertfunction_node.type=='function_definition'assertfunction_node.child_by_field_name('name').type=='identifier'function_name_node=function_node.children[1]assertfunction_name_node.type=='identifier'assertfunction_name_node.start_point==(1,4)assertfunction_name_node.end_point==(1,7)assertroot_node.sexp()=="(module ""(function_definition ""name: (identifier) ""parameters: (parameters) ""body: (block ""(if_statement ""condition: (identifier) ""consequence: (block ""(expression_statement (call ""function: (identifier) ""arguments: (argument_list))))))))"Walking Syntax TreesIf you need to traverse a large number of nodes efficiently, you can use
aTreeCursor:cursor=tree.walk()assertcursor.node.type=="module"assertcursor.goto_first_child()assertcursor.node.type=="function_definition"assertcursor.goto_first_child()assertcursor.node.type=="def"# Returns `False` because the `def` node has no childrenassertnotcursor.goto_first_child()assertcursor.goto_next_sibling()assertcursor.node.type=="identifier"assertcursor.goto_next_sibling()assertcursor.node.type=="parameters"assertcursor.goto_parent()assertcursor.node.type=="function_definition"EditingWhen a source file is edited, you can edit the syntax tree to keep it in sync with
the source:tree.edit(start_byte=5,old_end_byte=5,new_end_byte=5+2,start_point=(0,5),old_end_point=(0,5),new_end_point=(0,5+2),)Then, when you're ready to incorporate the changes into a new syntax tree,
you can callParser.parseagain, but pass in the old tree:new_tree=parser.parse(new_source,tree)This will run much faster than if you were parsing from scratch.TheTree.get_changed_rangesmethod can be called on theoldtree to return
the list of ranges whose syntactic structure has been changed:forchanged_rangeintree.get_changed_ranges(new_tree):print("Changed range:")print(f"  Start point{changed_range.start_point}")print(f"  Start byte{changed_range.start_byte}")print(f"  End point{changed_range.end_point}")print(f"  End byte{changed_range.end_byte}")Pattern-matchingYou can search for patterns in a syntax tree using atree query:query=PY_LANGUAGE.query("""(function_definitionname: (identifier) @function.def)(callfunction: (identifier) @function.call)""")captures=query.captures(tree.root_node)assertlen(captures)==2assertcaptures[0][0]==function_name_nodeassertcaptures[0][1]=="function.def"TheQuery.captures()method takes optionalstart_point,end_point,start_byteandend_bytekeyword arguments which can be used to restrict the
query's range. Only one of the..._byteor..._pointpairs need to be given
to restrict the range. If all are omitted, the entire range of the passed node
is used.