Package: abch-tree-sitter
py-tree-sitterThis module provides Python bindings to thetree-sitterparsing library.InstallationThis package currently only works with Python 3. There are no library dependencies, the package is distributed as a CPython-compiled wheel.pip3installabch-tree-sitterAcknowledgmentsThis is a fork ofpy-tree-sitterby Max Brunsfeld with UTF-16 support added and distributed as CPython-compiled wheels.UsageSetupFirst you'll need a Tree-sitter language implementation for each language that you want to parse. You can clone some of theexisting language reposorcreate your own:gitclonehttps://github.com/tree-sitter/tree-sitter-go
gitclonehttps://github.com/tree-sitter/tree-sitter-javascript
gitclonehttps://github.com/tree-sitter/tree-sitter-pythonUse theLanguage.build_librarymethod to compile these into a library that's usable from Python. This function will return immediately if the library has already been compiled since the last time its source code was modified:fromtree_sitterimportLanguage,ParserLanguage.build_library(# Store the library in the `build` directory'build/my-languages.so',# Include one or more languages['vendor/tree-sitter-go','vendor/tree-sitter-javascript','vendor/tree-sitter-python'])Load the languages into your app asLanguageobjects:GO_LANGUAGE=Language('build/my-languages.so','go')JS_LANGUAGE=Language('build/my-languages.so','javascript')PY_LANGUAGE=Language('build/my-languages.so','python')Basic ParsingCreate aParserand configure it to use one of the languages:parser=Parser()parser.set_language(PY_LANGUAGE)Parse some source code:tree=parser.parse(bytes("""def foo():if bar:baz()""","utf8"))If you have your source code in some data structure other than a bytes object,
you can pass a "read" callable to the parse function.The read callable can use either the byte offset or point tuple to read from
buffer and return source code as bytes object. An empty bytes object or None
terminates parsing for that line. The default encoding is utf8.For example, to use the byte offset:src=bytes("""def foo():if bar:baz()""","utf8")defread_callable(byte_offset,point):returnsrc[byte_offset:byte_offset+1]tree=parser.parse(read_callable)And to use the point:src_lines=["def foo():\n","    if bar:\n","        baz()"]defread_callable(byte_offset,point):row,column=pointifrow>=len(src_lines)orcolumn>=len(src_lines[row]):returnNonereturnsrc_lines[row][column:].encode('utf8')tree=parser.parse(read_callable)Or with utf16 encoding:tree=parser.parse(bytes("""def foo():if bar:baz()""","utf16"),encoding="utf16")src=bytes("""def foo():if bar:baz()""","utf16")defread_callable(byte_offset,point):returnsrc[byte_offset:byte_offset+2]tree=parser.parse(read_callable,encoding="utf16")src_lines=["def foo():\n","    if bar:\n","        baz()"]defread_callable(byte_offset,point):row,column=pointifrow>=len(src_lines)orcolumn>=len(src_lines[row].encode("utf-16-le")):returnNoneret=src_lines[row].encode("utf-16-le")[column:]returnrettree=parser.parse(read_callable,encoding="utf16")Inspect the resultingTree:root_node=tree.root_nodeassertroot_node.type=='module'assertroot_node.start_point==(1,0)assertroot_node.end_point==(3,13)function_node=root_node.children[0]assertfunction_node.type=='function_definition'assertfunction_node.child_by_field_name('name').type=='identifier'function_name_node=function_node.children[1]assertfunction_name_node.type=='identifier'assertfunction_name_node.start_point==(1,4)assertfunction_name_node.end_point==(1,7)assertroot_node.sexp()=="(module ""(function_definition ""name: (identifier) ""parameters: (parameters) ""body: (block ""(if_statement ""condition: (identifier) ""consequence: (block ""(expression_statement (call ""function: (identifier) ""arguments: (argument_list))))))))"Walking Syntax TreesIf you need to traverse a large number of nodes efficiently, you can use
aTreeCursor:cursor=tree.walk()assertcursor.node.type=='module'assertcursor.goto_first_child()assertcursor.node.type=='function_definition'assertcursor.goto_first_child()assertcursor.node.type=='def'# Returns `False` because the `def` node has no childrenassertnotcursor.goto_first_child()assertcursor.goto_next_sibling()assertcursor.node.type=='identifier'assertcursor.goto_next_sibling()assertcursor.node.type=='parameters'assertcursor.goto_parent()assertcursor.node.type=='function_definition'EditingWhen a source file is edited, you can edit the syntax tree to keep it in sync with the source:tree.edit(start_byte=5,old_end_byte=5,new_end_byte=5+2,start_point=(0,5),old_end_point=(0,5),new_end_point=(0,5+2),)Then, when you're ready to incorporate the changes into a new syntax tree,
you can callParser.parseagain, but pass in the old tree:new_tree=parser.parse(new_source,tree)This will run much faster than if you were parsing from scratch.TheTree.get_changed_rangesmethod can be called on theoldtree to return
the list of ranges whose syntactic structure has been changed:forchanged_rangeintree.get_changed_ranges(new_tree):print('Changed range:')print(f'  Start point{changed_range.start_point}')print(f'  Start byte{changed_range.start_byte}')print(f'  End point{changed_range.end_point}')print(f'  End byte{changed_range.end_byte}')Pattern-matchingYou can search for patterns in a syntax tree using atree query:query=PY_LANGUAGE.query("""(function_definitionname: (identifier) @function.def)(callfunction: (identifier) @function.call)""")captures=query.captures(tree.root_node)assertlen(captures)==2assertcaptures[0][0]==function_name_nodeassertcaptures[0][1]=="function.def"TheQuery.captures()method takes optionalstart_point,end_point,start_byteandend_bytekeyword arguments which can be used to restrict the
query's range. Only one of the..._byteor..._pointpairs need to be given
to restrict the range. If all are omitted, the entire range of the passed node
is used.