Package: cstq
cstqA very simple and, at least according to the author, intuitive library to navigate Python source code, and code modeling
based onlibcst.Enough said, I need some action!In code as in screenwriting, it's better to show rather than tell. So, here are a couple of examples that scratch the surface of
this library.InstallationFirst things first, let's install the librarypip install cstqExample base codeTo start working with cstq, you can pass the path to a Python file or pass the module directly by:In[1]:fromcstqimportQuery...:...:q=Query(...:"""...: import sys...:...: def main() -> None:...:     import os...:     print('hello world' if os.environ.get("USER") else "who are you?")...:...: if __name__ == "__main__":...:     main()...: """...:)Now lets get down to businessBasic NavigationThere are 3 types of basic navigation:1. Direct Attribute AccessYou can navigate by referencing attributes directly.In[2]:qOut[2]:<CollectionOfNodesnodes=['$(Module)']># access the body attribute of the main moduleIn[3]:q.bodyOut[3]:<CollectionOfNodesnodes=['$(Module).body']># you can get all elementsIn[4]:q.body[:]Out[4]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine)','$(Module).body[1](FunctionDef)','$(Module).body[2](If)']># and get the body element of every element in the body of module# (if they have one)In[5]:q.body[:].bodyOut[5]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine).body','$(Module).body[1](FunctionDef).body(IndentedBlock)','$(Module).body[2](If).body(IndentedBlock)']># or just the body elements of the first element of the moduleIn[6]:q.body[0].body[:]Out[6]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine).body[0](Import)']>and then get the node (and the code) back asIn[7]:q.body[0].body[0].node()Out[7]:Import(names=[ImportAlias(name=Name(value='sys',lpar=[],rpar=[],),asname=None,comma=MaybeSentinel.DEFAULT,),],semicolon=MaybeSentinel.DEFAULT,whitespace_after_import=SimpleWhitespace(value=' ',),)# and then get the code for that nodeIn[8]:q.body[0].body[0].code_for_node()Out[8]:importsysIn[9]:q.body[0].body[:].names[0].name.node()Out[9]:Name(value='sys',lpar=[],rpar=[],)2. FilteringFiltering allows you to "filter" the current selection of nodes to specific ones. Each one of these accepts either alibcst.matchersor a callback (more on callbacks later).libcst.matcherspresents a very powerful query language, and when that's not enough, you can always fall back to a custom callback.In[10]:importlibcst.matchersasm# main root nodesIn[11]:q.body[:]Out[11]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine)','$(Module).body[1](FunctionDef)','$(Module).body[2](If)']># filter out function definitions using explicit filterIn[12]:q.body[:].filter(m.FunctionDef())Out[12]:<CollectionOfNodesnodes=['$(Module).body[1](FunctionDef)']># filter out function definitions using implicit filterIn[13]:q.body[m.FunctionDef()]Out[13]:<CollectionOfNodesnodes=['$(Module).body[1](FunctionDef)']>By using the.filtermethod, you can filter any selection. For instance,q.body[:]represents the elements of the body of
the module. However, you can also filter out by using the__getitem__operation ([.. add your filter here ...]),
making it a bit more compact.3. SearchingIf you want to search, not only in the current selection but also at every level, you can use.search. Similar
to.filter, it accepts alibcst.matchersor a callback.In[14]:q.search(m.Import())Out[14]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine).body[0](Import)','$(Module).body[1](FunctionDef).body(IndentedBlock).body[0](SimpleStatementLine).body[0](Import)']># get the __name__ == "__main__" using search and filterIn[15]:q.search(m.If()).filter(lambdan:n.test.code()=='__name__ == "__main__"')Out[15]:<CollectionOfNodesnodes=['$(Module).body[2](If)']># combining multiple search and filters into a single statementIn[16]:q.search(m.If(),lambdan:n.test.code()=='__name__ == "__main__"')Out[16]:<CollectionOfNodesnodes=['$(Module).body[2](If)']>using callbacks.filterand.searchcan take a callback method that takes anextended versionof a CSTNode and returns true or false.
The extended version of the CSTNode its the regular CSTNode with a couple of extra methods like.parent()to give your the parent,
and.code()to generate the code that node represents, soo far I haven't added anything else.codemod (changes)If you want to change your python document, it's not hard to do, we provide a couple of simple methods like.change(callable, **kwargs): to change the contents of a collection of nodes..replace(node): change the collection of nodes for a new one..remove(): removes every node from the collection of nodes..insert(index, node): if the collection of nodes are a range, this insert the node, in the position index..append(node): add the node to the end of the range of nodes.extend(list[node] | collection of nodes):extend the range to the collection or list of nodesfor instance removing the first import would be as easy as finding it
(using any combination of.searchand/or.filter) and then call.remove()In[17]:q.body[0].body[0].remove()# print the code on topIn[18]:q.code()Out[18]:defmain()->None:importosprint('hello world'ifos.environ.get("USER")else"who are you?")if__name__=="__main__":main()As you can see, there are several empty lines at the top. This is because of 2 things, first the module object is
defined to have 1 empty line as a header, and the funtion def has the leading_lines attribute set to also have a empty lineIn[19]:q.header[:].node()Out[19]:EmptyLine(indent=True,whitespace=SimpleWhitespace(value='',),comment=None,newline=Newline(value=None,),)In[20]:q.body[0]Out[20]:<CollectionOfNodesnodes=['$(Module).body[0](FunctionDef)']>In[21]:q.body[0].leading_lines[:].node()Out[21]:EmptyLine(indent=True,whitespace=SimpleWhitespace(value='',),comment=None,newline=Newline(value=None,),)Let's address this by simply changing the attributeleading_linesin that function definition using the method.changeIn[22]:q.body[0].change(leading_lines=[])Out[22]:<CollectionOfNodesnodes=['$(Module).body[0](FunctionDef)']>In[23]:q.code()Out[23]:defmain()->None:importosprint('hello world'ifos.environ.get("USER")else"who are you?")if__name__=="__main__":main()for more complex changes, instead of passing the attributes to change, you can pass a callback# reverse the order of leading linesIn[24]:q.body[0].change(lambdanode:node.with_changes(leading_lines=node.leading_lines[::-1]))Out[24]:<CollectionOfNodesnodes=['$(Module).body[0](FunctionDef)']>you can replace a node with another one# Lets create an "import from" node and using the serach and node function,# and then lets use that node to replace the "import" on our module.In[25]:import_from=Query("from python_wrapper import os").search(m.ImportFrom()).node()...:...:q.search(m.Import()).replace(import_from)...:q.code()Out[25]:defmain()->None:frompython_wrapperimportosprint('hello world'ifos.environ.get("USER")else"who are you?")if__name__=="__main__":main()To add a import at the top of the file, we can.insertthe new node at the top, like this# Let's add the import at the top levelIn[26]:importlibcstascst...:...:q.body.insert(0,cst.SimpleStatementLine(body=[import_from]))# Let's remove the inner importIn[27]:q.search(m.FunctionDef()).search(m.ImportFrom()).remove()# Let's print the resultIn[28]:q.code()Out[28]:frompython_wrapperimportosdefmain()->None:print('hello world'ifos.environ.get("USER")else"who are you?")if__name__=="__main__":main()adding a call at the end would be as easy as# using extend to add a few lines at the end of the documentIn[29]:EXTRA_LINES=Query(...:"""...: import my_custom_logging...: my_custom_logging.log(__file__)...:     """...:)...:...:q.body.extend(EXTRA_LINES.body[:])...:...:q.code()Out[29]:frompython_wrapperimportosdefmain()->None:print('hello world'ifos.environ.get("USER")else"who are you?")if__name__=="__main__":main()importmy_custom_loggingmy_custom_logging.log(__file__)CST to objects and object to CSTOne particular thing this library tries to do, its to give you many representations of the data, so you
can interact with the source code as conformable as you can, for this sometimes you want to grab a python
object from the source code and treat it as its regular object and then turn the result back to its
CST representation this is not extremely hard to doIn[30]:fromcstqimportQuery,obj2cst...:importlibcst.matchersasm...:...:q=Query(...:"""...: X = [1, 2, 3, "foo", 3j]...: """...:)...:...:the_list=q.search(m.List())...:the_listOut[30]:<CollectionOfNodesnodes=['$(Module).body[0](SimpleStatementLine).body[0](Assign).value(List)']># now we can turn that cst.List object into a python list using using `literal_eval_for_node`In[31]:real_list=the_list.literal_eval_for_node()...:f"type({real_list}) ={type(real_list)}"Out[31]:type([1,2,3,'foo',3j])=<class'list'># lets remove non integers for it and lets write it back to the cstIn[32]:only_int_list=obj2cst([eforeinreal_listifisinstance(e,int)])...:...:the_list.replace(only_int_list)...:...:q.code()Out[32]:X=[1,2,3]Licensecstqis distributed under the terms of theMITlicense.