Package: plusplus
plusplusEnable the++xand--xexpressions in PythonWhat's this?By default, Python does not support neither pre-increments (like++x) nor post-increments (likex++).
However, the first ones aresyntactically correctsince Python parses them as two subsequent+xoperations,
where+is theunary plus operator(same with--xand theunary minus).
They both have no effect, since in practice-(-x) == +(+x) == x.This module turns the++x-like expressions intox += 1at the bytecode level.
Increments and decrements of collection items and object attributes are supported as well, for example:dictionary={'key':42}++dictionary['key']assertdictionary['key']==43Unlikex += 1,++xis still an expression, so the increments work fine inside other expressions,if/whileconditions, lambda functions, and list/dict comprehensions:array[++index]=new_valueif--connection.num_users==0:connection.close()button.add_click_callback(lambda:++counter)index=0indexed_cells={++index:cellforrowintableforcellinrow}Seetestsfor more sophisticated examples.[How it works] [Installation]Why?This module is made for fun, as a demonstration of Python flexibility and bytecode manipulation techniques.
Note that enabling increments in real projects may be risky: such code may confuse new developers and
behave differently if copied to environments without theplusplusmodule. Also, this feature gives more opportunities
to write unreadable code in general.Nevertheless, there are situations where increments (if used with care) may allow to avoid repetitions or
make code more readable. Some of them are listedherewith the examples from
the source code of the Python standard library.Also, having the increment expressions seems consistent withPEP 572 "Assignment Expressions"that introduced thex := valueexpressions in Python 3.8+.
They can be used insideif/whileconditions and lambda functions as well.How it works?Patching bytecodePython compiles all source code to a low-levelbytecodeexecuted on the Python's stack-based virtual machine. Each bytecode instruction consumes a few items from the stack,
does something with them, and pushes the results back to the stack.The++xexpressions are compiled into two consecutiveUNARY_POSITIVEinstructions
that do not save the intermediate result in between (same with--xand twoUNARY_NEGATIVEinstructions).
No other expressions produce a similar bytecode pattern.plusplusreplaces these patterns with the bytecode forx += 1, then adds the bytecode for storing
the resulting value to the place where the initial value was taken.This is what happens for they = ++xline:A similar but more complex transformation happens for the code with subscription expressions
likevalue = ++dictionary['key']. We need the instructions from the yellow boxes to save the initial location and
recall it when the increment is done (see the explanation below):This bytecode is similar to what the stringdictionary['key'] += 1compiles to. The only difference is that it
keeps an extra copy of the incremented value,
so we can return it from the expression and assign it to thevaluevariable.Arguably, the least clear part here is the second yellow box. Actually, it is only needed to reorder
the top 4 items of the stack. If we need to reorder the top 2 or 3 items of the stack, we can just use
theROT_TWOandROT_THREEinstructions (they do a circular shift
of the specified number of items of the stack). If we had aROT_FOURinstruction, we would be able to just
replace the second yellow box with twoROT_FOURs to achieve the desired order.However,ROT_FOURwas removed in Python 3.2
(since it wasrarely usedby the compiler) and
recovered back only in Python 3.8. If we want to support Python 3.3 - 3.7, we need to use a workaround,
e.g. theBUILD_TUPLEandUNPACK_SEQUENCEinstructions.
The first one replaces the top N items of the stack with a tuple made of these N items. The second unpacks the tuple
putting the values on the stack right-to-left, i.e.in reverse order. We use them to reverse the top 4 items,
then swap the top two to achieve the desired order.[Source code]The @enable_increments decoratorThe first way to enable the increments is to use a decorator that would patch the bytecode of a given function.The decorator disassembles the bytecode, patches the patterns described above, and recursively calls itself
for any nested bytecode objects (this way, the nested function and class definitions are also patched).The bytecode is disassembled and assembled back
using theMatthieuDartiailh/bytecodelibrary.[Source code]Enabling increments in the whole packageThe Python import system allows loading modules not only from files but from any reasonable place
(e.g. there was amodulethat enables importing code
from Stack Overflow answers). The only thing you need is to provide module contents, including its bytecode.We can leverage this to implement a wrapping loader that imports the module as usual but patching its bytecode
as described above. To do this, we can create a newMetaPathFinderand install it
tosys.meta_path.[Source code]Why not just override unary plus operator?Overriding operators via magic methods
(such as__pos__()and__neg__())
do not work for built-in Python types likeint,float, etc.
In contrast,plusplusworks with all built-in and user-defined types.Caveatspytestdoes its own bytecode modifications in tests, adding the code to save intermediate expression results
to theassertstatements. This is necessary to show these results if the test fails
(seepytest docs).By default, this breaks thepluspluspatcher because the twoUNARY_POSITIVEinstructions become
separated by the code saving the result of the firstUNARY_POSITIVE.We fix that by removing the code saving some of the intermediate results, which does not break
the pytest introspection.[Source code]How to use it?You can install this module with pip:pip install plusplusFor a particular function or methodAdd a decorator:fromplusplusimportenable_increments@enable_incrementsdefincrement_and_return(x):return++xThis enables increments for all code inside the function, including nested function and class definitions.For all code in your packageInpackage/__init__.py, make this callbeforeyou import submodules:fromplusplusimportenable_incrementsenable_increments(__name__)# Import submodules here...This enables increments in the submodules, but not in thepackage/__init__.pycode itself.See alsocpmoptimize— a module that optimizes a Python code
calculating linear recurrences, reducing the time complexity from O(n) to O(log n).dontasq— a module that adds functional-style methods
(such as.where(),.group_by(),.order_by()) to built-in Python collections.AuthorsCopyright © 2021Alexander Borzunov