Package: kwonly-args
Providing keyword-only argument emulation for python2. The resulting code is python3 compatible.You have to read only the shortUsagesection of the document to get started, the rest is optional for curious people and
bored time billionaires.ContentsUsageInstallationBrief how-to with code examplesCode style/design: why use keyword-only arguments?Code readabilityEasier maintenance and refactorizationImplementation detailsPython 2 function signature anatomyEmulated keyword-only args VS static analyzersPoor man’s python2 keyword-only argumentsWhy does this “library” exist?UsageInstallationpipinstallkwonly-argsAlternatively you can download the zipped library fromhttps://pypi.python.org/pypi/kwonly-argsBrief how-to with code examplesWith this library you can turn some or all of the default arguments of your function into keyword-only arguments.Decorate your function withkwonly_args.first_kwonly_argand select one of the default arguments of your function
with thenameparameter of the decorator. The selected argument along with all default arguments on its right
side will be treated as keyword-only arguments.All keyword-only arguments have a default value and they aren’t required args by default. You can make a
keyword-only argument required by usingkwonly_args.KWONLY_REQUIREDas its default value.Your new-born keyword-only args are no longer treated as positional arguments and varargs still work if your function
has*argsor something like that.fromkwonly_argsimportfirst_kwonly_arg,KWONLY_REQUIRED# This turns default1 and default2 into keyword-only arguments.# They are no longer handled as positional arguments.@first_kwonly_arg('default1')deffunc(arg0,arg1,default0='d0',default1='d1',default2='d2',*args):print('arg0=%sarg1=%sdefault0=%sdefault1=%sdefault2=%sargs=%s'%(arg0,arg1,default0,default1,default2,args))func(0,1,2,3,4)# Output:# arg0=0 arg1=1 default0=2 default1=d1 default=d2 args=(3, 4)# The default1 and default2 args can be passed only as keyword arguments:func(0,1,2,3,4,default1='kwonly_param')# Output:# arg0=0 arg1=1 default0=2 default1=kwonly_param default=d2 args=(3, 4)# In this example all three args are keyword-only args and default1 is required.@first_kwonly_arg('default0')deffunc2(default0='d0',default1=KWONLY_REQUIRED,default2='d2'):...You can also decorate class methods (including both old and new style classes):fromkwonly_argsimportfirst_kwonly_argclassMyClass:# turning d1 and d2 into keyword-only arguments@first_kwonly_arg('d1')defmy_instance_method(self,a0,a1,d0='d0',d1='d1',d2='d2',*args):...# You have to apply @first_kwonly_arg before @classmethod!@classmethod@first_kwonly_arg('d1')defmy_class_method(cls,a0,a1,d0='d0',d1='d1',d2='d2',*args):...# You have to apply @first_kwonly_arg before @staticmethod!@staticmethod@first_kwonly_arg('d1')defmy_static_method(a0,a1,d0='d0',d1='d1',d2='d2',*args):...If you want to turn all default arguments into keyword-only arguments then the following convenience API may be useful:fromkwonly_argsimportfirst_kwonly_arg,FIRST_DEFAULT_ARG,kwonly_defaults# The FIRST_DEFAULT_ARG constant automatically selects the first default# argument (default0) so it turns all default arguments into keyword-only.@first_kwonly_arg(FIRST_DEFAULT_ARG)deffunc(arg0,arg1,default0='d0',default1='d1',*args):...# As an equivalent shortcut you can use @kwonly_defaults.@kwonly_defaultsdeffunc(arg0,arg1,default0='d0',default1='d1',*args):...Code style/design: why use keyword-only arguments?You may have an understanding of this topic. If not then read along.
Using keyword-only arguments provides the following benefits:Code readabilityIt can make code that calls your function more readable. This is especially true if you have several functions with
long argument lists like some of the python standard library APIs. For examplesubprocess.Popen()has more than
10 arguments.subprocess.Popen()is a legacy function from python2 (so it couldn’t make use of keyword-only
arguments despite being a very good candidate for that) but some newer python3 APIs make use of keyword-only
arguments with a good reason. For example the python3subprocess.run()has about 10 arguments but only
the firstargvargument can be passed as positional, the rest are keyword-only.defdraw_circle(x,y,radius,filled=False):...defdraw_ellipse(x,y,radius_x,radius_y,filled=False):...# 1. calling without using keyword arguments:draw_circle(100,200,50,True)draw_ellipse(200,100,100,50)# 2. calling using keyword arguments:draw_circle(x=100,y=200,radius=50,filled=True)draw_ellipse(x=200,y=100,radius_x=100,radius_y=50)Without keyword-only arguments users of your function will be able to use both of the above conventions. If you
employ keyword-only arguments then they can use only #2. In case of a simple function like mydraw_circle()it
may not seem reasonable enough to force keyword-only arguments. But imagine what happens if you start having many
similar functions likedraw_ellpise(),draw_rectangle(), etc.. and you have to read code that calls these
without keyword arguments with a bunch of listed numbers and bools mixed together as their input… The above
example in section #1 is relatively lightweight compared to what it can look in real life.When a function has more than 3-4 arguments (likesubprocess.Popen()) I think it is a very good practice to
allow at most the first few (or none of the) arguments to be passed as positional ones and make the rest kw-only
(like the standard python3subprocess.run()).
It isn’t a problem if a function has a lot of parameters (especially default ones) as long as the code that calls
the function remains readable by using keyword argument passing and you can enforce/guarantee that by making the
most of the arguments keyword-only:importsubprocessargv=['ls','-l']# BAD! I think I don't really have to explain why...p=subprocess.Popen(argv,-1,None,subprocess.PIPE,subprocess.PIPE,subprocess.STDOUT,None,True,True)# GOOD! And this has the same behavior as the previous call.# I think it is well worth enforcing this form with keyword-only args.p=subprocess.Popen(argv,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,shell=True)# If the number of passed arguments exceeds my threshold# I switch to the following format for readability:p=subprocess.Popen(argv,stdin=subprocess.PIPE,stdout=subprocess.PIPE,stderr=subprocess.STDOUT,shell=True,)Easier maintenance and refactorizationKeyword-only args have an extremely useful property: you can declare them in any order in your function signature and
the code that calls your function can also pass them in any order.
Later you can change the order of declaration of your keyword-only arguments for cosmetic and readability reasons
without affecting behavior and without having to refactor code that calls this function. This comes in handy not only
in case of code cosmetics but also makes it easier to add new keyword-only args and to remove old ones if necessary.
Let’s review these scenarios with code examples.Imagine a scenario where you have adraw_circle(x, y, radius, outline_color=black, filled=False, fill_color=None)function. It already looks bad enough without keyword-only args. Let’s imagine that someone asks you to add anoutline_widthargument. Since all parameters can be passed as positional arguments you have to keep backward
compatibility and you have to append this argument to the end of the current arg list with a default value. This
introduces another ugly thing: the arguments that belong to the outline aren’t adjacent. There will be two unrelated
args betweenoutline_colorand the newly addedoutline_width. If these args were keyword-only arguments then
the arbitrary argument order would allow you to insert the newoutline_widtharg right afteroutline_color.Another typical and similar scenario is having a function that makes use of 2 or more other functions. For this reason
it receives input args and passes them through to the two other functions. Let’s say you start out with something like
this at the beginning of your project:# lower level workhorse functions used by the higher level ``my_func()``defworkhorse1(wh1_1,wh1_2):...defworkhorse2(wh2_1,wh2_2):...# And your function looks like thisdefmy_func(wh1_1,wh2_1,wh2_2):# TODO: perhaps manipulate the input args...workhorse1(wh1_1,8)workhorse2(wh2_1,wh2_2)Then for some reason someone introduces a newwh1_3parameter forworkhorse1()and you have to pass it through
your higher levelmy_func(). It will look like this:# One arg for wh1, then two args for wh2 and then another arg for wh1... Nice.defmy_func(wh1_1,wh2_1,wh2_2,wh1_3):# TODO: perhaps manipulate the input args...workhorse1(wh1_1,8)workhorse2(wh2_1,wh2_2)In python you can avoid such scenarios by passing such arguments in**kwargsor in separate dictionaries but it
often makes the code less explicit and readable:# It is more difficult to find out what's going on with ``*args``# and ``**kwargs`` then with explicitly named arguments.defmy_func(**kwargs):# Let the workhorses to cherry pick the parameters they# need and ignore the rest that they don't need.workhorse1(**kwargs)workhorse2(**kwargs)You can also use two separate dictionaries or data objects to pass the arguments to the workhorses. This technique
is better than keyword only argument passing when the workhorses have a lot of parameters and/or you have to pass
the arguments deeply through several calls but this solution is an an overkill in many simpler situations where the
number of parameters isn’t too high and there is no deep arg passing:defmy_func(wh1_args,wh2_args):# TODO: perhaps manipulate the input args...workhorse1(wh1_args)workhorse2(wh2_args)With keyword-only arguments the above problems don’t exist. The newwh1_3argument can be placed anywhere in the
keyword-only argument part of the argument list (e.g.: afterwh1_1) without affecting the rest of the code that
already calls this functions with other keyword-only args (given that they don’t want to use the newly added arg).Implementation detailsPython 2 function signature anatomyA python2 function signature consists of the following optional parts. Any optional parts that are present in
a function signature appear in the listed order:Positional argumentsRequired arguments (positional arguments without default value)Default arguments (positional arguments with default value)Keyword-only arguments (non-standard, emulated/provided by this library)VarArgs (*args)VarKWArgs (**kwargs)As you see in standard python2 your positional argument list consists of zero or more required arguments followed by
zero or more default arguments. This library can turn the last N default arguments (all/some of them) into keyword-only
arguments. With the help of this library you can now split the positional argument list of your python2 function
signatures into 3 parts instead of the standard 2.In python3 the keyword-only arguments reside between VarArgs and VarKWArgs but in python2 you can’t put anything
between those (it would be a syntax error) so your best bet to emulate keyword-only arguments is turning some of your
positional arguments into keyword-only args.Emulated keyword-only args VS static analyzersAs discussed previously unfortunately we can declare our emulated python2 keyword-only arguments only before the
VarArgs (*args) of the function. This means that our signature can have positional arguments not only before our
keyword-only args, but also after them (because VarArgs are positional). This may lead to false-positive
warnings/errors with static analyzers in the following case:If you have a function with both keyword-only arguments and VarArgs then static analyzers may treat some
of the calls to this function suspicious (resulting in a false positive warning/error).@first_kwonly_arg('ko0')deffunc(a0,d0=-1,ko0=-1,ko1=-1,*args):...# No problem: a0=0func(0)# No problem: a0=0, d0=1func(0,1)# No problem: a0=0 d0=1 args=(2,)func(0,1,2)# The static analyzer will probably treat this as an error. It thinks that# you pass both the positional argument 2 and ko0=3 to the ko0 arg of the# function because it can't track down the magic done by the @first_kwonly_arg# decorator and binds the passed parameters to the function args using standard# python2 rules. If func() didn't have our @first_kwonly_arg decorator then# this function call would probably cause an error like:# TypeError: func() got multiple values for argument 'ko0'## However what actually happens as a result of the magic done by the# decorator is: a0=0 d0=1 ko0=3 ko1=-1 args=(2,)# The decorator ensures that positional parameters passed by function calls# are bound only to positional non-keyword-only arguments and the VarArgs# of the function.func(0,1,2,ko0=3)# No problem despite the fact that the static analyzer probably assumes# something different than what actually happens. According to standard# python2 arg binding rules the static analyzer probably thinks that:# a0=0 d0=1 ko0=2 ko1=3 args=()## However the actual outcome caused by our decorator is:# a0=0 d0=1 ko0=-1 ko1=3 args=(2,)func(0,1,2,ko1=3)Despite the above issue a decorator like this can still be very useful. The reason for this is that for me it happens
quite rarely that in a function I need both keyword-only arguments and VarArgs. I need VarArgs quite rarely in general
while keyword-only arguments come in handy quite often. If this is the same for you then go on using this decorator in
your python2 projects and in the rare cases where you need both keyword-only arguments and VarArgs use one of the
following workarounds to aid this issue:Static analyzers are stupid! Don’t use them! (Well, this was only a joke. :-D)In your static analyzer tool or service ignore the individual instances of these false positive warnings.UsePoor man’s python2 keyword-only argumentswith these problematic cases instead of decorating them and use the
decorator only with the rest (probably the majority) of the functions that don’t have VarArgs.Poor man’s python2 keyword-only argumentsI really like the benefits brought by keyword-only arguments. Long ago before extensively working with python I’ve
already forged some coding-convention rules that have similar advantages (unordered arguments, specifying arg names
while calling the function for readability) in other languages (e.g.: C/C++). Before thinking about using a python2
solution like the one provided by this library I’ve used a “manually implemented poor man’s python2 keyword-only args”
solution like this:deffunc(arg0,arg1,default0='d0',default1='d1',**kwargs):# Keyword-only arg with a default value:optional_kwonly0=kwargs.pop('kwonly0','ko0')# Required keyword-only arg:required_kwonly1=kwargs.pop('kwonly1')# Checking whether the caller has passed an unexpected keyword argument.# Sometimes passing an unexpected keyword argument is simply the result# of a typo in the name of an expected arg. E.g.: kwnly0 instead of kwonly0check_no_kwargs_left(func,kwargs)# ... the rest of the function body# utility function far away somewhere in a central place...defcheck_no_kwargs_left(func_or_func_name,kwargs):ifnotkwargs:returnfunc_name=func_or_func_name.__name__ifcallable(func_or_func_name)elsefunc_or_func_namearg_names=', '.join(repr(k)forkinsorted(kwargs.keys()))raiseTypeError('%s() got unexpected keyword argument(s):%s'%(func_name,arg_names))While I think the above solution if fairly good it still requires checking the function body too in order to see the
full signature and sometimes people may forget to check for leftover kwargs after popping the kwonly args.Why does this “library” exist?I’ve checked out some other python2 keyword-only argument emulator code snippets and decided to roll my own just for
fun and also for the following reasons:Some of those implementations provide you with a decorator with which you have to specify your keyword-only arguments
with their (usually zero based) index in the arg list of the function. This is error prone, I never liked the
idea of identifying arguments with indexes. The only minor disadvantage of using arg names instead of arg indexes
is that using arg names requires direct access to the signature of theoriginalwrapped function.
If there are other decorators between our decorator and the original function then under python2 using names isn’t
really possible (becausefunctools.update_wrapper()and decorators in general don’t have/support the__wrapped__attribute to maintain a chain back to the originally wrapped function).Some implementations allow you to pick an arbitrary set of positional arguments by specifying their indexes or names.
I don’t like the idea of promoting arbitrary positional arguments into keyword-only arguments by scattering
keyword-only args through the remaining positional args. It degrades code readability a lot. This is why I decided
to keep positional arguments of the same type (required/default/kwonly) together in a well defined slice of the
positional argument list.The implementation of this solutionis brief (~40 lines of logic), simple, and well tested.