Package: wrappingpaper
wrappingpaperA collection of Python decorators and utilities to abstract away common/tedious Python patterns.NotesThis package is more about providing interesting abstractions and trying to flesh out the possibilities of Python code organization. I am in no way saying that using these functions will provide "good" code and I am in no way condoning their use for creating evil Python code ;).Some of the functions in here may incentivize less understandable code, but that's okay. I want to give them space to exist and hopefully we can develop them further to where they will be more understandable and provide more intuitive and familiar abstractions.This package is about experimentation and trying to create basic, interesting, natural feeling, and convenient abstractions while sidelining the scrutiny of Python purists, and potentially people with more sense (!!). I want this to try to push the limits of the language to see what other interesting constructs we can facilitate.So, I guess the motto of this package is to develop freely, but use responsibly. <3One other thing to note though, some of these don't play nice with linters 😢Simple Exampleimportwrappingpaperaswp@wp.contextdecoratordefdoing_something(a,b):print(a)yieldprint(b)# por que no los dos?# you can do thiswithdoing_something(4,5):print(1)# prints 4 1 5# as well as this@doing_something(4,5)defsomething():print(1)something()# prints 4 1 5Includeshelper modulesreal implementations of faux imports, meant as case-studies for import mechanic classes provided.logging / error handlingcatch errors thrown in a function and redirect to loggercontext managerscontext managers that double as function wrappersobject propertiesclass and instance cachingdynamic property objects - give properties nested attributes and methods !!function signature helpersoverride and apply updates to function signaturesfilter function arguments that are outside the function schemapartial that actually updates the wrapperimport mechanicscreate faux modules and customize how modules are imported (I did some of the confusing bits for us thankfully)iterablesincludes some basic iterable functions that I've pulled from other projects so I don't have to keep duplicating them everywheremiscstuff I just haven't sorted. ya know?retry on exceptioncheck circular referencesInstallpipinstallwrappingpaperUsageimportwrappingpaperaswpHelper ModulesThese are faux modules that utilizewrappingpaper'simport mechanicsto alter modules that are imported from them.lazyimportThis is a simple implementation of lazy importing using the defined import mechanics.fromlazyimportimportsklearnimportlibrosa# sklearn imports will be lazypresetsThis is a re-implimentation ofbmcfee/presetsthat includes the import mechanics, instead of having to wrap modules afterwards. I may add a PR to that package, but implementing it here was trivial for the time being and I didn't feel like it was important enough to push it thru the review process.frompresetsimportlibrosalibrosa.update(sr=44100)# now functions will default to sr=44100LoggingNOTE: I haven't put in the work to mock logging objects for testing so beware that in their current form they are untested and most likely have 1 or 2 bugs in there.I was working on a project that was full of error suppression and logging. There would be functions wrapped in try except blocks, logging calls, and a lot of redundancy in the scaffolding needed.So I did work to factor that out and perform many of the common patterns in decorators.The logging decorators here are primarily for functions that can be permitted to fail and return a default/empty value without the rest of the program breaking.It also has utilities for pulling information from tracebacks. I haven't done anything about the logging Handlers and Formatters so that's a TODO.importlogginglog=logging.getLogger(__name__)# handle and log error@wp.log_error_as_warning(log,default=dict)defget_stats(x=None):ifxisTrue:raiseValueError()# some error happensreturn{'a':5,'b':6}assertget_stats()=={'a':5,'b':6}assertget_stats(True)=={}Roughly equivalent to:defget_stats(x=None):try:ifxisTrue:raiseValueError()# some error happensreturn{'a':5,'b':6}exceptValueErrorase:log.warning('Exception in get_stats:%s',e)return{}Context ManagersTwo common patterns in Python are context managers and decorators. Often, they have the same basic structure: do some initialization, run a function, and do some cleanup.And both can be useful in different contexts to give you clean code, but to use both, I often find myself writing an additional wrapper function around the context manager, and then you have to give it a slightly different name and it can get confusing.So, in comescontextdecoratorwhich works the same ascontextlib.contextmanager, but it also doubles as a function decorator. When used as a decorator, it will call the function inside the context manager.@wp.contextdecoratordefdoing_something(a,b):print(a)yieldprint(b)# por que no los dos?# you can do thiswithdoing_something(4,5):print(1)# as well as this@doing_something(4,5)defsomething():print(1)something()Sometimes, your decorator isn't as simple and you need to do things a bit differently in the decorator (e.g. you need the name of the wrapped function).@doing_something.caller# override default decoratordefdoing_something(func,a,b):# wrapped function, decorator arguments# change argumentsname=func.__name__a='calling{}:{}'.format(name,a)b='calling{}:{}'.format(name,b)# return the wrapped function@functools.wraps(func)definner(*args,**kw):withdoing_something(a,b):returnfunc(*args,**kw)returninnerRoughly equivalent to:importfunctoolsfromcontextlibimportcontextmanager@contextmanagerdefdoing_something(a,b):print(a)yieldprint(b)defdoing_something2(a,b):defouter(func):@functools.wraps(func)definner(*a,**kw):withdoing_something(a,b):returnfunc(*a,**kw)returninnerreturnouter# used like:withdoing_something(4,5):print(1)@doing_something2(4,5)defsomething():print(1)something()PropertiesPython property objects are incredibly useful as they allow you to create natural feeling objects with some complex stuff all bundled up in a nice unsuspecting interface.But using them, there are often times where I find myself writing the same classes stored many times over in utility files.One use-case is caching. There are different levels of caching that you can provide.cachedproperty: cached on the instance object - runs once per instanceonceproperty: cached on the class object - runs once per class/baseclassoverridable_property: works as a normal property (calls the wrapped function), until the property is assigned to. Then it returns the assigned value.overridable_method: works as a normal method (calls the wrapped function), until the function is called as a decorator. Then it calls the wrapped function. Works on an instance level.importtimeclassSomeClass:@wp.cachedpropertydefinstance_prop(self):'''This is run once per object instance.'''returntime.time()@wp.oncepropertydefclass_prop(self):'''This is run once. It is cached in the propertyobject itself.'''returntime.time()@wp.overridable_propertydefoverridable(self):'''This property is run normally, until another value is assigned on top.'''returntime.time()def__init__(self,overridable=None):ifoverridable:# override the property value# stores at self._overridableself.overridable=overridable# otherwise it just uses the property function like usuala=SomeClass()b=SomeClass()asserta.instance_prop!=b.instance_prop# prop runs once per objectasserta.class_prop==b.class_prop# prop runs only onceasserta.overridable!=a.overridable# gets called twice, shouldn't be the samea.overridable=5asserta.overridable==5# now the value is overriddenassertSomeClass(5).overridable==5# overriding inside classFunction SignatureThis is something that I'm looking for constantly.Personally, I like the idea of config files that wrap up a bunch of function arguments into a file.I also hate having to duplicate arguments when passing variables down 5 levels of nested function calls.I like to just pass keyword arguments (**kw) down to the next function.But there are cases, where there are extra config values in your keyword dict and you only want to pass the values that your function takes.# dynamic function defaults@wp.configfunctiondefasdf(a=5,b=6,c=7):returna+b+cassertasdf()==5+6+7# normal behaviorasdf.update(a=1)assertasdf()==1+6+7# updated defaultassertasdf(3)==3+6+7# automatically resolves kwargs and posargsasdf.clear()assertasdf()==5+6+7# back to normal behavior# filter out kwargs not in the signature (if **kw, it's a no-op).@wp.filterkwdefasdf(a=5,b=6,c=7):returna+b+cassertasdf(b=10,d=1234)==5+10+7ObjectsMonkeypatchingclassBlah:defasdf(self):return10b=Blah()@wp.monkeypatch(b)defasdf():return11assertasdf()==11asdf.reset()# remove patchassertasdf()==10asdf.repatch()# re-place the patchassertasdf()==11Namespaceclasssomething(metaclass=wp.namespace):one_thing=5other_thing=6defblah(x):returnone_thing+other_thing+xassertsomething.blah(10)==5+6+10Iterables###################### loop breaking#####################items=wp.until(xifx!=7elsewp.doneforxinrange(10))assertlist(items)==list(range(0,6))##################### loop throttling##################### make sure that a for loop doesn't go too fast.# limit the time one iteration takes.t0=time.time()forxinwp.throttled(range(10),1):print(x)asserttime.time()-t0>10# limiting the number of iterations to 10.# with no iterable passed, it loops infinitely and# yields the total yield time and the time it had to sleep.fordt,time_asleepinwp.limit(wp.throttled(secs=1),10):print('Iteration took{}s. Had to sleep for{}s.'.format(dt,time_asleep))print('-'*10)################################# Use `while True:` in a loop################################for_inwp.infinite():print('this is gonna be a while...')########################## pre-check an iterable########################## check the first n items in an iterable, without removing them.it=iter(range(6))items,it=wp.pre_check_iter(it,3)assertitems==[0,1,2]assertlist(it)==[0,1,2,3,4,5,6]############################################ repeat and chain a function infinitely###########################################importrandomdefget_numbers():# function returns an iterablereturn[random.random()for_inrange(10)]numbers=wp.run_iter_forever(get_numbers)# repeat get_numbers() and chain iterable outputs togetherall_numbers=list(wp.limit(numbers,100))assertall(isinstance(x,float)forxinall_numbers)# If no items are returned by a call, instead of the iterable hanging# indefinitely waiting for an item, return None.defget_numbers():ifrandom.random()>0.8:# make random breaksreturn# returns emptyreturn[random.random()for_inrange(10)]numbers=wp.run_iter_forever(get_numbers,none_if_empty=True)# this SHOULD contain sporadic None's at a multiple of 10all_numbers=list(wp.limit(numbers,5000))assertNoneinall_numbersImport MechanicsThis is probably the most dangerous thing to be playing with in here.Python exposes a lot of its internal mechanics including its import system.So we can take advantage of that to provide import wrappers that modify module behavior.A basic example - lazy loading:# lazyimport/__init__.pyimportwrappingpaperaswpwp.lazy_loader.activate(__name__)# main.pyfromlazyimportimportsklearn.model_selection# sklearn is not currently loadedsklearn.model_selection.train_test_split()# now it's loaded.Modify a module after it has been imported from your pseudo-moduleimportwrappingpaperaswp@wp.PseudoImportFinder.modulemodifierdefmy_loader(module):module.sneakything='......hi'my_loader.activate('somethingrandom')# now somewhere else, you can dofromsomethingrandomimportnumpyasnpassertnp.sneakything=='......hi'Wrap a module to modify the module's contentsimportimportlibimportwrappingpaperaswp# create the module wrapper that will traverse and modify the module when it is loaded.classModule(wp.ModuleWrapper):# this is called for each item in the moduledef_wrapattr(self,attr,value):# do whatever you want with the valueifcallable(value)andgetattr(value,'__doc__',None)isnotNone:value.__doc__+='\nI was here.'elifself._is_submodule(value):value=Module(value)# always pass attr and modified value to be set,# otherwise it will be undefined.super()._wrapattr(attr,value)# applies the module wrapper on load@wp.PseudoImportFinder.moduleloaderdefmy_loader(spec):returnModule(importlib.util.module_from_spec(spec))# somewhere else (or in the same place. I'm not ur mom), actually use itwithmy_loader.activated('somethingrandom'):# activated only inside contextfromsomethingrandomimportglobprint(glob.glob.__doc__)assertglob.glob.__doc__.endswith('I was here.')MiscSome other miscellaneous stuff that I have yet to organize.importrandom# retry a function if an exception is raised@wp.retry_on_failure(10)defasdf():x=random.random()ifx<0.5:raiseValueErrorreturnx# will either return a number that is definitely > 0.5# or every number in the first 10 tries were below 0.5try:assertasdf()>0.5exceptValueError:print("Couldn't get a number :/")# ignore errorwithwp.ignore():a,b=5,0c=a/b# throws divide by zeroa=10# never runasserta==5