Package: humpack
This package contains several high level utilities for python development, including:Packing: Human Readable Serialization of Python ObjectsTransactions: Grouping multiple python statements into an atomic operation that can be committed or abortedContainers: dropin replacements for several common python types (includingdict,list,set, etc.)Cryptography: simple utilities for common cryptography functionality (very high-level on top of standard python libraries)InstallEverything is tested with Python 3.7 on Ubuntu 18.04, but there is no reason it shouldn’t also work for Windows.You can install this package through pip:pipinstallhumpackAlternatively, you can clone this repo and install the local version for development:gitclonehttps://github.com/felixludos/HumPackpipinstall-e./HumPackQuick StartContainersThe provided containers:tdict,tlist, andtsetserve as drop-in replacements for pythonsdict,list, andsettypes that areTransactionableandPackable(more info below). Furthermore, all keys inadictthat are valid attribute names, can be treated as attributes.A few examples:fromhumpackimportadict,tdict,tlist,tsetfromhumpackimportjson_pack,json_unpackfromhumpackimportAbortTransactiond=adict({'apple':1,'orange':10,'pear':3})d.apple+=10d.update({'non-det banana':tset({2,3,7}),'orange':None})deld.pearassertd.apple==11and2ind['non-det banana']and'pear'notindoptions=tlist(d.keys())options.sort()first=options[0]assertfirst=='apple'd.order=optionsjson_d=json_pack(d)assertisinstance(json_d,str)d.begin()# starts a transaction (tracking all changes)assertoptions.in_transaction()d['non-det banana'].discard(7)d.cherry=4.2assert'cherry'indandlen(d['non-det banana'])==2d['order'].extend(['grape','lemon','apricot'])assert'grape'inoptionsdeld.order[0]deld['orange']d.order.sort()assertoptions[0]=='apricot'd.abort()assert'cherry'notindand7ind['non-det banana']assert'grape'notinoptionswithd:assertd['non-det banana'].in_transaction()d.clear()assertlen(d)==0d.melon=100jassert'melon'indandd['melon'].real==0raiseAbortTransactionassert'melon'notindassertjson_pack(d)==json_dassertsum(d['non-det banana'])==sum(json_unpack(json_d)['non-det banana'])withd:assert'cherry'notindd.cherry=5# automatically commits transaction on exiting the context if no exception is thrownassert'cherry'indWhen starting with data in standard python, it can be converted to using the “t” series counter parts usingcontainerify.fromhumpackimportcontainerifyfromhumpackimportAbortTransactionx={'one':1,1:2,None:['hello',123j,{1,3,4,5}]}d=containerify(x)assertlen(x)==len(d)assertlen(x[None])==len(d[None])assertx['one']==d.onewithd:assertd[None][-1].in_transaction()deld.oned.two=2d[None][-1].add(1000)assertd['two']==2and'one'notindandsum(d[None][-1])>1000raiseAbortTransactionassert1000notind[None][-1]and'one'indand'two'notindFinally, there are a few useful containers which don’t have explicit types in standard python are also provided including heaps and stacks:theapandtstack.Packing (serialization)To serializing an object into a human-readable, json compliant format, this library implements packing and unpacking. When an object is packed, it can still be read (and manipulated, although that not recommended), converted to a valid json string, or encrypted/decrypted (see the Security section below). However for an obejct to be packable it and all of it’s submembers (recursively) must either be primitives (int,float,str,bool,None) or registered as aPackable, which can be donePacking and unpacking is primarily done using thepackandunpackfunctions, however, several higher level functions are provided to combine packing and unpacking with other common features in object serialization. For custom classes to bePackable, they must implement three methods:__pack__,__create__,__unpack__(for more info see the documentation forPackable). When implementing these methods, all members of the objects that should be packed/unpacked, must usepack_memberandunpack_memberto avoid reference loops.fromhumpackimportpack,unpackx={'one':1,1:2,None:['hello',123j,{1,3,4,5}]}p=pack(x)# several standard python types are already packableassertisinstance(p,dict)deepcopy_x=unpack(p)assertrepr(x)==repr(deepcopy_x)fromhumpackimportjson_pack,json_unpack# Convert to/from json stringj=json_pack(x)assertisinstance(j,str)deepcopy_x=json_unpack(j)assertrepr(x)==repr(deepcopy_x)fromhumpackimportsave_pack,load_pack# Save/load packed object to disk as json fileimportos,tempfilefd,path=tempfile.mkstemp()try:withopen(path,'w')astmp:save_pack(x,tmp)withopen(path,'r')astmp:deepcopy_x=load_pack(tmp)finally:os.remove(path)assertrepr(x)==repr(deepcopy_x)For examples of how to any types can registered to bePackableor objects can be wrapped inPackablewrappers, see thehumpack/common.pyandhumpack/wrappers.pyscripts.TransactionsFor examples of howTransactionableobjects behave see the “Containers” section above.To enable transactions for a class, it must be a subclass ofTransactionableand implement the four required functions:begin,in_transaction,commit, andabort. Assuming these functions are implemented as specified (see documentation), you can manipulate instances of these classes in a transaction and then roll back all the changes by aborting the transaction.One important thing to note with subclassingTransactionable: any members of instances ofTransactionablesubclasses should be checked for if they are alsoTransactionable, and if so, they the call should be delegated. In the example below,Accounthas to take into account that its attributeusercould beTransactionable.fromhumpackimportTransactionableclassAccount(Transactionable):def__init__(self,user,balance=0.):super().__init__()self._in_transaction=Falseself._shadow_user=Noneself.user=userself.balance=balancedefchange(self,delta):ifself.balance+delta<0.:raiseValueErrorself.balance+=deltadefbegin(self):# FIRST: begin the transaction in selfself._shadow_user=self.user.copy(),self.balance# Assuming `user` can be shallow copied with `copy()`self._in_transaction=True# THEN: begin transactions in any members that are Transactionableifisinstance(self.user,Transactionable):self.user.begin()# To be extra safe, you could also check `self.balance`, but we'll assume it's always a primitive (eg. float)defin_transaction(self):returnself._in_transactiondefcommit(self):# FIRST: commit the transaction in selfself._in_transaction=Falseself._shadow_user=None# THEN: commit transactions in any members that are Transactionableifisinstance(self.user,Transactionable):self.user.commit()defabort(self):# FIRST: abort the transaction in selfifself.in_transaction():# Note that this call only has an effect if self was in a transaction.self.user,self.balance=self._shadow_userself._in_transaction=Falseself._shadow_user=None# THEN: abort transactions in any members that are Transactionableifisinstance(self.user,Transactionable):self.user.abort()Optionally, for a more pythonic implementation, you can usetry/exceptstatements instead of type checking withisinstance.SecurityThere are a few high-level cryptography routines. Nothing special, just meant to make integration in larger projects simple and smooth.TODOFeatures that could be added/improved:Enable simple conversion from containers to standard python (eg. decontainerify)Add security functions to encrypt/decrypt files and directories (collecting/zipping contents in a tar)Add Transactionable/Packable replacements for more standard python types (especially tuples)Possibly add 1-2 tutorialsWrite more comprehensive unit tests and report test coverageAllow packing bound methods of Packable typesAdd option to save class attributesContributions and suggestions are always welcome.