Package: slots-factory
slots_factoryFactory functions and decorators for creating slot objectsSlots are a python construct that allows users to create an object that doesn't contain__dict__or__weakref__attributes. The benefit to a slots object is that it has faster attribute access and it saves on memory use, which make slots objects ideal for when you have lots of instances of a single python object.I've never been a huge fan of the syntax though, as it requires repetitive code for definition as well as instantiation.yuck.classSlotsObject:__slots__=('x','y','z')def__init__(self,x,y,z):self.x=xself.y=yself.z=zdef__repr__(self):contents=", ".join([f"{key}={getattr(self,key)}"forkeyinself.__slots__])returnf"SlotsObject({contents})"For funsies, I wanted to see if I could create a different way to instantiate these objects, with less jargon. Something likecollections.namedtuple, but again without redundant definitions and with the benefits of__slots__. This repo is the results of such endeavor.TL;DR- the@dataslotsdecorator ends up being the most useful implementation, free to skip to it if you want to see the fireworks.slots_factory()The first factory function made available isslots_factory. Simply import the function, and all **kwargs are assigned as attributes to an instance of a slots object. Type definitions are handled internally by the function, so successive calls toslots_factorywith the same_nameand**kwargskeys will return new instances of the same type.For example:In[1]:fromslots_factoryimportslots_factoryIn[2]:this=slots_factory(x=1,y=2,z=3)In[3]:thisOut[3]:SlotsObject(x=1,y=2,z=3)In[4]:that=slots_factory(x=4,y=5,z=6)In[5]:thatOut[5]:SlotsObject(x=4,y=5,z=6)In[6]:fizzbuzz=slots_factory(_name="fizzbuzz",fizz="fizz",buzz="buzz")In[7]:fizzbuzzOut[7]:fizzbuzz(fizz=fizz,buzz=buzz)In[8]:slots_factory.__dict__Out[8]:{13844952821349480973:slots_factory.slots_factory.SlotsObject,7572372383060875:slots_factory.slots_factory.fizzbuzz}As we can see, we created three instances,this,that, andfizzbuzz.thisandthatare instances of the same type, since the function args were the same.fizzbuzzis a different type however, since its function arguments were different.In[9]:type(this)==type(that)Out[9]:TrueIn[10]:type(this)==type(fizzbuzz)Out[10]:FalseAnother benefit to thisSlotsObjectis that, as the underlying type is a slots object, the attributes are dynamic, unlike thenamedtuple.In[11]:this.x=4In[12]:thisOut[12]:SlotsObject(x=4,y=2,z=3)The type identification and attribute setting is all done in C, in attempt to make instantiation as fast as possible. Instantiation of aSlotObjectis still about 80% slower than the instantiation of anamedtuple(mainly because it handles type definitions internally). Attribute access is on par however, and faster than a normal object as expected.In[13]:fromcollectionsimportnamedtupleIn[14]:This=namedtuple('This',['x','y','z'])In[15]:%timeitthis=This(x=1,y=2,z=3)315ns±1.58nsperloop(mean±std.dev.of7runs,1000000loopseach)In[16]:%timeitthat=slots_factory('that',x=1,y=2,z=3)597ns±1.38nsperloop(mean±std.dev.of7runs,1000000loopseach)In[17]:%timeitthis.c24.6ns±0.132nsperloop(mean±std.dev.of7runs,10000000loopseach)In[18]:%timeitthat.c25.8ns±0.13nsperloop(mean±std.dev.of7runs,10000000loopseach)%timefast_slots()There's a second factory function,fast_slots, which is, obviously, faster. Instead of using the builtin hashing algorithm to generate an ID, it simply uses the object name and assumes that all objects named the same, are the same. Since it skips the hashing step, it builds slot instances much faster.In[4]:fromslots_factoryimportfast_slotsIn[5]:%timeitthat=fast_slots('that',x=1,y=2,z=3)442ns±3.71nsperloop(mean±std.dev.of7runs,1000000loopseach)Instead of relying on an internal ID mechanism,fast_slotsleverages python'stry/exceptfunctionality. The internal_slots_factory_setattrsmethod throws an exception when the object attributes are thought to be different, so when this happensfast_slotsdeletes its old internalized type definition and then builds a new one. As such, if you expect to be redefining the same type over and over again, it's best to useslots_factoryfor better overall performance. If however you're certain to be creating identical instances of the same type (with differing attribute variables of course, that is indeed allowed byfast_slots), then you'll be better of usingfast_slotsto do this.fromslots_factoryimportslots_factory,fast_slots# use `slots_factory` like so:this=slots_factory(x=1)that=slots_factory(y=2)# use `fast_slots` like so:category=fast_slots('category',id=1,name='category 1')category=fast_slots('category',id=2,name='category 2')type_factory()Finally, if we're really craving the speeds, the most efficient way to use this module is to individually define your types and then manually spin up instances of these objects. This can be done by importing thetype_factoryandslots_from_typefunctions.fromslots_factoryimporttype_factory,slots_from_typetype_=type_factory(['x','y','z'],_name="SlotsObject")instance=slots_from_type(type_,x=1,y=2,z=3,)In[6]:%timeitinstance=slots_from_type(type_,x=1,y=2,z=3)323ns±10.4nsperloop(mean±std.dev.of7runs,1000000loopseach)@dataslotsThere's a new decorator provided in theslots_factorymodule which attempts to improve upon Python'sdataclasses.dataclass. Class definitions can be decorated with the@dataslotsdecorator to generate instances of analogous types with__slots__. I sayanalogousbecause at runtime the decorator instantiates a new type instead of modifying the user's defined type. The user's type is simply used as a sort of blueprint for generating the desired type with__slots__.In[1]:fromslots_factoryimportdataslots@dataslotsclassThis:x:inty:intz:intIn[2]:%timeitThis(x=1,y=2,z=3)397ns±1.51nsperloop(mean±std.dev.of7runs,1000000loopseach)@dataslotsclassThis:x:int=1y:int=2z:int=3In[2]:%timeitThis()261ns±1.2nsperloop(mean±std.dev.of7runs,1000000loopseach)The@dataslotsdecorator allows for users to set default values using standard python syntax, and defaults can be overwritten simply by defining a new value at instantiation. There is no ordering restrictions on default definitions. It's also worth noting that, normally, when writing__slots__classes, we can't define class attributes which conflict with the__slots__structure that Python creates. However due to the internal mechanics of@dataslots, we can set__slots__object defaults absent any annotations.@dataslotsclassFizzBuzz:fizz='fizz'buzz:strfizzbuzz:str='spam'In[5]:this=FizzBuzz(buzz='buzz',fizzbuzz='fizzbuzz')Out[5]:FizzBuzz(fizz=fizz,buzz=buzz,fizzbuzz=fizzbuzz)optional arguments for@dataslots@dataslotsprovides afrozenkeyword argument as a boolean. Passingfrozen=Trueto the@dataslotsdecorator forces instances to be immutable.@dataslots(frozen=True)classFizzBuzz:fizz:str='fizz'buzz:str='buzz'In[7]:fb=FizzBuzz()In[8]:fbOut[8]:FizzBuzz(fizz=fizz,buzz=buzz)In[9]:fb.fizz='buzz'-----------------------------------------------------------------------AttributeErrorTraceback(mostrecentcalllast)<ipython-input-9-63a20d67080e>in<module>---->1fb.fizz='buzz'~/programming/python/slots_factory/src/slots_factory/slots_factory.pyin_frozen(self,*_,**__)127def_frozen(self,*_,**__):128raiseAttributeError("instance is immutable.")-->129methods.update({130"__setattr__":_frozen,131"__delattr__":_frozenAttributeError:instanceisimmutable.@dataslotsalso provides anorderkeyword argument as either a boolean or an iterable. If passed as a boolean, items are iterated over in whatever manner Python decides to sort the attribute names. Order can be made explicit by passing an iterable of attribute names for yielding.@dataslots(order=True)classThis:x:inty:intz:intIn[1]:this=This(x=1,y=2,z=3)In[2]:[xforxinthis]Out[2]:[('x',1),('y',2),('z',3)]@dataslots(order=['x','z','y'])classThis:x:inty:intz:intIn[3]:this=This(x=1,y=2,z=3)In[4]:[xforxinthis]Out[4]:[('x',1),('z',3),('y',2)]Ordering implies hierarchy, and hierarchy provides a means for rich comparisons. Instances that are ordered can be compared using Python's builtin comparison operators. Comparison is done by applying the respected operator's method as defined on theselfof the pair of objects, in order, across attributes. Comparison is resolved at first instance of inequality.@dataslots(order=True)classThis:x:int=1y:int=2z:int=3@dataslots(order=True)classThat:x:int=4y:int=5z:int=6In[1]:this,that=This(),That()In[2]:this<thatOut[2]:TrueIn[3]:this=This(x=6)In[4]:this<thatOut[4]:FalseThough dataslots are not dictionaries, they have many of the properties you would expect from a dictionary object. As such, conversion to and from dictionaries is built in. And as dictionaries are ordered in Python 3.6+, we make sure to preserve order between conversions.@dataslots(order=["x","z","y"])classThis:x:inty:intz:intIn[1]:this=This(x=1,y=2,z=3)In[2]:that=dict(this)In[3]:thatOut[3]:{'x':1,'z':3,'y':2}In[4]:dataslots.from_dict(that)Out[4]:SlotsObject(x=1,z=3,y=2)Dataslots also supports user-defined methods and properties. They can be defined as normal on the class, and @dataslots will be sure to carry these objects over to the__slots__object.@dataslotsclassFizzBuzz:fizz='fizz'buzz:str='buzz'deffizzbuzz(self):returnself.fizz+self.buzzIn[1]:fizzbuzz=FizzBuzz()In[2]:fizzbuzz.fizzbuzz()Out[2]:"fizzbuzz"@dataslotsclassFizzBuzz:fizz='fizz'buzz:str='buzz'@propertydeffizzbuzz(self):returnself.fizz+self.buzz@fizzbuzz.setterdeffizzbuzz(self,item):self.fizz,self.buzz=itemIn[1]:fizzbuzz=FizzBuzz()In[2]:fizzbuzz.fizzbuzzOut[2]:'fizzbuzz'In[3]:fizzbuzz.fizzbuzz=("This","That")In[4]:fizzbuzz.fizzbuzzOut[4]:'ThisThat'Mutable default types in@dataslotsvialambdaGiven the nature of mutable types in Python, it's always been considered gauche to define default values as mutable types within object definitions. In order to allow for mutable defaults whose references aren't shared across instances,@dataslotsdefault values can be assigned as eithertypetype or alambdaexpression with no arguments. These defaults are then called on instantiation, and instances assigned the result of the callable.@dataslotsclassRecordsCollection:list_of_records=lambda:[{"record_id":0,"name":"Terminal Record"}]record_count:int=1defadd_record(self,_id,name):self.record_count+=1self.list_of_records.append({"record_id":_id,"name":name})@dataslotsclassRecordIds:ids=setdefingest_record(self,record):foriteminrecord.list_of_records:self.ids.add(item["record_id"])In[1]:n1=RecordsCollection()In[2]:%timeitRecordsCollection()Out[2]:496ns±1.95nsperloop(mean±std.dev.of7runs,1000000loopseach)In[3]:n2=RecordsCollection()In[4]:n1.add_record(5,"New Record")In[5]:n1.list_of_recordsOut[5]:[{'record_id':0,'name':'Terminal Record'},{'record_id':5,'name':'New Record'}]In[6]:n2.list_of_recordsOut[6]:[{'record_id':0,'name':'Terminal Record'}]In[7]:rec_ids=RecordIds()In[8]:rec_ids.ingest_record(n1)In[9]:rec_ids.idsOut[9]:{0,5}Inheritance and Composition in@dataslots@dataslotsobjects can inherit artifacts from other dataslots. However, given that@dataslotsis regenerating new datatypes on the fly, it currently doesn't have any concept of method resolution order, nor does it understand the concept ofsuper(). A derived class simply updates its default values with preference given to the first base class in queue. Given this, class composition is generally regarded as a better implementation strategy, given@dataslots's compatibility with default type instantiations."""inheritance"""@dataslotsclassA:a:list=lambda:[1,2,3]@dataslotsclassB:a=list@dataslotsclassDerivedOne(A,B):defget_list(self):returnself.a@dataslotsclassDerivedTwo(B,A):defget_list(self):returnself.aIn[1]:instance_one=DerivedOne()In[2]:instance_two=DerivedTwo()In[3]:instance_one.get_list()Out[3]:[1,2,3]In[4]:instance_two.get_list()Out[4]:[]"""composition"""@dataslotsclassSubcomponentOne:x=1@dataslotsclassSubcomponentTwo:x=lambda:[1,2,3]@dataslotsclassRootClass:s1=SubcomponentOnes2=SubcomponentTwoIn[1]:instance=RootClass()In[2]:repr(instance)Out[2]:'RootClass(s1=SubcomponentOne(x=1), s2=SubcomponentTwo(x=[1, 2, 3]))'In[3]:instance.s2.xOut[3]:[1,2,3]Dependent defaults in@dataslotsAttributes oftentimes depend on the state of other attributes within an object. This can be tricky when it comes to default values inslots, as if you set values at type definition, those attributes become read-only. One solution to this is to define the attribute as a@property, so that thepropertyhas access to the instance when referenced.@dataslotsprovides a leaner alternative, once again using thelambdafunction as a means for default assignments. lambda functions assigned to attributes can take a single argument,self. At instantiation the lambda is called and the resultant is assigned to the instance attribute.importpymongoimportredisfromslots_factoryimportdataslots@dataslotsclassRedis:queue=redis.Redis(host="redis-queue")@dataslotsclassMongo:client=pymongo.MongoClient("mongodb://mongo:27017")database=lambdaself:self.client.get_database("primary")@dataslotsclassConnections:mongo=Mongoredis=RedisIn[1]:conn=Connections()In[2]:conn.mongo.databaseOut[2]:Database(MongoClient(host=['mongo:27017'],document_class=dict,tz_aware=False,connect=True),'primary')Appendix: Some pure-Python implementationsThis module uses custom C extensions for trying to speed up attribute write times. However the inclusion of this requiresslots_factoryto be installed and the extensions compiled. If that seems undesirable, here are some pure-Python implementations that can simply be copied into a codebase.defslots_factory(_name="SlotsObject",**kwargs):stores=slots_factory.__dict___keys=frozenset(kwargs)if_name=="SlotsObject":_id=hash(_keys)_type=stores.get(_id)else:_id=hash(_name)^hash(_keys)_type=stores.get(_id)ifnot_type:def__repr__(self):contents=", ".join([f"{key}={getattr(self,key)}"forkeyinself.__slots__])returnf"{self.__class__.__name__}({contents})"_type=type(_name,(),{"__slots__":_keys,"__repr__":__repr__})stores[_id]=_typeinstance=_type()forkey,valueinkwargs.items():setattr(instance,key,value)returninstancedeffast_slots(_name="SlotsObject",**kwargs):_type=fast_slots.__dict__.get(_name)ifnot_type:def__repr__(self):contents=", ".join([f"{key}={getattr(self,key)}"forkeyinself.__slots__])returnf"{self.__class__.__name__}({contents})"_type=type(_name,(),{"__slots__":kwargs.keys(),"__repr__":__repr__})fast_slots.__dict__[_name]=_typeinstance=_type()try:forkey,valueinkwargs.items():setattr(instance,key,value)returninstanceexceptAttributeError:delfast_slots.__dict__[_name]returnfast_slots(_name,**kwargs)