Package: arrex
ArrexdocumentationrepositoryArrex is a module that allows to create typed arrays much likenumpy.ndarrayandarray.array, but resizeable and using any kind of element, not only numbers. Its dtype system is extremely flexible and makes it ideal to work and share structured data with compiled code.The elements can be many different things, there is just 2 requirements:they must be of a fixed binary sizethey must be byte copiable, therefore without any reference or pointer to something elseinterestsmuch smaller memory footprint (an arrex array is at most 30x smaller than pure python data storage)content can be directly shared with compiled code which improves computation performancesslice & view without a copycompatible with standard python librariesbasic usage:>>>fromarreximport*>>>a=typedlist([myclass(...),myclass(...),],dtype=myclass)>>>a[0]myclass(...)in that example,myclasscan be a primitive numpy type, likenp.float64>>>importarrex.numpy# this is enabling numpy dtypes for arrex>>>typedlist(dtype=np.float64)it can be a more complex type, from modulepyglmfor instance>>>importarrex.glm# this is enabling glm dtypes for arrex>>>typedlist(dtype=glm.vec4)typedlistis a dynamically sized, borrowing array, which mean the internal buffer of data is reallocated on append, but can be used to view and extract from any buffer without a copy.Use it as a list:>>>a=typedlist(dtype=vec3)# build from an iterable>>>a=typedlist([],dtype=vec3)# append some data>>>a.append(vec3(1,2,3))# extend with an iterable>>>a.extend(vec3(i)foriinrange(5))>>>len(a)# the current number of elements6>>>a.owner# the current data bufferb'.........'>>>a[0]vec3(1,2,3)Use it as a slice:>>>myslice=a[:5]# no data is copiedtypedlist(....)Use it as a view on top of a random buffer>>>a=np.ones((6,3),dtype='f4')>>>myslice=typedlist(a,dtype=vec3)buffer protocolIt does support the buffer protocol, so it can be converted into a great variety of well known arrays, even without any copy>>>np.array(typedlist([....]))Which dtype is allowedanswer is:whatever you want, but here is some examples:# an extension type, previously declaredtypedlist(dtype=glm.vec3)# a Structtypedlist(dtype='ffxI')# a ctypetypedlist(dtype=ctypes.c_int*5)# a pure python classclasstest_class:__packlayout__='ff'_struct=struct.Struct(__packlayout__)def__init__(self,x,y):self.x=xself.y=ydef__bytes__(self):returnself._struct.pack(self.x,self.y)@classmethoddeffrombytes(cls,b):returncls(*cls._struct.unpack(b))typedlist(dtype=test_class)# and so much more !performancesTime performances comparison betweenlist,numpy.ndarray,  andarrex.typedlist(seebenchmark)execution time (s) for 10k elements (dvec3)set item
  list:         7.9847e-04 s
  numpy:        1.2727e-02 s
  arrex:        1.0481e-03 s  (10x faster than numpy)

get item
  creation:     1.0655e-03 s
  list:         5.1503e-04 s
  numpy:        1.8619e-03 s
  arrex:        8.0111e-04 s   (2x faster than numpy)RoadmapThere is additionnal features planned, but no precise schedul yet:typedarraya n-dim array view much like numpy arrays but using dtypes as intypedlist.
Its purpose is mostly to access its items with n-dim indices and slices.a ufunc systemto collect and put defaults to any kind of array scale operations, like__add__,__mul__,__matmul__, ... The goal would be to have a standard way to apply any function to every element of one or more array, that defaults to the python implementation, but can be overloaded with a compiled implementationmaybeeven extend to the complete API ofnumcy