Package: cythonuniquedim
Finds unique dimensions in multi-dimensional arrays, compresses and decompresses arrays ...pip install cythonuniquedimTested against Windows / Python 3.11 / AnacondaCython (and a C/C++ compiler) must be installedThis module provides functions for working with multi-dimensional arrays and performing operations
such as grouping rows with the same values, compressing and decompressing arrays, and
finding unique dimensions in multi-dimensional arrays.FUNCTIONScompress_rows_as_list(a,bitshift_dtype=None)Compressrowsofa2Dnumpyarrayintoalistofintegersusingbitshifting.Args:a(numpy.ndarray):Theinput2Darray.bitshift_dtype(numpy.dtype,optional):Thedatatypeforbitshifting.DefaultstoNone(willfindthefastest).Returns:Tuple[List[mpz],int,Tuple[int,int]]:Atuplecontainingthecompressedlistofintegers,thebitshiftvalueused,andtheshapeoftheinputarray.group_same_rows_as_dict(a,bitshift_dtype=None,fullkeys=True)Grouprowswiththesamevaluesina2Dnumpyarrayandreturntheresultasadictionary.Args:a(numpy.ndarray):Theinput2Darray.bitshift_dtype(numpy.dtype,optional):Thedatatypeforbitshifting.DefaultstoNone(willfindthefastest).fullkeys(bool,optional):IfTrue,usesfullkeys(mpz)intheresultdictionary.IfFalse,usesintegerkeys.DefaultstoTrue.Returns:Tuple[Dict[Union[int,mpz],List[int]],int,Tuple[int,int]]:Atuplecontainingtheresultdictionary,thebitshiftvalueused,andtheshapeoftheinputarray.uncompress_list(mpz_list,usedbitshift,elements_per_row,numpy_dtype=None)Decompressalistofintegersintoa2Dnumpyarray.Args:mpz_list(List[mpz]):Thecompressedlistofintegers.usedbitshift(int):Thebitshiftvalueusedduringcompression.elements_per_row(int):Thenumberofelementsperrowintheoriginalarray.numpy_dtype(numpy.dtype,optional):Thedesireddatatypeoftheoutputarray.DefaultstoNone.Returns:numpy.ndarray:Thedecompressed2Dnumpyarray.unique_dimensions(a,last_dim,unordered=True,iterray=())Finduniquedimensionsinamulti-dimensionalnumpyarray.Args:a(numpy.ndarray):Theinputmulti-dimensionalarray.last_dim(int):Thelastdimensiontoconsiderwhenfindinguniquedimensions.unordered(bool,optional):IfTrue,usesmultiprocessingtocreatetheindex.DefaultstoTrue.iterray(Any,optional):Customiteratorarray.Defaultstoanemptytuple.Returns:numpy.ndarray:Thearraycontaininguniquedimensionsbasedonthespecifiedcriteria.importnumpyasnpfromcythonuniquedimimportgroup_same_rows_as_dict,compress_rows_as_list,uncompress_list,unique_dimensionsshaha=(3,5)dtype=np.uint8b=np.arange(np.product(shaha),dtype=dtype)a=b.reshape(shaha)a=np.tile(a,3).Tacp=a.copy()resultdict0,usedbitshift0,arrayviewshape0=group_same_rows_as_dict(a,bitshift_dtype=None,fullkeys=True)resultdict1,usedbitshift1,arrayviewshape1=group_same_rows_as_dict(a,bitshift_dtype=None,fullkeys=False)print(f'{resultdict0,usedbitshift0,arrayviewshape0=}')print(f'{resultdict1,usedbitshift1,arrayviewshape1=}')mpz_list100,mpz_list_usedbitshift100,arrayviewshape100=compress_rows_as_list(a,bitshift_dtype=None,)mpz_list111,mpz_list_usedbitshift111,arrayviewshape111=compress_rows_as_list(a,bitshift_dtype=np.uint8,)print(f'{mpz_list100,mpz_list_usedbitshift100,arrayviewshape100=}')print(f'{mpz_list111,mpz_list_usedbitshift111,arrayviewshape111=}')unclist=(uncompress_list(mpz_list=mpz_list111,usedbitshift=mpz_list_usedbitshift111,elements_per_row=arrayviewshape111[1],numpy_dtype=np.uint32))print(f'{unclist=}')# resultdict0,usedbitshift0,arrayviewshape0=({mpz(656640): [0, 5, 10], mpz(722433): [1, 6, 11], mpz(788226): [2, 7, 12], mpz(854019): [3, 8, 13], mpz(919812): [4, 9, 14]}, 256, (15, 3))# resultdict1,usedbitshift1,arrayviewshape1=({0: [0, 5, 10], 1: [1, 6, 11], 2: [2, 7, 12], 3: [3, 8, 13], 4: [4, 9, 14]}, 256, (15, 3))# mpz_list100,mpz_list_usedbitshift100,arrayviewshape100=([mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812), mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812), mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812)], 256, (15, 3))# mpz_list111,mpz_list_usedbitshift111,arrayviewshape111=([mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812), mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812), mpz(656640), mpz(722433), mpz(788226), mpz(854019), mpz(919812)], 256, (15, 3))# unclist=array([[ 0,  5, 10],#        [ 1,  6, 11],#        [ 2,  7, 12],#        [ 3,  8, 13],#        [ 4,  9, 14],#        [ 0,  5, 10],#        [ 1,  6, 11],#        [ 2,  7, 12],#        [ 3,  8, 13],#        [ 4,  9, 14],#        [ 0,  5, 10],#        [ 1,  6, 11],#        [ 2,  7, 12],#        [ 3,  8, 13],#        [ 4,  9, 14]], dtype=uint32)shaha=(4,3,4,3,3,2,3)dtype=np.float64b=np.arange(np.product(shaha),dtype=dtype)a=b.reshape(shaha)a=np.tile(a,3).T.copy()unidims=unique_dimensions(a,last_dim=3,unordered=True)print(f'{unidims=}')print(f'{unidims.shape=}')unidims1=unique_dimensions(a,last_dim=4,unordered=True)print(f'{unidims1=}')print(f'{unidims1.shape=}')unidims2=unique_dimensions(a,last_dim=5,unordered=True)print(f'{unidims2=}')print(f'{unidims2.shape=}')unidims3=unique_dimensions(a,last_dim=6,unordered=True)print(f'{unidims3=}')print(f'{unidims3.shape=}')# unidims=array([[[[[[0.000e+00, 6.480e+02, 1.296e+03, 1.944e+03],#            [2.160e+02, 8.640e+02, 1.512e+03, 2.160e+03],#            [4.320e+02, 1.080e+03, 1.728e+03, 2.376e+03]],#           [[5.400e+01, 7.020e+02, 1.350e+03, 1.998e+03],#            [2.700e+02, 9.180e+02, 1.566e+03, 2.214e+03],#            [4.860e+02, 1.134e+03, 1.782e+03, 2.430e+03]],#           [[1.080e+02, 7.560e+02, 1.404e+03, 2.052e+03],#            [3.240e+02, 9.720e+02, 1.620e+03, 2.268e+03],#            [5.400e+02, 1.188e+03, 1.836e+03, 2.484e+03]],#           [[1.620e+02, 8.100e+02, 1.458e+03, 2.106e+03],#            [3.780e+02, 1.026e+03, 1.674e+03, 2.322e+03],#            [5.940e+02, 1.242e+03, 1.890e+03, 2.538e+03]]],#          [[[1.800e+01, 6.660e+02, 1.314e+03, 1.962e+03],#            [2.340e+02, 8.820e+02, 1.530e+03, 2.178e+03],#            [4.500e+02, 1.098e+03, 1.746e+03, 2.394e+03]],#           [[7.200e+01, 7.200e+02, 1.368e+03, 2.016e+03],#            [2.880e+02, 9.360e+02, 1.584e+03, 2.232e+03],#            [5.040e+02, 1.152e+03, 1.800e+03, 2.448e+03]],#           [[1.260e+02, 7.740e+02, 1.422e+03, 2.070e+03],#            [3.420e+02, 9.900e+02, 1.638e+03, 2.286e+03],#            [5.580e+02, 1.206e+03, 1.854e+03, 2.502e+03]],#           [[1.800e+02, 8.280e+02, 1.476e+03, 2.124e+03],#            [3.960e+02, 1.044e+03, 1.692e+03, 2.340e+03],#            [6.120e+02, 1.260e+03, 1.908e+03, 2.556e+03]]],# ......