Package: treevalue
TreeValueis a generalized tree-based data structure mainly developed byOpenDILab Contributors.Almost all the operations can be supported in the form of trees in a convenient way to simplify the structure processing when the calculation is tree-based.OutlineOverviewGetting StartedPrerequisiteInstallationQuick UsageTutorialsExternalSpeed PerformanceChange LogFeedback and ContributeCitationLicenseOverviewWhen we build a complex nested structure, we need to model it as a tree structure, and the native list and dict in Python are often used to solve this problem. However, it takes a lot of codes and some complex and non-intuitive calculation logic, which is not easy to modify and extend related code and data, and parallelization is impossible.Therefore, we need a kind of more proper data container, namedTreeValue. It is designed for solving the following problems:Ease of Use: When the existing operations are applied to tree structures such as dict, they will become completely unrecognizable, with really low readability and maintainability.Diversity of Data: In the tree structure operation, various abnormal conditions (structure mismatch, missing key-value, type mismatch, etc.) occur from time to time, and the code will be more complicated if it needs to be handled properly.Scalability and Parallelization: When any multivariate operation is performed, the calculation logic needs to be redesigned under the native Python code implementation, and the processing will be more complicated and confusing, and the code quality is difficult to control.Getting StartedPrerequisitetreevaluehas been fully tested in the Linux, macOS and Windows environments and with multiple Python versions, and it works properly on all these platforms.However,treevaluecurrently does not support PyPy, so just pay attention to this when using it.InstallationYou can simply install it withpipcommand line from the official PyPI site.pipinstalltreevalueOr just from the source code on githubpipinstallgit+https://github.com/opendilab/treevalue.git@mainFor more information about installation, you can refer to theinstallation guide.After this, you can check if the installation is processed properly with the following codefromtreevalueimport__version__print('TreeValue version is',__version__)Quick UsageYou can easily create a tree value object based onFastTreeValue.fromtreevalueimportFastTreeValueif__name__=='__main__':t=FastTreeValue({'a':1,'b':2.3,'x':{'c':'str','d':[1,2,None],'e':b'bytes',}})print(t)The result should be<FastTreeValue 0x7f6c7df00160 keys: ['a', 'b', 'x']>
├── 'a' --> 1
├── 'b' --> 2.3
└── 'x' --> <FastTreeValue 0x7f6c81150860 keys: ['c', 'd', 'e']>
    ├── 'c' --> 'str'
    ├── 'd' --> [1, 2, None]
    └── 'e' --> b'bytes'Andtis structure should be like thisNot only a visible tree structure, but abundant operation supports is provided.
You can just put objects (such astorch.Tensor, or any other types) here and just
call their methods, like thisimporttorchfromtreevalueimportFastTreeValuet=FastTreeValue({'a':torch.rand(2,5),'x':{'c':torch.rand(3,4),}})print(t)# <FastTreeValue 0x7f8c069346a0># ├── a --> tensor([[0.3606, 0.2583, 0.3843, 0.8611, 0.5130],# │                 [0.0717, 0.1370, 0.1724, 0.7627, 0.7871]])# └── x --> <FastTreeValue 0x7f8ba6130f40>#     └── c --> tensor([[0.2320, 0.6050, 0.6844, 0.3609],#                       [0.0084, 0.0816, 0.8740, 0.3773],#                       [0.6523, 0.4417, 0.6413, 0.8965]])print(t.shape)# property access# <FastTreeValue 0x7f8c06934ac0># ├── a --> torch.Size([2, 5])# └── x --> <FastTreeValue 0x7f8c069346d0>#     └── c --> torch.Size([3, 4])print(t.sin())# method call# <FastTreeValue 0x7f8c06934b80># ├── a --> tensor([[0.3528, 0.2555, 0.3749, 0.7586, 0.4908],# │                 [0.0716, 0.1365, 0.1715, 0.6909, 0.7083]])# └── x --> <FastTreeValue 0x7f8c06934b20>#     └── c --> tensor([[0.2300, 0.5688, 0.6322, 0.3531],#                       [0.0084, 0.0816, 0.7669, 0.3684],#                       [0.6070, 0.4275, 0.5982, 0.7812]])print(t.reshape((2,-1)))# method with arguments# <FastTreeValue 0x7f8c06934b80># ├── a --> tensor([[0.3606, 0.2583, 0.3843, 0.8611, 0.5130],# │                 [0.0717, 0.1370, 0.1724, 0.7627, 0.7871]])# └── x --> <FastTreeValue 0x7f8c06934b20>#     └── c --> tensor([[0.2320, 0.6050, 0.6844, 0.3609, 0.0084, 0.0816],#                       [0.8740, 0.3773, 0.6523, 0.4417, 0.6413, 0.8965]])print(t[:,1:-1])# index operator# <FastTreeValue 0x7f8ba5c8eca0># ├── a --> tensor([[0.2583, 0.3843, 0.8611],# │                 [0.1370, 0.1724, 0.7627]])# └── x --> <FastTreeValue 0x7f8ba5c8ebe0>#     └── c --> tensor([[0.6050, 0.6844],#                       [0.0816, 0.8740],#                       [0.4417, 0.6413]])print(1+(t-0.8)**2*1.5)# math operators# <FastTreeValue 0x7fdfa5836b80># ├── a --> tensor([[1.6076, 1.0048, 1.0541, 1.3524, 1.0015],# │                 [1.0413, 1.8352, 1.2328, 1.7904, 1.0088]])# └── x --> <FastTreeValue 0x7fdfa5836880>#     └── c --> tensor([[1.1550, 1.0963, 1.3555, 1.2030],#                       [1.0575, 1.4045, 1.0041, 1.0638],#                       [1.0782, 1.0037, 1.5075, 1.0658]])TutorialsFor more examples, explanations and further usages, take a look at:Quick StartCreate a Simplest TreeCreate a Slightly Complex TreeBasic UsageCreate a TreeEdit a TreeDo Index or Slice Calculation on The TreeDo Math Calculation on The TreeMake Function Tree-SupportedAdvanced UsageFunction ModesInheriting on TreesProcess Missing ValuesFunctional UtilitiesStructural UtilitiesTree UtilitiesFlatten UtilitiesIO UtilitiesObject Oriented UsageCostumize My TreeValue ClassVisualization of TreeValueExternalWe provide an official treevalue-based-wrapper for numpy and torch calledDI-treetensorsince thetreevalueis often used with libraries likenumpyandtorch. It will actually be helpful while working with AI fields.Speed PerformanceHere is the speed performance of all the operations inFastTreeValue; the following table is the performance comparison result withdm-tree.
(In DM-Tree, theunflattenoperation is different from that in TreeValue, see:Comparison Between TreeValue and DM-Treefor more details.)flattenflatten(with path)mappingmapping(with path)treevalue---511 ns ± 6.92 ns3.16 µs ± 42.8 ns1.58 µs ± 30 nsflattenflatten_with_pathmap_structuremap_structure_with_pathdm-tree830 ns ± 8.53 ns11.9 µs ± 358 ns13.3 µs ± 87.2 ns62.9 µs ± 2.26 µsThe following 2 tables are the performance comparison result withjax pytree.mappingmapping(with path)flattenunflattenflatten_valuesflatten_keystreevalue2.21 µs ± 32.2 ns2.16 µs ± 123 ns515 ns ± 7.53 ns601 ns ± 5.99 ns301 ns ± 12.9 ns451 ns ± 17.3 nstree_map(Not Implemented)tree_flattentree_unflattentree_leavestree_structurejax pytree4.67 µs ± 184 ns---1.29 µs ± 27.2 ns742 ns ± 5.82 ns1.29 µs ± 22 ns1.27 µs ± 16.5 nsflatten + allflatten + reduceflatten + reduce(with init)rise(given structure)rise(automatic structure)treevalue425 ns ± 9.33 ns702 ns ± 5.93 ns793 ns ± 13.4 ns9.14 µs ± 129 ns11.5 µs ± 182 nstree_alltree_reducetree_reduce(with init)tree_transpose(Not Implemented)jax pytree1.47 µs ± 37 ns1.88 µs ± 27.2 ns1.91 µs ± 47.4 ns10 µs ± 117 ns---This is the comparison between dm-tree, jax-libtree and us, withflattenandmappingoperations (lower value means less time cost and runs faster)The following table is the performance comparison result withtianshou Batch.getsetinitdeepcopystackcatsplittreevalue51.6 ns ± 0.609 ns64.4 ns ± 0.564 ns750 ns ± 14.2 ns88.9 µs ± 887 ns50.2 µs ± 771 ns40.3 µs ± 1.08 µs62 µs ± 1.2 µstianshou Batch43.2 ns ± 0.698 ns396 ns ± 8.99 ns11.1 µs ± 277 ns89 µs ± 1.42 µs119 µs ± 1.1 µs194 µs ± 1.81 µs653 µs ± 17.8 µsAnd this is the comparison between Tianshou Batch and us, withcat,stackandsplitoperations (lower value means less time cost and runs faster)Test benchmark code can be found here:Comparison with dm-treeComparison with jax-libtreeComparison with tianshou BatchChange LogVersion History[click to expand]2022-05-03
1.3.1: Change definition of getitem, setitem and delitem; add pop method for TreeValue class.2022-03-15
1.3.0: Add getitem, setitem and delitem for adding, editing and removing items in TreeValue class.2022-02-22
1.2.2: Optimize union function; add walk utility method.2022-01-26
1.2.1: Update tree printing; add keys, values, items on TreeValue; add comparision to facebook nest library.2022-01-04
1.2.0: Add flatten_values and flatten_keys; fix problem in mapping function; add support for potc.2021-12-03
1.1.0: Add version information; fix bug of default value; add flatten and unflatten; optimization speed performance.2021-10-24
1.0.0: Greatly optimize the speed performance using cython, overhead has been reduced to a negligible level.Feedback and ContributeWelcome toOpenDILabcommunity - treevalue!If you meet some problem or have some brilliant ideas, you canfile an issue.Scan the QR code and add us on Wechat:Or just contact us withslackor email (opendilab.contact@gmail.com).Please checkContributing Guidances.Thanks to the following contributors!Citation@misc{treevalue,
    title={{TreeValue} - Tree-Structure Computing Solution},
    author={TreeValue Contributors},
    publisher = {GitHub},
    howpublished = {\url{https://github.com/opendilab/treevalue}},
    year={2021},
}Licensetreevaluereleased under the Apache 2.0 license. See theLICENSEfile for details.