Package: hypod
Hypod: A dataclass-based hyperparameter managing systemOverviewDeep learning models are often composed of multiple networks, where each of the networks is composed of multiple layers, and the class of each layer or its hyperparameters differ by experiments.Hypodsimplifies managing this complex hierarchy of hyperparameters utilizing the built-indataclass.dataclassderives the following benefits.Defining a group of hyperaparameters is easy withtype annotationsupport.Creating it is easy with the auto-defined__init__.Many IDE's (e.g., PyCharm, VSCode) support "jump to the definition".However, difficulties for using it "as-is" in hyperparameter-managing are:Handling nested dataclasses is not so good.Parsing strings or sys.argv to create a dataclass is not natively supported.Switching between multiple child dataclasses using a simple "tag" is cumbersome.Hypodwill handle all these difficulties for you with following advantages.Fast, lightweight implementation using the built-indataclassanddescriptorsMinimal dependencyType-checking (based on annotation)Parsing a stringified object or YAML to create the corresponding nested dataclassAbility to auto-make the corresponding module (e.g., layer, network), when defined as an inner class of that module.Comparison with other packagesHydra: Hydra is a popular package with the same purpose. Itsstructured configmode allows defining a config withdataclassas well. However, Hydra converts a config intoDictConfigobject (from another packageomegaconf) even when it is originally defined withdataclass. Thus, all the operations (modifying, getting values, merging) are done withDictConfigand this brings the following drawbacks compared to Hypod, which usesdataclassobject all the time.DictConfigis not type annotated."Go to the definition" in IDE cannot be done withDictConfig.Inheritance strcuture of configs cannot be checked.Complex value interpolation is difficult or impossible to implement. In Hypoddataclass, it can be done using built-in__post_init__()function (seeexamples/advanced_usage.py).EtymologyHypod stands for"A Pod of Hyperparameters, or A Hyperparameters-Pod". You can put various Hyperparameters as a Pod and compose them with other Pods as you wish.Quick startInstall the packagepip install hypodBasic Usagefromdataclassesimportdataclassfromhypodimporthypod@dataclass@hypodclassLayerHP:in_features:intout_features:intinit_scale:float=0.9@dataclass@hypodclassNetworkHP:num_layers:intlayer_hp:LayerHPif__name__=="__main__":# Hypod can be created in the same manner as dataclass.net_hp1=NetworkHP(num_layers=3,layer_hp=LayerHP(64,32))print(net_hp1)# Hypod can be also created from parsing stringified objects.net_hp2=NetworkHP(num_layers="3",layer_hp=LayerHP("64",32))print(net_hp2)# Hypod class itself can be also created from a dictionary.net_hp3=NetworkHP(num_layers=3,layer_hp=dict(in_features=64,out_features=32))print(net_hp3)Command-line UsageWith the Hypod classes defined the same as above, define the main function as follows.# main.pyfromhypodimporthypod_main@hypod_main()# parses sys.argv to construct the hypod in the first argument, `net_hp`defmain(net_hp:NetworkHP):print(net_hp)if__name__=="__main__":main()Then, on the command-line, type as follows to obtain the same results as before.python main.py num_layers=3 layer_hp.in_features=64 layer_hp.out_features=32InheritanceHypod can be subclassed with atag.fromdataclassesimportdataclassfromhypodimporthypod@dataclass@hypodclassData:path:strbatch_size:int=4@dataclass@hypodclassFFHQData(Data,tag="ffhq"):path:str="/path/to/FFHQ"meta:str="Flicker-Faces HQ Data, containing 70k images"@dataclass@hypodclassFFHQDataLargeBatch(FFHQData,tag="ffhq_lg"):batch_size:int=16@dataclass@hypodclassModel:data:Datanet:NetworkHPif__name__=="__main__":model_with_ffhq=Model(data="ffhq",net=NetworkHP(num_layers=3,layer_hp=LayerHP(64,32)),)print(model_with_ffhq)model_with_ffhq_lg=Model(data=dict(_tag="ffhq_lg"),# Also can be created from dict with "_tag" key.net=NetworkHP(num_layers=3,layer_hp=LayerHP(64,32)),)print(model_with_ffhq_lg)model_with_cifar10=Model(data=dict(path="/path/to/cifar10"),net=NetworkHP(num_layers=3,layer_hp=LayerHP(64,32)),)print(model_with_cifar10)Command-line Usage with InheritanceWith the Hypod classes defined the same as above, define the main function as follows.# main.pyfromhypodimporthypod_main@hypod_main()# parses sys.argv to construct the hypod in the first argument, `model`defmain(model:Model):print(model)if__name__=="__main__":main()Then, in the command-line type as follows to obtain the same results as before.python main.py model.data=ffhq model.net.num_layers=3 model.net.layer_hp.in_features=64 model.net.layer_hp.out_features=32I.e., tag can be fed in the CLI as the root argument.Advanced Usage (Value Interpolation)Please refer toadvanced_usage.pyin theexamplesdirectory.Please note that manually updating the fields of a Hypod object is prohibited. You should always usereplacewhich will automatically call__post_init__()to correctly process the value interpolation.FAQ(Q) When creating Hypod from dict, how does it know which hypod to create?(A) Via type annotation.(Q) Istyping.Unionsupported?(A) Yes. But when it is a union ofstrordict, there are ambiguities in parsing the stringified objects. A warning or an error will be raised in this case.(Q) Does tag-based creation still work whentyping.Unionis used?(A) Yes. The tag search will be done over all the types in the union.(Q) Aretyping.Listor other generic types supported?(A) Yes.(Q) How can I use value interpolation?(A)dataclassprovides__post_init__()function where you can define interpolation rules inPython. E.g., if you want to make a hyperparameterathat has to be twice the value ofbin some Hypod, writeself.a = 2 * self.bin__post_init__()of that Hypod.(Q) Why not usingtyping.dataclass_transform()in implementing Hypod?(A) Because requirement of Python>=3.11 is too strict for now.