Package: rayconfig
rayconfigPolymorphic structuredjsonargparse-friendly configurations asdataclassesforrayjobs.
WritejsonargparseCLI's forrayjobs.
Pass configurations as typed objects.
Configure hyperparameterssearch space.Mainlyrayconfigdoes the following:Enables conversion between pythondictand corresponding polymorphicdataclassby using the convention ofjsonargparseusingclass_pathandinit_argsconfig fieldsrayconfigadaptsdacite'sfrom_dictfunction to convert pythondictto corresponding pythondataclass.rayconfigadapts python'sdataclasses.asdictfunction to convert adataclassto a pythondict.IntroducesParam[T]type to enable configuring hyperparameters of typeTusingRay Tune Search Space APIInstallation$ pip install rayconfigQuick startfromdataclassesimportdataclassfromrayconfigimportasdict,from_dict@dataclassclassConfig:user:strport:intconfig_dict={'user':'Ahmed','port':8080,}config_obj=Config(user='Ahmed',port=8080)assertconfig_dict==asdict(config_obj)assertconfig_obj==from_dict(data_class=Config,data=config_dict)Basic ExampleWrite polymorphic nested config types with tunable hyperparameters:All classes should be annotated with@dataclass.Extendable classes must inherit fromabc.ABC.Tunable hyperparameters of typeTshould useParam[T].fromabcimportABCfromdataclassesimportdataclassfromrayconfigimportParam@dataclassclassModelConfig(ABC):name:str@dataclassclassModelAConfig(ModelConfig):n1_dropout:Param[float]hidden_dim:Param[int]@dataclassclassModelBConfig(ModelConfig):fc_out:Param[int]embed_dim:Param[int]@dataclassclassOptimizerConfig(ABC):name:str@dataclassclassOptimizerAConfig(OptimizerConfig):lr:Param[float]@dataclassclassOptimizerBConfig(OptimizerConfig):lr:Param[float]momentum:Param[float]@dataclassclassTrainConfig:model:ModelConfigoptimizer:OptimizerConfigbatch_size:Param[int]WritejsonargparseCLIfromjsonargparseimportArgumentParser,ActionConfigFiledefmain():parser_train=ArgumentParser()parser_train.add_argument("config",type=TrainConfig)parser=ArgumentParser(prog="app")parser.add_argument("--config",action=ActionConfigFile)subcommands=parser.add_subcommands()subcommands.add_subcommand("train",parser_train)subcommands.add_subcommand("tune",parser_train)config=parser.instantiate_classes(parser.parse_args())if"train"inconfig:train(config.train.config)elif"tune"inconfig:tune(config.tune.config)if__name__=="__main__":main()Convert config object to dictionary when needed usingasdictimportrayfromrayconfigimportasdictdeftrain(config:TrainConfig):...trainer=ray.train.torch.TorchTrainer(train_func,train_loop_config=asdict(config)# <--- HERE)...result=trainer.fit()...deftune(config:TrainConfig):...trainer=ray.train.torch.TorchTrainer(train_func,)...tuner=ray.tune.Tuner(trainer,param_space={"train_loop_config":asdict(config)# <--- HERE},)...result_grid=tuner.fit()...Convert config dictionary to object when needed usingfrom_dictfromrayconfigimportfrom_dictdeftrain_func(config_dict):...config=from_dict(data_class=TrainConfig,data=config_dict)# <--- HERE...optimizer=get_optimizer(config.optimizer)...model=get_model(config.model)...Write configurations, e.g. yaml file:Classes extending from base classes must be qualified byclass_path, a python import path,
and fields grouped underinit_argsHyperparameters can take a single value or to be configured usingRay Tune Search Space APIWARNING: currentlyrayconfigusesevalto evaluate ray tune param configurations.
This can introduce security risks. So, use with caution if you only trust your execution environment.tune:config:model:class_path:app.ModelAConfiginit_args:n1_dropout:ray.tune.uniform(0.4, 0.6)hidden_dim:"ray.tune.sample_from(lambda_:2**__import__('numpy').random.randint(7,9))"optimizer:class_path:app.OptimizerBConfiginit_args:lr:ray.tune.grid_search([0.001, 0.01, 0.1])momentum:0.9batch_size:ray.tune.grid_search([32, 64, 128])