Package: prefigure
prefigureRun-configuration management utils: combines configparser, argparse, and wandb.APICapabilities for archiving run settings and pulling configurations from previous runs.  With just 3 lines of code 😎 : the import, the arg setup, & the wandb push.Combines argparse, configparser, and wandb.API.  WandB logging is done viapytorch_lightning'sWandBLogger.Install:pipinstallprefigureInstructions:All your usual command line args (with the exception of--nameand--training-dir) are now to be specified in adefaults.inifile -- seeexamples/for an example.A different.inifile can be specified via--config-file.Versions 0.0.9 and later: A.gincan be instead be used for--config-file, in which case the sytemonly runs gin and nothing else.The option--wandb-config <url>pulls previous runs' configs off wandb, where<url> is the url of any one of your runs to override those defaults: e.g.--wandb-config='https://wandb.ai/drscotthawley/delete-me/runs/1m2gh3o1?workspace=user-drscotthawley'`(i.e., whatever URL you grab from your browser window when looking at an individual run.)NOTE: the--wandb-configthing can only pull from WandB runs that used prefigure, i.e. that have logged a "wandb config push".Any command line args you specify will override any settings from WandB and/or the.inifile.The order of precedence is "command line args override WandB, which overrides the .ini file".1st line to addIn your run/training code, add this near the top:fromprefigureimportget_all_args,push_wandb_config2nd line to addNear the top of yourmain(), add this:args=get_all_args()Further down in your code, comment-out (or delete)allyour command-line arguments (e.g. ArgParse calls). If you want different command-line arguments, then add or change them in defaults.ini.  The 'help' string for these is provided via  comment in the line preceding your variable. Seeexamples/defaults.inifor examples.3rd line to addand then right after you define thewandb logger, runpush_wandb_config(wandb_logger,args)(Optional:) 4th & 5ths line to add: OFCStarting withprefigurev0.0.8, there is an On-the-Fly Control (OFC,pronounced likewhat you say when you realize you forget to set a variable properly).
This tracks any changes to arguments listed as "steerable" by logging to a separate file (by defaultofc.ini) and
updates those args dyanmically when changes to that file are made. It can also (optionally) log those changes to WandB (and when they occur); see sample usage below.fromprefigureimportOFC...ofc=OFC(args,steerables=vars(args).keys())# allow all args to be steerableor fancier: with the Gradio GUI, and only allowing OFC steering for certain variables (default is all are steerable), and only launch one GUI for a DDP PyTorch Lightning process:ofc = OFC(args, gui=(trainer.global_rank==0), steerables=['lr','demo_every','demo_steps', 'num_demos','checkpoint_every'])If the GUI is enabled, you get a Gradio URL, which is also pushed towandb(as "Media").  By default this URL is onlocalhost, however,
if environment variablesOFC_USERNAMEandOFC_PASSWORDare set, then a temporary public Gradio is obtained. (Since these temporary public URLs expire after 72 hours, we re-launch the GUI every 71 hours and update the link on WandB.)Also, if you setsliders=Truewhen callingOFC(), the float and int variables will get sliders (with max & min guessed at by arg values).  Otherwise, the default is that all variables (excepbooltypes) are expressed via text fields.Sample usage:Here's a rough outline of some pytorch code. Seeexamples/for more.importtorchimporttorch.utils.dataasdatafromprefigureimportget_all_args,push_wandb_config,OFCimportpytorch_lightningasplimportwandbdefmain():# Config setup. Order of preference will be:#   1. Default settings are in defaults.ini file or whatever you specify via --config-file#   2. if --wandb-config is given, pull config from wandb to override defaults#   3. Any new command-line arguments override whatever was set earlierargs=get_all_args()ofc=OFC(args)# optionaldevice=torch.device('cuda'iftorch.cuda.is_available()else'cpu')torch.manual_seed(args.seed)train_set=SampleDataset([args.training_dir],args)train_dl=data.DataLoader(train_set,args.batch_size,shuffle=True,num_workers=args.num_workers,persistent_workers=True,pin_memory=True)wandb_logger=pl.loggers.WandbLogger(project=args.name)# push config to wandb for archiving, but don't push --training-dir value to WandBpush_wandb_config(wandb_logger,args,omit=['training_dir'])demo_dl=data.DataLoader(train_set,args.num_demos,shuffle=True)...#inside training loop# OFC usage (optional)ifhasattr(args,'check_ofc_every')and(step>0)and(step%args.check_ofc_every==0):changes_dict=ofc.update()# check for changes. NOTE: all "args" updated automaticallyif{}!=changes_dict:# other things to do with changes: log to wandbwandb.log({'args/'+k:vfork,vinchanges_dict.items()},step=step)# For easy drop-in OFC capability, keep using args.XXXX for all variables....)if(step>0)and(step%args.checkpoint_every==0):...lr=args.learning_rate# for example. args.learning_rate gets updated by ofc.update()do_stuff(lr)# your code hereExtra TricksImports & Other File Formatsprefiguredefaults to .ini files, but will also read .json and .gin files.  It will also import
said files that are specified as values --ifthese parameters are listed via a separate "imports" parameter, as in the following example:$ cat examples/harmonai-tools.ini 
[DEFAULTS]
# model config fle
model_config = ../../harmonai-tools/harmonai_tools/configs/model_configs/diffusion_autoencoders/seanet_32_32_diffae.json

# dataset config file 
dataset_config = ../../harmonai-tools/harmonai_tools/configs/dataset_configs/s3_wds_example.json

imports = model_config, dataset_configIn this case, bothargs.model_configandargs.dataset_configwill have their filename value stringreplacedby the dict(s) specified in the .json files given.  If they were not listed underimports, then the filename value will remain and no import will occur.LightningIf you want to pass around theofcobject deep inside other libraries, e.g., PyTorch Lightning, I've had success overloading Lightning'sTrainerobject, e.g.trainer.ofc = ofc.  Then do something likemodule.ofc.update()inside the training routine.  For example, cf.my tweet about this.