Package: codesave-library
CodesaveTL;DRcodesave.-ocodebase.zip
code_from_wandb"wandb_username/project_name/run_id"-ocodesave.zip### When trainingfromcodesaveimportcheckpoint_to_wandbwandb.init(...)checkpoint_to_wandb(codebase_dirname)# or wandb.run.log_code(codebase_dirname)### Time to recoverfromcodesaveimportWandBCodebasewithWandBCodebase("wandb_username/project_name/run_id"):fromsrcimportmodelmodel.whatever_you_wantInstallation:pip install codesave_librarypip install git+https://github.com/dibyaghosh/codesave.gitBasic UsageThe easiest way to use codesave is usingcheckpointorcheckpoint_to_wandb, which will take all the files in the directory and create a zip file.For example, suppose we have some repository with the following structure:codebase/
    src/
        model.py
        utils.py
    train.py
    eval.pyfrom codesave import checkpoint, checkpoint_to_wandb
checkpoint('codebase/', output_zipname='codebase.zip')
checkpoint_to_wandb('codebase/') # Stores it in the current wandb run (on the cloud)This creates a zipfile that we can use anywhere else  (even if we don't have codesave installed):import sys
sys.path.append('codebase.zip')
import src.model # This will load the library directly from the zip fileLoading a CodebaseUsingsys.pathThe easiest way, doesn't requirecodesaveto be installedimportsyssys.path.append('codebase.zip')# Or PYTHONPATH=codebase.zip:$PYTHONPATHimportsrc.model# This will load the library directly from the zip fileUsingCodebaseThe second easiest way, butcannot be run multiple times in the same process(because old modules are not unloaded)fromcodesaveimportCodebasewithCodebase('codebase.zip'):fromsrcimportmodelmodel.whatever_you_wantUsingUniqueCodebase(general, recommended)Using UniqueCodebase, we can load as many versions of a codebase as we want, and they can interact with each other.The import notation is a little less clean, but it's necessary to avoid conflicts.fromcodesaveimportUniqueCodebasecodebase1=UniqueCodebase('codebase.zip')codebase2=UniqueCodebase('codebase_v2.zip')model_v1=codebase1.import_module('src.model')# equivalent to (from src import model as model_v1) from codebase.zipmodel_v2=codebase2.import_module('src.model')# equivalent to (from src import model as model_v2) from codebase_v2.zipThere are some syntactic sugar options which do some magic juju to automatically assign the module to a variable w/out needing to do explicit assignmentcodebase1.import_('src.model')# equivalent to `import src.model`codebase1.import_('src.model',as_='model_v1')# equivalent to `import src.model as model_v1`codebase1.from_import('src',('model','utils'))# equivalent to `from src import model, utils`codebase1.from_import('src','model',as_='model_v1')# equivalent to `from src import model as model_v1`Just Unzip ItYou can always unzip the zip file, and treat it as a normal directory.Integration w/ wandbWhen training,fromcodesaveimportcheckpoint_codebase_for_wandbwandb.init(...)checkpoint_codebase_for_wandb('codebase/')Now at any point,fromcodesaveimportload_codebase_from_wandbload_codebase_from_wandb(wandb_run_name,output_zipname='codesave.zip')# Now we can add to sys.path or ZipCodebase as beforeAPI# These provide general routines to save codebasefrom.routinesimport(checkpoint,# Save codebase to a zip filecheckpoint_to_wandb,# Save codebase to a zip file and upload to wandb)# These allow you to load a codebase from a zip filefrom.baseimport(Codebase,# Easy load code from a zip fileUniqueCodebase,# General way to load code from a zip file)# These are routines to download codebase from wandbfrom.routinesimport(download_from_wandb,# download something saved with checkpoint_to_wandbzip_from_wandb_artifact,# download something saved with wandb.run.log_code)defcheckpoint(main_folder,output_zipname=None,extra_libraries=tuple(),extra_pythonpath=tuple(),ignore=tuple(),ignore_patterns=tuple(),py_only=False,ignore_larger_than=None,verbose=True,):# Saves a codebase to a zip filedefcheckpoint_to_wandb(main_folder,output_directory=None,extra_libraries=tuple(),extra_pythonpath=tuple(),ignore=tuple(),ignore_patterns=tuple(),py_only=False,ignore_larger_than=None,verbose=True,):# Saves a codebase to output_directory/codebase.zip file and uploads it to wandb