Package: brane
Brane: Extensible data I/O interface with hook systems and data trackingBrane is designed to provide a rich I/O management system in the Python programming.
The goals areto avoid setting the common arguments or processings on the I/O eventsto be allowed to track data flowsThen, we can save time, get less human error and make codes readable.InstallationDependenciesRequirementsPyYAMLfsspectyping-extension (Python 3.7)You also need other modules for reading or writing depending on use cases.Via PyPIYou can use any tools such as pip, pipenv or poetry:$pipinstallbrane
$pipenvinstallbrane
$poetryaddbraneCurrently, the conda installation is not supported yet.Via Github Source$pipinstallgit+https://github.com/nrakwtnb/brane.git
$pipenvinstallgit+https://github.com/nrakwtnb/brane.git#egg=brane
$poetryaddgit+https://github.com/nrakwtnb/brane.gitIf you'd like to specify the version, try$pipinstallgit+https://github.com/nrakwtnb/brane.git@<version>#egg=brane
$pipenvinstallgit+https://github.com/nrakwtnb/brane.git@<version>#egg=brane
$poetryaddgit+https://github.com/nrakwtnb/brane.git#<version>where the version can be one of0.0.dev0DocumentSeeDocument.FeaturesUsageUnified I/O InterfacefrombraneimportExtendedIOasxio# read a single filepath:str="<path to load>"obj=xio.read(path)# read from remote storagesremote_path:str="s3://<bucket>/<blob>"# valid only if your environment detects AWS configremote_path:str="gcs://<bucket>/<blob>"# valid only if your environment detects GCP configread_kwargs=dict(arg1=param1,arg2=param2,...)# arg1, arg2, ... are arguments for the read method of the internally loaded module depending on the extensionobj=xio.read(path=remote_path,**read_kwargs)# read multiple filespath_list:list[str]=["<path1>","<path2>",...]objs=xio.read_all_as_list(path_list)# writesave_path:str="<path to save>"xio.write(obj,save_path)# write into remote storagesremote_path:str="s3://<bucket>/<blob>"# valid only if your environment detects AWS configremote_path:str="gcs://<bucket>/<blob>"# valid only if your environment detects GCP configwrite_kwargs=dict(arg1=param1,arg2=param2,...)# arg1, arg2, ... are arguments for the write method of the internally loaded module depending on the extension and the object to savexio.write(obj=obj,path=remote_path,**write_kwargs)# write multiple objectsoutput_dir:str="<output_dir>"name2obj={"<name1>":<obj1>,"<name2>":<obj2>,...}xio.write_all_from_dict(name2obj,output_dir=output_dir)Hook systemfrombraneimportExtendedIOasxio# set a hook of postprocess after loading an objectdefpostprocess(context):obj=context["object"]# ... some postprocess on objreturnprocessed_objdefcheck_condition(context)->bool:obj=context["object"]path=context["path"]# ... determine whether the hook is called or notreturncalledxio.register_post_read_hook(hook=postprocess,condition_func=check_condition)# set a hook of optimiation before objects with the certain type are savedtarget_obj_type=<objecttype>defoptimize(context):obj=context["object"]# ... some optimization process on objreturnoptimized_objxio.register_pre_write_hook(hook=optimize,object_type=target_obj_type)ExamplesSeeExample Page.InformationLicenseApache 2.0ChangelogDevelopmentContribution