Package: research-toolbox
Motivation for a research toolboxThis repo contains a number of Python tools that I developed for doing experimental research in machine learning.
It includes a broad set of functionality:file system manipulation, e.g., creating and deleting a file or a folder, checking if a given file or folder exists, and listing files or folders in a directory.interacting with a remote server, e.g., synchronizing files and folders to and from the server, and running commands on the server from the local machine.writing and reading simple file types, such as JSON, and pickle files.creating folders with experiment configurations that can then be easily ran locally or on the server.logging functionality for keeping track of important information when running code, e.g., memory usage or time since start.While Python has a broad set of functionality, using directly this functionality has an unnecessary high cognitive load because the functions necessary to implement the desired functionality are spread across multiple libraries and use different API and different design principles.
Existing APIs are often unnecessarily flexible for the most common use-cases needed by a particular user.
Developing your own wrapper APIs reduces cognitive load by making common use-cases more explicit.
These APIs are easy to use because they are high-level, coherent, and adjusted to the needs of that particular user.
These wrapper APIs can include high-level error-checking for each use-case, which would require considerably higher cognitive load to implement from scratch using existing APIs.I am not claiming that this library solves all problems that you may have.
I am suggesting that creating and maintaining your own research toolbox is convenient and should lead to being able to get things done faster and an overall more pleasant experience.
I recommend extending this toolbox or develop your own to suit your needs.
This library is work in progress.
The ultimate goal is to go from research idea to results as fast as possible.File descriptiontb_augmentation.py:
simple data augmentation.tb_data.py:
data loaders and data related functionality.tb_debugging.py:
error checking and debugging functionality .tb_experiments.py:
writing experiment folders with configurations for running different experiments.tb_filesystem.py:
creating, copying, and testing for existence of files and folders.tb_interact.py:
interactive commands for running jobs on the server or locally.tb_io.py:
reading and writing simple file types.tb_logging.py:
common logging funtionality.tb_plotting.py:
wrappers around plotting libraries such as matplotlib to make simple plot generation easier.tb_preprocessing.py:
simple preprocessing functionality for going from raw data to data that is more ameanable for the application of machine learning.tb_project.py:
creation of the typical project structure for a machine learning project.tb_random.py:
simple random functionality for shuffling, sorting, and sampling.tb_remote.py:
interaction with remote servers, such as syncing folders to and from the local machine, and submitting jobs to the server.tb_resource.py:
getting information about available resources in a machine, such as the number of CPUs or GPUS.tb_training.py:
learning rate schedules and additional logic that is often employed in training machine learning models such as saving and loading the best model found during training.Example code### retrieving certain keys from a dictionary (example from tb_utils.py)defsubset_dict_via_selection(d,ks):return{k:d[k]forkinks}### sorting and randomness tools (examples from tb_random.py)defargsort(xs,fns,increasing=True):"""The functions in fns are used to compute a key which are then used toconstruct a tuple which is then used to sort. The earlier keys are moreimportant than the later ones."""defkey_fn(x):returntuple([f(x)forfinfns])idxs,_=tb_ut.zip_toggle(sorted(enumerate(xs),key=lambdax:key_fn(x[1]),reverse=notincreasing))returnidxsdefsort(xs,fns,increasing=True):idxs=argsort(xs,fns,increasing)returnapply_permutation(xs,idxs)defapply_permutation(xs,idxs):assertlen(set(idxs).intersection(range(len(xs))))==len(xs)return[xs[i]foriinidxs]defapply_inverse_permutation(xs,idxs):assertlen(set(idxs).intersection(range(len(xs))))==len(xs)out_xs=[None]*len(xs)fori_from,i_toinenumerate(idxs):out_xs[i_to]=xs[i_from]returnout_xsdefshuffle_tied(xs_lst):assertlen(xs_lst)>0andlen(map(len,xs_lst))==1n=len(xs_lst[0])idxs=random_permutation(n)ys_lst=[apply_permutation(xs,idxs)forxsinxs_lst]returnys_lst### io tools (examples from tb_io.py)defread_textfile(filepath,strip=True):withopen(filepath,'r')asf:lines=f.readlines()ifstrip:lines=[line.strip()forlineinlines]returnlinesdefwrite_textfile(filepath,lines,append=False,with_newline=True):mode='a'ifappendelse'w'withopen(filepath,mode)asf:forlineinlines:f.write(line)ifwith_newline:f.write("\n")defread_jsonfile(filepath):withopen(filepath,'r')asf:d=json.load(f)returnddefwrite_jsonfile(d,filepath,sort_keys=False):withopen(filepath,'w')asf:json.dump(d,f,indent=4,sort_keys=sort_keys)defread_picklefile(filepath):withopen(filepath,'rb')asf:returnpickle.load(f)defwrite_picklefile(x,filepath):withopen(filepath,'wb')asf:pickle.dump(x,f)### path tools (examples from tb_filesystem.py)defpath_prefix(path):returnos.path.split(path)[0]defpath_last_element(path):returnos.path.split(path)[1]defpath_relative_to_absolute(path):returnos.path.abspath(path)defpath_exists(path):returnos.path.exists(path)deffile_exists(path):returnos.path.isfile(path)deffolder_exists(path):returnos.path.isdir(path)defcreate_file(filepath,abort_if_exists=True,create_parent_folders=False):assertcreate_parent_foldersorfolder_exists(path_prefix(filepath))assertnot(abort_if_existsandfile_exists(filepath))ifcreate_parent_folders:create_folder(path_prefix(filepath),abort_if_exists=False,create_parent_folders=True)withopen(filepath,'w'):passdefcreate_folder(folderpath,abort_if_exists=True,create_parent_folders=False):assertnotfile_exists(folderpath)assertcreate_parent_foldersorfolder_exists(path_prefix(folderpath))assertnot(abort_if_existsandfolder_exists(folderpath))ifnotfolder_exists(folderpath):os.makedirs(folderpath)defcopy_file(src_filepath,dst_filepath,abort_if_dst_exists=True,create_parent_folders=False):assertfile_exists(src_filepath)assertsrc_filepath!=dst_filepathassertnot(abort_if_dst_existsandfile_exists(dst_filepath))src_filename=path_last_element(src_filepath)dst_folderpath=path_prefix(dst_filepath)dst_filename=path_last_element(dst_filepath)assertcreate_parent_foldersorfolder_exists(dst_folderpath)ifnotfolder_exists(dst_folderpath):create_folder(dst_folderpath,create_parent_folders=True)shutil.copyfile(src_filepath,dst_filepath)defcopy_folder(src_folderpath,dst_folderpath,ignore_hidden_files=False,ignore_hidden_folders=False,ignore_file_exts=None,abort_if_dst_exists=True,create_parent_folders=False):assertfolder_exists(src_folderpath)assertsrc_folderpath!=dst_folderpathassertnot(abort_if_dst_existsandfolder_exists(dst_folderpath))if(notabort_if_dst_exists)andfolder_exists(dst_folderpath):delete_folder(dst_folderpath,abort_if_nonempty=False)pref_dst_fo=path_prefix(dst_folderpath)assertcreate_parent_foldersorfolder_exists(pref_dst_fo)create_folder(dst_folderpath,create_parent_folders=create_parent_folders)# create all folders in the destination.args=subset_dict_via_selection(locals(),['ignore_hidden_folders','ignore_hidden_files'])fos=list_folders(src_folderpath,use_relative_paths=True,recursive=True,**args)forfoinfos:fo_path=join_paths([dst_folderpath,fo])create_folder(fo_path,create_parent_folders=True)# copy all files to the destination.args=subset_dict_via_selection(locals(),['ignore_hidden_folders','ignore_hidden_files','ignore_file_exts'])fis=list_files(src_folderpath,use_relative_paths=True,recursive=True,**args)forfiinfis:src_fip=join_paths([src_folderpath,fi])dst_fip=join_paths([dst_folderpath,fi])copy_file(src_fip,dst_fip)defdelete_file(filepath,abort_if_notexists=True):assertfile_exists(filepath)or(notabort_if_notexists)iffile_exists(filepath):os.remove(filepath)defdelete_folder(folderpath,abort_if_nonempty=True,abort_if_notexists=True):assertfolder_exists(folderpath)or(notabort_if_notexists)iffolder_exists(folderpath):assertlen(os.listdir(folderpath))==0or(notabort_if_nonempty)shutil.rmtree(folderpath)else:assertnotabort_if_notexists