Package: iotools-wrappers
iotoolsiotoolsprovides a unified syntax for reading and writing data to files.Example:text=load_txt("filename.txt")data=load_json("filename.json")table=load_csv("filename.csv")image=load_jpg("filename.jpg")config=load_yaml("filename.yaml")FunctionsReadingdata = load_*(filename): filename is a path to an existing filedata = read_*(openfile): openfile is a file-like objectdata = decode_*(rawbytes): rawbytes is the file's bytesWritingsave_*(filename, data): filename is a path to an existing filewrite_*(openfile, data): openfile is a file-like objectrawbytes = encode_*(data): returns bytesHelpinghelp_*(): prints simple examples and simple implementations on how to use this file format without iotools functionsInstallationpipinstalliotools-wrappersSupported formatsavibytes: not really a format, but use this to load raw bytescsvgifgz: gzipjpgjsonmp4npy: numpypicklepngtartxt: text filesxmlyamlzipzst: zstandardHow it worksiotoolsis nothing but wrappers on other python librairies, for exampleload_jsonwill calljson.loadinternally.
The backend used is specified in each function documentation.iotoolsgoal is to provide a memorable io syntax, on as many file formats as possible.For more format-specific functions, or more complex io features, please use the library specific to that file format (help_*()may provide a good starting point)ExamplesTextfromiotoolsimportload_txt,read_txt# Load text in one line:text=load_txt("filename.txt")# Or if you prefer the 'with open()' syntax:withopen("filename.txt","r")asf:text=read_txt(f)# same as f.read()Images# Example where image colors are inversedfromiotoolsimportload_image,save_imageimage=load_image("input.jpg")save_image("output.jpg",image[:,:,::-1])Archives# Load a tar.gz archivefromiotoolsimportload_gz,decode_tardata=decode_tar(load_gz("archive.tar.gz"))# If your archive is made of images, you can even do:fromiotoolsimportload_gz,decode_tar,decode_imagedata={filename:decode_image(img)forfilename,imgindecode_tar(load_gz("archive.tar.gz")).items()}GIFsimportnumpyasnpfromiotoolsimportsave_gifgif=[np.ones((180,320),dtype=np.uint8)*i*10foriinrange(26)]save_gif("filename.gif",gif,fps=6)Additionnal featuresFile systemsiotoolsis compatible withfsspec:load_*andsave_*functions have anfskwarg in order to use different file systems.Example:# Load a tar.gz archive containing imagesfromiotoolsimportload_imagefromfsspec.implementations.tarimportTarFileSystemtarfs=TarFileSystem("archive.tar.gz")data={filename:load_image(filename,fs=tarfs)forfilenameintarfs.glob("*")}Makedirssave_*(filename, data, fs=None, makedirs=False)have a kwargmakedirsthat allows to automatically createfilename's parent directory if it does not exist.Format specific optionsSinceiotoolsfunctions are mostly wrappers of other python libraires, most of formats specific options can be used throughiotoolsfunctions.
For example you can dosave_json("filename.json", data, indent=4).Use your prefered librarySometimes several python librairies can be used to load/save data to a format. To let you use your preferred library,iotoolsfunctions are declined in several forms with the keywordusing. Here is an example with jpg images:fromiotools.jpgioimportload_jpg_using_imageio,load_jpg_using_pil,load_jpg_using_cv2image=load_jpg_using_imageio("filename.jpg")# uses imageio libraryimage=load_jpg_using_pil("filename.jpg")# uses PIL (Pillow) libraryimage=load_jpg_using_cv2("filename.jpg")# uses cv2 (opencv-python) libraryNote that the output image can be different based on which library is used internallyGeneric formatsiotoolssupports 2 generic formats: images and videos. Thus there are functions likeload_imageorload_videoto load any image or video format (formats not enumerated above were not tested).save_imageorsave_videoalso exists, and a kwargformatis there for you to choose which image/video format to save in, otherwise it will be guessed from the filename.Other notesImport syntaxSeveral import syntaxs can be used:fromiotoolsimportload_csvfromiotoolsimportcsvio# --> csvio.load_csvfromiotools.csvioimportload_csv,load_csv_using_pandasOS Supportiotoolswas only tested under Linux. Under Windows, it seems that there are still issues with csvs, images and videos, but the other formats should be fine.DependenciesInstallingiotoolsdoes not install any format-specific backends, thus you may not be able to use a file format if the backend is not already installed. Please refer topyproject.tomlfor backends supported versions.