Package: virtual-dataset
Virtual DatasetA small utility that allows to mount virtually a list of files as a folder.InstallationVirtual-Dataset can be installed as a normal python package just runpip install ...UsageThere are two main functionsmount,umountMount:allows creating the temporary mount with all the necessary files inside.Umount:allows deletion of temporary mount foldersSimple Example :fromvdatasetimportmount,unmountfile_list=['data/file1.txt','data/file2.txt','other_data/file6.txt','/data/file.txt']location=mount(file_list)# location folder contains symlinks with all the files specified# cleans up mount and removes all created filesunmount(location)All files are created as symbolic links to the originals so no data copy happens on mountunmount allows a safe option that when turned on will fail to delete the directory if any file in it is not a symlinkMount InputInput can be of multiple format:A simple list of filesList[Path, str]that can be anywhere on your machine.A dictionary allows preserving (or specifying) a file structure inside the created folder
ex:obj={"dir1":['file1.txt','file2.txt','file3.txt','file4.txt'],"dir2":['file5.txt','file6.txt','file7.txt','file8.txt','file9.txt'],"dir3":{"subDir1":['file10.txt','file11.txt','file12.txt','file13.txt','file14.txt','file15.txt'],"subDir2":'file19.txt',},"dir4":['file16.txt','file17.txt','file18.txt']}Creates the following temp folder :tmpdir
├── dir1
│   ├── file1.txt -> source_dir/file1.txt
│   ├── file2.txt -> source_dir/file2.txt
│   ├── file3.txt -> source_dir/file3.txt
│   └── file4.txt -> source_dir/file4.txt
├── dir2
│   ├── file5.txt -> source_dir/file5.txt
│   ├── file6.txt -> source_dir/file6.txt
│   ├── file7.txt -> source_dir/file7.txt
│   ├── file8.txt -> source_dir/file8.txt
│   └── file9.txt -> source_dir/file9.txt
├── dir3
│   ├── subDir1
│   │   ├── file10.txt -> source_dir/file10.txt
│   │   ├── file11.txt -> source_dir/file11.txt
│   │   ├── file12.txt -> source_dir/file12.txt
│   │   ├── file13.txt -> source_dir/file13.txt
│   │   ├── file14.txt -> source_dir/file14.txt
│   │   └── file15.txt -> source_dir/file15.txt
│   └── subDir2
│       └── file19.txt -> source_dir/file19.txt
└── dir4
    ├── file16.txt -> source_dir/file16.txt
    ├── file17.txt -> source_dir/file17.txt
    └── file18.txt -> source_dir/file18.txtInput can be taken from yaml or json files (using mount_from_index_file function)Input can also be another directory (using mount_from_location function)CLIA command line utility exists to allow usage from outside python.When installing the package thevmountcommand is installed in the current environment.❯vmount
usage:vmount[-h][-uUMOUNT][-iMOUNT_FROM_INDEX][-dMOUNT_FROM_DIR][--unsafe][-kINDEX_KEY][-tTMP_PREFIX][-sKEEP_STRUCTURE][-pPATTERN]optionalarguments:-h,--helpshowthishelpmessageandexit-uUMOUNT,--umountUMOUNTunmountlocation-iMOUNT_FROM_INDEX,--mount-from-indexMOUNT_FROM_INDEXmountfromanindexfile[yaml,json]-dMOUNT_FROM_DIR,--mount-from-dirMOUNT_FROM_DIRmountfromadirectory--unsafepracticeunsafeunmountingtechniques(default:false)-kINDEX_KEY,--index-keyINDEX_KEYpathtosub-itemwhenloadingindexobject(delimitedbydotsex:key1.item3)-tTMP_PREFIX,--tmp-prefixTMP_PREFIXUsethislocationasaprefixforcreatingmountpoint-sKEEP_STRUCTURE,--keep-structureKEEP_STRUCTUREKeepdirectorystructurewhenmountingfromdir-pPATTERN,--patternPATTERNPatterntomatchwhenmountingfromdir(list)