Package: catcli
CATCLIThe command line catalog tool for your offline data[!WARNING]
catcli has been superseded bygocatcliwhich provides all features of catcli and more...Did you ever wanted to find back that specific file that should be on one of your
backup DVDs or one of your external hard drives? You usually go through all
of them hoping to find the right one on the first try?Catcliindexes external media
in a catalog file and allows to quickly find specific files or even navigate in the
catalog of indexed files while these are not connected to your host.Features:Index any directories in a catalogAbility to search for files by name in the catalogAbility to navigate through indexed data à lalsSupport for fuse to mount the indexed data as a virtual filesystemHandle archive files (zip, tar, ...) and index their contentSave catalog to json for easy versioning with gitCommand line interface FTWStore files and directories sizesStore md5 hash of filesAbility to update the catalogSupport forfzffor finding filesTag your different storages with additional informationExport catalog to CSVQuick start:# install catcli with pippip3installcatcli--user# index a directory in the catalogcatcliindex--meta='some description'log/var/log# display the contentcatclils-r# navigatecatclilslog# find files/directories named '*log*'catclifindlog# show directories sizescatclidulogseeusagefor specific infoWhy catcli?Catcligives the ability to navigate,
explore and find your files that are stored on external media
(DVDs, hard drives, USB sticks, etc) when those are not connected.
Catcli can just as easily index any arbitrary directories.See theexamplesfor an overview of the available features.Table of ContentsInstallationUsageIndex dataIndex archive filesWalk indexed files with lsFind filesMount catalogDisplay entire hierarchyDisk usageCatalog graphEdit storageUpdate catalogCSV formatExamplesContributionThank youInstallationInstall from Pypi$pip3installcatcli--userOr from github directly$cd/tmp;gitclonehttps://github.com/deadc0de6/catcli&&cdcatcli
$sudopython3setup.pyinstall
$catcli--helpTo work with catcli without installing it, you can do the following$cd/tmp;gitclonehttps://github.com/deadc0de6/catcli&&cdcatcli
$pip3install-rrequirements.txt--user
$python3-mcatcli.catcli--helpor install it in a virtualenv$cd/tmp;gitclonehttps://github.com/deadc0de6/catcli&&cdcatcli
$virtualenv-ppython3env
$sourceenv/bin/activate
$pythonsetup.pyinstall
$catcli--helpCatcli is also available on aur:https://aur.archlinux.org/packages/catcli-git/UsageEach indexed directory is stored in the catalog. Multiple directories can be indexed
and they are all available through the command line interface of catcli.Five different types of entry are present in a catalog:top node: this is the root of the hierarchystorage node: this represents an indexed storage (a DVD, an external
hard drive, an USB drive, some arbitrary directory, etc).dir node: this is a directoryfile node: this is a filearchive node: this is a file contained in an archive (tar, zip, etc)Following environment variables are supported:CATCLI_CATALOG_PATH: define the catalog path (--catalog=<path>)CATCLI_NO_BANNER: disable the banner (--no-banner)CATCLI_VERBOSE: enable verbose mode (--verbose)CATCLI_FORMAT: define the output format (-F --format=<fmt>)Index dataLet's say the DVD or external hard drive that needs to be indexed
is mounted on/media/mnt. The following command
will index the entire directory/media/mntand store that in your catalog under the name<short-name>.$catcliindex--meta=<some-description><short-name>/media/mntIf not specified otherwise (with the switch--catalog), the catalog is saved in the current
directory undercatcli.catalog.The--metaswitch allows to add any additional information to store along in
the catalog like for examplethe blue disk in my office.Using the-a --archiveswitch allows to also index archive files as explainedbelow.Index archive filesCatcli is able to index and explore the content of archive files.
Following archive formats are supported:tar,tar.gz,tar.xz,lzma,tar.bz2,zip.
Catcli is also able to find files within indexed archive files.See thearchive examplefor more.Walk indexed files with lsA catalog can be walked using the commandlsas if the media
is mounted (File/directories separator is/).$catclilstmp/a/b/cResulting files can be sorted by size using-S --sortsize.
See theexamplesfor more.Find filesFiles and directories can be found based on their names
using thefindcommand.Findsupport two formats that allow to usefzffor
searching:--format=fzf-native: display the result in native format--format=fzf-csv: display the result in csvSee theexamplesfor more.Mount catalogThe catalog can be mounted withfuseand navigate like any filesystem.$mkdir/tmp/mnt
$catcliindex-cgithub.github
$catclimount/tmp/mnt
$ls-laR/tmp/mnt
drwxrwxrwx-user8Mar22:08github

mnt/github:
.rwxrwxrwx17user19Oct2022FUNDING.yml
drwxrwxrwx-user2Mar10:15workflows

mnt/github/workflows:
.rwxrwxrwx691user19Oct2022pypi-release.yml
.rwxrwxrwx635user8Mar21:08testing.ymlDisplay entire hierarchyThe entire catalog can be shown using thels -rcommand.
Resulting files can be sorted by size using the-S --sortsizeswitch.See theexamplesfor more.Disk usageYou can get the disk usage with theducommand.
Resulting files can be sorted by size using the-S --sortsizeswitch.Catalog graphThe catalog can be exported in a dot file that can be used to
generate a graph of the indexed files.$catcligraph
dotfilecreatedunder"/tmp/catcli.dot"creategraphwith"dot /tmp/catcli.dot -T png -o /tmp/tree.png"(youneedgraphviz)$dot/tmp/catcli.dot-Tpng-o/tmp/tree.pngEdit storageStorage entry can be edited with following catcli commands:rename- rename the storageedit- edit storage metadataUpdate catalogThe catalog can be updated with theupdatecommand.
Updates are based on the access time of each of the files and on the
hash checksum if present (catalog was indexed with-c --hashandupdateis called with the switch-c --hash).CSV formatResults can be printed to CSV using--format=csv.
Fields are separated by a comma (,) and are quoted with double quotes (").Each line contains the following fields:name: the entry nametype: the entry type (file, directory, storage, etc)path: the entry pathsize: the entry sizeindexed_at: when this entry was indexedmaccess: the entry modification date/timemd5: the entry checksum (if any)nbfiles: the number of children (empty for nodes that are not storage or directory)free_space: free space (empty for not storage nodes)total_space: total space (empty for not storage nodes)meta: meta information (empty for not storage nodes)ExamplesSimple exampleLet's first create some files and directories:$mkdir-p/tmp/test/{a,b,c}$echo'something in files in a'>/tmp/test/a/{1,2,3}$echo'something else in files in b'>/tmp/test/b/{4,5,6}$echo'some bytes'>/tmp/test/c/{7,8,9}$tree/tmp/test
/tmp/test
├──a
│├──1│├──2│└──3├──b
│├──4│├──5│└──6└──c├──7├──8└──93directories,9filesFirst this directory is indexed withcatclias if it was some kind of
external storage:$catcliindex--meta='my test directory'tmptest/tmp/testCatcli creates its catalog file in the current directory ascatcli.catalog.Printing the entire catalog as a tree is done with the commandls -r$ catcli ls -r
top
└── storage: tmptest (my test directory) (nbfiles:3, free:3.7G/3.7G, date:2019-01-26 19:59:47)
    ├── a [nbfiles:3, totsize:72]
    │   ├── 1 [size:24]
    │   ├── 2 [size:24]
    │   └── 3 [size:24]
    ├── b [nbfiles:3, totsize:87]
    │   ├── 4 [size:29]
    │   ├── 5 [size:29]
    │   └── 6 [size:29]
    └── c [nbfiles:3, totsize:33]
        ├── 7 [size:11]
        ├── 8 [size:11]
        └── 9 [size:11]The catalog can be walked withlsas if it was a normal directory$ catcli ls
top
- storage: tmptest (my test directory) (nbfiles:3, free:3.7G/3.7G, date:2019-01-26 19:59:47)

$ catcli ls tmptest
storage: tmptest (my test directory) (nbfiles:3, free:3.7G/3.7G, date:2019-01-26 19:59:47)
- a [nbfiles:3, totsize:72]
- b [nbfiles:3, totsize:87]
- c [nbfiles:3, totsize:33]

$ catcli ls tmptest/b
b [nbfiles:3, totsize:87]
- 4 [size:29]
- 5 [size:29]
- 6 [size:29]And files can be found using the commandfind$catclifind9c/9[size:11,storage:tmptest]When using the-b --scriptswitch, a one-liner is generated
that allows to handle the found file(s)$ catcli find 9 --script

c/9 [size:11, storage:tmptest]
op=file; source=/media/mnt; $op ${source}/c/9Archive exampleLet's consider a directory containing archive files:$ls-1/tmp/catcli
catcli-0.3.1
v0.3.1.tar.gz
v0.3.1.zipTo enable the indexing of archive contents use
the-a --archiveswitch$catcliindex-ausome-name/tmp/catcliThen any command can be used to explore the catalog as for normal
files but, by providing the-a --archiveswitch, archive content are displayed.$catclilssome-namestorage:some-name(free:800G,total:1T)-catcli-0.3.1[nbfiles:11,totsize:80.5K]-v0.3.1.tar.gz[size:24.2K]-v0.3.1.zip[size:31.2K]$catclils-rsome-name/v0.3.1.zipv0.3.1.zip[size:31.2K]$catclils-arsome-name/v0.3.1.zipv0.3.1.zip[size:31.2K]├──catcli-0.3.1[archive:v0.3.1.zip]│├──catcli[archive:v0.3.1.zip]││├──__init__.py[archive:v0.3.1.zip]││├──catalog.py[archive:v0.3.1.zip]││├──catcli.py[archive:v0.3.1.zip]││├──logger.py[archive:v0.3.1.zip]││├──noder.py[archive:v0.3.1.zip]││├──utils.py[archive:v0.3.1.zip]││└──walker.py[archive:v0.3.1.zip]│├──.gitignore[archive:v0.3.1.zip]│├──LICENSE[archive:v0.3.1.zip]│├──MANIFEST.in[archive:v0.3.1.zip]│├──README.md[archive:v0.3.1.zip]│├──requirements.txt[archive:v0.3.1.zip]│├──setup.cfg[archive:v0.3.1.zip]│├──setup.py[archive:v0.3.1.zip]│├──tests[archive:v0.3.1.zip]││├──__init__.py[archive:v0.3.1.zip]││├──helpers.py[archive:v0.3.1.zip]││├──test_find.py[archive:v0.3.1.zip]││├──test_graph.py[archive:v0.3.1.zip]││├──test_index.py[archive:v0.3.1.zip]││├──test_ls.py[archive:v0.3.1.zip]││├──test_rm.py[archive:v0.3.1.zip]││└──test_tree.py[archive:v0.3.1.zip]│├──tests.sh[archive:v0.3.1.zip]│└──.travis.yml[archive:v0.3.1.zip]└──catcli-0.3.1/[archive:v0.3.1.zip]ContributionIf you are having trouble installing or using catcli, open an issue.If you want to contribute, feel free to do a PR (please follow PEP8).Thetests.shscript can be run to check the code.Thank youIf you like catcli,buy me a coffee.LicenseThis project is licensed under the terms of the GPLv3 license.