Package: unifs
unifsUnified FS-like CLI for S3, GCS, ADLS, HDFS, SMB, Dropbox, Google Drive, and
dozens of other "file systems".unifs conf use my-s3-bucket
unifs ls -l /
unifs mv /foo.txt /bar.txt
unifs download /bar.txt ~/Downloads/local.copy.txtunifsuses the term "file system" in an open sense for anything that can be
represented as a set of files and directories and be manipulated with the
commands likels,cat,cp, andmvfor example (list is not exhaustive).unifsalso allows data upload and download when working with remote back-ends
(e.g., a cloud-based BLOB storage).unifssupports multiple back-ends, such as a local file system, (S)FTP,
Google Drive, various blob storage such as S3, GCS, ADLS, and dozens of other
implementations. Useunifs impl listto list supported protocols, but know
that other protocols can be added, including any custom implementations users
may provide.unifsis different from FUSE implementations in that it doesn't mount a file
system. Instead, it provides a unified CLI that uses target back-end API to
execute the issued commands.Installationunifsis a Python package:pip install unifsDefaultunifsinstallation only supports a few basic protocols (e.g., a local
file system). To support other protocols you may need to install their
implementation packages. Because there are too many,unifsdoesn't install
them for you by default, but it will tell which packages are missing if you
attempt to use a protocol that is not supported out of the box.For example, to add the support for the GCS:pip install gcsfsMake sure to install the additional packages to the same (virtual) environment
whereunifsis installed.To list known implementations and their prerequisites, use:unifs impl list
unifs impl info NAMETo avoid conflicts with other Python packages, it is recommended to install
this application into a dedicated virtual environment. For example, you may usepipx, or create a virtual environment manually. At very least, install with a--useroption (pip install --user unifs).Quick startBy default,unifswill use the local file system and will behave much like
issuing the similar commands directly in the shell:unifs ls -l /
unifs cat /tmp/foo.txt
unifs mv /tmp/foo.tx /tmp/bar.txt
unifs --helpYou need to configureunifsto let it know about other file systems you will
use.ConfigurationYou may either modify the configuration file, or useunifs confcommand to
manipulate it.Usingunifs confGet the list of configured file systems (currently active one is highlighted):unifs conf listSet the active file system:unifs conf use NAMEConfiguration fileunifsconfiguration is stored in the default OS configuration directory. You
can obtain a configuration file path with:unifs conf pathAlternatively, you can pin the configuration file location with aUNIFS_CONFIG_PATHenvironment variable.If you didn't change your default OS settings, most likely it will
be:~/.local/share/unifs/config.toml  # Linux
~/Library/Application Support/unifs/config.toml  # MacOS
~\AppData\Local\unifs\Config\config.toml # WindowsConfiguration file is a TOML file that consists of:a single[unifs]section where the currently active file system is setany number of[unifs.fs.NAME]sections that declare the file systemsExample:[unifs]
current = "local"

[unifs.fs.local]
protocol = "file"
auto_mkdir = falseFile system configuration is a set of key-value pairs.protocolkey is
mandatory and is used to select the implementation, all other values are passed
to the specific implementation. Useunifs impl info NAMEto the list of
accepted parameters for any protocol.For example, for a GCS bucket:[unifs.fs.my-gcs-bucket]
protocol = gcs
project = "my-gcp-project"
token = "/path/to/token.json"StatusAvailableunifsfeatures are considered stable.unifsis being actively
developed and more features are coming.Error reportingIf you happen to encounter an error ("An unexpected error" in the output),
please, feel free to report it on the Issues page. In this case, you may find
the detailed error message in the log file located in the same directory as the
application configuration file.Word of cautionBeware thatunifsmay change (copy, move, remove, etc.) the data in a "file
system" (as understood above).unifsis only a command-line layer between the
user and the target "file system".unifstries its best to prevent errors
(e.g., uses interactive confirmations for some commands), but ultimately the
user is responsible for the operations performed by this program.unifsis designed to be used in an interactive shell, not in headless mode.