|
Metadata-Version: 2.1 |
|
Name: fsspec |
|
Version: 2023.12.2 |
|
Summary: File-system specification |
|
Home-page: https://github.com/fsspec/filesystem_spec |
|
Maintainer: Martin Durant |
|
Maintainer-email: mdurant@anaconda.com |
|
License: BSD |
|
Project-URL: Changelog, https://filesystem-spec.readthedocs.io/en/latest/changelog.html |
|
Project-URL: Documentation, https://filesystem-spec.readthedocs.io/en/latest/ |
|
Keywords: file |
|
Classifier: Development Status :: 4 - Beta |
|
Classifier: Intended Audience :: Developers |
|
Classifier: License :: OSI Approved :: BSD License |
|
Classifier: Operating System :: OS Independent |
|
Classifier: Programming Language :: Python :: 3.8 |
|
Classifier: Programming Language :: Python :: 3.9 |
|
Classifier: Programming Language :: Python :: 3.10 |
|
Classifier: Programming Language :: Python :: 3.11 |
|
Requires-Python: >=3.8 |
|
Description-Content-Type: text/markdown |
|
License-File: LICENSE |
|
Provides-Extra: abfs |
|
Requires-Dist: adlfs ; extra == 'abfs' |
|
Provides-Extra: adl |
|
Requires-Dist: adlfs ; extra == 'adl' |
|
Provides-Extra: arrow |
|
Requires-Dist: pyarrow >=1 ; extra == 'arrow' |
|
Provides-Extra: dask |
|
Requires-Dist: dask ; extra == 'dask' |
|
Requires-Dist: distributed ; extra == 'dask' |
|
Provides-Extra: devel |
|
Requires-Dist: pytest ; extra == 'devel' |
|
Requires-Dist: pytest-cov ; extra == 'devel' |
|
Provides-Extra: dropbox |
|
Requires-Dist: dropboxdrivefs ; extra == 'dropbox' |
|
Requires-Dist: requests ; extra == 'dropbox' |
|
Requires-Dist: dropbox ; extra == 'dropbox' |
|
Provides-Extra: entrypoints |
|
Provides-Extra: full |
|
Requires-Dist: adlfs ; extra == 'full' |
|
Requires-Dist: aiohttp !=4.0.0a0,!=4.0.0a1 ; extra == 'full' |
|
Requires-Dist: dask ; extra == 'full' |
|
Requires-Dist: distributed ; extra == 'full' |
|
Requires-Dist: dropbox ; extra == 'full' |
|
Requires-Dist: dropboxdrivefs ; extra == 'full' |
|
Requires-Dist: fusepy ; extra == 'full' |
|
Requires-Dist: gcsfs ; extra == 'full' |
|
Requires-Dist: libarchive-c ; extra == 'full' |
|
Requires-Dist: ocifs ; extra == 'full' |
|
Requires-Dist: panel ; extra == 'full' |
|
Requires-Dist: paramiko ; extra == 'full' |
|
Requires-Dist: pyarrow >=1 ; extra == 'full' |
|
Requires-Dist: pygit2 ; extra == 'full' |
|
Requires-Dist: requests ; extra == 'full' |
|
Requires-Dist: s3fs ; extra == 'full' |
|
Requires-Dist: smbprotocol ; extra == 'full' |
|
Requires-Dist: tqdm ; extra == 'full' |
|
Provides-Extra: fuse |
|
Requires-Dist: fusepy ; extra == 'fuse' |
|
Provides-Extra: gcs |
|
Requires-Dist: gcsfs ; extra == 'gcs' |
|
Provides-Extra: git |
|
Requires-Dist: pygit2 ; extra == 'git' |
|
Provides-Extra: github |
|
Requires-Dist: requests ; extra == 'github' |
|
Provides-Extra: gs |
|
Requires-Dist: gcsfs ; extra == 'gs' |
|
Provides-Extra: gui |
|
Requires-Dist: panel ; extra == 'gui' |
|
Provides-Extra: hdfs |
|
Requires-Dist: pyarrow >=1 ; extra == 'hdfs' |
|
Provides-Extra: http |
|
Requires-Dist: requests ; extra == 'http' |
|
Requires-Dist: aiohttp !=4.0.0a0,!=4.0.0a1 ; extra == 'http' |
|
Provides-Extra: libarchive |
|
Requires-Dist: libarchive-c ; extra == 'libarchive' |
|
Provides-Extra: oci |
|
Requires-Dist: ocifs ; extra == 'oci' |
|
Provides-Extra: s3 |
|
Requires-Dist: s3fs ; extra == 's3' |
|
Provides-Extra: sftp |
|
Requires-Dist: paramiko ; extra == 'sftp' |
|
Provides-Extra: smb |
|
Requires-Dist: smbprotocol ; extra == 'smb' |
|
Provides-Extra: ssh |
|
Requires-Dist: paramiko ; extra == 'ssh' |
|
Provides-Extra: tqdm |
|
Requires-Dist: tqdm ; extra == 'tqdm' |
|
|
|
|
|
|
|
[](https://pypi.python.org/pypi/fsspec/) |
|
[](https://anaconda.org/conda-forge/fsspec) |
|
 |
|
[](https://filesystem-spec.readthedocs.io/en/latest/?badge=latest) |
|
[](https://pepy.tech/project/fsspec) |
|
|
|
A specification for pythonic filesystems. |
|
|
|
|
|
|
|
```bash |
|
pip install fsspec |
|
``` |
|
|
|
would install the base fsspec. Various optionally supported features might require specification of custom |
|
extra require, e.g. `pip install fsspec[ssh]` will install dependencies for `ssh` backends support. |
|
Use `pip install fsspec[full]` for installation of all known extra dependencies. |
|
|
|
Up-to-date package also provided through conda-forge distribution: |
|
|
|
```bash |
|
conda install -c conda-forge fsspec |
|
``` |
|
|
|
|
|
|
|
|
|
To produce a template or specification for a file-system interface, that specific implementations should follow, |
|
so that applications making use of them can rely on a common behaviour and not have to worry about the specific |
|
internal implementation decisions with any given backend. Many such implementations are included in this package, |
|
or in sister projects such as `s3fs` and `gcsfs`. |
|
|
|
In addition, if this is well-designed, then additional functionality, such as a key-value store or FUSE |
|
mounting of the file-system implementation may be available for all implementations "for free". |
|
|
|
|
|
|
|
Please refer to [RTD](https://filesystem-spec.readthedocs.io/en/latest/?badge=latest) |
|
|
|
|
|
|
|
fsspec uses GitHub Actions for CI. Environment files can be found |
|
in the "ci/" directory. Note that the main environment is called "py38", |
|
but it is expected that the version of python installed be adjustable at |
|
CI runtime. For local use, pick a version suitable for you. |
|
|
|
|
|
|
|
Tests can be run in the dev environment, if activated, via ``pytest fsspec``. |
|
|
|
The full fsspec suite requires a system-level docker, docker-compose, and fuse |
|
installation. If only making changes to one backend implementation, it is |
|
not generally necessary to run all tests locally. |
|
|
|
It is expected that contributors ensure that any change to fsspec does not |
|
cause issues or regressions for either other fsspec-related packages such |
|
as gcsfs and s3fs, nor for downstream users of fsspec. The "downstream" CI |
|
run and corresponding environment file run a set of tests from the dask |
|
test suite, and very minimal tests against pandas and zarr from the |
|
test_downstream.py module in this repo. |
|
|
|
|
|
|
|
fsspec uses [Black](https://black.readthedocs.io/en/stable) to ensure |
|
a consistent code format throughout the project. |
|
Run ``black fsspec`` from the root of the filesystem_spec repository to |
|
auto-format your code. Additionally, many editors have plugins that will apply |
|
``black`` as you edit files. ``black`` is included in the ``tox`` environments. |
|
|
|
Optionally, you may wish to setup [pre-commit hooks](https://pre-commit.com) to |
|
automatically run ``black`` when you make a git commit. |
|
Run ``pre-commit install --install-hooks`` from the root of the |
|
filesystem_spec repository to setup pre-commit hooks. ``black`` will now be run |
|
before you commit, reformatting any changed files. You can format without |
|
committing via ``pre-commit run`` or skip these checks with ``git commit |
|
--no-verify``. |
|
|
