Package: s5commander
s5commander◦ S5commander: Unleash the Power ofs5cmddirectly in Python📒 Table of Contents📒 Table of Contents📍 Overview⚙️ Features📂 Project Structure🧩 Modules🚀 Getting Started🗺 Roadmap🤝 Contributing📄 License👏 Acknowledgments📍 OverviewThe s5commander project is a Python library that allows users to interact with S3 buckets using the awesome s5cmd command-line tool.
To quote its description,s5cmdis a very fast S3 and local filesystem execution tool.
It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files.In short, s5cmd offers a very fast speed. For uploads, s5cmd is 32x faster than s3cmd and 12x faster than aws-cli. For downloads, s5cmd can saturate a 40Gbps link (~4.3 GB/s), whereas s3cmd and aws-cli can only reach 85 MB/s and 375 MB/s respectively.The goal of this library is to offer a wrapper around this software.
It offers functionality for performing various operations like listing, copying, and removing files, creating and removing buckets, checking disk usage, reading files, and utilizing data pipes.This project simplifies management and manipulation of S3 data usings5cmdinside a Python project, making it easier for users to work with S3 buckets and improve their productivity.⚙️ FeaturesFeatureDescription⚙️ ArchitectureThe codebase follows a modular and object-oriented design. It provides a wrapper for the s5cmd command-line tool and uses classes and functions to handle different actions related to S3 bucket management and file operations.📖 DocumentationThe documentation will come later...🔗 DependenciesThe codebase depends on the s5cmd command-line tool, which is an external dependency for interacting with the AWS S3 service. It relies on necessary AWS credentials and configuration provided by the user.🧩 ModularityThe codebase is organized into separate modules:commander.pycontains classes and functions related to S3 bucket operations and file management. It allows components to be reused or extended for different S3 operations.✔️ TestingTests will also come later :)⚡️ PerformanceThe performance of the codebase is directly dependent on the underlying s5cmd tool and AWS S3 service. As the codebase primarily acts as a wrapper, its own performance impact is limited.🔐 SecurityThe codebase relies on AWS credentials for accessing and interacting with S3 buckets, ensuring proper authentication and authorization. Proper handling and protection of these credentials is crucial for maintaining security.🔀 Version ControlThe codebase is hosted on GitHub and follows a standard Git version control workflow. Commits are tracked, and the repository can be cloned, branched, and versions can be audited using Git tools and techniques.🔌 IntegrationsThe codebase is designed to interact with cloud providers using an S3 protocol, including other providers than AWS services.📂 Project Structure🧩 ModulesS5commanderFileSummarycommander.pyThe code provides a wrapper for the s5cmd command-line tool, allowing users to interact with an S3 bucket. It supports functionalities like listing files, copying files, removing files, moving files, creating/removing buckets, selecting buckets, checking disk usage, reading files, and using data pipe.🚀 Getting Started✔️ PrerequisitesBefore you begin, ensure that you have the following prerequisites installed:s5cmd📦 InstallationClone the s5commander repository:gitclonehttps://github.com/clementpoiret/s5commanderChange to the project directory:cds5commanderInstall the dependencies:pipinstall-rrequirements.txt🎮 Using s5commander</code></pre><h3>🧪RunningTests</h3><prelang="sh"><code>pytest🗺 Roadmap[X]list buckets and objects[X]download method[X]upload method[ ]copy objects[ ]remove objects[ ]move/rename objects[ ]make bucket[ ]remove bucket[ ]run SQL queries on objects[ ]show object size usage[ ]print remote object content[ ]stream to remote from stdin[ ]run commands in batch[ ]sync objects🤝 ContributingContributions are always welcome! Please follow these steps:Fork the project repository. This creates a copy of the project on your account that you can modify without affecting the original project.Clone the forked repository to your local machine using a Git client like Git or GitHub Desktop.Create a new branch with a descriptive name (e.g.,new-feature-branchorbugfix-issue-123).gitcheckout-bnew-feature-branchMake changes to the project's codebase.Commit your changes to your local branch with a clear commit message that explains the changes you've made.gitcommit-m'Implemented new feature.'Push your changes to your forked repository on GitHub using the following commandgitpushoriginnew-feature-branchCreate a new pull request to the original project repository. In the pull request, describe the changes you've made and why they're necessary.
The project maintainers will review your changes and provide feedback or merge them into the main branch.📄 LicenseThis project is licensed under the MIT License. See theLICENSEfile for additional info.