smart_open (conda): smart_open is a Python 3 library for efficient streaming of very large files from/to storages such as S3, GCS, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, or local filesystem.  It supports transparent, on-the-fly (de-)compression for a variety of different formats.
Keywords: boto|bz2|file|gzip-stream|hacktoberfest|hdfs|python|s3|streaming|streaming-data|webhdfs