smart_open (conda): smart_open is a Python 3 library for efficient streaming of very large files from/to S3, HDFS, WebHDFS or local (compressed) files.
Keywords: boto|bz2|file|gzip-stream|hacktoberfest|hdfs|python|s3|streaming|streaming-data|webhdfs