Package: s3bz
S3Bzsave and load dictionary to s3 using bz compressionfull docs herehttps://thanakijwanavit.github.io/s3bz/Installpip install s3bzHow to useCreate a bucket and make sure that it has transfer acceleration enabledcreate a buketaws s3 mb s3://<bucketname>put transfer accelerationaws s3api put-bucket-accelerate-configuration --bucket <bucketname> --accelerate-configuration Status=EnabledFirst, import the s3 moduleimport packagefrom importlib import reload
from s3bz.s3bz import S3set up dummy dataBZ2 compressionsave object using bz2 compressionresult = S3.save(key = key, 
       objectToSave = sampleDict,
       bucket = bucket,
       user=USER,
       pw = PW,
       accelerate = True)
print(('failed', 'success')[result])successload object with bz2 compressionresult = S3.load(key = key,
       bucket = bucket,
       user = USER,
       pw = PW,
       accelerate = True)
print(result[0]){'ib_prcode': '23238', 'ib_brcode': '1015', 'ib_cf_qty': '703', 'new_ib_vs_stock_cv': '768'}other compressionsZl : zlib compression with json string encoding
pklzl : zlib compression with pickle encodingprint(bucket)
%time S3.saveZl(key,sampleDict,bucket)
%time S3.loadZl(key,bucket)
%time S3.savePklZl(key,sampleDict,bucket)
%time result =S3.loadPklZl(key,bucket)pybz-test
CPU times: user 23.9 ms, sys: 559 µs, total: 24.5 ms
Wall time: 155 ms
CPU times: user 28.3 ms, sys: 3.04 ms, total: 31.4 ms
Wall time: 154 ms
CPU times: user 21.6 ms, sys: 228 µs, total: 21.9 ms
Wall time: 151 ms
CPU times: user 31.6 ms, sys: 0 ns, total: 31.6 ms
Wall time: 114 msBring your own compressor and encoderimport gzip, json
compressor=lambda x: gzip.compress(x)
encoder=lambda x: json.dumps(x).encode()
decompressor=lambda x: gzip.decompress(x)
decoder=lambda x: json.loads(x.decode())

%time S3.generalSave(key, sampleDict, bucket = bucket, compressor=compressor, encoder=encoder )
%time result = S3.generalLoad(key, bucket , decompressor=decompressor, decoder=decoder)
assert result == sampleDict, 'not the same as sample dict'CPU times: user 31 ms, sys: 0 ns, total: 31 ms
Wall time: 155 ms
CPU times: user 32.5 ms, sys: 51 µs, total: 32.5 ms
Wall time: 115 mscheck if an object existresult = S3.exist('', bucket, user=USER, pw=PW, accelerate = True)
print(('doesnt exist', 'exist')[result])existpresign download objecturl = S3.presign(key=key,
              bucket=bucket,
              expiry = 1000,
              user=USER,
              pw=PW)
print(url)https://pybz-test.s3-accelerate.amazonaws.com/test.dict?AWSAccessKeyId=AKIAVX4Z5TKDSNNNULGB&Signature=BR8Laz3uvkNKGh%2FBZ8x7IhRE3OU%3D&Expires=1616667887download using signed linkfrom s3bz.s3bz import Requests
result = Requests.getContentFromUrl(url)File operationssave without compressioninputPath = '/tmp/tmpFile.txt'
key = 'tmpFile'
downloadPath = '/tmp/downloadTmpFile.txt'
with open(inputPath , 'w')as f:
  f.write('hello world')S3.saveFile(key =key ,path = inputPath,bucket = bucket)
##test
S3.exist(key,bucket)Trueload without compressionS3.loadFile(key= key , path = downloadPath, bucket = bucket)##test
with open(downloadPath, 'r') as f:
  print(f.read())hello worlddeleteresult = S3.deleteFile(key, bucket)
## test
S3.exist(key,bucket)Falsesave and load pandas dataframe### please install in pandas, 
### this is not include in the requirements to minimize the size impact
import pandas as pd
df = pd.DataFrame({'test':[1,2,3,4,5],'test2':[2,3,4,5,6]})
S3.saveDataFrame(bucket,key,df)
S3.loadDataFrame(bucket,key)<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}</style>Unnamed: 0testtest200121123223433454456presign post with conditionsfrom s3bz.s3bz import ExtraArgs, S3bucket = 'pybz-test'
key = 'test.dict'
fields = {**ExtraArgs.jpeg}
S3.presignUpload(bucket, key, fields=fields){'url': 'https://pybz-test.s3-accelerate.amazonaws.com/',
 'fields': {'Content-Type': 'image/jpeg',
  'key': 'test.dict',
  'AWSAccessKeyId': 'AKIAVX4Z5TKDSNNNULGB',
  'policy': 'eyJleHBpcmF0aW9uIjogIjIwMjEtMDMtMjVUMTA6MjQ6NTJaIiwgImNvbmRpdGlvbnMiOiBbeyJidWNrZXQiOiAicHliei10ZXN0In0sIHsia2V5IjogInRlc3QuZGljdCJ9XX0=',
  'signature': 'hwC8kIjmjNPU0KT3BE54/TUQ/7w='}}