Package: cannerflow-jupyter-contents
CannerFlow Jupyter ContentsAn S3, GCS, Blob backed ContentsManager implementation for Jupyter. Based ons3ContensInstallation$ pip install cannerflow-jupyter-contentsJupyter configEdit~/.jupyter/jupyter_notebook_config.pybased on the backend you want to
based on the examples below. Replace credentials as needed.AWS S3froms3contentsimportS3ContentsManagerc=get_config()# Tell Jupyter to use S3ContentsManager for all storage.c.NotebookApp.contents_manager_class=S3ContentsManagerc.S3ContentsManager.access_key_id="{{ AWS Access Key ID / IAM Access Key ID }}"c.S3ContentsManager.secret_access_key="{{ AWS Secret Access Key / IAM Secret Access Key }}"c.S3ContentsManager.session_token="{{ AWS Session Token / IAM Session Token }}"c.S3ContentsManager.bucket="{{ S3 bucket name }}"# Optional settings:c.S3ContentsManager.prefix="this/is/a/prefix/on/the/s3/bucket"c.S3ContentsManager.sse="AES256"c.S3ContentsManager.signature_version="s3v4"c.S3ContentsManager.init_s3_hook=init_function# See AWS key refreshExample forplay.minio.io:9000:froms3contentsimportS3ContentsManagerc=get_config()# Tell Jupyter to use S3ContentsManager for all storage.c.NotebookApp.contents_manager_class=S3ContentsManagerc.S3ContentsManager.access_key_id="Q3AM3UQ867SPQQA43P2F"c.S3ContentsManager.secret_access_key="zuf+tfteSlswRu7BJ86wekitnifILbZam1KYY3TG"c.S3ContentsManager.endpoint_url="http://play.minio.io:9000"c.S3ContentsManager.bucket="s3contents-demo"c.S3ContentsManager.prefix="notebooks/test"AWS EC2 role auth setupIt also possible to use IAM Role-based access to the S3 bucket from an Amazon EC2 instance.To do that just leaveaccess_key_idandsecret_access_keyset to their default values (None),
and ensure that the EC2 instance has an IAM role which provides sufficient permissions for the bucket and the operations necessary.AWS key refreshThe optionalinit_s3_hookconfiguration can be used to enable AWS key rotation (describedhereandhere) as follows:froms3contentsimportS3ContentsManagerfrombotocore.credentialsimportRefreshableCredentialsfrombotocore.sessionimportget_sessionimportbotocoreimportboto3fromconfigparserimportConfigParserdefrefresh_external_credentials():config=ConfigParser()config.read('/home/jovyan/.aws/credentials')return{"access_key":config['default']['aws_access_key_id'],"secret_key":config['default']['aws_secret_access_key'],"token":config['default']['aws_session_token'],"expiry_time":config['default']['aws_expiration']}session_credentials=RefreshableCredentials.create_from_metadata(metadata=refresh_external_credentials(),refresh_using=refresh_external_credentials,method='custom-refreshing-key-file-reader')defmake_key_refresh_boto3(this_s3contents_instance):refresh_session=get_session()# from botocore.sessionrefresh_session._credentials=session_credentialsmy_s3_session=boto3.Session(botocore_session=refresh_session)this_s3contents_instance.boto3_session=my_s3_session# Tell Jupyter to use S3ContentsManager for all storage.c.NotebookApp.contents_manager_class=S3ContentsManagerc.S3ContentsManager.init_s3_hook=make_key_refresh_boto3GCP Cloud Storagefroms3contentsimportGCSContentsManagerc=get_config(c.NotebookApp.contents_manager_class=GCSContentsManagerc.GCSContentsManager.project="{{ your-project }}"c.GCSContentsManager.token="~/.config/gcloud/application_default_credentials.json"c.GCSContentsManager.bucket="{{ GCP bucket name }}"Note that the file~/.config/gcloud/application_default_credentials.jsonassumes a posix system
when you didgcloud initAzure blob storage with SASimport osfrom s3contents import BlobContentsManager
from s3contents.blobmanager import auth_shared_access_signature
token = auth_shared_access_signature(
account_key=os.environ["AZURE_ACCOUNT_KEY"],
account_name=os.environ["AZURE_ACCOUNT_NAME"]
)
c.NotebookApp.contents_manager_class = BlobContentsManager
c.BlobContentsManager.credential=token
c.BlobContensManager.account_key=os.environ["AZURE_ACCOUNT_KEY"]
c.BlobContentsManager.container_name=os.environ["AZURE_CONTAINER_NAME"]
c.BlobContentsManager.account_name=os.environ["AZURE_ACCOUNT_NAME"]Access local filesTo access local file as well as remote files in S3 you can usehybridcontents.First install it:pip install hybridcontentsUse a configuration similar to this:froms3contentsimportS3ContentsManagerfromhybridcontentsimportHybridContentsManagerfromIPython.html.services.contents.filemanagerimportFileContentsManagerc=get_config()c.NotebookApp.contents_manager_class=HybridContentsManagerc.HybridContentsManager.manager_classes={# Associate the root directory with an S3ContentsManager.# This manager will receive all requests that don"t fall under any of the# other managers."":S3ContentsManager,# Associate /directory with a FileContentsManager."local_directory":FileContentsManager,}c.HybridContentsManager.manager_kwargs={# Args for root S3ContentsManager."":{"access_key_id":"{{ AWS Access Key ID / IAM Access Key ID }}","secret_access_key":"{{ AWS Secret Access Key / IAM Secret Access Key }}","bucket":"{{ S3 bucket name }}",},# Args for the FileContentsManager mapped to /directory"local_directory":{"root_dir":"/Users/danielfrg/Downloads",},}