Package: jupyters3
Jupyter S3Jupyter Notebook Contents Manager for AWS S3.Installationpip install jupyters3ConfigurationTo configure Jupyter Notebook to use JupyterS3, you can add the following to your notebook config file.fromjupyters3importJupyterS3,JupyterS3SecretAccessKeyAuthenticationc=get_config()c.NotebookApp.contents_manager_class=JupyterS3andmustalso set the following settings onc.JupyterS3in your config file.SettingDescriptionExampleaws_regionThe AWS region in which the bucket is located'eu-west-1'aws_s3_bucketThe name of the S3 bucket.'my-example-bucket'aws_s3_hostThe hostname of the AWS S3 API. Typically, this is of the forms3-<aws_region>.amazonaws.com.'s3-eu-west-1.amazonaws.com'prefixThe prefix to all keys used to store notebooks and checkpoints. This can be the empty string''. If non-empty, typically this would end in a forward slash/.'some-prefix/'You must also, either, authenticate using a secret key, in which case you must have the following configurationfromjupyters3importJupyterS3SecretAccessKeyAuthenticationc.JupyterS3.authentication_class=JupyterS3SecretAccessKeyAuthenticationandthe following settings onc.JupyterS3SecretAccessKeyAuthenticationSettingDescriptionExampleaws_access_key_idThe ID of the AWS access key used to sign the requests to the AWS S3 API.ommittedaws_secret_access_keyThe secret part of the AWS access key used to sign the requests to the AWS S3 API.ommittedorauthenticate using a role in an ECS container, in which case you must have the following configurationfromjupyters3importJupyterS3ECSRoleAuthenticationc.JupyterS3.authentication_class=JupyterS3ECSRoleAuthenticationwhere JupyterS3ECSRoleAuthentication does not have configurable options,orwrite your own authentication class, such as the one below for EC2/IAM role-based authenticationimportdatetimeimportjsonfromjupyters3import(AwsCreds,JupyterS3Authentication,)fromtornadoimportgenfromtornado.httpclientimport(AsyncHTTPClient,HTTPErrorasHTTPClientError,HTTPRequest,)classJupyterS3EC2RoleAuthentication(JupyterS3Authentication):role_name=Unicode(config=True)aws_access_key_id=Unicode()aws_secret_access_key=Unicode()pre_auth_headers=Dict()expiration=Datetime()@gen.coroutinedefget_credentials(self):now=datetime.datetime.now()ifnow>self.expiration:request=HTTPRequest('http://169.254.169.254/latest/meta-data/iam/security-credentials/'+self.role_name,method='GET')creds=json.loads((yieldAsyncHTTPClient().fetch(request)).body.decode('utf-8'))self.aws_access_key_id=creds['AccessKeyId']self.aws_secret_access_key=creds['SecretAccessKey']self.pre_auth_headers={'x-amz-security-token':creds['Token'],}self.expiration=datetime.datetime.strptime(creds['Expiration'],'%Y-%m-%dT%H:%M:%SZ')returnAwsCreds(access_key_id=self.aws_access_key_id,secret_access_key=self.aws_secret_access_key,pre_auth_headers=self.pre_auth_headers,)configured usingc.JupyterS3.authentication_class=JupyterS3EC2RoleAuthenticationc.JupyterS3EC2RoleAuthentication.role_name='my-iam-role-name'Differences from S3ContentsThere are no extra dependencies over those already required for Jupyter Notebook. Specifically, there is no virtual filesystem library such as S3FS used, boto3 is not used, and Tornado is used as the HTTP client.Checkpoints are also saved to S3, under the key<file_name>/.checkpoints/.Multiple checkpoints are saved.The event loop is mostly not blocked during requests to S3. There are some exceptions due to Jupyter Notebook expecting certain requests to block.Uploading arbitrary files, such as JPEGs, and viewing them in Jupyter or downloading them, works.Copying and renaming files don't download or re-upload object data from or to S3. "PUT Object - Copy" is used instead.Folders are created using a 0 byte object with key suffix/(forward slash). A single forward slash suffix is consistent with both the AWS Console and AWS AppStream.