Package: requests-aws4auth
Amazon Web Services version 4 authentication for the PythonRequestslibrary.FeaturesRequests authentication for all AWS services that support AWS auth v4Independent signing key objectsAutomatic regeneration of keys when scope date boundary is passedSupport for STS temporary credentialsImplements header-based authentication, GET URL parameter and POST
parameter authentication are not supported.Supported ServicesThis package has been tested as working against:AppStream, AppSync, Auto-Scaling, CloudFormation, CloudFront, CloudHSM,
CloudSearch, CloudTrail, CloudWatch Monitoring, CloudWatch Logs,
CodeDeploy, Cognito Identity, Cognito Sync, Config, DataPipeline, Direct
Connect, DynamoDB, Elastic Beanstalk, ElastiCache, EC2, EC2 Container
Service, Elastic Load Balancing, Elastic MapReduce, ElasticSearch,
Elastic Transcoder, Glacier, Identity and Access Management (IAM), Key
Management Service (KMS), Kinesis, Lambda, Opsworks, Redshift,
Relational Database Service (RDS), Route 53, Simple Storage Service
(S3), Simple Notification Service (SNS), Simple Queue Service (SQS),
Storage Gateway, Security Token Service (STS)The following services do not support AWS auth version 4 and are not
usable with this package:Simple Email Service (SES), Simple Workflow Service (SWF),
Import/Export, SimpleDB, DevPay, Mechanical TurkThe AWS Support API has not been tested as it requires a premium
subscription.Python versionsIn the 1.x semantic versions, the minimum python support will be gradually raised:1.0.x: Support python2.7 and python3.3+.1.1.x: python2.7 is not supported, is best-effort. Support python3.3+.1.2.x:Requires-Pythonwill be set to python3.3+, explicitly removing earlier versions. python<3.7 is not supported, is best-effort.1.3.x:Requires-Pythonwill be set to python3.7+, explicitly removing earlier versions. (best-effort is TBD)InstallationInstall via pip:$ pip install requests-aws4authrequests-aws4auth requires theRequestslibrary by Kenneth
Reitz.requests-aws4auth is tested on Python 2.7 and 3.5 and up.Behaviour changes in 0.8Version 0.8 introduces request date checking and automatic key
regeneration behaviour as default. This has implications for sharing
authentication objects between threads, and for storage of secret keys.
See the relevant sections below for details. See also the discussion inGitHub issue
#10.Basic usage>>> import requests
>>> from requests_aws4auth import AWS4Auth
>>> endpoint = 'http://s3-eu-west-1.amazonaws.com'
>>> auth = AWS4Auth('<ACCESS ID>', '<ACCESS KEY>', 'eu-west-1', 's3')
>>> response = requests.get(endpoint, auth=auth)
>>> response.text
<?xml version="1.0" encoding="UTF-8"?>
    <ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01">
        <Owner>
        <ID>bcaf1ffd86f461ca5fb16fd081034f</ID>
        <DisplayName>webfile</DisplayName>
        ...This example would list your buckets in theeu-west-1region of the
Amazon S3 service.STS Temporary Credentials>>> from requests_aws4auth import AWS4Auth
>>> auth = AWS4Auth('<ACCESS ID>', '<ACCESS KEY>', 'eu-west-1', 's3',
                    session_token='<SESSION TOKEN>')
...This example shows how to construct an AWS4Auth object for use with STS
temporary credentials. Thex-amz-security-tokenheader is added with
the session token. Temporary credential timeouts are not managed -- in
case the temporary credentials expire, they need to be re-generated and
the AWS4Auth object re-constructed with the new credentials.Dynamic STS Credentials using botocore RefreshableCredentials>>> from requests_aws4auth import AWS4Auth
>>> from botocore.session import Session
>>> credentials = Session().get_credentials()
>>> auth = AWS4Auth(region='eu-west-1', service='es',
                    refreshable_credentials=credentials)
...This example shows how to construct an AWS4Auth instance with
automatically refreshing credentials, suitable for long-running
applications using AWS IAM assume-role.
The RefreshableCredentials instance is used to generate valid static
credentials per-request, eliminating the need to recreate the AWS4Auth
instance when temporary credentials expire.Date handlingIf an HTTP request to be authenticated contains aDateorX-Amz-Dateheader, AWS will only accept the authorised request if the date in the
header matches the scope date of the signing key (see theAWS REST API date
docs.)).From version 0.8 of requests-aws4auth, if the header date does not match
the scope date, anAWS4Authinstance will automatically regenerate its
signing key, using the same scope parameters as the previous key except
for the date, which will be changed to match the request date. If a
request does not include a date, the current date is added to the
request in anX-Amz-Dateheader, and the signing key is regenerated if
this differs from the scope date.This means thatAWS4Authnow extracts and parses dates from the values
ofX-Amz-DateandDateheaders. Supported date formats are:RFC 7231 (e.g. Mon, 09 Sep 2011 23:36:00 GMT)RFC 850 (e.g. Sunday, 06-Nov-94 08:49:37 GMT)C time (e.g. Wed Dec 4 00:00:00 2002)Amz-Date format (e.g. 20090325T010101Z)ISO 8601 / RFC 3339 (e.g. 2009-03-25T10:11:12.13-01:00)If either header is present butAWS4Authcannot extract a date because
all present date headers are in an unrecognisable format,AWS4Authwill delete anyX-Amz-DateandDateheaders present and replace with
a singleX-Amz-Dateheader containing the current date. This behaviour
can be modified using theraise_invalid_datekeyword argument of theAWS4Authconstructor.Automatic key regenerationIf you do not want the signing key to be automatically regenerated when
a mismatch between the request date and the scope date is encountered,
use the alternativeStrictAWS4Authclass, which is identical toAWS4Authexcept that upon encountering a date mismatch it just raises
aDateMismatchError. You can also use thePassiveAWS4Authclass,
which mimics theAWS4Authbehaviour prior to version 0.8 and just
signs and sends the request, whether the date matches or not. In this
case it is up to the calling code to handle an authentication failure
response from AWS caused by the date mismatch.Secret key storageTo allow automatic key regeneration, the secret key is stored in theAWS4Authinstance, in the signing key object. If you do not want this
to occur, instantiate the instance using anAWS4Signingkey which was
created with thestore_secret_keyparameter set to False:>>> sig_key = AWS4SigningKey(secret_key, region, service, date, False)
>>> auth = StrictAWS4Auth(access_id, sig_key)TheAWS4Authclass will then raise aNoSecretKeyErrorwhen it
attempts to regenerate its key. A slightly more conceptually elegant way
to handle this is to use the alternativeStrictAWS4Authclass, again
instantiating it with anAWS4SigningKeyinstance created withstore_secret_key = False.MultithreadingIf you shareAWS4Auth(or evenStrictAWS4Auth) instances between
threads you are likely to encounter problems. BecauseAWS4Authinstances may unpredictably regenerate their signing key as part of
signing a request, threads using the same instance may find the key
changed by another thread halfway through the signing process, which may
result in undefined behaviour.It may be possible to rig up a workable instance sharing mechanism using
locking primitives and theStrictAWS4Authclass, however this poor
author can't think of a scenario which works safely yet doesn't suffer
from at some point blocking all threads for at least the duration of an
HTTP request, which could be several seconds. If several requests come
in in close succession which all require key regenerations then the
system could be forced into serial operation for quite a length of time.In short, it's probably best to create a thread-local instance ofAWS4Authfor each thread that needs to do authentication.API referenceSee the doctrings inaws4auth.pyandaws4signingkey.py.TestingA test suite is included in the test folder.The package passes all tests in the AWS auth v4test_suite,
and contains tests against the supported live services. See docstrings
intest/requests_aws4auth_test.pyfor details about running the tests.Connection parameters are included in the tests for the AWS Support API,
should you have access and want to try it. The documentation says it
supports auth v4 so it should work if you have a subscription. Do pass
on your results!Unsupported AWS features / todoCurrently does not support Amazon S3 chunked uploadsTests for new AWS servicesRequires Requests library to be present even if only using
AWS4SigningKeyCoherent documentationVersion release notesupdateHISTORY.mdupdaterequests_aws4auth/__init__.pycreate areleaseon githubprep:python3 -m pip install --user --upgrade setuptools wheel testresources twinebuild and release, creds in~/.pypirc:rm -f dist/*; \
python3 setup.py sdist bdist_wheel && \
python3 -m twine upload --repository testpypi_requests_aws4auth dist/* && \
python3 -m twine upload --repository pypi dist/*1.2.3 (2023-05-03)ChangesAdd manifest file so tarball installs succeed, #66. Thanks @jantman.1.2.2 (2023-02-02)BugfixesThe 1.2.0/1.2.1 releases had a regression error. The fix of #63 has been reverted.1.2.1 (2023-01-25)BugfixesActually fix #34. Build 1.2.0 was not fully released.1.2.0 (2023-01-20)BugfixesFix #34, port numbers on header, with #63. Thanks @phillipberndt.Changestest against 3.10. Currently supporting 3.8-3.10.small fixup to flake8 config1.1.2 (2022-03-24)Changesdon't install markdown files, and especially not outside of our lib dir, see #51. Thanks @benjaminp.prevent unquoting of query string reserved characters, see #60. Thanks @mliarakos.Add support for files as request body, see see #58. Thanks @USSRLivesOn.remove deprecated python 2.7 and python 3.5. Currently testing against 3.8 and 3.9.renamed main branch to 'main'1.1.1 (2021-06-04)Bugfixessecondary fix to query string ordering, documented in #49. It would fail with multiple values for the same key. Thanks @martinamps.fix minor deprecation warning in a regex.Package changesnoneTestsAdded tests for #49 secondary fix for string ordering.1.1.0 (2021-05-21)New featuresquery string ordering has been fixed. Documented in #21, fixed in #23. Thanks @zen4ever.test for spaces before calling shlex on them inamz_norm_whitespace. shlex doesn't like to split whitespace on a string without whitespace, taking several orders of magnitude longer to parse through it. #35, thanks @noamkush.addedrefreshable_credentials, see #37, thanks @teemuy.Package changesRemoved python2.7 support. Usage of py2.7 is not supported.Requires-Pythonwill be set to py3.3+ in the next minor release.Testsnone1.0.1 (2020-09-28)New featuresnonePackage changesbump to proper X.Y.Z semver syntaxbump project to stable (#33)Testsnone1.0 (2020-06-06)New featuresnone!Package changesadd flake8 configconvert docs to markdownRemoved bundled six.py.Taken over ownership viaPEP451;pypi issue here,successfully contacted Sam.Testsremove sys path hacking from tests.Ensure they work now.Github Actions are in place for continuous integration.Allow longer flake8 line length in tests.0.9 (2016-02-07)New featuresSupport for STS temporary credentials. Thanks tohttps://github.com/magdaleneTestsTests for the STS temporary credentials functionalityFixedAWS4SigningKey.amz_datedeprecation warning testElastic MapReduce live service test no longer using deprecated DescribeJobFlows action0.8 (2015-12-31)This version introduces some behaviour changes designed to reduce the legwork needed when a signing key goes out of date. This has implications for multithreading and secret key storage. See the README for further details.New featuresAWS4Auth class now checks request header date against signing key
scope date, and automatically regenerates the signing key with the
request date if they don't matchAdded exceptions module with new exceptions:
RequestsAWS4AuthException, DateMismatchError, NoSecretKeyError, DateFormatErrorAdded StrictAWS4Auth and PassiveAWS4Auth classesAWS4Auth changesAddedregenerate_signing_key()method, to allow regeneration of
current signing key with parameter overridesAdded methods for checking and extracting dates from requests:get_request_date(),parse_date(),handle_date_mismatch()__call__()now checks for a date header in the request and
attempts to automatically regenerate the signing key with the
request date if request date differs from the signing key dateCan now supply a date to the constructorChanged default included sig headers to includeDateheader if
presentAWS4SigningKey changesAdded newstore_secret_keyinstantiation parameter which allows
control of whether the secret key is stored in the instanceDeprecated theamz_dateproperty in favour of justdateSpelling typo fix in AWS4AuthSigningKey module docstring. Thanks to jhgorrellPackage changesDropped support for Python 3.2. Now only supported on Python 2.7 and
3.3 and up, to match versions supported by Requests.TestsMany new tests for the new functionalityAdded tests for generating canonical path, including test for fix
added in 0.7 for percent encoding of pathsAdded tests for generating canonical querystrings0.7 (2015-11-02)BugfixesFixed percent encoded characters in URL paths not being encoded
again for signature generation, as is expected for all services
except S3. This was causing authentication failures whenever these
characters appeared in a URL. Thanks to ipartola and cristi23 for
the report.Two bugfixes for ElasticSearch, thanks to Matthew Thompson for both:No longer setting body tob''during signing if it's NoneNow stripping port from URL netloc for signature generationModulesUpgraded the included version of six.py to 1.10TestsFixed a couple of broken Unicode tests on Python 2Added a couple more tests for encoding Unicode request bodies0.6 (2015-09-07)BugfixesIncluded HISTORY.rst in built package to fix pip source install
failure. Thanks to Beirdo for the bug report.0.5 (2015-04-29)BugfixesFixed bug when uploading to S3 with x-amz-acl header which caused
authentication failure - headers used in signature are now: host,
content-type and allx-amz-*headers (except for
x-amz-client-context which breaks Mobile Analytics auth if included)DocsMinor docstring and comment updatesLicenseChanged content of LICENSE to vanilla MIT license